libbsd development policy clarification needed?

Fri Feb 3 21:12:06 UTC 2023

On Fri, Feb 3, 2023 at 12:52 PM <oss at c-mauderer.de> wrote:
>
> Hello Gedare,
>
> Am 03.02.23 um 19:51 schrieb Gedare Bloom:
> > On Thu, Feb 2, 2023 at 11:24 PM Christian MAUDERER
> > <christian.mauderer at embedded-brains.de> wrote:
> >>
> >> Hello Karel,
> >>
> >> On 2023-02-02 12:43, Karel Gardas wrote:
> >>>
> >>>     Guys,
> >>>
> >>> recently I needed to work with RTEMS/NFS. As this is provided by libbsd
> >>> I took this and following two sentences below from master branch
> >>> description provided in README I took as granted that master does have
> >>> all the features which are currently available and provided by the project:
> >>>
> >>> "This branch must be used for libbsd development. Back ports to the
> >>> 6-freebsd-12 are allowed."
> >>>
> >>> I was surprised to be proven wrong then by Fabrizio here:
> >>> https://devel.rtems.org/ticket/4723
> >>>
> >>> and by later investigation which shows that 6-freebsd-12 branch
> >>> accumulated NFS work by Chris done in 2021 which is not presented on
> >>> master. I've investigated just NFS as this was my focus here.
> >>>
> >>> So if 6-freebsd-12 became development branch of some sort, then it would
> >>> be great to have that clarified in the project README file to prevent
> >>> users confusion? Or if the policy is still the same, then perhaps some
> >>> branch sync is needed here?
> >>
> >> That currently is an open issue. Basically there is a pending patch set
> >> that should fix that since several months. But there is a disagreement
> >> about some of the changes in that patch set (and about the patches
> >> checked in to 6-freebsd-12). Therefore, it still hasn't been merged.
> >>
> >> If you want to know some more about the problematic points, I recommend
> >> reading this (long) thread:
> >>
> >> https://lists.rtems.org/pipermail/devel/2023-January/074164.html
> >>
> >> The statement that development has to happen on the master branch is
> >> still true. The master is intended to track the FreeBSD upstream
> >> development. Only changes on that branch are guaranteed to live through
> >> an upgrade to a newer base version of FreeBSD. It's very unfortunate,
> >> that there are some patches on the 6-freebsd-12 branch only. On the long
> >> term, that issue has to be resolved.
> >>
> >
> > I have been investigating this problem in the background, and I have
> > some findings and some questions. First, I have found that there is a
> > most-common ancestor between master and 6-freebsd-12 at commit
> > https://git.rtems.org/rtems-libbsd/commit/?h=6-freebsd-12&id=2ce13cf6dc73855f28bc7edbbc64dc4b482a4976
> > This is at least promising that the discrepancy between the branches
> > can be resolved.
> >
> > The proposed pending patch set to "fix" the NFS issue does not fix the
> > underlying problem. Instead, it introduces further divergence between
> > the branches. I would instead suggest that we should resolve to fix
> > the underlying problem. I can see two paths forward.
> >
> > 1. Abandon 6-freebsd-12 after releasing 6. This is probably not ideal
> > since what I understand is some users have projects based on
> > 6-freebsd-12 and would like an upgrade path. (I guess there is also
> > the option to abandon master, which also makes little sense.)
>
> A variant for this would be to introduce a 6-freebsd-13 that is based on
> the master branch as soon as we have one. That would allow a longer
> maintenance because FreeBSD 12 reaches it's EoL December 2023.
>
> >
> > 2. Pull commits from 6-freebsd-12 into master to make sure master is
> > the development head. in the future, reject patches that only go
> > toward release branches. This has its own problems too. It can
> > realistically only be done in three ways:
>
> Please note that Sebastian mentioned that the file descriptors broke the
> NTP support (at least I think it was NTP; possible that it was another
> submodule). So picking the current version of the patches into the
> master without adding fixes makes the master unusable for some cases.
>
Fixing the problems after making the branches the same will be better
for the long-term, if we can find a path to do it.

> >
> > 2a: Rebase master and cherry-pick commits from 6-freebsd-12 and master
> > back into master. This rewrites the history of master, and
> > unfortunately will cause the head of 5-freebsd-12 and the tags for
> > rtems-5 to no longer exist on the master branch. They will still exist
> > in the '5' branch. The advantage is in the end there will be a linear
> > history of development on master that reflects the timeline of actual
> > development that spanned both branches. Theoretically, this should
> > make it easier to git-bisect.
> >
> > 2b: Cherry-pick commits from 6-freebsd-12 to master and fix conflicts.
> > This puts all the missing commits from 6-freebsd-12 on to the current
> > head of master. I don't know how messy this would be. It ends up
> > making the history of master convoluted to understand, with fairly old
> > commits from 2018 being placed on top of newer commits from 2020s.
> >
> > 2c: Merge 6-freebsd-12 into master and fixup conflicts in the merge
> > commit. This is pretty similar to 2a but ends up with a non-linear
> > history and a merge commit. It may be a fairly complex merge commit.
>
> For all of the 2x solutions: The commits from 6-freebsd-12 can't just be
> cherry-picked. You have to re-import the NFS files from the FreeBSD
> master version that is used as base for the current libbsd master.
> Otherwise we mix different FreeBSD source versions. We had that some
> time back in libbsd and Sebastian needed a lot of time cleaning that up.
>
Understood.

> >
> > To get a sense of the difference between the two branches, I have done
> > the following command:
> > $ git log --pretty=oneline master...6-freebsd-12 > ../log.txt
> > This uses the ... (three-dot) Symmetric Difference Notation. The
> > result of that is a 750 line file, so 750 commits are different
> > between the two branches. Some of those commits are actually the same
> > content, but they have different parents so different hashes. In a
> > rebase or merge situation, those commits should end up the same. There
> > may be other git-fu to find just the patches that are unique in the
> > two branches.
>
> 750 commits are a bit too much. The order of patches on master and
> 6-freebsd-12 isn't always the same. I wrote a small python script to
> find the differences somewhere in 2021 when I needed the differences in
> a discussion with (I think) Joel. It compares based on author and
> subject which gives a quite good estimate for libbsd. You can find the
> script here:
>
>    https://gist.github.com/a82e21eb250cb96c3a36f107b92dab09
>
> That shows 138 different commits. 86 are only on 6-freebsd-12. Complete
> output is here:
>
>    https://gist.github.com/93a2b19a5bf4cd8a6263ae29aa359ac2
>
>  From these you can ignore the "Update to FreeBSD..." commits and some
> of the cleanup patches. I assume that quite some of them can be
> cherry-picked with only small or no changes. For example patches that
> only add or update drivers in rtemsbsd should work without problems. So
> I think a more realistic number of problematic patches should be 30 to 50.
>
Oh, thanks, this is nice. It may be a reasonable place to start by
looking at what is "Only on master" and to consider back-porting to
6-freebsd-12. That may reduce the delta somewhat in a way that makes
sense.

> Best regards
>
> Christian
>
> >
> > In any case, doing this in a way that ensures the commits build and
> > tests run is challenging due to the interactions with the rtems.git,
> > toolchain, and the submodules. >
> > After the 6-freebsd-12 and master are made consistent, then it becomes
> > possible to update freebsd and also to consider what ways can be
> > chosen to fix problems in 6-freebsd-12.
> >
> > -Gedare
> >
> >> Best regards
> >>
> >> Christian
> >>
> >>>
> >>> I'm fine with either way, as a user I just need clear not confusing
> >>> project message...
> >>>
> >>> Thanks!
> >>> Karel
> >>>
> >>>
> >>> _______________________________________________
> >>> devel mailing list
> >>> devel at rtems.org
> >>> http://lists.rtems.org/mailman/listinfo/devel
> >>
> >> --
> >> --------------------------------------------
> >> embedded brains GmbH
> >> Herr Christian MAUDERER
> >> Dornierstr. 4
> >> 82178 Puchheim
> >> Germany
> >> email:  christian.mauderer at embedded-brains.de
> >> phone:  +49-89-18 94 741 - 18
> >> mobile: +49-176-152 206 08
> >>
> >> Registergericht: Amtsgericht München
> >> Registernummer: HRB 157899
> >> Vertretungsberechtigte Geschäftsführer: Peter Rasmussen, Thomas Dörfler
> >> Unsere Datenschutzerklärung finden Sie hier:
> >> https://embedded-brains.de/datenschutzerklaerung/
> >> _______________________________________________
> >> devel mailing list
> >> devel at rtems.org
> >> http://lists.rtems.org/mailman/listinfo/devel
> > _______________________________________________
> > devel mailing list
> > devel at rtems.org
> > http://lists.rtems.org/mailman/listinfo/devel