libbsd development policy clarification needed?

[Gedare Bloom]

On Mon, Feb 6, 2023 at 2:26 AM Christian MAUDERER
<christian.mauderer at embedded-brains.de> wrote:
>
> On 2023-02-06 10:02, Christian MAUDERER wrote:
> > On 2023-02-05 11:38, Christian Mauderer wrote:
> >>
> >>
> >> Am 04.02.23 um 01:25 schrieb Gedare Bloom:
> >>> On Fri, Feb 3, 2023 at 3:29 PM Christian Mauderer <oss at c-mauderer.de>
> >>> wrote:
> >>>>
> >>>>
> >>>>
> >>>> Am 3. Februar 2023 22:52:48 MEZ schrieb Gedare Bloom
> >>>> <gedare at rtems.org>:
> >>>>> On Fri, Feb 3, 2023 at 2:39 PM Christian Mauderer
> >>>>> <oss at c-mauderer.de> wrote:
> >>>>>>
> >>>>>>
> >>>>>>
> >>>>>> Am 3. Februar 2023 22:12:06 MEZ schrieb Gedare Bloom
> >>>>>> <gedare at rtems.org>:
> >>>>>>> On Fri, Feb 3, 2023 at 12:52 PM <oss at c-mauderer.de> wrote:
> >>>>>>>>
> >>>>>>>> Hello Gedare,
> >>>>>>>>
> >>>>>>>> Am 03.02.23 um 19:51 schrieb Gedare Bloom:
> >>>>>>>>> On Thu, Feb 2, 2023 at 11:24 PM Christian MAUDERER
> >>>>>>>>> <christian.mauderer at embedded-brains.de> wrote:
> >>>>>>>>>>
> >>>>>>>>>> Hello Karel,
> >>>>>>>>>>
> >>>>>>>>>> On 2023-02-02 12:43, Karel Gardas wrote:
> >>>>>>>>>>>
> >>>>>>>>>>>      Guys,
> >>>>>>>>>>>
> >>>>>>>>>>> recently I needed to work with RTEMS/NFS. As this is provided
> >>>>>>>>>>> by libbsd
> >>>>>>>>>>> I took this and following two sentences below from master branch
> >>>>>>>>>>> description provided in README I took as granted that master
> >>>>>>>>>>> does have
> >>>>>>>>>>> all the features which are currently available and provided
> >>>>>>>>>>> by the project:
> >>>>>>>>>>>
> >>>>>>>>>>> "This branch must be used for libbsd development. Back ports
> >>>>>>>>>>> to the
> >>>>>>>>>>> 6-freebsd-12 are allowed."
> >>>>>>>>>>>
> >>>>>>>>>>> I was surprised to be proven wrong then by Fabrizio here:
> >>>>>>>>>>> https://devel.rtems.org/ticket/4723
> >>>>>>>>>>>
> >>>>>>>>>>> and by later investigation which shows that 6-freebsd-12 branch
> >>>>>>>>>>> accumulated NFS work by Chris done in 2021 which is not
> >>>>>>>>>>> presented on
> >>>>>>>>>>> master. I've investigated just NFS as this was my focus here.
> >>>>>>>>>>>
> >>>>>>>>>>> So if 6-freebsd-12 became development branch of some sort,
> >>>>>>>>>>> then it would
> >>>>>>>>>>> be great to have that clarified in the project README file to
> >>>>>>>>>>> prevent
> >>>>>>>>>>> users confusion? Or if the policy is still the same, then
> >>>>>>>>>>> perhaps some
> >>>>>>>>>>> branch sync is needed here?
> >>>>>>>>>>
> >>>>>>>>>> That currently is an open issue. Basically there is a pending
> >>>>>>>>>> patch set
> >>>>>>>>>> that should fix that since several months. But there is a
> >>>>>>>>>> disagreement
> >>>>>>>>>> about some of the changes in that patch set (and about the
> >>>>>>>>>> patches
> >>>>>>>>>> checked in to 6-freebsd-12). Therefore, it still hasn't been
> >>>>>>>>>> merged.
> >>>>>>>>>>
> >>>>>>>>>> If you want to know some more about the problematic points, I
> >>>>>>>>>> recommend
> >>>>>>>>>> reading this (long) thread:
> >>>>>>>>>>
> >>>>>>>>>> https://lists.rtems.org/pipermail/devel/2023-January/074164.html
> >>>>>>>>>>
> >>>>>>>>>> The statement that development has to happen on the master
> >>>>>>>>>> branch is
> >>>>>>>>>> still true. The master is intended to track the FreeBSD upstream
> >>>>>>>>>> development. Only changes on that branch are guaranteed to
> >>>>>>>>>> live through
> >>>>>>>>>> an upgrade to a newer base version of FreeBSD. It's very
> >>>>>>>>>> unfortunate,
> >>>>>>>>>> that there are some patches on the 6-freebsd-12 branch only.
> >>>>>>>>>> On the long
> >>>>>>>>>> term, that issue has to be resolved.
> >>>>>>>>>>
> >>>>>>>>>
> >>>>>>>>> I have been investigating this problem in the background, and I
> >>>>>>>>> have
> >>>>>>>>> some findings and some questions. First, I have found that
> >>>>>>>>> there is a
> >>>>>>>>> most-common ancestor between master and 6-freebsd-12 at commit
> >>>>>>>>> https://git.rtems.org/rtems-libbsd/commit/?h=6-freebsd-12&id=2ce13cf6dc73855f28bc7edbbc64dc4b482a4976
> >>>>>>>>> This is at least promising that the discrepancy between the
> >>>>>>>>> branches
> >>>>>>>>> can be resolved.
> >>>>>>>>>
> >>>>>>>>> The proposed pending patch set to "fix" the NFS issue does not
> >>>>>>>>> fix the
> >>>>>>>>> underlying problem. Instead, it introduces further divergence
> >>>>>>>>> between
> >>>>>>>>> the branches. I would instead suggest that we should resolve to
> >>>>>>>>> fix
> >>>>>>>>> the underlying problem. I can see two paths forward.
> >>>>>>>>>
> >>>>>>>>> 1. Abandon 6-freebsd-12 after releasing 6. This is probably not
> >>>>>>>>> ideal
> >>>>>>>>> since what I understand is some users have projects based on
> >>>>>>>>> 6-freebsd-12 and would like an upgrade path. (I guess there is
> >>>>>>>>> also
> >>>>>>>>> the option to abandon master, which also makes little sense.)
> >>>>>>>>
> >>>>>>>> A variant for this would be to introduce a 6-freebsd-13 that is
> >>>>>>>> based on
> >>>>>>>> the master branch as soon as we have one. That would allow a longer
> >>>>>>>> maintenance because FreeBSD 12 reaches it's EoL December 2023.
> >>>>>>>>
> >>>>>>>>>
> >>>>>>>>> 2. Pull commits from 6-freebsd-12 into master to make sure
> >>>>>>>>> master is
> >>>>>>>>> the development head. in the future, reject patches that only go
> >>>>>>>>> toward release branches. This has its own problems too. It can
> >>>>>>>>> realistically only be done in three ways:
> >>>>>>>>
> >>>>>>>> Please note that Sebastian mentioned that the file descriptors
> >>>>>>>> broke the
> >>>>>>>> NTP support (at least I think it was NTP; possible that it was
> >>>>>>>> another
> >>>>>>>> submodule). So picking the current version of the patches into the
> >>>>>>>> master without adding fixes makes the master unusable for some
> >>>>>>>> cases.
> >>>>>>>>
> >>>>>>> Fixing the problems after making the branches the same will be
> >>>>>>> better
> >>>>>>> for the long-term, if we can find a path to do it.
> >>>>>>>
> >>>>>>>>>
> >>>>>>>>> 2a: Rebase master and cherry-pick commits from 6-freebsd-12 and
> >>>>>>>>> master
> >>>>>>>>> back into master. This rewrites the history of master, and
> >>>>>>>>> unfortunately will cause the head of 5-freebsd-12 and the tags for
> >>>>>>>>> rtems-5 to no longer exist on the master branch. They will
> >>>>>>>>> still exist
> >>>>>>>>> in the '5' branch. The advantage is in the end there will be a
> >>>>>>>>> linear
> >>>>>>>>> history of development on master that reflects the timeline of
> >>>>>>>>> actual
> >>>>>>>>> development that spanned both branches. Theoretically, this should
> >>>>>>>>> make it easier to git-bisect.
> >>>>>>>>>
> >>>>>>>>> 2b: Cherry-pick commits from 6-freebsd-12 to master and fix
> >>>>>>>>> conflicts.
> >>>>>>>>> This puts all the missing commits from 6-freebsd-12 on to the
> >>>>>>>>> current
> >>>>>>>>> head of master. I don't know how messy this would be. It ends up
> >>>>>>>>> making the history of master convoluted to understand, with
> >>>>>>>>> fairly old
> >>>>>>>>> commits from 2018 being placed on top of newer commits from 2020s.
> >>>>>>>>>
> >>>>>>>>> 2c: Merge 6-freebsd-12 into master and fixup conflicts in the
> >>>>>>>>> merge
> >>>>>>>>> commit. This is pretty similar to 2a but ends up with a non-linear
> >>>>>>>>> history and a merge commit. It may be a fairly complex merge
> >>>>>>>>> commit.
> >>>>>>>>
> >>>>>>>> For all of the 2x solutions: The commits from 6-freebsd-12 can't
> >>>>>>>> just be
> >>>>>>>> cherry-picked. You have to re-import the NFS files from the FreeBSD
> >>>>>>>> master version that is used as base for the current libbsd master.
> >>>>>>>> Otherwise we mix different FreeBSD source versions. We had that
> >>>>>>>> some
> >>>>>>>> time back in libbsd and Sebastian needed a lot of time cleaning
> >>>>>>>> that up.
> >>>>>>>>
> >>>>>>> Understood.
> >>>>>>>
> >>>>>>>>>
> >>>>>>>>> To get a sense of the difference between the two branches, I
> >>>>>>>>> have done
> >>>>>>>>> the following command:
> >>>>>>>>> $ git log --pretty=oneline master...6-freebsd-12 > ../log.txt
> >>>>>>>>> This uses the ... (three-dot) Symmetric Difference Notation. The
> >>>>>>>>> result of that is a 750 line file, so 750 commits are different
> >>>>>>>>> between the two branches. Some of those commits are actually
> >>>>>>>>> the same
> >>>>>>>>> content, but they have different parents so different hashes. In a
> >>>>>>>>> rebase or merge situation, those commits should end up the
> >>>>>>>>> same. There
> >>>>>>>>> may be other git-fu to find just the patches that are unique in
> >>>>>>>>> the
> >>>>>>>>> two branches.
> >>>>>>>>
> >>>>>>>> 750 commits are a bit too much. The order of patches on master and
> >>>>>>>> 6-freebsd-12 isn't always the same. I wrote a small python
> >>>>>>>> script to
> >>>>>>>> find the differences somewhere in 2021 when I needed the
> >>>>>>>> differences in
> >>>>>>>> a discussion with (I think) Joel. It compares based on author and
> >>>>>>>> subject which gives a quite good estimate for libbsd. You can
> >>>>>>>> find the
> >>>>>>>> script here:
> >>>>>>>>
> >>>>>>>>     https://gist.github.com/a82e21eb250cb96c3a36f107b92dab09
> >>>>>>>>
> >>>>>>>> That shows 138 different commits. 86 are only on 6-freebsd-12.
> >>>>>>>> Complete
> >>>>>>>> output is here:
> >>>>>>>>
> >>>>>>>>     https://gist.github.com/93a2b19a5bf4cd8a6263ae29aa359ac2
> >>>>>>>>
> >>>>>>>>   From these you can ignore the "Update to FreeBSD..." commits
> >>>>>>>> and some
> >>>>>>>> of the cleanup patches. I assume that quite some of them can be
> >>>>>>>> cherry-picked with only small or no changes. For example patches
> >>>>>>>> that
> >>>>>>>> only add or update drivers in rtemsbsd should work without
> >>>>>>>> problems. So
> >>>>>>>> I think a more realistic number of problematic patches should be
> >>>>>>>> 30 to 50.
> >>>>>>>>
> >>>>>>> Oh, thanks, this is nice. It may be a reasonable place to start by
> >>>>>>> looking at what is "Only on master" and to consider back-porting to
> >>>>>>> 6-freebsd-12. That may reduce the delta somewhat in a way that makes
> >>>>>>> sense.
> >>>>>>
> >>>>>> We really should do the other direction. Not all features oft
> >>>>>> master have to be on the stable branch. But all features of the
> >>>>>> stable one should be on master.
> >>>>>>
> >>>>> I agree, but since there are many fewer changes only on master, if we
> >>>>> can merge those into 6-freebsd-12, then it becomes simpler to just
> >>>>> call 6-freebsd-12 as the new master. We can replace the current master
> >>>>> with this branch, and then update freebsd versions. I think this can
> >>>>> be a lower burden process to follow, except it requires acceptance of
> >>>>> the commits that are on 6-freebsd-12, which is a different matter.
> >>>>
> >>>> You can't just make 6-freebsd-12 a new master. It tracks the FreeBSD
> >>>> 12 branch.
> >>>>
> >>>> Every block of updates to a newer FreeBSD base revision was several
> >>>> days of work for Sebastian. There are a few on master.
> >>>>
> >>>> An upgrade from a branch tracking FreeBSD 12 to one tracking FreeBSD
> >>>> 13 is most likely more something in the several week range of work.
> >>>>
> >>>>>
> >>>>>> master tracks the FreeBSD master. As soon as it reaches the point
> >>>>>> where FreeBSD branched off version 13, we can create a
> >>>>>> 6-freebsd-13 or 7-freebsd-13 from master (depending on which RTEMS
> >>>>>> version we have then). All features not on master would be lost
> >>>>>> when we do that.
> >>>>>>
> >>>>>> The current 6-freebsd-12 on the other hand should be considered
> >>>>>> stable. We won't upgrade it to a 7-freensd-12 or similar. It will
> >>>>>> be a maintenance branch as soon as we reach the release. We can
> >>>>>> backport features that have been tested on master if there are
> >>>>>> users for that. But we don't have to. Just like we don't backport
> >>>>>> every new feature from RTEMS 6 to 5.
> >>>>>>
> >>>>> I view this as equivalent to the proposals to revert commits that are
> >>>>> on 6-freebsd-12. Doing this means that we are discarding quite a lot
> >>>>> of software engineering effort that was done on 6-freebsd-12. I would
> >>>>> rather that we determine a way that can retain that effort. At this
> >>>>> point, there is probably little way to make everyone happy.
> >>>>
> >>>> I think you misunderstood my intention. We definitively should port
> >>>> the patches from 6-freebsd-12 to master so that they are on a future
> >>>> libbsd branch. Like said above: I just don't think it's realistic to
> >>>> base a new developed branch on the 12 one.
> >>>>
> >>> I see. This is good to consider.
> >>>
> >>>> If I remember correctly, Sebastian's branch that contains his
> >>>> suggested fixes already ported Chris NFS work to master first and
> >>>> then added Sebastian's patches that revert the parts where he
> >>>> disagrees.
> >>>>
> >>> I would leave it to them to determine if the NFSv4 is fully supported.
> >>>
> >>>> That leaves some driver patches on the 12 branch that are most
> >>>> likely a lot simpler to port to master.
> >>>>
> >>>> My guess would be: Porting the stuff from 6-freebsd-12 to master
> >>>> should be a few days maximum to a compile clean state. Upgrading
> >>>> 6-freebsd-12 to something that would follow FreeBSD master again
> >>>> should be something in the range of weeks.
> >>>>
> >>> OK, then we should likely follow the path of porting from 6-freebsd-12
> >>> to master. I would leave the issue of what to do with FDs as a
> >>> separate concern that can be addressed once we solve this underlying
> >>> problem.
> >>>
> >>> What would you think is the best way to handle the process though? It
> >>> would seem to still require some kind of cherry-picking + patching
> >>> FreeBSD and passing some tests? Perhaps it is best to do it that way,
> >>> on top of the current master branch, attempting to pick up patches
> >>> from the shared ancestor. ("Only in 6-freebsd-12").
> >>
> >> I would usually just cherry-pick commits that don't import anything
> >> from FreeBSD. All commits that do import files need a bit more work:
> >> For these, the files from FreeBSD master have to be re-importet.
> >>
> >> It's basically the same process that I use to back-port something from
> >> master to 6-freebsd-12. And it's why CONTRIBUTING.md tells that you
> >> should have one commit that only adds the unmodified imported files.
> >> With that it's easy to re-import the files from a different version
> >> and re-apply the patches.
> >>
> >> Best regards
> >>
> >> Christian
> >
> > I tried to cherry-pick some of the commits yesterday (hobby time on
> > Sunday!). The first few patches should work quite well. There are some
> > minor problems that still have to be cleaned up to make it compile clean.
> >
> > One of the early ones that makes problems is the path-mappings feature
> > from Jan Sommer because on master something changed the same code parts.
> > We need someone who understands both changes to port that.
> >
> > About starting with Chris kernel symbol namespace tool patch
> > (59f652fe88) the changes start to get bigger and the patches are harder
> > to apply. I think the kernel symbol namespace is one of the core patches
> > that would be important to port, but I don't know the code surrounding
> > it well enough that I can do it fast.
> >
> > How should we approach that work? One patch at a time? Maybe together
> > with pinging the original authors to ask them whether they can port
> > their patches to master or at least test them there?
> >
> > For the VFS and NFS parts, it depends a bit on the solution of the
> > discussion. Sebastian already worked on a re-implementation of NFS on
> > master (not yet entirely finished):
> >
> >    https://github.com/sebhub/rtems-libbsd/commits/master-update
> >
> > Of course that is based on his solution without the file descriptors and
> > without the central system calls. So it depends on the results of the
> > discussion between Sebastian and Chris whether that patch can be used to
> > have the NFS functionality in master.
> >
>
> When re-reading the text I noted that mentioning that branch here can
> lead into a wrong direction. To make my intention clear: I think we
> should concentrate on all unrelated patches at the moment. Parallel we
> should try to push the NFS/VFS discussion and help Chris and Sebastian
> to agree on one solution that is acceptable for both. Depending on that
> solution, it could be simpler to just use that branch and maybe port
> only small parts to have a functionally equivalent solution on both
> branches with a slightly different history.
>
Yes, thanks. This looks like a good analysis. I would definitely
prefer to get master and 6-freebsd-12 into harmony. Then it is a
little simpler to discuss the other problems in libbsd.

Vijay had similar kind of success (and problems) as you did with
cherry-picking off of 6-freebsd-12. I think he made it a little
further. I guess one possible route forward would be to begin working
on this effort and share results, pushing to master when some
relatively stable milestones are reached.

Gedare

> Best regards
>
> Christian
>
> > Note that Sebastian also mentioned, that FreeBSD changed a lot in the
> > NFS/VFS in the last few months. So it's possible that both solutions
> > need quite some changes after we reach that version of FreeBSD.
> >
> > Best regards
> >
> > Christian
> >
> >>
> >>>
> >>>> Best regards
> >>>>
> >>>> Christian
> >>>>
> >>>>>
> >>>>> it will be much easier to debate the merits of proposed patches to
> >>>>> address optimization etc., if we have a shared basis of features and
> >>>>> development. The way this has been heading is going to lead us
> >>>>> directly into a non-viable mode of trying to support several open
> >>>>> development branches of libbsd. I can easily see someone wanting to
> >>>>> carry forward 6-freebsd-12 commits to rtems7. If we don't have a way
> >>>>> to do that, we will definitely end up with something like
> >>>>> 7-freebsd-12.
> >>>>>
> >>>>> We have to deal with the fact that there are legitimate developments
> >>>>> on 6-freebsd-12 that users may expect in future RTEMS releases. I
> >>>>> cannot agree with having a release of 6-freebsd-12 with rtems6 that is
> >>>>> followed by a release of 7-freebsd-1x that drops some features such as
> >>>>> support for NFSv4. It is not pragmatic. But we also don't have the
> >>>>> resources to be maintaining multiple branches of libbsd.
> >>>>>
> >>>>>> For example, I don't know of a user for the BBB framebuffer on
> >>>>>> RTEMS 6. It's nice that it is there on master. It will become a
> >>>>>> stable feature on the next major branch (one of 6 or 7-freebsd-13
> >>>>>> or 14). But there is no necessity to backport it to the mostly
> >>>>>> stable 6-freebsd-12.
> >>>>>>
> >>>>> I have asked Vijay to look at backporting it anyway, to reduce the
> >>>>> difference between the branches. I want to get us out of this backward
> >>>>> way of development / change management that has evolved over time in
> >>>>> libbsd. How best to get us to a state where new development only goes
> >>>>> to master is for me still the main question. Once we have this fixed,
> >>>>> it is my intention to thoroughly reject any development that is only
> >>>>> submitted to a "stable" branch in the future.
> >>>>>
> >>>>> Gedare
> >>>>>
> >>>>>> Best regards
> >>>>>>
> >>>>>> Christian
> >>>>>>
> >>>>>>>
> >>>>>>>> Best regards
> >>>>>>>>
> >>>>>>>> Christian
> >>>>>>>>
> >>>>>>>>>
> >>>>>>>>> In any case, doing this in a way that ensures the commits build
> >>>>>>>>> and
> >>>>>>>>> tests run is challenging due to the interactions with the
> >>>>>>>>> rtems.git,
> >>>>>>>>> toolchain, and the submodules. >
> >>>>>>>>> After the 6-freebsd-12 and master are made consistent, then it
> >>>>>>>>> becomes
> >>>>>>>>> possible to update freebsd and also to consider what ways can be
> >>>>>>>>> chosen to fix problems in 6-freebsd-12.
> >>>>>>>>>
> >>>>>>>>> -Gedare
> >>>>>>>>>
> >>>>>>>>>> Best regards
> >>>>>>>>>>
> >>>>>>>>>> Christian
> >>>>>>>>>>
> >>>>>>>>>>>
> >>>>>>>>>>> I'm fine with either way, as a user I just need clear not
> >>>>>>>>>>> confusing
> >>>>>>>>>>> project message...
> >>>>>>>>>>>
> >>>>>>>>>>> Thanks!
> >>>>>>>>>>> Karel
> >>>>>>>>>>>
> >>>>>>>>>>>
> >>>>>>>>>>> _______________________________________________
> >>>>>>>>>>> devel mailing list
> >>>>>>>>>>> devel at rtems.org
> >>>>>>>>>>> http://lists.rtems.org/mailman/listinfo/devel
> >>>>>>>>>>
> >>>>>>>>>> --
> >>>>>>>>>> --------------------------------------------
> >>>>>>>>>> embedded brains GmbH
> >>>>>>>>>> Herr Christian MAUDERER
> >>>>>>>>>> Dornierstr. 4
> >>>>>>>>>> 82178 Puchheim
> >>>>>>>>>> Germany
> >>>>>>>>>> email:  christian.mauderer at embedded-brains.de
> >>>>>>>>>> phone:  +49-89-18 94 741 - 18
> >>>>>>>>>> mobile: +49-176-152 206 08
> >>>>>>>>>>
> >>>>>>>>>> Registergericht: Amtsgericht München
> >>>>>>>>>> Registernummer: HRB 157899
> >>>>>>>>>> Vertretungsberechtigte Geschäftsführer: Peter Rasmussen,
> >>>>>>>>>> Thomas Dörfler
> >>>>>>>>>> Unsere Datenschutzerklärung finden Sie hier:
> >>>>>>>>>> https://embedded-brains.de/datenschutzerklaerung/
> >>>>>>>>>> _______________________________________________
> >>>>>>>>>> devel mailing list
> >>>>>>>>>> devel at rtems.org
> >>>>>>>>>> http://lists.rtems.org/mailman/listinfo/devel
> >>>>>>>>> _______________________________________________
> >>>>>>>>> devel mailing list
> >>>>>>>>> devel at rtems.org
> >>>>>>>>> http://lists.rtems.org/mailman/listinfo/devel
> >>>>>>>
> >>>>>
> >>>
> >
>
> --
> --------------------------------------------
> embedded brains GmbH
> Herr Christian MAUDERER
> Dornierstr. 4
> 82178 Puchheim
> Germany
> email:  christian.mauderer at embedded-brains.de
> phone:  +49-89-18 94 741 - 18
> mobile: +49-176-152 206 08
>
> Registergericht: Amtsgericht München
> Registernummer: HRB 157899
> Vertretungsberechtigte Geschäftsführer: Peter Rasmussen, Thomas Dörfler
> Unsere Datenschutzerklärung finden Sie hier:
> https://embedded-brains.de/datenschutzerklaerung/