libbsd development policy clarification needed?

Fri Feb 3 21:39:41 UTC 2023

Am 3. Februar 2023 22:12:06 MEZ schrieb Gedare Bloom <gedare at rtems.org>:
>On Fri, Feb 3, 2023 at 12:52 PM <oss at c-mauderer.de> wrote:
>>
>> Hello Gedare,
>>
>> Am 03.02.23 um 19:51 schrieb Gedare Bloom:
>> > On Thu, Feb 2, 2023 at 11:24 PM Christian MAUDERER
>> > <christian.mauderer at embedded-brains.de> wrote:
>> >>
>> >> Hello Karel,
>> >>
>> >> On 2023-02-02 12:43, Karel Gardas wrote:
>> >>>
>> >>>     Guys,
>> >>>
>> >>> recently I needed to work with RTEMS/NFS. As this is provided by libbsd
>> >>> I took this and following two sentences below from master branch
>> >>> description provided in README I took as granted that master does have
>> >>> all the features which are currently available and provided by the project:
>> >>>
>> >>> "This branch must be used for libbsd development. Back ports to the
>> >>> 6-freebsd-12 are allowed."
>> >>>
>> >>> I was surprised to be proven wrong then by Fabrizio here:
>> >>> https://devel.rtems.org/ticket/4723
>> >>>
>> >>> and by later investigation which shows that 6-freebsd-12 branch
>> >>> accumulated NFS work by Chris done in 2021 which is not presented on
>> >>> master. I've investigated just NFS as this was my focus here.
>> >>>
>> >>> So if 6-freebsd-12 became development branch of some sort, then it would
>> >>> be great to have that clarified in the project README file to prevent
>> >>> users confusion? Or if the policy is still the same, then perhaps some
>> >>> branch sync is needed here?
>> >>
>> >> That currently is an open issue. Basically there is a pending patch set
>> >> that should fix that since several months. But there is a disagreement
>> >> about some of the changes in that patch set (and about the patches
>> >> checked in to 6-freebsd-12). Therefore, it still hasn't been merged.
>> >>
>> >> If you want to know some more about the problematic points, I recommend
>> >> reading this (long) thread:
>> >>
>> >> https://lists.rtems.org/pipermail/devel/2023-January/074164.html
>> >>
>> >> The statement that development has to happen on the master branch is
>> >> still true. The master is intended to track the FreeBSD upstream
>> >> development. Only changes on that branch are guaranteed to live through
>> >> an upgrade to a newer base version of FreeBSD. It's very unfortunate,
>> >> that there are some patches on the 6-freebsd-12 branch only. On the long
>> >> term, that issue has to be resolved.
>> >>
>> >
>> > I have been investigating this problem in the background, and I have
>> > some findings and some questions. First, I have found that there is a
>> > most-common ancestor between master and 6-freebsd-12 at commit
>> > https://git.rtems.org/rtems-libbsd/commit/?h=6-freebsd-12&id=2ce13cf6dc73855f28bc7edbbc64dc4b482a4976
>> > This is at least promising that the discrepancy between the branches
>> > can be resolved.
>> >
>> > The proposed pending patch set to "fix" the NFS issue does not fix the
>> > underlying problem. Instead, it introduces further divergence between
>> > the branches. I would instead suggest that we should resolve to fix
>> > the underlying problem. I can see two paths forward.
>> >
>> > 1. Abandon 6-freebsd-12 after releasing 6. This is probably not ideal
>> > since what I understand is some users have projects based on
>> > 6-freebsd-12 and would like an upgrade path. (I guess there is also
>> > the option to abandon master, which also makes little sense.)
>>
>> A variant for this would be to introduce a 6-freebsd-13 that is based on
>> the master branch as soon as we have one. That would allow a longer
>> maintenance because FreeBSD 12 reaches it's EoL December 2023.
>>
>> >
>> > 2. Pull commits from 6-freebsd-12 into master to make sure master is
>> > the development head. in the future, reject patches that only go
>> > toward release branches. This has its own problems too. It can
>> > realistically only be done in three ways:
>>
>> Please note that Sebastian mentioned that the file descriptors broke the
>> NTP support (at least I think it was NTP; possible that it was another
>> submodule). So picking the current version of the patches into the
>> master without adding fixes makes the master unusable for some cases.
>>
>Fixing the problems after making the branches the same will be better
>for the long-term, if we can find a path to do it.
>
>> >
>> > 2a: Rebase master and cherry-pick commits from 6-freebsd-12 and master
>> > back into master. This rewrites the history of master, and
>> > unfortunately will cause the head of 5-freebsd-12 and the tags for
>> > rtems-5 to no longer exist on the master branch. They will still exist
>> > in the '5' branch. The advantage is in the end there will be a linear
>> > history of development on master that reflects the timeline of actual
>> > development that spanned both branches. Theoretically, this should
>> > make it easier to git-bisect.
>> >
>> > 2b: Cherry-pick commits from 6-freebsd-12 to master and fix conflicts.
>> > This puts all the missing commits from 6-freebsd-12 on to the current
>> > head of master. I don't know how messy this would be. It ends up
>> > making the history of master convoluted to understand, with fairly old
>> > commits from 2018 being placed on top of newer commits from 2020s.
>> >
>> > 2c: Merge 6-freebsd-12 into master and fixup conflicts in the merge
>> > commit. This is pretty similar to 2a but ends up with a non-linear
>> > history and a merge commit. It may be a fairly complex merge commit.
>>
>> For all of the 2x solutions: The commits from 6-freebsd-12 can't just be
>> cherry-picked. You have to re-import the NFS files from the FreeBSD
>> master version that is used as base for the current libbsd master.
>> Otherwise we mix different FreeBSD source versions. We had that some
>> time back in libbsd and Sebastian needed a lot of time cleaning that up.
>>
>Understood.
>
>> >
>> > To get a sense of the difference between the two branches, I have done
>> > the following command:
>> > $ git log --pretty=oneline master...6-freebsd-12 > ../log.txt
>> > This uses the ... (three-dot) Symmetric Difference Notation. The
>> > result of that is a 750 line file, so 750 commits are different
>> > between the two branches. Some of those commits are actually the same
>> > content, but they have different parents so different hashes. In a
>> > rebase or merge situation, those commits should end up the same. There
>> > may be other git-fu to find just the patches that are unique in the
>> > two branches.
>>
>> 750 commits are a bit too much. The order of patches on master and
>> 6-freebsd-12 isn't always the same. I wrote a small python script to
>> find the differences somewhere in 2021 when I needed the differences in
>> a discussion with (I think) Joel. It compares based on author and
>> subject which gives a quite good estimate for libbsd. You can find the
>> script here:
>>
>>    https://gist.github.com/a82e21eb250cb96c3a36f107b92dab09
>>
>> That shows 138 different commits. 86 are only on 6-freebsd-12. Complete
>> output is here:
>>
>>    https://gist.github.com/93a2b19a5bf4cd8a6263ae29aa359ac2
>>
>>  From these you can ignore the "Update to FreeBSD..." commits and some
>> of the cleanup patches. I assume that quite some of them can be
>> cherry-picked with only small or no changes. For example patches that
>> only add or update drivers in rtemsbsd should work without problems. So
>> I think a more realistic number of problematic patches should be 30 to 50.
>>
>Oh, thanks, this is nice. It may be a reasonable place to start by
>looking at what is "Only on master" and to consider back-porting to
>6-freebsd-12. That may reduce the delta somewhat in a way that makes
>sense.

We really should do the other direction. Not all features oft master have to be on the stable branch. But all features of the stable one should be on master.

master tracks the FreeBSD master. As soon as it reaches the point where FreeBSD branched off version 13, we can create a 6-freebsd-13 or 7-freebsd-13 from master (depending on which RTEMS version we have then). All features not on master would be lost when we do that.

The current 6-freebsd-12 on the other hand should be considered stable. We won't upgrade it to a 7-freensd-12 or similar. It will be a maintenance branch as soon as we reach the release. We can backport features that have been tested on master if there are users for that. But we don't have to. Just like we don't backport every new feature from RTEMS 6 to 5.

For example, I don't know of a user for the BBB framebuffer on RTEMS 6. It's nice that it is there on master. It will become a stable feature on the next major branch (one of 6 or 7-freebsd-13 or 14). But there is no necessity to backport it to the mostly stable 6-freebsd-12.

Best regards

Christian

>
>> Best regards
>>
>> Christian
>>
>> >
>> > In any case, doing this in a way that ensures the commits build and
>> > tests run is challenging due to the interactions with the rtems.git,
>> > toolchain, and the submodules. >
>> > After the 6-freebsd-12 and master are made consistent, then it becomes
>> > possible to update freebsd and also to consider what ways can be
>> > chosen to fix problems in 6-freebsd-12.
>> >
>> > -Gedare
>> >
>> >> Best regards
>> >>
>> >> Christian
>> >>
>> >>>
>> >>> I'm fine with either way, as a user I just need clear not confusing
>> >>> project message...
>> >>>
>> >>> Thanks!
>> >>> Karel
>> >>>
>> >>>
>> >>> _______________________________________________
>> >>> devel mailing list
>> >>> devel at rtems.org
>> >>> http://lists.rtems.org/mailman/listinfo/devel
>> >>
>> >> --
>> >> --------------------------------------------
>> >> embedded brains GmbH
>> >> Herr Christian MAUDERER
>> >> Dornierstr. 4
>> >> 82178 Puchheim
>> >> Germany
>> >> email:  christian.mauderer at embedded-brains.de
>> >> phone:  +49-89-18 94 741 - 18
>> >> mobile: +49-176-152 206 08
>> >>
>> >> Registergericht: Amtsgericht München
>> >> Registernummer: HRB 157899
>> >> Vertretungsberechtigte Geschäftsführer: Peter Rasmussen, Thomas Dörfler
>> >> Unsere Datenschutzerklärung finden Sie hier:
>> >> https://embedded-brains.de/datenschutzerklaerung/
>> >> _______________________________________________
>> >> devel mailing list
>> >> devel at rtems.org
>> >> http://lists.rtems.org/mailman/listinfo/devel
>> > _______________________________________________
>> > devel mailing list
>> > devel at rtems.org
>> > http://lists.rtems.org/mailman/listinfo/devel
>