libbsd development policy clarification needed?

Fri Feb 3 19:52:05 UTC 2023

Hello Gedare,

Am 03.02.23 um 19:51 schrieb Gedare Bloom:
> On Thu, Feb 2, 2023 at 11:24 PM Christian MAUDERER
> <christian.mauderer at embedded-brains.de> wrote:
>>
>> Hello Karel,
>>
>> On 2023-02-02 12:43, Karel Gardas wrote:
>>>
>>>     Guys,
>>>
>>> recently I needed to work with RTEMS/NFS. As this is provided by libbsd
>>> I took this and following two sentences below from master branch
>>> description provided in README I took as granted that master does have
>>> all the features which are currently available and provided by the project:
>>>
>>> "This branch must be used for libbsd development. Back ports to the
>>> 6-freebsd-12 are allowed."
>>>
>>> I was surprised to be proven wrong then by Fabrizio here:
>>> https://devel.rtems.org/ticket/4723
>>>
>>> and by later investigation which shows that 6-freebsd-12 branch
>>> accumulated NFS work by Chris done in 2021 which is not presented on
>>> master. I've investigated just NFS as this was my focus here.
>>>
>>> So if 6-freebsd-12 became development branch of some sort, then it would
>>> be great to have that clarified in the project README file to prevent
>>> users confusion? Or if the policy is still the same, then perhaps some
>>> branch sync is needed here?
>>
>> That currently is an open issue. Basically there is a pending patch set
>> that should fix that since several months. But there is a disagreement
>> about some of the changes in that patch set (and about the patches
>> checked in to 6-freebsd-12). Therefore, it still hasn't been merged.
>>
>> If you want to know some more about the problematic points, I recommend
>> reading this (long) thread:
>>
>> https://lists.rtems.org/pipermail/devel/2023-January/074164.html
>>
>> The statement that development has to happen on the master branch is
>> still true. The master is intended to track the FreeBSD upstream
>> development. Only changes on that branch are guaranteed to live through
>> an upgrade to a newer base version of FreeBSD. It's very unfortunate,
>> that there are some patches on the 6-freebsd-12 branch only. On the long
>> term, that issue has to be resolved.
>>
> 
> I have been investigating this problem in the background, and I have
> some findings and some questions. First, I have found that there is a
> most-common ancestor between master and 6-freebsd-12 at commit
> https://git.rtems.org/rtems-libbsd/commit/?h=6-freebsd-12&id=2ce13cf6dc73855f28bc7edbbc64dc4b482a4976
> This is at least promising that the discrepancy between the branches
> can be resolved.
> 
> The proposed pending patch set to "fix" the NFS issue does not fix the
> underlying problem. Instead, it introduces further divergence between
> the branches. I would instead suggest that we should resolve to fix
> the underlying problem. I can see two paths forward.
> 
> 1. Abandon 6-freebsd-12 after releasing 6. This is probably not ideal
> since what I understand is some users have projects based on
> 6-freebsd-12 and would like an upgrade path. (I guess there is also
> the option to abandon master, which also makes little sense.)

A variant for this would be to introduce a 6-freebsd-13 that is based on 
the master branch as soon as we have one. That would allow a longer 
maintenance because FreeBSD 12 reaches it's EoL December 2023.

> 
> 2. Pull commits from 6-freebsd-12 into master to make sure master is
> the development head. in the future, reject patches that only go
> toward release branches. This has its own problems too. It can
> realistically only be done in three ways:

Please note that Sebastian mentioned that the file descriptors broke the 
NTP support (at least I think it was NTP; possible that it was another 
submodule). So picking the current version of the patches into the 
master without adding fixes makes the master unusable for some cases.

> 
> 2a: Rebase master and cherry-pick commits from 6-freebsd-12 and master
> back into master. This rewrites the history of master, and
> unfortunately will cause the head of 5-freebsd-12 and the tags for
> rtems-5 to no longer exist on the master branch. They will still exist
> in the '5' branch. The advantage is in the end there will be a linear
> history of development on master that reflects the timeline of actual
> development that spanned both branches. Theoretically, this should
> make it easier to git-bisect.
> 
> 2b: Cherry-pick commits from 6-freebsd-12 to master and fix conflicts.
> This puts all the missing commits from 6-freebsd-12 on to the current
> head of master. I don't know how messy this would be. It ends up
> making the history of master convoluted to understand, with fairly old
> commits from 2018 being placed on top of newer commits from 2020s.
> 
> 2c: Merge 6-freebsd-12 into master and fixup conflicts in the merge
> commit. This is pretty similar to 2a but ends up with a non-linear
> history and a merge commit. It may be a fairly complex merge commit.

For all of the 2x solutions: The commits from 6-freebsd-12 can't just be 
cherry-picked. You have to re-import the NFS files from the FreeBSD 
master version that is used as base for the current libbsd master. 
Otherwise we mix different FreeBSD source versions. We had that some 
time back in libbsd and Sebastian needed a lot of time cleaning that up.

> 
> To get a sense of the difference between the two branches, I have done
> the following command:
> $ git log --pretty=oneline master...6-freebsd-12 > ../log.txt
> This uses the ... (three-dot) Symmetric Difference Notation. The
> result of that is a 750 line file, so 750 commits are different
> between the two branches. Some of those commits are actually the same
> content, but they have different parents so different hashes. In a
> rebase or merge situation, those commits should end up the same. There
> may be other git-fu to find just the patches that are unique in the
> two branches.

750 commits are a bit too much. The order of patches on master and 
6-freebsd-12 isn't always the same. I wrote a small python script to 
find the differences somewhere in 2021 when I needed the differences in 
a discussion with (I think) Joel. It compares based on author and 
subject which gives a quite good estimate for libbsd. You can find the 
script here:

   https://gist.github.com/a82e21eb250cb96c3a36f107b92dab09

That shows 138 different commits. 86 are only on 6-freebsd-12. Complete 
output is here:

   https://gist.github.com/93a2b19a5bf4cd8a6263ae29aa359ac2

 From these you can ignore the "Update to FreeBSD..." commits and some 
of the cleanup patches. I assume that quite some of them can be 
cherry-picked with only small or no changes. For example patches that 
only add or update drivers in rtemsbsd should work without problems. So 
I think a more realistic number of problematic patches should be 30 to 50.

Best regards

Christian

> 
> In any case, doing this in a way that ensures the commits build and
> tests run is challenging due to the interactions with the rtems.git,
> toolchain, and the submodules. >
> After the 6-freebsd-12 and master are made consistent, then it becomes
> possible to update freebsd and also to consider what ways can be
> chosen to fix problems in 6-freebsd-12.
> 
> -Gedare
> 
>> Best regards
>>
>> Christian
>>
>>>
>>> I'm fine with either way, as a user I just need clear not confusing
>>> project message...
>>>
>>> Thanks!
>>> Karel
>>>
>>>
>>> _______________________________________________
>>> devel mailing list
>>> devel at rtems.org
>>> http://lists.rtems.org/mailman/listinfo/devel
>>
>> --
>> --------------------------------------------
>> embedded brains GmbH
>> Herr Christian MAUDERER
>> Dornierstr. 4
>> 82178 Puchheim
>> Germany
>> email:  christian.mauderer at embedded-brains.de
>> phone:  +49-89-18 94 741 - 18
>> mobile: +49-176-152 206 08
>>
>> Registergericht: Amtsgericht München
>> Registernummer: HRB 157899
>> Vertretungsberechtigte Geschäftsführer: Peter Rasmussen, Thomas Dörfler
>> Unsere Datenschutzerklärung finden Sie hier:
>> https://embedded-brains.de/datenschutzerklaerung/
>> _______________________________________________
>> devel mailing list
>> devel at rtems.org
>> http://lists.rtems.org/mailman/listinfo/devel
> _______________________________________________
> devel mailing list
> devel at rtems.org
> http://lists.rtems.org/mailman/listinfo/devel