[PATCH rtems-libbsd] CONTRIBUTING: Sharpen priority development goals

[Chris Johns]

On 6/2/2023 7:07 pm, Christian MAUDERER wrote:
> Hello Chris,
> 
> thanks for your feedback on the patch.

No problem :)

> On 2023-02-06 05:16, Chris Johns wrote:
>> On 3/2/2023 6:31 pm, Christian Mauderer wrote:
>>> This patch tries to make the most important goals of LibBSD development
>>> more clear based on the results of the discussion on the mailing list:
>>>
>>> https://lists.rtems.org/pipermail/devel/2023-January/074164.html
>>> ---
>>>   CONTRIBUTING.rst | 39 ++++++++++++++++++++++++++++++---------
>>>   1 file changed, 30 insertions(+), 9 deletions(-)
>>>
>>> diff --git a/CONTRIBUTING.rst b/CONTRIBUTING.rst
>>> index 0b6fc7a0..31561f6a 100644
>>> --- a/CONTRIBUTING.rst
>>> +++ b/CONTRIBUTING.rst
>>> @@ -21,15 +21,36 @@ RTEMS specific support files, general guidelines on what
>>> modifications to the
>>>   FreeBSD source are permitted, and some other topics.  For building the
>>> library,
>>>   see the `README <README.rst>`_.
>>>   -Goals of the LibBSD activity are
>>> -
>>> -* provide functionality from FreeBSD to RTEMS,
>>> -* ease updating to future FreeBSD versions,
>>> -* ease tracking changes in FreeBSD code,
>>> -* minimize manual changes in FreeBSD code.
>>> -
>>> -We will work to push our changes upstream to the FreeBSD Project and minimize
>>> -changes required at each update point.
>>> +Every change to LibBSD has to consider the following points in groups of
>>> +descending priority:
>>> +
>>> +#. Real-time Impacts + Maintainability Loss
>>> +   * LibBSD itself doesn't make any real time claims because it is basically
>>> +     FreeBSD and they don't make any real time claims either.  But all
>>> +     development in LibBSD must make sure that the real time ability of the
>>> +     RTEMS core system or the application is not affected.
>>> +   * It's utterly important that LibBSD is simple to maintain.  One of the most
>>> +     important points for that is that upgrades to newer FreeBSD versions have
>>> +     to be easy.
>>
>> Correct and it is important we adopt and use what FreeBSD provides rather than
>> adding extra complexity. I wrote about this in the FreeBSD journal years ago.
>> The bespoke handing of fd and file objects is something I consider an unneeded
>> complexity for specific system related reasons. We need NFSv4 and that uses VFS
>> and that in turn uses the FreeBSD fd and file objects.
>>
> 
> I'm trying to find out what rules are agreed by most of us.

I feel we need simplicity. I was surprised by the number of undefined rules my
changes tripped over and to be honest I still have no idea what they are, why
they exist and who made them?

> It's of course OK to
> bring examples. But I think we should try to evaluate the currently discussed
> patch set based on the results together with as many of the other maintainers as
> possible as soon as we agree on the rules that we want to use to evaluate them.

The rules for the changes I made are simple. It is transparency of the FreeBSD code.

> The file descriptors are an example where I know that Sebastian and you will
> argument that their solution is the right one.

He may but his view is only one view and fixed and I am sorry but the forceful
and sometimes short nature of his responses has not helped his cause.

I found the port of NFSv4 easy until I hit the fd/file parts. The complexity
then grew and as I attempted to sort one piece out I hit another location and in
the end I had little confidence about the changes I had made. Each change
started to appear like a slight variation of the same thing for each descriptor
type. In the end I decided the size of changes had grown to big so I decided to
switch direction and bring across the FreeBSD code as close as possible and to
move libbsd in the that direction because the closer we had VFS to FreeBSD the
simpler the testing of complex pieces like the vnode cache and lookup would be.

It is important to note we need code the project can maintain and as a result of
my deep dive I now question who can maintain the fd/file support as it was?

> I think that can only be decided with the help of some more people.

I agree. This is about the long term and beyond any one of us.

>> Source transparency is what matters and as a test of what is acceptable it must
>> be preferred between competing implementations.
>>
>>> +#. Transparency Loss + Modularity Loss + Code/RAM Size Increase
>>> +   * LibBSD code should be easy to read and easy to debug.  Changes should be
>>> +     integrated in a way that are easy to understand.  Of course that's a
>>> +     subjective goal.  As a general rule: If it adds lot's of extra code or
>>> even
>>> +     more layers than already exist in FreeBSD, it's harder to understand.
>>> +   * There are a number of methods used in LibBSD to make it modular.  If you
>>> +     add new functionality, use one of the existing methods to allow
>>> enabling or
>>> +     disabling your module.  For example make sure that the linker can remove
>>> +     unused modules.  If that doesn't work for your feature, try to use the
>>> +     buildsets to allow to switch a module on or off.
>>> +   * A lot of different hardware uses LibBSD as network or USB stack or
>>> maybe in
>>> +     the future even only for other subsystems.  Some of the targets have
>>> +     hundreds of megabytes memory.  Others can only have a few megabytes (like
>>> +     the ATSAMV71).  Make sure that changes don't increase the RAM / Flash size
>>> +     of the default build so that it can't be used on the small targets.
>>
>> This is not realistic or achievable and I find confusing the reasons it is being
>> pushed over and over. I would have not have agreed to this being added before
>> now and nothing has changed. The central reason for rejecting this statement is
>> a change in FreeBSD may add a few meg of memory to the footprint and this type
>> of statement would conflict with that addition and that in turn would conflict
>> with the need for transparency of source. And as stated before transparency must
>> be preferred.
> 
> Maintainability is the point that has the highest priority in my suggestion. I
> grouped transparency, modularity and size like Gedare suggested because there
> might be cases where we want to prefer one over the other.
> 
> Transparency is also a bit difficult and often subjective. I defined it here as
> 'easy to read' and 'easy to debug'. These two alone can conflict each other. For
> example more layers can sometimes make something simpler to read because it is
> nicely abstracted, but it can make it horrible hard to debug a problem because
> you have to step through all the layers.

I use the term transparency to reflect how much of the code we bring in from
FreeBSD remains unchanged. It is a crude measure because a small change we make
could be complicated but as a simple measure it tends to work. It does not
reflect the complexity of the FreeBSD code and that is a key reason we want the
code to be used as transparently as it can be. I documented all this in ..

https://freebsdfoundation.org/wp-content/uploads/2016/08/FreeBSD-and-RTEMS-Unix-in-a-Real-Time-Operating-System.pdf

What I wrote in 2016 is still valid. FYI Joel posted this link last year in
relation to this topic.

> Can you suggest a better choice or a better goal instead of the main point
> 'transparency, modularity and size' that I wanted to add here?

I am sensitive to statements like the one you have suggested. I made a large
change to bring in important changes the project needs and I have been hit first
with a patch with revert and then further rules I never knew existed and I would
have never agreed to for this project.

>> System requirements are for the developers of those systems and not RTEMS.
>> Derating designs is an important part of system design and not the domain of
>> this project. Memory constrained systems can consider another networking stack
>> option or bespoke changes internally maintained. That is a cost trade off no one
>> here can help make.
> 
> As said in the other discussion: LibBSD is more than a network stack. It started
> as a pure USB stack, and it can still be that. 

My understanding of the history of LibBSD has always been an effort to move on
from the legacy stack taking the lessons we learned. I still remember the GSoC
meeting when it happened. USB was one of the first working milestone but it was
networking and IPv6 that was the original discussion point. I pushed for libbsd
as the name so it was not just networking.

> We don't have another USB stack
> to choose from. It now can also be used as a SDMMC stack. We don't have another
> one of those too. I think LibBSD is now more a driver framework than a pure
> network stack.

I shipped a FreeBSD USB stack on a resource limited NIOS-II product years before
libbsd existed so I am across what is needed. As I said in the article it helped
shape how we approach libbsd.

> Even if we take a look at the pure network functionality: lwip was added as
> another network stack for RTEMS not too long ago. I think it's great, and it
> will be very useful for small targets that need limited network functionality.
> But does it provide stuff like VLAN, packet filter, IPSec, WiFi?

It does not and if you are going to use those features then I suggest memory and
performance is considered up front when selected the platform to use.

> I know of at least one user who uses VLAN, packet filter and IPSec on an ATSAM
> BSP and another one who uses USB and WiFi on another variant of the ATSAM. SDMMC
> is used on the STM32H7. I think pure network is used on some of the older MPC*
> or ColdFire BSPs where it replaced the legacy stack before lwip was added to RTEMS.

These are constraints I never knew existed so are you saying only EB makes major
changes to libbsd and you are the gate keeper because you are the only people
who understand these bounds? Knowing you all I doubt this is the case but this
is the optics of what you are saying.

> The users used it because it currently works well. Let's assume we just tell the
> existing users, that it won't work in the next RTEMS version any more (which is
> an unusual step for RTEMS): Where would you see the minimum requirements for
> LibBSD? We should agree on some minimum requirements and add it to the
> documentation as 'If your system has less than that: Don't use LibBSD! It might
> not work with that in the future.'

I am sorry you have lost me.

>>> +#. Performance Loss
>>> +   * There are applications, that require (for example) a high network
>>> +     throughput or a fast storage access.  Make sure to take that into account
>>> +     if adding changes.
>>
>> I prefer we do not add statements that have no definition or boundaries someone
>> could use to test against. Selection of RTEMS in a systems is the choice of the
>> system designer. There are systems RTEMS is not a good fit for.
> 
> Do we give any good guidance to a user where RTEMS is a good fit? If not: What
> should be added to the documentation? At the moment a new user would take a look
> at the supported BSPs and would see that we support low memory targets like
> STM32F4 (not with LibBSD but with RTEMS) as well as high performance targets
> with 24 cores. So he will assume that RTEMS works for all of these.  If we say
> that RTEMS is not a good fit for some of the systems, we should think about
> removing all BSPs that are lower than the minimum or higher than the maximum
> requirements. Note that this is of course a provocative statement to find the
> limits that we want to support. So please don't take it as a serious suggestion
> to remove STM32F4.

The suitability is for the system designers to answer. It is an open source
project and not a product with a product specification.

> An example for the upper end of the requirements: I know of a system with a
> 10GBit Ethernet with a high load and low delays in the processing that is
> currently supported by RTEMS. It's a real time application so RTEMS should be a
> good fit for it. Network performance is an important point for it. 

Real-time and networking? This sounds like something specialised I often refer
to as "engineered networking". Engineered networks are specialised and
engineered at all layers unlike an office or home network. Should we all be
subject to these limitation now and into the future in case there is still a need?

> If you want,
> I can find some of the requirements of that system (network packet processed in
> this many clocks, minimum throughput on platform X, ...) and add it here so that
> we have a clear boundary.
> 
> Let's assume a change reduces the network throughput of libbsd to 50%. But it
> adds a new feature for some use case: Should we really not take a look at
> performance and just add it? Where would you put the limit for performance
> impacts for LibBSD?

FreeBSD.

Chris