[PATCH rtems-libbsd] CONTRIBUTING: Sharpen priority development goals

Mon Feb 6 08:07:20 UTC 2023

Hello Chris,

thanks for your feedback on the patch.

On 2023-02-06 05:16, Chris Johns wrote:
> On 3/2/2023 6:31 pm, Christian Mauderer wrote:
>> This patch tries to make the most important goals of LibBSD development
>> more clear based on the results of the discussion on the mailing list:
>>
>> https://lists.rtems.org/pipermail/devel/2023-January/074164.html
>> ---
>>   CONTRIBUTING.rst | 39 ++++++++++++++++++++++++++++++---------
>>   1 file changed, 30 insertions(+), 9 deletions(-)
>>
>> diff --git a/CONTRIBUTING.rst b/CONTRIBUTING.rst
>> index 0b6fc7a0..31561f6a 100644
>> --- a/CONTRIBUTING.rst
>> +++ b/CONTRIBUTING.rst
>> @@ -21,15 +21,36 @@ RTEMS specific support files, general guidelines on what modifications to the
>>   FreeBSD source are permitted, and some other topics.  For building the library,
>>   see the `README <README.rst>`_.
>>   
>> -Goals of the LibBSD activity are
>> -
>> -* provide functionality from FreeBSD to RTEMS,
>> -* ease updating to future FreeBSD versions,
>> -* ease tracking changes in FreeBSD code,
>> -* minimize manual changes in FreeBSD code.
>> -
>> -We will work to push our changes upstream to the FreeBSD Project and minimize
>> -changes required at each update point.
>> +Every change to LibBSD has to consider the following points in groups of
>> +descending priority:
>> +
>> +#. Real-time Impacts + Maintainability Loss
>> +   * LibBSD itself doesn't make any real time claims because it is basically
>> +     FreeBSD and they don't make any real time claims either.  But all
>> +     development in LibBSD must make sure that the real time ability of the
>> +     RTEMS core system or the application is not affected.
>> +   * It's utterly important that LibBSD is simple to maintain.  One of the most
>> +     important points for that is that upgrades to newer FreeBSD versions have
>> +     to be easy.
> 
> Correct and it is important we adopt and use what FreeBSD provides rather than
> adding extra complexity. I wrote about this in the FreeBSD journal years ago.
> The bespoke handing of fd and file objects is something I consider an unneeded
> complexity for specific system related reasons. We need NFSv4 and that uses VFS
> and that in turn uses the FreeBSD fd and file objects.
> 

I'm trying to find out what rules are agreed by most of us. It's of 
course OK to bring examples. But I think we should try to evaluate the 
currently discussed patch set based on the results together with as many 
of the other maintainers as possible as soon as we agree on the rules 
that we want to use to evaluate them. The file descriptors are an 
example where I know that Sebastian and you will argument that their 
solution is the right one. I think that can only be decided with the 
help of some more people.

> Source transparency is what matters and as a test of what is acceptable it must
> be preferred between competing implementations.
> 
>> +#. Transparency Loss + Modularity Loss + Code/RAM Size Increase
>> +   * LibBSD code should be easy to read and easy to debug.  Changes should be
>> +     integrated in a way that are easy to understand.  Of course that's a
>> +     subjective goal.  As a general rule: If it adds lot's of extra code or even
>> +     more layers than already exist in FreeBSD, it's harder to understand.
>> +   * There are a number of methods used in LibBSD to make it modular.  If you
>> +     add new functionality, use one of the existing methods to allow enabling or
>> +     disabling your module.  For example make sure that the linker can remove
>> +     unused modules.  If that doesn't work for your feature, try to use the
>> +     buildsets to allow to switch a module on or off.
>> +   * A lot of different hardware uses LibBSD as network or USB stack or maybe in
>> +     the future even only for other subsystems.  Some of the targets have
>> +     hundreds of megabytes memory.  Others can only have a few megabytes (like
>> +     the ATSAMV71).  Make sure that changes don't increase the RAM / Flash size
>> +     of the default build so that it can't be used on the small targets.
> 
> This is not realistic or achievable and I find confusing the reasons it is being
> pushed over and over. I would have not have agreed to this being added before
> now and nothing has changed. The central reason for rejecting this statement is
> a change in FreeBSD may add a few meg of memory to the footprint and this type
> of statement would conflict with that addition and that in turn would conflict
> with the need for transparency of source. And as stated before transparency must
> be preferred.

Maintainability is the point that has the highest priority in my 
suggestion. I grouped transparency, modularity and size like Gedare 
suggested because there might be cases where we want to prefer one over 
the other.

Transparency is also a bit difficult and often subjective. I defined it 
here as 'easy to read' and 'easy to debug'. These two alone can conflict 
each other. For example more layers can sometimes make something simpler 
to read because it is nicely abstracted, but it can make it horrible 
hard to debug a problem because you have to step through all the layers.

Can you suggest a better choice or a better goal instead of the main 
point 'transparency, modularity and size' that I wanted to add here?

> 
> System requirements are for the developers of those systems and not RTEMS.
> Derating designs is an important part of system design and not the domain of
> this project. Memory constrained systems can consider another networking stack
> option or bespoke changes internally maintained. That is a cost trade off no one
> here can help make.

As said in the other discussion: LibBSD is more than a network stack. It 
started as a pure USB stack, and it can still be that. We don't have 
another USB stack to choose from. It now can also be used as a SDMMC 
stack. We don't have another one of those too. I think LibBSD is now 
more a driver framework than a pure network stack.

Even if we take a look at the pure network functionality: lwip was added 
as another network stack for RTEMS not too long ago. I think it's great, 
and it will be very useful for small targets that need limited network 
functionality. But does it provide stuff like VLAN, packet filter, 
IPSec, WiFi?

I know of at least one user who uses VLAN, packet filter and IPSec on an 
ATSAM BSP and another one who uses USB and WiFi on another variant of 
the ATSAM. SDMMC is used on the STM32H7. I think pure network is used on 
some of the older MPC* or ColdFire BSPs where it replaced the legacy 
stack before lwip was added to RTEMS.

The users used it because it currently works well. Let's assume we just 
tell the existing users, that it won't work in the next RTEMS version 
any more (which is an unusual step for RTEMS): Where would you see the 
minimum requirements for LibBSD? We should agree on some minimum 
requirements and add it to the documentation as 'If your system has less 
than that: Don't use LibBSD! It might not work with that in the future.'

> 
>> +#. Performance Loss
>> +   * There are applications, that require (for example) a high network
>> +     throughput or a fast storage access.  Make sure to take that into account
>> +     if adding changes.
> 
> I prefer we do not add statements that have no definition or boundaries someone
> could use to test against. Selection of RTEMS in a systems is the choice of the
> system designer. There are systems RTEMS is not a good fit for.

Do we give any good guidance to a user where RTEMS is a good fit? If 
not: What should be added to the documentation? At the moment a new user 
would take a look at the supported BSPs and would see that we support 
low memory targets like STM32F4 (not with LibBSD but with RTEMS) as well 
as high performance targets with 24 cores. So he will assume that RTEMS 
works for all of these.  If we say that RTEMS is not a good fit for some 
of the systems, we should think about removing all BSPs that are lower 
than the minimum or higher than the maximum requirements. Note that this 
is of course a provocative statement to find the limits that we want to 
support. So please don't take it as a serious suggestion to remove STM32F4.

An example for the upper end of the requirements: I know of a system 
with a 10GBit Ethernet with a high load and low delays in the processing 
that is currently supported by RTEMS. It's a real time application so 
RTEMS should be a good fit for it. Network performance is an important 
point for it. If you want, I can find some of the requirements of that 
system (network packet processed in this many clocks, minimum throughput 
on platform X, ...) and add it here so that we have a clear boundary.

Let's assume a change reduces the network throughput of libbsd to 50%. 
But it adds a new feature for some use case: Should we really not take a 
look at performance and just add it? Where would you put the limit for 
performance impacts for LibBSD?

Best regards

Christian

> 
> Chris

-- 
--------------------------------------------
embedded brains GmbH
Herr Christian MAUDERER
Dornierstr. 4
82178 Puchheim
Germany
email:  christian.mauderer at embedded-brains.de
phone:  +49-89-18 94 741 - 18
mobile: +49-176-152 206 08

Registergericht: Amtsgericht München
Registernummer: HRB 157899
Vertretungsberechtigte Geschäftsführer: Peter Rasmussen, Thomas Dörfler
Unsere Datenschutzerklärung finden Sie hier:
https://embedded-brains.de/datenschutzerklaerung/