[PATCH rtems-libbsd] CONTRIBUTING: Sharpen priority development goals
Christian MAUDERER
christian.mauderer at embedded-brains.de
Tue Feb 7 09:41:08 UTC 2023
Hello Chris,
On 2023-02-07 05:42, Chris Johns wrote:
> On 6/2/2023 7:07 pm, Christian MAUDERER wrote:
>> Hello Chris,
>>
>> thanks for your feedback on the patch.
>
> No problem :)
>
>> On 2023-02-06 05:16, Chris Johns wrote:
>>> On 3/2/2023 6:31 pm, Christian Mauderer wrote:
>>>> This patch tries to make the most important goals of LibBSD development
>>>> more clear based on the results of the discussion on the mailing list:
>>>>
>>>> https://lists.rtems.org/pipermail/devel/2023-January/074164.html
>>>> ---
>>>> CONTRIBUTING.rst | 39 ++++++++++++++++++++++++++++++---------
>>>> 1 file changed, 30 insertions(+), 9 deletions(-)
>>>>
>>>> diff --git a/CONTRIBUTING.rst b/CONTRIBUTING.rst
>>>> index 0b6fc7a0..31561f6a 100644
>>>> --- a/CONTRIBUTING.rst
>>>> +++ b/CONTRIBUTING.rst
>>>> @@ -21,15 +21,36 @@ RTEMS specific support files, general guidelines on what
>>>> modifications to the
>>>> FreeBSD source are permitted, and some other topics. For building the
>>>> library,
>>>> see the `README <README.rst>`_.
>>>> -Goals of the LibBSD activity are
>>>> -
>>>> -* provide functionality from FreeBSD to RTEMS,
>>>> -* ease updating to future FreeBSD versions,
>>>> -* ease tracking changes in FreeBSD code,
>>>> -* minimize manual changes in FreeBSD code.
>>>> -
>>>> -We will work to push our changes upstream to the FreeBSD Project and minimize
>>>> -changes required at each update point.
>>>> +Every change to LibBSD has to consider the following points in groups of
>>>> +descending priority:
>>>> +
>>>> +#. Real-time Impacts + Maintainability Loss
>>>> + * LibBSD itself doesn't make any real time claims because it is basically
>>>> + FreeBSD and they don't make any real time claims either. But all
>>>> + development in LibBSD must make sure that the real time ability of the
>>>> + RTEMS core system or the application is not affected.
>>>> + * It's utterly important that LibBSD is simple to maintain. One of the most
>>>> + important points for that is that upgrades to newer FreeBSD versions have
>>>> + to be easy.
>>>
>>> Correct and it is important we adopt and use what FreeBSD provides rather than
>>> adding extra complexity. I wrote about this in the FreeBSD journal years ago.
>>> The bespoke handing of fd and file objects is something I consider an unneeded
>>> complexity for specific system related reasons. We need NFSv4 and that uses VFS
>>> and that in turn uses the FreeBSD fd and file objects.
>>>
>>
>> I'm trying to find out what rules are agreed by most of us.
>
> I feel we need simplicity. I was surprised by the number of undefined rules my
> changes tripped over and to be honest I still have no idea what they are, why
> they exist and who made them?
>
>> It's of course OK to
>> bring examples. But I think we should try to evaluate the currently discussed
>> patch set based on the results together with as many of the other maintainers as
>> possible as soon as we agree on the rules that we want to use to evaluate them.
>
> The rules for the changes I made are simple. It is transparency of the FreeBSD code.
>
>> The file descriptors are an example where I know that Sebastian and you will
>> argument that their solution is the right one.
>
> He may but his view is only one view and fixed and I am sorry but the forceful
> and sometimes short nature of his responses has not helped his cause.
Then let's hope that my tendency to write long mails can help to bring
the discussion a bit forward ;-)
>
> I found the port of NFSv4 easy until I hit the fd/file parts. The complexity
> then grew and as I attempted to sort one piece out I hit another location and in
> the end I had little confidence about the changes I had made. Each change
> started to appear like a slight variation of the same thing for each descriptor
> type. In the end I decided the size of changes had grown to big so I decided to
> switch direction and bring across the FreeBSD code as close as possible and to
> move libbsd in the that direction because the closer we had VFS to FreeBSD the
> simpler the testing of complex pieces like the vnode cache and lookup would be.
>
As far as I understood Sebastian, the fd/file parts add quite a bit of
code which has to be maintained too and which duplicates part of RTEMS
core infrastructure.
Is it really simpler to understand more code and an extra layer? Maybe
that also depends on the background of the person who looks at the code.
If someone reads that code first time: Let's assume he or she has worked
with FreeBSD sources before. In that case it might is simpler to
understand the code if the FreeBSD file descriptors are in the sources.
Let's assume the other case: He or she has worked only with small real
time systems. In that case it might is quite odd that there are two code
parts that more or less do the same and handle files.
It is a trade-off whether it's simpler to cut above the fd/file parts or
below them. And most likely it's one that is very subjective.
> It is important to note we need code the project can maintain and as a result of
> my deep dive I now question who can maintain the fd/file support as it was?
>
You could ask the same question: Who (except for you) can maintain
fd/file support as it is now on 6-freebsd-12? That question can most
likely only be answered if someone who isn't influenced too much by one
of the solutions takes a look at that problem.
>> I think that can only be decided with the help of some more people.
>
> I agree. This is about the long term and beyond any one of us.
>
>>> Source transparency is what matters and as a test of what is acceptable it must
>>> be preferred between competing implementations.
>>>
>>>> +#. Transparency Loss + Modularity Loss + Code/RAM Size Increase
>>>> + * LibBSD code should be easy to read and easy to debug. Changes should be
>>>> + integrated in a way that are easy to understand. Of course that's a
>>>> + subjective goal. As a general rule: If it adds lot's of extra code or
>>>> even
>>>> + more layers than already exist in FreeBSD, it's harder to understand.
>>>> + * There are a number of methods used in LibBSD to make it modular. If you
>>>> + add new functionality, use one of the existing methods to allow
>>>> enabling or
>>>> + disabling your module. For example make sure that the linker can remove
>>>> + unused modules. If that doesn't work for your feature, try to use the
>>>> + buildsets to allow to switch a module on or off.
>>>> + * A lot of different hardware uses LibBSD as network or USB stack or
>>>> maybe in
>>>> + the future even only for other subsystems. Some of the targets have
>>>> + hundreds of megabytes memory. Others can only have a few megabytes (like
>>>> + the ATSAMV71). Make sure that changes don't increase the RAM / Flash size
>>>> + of the default build so that it can't be used on the small targets.
>>>
>>> This is not realistic or achievable and I find confusing the reasons it is being
>>> pushed over and over. I would have not have agreed to this being added before
>>> now and nothing has changed. The central reason for rejecting this statement is
>>> a change in FreeBSD may add a few meg of memory to the footprint and this type
>>> of statement would conflict with that addition and that in turn would conflict
>>> with the need for transparency of source. And as stated before transparency must
>>> be preferred.
>>
>> Maintainability is the point that has the highest priority in my suggestion. I
>> grouped transparency, modularity and size like Gedare suggested because there
>> might be cases where we want to prefer one over the other.
>>
>> Transparency is also a bit difficult and often subjective. I defined it here as
>> 'easy to read' and 'easy to debug'. These two alone can conflict each other. For
>> example more layers can sometimes make something simpler to read because it is
>> nicely abstracted, but it can make it horrible hard to debug a problem because
>> you have to step through all the layers.
>
> I use the term transparency to reflect how much of the code we bring in from
> FreeBSD remains unchanged. It is a crude measure because a small change we make
> could be complicated but as a simple measure it tends to work. It does not
> reflect the complexity of the FreeBSD code and that is a key reason we want the
> code to be used as transparently as it can be. I documented all this in ..
>
> https://freebsdfoundation.org/wp-content/uploads/2016/08/FreeBSD-and-RTEMS-Unix-in-a-Real-Time-Operating-System.pdf
>
> What I wrote in 2016 is still valid. FYI Joel posted this link last year in
> relation to this topic.
Thanks for the article. You give a very specific definition of "source
transparency" there as the amount of code that is changed. I might have
used transparency different in the discussion as 'easy to read /
understand / debug'. So we clearly used that differently which might has
lead to misunderstandings. I'll try to adapt to your definition starting
from this mail.
Off-topic question: The article contains some numbers like an 'opacity
level of 1.6%'. Do we have tools to check these numbers or where they
calculated by hand? It would be interesting to have such tools to
analyze different subsystems of LibBSD.
Back to topic: Beneath transparency (fewest possible changes): Isn't it
also one goal to bring in only the absolutely necessary parts so that
there is fewer code that has to be maintained? Otherwise, we could
theoretically end up with a complete FreeBSD kernel running in parallel
to RTEMS, and it would be better based on only the transparency metric
because it has fewer changes. More code would even improve the
transparency metric as long as it is not changed.
The article also mentions, that there had always been a focus on small
executable size on page 5 in the sysinit area. It mentions modularity
multiple times, which is also part of allowing small configurations.
You more or less give FreeBSD system requirements as a minimum
recommended system further below. So did the goal of a small executable
size change from your point of view? If we target only big systems that
can run FreeBSD neither size nor modularity is important.
>
>> Can you suggest a better choice or a better goal instead of the main point
>> 'transparency, modularity and size' that I wanted to add here?
>
> I am sensitive to statements like the one you have suggested. I made a large
> change to bring in important changes the project needs and I have been hit first
> with a patch with revert and then further rules I never knew existed and I would
> have never agreed to for this project.
>
A few years back I added the build sets with a lot of support from you
(thanks again for all the help). One of the goals was that it is
possible to bring in IPSec. Enabling IPSec adds a processing overhead to
every network package. That would have reduced the performance of every
BSP quite significantly if it had been enabled for all cases. We could
have decided that we only target FreeBSD capable systems back then and
just enable it for everyone because processing power is not a problem on
these systems.
Another goal of the buildsets was to add a minimal buildset. I don't
want to search and cite any mails, but I think you worked with me
(together with some others) to define the goals for these buildsets. For
minimal we defined the following (in the header of the file):
#
# This configuration has the target to provide the smallest possible libbsd
# during link time. It should disable everything that can be disabled
without
# loosing basic functionality. As a target, it should once only provide the
# following functions:
#
# - basic IPv4 only networking
# - basic USB support
# - all device drivers that don't increase the application size without
being
# explicitly linked in
#
# ATTENTION: This configuration will loose functionality in the future
as soon
# as it is possible to disable the functionality.
#
In the discussion the target was that it should be (in the optimal case)
a replacement for the legacy stack. So there was a clear goal that
LibBSD should be able to run on similar systems like the legacy stack.
We never fully reached that goal, but I would still see it as relevant.
>>> System requirements are for the developers of those systems and not RTEMS.
>>> Derating designs is an important part of system design and not the domain of
>>> this project. Memory constrained systems can consider another networking stack
>>> option or bespoke changes internally maintained. That is a cost trade off no one
>>> here can help make.
>>
>> As said in the other discussion: LibBSD is more than a network stack. It started
>> as a pure USB stack, and it can still be that.
>
> My understanding of the history of LibBSD has always been an effort to move on
> from the legacy stack taking the lessons we learned. I still remember the GSoC
> meeting when it happened. USB was one of the first working milestone but it was
> networking and IPv6 that was the original discussion point. I pushed for libbsd
> as the name so it was not just networking.
>
Which is one point for modularity and size: We should make sure that
LibBSD still runs in as many configurations as possible on as many BSPs
as possible. We will only be able to maintain a small amount of defined
configurations. But we should try to still allow others.
>> We don't have another USB stack
>> to choose from. It now can also be used as a SDMMC stack. We don't have another
>> one of those too. I think LibBSD is now more a driver framework than a pure
>> network stack.
>
> I shipped a FreeBSD USB stack on a resource limited NIOS-II product years before
> libbsd existed so I am across what is needed. As I said in the article it helped
> shape how we approach libbsd.
>
Yes, and the article clearly states that it is one goal of LibBSD to
collect the separated ports like USB, Network, ... into one library.
Should we drop the ability to use LibBSD as only a USB stack on some
small BSP because the parts are collected now?
I would rather see a goal that LibBSD should become more modular. In
another discussion someone mentioned that it would be interesting for
lwIP to support FreeBSD network drivers. In my opinion it would be
optimal if libBSD could be configured in a way that it can compile these
drivers. That would be maximum modularity.
>> Even if we take a look at the pure network functionality: lwip was added as
>> another network stack for RTEMS not too long ago. I think it's great, and it
>> will be very useful for small targets that need limited network functionality.
>> But does it provide stuff like VLAN, packet filter, IPSec, WiFi?
>
> It does not and if you are going to use those features then I suggest memory and
> performance is considered up front when selected the platform to use.
>
We never documented minimum requirements for LibBSD as far as I know.
There is no guidance what should be considered a good platform to use if
you want to have feature X. Currently users (as well as we at embedded
brains) do just what works well: Use it for targets with only a few
megabytes of Flash and RAM.
>> I know of at least one user who uses VLAN, packet filter and IPSec on an ATSAM
>> BSP and another one who uses USB and WiFi on another variant of the ATSAM. SDMMC
>> is used on the STM32H7. I think pure network is used on some of the older MPC*
>> or ColdFire BSPs where it replaced the legacy stack before lwip was added to RTEMS.
>
> These are constraints I never knew existed so are you saying only EB makes major
> changes to libbsd and you are the gate keeper because you are the only people
> who understand these bounds? Knowing you all I doubt this is the case but this
> is the optics of what you are saying.
>
I don't think that embedded brains should be the only maintainer of
LibBSD. A software project grows stronger by more users and maintainers.
But you make some assumptions about the accepted minimal requirements
too. You currently assume the minimum requirement is something similar
to a system that could run FreeBSD.
If there is no definition that all agreed before: Shouldn't existing use
cases define a minimum that should only be changed after a discussion?
I'm sure there are a lot of applications for RTEMS or LibBSD that both
of us don't know anything about. For example I have no idea which
platforms Joel uses to run LibBSD. But if we make a change, and he tells
us that it breaks one of his use cases, we most likely have to discuss
whether we can make another, better change that allows these use cases.
We have a lot of unwritten requirements in RTEMS. Isn't it usual to find
a way to support all cases that we know of?
>> The users used it because it currently works well. Let's assume we just tell the
>> existing users, that it won't work in the next RTEMS version any more (which is
>> an unusual step for RTEMS): Where would you see the minimum requirements for
>> LibBSD? We should agree on some minimum requirements and add it to the
>> documentation as 'If your system has less than that: Don't use LibBSD! It might
>> not work with that in the future.'
>
> I am sorry you have lost me.
>
Core of that was: If we don't write down any requirements, users and
developers will use a system for whatever it works well. Defining new
requirements later - just because we never intended the software for
that use case - will make problems. Just like the one we are currently
discussing.
>>>> +#. Performance Loss
>>>> + * There are applications, that require (for example) a high network
>>>> + throughput or a fast storage access. Make sure to take that into account
>>>> + if adding changes.
>>>
>>> I prefer we do not add statements that have no definition or boundaries someone
>>> could use to test against. Selection of RTEMS in a systems is the choice of the
>>> system designer. There are systems RTEMS is not a good fit for.
>>
>> Do we give any good guidance to a user where RTEMS is a good fit? If not: What
>> should be added to the documentation? At the moment a new user would take a look
>> at the supported BSPs and would see that we support low memory targets like
>> STM32F4 (not with LibBSD but with RTEMS) as well as high performance targets
>> with 24 cores. So he will assume that RTEMS works for all of these. If we say
>> that RTEMS is not a good fit for some of the systems, we should think about
>> removing all BSPs that are lower than the minimum or higher than the maximum
>> requirements. Note that this is of course a provocative statement to find the
>> limits that we want to support. So please don't take it as a serious suggestion
>> to remove STM32F4.
>
> The suitability is for the system designers to answer. It is an open source
> project and not a product with a product specification.
>
So we don't give any guides to decide. But we also don't care for
existing applications because a system designer should have estimated
that we might change stuff?
>> An example for the upper end of the requirements: I know of a system with a
>> 10GBit Ethernet with a high load and low delays in the processing that is
>> currently supported by RTEMS. It's a real time application so RTEMS should be a
>> good fit for it. Network performance is an important point for it.
>
> Real-time and networking? This sounds like something specialised I often refer
> to as "engineered networking". Engineered networks are specialised and
> engineered at all layers unlike an office or home network. Should we all be
> subject to these limitation now and into the future in case there is still a need?
>
That's one application where the stack is pushed to its limits. I didn't
want to define a hard limit. Therefore, I only wrote a very soft
statement. You asked for a hard limit and I provided an example that
could be one.
>> If you want,
>> I can find some of the requirements of that system (network packet processed in
>> this many clocks, minimum throughput on platform X, ...) and add it here so that
>> we have a clear boundary.
>>
>> Let's assume a change reduces the network throughput of libbsd to 50%. But it
>> adds a new feature for some use case: Should we really not take a look at
>> performance and just add it? Where would you put the limit for performance
>> impacts for LibBSD?
>
> FreeBSD.
>
Caution! The following is a deliberately exaggerated statement that I
don't want to implement: In that case we maybe should remove the
buildsets and just include everything into the base. It will make
maintenance simpler and FreeBSD most likely has WLAN and IPSec enabled
by default anyway.
We don't reach FreeBSD performance for a lot of targets. Others are
faster than it. On some targets it's not even possible to compare with
FreeBSD because the targets can't run it due to a missing MMU (like
i.MXRT which is otherwise a performant system) or maybe because FreeBSD
just hasn't ported to that platform. How could we use FreeBSD for these
as a performance indicator?
I would rather have a rule that says that if a change has a relevant
influence on performance, it should be thoroughly discussed. Even if it
is a soft statement.
Best regards
Christian
> Chris
--
--------------------------------------------
embedded brains GmbH
Herr Christian MAUDERER
Dornierstr. 4
82178 Puchheim
Germany
email: christian.mauderer at embedded-brains.de
phone: +49-89-18 94 741 - 18
mobile: +49-176-152 206 08
Registergericht: Amtsgericht München
Registernummer: HRB 157899
Vertretungsberechtigte Geschäftsführer: Peter Rasmussen, Thomas Dörfler
Unsere Datenschutzerklärung finden Sie hier:
https://embedded-brains.de/datenschutzerklaerung/
More information about the devel
mailing list