Prioritizing and documenting libbsd development goals (was: libbsd updates)
Christian MAUDERER
christian.mauderer at embedded-brains.de
Mon Jan 23 13:33:07 UTC 2023
Hello,
like suggested earlier in the original discussion I would suggest to
prioritize our development goals for libbsd and later evaluate the two
problems discussed in the thread based on this. Let's first agree on
development goals (that also can be added to the libbsd documentation).
From the thread and responses, I currently have collected the following
goals / priorities. I replaced my original extensibility and
debuggability with Joels transparency because it's a good summary term
for these two points.
From highest to lowest priority:
- Maintainability: How easy is it for the people doing the main
maintenance tasks to do that work.
- Transparency: How easy it is to understand the code? Relevant for
extending and debugging.
- Code and RAM sizes (or other hardware requirements): Whether we meet
the minimum hardware requirements.
- Modularity: How well and easy the system can be adapted to target
applications. Have only few official ways to enable / disable modules in
the subsystem.
- Real time capabilities: No hard real time requirements for libbsd core
but we have to make sure that it doesn't have a negative impact on other
subsystems.
- Performance: Whether libbsd performes well enough in the typical use
cases.
That means (for example): If it makes the system more maintainable, the
performance isn't that important. If it reduces the size, we can trade
off some performance or real time capabilities.
Would you prioritize different? Did I miss some additional points from
the discussion?
Best regards
Christian
On 2023-01-20 08:32, Christian MAUDERER wrote:
> Hello,
>
> recently an internal discussion about updates in the libbsd started. All
> who participated in the discussion agreed that we should move the
> discussion to a public one. Below you can find the mail thread. It's
> oldest mail first. Mails are split with lines of = signs. I removed some
> duplicated mail text in case there were no inline comments. The
> date/time of the mails are in my current locale which is UTC+1.
>
> Best regards
>
> Christian
>
> =====================================================================
> On 2023-01-03 16:52, Thomas DOERFLER wrote:
>> Hello,
>>
>> first of all I wish you all a healthy, happy and successful new year.
>>
>> Unfortunately I already have to re-raise an issue: One of our customers
>> has asked us about the RTEMS libbsd update policy. Since we still use a
>> rather old libbsd version, it contains "OpenSSL 1.1.1d-freebsd 10 Sep
>> 2019", there are many openSSL fixes missing when using RTEMS, and this
>> most likely is true for other parts of libbsd.
>>
>> IMHO we should find a way to overcome the current libbsd update blocking
>> points and try to stay close to the FreeBSD code (maybe in a 1-3 month
>> interval).
>>
>> AFAIK the big blocking point are the different views around changes of
>> the file descriptor handling between RTEMS and BSD (this may be a bit
>> off the real topic, I am not yet into the details). In the next week I
>> would like to get things rolling to see the pros and cons of both
>> possible implementations and I would be happy if those closely involved
>> would support this.
>>
>> Apart from the current blocking point, can we agree that staying close
>> the the FreeBSD code (with a 1-3 month update/sync interval) would be
>> desirable?
>>
>> Kind regards to you all,
>>
>> Thomas.
>>
>
> =====================================================================
> On 2023-01-12 01:24, Gedare Bloom wrote:
>> Hi Thomas,
>>
>> I think the goal you have stated is laudable and fits with the initial
>> design goals of libbsd. I will take a closer look at this topic and
>> report back soon, I hope. I have remained (purposefully) ignorant
>> about libbsd evolution over time, but I guess it is time for me to
>> learn a bit more and catch up.
>>
>> From what I have understood about the current blocking issue, it has
>> to do with the changes that were made to use FreeBSD File Descriptors
>> and the VFS. However, one of the main problems that was noted actually
>> appears to be just related to the size increase caused by the syscall
>> glue implemented in a single .c file. So, I will plan to start my
>> libbsd learning adventure by trying to split that .c file apart into
>> separate build components to see how/if that alleviates the linked
>> image size. If successful, that should get us closer.
>>
>> I think we can then revisit whether or not the FreeBSD File
>> Descriptors themselves are in fact problematic.
>>
>> It would also be helpful to decide if we want to clarify any
>> requirements for libbsd maintenance. So, for example, the ability to
>> keep it updated on a rolling basis would require good automation and
>> validation that changes/updates remain usable. It would appear that
>> there are some minimum resource requirements that some users of libbsd
>> would like to remain under if possible. To ensure that would require
>> establishing what platform(s) have resource requirements for using
>> libbsd, and adding configurations/testing to check for when those
>> resources are exceeded by an update. Because it is entirely plausible
>> that an update pulls in changes from upstream that causes the
>> resources to grow.
>>
>> Gedare
>
> =====================================================================
> On 2023-01-12 08:46, Sebastian Huber wrote:
>> Hello Gedare,
>>
>> On 12.01.23 01:24, Gedare Bloom wrote:
>>> Hi Thomas,
>>>
>>> I think the goal you have stated is laudable and fits with the initial
>>> design goals of libbsd. I will take a closer look at this topic and
>>> report back soon, I hope. I have remained (purposefully) ignorant
>>> about libbsd evolution over time, but I guess it is time for me to
>>> learn a bit more and catch up.
>>
>> yes, one of the goals of libbsd was to follow the FreeBSD upstream,
>> however, I was the only one doing this work in the past and I was busy
>> with other things in the last three years (mainly the RTEMS
>> pre-qualification). We need a more automated way to keep libbsd up to
>> date. My proposal is to focus on the master branch since the FreeBSD EOL
>> is 31. December 2023. We could use FreeBSD 14 for the next production
>> branch. The current libbsd synchronization support was designed to do
>> updates from Subversion. With Git we can do things more efficiently. I
>> would replace the libbsd build system with the one from RTEMS (waf +
>> spec items). With the spec items we can more easily read the data in
>> other scripts. For continuous and automatic updates I would do the
>> following:
>>
>> 1. Check out the FreeBSD X branch.
>>
>> 2. Move all FreeBSD top-level directories under the new "freebsd"
>> directory and merge in all updates from FreeBSD.
>>
>> 3. Gather the set of files imported from FreeBSD. Name the set F.
>>
>> 4. Pick a start FreeBSD commit C.
>>
>> 5. Make a patch from C to C + 1 for the file set F.
>>
>> 6. Cherry pick the patch to the associated libbsd branch, if this fails,
>> send a message and stop.
>>
>> 7. Build libbsd for the reference BSP, if this fails, send a message and
>> stop.
>>
>> 8. Run the tests for the reference BSP, if this fails, send a message
>> and stop.
>>
>> 9. Set C to C + 1, go to 5.
>>
>> Update problems can be more easily fixed if you just have a single
>> commit. This approach makes it easier to fix things directly in FreeBSD
>> and then get the fixes automatically into libbsd. Individual commits
>> make it possible to use git bisect.
>>
>> I already started to work on this.
>>
>> https://github.com/sebhub/rtems-libbsd/tree/master-update
>>
>> I updated the FreeBSD baseline from 2019-09-24 to 2020-02-09 (again no
>> issues with the system calls). The problem areas during updates are UMA,
>> sleep queues, callouts, and the introduction of SMP optimizations
>> through per-processor data structures. I added the ntpd support and
>> started with the integration of the VFS/NFS support. This was not easy
>> since FreeBSD was quite active in these five months and touch a lot of
>> code in the VFS/NFS area.
>>
>> All the embedded brains contributions are in the 6-freebsd-12 and the
>> master branch. This is not the case for other contributions. If I
>> proceed with the update of the master branch moving this stuff from the
>> 6-freebsd-12 branch to master will get more difficult.
>>
>>>
>>> From what I have understood about the current blocking issue, it has
>>> to do with the changes that were made to use FreeBSD File Descriptors
>>> and the VFS. However, one of the main problems that was noted actually
>>> appears to be just related to the size increase caused by the syscall
>>> glue implemented in a single .c file. So, I will plan to start my
>>> libbsd learning adventure by trying to split that .c file apart into
>>> separate build components to see how/if that alleviates the linked
>>> image size. If successful, that should get us closer.
>>
>> I have two issues:
>>
>> 1. The system calls in a single RTEMS-specific file.
>>
>> 2. The use of FreeBSD file descriptors.
>>
>> Both issues are fixed by the patch set. I don't see a benefit in
>> splitting the RTEMS-specific file into one file per system call. The
>> original approach is still the best approach. I don't know why it was
>> changed without a discussion on the RTEMS mailing list. It worked for
>> years, it allows compiler optimizations, it is easy to review, and there
>> are no issues during updates.
>>
>> I updated the patch set so that it doesn't require changes in RTEMS:
>>
>> https://github.com/sebhub/rtems-libbsd/tree/filedesc
>>
>>>
>>> I think we can then revisit whether or not the FreeBSD File
>>> Descriptors themselves are in fact problematic.
>>
>> File descriptors on top of file descriptors make little sense. Also
>> FreeBSD has some implementation overheads which are unnecessary in the
>> RTEMS context in which the maximum file descriptor count is fixed at
>> link time.
>>
>>>
>>> It would also be helpful to decide if we want to clarify any
>>> requirements for libbsd maintenance. So, for example, the ability to
>>> keep it updated on a rolling basis would require good automation and
>>> validation that changes/updates remain usable. It would appear that
>>> there are some minimum resource requirements that some users of libbsd
>>> would like to remain under if possible. To ensure that would require
>>> establishing what platform(s) have resource requirements for using
>>> libbsd, and adding configurations/testing to check for when those
>>> resources are exceeded by an update. Because it is entirely plausible
>>> that an update pulls in changes from upstream that causes the
>>> resources to grow.
>>
>> Yes, it is possible that changes in FreeBSD lead to a higher resource
>> usage and it would be good to have tests which catch this. However, the
>> current problems were not caused by FreeBSD changes. The really big
>> issue is that we didn't manage to integrate FreeBSD changes in the last
>> three years.
>>
>
> =====================================================================
> On 2023-01-12 19:12, Gedare Bloom wrote:
>> On Thu, Jan 12, 2023 at 12:46 AM Sebastian Huber
>> <sebastian.huber at embedded-brains.de> wrote:
>>>
>>> Hello Gedare,
>>>
>>> On 12.01.23 01:24, Gedare Bloom wrote:
>>>> Hi Thomas,
>>>>
>>>> I think the goal you have stated is laudable and fits with the initial
>>>> design goals of libbsd. I will take a closer look at this topic and
>>>> report back soon, I hope. I have remained (purposefully) ignorant
>>>> about libbsd evolution over time, but I guess it is time for me to
>>>> learn a bit more and catch up.
>>>
>>> yes, one of the goals of libbsd was to follow the FreeBSD upstream,
>>> however, I was the only one doing this work in the past and I was busy
>>> with other things in the last three years (mainly the RTEMS
>>> pre-qualification). We need a more automated way to keep libbsd up to
>>> date. My proposal is to focus on the master branch since the FreeBSD EOL
>>> is 31. December 2023. We could use FreeBSD 14 for the next production
>>> branch. The current libbsd synchronization support was designed to do
>>> updates from Subversion. With Git we can do things more efficiently. I
>>> would replace the libbsd build system with the one from RTEMS (waf +
>>> spec items). With the spec items we can more easily read the data in
>>> other scripts. For continuous and automatic updates I would do the
>>> following:
>>>
>>> 1. Check out the FreeBSD X branch.
>>>
>>> 2. Move all FreeBSD top-level directories under the new "freebsd"
>>> directory and merge in all updates from FreeBSD.
>>>
>>> 3. Gather the set of files imported from FreeBSD. Name the set F.
>>>
>>> 4. Pick a start FreeBSD commit C.
>>>
>>> 5. Make a patch from C to C + 1 for the file set F.
>>>
>>> 6. Cherry pick the patch to the associated libbsd branch, if this fails,
>>> send a message and stop.
>>>
>>> 7. Build libbsd for the reference BSP, if this fails, send a message and
>>> stop.
>>>
>>> 8. Run the tests for the reference BSP, if this fails, send a message
>>> and stop.
>>>
>>> 9. Set C to C + 1, go to 5.
>>>
>>> Update problems can be more easily fixed if you just have a single
>>> commit. This approach makes it easier to fix things directly in FreeBSD
>>> and then get the fixes automatically into libbsd. Individual commits
>>> make it possible to use git bisect.
>>>
>>> I already started to work on this.
>>>
>>> https://github.com/sebhub/rtems-libbsd/tree/master-update
>>>
>> Thanks for that description. It makes sense, and I think this is a
>> good direction for us to be heading. However, we do have to resolve
>> the "fork" that currently exists between master and 6-freebsd-12.
>>
>>> I updated the FreeBSD baseline from 2019-09-24 to 2020-02-09 (again no
>>> issues with the system calls). The problem areas during updates are UMA,
>>> sleep queues, callouts, and the introduction of SMP optimizations
>>> through per-processor data structures. I added the ntpd support and
>>> started with the integration of the VFS/NFS support. This was not easy
>>> since FreeBSD was quite active in these five months and touch a lot of
>>> code in the VFS/NFS area.
>>>
>>> All the embedded brains contributions are in the 6-freebsd-12 and the
>>> master branch. This is not the case for other contributions. If I
>>> proceed with the update of the master branch moving this stuff from the
>>> 6-freebsd-12 branch to master will get more difficult.
>>>
>> Yes, we need to resolve this issue. Thank you for having patience so
>> far with it. I have not been able to put much thought or energy to it
>> before now.
>>
>>>>
>>>> From what I have understood about the current blocking issue, it has
>>>> to do with the changes that were made to use FreeBSD File Descriptors
>>>> and the VFS. However, one of the main problems that was noted actually
>>>> appears to be just related to the size increase caused by the syscall
>>>> glue implemented in a single .c file. So, I will plan to start my
>>>> libbsd learning adventure by trying to split that .c file apart into
>>>> separate build components to see how/if that alleviates the linked
>>>> image size. If successful, that should get us closer.
>>>
>>> I have two issues:
>>>
>>> 1. The system calls in a single RTEMS-specific file.
>>>
>>> 2. The use of FreeBSD file descriptors.
>>>
>>> Both issues are fixed by the patch set. I don't see a benefit in
>>> splitting the RTEMS-specific file into one file per system call. The
>>> original approach is still the best approach. I don't know why it was
>>
>> It looks to me like it is reverted rather than fixed. Since Chris
>> apparently found it necessary to change, it is best if we can resolve
>> the difference between what he needed and what you need. Simply
>> returning back to the prior state is not respectful to him. Similarly,
>> to be respectful to you, we should understand the pros/cons of each
>> approach and try to resolve them. I am willing to do some of this work
>> on a voluntary (unpaid) basis to help the community move forward, but
>> I suspect it will eventually require compromise (and code) from both
>> of you.
>>
>> From what I have seen, the reason that the syscalls were consolidated
>> was to make it easier to work across related syscalls. Maintainability
>> is decided by the *next* person to work on the code. It looks to me
>> like it is considerably easier to find the RTEMS syscall layer in the
>> consolidated approach, and to work across layers.
>>
>> There is also a performance benefit to allow the compiler to optimize
>> the syscall away, but that also impacts debuggability if our syscall
>> layer disappears. The tradeoffs between maintainability and
>> debuggability vs performance appear to be fairly complicated in this
>> space. There's not a great description to me yet of what is required
>> for performance, while the documentation of libbsd has mostly favored
>> maintainability as a primary objective. So I think that any argument
>> to do something for performance reasons has to be accompanied by clear
>> evidence that the performance improvement matters.
>>
>>> changed without a discussion on the RTEMS mailing list. It worked for
>>> years, it allows compiler optimizations, it is easy to review, and there
>>> are no issues during updates.
>>>
>> I think the change was announced:
>> https://lists.rtems.org/pipermail/devel/2021-July/068573.html
>>
>> This is just another outcome of our current way of handling change
>> management. Changes can easily be lost in the email exchange.
>>
>>> I updated the patch set so that it doesn't require changes in RTEMS:
>>>
>>> https://github.com/sebhub/rtems-libbsd/tree/filedesc
>>>
>>>>
>>>> I think we can then revisit whether or not the FreeBSD File
>>>> Descriptors themselves are in fact problematic.
>>>
>>> File descriptors on top of file descriptors make little sense. Also
>>> FreeBSD has some implementation overheads which are unnecessary in the
>>> RTEMS context in which the maximum file descriptor count is fixed at
>>> link time.
>>>
>>
>> Are there any performance analyses that quantify the impact of the
>> implementation overhead of FreeBSD File Descriptors? Again, this is
>> about the tradeoff between maintainability and performance. i can't
>> make value judgments about performance without understanding the
>> resource constraints of the target platforms, and the performance
>> metrics (code size, heap size, throughput?) that are used to decide if
>> an optimization is worth sacrificing maintainability. It has always
>> been my understanding that the libbsd strives for maintainability
>> first and foremost. So to unilaterally reject code changes that reduce
>> the maintenance burden (by keeping us closer to the vanilla FreeBSD
>> code base) in the name of performance requires two things:
>> 1. Evidence that the performance improvement is substantial.
>> 2. Argument that the implementation of the performance improvement is
>> the most maintainable way to achieve it.
>>
>> So far, I haven't seen either of these with respect to the runtime
>> overheads. And I haven't seen a good argument that injecting the
>> syscall implementations into FreeBSD is better than having them live
>> separately in the rtemsbsd layer.
>>
>
> =====================================================================
> On 2023-01-13 08:26, Sebastian Huber wrote:
>> On 12.01.23 19:12, Gedare Bloom wrote:
>>>> I have two issues:
>>>>
>>>> 1. The system calls in a single RTEMS-specific file.
>>>>
>>>> 2. The use of FreeBSD file descriptors.
>>>>
>>>> Both issues are fixed by the patch set. I don't see a benefit in
>>>> splitting the RTEMS-specific file into one file per system call. The
>>>> original approach is still the best approach. I don't know why it was
>>> It looks to me like it is reverted rather than fixed. Since Chris
>>> apparently found it necessary to change, it is best if we can resolve
>>> the difference between what he needed and what you need. Simply
>>> returning back to the prior state is not respectful to him. Similarly,
>>> to be respectful to you, we should understand the pros/cons of each
>>> approach and try to resolve them. I am willing to do some of this work
>>> on a voluntary (unpaid) basis to help the community move forward, but
>>> I suspect it will eventually require compromise (and code) from both
>>> of you.
>>
>> I would not have looked into that if I had not experienced two problems:
>>
>> 1. The ntpd crashed somewhere in the area of file descriptors. I tried
>> to debug this, but the code is too complex compared to just using the
>> RTEMS file descriptors. This is why I removed the FreeBSD file
> descriptors.
>>
>> 2. I updated a project using the old network stack and libusb to just
>> use libbsd. Here I have a RAM limitation of 16MiB. Wasting at least 4MiB
>> for unused stuff is not that great. This lead to the system call
>> aggregation issue.
>>
>>>
>>> From what I have seen, the reason that the syscalls were consolidated
>>> was to make it easier to work across related syscalls.
>>
>> What do you mean with "work across related syscalls"? For example, the
>> socket related system call implementations are in uipc_syscalls.c. Here
>> we had:
>>
>> #ifdef __rtems__
>> static
>> #endif /* __rtems__ */
>> int
>> sys_sendto(struct thread *td, struct sendto_args *uap)
>> {
>> struct msghdr msg;
>> struct iovec aiov;
>>
>> msg.msg_name = __DECONST(void *, uap->to);
>> msg.msg_namelen = uap->tolen;
>> msg.msg_iov = &aiov;
>> msg.msg_iovlen = 1;
>> msg.msg_control = 0;
>> #ifdef COMPAT_OLDSOCK
>> if (SV_PROC_FLAG(td->td_proc, SV_AOUT))
>> msg.msg_flags = 0;
>> #endif
>> aiov.iov_base = __DECONST(void *, uap->buf);
>> aiov.iov_len = uap->len;
>> return (sendit(td, uap->s, &msg, uap->flags));
>> }
>> #ifdef __rtems__
>> ssize_t
>> sendto(int socket, const void *message, size_t length, int flags,
>> const struct sockaddr *dest_addr, socklen_t dest_len)
>> {
>> struct thread *td = rtems_bsd_get_curthread_or_null();
>> struct sendto_args ua = {
>> .s = socket,
>> .buf = (caddr_t) message,
>> .len = length,
>> .flags = flags,
>> .to = dest_addr,
>> .tolen = dest_len
>> };
>> int error;
>>
>> if (td != NULL) {
>> error = sys_sendto(td, &ua);
>> } else {
>> error = ENOMEM;
>> }
>>
>> if (error == 0) {
>> return td->td_retval[0];
>> } else {
>> rtems_set_errno_and_return_minus_one(error);
>> }
>> }
>> [...]
>> #endif /* __rtems__ */
>>
>> The sys_sendto() is unmodified and the RTEMS adoption is below the
>> implementation. This is easy to review and understand. Everything is in
>> one place. There are no issues during updates and I know what I am
>> talking about:
>>
>> git log ./freebsd/sys/kern/uipc_syscalls.c | grep 'Update to F'
>> Update to FreeBSD stable/12 2019-09-23
>> Update to FreeBSD stable/12 2019-02-11
>> Update to FreeBSD stable/12 2019-01-16
>> Update to FreeBSD head 2018-11-15
>> Update to FreeBSD head 2018-09-17
>> Update to FreeBSD head 2018-06-01
>> Update to FreeBSD head 2018-04-01
>> Update to FreeBSD head 2017-12-01
>> Update to FreeBSD head 2017-08-01
>> Update to FreeBSD head 2017-04-04
>> Update to FreeBSD head 2016-12-10
>> Update to FreeBSD head 2016-08-23
>> Update to FreeBSD 9.3
>> Update to FreeBSD 9.2
>> Update to FreeBSD 8.4
>>
>> In general, the system call related area is not an issue during updates
>> since the system call interface is pretty stable. System call stability
>> is a very important thing for FreeBSD/Linux/etc.
>>
>>> Maintainability
>>> is decided by the*next* person to work on the code.
>>
>> So far I am the only one who expressed plans to synchronize libbsd with
>> the upstream FreeBSD development.
>>
>>> It looks to me
>>> like it is considerably easier to find the RTEMS syscall layer in the
>>> consolidated approach, and to work across layers.
>>
>> Moving things to a separate file instead of having it directly under the
>> implementation function is considerably easier to find?
>>
>>>
>>> There is also a performance benefit to allow the compiler to optimize
>>> the syscall away, but that also impacts debuggability if our syscall
>>> layer disappears.
>>
>> If you want to debug you have to reduce the optimization level anyway.
>> You can use for example -fno-inline.
>>
>>> The tradeoffs between maintainability and
>>> debuggability vs performance appear to be fairly complicated in this
>>> space. There's not a great description to me yet of what is required
>>> for performance, while the documentation of libbsd has mostly favored
>>> maintainability as a primary objective. So I think that any argument
>>> to do something for performance reasons has to be accompanied by clear
>>> evidence that the performance improvement matters.
>>
>> Talking about maintainability here sounds fairly abstract to me. This
>> should be underlined with concrete examples. In the master branch there
>> is no kern_descrip.c which is a 4k LOC source file. I don't know how
>> adding this file improves maintainability. The VFS/NFS support works
>> without the FreeBSD file descriptors.
>>
>>>
>>>> changed without a discussion on the RTEMS mailing list. It worked for
>>>> years, it allows compiler optimizations, it is easy to review, and
> there
>>>> are no issues during updates.
>>>>
>>> I think the change was announced:
>>> https://lists.rtems.org/pipermail/devel/2021-July/068573.html
>>>
>>> This is just another outcome of our current way of handling change
>>> management. Changes can easily be lost in the email exchange.
>>
>> Sorry, I didn't notice this e-mail before. In this time frame I was
>> quite busy with the RTEMS pre-qualification activity and holidays.
>>
>>>
>>>> I updated the patch set so that it doesn't require changes in RTEMS:
>>>>
>>>> https://github.com/sebhub/rtems-libbsd/tree/filedesc
>>>>
>>>>> I think we can then revisit whether or not the FreeBSD File
>>>>> Descriptors themselves are in fact problematic.
>>>> File descriptors on top of file descriptors make little sense. Also
>>>> FreeBSD has some implementation overheads which are unnecessary in the
>>>> RTEMS context in which the maximum file descriptor count is fixed at
>>>> link time.
>>>>
>>> Are there any performance analyses that quantify the impact of the
>>> implementation overhead of FreeBSD File Descriptors? Again, this is
>>> about the tradeoff between maintainability and performance. i can't
>>> make value judgments about performance without understanding the
>>> resource constraints of the target platforms, and the performance
>>> metrics (code size, heap size, throughput?) that are used to decide if
>>> an optimization is worth sacrificing maintainability. It has always
>>> been my understanding that the libbsd strives for maintainability
>>> first and foremost. So to unilaterally reject code changes that reduce
>>> the maintenance burden (by keeping us closer to the vanilla FreeBSD
>>> code base) in the name of performance requires two things:
>>> 1. Evidence that the performance improvement is substantial.
>>> 2. Argument that the implementation of the performance improvement is
>>> the most maintainable way to achieve it.
>>>
>>> So far, I haven't seen either of these with respect to the runtime
>>> overheads. And I haven't seen a good argument that injecting the
>>> syscall implementations into FreeBSD is better than having them live
>>> separately in the rtemsbsd layer.
>>
>> If you are interested I suggest to build libbsd for the
>> arm/xilinx_zynq_a9_qemu BSP with optimization disabled. Then step
>> through a socket send/receive system call on the master and 6-freebsd-12
>> branch and in particular getsock_cap().
>>
>
> =====================================================================
> On 2023-01-16 14:30, Christian MAUDERER wrote:
>> Hello,
>>
>> I think we have a bit of a new situation here: There are two approaches
>> to a problem, and it seems that no consent can be found with the usual
>> discussions. May I suggest that we try a more top-down approach: Let's
>> agree what are the important points that we want to reach and then try
>> to evaluate the problem based on these?
>>
>> What I have seen in the discussions, the following points should be
>> evaluated:
>>
>> - Maintainability: Who does the main maintenance work and how easy is it
>> for them to do that work?
>>
>> - Extensibility: Who works with the code and how easy is it for them to
>> contribute?
>>
>> - Debuggability: How easy is it to debug / understand a problem and the
>> related code?
>>
>> - Code and RAM sizes (or other hardware requirements): What are the
>> minimum requirements and do we meet them?
>>
>> - Performance: Which are our use cases and does it perform well enough
>> in these?
>>
>> - Real time capabilities: Does the subsystem have any real time
>> requirements?
>>
>> These could be valid for basically any part of RTEMS (so that list could
>> be useful for further similar problems). We should try to find out where
>> our focus for libbsd is. For example, I think that real time capability
>> was never important for libbsd.
>>
>> Where do you see the focus for libbsd? Please feel free to add further
>> points and rate the ones above.
>>
>>
>> Based on what we agree as most important points, we then should analyze
>> the suggested solutions. From my point of view, we have at least three
>> mostly independent parts in this discussion:
>>
>> - Updated NFS support. I think it's clear that the target for all of us
>> is that it works.
>>
>> - Libbsd file descriptors: Chris imported them to the 6-freebsd-12
>> branch when adding the NFS support. Sebastian wants to get rid of them
>> again.
>>
>> - Collected system calls: Different opinions where the system calls
>> should be located.
>>
>> Did I miss a fourth part of the problem?
>>
>> Please note: Of course I'm biased because I talk a lot more with
>> Sebastian than with Chris. Despite that I'll try to be as objective as
>> possible in this discussion.
>>
>> Best regards
>>
>> Christian
>>
>
> =====================================================================
> On 2023-01-16 17:33, Thomas DOERFLER wrote:
>> Hello,
>>
>> just my two cents regarding real time capability:
>>
>> The point should not only consider, whether a certain subsystem has real
>> time requirements, but also whether a certain patch/solution influences
>> the real time capabilities of RTEMS as a whole.
>>
>> wkr,
>>
>> Thomas.
>
> =====================================================================
> On 2023-01-16 17:55, Joel Sherrill wrote:
>>
>>
>> On Mon, Jan 16, 2023 at 10:33 AM Thomas DOERFLER
>> <Thomas.Doerfler at embedded-brains.de
>> <mailto:Thomas.Doerfler at embedded-brains.de>> wrote:
>>
>> Hello,
>>
>> just my two cents regarding real time capability:
>>
>> The point should not only consider, whether a certain subsystem has
>> real
>> time requirements, but also whether a certain patch/solution
> influences
>> the real time capabilities of RTEMS as a whole.
>>
>>
>> libbsd made us realize that source transparency and compatibility is
> a big
>> factor and must be considered any time we import/reuse third party code.
>> libbsd is a huge amount of code and it is of paramount importance to
>> minimize changes and RTEMS special code. Doing so increases the
>> maintenance burden and adds risk of introducing incompatibility with
>> FreeBSD.
>>
>> If something from FreeBSD cannot be supported on RTEMS or (in this
>> case) is too large for some desired targets, then a way must be found to
>> make the FreeBSD import leaner. Perhaps a configuration option to avoid
>> some memory allocations. Or a configuration option to disable the part of
>> the code in question that pulls in what is not needed for the specific
>> application and target in question.
>>
>> The RTEMS way is to make features optional and drop out cleanly.
>> We do not have a history of hacking up ported package.
>>
>> libbsd's first order goal is source transparency to ease updates,
>> maintenance, and avoid introducing bugs/incompatibilities.
>>
>> I have trouble believing that this is about performance and we don't
>> make real-time claims about libbsd because that would mean FreeBSD
>> made those claims and they don't. It is about functionality and
>> compatibility. I'm sure FreeBSD has made their implementation as
>> performant as possible but without a worry about memory consumption.
>>
>> The unnecessary code for this specific application/target needs to be
>> identified and simply configured out.
>>
>> Transparency and compatibility override any other considerations or
>> options.
>>
>> --joel
>> .
>>
>
> =====================================================================
> On 2023-01-17 11:38, Christian MAUDERER wrote:
>> Hello Joel,
>>
>> Am 16.01.23 um 17:55 schrieb Joel Sherrill:
>>>
>>>
>>> On Mon, Jan 16, 2023 at 10:33 AM Thomas DOERFLER
>>> <Thomas.Doerfler at embedded-brains.de
>>> <mailto:Thomas.Doerfler at embedded-brains.de>> wrote:
>>>
>>> Hello,
>>>
>>> just my two cents regarding real time capability:
>>>
>>> The point should not only consider, whether a certain subsystem has
>>> real
>>> time requirements, but also whether a certain patch/solution
>>> influences
>>> the real time capabilities of RTEMS as a whole.
>>>
>>>
>>> libbsd made us realize that source transparency and compatibility is a
>>> big
>>> factor and must be considered any time we import/reuse third party code.
>>> libbsd is a huge amount of code and it is of paramount importance to
>>> minimize changes and RTEMS special code. Doing so increases the
>>> maintenance burden and adds risk of introducing incompatibility with
>>> FreeBSD.
>>>
>>> If something from FreeBSD cannot be supported on RTEMS or (in this
>>> case) is too large for some desired targets, then a way must be found to
>>> make the FreeBSD import leaner. Perhaps a configuration option to avoid
>>> some memory allocations. Or a configuration option to disable the
> part of
>>> the code in question that pulls in what is not needed for the specific
>>> application and target in question.
>>>
>>> The RTEMS way is to make features optional and drop out cleanly.
>>> We do not have a history of hacking up ported package.
>>>
>>> libbsd's first order goal is source transparency to ease updates,
>>> maintenance, and avoid introducing bugs/incompatibilities.
>>>
>>> I have trouble believing that this is about performance and we don't
>>> make real-time claims about libbsd because that would mean FreeBSD
>>> made those claims and they don't. It is about functionality and
>>> compatibility. I'm sure FreeBSD has made their implementation as
>>> performant as possible but without a worry about memory consumption.
>>>
>>> The unnecessary code for this specific application/target needs to be
>>> identified and simply configured out.
>>>
>>> Transparency and compatibility override any other considerations or
>>> options.
>>
>> Is the transparency and compatibility a new point? From how I read your
>> text it is mostly part of maintainability and debugability as well as a
>> bit extensibility that I mentioned in my list earlier.
>>
>> Beneath that you added a point to the list that is related but not the
>> same as code and RAM size:
>>
>> - Modularity: How well and easy the system can be adapted to target
>> applications. Additionally: What is the official way to enable / disable
>> modules in the subsystem.
>>
>> Best regards
>>
>> Christian
>>
>
> =====================================================================
>
> After this mail it has been suggested to continue the discussion on
> devel and all agreed that the mails can be re-posted.
> _______________________________________________
> devel mailing list
> devel at rtems.org
> http://lists.rtems.org/mailman/listinfo/devel
--
--------------------------------------------
embedded brains GmbH
Herr Christian MAUDERER
Dornierstr. 4
82178 Puchheim
Germany
email: christian.mauderer at embedded-brains.de
phone: +49-89-18 94 741 - 18
mobile: +49-176-152 206 08
Registergericht: Amtsgericht München
Registernummer: HRB 157899
Vertretungsberechtigte Geschäftsführer: Peter Rasmussen, Thomas Dörfler
Unsere Datenschutzerklärung finden Sie hier:
https://embedded-brains.de/datenschutzerklaerung/
More information about the devel
mailing list