[libbsd 00/22] Remove FreeBSD file descriptors and avoid VFS

[Chris Johns]

On 20/10/2022 7:01 pm, Sebastian Huber wrote:
> On 29/08/2022 10:27, Sebastian Huber wrote:
>> Hello Chris,
>>
>> On 25/07/2022 08:12, Sebastian Huber wrote:
>>> On 11/07/2022 15:04, Sebastian Huber wrote:
>>>> On 24/06/2022 08:33, Sebastian Huber wrote:
>>>>> This patch set removes the FreeBSD file descriptors.  The VFS is no longer
>>>>> used
>>>>> if only the USB, SD/MMC, network, PCI, and NVMe support is used by the
>>>>> application.  This change significantly reduce the memory usage of LibBSD for
>>>>> these applications.  Using the media01 test case for the arm/lpc32xx BSP as a
>>>>> benchmark, the heap usage dropped from 14.3MiB to 10.2MiB.  The "_BSD
>>>>> bufdaemon", "_BSD vnlru", "_BSD syncer", and "_BSD bufspacedaemon-" tasks are
>>>>> no longer present in media01.  The code size is reduced by about 8KiB.  The
>>>>> data size is reduced by about 30KiB.  The throughput with a simple FTP test
>>>>> increased by about 1%.
>>>>>
>>>>> The "Remove FreeBSD file descriptors" change removes more lines than there are
>>>>> added.
>>>>>
>>>>> This change makes it easier to port the NFS support to the master branch since
>>>>> now the changes are more localized.
>>>>
>>>> I have a target with only 8MiB of RAM (for code and data). So, this patch
>>>> set is not just a micro optimization.
>>>
>>> This pending patch set is currently a blocker for me. I would like to work on
>>> the NTP daemon integration and an update of the libbsd master branch to a
>>> more recent FreeBSD version. This patch set just restores what worked well
>>> for several years.
>>
>> did you have time to review this patch set in the meantime? The NTP client no
>> longer crashes with this patch set.
> 
> This patch set is now pending for about four months. From my point of view it is
> a straight forward fix of some severe issues which prevent the use of libbsd on
> lower end targets.

Is there a repo somewhere with the patches applied? I am being lazy asking as I
would like to avoid downloading the 22 patches and applying them by hand.

It seems the media test cannot grow by more than 9K of code and the heap by 5M!
Is my understanding correct? Anything else we need to understand about the low
end limits?

I have not seen any requirements on the lower end for libbsd before now. How do
we manage the competing requirement of low end support and adding functionality
to libbsd in the future if they clash?

Adding VFS support as FreeBSD implements it has pushed you over the edge but
that edge is not defined so it makes it difficult for anyone other than you to
work on this code base in a major way. I was not aware this was an issue and for
me the extra 9K of code is not a problem and a 5M change in heap is acceptable.

> It is not really good if our internal libbsd branches diverge
> too much from the RTEMS mainline.

Yes I agree. It is not in anyone's interest and that only benefits you.

I would like to reiterate we have an obligation to maintain the code base as
close to FreeBSD as possible. Some of these changes push that boundary. We need
to find a suitable path forward.

Reviewing the changes:

1. The break up of the syscalls file. The related comments are:

 "Collecting all system calls in a single translation unit is not good
  due to the library initialization through linker sets."

Could you please explain what the linker set issue is?

Why the need to change FreeBSD calls to static? Is it solely a performance
optimisation?

We are required to use function and data sections so I assume the syscall file
will be broken up by the linker where possible?

I cannot tell from the patches if the conditional syscall trace support is still
present?

I do not agree with your statement in the patches separation helps porting. My
experience was not that. When adding VFS and the NFS client I found the syscall
code all over the place and not easy to find, hard to review and it added
complexity. It adds unnecessary changes to the FreeBSD code against a base
requirement of the porting effort.

2. FreeBSD file descriptor removal

Does this mean we can never have them in libbsd? Does that limit what can be
brought across from FreeBSD in the future?

3. select, poll, and kqueue

Do these calls work with an NFS client fd?

4. Types of RTEMS fd's in libbsd

The change I made brought the types into a single location so it is easier to
review and understand. There is:

 - rtems_bsd_sysgen_dirops
 - rtems_bsd_sysgen_fileops
 - rtems_bsd_sysgen_nodeops
 - rtems_bsd_sysgen_imfsnodeops

VFS is now left to handle the specific vnode ops as FreeBSD is designed to do.

[
https://git.rtems.org/rtems-libbsd/tree/rtemsbsd/rtems/rtems-bsd-syscall-api.c?h=6-freebsd-12
]

Chris