[libbsd 00/22] Remove FreeBSD file descriptors and avoid VFS

[Joel Sherrill]

On Thu, Oct 20, 2022 at 7:30 PM Chris Johns <chrisj at rtems.org> wrote:

> On 20/10/2022 7:01 pm, Sebastian Huber wrote:
> > On 29/08/2022 10:27, Sebastian Huber wrote:
> >> Hello Chris,
> >>
> >> On 25/07/2022 08:12, Sebastian Huber wrote:
> >>> On 11/07/2022 15:04, Sebastian Huber wrote:
> >>>> On 24/06/2022 08:33, Sebastian Huber wrote:
> >>>>> This patch set removes the FreeBSD file descriptors.  The VFS is no
> longer
> >>>>> used
> >>>>> if only the USB, SD/MMC, network, PCI, and NVMe support is used by
> the
> >>>>> application.  This change significantly reduce the memory usage of
> LibBSD for
> >>>>> these applications.  Using the media01 test case for the arm/lpc32xx
> BSP as a
> >>>>> benchmark, the heap usage dropped from 14.3MiB to 10.2MiB.  The "_BSD
> >>>>> bufdaemon", "_BSD vnlru", "_BSD syncer", and "_BSD bufspacedaemon-"
> tasks are
> >>>>> no longer present in media01.  The code size is reduced by about
> 8KiB.  The
> >>>>> data size is reduced by about 30KiB.  The throughput with a simple
> FTP test
> >>>>> increased by about 1%.
>

Only the decrease in heap size is significant enough to discuss. The others
are noise
when considered in light of the source transparency we aimed for.

https://freebsdfoundation.org/wp-content/uploads/2016/08/FreeBSD-and-RTEMS-Unix-in-a-Real-Time-Operating-System.pdf

This guideline which is intended to increase compatibility and lower
long-term
maintenance is at odds with this set of patches.

That leaves us with the 4MB of heap. That alone is your problem. What is
allocating it? Is there a hint or sysctl value that can turn it down? Is it
representative of a single feature that could just be disabled in your
low memory situation [1].

[1] I agree with Chris' statements that there are no requirements for
a set of functionality to fit within a certain amount of memory. And
if you are this close to the edge, almost anything that happens will
break it again.


> >>>>>
> >>>>> The "Remove FreeBSD file descriptors" change removes more lines than
> there are
> >>>>> added.
> >>>>>
> >>>>> This change makes it easier to port the NFS support to the master
> branch since
> >>>>> now the changes are more localized.
> >>>>
> >>>> I have a target with only 8MiB of RAM (for code and data). So, this
> patch
> >>>> set is not just a micro optimization.
> >>>
> >>> This pending patch set is currently a blocker for me. I would like to
> work on
> >>> the NTP daemon integration and an update of the libbsd master branch
> to a
> >>> more recent FreeBSD version. This patch set just restores what worked
> well
> >>> for several years.
> >>
> >> did you have time to review this patch set in the meantime? The NTP
> client no
> >> longer crashes with this patch set.
> >
> > This patch set is now pending for about four months. From my point of
> view it is
> > a straight forward fix of some severe issues which prevent the use of
> libbsd on
> > lower end targets.
>
> Is there a repo somewhere with the patches applied? I am being lazy asking
> as I
> would like to avoid downloading the 22 patches and applying them by hand.
>
> It seems the media test cannot grow by more than 9K of code and the heap
> by 5M!
> Is my understanding correct? Anything else we need to understand about the
> low
> end limits?
>
> I have not seen any requirements on the lower end for libbsd before now.
> How do
> we manage the competing requirement of low end support and adding
> functionality
> to libbsd in the future if they clash?
>
> Adding VFS support as FreeBSD implements it has pushed you over the edge
> but
> that edge is not defined so it makes it difficult for anyone other than
> you to
> work on this code base in a major way. I was not aware this was an issue
> and for
> me the extra 9K of code is not a problem and a 5M change in heap is
> acceptable.
>

Again that increased heap usage is clearly a memory allocation and there is
code
which is doing it. Can it be disabled and we avoid this entire set of
invasive patches?


>
> > It is not really good if our internal libbsd branches diverge
> > too much from the RTEMS mainline.
>
> Yes I agree. It is not in anyone's interest and that only benefits you.
>
> I would like to reiterate we have an obligation to maintain the code base
> as
> close to FreeBSD as possible. Some of these changes push that boundary. We
> need
> to find a suitable path forward.
>
> Reviewing the changes:
>
> 1. The break up of the syscalls file. The related comments are:
>
>  "Collecting all system calls in a single translation unit is not good
>   due to the library initialization through linker sets."
>
> Could you please explain what the linker set issue is?
>
> Why the need to change FreeBSD calls to static? Is it solely a performance
> optimisation?
>
> We are required to use function and data sections so I assume the syscall
> file
> will be broken up by the linker where possible?
>
> I cannot tell from the patches if the conditional syscall trace support is
> still
> present?
>
> I do not agree with your statement in the patches separation helps
> porting. My
> experience was not that. When adding VFS and the NFS client I found the
> syscall
> code all over the place and not easy to find, hard to review and it added
> complexity. It adds unnecessary changes to the FreeBSD code against a base
> requirement of the porting effort.
>
> 2. FreeBSD file descriptor removal
>
> Does this mean we can never have them in libbsd? Does that limit what can
> be
> brought across from FreeBSD in the future?
>
> 3. select, poll, and kqueue
>
> Do these calls work with an NFS client fd?
>
> 4. Types of RTEMS fd's in libbsd
>
> The change I made brought the types into a single location so it is easier
> to
> review and understand. There is:
>
>  - rtems_bsd_sysgen_dirops
>  - rtems_bsd_sysgen_fileops
>  - rtems_bsd_sysgen_nodeops
>  - rtems_bsd_sysgen_imfsnodeops
>
> VFS is now left to handle the specific vnode ops as FreeBSD is designed to
> do.
>
> [
>
> https://git.rtems.org/rtems-libbsd/tree/rtemsbsd/rtems/rtems-bsd-syscall-api.c?h=6-freebsd-12
> ]
>
> Chris
> _______________________________________________
> devel mailing list
> devel at rtems.org
> http://lists.rtems.org/mailman/listinfo/devel
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.rtems.org/pipermail/devel/attachments/20221021/a3ce319b/attachment.htm>