#4328: New APIs added to POSIX Standard (2021)

Matthew Joyce mfjoyce2004 at gmail.com
Wed Mar 24 12:38:22 UTC 2021 

Hi Dr. Joel,

I've gone over the list a few times now and see a few categories shaping up:

1) Already done (In Newlib source, defined in libc.a):
a) reallocarray
b) qsort_r
c) memmem
d) strlcat / strlcpy
d) wcslcat / wcslcpy
*Out of this group, strlcat and strlcpy also show up in
src/rtems/cpukit. Why is that?

2) Not done yet (Do not show up in Newlib source or RTEMS):
a) getlocalename_l
b) posix_getdents
c) sem_clockwait
d) sig2str / str2sig

3) Not in Newlib; Referenced in RTEMS but hidden behind #ifdef:
a) pthread_cond_clockwait
(rtems/6/lib/gcc/sparc-rtems6/10.2.1/include/c++/condition_variable)
b) pthread_mutex_clocklock
(rtems/6/lib/gcc/sparc-rtems6/10.2.1/include/c++/mutex)
c) pthread_rwlock_clockrdlock
(rtems/6/lib/gcc/sparc-rtems6/10.2.1/include/c++/shared_mutex)
c) pthread_rwlock_clockwrlock
(rtems/6/lib/gcc/sparc-rtems6/10.2.1/include/c++/shared_mutex)
*It looks like some groundwork was done, but the methods are not yet supported.

4) Misc (In Newlib source, not defined in libc.a, appear in RTEMS in
various ways)
a) getentropy (an alternate version is defined in RTEMS librtemsbsd.a,
in src/rtems/bsps/shared/dev/getentropy/getentropy-cpucounter.c. The
comments note that it is not cryptographically secure, so it may not
fit the bill for the getentropy() mentioned in the Open Group
document)
b) ppoll (appears in rtems/6/share/gdb/syscalls)
c) dladdr (appears in rtems/cpukit but not defined)

5) Others?
It looks like there was work done on methods like sockatmark and
pselect, but I don't see them supported as yet. Should those be added
to the list or are they still being worked on?

As you suggested, I'll look into NetBSD for dladdr and do some digging
on the implementation of the other outstanding methods. You mentioned
that the "clock" ones have to be strictly added to rtems/cpukit, but
the references I found above are all in lib/gcc/sparc-rtems6/10.2.1.
Why is that the case and what is 10.2.1? Also, I'm not sure what to
make of getentropy and ppoll based on what I found above...at your
convenience could you please advise?

Thank you very much!

Matt




On Sun, Mar 21, 2021 at 6:38 PM Joel Sherrill <joel at rtems.org> wrote:
>
>
>
> On Sun, Mar 21, 2021 at 2:28 AM Matthew Joyce <mfjoyce2004 at gmail.com> wrote:
>>
>> Gentlemen,
>>
>> Awesome, thanks!  I see how that works now...I'll give it a thorough
>> look tomorrow and will update the spreadsheet accordingly. I'll pipe
>> back up when I have a more accurate look of what's currently there.
>
>
> Knowing what doesn't have to be done is the first step. (rtems, newlib, and libbsd)
>
> I'd be prone to look for things that are easy to add first.
>
> Some may not be implementable on RTEMS due to only supporting a
> single process and no virtual memory. If you have doubts on whether it
> is possible to support a specific method, speak up and let's try to decide.
>
> Then find upstream places for an implementation where possible. I suspect
> all the new "clock" methods will require discussion on an implementation
> pattern but those must strictly be added to rtems/cpukit with tests and
> documentation. At least I can throw you that much. :)
>
>>
>> Thanks again and have a great Sunday!
>>
>> Matt
>>
>> On Fri, Mar 19, 2021 at 8:27 PM Joel Sherrill <joel at rtems.org> wrote:
>> >
>> >
>> >
>> > On Fri, Mar 19, 2021 at 1:08 PM Gedare Bloom <gedare at rtems.org> wrote:
>> >>
>> >> On Fri, Mar 19, 2021 at 11:16 AM Matthew Joyce <mfjoyce2004 at gmail.com> wrote:
>> >> >
>> >> > Dr. Joel,
>> >> >
>> >> > Thanks very much...I'll keep working to get a sense of what goes
>> >> > where! In the meantime, where can I look to get the ground truth of
>> >> > which methods are "in RTEMS" as opposed to those in newlib?
>> >> >
>> >> There is only one ground truth:
>> >> git://git.rtems.org/rtems.git
>> >>
>> >> And for newlib
>> >>
>> >> git://sourceware.org/git/newlib-cygwin.git
>> >>
>> >> That said, searching for the function name symbols in compiled
>> >> libraries is a good first step to rule out newlib. Then, you can
>> >> 'grep' the RTEMS source code for the function names to see if they
>> >> exist there.
>> >
>> >
>> > rtems/cpukit to be specitic. It won't be implemented anywhere else.
>> >
>> > And clearly we both have forgotten that networking APIs are in the
>> > rtems-libbsd repository.
>> >
>> > https://git.rtems.org/rtems-libbsd/
>> >
>> > I suspect ppoll() might already be in there. Or at least supported by
>> > FreeBSD.
>> >
>> > You should clone everything and grep the sources. newlib already has
>> > qsort_r. This is the nm I used:
>> >
>> > $ ~/rtems-work/tools/6/bin/sparc-rtems6-nm ~/rtems-work/tools/6/sparc-rtems6/lib/libc.a | grep qsort_r
>> > lib_a-bsd_qsort_r.o:
>> > 00000000 T __bsd_qsort_r
>> > lib_a-qsort_r.o:
>> > 00000000 T qsort_r
>> >
>> > Notice the last line has "T qsort_r" which says it is defined.
>> >
>> > grep -r in the newlib source shows it is in ./libc/search/qsort_r.c
>> >
>> > dladdr() looks to be prototyped in RTEMS but hidden behind an ifdef like it
>> > wasn't ported from NetBSD so that looks possible. It is in rtems.
>> >
>> > Those two examples should help you figure out why you missed
>> > finding some things that were implemented.
>> >
>> > I need to figure out what this next POSIX version is to be called
>> > so I can update the tracking spreadsheet that generates the RTEMS
>> > POSIX Compliance Guide, :)
>> >
>> > --joel
>> >>
>> >>
>> >> > Thanks again!
>> >> >
>> >> > Matt
>> >> >
>> >> > On Fri, Mar 19, 2021 at 1:58 PM Joel Sherrill <joel at rtems.org> wrote:
>> >> > >
>> >> > > Keep devel@ on the list. :)
>> >> > >
>> >> > > On Fri, Mar 19, 2021 at 7:51 AM Matthew Joyce <mfjoyce2004 at gmail.com> wrote:
>> >> > >>
>> >> > >> Sir,
>> >> > >>
>> >> > >> Thank you for the link! I see that you're right, those last four are
>> >> > >> in newlib, plus memmem(). I updated those in the Google Sheet.
>> >> > >>
>> >> > >> Now I see the newlib part, but where are you referring to specifically
>> >> > >> when you say RTEMS, as in "POSIX support comes from a mix of RTEMS and
>> >> > >> newlib"?
>> >> > >
>> >> > >
>> >> > > POSIX is a HUGE HUGE standard and references other standards. One
>> >> > > it references and pulls in is the C99 Standard C Library which is libc and
>> >> > > libm. RTEMS mostly does not implement this functionality and relies on
>> >> > > another open source project for those APIs. Newlib is an open source
>> >> > > C Library used by RTEMS, Cygwin, and most embedded systems GNU tools
>> >> > > chains.
>> >> > >
>> >> > > Most of the POSIX header files with RTEMS are actually in Newlib even
>> >> > > if they originated with RTEMS. Many are shared with Cygwin.
>> >> > >
>> >> > > So methods like the string, memory, and *printf come from Newlib since they
>> >> > > are in C99. We provide POSIX like threading, signals, core file access, and
>> >> > > much more.
>> >> > >
>> >> > > It's a complementary relationship but it takes a bit to figure out when
>> >> > > something should be in one or the other. The line gets blurred at times.
>> >> > >
>> >> > > Say you added a new CPU architecture implementation of a math
>> >> > > method (like Eshan did last year), then it goes in newlib. But he also
>> >> > > added some POSIX methods which go in RTEMS. In either case,
>> >> > > we like tests for them in RTEMS to show they work in our environment.
>> >> > >
>> >> > > --joel
>> >> > >
>> >> > >
>> >> > >
>> >> > >>
>> >> > >> Thanks again!
>> >> > >>
>> >> > >> Matt
>> >> > >>
>> >> > >> On Fri, Mar 19, 2021 at 1:13 PM Joel Sherrill <joel at rtems.org> wrote:
>> >> > >> >
>> >> > >> >
>> >> > >> >
>> >> > >> > On Fri, Mar 19, 2021, 6:40 AM Joel Sherrill <joel at rtems.org> wrote:
>> >> > >> >>
>> >> > >> >>
>> >> > >> >>
>> >> > >> >> On Fri, Mar 19, 2021, 5:48 AM Matthew Joyce <mfjoyce2004 at gmail.com> wrote:
>> >> > >> >>>
>> >> > >> >>> https://docs.google.com/spreadsheets/d/1reCNOIZC5JTwQENgl-hvG8THfQqNtlUDVy_07PYodic/edit?usp=sharing
>> >> > >> >>>
>> >> > >> >>> Hello,
>> >> > >> >>>
>> >> > >> >>> As suggested by Dr. Sherril, I've taken an initial look through this
>> >> > >> >>> document https://www.opengroup.org/austin/docs/austin_1110.pdf and
>> >> > >> >>> added the new methods  to a Googe Sheet, linked above.
>> >> > >> >>>
>> >> > >> >>> None of them appear to be in the RTEMS POSIX API Users Guide, but
>> >> > >> >>> maybe that's not the right place to look. I'll stand by for your
>> >> > >> >>> feedback regarding what's possible / desirable to add to RTEMS.
>> >> > >> >>
>> >> > >> >>
>> >> > >> >> It is possible they are in our C Library or Math Library.  Or just not in the manual. The POSIX manual tends to be sparse since you can always use man pages or the POSIX standard.
>> >> > >> >>
>> >> > >> >> Since you have RTEMS and tools built. Find one of the libc.a and libm.a files in the tools install and librtemscpu.a in the RTEMS build or install. Then try a command something like this:
>> >> > >> >>
>> >> > >> >> CPU-rtems6-nm LIBRARY | grep SYMBOL
>> >> > >> >>
>> >> > >> >> If you see it list with T then it is in the text section and there.
>> >> > >> >
>> >> > >> >
>> >> > >> > Following up, I initially answered from my phone and didn't look at source.  I am still on my phone but looked through the list and think the last four methods are probably the only ones currently supported.
>> >> > >> >
>> >> > >> > https://sourceware.org/git/?p=newlib-cygwin.git;a=tree;f=newlib/libc/string;h=ceeec602cdd0e6b5c6b002b741bda9b41da4e441;hb=HEAD
>> >> > >> >
>> >> > >> > POSIX support comes from a mix of RTEMS and newlib. That's key to this type of project.
>> >> > >> >
>> >> > >> > --joel
>> >> > >> >>
>> >> > >> >>
>> >> > >> >>
>> >> > >> >>>
>> >> > >> >>> Thanks very much for your time!
>> >> > >> >>>
>> >> > >> >>> Sincerely,
>> >> > >> >>>
>> >> > >> >>> Matt
>> >> > _______________________________________________
>> >> > devel mailing list
>> >> > devel at rtems.org
>> >> > http://lists.rtems.org/mailman/listinfo/devel

More information about the devel mailing list