Pre-Qualified RTEMS Build

[Sebastian Huber]

Hello,

for the ESA pre-qualification activity we would like to have an RTEMS 
build which includes the pre-qualified parts and in addition give 
applications the option to use most of the remaining functionality 
provided by a standard RTEMS (excluded are POSIX signals and timers). To 
achieve this I wrote a new build system for RTEMS based on specification 
items last year as a task of this ESA activity. The RTEMS Project 
decided to not integrate it in RTEMS 5 and postpone this to RTEMS 6 
(maybe). To avoid further delays in the ESA activity I decided to start 
with the modifications for a pre-qualified build on top of the not yet 
integrated new build system.

I can now build an RTEMS sparc/leon3 BSP which provides potentially 
pre-qualified start.o, librtemsbsp.a, librtemscpu.a, and librtemstest.a 
artifacts. It offers the full set of RTEMS header files and provides 
librtemsbspextra.a, librtemscpuextra.a, librtemstestextra.a, etc. 
libraries as non-pre-qualified components. In the build specification, 
every header and source file belongs exactly one of two pairwise 
disjoint sets (pre-qualified and non-pre-qualified). The source and 
header files necessary for the pre-qualified artifacts are determined by 
the new space profile validation test:

https://git.rtems.org/sebh/rtems.git/tree/testsuites/validation/ts-profile.c?h=qual

This test includes all functions of the space profile proposal (except 
device drivers and the interrupt controller API).

https://ftp.rtems.org/pub/rtems/people/sebh/tn-space-profile-r2-18072019.pdf

In the pre-qualified set of files for sparc/leon3, there are 242 header 
files, 255 C source files, and 8 assembler source files.

The pre-qualified set has no reference to the heap free function. To 
achieve this, I wrote more than a hundred patches for RTEMS in 2019 and 
2020 (thanks for reviewing all this stuff and all the help to find 
better solutions). A change in the space profile is necessary for this 
also. We have to replace the rtems_task_create() function with an 
rtems_task_build() function. This needs to be discussed:

https://devel.rtems.org/ticket/3959

The application configuration using the pre-qualified artifacts is done 
through the standard <rtems/confdefs.h> header file. To achieve this, I 
had to rework a nearly 4k lines of code header file. I split it into 22 
different header files:

https://git.rtems.org/rtems/tree/cpukit/include/rtems/confdefs

All 149 application configuration options are now documented and 
(partly) specified:

https://git.rtems.org/sebh/rtems-qual.git/tree/spec/acfg/opt

How can we verify which source and header files are required for the 
space profile? For this I added a verification build. All files of the 
pre-qualified set are copied to a separate directory, then a build of 
only the pre-qualified artifacts is done in this separate directory 
using the build specification of the corresponding RTEMS sources. Only 
the artifacts and header file in this separate directory are used to 
link the space profile validation test. You can install this build and 
get a BSP installation without the extras. Doing this specialized build 
is done with 11 lines of Python code. Gathering the files necessary for 
this build is done with 57 lines of Python code. You find it here:

https://git.rtems.org/sebh/rtems-qual.git/

The specification items are in YAML format. I am very happy with this 
file format, however, the Python YAML parser is quite slow. So, I wrote 
an item cache in Python pickle format. It loads the current 
specification (2439 files) in about 0.2 seconds compared to several 
seconds if loaded from YAML.

The code was written according to the new RTEMS Python Development 
Guidelines:

https://docs.rtems.org/branches/master/eng/python-devel.html

The code is formatted by yapf and analyzed by flake8, mypy, and pylint. 
It is tested with pytest. The code coverage through unit tests is:

coverage report -m 
--include=rtems_spec_to_x.py,rtemsqual/util.py,rtemsqual/items.py,rtemsqual/__init__.py,rtemsqual/glossary.py,rtemsqual/content.py,rtemsqual/build.py,rtemsqual/applconfig.py
Name                      Stmts   Miss  Cover   Missing
-------------------------------------------------------
rtemsqual/__init__.py         8      0   100%
rtemsqual/applconfig.py     130      0   100%
rtemsqual/build.py           36      0   100%
rtemsqual/content.py        133      0   100%
rtemsqual/glossary.py        71      0   100%
rtemsqual/items.py          126      0   100%
rtemsqual/util.py            26      0   100%
-------------------------------------------------------
TOTAL                       530      0   100%

I found that using the analyzers helped during development and is not a 
burden. I think it improves productivity. You just need to get used to it.