[PATCH] build: Add RTEMS_QUALIFIED

Fri Nov 10 08:39:21 UTC 2023

On 09.11.23 06:28, Chris Johns wrote:
> On 8/11/2023 6:44 pm, Sebastian Huber wrote:
>> On 06.11.23 21:55, Chris Johns wrote:
>>> On 6/11/2023 8:27 pm, Sebastian Huber wrote:
>>>> On 06.11.23 01:14, Chris Johns wrote:
>>>>> On 4/11/2023 1:31 am, Sebastian Huber wrote:
>>>>>> On 03.11.23 15:08, Joel Sherrill wrote:
>>>>>>> On Fri, Nov 3, 2023 at 3:58 AM Sebastian Huber
>>>>>>> <sebastian.huber at embedded-brains.de
>>>>>>> <mailto:sebastian.huber at embedded-brains.de>> wrote:
>>>>>>>
>>>>>>>        The goal of the RTEMS pre-qualification activity #3701 is a specified
>>>>>>>        and validated subset of RTEMS.  For users of the pre-qualified
>>>>>>> subset of
>>>>>>>        RTEMS it is important to not accidentally use not pre-qualified
>>>>>>>        features.  One way to achieve this, is to build only the sources of the
>>>>>>>        pre-qualified feature set. This customized build is enabled by the new
>>>>>>>        build configuration option RTEMS_QUALIFIED.  If it is enabled, then
>>>>>>> only
>>>>>>>        the pre-qualified subset of RTEMS is built and installed.
>>>>>>>
>>>>>>>        Building with RTEMS_QUALIFIED enable is currently only supported for
>>>>>>> the
>>>>>>>        sparc/leon3 BSP family.  To support an RTEMS_QUALIFIED enabled build,
>>>>>>>        changes in the CPU port and the BSP are required to only use
>>>>>>> features of
>>>>>>>        the pre-qualified feature set.
>>>>>>>
>>>>>>>
>>>>>>> Where is this documented?
>>>>>> You mean a documentation of what needs to be done to create pre-qualified
>>>>>> BSP? I
>>>>>> don't have it available yet. A good place to add this would be the how-to
>>>>>> section in the RTEMS Software Engineering manual.
>>>>>>
>>>>>>> This is a very large patch. Are you assuming that if "not qualified" is
>>>>>>> specified,
>>>>>>> then it is in the qualified set?
>>>>>> No, the logic is reversed. Everything is built by default. Some parts are only
>>>>>> enabled if RTEMS_QUALIFIED is not enabled, for example
>>>>>> (spec/build/cpukit/objextra.yml):
>>>>>>
>>>>>> enabled-by:
>>>>>>      not: RTEMS_QUALIFIED
>>>>> It seems counter intuitive to me. I have no idea about qual work but my limited
>>>>> understanding is everything is controlled yet this is the inverse of that and
>>>>> anything anyone adds will by default be qualified?
>>>>
>>>> The goal of this patch set is to place each source and header file of RTEMS into
>>>> two buckets, the pre-qualified bucket and the extra bucket. For two buckets you
>>>> need just one option with two values. In the current patch it is RTEMS_QUALIFIED
>>>> enabled or disabled.
>>>
>>> I think we should also include in our discussion code contained within this
>>> define (or defines) in shared files and how we manage changes to that code? In
>>> an ideal world we would not have any need for conditional code however I
>>> appreciate this may not be possible or initial achievable. We should however
>>> look for approaches that try to avoid this and understand the constraints the
>>> define brings. For example can I change code in a qualification or FACE define
>>> when I do not know the standards they support?
>>
>> We should avoid using the C preprocessor for the profile customization. For the
>> ECSS pre-qualification, there was no need to do this (with the exception of
>> testbeginend.c, but this could be changed).
> 
> +1
> 
>>>> Since Joel already brought the FACE profile into play, an alternative could be
>>>> this approach. Lets assume that we pre-qualified the FACE profile interfaces of
>>>> RTEMS. Use RTEMS_EVERYTHING for the I want everything profile. Use
>>>> RTEMS_QUALIFIED for the pre-qualified profile. Use RTEMS_FACE_PROFILE for
>>>> interfaces of the FACE profile.
>>>
>>> If these are profiles I suggest RTEMS_PROFILE_EVERYTHING so RTEMS_PROFILE_.* can
>>> find all profiles?
>>
>> Yes, this make sense.
>>
>>>
>>>> The objects which are not pre-qualified would use this:
>>>>
>>>> enabled-by: RTEMS_EVERYTHING
>>>
>>> RTEMS_PROFILE_COMMERCIAL ?
>>> RTEMS_PROFILE_INDUSTRIAL ?
>>> RTEMS_PROFILE_RTEMS ?
>>>
>>> ... or something that reflects what we have always shipped? I borrowed those
>>> labels from the way parts are classed. I tend to keep away from labels like new,
>>> next, everything etc because they quickly date.
>>>
>>>>
>>>> The objects which are pre-qualified would use this:
>>>>
>>>> enabled-by:
>>>> - RTEMS_EVERYTHING
>>>> - RTEMS_QUALIFIED
>>>
>>> Qualified to what? RTEMS_PROFILE_FACE defines something useful and concrete. Is
>>> there an equivalent we could use for the work you have been doing?
>>
>> The selection of the feature set for the ECSS pre-qualification was based on a
>> survey:
>>
>> https://ftp.rtems.org/pub/rtems/people/sebh/rtems-smp-userconsultation-27032019-release.pdf
>>
>> https://ftp.rtems.org/pub/rtems/people/sebh/tn-space-profile-r6-23062019.pdf
>>
> 
> RTEMS_PROFILE_SPACE ? I just used the name on page 1 of this document.

The profile in the patch set is by no means limited to the European 
space industry. We need a name for the subset of RTEMS which fulfills 
the 4 rules below. This subset may grow over time and include for 
example the POSIX threads API. We already have plans to work on this.

> 
>>> I do not like the use of QUALIFIED as mentioned in my other post in this thread
>>> because it implies something the open project should not and that is the code is
>>> qualified. We will only provide pre-qualification data packs from the public
>>> repos unless that has changed?
>>
>> In the context of this patch set and the Software Requirements Engineering
>> chapter of the RTEMS Software Engineering manual
>>
>> https://docs.rtems.org/branches/master/eng/req/index.html
>>
>> "qualified" means that:
>>
>> 1. An interface specification exists for each qualified API element.
>>
>> 2. Each qualified interface has a functional specification.
>>
>> 3. The function is validated by a test.
>>
>> 4. You get 100% branch and code coverage from the validation tests.  For some
>> justified exceptions you can use unit tests.
> 
> Yes but there is no public proof so it is a promise until it is a fact and that
> is achieved at the end of the process. What is written is correct but a repo is
> in the present tense.

I continuously work on integrating the work into the RTEMS Project. This 
patch set is a part of this work. It would reduce the maintenance cost 
on my side and so help to free time to work on other parts of the 
integration TODO list.

> 
>> If someone has a better name than "qualified" for this, I am happy to use it.
> 
> Good question. If RTEMS_PROFILE_SPACE is used can RTEMS_QUALIFIED be avoided?

I think "space" is too specific.

> 
>>>> The objects which are pre-qualified or in the FACE profile would use this:
>>>>
>>>> enabled-by:
>>>> - RTEMS_EVERYTHING
>>>> - RTEMS_QUALIFIED
>>>> - RTEMS_FACE_PROFILE
>>>
>>> The addition of these defines explodes the builds and testing we need to perform
>>> for each commit. We need to discuss this some more and look for boundaries that
>>> could help limit the builds and testing for each profile.
>>
>> Yes, all these options lead to a combinatorial explosion. We have to be
>> pragmatic and focus on a useful subset for tests.
> 
> I know for RTEMS_PROFILE_SPACE it will be a specific set of LEON archs? I see no
> point building that profile for anything else. The point is a profile should
> have a list of arches or BSPs associated to it as part of being accepted.

This profile is not specific to a particular BSP. However, for each 
pre-qualified BSP there is some work to do. Firstly, it needs to be 
restructured to use only features of the pre-qualified feature set. 
Secondly, you may have to specify BSP-specific functions and validate 
them. Thirdly, you have to review all chip errata, implement 
workarounds, justify why erratas are not applicable to the BSP.

> 
> Chris
> 
>>>> Of RTEMS_EVERYTHING, RTEMS_QUALIFIED, and RTEMS_FACE_PROFILE exactly one option
>>>> shall be enabled.
>>>
>>> Yes this makes sense. Should a source file not in a profile be considered an
>>> error?

Yes, this would be an error.

-- 
embedded brains GmbH
Herr Sebastian HUBER
Dornierstr. 4
82178 Puchheim
Germany
email: sebastian.huber at embedded-brains.de
phone: +49-89-18 94 741 - 16
fax:   +49-89-18 94 741 - 08

Registergericht: Amtsgericht München
Registernummer: HRB 157899
Vertretungsberechtigte Geschäftsführer: Peter Rasmussen, Thomas Dörfler
Unsere Datenschutzerklärung finden Sie hier:
https://embedded-brains.de/datenschutzerklaerung/