[PATCH] build: Add RTEMS_QUALIFIED

Thu Nov 9 05:28:01 UTC 2023

On 8/11/2023 6:44 pm, Sebastian Huber wrote:
> On 06.11.23 21:55, Chris Johns wrote:
>> On 6/11/2023 8:27 pm, Sebastian Huber wrote:
>>> On 06.11.23 01:14, Chris Johns wrote:
>>>> On 4/11/2023 1:31 am, Sebastian Huber wrote:
>>>>> On 03.11.23 15:08, Joel Sherrill wrote:
>>>>>> On Fri, Nov 3, 2023 at 3:58 AM Sebastian Huber
>>>>>> <sebastian.huber at embedded-brains.de
>>>>>> <mailto:sebastian.huber at embedded-brains.de>> wrote:
>>>>>>
>>>>>>       The goal of the RTEMS pre-qualification activity #3701 is a specified
>>>>>>       and validated subset of RTEMS.  For users of the pre-qualified
>>>>>> subset of
>>>>>>       RTEMS it is important to not accidentally use not pre-qualified
>>>>>>       features.  One way to achieve this, is to build only the sources of the
>>>>>>       pre-qualified feature set. This customized build is enabled by the new
>>>>>>       build configuration option RTEMS_QUALIFIED.  If it is enabled, then
>>>>>> only
>>>>>>       the pre-qualified subset of RTEMS is built and installed.
>>>>>>
>>>>>>       Building with RTEMS_QUALIFIED enable is currently only supported for
>>>>>> the
>>>>>>       sparc/leon3 BSP family.  To support an RTEMS_QUALIFIED enabled build,
>>>>>>       changes in the CPU port and the BSP are required to only use
>>>>>> features of
>>>>>>       the pre-qualified feature set.
>>>>>>
>>>>>>
>>>>>> Where is this documented?
>>>>> You mean a documentation of what needs to be done to create pre-qualified
>>>>> BSP? I
>>>>> don't have it available yet. A good place to add this would be the how-to
>>>>> section in the RTEMS Software Engineering manual.
>>>>>
>>>>>> This is a very large patch. Are you assuming that if "not qualified" is
>>>>>> specified,
>>>>>> then it is in the qualified set?
>>>>> No, the logic is reversed. Everything is built by default. Some parts are only
>>>>> enabled if RTEMS_QUALIFIED is not enabled, for example
>>>>> (spec/build/cpukit/objextra.yml):
>>>>>
>>>>> enabled-by:
>>>>>     not: RTEMS_QUALIFIED
>>>> It seems counter intuitive to me. I have no idea about qual work but my limited
>>>> understanding is everything is controlled yet this is the inverse of that and
>>>> anything anyone adds will by default be qualified?
>>>
>>> The goal of this patch set is to place each source and header file of RTEMS into
>>> two buckets, the pre-qualified bucket and the extra bucket. For two buckets you
>>> need just one option with two values. In the current patch it is RTEMS_QUALIFIED
>>> enabled or disabled.
>>
>> I think we should also include in our discussion code contained within this
>> define (or defines) in shared files and how we manage changes to that code? In
>> an ideal world we would not have any need for conditional code however I
>> appreciate this may not be possible or initial achievable. We should however
>> look for approaches that try to avoid this and understand the constraints the
>> define brings. For example can I change code in a qualification or FACE define
>> when I do not know the standards they support?
> 
> We should avoid using the C preprocessor for the profile customization. For the
> ECSS pre-qualification, there was no need to do this (with the exception of
> testbeginend.c, but this could be changed).

+1

>>> Since Joel already brought the FACE profile into play, an alternative could be
>>> this approach. Lets assume that we pre-qualified the FACE profile interfaces of
>>> RTEMS. Use RTEMS_EVERYTHING for the I want everything profile. Use
>>> RTEMS_QUALIFIED for the pre-qualified profile. Use RTEMS_FACE_PROFILE for
>>> interfaces of the FACE profile.
>>
>> If these are profiles I suggest RTEMS_PROFILE_EVERYTHING so RTEMS_PROFILE_.* can
>> find all profiles?
> 
> Yes, this make sense.
> 
>>
>>> The objects which are not pre-qualified would use this:
>>>
>>> enabled-by: RTEMS_EVERYTHING
>>
>> RTEMS_PROFILE_COMMERCIAL ?
>> RTEMS_PROFILE_INDUSTRIAL ?
>> RTEMS_PROFILE_RTEMS ?
>>
>> ... or something that reflects what we have always shipped? I borrowed those
>> labels from the way parts are classed. I tend to keep away from labels like new,
>> next, everything etc because they quickly date.
>>
>>>
>>> The objects which are pre-qualified would use this:
>>>
>>> enabled-by:
>>> - RTEMS_EVERYTHING
>>> - RTEMS_QUALIFIED
>>
>> Qualified to what? RTEMS_PROFILE_FACE defines something useful and concrete. Is
>> there an equivalent we could use for the work you have been doing?
> 
> The selection of the feature set for the ECSS pre-qualification was based on a
> survey:
> 
> https://ftp.rtems.org/pub/rtems/people/sebh/rtems-smp-userconsultation-27032019-release.pdf
> 
> https://ftp.rtems.org/pub/rtems/people/sebh/tn-space-profile-r6-23062019.pdf
> 

RTEMS_PROFILE_SPACE ? I just used the name on page 1 of this document.

>> I do not like the use of QUALIFIED as mentioned in my other post in this thread
>> because it implies something the open project should not and that is the code is
>> qualified. We will only provide pre-qualification data packs from the public
>> repos unless that has changed?
> 
> In the context of this patch set and the Software Requirements Engineering
> chapter of the RTEMS Software Engineering manual
> 
> https://docs.rtems.org/branches/master/eng/req/index.html
> 
> "qualified" means that:
> 
> 1. An interface specification exists for each qualified API element.
> 
> 2. Each qualified interface has a functional specification.
> 
> 3. The function is validated by a test.
> 
> 4. You get 100% branch and code coverage from the validation tests.  For some
> justified exceptions you can use unit tests.

Yes but there is no public proof so it is a promise until it is a fact and that
is achieved at the end of the process. What is written is correct but a repo is
in the present tense.

> If someone has a better name than "qualified" for this, I am happy to use it.

Good question. If RTEMS_PROFILE_SPACE is used can RTEMS_QUALIFIED be avoided?

>>> The objects which are pre-qualified or in the FACE profile would use this:
>>>
>>> enabled-by:
>>> - RTEMS_EVERYTHING
>>> - RTEMS_QUALIFIED
>>> - RTEMS_FACE_PROFILE
>>
>> The addition of these defines explodes the builds and testing we need to perform
>> for each commit. We need to discuss this some more and look for boundaries that
>> could help limit the builds and testing for each profile.
> 
> Yes, all these options lead to a combinatorial explosion. We have to be
> pragmatic and focus on a useful subset for tests.

I know for RTEMS_PROFILE_SPACE it will be a specific set of LEON archs? I see no
point building that profile for anything else. The point is a profile should
have a list of arches or BSPs associated to it as part of being accepted.

Chris

>>> Of RTEMS_EVERYTHING, RTEMS_QUALIFIED, and RTEMS_FACE_PROFILE exactly one option
>>> shall be enabled.
>>
>> Yes this makes sense. Should a source file not in a profile be considered an
>> error?
>>
>> Chris
> 