[PATCH] build: Add RTEMS_QUALIFIED

Wed Nov 8 07:44:53 UTC 2023

On 06.11.23 21:55, Chris Johns wrote:
> On 6/11/2023 8:27 pm, Sebastian Huber wrote:
>> On 06.11.23 01:14, Chris Johns wrote:
>>> On 4/11/2023 1:31 am, Sebastian Huber wrote:
>>>> On 03.11.23 15:08, Joel Sherrill wrote:
>>>>> On Fri, Nov 3, 2023 at 3:58 AM Sebastian Huber
>>>>> <sebastian.huber at embedded-brains.de
>>>>> <mailto:sebastian.huber at embedded-brains.de>> wrote:
>>>>>
>>>>>       The goal of the RTEMS pre-qualification activity #3701 is a specified
>>>>>       and validated subset of RTEMS.  For users of the pre-qualified subset of
>>>>>       RTEMS it is important to not accidentally use not pre-qualified
>>>>>       features.  One way to achieve this, is to build only the sources of the
>>>>>       pre-qualified feature set. This customized build is enabled by the new
>>>>>       build configuration option RTEMS_QUALIFIED.  If it is enabled, then only
>>>>>       the pre-qualified subset of RTEMS is built and installed.
>>>>>
>>>>>       Building with RTEMS_QUALIFIED enable is currently only supported for the
>>>>>       sparc/leon3 BSP family.  To support an RTEMS_QUALIFIED enabled build,
>>>>>       changes in the CPU port and the BSP are required to only use features of
>>>>>       the pre-qualified feature set.
>>>>>
>>>>>
>>>>> Where is this documented?
>>>> You mean a documentation of what needs to be done to create pre-qualified BSP? I
>>>> don't have it available yet. A good place to add this would be the how-to
>>>> section in the RTEMS Software Engineering manual.
>>>>
>>>>> This is a very large patch. Are you assuming that if "not qualified" is
>>>>> specified,
>>>>> then it is in the qualified set?
>>>> No, the logic is reversed. Everything is built by default. Some parts are only
>>>> enabled if RTEMS_QUALIFIED is not enabled, for example
>>>> (spec/build/cpukit/objextra.yml):
>>>>
>>>> enabled-by:
>>>>     not: RTEMS_QUALIFIED
>>> It seems counter intuitive to me. I have no idea about qual work but my limited
>>> understanding is everything is controlled yet this is the inverse of that and
>>> anything anyone adds will by default be qualified?
>>
>> The goal of this patch set is to place each source and header file of RTEMS into
>> two buckets, the pre-qualified bucket and the extra bucket. For two buckets you
>> need just one option with two values. In the current patch it is RTEMS_QUALIFIED
>> enabled or disabled.
> 
> I think we should also include in our discussion code contained within this
> define (or defines) in shared files and how we manage changes to that code? In
> an ideal world we would not have any need for conditional code however I
> appreciate this may not be possible or initial achievable. We should however
> look for approaches that try to avoid this and understand the constraints the
> define brings. For example can I change code in a qualification or FACE define
> when I do not know the standards they support?

We should avoid using the C preprocessor for the profile customization. 
For the ECSS pre-qualification, there was no need to do this (with the 
exception of testbeginend.c, but this could be changed).

> 
>> Since Joel already brought the FACE profile into play, an alternative could be
>> this approach. Lets assume that we pre-qualified the FACE profile interfaces of
>> RTEMS. Use RTEMS_EVERYTHING for the I want everything profile. Use
>> RTEMS_QUALIFIED for the pre-qualified profile. Use RTEMS_FACE_PROFILE for
>> interfaces of the FACE profile.
> 
> If these are profiles I suggest RTEMS_PROFILE_EVERYTHING so RTEMS_PROFILE_.* can
> find all profiles?

Yes, this make sense.

> 
>> The objects which are not pre-qualified would use this:
>>
>> enabled-by: RTEMS_EVERYTHING
> 
> RTEMS_PROFILE_COMMERCIAL ?
> RTEMS_PROFILE_INDUSTRIAL ?
> RTEMS_PROFILE_RTEMS ?
> 
> ... or something that reflects what we have always shipped? I borrowed those
> labels from the way parts are classed. I tend to keep away from labels like new,
> next, everything etc because they quickly date.
> 
>>
>> The objects which are pre-qualified would use this:
>>
>> enabled-by:
>> - RTEMS_EVERYTHING
>> - RTEMS_QUALIFIED
> 
> Qualified to what? RTEMS_PROFILE_FACE defines something useful and concrete. Is
> there an equivalent we could use for the work you have been doing?

The selection of the feature set for the ECSS pre-qualification was 
based on a survey:

https://ftp.rtems.org/pub/rtems/people/sebh/rtems-smp-userconsultation-27032019-release.pdf

https://ftp.rtems.org/pub/rtems/people/sebh/tn-space-profile-r6-23062019.pdf

> 
> I do not like the use of QUALIFIED as mentioned in my other post in this thread
> because it implies something the open project should not and that is the code is
> qualified. We will only provide pre-qualification data packs from the public
> repos unless that has changed?

In the context of this patch set and the Software Requirements 
Engineering chapter of the RTEMS Software Engineering manual

https://docs.rtems.org/branches/master/eng/req/index.html

"qualified" means that:

1. An interface specification exists for each qualified API element.

2. Each qualified interface has a functional specification.

3. The function is validated by a test.

4. You get 100% branch and code coverage from the validation tests.  For 
some justified exceptions you can use unit tests.

If someone has a better name than "qualified" for this, I am happy to 
use it.

> 
>> The objects which are pre-qualified or in the FACE profile would use this:
>>
>> enabled-by:
>> - RTEMS_EVERYTHING
>> - RTEMS_QUALIFIED
>> - RTEMS_FACE_PROFILE
> 
> The addition of these defines explodes the builds and testing we need to perform
> for each commit. We need to discuss this some more and look for boundaries that
> could help limit the builds and testing for each profile.

Yes, all these options lead to a combinatorial explosion. We have to be 
pragmatic and focus on a useful subset for tests.

> 
>> Of RTEMS_EVERYTHING, RTEMS_QUALIFIED, and RTEMS_FACE_PROFILE exactly one option
>> shall be enabled.
> 
> Yes this makes sense. Should a source file not in a profile be considered an error?
> 
> Chris

-- 
embedded brains GmbH
Herr Sebastian HUBER
Dornierstr. 4
82178 Puchheim
Germany
email: sebastian.huber at embedded-brains.de
phone: +49-89-18 94 741 - 16
fax:   +49-89-18 94 741 - 08

Registergericht: Amtsgericht München
Registernummer: HRB 157899
Vertretungsberechtigte Geschäftsführer: Peter Rasmussen, Thomas Dörfler
Unsere Datenschutzerklärung finden Sie hier:
https://embedded-brains.de/datenschutzerklaerung/