[PATCH] build: Add RTEMS_QUALIFIED

Wed Nov 8 07:26:14 UTC 2023

On 08.11.23 00:17, Gedare Bloom wrote:
> On Mon, Nov 6, 2023 at 1:55 PM Chris Johns <chrisj at rtems.org> wrote:
>>
>> On 6/11/2023 8:27 pm, Sebastian Huber wrote:
>>> On 06.11.23 01:14, Chris Johns wrote:
>>>> On 4/11/2023 1:31 am, Sebastian Huber wrote:
>>>>> On 03.11.23 15:08, Joel Sherrill wrote:
>>>>>> On Fri, Nov 3, 2023 at 3:58 AM Sebastian Huber
>>>>>> <sebastian.huber at embedded-brains.de
>>>>>> <mailto:sebastian.huber at embedded-brains.de>> wrote:
>>>>>>
>>>>>>       The goal of the RTEMS pre-qualification activity #3701 is a specified
>>>>>>       and validated subset of RTEMS.  For users of the pre-qualified subset of
>>>>>>       RTEMS it is important to not accidentally use not pre-qualified
>>>>>>       features.  One way to achieve this, is to build only the sources of the
>>>>>>       pre-qualified feature set. This customized build is enabled by the new
>>>>>>       build configuration option RTEMS_QUALIFIED.  If it is enabled, then only
>>>>>>       the pre-qualified subset of RTEMS is built and installed.
>>>>>>
>>>>>>       Building with RTEMS_QUALIFIED enable is currently only supported for the
>>>>>>       sparc/leon3 BSP family.  To support an RTEMS_QUALIFIED enabled build,
>>>>>>       changes in the CPU port and the BSP are required to only use features of
>>>>>>       the pre-qualified feature set.
>>>>>>
>>>>>>
>>>>>> Where is this documented?
>>>>> You mean a documentation of what needs to be done to create pre-qualified BSP? I
>>>>> don't have it available yet. A good place to add this would be the how-to
>>>>> section in the RTEMS Software Engineering manual.
>>>>>
>>>>>> This is a very large patch. Are you assuming that if "not qualified" is
>>>>>> specified,
>>>>>> then it is in the qualified set?
>>>>> No, the logic is reversed. Everything is built by default. Some parts are only
>>>>> enabled if RTEMS_QUALIFIED is not enabled, for example
>>>>> (spec/build/cpukit/objextra.yml):
>>>>>
>>>>> enabled-by:
>>>>>     not: RTEMS_QUALIFIED
>>>> It seems counter intuitive to me. I have no idea about qual work but my limited
>>>> understanding is everything is controlled yet this is the inverse of that and
>>>> anything anyone adds will by default be qualified?
>>>
>>> The goal of this patch set is to place each source and header file of RTEMS into
>>> two buckets, the pre-qualified bucket and the extra bucket. For two buckets you
>>> need just one option with two values. In the current patch it is RTEMS_QUALIFIED
>>> enabled or disabled.
>>
>> I think we should also include in our discussion code contained within this
>> define (or defines) in shared files and how we manage changes to that code? In
> +1
> 
>> an ideal world we would not have any need for conditional code however I
>> appreciate this may not be possible or initial achievable. We should however
>> look for approaches that try to avoid this and understand the constraints the
>> define brings. For example can I change code in a qualification or FACE define
>> when I do not know the standards they support?
>>
> +1
> 
> For now these are the two things that stand out to me about this
> patch. Historically, we have tried to get away from CPP defines to
> tailor builds at a global level. I'm not sure the extent of what must
> be split within a file, versus what can be split/qualified at a file
> level. Assuming one qualifies API subsets, then I would imagine it
> would be the entire set of files brought in by those subsets. I'm not
> interested in maintenance burden of two versions of basically the same
> code. This will eventually lead to inconsistencies. I understand the
> intent may be to have a code base closer to what can be ultimately
> qualified, but I see this as problematic from a long-term maintenance
> perspective.

I share your concern with respect to the customization based on C 
preprocessor defines. To avoid this, a couple of files were restructured 
to avoid unnecessary dependencies. In this patch, there is a single file 
with #ifdef RTEMS_QUALIFIED. After having a second view on this file, I 
think that I could get rid of this #ifdef. So, the profile customization 
would be entirely done by the build system.

> 
> What I see here is that certain APIs are being circumvented presumably
> because they are not part of someone's qualified package. As far as I
> know, the goal of pre-qual was never to create qualified packages, but
> to make it easier for downstream users to make qualified packages of
> their choosing. Pushing constraints back into rtems.git because of an
> inability to qualify some parts of the code is opposite of the goal.
> It is like a somewhat kinder form of stripping down RTEMS to what is
> qualified and shipping that, which is what I understood to be the
> previous methodology that should be avoided.
> 
> I would also not like to see variations of the same files for
> different "profiles" or qualified targets.

In the context of this patch set and the Software Requirements 
Engineering chapter of the RTEMS Software Engineering manual

https://docs.rtems.org/branches/master/eng/req/index.html

"qualified" means that:

1. An interface specification exists for each qualified API element.

2. Each qualified interface has a functional specification.

3. The function is validated by a test.

4. You get 100% branch and code coverage from the validation tests.  For 
some justified exceptions you can use unit tests.

It would be multi person years of work to do this for everything in the 
RTEMS sources, the Newlib C library and the GCC provided libraries such 
as libgcc. So, the only feasible approach is to strip the feature set 
down and then gradually extend it based on user needs. I maintained this 
patch set since 2021. It was simplified over time. However, maintaining 
this patch set outside of the RTEMS mainline is quite time consuming.

If you want to qualify against a particular standard, then additional 
things need to be done (code reviews, static code analysis, code 
metrics, etc.).

-- 
embedded brains GmbH
Herr Sebastian HUBER
Dornierstr. 4
82178 Puchheim
Germany
email: sebastian.huber at embedded-brains.de
phone: +49-89-18 94 741 - 16
fax:   +49-89-18 94 741 - 08

Registergericht: Amtsgericht München
Registernummer: HRB 157899
Vertretungsberechtigte Geschäftsführer: Peter Rasmussen, Thomas Dörfler
Unsere Datenschutzerklärung finden Sie hier:
https://embedded-brains.de/datenschutzerklaerung/