Traceability from specification to source code?

Tue Mar 31 06:34:30 UTC 2020

On 31/03/2020 06:32, Chris Johns wrote:

> On 2020-03-31 05:52, Sebastian Huber wrote:
>> Hello Gedare,
>>
>> On 30/03/2020 18:17, Gedare Bloom wrote:
>>> This question is related to
>>> https://lists.rtems.org/pipermail/devel/2019-September/055519.html
>>> about code annotations, and similar discussion raised there applies
>>> here.
>>
>> thanks for referencing this thread. Unfortunately, the outcomes of 
>> this thread are not written down in the Software Engineering Manual 
>> (definitely our fault). I read the thread again, but I think the 
>> situation here is a bit different.
>>
>> For the traceability between source code and the specification we 
>> likely don't need additional complex third-party tools (I depends a 
>> bit if we want to use a C statement as a scope for an annotation for 
>> example, but this is more a comfort feature). Also in contrast to the 
>> annotations for the formal methods tools, the link annotations are 
>> simple and you don't need expert knowledge to understand them.
>
> I think the 3 major headings Joel listed are valid, that is 
> Annotation, Supporting Tools, and Maintenance.
Yes, these are important criteria to select one method or another.
>
> How smart is the scanner used to extract these strings and how 
> resilient is the expected qual out come to localised changes in the 
> code around these label? I have looked at your example for a while now 
> to determine the relationship between the opening label and the 
> default value and it seems complex to me and I would not like to be 
> making changes around this area without either a clear understanding 
> of what is happening or a disregard for the quality of the qual 
> information. I do not like either of these options. Add to this the C 
> language can be rather tricky to parse at the best of times.

If we use a C preprocessor or C language construct to define the scope 
of an annotation, then the scanner needs to be much more complex. A 
simple annotation could just use a number of lines (the {6} in the example):

/* 
$satisfy{RTEMS-ACFG-OPT-MAXBARRIERS:appl-config-option-default-value}{6} */
OBJECTS_INFORMATION_DEFINE_ZERO(
   _Barrier,
   OBJECTS_CLASSIC_API,
   OBJECTS_RTEMS_BARRIERS,
   OBJECTS_NO_STRING_NAME
);

>
> How do we limit what can be added? Joel refers to this as 
> "Invasive/Intruding/Bulk". We currently have doxygen, we then add this 
> and then maybe Frama-C, i.e. signal-to-noise ratio starts to make 
> seeing the actual code hard.

We need also traceability to the software architecture (Doxygen groups) 
and software components. I updated the glossary to include these terms:

https://lists.rtems.org/pipermail/devel/2020-March/058765.html

For the things represented by Doxygen markup, the proposal is to add an 
@satisfy{} custom command:

https://docs.rtems.org/branches/master/eng/req-eng.html#traceability-between-software-requirements-architecture-and-design

Yes, adding too much annotation could make the code hard to read.

>
>> The formal methods support is optional for a pre-qualification and 
>> only required for higher criticality categories. The traceability, 
>> however, is a basic requirement of the standards. This is something 
>> we have establish, otherwise there will be no RTEMS Qualification 
>> project at all.
>
> I would like to understand what the requirement for traceability is 
> you need to satisfy? 
For example ECSS-E-ST-40C Clause 5.8.3.5a:

"The supplier shall verify the software code ensuring that: 1. the code 
is externally consistent with the requirements and design of the 
software item; 2. there is internal consistency between software units; 
3. the code is traceable to design and requirements, testable, correct, 
and in conformity to software requirements and coding standards; 4. the 
code that is not traced to the units is justified; 5. the code 
implements proper events sequences, consistent interfaces, correct data 
and control flow, completeness, appropriate allocation of timing and 
sizing budgets, and error handling; 6. the code implements safety, 
security, and other critical requirements correctly as shown by 
appropriate methods; 7. the effects of run-time errors are controlled; 
8. there are no memory leaks; 9. numerical protection mechanisms are 
implemented. {NOTE: "AM" means that the value is agreed with the 
customer and measured as per ECSS-Q-ST-80 clause 6.3.5.2.}"

In item 3 we have "the code is traceable to design and requirements, 
testable, correct, and in conformity to software requirements and coding 
standards".

Object code is mentioned only in ECSS-E-ST-40C Clause 5.8.3.5e:

"In case the traceability between source code and object code cannot be 
verified (e.g. use of compiler optimization), the supplier shall perform 
additional code coverage analysis on object code level as follows 
(category=coverage). Object code coverage: A=100% B=N/A C=N/A D=N/A. 
{NOTE: N/A means not applicable.}"

It is only required for criticality category A, so not needed for the 
current ESA activity. However, we still should consider how this could 
be achieved.

> In your example is it to know the definition, i.e. file:line, for..
>
>  CONFIGURE_MAXIMUM_BARRIERS
>  _Objects_.*_Barrier_.*      (A guess at expanded label)
>
> ?
>
> Have you considered a gcc plugin [1] or some other compiler generated 
> data format that contains this detail, ie DWARF?
No, but thanks for the hint. This is something to look at.
> In other words, are you adding tags and related detail because parsing 
> the C code is hard? Or I am mistaken in assuming the compiler sees and 
> understands all this stuff? I am assuming it has to by definition or 
> there is not need to track it?

I think we first have to clarify up to which level we would like to have 
the traceability. In the example above, the requirement is "The 
application configuration option CONFIGURE_MAXIMUM_BARRIERS shall have a 
default value of 0." So there should be some code in the system which 
satisfies this, here in particular:

OBJECTS_INFORMATION_DEFINE_ZERO(
   _Barrier,
   OBJECTS_CLASSIC_API,
   OBJECTS_RTEMS_BARRIERS,
   OBJECTS_NO_STRING_NAME
);

Tracing this to the object level:

         .global _Barrier_Information
         .section        .data._Barrier_Information,"aw", at progbits
         .align 4
         .type   _Barrier_Information, #object
         .size   _Barrier_Information, 48
_Barrier_Information:
         .long   1375797248
         .long   0
         .long   _Objects_Allocate_none
         .long   0
         .half   0
         .half   0
         .half   0
         .half   0
         .long   _Barrier_Information+28
         .long   0
         .long   _Barrier_Information+24
         .long   0
         .long   0
         .long   0

Is another step. I guess, for this we would need a sophisticated 
tooling. In the current ESA activity, traceability to object code is not 
needed.

>
> Chris
>
> [1] https://github.com/davidmalcolm/gcc-python-plugin