Traceability from specification to source code?

[Joel Sherrill]

On Fri, Apr 17, 2020 at 9:06 AM Sebastian Huber <
sebastian.huber at embedded-brains.de> wrote:

> Hello,
>
> with the pre-qualified build of RTEMS
>
> https://lists.rtems.org/pipermail/devel/2020-April/059445.html
>
> I try out the following approach. I run Doxygen on the set of
> pre-qualified files to produce the documentation and a tagfile:
>
> # The TAGFILES tag can be used to specify one or more tag files. For
> each tag
> # file the location of the external documentation should be added. The
> format of
> # a tag file without this location is as follows:
> # TAGFILES = file1 file2 ...
> # Adding location for the tag files is done as follows:
> # TAGFILES = file1=loc1 "file2 = loc2" ...
> # where loc1 and loc2 can be relative or absolute paths or URLs. See the
> # section "Linking to external documentation" for more information about
> the use
> # of tag files.
> # Note: Each tag file must have a unique name (where the name does NOT
> include
> # the path). If a tag file is not located in the directory in which
> doxygen is
> # run, you must also specify the path to the tagfile here.
>
> The tagfile is in XML and contains things like this:
>
>      <member kind="function">
>        <type>RTEMS_INLINE_ROUTINE uint64_t</type>
>        <name>_Watchdog_Ticks_from_seconds</name>
> <anchorfile>group__RTEMSScoreWatchdog.html</anchorfile>
> <anchor>gae368bd39a2a612b0e7701d954d363ce6</anchor>
>        <arglist>(uint32_t seconds)</arglist>
>      </member>
>
> In the specification, I place a reference like this:
>
> ref:
> - kind: function
>    name: _Watchdog_Ticks_from_seconds
>
> I can then use the tagfile to lookup this reference. With the
> information in the tagfile, I can generate additional documentation
> content, for example:
>
> /**
>   * @fn RTEMS_INLINE_ROUTINE uint64_t _Watchdog_Ticks_from_seconds
> (uint32_t seconds)
>   *
>   * bla blub
>   */
>
> In a second Doxygen run, this information is added to the
> _Watchdog_Ticks_from_seconds() documentation:
>
> Converts the seconds to ticks.
>
> Parameters
>      seconds    The seconds to convert to ticks.
>
> Returns
>      seconds converted to ticks.
>
> bla blub
>
> The benefits of this approach is that we don't have to modify the RTEMS
> sources for the specification traceability and the references in the
> specification can be done at a high level without a lot of details.
>

I'm wading in late and the comments early on about annotations, tools, etc
still  applies.

But based on some non-RTEMS work OAR is currently doing and use of a
commercial tool which is approved for requirements and coverage analysis
by various certification authorities, I have more practical experience than
I
did when we started this.

Code annotations are a must. Sebastian is right that putting something
above the body is the way to go. We are using a requirements comment
block above the entry point to the method. A script integrates with the
commercial tool. They analyse the source and provide file, line number,
and signature (e.g. tags). From that, you can easily find those special
comments with requirements. You do have to address the case where
there are no requirements or an incorrectly formatted set. But that's
the practical matter of what you have to do.

If you put in a bad requirement tag, then it will just show up as an
error when you process and some requirement will be uncovered
in the traceability report.

There are a lot of details to work out. Doxygen is slow. Would ctags
or something similar be faster?

We also do not have Level A requirements on our project but are being
very conscious of what we are missing to get there and are trying to
do things right so no rework is required to bump the level, just more work.
For example, the commercial tool generated Statement, Decision Coverage,
and MCDC. When an area hits 100% statement, we plan to set the other
coverage types acceptable minimum percentage at that point to what it
just happens to be.  I think figuring out the RTEMS Project approach to
this is a wise thing.

Even though it is a largely ignored tool in the RTEMS Project quiver,
covoar is quite capable of being extended from doing object coverage
to tracing the requirements. That was part of what it was conceived of
doing. A bit of love to tidy up a few issues and make sure we like the
reports would be a valuable thing.

FWIW I have finally confirmed what I long believed. That writing simple
single logic expression if conditions makes statement coverage results
approach those of the higher levels. Those address if conditions with
multiple sub-expressions. Avoid using multiple sub-expressions and
coverage is much simpler.

I hope I actually helped on this thread.

>
> _______________________________________________
> devel mailing list
> devel at rtems.org
> http://lists.rtems.org/mailman/listinfo/devel
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.rtems.org/pipermail/devel/attachments/20200417/f62b9ef0/attachment.html>