Use specification items to generate documentation

[Sebastian Huber]

On 08/01/2020 19:49, Gedare Bloom wrote:
> On Wed, Jan 8, 2020 at 7:55 AM Sebastian Huber
> <sebastian.huber at embedded-brains.de>  wrote:
>> Hello,
>>
>> for the RTEMS pre-qualification activity we need a specification of
>> RTEMS. I would like to use this opportunity to move structured
>> information which is present in hand written and hand formatted content
>> in the RTEMS documentation into a structured data format (YAML).
>> Documentation source files can be generated from the YAML files. Having
>> information available in an easily machine readable format with
>> meta-information gives us a lot of flexibility. For example to adjust
>> the presentation we just have to change the generator program and not
>> the hand written content. The presentation of things may depend on the
>> standard of interest, e.g. ECSS, IEC 61508, ISO 26262, DO-178C, etc.
>>
> The proposal makes sense to me.
> 
>> To get started, I would like to use the glossary of terms and the
>> application configuration as a use case. The conversion from hand
>> written and hand formatted content to generated content needs three things:
>>
>> 1. A definition of the data format (YAML with Doorstop attributes)
>>
>> 2. The specification items in the defined data format.
>>
>> 3. A generator program which uses the specification items as input and
>> outputs the documentation source files.
>>
>> The new build system introduced build specification items in the RTEMS
>> sources under "spec/build". I would like to place all of the RTEMS
>> specification under the "spec" directory. For the glossary I propose to
>> use "spec/glossary". For the application configuration categories I
>> propose "spec/cfg" and for the application configuration options
>> "spec/cfg/opt". The layout of the application configuration items
>> enables an easy reassignment of options to categories.
>>
> Will there also be specifications for RTEMS configuration? (I.e.,
> options specified during RTEMS build such as --enable-networking.) If
> so, maybe spec/cfg needs to be separated or subdirectoried for 'app'
> vs 'rtems'?

The new build system doesn't use a --enable-networking command line 
option. For each BSP configuration a list of key-value pairs in 
INI-style files is used. The RTEMS_NETWORKING related option is defined 
like this:

https://git.rtems.org/sebh/rtems.git/tree/spec/build/cpukit/RTEMS-BUILD-CPUKIT-OPTNET.yml?h=build

One remark: the interpretation of the "links" attribute is item 
type-dependent. In the build specification items the links are 
interpreted as "depends on". In the requirements related specification 
items the links are interpreted as "is a refinement of". In validation 
test items the links are interpreted as "validates".

> 
>> The definition of the data format should be added to the RTEMS Software
>> Engineering manual to the Software Requirements Engineering chapter.
>>
>> The generator program should be written in Python and placed into the
>> rtems-tools repository. The generator program needs paths to the RTEMS
>> sources and the RTEMS documentation sources. The paths should be
>> specified via the command line.
>>
> Similar paths are specified as options for other tools (e.g.,
> rtems-test). Aim for consistency (and code reuse) in these interfaces.

Yes, we should try to name the command line options consistently across 
our tools.

> 
>> For a proof-of-concept please have a look at the attached two patches.
>>
> The .doorstop.yml are "hidden" files in typical *nix filesystems. Are
> these automatically generated files and not to be modified or
> necessarily read by humans? If so, fine, but otherwise they should not
> be "hidden"

The .doorstop.yml is defined by the Doorstop tool. A maintainer may have 
to edit it if new item attributes are added which contribute to the item 
fingerprint:

https://doorstop.readthedocs.io/en/latest/cli/creation/#document-configuration

> 
>> What do you think of it?
>>
> Cursory glance of the patches makes sense. Is there a good way to
> validate the format or error-check the configuration yml files, or it
> is a manual job to create the file and check correctness?

At some point we need also a tool to check the specification for 
consistency. Maybe we could use

https://json-schema.org/

The YAML/JSON world seems to be a bit scattered to me with lots of home 
grown solutions.

-- 
Sebastian Huber, embedded brains GmbH

Address : Dornierstr. 4, D-82178 Puchheim, Germany
Phone   : +49 89 189 47 41-16
Fax     : +49 89 189 47 41-09
E-Mail  : sebastian.huber at embedded-brains.de
PGP     : Public key available on request.

Diese Nachricht ist keine geschäftliche Mitteilung im Sinne des EHUG.