[PATCH 0/1] Generate application configuration option documentation
sebastian.huber at embedded-brains.de
Wed Jul 8 05:39:12 UTC 2020
[It seems Thunderbird misformatted the e-mail, maybe this one looks better.]
thanks for the detailed response.
On 08/07/2020 03:55, Chris Johns wrote:
> Hi Sebastian,
> Thank you for this patch. I am sorry but I would like to see this
> central repo
> issue resolved before any generated files are added to any of the
> project's repos.
> I understand to some level the path you are taking and moving along
> but these
> generated files are coming from a personal repo that has changes,
> processes and
> tools that are not being reviewed. I think it would be wise for us to
> agree on
> what is behind all this before we agree to what is being generated.
I rescheduled some work to be able to provide this patch earlier. Richi
Dubey asked a couple of questions with respect to the scheduler
implementation and some of them are related to how application
configuration options define internal scheduler data structures. So, It
would be nice to be able to reference them in Doxygen. You can review
this patch independent from the way it is produced.
> On 7/7/20 9:25 pm, Sebastian Huber wrote:
>> This patch adds the application configuration option documentation
>> from specification items:
> <gulp> That is a lot of files. I am not sure I am ready to accept
> as a lot of small files. I am happy to be persuaded otherwise.
> It is not clear to me if these YAML files generate documentation,
> headers, are
> requirements or something else.
Each application configuration option and group is provided as a
separate specification item (YAML file). The type of the specification
items is documented here:
Application configuration options are things on their own, so they have
their own items. This enables other items to reference application
configuration options. In the manual you have now a lot more cross
references for example if you compare it with the RTEMS 4.11
documentation. Items have their own history and can be reviewed
individually. I see this as benefits.
> There is nothing I can see in them that defines
> their `role` or `roles`.
When you look at an item
you see attributes like
This defines the type of an item. I corresponds to the specification
All attributes and the value types are defined by specification item
types. The specification item types are defined by specification items:
The type information is used to generate the documentation in the RTEMS
Software Engineering manual and is used to verify that the items are in
line with the type specification:
> If they do generate files how does that happen and how
> are those generated header files related to the C code they are an
The Doxygen header file of this patch is not read by the C compiler.
However, I would like to generate also the API header files from
specification items. I am now able to generate <rtems.h> and all header
files included by this header which define API elements. About 585 items
describe the API defined by <rtems.h>:
This item describes rtems_semaphore_create():
It is worth to have a closer look at this item.
The interface specification items can be used to generate the header
files with Doxygen markup, Sphinx sources for the manuals, and an
Interface Control Document (ICD, a document required by ECSS).
> How do I keep each piece in sync?
The basic work flow would be to update an item, then call one or more
scripts, run make or ./waf, or whatever, to generate the derived
content. This will alter the state of Git submodules. Then you create
patch sets from the changes in the submodules and send them for review.
After acceptance you push the changes and update the submodules.
> A compile error seems a long way from one
> of these YAML files.
Yes, this is a bit harder to correlate. I don't think this will be a
real issue. It is for example easy to figure out with "grep" which item
> For example, if I was to add a YAML file to this directory I have no
> idea where
> I look to find the naming, what roles are assumed, how it integrates
> into the
> existing documentation, doxygen or header files? I also have no idea
> how I would
> create a new API header file? What if I push a new header file to the
> repo with a new #include in rtems.h. Does a regenerate over write it?
Yes, this is a real issue. We have to think about how we want to
continue with the specification stuff in general. I see clear benefits
to generate content from a project-wide master data set to avoid the
copy and paste stuff we have right now.
> It seems to me the key is the central repo and how it functions, how
> easy or
> hard it is to learn about it and how well it will work in practice. A key
> concern is taking it from a single person repo, ie you, to a project
> wide repo
> with concurrent updates ad lots of moving pieces. It is easy to ask these
> questions and I appreciate they are not all easy to answer but I think
> we need
> to try.
My approach would be to document common tasks for the every day RTEMS
You also have to consider the future rate of change. The API parts,
application configuration options, and glossary terms for example are
all things which don't change every week.
With respect to the code which generates the content. It is written in
Python 3 with the new Python guidelines in mind:
It uses yapf for the code formatting. It uses the static analyzers
flake8, mypy, and pylint. There are unit tests using pytest which yield
100% branch coverage. It is not finished work. It was not reviewed, but
I think it is a good start.
>> The header file is generated by the following script and module:
>> The module uses currently a hack to resolve references external to
>> items, e.g. sections in the RTEMS Classic API Guide or URLs. I think
>> we need
>> specialized specification items for these external references.
> I do not understand what this means?
Being able to generate Doxygen markup and Sphinx sources from the same
data set was not a small task. We talk here about several weeks of work.
You can't take it for granted that this works at all in an area with
many cross references such as the application configuration options. For
all external references I had to create a table in the source code:
This is not really elegant.
>> Sebastian Huber (1):
>> Document application configuration options
>> cpukit/doxygen/appl-config.h | 4133 ++++++++++++++++++++++++++++++++++
>> 1 file changed, 4133 insertions(+)
>> create mode 100644 cpukit/doxygen/appl-config.h
> Is this information found in the user manual as well? I am struggling to
> understand the relationships and what is being presented where.
Yes, you find it here for example:
More information about the devel