Doorstop Issues

[Joel Sherrill]

On Sun, Apr 19, 2020 at 10:14 AM Sebastian Huber <
sebastian.huber at embedded-brains.de> wrote:

> Hello,
>
> I am about to add the next couple of hundred specification items for the
> RTEMS pre-qualification activity. Before I do this, I would like to
> present some issues that popped up during my daily work with Doorstop.
> Doorstop is a requirements management tool which was selected for the
> pre-qualification activity:
>
>
> https://docs.rtems.org/branches/master/eng/req-eng.html#selected-tool-doorstop
>
> I am quite happy that Jan Sommer suggested to use this tool. I has some
> pretty nice features. In particular, I think the file based organization
> in YAML format forming a directed acyclic graph is really the right
> thing for this project. However, it turned out that it lacks the
> required flexibility.
>
> 1. Its use case is requirements management. So it has some standard
> attributes useful in this domain, like derived, header, level,
> normative, ref, reviewed, and text. However, I want to use it more
> generally for specification items and these attributes make not always
> sense. Having them in every item is just overhead and may cause confusion.
>

If they are documented well and their flow through the process is clear,
there should not be huge confusion. Performance overhead maybe.

I have seen project requirements that had schedule phase or HW/SW associated
with each requirement. On those projects, it seemed to me that the primary
problem was that the tooling couldn't check that parent requirement
custom attributes had an relationship to those of the children. Schedule
phases and HW/SW didn't match across the levels of requirements and
there was no way to know if it was intentional or just 100s of mistakes.

>
> 2. The links cannot have custom attributes, e.g. role, enabled-by.
>

i assume by enabled-by you are tracking to an RTEMS compile or condefs.h
configure option.

The FACE Technical Standard uses the term conditional to describe
capabilities
an implementation of the standard may provide. The term optional is also
used
in the general requirements community but these. I recall that is what
POSIX
calls them. Interestingly, neither standard uses will or may in those
requirements
as the textbooks suggest but collects them under optional feature
categories.


> 3. Inside a document (directory) items are supposed to have a common
> type. I would like to store at a hierarchy level also distinct
> specializations.
>
> 4. The verification if the items is quite limited. We need verification
> with type-based rules.
>

This sounds like you have found the problem I was referring to above.
Parent and child should follow rules on custom attributes.

>
> 5. The UIDs in combination with the document hierarchy lead to
> duplication, e.g. a/b/c/a-b-c-d.yml. You have the path (a/b/c) also in
> the file name (a-b-c). You cannot have relative UIDs in links (e.g.
> ../parent-req) . The specification items may contain multiple
> requirements, e.g. min/max attributes. There is no way to identify them.
>

This seems fixable.

>
> I will probably stop using Doorstop and use a custom tooling. With
> Python, this is quite easy. I already have the infrastructure for this in:
>
> https://git.rtems.org/sebh/rtems-qual.git/
>
> I will try this out with the new build system.
>

You are indeed pushing beyond what a requirements management tool
is usually scoped to do. Which is funny because some of the commercial
tools have capabilities not discussed so far. They are usually metrics,
static analysis, and/or code coverage. We address two of the three with
custom tools.

I agree with Chris that you should discuss this with the Doorstop project.
RTEMS has certainly grown with odd requirements thrown at it. It would
be rude to not give another open source project the same opportunity.
Of course, that may not change the end result.

If we do end up beyond Doorstop's mission, then it is worth considering
a sibling tool to do the generation. But I also know that Doorstop is not
a huge body of Python either.

--joel


>
> _______________________________________________
> devel mailing list
> devel at rtems.org
> http://lists.rtems.org/mailman/listinfo/devel
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.rtems.org/pipermail/devel/attachments/20200420/53e5778a/attachment.html>