Requirement Document generator tool

[Joel Sherrill]

On Fri, Jan 3, 2020, 4:49 AM Jose Valdez <Jose.Valdez at edisoft.pt> wrote:

> Hello Gedare and Joel,
>
> Please see my answers below for both of you (see JV02012019).
>
> Please tell me, if you need further clarification.
>
> Best regards
>
> José
>
> -----Original Message-----
> From: Gedare Bloom [mailto:gedare at rtems.org]
> Sent: quinta-feira, 2 de janeiro de 2020 17:21
> To: Joel Sherrill
> Cc: Jose Valdez; devel at rtems.org
> Subject: Re: Requirement Document generator tool
>
> Hello José et al.,
>
> I have only a few points I would like clarified, below:
>
> On Mon, Dec 16, 2019 at 3:35 PM Joel Sherrill <joel at rtems.org> wrote:
> >
> >
> >
> > On Mon, Dec 16, 2019 at 7:29 AM Jose Valdez <Jose.Valdez at edisoft.pt>
> wrote:
> >>
> >> Hello Chris,
> >>
> >> Thank you for your reply.
> >>
> >> Please find below my answers.
> >>
> >> José
> >>
> >> -----Original Message-----
> >> From: Chris Johns [mailto:chrisj at rtems.org]
> >> Sent: sexta-feira, 13 de dezembro de 2019 19:09
> >> To: Jose Valdez; devel at rtems.org
> >> Subject: Re: Requirement Document generator tool
> >>
> >> On 14/12/19 3:09 am, Jose Valdez wrote:
> >> > In the scope of ESA's RTEMS SMP project, we are developing a tool
> which will
> >> > allow to generate RTEMS software documentation
> >>
> >> I assume this is the pre-qual effort.
> >>
> >> JV: That's correct.
> >>
> >> > (Requirements, Design, Test Specification and other software
> documents).
> >>
> >> How do these documents relate to the project's current documentation?
> It may
> >> help to explain the purpose of each of these documents and the target
> user.
> >
> >
> >
> >>
> >> JV: The idea of the project is to allow an user to pre-qualify RTEMS
> for space applications. In that sense, the goal of the tool is to save the
> pre-qualification effort, by creating automatically the parts of the
> necessary documentation, which may be automatized. Generically parts which
> may be automatized are:
> >> -> Traceabilities (ex: Requirements against Architecture)
> >> -> Requirement coverage (Tests against requirements)
> >> -> Test results verification
> >> -> Creation of document templates
> >> -> source code metrics collection
> >> -> etc.
> >>
> >> The current scope is to create the documentation in accordance with ESA
> standard (ECSS), which defines the following necessary documents (and also
> the template) for a software product (note: the description text of each
> document was taken from ECSS, for helping you to understand the goal of
> each document):
> >
> >
> > This list of documents has some of which I would expect to
> > be hand-written, one-time documents. Others related to requirements,
> traceability,
> > and tests, I would expect to be generated. Can you clarify which
> documents
> > fall into which category?
> >
> > And how these documents related to RTEMS Pre-Qualification. For example,
> I don't see the
> > need for a Software Reuse File for RTEMS. That would seem to be
> something a project adopting
> > RTEMS might need.
> >
> > These documents also show an intentional bias to ECSS which is OK for
> you guys but
> > doesn't help in the RTEMS.org broader goal of having technical data for
> qualification which
> > fulfills the needs for multiple standards (NASA Quality, DO-178,
> automotive, etc.) Please
> > keep in mind that RTEMS is a world-wide project, independent of ESA and
> we would
> > like this effort to be able to address the multiple stakeholders.
> >
> > Some of these have the same names as other quality standard, others
> don't.
>
>
> JV02012019: Regarding your first question, a document itself may have both
> parts hand-written and generated. I understand that probably we missed to
> give you the full technical scope of the project (which is difficult by
> only exchanging e-mails). Please wait for our first input (Requirement
> Manager), which will include instructions on how to use and then you may
> perform your assessments and clarify your technical doubts.
>

One of the things that has unfortunately been lost over the years in
computer documentation is a Theory of Operations. This is independent of
the implementation.

In this case, you guys are trying to automate parts of an information flow
and traceability of that flow. What would help is an implementation
independent description of that information flow so we can say for
information X, traceability link Y,  etc, this is what we will do.

This is a software engineering process and there are lots of ways to
achieve it from Excel spreadsheets and pain to hugely expensive proprietary
tools. All achieve the same information and trace links.


>
> JV02012019: The documents which are not applicable to RTEMS project (like
> Software Reuse File) will be created, but will not have any content (the
> sections will have text like "Not Applicable").
>

What about code in RTEMS from third parties?

Doesn't this document have a perspective of a code base?


> JV02012019: I understand your concern about having the tool producing
> output for other standards/applications. Unfortunately this is not possible
> in this project (due to budget). However, the philosophy for this project
> should be to open the possibility to extend the tool, and hence other users
> may add the functionality to produce the documentation according with their
> standards.
>

If we understand the abstract process, then adapting it to other processes
should be easier. At least, we are less likely to miss things.

Also if the RTEMS Project has a defined process that is implemented in part
with certain tools and documents, knowing the abstract process, goals, etc.
makes it easier in the future to say ESA X is the same as FUTURE Standard Y.

--joel




> >
> >>
> >>
> >>
> ------------------------------------------------------------------------------------------------
> >> Software Development Plan (SDP) - Its purpose is to describe the
> established management and development approach for the software items to
> be defined by a software supplier to set up a software project in
> accordance with the customer requirements.
> >>
> >> Software Product Assurance Plan (SPAP) - The purpose of the software
> product assurance plan is to provide information on the organizational
> aspects and the technical approach to the execution of the software product
> assurance programme
> >>
> >> Software Configuration Management Plan (SCMP) - The objective of the
> configuration management plan is to provide in a single document all
> elements necessary to ensure that the implementation of configuration
> management meets customer requirements and is commensurate with the
> programme or project, organization, and management structure.
> >>
> >> Software Configuration File (SCF) - The objective of the software
> configuration file is to provide the configuration status of the software
> configuration item. It controls its evolution during the programme or
> project life cycle.
> >>
> >> Software Reuse File (SRF) - Its purpose is to document the analysis to
> be performed on existing software intended to be reused.
> >> The global objectives of the software reuse file are to document all
> the information used to decide about the reuse (or not) of existing
> software and to plan the specific actions undertaken to ensure that the
> reused software meets the project requirements.
> >> The SRF is also used to document software developed for intended reuse,
> such that it is ready when the software is actually reused.
> >>
> >> Software User Manual (SUM) - Its purpose is to provide instructions for
> the users of the software.
> >> For flight software, the relevant parts of the SUM are a contribution
> to the flight operation manual (FOM).
> >>
> >> Software Validation Specification (SVS) - The purpose of this DRD is to
> describe the testing, analysis, inspection and
> >> review of design specifications, and is used to document:
> >> -> the software validation specification with respect to the technical
> specification (TS), and
> >> -> the software validation specification with respect to the
> requirements baseline (RB).
> >> It provides a unique template for the software validation specification
> document, to be instantiated for, either the technical specification, or the
> >> requirement baseline. The acronym SVS w.r.t. TS is used to designate
> the software validation specification with respect to the technical
> specification whilst SVS w.r.t. RB is used to designate the software
> validation specification with respect to the requirement baseline.
> >>
> >> Software Design Document (SDD) - It provides description of the
> software architectural design and the software detailed design. Internal
> interfaces design is also included in this document.
> >>
> >> Software Release Document (SRelD) - Its purpose is to describe a given
> software version in terms of known problems, limitations or restrictions
> with respect to its approved baseline.
> >>
> >> Interface Control Document (ICD) - It describes all the (preliminary
> and update) external interfaces.
> >>
> >> Software Product Assurance Milestone Report (SPAMR) - The main purpose
> of the software product assurance milestone report is to collect and
> present at project milestones the reporting on the software product
> assurance activities performed during the past project phases.
> >>
> >> Software Review Plan (SRevP) - This document provides means for
> identifying and structuring all of the activities and information required
> in a project review. It identifies the information outputs and activities
> necessary to complete the process. It also provides a check-list of
> activities and information required for each of the project reviews
> identified in the ECSS Management Standards.
> >>
> >> Software Verification Report (SVR) - Its purpose is to present gathered
> results of all the software verification activities that have to be
> executed along the software development life cycle according to the SVerP.
> It is organized per process, with the exception of the timing and sizing
> issues which are gathered in a separate section. Each process verification
> report can be placed into a separate document.
> >>
> >> Software Validation Plan (SVP) - Its purpose is to provide the
> definition of organizational aspects and management approach to the
> implementation of the validation tasks. The objective of the software
> validation plan is to describe the approach to the implementation of the
> validation process for a software product.
> >>
> >> Software Requirements Specification (SRS) - It describes the functional
> and non functional requirements applicable to the software item.
> >>
> >> Software Unit/Integration Test Plan (SUITP) - The purpose of this
> document is to describe the tests plans, and is utilized for the following
> documents:
> >> -> the software unit test plan;
> >> -> the software integration test plan.
> >> It provides a unique template for unit and integration testing, to be
> instantiated for the software test plans specified in the document
> requirement list, either for a software unit test plan, or for a software
> integration test plan.
> >>
> ---------------------------------------------------------------------------------------
> >>
> >> This documentation will re-direct to information already existent in
> the RTEMS current documentation (ex: the case of the RTEMS API User
> Manual). Repeating what was said above, the goal is to create ESA needed
> documentation to facilitate the RTEMS pre-qualification for space missions
> (according with ESA standard). However, the idea is not to close the tool
> for ESA space applications. We have performed a study, comparing the ECSS
> standard with other standards (DO, IEC and ISO) which aimed to assess the
> needed effort to adapt/extend the tool for other standards. The conclusion
> was that the tool could be indeed adapted for other standards, so this
> opens potential for the tool to be extended and used for other space
> agencies (ex: NASA) and also for other possible RTEMS markets (ex:
> automotive standards).
> >>
> >> Note that this work includes also a Test Executor, which will allow to
> execute the tests in the space platforms. Although some development was
> already done, we are currently studying how we could integrate our tool
> with RTEMS tester or even if we need our Test Executor at all, since the
> RTEMS tester may be doing already all the intended functionality.
> >>
> >> > Our first task will be to develop and integrate the SRS Manager into
> RTEMS
> >> > project.
> >>
> >> What does SRS mean? What is it managing?
> >>
> >> JV: Software Requirement Specification (as explained above). The SRS
> Manager software will allow to collect the requirements written in doorstop
> format and create the SRS document according with ESA standard. It may
> manage the following information:
> >> -> Creation of a Requirement document according with a certain template
> >> -> Verification of requirements traceability against higher-level
> requirements
> >> -> Certain checks in the requirements (ex: unique ID, if it has a
> verification method defined, if it is deprecated, etc)
> >>
>
> The scope of SRS Manager subsumes several distinct activities within
> pre-qual effort. It would be beneficial to define those "generically"
> described terms for the activities that go into the SRS, and provide
> these in a modular way for RTEMS Project adoption and maintenance.
> This would make it more likely those modules could be used with other
> qualification. For example, we could have a Requirements Document
> Generator, Requirements Traceability Generator, Requirement Coverage
> Generator, Requirement Error Checker, etc.
>
> JV02012019: Having modular design is our idea in theory. In practice I
> think that we will iterate until we have something that it is acceptable by
> the community. We will be working on this integration effort during the
> project.
>
> >> Could you please explain in more detail how you are planing to do the
> >> development? As an open source project we prefer to see things that end
> up in
> >> RTEMS being developed publicly and on our devel@ list. It makes the
> process of
> >> merging smoother.
> >>
> >> JV: The tool development will be made in a module-by-module basis (that
> is first we start with SRS Manager, then TestManager and so on), this
> development will be approximately 1 month development cycle per manager,
> during the next year. We will have the RTEMS SMP qualification (done by
> Sebastian Huber) as our first user. My suggestion is that each manager
> could be integrated in the RTEMS project as long as it is finished and
> Sebastian gives us positive feedback of his tool user-experience..
> >
> >
> > Existence is not a sufficient condition for any tool to be accepted by
> the RTEMS Project.
> >
> > There are requirements on licensing, cross-platform support,
> dependencies to manage,
> > ease of integration into automated systems, etc. If these tools are to
> be supported, they
> > need to be made available for review from the initial decision points.
> By the time you have
> > code, you may already be too committed down a path to produce something
> the RTEMS
> > Project would find acceptable.
> >
>
> JV02012019: The idea is to develop the tool as much as possible according
> with the RTEMS community rules from beginning (for that I will have the
> help of Sebastian Huber). If still, there are divergence points, we will
> iterate the tool, until it gets accepted by the community.
>
>
> >>
> >>
> >> > This SRS Manager reads requirements written in doorstop format (each
> >> > topic is a doorstop document) and produces the SRS document according
> with ECSS
> >> > format (the output of the tool are sphinx files, which then are
> compiled to pdf
> >> > document).
> >>
> >> I do not understand what is meant by "doorstop document"? A work flow
> may help
> >> where we see the types of data used. I ask this because a document
> generator
> >> needs to have the requirements data being held somewhere and I do not
> know where
> >> that is.
> >>
> >>
> >> JV: You may think as "doorstop document" a set of doorstop yaml files.
> Each set is a requirements' topic (ex: requirements for task manager,
> requirements for semaphore manager, etc). This is explained in our
> documentation, which probably we should provide to you.
> >
> >
> > This is a requirement set then which is implemented as a collection of
> Doorstop YAML files.
> >
> > Can this set be changed? Can multiple sets be managed? For example, the
> ESA SMP Profile
> > and another POSIX focused profile.
> >
>
> It would be nice if a simple example would be provided that is
> parallel to the ECSS. This would also help to understand how the tool
> may be adopted for other standards.
>
> JV02012019, Joel and Gedare: Several sets of requirements can be managed
> (based on flags). When we start the RTEMS specification for our project
> (our use-case) it will be more clear to you.
>
> >>
> >>
> >> Does the tool generate ReST format output?
> >>
> >> JV: Yes
> >>
> >> Is the ECSS format a template given to sphinx? We would need to be able
> to
> >> support other formats.
> >>
> >> JV: It is our tool which creates the ECSS format. I agree (other
> formats should be supported), and it is in line with I explained above. The
> scope of this project is to just create automatically the documentation for
> the ECSS format. But as a future continuation of this work other
> formats/standars could be included.
> >
> >
> > If I read this correctly, without this tool (which has issues mentioned
> earlier),
> > there are no requirements and traceability documents produced. The RTEMS
> > Project would just have a set of Doorstop files without much value.
>
> JV02012019:  The information will be still there (requirements,
> traceabilities) but in raw format, which may still used if needed.
>
> >
> >>
> >>
> >> We typically require HTML _and_ PDF.
> >>
> >> JV: We are generating PDF. HTML is not considered to be in the scope of
> this project. Although I would say that it would not be hard to generate
> html from what we have now, since sphinx has built-in capability to
> generate html.
> >
> >
> > Ironically, I know of a DO-178 Level A product that delivers
> requirements in browsable,
> > hyperlinked HTML. It is really a beauty in action.
>
> JV02012019: Yes, probably there are already products which do all or parts
> of what we intend to do in this project. I would say that most of them are
> closed source. I think the benefit of this project for RTEMS is to have the
> possibility to pre-qualify several customized RTEMS profiles more quickly
> (encompassing more users), instead of having a fixed pre-qualified profile
> which only fits a limited set of market.
>
> Since it outputs to ReST and Sphinx, I don't think it is a big deal
> whether or not it supports HTML immediately. If other qualification
> users want HTML (or any other format) they should be able to add it
> with incremental cost. That is the important thing.
>
> JV02012019: Yes, I agree.
>
> >>
> >>
> >> > Could you indicate how I can integrate our tool into RTEMS git
> repository?
> >>
> >> Commit access to the repositories is given to people who work with the
> project
> >> and are active for a period of time. You will need to post changes and
> get the
> >> support of a core development before the code can be merged.
> >
> >
> > Commit privileges are earned as Chris points out.
> >
> > You will have to submit your work in manageable pieces under an
> acceptable license.
> > You will have to "socialize" it before dumping any code on the RTEMS
> Project. The
> > core developers will have to understand where the tools fit into the
> project, development
> > process, and that they have a chance to be technically acceptably.
>
> JV02012019: That is out idea to do through the project :) This
> conversation is the first stone to that goal.
>
> >
> >>
> >>
> >> > Should be in rtems-tools? If so, in which place?
> >>
> >> I think this is the best place given the information I have. I would
> need to
> >> have more detail before I could provide any specific direction here.
> >
> >
> > All host tools are in rtems-tools so at this point, there isn't another
> place.
> >
> > But I agree with Chris. We need more information.
> >
>
> We need to know:
> * Language(s) used for the tool
> * Host requirements to run the tool
> * Licensing of the tool
>
> Putting the pre-qual tools into rtems-tools seems reasonable, but a
> discussion should happen within the RTEMS Project
> community/maintainers to do that.
>
> JV02012019, Joel and Gedare:
>
> We need to know:
> * Language(s) used for the tool - python (>= 3.7)
> * Host requirements to run the tool - Debian 10
> * Licensing of the tool - BSD-2-Clause
>
> Please read the section 4 of the document SDD-301.pdf, I am sending in
> attach. This provides an overview of the tool (I think we missed to give
> you this overview).
> For more information you may also take a look in the SRS-300.pdf and
> TI-003 (just the conclusions on each section).
> Note that the information presented in these documents is subjected to be
> changed, but as I said, the idea is for you to have a better understanding
> about the big picture of the project. The details will be discussed (and
> iterated) alongside the project execution.
>
> >>
> >> JV: I hope I was able to clarify your points. I predict the first
> integration/commit will be for the SRS Manager, after Sebastian positive
> user-experience.
> >>
> >> I would like to clearly state I fully support the pre-qual effort and I
> thank
> >> ESA for making this happen. It is exciting for RTEMS to have this
> activity.
> >
> >
> > I couldn't agree with this sentiment more. I am thrilled this is
> happening.
> > We just have to be careful to not "break the project". We have open
> source
> > principles to uphold along with the requirement not to make working with
> or
> > on RTEMS burdensome. We still want a college student to be able to make
> > meaningful contributions without being buried or terrified with flight
> safety.
> >
> +1
>
> JV02012019: Yes, I also agree. As pointed out, we do not intend to break
> the project, but instead to be the more aligned possible with the community
> philosophy.
>
> >>
> >>
> >> JV: Thank you for your words. We hope this tool serves the RTEMS
> community as much as possible.
> >>
> >> Chris
> >> _______________________________________________
> >> devel mailing list
> >> devel at rtems.org
> >> http://lists.rtems.org/mailman/listinfo/devel
> >
> > _______________________________________________
> > devel mailing list
> > devel at rtems.org
> > http://lists.rtems.org/mailman/listinfo/devel
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.rtems.org/pipermail/devel/attachments/20200103/1f4da32b/attachment-0001.html>