Requirement Document generator tool

Tue Jan 21 16:31:15 UTC 2020

On Tue, Jan 21, 2020 at 2:10 AM Christian Mauderer
<christian.mauderer at embedded-brains.de> wrote:
>
> On 20/01/2020 22:37, Chris Johns wrote:
> > On 14/1/20 7:18 pm, Christian Mauderer wrote:
> >> On 13/01/2020 18:03, Joel Sherrill wrote:
> >>>
> >>>
> >>> On Mon, Jan 13, 2020 at 10:04 AM Christian Mauderer
> >>> <christian.mauderer at embedded-brains.de
> >>> <mailto:christian.mauderer at embedded-brains.de>> wrote:
> >>>
> >>>     Hello Jose,
> >>>
> >>>     On 10/01/2020 17:33, Jose Valdez wrote:
> >>>     > Hello,
> >>>     >
> >>>     > Regarding this topic and to start to define the python tools that
> >>>     are more appropriate for the RTEMS community python developments, I
> >>>     would like to propose the following tools (to be placed in RTEMS
> >>>     Software Engineering manual):
> >>>     >
> >>>     > Python language version: 3.6 (minimum version for Doorstop 2.0.post2)
> >>>     > Python coding style/standard - Google (as suggested by Gedare)
> >>>     > Python documentation style - Google docstrings (to be in
> >>>     accordance with coding style. Note it is supported by sphinx)
> >>>     > Python test approach - unittest (seems to be the standard used by
> >>>     python)
> >>>     > Python static analysis - pylint (recommended by Google style, see
> >>>     http://google.github.io/styleguide/pyguide.html) and mypy (catch
> >>>     additional errors) Python code coverage - Coverage.py (seems to be
> >>>     the standard used by python)
> >>>     > Python third party packages - For now we are using: paramiko,
> >>>     pyserial, pexpect, gitpython, but the list is expected to be
> >>>     changed. I think here any third package is allowed, as long as the
> >>>     license of the package complies with RTEMS project license.
> >>>
> >>>     To be future-proof: python3 compatibility would be nearly a must-have
> >>>     for any new library.
> >>>
> >>>
> >>> We discussed this somewhere earlier and I thought we decided that if
> >>> it was a core tool required for a "normal user", then it would have to
> >>> be Python 2.x and 3.x compatible. Tools for "requirements user" or
> >>> "pre-qualification user" (pick a name for this role), can be 3.x only
> >>> mainly because there was so much infrastructure required for the
> >>> "requirements user" that already needed Python 3.x.
> >
> > Yes this is correct. Python is used in the rtems-tools repo and any program in
> > the rtems-tools repo needs to support Python 2 and Python 3 no matter what role
> > it performs. The rtems-tools repo is not the same as doorstop or similar tools
> > used to develop RTEMS, they are user tools used to develop application to run on
> > RTEMS.
>
> Does that mean you don't want to have any pre-qualification tools in the
> rtems-tools repo? If yes: Where else?
>

Maybe we need a new repo, e.g., 'rtems-devtools.git' to hold
developer-only tooling.

> I fully agree that every tool that is thought for every day use for 99%
> of the RTEMS users should have Python 2 and 3 support. With no official
> Python 2 support since this year I don't think that this would be a good
> requirement for new specialized areas like pre-qualification. I think
> you agree there?
>
> On the other hand I wouldn't know a better place then rtems-tools for
> everything produced through the pre-qualification effort. So I'm not a
> fan of saying: Everything in rtems-tools should be Python 2 + 3.
>
> >
> > It is not clear to me what role the original python specification Jose listed is
> > for. I will restrict my comments to rtems-tools or more generally any tool
> > installed into the user's tools prefix for use when developing RTEMS applications.
> >
> > For a user focused tool:
> >
> > - Python is 2 and 3
> > - No external third party packages, all support needs to be internal.
>
> What about stuff like pygdb (which is already used in rtems-tools)?
>
> > - Fully portable for all supported host platforms, that is Windows, MacOS,
> > FreeBSD and Linux.
> >
> > The second item is important. If user installed tools start to depend on 3rd
> > party packages the deployment complexity for us rises and the process becomes
> > complex.
> >
> > While on the topic of 3rd party packages I do not agree with Jose's view any
> > package can be used if the license it suitable. Package complexity, it's
> > dependencies and portability need to be considered, for example gitpython uses
> > gitdb and this is python code to directly read and write a git repo. I would not
> > like any of our repo's being modified by that code so I would prefer we do not
> > use it. If we need to extract data I prefer the solution in the rtems-tools
> > toolkit so we should also consider using existing support first.
>
> OK. So rule for third party is:
>
> For everything that is used by 99% of our users: Basically don't use any
> third party. Exceptions for that user group have to be discussed (and
> are most likely rejected) on the mailing list.
>
> For special use cases:
> - Should be avoided if a build-in can be used instead.
> - Only if there isn't another package for the same purpose already in use.
> - Open source license that is not putting restrictions on our code (like
> a code generator that would put the generated code under GPL).
> - Portable to all mayor host systems (for RTEMS that's Windows, Linux,
> FreeBSD, MacOS).
> - Libraries that touch critical infrastructure would have to discussed
> thoroughly with the community before it can be added.
>
> About right?
>
> Best regards
>
> Christian
>
> >
> >>
> >> Yes, sorry. I used the wrong wording. What I meant was that we should
> >> avoid libraries that support only python 2. But you have added a lot of
> >> relevant details to this.
> >>
> >>>
> >>> I think this discussion should also include "documentation producer"
> >>> because that requires Python 3.x as well.
> >>>
> >>> We just need to be conscious of the user roles that the tools are
> >>> supporting and justify the minimum. This gives us at least 3 categories
> >>> of tools with potentially different minimum hosting requirements. For
> >>> example, we may not care if a certain set of tools runs on Mingw.
> >>>
> >>> Obviously documented somewhere.
> >>>
> >>>
> >>>
> >>>     I don't think that there are heavy license problems. The python packages
> >>>     are most likely only used in tools. So as long as they are open enough
> >>>     to use them on all host platforms for all development situations that
> >>>     shouldn't be a problem. Exception: If the package is used to generate
> >>>     code with a specific license. That generated code would have to be
> >>>     compatible with the RTEMS license.
> >>>
> >>>
> >>> +1 and very much non-viral and no advertising for generated code.
> >>>
> >>>
> >>>     >
> >>>     > Please tell me your opinions for each python tool and then we can
> >>>     start to agree on what to use. Note that once this is defined, our
> >>>     Qualification software will comply with the RTEMS community chosen
> >>>     tools.
> >>>
> >>>     If no one objects I would suggest to start documenting them in the RTEMS
> >>>     Software Engineering manual. A small example project in the rtems-tools
> >>>     repository that uses the suggested code checkers (meaning: calls them
> >>>     during a waf build or a special 'waf check' target) would be great too.
> >>>     That could be used for all future python based tools that are added to
> >>>     rtems-tools (which includes the whole qualification tools).
> >>>
> >>>
> >>> Yes. It must be enforced. Whether it is a special step or not is up for
> >>> discussion.
> >>> My first thought is that a special step is easy to skip.
> >>>
> >>> Also can "check" be done as a pre-commit hook?
> >>
> >> Possible. But I'm not sure. We might can try it as soon as we have it
> >> running in an example.
> >
> > The problem I see with git hooks is the catch is too late. A patch is tested
> > then reviewed only for a push to fail and the review process needs to start
> > again or be restarted.
> >
> >>> And when the style is agreed to, should existing code be auto reformatted
> >>> to meet it?
> >>
> >> I think this could be migrated on a per-tool basis. If we have one
> >> project that is a sample for all future tools the checkers can be back
> >> ported to existing tools when there is time or a good opportunity.
> >
> > You need to convince me a Python reformatting tool does not break existing
> > python code. A misaligned indent changes the logic.
> >
> > Chris
> >
>
> --
> --------------------------------------------
> embedded brains GmbH
> Herr Christian Mauderer
> Dornierstr. 4
> D-82178 Puchheim
> Germany
> email: christian.mauderer at embedded-brains.de
> Phone: +49-89-18 94 741 - 18
> Fax:   +49-89-18 94 741 - 08
> PGP: Public key available on request.
>
> Diese Nachricht ist keine geschäftliche Mitteilung im Sinne des EHUG.