Requirement Document generator tool

Tue Jan 21 09:10:11 UTC 2020

On 20/01/2020 22:37, Chris Johns wrote:
> On 14/1/20 7:18 pm, Christian Mauderer wrote:
>> On 13/01/2020 18:03, Joel Sherrill wrote:
>>>
>>>
>>> On Mon, Jan 13, 2020 at 10:04 AM Christian Mauderer
>>> <christian.mauderer at embedded-brains.de
>>> <mailto:christian.mauderer at embedded-brains.de>> wrote:
>>>
>>>     Hello Jose,
>>>
>>>     On 10/01/2020 17:33, Jose Valdez wrote:
>>>     > Hello,
>>>     >
>>>     > Regarding this topic and to start to define the python tools that
>>>     are more appropriate for the RTEMS community python developments, I
>>>     would like to propose the following tools (to be placed in RTEMS
>>>     Software Engineering manual):
>>>     >
>>>     > Python language version: 3.6 (minimum version for Doorstop 2.0.post2)
>>>     > Python coding style/standard - Google (as suggested by Gedare)
>>>     > Python documentation style - Google docstrings (to be in
>>>     accordance with coding style. Note it is supported by sphinx)
>>>     > Python test approach - unittest (seems to be the standard used by
>>>     python)
>>>     > Python static analysis - pylint (recommended by Google style, see
>>>     http://google.github.io/styleguide/pyguide.html) and mypy (catch
>>>     additional errors) Python code coverage - Coverage.py (seems to be
>>>     the standard used by python)
>>>     > Python third party packages - For now we are using: paramiko,
>>>     pyserial, pexpect, gitpython, but the list is expected to be
>>>     changed. I think here any third package is allowed, as long as the
>>>     license of the package complies with RTEMS project license.
>>>
>>>     To be future-proof: python3 compatibility would be nearly a must-have
>>>     for any new library.
>>>
>>>
>>> We discussed this somewhere earlier and I thought we decided that if
>>> it was a core tool required for a "normal user", then it would have to
>>> be Python 2.x and 3.x compatible. Tools for "requirements user" or
>>> "pre-qualification user" (pick a name for this role), can be 3.x only 
>>> mainly because there was so much infrastructure required for the
>>> "requirements user" that already needed Python 3.x.
> 
> Yes this is correct. Python is used in the rtems-tools repo and any program in
> the rtems-tools repo needs to support Python 2 and Python 3 no matter what role
> it performs. The rtems-tools repo is not the same as doorstop or similar tools
> used to develop RTEMS, they are user tools used to develop application to run on
> RTEMS.

Does that mean you don't want to have any pre-qualification tools in the
rtems-tools repo? If yes: Where else?

I fully agree that every tool that is thought for every day use for 99%
of the RTEMS users should have Python 2 and 3 support. With no official
Python 2 support since this year I don't think that this would be a good
requirement for new specialized areas like pre-qualification. I think
you agree there?

On the other hand I wouldn't know a better place then rtems-tools for
everything produced through the pre-qualification effort. So I'm not a
fan of saying: Everything in rtems-tools should be Python 2 + 3.

> 
> It is not clear to me what role the original python specification Jose listed is
> for. I will restrict my comments to rtems-tools or more generally any tool
> installed into the user's tools prefix for use when developing RTEMS applications.
> 
> For a user focused tool:
> 
> - Python is 2 and 3
> - No external third party packages, all support needs to be internal.

What about stuff like pygdb (which is already used in rtems-tools)?

> - Fully portable for all supported host platforms, that is Windows, MacOS,
> FreeBSD and Linux.
> 
> The second item is important. If user installed tools start to depend on 3rd
> party packages the deployment complexity for us rises and the process becomes
> complex.
> 
> While on the topic of 3rd party packages I do not agree with Jose's view any
> package can be used if the license it suitable. Package complexity, it's
> dependencies and portability need to be considered, for example gitpython uses
> gitdb and this is python code to directly read and write a git repo. I would not
> like any of our repo's being modified by that code so I would prefer we do not
> use it. If we need to extract data I prefer the solution in the rtems-tools
> toolkit so we should also consider using existing support first.

OK. So rule for third party is:

For everything that is used by 99% of our users: Basically don't use any
third party. Exceptions for that user group have to be discussed (and
are most likely rejected) on the mailing list.

For special use cases:
- Should be avoided if a build-in can be used instead.
- Only if there isn't another package for the same purpose already in use.
- Open source license that is not putting restrictions on our code (like
a code generator that would put the generated code under GPL).
- Portable to all mayor host systems (for RTEMS that's Windows, Linux,
FreeBSD, MacOS).
- Libraries that touch critical infrastructure would have to discussed
thoroughly with the community before it can be added.

About right?

Best regards

Christian

> 
>>
>> Yes, sorry. I used the wrong wording. What I meant was that we should
>> avoid libraries that support only python 2. But you have added a lot of
>> relevant details to this.
>>
>>>
>>> I think this discussion should also include "documentation producer"
>>> because that requires Python 3.x as well.
>>>
>>> We just need to be conscious of the user roles that the tools are
>>> supporting and justify the minimum. This gives us at least 3 categories
>>> of tools with potentially different minimum hosting requirements. For
>>> example, we may not care if a certain set of tools runs on Mingw.
>>>
>>> Obviously documented somewhere.
>>>  
>>>
>>>
>>>     I don't think that there are heavy license problems. The python packages
>>>     are most likely only used in tools. So as long as they are open enough
>>>     to use them on all host platforms for all development situations that
>>>     shouldn't be a problem. Exception: If the package is used to generate
>>>     code with a specific license. That generated code would have to be
>>>     compatible with the RTEMS license.
>>>
>>>
>>> +1 and very much non-viral and no advertising for generated code.
>>>
>>>
>>>     >
>>>     > Please tell me your opinions for each python tool and then we can
>>>     start to agree on what to use. Note that once this is defined, our
>>>     Qualification software will comply with the RTEMS community chosen
>>>     tools.
>>>
>>>     If no one objects I would suggest to start documenting them in the RTEMS
>>>     Software Engineering manual. A small example project in the rtems-tools
>>>     repository that uses the suggested code checkers (meaning: calls them
>>>     during a waf build or a special 'waf check' target) would be great too.
>>>     That could be used for all future python based tools that are added to
>>>     rtems-tools (which includes the whole qualification tools).
>>>
>>>
>>> Yes. It must be enforced. Whether it is a special step or not is up for
>>> discussion.
>>> My first thought is that a special step is easy to skip.
>>>
>>> Also can "check" be done as a pre-commit hook? 
>>
>> Possible. But I'm not sure. We might can try it as soon as we have it
>> running in an example.
> 
> The problem I see with git hooks is the catch is too late. A patch is tested
> then reviewed only for a push to fail and the review process needs to start
> again or be restarted.
> 
>>> And when the style is agreed to, should existing code be auto reformatted
>>> to meet it?
>>
>> I think this could be migrated on a per-tool basis. If we have one
>> project that is a sample for all future tools the checkers can be back
>> ported to existing tools when there is time or a good opportunity.
> 
> You need to convince me a Python reformatting tool does not break existing
> python code. A misaligned indent changes the logic.
> 
> Chris
> 

-- 
--------------------------------------------
embedded brains GmbH
Herr Christian Mauderer
Dornierstr. 4
D-82178 Puchheim
Germany
email: christian.mauderer at embedded-brains.de
Phone: +49-89-18 94 741 - 18
Fax:   +49-89-18 94 741 - 08
PGP: Public key available on request.

Diese Nachricht ist keine geschäftliche Mitteilung im Sinne des EHUG.