Addition of Rule Checkers

[Sebastian Huber]

On 30/07/2019 01:01, Joel Sherrill wrote:
> 
> 
> On Mon, Jul 29, 2019 at 5:23 PM Gedare Bloom <gedare at rtems.org 
> <mailto:gedare at rtems.org>> wrote:
> 
>     On Wed, Jul 24, 2019 at 7:16 AM Joel Sherrill <joel at rtems.org
>     <mailto:joel at rtems.org>> wrote:
>      >
>      > Hi
>      >
>      > I just wanted to make sure we followed proper procedures and
>     policies when considering rule checkers.
>      >
>     I guess these comments are germane to the other thread, but I want to
>     reflect here a bit.
> 
>      > 1. The license must appropriate.
>      >
>     Can you expand what you mean by this?
> 
> 
> It must be a FLOSS tool -- it can't be proprietary. We use Scan but it 
> has issues.
> 
> It also must be available on the normal set of commonly used hosts.
> 
> 
>      > 2. There should be some basic requirements that the tool is
>     expected to meet to be fit for purpose
>      >
>      > 3. The tool must support the range of development hosts used in
>     the Community.
>      >
>     Only if we make it a requirement for developers to check these rules
>     locally before submitting patches/commits. If we had a server for
>     example that could accept submissions from developers, this would be
>     irrelevant?
> 
> 
> If we relax this rule, then we will end up with a one host tool and have 
> to have
> a server that runs it. Our infrastructure is FreeBSD so saying it 
> supports the range
> of hosts is likely saying it supports our servers, Linux, MacOS, Cygwin, 
> and MingW

I would not make it a requirement that every user must be able to run 
the tools on a wide range of platforms. We should clearly identify the 
tools (e.g. version) and the tool configuration (rules, scope). It 
should enable someone else to set up the tool and run it in a 
configuration used by the RTEMS project.

What is the overall goal of this checker stuff? We want that they find 
bugs for us which we can fix. For the checker tools we have at least to 
categories: 1. static analysis and 2. coding rule checkers.

1. For the static analysis tools I think it doesn't matter how they are 
run or by whom. If they find a bug, we fix it.

2. This is a bit different to the old style coding rule checkers. There 
aim is to write the code with rules so that bugs are prevented in the 
first place (e.g. MISRA). I think these tools must be freely available. 
Rule violations will result in patches which don't fix bugs.

> 
> 
>      > As a detail, once selected the tool needs to have an RSB recipe.
>      >
>     Only if you require that checkers must be built-from-source.
> 
> 
> If we can't get source, then they aren't FLOSS and we are depending on
> them coming with distributions and being provided for every host we care
> about.
> 
> We need something like Phabricator so things can be run against the code
> as part of the review/submission process but I want every tool to be 
> available
> to every user.

If you set up a project infrastructure which automatically checks 
patches, then not everyone needs to install the tools used for the 
automatic checks. My experience is that in projects with good automatic 
checks in the Git pull pipeline the developers learn to rely on this and 
stop to check their patches locally. They just wait for the reports from 
the automatic checkers and if a red flag appears, they get active.

> 
> 
>      > For style checkers, I am concerned that they are not integrated
>     into the normal development and build process. There is already
>     enough "garbage collection" to do and making a special pass. If
>     that's the case, it will just be a burden.
>      >
>      > Just some thoughts. I'm not opposed to this, I just don't want
>     something like Coverity where it is only run periodically and only a
>     few people even look at it much less fix problems.
>      >
>     I think that the problem of only running periodically and having few
>     eyes/fixes are a bit orthogonal to things you pointed out earlier. I
>     think the inclusion of rule checkers will be a good thing, but I also
>     agree that how it gets done is important for long-term viability.
> 
> 
> The less available and less open the tool is, the less likely it is that 
> anyone
> will look at it. I guess I think the two issues are tied together.

Yes, the first burden is a need to log in to see something. The next 
burden is if you cannot copy and paste things from the report to discuss 
things with other developers. Inaccessible documentation and no good 
support channel is another burden.

> 
> If we can add it to the RSB, then it can be built locally and integrated 
> by us into
> the normal build process. Otherwise, we just become a project which is 
> hard to
> submit code to because we are picky and have our own tools at the end.
> 
> What does Chromium do with their checker? That would be a good reference to
> see where on the spectrum a big project falls.

Yes, I think we could learn a lot from the Chromium project.

-- 
Sebastian Huber, embedded brains GmbH

Address : Dornierstr. 4, D-82178 Puchheim, Germany
Phone   : +49 89 189 47 41-16
Fax     : +49 89 189 47 41-09
E-Mail  : sebastian.huber at embedded-brains.de
PGP     : Public key available on request.

Diese Nachricht ist keine geschäftliche Mitteilung im Sinne des EHUG.