Improve Coverity Scan Integration: GSOC project details

[Gedare Bloom]

On Sun, Mar 1, 2020 at 6:31 PM Joel Sherrill <joel at rtems.org> wrote:
>
>
>
> On Sun, Mar 1, 2020, 1:02 PM suyash singh <suyashsingh234 at gmail.com> wrote:
>>
>> This is regarding the project I would like to work in
>>
>> Improve Coverity Scan Integration
>> https://devel.rtems.org/ticket/3710
>>
>> FIRST POINT-
>> I was thinking about implementing Coverity or clang analyzer as an offline analyzer. Would it be a suitable idea to make a separate repo just for testing purposes?
>> This "testing repository" will have the updated rtems code as well the analyzer in it. If possible this repo will be updated automatically from the main repo.
>
>
> I would have to see how the clang analyser is run against the RTEMS source. My expectation would be that ultimately something would be added to rtems-tools to run the analysis process and generate reports.
>
> The reports could then be locally generated or run on an rtems.org server for publication.
>
> I don't see needing a private RTEMS repository. This should be an rtems-tools project and you would only have a personal copy of that for the purposes of getting reviews. Usually patches just get submitted and you don't need a public facing repo at all.
>
>

I have someone attempting to replicate the recent instructions how to
use clang-analyzer. I will share results if we succeed.

It is possible to run open-source analyzers (like clang, cppcheck) on
a repo. For a GSoC, it would be fine to work on an RTEMS/RTEMS-Tools
fork, with the intent to deliver upstream.

Unfortunately, you can't easily run Coverity offline or on your own
repository. To use the free version you need to run it through their
interface. At least, I don't know if you can download/run
coverity-scan free version. That is something you could take a look
at. Longer term, I don't think Coverity is the right tool for the
community because of the restrictions on the open version. However, we
should continue to improve how we use it. I'm becoming more convinced
that we need to identify suitable open-source static analysis tools
that help us to improve our products.

>>
>> SECOND POINT-
>> Using Travis CI for github. I have seen Travis CI checking the build of pull requests in other repos.
>> Also making a special coverity branch to trigger coverity scan
>
>
> Amar should comment on this. RTEMS.org is using buildbot.

There is special support to integrate Coverity with Travis on GitHub
with the primary advantage being the ability to trigger scans on a
special branch, as you note. However, from a resources perspective, I
don't think we want to invest time in Travis on GitHub. We provide
GitHub mirrors as a convenience only.

Instead, I think we would like to have better automation to run our
own builds on our infrastructure periodically (e.g., submit a weekly
buildbot upload to coverity).

>>
>>
>> THIRD POINT-
>> Enabling and testing MISRA rules.
>
>
> Do you know of a free tool that checks Misra C rules? There are closed source products that do but I don't know of a FLOSS one.
>
> Assuming you have identified a free software tool to do the analysis, the impact and applicability of each rule to the RTEMS source base needs evaluation. I know some of the rules we agree with but others we will not want to adopt. For example, Misra defines a subset of C Library header files you can include. RTEMS will violate that regularly and intentionally.
>
>
>>
>> FOURTH POINT-
>> Reducing false positives.
>> I have yet to figure out how to do this one.
>> I noticed a pattern where value overwrite errors are given for unused values. There is email thread about it with subject "Coverity false positive pattern"
>
>
> If the tool flags a pattern in code incorrectly, then it needs to be reported to the tool owner.
>
> If there is an intentional violation, then the code may have to be annotated with an explanation. Also documenting the allowed types of exceptions is a critical part of the software engineering side of this.
>>
>>
>> FIFTH POINT-
>> Solving reported issues by coverity.
>
>
> This is always good and as recent traffic on the mailing list has shown is often more difficult than at first glance.
>
> It is good work and appreciated. Just can't be measured by number of lines of code since you are usually fixing with a scalpel something subtle
>>
>>
>>
>> _______________________________________________
>> devel mailing list
>> devel at rtems.org
>> http://lists.rtems.org/mailman/listinfo/devel
>
> _______________________________________________
> devel mailing list
> devel at rtems.org
> http://lists.rtems.org/mailman/listinfo/devel