GCOV

[Joel Sherrill]

Chris Johns wrote:
> Aitor Viana wrote:
>   
>> Hi everybody,
>>
>> Does anybody knows how RTEMS is handling (if it does) test coverage
>> analysis in the embedded platforms? either for applications or RTEMS itself?
>>
>> I am making that question because I started not to far ago to write
>> something allowing embedded SW coverage analysis by means of gcov. I
>> already get something running for ERC32 and LEON2 allowing the coverage
>> analysis for one source file...and I am about continue (in the spare
>> time) the development to address the full implementation.
>>
>> When you want to perform gcov analysis, you compile the source files
>> with the "-fprofile-arcs" and "-ftest-coverage" flags. GCC then will
>> instrument the code adding the line counting and hooks to the "libgcov"
>> library, which is the one retrieving the coverage information after
>> execution.
>> The calls being hooked by gcc are empty, so only doing this, the SW
>> would actually would do nothing when executing (we would not retrieve
>> the coverage information).
>> This needs to be done by the libgcov library, which needs to be linked
>> with the already hooked software.
>>
>> This library is the one I am  implementing for embedded systems, as the
>> normal one just dumps the coverage information in files (not good for
>> embedded systems)
>>
>> Basically what I am planning is to send the .gcda (is the file which
>> will hold the coverage analysis data) through for instance a serial
>> line, though I think will be to slow interface. That's why I am also
>> planning to support any other high speed comm. links, like Ethernet,
>> spacewire, etc. An also let the gcov library software being as scalable
>> and portable as possible.
>>
>> The idea is to have a client in the embedded target sending the
>> information to the host where a server is being executed and is putting
>> together everything again in files. (.gcda files). Those files will be
>> then analyzed by gcov after execution.
>>
>> I think, this could be a tool that might be integrated within RTEMS
>> distribution for application testing purposes. Even more, it may also be
>> tuned to support RTEMS overage analysis....
>>
>> I am gonna do it anyway...so if RTEMS want benefit from the tool...just
>> let me know.
>>
>>     
>
> Yes I think this would be useful. Have you seen the GSOC project relating to
> the real time tracing ? Please check the RTEMS Wiki. The Capture Engine will
> contain the buffering and transport code so it would be nice to have your code
> use this software. We want to a way for user to add their own transport if the
> standard ones in RTEMS are not suitable.
>
> I would consider the GCOV data as a single trace point of a specific class.
> The Capture Engine can provide a call for you to place the data into the
> buffer and to transport it to host machine. Here we can extract that data for
> you to handle with the existing GCOV tools. It will also handle the overflow
> issues, an issue with embedded systems and slow transports.
>
> Please note this project depends on GSOC acceptance so here hoping we get it.
>   

The above is a good approach for cross-target gcov support.

My coverage analysis efforts have been on tsim and using
its coverage capabilities.  With a library/linker trick combined
with a couple of special helper utilities, I can get a coverage
analysis report at the assembly statement for rtems which
is a union of all tests.

I have encouraged the Skyeye developers to look at providing
coverage information reports similar to the information from
tsim.  That would open up a few more CPU architectures for
coverage analysis.

Also again in the shameless plug department, test coverage
improvement is an ongoing effort which I am very happy to
get any consulting hours to work on.  OAR did the initial work
using some internal R&D funds.  We got the infrastructure built
and established a process.  The test coverage did improve a LOT
due to our work but more importantly, we also learned that
there is no silver bullet to getting 100% binary coverage.  You
need lots of tests but that alone isn't the entire answer.

+ layered software sometimes has paths in lower levels that
are unreachable because they are errors caught in upper layers.
+ lower layers may have functionality unused in a particular
configuration.  Think SuperCore functionality only used by POSIX.
+ switches sometimes result in the compiler generating code
which is unreached even when all paths in the switch are reached.
+ -Os is easier to analyze than -O2.
+ some necessary interrupt logic is difficult to hit
+ inlining code with a number of if's in it can lead to unreachable code.
Not every inlined location may hit all the paths.

In short, blindly adding tests is wrong.  Each uncovered range has to
carefully analyzed.

But it is critically important to ALL RTEMS users that coverage
improvements come back to the main distribution.  More testing
and more coverage is more better. :-D

--joel
> Regards
> Chris
> _______________________________________________
> rtems-users mailing list
> rtems-users at rtems.com
> http://rtems.rtems.org/mailman/listinfo/rtems-users
>   


-- 
Joel Sherrill, Ph.D.             Director of Research & Development
joel.sherrill at OARcorp.com        On-Line Applications Research
Ask me about RTEMS: a free RTOS  Huntsville AL 35805
   Support Available             (256) 722-9985