rtems-5.1 source code test coverage failed

[smallphd at aliyun.com]

Thank you once again. Your reply is very very fast.



smallphd at aliyun.com
 
From: Joel Sherrill
Date: 2020-10-23 10:03
To: smallphd
CC: devel
Subject: Re: Re: rtems-5.1 source code test coverage failed
Hit send too soon

On Thu, Oct 22, 2020, 8:59 PM Joel Sherrill <joel at rtems.org> wrote:


On Thu, Oct 22, 2020, 8:53 PM smallphd at aliyun.com <smallphd at aliyun.com> wrote:
Thanks joel.

I google the Couverture-Qemu and find many useful document about rtems test coverage.
The conventional approach will compile source codes to create *.gcno files. But the Couverture-Qemu method needn't. 
In the final report, the Couverture-Qemu method will create an annotated.html file in which the instructions are listed as "NOT EXECUTED" or executed.
The main different is only assembly language is used, while the conventional approach will list the souce code in C language.

The reports from covoar include the source code. The reports are from a developer perspective and we haven't had any feedback from iv&v folks.  If the reports don't match what you need, there is always the source code . :)

This is an area where feedback, contributions, and supporting developers is really appreciated.

And there is an RSB recipe for couverture qemu which should make it easier to build.

I'd love to see this area of our tooling and ecosystem get love and grow.  :)

--joel




smallphd at aliyun.com
 
From: Joel Sherrill
Date: 2020-10-22 23:49
To: smallphd at aliyun.com
CC: devel
Subject: Re: rtems-5.1 source code test coverage failed


On Thu, Oct 22, 2020 at 4:02 AM smallphd at aliyun.com <smallphd at aliyun.com> wrote:
hi, all
I want test source code coverage in qemu. There are some questions I met.
1. If compiling without -ftest-coverage, the test only produce a percent coverage, no *.gcov file which is important to recognize which lines not execute.
2. If compiling with -ftest-coverage, *.gcno are created in the same directory with *.o and these *.gcno file are included in xilinx_zynq_a9_qemu-symbols.ini. However this ini file could only include *.o. So the execution will raise error ----- File format not ELF or archive. 

Is there a tutorial introducing how to get rtems-5.1 source code test coverage?

We don't do coverage that way. Some simulators can directly produce gcov data. 
I think tsim can do that for example.

gcov foundationally assumes that it can run, collect data, and write it to a file
at program exit. That model doesn't work on embedded targets. There are 
some write ups on the net of people forcing gcov data to be written sporadically
and magic to get it off target. But this isn't how RTEMS has done coverage.

We depend on simulators that write trace logs and a utility called covoar which
can correlate multiple trace logs with the program executables and product
a unified report. Remember that the same function may be in two RTEMS
executables but they will not be at the same physical address. The trace/log
execution information produced by the simulators will be in terms of which
physical addresses are executed.

The RTEMS Tester has support for running coverage and hopefully the
documentation is good at explaining that. 

The qemu couverture project (a separate repo from regular qemu), sis,
and tsim all produce trace/coverage info for an executable that covoar
should be able to process. 

FWIW there is at least one known issue with covoar right now. Sometimes
the reported size of a method differs between two executables. Historically,
this has been due to padding after a method so the next adjacent 
method in memory is cache line aligned. covoar will reject information
when a method has different sizes in different executables. But can
adjust if it is nop padding to ignore at the end. No one has produced
a reproducible test case to investigate for this latest one. Given
two executables where function X appears to be different sizes, it
is usually easy to see by hand what the program is missing. But you
need that test case.

--joel
 
Thank you very much!



smallphd at aliyun.com
_______________________________________________
devel mailing list
devel at rtems.org
http://lists.rtems.org/mailman/listinfo/devel
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.rtems.org/pipermail/devel/attachments/20201023/5b92ba71/attachment-0001.html>