[PATCH] rtems-tools/covoar: Wrong coverage computed

[Joel Sherrill]

Sorry for the duplicate email. I didn't use the right email address and the
response did not go to devel@

On Fri, Jan 25, 2019 at 8:43 AM Jiri Gaisler <jiri at gaisler.se> wrote:

> Sorry for the noise, but I have a better patch that actually solves the
> root problem. I ran the full testsuite on sis and covoar against libscore.a
> like this:
>
> $ sparc-rtems5-ar -rcs sparc-rtems5/c/leon3/cpukit/score/libscore.a
> sparc-rtems5/c/leon3/cpukit/score/src/*.o
>
> $ rtems-test --rtems-bsp=leon3-sis-cov sparc-rtems5/c/leon3/testsuites
>
> $ /opt/rtems/5/share/rtems/tester/bin/covoar -v -f TSIM -S
> /opt/rtems/5/share/rtems/tester/rtems/testing/coverage/score-symbols.ini -O
> coverage -E
> /opt/rtems/5/share/rtems/tester/rtems/testing/coverage/Explanations.txt -p
> RTEMS-5  sparc-rtems5/c/leon3/testsuites/*/*.exe
>
> summary.txt:
>
> Bytes Analyzed           : 39376 Bytes Not Executed       : 2864
> Percentage Executed      : 92.73 Percentage Not Executed  : 7.273 Uncovered
> ranges found   : 108 Total branches found     : 860 Uncovered branches
> found : 200    61 branches always taken    139 branches never taken
>
> All symbol sizes and ranges are now correctly reported/calculated in the
> reports. There are two remaining issues to be solved:
>
> 1. File names are printed as 'unknown' in reports (minor problem but
> useful)
>
Originally we got the info via the binutils command line tools. I think the
filename came from addr2line. When we designed it originally, Ian Lance
Taylor said parsing the tool output was more easier, reliable, and more
stable than tracking any elf reading library.

But it was slower than accessing the data directly via ELF and DWARF.
Since Chris added this support for the dynamic loader, it was natural to
begin to replace that code. Unfortunately, there are some warts left from
the transition. The name may be one of them.

We also think the beginning and end of a method are not marked properly.

And when something is inlined, it isn't considered part of the main method.
The original method considered assembly from inlined code part of the
caller. It is now reported as part of the callee. We have identified a
needed
change to (1) account for the inlined assembly as part of the caller and
(2) add additional tracking of the coverage of inlined methods.

The immediate need is to get the reporting so it accurately annoates
the generated assembly from a method and anything inlined into it.
This should let us do DO-178 Level A style coverage -- we are
sure we use ever instruction and take/not take every branch.

If there is unreachable code or branches where both paths are not
exercised, we need tests to exercise them OR the code is useless
and can be removed. Either way, we win.

> 2. We should add RISC-V support to covoar ...
>

I am sure you have seen the Target* files. They are not large. Some is
trivial. Other parts are to recognize nops and branch instruction
variants.

 On a RISC architecture, I wouldn't expect a lot of variation in nops.
The x86 has an awful set that range in size. :(

Let me know if you agree to the patch ..
>

The patch looks good to me.

> _______________________________________________
> devel mailing list
> devel at rtems.org
> http://lists.rtems.org/mailman/listinfo/devel
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.rtems.org/pipermail/devel/attachments/20190125/0602b987/attachment-0002.html>