rtems-5.1 source code test coverage failed

[Joel Sherrill]

Hit send too soon

On Thu, Oct 22, 2020, 8:59 PM Joel Sherrill <joel at rtems.org> wrote:

>
>
> On Thu, Oct 22, 2020, 8:53 PM smallphd at aliyun.com <smallphd at aliyun.com>
> wrote:
>
>> Thanks joel.
>>
>> I google the Couverture-Qemu and find many useful document about rtems
>> test coverage.
>> The conventional approach will compile source codes to create *.gcno
>> files. But the Couverture-Qemu method needn't.
>> In the final report, the Couverture-Qemu method will create an
>> annotated.html file in which the instructions are listed as "NOT EXECUTED"
>> or executed.
>> The main different is only assembly language is used, while the
>> conventional approach will list the souce code in C language.
>>
>
> The reports from covoar include the source code. The reports are from a
> developer perspective and we haven't had any feedback from iv&v folks.  If
> the reports don't match what you need, there is always the source code . :)
>

This is an area where feedback, contributions, and supporting developers is
really appreciated.

And there is an RSB recipe for couverture qemu which should make it easier
to build.

I'd love to see this area of our tooling and ecosystem get love and grow.
:)

--joel

>
>
>> ------------------------------
>> smallphd at aliyun.com
>>
>>
>> *From:* Joel Sherrill <joel at rtems.org>
>> *Date:* 2020-10-22 23:49
>> *To:* smallphd at aliyun.com
>> *CC:* devel <devel at rtems.org>
>> *Subject:* Re: rtems-5.1 source code test coverage failed
>>
>>
>> On Thu, Oct 22, 2020 at 4:02 AM smallphd at aliyun.com <smallphd at aliyun.com>
>> wrote:
>>
>>> hi, all
>>> I want test source code coverage in qemu. There are some questions I met.
>>> 1. If compiling without -ftest-coverage, the test only produce a percent
>>> coverage, no *.gcov file which is important to recognize which lines not
>>> execute.
>>> 2. If compiling with -ftest-coverage, *.gcno are created in the same
>>> directory with *.o and these *.gcno file are included
>>> in xilinx_zynq_a9_qemu-symbols.ini. However this ini file could only
>>> include *.o. So the execution will raise error ----- File format not ELF or
>>> archive.
>>>
>>> Is there a tutorial introducing how to get rtems-5.1 source code test
>>> coverage?
>>>
>>
>> We don't do coverage that way. Some simulators can directly produce gcov
>> data.
>> I think tsim can do that for example.
>>
>> gcov foundationally assumes that it can run, collect data, and write it
>> to a file
>> at program exit. That model doesn't work on embedded targets. There are
>> some write ups on the net of people forcing gcov data to be written
>> sporadically
>> and magic to get it off target. But this isn't how RTEMS has done
>> coverage.
>>
>> We depend on simulators that write trace logs and a utility called
>> covoar which
>> can correlate multiple trace logs with the program executables and product
>> a unified report. Remember that the same function may be in two RTEMS
>> executables but they will not be at the same physical address. The
>> trace/log
>> execution information produced by the simulators will be in terms of which
>> physical addresses are executed.
>>
>> The RTEMS Tester has support for running coverage and hopefully the
>> documentation is good at explaining that.
>>
>> The qemu couverture project (a separate repo from regular qemu), sis,
>> and tsim all produce trace/coverage info for an executable that covoar
>> should be able to process.
>>
>> FWIW there is at least one known issue with covoar right now. Sometimes
>> the reported size of a method differs between two executables.
>> Historically,
>> this has been due to padding after a method so the next adjacent
>> method in memory is cache line aligned. covoar will reject information
>> when a method has different sizes in different executables. But can
>> adjust if it is nop padding to ignore at the end. No one has produced
>> a reproducible test case to investigate for this latest one. Given
>> two executables where function X appears to be different sizes, it
>> is usually easy to see by hand what the program is missing. But you
>> need that test case.
>>
>> --joel
>>
>>
>>> Thank you very much!
>>>
>>> ------------------------------
>>> smallphd at aliyun.com
>>> _______________________________________________
>>> devel mailing list
>>> devel at rtems.org
>>> http://lists.rtems.org/mailman/listinfo/devel
>>
>>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.rtems.org/pipermail/devel/attachments/20201022/478d8900/attachment.html>