rtems-test and test coverage doubts

Fri Feb 24 06:37:58 UTC 2017

2017-02-22 15:12 GMT+01:00 Joel Sherrill <joel at rtems.org>:

>
>
> On Wed, Feb 22, 2017 at 1:32 AM, Sergio Paracuellos <
> sparacuellos at orbitalcs.com> wrote:
>
>> Hello All,
>>
>> First of all sorry for my english. I am new in this list and I have been
>> playing a couple of weeks  with rtems. I am using branch 4.11 of the git
>> repository and I have several doubts in how the testing should be done and
>> which is the way to do a good coverage testing.
>>
>> So, I start from the beginning.
>>
>> First, I have noted that the rtems source builder in branch 4.11 uses, as
>> expected, branch 4.11 of the rtems-tools git repository. This branch seems
>> to have some problems with rtems-test to be working well on linux. Looking
>> at the repository I could see that this problems have been fixed in 4.10
>> branch (and also another changes applied which has not been applied for
>> 4.11)  and master but not in 4.11 and I was wondering If version 4.11 is a
>> good start point for doing something. Of course, fix this problems is
>> straightforward and I can run rtems-tests without problems, but just to
>> know, I am asking now.
>>
>> The other doubt is what would be a good approach to do coverage tests. It
>> seems to exist two approachs to do this process: one is using rtems-testing
>> repository and other one is using rtems-tools. I am confused because in
>> branch 4.11 rtems-testing repository with the configure options used in the
>> scripts, rtems does not compile because of RTEMS_DO_NOT_INLINE_CORE_MUTEX_SEIZE.
>> For working this correctly you have to change "ISR_Level" to
>> ISR_lock_Context * in cpukit/score/src/coremutexseizeintr.c and
>> cpukit/score/src/coremutexseize.c. Because of this, I am thinking that
>> this is not the best way to do coverage tests (or nobody is using this).
>> The other approach is using rtems-tools repository and in some version of
>> the past there is a patch (https://devel.rtems.org/attac
>> hment/wiki/SOCIS/2015/CoverageAnalysis/0001-Create_coverage_
>> report_incl_debug_output_for_byte_size_diff.patch) to add a --coverage
>> option to rtems-tester. Again, this patch has not been applied to
>> rtems-tester in any version and I was wondering why. Also, this coverture
>> tests uses coverture-qemu (only i386?) and it seems that there is nothing
>> included in these days to do coverture testing in a real board. Am I
>> correct with these things? Is coverage testing in rtems being done in these
>> days? (last public reports are from 2014?) Rtems uses covoar utility
>> instead of just using gcov for example and I'd like to know or where I can
>> read the differences and reasons for this. And with all of this questions
>> (sorry, I know there are a lot) the main question could be the following:
>> What is a good way to do real coverage testing for rtems?
>>
>>
Hi Joel,

Thanks for your response :).

>
> covoar was moved to rtems-tools to be part of the rtems-testing suite.
> There was
> a project to update the rtems-tester to do the coverage runs and reports
> that
> were formerly done by scripts in the rtems-testing git repo. This project
> was
> not completed and I don't know the precise status offhand. It needs to be
> completed.
> As a result, there have been no coverage reports on 4.11.
>

I see, that is what I thought at first. I wasn't wrong.

> 4.11 and newer have significant changes so it doesn't surprise me that the
> code doesn't compile when you define RTEMS_DO_NOT_INLINE_CORE_MUTEX_SEIZE.
> The rationale for these conditionals when building for coverage is that it
> ends up
> inlining code that has compare/branches and you end up with more assembly
> code
> to cover but it is the same functionality. A patch for any code which
> doesn't compile
> is appreciated. Also it is quite possible that the build options may need
> to be tweaked
> for coverage builds for 4.11 and newer. But this is minor in comparison to
> getting
> the reports in the first place.
>

I'll read how to submit patches :-).

>
> As I recall, there were hard-coded paths in the SOCIS patches which had
> not been
> resolved but that may just be a memory of an earlier code review. There
> may also
> have been issues that it was only tested on a single BSP and possibly
> hard-coded
> options on that.
>

If I'm not wrong all of coverage stuff was only oriented to x86
architecture? It seems all is based in coverture-qemu for
get the cov files to get final coverture report with covoar... As far as I
know coverture-qemu is a modified
qemu for x86 architectures, right? Please correct me if I'm wrong. If this
is true I don't understand in which way coverage
testing was thought at first, because with this approach you are not be
able to run and get coverage files in a real board (for example).

Why not just use libgcov with modifications for embedded?

>
> It would be greatly appreciated and of significant value to the project if
> you could
> try out the rtems-tester coverage addition patches and help get them in
> shape
> to merge. They should be close but someone with a view to using them for
> real work will have the right perspective. :)
>
>
>> Thank you very much for your time. I really apreciate your help.
>>
>>
>
> We hope to appreciate your help. :)
>
> Thanks.
>

Thanks again, for your time.

Sergio

>
>
>>
>>
>>
>> _______________________________________________
>> users mailing list
>> users at rtems.org
>> http://lists.rtems.org/mailman/listinfo/users
>>
>
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.rtems.org/pipermail/users/attachments/20170224/08b58760/attachment-0002.html>