rtems-test and test coverage doubts

[Joel Sherrill]

On Wed, Feb 22, 2017 at 1:32 AM, Sergio Paracuellos <
sparacuellos at orbitalcs.com> wrote:

> Hello All,
>
> First of all sorry for my english. I am new in this list and I have been
> playing a couple of weeks  with rtems. I am using branch 4.11 of the git
> repository and I have several doubts in how the testing should be done and
> which is the way to do a good coverage testing.
>
> So, I start from the beginning.
>
> First, I have noted that the rtems source builder in branch 4.11 uses, as
> expected, branch 4.11 of the rtems-tools git repository. This branch seems
> to have some problems with rtems-test to be working well on linux. Looking
> at the repository I could see that this problems have been fixed in 4.10
> branch (and also another changes applied which has not been applied for
> 4.11)  and master but not in 4.11 and I was wondering If version 4.11 is a
> good start point for doing something. Of course, fix this problems is
> straightforward and I can run rtems-tests without problems, but just to
> know, I am asking now.
>
> The other doubt is what would be a good approach to do coverage tests. It
> seems to exist two approachs to do this process: one is using rtems-testing
> repository and other one is using rtems-tools. I am confused because in
> branch 4.11 rtems-testing repository with the configure options used in the
> scripts, rtems does not compile because of RTEMS_DO_NOT_INLINE_CORE_MUTEX_SEIZE.
> For working this correctly you have to change "ISR_Level" to
> ISR_lock_Context * in cpukit/score/src/coremutexseizeintr.c and
> cpukit/score/src/coremutexseize.c. Because of this, I am thinking that
> this is not the best way to do coverage tests (or nobody is using this).
> The other approach is using rtems-tools repository and in some version of
> the past there is a patch (https://devel.rtems.org/
> attachment/wiki/SOCIS/2015/CoverageAnalysis/0001-Create_
> coverage_report_incl_debug_output_for_byte_size_diff.patch) to add a
> --coverage option to rtems-tester. Again, this patch has not been applied
> to rtems-tester in any version and I was wondering why. Also, this
> coverture tests uses coverture-qemu (only i386?) and it seems that there is
> nothing included in these days to do coverture testing in a real board. Am
> I correct with these things? Is coverage testing in rtems being done in
> these days? (last public reports are from 2014?) Rtems uses covoar utility
> instead of just using gcov for example and I'd like to know or where I can
> read the differences and reasons for this. And with all of this questions
> (sorry, I know there are a lot) the main question could be the following:
> What is a good way to do real coverage testing for rtems?
>
>
covoar was moved to rtems-tools to be part of the rtems-testing suite.
There was
a project to update the rtems-tester to do the coverage runs and reports
that
were formerly done by scripts in the rtems-testing git repo. This project
was
not completed and I don't know the precise status offhand. It needs to be
completed.
As a result, there have been no coverage reports on 4.11.

4.11 and newer have significant changes so it doesn't surprise me that the
code doesn't compile when you define RTEMS_DO_NOT_INLINE_CORE_MUTEX_SEIZE.
The rationale for these conditionals when building for coverage is that it
ends up
inlining code that has compare/branches and you end up with more assembly
code
to cover but it is the same functionality. A patch for any code which
doesn't compile
is appreciated. Also it is quite possible that the build options may need
to be tweaked
for coverage builds for 4.11 and newer. But this is minor in comparison to
getting
the reports in the first place.

As I recall, there were hard-coded paths in the SOCIS patches which had not
been
resolved but that may just be a memory of an earlier code review. There may
also
have been issues that it was only tested on a single BSP and possibly
hard-coded
options on that.

It would be greatly appreciated and of significant value to the project if
you could
try out the rtems-tester coverage addition patches and help get them in
shape
to merge. They should be close but someone with a view to using them for
real work will have the right perspective. :)


> Thank you very much for your time. I really apreciate your help.
>
>
We hope to appreciate your help. :)

Thanks.

--joel


> Cheers,
>     Sergio Paracuellos
>
>
>
>
> _______________________________________________
> users mailing list
> users at rtems.org
> http://lists.rtems.org/mailman/listinfo/users
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.rtems.org/pipermail/users/attachments/20170222/e095f41e/attachment.html>