rtems-tools: coverage covoar GSoC merge

Thu May 3 23:04:07 UTC 2018

On Thu, May 3, 2018 at 1:45 PM, Vijay Kumar Banerjee <
vijaykumar9597 at gmail.com> wrote:

>
> On 3 May 2018 at 22:58, Cillian O'Donnell <cpodonnell8 at gmail.com> wrote:
>
>>
>>
>> On Thu, 3 May 2018, 16:23 Vijay Kumar Banerjee, <vijaykumar9597 at gmail.com>
>> wrote:
>>
>>> Hello,
>>>
>>> I want to ask some things about the project to get a clear understanding
>>> of the objectives/milestones and current status of the project. I also seek
>>> advice on my Tasks/obectives.
>>>
>>> 1. The covoar has been updated to read symbols from the library and the
>>> next milestone is to remove covoar's dependancy on the external tools,
>>> which Chris is working on . ( Is that correct? )
>>>
>>
>> Looks like it won't be necessary for gsoc, so we won't have to wait for
>> their removal. Chris might still have some other changes to make though and
>> then we can pull master and branch off from there.
>>
> Understood.
>

If it is working as is, you are OK to work on GSoC objectives. Emphasis on
the "working" part.
If something is broken right now, we want to fix it. :)

We also want to make sure all of the previous work is merged into the
master. There may be
clean up left for this. Cillian is the best person to answer this one.

Chris has identified things to improve covoar which are not all required to
be done now.

FWIW some history which might help get some perspective.

covoar was functional about a decade ago and driven by shell scripts that
are still
in the rtems-testing repository.  That is executable evidence of the work
flow. I have some
odd figures and presentations from about the same time frame. That means it
predates
the RSB and rtems-tools. Consequently, it predates using Python and C++ on
the host
side for RTEMS tools. The shell scripts driving the process and invoking
covoar should
now be all in Python. covoar itself needs some clean up to match current
C++ practices.

The use of external tools was actually recommended by a binutils maintainer
at the
time because the output of the tools is stable. That let us focus on the
algorithms
to analyse the coverage. Now we can replace the use of those where
possible. The
RTEMS Project did not use the ELF or DWARF libraries back then.

My shell scripts only did a few subsets of code to run reports on. One of
the big
changes in the move to Python was a move to reporting on a larger number of
small sets. For example, report coverage on score, rtems, dosfs, etc.
rather than
just "core" or "all".

I think this is the third GSoC project to work on these tools. That doesn't
count
at least two students working on RTEMS tests to improve the actual coverage.

Hopefully Cillian's work last summer got us over the hurdle of being able
to generate
the reports using rtems-tester. Pushing to get all that merged is an
important and
critical precondition for this summer. All outstanding work should be in the
RTEMS.org repository.

And all your work should land there as well as soon as it is ready. :)

>
>>> 2. after it is done , the next step,I think, would be to update the
>>> coverage.py and test.py with the changes in covoar.
>>>
>>
>> Yeah getting all the rtems tester code up to a standard that Chris will
>> be happy to merge it will be the next step.
>>
> So basically we wait for Chris to make the changes to covoar, needed for
> us to start working on coverage code to make it running and up to the
> standards.
>

Chris can answer this. But if it works and produces coverage reports, it is
ready.
If it is broken, report it.

All clean up and removal of external tools should not impact your project
if the
code is working now. :)

>>
>>> While the Covoar is being updated, shall I be looking into the
>>> documentations and read the code to gain better understanding? Do you
>>> suggest me to work on some specific task ?
>>>
>>
>> Understanding how covoar is working will be useful. Running covoar with
>> -v option and reading down through covoar.cc should help you get an
>> overview of what's going on. You don't need to understand every detail,
>> just get a general sense of it.
>>
>> okay, I'll follow this.
>

There is a Word document in the covoar directory. Cillian does that flow
look right to you?

> Looks like gcov is going to feature heavily in your project. So reading up
>> on that and understanding what state the gcov support in covoar is like
>> should be useful.
>>
>> Understood. I'll read about it.
> Thanks for the suggestions.
>

My understanding is that it works (or worked) and that the reports
generated by
running "gcov" on those output files did not match the results generated by
covoar's own reports.

The challenge is to find one of those disagreements and let's make sure
covoar
is right. Then we can bring in the GCC guy who offered to help.

--joel

>
>>> Please suggest resources to gain better understanding of how coverage
>>> analysis works/supposed to work with the rtems tester. (I can even try
>>> updating the documentations with some help, this will also help me get
>>> better understanding ).
>>>
>>> Thank you.
>>> _______________________________________________
>>> devel mailing list
>>> devel at rtems.org
>>> http://lists.rtems.org/mailman/listinfo/devel
>>
>>
>
> _______________________________________________
> devel mailing list
> devel at rtems.org
> http://lists.rtems.org/mailman/listinfo/devel
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.rtems.org/pipermail/devel/attachments/20180503/a9e56878/attachment-0002.html>