rtems-tools: coverage covoar GSoC merge

[Vijay Kumar Banerjee]

On 4 May 2018 at 04:34, Joel Sherrill <joel at rtems.org> wrote:

>
>
> On Thu, May 3, 2018 at 1:45 PM, Vijay Kumar Banerjee <
> vijaykumar9597 at gmail.com> wrote:
>
>>
>> On 3 May 2018 at 22:58, Cillian O'Donnell <cpodonnell8 at gmail.com> wrote:
>>
>>>
>>>
>>> On Thu, 3 May 2018, 16:23 Vijay Kumar Banerjee, <
>>> vijaykumar9597 at gmail.com> wrote:
>>>
>>>> Hello,
>>>>
>>>> I want to ask some things about the project to get a clear
>>>> understanding of the objectives/milestones and current status of the
>>>> project. I also seek advice on my Tasks/obectives.
>>>>
>>>> 1. The covoar has been updated to read symbols from the library and the
>>>> next milestone is to remove covoar's dependancy on the external tools,
>>>> which Chris is working on . ( Is that correct? )
>>>>
>>>
>>> Looks like it won't be necessary for gsoc, so we won't have to wait for
>>> their removal. Chris might still have some other changes to make though and
>>> then we can pull master and branch off from there.
>>>
>> Understood.
>>
>
> If it is working as is, you are OK to work on GSoC objectives. Emphasis on
> the "working" part.
> If something is broken right now, we want to fix it. :)
>
> Okay, got the point.

> We also want to make sure all of the previous work is merged into the
> master. There may be
> clean up left for this. Cillian is the best person to answer this one.
>
> Chris has identified things to improve covoar which are not all required
> to be done now.
>
> FWIW some history which might help get some perspective.
>
> covoar was functional about a decade ago and driven by shell scripts that
> are still
> in the rtems-testing repository.  That is executable evidence of the work
> flow. I have some
> odd figures and presentations from about the same time frame. That means
> it predates
> the RSB and rtems-tools. Consequently, it predates using Python and C++ on
> the host
> side for RTEMS tools. The shell scripts driving the process and invoking
> covoar should
> now be all in Python. covoar itself needs some clean up to match current
> C++ practices.
>
> The use of external tools was actually recommended by a binutils
> maintainer at the
> time because the output of the tools is stable. That let us focus on the
> algorithms
> to analyse the coverage. Now we can replace the use of those where
> possible. The
> RTEMS Project did not use the ELF or DWARF libraries back then.
>
> My shell scripts only did a few subsets of code to run reports on. One of
> the big
> changes in the move to Python was a move to reporting on a larger number of
> small sets. For example, report coverage on score, rtems, dosfs, etc.
> rather than
> just "core" or "all".
>
> I think this is the third GSoC project to work on these tools. That
> doesn't count
> at least two students working on RTEMS tests to improve the actual
> coverage.
>
> Hopefully Cillian's work last summer got us over the hurdle of being able
> to generate
> the reports using rtems-tester. Pushing to get all that merged is an
> important and
> critical precondition for this summer. All outstanding work should be in
> the
> RTEMS.org repository.
>
> And all your work should land there as well as soon as it is ready. :)
>
Thanks for the history about covoar, I will look into covoar code more
closely to get a clear picture.

>
>
>
>>
>>>> 2. after it is done , the next step,I think, would be to update the
>>>> coverage.py and test.py with the changes in covoar.
>>>>
>>>
>>> Yeah getting all the rtems tester code up to a standard that Chris will
>>> be happy to merge it will be the next step.
>>>
>> So basically we wait for Chris to make the changes to covoar, needed for
>> us to start working on coverage code to make it running and up to the
>> standards.
>>
>
> Chris can answer this. But if it works and produces coverage reports, it
> is ready.
> If it is broken, report it.
>
> All clean up and removal of external tools should not impact your project
> if the
> code is working now. :)
>
understood. I'll follow the updates
and work on those scripts accordingly to make them work with the updates.

>
>
>>>
>>>> While the Covoar is being updated, shall I be looking into the
>>>> documentations and read the code to gain better understanding? Do you
>>>> suggest me to work on some specific task ?
>>>>
>>>
>>> Understanding how covoar is working will be useful. Running covoar with
>>> -v option and reading down through covoar.cc should help you get an
>>> overview of what's going on. You don't need to understand every detail,
>>> just get a general sense of it.
>>>
>>> okay, I'll follow this.
>>
>
> There is a Word document in the covoar directory. Cillian does that flow
> look right to you?
>
>
>
>> Looks like gcov is going to feature heavily in your project. So reading
>>> up on that and understanding what state the gcov support in covoar is like
>>> should be useful.
>>>
>>> Understood. I'll read about it.
>> Thanks for the suggestions.
>>
>
> My understanding is that it works (or worked) and that the reports
> generated by
> running "gcov" on those output files did not match the results generated
> by
> covoar's own reports.
>
> The challenge is to find one of those disagreements and let's make sure
> covoar
> is right. Then we can bring in the GCC guy who offered to help.
>
seeing it working first will help get a picture of the discrepancies. This
certainly will be a challenge and making it run as soon as possible will be
very necessary to get started.

>
> --joel
>
>
>
>>
>>>> Please suggest resources to gain better understanding of how coverage
>>>> analysis works/supposed to work with the rtems tester. (I can even try
>>>> updating the documentations with some help, this will also help me get
>>>> better understanding ).
>>>>
>>>> Thank you.
>>>> _______________________________________________
>>>> devel mailing list
>>>> devel at rtems.org
>>>> http://lists.rtems.org/mailman/listinfo/devel
>>>
>>>
>>
>> _______________________________________________
>> devel mailing list
>> devel at rtems.org
>> http://lists.rtems.org/mailman/listinfo/devel
>>
>
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.rtems.org/pipermail/devel/attachments/20180504/2e38dd96/attachment-0002.html>