rtems-tools: coverage covoar GSoC merge

[Vijay Kumar Banerjee]

On Fri, 4 May 2018, 12:17 Cillian O'Donnell, <cpodonnell8 at gmail.com> wrote:

>
>
> On Fri, 4 May 2018, 00:04 Joel Sherrill, <joel at rtems.org> wrote:
>
>>
>>
>> On Thu, May 3, 2018 at 1:45 PM, Vijay Kumar Banerjee <
>> vijaykumar9597 at gmail.com> wrote:
>>
>>>
>>> On 3 May 2018 at 22:58, Cillian O'Donnell <cpodonnell8 at gmail.com> wrote:
>>>
>>>>
>>>>
>>>> On Thu, 3 May 2018, 16:23 Vijay Kumar Banerjee, <
>>>> vijaykumar9597 at gmail.com> wrote:
>>>>
>>>>> Hello,
>>>>>
>>>>> I want to ask some things about the project to get a clear
>>>>> understanding of the objectives/milestones and current status of the
>>>>> project. I also seek advice on my Tasks/obectives.
>>>>>
>>>>> 1. The covoar has been updated to read symbols from the library and
>>>>> the next milestone is to remove covoar's dependancy on the external tools,
>>>>> which Chris is working on . ( Is that correct? )
>>>>>
>>>>
>>>> Looks like it won't be necessary for gsoc, so we won't have to wait for
>>>> their removal. Chris might still have some other changes to make though and
>>>> then we can pull master and branch off from there.
>>>>
>>> Understood.
>>>
>>
>> If it is working as is, you are OK to work on GSoC objectives. Emphasis
>> on the "working" part.
>> If something is broken right now, we want to fix it. :)
>>
>> We also want to make sure all of the previous work is merged into the
>> master. There may be
>> clean up left for this. Cillian is the best person to answer this one.
>>
>> Chris has identified things to improve covoar which are not all required
>> to be done now.
>>
>> FWIW some history which might help get some perspective.
>>
>> covoar was functional about a decade ago and driven by shell scripts that
>> are still
>> in the rtems-testing repository.  That is executable evidence of the work
>> flow. I have some
>> odd figures and presentations from about the same time frame. That means
>> it predates
>> the RSB and rtems-tools. Consequently, it predates using Python and C++
>> on the host
>> side for RTEMS tools. The shell scripts driving the process and invoking
>> covoar should
>> now be all in Python. covoar itself needs some clean up to match current
>> C++ practices.
>>
>> The use of external tools was actually recommended by a binutils
>> maintainer at the
>> time because the output of the tools is stable. That let us focus on the
>> algorithms
>> to analyse the coverage. Now we can replace the use of those where
>> possible. The
>> RTEMS Project did not use the ELF or DWARF libraries back then.
>>
>> My shell scripts only did a few subsets of code to run reports on. One of
>> the big
>> changes in the move to Python was a move to reporting on a larger number
>> of
>> small sets. For example, report coverage on score, rtems, dosfs, etc.
>> rather than
>> just "core" or "all".
>>
>> I think this is the third GSoC project to work on these tools. That
>> doesn't count
>> at least two students working on RTEMS tests to improve the actual
>> coverage.
>>
>> Hopefully Cillian's work last summer got us over the hurdle of being able
>> to generate
>> the reports using rtems-tester. Pushing to get all that merged is an
>> important and
>> critical precondition for this summer. All outstanding work should be in
>> the
>> RTEMS.org repository.
>>
>> And all your work should land there as well as soon as it is ready. :)
>>
>>
>>
>>>
>>>>> 2. after it is done , the next step,I think, would be to update the
>>>>> coverage.py and test.py with the changes in covoar.
>>>>>
>>>>
>>>> Yeah getting all the rtems tester code up to a standard that Chris will
>>>> be happy to merge it will be the next step.
>>>>
>>> So basically we wait for Chris to make the changes to covoar, needed for
>>> us to start working on coverage code to make it running and up to the
>>> standards.
>>>
>>
>> Chris can answer this. But if it works and produces coverage reports, it
>> is ready.
>> If it is broken, report it.
>>
>> All clean up and removal of external tools should not impact your project
>> if the
>> code is working now. :)
>>
>>
>>>>
>>>>> While the Covoar is being updated, shall I be looking into the
>>>>> documentations and read the code to gain better understanding? Do you
>>>>> suggest me to work on some specific task ?
>>>>>
>>>>
>>>> Understanding how covoar is working will be useful. Running covoar with
>>>> -v option and reading down through covoar.cc should help you get an
>>>> overview of what's going on. You don't need to understand every detail,
>>>> just get a general sense of it.
>>>>
>>>> okay, I'll follow this.
>>>
>>
>> There is a Word document in the covoar directory. Cillian does that flow
>> look right to you?
>>
>
> Not seeing the txt file, what's it called?
>
https://github.com/RTEMS/rtems-tools/blob/master/tester/covoar/covoar_flow.doc

>
>>
>>
>>> Looks like gcov is going to feature heavily in your project. So reading
>>>> up on that and understanding what state the gcov support in covoar is like
>>>> should be useful.
>>>>
>>>> Understood. I'll read about it.
>>> Thanks for the suggestions.
>>>
>>
>> My understanding is that it works (or worked) and that the reports
>> generated by
>> running "gcov" on those output files did not match the results generated
>> by
>> covoar's own reports.
>>
>> The challenge is to find one of those disagreements and let's make sure
>> covoar
>> is right. Then we can bring in the GCC guy who offered to help.
>>
>> --joel
>>
>>
>>
>>>
>>>>> Please suggest resources to gain better understanding of how coverage
>>>>> analysis works/supposed to work with the rtems tester. (I can even try
>>>>> updating the documentations with some help, this will also help me get
>>>>> better understanding ).
>>>>>
>>>>> Thank you.
>>>>> _______________________________________________
>>>>> devel mailing list
>>>>> devel at rtems.org
>>>>> http://lists.rtems.org/mailman/listinfo/devel
>>>>
>>>>
>>>
>>> _______________________________________________
>>> devel mailing list
>>> devel at rtems.org
>>> http://lists.rtems.org/mailman/listinfo/devel
>>>
>>
>>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.rtems.org/pipermail/devel/attachments/20180504/e159dbbd/attachment-0002.html>