rtems-tools: coverage covoar GSoC merge

Fri May 4 06:57:02 UTC 2018

On Fri, 4 May 2018, 07:51 Vijay Kumar Banerjee, <vijaykumar9597 at gmail.com>
wrote:

>
>
> On Fri, 4 May 2018, 12:17 Cillian O'Donnell, <cpodonnell8 at gmail.com>
> wrote:
>
>>
>>
>> On Fri, 4 May 2018, 00:04 Joel Sherrill, <joel at rtems.org> wrote:
>>
>>>
>>>
>>> On Thu, May 3, 2018 at 1:45 PM, Vijay Kumar Banerjee <
>>> vijaykumar9597 at gmail.com> wrote:
>>>
>>>>
>>>> On 3 May 2018 at 22:58, Cillian O'Donnell <cpodonnell8 at gmail.com>
>>>> wrote:
>>>>
>>>>>
>>>>>
>>>>> On Thu, 3 May 2018, 16:23 Vijay Kumar Banerjee, <
>>>>> vijaykumar9597 at gmail.com> wrote:
>>>>>
>>>>>> Hello,
>>>>>>
>>>>>> I want to ask some things about the project to get a clear
>>>>>> understanding of the objectives/milestones and current status of the
>>>>>> project. I also seek advice on my Tasks/obectives.
>>>>>>
>>>>>> 1. The covoar has been updated to read symbols from the library and
>>>>>> the next milestone is to remove covoar's dependancy on the external tools,
>>>>>> which Chris is working on . ( Is that correct? )
>>>>>>
>>>>>
>>>>> Looks like it won't be necessary for gsoc, so we won't have to wait
>>>>> for their removal. Chris might still have some other changes to make though
>>>>> and then we can pull master and branch off from there.
>>>>>
>>>> Understood.
>>>>
>>>
>>> If it is working as is, you are OK to work on GSoC objectives. Emphasis
>>> on the "working" part.
>>> If something is broken right now, we want to fix it. :)
>>>
>>> We also want to make sure all of the previous work is merged into the
>>> master. There may be
>>> clean up left for this. Cillian is the best person to answer this one.
>>>
>>> Chris has identified things to improve covoar which are not all required
>>> to be done now.
>>>
>>> FWIW some history which might help get some perspective.
>>>
>>> covoar was functional about a decade ago and driven by shell scripts
>>> that are still
>>> in the rtems-testing repository.  That is executable evidence of the
>>> work flow. I have some
>>> odd figures and presentations from about the same time frame. That means
>>> it predates
>>> the RSB and rtems-tools. Consequently, it predates using Python and C++
>>> on the host
>>> side for RTEMS tools. The shell scripts driving the process and invoking
>>> covoar should
>>> now be all in Python. covoar itself needs some clean up to match current
>>> C++ practices.
>>>
>>> The use of external tools was actually recommended by a binutils
>>> maintainer at the
>>> time because the output of the tools is stable. That let us focus on the
>>> algorithms
>>> to analyse the coverage. Now we can replace the use of those where
>>> possible. The
>>> RTEMS Project did not use the ELF or DWARF libraries back then.
>>>
>>> My shell scripts only did a few subsets of code to run reports on. One
>>> of the big
>>> changes in the move to Python was a move to reporting on a larger number
>>> of
>>> small sets. For example, report coverage on score, rtems, dosfs, etc.
>>> rather than
>>> just "core" or "all".
>>>
>>> I think this is the third GSoC project to work on these tools. That
>>> doesn't count
>>> at least two students working on RTEMS tests to improve the actual
>>> coverage.
>>>
>>> Hopefully Cillian's work last summer got us over the hurdle of being
>>> able to generate
>>> the reports using rtems-tester. Pushing to get all that merged is an
>>> important and
>>> critical precondition for this summer. All outstanding work should be in
>>> the
>>> RTEMS.org repository.
>>>
>>> And all your work should land there as well as soon as it is ready. :)
>>>
>>>
>>>
>>>>
>>>>>> 2. after it is done , the next step,I think, would be to update the
>>>>>> coverage.py and test.py with the changes in covoar.
>>>>>>
>>>>>
>>>>> Yeah getting all the rtems tester code up to a standard that Chris
>>>>> will be happy to merge it will be the next step.
>>>>>
>>>> So basically we wait for Chris to make the changes to covoar, needed
>>>> for us to start working on coverage code to make it running and up to the
>>>> standards.
>>>>
>>>
>>> Chris can answer this. But if it works and produces coverage reports, it
>>> is ready.
>>> If it is broken, report it.
>>>
>>> All clean up and removal of external tools should not impact your
>>> project if the
>>> code is working now. :)
>>>
>>>
>>>>>
>>>>>> While the Covoar is being updated, shall I be looking into the
>>>>>> documentations and read the code to gain better understanding? Do you
>>>>>> suggest me to work on some specific task ?
>>>>>>
>>>>>
>>>>> Understanding how covoar is working will be useful. Running covoar
>>>>> with -v option and reading down through covoar.cc should help you get an
>>>>> overview of what's going on. You don't need to understand every detail,
>>>>> just get a general sense of it.
>>>>>
>>>>> okay, I'll follow this.
>>>>
>>>
>>> There is a Word document in the covoar directory. Cillian does that flow
>>> look right to you?
>>>
>>
>> Not seeing the txt file, what's it called?
>>
>
> https://github.com/RTEMS/rtems-tools/blob/master/tester/covoar/covoar_flow.doc
>

Thanks Vijay. Can't open it on a phone unfortunately. I'll have a look
later.

>
>
>>>
>>>
>>>> Looks like gcov is going to feature heavily in your project. So reading
>>>>> up on that and understanding what state the gcov support in covoar is like
>>>>> should be useful.
>>>>>
>>>>> Understood. I'll read about it.
>>>> Thanks for the suggestions.
>>>>
>>>
>>> My understanding is that it works (or worked) and that the reports
>>> generated by
>>> running "gcov" on those output files did not match the results generated
>>> by
>>> covoar's own reports.
>>>
>>> The challenge is to find one of those disagreements and let's make sure
>>> covoar
>>> is right. Then we can bring in the GCC guy who offered to help.
>>>
>>> --joel
>>>
>>>
>>>
>>>>
>>>>>> Please suggest resources to gain better understanding of how coverage
>>>>>> analysis works/supposed to work with the rtems tester. (I can even try
>>>>>> updating the documentations with some help, this will also help me get
>>>>>> better understanding ).
>>>>>>
>>>>>> Thank you.
>>>>>> _______________________________________________
>>>>>> devel mailing list
>>>>>> devel at rtems.org
>>>>>> http://lists.rtems.org/mailman/listinfo/devel
>>>>>
>>>>>
>>>>
>>>> _______________________________________________
>>>> devel mailing list
>>>> devel at rtems.org
>>>> http://lists.rtems.org/mailman/listinfo/devel
>>>>
>>>
>>>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.rtems.org/pipermail/devel/attachments/20180504/b0d21660/attachment-0002.html>