[PATCH v2] tester: Add script to generate html coverage report from covoar output

[Vijay Kumar Banerjee]

On 2 June 2018 at 00:48, Joel Sherrill <joel at rtems.org> wrote:

>
>
> On Fri, Jun 1, 2018, 11:21 AM Vijay Kumar Banerjee <
> vijaykumar9597 at gmail.com> wrote:
>
>> On 1 June 2018 at 20:30, Gedare Bloom <gedare at rtems.org> wrote:
>>
>>> On Fri, Jun 1, 2018 at 10:28 AM, Vijay Kumar Banerjee
>>> <vijaykumar9597 at gmail.com> wrote:
>>> > On 1 June 2018 at 19:24, Joel Sherrill <joel at rtems.org> wrote:
>>> >>
>>> >>
>>> >>
>>> >> On Fri, Jun 1, 2018 at 2:46 AM, Vijay Kumar Banerjee
>>> >> <vijaykumar9597 at gmail.com> wrote:
>>> >>>
>>> >>> Here's the list of Ideas for improvements:
>>> >>>
>>> >>> 1. Include the section coverage in the bsp config file.
>>> >>>     If the section is not found then the script will show
>>> >>>     proper error showing coverage is not supported for the
>>> >>>     provided bsp config file.
>>> >>>
>>> >>> 2. Update covoar to add support for separate coverage report
>>> >>>     for each symbol set.
>>> >>>
>>> >>> 3. Add a method somewhere in covoar to get the size of an instruction
>>> >>>     and fix the hard coded size 4 in ObjdumpProcessor.cc
>>> >>
>>> >>
>>> >> What about a single XXX_run command? What about that suggestion?
>>> >>
>>> > The suggestion was to turn test_run and coverage_run into a single
>>> command.
>>> > I have kept them separate so that there's a possibility to just run the
>>> > test.
>>> >
>>> > If we want to run coverage everytime we run the test. we can do it.
>>> > Then I think the --coverage option can be changed to --coverage-sets
>>> > to mention the sets.
>>> > If that's what we're looking for then I don't think a separate ticket
>>> is
>>> > needed,
>>> > I can try to do it within tomorrow and submit an updated patch.
>>> >
>>> >>
>>> >> Will there be an update to this patch?
>>> >>
>>> > This patch is working an showing results. I don't have any work
>>> > going related to this patch currently.
>>> > If there are any suggestions, I'll try to include all the suggested
>>> updates
>>> > as soon as possible and submit. So that we can get it merged.
>>> >
>>>
>>> I get confused by the similarity between test_run() and coverage_run()
>>> names, and now I'm also seeing some confusion because there is a
>>> function coverage_run() and a class coverage_run. I suggest you remove
>>> this function coverage_run(), and make either coverage_run.__init__()
>>> or coverage_run.run() take the executables as a parameter directly.
>>>
>>> Thank you for the suggestion. :)
>> I have removed the function and taken executables as a parameter in
>> coverage_run.__init__()
>>
>> I have a question.
>> The ini file that is fed to covoar is written by the script according to
>> the
>> symbols mentioned by the user. I haven't include the ini file in the
>> patch.
>> I'm planning to delete the file after the run, unless --no-clean option
>> is given,
>> which currently deletes the .cov trace files after the run.
>>
>
> That makes sense.
>
>
>> Can I proceed with this ?
>>
>
> Yes.
>
Added. Thanks. :)

> also, shall I include that in the .gitignore ?
>>
>
> Is the name of the file constant? The same across multiple BSPs? If so,
> then this will be a problem doing automated testing of multiple BSPs in
> parallel.
>
> The ini file I'm talking about is the symbol sets config file not the bsp
config file. yes the name is constant. Should it be unique to the bsp ?
something like, leon3-symbols.ini ?
How does the automated testing work?

> I think the name needs to be unique enough.to support running testing
> with coverage on multiple BSPs in parallel. That means you can't add it to
> gitignore. And can add another issue and FIXME to the code.
>
>> If it is needed then I have a fix in mind. I can take the bsp name and add
'-symbols.ini' to it. and add *-symbols.ini to .gitignore .
Can we do this ?

>
>>
>>> -Gedare
>>>
>>
>>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.rtems.org/pipermail/devel/attachments/20180602/5251da57/attachment-0002.html>