Improve coverage analysis toolset

Joel Sherrill joel at rtems.org
Wed Mar 21 23:00:13 UTC 2018 

Glad you have tests running. Hopefully as Cillian suggested, you can fix the
remaining issues to see coverage.

We certainly need to get this merged if Chris is OK with it and it doesn't
break anything else.

I am reviewing proposals now. Seems to be a queue. :)

--joel

On Wed, Mar 21, 2018 at 4:42 AM, Vijay Kumar Banerjee <
vijaykumar9597 at gmail.com> wrote:

> Sir ,
> I have done the changes in the proposal , based on the comments in the
> google doc , please review it and suggest any further changes if required
>
> Thank you ,
> -- vijay
>
> P.S : the previous version is in parentheses , I will remove them after
> you review the changes .
>
> On 18 March 2018 at 16:05, Vijay Kumar Banerjee <vijaykumar9597 at gmail.com>
> wrote:
>
>> Thanks Cillian :)
>>
>> It's great to see it running ,  I'll do some background reading about
>> RSB, and RTEMS-Tools along with the covoar code . And also work on my
>> python skills.
>>
>> to try rtems-test on a bsp that runs gdb , I tried it on erc32 and that
>> also worked.
>>
>> I'm also waiting for Joel and other mentors' review on my draft proposal,
>> so that I can also work on it and make any changes if needed.
>>
>> Thanks .
>>
>> -- vijay
>>
>> On 18 March 2018 at 14:17, Cillian O'Donnell <cpodonnell8 at gmail.com>
>> wrote:
>>
>>>
>>>
>>> On 18 March 2018 at 06:47, Vijay Kumar Banerjee <
>>> vijaykumar9597 at gmail.com> wrote:
>>>
>>>> It worked !
>>>>
>>>
>>> That's great!  That's the good news then. I was using an old leon3 build
>>> and maybe some older Qemu too and I think that's why I didn't see any
>>> issues initially. I was hoping you could see the coverage running and the
>>> reports generated from that but it looks like the full update to the
>>> current master will be necessary to have a look. So until the parsing for
>>> the INI files is added to the coverage.py we won't see the coverage
>>> running. Unfortunately, I'm in the middle of exams until the following
>>> Monday so I won't be able to sink any more time into it until then. You can
>>> still figure out the RSB problem you're having and do some background
>>> reading, brush up on your Python skills, have a read of the covoar code
>>>
>>> https://github.com/RTEMS/rtems-tools/blob/master/tester/covoar/covoar.cc
>>>
>>> just skim through, read the comments, get a sense of the what it's doing
>>> and in what order.
>>>
>>>
>>>> It's great to see it running ! I have attached the result .
>>>>
>>>> -- vijay
>>>>
>>>> On 18 March 2018 at 02:31, Cillian O'Donnell <cpodonnell8 at gmail.com>
>>>> wrote:
>>>>
>>>>>
>>>>>
>>>>> On 17 March 2018 at 20:08, Vijay Kumar Banerjee <
>>>>> vijaykumar9597 at gmail.com> wrote:
>>>>>
>>>>>> yes it prints hello world
>>>>>>
>>>>>
>>>>> Alright I've added an .ini for leon3-qemu to the current rtems-tools.
>>>>> Pull this branch
>>>>>
>>>>> https://github.com/cillianodonnell/rtems-tools/tree/ini-update
>>>>>
>>>>> and try
>>>>>
>>>>> $HOME/development/rtems/test/rtems-tools/tester/rtems-test
>>>>> --rtems-tools=$HOME/development/rtems/5 --log=coverage-analysis.log
>>>>> --rtems-bsp=leon3_qemu $HOME/development/rtems/leon3/
>>>>> sparc-rtems5/c/leon3/testsuites/samples
>>>>>
>>>>>>
>>>>>> -- vijay
>>>>>>
>>>>>> On 18 March 2018 at 01:31, Cillian O'Donnell <cpodonnell8 at gmail.com>
>>>>>> wrote:
>>>>>>
>>>>>>> If you run just one test by itself without rtems-test
>>>>>>>
>>>>>>> qemu-system-sparc -no-reboot -monitor null -serial stdio -nographic
>>>>>>> -M leon3_generic -kernel $HOME/development/rtems/leon3/
>>>>>>> sparc-rtems5/c/leon3/testsuites/samples/hello/hello.exe
>>>>>>>
>>>>>>> Does the hello world print out?
>>>>>>>
>>>>>>> On 17 March 2018 at 14:46, Vijay Kumar Banerjee <
>>>>>>> vijaykumar9597 at gmail.com> wrote:
>>>>>>>
>>>>>>>> I built it manually
>>>>>>>>
>>>>>>>> the environment variable PATH looks like this
>>>>>>>> /home/lunatic/qemu/install/bin:/home/lunatic/development/rte
>>>>>>>> ms/5/bin:/usr/local/bin:/usr/local/sbin:/usr/bin:/usr/sbin:/
>>>>>>>> home/lunatic/.local/bin:/home/lunatic/bin
>>>>>>>>
>>>>>>>> I tried to run rtems-test again without --coverage , it gives the
>>>>>>>> same result
>>>>>>>> I have attached the log.
>>>>>>>>
>>>>>>>> -- vijay
>>>>>>>>
>>>>>>>> On 17 March 2018 at 00:55, Cillian O'Donnell <cpodonnell8 at gmail.com
>>>>>>>> > wrote:
>>>>>>>>
>>>>>>>>> Yes this is something more than my build, I'll need someone a bit
>>>>>>>>> more expert in the RSB to step in there. In the meantime, lets just build
>>>>>>>>> couverture-qemu manually so we can see is everything else working.
>>>>>>>>>
>>>>>>>>> git clone https://github.com/AdaCore/qemu
>>>>>>>>>
>>>>>>>>> cd qemu
>>>>>>>>>
>>>>>>>>> ./configure --target-list=sparc-softmmu
>>>>>>>>> --prefix=$HOME/qemu/install --disable-docs --disable-virtfs --disable-werror
>>>>>>>>>
>>>>>>>>> make
>>>>>>>>>
>>>>>>>>> make install
>>>>>>>>>
>>>>>>>>> then add the prefix to $PATH in .bashrc as well like before.
>>>>>>>>>
>>>>>>>>> export PATH=$HOME/qemu/install/bin:$PATH
>>>>>>>>>
>>>>>>>>> Then run rtem-test and see what happens
>>>>>>>>>
>>>>>>>>>
>>>>>>>>> On 16 March 2018 at 19:13, Vijay Kumar Banerjee <
>>>>>>>>> vijaykumar9597 at gmail.com> wrote:
>>>>>>>>>
>>>>>>>>>> the same error comes when I try to build qemu from the
>>>>>>>>>> RTEMS/rtems-source-builder as well
>>>>>>>>>>
>>>>>>>>>> is the issue coming from my system ? I'm using fedora 27 64bit
>>>>>>>>>>
>>>>>>>>>> On 17 Mar 2018 12:39 a.m., "Vijay Kumar Banerjee" <
>>>>>>>>>> vijaykumar9597 at gmail.com> wrote:
>>>>>>>>>>
>>>>>>>>>>> yes , the same thing happens
>>>>>>>>>>>
>>>>>>>>>>> On 17 Mar 2018 12:13 a.m., "Cillian O'Donnell" <
>>>>>>>>>>> cpodonnell8 at gmail.com> wrote:
>>>>>>>>>>>
>>>>>>>>>>>> If you build regular qemu with the RSB, does the same thing
>>>>>>>>>>>> happen?
>>>>>>>>>>>>
>>>>>>>>>>>> Try
>>>>>>>>>>>>
>>>>>>>>>>>> ../source-builder/sb-set-builder --log=qemu_log.txt
>>>>>>>>>>>> --prefix=$HOME/development/5 devel/qemu
>>>>>>>>>>>>
>>>>>>>>>>>>
>>>>>>>>>>>> On 16 March 2018 at 16:48, Vijay Kumar Banerjee <
>>>>>>>>>>>> vijaykumar9597 at gmail.com> wrote:
>>>>>>>>>>>>
>>>>>>>>>>>>> still the same error
>>>>>>>>>>>>>
>>>>>>>>>>>>> -- vijay
>>>>>>>>>>>>>
>>>>>>>>>>>>> On 16 March 2018 at 21:39, Cillian O'Donnell <
>>>>>>>>>>>>> cpodonnell8 at gmail.com> wrote:
>>>>>>>>>>>>>
>>>>>>>>>>>>>> Just checked and the build was failing because one of the
>>>>>>>>>>>>>> patches needed its hash to be updated to sha256. Just pushed that change.
>>>>>>>>>>>>>> The build finishes successfully on my end. Pull that change into
>>>>>>>>>>>>>> couverture-build branch and try it again. I'm not seeing any automake stuff
>>>>>>>>>>>>>> here, so just check that and let me know.
>>>>>>>>>>>>>>
>>>>>>>>>>>>>> On 16 March 2018 at 14:59, Vijay Kumar Banerjee <
>>>>>>>>>>>>>> vijaykumar9597 at gmail.com> wrote:
>>>>>>>>>>>>>>
>>>>>>>>>>>>>>> building couverture-qemu from rtems-source-builder (
>>>>>>>>>>>>>>> https://github.com/cillianodonnell/rtems-source-builder/tr
>>>>>>>>>>>>>>> ee/couverture-build )
>>>>>>>>>>>>>>> gives error building auromake-1.12.6-x86_64-linux-gnu-1 .
>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>> I have attached the error report .
>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>> -- vijay
>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>> On 15 March 2018 at 19:17, Cillian O'Donnell <
>>>>>>>>>>>>>>> cpodonnell8 at gmail.com> wrote:
>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>> On 15 March 2018 at 12:26, Vijay Kumar Banerjee <
>>>>>>>>>>>>>>>> vijaykumar9597 at gmail.com> wrote:
>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>> It runs with a bunch of errors . I have attached the log
>>>>>>>>>>>>>>>>> file
>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>> Ok, I'm guessing you didn't set up Couverture-Qemu (special
>>>>>>>>>>>>>>>> version of qemu designed for generating extra trace data for coverage
>>>>>>>>>>>>>>>> analysis). That's what those errors are about. I have an RSB build for that.
>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>> https://github.com/cillianodon
>>>>>>>>>>>>>>>> nell/rtems-source-builder/tree/couverture-build
>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>> and the instructions for building it are
>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>> https://devel.rtems.org/wiki/G
>>>>>>>>>>>>>>>> SoC/2017/coveragetools#BuildingCouverture-QemuwiththeRSB
>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>> I know what the other problem is too. I have a specific
>>>>>>>>>>>>>>>> environment variable defined for the path, sorry I can't even remember
>>>>>>>>>>>>>>>> putting it there, I thought that was automatically generated (probably
>>>>>>>>>>>>>>>> should be, another thing to add to the list :)... ). So wherever you stuck
>>>>>>>>>>>>>>>> the export path for where the rsb built the tools, in .bashrc or whatever
>>>>>>>>>>>>>>>> you're using. Also put something like:
>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>> export PATH=$HOME/development/rtems/5
>>>>>>>>>>>>>>>> /bin:$PATH
>>>>>>>>>>>>>>>> export PATH=$HOME/development/rtems/t
>>>>>>>>>>>>>>>> est/rtems-tools/build/tester/covoar:$PATH
>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>> or you could just copy covoar into the /bin directory with
>>>>>>>>>>>>>>>> all the other rsb tools gcc and all that, it'll find it either way.
>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>> -- vijay
>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>> On 15 March 2018 at 16:58, Cillian O'Donnell <
>>>>>>>>>>>>>>>>> cpodonnell8 at gmail.com> wrote:
>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>> Looks good. If you run the samples without coverage is
>>>>>>>>>>>>>>>>>> everything ok?
>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>> So removing --coverage and tacking on /samples
>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>> $HOME/development/rtems-tools/tester/rtems-test
>>>>>>>>>>>>>>>>>> --rtems-bsp=leon3-qemu --log=log-leon3.log --rtems-tools=$HOME/development/rtems/5
>>>>>>>>>>>>>>>>>> --rtems-builddir=$HOME/development/rtems/kernel/leon3
>>>>>>>>>>>>>>>>>> sparc-rtems5/c/leon3/testsuites/samples
>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>> Do the tests run?
>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>> On 15 March 2018 at 10:53, Vijay Kumar Banerjee <
>>>>>>>>>>>>>>>>>> vijaykumar9597 at gmail.com> wrote:
>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>> I have attached the output of the ls of that directory
>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>> -- vijay
>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>> On 15 March 2018 at 15:52, Cillian O'Donnell <
>>>>>>>>>>>>>>>>>>> cpodonnell8 at gmail.com> wrote:
>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>> On 15 March 2018 at 03:58, Vijay Kumar Banerjee <
>>>>>>>>>>>>>>>>>>>> vijaykumar9597 at gmail.com> wrote:
>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>> hello ,
>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>> as told by Joel , I started this thread to further
>>>>>>>>>>>>>>>>>>>>> discuss the coverage analysis toolset .
>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>> Current status is , I'm trying to builld and run
>>>>>>>>>>>>>>>>>>>>> rtems-test from the coverage-merge branch of the previous GSoC student
>>>>>>>>>>>>>>>>>>>>> Cillian .
>>>>>>>>>>>>>>>>>>>>> https://github.com/cillianodon
>>>>>>>>>>>>>>>>>>>>> nell/rtems-tools/tree/coverage-merge
>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>> I'm getting an error that says .
>>>>>>>>>>>>>>>>>>>>>  "Covoar not found !"
>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>> It's supposed to find it in
>>>>>>>>>>>>>>>>>>>> rtems-tools/build/tester/covoar/ If it's in there it
>>>>>>>>>>>>>>>>>>>> should be fine. Can you show me the contents of that directory?
>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>> cpod at cpod ~/development/rtems/test/rtems-tools/build/tester/covoar
>>>>>>>>>>>>>>>>>>>> $ ls
>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>> the Covoar appeared in rtems-tools/tester/covoar .
>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>> --
>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>> Vijay
>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>> _______________________________________________
>>>>>>>>>>>>>>>>>>>>> devel mailing list
>>>>>>>>>>>>>>>>>>>>> devel at rtems.org
>>>>>>>>>>>>>>>>>>>>> http://lists.rtems.org/mailman/listinfo/devel
>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>
>>>>>>>>>>>>>
>>>>>>>>>>>>
>>>>>>>>>
>>>>>>>>
>>>>>>>
>>>>>>
>>>>>
>>>>
>>>
>>
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.rtems.org/pipermail/devel/attachments/20180321/6f6ac2a6/attachment-0001.html>

More information about the devel mailing list