Improve coverage analysis toolset

Sat Mar 24 16:07:20 UTC 2018

hello mentors and other developers of the community , there are only three
days remaining to the GSoC proposal submission deadline.

I request the mentors and everyone in the community who has experience in
RTEMS, to please go through my proposal for Coverage Analysis tool
improvement project , if they have time , and suggest any kind of
improvements and changes .
here is the link to the proposal
https://docs.google.com/document/d/1UjCWE1ojrpDQLQ2fQqxHU0SCrLETlerxD6t0SOcQNLQ/edit?usp=sharing
Thank you

-- vijay

On 22 March 2018 at 04:30, Joel Sherrill <joel at rtems.org> wrote:

> Glad you have tests running. Hopefully as Cillian suggested, you can fix
> the
> remaining issues to see coverage.
>
> We certainly need to get this merged if Chris is OK with it and it doesn't
> break anything else.
>
> I am reviewing proposals now. Seems to be a queue. :)
>
> --joel
>
> On Wed, Mar 21, 2018 at 4:42 AM, Vijay Kumar Banerjee <
> vijaykumar9597 at gmail.com> wrote:
>
>> Sir ,
>> I have done the changes in the proposal , based on the comments in the
>> google doc , please review it and suggest any further changes if required
>>
>> Thank you ,
>> -- vijay
>>
>> P.S : the previous version is in parentheses , I will remove them after
>> you review the changes .
>>
>> On 18 March 2018 at 16:05, Vijay Kumar Banerjee <vijaykumar9597 at gmail.com
>> > wrote:
>>
>>> Thanks Cillian :)
>>>
>>> It's great to see it running ,  I'll do some background reading about
>>> RSB, and RTEMS-Tools along with the covoar code . And also work on my
>>> python skills.
>>>
>>> to try rtems-test on a bsp that runs gdb , I tried it on erc32 and that
>>> also worked.
>>>
>>> I'm also waiting for Joel and other mentors' review on my draft
>>> proposal, so that I can also work on it and make any changes if needed.
>>>
>>> Thanks .
>>>
>>> -- vijay
>>>
>>> On 18 March 2018 at 14:17, Cillian O'Donnell <cpodonnell8 at gmail.com>
>>> wrote:
>>>
>>>>
>>>>
>>>> On 18 March 2018 at 06:47, Vijay Kumar Banerjee <
>>>> vijaykumar9597 at gmail.com> wrote:
>>>>
>>>>> It worked !
>>>>>
>>>>
>>>> That's great!  That's the good news then. I was using an old leon3
>>>> build and maybe some older Qemu too and I think that's why I didn't see any
>>>> issues initially. I was hoping you could see the coverage running and the
>>>> reports generated from that but it looks like the full update to the
>>>> current master will be necessary to have a look. So until the parsing for
>>>> the INI files is added to the coverage.py we won't see the coverage
>>>> running. Unfortunately, I'm in the middle of exams until the following
>>>> Monday so I won't be able to sink any more time into it until then. You can
>>>> still figure out the RSB problem you're having and do some background
>>>> reading, brush up on your Python skills, have a read of the covoar code
>>>>
>>>> https://github.com/RTEMS/rtems-tools/blob/master/tester/covo
>>>> ar/covoar.cc
>>>>
>>>> just skim through, read the comments, get a sense of the what it's
>>>> doing and in what order.
>>>>
>>>>
>>>>> It's great to see it running ! I have attached the result .
>>>>>
>>>>> -- vijay
>>>>>
>>>>> On 18 March 2018 at 02:31, Cillian O'Donnell <cpodonnell8 at gmail.com>
>>>>> wrote:
>>>>>
>>>>>>
>>>>>>
>>>>>> On 17 March 2018 at 20:08, Vijay Kumar Banerjee <
>>>>>> vijaykumar9597 at gmail.com> wrote:
>>>>>>
>>>>>>> yes it prints hello world
>>>>>>>
>>>>>>
>>>>>> Alright I've added an .ini for leon3-qemu to the current rtems-tools.
>>>>>> Pull this branch
>>>>>>
>>>>>> https://github.com/cillianodonnell/rtems-tools/tree/ini-update
>>>>>>
>>>>>> and try
>>>>>>
>>>>>> $HOME/development/rtems/test/rtems-tools/tester/rtems-test
>>>>>> --rtems-tools=$HOME/development/rtems/5 --log=coverage-analysis.log
>>>>>> --rtems-bsp=leon3_qemu $HOME/development/rtems/leon3/
>>>>>> sparc-rtems5/c/leon3/testsuites/samples
>>>>>>
>>>>>>>
>>>>>>> -- vijay
>>>>>>>
>>>>>>> On 18 March 2018 at 01:31, Cillian O'Donnell <cpodonnell8 at gmail.com>
>>>>>>> wrote:
>>>>>>>
>>>>>>>> If you run just one test by itself without rtems-test
>>>>>>>>
>>>>>>>> qemu-system-sparc -no-reboot -monitor null -serial stdio -nographic
>>>>>>>> -M leon3_generic -kernel $HOME/development/rtems/leon3/
>>>>>>>> sparc-rtems5/c/leon3/testsuites/samples/hello/hello.exe
>>>>>>>>
>>>>>>>> Does the hello world print out?
>>>>>>>>
>>>>>>>> On 17 March 2018 at 14:46, Vijay Kumar Banerjee <
>>>>>>>> vijaykumar9597 at gmail.com> wrote:
>>>>>>>>
>>>>>>>>> I built it manually
>>>>>>>>>
>>>>>>>>> the environment variable PATH looks like this
>>>>>>>>> /home/lunatic/qemu/install/bin:/home/lunatic/development/rte
>>>>>>>>> ms/5/bin:/usr/local/bin:/usr/local/sbin:/usr/bin:/usr/sbin:/
>>>>>>>>> home/lunatic/.local/bin:/home/lunatic/bin
>>>>>>>>>
>>>>>>>>> I tried to run rtems-test again without --coverage , it gives the
>>>>>>>>> same result
>>>>>>>>> I have attached the log.
>>>>>>>>>
>>>>>>>>> -- vijay
>>>>>>>>>
>>>>>>>>> On 17 March 2018 at 00:55, Cillian O'Donnell <
>>>>>>>>> cpodonnell8 at gmail.com> wrote:
>>>>>>>>>
>>>>>>>>>> Yes this is something more than my build, I'll need someone a bit
>>>>>>>>>> more expert in the RSB to step in there. In the meantime, lets just build
>>>>>>>>>> couverture-qemu manually so we can see is everything else working.
>>>>>>>>>>
>>>>>>>>>> git clone https://github.com/AdaCore/qemu
>>>>>>>>>>
>>>>>>>>>> cd qemu
>>>>>>>>>>
>>>>>>>>>> ./configure --target-list=sparc-softmmu
>>>>>>>>>> --prefix=$HOME/qemu/install --disable-docs --disable-virtfs --disable-werror
>>>>>>>>>>
>>>>>>>>>> make
>>>>>>>>>>
>>>>>>>>>> make install
>>>>>>>>>>
>>>>>>>>>> then add the prefix to $PATH in .bashrc as well like before.
>>>>>>>>>>
>>>>>>>>>> export PATH=$HOME/qemu/install/bin:$PATH
>>>>>>>>>>
>>>>>>>>>> Then run rtem-test and see what happens
>>>>>>>>>>
>>>>>>>>>>
>>>>>>>>>> On 16 March 2018 at 19:13, Vijay Kumar Banerjee <
>>>>>>>>>> vijaykumar9597 at gmail.com> wrote:
>>>>>>>>>>
>>>>>>>>>>> the same error comes when I try to build qemu from the
>>>>>>>>>>> RTEMS/rtems-source-builder as well
>>>>>>>>>>>
>>>>>>>>>>> is the issue coming from my system ? I'm using fedora 27 64bit
>>>>>>>>>>>
>>>>>>>>>>> On 17 Mar 2018 12:39 a.m., "Vijay Kumar Banerjee" <
>>>>>>>>>>> vijaykumar9597 at gmail.com> wrote:
>>>>>>>>>>>
>>>>>>>>>>>> yes , the same thing happens
>>>>>>>>>>>>
>>>>>>>>>>>> On 17 Mar 2018 12:13 a.m., "Cillian O'Donnell" <
>>>>>>>>>>>> cpodonnell8 at gmail.com> wrote:
>>>>>>>>>>>>
>>>>>>>>>>>>> If you build regular qemu with the RSB, does the same thing
>>>>>>>>>>>>> happen?
>>>>>>>>>>>>>
>>>>>>>>>>>>> Try
>>>>>>>>>>>>>
>>>>>>>>>>>>> ../source-builder/sb-set-builder --log=qemu_log.txt
>>>>>>>>>>>>> --prefix=$HOME/development/5 devel/qemu
>>>>>>>>>>>>>
>>>>>>>>>>>>>
>>>>>>>>>>>>> On 16 March 2018 at 16:48, Vijay Kumar Banerjee <
>>>>>>>>>>>>> vijaykumar9597 at gmail.com> wrote:
>>>>>>>>>>>>>
>>>>>>>>>>>>>> still the same error
>>>>>>>>>>>>>>
>>>>>>>>>>>>>> -- vijay
>>>>>>>>>>>>>>
>>>>>>>>>>>>>> On 16 March 2018 at 21:39, Cillian O'Donnell <
>>>>>>>>>>>>>> cpodonnell8 at gmail.com> wrote:
>>>>>>>>>>>>>>
>>>>>>>>>>>>>>> Just checked and the build was failing because one of the
>>>>>>>>>>>>>>> patches needed its hash to be updated to sha256. Just pushed that change.
>>>>>>>>>>>>>>> The build finishes successfully on my end. Pull that change into
>>>>>>>>>>>>>>> couverture-build branch and try it again. I'm not seeing any automake stuff
>>>>>>>>>>>>>>> here, so just check that and let me know.
>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>> On 16 March 2018 at 14:59, Vijay Kumar Banerjee <
>>>>>>>>>>>>>>> vijaykumar9597 at gmail.com> wrote:
>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>> building couverture-qemu from rtems-source-builder (
>>>>>>>>>>>>>>>> https://github.com/cillianodonnell/rtems-source-builder/tr
>>>>>>>>>>>>>>>> ee/couverture-build )
>>>>>>>>>>>>>>>> gives error building auromake-1.12.6-x86_64-linux-gnu-1 .
>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>> I have attached the error report .
>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>> -- vijay
>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>> On 15 March 2018 at 19:17, Cillian O'Donnell <
>>>>>>>>>>>>>>>> cpodonnell8 at gmail.com> wrote:
>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>> On 15 March 2018 at 12:26, Vijay Kumar Banerjee <
>>>>>>>>>>>>>>>>> vijaykumar9597 at gmail.com> wrote:
>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>> It runs with a bunch of errors . I have attached the log
>>>>>>>>>>>>>>>>>> file
>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>> Ok, I'm guessing you didn't set up Couverture-Qemu
>>>>>>>>>>>>>>>>> (special version of qemu designed for generating extra trace data for
>>>>>>>>>>>>>>>>> coverage analysis). That's what those errors are about. I have an RSB build
>>>>>>>>>>>>>>>>> for that.
>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>> https://github.com/cillianodon
>>>>>>>>>>>>>>>>> nell/rtems-source-builder/tree/couverture-build
>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>> and the instructions for building it are
>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>> https://devel.rtems.org/wiki/G
>>>>>>>>>>>>>>>>> SoC/2017/coveragetools#BuildingCouverture-QemuwiththeRSB
>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>> I know what the other problem is too. I have a specific
>>>>>>>>>>>>>>>>> environment variable defined for the path, sorry I can't even remember
>>>>>>>>>>>>>>>>> putting it there, I thought that was automatically generated (probably
>>>>>>>>>>>>>>>>> should be, another thing to add to the list :)... ). So wherever you stuck
>>>>>>>>>>>>>>>>> the export path for where the rsb built the tools, in .bashrc or whatever
>>>>>>>>>>>>>>>>> you're using. Also put something like:
>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>> export PATH=$HOME/development/rtems/5
>>>>>>>>>>>>>>>>> /bin:$PATH
>>>>>>>>>>>>>>>>> export PATH=$HOME/development/rtems/t
>>>>>>>>>>>>>>>>> est/rtems-tools/build/tester/covoar:$PATH
>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>> or you could just copy covoar into the /bin directory with
>>>>>>>>>>>>>>>>> all the other rsb tools gcc and all that, it'll find it either way.
>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>> -- vijay
>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>> On 15 March 2018 at 16:58, Cillian O'Donnell <
>>>>>>>>>>>>>>>>>> cpodonnell8 at gmail.com> wrote:
>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>> Looks good. If you run the samples without coverage is
>>>>>>>>>>>>>>>>>>> everything ok?
>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>> So removing --coverage and tacking on /samples
>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>> $HOME/development/rtems-tools/tester/rtems-test
>>>>>>>>>>>>>>>>>>> --rtems-bsp=leon3-qemu --log=log-leon3.log --rtems-tools=$HOME/development/rtems/5
>>>>>>>>>>>>>>>>>>> --rtems-builddir=$HOME/development/rtems/kernel/leon3
>>>>>>>>>>>>>>>>>>> sparc-rtems5/c/leon3/testsuites/samples
>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>> Do the tests run?
>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>> On 15 March 2018 at 10:53, Vijay Kumar Banerjee <
>>>>>>>>>>>>>>>>>>> vijaykumar9597 at gmail.com> wrote:
>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>> I have attached the output of the ls of that directory
>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>> -- vijay
>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>> On 15 March 2018 at 15:52, Cillian O'Donnell <
>>>>>>>>>>>>>>>>>>>> cpodonnell8 at gmail.com> wrote:
>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>> On 15 March 2018 at 03:58, Vijay Kumar Banerjee <
>>>>>>>>>>>>>>>>>>>>> vijaykumar9597 at gmail.com> wrote:
>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>> hello ,
>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>> as told by Joel , I started this thread to further
>>>>>>>>>>>>>>>>>>>>>> discuss the coverage analysis toolset .
>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>> Current status is , I'm trying to builld and run
>>>>>>>>>>>>>>>>>>>>>> rtems-test from the coverage-merge branch of the previous GSoC student
>>>>>>>>>>>>>>>>>>>>>> Cillian .
>>>>>>>>>>>>>>>>>>>>>> https://github.com/cillianodon
>>>>>>>>>>>>>>>>>>>>>> nell/rtems-tools/tree/coverage-merge
>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>> I'm getting an error that says .
>>>>>>>>>>>>>>>>>>>>>>  "Covoar not found !"
>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>> It's supposed to find it in
>>>>>>>>>>>>>>>>>>>>> rtems-tools/build/tester/covoar/ If it's in there it
>>>>>>>>>>>>>>>>>>>>> should be fine. Can you show me the contents of that directory?
>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>> cpod at cpod ~/development/rtems/test/rtems-tools/build/tester/covoar
>>>>>>>>>>>>>>>>>>>>> $ ls
>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>> the Covoar appeared in rtems-tools/tester/covoar .
>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>> --
>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>> Vijay
>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>> _______________________________________________
>>>>>>>>>>>>>>>>>>>>>> devel mailing list
>>>>>>>>>>>>>>>>>>>>>> devel at rtems.org
>>>>>>>>>>>>>>>>>>>>>> http://lists.rtems.org/mailman/listinfo/devel
>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>
>>>>>>>>>>>>>
>>>>>>>>>>
>>>>>>>>>
>>>>>>>>
>>>>>>>
>>>>>>
>>>>>
>>>>
>>>
>>
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.rtems.org/pipermail/devel/attachments/20180324/bfd53ecc/attachment-0001.html>