error while running rtems-test with coverage

Vijay Kumar Banerjee vijaykumar9597 at gmail.com
Fri Apr 13 17:51:25 UTC 2018 

can we set it in [coverage] ?

-- vijay

On 13 April 2018 at 23:15, Cillian O'Donnell <cpodonnell8 at gmail.com> wrote:

> Alright so we're back to ini update stuff. That error is because in the
> old leon3-qemu.mc we had this global variable set. So you need to find
> out how to get that global variable set in the new leon3-qemu.ini
>
> Heres the old version from the .MC file
>
> [global]
> bsp:                      none,    none,     'leon3'
> coverage_supported:       none,    none,     '1'
>
>
> On Fri, 13 Apr 2018, 18:29 Vijay Kumar Banerjee, <vijaykumar9597 at gmail.com>
> wrote:
>
>> It shows the following error
>>
>> .......
>> RTEMS Testing - Tester, 5 (58acb6d61ac7 modified)
>> Coverage environment prepared
>> [ 3/13] p:0  f:0  u:0  e:0  I:0  B:0  t:0  i:0  W:0  | sparc/leon3_qemu:
>> cdtest.exe
>> [ 1/13] p:0  f:0  u:0  e:0  I:0  B:0  t:0  i:0  W:0  | sparc/leon3_qemu:
>> base_sp.exe
>> [ 2/13] p:0  f:0  u:0  e:0  I:0  B:0  t:0  i:0  W:0  | sparc/leon3_qemu:
>> capture.exe
>> [ 4/13] p:0  f:0  u:0  e:0  I:0  B:0  t:0  i:0  W:0  | sparc/leon3_qemu:
>> fileio.exe
>> error: qemu.cfg:77: macro '%{coverage_supported}' not found
>> warning: switched to dry run due to errors
>> error: qemu.cfg:77: invalid if bool value:  %if %{coverage_supported}
>> error: qemu.cfg:77: macro '%{coverage_supported}' not found
>> warning: switched to dry run due to errors
>> error: qemu.cfg:77: invalid if bool value:  %if %{coverage_supported}
>> Traceback (most recent call last):
>>   File "/home/lunatic/development/rtems/test/rtems-tools/tester/rtems-test",
>> line 40, in <module>
>>     rt.test.run()
>>   File "/home/lunatic/development/rtems/test/rtems-tools/tester/rt/test.py",
>> line 353, in run
>>     tst.reraise()
>>   File "/home/lunatic/development/rtems/test/rtems-tools/tester/rt/test.py",
>> line 160, in reraise
>>     _test_reraise(*self.result)
>> NameError: global name '_test_reraise' is not defined
>> error: qemu.cfg:77: macro '%{coverage_supported}' not found
>> warning: switched to dry run due to errors
>> error: qemu.cfg:77: invalid if bool value:  %if %{coverage_supported}
>> error: qemu.cfg:77: macro '%{coverage_supported}' not found
>> warning: switched to dry run due to errors
>> error: qemu.cfg:77: invalid if bool value:  %if %{coverage_supported}
>>
>>
>>
>> -- vijay
>>
>> On 13 April 2018 at 22:54, Cillian O'Donnell <cpodonnell8 at gmail.com>
>> wrote:
>>
>>> Ah yes, your missing the qemu arguments to run the coverage. Try this
>>>
>>> #
>>> # Coverage analysis
>>> #
>>> %define coverage_arg %{nil}
>>> %if %{defined _coverage}
>>>     %if %{coverage_supported}
>>>         %define coverage_arg -exec-trace %{test_executable}.cov
>>>     %else
>>>         %error "Coverage analysis unsupported for %{bsp}"
>>>     %endif
>>> %endif
>>>
>>> #
>>> # Executable
>>> #
>>> %execute %{qemu_cmd} %{qemu_opts_base} %{qemu_opts} -kernel %{test_executable} %{coverage_arg}
>>>
>>>
>>> On Fri, 13 Apr 2018, 18:18 Vijay Kumar Banerjee, <
>>> vijaykumar9597 at gmail.com> wrote:
>>>
>>>> sure
>>>>
>>>> .........
>>>>
>>>> #
>>>> # RTEMS Tools Project (http://www.rtems.org/)
>>>> # Copyright 2010-2014 Chris Johns (chrisj at rtems.org)
>>>> # All rights reserved.
>>>> #
>>>> # This file is part of the RTEMS Tools package in 'rtems-tools'.
>>>> #
>>>> # Redistribution and use in source and binary forms, with or without
>>>> # modification, are permitted provided that the following conditions
>>>> are met:
>>>> #
>>>> # 1. Redistributions of source code must retain the above copyright
>>>> notice,
>>>> # this list of conditions and the following disclaimer.
>>>> #
>>>> # 2. Redistributions in binary form must reproduce the above copyright
>>>> notice,
>>>> # this list of conditions and the following disclaimer in the
>>>> documentation
>>>> # and/or other materials provided with the distribution.
>>>> #
>>>> # THIS SOFTWARE IS PROVIDED BY THE COPYRIGHT HOLDERS AND CONTRIBUTORS
>>>> "AS IS"
>>>> # AND ANY EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT LIMITED TO,
>>>> THE
>>>> # IMPLIED WARRANTIES OF MERCHANTABILITY AND FITNESS FOR A PARTICULAR
>>>> PURPOSE
>>>> # ARE DISCLAIMED. IN NO EVENT SHALL THE COPYRIGHT HOLDER OR
>>>> CONTRIBUTORS BE
>>>> # LIABLE FOR ANY DIRECT, INDIRECT, INCIDENTAL, SPECIAL, EXEMPLARY, OR
>>>> # CONSEQUENTIAL DAMAGES (INCLUDING, BUT NOT LIMITED TO, PROCUREMENT OF
>>>> # SUBSTITUTE GOODS OR SERVICES; LOSS OF USE, DATA, OR PROFITS; OR
>>>> BUSINESS
>>>> # INTERRUPTION) HOWEVER CAUSED AND ON ANY THEORY OF LIABILITY, WHETHER
>>>> IN
>>>> # CONTRACT, STRICT LIABILITY, OR TORT (INCLUDING NEGLIGENCE OR
>>>> OTHERWISE)
>>>> # ARISING IN ANY WAY OUT OF THE USE OF THIS SOFTWARE, EVEN IF ADVISED
>>>> OF THE
>>>> # POSSIBILITY OF SUCH DAMAGE.
>>>> #
>>>>
>>>> #
>>>> # QEMU
>>>> #
>>>> # Use a qemu command to run the executable in the qemu simulator.
>>>> #
>>>>
>>>> %include %{_configdir}/base.cfg
>>>> %include %{_configdir}/checks.cfg
>>>>
>>>> #
>>>> # Console.
>>>> #
>>>> %define console_stdio
>>>> %include %{_configdir}/console.cfg
>>>>
>>>> #
>>>> # RTEMS version
>>>> #
>>>> %include %{_rtdir}/rtems/version.cfg
>>>>
>>>> #
>>>> # Qemu common option patterns.
>>>> #
>>>> #%define qemu_opts_base   -no-reboot -monitor none -serial stdio
>>>> -nographic
>>>> #%define qemu_opts_base   -no-reboot -serial null -serial mon:stdio
>>>> -nographic
>>>> %define qemu_opts_base  -no-reboot -monitor null -serial stdio
>>>> -nographic
>>>> %define qemu_opts_no_net -net none
>>>> #
>>>> # Qemu executable
>>>> #
>>>> %ifn %{defined bsp_qemu_opts}
>>>>  %define bsp_qemu_opts %{nil}
>>>> %endif
>>>> %define qemu_cmd  qemu-system-%{bsp_arch}
>>>> %define qemu_opts %{bsp_qemu_opts}
>>>>
>>>> #
>>>> # Executable
>>>> #
>>>> %execute %{qemu_cmd} %{qemu_opts} -kernel %{test_executable}
>>>>
>>>> -- vijay
>>>>
>>>> On 13 April 2018 at 22:19, Cillian O'Donnell <cpodonnell8 at gmail.com>
>>>> wrote:
>>>>
>>>>> Yeah that's fine but can you just paste the contents here anyway. I'm
>>>>> not near a laptop and I can't remember which branch your starting from.
>>>>>
>>>>> On Fri, 13 Apr 2018, 13:51 Vijay Kumar Banerjee, <
>>>>> vijaykumar9597 at gmail.com> wrote:
>>>>>
>>>>>> I haven't made any local changes to qemu.cfg
>>>>>>
>>>>>> -- vijay
>>>>>>
>>>>>> On 13 April 2018 at 11:43, Cillian O'Donnell <cpodonnell8 at gmail.com>
>>>>>> wrote:
>>>>>>
>>>>>>> No that's the right one. I think it might be something to do with
>>>>>>> qemu.cfg in rtems-tools then. What's in your qemu.cfg? It's in
>>>>>>> rtems-tools/tester/rtems/testing
>>>>>>>
>>>>>>> On Fri, 13 Apr 2018, 06:39 Vijay Kumar Banerjee, <
>>>>>>> vijaykumar9597 at gmail.com> wrote:
>>>>>>>
>>>>>>>> I manually built qemu from here
>>>>>>>>
>>>>>>>> https://github.com/AdaCore/qemu
>>>>>>>>
>>>>>>>> I guess this is regular qemu , please send a link for couverture
>>>>>>>> qemu for manual build.
>>>>>>>>
>>>>>>>> -- vijay
>>>>>>>>
>>>>>>>> On 12 April 2018 at 22:23, Cillian O'Donnell <cpodonnell8 at gmail.com
>>>>>>>> > wrote:
>>>>>>>>
>>>>>>>>> Ah that's good progress, those .cov files are supposed to be
>>>>>>>>> generated by couverture-qemu as the test are run. They contain the trace
>>>>>>>>> data. Are you definitely running couverture-qemu and not regular qemu?
>>>>>>>>>
>>>>>>>>> On Thu, 12 Apr 2018, 17:41 Vijay Kumar Banerjee, <
>>>>>>>>> vijaykumar9597 at gmail.com> wrote:
>>>>>>>>>
>>>>>>>>>> I got the parsing of the coverage from the ini file running , now
>>>>>>>>>> I'm getting the following error and warnings .
>>>>>>>>>>
>>>>>>>>>>
>>>>>>>>>> ....
>>>>>>>>>>
>>>>>>>>>> Running covoar for score
>>>>>>>>>> WARNING: Unable to read coverage file /home/lunatic/development/
>>>>>>>>>> rtems/kernel/leon3/sparc-rtems5/c/leon3/testsuites/
>>>>>>>>>> samples/base_sp/base_sp.exe.cov
>>>>>>>>>> WARNING: Unable to read coverage file /home/lunatic/development/
>>>>>>>>>> rtems/kernel/leon3/sparc-rtems5/c/leon3/testsuites/
>>>>>>>>>> samples/capture/capture.exe.cov
>>>>>>>>>> WARNING: Unable to read coverage file /home/lunatic/development/
>>>>>>>>>> rtems/kernel/leon3/sparc-rtems5/c/leon3/testsuites/
>>>>>>>>>> samples/cdtest/cdtest.exe.cov
>>>>>>>>>> WARNING: Unable to read coverage file /home/lunatic/development/
>>>>>>>>>> rtems/kernel/leon3/sparc-rtems5/c/leon3/testsuites/
>>>>>>>>>> samples/fileio/fileio.exe.cov
>>>>>>>>>> WARNING: Unable to read coverage file /home/lunatic/development/
>>>>>>>>>> rtems/kernel/leon3/sparc-rtems5/c/leon3/testsuites/
>>>>>>>>>> samples/hello/hello.exe.cov
>>>>>>>>>> WARNING: Unable to read coverage file /home/lunatic/development/
>>>>>>>>>> rtems/kernel/leon3/sparc-rtems5/c/leon3/testsuites/
>>>>>>>>>> samples/iostream/cxx_iostream.exe.cov
>>>>>>>>>> WARNING: Unable to read coverage file /home/lunatic/development/
>>>>>>>>>> rtems/kernel/leon3/sparc-rtems5/c/leon3/testsuites/
>>>>>>>>>> samples/loopback/loopback.exe.cov
>>>>>>>>>> WARNING: Unable to read coverage file /home/lunatic/development/
>>>>>>>>>> rtems/kernel/leon3/sparc-rtems5/c/leon3/testsuites/
>>>>>>>>>> samples/minimum/minimum.exe.cov
>>>>>>>>>> WARNING: Unable to read coverage file /home/lunatic/development/
>>>>>>>>>> rtems/kernel/leon3/sparc-rtems5/c/leon3/testsuites/
>>>>>>>>>> samples/nsecs/nsecs.exe.cov
>>>>>>>>>> WARNING: Unable to read coverage file /home/lunatic/development/
>>>>>>>>>> rtems/kernel/leon3/sparc-rtems5/c/leon3/testsuites/
>>>>>>>>>> samples/paranoia/paranoia.exe.cov
>>>>>>>>>> WARNING: Unable to read coverage file /home/lunatic/development/
>>>>>>>>>> rtems/kernel/leon3/sparc-rtems5/c/leon3/testsuites/
>>>>>>>>>> samples/pppd/pppd.exe.cov
>>>>>>>>>> WARNING: Unable to read coverage file /home/lunatic/development/
>>>>>>>>>> rtems/kernel/leon3/sparc-rtems5/c/leon3/testsuites/
>>>>>>>>>> samples/ticker/ticker.exe.cov
>>>>>>>>>> WARNING: Unable to read coverage file /home/lunatic/development/
>>>>>>>>>> rtems/kernel/leon3/sparc-rtems5/c/leon3/testsuites/
>>>>>>>>>> samples/unlimited/unlimited.exe.cov
>>>>>>>>>> ERROR: No information to analyze
>>>>>>>>>> ***Cleaning tempfiles***
>>>>>>>>>> error: covoar failure exit code: 1
>>>>>>>>>>
>>>>>>>>>>
>>>>>>>>>>
>>>>>>>>>>
>>>>>>>>>> -- vijay
>>>>>>>>>>
>>>>>>>>>> On 10 April 2018 at 18:10, Vijay Kumar Banerjee <
>>>>>>>>>> vijaykumar9597 at gmail.com> wrote:
>>>>>>>>>>
>>>>>>>>>>> I managed to get the tests running with the following two
>>>>>>>>>>> patches .
>>>>>>>>>>> Please review them .
>>>>>>>>>>>
>>>>>>>>>>> -- vijay
>>>>>>>>>>>
>>>>>>>>>>> On 9 April 2018 at 15:28, Vijay Kumar Banerjee <
>>>>>>>>>>> vijaykumar9597 at gmail.com> wrote:
>>>>>>>>>>>
>>>>>>>>>>>>
>>>>>>>>>>>>
>>>>>>>>>>>> On Fri, 6 Apr 2018, 02:24 Vijay Kumar Banerjee, <
>>>>>>>>>>>> vijaykumar9597 at gmail.com> wrote:
>>>>>>>>>>>>
>>>>>>>>>>>>> I tried to solve the empty get in the way shown in the patch ,
>>>>>>>>>>>>> but it doesn't seems to work, I'm getting the following error now
>>>>>>>>>>>>>
>>>>>>>>>>>>> RTEMS Testing - Tester, 5 (1ed6e2e64090)
>>>>>>>>>>>>> Traceback (most recent call last):
>>>>>>>>>>>>>   File "/home/lunatic/development/rtems/rtems-tools/tester/rtems-test",
>>>>>>>>>>>>> line 40, in <module>
>>>>>>>>>>>>>     rt.test.run()
>>>>>>>>>>>>>   File "/home/lunatic/development/rtems/rtems-tools/tester/rt/test.py",
>>>>>>>>>>>>> line 303, in run
>>>>>>>>>>>>>     bsp = config.load(bsp[1], opts)
>>>>>>>>>>>>>   File "/home/lunatic/development/rtems/rtems-tools/tester/rt/config.py",
>>>>>>>>>>>>> line 461, in load
>>>>>>>>>>>>>     return opts.defaults['bsp']
>>>>>>>>>>>>>   File "/home/lunatic/development/rtems/rtems-tools/rtemstoolkit/macros.py",
>>>>>>>>>>>>> line 190, in __getitem__
>>>>>>>>>>>>>     raise IndexError('key: %s' % (key))
>>>>>>>>>>>>> IndexError: key: bsp
>>>>>>>>>>>>>
>>>>>>>>>>>>> I still Couldn't figure out why it's happening, when I use
>>>>>>>>>>>> RawConfigParse , it seems to run nicely though .
>>>>>>>>>>>>
>>>>>>>>>>>> Any Ideas on solving it without using RawConfigParser ?
>>>>>>>>>>>>
>>>>>>>>>>>>
>>>>>>>>>>>>> -- vijay
>>>>>>>>>>>>>
>>>>>>>>>>>>> On 5 April 2018 at 11:51, Cillian O'Donnell <
>>>>>>>>>>>>> cpodonnell8 at gmail.com> wrote:
>>>>>>>>>>>>>
>>>>>>>>>>>>>>
>>>>>>>>>>>>>>
>>>>>>>>>>>>>> On Wed, 4 Apr 2018, 11:18 Vijay Kumar Banerjee, <
>>>>>>>>>>>>>> vijaykumar9597 at gmail.com> wrote:
>>>>>>>>>>>>>>
>>>>>>>>>>>>>>> I got the rtems-test running
>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>> I found that get_item_names(section, err) in
>>>>>>>>>>>>>>> configuration.py  was returning an empty list to the
>>>>>>>>>>>>>>> item_names in load(bsp,opts) for config.py this was giving
>>>>>>>>>>>>>>> out the error mandatory item not found .
>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>> I have attached the patch file .
>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>> current status :
>>>>>>>>>>>>>>> rtems-test is running without coverage
>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>> running coverage gives the following error :
>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>> RTEMS Testing - Tester, 5 (092c569e81ae)
>>>>>>>>>>>>>>> Traceback (most recent call last):
>>>>>>>>>>>>>>>   File "/home/lunatic/development/rtems/rtems-tools/tester/rtems-test",
>>>>>>>>>>>>>>> line 40, in <module>
>>>>>>>>>>>>>>>     rt.test.run()
>>>>>>>>>>>>>>>   File "/home/lunatic/development/
>>>>>>>>>>>>>>> rtems/rtems-tools/tester/rt/test.py", line 310, in run
>>>>>>>>>>>>>>>     coverage = coverage_get_obj(opts, path_to_builddir[1])
>>>>>>>>>>>>>>>   File "/home/lunatic/development/
>>>>>>>>>>>>>>> rtems/rtems-tools/tester/rt/test.py", line 230, in
>>>>>>>>>>>>>>> coverage_get_obj
>>>>>>>>>>>>>>>     coverage_obj = coverage.coverage_run(opts.defaults,
>>>>>>>>>>>>>>> path_to_builddir)
>>>>>>>>>>>>>>>   File "/home/lunatic/development/
>>>>>>>>>>>>>>> rtems/rtems-tools/tester/rt/coverage.py", line 335, in
>>>>>>>>>>>>>>> __init__
>>>>>>>>>>>>>>>     self.report_format = self.config_map['report_format'][2]
>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>
>>>>>>>>>>>>>> This is back to the difference in cfg formats. It's looking
>>>>>>>>>>>>>> for the last line here
>>>>>>>>>>>>>>
>>>>>>>>>>>>>> https://github.com/cillianodonnell/rtems-tools/
>>>>>>>>>>>>>> blob/coverage/tester/rtems/testing/bsps/leon3-qemu.mc
>>>>>>>>>>>>>>
>>>>>>>>>>>>>> Which is now
>>>>>>>>>>>>>>
>>>>>>>>>>>>>> https://github.com/cillianodonnell/rtems-tools/
>>>>>>>>>>>>>> blob/ini-update/tester/rtems/testing/bsps/leon3_qemu.ini
>>>>>>>>>>>>>>
>>>>>>>>>>>>>> It's trying to grab the 'html' text so it can trigger the
>>>>>>>>>>>>>> right section of code in coverage.py
>>>>>>>>>>>>>>
>>>>>>>>>>>>>> It's something just to tell which format we want the report
>>>>>>>>>>>>>> to output
>>>>>>>>>>>>>>
>>>>>>>>>>>>>
>>>>>>>>>>>>> Understood , I'll come to that after I manage to get the test
>>>>>>>>>>>>> running first .
>>>>>>>>>>>>>
>>>>>>>>>>>>>>
>>>>>>>>>>>>>> TypeError: 'NoneType' object has no attribute '__getitem__'
>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>> Thanks
>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>> -- vijay
>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>> On 4 April 2018 at 11:16, Chris Johns <chrisj at rtems.org>
>>>>>>>>>>>>>>> wrote:
>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>> On 04/04/2018 06:36, Vijay Kumar Banerjee wrote:
>>>>>>>>>>>>>>>> > after pulling the ini-update branch from
>>>>>>>>>>>>>>>> > https://github.com/cillianodonnell/rtems-tools/
>>>>>>>>>>>>>>>> tree/ini-update
>>>>>>>>>>>>>>>> >
>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>> Any local changes?
>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>> > I tried running the rtems-test with the following command
>>>>>>>>>>>>>>>> >
>>>>>>>>>>>>>>>> > $HOME/development/rtems/rtems-tools/tester/rtems-test
>>>>>>>>>>>>>>>> > --rtems-tools=$HOME/development/rtems/5
>>>>>>>>>>>>>>>> --log=coverage_analysis.log
>>>>>>>>>>>>>>>> > --rtems-bsp=leon3_qemu --rtems-builddir=$HOME/
>>>>>>>>>>>>>>>> development/rtems/kernel/leon3
>>>>>>>>>>>>>>>> > sparc-rtems5/c/leon3/testsuites/samples
>>>>>>>>>>>>>>>> >
>>>>>>>>>>>>>>>> > was getting the KeyError for 'coverage'
>>>>>>>>>>>>>>>> >
>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>> Please paste the command and output as it appears in the
>>>>>>>>>>>>>>>> terminal?
>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>> > current status :
>>>>>>>>>>>>>>>> > the same command gives the error :
>>>>>>>>>>>>>>>> > "mandatory item not found in bsp section : bsp,"
>>>>>>>>>>>>>>>> > to make sure I didn't break something , I also tried
>>>>>>>>>>>>>>>> running from a freshly
>>>>>>>>>>>>>>>> > downloaded directory as well, it gives the same error.
>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>> Did you try running the following command from the top of
>>>>>>>>>>>>>>>> the source tree:
>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>  $ grep -r "mandatory item not found" *
>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>> ?
>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>> Chris
>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>
>>>>>>>>>>>>>
>>>>>>>>>>>
>>>>>>>>>>
>>>>>>>>
>>>>>>
>>>>
>>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.rtems.org/pipermail/devel/attachments/20180413/2e92c1ef/attachment-0002.html>

More information about the devel mailing list