error while running rtems-test with coverage

Cillian O'Donnell cpodonnell8 at gmail.com
Fri Apr 13 17:45:27 UTC 2018 

Alright so we're back to ini update stuff. That error is because in the old
leon3-qemu.mc we had this global variable set. So you need to find out how
to get that global variable set in the new leon3-qemu.ini

Heres the old version from the .MC file

[global]
bsp:                      none,    none,     'leon3'
coverage_supported:       none,    none,     '1'


On Fri, 13 Apr 2018, 18:29 Vijay Kumar Banerjee, <vijaykumar9597 at gmail.com>
wrote:

> It shows the following error
>
> .......
> RTEMS Testing - Tester, 5 (58acb6d61ac7 modified)
> Coverage environment prepared
> [ 3/13] p:0  f:0  u:0  e:0  I:0  B:0  t:0  i:0  W:0  | sparc/leon3_qemu:
> cdtest.exe
> [ 1/13] p:0  f:0  u:0  e:0  I:0  B:0  t:0  i:0  W:0  | sparc/leon3_qemu:
> base_sp.exe
> [ 2/13] p:0  f:0  u:0  e:0  I:0  B:0  t:0  i:0  W:0  | sparc/leon3_qemu:
> capture.exe
> [ 4/13] p:0  f:0  u:0  e:0  I:0  B:0  t:0  i:0  W:0  | sparc/leon3_qemu:
> fileio.exe
> error: qemu.cfg:77: macro '%{coverage_supported}' not found
> warning: switched to dry run due to errors
> error: qemu.cfg:77: invalid if bool value:  %if %{coverage_supported}
> error: qemu.cfg:77: macro '%{coverage_supported}' not found
> warning: switched to dry run due to errors
> error: qemu.cfg:77: invalid if bool value:  %if %{coverage_supported}
> Traceback (most recent call last):
>   File
> "/home/lunatic/development/rtems/test/rtems-tools/tester/rtems-test", line
> 40, in <module>
>     rt.test.run()
>   File
> "/home/lunatic/development/rtems/test/rtems-tools/tester/rt/test.py", line
> 353, in run
>     tst.reraise()
>   File
> "/home/lunatic/development/rtems/test/rtems-tools/tester/rt/test.py", line
> 160, in reraise
>     _test_reraise(*self.result)
> NameError: global name '_test_reraise' is not defined
> error: qemu.cfg:77: macro '%{coverage_supported}' not found
> warning: switched to dry run due to errors
> error: qemu.cfg:77: invalid if bool value:  %if %{coverage_supported}
> error: qemu.cfg:77: macro '%{coverage_supported}' not found
> warning: switched to dry run due to errors
> error: qemu.cfg:77: invalid if bool value:  %if %{coverage_supported}
>
>
>
> -- vijay
>
> On 13 April 2018 at 22:54, Cillian O'Donnell <cpodonnell8 at gmail.com>
> wrote:
>
>> Ah yes, your missing the qemu arguments to run the coverage. Try this
>>
>> #
>> # Coverage analysis
>> #
>> %define coverage_arg %{nil}
>> %if %{defined _coverage}
>>     %if %{coverage_supported}
>>         %define coverage_arg -exec-trace %{test_executable}.cov
>>     %else
>>         %error "Coverage analysis unsupported for %{bsp}"
>>     %endif
>> %endif
>>
>> #
>> # Executable
>> #
>> %execute %{qemu_cmd} %{qemu_opts_base} %{qemu_opts} -kernel %{test_executable} %{coverage_arg}
>>
>>
>> On Fri, 13 Apr 2018, 18:18 Vijay Kumar Banerjee, <
>> vijaykumar9597 at gmail.com> wrote:
>>
>>> sure
>>>
>>> .........
>>>
>>> #
>>> # RTEMS Tools Project (http://www.rtems.org/)
>>> # Copyright 2010-2014 Chris Johns (chrisj at rtems.org)
>>> # All rights reserved.
>>> #
>>> # This file is part of the RTEMS Tools package in 'rtems-tools'.
>>> #
>>> # Redistribution and use in source and binary forms, with or without
>>> # modification, are permitted provided that the following conditions are
>>> met:
>>> #
>>> # 1. Redistributions of source code must retain the above copyright
>>> notice,
>>> # this list of conditions and the following disclaimer.
>>> #
>>> # 2. Redistributions in binary form must reproduce the above copyright
>>> notice,
>>> # this list of conditions and the following disclaimer in the
>>> documentation
>>> # and/or other materials provided with the distribution.
>>> #
>>> # THIS SOFTWARE IS PROVIDED BY THE COPYRIGHT HOLDERS AND CONTRIBUTORS
>>> "AS IS"
>>> # AND ANY EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT LIMITED TO,
>>> THE
>>> # IMPLIED WARRANTIES OF MERCHANTABILITY AND FITNESS FOR A PARTICULAR
>>> PURPOSE
>>> # ARE DISCLAIMED. IN NO EVENT SHALL THE COPYRIGHT HOLDER OR CONTRIBUTORS
>>> BE
>>> # LIABLE FOR ANY DIRECT, INDIRECT, INCIDENTAL, SPECIAL, EXEMPLARY, OR
>>> # CONSEQUENTIAL DAMAGES (INCLUDING, BUT NOT LIMITED TO, PROCUREMENT OF
>>> # SUBSTITUTE GOODS OR SERVICES; LOSS OF USE, DATA, OR PROFITS; OR
>>> BUSINESS
>>> # INTERRUPTION) HOWEVER CAUSED AND ON ANY THEORY OF LIABILITY, WHETHER IN
>>> # CONTRACT, STRICT LIABILITY, OR TORT (INCLUDING NEGLIGENCE OR OTHERWISE)
>>> # ARISING IN ANY WAY OUT OF THE USE OF THIS SOFTWARE, EVEN IF ADVISED OF
>>> THE
>>> # POSSIBILITY OF SUCH DAMAGE.
>>> #
>>>
>>> #
>>> # QEMU
>>> #
>>> # Use a qemu command to run the executable in the qemu simulator.
>>> #
>>>
>>> %include %{_configdir}/base.cfg
>>> %include %{_configdir}/checks.cfg
>>>
>>> #
>>> # Console.
>>> #
>>> %define console_stdio
>>> %include %{_configdir}/console.cfg
>>>
>>> #
>>> # RTEMS version
>>> #
>>> %include %{_rtdir}/rtems/version.cfg
>>>
>>> #
>>> # Qemu common option patterns.
>>> #
>>> #%define qemu_opts_base   -no-reboot -monitor none -serial stdio
>>> -nographic
>>> #%define qemu_opts_base   -no-reboot -serial null -serial mon:stdio
>>> -nographic
>>> %define qemu_opts_base  -no-reboot -monitor null -serial stdio -nographic
>>> %define qemu_opts_no_net -net none
>>> #
>>> # Qemu executable
>>> #
>>> %ifn %{defined bsp_qemu_opts}
>>>  %define bsp_qemu_opts %{nil}
>>> %endif
>>> %define qemu_cmd  qemu-system-%{bsp_arch}
>>> %define qemu_opts %{bsp_qemu_opts}
>>>
>>> #
>>> # Executable
>>> #
>>> %execute %{qemu_cmd} %{qemu_opts} -kernel %{test_executable}
>>>
>>> -- vijay
>>>
>>> On 13 April 2018 at 22:19, Cillian O'Donnell <cpodonnell8 at gmail.com>
>>> wrote:
>>>
>>>> Yeah that's fine but can you just paste the contents here anyway. I'm
>>>> not near a laptop and I can't remember which branch your starting from.
>>>>
>>>> On Fri, 13 Apr 2018, 13:51 Vijay Kumar Banerjee, <
>>>> vijaykumar9597 at gmail.com> wrote:
>>>>
>>>>> I haven't made any local changes to qemu.cfg
>>>>>
>>>>> -- vijay
>>>>>
>>>>> On 13 April 2018 at 11:43, Cillian O'Donnell <cpodonnell8 at gmail.com>
>>>>> wrote:
>>>>>
>>>>>> No that's the right one. I think it might be something to do with
>>>>>> qemu.cfg in rtems-tools then. What's in your qemu.cfg? It's in
>>>>>> rtems-tools/tester/rtems/testing
>>>>>>
>>>>>> On Fri, 13 Apr 2018, 06:39 Vijay Kumar Banerjee, <
>>>>>> vijaykumar9597 at gmail.com> wrote:
>>>>>>
>>>>>>> I manually built qemu from here
>>>>>>>
>>>>>>> https://github.com/AdaCore/qemu
>>>>>>>
>>>>>>> I guess this is regular qemu , please send a link for couverture
>>>>>>> qemu for manual build.
>>>>>>>
>>>>>>> -- vijay
>>>>>>>
>>>>>>> On 12 April 2018 at 22:23, Cillian O'Donnell <cpodonnell8 at gmail.com>
>>>>>>> wrote:
>>>>>>>
>>>>>>>> Ah that's good progress, those .cov files are supposed to be
>>>>>>>> generated by couverture-qemu as the test are run. They contain the trace
>>>>>>>> data. Are you definitely running couverture-qemu and not regular qemu?
>>>>>>>>
>>>>>>>> On Thu, 12 Apr 2018, 17:41 Vijay Kumar Banerjee, <
>>>>>>>> vijaykumar9597 at gmail.com> wrote:
>>>>>>>>
>>>>>>>>> I got the parsing of the coverage from the ini file running , now
>>>>>>>>> I'm getting the following error and warnings .
>>>>>>>>>
>>>>>>>>>
>>>>>>>>> ....
>>>>>>>>>
>>>>>>>>> Running covoar for score
>>>>>>>>> WARNING: Unable to read coverage file
>>>>>>>>> /home/lunatic/development/rtems/kernel/leon3/sparc-rtems5/c/leon3/testsuites/samples/base_sp/base_sp.exe.cov
>>>>>>>>> WARNING: Unable to read coverage file
>>>>>>>>> /home/lunatic/development/rtems/kernel/leon3/sparc-rtems5/c/leon3/testsuites/samples/capture/capture.exe.cov
>>>>>>>>> WARNING: Unable to read coverage file
>>>>>>>>> /home/lunatic/development/rtems/kernel/leon3/sparc-rtems5/c/leon3/testsuites/samples/cdtest/cdtest.exe.cov
>>>>>>>>> WARNING: Unable to read coverage file
>>>>>>>>> /home/lunatic/development/rtems/kernel/leon3/sparc-rtems5/c/leon3/testsuites/samples/fileio/fileio.exe.cov
>>>>>>>>> WARNING: Unable to read coverage file
>>>>>>>>> /home/lunatic/development/rtems/kernel/leon3/sparc-rtems5/c/leon3/testsuites/samples/hello/hello.exe.cov
>>>>>>>>> WARNING: Unable to read coverage file
>>>>>>>>> /home/lunatic/development/rtems/kernel/leon3/sparc-rtems5/c/leon3/testsuites/samples/iostream/cxx_iostream.exe.cov
>>>>>>>>> WARNING: Unable to read coverage file
>>>>>>>>> /home/lunatic/development/rtems/kernel/leon3/sparc-rtems5/c/leon3/testsuites/samples/loopback/loopback.exe.cov
>>>>>>>>> WARNING: Unable to read coverage file
>>>>>>>>> /home/lunatic/development/rtems/kernel/leon3/sparc-rtems5/c/leon3/testsuites/samples/minimum/minimum.exe.cov
>>>>>>>>> WARNING: Unable to read coverage file
>>>>>>>>> /home/lunatic/development/rtems/kernel/leon3/sparc-rtems5/c/leon3/testsuites/samples/nsecs/nsecs.exe.cov
>>>>>>>>> WARNING: Unable to read coverage file
>>>>>>>>> /home/lunatic/development/rtems/kernel/leon3/sparc-rtems5/c/leon3/testsuites/samples/paranoia/paranoia.exe.cov
>>>>>>>>> WARNING: Unable to read coverage file
>>>>>>>>> /home/lunatic/development/rtems/kernel/leon3/sparc-rtems5/c/leon3/testsuites/samples/pppd/pppd.exe.cov
>>>>>>>>> WARNING: Unable to read coverage file
>>>>>>>>> /home/lunatic/development/rtems/kernel/leon3/sparc-rtems5/c/leon3/testsuites/samples/ticker/ticker.exe.cov
>>>>>>>>> WARNING: Unable to read coverage file
>>>>>>>>> /home/lunatic/development/rtems/kernel/leon3/sparc-rtems5/c/leon3/testsuites/samples/unlimited/unlimited.exe.cov
>>>>>>>>> ERROR: No information to analyze
>>>>>>>>> ***Cleaning tempfiles***
>>>>>>>>> error: covoar failure exit code: 1
>>>>>>>>>
>>>>>>>>>
>>>>>>>>>
>>>>>>>>>
>>>>>>>>> -- vijay
>>>>>>>>>
>>>>>>>>> On 10 April 2018 at 18:10, Vijay Kumar Banerjee <
>>>>>>>>> vijaykumar9597 at gmail.com> wrote:
>>>>>>>>>
>>>>>>>>>> I managed to get the tests running with the following two patches
>>>>>>>>>> .
>>>>>>>>>> Please review them .
>>>>>>>>>>
>>>>>>>>>> -- vijay
>>>>>>>>>>
>>>>>>>>>> On 9 April 2018 at 15:28, Vijay Kumar Banerjee <
>>>>>>>>>> vijaykumar9597 at gmail.com> wrote:
>>>>>>>>>>
>>>>>>>>>>>
>>>>>>>>>>>
>>>>>>>>>>> On Fri, 6 Apr 2018, 02:24 Vijay Kumar Banerjee, <
>>>>>>>>>>> vijaykumar9597 at gmail.com> wrote:
>>>>>>>>>>>
>>>>>>>>>>>> I tried to solve the empty get in the way shown in the patch ,
>>>>>>>>>>>> but it doesn't seems to work, I'm getting the following error now
>>>>>>>>>>>>
>>>>>>>>>>>> RTEMS Testing - Tester, 5 (1ed6e2e64090)
>>>>>>>>>>>> Traceback (most recent call last):
>>>>>>>>>>>>   File
>>>>>>>>>>>> "/home/lunatic/development/rtems/rtems-tools/tester/rtems-test", line 40,
>>>>>>>>>>>> in <module>
>>>>>>>>>>>>     rt.test.run()
>>>>>>>>>>>>   File
>>>>>>>>>>>> "/home/lunatic/development/rtems/rtems-tools/tester/rt/test.py", line 303,
>>>>>>>>>>>> in run
>>>>>>>>>>>>     bsp = config.load(bsp[1], opts)
>>>>>>>>>>>>   File
>>>>>>>>>>>> "/home/lunatic/development/rtems/rtems-tools/tester/rt/config.py", line
>>>>>>>>>>>> 461, in load
>>>>>>>>>>>>     return opts.defaults['bsp']
>>>>>>>>>>>>   File
>>>>>>>>>>>> "/home/lunatic/development/rtems/rtems-tools/rtemstoolkit/macros.py", line
>>>>>>>>>>>> 190, in __getitem__
>>>>>>>>>>>>     raise IndexError('key: %s' % (key))
>>>>>>>>>>>> IndexError: key: bsp
>>>>>>>>>>>>
>>>>>>>>>>>> I still Couldn't figure out why it's happening, when I use
>>>>>>>>>>> RawConfigParse , it seems to run nicely though .
>>>>>>>>>>>
>>>>>>>>>>> Any Ideas on solving it without using RawConfigParser ?
>>>>>>>>>>>
>>>>>>>>>>>
>>>>>>>>>>>> -- vijay
>>>>>>>>>>>>
>>>>>>>>>>>> On 5 April 2018 at 11:51, Cillian O'Donnell <
>>>>>>>>>>>> cpodonnell8 at gmail.com> wrote:
>>>>>>>>>>>>
>>>>>>>>>>>>>
>>>>>>>>>>>>>
>>>>>>>>>>>>> On Wed, 4 Apr 2018, 11:18 Vijay Kumar Banerjee, <
>>>>>>>>>>>>> vijaykumar9597 at gmail.com> wrote:
>>>>>>>>>>>>>
>>>>>>>>>>>>>> I got the rtems-test running
>>>>>>>>>>>>>>
>>>>>>>>>>>>>> I found that get_item_names(section, err) in
>>>>>>>>>>>>>> configuration.py  was returning an empty list to the
>>>>>>>>>>>>>> item_names in load(bsp,opts) for config.py this was giving
>>>>>>>>>>>>>> out the error mandatory item not found .
>>>>>>>>>>>>>>
>>>>>>>>>>>>>> I have attached the patch file .
>>>>>>>>>>>>>>
>>>>>>>>>>>>>> current status :
>>>>>>>>>>>>>> rtems-test is running without coverage
>>>>>>>>>>>>>>
>>>>>>>>>>>>>> running coverage gives the following error :
>>>>>>>>>>>>>>
>>>>>>>>>>>>>> RTEMS Testing - Tester, 5 (092c569e81ae)
>>>>>>>>>>>>>> Traceback (most recent call last):
>>>>>>>>>>>>>>   File
>>>>>>>>>>>>>> "/home/lunatic/development/rtems/rtems-tools/tester/rtems-test", line 40,
>>>>>>>>>>>>>> in <module>
>>>>>>>>>>>>>>     rt.test.run()
>>>>>>>>>>>>>>   File
>>>>>>>>>>>>>> "/home/lunatic/development/rtems/rtems-tools/tester/rt/test.py", line 310,
>>>>>>>>>>>>>> in run
>>>>>>>>>>>>>>     coverage = coverage_get_obj(opts, path_to_builddir[1])
>>>>>>>>>>>>>>   File
>>>>>>>>>>>>>> "/home/lunatic/development/rtems/rtems-tools/tester/rt/test.py", line 230,
>>>>>>>>>>>>>> in coverage_get_obj
>>>>>>>>>>>>>>     coverage_obj = coverage.coverage_run(opts.defaults,
>>>>>>>>>>>>>> path_to_builddir)
>>>>>>>>>>>>>>   File
>>>>>>>>>>>>>> "/home/lunatic/development/rtems/rtems-tools/tester/rt/coverage.py", line
>>>>>>>>>>>>>> 335, in __init__
>>>>>>>>>>>>>>     self.report_format = self.config_map['report_format'][2]
>>>>>>>>>>>>>>
>>>>>>>>>>>>>
>>>>>>>>>>>>> This is back to the difference in cfg formats. It's looking
>>>>>>>>>>>>> for the last line here
>>>>>>>>>>>>>
>>>>>>>>>>>>>
>>>>>>>>>>>>> https://github.com/cillianodonnell/rtems-tools/blob/coverage/tester/rtems/testing/bsps/leon3-qemu.mc
>>>>>>>>>>>>>
>>>>>>>>>>>>> Which is now
>>>>>>>>>>>>>
>>>>>>>>>>>>>
>>>>>>>>>>>>> https://github.com/cillianodonnell/rtems-tools/blob/ini-update/tester/rtems/testing/bsps/leon3_qemu.ini
>>>>>>>>>>>>>
>>>>>>>>>>>>> It's trying to grab the 'html' text so it can trigger the
>>>>>>>>>>>>> right section of code in coverage.py
>>>>>>>>>>>>>
>>>>>>>>>>>>> It's something just to tell which format we want the report to
>>>>>>>>>>>>> output
>>>>>>>>>>>>>
>>>>>>>>>>>>
>>>>>>>>>>>> Understood , I'll come to that after I manage to get the test
>>>>>>>>>>>> running first .
>>>>>>>>>>>>
>>>>>>>>>>>>>
>>>>>>>>>>>>> TypeError: 'NoneType' object has no attribute '__getitem__'
>>>>>>>>>>>>>>
>>>>>>>>>>>>>>
>>>>>>>>>>>>>> Thanks
>>>>>>>>>>>>>>
>>>>>>>>>>>>>> -- vijay
>>>>>>>>>>>>>>
>>>>>>>>>>>>>> On 4 April 2018 at 11:16, Chris Johns <chrisj at rtems.org>
>>>>>>>>>>>>>> wrote:
>>>>>>>>>>>>>>
>>>>>>>>>>>>>>> On 04/04/2018 06:36, Vijay Kumar Banerjee wrote:
>>>>>>>>>>>>>>> > after pulling the ini-update branch from
>>>>>>>>>>>>>>> >
>>>>>>>>>>>>>>> https://github.com/cillianodonnell/rtems-tools/tree/ini-update
>>>>>>>>>>>>>>> >
>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>> Any local changes?
>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>> > I tried running the rtems-test with the following command
>>>>>>>>>>>>>>> >
>>>>>>>>>>>>>>> > $HOME/development/rtems/rtems-tools/tester/rtems-test
>>>>>>>>>>>>>>> > --rtems-tools=$HOME/development/rtems/5
>>>>>>>>>>>>>>> --log=coverage_analysis.log
>>>>>>>>>>>>>>> > --rtems-bsp=leon3_qemu
>>>>>>>>>>>>>>> --rtems-builddir=$HOME/development/rtems/kernel/leon3
>>>>>>>>>>>>>>> > sparc-rtems5/c/leon3/testsuites/samples
>>>>>>>>>>>>>>> >
>>>>>>>>>>>>>>> > was getting the KeyError for 'coverage'
>>>>>>>>>>>>>>> >
>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>> Please paste the command and output as it appears in the
>>>>>>>>>>>>>>> terminal?
>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>> > current status :
>>>>>>>>>>>>>>> > the same command gives the error :
>>>>>>>>>>>>>>> > "mandatory item not found in bsp section : bsp,"
>>>>>>>>>>>>>>> > to make sure I didn't break something , I also tried
>>>>>>>>>>>>>>> running from a freshly
>>>>>>>>>>>>>>> > downloaded directory as well, it gives the same error.
>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>> Did you try running the following command from the top of
>>>>>>>>>>>>>>> the source tree:
>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>  $ grep -r "mandatory item not found" *
>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>> ?
>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>> Chris
>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>
>>>>>>>>>>>>>>
>>>>>>>>>>>>
>>>>>>>>>>
>>>>>>>>>
>>>>>>>
>>>>>
>>>
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.rtems.org/pipermail/devel/attachments/20180413/7db5f0f7/attachment-0002.html>

More information about the devel mailing list