error while running rtems-test with coverage

Cillian O'Donnell cpodonnell8 at gmail.com
Fri Apr 13 17:24:04 UTC 2018 

Ah yes, your missing the qemu arguments to run the coverage. Try this

#
# Coverage analysis
#
%define coverage_arg %{nil}
%if %{defined _coverage}
    %if %{coverage_supported}
        %define coverage_arg -exec-trace %{test_executable}.cov
    %else
        %error "Coverage analysis unsupported for %{bsp}"
    %endif
%endif

#
# Executable
#
%execute %{qemu_cmd} %{qemu_opts_base} %{qemu_opts} -kernel
%{test_executable} %{coverage_arg}


On Fri, 13 Apr 2018, 18:18 Vijay Kumar Banerjee, <vijaykumar9597 at gmail.com>
wrote:

> sure
>
> .........
>
> #
> # RTEMS Tools Project (http://www.rtems.org/)
> # Copyright 2010-2014 Chris Johns (chrisj at rtems.org)
> # All rights reserved.
> #
> # This file is part of the RTEMS Tools package in 'rtems-tools'.
> #
> # Redistribution and use in source and binary forms, with or without
> # modification, are permitted provided that the following conditions are
> met:
> #
> # 1. Redistributions of source code must retain the above copyright notice,
> # this list of conditions and the following disclaimer.
> #
> # 2. Redistributions in binary form must reproduce the above copyright
> notice,
> # this list of conditions and the following disclaimer in the documentation
> # and/or other materials provided with the distribution.
> #
> # THIS SOFTWARE IS PROVIDED BY THE COPYRIGHT HOLDERS AND CONTRIBUTORS "AS
> IS"
> # AND ANY EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT LIMITED TO, THE
> # IMPLIED WARRANTIES OF MERCHANTABILITY AND FITNESS FOR A PARTICULAR
> PURPOSE
> # ARE DISCLAIMED. IN NO EVENT SHALL THE COPYRIGHT HOLDER OR CONTRIBUTORS BE
> # LIABLE FOR ANY DIRECT, INDIRECT, INCIDENTAL, SPECIAL, EXEMPLARY, OR
> # CONSEQUENTIAL DAMAGES (INCLUDING, BUT NOT LIMITED TO, PROCUREMENT OF
> # SUBSTITUTE GOODS OR SERVICES; LOSS OF USE, DATA, OR PROFITS; OR BUSINESS
> # INTERRUPTION) HOWEVER CAUSED AND ON ANY THEORY OF LIABILITY, WHETHER IN
> # CONTRACT, STRICT LIABILITY, OR TORT (INCLUDING NEGLIGENCE OR OTHERWISE)
> # ARISING IN ANY WAY OUT OF THE USE OF THIS SOFTWARE, EVEN IF ADVISED OF
> THE
> # POSSIBILITY OF SUCH DAMAGE.
> #
>
> #
> # QEMU
> #
> # Use a qemu command to run the executable in the qemu simulator.
> #
>
> %include %{_configdir}/base.cfg
> %include %{_configdir}/checks.cfg
>
> #
> # Console.
> #
> %define console_stdio
> %include %{_configdir}/console.cfg
>
> #
> # RTEMS version
> #
> %include %{_rtdir}/rtems/version.cfg
>
> #
> # Qemu common option patterns.
> #
> #%define qemu_opts_base   -no-reboot -monitor none -serial stdio -nographic
> #%define qemu_opts_base   -no-reboot -serial null -serial mon:stdio
> -nographic
> %define qemu_opts_base  -no-reboot -monitor null -serial stdio -nographic
> %define qemu_opts_no_net -net none
> #
> # Qemu executable
> #
> %ifn %{defined bsp_qemu_opts}
>  %define bsp_qemu_opts %{nil}
> %endif
> %define qemu_cmd  qemu-system-%{bsp_arch}
> %define qemu_opts %{bsp_qemu_opts}
>
> #
> # Executable
> #
> %execute %{qemu_cmd} %{qemu_opts} -kernel %{test_executable}
>
> -- vijay
>
> On 13 April 2018 at 22:19, Cillian O'Donnell <cpodonnell8 at gmail.com>
> wrote:
>
>> Yeah that's fine but can you just paste the contents here anyway. I'm not
>> near a laptop and I can't remember which branch your starting from.
>>
>> On Fri, 13 Apr 2018, 13:51 Vijay Kumar Banerjee, <
>> vijaykumar9597 at gmail.com> wrote:
>>
>>> I haven't made any local changes to qemu.cfg
>>>
>>> -- vijay
>>>
>>> On 13 April 2018 at 11:43, Cillian O'Donnell <cpodonnell8 at gmail.com>
>>> wrote:
>>>
>>>> No that's the right one. I think it might be something to do with
>>>> qemu.cfg in rtems-tools then. What's in your qemu.cfg? It's in
>>>> rtems-tools/tester/rtems/testing
>>>>
>>>> On Fri, 13 Apr 2018, 06:39 Vijay Kumar Banerjee, <
>>>> vijaykumar9597 at gmail.com> wrote:
>>>>
>>>>> I manually built qemu from here
>>>>>
>>>>> https://github.com/AdaCore/qemu
>>>>>
>>>>> I guess this is regular qemu , please send a link for couverture
>>>>> qemu for manual build.
>>>>>
>>>>> -- vijay
>>>>>
>>>>> On 12 April 2018 at 22:23, Cillian O'Donnell <cpodonnell8 at gmail.com>
>>>>> wrote:
>>>>>
>>>>>> Ah that's good progress, those .cov files are supposed to be
>>>>>> generated by couverture-qemu as the test are run. They contain the trace
>>>>>> data. Are you definitely running couverture-qemu and not regular qemu?
>>>>>>
>>>>>> On Thu, 12 Apr 2018, 17:41 Vijay Kumar Banerjee, <
>>>>>> vijaykumar9597 at gmail.com> wrote:
>>>>>>
>>>>>>> I got the parsing of the coverage from the ini file running , now
>>>>>>> I'm getting the following error and warnings .
>>>>>>>
>>>>>>>
>>>>>>> ....
>>>>>>>
>>>>>>> Running covoar for score
>>>>>>> WARNING: Unable to read coverage file
>>>>>>> /home/lunatic/development/rtems/kernel/leon3/sparc-rtems5/c/leon3/testsuites/samples/base_sp/base_sp.exe.cov
>>>>>>> WARNING: Unable to read coverage file
>>>>>>> /home/lunatic/development/rtems/kernel/leon3/sparc-rtems5/c/leon3/testsuites/samples/capture/capture.exe.cov
>>>>>>> WARNING: Unable to read coverage file
>>>>>>> /home/lunatic/development/rtems/kernel/leon3/sparc-rtems5/c/leon3/testsuites/samples/cdtest/cdtest.exe.cov
>>>>>>> WARNING: Unable to read coverage file
>>>>>>> /home/lunatic/development/rtems/kernel/leon3/sparc-rtems5/c/leon3/testsuites/samples/fileio/fileio.exe.cov
>>>>>>> WARNING: Unable to read coverage file
>>>>>>> /home/lunatic/development/rtems/kernel/leon3/sparc-rtems5/c/leon3/testsuites/samples/hello/hello.exe.cov
>>>>>>> WARNING: Unable to read coverage file
>>>>>>> /home/lunatic/development/rtems/kernel/leon3/sparc-rtems5/c/leon3/testsuites/samples/iostream/cxx_iostream.exe.cov
>>>>>>> WARNING: Unable to read coverage file
>>>>>>> /home/lunatic/development/rtems/kernel/leon3/sparc-rtems5/c/leon3/testsuites/samples/loopback/loopback.exe.cov
>>>>>>> WARNING: Unable to read coverage file
>>>>>>> /home/lunatic/development/rtems/kernel/leon3/sparc-rtems5/c/leon3/testsuites/samples/minimum/minimum.exe.cov
>>>>>>> WARNING: Unable to read coverage file
>>>>>>> /home/lunatic/development/rtems/kernel/leon3/sparc-rtems5/c/leon3/testsuites/samples/nsecs/nsecs.exe.cov
>>>>>>> WARNING: Unable to read coverage file
>>>>>>> /home/lunatic/development/rtems/kernel/leon3/sparc-rtems5/c/leon3/testsuites/samples/paranoia/paranoia.exe.cov
>>>>>>> WARNING: Unable to read coverage file
>>>>>>> /home/lunatic/development/rtems/kernel/leon3/sparc-rtems5/c/leon3/testsuites/samples/pppd/pppd.exe.cov
>>>>>>> WARNING: Unable to read coverage file
>>>>>>> /home/lunatic/development/rtems/kernel/leon3/sparc-rtems5/c/leon3/testsuites/samples/ticker/ticker.exe.cov
>>>>>>> WARNING: Unable to read coverage file
>>>>>>> /home/lunatic/development/rtems/kernel/leon3/sparc-rtems5/c/leon3/testsuites/samples/unlimited/unlimited.exe.cov
>>>>>>> ERROR: No information to analyze
>>>>>>> ***Cleaning tempfiles***
>>>>>>> error: covoar failure exit code: 1
>>>>>>>
>>>>>>>
>>>>>>>
>>>>>>>
>>>>>>> -- vijay
>>>>>>>
>>>>>>> On 10 April 2018 at 18:10, Vijay Kumar Banerjee <
>>>>>>> vijaykumar9597 at gmail.com> wrote:
>>>>>>>
>>>>>>>> I managed to get the tests running with the following two patches .
>>>>>>>> Please review them .
>>>>>>>>
>>>>>>>> -- vijay
>>>>>>>>
>>>>>>>> On 9 April 2018 at 15:28, Vijay Kumar Banerjee <
>>>>>>>> vijaykumar9597 at gmail.com> wrote:
>>>>>>>>
>>>>>>>>>
>>>>>>>>>
>>>>>>>>> On Fri, 6 Apr 2018, 02:24 Vijay Kumar Banerjee, <
>>>>>>>>> vijaykumar9597 at gmail.com> wrote:
>>>>>>>>>
>>>>>>>>>> I tried to solve the empty get in the way shown in the patch ,
>>>>>>>>>> but it doesn't seems to work, I'm getting the following error now
>>>>>>>>>>
>>>>>>>>>> RTEMS Testing - Tester, 5 (1ed6e2e64090)
>>>>>>>>>> Traceback (most recent call last):
>>>>>>>>>>   File
>>>>>>>>>> "/home/lunatic/development/rtems/rtems-tools/tester/rtems-test", line 40,
>>>>>>>>>> in <module>
>>>>>>>>>>     rt.test.run()
>>>>>>>>>>   File
>>>>>>>>>> "/home/lunatic/development/rtems/rtems-tools/tester/rt/test.py", line 303,
>>>>>>>>>> in run
>>>>>>>>>>     bsp = config.load(bsp[1], opts)
>>>>>>>>>>   File
>>>>>>>>>> "/home/lunatic/development/rtems/rtems-tools/tester/rt/config.py", line
>>>>>>>>>> 461, in load
>>>>>>>>>>     return opts.defaults['bsp']
>>>>>>>>>>   File
>>>>>>>>>> "/home/lunatic/development/rtems/rtems-tools/rtemstoolkit/macros.py", line
>>>>>>>>>> 190, in __getitem__
>>>>>>>>>>     raise IndexError('key: %s' % (key))
>>>>>>>>>> IndexError: key: bsp
>>>>>>>>>>
>>>>>>>>>> I still Couldn't figure out why it's happening, when I use
>>>>>>>>> RawConfigParse , it seems to run nicely though .
>>>>>>>>>
>>>>>>>>> Any Ideas on solving it without using RawConfigParser ?
>>>>>>>>>
>>>>>>>>>
>>>>>>>>>> -- vijay
>>>>>>>>>>
>>>>>>>>>> On 5 April 2018 at 11:51, Cillian O'Donnell <
>>>>>>>>>> cpodonnell8 at gmail.com> wrote:
>>>>>>>>>>
>>>>>>>>>>>
>>>>>>>>>>>
>>>>>>>>>>> On Wed, 4 Apr 2018, 11:18 Vijay Kumar Banerjee, <
>>>>>>>>>>> vijaykumar9597 at gmail.com> wrote:
>>>>>>>>>>>
>>>>>>>>>>>> I got the rtems-test running
>>>>>>>>>>>>
>>>>>>>>>>>> I found that get_item_names(section, err) in configuration.py
>>>>>>>>>>>> was returning an empty list to the
>>>>>>>>>>>> item_names in load(bsp,opts) for config.py this was giving out
>>>>>>>>>>>> the error mandatory item not found .
>>>>>>>>>>>>
>>>>>>>>>>>> I have attached the patch file .
>>>>>>>>>>>>
>>>>>>>>>>>> current status :
>>>>>>>>>>>> rtems-test is running without coverage
>>>>>>>>>>>>
>>>>>>>>>>>> running coverage gives the following error :
>>>>>>>>>>>>
>>>>>>>>>>>> RTEMS Testing - Tester, 5 (092c569e81ae)
>>>>>>>>>>>> Traceback (most recent call last):
>>>>>>>>>>>>   File
>>>>>>>>>>>> "/home/lunatic/development/rtems/rtems-tools/tester/rtems-test", line 40,
>>>>>>>>>>>> in <module>
>>>>>>>>>>>>     rt.test.run()
>>>>>>>>>>>>   File
>>>>>>>>>>>> "/home/lunatic/development/rtems/rtems-tools/tester/rt/test.py", line 310,
>>>>>>>>>>>> in run
>>>>>>>>>>>>     coverage = coverage_get_obj(opts, path_to_builddir[1])
>>>>>>>>>>>>   File
>>>>>>>>>>>> "/home/lunatic/development/rtems/rtems-tools/tester/rt/test.py", line 230,
>>>>>>>>>>>> in coverage_get_obj
>>>>>>>>>>>>     coverage_obj = coverage.coverage_run(opts.defaults,
>>>>>>>>>>>> path_to_builddir)
>>>>>>>>>>>>   File
>>>>>>>>>>>> "/home/lunatic/development/rtems/rtems-tools/tester/rt/coverage.py", line
>>>>>>>>>>>> 335, in __init__
>>>>>>>>>>>>     self.report_format = self.config_map['report_format'][2]
>>>>>>>>>>>>
>>>>>>>>>>>
>>>>>>>>>>> This is back to the difference in cfg formats. It's looking for
>>>>>>>>>>> the last line here
>>>>>>>>>>>
>>>>>>>>>>>
>>>>>>>>>>> https://github.com/cillianodonnell/rtems-tools/blob/coverage/tester/rtems/testing/bsps/leon3-qemu.mc
>>>>>>>>>>>
>>>>>>>>>>> Which is now
>>>>>>>>>>>
>>>>>>>>>>>
>>>>>>>>>>> https://github.com/cillianodonnell/rtems-tools/blob/ini-update/tester/rtems/testing/bsps/leon3_qemu.ini
>>>>>>>>>>>
>>>>>>>>>>> It's trying to grab the 'html' text so it can trigger the right
>>>>>>>>>>> section of code in coverage.py
>>>>>>>>>>>
>>>>>>>>>>> It's something just to tell which format we want the report to
>>>>>>>>>>> output
>>>>>>>>>>>
>>>>>>>>>>
>>>>>>>>>> Understood , I'll come to that after I manage to get the test
>>>>>>>>>> running first .
>>>>>>>>>>
>>>>>>>>>>>
>>>>>>>>>>> TypeError: 'NoneType' object has no attribute '__getitem__'
>>>>>>>>>>>>
>>>>>>>>>>>>
>>>>>>>>>>>> Thanks
>>>>>>>>>>>>
>>>>>>>>>>>> -- vijay
>>>>>>>>>>>>
>>>>>>>>>>>> On 4 April 2018 at 11:16, Chris Johns <chrisj at rtems.org> wrote:
>>>>>>>>>>>>
>>>>>>>>>>>>> On 04/04/2018 06:36, Vijay Kumar Banerjee wrote:
>>>>>>>>>>>>> > after pulling the ini-update branch from
>>>>>>>>>>>>> >
>>>>>>>>>>>>> https://github.com/cillianodonnell/rtems-tools/tree/ini-update
>>>>>>>>>>>>> >
>>>>>>>>>>>>>
>>>>>>>>>>>>> Any local changes?
>>>>>>>>>>>>>
>>>>>>>>>>>>> > I tried running the rtems-test with the following command
>>>>>>>>>>>>> >
>>>>>>>>>>>>> > $HOME/development/rtems/rtems-tools/tester/rtems-test
>>>>>>>>>>>>> > --rtems-tools=$HOME/development/rtems/5
>>>>>>>>>>>>> --log=coverage_analysis.log
>>>>>>>>>>>>> > --rtems-bsp=leon3_qemu
>>>>>>>>>>>>> --rtems-builddir=$HOME/development/rtems/kernel/leon3
>>>>>>>>>>>>> > sparc-rtems5/c/leon3/testsuites/samples
>>>>>>>>>>>>> >
>>>>>>>>>>>>> > was getting the KeyError for 'coverage'
>>>>>>>>>>>>> >
>>>>>>>>>>>>>
>>>>>>>>>>>>> Please paste the command and output as it appears in the
>>>>>>>>>>>>> terminal?
>>>>>>>>>>>>>
>>>>>>>>>>>>> > current status :
>>>>>>>>>>>>> > the same command gives the error :
>>>>>>>>>>>>> > "mandatory item not found in bsp section : bsp,"
>>>>>>>>>>>>> > to make sure I didn't break something , I also tried running
>>>>>>>>>>>>> from a freshly
>>>>>>>>>>>>> > downloaded directory as well, it gives the same error.
>>>>>>>>>>>>>
>>>>>>>>>>>>> Did you try running the following command from the top of the
>>>>>>>>>>>>> source tree:
>>>>>>>>>>>>>
>>>>>>>>>>>>>  $ grep -r "mandatory item not found" *
>>>>>>>>>>>>>
>>>>>>>>>>>>> ?
>>>>>>>>>>>>>
>>>>>>>>>>>>> Chris
>>>>>>>>>>>>>
>>>>>>>>>>>>
>>>>>>>>>>>>
>>>>>>>>>>
>>>>>>>>
>>>>>>>
>>>>>
>>>
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.rtems.org/pipermail/devel/attachments/20180413/3292722a/attachment-0001.html>

More information about the devel mailing list