the generated coverage report doesn't contain any data

Cillian O'Donnell cpodonnell8 at gmail.com
Thu Apr 19 13:51:22 UTC 2018 

Oh great!. So the coverage-merge branch worked. The stuff about symbol-sets
and a fix to get rid of a dependency of one of the external tools is all
that hasn't been merged there. Like Joel said there, try run it again with
covoar built with less optimization. The option is in covoar/wscript.

On Wed, 18 Apr 2018, 17:32 Joel Sherrill, <joel at rtems.org> wrote:

>
>
> On Wed, Apr 18, 2018 at 10:49 AM, Vijay Kumar Banerjee <
> vijaykumar9597 at gmail.com> wrote:
>
>> It's great to see it produce some data .
>> I have attached a screenshot of the report generated from testsuites
>>
>
> It is good to see results. I have no idea about the "goodness" of them.
> That depends
> on the configuration, tests run, optimization level, etc. We normally use
> -Os for
> coverage analysis because -O2 unrolls loops and duplicates code. -Os has
> the
> generated code more closely match the source.
>
> I suppose the next step is to work to get Cillian's updated work merged.
>
> Then for me to try to duplicate the results.
>
> FWIW I am teaching an RTEMS class next week and won't be around as much.
>
> --joel
>
>
>>
>>
>>
>> On 18 April 2018 at 02:07, Cillian O'Donnell <cpodonnell8 at gmail.com>
>> wrote:
>>
>>> Could be still missing something, try using the whole covoar directory
>>> from coverage-merge. I spent a good bit of time fixing errors like that,
>>> would be strange if it popped back up again but it could be right. Either
>>> way we're in pretty good shape now. Report looks good, try and run it with
>>> the full testsuite to see what happens, it takes a while, a hour-ish.
>>>
>>> On 17 April 2018 at 20:32, Vijay Kumar Banerjee <
>>> vijaykumar9597 at gmail.com> wrote:
>>>
>>>>
>>>>
>>>>
>>>> On 18 April 2018 at 00:57, Vijay Kumar Banerjee <
>>>> vijaykumar9597 at gmail.com> wrote:
>>>>
>>>>> The report is showing some data !!
>>>>>
>>>>> I merged a lot of changes form coverage-merge , it's showing some data
>>>>> now , but I'm still getting a set of errors. I'll paste them below.
>>>>>
>>>>> I have attached a screenshot of the report.html
>>>>>
>>>>> errors :
>>>>> ............
>>>>> INFO: DesiredSymbols::createCoverageMap - Attempt to create unified
>>>>> coverage maps for _Workspace_Allocate_or_fatal_error with different sizes
>>>>> (/home/lunatic/development/rtems/kernel/leon3/sparc-rtems5/c/leon3/testsuites/samples/capture/capture.exe/84
>>>>> !=
>>>>> /home/lunatic/development/rtems/kernel/leon3/sparc-rtems5/c/leon3/testsuites/samples/base_sp/base_sp.exe/60)
>>>>>
>>>>> INFO: DesiredSymbols::mergeCoverageMap - Unable to merge coverage map
>>>>> for _Workspace_Allocate_or_fatal_error because the sizes are different
>>>>>
>>>>> Am I still possibly missing something ?
>>>>
>>>>>
>>>>>
>>>> On 17 April 2018 at 11:59, Cillian O'Donnell <cpodonnell8 at gmail.com>
>>>>> wrote:
>>>>>
>>>>>> Also if there's any file in covoar directory of coverage-merge branch
>>>>>> that's called symbol-set.c symbol_set_reader.h or any variation of that,
>>>>>> that you don't already have pull all of them into your current branch.
>>>>>>
>>>>>> On Tue, 17 Apr 2018, 07:19 Cillian O'Donnell, <cpodonnell8 at gmail.com>
>>>>>> wrote:
>>>>>>
>>>>>>> Some things are definitely missing there. Git checkout main.c
>>>>>>> coverage-merge. If you have that branch lying around, that definitely has
>>>>>>> everything in it. I must of missed of some things updating to current
>>>>>>> master. Fingers crossed this solves our problems
>>>>>>>
>>>>>>>
>>>>>>> On Mon, 16 Apr 2018, 22:52 Vijay Kumar Banerjee, <
>>>>>>> vijaykumar9597 at gmail.com> wrote:
>>>>>>>
>>>>>>>>
>>>>>>>>
>>>>>>>> On Tue, 17 Apr 2018, 03:19 Joel Sherrill, <joel at rtems.org> wrote:
>>>>>>>>
>>>>>>>>>
>>>>>>>>>
>>>>>>>>> On Mon, Apr 16, 2018 at 4:47 PM, Vijay Kumar Banerjee <
>>>>>>>>> vijaykumar9597 at gmail.com> wrote:
>>>>>>>>>
>>>>>>>>>>
>>>>>>>>>>
>>>>>>>>>> On 17 April 2018 at 01:43, Cillian O'Donnell <
>>>>>>>>>> cpodonnell8 at gmail.com> wrote:
>>>>>>>>>>
>>>>>>>>>>>
>>>>>>>>>>>
>>>>>>>>>>> On 16 April 2018 at 17:46, Joel Sherrill <joel at rtems.org> wrote:
>>>>>>>>>>>
>>>>>>>>>>>>
>>>>>>>>>>>>
>>>>>>>>>>>> On Mon, Apr 16, 2018 at 2:33 AM, Vijay Kumar Banerjee <
>>>>>>>>>>>> vijaykumar9597 at gmail.com> wrote:
>>>>>>>>>>>>
>>>>>>>>>>>>> current status :
>>>>>>>>>>>>> the coverage is running now with rtems-test and generating the
>>>>>>>>>>>>> report, however, the report doesn't show any data.
>>>>>>>>>>>>>
>>>>>>>>>>>>
>>>>>>>>>>>> I've been lurking as you have been making progress. Now you
>>>>>>>>>>>> have crossed
>>>>>>>>>>>> into something you probably need some hints at. Some things to
>>>>>>>>>>>> check:
>>>>>>>>>>>>
>>>>>>>>>>>> + Obviously, check output for signs that something didn't
>>>>>>>>>>>> happen right.
>>>>>>>>>>>> Anything from a path wrong, etc. The configuration has to be
>>>>>>>>>>>> right to
>>>>>>>>>>>> point to the source code, object code, executables, etc.
>>>>>>>>>>>>
>>>>>>>>>>>> + A lot of source code has moved around since last summer. Check
>>>>>>>>>>>> that it is looking in the right places inside RTEMS.
>>>>>>>>>>>>
>>>>>>>>>>>> + Does the mechanism to get debug information actually work?
>>>>>>>>>>>> Cillian?
>>>>>>>>>>>>
>>>>>>>>>>>
>>>>>>>>>>> The covoar debug option just disables the cleaning of the
>>>>>>>>>>> tempfiles to take a look, so it's not as powerful as it might seem :)... I
>>>>>>>>>>> always used gdb here so that's probably the way to go.
>>>>>>>>>>>
>>>>>>>>>>>>
>>>>>>>>>>>> + There is a utility named trace-converter. Make sure your qemu
>>>>>>>>>>>> traces
>>>>>>>>>>>> have information in them.
>>>>>>>>>>>>
>>>>>>>>>>>> + Cillian probably has guidance on running it just on one test
>>>>>>>>>>>> (say ticker)
>>>>>>>>>>>> so you can see every step.
>>>>>>>>>>>>
>>>>>>>>>>>> + Check that the executables have symbolic information. "file"
>>>>>>>>>>>> should
>>>>>>>>>>>> show if they are stripped or not.
>>>>>>>>>>>>
>>>>>>>>>>>> I recall covoar has a verbose mode which should be of use.
>>>>>>>>>>>>
>>>>>>>>>>>> Cillian .. do you have instructions on running covoar in gdb?
>>>>>>>>>>>>
>>>>>>>>>>>
>>>>>>>>>>> Alright so run rtems-test with --no-clean option to leave the
>>>>>>>>>>> coverage files lying around
>>>>>>>>>>>
>>>>>>>>>>> $HOME/development/rtems/test/test/rtems-tools/tester/rtems-test
>>>>>>>>>>> --rtems-tools=$HOME/development/rtems/5 --log=coverage-analysis.log
>>>>>>>>>>> --rtems-bsp=leon3_qemu --coverage --no-clean
>>>>>>>>>>> --rtems-builddir=$HOME/development/rtems/leon3
>>>>>>>>>>> sparc-rtems5/c/leon3/testsuites/samples
>>>>>>>>>>>
>>>>>>>>>>> Then run
>>>>>>>>>>>
>>>>>>>>>>> gdb covoar
>>>>>>>>>>>
>>>>>>>>>>> from gdb prompt
>>>>>>>>>>>
>>>>>>>>>>> run -S /home/cpod/coverage_test/leon3/coverage/score.symcfg -O
>>>>>>>>>>> /home/cpod/coverage_test/leon3/test/score -fQEMU -Tsparc-rtems5
>>>>>>>>>>> -E/home/cpod/development/rtems/test/test/vijay/rtems-tools/tester/rtems/testing/coverage/Explanations.txt
>>>>>>>>>>> -c.cov -eexe -pRTEMS-5
>>>>>>>>>>> /home/cpod/development/rtems/leon3/sparc-rtems5/c/leon3/testsuites/samples/base_sp/base_sp.exe
>>>>>>>>>>>
>>>>>>>>>>> The options there are ( if you're wondering )
>>>>>>>>>>>
>>>>>>>>>>>  -c COVERAGEFILE_EXTENSION EXECUTABLE1 ... EXECUTABLE2
>>>>>>>>>>>
>>>>>>>>>>>  -v                  - verbose output
>>>>>>>>>>>  -T TARGET           - architecture target name
>>>>>>>>>>>  -f FORMAT           - simulator format
>>>>>>>>>>> (RTEMS, QEMU, TSIM or Skyeye)
>>>>>>>>>>>  -E EXPLANATIONS     - file of explanations
>>>>>>>>>>>  -s SYMBOLS_FILE     - symbols of interest
>>>>>>>>>>>  -S SYMBOL_SET_FILE  - path to symbol_sets.cfg
>>>>>>>>>>>  -1 EXECUTABLE       - executable to get symbols from
>>>>>>>>>>>  -e EXE_EXTENSION    - suffix for executables
>>>>>>>>>>>  -c COVERAGEFILE_EXT - coverage file suffix
>>>>>>>>>>>  -g GCNOS_LIST       - list of *.gcno files
>>>>>>>>>>>  -p PROJECT_NAME     - name of the project
>>>>>>>>>>>  -O Output_Directory - output directory default=.
>>>>>>>>>>>  -d debug            - disable cleaning of tempfiles.
>>>>>>>>>>>
>>>>>>>>>>>
>>>>>>>>>> maybe the problem is here . I am getting an error for the -S .
>>>>>>>>>> Here are the list of options that show up , capital S for symbol_set_file
>>>>>>>>>> is not one of them .
>>>>>>>>>>
>>>>>>>>>> Usage: /home/lunatic/development/rtems/5/bin/covoar [-v] -T
>>>>>>>>>> TARGET -f FORMAT [-E EXPLANATIONS] -e EXE_EXTENSION -c
>>>>>>>>>> COVERAGEFILE_EXTENSION EXECUTABLE1 ... EXECUTABLE2
>>>>>>>>>>
>>>>>>>>>>   -v                        - verbose at initialization
>>>>>>>>>>   -T TARGET                 - target name
>>>>>>>>>>   -f FORMAT                 - coverage file format (RTEMS, QEMU,
>>>>>>>>>> TSIM or Skyeye)
>>>>>>>>>>   -E EXPLANATIONS           - name of file with explanations
>>>>>>>>>>   -s SYMBOLS_FILE           - name of file with symbols of
>>>>>>>>>> interest
>>>>>>>>>>   -1 EXECUTABLE             - name of executable to get symbols
>>>>>>>>>> from
>>>>>>>>>>   -e EXE_EXTENSION          - extension of the executables to
>>>>>>>>>> analyze
>>>>>>>>>>   -c COVERAGEFILE_EXTENSION - extension of the coverage files to
>>>>>>>>>> analyze
>>>>>>>>>>   -g GCNOS_LIST             - name of file with list of *.gcno
>>>>>>>>>> files
>>>>>>>>>>   -p PROJECT_NAME           - name of the project
>>>>>>>>>>   -C ConfigurationFileName  - name of configuration file
>>>>>>>>>>   -O Output_Directory       - name of output directory (default=.
>>>>>>>>>>
>>>>>>>>>
>>>>>>>>> Is it processed by the getopts switch statement in main() but not
>>>>>>>>> listed in the usage? That is
>>>>>>>>> an easy mistake to creep in.
>>>>>>>>>
>>>>>>>> It gives an "invalid option" error
>>>>>>>>
>>>>>>>>>
>>>>>>>>>
>>>>>>>>>>
>>>>>>>>>>
>>>>>>>>>>> Just trying it there, it runs into segmentation fault. So trying
>>>>>>>>>>> to access memory it shouldn't.
>>>>>>>>>>>
>>>>>>>>>>> Starting program: /home/cpod/covoar -S
>>>>>>>>>>> /home/cpod/coverage_test/leon3/coverage/score.symcfg -O
>>>>>>>>>>> /home/cpod/coverage_test/leon3/test/score -fQEMU -Tsparc-rtems5
>>>>>>>>>>> -E/home/cpod/development/rtems/test/test/vijay/rtems-tools/tester/rtems/testing/coverage/Explanations.txt
>>>>>>>>>>> -c.cov -eexe -pRTEMS-5
>>>>>>>>>>> /home/cpod/development/rtems/leon3/sparc-rtems5/c/leon3/testsuites/samples/base_sp/base_sp.exe
>>>>>>>>>>> Reading configuration symbol set file:
>>>>>>>>>>> /home/cpod/coverage_test/leon3/coverage/score.symcfg
>>>>>>>>>>>
>>>>>>>>>>> Program received signal SIGSEGV, Segmentation fault.
>>>>>>>>>>> 0x00007ffff7b769bb in std::__cxx11::basic_string<char,
>>>>>>>>>>> std::char_traits<char>, std::allocator<char>
>>>>>>>>>>> >::basic_string(std::__cxx11::basic_string<char, std::char_traits<char>,
>>>>>>>>>>> std::allocator<char> > const&) ()
>>>>>>>>>>>    from /usr/lib/x86_64-linux-gnu/libstdc++.so.6
>>>>>>>>>>>
>>>>>>>>>>> Ahhh feels good to have error messages back.
>>>>>>>>>>>
>>>>>>>>>>> Oh also build covoar with no optimization and you'll have an
>>>>>>>>>>> easier time looking at stuff in gdb
>>>>>>>>>>>
>>>>>>>>>>> cd rtems-tools/tester/covoar
>>>>>>>>>>>
>>>>>>>>>>> vim wscript and change the '-O2' to '-O0' and then build again
>>>>>>>>>>> with waf and use that covoar to with gdb
>>>>>>>>>>>
>>>>>>>>>>>
>>>>>>>>>>>>
>>>>>>>>>>>>
>>>>>>>>>>>>>
>>>>>>>>>>>>> -- vijay
>>>>>>>>>>>>>
>>>>>>>>>>>>
>>>>>>>>>>>>
>>>>>>>>>>>
>>>>>>>>>>
>>>>>>>>>
>>>>>
>>>>
>>>
>>
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.rtems.org/pipermail/devel/attachments/20180419/55d10808/attachment-0002.html>

More information about the devel mailing list