the generated coverage report doesn't contain any data

Joel Sherrill joel at rtems.org
Wed Apr 18 16:32:00 UTC 2018 

On Wed, Apr 18, 2018 at 10:49 AM, Vijay Kumar Banerjee <
vijaykumar9597 at gmail.com> wrote:

> It's great to see it produce some data .
> I have attached a screenshot of the report generated from testsuites
>

It is good to see results. I have no idea about the "goodness" of them.
That depends
on the configuration, tests run, optimization level, etc. We normally use
-Os for
coverage analysis because -O2 unrolls loops and duplicates code. -Os has the
generated code more closely match the source.

I suppose the next step is to work to get Cillian's updated work merged.

Then for me to try to duplicate the results.

FWIW I am teaching an RTEMS class next week and won't be around as much.

--joel


>
>
>
> On 18 April 2018 at 02:07, Cillian O'Donnell <cpodonnell8 at gmail.com>
> wrote:
>
>> Could be still missing something, try using the whole covoar directory
>> from coverage-merge. I spent a good bit of time fixing errors like that,
>> would be strange if it popped back up again but it could be right. Either
>> way we're in pretty good shape now. Report looks good, try and run it with
>> the full testsuite to see what happens, it takes a while, a hour-ish.
>>
>> On 17 April 2018 at 20:32, Vijay Kumar Banerjee <vijaykumar9597 at gmail.com
>> > wrote:
>>
>>>
>>>
>>>
>>> On 18 April 2018 at 00:57, Vijay Kumar Banerjee <
>>> vijaykumar9597 at gmail.com> wrote:
>>>
>>>> The report is showing some data !!
>>>>
>>>> I merged a lot of changes form coverage-merge , it's showing some data
>>>> now , but I'm still getting a set of errors. I'll paste them below.
>>>>
>>>> I have attached a screenshot of the report.html
>>>>
>>>> errors :
>>>> ............
>>>> INFO: DesiredSymbols::createCoverageMap - Attempt to create unified
>>>> coverage maps for _Workspace_Allocate_or_fatal_error with different
>>>> sizes (/home/lunatic/development/rtems/kernel/leon3/sparc-rtems5/c
>>>> /leon3/testsuites/samples/capture/capture.exe/84 !=
>>>> /home/lunatic/development/rtems/kernel/leon3/sparc-rtems5/c/
>>>> leon3/testsuites/samples/base_sp/base_sp.exe/60)
>>>>
>>>> INFO: DesiredSymbols::mergeCoverageMap - Unable to merge coverage map
>>>> for _Workspace_Allocate_or_fatal_error because the sizes are different
>>>>
>>>> Am I still possibly missing something ?
>>>
>>>>
>>>>
>>> On 17 April 2018 at 11:59, Cillian O'Donnell <cpodonnell8 at gmail.com>
>>>> wrote:
>>>>
>>>>> Also if there's any file in covoar directory of coverage-merge branch
>>>>> that's called symbol-set.c symbol_set_reader.h or any variation of that,
>>>>> that you don't already have pull all of them into your current branch.
>>>>>
>>>>> On Tue, 17 Apr 2018, 07:19 Cillian O'Donnell, <cpodonnell8 at gmail.com>
>>>>> wrote:
>>>>>
>>>>>> Some things are definitely missing there. Git checkout main.c
>>>>>> coverage-merge. If you have that branch lying around, that definitely has
>>>>>> everything in it. I must of missed of some things updating to current
>>>>>> master. Fingers crossed this solves our problems
>>>>>>
>>>>>>
>>>>>> On Mon, 16 Apr 2018, 22:52 Vijay Kumar Banerjee, <
>>>>>> vijaykumar9597 at gmail.com> wrote:
>>>>>>
>>>>>>>
>>>>>>>
>>>>>>> On Tue, 17 Apr 2018, 03:19 Joel Sherrill, <joel at rtems.org> wrote:
>>>>>>>
>>>>>>>>
>>>>>>>>
>>>>>>>> On Mon, Apr 16, 2018 at 4:47 PM, Vijay Kumar Banerjee <
>>>>>>>> vijaykumar9597 at gmail.com> wrote:
>>>>>>>>
>>>>>>>>>
>>>>>>>>>
>>>>>>>>> On 17 April 2018 at 01:43, Cillian O'Donnell <
>>>>>>>>> cpodonnell8 at gmail.com> wrote:
>>>>>>>>>
>>>>>>>>>>
>>>>>>>>>>
>>>>>>>>>> On 16 April 2018 at 17:46, Joel Sherrill <joel at rtems.org> wrote:
>>>>>>>>>>
>>>>>>>>>>>
>>>>>>>>>>>
>>>>>>>>>>> On Mon, Apr 16, 2018 at 2:33 AM, Vijay Kumar Banerjee <
>>>>>>>>>>> vijaykumar9597 at gmail.com> wrote:
>>>>>>>>>>>
>>>>>>>>>>>> current status :
>>>>>>>>>>>> the coverage is running now with rtems-test and generating the
>>>>>>>>>>>> report, however, the report doesn't show any data.
>>>>>>>>>>>>
>>>>>>>>>>>
>>>>>>>>>>> I've been lurking as you have been making progress. Now you have
>>>>>>>>>>> crossed
>>>>>>>>>>> into something you probably need some hints at. Some things to
>>>>>>>>>>> check:
>>>>>>>>>>>
>>>>>>>>>>> + Obviously, check output for signs that something didn't happen
>>>>>>>>>>> right.
>>>>>>>>>>> Anything from a path wrong, etc. The configuration has to be
>>>>>>>>>>> right to
>>>>>>>>>>> point to the source code, object code, executables, etc.
>>>>>>>>>>>
>>>>>>>>>>> + A lot of source code has moved around since last summer. Check
>>>>>>>>>>> that it is looking in the right places inside RTEMS.
>>>>>>>>>>>
>>>>>>>>>>> + Does the mechanism to get debug information actually work?
>>>>>>>>>>> Cillian?
>>>>>>>>>>>
>>>>>>>>>>
>>>>>>>>>> The covoar debug option just disables the cleaning of the
>>>>>>>>>> tempfiles to take a look, so it's not as powerful as it might seem :)... I
>>>>>>>>>> always used gdb here so that's probably the way to go.
>>>>>>>>>>
>>>>>>>>>>>
>>>>>>>>>>> + There is a utility named trace-converter. Make sure your qemu
>>>>>>>>>>> traces
>>>>>>>>>>> have information in them.
>>>>>>>>>>>
>>>>>>>>>>> + Cillian probably has guidance on running it just on one test
>>>>>>>>>>> (say ticker)
>>>>>>>>>>> so you can see every step.
>>>>>>>>>>>
>>>>>>>>>>> + Check that the executables have symbolic information. "file"
>>>>>>>>>>> should
>>>>>>>>>>> show if they are stripped or not.
>>>>>>>>>>>
>>>>>>>>>>> I recall covoar has a verbose mode which should be of use.
>>>>>>>>>>>
>>>>>>>>>>> Cillian .. do you have instructions on running covoar in gdb?
>>>>>>>>>>>
>>>>>>>>>>
>>>>>>>>>> Alright so run rtems-test with --no-clean option to leave the
>>>>>>>>>> coverage files lying around
>>>>>>>>>>
>>>>>>>>>> $HOME/development/rtems/test/test/rtems-tools/tester/rtems-test
>>>>>>>>>> --rtems-tools=$HOME/development/rtems/5
>>>>>>>>>> --log=coverage-analysis.log --rtems-bsp=leon3_qemu --coverage --no-clean
>>>>>>>>>> --rtems-builddir=$HOME/development/rtems/leon3
>>>>>>>>>> sparc-rtems5/c/leon3/testsuites/samples
>>>>>>>>>>
>>>>>>>>>> Then run
>>>>>>>>>>
>>>>>>>>>> gdb covoar
>>>>>>>>>>
>>>>>>>>>> from gdb prompt
>>>>>>>>>>
>>>>>>>>>> run -S /home/cpod/coverage_test/leon3/coverage/score.symcfg -O
>>>>>>>>>> /home/cpod/coverage_test/leon3/test/score -fQEMU -Tsparc-rtems5
>>>>>>>>>> -E/home/cpod/development/rtems/test/test/vijay/rtems-tools/t
>>>>>>>>>> ester/rtems/testing/coverage/Explanations.txt -c.cov -eexe
>>>>>>>>>> -pRTEMS-5 /home/cpod/development/rtems/l
>>>>>>>>>> eon3/sparc-rtems5/c/leon3/testsuites/samples/base_sp/base_sp.exe
>>>>>>>>>>
>>>>>>>>>> The options there are ( if you're wondering )
>>>>>>>>>>
>>>>>>>>>>  -c COVERAGEFILE_EXTENSION EXECUTABLE1 ... EXECUTABLE2
>>>>>>>>>>
>>>>>>>>>>  -v                  - verbose output
>>>>>>>>>>  -T TARGET           - architecture target name
>>>>>>>>>>  -f FORMAT           - simulator format
>>>>>>>>>> (RTEMS, QEMU, TSIM or Skyeye)
>>>>>>>>>>  -E EXPLANATIONS     - file of explanations
>>>>>>>>>>  -s SYMBOLS_FILE     - symbols of interest
>>>>>>>>>>  -S SYMBOL_SET_FILE  - path to symbol_sets.cfg
>>>>>>>>>>  -1 EXECUTABLE       - executable to get symbols from
>>>>>>>>>>  -e EXE_EXTENSION    - suffix for executables
>>>>>>>>>>  -c COVERAGEFILE_EXT - coverage file suffix
>>>>>>>>>>  -g GCNOS_LIST       - list of *.gcno files
>>>>>>>>>>  -p PROJECT_NAME     - name of the project
>>>>>>>>>>  -O Output_Directory - output directory default=.
>>>>>>>>>>  -d debug            - disable cleaning of tempfiles.
>>>>>>>>>>
>>>>>>>>>>
>>>>>>>>> maybe the problem is here . I am getting an error for the -S .
>>>>>>>>> Here are the list of options that show up , capital S for symbol_set_file
>>>>>>>>> is not one of them .
>>>>>>>>>
>>>>>>>>> Usage: /home/lunatic/development/rtems/5/bin/covoar [-v] -T
>>>>>>>>> TARGET -f FORMAT [-E EXPLANATIONS] -e EXE_EXTENSION -c
>>>>>>>>> COVERAGEFILE_EXTENSION EXECUTABLE1 ... EXECUTABLE2
>>>>>>>>>
>>>>>>>>>   -v                        - verbose at initialization
>>>>>>>>>   -T TARGET                 - target name
>>>>>>>>>   -f FORMAT                 - coverage file format (RTEMS, QEMU,
>>>>>>>>> TSIM or Skyeye)
>>>>>>>>>   -E EXPLANATIONS           - name of file with explanations
>>>>>>>>>   -s SYMBOLS_FILE           - name of file with symbols of interest
>>>>>>>>>   -1 EXECUTABLE             - name of executable to get symbols
>>>>>>>>> from
>>>>>>>>>   -e EXE_EXTENSION          - extension of the executables to
>>>>>>>>> analyze
>>>>>>>>>   -c COVERAGEFILE_EXTENSION - extension of the coverage files to
>>>>>>>>> analyze
>>>>>>>>>   -g GCNOS_LIST             - name of file with list of *.gcno
>>>>>>>>> files
>>>>>>>>>   -p PROJECT_NAME           - name of the project
>>>>>>>>>   -C ConfigurationFileName  - name of configuration file
>>>>>>>>>   -O Output_Directory       - name of output directory (default=.
>>>>>>>>>
>>>>>>>>
>>>>>>>> Is it processed by the getopts switch statement in main() but not
>>>>>>>> listed in the usage? That is
>>>>>>>> an easy mistake to creep in.
>>>>>>>>
>>>>>>> It gives an "invalid option" error
>>>>>>>
>>>>>>>>
>>>>>>>>
>>>>>>>>>
>>>>>>>>>
>>>>>>>>>> Just trying it there, it runs into segmentation fault. So trying
>>>>>>>>>> to access memory it shouldn't.
>>>>>>>>>>
>>>>>>>>>> Starting program: /home/cpod/covoar -S
>>>>>>>>>> /home/cpod/coverage_test/leon3/coverage/score.symcfg -O
>>>>>>>>>> /home/cpod/coverage_test/leon3/test/score -fQEMU -Tsparc-rtems5
>>>>>>>>>> -E/home/cpod/development/rtems/test/test/vijay/rtems-tools/t
>>>>>>>>>> ester/rtems/testing/coverage/Explanations.txt -c.cov -eexe
>>>>>>>>>> -pRTEMS-5 /home/cpod/development/rtems/l
>>>>>>>>>> eon3/sparc-rtems5/c/leon3/testsuites/samples/base_sp/base_sp.exe
>>>>>>>>>> Reading configuration symbol set file:
>>>>>>>>>> /home/cpod/coverage_test/leon3/coverage/score.symcfg
>>>>>>>>>>
>>>>>>>>>> Program received signal SIGSEGV, Segmentation fault.
>>>>>>>>>> 0x00007ffff7b769bb in std::__cxx11::basic_string<char,
>>>>>>>>>> std::char_traits<char>, std::allocator<char> >::basic_string(std::__cxx11::basic_string<char,
>>>>>>>>>> std::char_traits<char>, std::allocator<char> > const&) ()
>>>>>>>>>>    from /usr/lib/x86_64-linux-gnu/libstdc++.so.6
>>>>>>>>>>
>>>>>>>>>> Ahhh feels good to have error messages back.
>>>>>>>>>>
>>>>>>>>>> Oh also build covoar with no optimization and you'll have an
>>>>>>>>>> easier time looking at stuff in gdb
>>>>>>>>>>
>>>>>>>>>> cd rtems-tools/tester/covoar
>>>>>>>>>>
>>>>>>>>>> vim wscript and change the '-O2' to '-O0' and then build again
>>>>>>>>>> with waf and use that covoar to with gdb
>>>>>>>>>>
>>>>>>>>>>
>>>>>>>>>>>
>>>>>>>>>>>
>>>>>>>>>>>>
>>>>>>>>>>>> -- vijay
>>>>>>>>>>>>
>>>>>>>>>>>
>>>>>>>>>>>
>>>>>>>>>>
>>>>>>>>>
>>>>>>>>
>>>>
>>>
>>
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.rtems.org/pipermail/devel/attachments/20180418/65382d92/attachment-0002.html>

More information about the devel mailing list