How can I use the covoar tool?

[Sebastian Huber]

Hello,

thanks for your hints.

On 20/03/2019 16:31, Vijay Kumar Banerjee wrote:
>
>
>
>
> On Wed, 20 Mar 2019 at 17:46, Sebastian Huber 
> <sebastian.huber at embedded-brains.de 
> <mailto:sebastian.huber at embedded-brains.de>> wrote:
>
>     Hello,
>
>     the RTEMS project has a covoar tool:
>
>     https://git.rtems.org/rtems-tools/tree/tester/covoar
>
>     I have seen some reports generated by it, but I never used it
>     myself. I
>     have trouble to find out how it works. There is nothing in the
>     documentation:
>
>     cd rtems-docs
>     grep -ri covoar .
>
>     There is some documentation available here:
>
>     https://devel.rtems.org/wiki/GCI/Documentation/CoverageAnalysis/Coverage
>
>     If I invoke the rtems-test like this in the erc32 BSP build
>     directory, I
>     get an error:
>
>     rtems-test --rtems-bsp=erc32-sis --rtems-tools=/build/rtems/5
>     --coverage
>     --target=sparc-rtems5 .
>     RTEMS Testing - Tester, 5.0.not_released
>       Command Line: /build/rtems/5/bin/rtems-test --rtems-bsp=erc32-sis
>     --rtems-tools=/build/rtems/5 --coverage --target=sparc-rtems5 .
>       Python: 3.6.5 (default, Mar 31 2018, 19:45:04) [GCC]
>     Host: Linux-4.12.14-lp150.12.48-default-x86_64-with-glibc2.3.4 (Linux
>     huber-nb-linux.suse 4.12.14-lp150.12.48-default #1 SMP Tue Feb 12
>     14:01:48 UTC 2019 (268f014) x86_64 x86_64)
>     Traceback (most recent call last):
>        File "/build/rtems/5/share/rtems/tester/rt/cmd-test.py", line
>     42, in
>     <module>
>          test.run(sys.argv[1:], command_path = base)
>        File "/build/rtems/5/share/rtems/tester/rt/test.py", line 300,
>     in run
>          trace = cov_trace)
>        File "/build/rtems/5/share/rtems/tester/rt/coverage.py", line
>     363, in
>     __init__
>          self.target = self.macros['target']
>        File "/build/rtems/5/share/rtems/rtemstoolkit/macros.py", line
>     181,
>     in __getitem__
>          raise IndexError('key: %s' % (key))
>     IndexError: key: target
>
> Hi,
>
> The covoar needs the .cov files to run which is supposed to be 
> generated by a command like
> this
> ====
> $HOME/development/rtems/5/bin/sparc-rtems5-sis -nouartrx -r -tlim 600 
> s -cov 
> $HOME/development/rtems/kernel/erc32/sparc-rtems5/c/erc32/testsuites/samples/hello.exe
> ====
> But this runs into a segmentation fault. If my command is right 
> (Please correct me if it isn't)
> then I think it's because the sis coverage is not supported for erc32. :/

Yes, I get also a segmentation fault. For the leon3 it seems to work:

sparc-rtems5-sis -leon3 -nouartrx -r -tlim 200 s -cov 
./sparc-rtems5/c/leon3/testsuites/samples/hello.exe

  SIS - SPARC/RISCV instruction simulator 2.13,  copyright Jiri Gaisler 2019
  Bug-reports to jiri at gaisler.se

  LEON3 emulation enabled, 1 cpus online, delta 50 clocks



*** BEGIN OF TEST HELLO WORLD ***
*** TEST VERSION: 5.0.0.45e418f922af81c9cb8d485c789ba73c95bda813
*** TEST STATE: EXPECTED-PASS
*** TEST BUILD: RTEMS_DEBUG RTEMS_NETWORKING RTEMS_POSIX_API RTEMS_SMP
*** TEST TOOLS: 7.4.0 20181206 (RTEMS 5, RSB 
c41b9d0df7e5b4a5056ca50c2534380a44e92769, Newlib 3e24fbf6f)
Hello World

*** END OF TEST HELLO WORLD ***

cpu 0 in error mode (tt = 0x80)
    165222  400103e0  91d02000  ta  0

saved code coverage to 
./sparc-rtems5/c/leon3/testsuites/samples/hello.exe.cov

>
> about rtems-test,
> I have attached a patch that adds erc32-sis-cov.ini, this will fix the 
> KeyError you're seeing,
> but it will return with error 10 as it's trying to run the above 
> command. If you run rtems-test
> with `--no-clean` and `--coverage` options, you'll have a 
> erc32-sis-symbols.ini file (and the .cov
> files, when they're supported, like in leon3), which can be used to 
> run covoar directly like
> below ( but this won't run now as we don't have .cov file )

I tried to use leon3-sis-cov and I get:

rtems-test --rtems-bsp=leon3-sis-cov --rtems-tools=/build/rtems/5 
--coverage .
RTEMS Testing - Tester, 5.0.not_released
  Command Line: /build/rtems/5/bin/rtems-test --rtems-bsp=leon3-sis-cov 
--rtems-tools=/build/rtems/5 --coverage .
  Python: 3.6.5 (default, Mar 31 2018, 19:45:04) [GCC]
Host: Linux-4.12.14-lp150.12.48-default-x86_64-with-glibc2.3.4 (Linux 
huber-nb-linux.suse 4.12.14-lp150.12.48-default #1 SMP Tue Feb 12 
14:01:48 UTC 2019 (268f014) x86_64 x86_64)
Traceback (most recent call last):
   File "/build/rtems/5/share/rtems/tester/rt/cmd-test.py", line 42, in 
<module>
     test.run(sys.argv[1:], command_path = base)
   File "/build/rtems/5/share/rtems/tester/rt/test.py", line 300, in run
     trace = cov_trace)
   File "/build/rtems/5/share/rtems/tester/rt/coverage.py", line 363, in 
__init__
     self.target = self.macros['target']
   File "/build/rtems/5/share/rtems/rtemstoolkit/macros.py", line 181, 
in __getitem__
     raise IndexError('key: %s' % (key))
IndexError: key: target

>
> covoar -S 
> $HOME/development/rtems/test/rtems-tools/tester/rtems/testing/coverage/erc32-sis-symbols.ini 
> \
> -O coverage 
> -E$HOME/development/rtems/test/rtems-tools/tester/rtems/testing/coverage/Explanations.txt 
> \
> -p RTEMS-5 sparc-rtems5/c/erc32/testsuites/samples/hello.exe

Do I have to call rtems-test first and then covoar or is it called by 
rtems-test itself?

-- 
Sebastian Huber, embedded brains GmbH

Address : Dornierstr. 4, D-82178 Puchheim, Germany
Phone   : +49 89 189 47 41-16
Fax     : +49 89 189 47 41-09
E-Mail  : sebastian.huber at embedded-brains.de
PGP     : Public key available on request.

Diese Nachricht ist keine geschäftliche Mitteilung im Sinne des EHUG.