How can I use the covoar tool?
Vijay Kumar Banerjee
vijaykumar9597 at gmail.com
Fri May 17 08:49:09 UTC 2019
On Fri, May 17, 2019 at 1:05 PM Sebastian Huber <
sebastian.huber at embedded-brains.de> wrote:
> On 20/03/2019 13:16, Sebastian Huber wrote:
> > Hello,
> >
> > the RTEMS project has a covoar tool:
> >
> > https://git.rtems.org/rtems-tools/tree/tester/covoar
> >
> > I have seen some reports generated by it, but I never used it myself.
> > I have trouble to find out how it works. There is nothing in the
> > documentation:
> >
> > cd rtems-docs
> > grep -ri covoar .
> >
> > There is some documentation available here:
> >
> > https://devel.rtems.org/wiki/GCI/Documentation/CoverageAnalysis/Coverage
> >
> > If I invoke the rtems-test like this in the erc32 BSP build directory,
> > I get an error:
> >
> > rtems-test --rtems-bsp=erc32-sis --rtems-tools=/build/rtems/5
> > --coverage --target=sparc-rtems5 .
> > RTEMS Testing - Tester, 5.0.not_released
> > Command Line: /build/rtems/5/bin/rtems-test --rtems-bsp=erc32-sis
> > --rtems-tools=/build/rtems/5 --coverage --target=sparc-rtems5 .
> > Python: 3.6.5 (default, Mar 31 2018, 19:45:04) [GCC]
> > Host: Linux-4.12.14-lp150.12.48-default-x86_64-with-glibc2.3.4 (Linux
> > huber-nb-linux.suse 4.12.14-lp150.12.48-default #1 SMP Tue Feb 12
> > 14:01:48 UTC 2019 (268f014) x86_64 x86_64)
> > Traceback (most recent call last):
> > File "/build/rtems/5/share/rtems/tester/rt/cmd-test.py", line 42, in
> > <module>
> > test.run(sys.argv[1:], command_path = base)
> > File "/build/rtems/5/share/rtems/tester/rt/test.py", line 300, in run
> > trace = cov_trace)
> > File "/build/rtems/5/share/rtems/tester/rt/coverage.py", line 363,
> > in __init__
> > self.target = self.macros['target']
> > File "/build/rtems/5/share/rtems/rtemstoolkit/macros.py", line 181,
> > in __getitem__
> > raise IndexError('key: %s' % (key))
> > IndexError: key: target
> >
>
> I made some progress to try it out myself. The next obstacle was this:
>
> rtems-test --rtems-tools=/build/rtems/5 --rtems-bsp=leon3-sis-cov
> --coverage --no-clean --log=leon3-test.log
> sparc-rtems5/c/leon3/testsuites/samples/hello.exe
> RTEMS Testing - Tester, 5.0.not_released
> [1/1] p:0 f:0 u:0 e:0 I:0 B:0 t:0 i:0 W:0 | sparc/leon3-sis: hello.exe
> Passed: 1
> Failed: 0
> User Input: 0
> Expected Fail: 0
> Indeterminate: 0
> Benchmark: 0
> Timeout: 0
> Invalid: 0
> Wrong Version: 0
> Wrong Build: 0
> Wrong Tools: 0
> ----------------
> Total: 1
> Average test time: 0:00:01.505144
> Testing time : 0:00:01.505144
> error: symbol parser write failed for score
>
> To figure out what went wrong, I removed the try/except block from
> tester/rt/coverage.py and got:
>
> rtems-test --rtems-tools=/build/rtems/5 --rtems-bsp=leon3-sis-cov
> --coverage --no-clean --log=leon3-test.log
> sparc-rtems5/c/leon3/testsuites/samples/hello.exe
> RTEMS Testing - Tester, 5.0.not_released
> [1/1] p:0 f:0 u:0 e:0 I:0 B:0 t:0 i:0 W:0 | sparc/leon3-sis: hello.exe
> Passed: 1
> Failed: 0
> User Input: 0
> Expected Fail: 0
> Indeterminate: 0
> Benchmark: 0
> Timeout: 0
> Invalid: 0
> Wrong Version: 0
> Wrong Build: 0
> Wrong Tools: 0
> ----------------
> Total: 1
> Average test time: 0:00:01.504073
> Testing time : 0:00:01.504073
> Traceback (most recent call last):
> File "/build/rtems/5/share/rtems/tester/rt/cmd-test.py", line 42, in
> <module>
> test.run(sys.argv[1:], command_path = base)
> File "/build/rtems/5/share/rtems/tester/rt/test.py", line 387, in run
> coverage_runner.run()
> File "/build/rtems/5/share/rtems/tester/rt/coverage.py", line 401, in
> run
> parser.write_ini(sset)
> File "/build/rtems/5/share/rtems/tester/rt/coverage.py", line 293, in
> write_ini
> object_files = [o for o in os.listdir(self.symbol_sets[sset]) if
> o[-1] == 'o']
> FileNotFoundError: [Errno 2] No such file or directory:
> '/sparc-rtems5/c/leon3/cpukit/score/src'
>
> This is more useful error output from my point of view. It seems you
> have to use an absolute path for the test directory.
>
> rtems-test --rtems-tools=/build/rtems/5 --rtems-bsp=leon3-sis-cov
> --coverage --no-clean --log=leon3-test.log
> /build/git-build/b-leon3/sparc-rtems5/c/leon3/testsuites/samples/hello.exe
> [....]
> Coverage generating reports
> error: copying tree: /build/rtems/5/share/rtems/tester/covoar/covoar.css
> -> /build/git-build/b-leon3/leon3-sis-coverage/score/covoar.css: [Errno
> 2] No such file or directory:
> '/build/rtems/5/share/rtems/tester/covoar/covoar.css'
>
> -rtems-tools also needs an absolute path.
> I copied some files by hand. A full test run ended up in:
>
> rtems-test --rtems-tools=/build/rtems/5 --rtems-bsp=leon3-sis-cov
> --coverage --no-clean --log=leon3-test.log
> /build/git-build/b-leon3/sparc-rtems5/c/leon3/testsuites
> RTEMS Testing - Tester, 5.0.not_released
> [ 3/710] p:0 f:0 u:0 e:0 I:0 B:0 t:0 i:0 W:0 |
> sparc/leon3-sis: ada_nsecs.exe
> [...]
> [710/710] p:684 f:5 u:6 e:0 I:0 B:3 t:0 i:2 W:0 |
> sparc/leon3-sis: tmttest01.exe
> Passed: 692
> Failed: 6
> User Input: 6
> Expected Fail: 0
> Indeterminate: 0
> Benchmark: 3
> Timeout: 1
> Invalid: 2
> Wrong Version: 0
> Wrong Build: 0
> Wrong Tools: 0
> ------------------
> Total: 710
> Average test time: 0:00:00.427096
> Testing time : 0:05:03.237825
>
> Running coverage analysis: score
> (/build/git-build/b-leon3/leon3-sis-coverage/score)
> Coverage time: 0:17:53.695293
>
> Running coverage analysis: rtems
> (/build/git-build/b-leon3/leon3-sis-coverage/rtems)
> Coverage time: 0:12:55.898577
>
> Running coverage analysis: sapi
> (/build/git-build/b-leon3/leon3-sis-coverage/sapi)
> error: coverage: covoar failure:: 10
>
> This is actually a bug in covoar or a bug somewhere in spversion01.exe
We have a ticket for this https://devel.rtems.org/ticket/3515.
To track it down, I manually ran covoar with the trace files using the
following command ( note: you need --no-clean option in rtems-test for this)
covoar -S
/home/lunatic/development/rtems/rtems-tools/tester/rtems/testing/coverage/leon3-sis-symbols.ini
-O coverage
-E/home/lunatic/development/rtems/rtems-tools/tester/rtems/testing/coverage/Explanations.txt
-p RTEMS-5 sparc-rtems5/c/leon3/testsuites/sptests/spversion01.exe -v -fTSIM
====
.
.
.
Created unified coverage map for rtems_initialize_executive (0x0 - 0x4b)
Created unified coverage map for rtems_shutdown_executive (0x0 - 0x13)
Created unified coverage map for rtems_fatal_error_occurred (0x0 - 0x13)
Created unified coverage map for rtems_get_version_string (0x0 - 0xb)
Created unified coverage map for _IO_Initialize_all_drivers (0x0 - 0x4b)
Created unified coverage map for rtems_io_initialize (0x0 - 0x5b)
Created unified coverage map for rtems_version (0x0 - 0xb)
Created unified coverage map for rtems_version_major (0x0 - 0x7)
Created unified coverage map for rtems_version_minor (0x0 - 0x7)
error: ExecutableInfo::findCoverageMap: rtems_version_revision
====
So there's something in rtems_version_revision() in spversion01/
that's making it unhappy.
> --
> Sebastian Huber, embedded brains GmbH
>
> Address : Dornierstr. 4, D-82178 Puchheim, Germany
> Phone : +49 89 189 47 41-16
> Fax : +49 89 189 47 41-09
> E-Mail : sebastian.huber at embedded-brains.de
> PGP : Public key available on request.
>
> Diese Nachricht ist keine geschäftliche Mitteilung im Sinne des EHUG.
>
> _______________________________________________
> devel mailing list
> devel at rtems.org
> http://lists.rtems.org/mailman/listinfo/devel
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.rtems.org/pipermail/devel/attachments/20190517/54b2913b/attachment-0002.html>
More information about the devel
mailing list