How can I use the covoar tool?

[Chris Johns]

On 17/5/19 5:34 pm, Sebastian Huber wrote:
> 
> I made some progress to try it out myself. The next obstacle was this:
> 
> rtems-test --rtems-tools=/build/rtems/5 --rtems-bsp=leon3-sis-cov --coverage
> --no-clean --log=leon3-test.log sparc-rtems5/c/leon3/testsuites/samples/hello.exe
> RTEMS Testing - Tester, 5.0.not_released
> [1/1] p:0 f:0 u:0 e:0 I:0 B:0 t:0 i:0 W:0 | sparc/leon3-sis: hello.exe
> Passed:        1
> Failed:        0
> User Input:    0
> Expected Fail: 0
> Indeterminate: 0
> Benchmark:     0
> Timeout:       0
> Invalid:       0
> Wrong Version: 0
> Wrong Build:   0
> Wrong Tools:   0
> ----------------
> Total:         1
> Average test time: 0:00:01.505144
> Testing time     : 0:00:01.505144
> error: symbol parser write failed for score
> 
> To figure out what went wrong, I removed the try/except block from
> tester/rt/coverage.py and got:
> 
> rtems-test --rtems-tools=/build/rtems/5 --rtems-bsp=leon3-sis-cov --coverage
> --no-clean --log=leon3-test.log sparc-rtems5/c/leon3/testsuites/samples/hello.exe
> RTEMS Testing - Tester, 5.0.not_released
> [1/1] p:0 f:0 u:0 e:0 I:0 B:0 t:0 i:0 W:0 | sparc/leon3-sis: hello.exe
> Passed:        1
> Failed:        0
> User Input:    0
> Expected Fail: 0
> Indeterminate: 0
> Benchmark:     0
> Timeout:       0
> Invalid:       0
> Wrong Version: 0
> Wrong Build:   0
> Wrong Tools:   0
> ----------------
> Total:         1
> Average test time: 0:00:01.504073
> Testing time     : 0:00:01.504073
> Traceback (most recent call last):
>   File "/build/rtems/5/share/rtems/tester/rt/cmd-test.py", line 42, in <module>
>     test.run(sys.argv[1:], command_path = base)
>   File "/build/rtems/5/share/rtems/tester/rt/test.py", line 387, in run
>     coverage_runner.run()
>   File "/build/rtems/5/share/rtems/tester/rt/coverage.py", line 401, in run
>     parser.write_ini(sset)
>   File "/build/rtems/5/share/rtems/tester/rt/coverage.py", line 293, in write_ini
>     object_files = [o for o in os.listdir(self.symbol_sets[sset]) if o[-1] == 'o']
> FileNotFoundError: [Errno 2] No such file or directory:
> '/sparc-rtems5/c/leon3/cpukit/score/src'
> 
> This is more useful error output from my point of view. 

A traceback like this is only briefly useful to a developer. The error should be
trapped and handled or checks make before the call so a useful user message
displayed. Swallowing a specific error like this seems wrong.

Also direct calls to file related `os` module calls should be removed. All calls
like this _must_ go through `path.py` in the toolkit to make them portable. The
code internally maintains paths as a POSIX path, ie /here/there, and this is
translated to a host path, ie Windows and and maybe a UTF Windows path, when
call the python calls.

> It seems you have to use an absolute path for the test directory.

Hmmm something smells here and worth a closer look. The `rtems-test` command is
designed to be run as installed or in the git repo and the base path or where it
is needs to be managed to find its files and the any dependent working files.

Chris