Coverage Analysis on rtems 4.10 of switch case structures

Thu Apr 30 12:54:54 UTC 2015

On April 30, 2015 7:41:29 AM CDT, Juanma Gomez <jmgomez at iaa.es> wrote:
>Hola Aitor, 
>
>    Sorry I sent the email from the incorrect email address. 
>
>We are actually using the sparc-rtems-gcov tool, but same result for
>sparc-elf-gcov.
>
>$ which sparc-elf-gcov 
>/opt/bcc-src-1.0.42/bin/sparc-elf-gcov
>$ which sparc-rtems-gcov 
>/opt/rtems-4.10/bin/sparc-rtems-gcov
>
>I have prepared an example in order to show you the results:

First this is one of the reasons why the community produced coverage results start with the actual instructions executed and map those back to the source using covoar and trusting the reports it directly generates.

Also.. If you are using covoar to generate gcov data, that generator has not been analyzed thoroughly. It was written as the bonus part of a GSOC project. We could see places the output didn't match our expectations but never investigated. If there are issues or integration improvements, please submit them.

Have you looked at the code generated? Is it generating four separate sets of code corresponding to each case? I have seen cases where a switch turns into a range check and table lookup. 

I have also seen native gcc cases where I can step through code even at the assembly level so I know it is executed but gcov doesn't report it executed. This eliminates a lot of the SPARC, target, cross, etc. issues and places the blame strictly on the gcov and GCC side.

>This will be our test suit, both functions are executeCommand and
>executeCommand2 are in a static library.
>    i = 1;
>    while(i < 10){
>
>        wrValue = executeCommand(i);
>        wrValue = executeCommand2(i);
>
>        i ++;
>    }
>
>
>executeCommand always use case 0 and default, on the other side
>executeCommand2 is complete covered.
>0 2 : int executeCommand(int cmd){ 11 : unsigned char a; 12 2 :
>switch(cmd){ 13 : 14 : case 0: 15 1 : a = 10; 16 1 : break; 17 : case
>1: 18 0 : a = 16; 19 0 : break; 20 : case 2: 21 0 : a = 21; 22 0 :
>break; 23 : case 3: 24 0 : a = 31; 25 0 : break; 26 : default: 27 1 : a
>= -1; 28 : break; 29 : 30 : } 31 : 32 2 : return a; 33 : 34 : } 35 : 36
>1 : int executeCommand2(int cmd){ 37 : unsigned char a; 38 : 39 1 : a =
>-1; 40 : 41 1 : if(cmd == 0) 42 0 : a = 10; 43 : 44 1 : if (cmd==1 ) 45
>1 : a = 16; 46 : 47 1 : if (cmd == 2) 48 1 : a = 21; 49 : 50 1 : if
>(cmd == 3) 51 1 : a = 31; 52 : 53 1 : return a; 54 : 55 : } I use
>covoar to convert the coverage data generated from TSIM-LEON3 to lcov
>data.
>
>simpleSW.syms
>
>_GLOBAL__I_65535_0_executeCommand
>/media/data/proyecto/Nomad/leon3-sw/coverage-test0/simpleLib/simpleSW.o
>executeCommand
>/media/data/proyecto/Nomad/leon3-sw/coverage-test0/simpleLib/simpleSW.o
>executeCommand2
>/media/data/proyecto/Nomad/leon3-sw/coverage-test0/simpleLib/simpleSW.o
>
>simpleSW.gcnos
>
>/media/data/proyecto/Nomad/leon3-sw/coverage-test0/simpleLib/simpleSW.gcno
>
>This is part of the output of gcov tool. It looks like it is actually
>receiving the data from TSIM or Covoar with that error. 
>
>Processing file:
>/media/data/proyecto/Nomad/leon3-sw/coverage-test0/simpleLib/simpleSW.gcno
>Computing uncovered ranges and branches
>Branch always taken found in executeCommand (0x4000151c - 0x4000151f)
>Branch always taken found in executeCommand2 (0x40001690 - 0x40001693)
>
>Could you provide me some information of how to verify the correctness
>of the coversion of the covoar tool and the tsim output. Maybe I could
>check the file a locate if the problem comes from tsim, covoar or
>sparc-rtems-gcov. 
>
>Thanks, Juanma.
>
>
>On 04/30/2015 10:22 AM, Aitor Viana Sanchez wrote:
>
>Hola Juanma,
>
>are you using sparc-gcov? I don’t remember whether I ever had a problem
>with switch-case while developing/testing the tool. If you’re using it
>and its a bug send me some files so I can reproduce the problem and fix
>the bug.
>If you’re using another gcov wrapper, have a look at sparc-gcov and see
>whether it fixes the problem.
>
>regards
>
>//avs
>
>On 29/04/15 10:46, Juanma Mail List wrote:
>
>Hello, 
>
>I am doing a coverage analysis for a software code which is based on
>RTEMS 4.10 and runs on a sparc-leon3 platform from Gaisler. I have
>problem with the coverage of switch-case structures. 
>
>The software mostly processes the commands that are send to it, and do
>the actions associated to the them. To do the coverage, I have added
>some wrappers and compiled the software as a static library.  
>
>And finally a set of test application do the call to the library and
>check the coverage.
>
>My framework uses:
>
>   - TSIM/LEON3 SPARC simulator, version 2.0.19 (professional version).
>  - covoar tool and leon3 simulation script  from  rtems-testing tools.
>    - sparc-rtems-gcov, lcov and genhtml.
>
>I am not able to cover the switch-case structures. The coverage shows
>that the execution always hit the first case of the switch-case. 
>
>Does anyone seen this behavior before? 
>
>Many thanks, Juanma.
>
>$ sparc-rtems-gcc -v 
>     Reading specs from /opt/rtems-4.10/lib/gcc/sparc-rtems/4.4.6/specs
>        Target: sparc-rtems
>Configured with: ../../gcc/configure --target=sparc-rtems --with-gnu-as
>--with-gnu-ld --with-newlib --verbose --enable-threads
>--enable-languages=c,c++ --disable-nls         --prefix=/opt/rtems-4.10
>--enable-version-specific-runtime-libs --with-system-zlib
>--disable-libstdcxx-pch --disable-win32-registry
>--without-included-gettext
>        Thread model: rtems
>        gcc version 4.4.6 (GCC) 
>
>
>$ sparc-rtems-gcov -v
>    gcov (GCC) 4.4.6
>    Copyright (C) 2010 Free Software Foundation, Inc.
>    This is free software; see the source for copying conditions.
>    There is NO warranty; not even for MERCHANTABILITY or 
>    FITNESS FOR A PARTICULAR PURPOSE.
>
>The tests are compiled wth the flags
>
>        CCFLAGS =  -fprofile-arcs -ftest-coverage
>        LDFLAGS = -lgcov -coverage
>
>
>My coverage script:
>
>./leon3 -c -l 120 simpletest.exe
>echo "Cleaning counters and create intial..."
>lcov --zerocounters --directory "$SOURCE_DIR" > log/gcov.log 2>&1
>lcov --capture --initial --directory "$SOURCE_DIR" --output-file 
>"$OUTPUT_DIR/coverage.info" --gcov-tool "$GCOV" >> log/gcov.log 2>&1
>
>covoar -v -C config $TESTSUITE >> log/gcov.log 2>&1
>lcov --no-checksum -d $SOURCE_DIR  --capture --output-file
>"$OUTPUT_DIR/coverage.info" --gcov-tool $GCOV --config-file 
>../rtemsNOMAD/sfs.lcovrc >> log/gcov.log 2>&1
>
>echo "Generating report $OUTPUT_DIR/$REPORTDIR"
>genhtml --highlight --legend --output-directory
>"$OUTPUT_DIR"/"$REPORTDIR"  coverage.info --config-file 
>../rtemsNOMAD/sfs.lcovrc >> log/gcov.log 2>&1
>
>
>
>
>
>_______________________________________________ users mailing list
>users at rtems.org http://lists.rtems.org/mailman/listinfo/users 
>
>
>
>This message and any attachments are intended for the use of the
>addressee or addressees only. The unauthorised disclosure, use,
>dissemination or copying (either in whole or in part) of its content is
>not permitted. If you received this message in error, please notify the
>sender and delete it from your system. Emails can be altered and their
>integrity cannot be guaranteed by the sender. Please consider the
>environment before printing this email. 
>
>-- Juan Manuel Gómez López Instituto de Astrofísica de Andalucía (IAA)
>Unidad de Desarrollo Instrumental y Tecnológico (UDIT) Glorieta de la
>Astronomía s/n 18008 Granada Teléfono centralita: +34 958 121 311
>Teléfono directo: +34 958 230 574 FAX: +34 958 814 530 

--joel