[PATCH 2/2] rtems-tools/../testing: Add configuration files for coverage analysis.
Cillian O'Donnell
cpodonnell8 at gmail.com
Fri Apr 27 07:28:58 UTC 2018
Yeah the config files are definitely a bit of a pick-a-mix. I only really
made changes in the .ini and symbolset.cfg. The rest is mostly yours I
guess Joel. You probably have the best idea of what to do with it. I think
Chris wants to squash it together, add his changes and make new patches.
On Thu, 26 Apr 2018, 23:55 Joel Sherrill, <joel at rtems.org> wrote:
> This looks like a mixed bag of stuff. The qemu cfg file could be by itself.
>
> Two files (Jiri's email and details on writing an Explanations.txt) could
> go
> in a (currently non-existent) document on running coverage.
>
> The .css file is a support file for the output.
>
> Not sure where the else fall but these seems like a handful of small
> patches
> rather than one larger patch.
>
> --joel
>
> On Wed, Apr 25, 2018 at 3:33 PM, Cillian O'Donnell <cpodonnell8 at gmail.com>
> wrote:
>
>> Co-author: Krzysztof Miesowicz <krzysztof.miesowicz at gmail.com>
>> Co-author: Vijay Kumar Banerjee <vijaykumar9597 at gmail.com>
>> ---
>> tester/rtems/testing/bsps/leon3_qemu.ini | 52 ++++++
>> tester/rtems/testing/coverage.ini | 49 +++++
>> tester/rtems/testing/coverage/Categories.txt | 19 ++
>> tester/rtems/testing/coverage/Explanations.txt | 35 ++++
>> .../coverage/SPARC-Annul-Slot-Explanation.txt | 56 ++++++
>> tester/rtems/testing/coverage/style.css | 197
>> +++++++++++++++++++++
>> tester/rtems/testing/coverage/symbol_sets.cfg | 3 +
>> tester/rtems/testing/qemu.cfg | 14 +-
>> 8 files changed, 424 insertions(+), 1 deletion(-)
>> create mode 100644 tester/rtems/testing/bsps/leon3_qemu.ini
>> create mode 100644 tester/rtems/testing/coverage.ini
>> create mode 100644 tester/rtems/testing/coverage/Categories.txt
>> create mode 100644 tester/rtems/testing/coverage/Explanations.txt
>> create mode 100644
>> tester/rtems/testing/coverage/SPARC-Annul-Slot-Explanation.txt
>> create mode 100644 tester/rtems/testing/coverage/style.css
>> create mode 100644 tester/rtems/testing/coverage/symbol_sets.cfg
>>
>> diff --git a/tester/rtems/testing/bsps/leon3_qemu.ini
>> b/tester/rtems/testing/bsps/leon3_qemu.ini
>> new file mode 100644
>> index 0000000..9888008
>> --- /dev/null
>> +++ b/tester/rtems/testing/bsps/leon3_qemu.ini
>> @@ -0,0 +1,52 @@
>> +#
>> +# RTEMS Tools Project (http://www.rtems.org/)
>> +# Copyright 2010-2014 Chris Johns (chrisj at rtems.org)
>> +# All rights reserved.
>> +#
>> +# This file is part of the RTEMS Tools package in 'rtems-tools'.
>> +#
>> +# Redistribution and use in source and binary forms, with or without
>> +# modification, are permitted provided that the following conditions are
>> met:
>> +#
>> +# 1. Redistributions of source code must retain the above copyright
>> notice,
>> +# this list of conditions and the following disclaimer.
>> +#
>> +# 2. Redistributions in binary form must reproduce the above copyright
>> notice,
>> +# this list of conditions and the following disclaimer in the
>> documentation
>> +# and/or other materials provided with the distribution.
>> +#
>> +# THIS SOFTWARE IS PROVIDED BY THE COPYRIGHT HOLDERS AND CONTRIBUTORS
>> "AS IS"
>> +# AND ANY EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT LIMITED TO,
>> THE
>> +# IMPLIED WARRANTIES OF MERCHANTABILITY AND FITNESS FOR A PARTICULAR
>> PURPOSE
>> +# ARE DISCLAIMED. IN NO EVENT SHALL THE COPYRIGHT HOLDER OR CONTRIBUTORS
>> BE
>> +# LIABLE FOR ANY DIRECT, INDIRECT, INCIDENTAL, SPECIAL, EXEMPLARY, OR
>> +# CONSEQUENTIAL DAMAGES (INCLUDING, BUT NOT LIMITED TO, PROCUREMENT OF
>> +# SUBSTITUTE GOODS OR SERVICES; LOSS OF USE, DATA, OR PROFITS; OR
>> BUSINESS
>> +# INTERRUPTION) HOWEVER CAUSED AND ON ANY THEORY OF LIABILITY, WHETHER IN
>> +# CONTRACT, STRICT LIABILITY, OR TORT (INCLUDING NEGLIGENCE OR OTHERWISE)
>> +# ARISING IN ANY WAY OUT OF THE USE OF THIS SOFTWARE, EVEN IF ADVISED OF
>> THE
>> +# POSSIBILITY OF SUCH DAMAGE.
>> +#
>> +
>> +#
>> +# The Leon 3 QEMU BSP
>> +#
>> +[global]
>> +coverage_supported = 1
>> +bsp = leon3_qemu
>> +[leon3_qemu]
>> +bsp = leon3_qemu
>> +arch = sparc
>> +tester = %{_rtscripts}/qemu.cfg
>> +bsp_qemu_opts = %{qemu_opts_base} -M leon3_generic
>> +
>> +
>> +[coverage]
>> +format = QEMU
>> +target = sparc-rtems5
>> +explanations = %{_rtscripts}/coverage/Explanations.txt
>> +coverage_extension = .cov
>> +gcnos_file = %{_rtscripts}/coverage/rtems.gcnos
>> +executable_extension = exe
>> +project_name = RTEMS-5
>> +report_format = html
>> diff --git a/tester/rtems/testing/coverage.ini
>> b/tester/rtems/testing/coverage.ini
>> new file mode 100644
>> index 0000000..c1eef2e
>> --- /dev/null
>> +++ b/tester/rtems/testing/coverage.ini
>> @@ -0,0 +1,49 @@
>> +#
>> +# RTEMS Tools Project (http://www.rtems.org/)
>> +# Copyright 2010-2014 Chris Johns (chrisj at rtems.org)
>> +# All rights reserved.
>> +#
>> +# This file is part of the RTEMS Tools package in 'rtems-tools'.
>> +#
>> +# Redistribution and use in source and binary forms, with or without
>> +# modification, are permitted provided that the following conditions are
>> met:
>> +#
>> +# 1. Redistributions of source code must retain the above copyright
>> notice,
>> +# this list of conditions and the following disclaimer.
>> +#
>> +# 2. Redistributions in binary form must reproduce the above copyright
>> notice,
>> +# this list of conditions and the following disclaimer in the
>> documentation
>> +# and/or other materials provided with the distribution.
>> +#
>> +# THIS SOFTWARE IS PROVIDED BY THE COPYRIGHT HOLDERS AND CONTRIBUTORS
>> "AS IS"
>> +# AND ANY EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT LIMITED TO,
>> THE
>> +# IMPLIED WARRANTIES OF MERCHANTABILITY AND FITNESS FOR A PARTICULAR
>> PURPOSE
>> +# ARE DISCLAIMED. IN NO EVENT SHALL THE COPYRIGHT HOLDER OR CONTRIBUTORS
>> BE
>> +# LIABLE FOR ANY DIRECT, INDIRECT, INCIDENTAL, SPECIAL, EXEMPLARY, OR
>> +# CONSEQUENTIAL DAMAGES (INCLUDING, BUT NOT LIMITED TO, PROCUREMENT OF
>> +# SUBSTITUTE GOODS OR SERVICES; LOSS OF USE, DATA, OR PROFITS; OR
>> BUSINESS
>> +# INTERRUPTION) HOWEVER CAUSED AND ON ANY THEORY OF LIABILITY, WHETHER IN
>> +# CONTRACT, STRICT LIABILITY, OR TORT (INCLUDING NEGLIGENCE OR OTHERWISE)
>> +# ARISING IN ANY WAY OUT OF THE USE OF THIS SOFTWARE, EVEN IF ADVISED OF
>> THE
>> +# POSSIBILITY OF SUCH DAMAGE.
>> +#
>> +
>> +#
>> +# All paths in defaults must be Unix format. Do not store any Windows
>> format
>> +# paths in the defaults.
>> +#
>> +# Every entry must describe the type of checking a host must pass.
>> +#
>> +# Records:
>> +# key: type, attribute, value
>> +# type : none, dir, exe, triplet
>> +# attribute: none, required, optional
>> +# value : 'single line', '''multi line'''
>> +#
>> +
>> +#
>> +# Global defaults
>> +#
>> +[global]
>> +__covoar = covoar
>> +_coverage = 1
>> diff --git a/tester/rtems/testing/coverage/Categories.txt
>> b/tester/rtems/testing/coverage/Categories.txt
>> new file mode 100644
>> index 0000000..e19a456
>> --- /dev/null
>> +++ b/tester/rtems/testing/coverage/Categories.txt
>> @@ -0,0 +1,19 @@
>> +This is the list of Explanation Categories used when analyzing RTEMS
>> +Coverage report. By using standard categories, the table filter on
>> +the web site works better.
>> +
>> +Simple Test Case
>> +
>> +Hard Test Tase
>> +
>> +Uncalled Routine
>> +
>> +Interrupt Critical Section
>> +
>> +Simple Error Case
>> +
>> +Hard Error Case
>> +
>> +Allocation Error
>> +
>> +Bharath Suri
>> diff --git a/tester/rtems/testing/coverage/Explanations.txt
>> b/tester/rtems/testing/coverage/Explanations.txt
>> new file mode 100644
>> index 0000000..a5917f6
>> --- /dev/null
>> +++ b/tester/rtems/testing/coverage/Explanations.txt
>> @@ -0,0 +1,35 @@
>> +schedulerpriorityyield.c:47
>> +Simple Test Case
>> +Branch Never Taken
>> +New test where there is more than one thread at a priority with the
>> +executing thread being non-preemptive. Create a higher priority thread
>> +and then yield.
>> +
>> + init task at priority 2, non-preemptive
>> + create task at priority 2
>> + create task at priority 1
>> + yield
>> ++++
>> +
>> +schedulerpriorityyield.c:51
>> +Simple Test Case
>> +Branch Always Taken
>> +New test where only one thread at a priority (non-preemptive), create a
>> +thread at higher priority, then yield.
>> +
>> + init task at priority 2, non-preemptive
>> + create task at priority 1
>> + yield
>> ++++
>> +
>> +schedulerpriorityyield.c:52
>> +Simple Test Case
>> +Not Executed
>> +Same test case as schedulerpriorityyield.c:51
>> ++++
>> +
>> +coremsg.c:86
>> +Simple Test Case
>> +We need to request enough messages of a certain size that the math
>> +overflows to less than a single message.
>> ++++
>> diff --git
>> a/tester/rtems/testing/coverage/SPARC-Annul-Slot-Explanation.txt
>> b/tester/rtems/testing/coverage/SPARC-Annul-Slot-Explanation.txt
>> new file mode 100644
>> index 0000000..ef740d3
>> --- /dev/null
>> +++ b/tester/rtems/testing/coverage/SPARC-Annul-Slot-Explanation.txt
>> @@ -0,0 +1,56 @@
>> +The SPARC assembly is often hard to understand because a single
>> +instruction will show up as not executed. The instructions before
>> +and after it will be marked as executed. The instruction before
>> +the one not executed should be a "bxx,a" instruction which means
>> +that the instruction following the branch instruction is executed
>> +ONLY if the branch is taken. Otherwise it is "annulled" or skipped.
>> +
>> +So when you see these cases, it means the branch was NOT taken.
>> +
>> +===================================================================
>> +Subject: <offlist> annul slot explanation
>> +From: Jiri Gaisler <jiri at gaisler.com>
>> +Date: Wed, 3 Jun 2009 14:57:48 -0500
>> +To: Joel Sherrill <Joel.Sherrill at OARcorp.com>
>> +
>> +
>> +Joel Sherrill wrote:
>> +> > Hi,
>> +> >
>> +> > I am trying to look at more coverage cases and
>> +> > wanted to make sure I am reading things correctly.
>> +> >
>> +> > The code in question is:
>> +> >
>> +> >
>> +> > if ( the_thread->current_priority > interested_priority )
>> +> > 200fd00: d8 00 e0 14 ld [ %g3 + 0x14 ], %o4
>> +> > 200fd04: 80 a3 00 04 cmp %o4, %g4
>> +> > 200fd08: 38 80 00 1c bgu,a 200fd78 <killinfo+0x224>
>> +> > 200fd0c: 98 10 00 04 mov %g4,
>> +> > %o4 <== NOT EXECUTED
>> +> >
>> +> > /*
>> +> > * If this thread is not interested, then go on to the next
>> thread.
>> +> > */
>> +> >
>> +> > api = the_thread->API_Extensions[ THREAD_API_POSIX ];
>> +> > 200fd10: d4 00 e1 6c ld [ %g3 + 0x16c ], %o2
>> +> >
>> +> > Am I correct in interpreting this as meaning 0x200fd0c
>> +> > is not executed because the bgu,a is never taken. And it
>> +> > is not executed as part of falling through.
>> +
>> +Yes, this is correct. The branch delay slot is only executed
>> +when the branch is taken.
>> +
>> +Jiri.
>> +
>> +> >
>> +> > So in this case we need a test where the "if" condition
>> +> > is true if I am reading things correctly.
>> +> >
>> +> > Thanks. There are a number of these 4 byte cases which
>> +> > are probably easy to hit if I read the code correctly.
>> +> >
>> +> >
>> diff --git a/tester/rtems/testing/coverage/style.css
>> b/tester/rtems/testing/coverage/style.css
>> new file mode 100644
>> index 0000000..c715518
>> --- /dev/null
>> +++ b/tester/rtems/testing/coverage/style.css
>> @@ -0,0 +1,197 @@
>> +body {
>> + background: rgb(253,253,253);
>> + color: rgb(0,0,0);
>> + font-family: helvetica, sans-serif;
>> + font-size: 1em;
>> + line-height: 1.4;
>> + margin: 5px, 5px, 5px, 5px;
>> + padding: 0;
>> +}
>> +
>> +a:link {
>> + color: rgb(180, 50, 50);
>> + font-family: helvetica, sans-serif;
>> + font-size: 1.0em;
>> +}
>> +
>> +a:visited {
>> + color: purple;
>> + font-family: helvetica, sans-serif;
>> + font-size: 1.0em;
>> +}
>> +
>> +a:hover {
>> + color: rgb(0, 0, 0);
>> + font-family: helvetica, sans-serif;
>> + font-size: 1.0em;
>> +}
>> +
>> +a:active {
>> + color: red;
>> + font-family: helvetica, sans-serif;
>> + font-size: 1.0em;
>> +}
>> +
>> +.heading {
>> + background: rgb(250,250,250);
>> + background-image: url("http://www.rtems.org/logos/rtems_logo.jpg");
>> + background-repeat: no-repeat;
>> + color: rgb(55,55,55);
>> + font-size: 1.5em;
>> + height: 140px;
>> + padding-top: 20px;
>> + padding-left: 300px;
>> +}
>> +
>> +.heading-title {
>> + text-align: center;
>> + color: rgb(0,0,0);
>> + font-size: 0.9em;
>> + font-weight: bold;
>> + padding-top: 5px;
>> + padding-left: 0px;
>> + text-align: center;
>> + width: 100%;
>> +}
>> +
>> +.datetime {
>> + color: rgb(55,55,55);
>> + font-size: 0.8em;
>> + padding-top: 5px;
>> + padding-left: 0px;
>> + text-align: center;
>> + width: 100%;
>> +}
>> +
>> +.info {
>> + color: rgb(55,55,55);
>> + font-size: 0.6em;
>> + padding-top: 5px;
>> + padding-left: 00px;
>> + text-align: center;
>> + width: 100%;
>> +}
>> +
>> +.stats-table {
>> + background: rgb(225,225,225);
>> + font-size: 0.9em;
>> + border: 1px solid rgb(200, 200, 200);
>> + padding: 0;
>> + margin-top: 3px;
>> + margin-left: 10px;
>> + width: 70%;
>> +}
>> +
>> +.stats-table-target {
>> + background: rgb(243,243,243);
>> + font-size: 1.2em;
>> + padding-left: 10px;
>> + text-align: left;
>> +}
>> +
>> +.stats-target-results {
>> + background: rgb(243,243,243);
>> + font-size: 0.9em;
>> + text-align: right;
>> + padding-right: 10px;
>> +}
>> +
>> +.stats-target-good {
>> + background: rgb(30,230,30);
>> + font-size: 0.9em;
>> + text-align: right;
>> + padding-right: 10px;
>> +}
>> +
>> +.stats-target-good {
>> + background: rgb(50,180,50);
>> + color: rgb(230,230,230);
>> + font-size: 0.9em;
>> + text-align: center;
>> + padding-right: 10px;
>> +}
>> +
>> +.stats-target-bad {
>> + background: rgb(180,50,50);
>> + color: rgb(230,230,230);
>> + font-size: 0.9em;
>> + text-align: center;
>> + padding-right: 10px;
>> +}
>> +
>> +.stats-table-top {
>> + background: rgb(243,243,243);
>> + color: rgb(0,0,0);
>> + font-size: 0.9em;
>> + padding-left: 2px;
>> +}
>> +
>> +.stats-table-row {
>> + background: rgb(253,253,253);
>> + font-size: 0.9em;
>> + padding: 1px;
>> + text-align: right;
>> +}
>> +
>> +.error-table {
>> + font-size: 0.9em;
>> + border: 1px solid rgb(200, 200, 200);
>> + padding: 0;
>> + margin-left: 10px;
>> + width: 96%;
>> +}
>> +
>> +.error-table-top {
>> + background: rgb(225,225,225);
>> + color: rgb(0,0,0);
>> + font-size: 0.9em;
>> + padding-left: 2px;
>> +}
>> +
>> +.error-table-on {
>> + background: rgb(225,225,225);
>> + font-size: 0.9em;
>> + padding-left: 2px;
>> +}
>> +
>> +.error-table-off {
>> + background: rgb(253,253,253);
>> + font-size: 0.9em;
>> + padding-left: 2px;
>> +}
>> +
>> +.error-table-dups {
>> + text-align: right;
>> + padding-right: 2px;
>> +}
>> +
>> +.error-table-error {
>> + background: rgb(255,150,150);
>> + font-size: 0.9em;
>> + padding-left: 2px;
>> +}
>> +
>> +.error-table-warning {
>> + font-size: 0.9em;
>> + padding-left: 2px;
>> +}
>> +
>> +.navbar {
>> + margin-left: auto;
>> + margin-right: auto;
>> + margin-top: 10px;
>> + width: 40%;
>> +}
>> +th.table-sortable {
>> + background-image:url("unsorted.gif");
>> + cursor: pointer;
>> + background-position: center left;
>> + background-repeat: no-repeat;
>> + padding-left: 15px;
>> +}
>> +th.table-sorted-asc {
>> + background-image:url("descending.gif");
>> +}
>> +th.table-sorted-desc {
>> + background-image:url("ascending.gif");
>> +}
>> \ No newline at end of file
>> diff --git a/tester/rtems/testing/coverage/symbol_sets.cfg
>> b/tester/rtems/testing/coverage/symbol_sets.cfg
>> new file mode 100644
>> index 0000000..7f62cce
>> --- /dev/null
>> +++ b/tester/rtems/testing/coverage/symbol_sets.cfg
>> @@ -0,0 +1,3 @@
>> +symbolset:
>> + name=score
>> + lib=sparc-rtems5/c/leon3/cpukit/score/libscore.a
>> diff --git a/tester/rtems/testing/qemu.cfg b/tester/rtems/testing/qemu.cfg
>> index db5b6b2..7a65ddd 100644
>> --- a/tester/rtems/testing/qemu.cfg
>> +++ b/tester/rtems/testing/qemu.cfg
>> @@ -64,6 +64,18 @@
>> %define qemu_opts %{bsp_qemu_opts}
>>
>> #
>> +# Coverage analysis
>> +#
>> +%define coverage_arg %{nil}
>> +%if %{defined _coverage}
>> + %if %{coverage_supported}
>> + %define coverage_arg -exec-trace %{test_executable}.cov
>> + %else
>> + %error "Coverage analysis unsupported for %{bsp}"
>> + %endif
>> +%endif
>> +
>> +#
>> # Executable
>> #
>> -%execute %{qemu_cmd} %{qemu_opts} -kernel %{test_executable}
>> +%execute %{qemu_cmd} %{qemu_opts} -kernel %{test_executable}
>> %{coverage_arg}
>> --
>> 2.7.4
>>
>> _______________________________________________
>> devel mailing list
>> devel at rtems.org
>> http://lists.rtems.org/mailman/listinfo/devel
>>
>
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.rtems.org/pipermail/devel/attachments/20180427/8b80e7a9/attachment-0002.html>
More information about the devel
mailing list