<html>
<head>
<meta content="text/html; charset=UTF-8" http-equiv="Content-Type">
</head>
<body bgcolor="#FFFFFF" text="#000000">
Hopefully Chris Johns will pipe up and add more details.<br>
In particular, where his rtems-test is. I think it can be used<br>
on the beagle board and some gdb simulators now.<br>
<br>
<div class="moz-cite-prefix">On 5/7/2014 4:18 PM, Krzysztof
Mięsowicz wrote:<br>
</div>
<blockquote
cite="mid:CANNpagoeboDTYKD3r9fxq1EF_+aALO8mZ_UjNHseKkdmk8DPCQ@mail.gmail.com"
type="cite">
<div dir="ltr">Hi,
<div><br>
</div>
<div>My name is Krzysztof Mięsowicz. I am interested in
participating again in SOCIS this year and I am writing to you
in hope that you could provide me some more details about
possible projects related to new rtems-test infrastructure. </div>
<div><br>
</div>
<div>As dr Joel wrote on Rtems-Users facebook group, there are
some possible improvements: (cited)</div>
<div><br>
</div>
<div>
<blockquote style="margin:0px 0px 0px
0.8ex;border-left-width:1px;border-left-color:rgb(204,204,204);border-left-style:solid;padding-left:1ex"
class="gmail_quote">
This year we want to encourage the SOCIS students to focus
on improvements to our new rtems-test Python-based
framework. This<br>
will include:<br>
+ adding more BSPs (simulators and real boards) to
rtems-test<br>
</blockquote>
</div>
</div>
</blockquote>
<br>
Chris will have to point us to his rtems-test repository. I can't
seem to <br>
locate it. It is in Python and drives real hardware debuggers<br>
and simulators using the GDB/MI. <br>
<br>
rtems-testing/sim-scripts supports more simulators but does not
support<br>
hardware debuggers. rtems-test supports a small subset of the
simulators<br>
supported by rtems-testing/sim-scripts. rtems-test needs to be
worked<br>
on so it can replace all uses of sim-scripts and obsolete them.<br>
<br>
<blockquote
cite="mid:CANNpagoeboDTYKD3r9fxq1EF_+aALO8mZ_UjNHseKkdmk8DPCQ@mail.gmail.com"
type="cite">
<div dir="ltr">
<div>
<blockquote style="margin:0px 0px 0px
0.8ex;border-left-width:1px;border-left-color:rgb(204,204,204);border-left-style:solid;padding-left:1ex"
class="gmail_quote">+ building more simulators with the
RTEMS Source <br>
- if successful, sim-scripts should be obsolete<br>
</blockquote>
</div>
</div>
</blockquote>
That should be RTEMS Source Builder.
<a class="moz-txt-link-freetext" href="http://git.rtems.org/rtems-source-builder/">http://git.rtems.org/rtems-source-builder/</a><br>
RSB builds the basic tools and a couple of simulator configurations.<br>
RSB and rtems-tools work together to provide a framework to manage <br>
build instructions and patch sets for RTEMS tools. We want to expand<br>
it so it can build every tool one needs cross. Simulators are the
next<br>
step. <br>
<br>
Eventually, we would like to be able to even build add-on libraries<br>
to target a specific BSP. But that is longer down the road. If
someone<br>
gets to it sooner, great!<br>
<br>
RSB needs to be augmented to be able to build all the simulators <br>
external to gdb that we currently use. This may involve some odd <br>
cases like say an old qemu version plus a patch to do coldfire. <br>
That would give a baseline to upgrade against.<br>
<br>
<blockquote
cite="mid:CANNpagoeboDTYKD3r9fxq1EF_+aALO8mZ_UjNHseKkdmk8DPCQ@mail.gmail.com"
type="cite">
<div dir="ltr">
<div>
<blockquote style="margin:0px 0px 0px
0.8ex;border-left-width:1px;border-left-color:rgb(204,204,204);border-left-style:solid;padding-left:1ex"
class="gmail_quote">+ integrating coverage runs into
rtems-test<br>
</blockquote>
</div>
</div>
</blockquote>
<br>
rtems-testing/covoar (C++) and rtems-testing/rtems-coverage (shell)<br>
together do RTEMS test runs for coverage and generate the coverage<br>
reports on <a class="moz-txt-link-freetext" href="http://www.rtems.org/ftp/pub/rtems/people/joel/coverage/">http://www.rtems.org/ftp/pub/rtems/people/joel/coverage/</a><br>
<br>
The rtems-test tool needs to be able to do coverage runs. It needs
to know<br>
if a target supports coverage, how to turn on coverage for a target,
how<br>
to run the report generator, etc.<br>
<br>
<br>
<blockquote
cite="mid:CANNpagoeboDTYKD3r9fxq1EF_+aALO8mZ_UjNHseKkdmk8DPCQ@mail.gmail.com"
type="cite">
<div dir="ltr">
<div>
<blockquote style="margin:0px 0px 0px
0.8ex;border-left-width:1px;border-left-color:rgb(204,204,204);border-left-style:solid;padding-left:1ex"
class="gmail_quote">+ improvements to the coverage test
reporting<br>
- reporting should be on "module" level. Too coarse now.<br>
</blockquote>
</div>
</div>
</blockquote>
The report generator itself needs a refresh. Right now, it reports
on two<br>
"lumps" of code -- core and developmental. Read "critical core" and
<br>
"the kitchen sink". :)<br>
<br>
Ideally, the coverage reports should be generated based on modules<br>
or source directories in cpukit/. Then we can get the coverage on <br>
score/ separate from shell/, fatfs/, or imfs. What we have seen is
that<br>
each time we add a module to the "kitchen sink" set, the covered<br>
percentage drops because the newly added module needs improvement<br>
in test coverage.<br>
<br>
covoar can generate .gcov output but no one has ever validated it <br>
against "truth" and seen if gcov, lcov, etc. can produce trustworth<br>
reports with it.
<blockquote
cite="mid:CANNpagoeboDTYKD3r9fxq1EF_+aALO8mZ_UjNHseKkdmk8DPCQ@mail.gmail.com"
type="cite">
<div dir="ltr">
<div>
<blockquote style="margin:0px 0px 0px
0.8ex;border-left-width:1px;border-left-color:rgb(204,204,204);border-left-style:solid;padding-left:1ex"
class="gmail_quote">- some scripts could be migrated to
Python</blockquote>
<div><br>
</div>
</div>
</div>
</blockquote>
The older code is in bash. Migrating it to Python as it is reworked
is<br>
desirable. <br>
<blockquote
cite="mid:CANNpagoeboDTYKD3r9fxq1EF_+aALO8mZ_UjNHseKkdmk8DPCQ@mail.gmail.com"
type="cite">
<div dir="ltr">
<div>
<div>I wonder which of these are of highest priorities and
what will it mean exactly. Could you provide more details
about these ideas and maybe point some starting points to
get more familiar and prepare better proposal?</div>
<div><br>
</div>
</div>
</div>
</blockquote>
Everything is high priority. :)<br>
<br>
I guess IMO I would like to see one BSP/simulator pushed through the<br>
entire software stack, reporting improved, and shell transitioned to<br>
Python as it was convenient/expedient. Adding other simulators<br>
to the RSB and rtems-test/ is a bonus. So if you want to assume we<br>
get one student from SOCIS, that would be my suggestion.<br>
<br>
My thinking is that if SOCIS gives us two or more students, one
could <br>
easily start at the rtems-test/ level integrating coverage runs as
they <br>
are now. Another student could focus on the coverage report
generation. <br>
<br>
Transitioning more simulators from sim-scripts/ to rtems-test/<br>
and adding building more simulators to the RSB could be added <br>
easily. Each simulator instance is more or less independent so<br>
any number of students could work off a check list.<br>
<br>
If we get two or more students, the problem can be tackled from <br>
multiple ends. <br>
<blockquote
cite="mid:CANNpagoeboDTYKD3r9fxq1EF_+aALO8mZ_UjNHseKkdmk8DPCQ@mail.gmail.com"
type="cite">
<div dir="ltr">
<div>
<div>Regards,</div>
<div>Krzysztof </div>
</div>
</div>
</blockquote>
<br>
<pre class="moz-signature" cols="72">--
Joel Sherrill, Ph.D. Director of Research & Development
<a class="moz-txt-link-abbreviated" href="mailto:joel.sherrill@OARcorp.com">joel.sherrill@OARcorp.com</a> On-Line Applications Research
Ask me about RTEMS: a free RTOS Huntsville AL 35805
Support Available (256) 722-9985</pre>
</body>
</html>