[ESA summer of code, 2014] RTEMS testing

[Joel Sherrill]

On May 13, 2014 10:22 AM, Kumar Amit Mehta <gmate.amit at gmail.com> wrote:
>
> On Tue, May 13, 2014 at 06:21:08PM +0300, Kumar Amit Mehta wrote:
> > Hi,
> >
> > I've some queries regarding the testing framework for rtems.
> >
> > 1: There seem to be two different set of framework and set of tests.
> > First one [1] is part of rtems package itself and the second one [2]
> > seem to have a different git repository and is written in Python.
> > How are these two different ?

There is currently one set of tests and those are in the RTEMS tree.

There are two sets of scripts to run those tests. One is rtems-testing/sim-scripts in bash and the other next generation is rtems-test which is in Python. The newer code in Python drives gdb and can support automated testing on real hardware. The bash scripts just drove simulators but supported more bsps than the newer Python code. We are in transition and this is part of the project.

> > 2: I ran the simple hello world test case on the simulator, then
> > I also ran all those ~490 test cases that were part of rtems-tools
> > tests[2].

These are really from the RTEMS tree in the testsuites directory.

According to the information on the ESA summer of code page
> > of RTEMS [4], RTEMS testing using 'C' and 'C++' (seems like cmock is
> > the favorite right now [3]) is a prospective candidate for ESA summer
> > of code. I just want to be sure that It is still the case, Therefore,
> > CC'd mentors for RTEMS testing [4] here.

That is another forward looming next generation effort and Amar will have to comment on what remains to be done.

There is plenty of work to do getting the current suite in its current state automated on as many bsps as possible. This will work with whatever suite changes are made.

> >
> > 3: Is there a guide or link on running rtems test suite [1] (I ran
> > the modified hello world program on the simulator, but was wondering,
> > If those standalone test cases can be run as a batch, one after
> > another)

Using rtems-testing/sim-scripts, it is as simple as sis *.exe. The rtems-test scripts are also easy but I can't do the command from my phone.

So rtems-test needs maturing to replace sim-scripts.

Also there is instruction coverage testing support and that needs to be integrated into rtems-test and some improvements made. Those have been discussed on rtems-devel on ESA SOCIS threads.

> > [1] http://git.rtems.org/rtems/tree/testsuites
> > [2] git://git.rtems.org/rtems-tools.git
> > [3] http://www.rtems.org/wiki/index.php/TestingReview
> > [4] http://www.rtems.org/wiki/index.php/RTEMS_Testing
>
> Resending the mail, as the delivery to one of the mentor(Amar) failed
> due to typo mistake from my side. My apologies.
>
> Thanks,
> Kumar
> _______________________________________________
> rtems-users mailing list
> rtems-users at rtems.org
> http://www.rtems.org/mailman/listinfo/rtems-users
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.rtems.org/pipermail/users/attachments/20140513/86777541/attachment-0001.html>