[RTEMS Project] #2918: Conversion to new test suite.

[RTEMS trac]

#2918: Conversion to new test suite.
-------------------------+--------------------------
 Reporter:  Amar Takhar  |       Owner:  Amar Takhar
     Type:  project      |      Status:  assigned
 Priority:  normal       |   Milestone:  Indefinite
Component:  General      |     Version:
 Severity:  normal       |  Resolution:
 Keywords:  testing,SoC  |
-------------------------+--------------------------

Comment (by Sebastian Huber):

 The Unity testing framework is by no means good enough for the RTEMS
 testsuite. It lacks some important features like thread-safety at a basic
 level.

 See also:

 https://lists.rtems.org/pipermail/devel/2013-November/004965.html

 Nothing changed in this respect since 2013.

 I would rather use something like the Google C++ Testing Framework,
 however, it uses to much resources for our low level targets.

 https://git.rtems.org/sebh/rtems-gtest.git/

 I don't think there exists a ready to use testing framework which meets
 our requirements. What are our requirements by the way? This should be
 addressed first.

 What is the specification input for the tests itself?

 How is the output produced at the device level?

 How it is ensured the output processing doesn't influence the test outcome
 in unexpected ways across targets?

 How is the output produced for test suite status consumers? Maybe have a
 look at
 [https://www.freebsd.org/cgi/man.cgi?query=xo&sektion=1&apropos=0&manpath=FreeBSD+11.0-RELEASE+and+Ports
 XO(1)].

 How is this synchronized with RTEMS qualification efforts?

 How it is guaranteed that we don't drop test cases accidentally during the
 conversion? Some tests are hard to understand. Do we keep the existing
 test suite as is?

 And so on.

 It is a several man month project if you want to do this right.

--
Ticket URL: <http://devel.rtems.org/ticket/2918#comment:6>
RTEMS Project <http://www.rtems.org/>
RTEMS Project