[SOCIS] Testing Framework

[Marcin Bujar]

Hi all,

I thought I would update you on my progress with this project. So far I
have implemented several tests to demonstrate how I see the new testsuite
should be written. They are all committed to a separate repository which at
this point I am not sure if you all have access to. I have tried to take
existing tests and apart from using the Unity calls I also separated the
tests to smaller functions if possible. The ones I implemented combine
several tests for the RTEMS clock API, the corresponding POSIX clock API,
and I also started writing a dynamic memory test. All the code compiles
correctly on the sparc sis target using waf and runs in the simulator.

I have also modified a Unity ruby script for generating test runners and
included it in the repository. It allows for the main function (in this
case the Init task) to be generated with all the written test*() function
calls being automatically added with their correct line numbers to be
executed at runtime. The script generates a separate .c file which is
linked with the other test .c files. The usage of this script is not
necessary and an Init task can be added manually.

I have also been preparing a short document/manual with all the necessary
information on how to convert the rest of the tests to this new format. It
is currently in a pdf format but I can also include it on the wiki if that
is preferrable once it is finished.

I hope my work will be useful and I welcome feedback once you can have a
look at the code in the repo (Amar knows how to access it).


Marcin


On 23 November 2013 16:05, Gedare Bloom <gedare at rtems.org> wrote:

> The article [1] might be relevant.
>
> -Gedare
>
> [1] http://www.sciencedirect.com/science/article/pii/S0167642312000317
>
> On Thu, Nov 21, 2013 at 9:20 AM, Marcin Bujar <marcin.bujar at gmail.com>
> wrote:
> > Hi Sebastian,
> >
> > Unfortunately the project was on hold for a while now due to
> administrative
> > reasons. I am just about get back into it so expect more updates soon.
> >
> > As to the thread safety of Unity I was not aware that was an explicit
> > requirement for a testing framework. If you are referring to accessing
> the
> > Unity struct then this is something I could serialize, or even better I
> will
> > check if somebody has implemented this already.
> >
> > Thanks for pointing it out.
> >
> >
> > Marcin
> >
> >
> > On 20 November 2013 16:11, Sebastian Huber
> > <sebastian.huber at embedded-brains.de> wrote:
> >>
> >> Hello Marcin,
> >>
> >> I just checked out the Unity sources and had a brief look at it.  I see
> >> one major problem with this framework.  It is not thread-safe, e.g. you
> have
> >> functions like this:
> >>
> >> https://github.com/ThrowTheSwitch/Unity/blob/master/src/unity.c
> >>
> >> [...]
> >> //-----------------------------------------------
> >> void UnityAssertNumbersWithin( const _U_SINT delta,
> >>                                const _U_SINT expected,
> >>                                const _U_SINT actual,
> >>                                const char* msg,
> >>                                const UNITY_LINE_TYPE lineNumber,
> >>                                const UNITY_DISPLAY_STYLE_T style)
> >> {
> >>     UNITY_SKIP_EXECUTION;
> >>
> >>     if ((style & UNITY_DISPLAY_RANGE_INT) == UNITY_DISPLAY_RANGE_INT)
> >>     {
> >>         if (actual > expected)
> >>           Unity.CurrentTestFailed = ((actual - expected) > delta);
> >>         else
> >>           Unity.CurrentTestFailed = ((expected - actual) > delta);
> >>     }
> >>     else
> >>     {
> >>         if ((_U_UINT)actual > (_U_UINT)expected)
> >>             Unity.CurrentTestFailed = ((_U_UINT)(actual - expected) >
> >> (_U_UINT)delta);
> >>         else
> >>             Unity.CurrentTestFailed = ((_U_UINT)(expected - actual) >
> >> (_U_UINT)delta);
> >>     }
> >>
> >>     if (Unity.CurrentTestFailed)
> >>     {
> >>         UnityTestResultsFailBegin(lineNumber);
> >>         UnityPrint(UnityStrDelta);
> >>         UnityPrintNumberByStyle(delta, style);
> >>         UnityPrint(UnityStrExpected);
> >>         UnityPrintNumberByStyle(expected, style);
> >>         UnityPrint(UnityStrWas);
> >>         UnityPrintNumberByStyle(actual, style);
> >>         UnityAddMsgIfSpecified(msg);
> >>         UNITY_FAIL_AND_BAIL;
> >>     }
> >> }
> >> [...]
> >>
> >> So now if multiple threads call this function the result is
> unpredictable.
> >>
> >> Thread-safety is a mandatory feature for a unit test framework for RTEMS
> >> from my point of view, so either we add this to Unity or search for
> >> something else.
> >>
> >>
> >> --
> >> Sebastian Huber, embedded brains GmbH
> >>
> >> Address : Dornierstr. 4, D-82178 Puchheim, Germany
> >> Phone   : +49 89 189 47 41-16
> >> Fax     : +49 89 189 47 41-09
> >> E-Mail  : sebastian.huber at embedded-brains.de
> >> PGP     : Public key available on request.
> >>
> >> Diese Nachricht ist keine geschäftliche Mitteilung im Sinne des EHUG.
> >
> >
> >
> > _______________________________________________
> > rtems-devel mailing list
> > rtems-devel at rtems.org
> > http://www.rtems.org/mailman/listinfo/rtems-devel
> >
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.rtems.org/pipermail/devel/attachments/20131221/7dba4bb2/attachment.html>