[PATCH 1/1] eng: Update test framework chapter

Sebastian Huber sebastian.huber at embedded-brains.de
Tue Jul 21 14:49:46 UTC 2020

Document the dynamic text fixtures, utility functions, and the interrupt
test support.

Update #3199.
 eng/index.rst                 |   4 +-
 eng/test-framework.rst        | 185 ++++++++++++++++++++++++++++++++--
 images/eng/interrupt-test.odg | Bin 0 -> 14829 bytes
 images/eng/interrupt-test.pdf | Bin 0 -> 14153 bytes
 images/eng/interrupt-test.png | Bin 0 -> 75272 bytes
 5 files changed, 178 insertions(+), 11 deletions(-)
 create mode 100644 images/eng/interrupt-test.odg
 create mode 100644 images/eng/interrupt-test.pdf
 create mode 100644 images/eng/interrupt-test.png

diff --git a/eng/index.rst b/eng/index.rst
index 8f91c5e..f6b02ec 100644
--- a/eng/index.rst
+++ b/eng/index.rst
@@ -11,8 +11,8 @@ RTEMS Software Engineering (|version|)
 .. topic:: Copyrights and License
-    | |copy| 2018, 2019 embedded brains GmbH
-    | |copy| 2018, 2019 Sebastian Huber
+    | |copy| 2018, 2020 embedded brains GmbH
+    | |copy| 2018, 2020 Sebastian Huber
     | |copy| 1988, 2015 On-Line Applications Research Corporation (OAR)
     .. include:: ../common/license.rst
diff --git a/eng/test-framework.rst b/eng/test-framework.rst
index b6411b5..582718d 100644
--- a/eng/test-framework.rst
+++ b/eng/test-framework.rst
@@ -1,7 +1,7 @@
 .. SPDX-License-Identifier: CC-BY-SA-4.0
-.. Copyright (C) 2018, 2019 embedded brains GmbH
-.. Copyright (C) 2018, 2019 Sebastian Huber
+.. Copyright (C) 2018, 2020 embedded brains GmbH
+.. Copyright (C) 2018, 2020 Sebastian Huber
 Software Test Framework
@@ -144,13 +144,41 @@ macro followed by a function body:
 The test case `name` must be a valid C designator.  The test case names must be
 unique within the test suite.  The `fixture` must point to a statically
-initialized read-only object of type `T_fixture`.  The test fixture
-provides methods to setup, stop and tear down a test case.  A context is passed
-to the methods.  The initial context is defined by the read-only fixture
-object.  The context can be obtained by the `T_fixture_context()`
-function.  It can be set within the scope of one test case by the
-`T_set_fixture_context()` function.  This can be used for example to
-dynamically allocate a test environment in the setup method.
+initialized read-only object of type `T_fixture`.
+.. code-block:: c
+    typedef struct T_fixture {
+        void (*setup)(void *context);
+        void (*stop)(void *context);
+        void (*teardown)(void *context);
+        void (*scope)(void *context, char *buffer, size_t size);
+        void *initial_context;
+    } T_fixture;
+The test fixture provides methods to setup, stop, and teardown a test case as
+well as to give the scope for log messags.  A context is passed to each of the
+methods.  The initial context is defined by the read-only fixture object.  The
+context can be obtained by the `T_fixture_context()` function.  It can be set
+within the scope of one test case by the `T_set_fixture_context()` function.
+This can be used for example to dynamically allocate a test environment in the
+setup method.
+The test case fixtures of a test case are organized as a stack.  Fixtures can
+be dynamically added to a test case and removed from a test case via the
+`T_push_fixture()` and `T_pop_fixture()` functions.
+.. code-block:: c
+    void *T_push_fixture(T_fixture_node *node, const T_fixture *fixture);
+    void T_pop_fixture(void);
+The `T_push_fixture()` function needs an uninitialized fixture node which must
+exist until `T_pop_fixture()` is called.  It returns the initial context of the
+fixture.  At the end of a test case all pushed fixtures are popped
+automatically.  A call of `T_pop_fixture()` invokes the teardown method of the
+fixture and must correspond to a previous call to `T_push_fixture()`.
 .. code-block:: c
     :caption: Test Fixture Example
@@ -1028,6 +1056,34 @@ RTEMS, floating-point operations are only supported in special tasks and may be
 forbidden in interrupt context.  The formatted output functions provided by the
 test framework work in every context.
+You can stop a test case via the ``T_stop()`` function.  This function does not
+return.  You can indicate unreachable code paths with the ``T_unreachable()``
+function.  If this function is called, then the test case stops.
+You can busy wait with the ``T_busy()`` function:
+.. code-block:: c
+    void T_busy(uint_fast32_t count);
+It performs a busy loop with the specified iteration count.  This function is
+optimized to not perform memory accesses and should have a small jitter.
+You can get an interation count for the ``T_busy()`` function which corresponds
+roughly to one clock tick interval with the ``T_get_one_clock_tick_busy()``
+.. code-block:: c
+    uint_fast32_t T_get_one_clock_tick_busy(void);
+This function requires a clock driver.  It must be called from thread context
+with interrupts enabled.  It may return a different value each time it is
 Time Services
@@ -1353,6 +1409,117 @@ reported.
+Interrupt Tests
+In the operating system implementation you may have two kinds of critical
+sections.  Firstly, there are low-level critical sections protected by
+interrupts disabled and maybe also some SMP spin lock.  Secondly, there are
+high-level critical sections which are protected by disabled thread
+dispatching.  The high-level critical sections may contain several low-level
+critical sections.  Between these low-level critical sections interrupts may
+happen which could alter the code path taken in the high-level critical
+The test framework provides support to write test cases for high-level critical
+sections though the `T_interrupt_test()` function:
+.. code-block:: c
+    typedef enum {
+    } T_interrupt_test_state;
+    typedef struct {
+        void                   (*prepare)(void *arg);
+        void                   (*action)(void *arg);
+        T_interrupt_test_state (*interrupt)(void *arg);
+        void                   (*blocked)(void *arg);
+        uint32_t                 max_iteration_count;
+    } T_interrupt_test_config;
+    T_interrupt_test_state T_interrupt_test(
+        const T_interrupt_test_config *config,
+        void                          *arg
+    );
+This function returns ``T_INTERRUPT_TEST_DONE`` if the test condition was
+satisfied within the maximum iteration count, otherwise it returns
+``T_INTERRUPT_TEST_TIMEOUT``.  The interrupt test run uses the specified
+configuration and passes the specified argument to all configured handlers.
+The function shall be called from thread context with interrupts enabled.
+.. image:: ../images/eng/interrupt-test.*
+    :scale: 60
+    :align: center
+The optional *prepare* handler should prepare the system so that the *action*
+handler can be called.  It is called in a tight loop, so all the time consuming
+setup should be done before ``T_interrupt_test()`` is called.  During the
+preparation the test state is ``T_INTERRUPT_TEST_INITIAL``.  The preparation
+handler shall not change the test state.
+The *action* handler should call the function which executes the code section
+under test.  The execution path up to the code section under test should have a
+low jitter.  Otherwise, the adaptive interrupt time point algorithm may not
+find the right spot.
+The *interrupt* handler should check if the test condition is satisfied or a
+new iteration is necessary.  This handler is called in interrupt context.  It
+shall return ``T_INTERRUPT_TEST_DONE`` if the test condition is satisfied and
+the test run is done.  It shall return ``T_INTERRUPT_TEST_EARLY`` if the
+interrupt happened too early to satisfy the test condition.  It shall return
+``T_INTERRUPT_TEST_LATE`` if the interrupt happened too late to satisfy the
+test condition.  It shall return ``T_INTERRUPT_TEST_CONTINUE`` if the test
+should continue with the current timing settings.  Other states shall not be
+returned.  It is critical to return the early and late states if the test
+conditions was not satisfied, otherwise the adaptive bisection algorithm may
+not work.  The returned state is used to try to change the test state from
+``T_INTERRUPT_TEST_ACTION`` to the returned state.
+The optional *blocked* handler is invoked if the executing thread blocks during
+the action processing.  It should remove the blocking condition of the thread
+so that the next iteration can start.  It can use
+``T_interrupt_change_state()`` to change the interrupt test state.
+The *max iteration count* configuration member defines the maximum iteration
+count of the test loop.  If the maximum iteration count is reached before the
+test condition is satisfied, then ``T_interrupt_test()`` returns
+The *interrupt* and *blocked* handlers may be called in arbitrary test states.
+The *action*, *interrupt*, and *blocked* handlers can use
+``T_interrupt_test_get_state()`` to get the current test state:
+.. code-block:: c
+    T_interrupt_test_state T_interrupt_test_get_state(void);
+The *action*, *interrupt*, and *blocked* handlers can use
+``T_interrupt_test_change_state()`` to try to change the test state from an
+expected state to a desired state:
+.. code-block:: c
+    T_interrupt_test_state T_interrupt_test_change_state(
+        T_interrupt_test_state expected_state,
+        T_interrupt_test_state desired_state
+    );
+The function returns the previous state.  If it **differs from the expected
+state**, then the requested state **change to the desired state did not take
+place**.  In an SMP configuration, do not call this function in a tight loop.
+It could lock up the test run.  To busy wait for a state change, use
 Test Runner

More information about the devel mailing list