[PATCH 1/1] eng: Update test framework chapter
Gedare Bloom
gedare at rtems.org
Tue Jul 21 16:49:22 UTC 2020
Thanks, a couple typos, and some suggestions for clarification.
On Tue, Jul 21, 2020 at 8:49 AM Sebastian Huber
<sebastian.huber at embedded-brains.de> wrote:
>
> Document the dynamic text fixtures, utility functions, and the interrupt
> test support.
>
> Update #3199.
> ---
> eng/index.rst | 4 +-
> eng/test-framework.rst | 185 ++++++++++++++++++++++++++++++++--
> images/eng/interrupt-test.odg | Bin 0 -> 14829 bytes
> images/eng/interrupt-test.pdf | Bin 0 -> 14153 bytes
> images/eng/interrupt-test.png | Bin 0 -> 75272 bytes
> 5 files changed, 178 insertions(+), 11 deletions(-)
> create mode 100644 images/eng/interrupt-test.odg
> create mode 100644 images/eng/interrupt-test.pdf
> create mode 100644 images/eng/interrupt-test.png
>
> diff --git a/eng/index.rst b/eng/index.rst
> index 8f91c5e..f6b02ec 100644
> --- a/eng/index.rst
> +++ b/eng/index.rst
> @@ -11,8 +11,8 @@ RTEMS Software Engineering (|version|)
>
> .. topic:: Copyrights and License
>
> - | |copy| 2018, 2019 embedded brains GmbH
> - | |copy| 2018, 2019 Sebastian Huber
> + | |copy| 2018, 2020 embedded brains GmbH
> + | |copy| 2018, 2020 Sebastian Huber
> | |copy| 1988, 2015 On-Line Applications Research Corporation (OAR)
>
> .. include:: ../common/license.rst
> diff --git a/eng/test-framework.rst b/eng/test-framework.rst
> index b6411b5..582718d 100644
> --- a/eng/test-framework.rst
> +++ b/eng/test-framework.rst
> @@ -1,7 +1,7 @@
> .. SPDX-License-Identifier: CC-BY-SA-4.0
>
> -.. Copyright (C) 2018, 2019 embedded brains GmbH
> -.. Copyright (C) 2018, 2019 Sebastian Huber
> +.. Copyright (C) 2018, 2020 embedded brains GmbH
> +.. Copyright (C) 2018, 2020 Sebastian Huber
>
> Software Test Framework
> ***********************
> @@ -144,13 +144,41 @@ macro followed by a function body:
>
> The test case `name` must be a valid C designator. The test case names must be
> unique within the test suite. The `fixture` must point to a statically
> -initialized read-only object of type `T_fixture`. The test fixture
> -provides methods to setup, stop and tear down a test case. A context is passed
> -to the methods. The initial context is defined by the read-only fixture
> -object. The context can be obtained by the `T_fixture_context()`
> -function. It can be set within the scope of one test case by the
> -`T_set_fixture_context()` function. This can be used for example to
> -dynamically allocate a test environment in the setup method.
> +initialized read-only object of type `T_fixture`.
> +
> +.. code-block:: c
> +
> + typedef struct T_fixture {
> + void (*setup)(void *context);
> + void (*stop)(void *context);
> + void (*teardown)(void *context);
> + void (*scope)(void *context, char *buffer, size_t size);
> + void *initial_context;
> + } T_fixture;
> +
> +The test fixture provides methods to setup, stop, and teardown a test case as
> +well as to give the scope for log messags. A context is passed to each of the
typo: messages
> +methods. The initial context is defined by the read-only fixture object. The
> +context can be obtained by the `T_fixture_context()` function. It can be set
"setting" the context means changing the value of initial_context to
point elsewhere? This would mean the fixture is not read-only
> +within the scope of one test case by the `T_set_fixture_context()` function.
> +This can be used for example to dynamically allocate a test environment in the
> +setup method.
> +
> +The test case fixtures of a test case are organized as a stack. Fixtures can
> +be dynamically added to a test case and removed from a test case via the
> +`T_push_fixture()` and `T_pop_fixture()` functions.
> +
> +.. code-block:: c
> +
> + void *T_push_fixture(T_fixture_node *node, const T_fixture *fixture);
> +
> + void T_pop_fixture(void);
> +
> +The `T_push_fixture()` function needs an uninitialized fixture node which must
> +exist until `T_pop_fixture()` is called. It returns the initial context of the
> +fixture. At the end of a test case all pushed fixtures are popped
> +automatically. A call of `T_pop_fixture()` invokes the teardown method of the
> +fixture and must correspond to a previous call to `T_push_fixture()`.
>
> .. code-block:: c
> :caption: Test Fixture Example
> @@ -1028,6 +1056,34 @@ RTEMS, floating-point operations are only supported in special tasks and may be
> forbidden in interrupt context. The formatted output functions provided by the
> test framework work in every context.
>
> +Utility
> +-------
> +
> +You can stop a test case via the ``T_stop()`` function. This function does not
> +return. You can indicate unreachable code paths with the ``T_unreachable()``
> +function. If this function is called, then the test case stops.
> +
> +You can busy wait with the ``T_busy()`` function:
> +
> +.. code-block:: c
> +
> + void T_busy(uint_fast32_t count);
> +
> +It performs a busy loop with the specified iteration count. This function is
> +optimized to not perform memory accesses and should have a small jitter.
It should be clarified that an iteration has CPU-specific duration.
> +
> +You can get an interation count for the ``T_busy()`` function which corresponds
typo: iteration
> +roughly to one clock tick interval with the ``T_get_one_clock_tick_busy()``
> +function:
> +
> +.. code-block:: c
> +
> + uint_fast32_t T_get_one_clock_tick_busy(void);
> +
> +This function requires a clock driver. It must be called from thread context
> +with interrupts enabled. It may return a different value each time it is
> +called.
> +
> Time Services
> -------------
>
> @@ -1353,6 +1409,117 @@ reported.
> M:E:Empty:D:0.015188063
> E:measure_empty:N:1:F:0:D:14.284869
>
> +Interrupt Tests
> +---------------
> +
> +In the operating system implementation you may have two kinds of critical
> +sections. Firstly, there are low-level critical sections protected by
> +interrupts disabled and maybe also some SMP spin lock. Secondly, there are
> +high-level critical sections which are protected by disabled thread
> +dispatching. The high-level critical sections may contain several low-level
> +critical sections. Between these low-level critical sections interrupts may
> +happen which could alter the code path taken in the high-level critical
> +section.
This paragraph seems to belong somewhere else, although I guess we
don't have a great "kernel developers" guide.
It may be worth using the names we typically do for these: ISR
Critical Section and Dispatch Disable Critical Section or some such.
> +
> +The test framework provides support to write test cases for high-level critical
> +sections though the `T_interrupt_test()` function:
> +
> +.. code-block:: c
> +
> + typedef enum {
> + T_INTERRUPT_TEST_INITIAL,
> + T_INTERRUPT_TEST_ACTION,
> + T_INTERRUPT_TEST_BLOCKED,
> + T_INTERRUPT_TEST_CONTINUE,
> + T_INTERRUPT_TEST_DONE,
> + T_INTERRUPT_TEST_EARLY,
> + T_INTERRUPT_TEST_INTERRUPT,
> + T_INTERRUPT_TEST_LATE,
> + T_INTERRUPT_TEST_TIMEOUT
> + } T_interrupt_test_state;
> +
> + typedef struct {
> + void (*prepare)(void *arg);
> + void (*action)(void *arg);
> + T_interrupt_test_state (*interrupt)(void *arg);
> + void (*blocked)(void *arg);
> + uint32_t max_iteration_count;
> + } T_interrupt_test_config;
> +
> + T_interrupt_test_state T_interrupt_test(
> + const T_interrupt_test_config *config,
> + void *arg
> + );
> +
> +This function returns ``T_INTERRUPT_TEST_DONE`` if the test condition was
> +satisfied within the maximum iteration count, otherwise it returns
> +``T_INTERRUPT_TEST_TIMEOUT``. The interrupt test run uses the specified
> +configuration and passes the specified argument to all configured handlers.
> +The function shall be called from thread context with interrupts enabled.
> +
> +.. image:: ../images/eng/interrupt-test.*
> + :scale: 60
> + :align: center
> +
> +The optional *prepare* handler should prepare the system so that the *action*
> +handler can be called. It is called in a tight loop, so all the time consuming
> +setup should be done before ``T_interrupt_test()`` is called. During the
> +preparation the test state is ``T_INTERRUPT_TEST_INITIAL``. The preparation
> +handler shall not change the test state.
> +
> +The *action* handler should call the function which executes the code section
> +under test. The execution path up to the code section under test should have a
> +low jitter. Otherwise, the adaptive interrupt time point algorithm may not
I don't know what is "the adaptive interrupt time point algorithm" --
I get a sense of it in the following, but may be worth a short
paragraph before to explain the idea.
> +find the right spot.
> +
> +The *interrupt* handler should check if the test condition is satisfied or a
> +new iteration is necessary. This handler is called in interrupt context. It
> +shall return ``T_INTERRUPT_TEST_DONE`` if the test condition is satisfied and
> +the test run is done. It shall return ``T_INTERRUPT_TEST_EARLY`` if the
> +interrupt happened too early to satisfy the test condition. It shall return
> +``T_INTERRUPT_TEST_LATE`` if the interrupt happened too late to satisfy the
> +test condition. It shall return ``T_INTERRUPT_TEST_CONTINUE`` if the test
> +should continue with the current timing settings. Other states shall not be
> +returned. It is critical to return the early and late states if the test
> +conditions was not satisfied, otherwise the adaptive bisection algorithm may
"condition was" or "conditions were"
> +not work. The returned state is used to try to change the test state from
> +``T_INTERRUPT_TEST_ACTION`` to the returned state.
> +
> +The optional *blocked* handler is invoked if the executing thread blocks during
> +the action processing. It should remove the blocking condition of the thread
> +so that the next iteration can start. It can use
> +``T_interrupt_change_state()`` to change the interrupt test state.
> +
> +The *max iteration count* configuration member defines the maximum iteration
> +count of the test loop. If the maximum iteration count is reached before the
> +test condition is satisfied, then ``T_interrupt_test()`` returns
> +``T_INTERRUPT_TEST_TIMEOUT``.
> +
> +The *interrupt* and *blocked* handlers may be called in arbitrary test states.
> +
> +The *action*, *interrupt*, and *blocked* handlers can use
> +``T_interrupt_test_get_state()`` to get the current test state:
> +
> +.. code-block:: c
> +
> + T_interrupt_test_state T_interrupt_test_get_state(void);
> +
> +The *action*, *interrupt*, and *blocked* handlers can use
> +``T_interrupt_test_change_state()`` to try to change the test state from an
> +expected state to a desired state:
> +
> +.. code-block:: c
> +
> + T_interrupt_test_state T_interrupt_test_change_state(
> + T_interrupt_test_state expected_state,
> + T_interrupt_test_state desired_state
> + );
> +
> +The function returns the previous state. If it **differs from the expected
> +state**, then the requested state **change to the desired state did not take
> +place**. In an SMP configuration, do not call this function in a tight loop.
> +It could lock up the test run. To busy wait for a state change, use
> +``T_interrupt_test_get_state()``.
>
> Test Runner
> -----------
> --
> 2.26.2
>
> _______________________________________________
> devel mailing list
> devel at rtems.org
> http://lists.rtems.org/mailman/listinfo/devel
More information about the devel
mailing list