[PATCH 1/1] eng: Add software test framework chapter
sebastian.huber at embedded-brains.de
Tue Mar 19 08:25:12 UTC 2019
On 19/03/2019 06:21, Chris Johns wrote:
> On 14/3/19 9:23 pm, Sebastian Huber wrote:
>> Update #3199.
>> eng/index.rst | 3 +
>> eng/test-framework.rst | 2077 ++++++++++++++++++++++++++++++++++++++++++++++++
>> 2 files changed, 2080 insertions(+)
>> create mode 100644 eng/test-framework.rst
>> diff --git a/eng/index.rst b/eng/index.rst
>> index fe51e91..f29d0f3 100644
>> --- a/eng/index.rst
>> +++ b/eng/index.rst
>> @@ -11,6 +11,8 @@ RTEMS Software Engineering (|version|)
>> .. topic:: Copyrights and License
>> + | |copy| 2018, 2019 embedded brains GmbH
>> + | |copy| 2018, 2019 Sebastian Huber
>> | |copy| 1988, 2015 On-Line Applications Research Corporation (OAR)
>> .. include:: ../common/license.rst
>> @@ -26,6 +28,7 @@ RTEMS Software Engineering (|version|)
>> + test-framework
>> diff --git a/eng/test-framework.rst b/eng/test-framework.rst
>> new file mode 100644
>> index 0000000..dda13bc
>> --- /dev/null
>> +++ b/eng/test-framework.rst
>> @@ -0,0 +1,2077 @@
>> +.. SPDX-License-Identifier: CC-BY-SA-4.0
>> +.. Copyright (C) 2018, 2019 embedded brains GmbH
>> +.. Copyright (C) 2018, 2019 Sebastian Huber
>> +Software Test Framework
>> +.. _TheTTestFramework:
>> +The T Test Framework
>> +The `T Test Framework` helps to write test suites. A `test suite` is a
>> +collection of test cases. A `test case` consists of individual test actions
>> +and checks. A `test check` determines if the outcome of a test action meets
>> +its expectation. A `test action` is a program sequence with an observable
>> +outcome, for example a function invocation with a return status. If the test
>> +action outcome is all right, then the test check passes, otherwise the test
>> +check fails. The test check failures of a test case are summed up. A test
>> +case passes, if the failure count of this test case is zero, otherwise the test
>> +case fails. The test suite passes if all test cases pass, otherwise it fails.
> I think this needs a little more clarification. The density of that is being
> expressed is too compact and I think it requires some expansion. For example
> from this I cannot tell where an executable boundary is and given RTEMS has had
> 'testsuites' I feel expanding and explaining each of these is important in
> relationship to what RTEMS has and needs. For example:
> The `T Test Framework` helps to write test suites. A `test suite` is a
> collection of test cases ......
> and then ...
> ... where tests for specific functionality are grouped for example
This chapter should just describe the test framework and not an RTEMS
specific test suite. This is a different topic. The test framework
should enable you to organize your tests one way or another.
> Is a test executable a collection of "test cases" or a single case? How is this
A test executable can contain one or more test suites. Test cases using
the T_TEST_CASE() macro are registered via a linker set. However, you
can run test cases also individually. See the attached example which
contains two test suites. It is a conversion of
testsuites/sptests/spsysinit01/init.c. It demonstrates that you can test
the system initialization procedure with this framework. How you
organize the tests is up to you and a separate topic. The test framework
is just a tool to write test cases.
> What is the expected number of tests once this SMP qual work ends?
I don't know. I guess it will be similar to the existing test suite.
Probably a bit more performance tests.
> Is hardware
> testing a requirement?
Yes, although thanks to Jiri Gaisler's work we probably don't have to
run the tests often on real hardware. We need it for the runtime
> An outline of the problem domain being addressed would help?
What do you mean with problem domain? From my point of view the problem
domain is writing test cases?
> Can there please be some explanation of what the issues this framework is
> attempt to solve are, what it does well, what it does not do well or is not,
> plus things like target requirements, test duration, etc?
Ok, good idea. I will put an overview/features section at the top of the
> I feel if this is done the detail that follows will be just that the detail.
Sebastian Huber, embedded brains GmbH
Address : Dornierstr. 4, D-82178 Puchheim, Germany
Phone : +49 89 189 47 41-16
Fax : +49 89 189 47 41-09
E-Mail : sebastian.huber at embedded-brains.de
PGP : Public key available on request.
Diese Nachricht ist keine geschäftliche Mitteilung im Sinne des EHUG.
-------------- next part --------------
A non-text attachment was scrubbed...
Size: 21969 bytes
Desc: not available
More information about the devel