New validation test suites
Sebastian Huber
sebastian.huber at embedded-brains.de
Tue Dec 14 07:24:33 UTC 2021
Hello Chris,
On 13/12/2021 22:01, Chris Johns wrote:
> On 14/12/21 1:53 am, Sebastian Huber wrote:
>> Hello,
>>
>> the ESA activity to pre-qualify parts of RTEMS according to ECSS requirements is
>> nearly complete. There is a short presentation available here:
>>
>> https://indico.esa.int/event/374/timetable/
>
> Was the change in memory usage for 4.8 of 23812 bytes to 68896 explained?
The foot print increase has mainly five reasons:
* General GCC code generation
* Chip errata workarounds done by GCC
* More features used from RTEMS (for example uniprocessor
synchronization done via task priorities vs. mutex synchronization)
* SMP support of RTEMS
* New RTEMS features such as transitive priority inheritance
>> We finished the specification of the pre-qualified RTEMS feature set. The
>> specification is available in an RTEMS Project repository:
>>
>> https://git.rtems.org/rtems-central/tree/spec
>
> I had a quick look. Is there a more user friendly view of this data?
>
> I think the term "specification" is a little bit misleading because the data
> files are not easily read by a person. I understand this is the specification
> data set however it is not what I am traditionally use to seeing.
You can use the "./specview.py" script to get views of the
specification. For example, this command displays the transition map
for the rtems_signal_send() directive:
./specview.py --filter=action-table /rtems/signal/req/send
.. table::
:class: longtable
===== ========== ===== ======= ======= ======== ====== ======
============= =========
Entry Descriptor Task Set Handler ASR Nested Status
Handler Recursive
===== ========== ===== ======= ======= ======== ====== ======
============= =========
0 0 NoObj Zero Invalid Enabled Yes InvNum
NoCall No
1 0 NoObj Zero Invalid Enabled No InvNum
NoCall No
2 0 NoObj Zero Invalid Disabled Yes InvNum
NoCall No
3 0 NoObj Zero Invalid Disabled No InvNum
NoCall No
4 0 NoObj Zero Valid Enabled Yes InvNum
NoCall No
5 0 NoObj Zero Valid Enabled No InvNum
NoCall No
6 0 NoObj Zero Valid Disabled Yes InvNum
NoCall No
7 0 NoObj Zero Valid Disabled No InvNum
NoCall No
8 1 NoObj NonZero Invalid Enabled Yes InvId
NoCall No
9 1 NoObj NonZero Invalid Enabled No InvId
NoCall No
10 1 NoObj NonZero Invalid Disabled Yes InvId
NoCall No
11 1 NoObj NonZero Invalid Disabled No InvId
NoCall No
12 1 NoObj NonZero Valid Enabled Yes InvId
NoCall No
13 1 NoObj NonZero Valid Enabled No InvId
NoCall No
14 1 NoObj NonZero Valid Disabled Yes InvId
NoCall No
15 1 NoObj NonZero Valid Disabled No InvId
NoCall No
16 0 Self Zero Invalid Enabled Yes InvNum
NoCall No
17 0 Self Zero Invalid Enabled No InvNum
NoCall No
18 0 Self Zero Invalid Disabled Yes InvNum
NoCall No
19 0 Self Zero Invalid Disabled No InvNum
NoCall No
20 0 Self Zero Valid Enabled Yes InvNum
NoCall No
21 0 Self Zero Valid Enabled No InvNum
NoCall No
22 0 Self Zero Valid Disabled Yes InvNum
NoCall No
23 0 Self Zero Valid Disabled No InvNum
NoCall No
24 2 Self NonZero Invalid Enabled Yes NotDef
NoCall No
25 2 Self NonZero Invalid Enabled No NotDef
NoCall No
26 2 Self NonZero Invalid Disabled Yes NotDef
NoCall No
27 2 Self NonZero Invalid Disabled No NotDef
NoCall No
28 6 Self NonZero Valid Enabled Yes Ok
DuringSend Yes
29 4 Self NonZero Valid Enabled No Ok
DuringSend No
30 3 Self NonZero Valid Disabled Yes Ok
AfterEnable No
31 3 Self NonZero Valid Disabled No Ok
AfterEnable No
32 0 Other Zero Invalid Enabled Yes InvNum
NoCall No
33 0 Other Zero Invalid Enabled No InvNum
NoCall No
34 0 Other Zero Invalid Disabled Yes InvNum
NoCall No
35 0 Other Zero Invalid Disabled No InvNum
NoCall No
36 0 Other Zero Valid Enabled Yes InvNum
NoCall No
37 0 Other Zero Valid Enabled No InvNum
NoCall No
38 0 Other Zero Valid Disabled Yes InvNum
NoCall No
39 0 Other Zero Valid Disabled No InvNum
NoCall No
40 2 Other NonZero Invalid Enabled Yes NotDef
NoCall No
41 2 Other NonZero Invalid Enabled No NotDef
NoCall No
42 2 Other NonZero Invalid Disabled Yes NotDef
NoCall No
43 2 Other NonZero Invalid Disabled No NotDef
NoCall No
44 7 Other NonZero Valid Enabled Yes Ok
AfterDispatch Yes
45 5 Other NonZero Valid Enabled No Ok
AfterDispatch No
46 3 Other NonZero Valid Disabled Yes Ok
AfterEnable No
47 3 Other NonZero Valid Disabled No Ok
AfterEnable No
===== ========== ===== ======= ======= ======== ====== ======
============= =========
Here the same information in a different view, for each post-condition
set the pre-condition sets are displayed:
./specview.py --filter=action-list /rtems/signal/req/send
Status = Ok, Handler = DuringSend, Recursive = Yes
* Task = Self, Set = NonZero, Handler = Valid, ASR = Enabled,
Nested = Yes
Status = Ok, Handler = DuringSend, Recursive = No
* Task = Self, Set = NonZero, Handler = Valid, ASR = Enabled,
Nested = No
Status = Ok, Handler = AfterDispatch, Recursive = Yes
* Task = Other, Set = NonZero, Handler = Valid, ASR = Enabled,
Nested = Yes
Status = Ok, Handler = AfterDispatch, Recursive = No
* Task = Other, Set = NonZero, Handler = Valid, ASR = Enabled,
Nested = No
Status = Ok, Handler = AfterEnable, Recursive = No
* Task = { Self, Other }, Set = NonZero, Handler = Valid, ASR =
Disabled, Nested = { Yes, No }
Status = InvId, Handler = NoCall, Recursive = No
* Task = NoObj, Set = NonZero, Handler = { Invalid, Valid }, ASR =
{ Enabled, Disabled }, Nested = { Yes, No }
Status = NotDef, Handler = NoCall, Recursive = No
* Task = { Self, Other }, Set = NonZero, Handler = Invalid, ASR = {
Enabled, Disabled }, Nested = { Yes, No }
Status = InvNum, Handler = NoCall, Recursive = No
* Task = { NoObj, Self, Other }, Set = Zero, Handler = { Invalid,
Valid }, ASR = { Enabled, Disabled }, Nested = { Yes, No }
>
>> The validation tests are generated from the specification using the
>> "./spec2modules.py" script and end up in the RTEMS sources of a Git submodule. I
>> think the specification and the generation tool is now sufficiently stable so
>> that the validation test code can be integrated in the RTEMS master branch. The
>> patch set is too big for the mailing list, so you can review it here:
>>
>> https://git.rtems.org/sebh/rtems.git/log/?h=validation
>
> The link failed.
Yes, viewing my personal repository no longer works. I am not sure if
this is a temporary issue. This is why I added the github link.
>
>> https://github.com/sebhub/rtems/tree/validation
>
> The header in a test says the regeneration instructions are in the engineering
> manual but I could not quickly find them?
https://docs.rtems.org/branches/master/eng/req/howto.html#generate-content-after-changes
In an earlier version of the header, we had a link which you didn't like:
commit a6689fb147649a29ff5533cec8a53635e2c95ec6
Author: Sebastian Huber <sebastian.huber at embedded-brains.de>
Date: Fri Jan 22 16:01:46 2021 +0100
Improve file header comment in generated files
diff --git a/cpukit/include/rtems/rtems/types.h
b/cpukit/include/rtems/rtems/types.h
index 32b45113c8..a058eedea1 100644
--- a/cpukit/include/rtems/rtems/types.h
+++ b/cpukit/include/rtems/rtems/types.h
@@ -38,11 +38,15 @@
* worded better please post a report or patch to an RTEMS mailing list
* or raise a bug report:
*
- * https://docs.rtems.org/branches/master/user/support/bugs.html
+ * https://www.rtems.org/bugs.html
*
- * For information on updating and regenerating please refer to:
+ * For information on updating and regenerating please refer to the How-To
+ * section in the Software Requirements Engineering chapter of the
+ * RTEMS Software Engineering manual. The manual is provided as a part of
+ * a release. For development sources please refer to the online
+ * documentation at:
*
- * https://docs.rtems.org/branches/master/eng/req/howto.html
+ * https://docs.rtems.org
*/
/* Generated from spec:/rtems/type/if/header */
>
>> The patch set is organized so that general support code for the validation tests
>> is added first and then there are commits for each pre-qualified Classic API
>> Manager or subsystem.
>>
>> I did build all BSPs with the patch set. The validation tests use only
>> statically allocated resources.
>
> Are the validation tests compatible with rtems-test?
Yes.
> How many test executables does this add to the testsuite?
If RTEMS_SMP is disabled, then there are 19 new executables in the patch
set. If RTEMS_SMP is enable, then there are 28 new executables.
>
> What hardware have the validation tests been run on? Any tier 1 archs?
I tested with the sparc/leon3 BSPs and the arm/realview_pbx_a9_qemu. You
need a full implementation of the new Interrupt Manager directives and a
working Cache Manager implementation.
I noticed an issue with the thread restart on aarch64/a53_lp64_qemu.
On powerpc/psim there is an issue in one test case, due to:
#define CPU_ALL_TASKS_ARE_FP CPU_HARDWARE_FP
Another issue is that the tm27 interrupt must be independent of the
clock driver interrupt. This is not the case for powerpc/psim.
There is definitely some work left to cover all edge cases. Some tests
are quite complicated.
>
> Is there anything that interprets the new test output format? It looks like lots
> of great info but a little difficult to read.
EDISOFT worked on a test report generator, however, it is not yet in a
reviewable state.
>
>> Some low memory targets are not able to link all test suites.
>
> Are these excluded in the normal way?
Yes:
https://github.com/sebhub/rtems/commit/2feedb9e7805f483e35b7914b9bd7c808e31a8b4#diff-bf7b325198d4133c9e52f49d77447905ecd3c6ac5159d5cd85e7efeffa6da47e
--
embedded brains GmbH
Herr Sebastian HUBER
Dornierstr. 4
82178 Puchheim
Germany
email: sebastian.huber at embedded-brains.de
phone: +49-89-18 94 741 - 16
fax: +49-89-18 94 741 - 08
Registergericht: Amtsgericht München
Registernummer: HRB 157899
Vertretungsberechtigte Geschäftsführer: Peter Rasmussen, Thomas Dörfler
Unsere Datenschutzerklärung finden Sie hier:
https://embedded-brains.de/datenschutzerklaerung/
More information about the devel
mailing list