[PATCH 7/8] validation/ts-performance-0: Add test suite

[Chris Johns]

On 16/11/20 5:48 pm, Sebastian Huber wrote:
> 
> On 16/11/2020 00:53, Chris Johns wrote:
>> On 13/11/20 9:07 pm, Sebastian Huber wrote:
>>> Share a default test suite with ts-validation-0.
>>> ---
>>>   spec/build/testsuites/validation/grp.yml      |   2 +
>>>   .../testsuites/validation/performance-0.yml   |  19 ++
>>>   testsuites/validation/ts-default.h            | 237 ++++++++++++++++++
>>>   testsuites/validation/ts-performance-0.c      |  74 ++++++
>>>   testsuites/validation/ts-validation-0.c       | 155 +-----------
>>>   5 files changed, 333 insertions(+), 154 deletions(-)
>>>   create mode 100644 spec/build/testsuites/validation/performance-0.yml
>>>   create mode 100644 testsuites/validation/ts-default.h
>>>   create mode 100644 testsuites/validation/ts-performance-0.c
>>>
>>> diff --git a/spec/build/testsuites/validation/grp.yml
>>> b/spec/build/testsuites/validation/grp.yml
>>> index 390fb48803..1a903a4198 100644
>>> --- a/spec/build/testsuites/validation/grp.yml
>>> +++ b/spec/build/testsuites/validation/grp.yml
>>> @@ -10,6 +10,8 @@ includes:
>>>   install: []
>>>   ldflags: []
>>>   links:
>>> +- role: build-dependency
>>> +  uid: performance-0
>>>   - role: build-dependency
>>>     uid: validation-0
>>>   type: build
>>> diff --git a/spec/build/testsuites/validation/performance-0.yml
>>> b/spec/build/testsuites/validation/performance-0.yml
>>> new file mode 100644
>>> index 0000000000..ba14066dd3
>>> --- /dev/null
>>> +++ b/spec/build/testsuites/validation/performance-0.yml
>>> @@ -0,0 +1,19 @@
>>> +SPDX-License-Identifier: CC-BY-SA-4.0 OR BSD-2-Clause
>>> +build-type: test-program
>>> +cflags: []
>>> +copyrights:
>>> +- Copyright (C) 2020 embedded brains GmbH (http://www.embedded-brains.de)
>>> +cppflags: []
>>> +cxxflags: []
>>> +enabled-by: true
>>> +features: c cprogram
>>> +includes: []
>>> +ldflags: []
>>> +links: []
>>> +source:
>>> +- testsuites/validation/ts-performance-0.c
>>> +stlib: []
>>> +target: testsuites/validation/ts-performance-0.exe
>>> +type: build
>>> +use-after: []
>>> +use-before: []
>>> diff --git a/testsuites/validation/ts-default.h
>>> b/testsuites/validation/ts-default.h
>> Who's defaults are these?
> This is an implementation of the default validation test suite runner and
> application configuration. The goal is to use it for a wide range of test cases.

OK.

>>> new file mode 100644
>>> index 0000000000..0f7db65a8e
>>> --- /dev/null
>>> +++ b/testsuites/validation/ts-default.h
>>> @@ -0,0 +1,237 @@
>>> +/* SPDX-License-Identifier: BSD-2-Clause */
>>> +
>>> +/**
>>> + * @file
>>> + *
>>> + * @brief This header file provides the default validation test suite runner
>>> + *   and application configuration.
>>> + */
>>> [...]
>>> +#define CONFIGURE_SCHEDULER_EDF_SMP
>>> +
>>> +#include <rtems/scheduler.h>
>>> +
>>> +RTEMS_SCHEDULER_EDF_SMP(a);
>>> +
>>> +RTEMS_SCHEDULER_EDF_SMP(b);
>>> +
>>> +RTEMS_SCHEDULER_EDF_SMP(c);
>>> +
>>> +#define CONFIGURE_SCHEDULER_TABLE_ENTRIES \
>>> +  RTEMS_SCHEDULER_TABLE_EDF_SMP(a, rtems_build_name('A', ' ', ' ', ' ')), \
>>> +  RTEMS_SCHEDULER_TABLE_EDF_SMP(b, rtems_build_name('B', ' ', ' ', ' ')), \
>>> +  RTEMS_SCHEDULER_TABLE_EDF_SMP(c, rtems_build_name('C', ' ', ' ', ' '))
>>> +
>>> +#define CONFIGURE_SCHEDULER_ASSIGNMENTS \
>>> +  RTEMS_SCHEDULER_ASSIGN(0, RTEMS_SCHEDULER_ASSIGN_PROCESSOR_MANDATORY), \
>>> +  RTEMS_SCHEDULER_ASSIGN(1, RTEMS_SCHEDULER_ASSIGN_PROCESSOR_OPTIONAL), \
>>> +  RTEMS_SCHEDULER_ASSIGN(2, RTEMS_SCHEDULER_ASSIGN_PROCESSOR_OPTIONAL), \
>>> +  RTEMS_SCHEDULER_ASSIGN(2, RTEMS_SCHEDULER_ASSIGN_PROCESSOR_OPTIONAL)
>>> +
>> These seems specific but I ma not sure.
> This configuration provides enough resources to run a wide range of test cases.
> For example, the SMP scheduler configuration with three schedulers can be used
> to test most of the SMP support of RTEMS.
>>
>> Do tests depend on this configuration?
> Yes, if the configuration provides the necessary resources, they pass, otherwise
> they fail.

Any particular reason for those schedulers in that order?

>> Does the number of cores available effect this or the dependent tests? I assume
>> this is included in a number of tests.
> If a test needs three processors and a board has only two, then the test case
> should fail.

Will those BSPs have the failing tests excluded?

I would like to be comfortable the tests and these set ups are not solely
focused on the LEON.

Chris