[PATCH 7/8] validation/ts-performance-0: Add test suite

[Chris Johns]

On 17/11/20 6:02 pm, Sebastian Huber wrote:
> On 17/11/2020 00:15, Chris Johns wrote:
> 
>>>>> new file mode 100644
>>>>> index 0000000000..0f7db65a8e
>>>>> --- /dev/null
>>>>> +++ b/testsuites/validation/ts-default.h
>>>>> @@ -0,0 +1,237 @@
>>>>> +/* SPDX-License-Identifier: BSD-2-Clause */
>>>>> +
>>>>> +/**
>>>>> + * @file
>>>>> + *
>>>>> + * @brief This header file provides the default validation test suite runner
>>>>> + *   and application configuration.
>>>>> + */
>>>>> [...]
>>>>> +#define CONFIGURE_SCHEDULER_EDF_SMP
>>>>> +
>>>>> +#include <rtems/scheduler.h>
>>>>> +
>>>>> +RTEMS_SCHEDULER_EDF_SMP(a);
>>>>> +
>>>>> +RTEMS_SCHEDULER_EDF_SMP(b);
>>>>> +
>>>>> +RTEMS_SCHEDULER_EDF_SMP(c);
>>>>> +
>>>>> +#define CONFIGURE_SCHEDULER_TABLE_ENTRIES \
>>>>> +  RTEMS_SCHEDULER_TABLE_EDF_SMP(a, rtems_build_name('A', ' ', ' ', ' ')), \
>>>>> +  RTEMS_SCHEDULER_TABLE_EDF_SMP(b, rtems_build_name('B', ' ', ' ', ' ')), \
>>>>> +  RTEMS_SCHEDULER_TABLE_EDF_SMP(c, rtems_build_name('C', ' ', ' ', ' '))
>>>>> +
>>>>> +#define CONFIGURE_SCHEDULER_ASSIGNMENTS \
>>>>> +  RTEMS_SCHEDULER_ASSIGN(0, RTEMS_SCHEDULER_ASSIGN_PROCESSOR_MANDATORY), \
>>>>> +  RTEMS_SCHEDULER_ASSIGN(1, RTEMS_SCHEDULER_ASSIGN_PROCESSOR_OPTIONAL), \
>>>>> +  RTEMS_SCHEDULER_ASSIGN(2, RTEMS_SCHEDULER_ASSIGN_PROCESSOR_OPTIONAL), \
>>>>> +  RTEMS_SCHEDULER_ASSIGN(2, RTEMS_SCHEDULER_ASSIGN_PROCESSOR_OPTIONAL)
>>>>> +
>>>> These seems specific but I ma not sure.
>>> This configuration provides enough resources to run a wide range of test cases.
>>> For example, the SMP scheduler configuration with three schedulers can be used
>>> to test most of the SMP support of RTEMS.
>>>> Do tests depend on this configuration?
>>> Yes, if the configuration provides the necessary resources, they pass, otherwise
>>> they fail.
>> Any particular reason for those schedulers in that order?
> Most tests use one processor. With just two processors you can probably test
> more than 90% of the RTEMS SMP support. Only some very specific corner cases
> need more than two processors. This is a general purpose test suite and the
> configuration will be adapted if a new test case needs something special. If it
> is too special, then it can be added to a dedicated test suite.

I see and thanks for the explanation.

>>>> Does the number of cores available effect this or the dependent tests? I assume
>>>> this is included in a number of tests.
>>> If a test needs three processors and a board has only two, then the test case
>>> should fail.
>> Will those BSPs have the failing tests excluded?
> Currently, there is no support for the new test framework in the RTEMS Tester.
> You can only specify the expected state of a complete test suite. I hope to
> eventually add some support for the new test framework so that the expected
> state of individual test cases can be specified.

OK.

>>
>> I would like to be comfortable the tests and these set ups are not solely
>> focused on the LEON.
> I am not focused on the LEON. Why do you think something is LEON-specific in
> this test suite?

Nothing specific. I was wondering why that specific configuration over another.

Chris