[PATCH] tests: CONFIGURE_DISABLE_SMP_CONFIGURATION

[Chris Johns]

On 17/05/2017 10:34, Gedare Bloom wrote:
> On Tue, May 16, 2017 at 6:40 PM, Chris Johns <chrisj at rtems.org> wrote:
>> On 16/05/2017 18:35, Sebastian Huber wrote:
>>>
>>> On 16/05/17 10:28, Chris Johns wrote:
>>>>
>>>> What we do need to do is make sure the test results express the true
>>>> state. If a test is broken it should fail. If it is tagged an
>>>> expected-fail we do not consider it a regression.
>>>
>>>
>>> Loosing test coverage simply because a test uses a non-SMP feature somehow
>>> is not really great.
>>>
>>
>> I am sorry you have lost me. If the test is for something not available when
>> SMP is enabled why build it? This can be handled in the build system. If the
>> test is using something not available on SMP and it can be written to work
>> on SMP by not using that feature then the test is broken on SMP and I would
>> expect a fail until it is fixed.
>>
> There is value in retaining uniprocessor tests whose patterns break on
> SMP because the patterns are not safe for parallel threads. I would
> just leave those as expected-fail.
> 

I agree there is value and it is reasonable something only available on 
non-SMP builds should have a specific test. In these cases I would 
assume the build system would manage which tests are built depending on 
the configuration just like C++ was and networking is. I question adding 
alternative ways to do this.

The premise seems simple to me, if a test is built and it runs and 
passes, the functionality covered by the test as been tested in the 
configuration and mode a user would be expected to use it in.

Previously we had a horrible hack to work around tests that did not fit 
into memory. This distorted the results because of a number of tests 
that could not run appeared to run and pass. I have since fixed the 
build system to manage this and we now do not build those tests and they 
do not appear in the test results.

Look at cpuuse which has had this change applied, it is built in an SMP 
enabled kernel with SMP disabled. Looking at just the test results I 
would assume it is working on SMP with SMP enabled unless I look into 
the test's source and understand the implication of this define and then 
I cannot tell if the test is broken or cpuuse is not supported on SMP.

Chris