SMP06 test failure on LEON

[Daniel Hellstrom]

On 11/12/2013 10:57 AM, Sebastian Huber wrote:
> On 2013-11-11 14:14, Daniel Hellstrom wrote:
>> On 11/11/2013 11:25 AM, Sebastian Huber wrote:
>>> On 2013-11-11 02:22, Gedare Bloom wrote:
>>>> I think perhaps the scheduler was changed and no longer matches the assumptions
>>>> of this test..
>>>
>>> Yes, this is the case.  I didn't look at the test and relied on the "END OF
>>> TEST" messages which normally indicates a successful test completion.  I
>>> removed this test now.
>>
>> Okay. Thanks.
>>
>> Generally I have had some problems verifying the test results of the tests. I
>> would like to log the console output and compare it against the logs in the
>> .scn files, do you know how this is done? I'm guessing that the CPU printouts
>> depends on the platform, for example which CPU runs the Init task makes the
>> printout be different. Do you have a trick on the PPC or is it matching the
>> console logs? I guess this question is not only a SMP issue.
>
> Unfortunately the RTEMS test suite output is not that useful currently.  A test program should only print out "END OF TEST" if it was successful.  Most tests do this.  Some tests (e.g. the 
> smptests/smp06) have bugs and produce false positives.  An open project is to convert the RTEMS test suite to use a unit test framework with better human and machine readable output.
>
That is good news. I don't know how many hours in total I've spent over the last years examining and comparing test output..

I have seen other problems as well, for example that the SMP09 .scn file can't be right (zeros on TA2 and TA3) but on my hardware the result looks good, and the output of such a test will be different 
on different platforms/frequency. SMP05 and SMP08 isn't documented, so it is hard to tell exactly what is tested etc.

Daniel