[PATCH] tester: Expand special case for minimum.exe

[Kinsey Moore]

On 5/26/2021 19:22, Chris Johns wrote:
> On 27/5/21 12:06 pm, Joel Sherrill wrote:
>> On Wed, May 26, 2021, 7:03 PM Chris Johns <chrisj at rtems.org
>> <mailto:chrisj at rtems.org>> wrote:
>>
>>      On 26/5/21 1:52 am, Kinsey Moore wrote:
>>      > The minimum.exe test case is expected to fail as an "invalid" test in
>>      > the tester since it is completely stripped down and does not output the
>>      > normal test header and footer. When fatal error detection support was
>>      > added, this caught minimum.exe and started flagging it as "fatal"
>>      > instead of "invalid". The special-case detection of minimum.exe only
>>      > matched on "invalid" results and not "fatal" results and so began
>>      > flagging minimum.exe as an actual failure.>
>>      > This change adds the special-case handling to the "fatal" test state
>>      > handling.
>>
>>      Is this the right solution?
>>
>>      Is minimum.exe suppose to run and not fail? It would seem easy to make a
>>      minimum.exe with nothing in it, ie minimal, that seems to pass. It would make
>>      great marketing material.
>>
>>      What happens if minimum fails? I feel minimum needs to be able to run and not
>>      fail to be a valid minimum.
>>
>>
>> It is an empty thread body that doesn't print. I suppose we could add
>> rtems_shutdown_executive(0) if that helps
> What if the work to make it small removes something that is needed? Is minimum
> suppose to be run and if it is how do we know it was successfull?
>
> My point is about the purpose of minimum. If we can never tell a run failed
> should it be run? If we cannot tell then excluding it as a test to run for all
> BSPs may be a simpler option that this change.

hello.exe already provides a minimum test case for something that can be 
verified to be functional, so I'd lean toward excluding minimum.exe from 
test runs of all BSPs. If minimum.exe is kept in, it needs its expected 
test state to be set to fatal-error by default for all BSPs. 
Unfortunately, fatal-error is currently bundled in as a more generic 
failure when being reported.


Kinsey