[PATCH] tester: Expand special case for minimum.exe

[Chris Johns]

On 27/5/21 2:04 pm, Kinsey Moore wrote:
> On 5/26/2021 19:22, Chris Johns wrote:
>> On 27/5/21 12:06 pm, Joel Sherrill wrote:
>>> On Wed, May 26, 2021, 7:03 PM Chris Johns <chrisj at rtems.org
>>> <mailto:chrisj at rtems.org>> wrote:
>>>
>>>      On 26/5/21 1:52 am, Kinsey Moore wrote:
>>>      > The minimum.exe test case is expected to fail as an "invalid" test in
>>>      > the tester since it is completely stripped down and does not output the
>>>      > normal test header and footer. When fatal error detection support was
>>>      > added, this caught minimum.exe and started flagging it as "fatal"
>>>      > instead of "invalid". The special-case detection of minimum.exe only
>>>      > matched on "invalid" results and not "fatal" results and so began
>>>      > flagging minimum.exe as an actual failure.>
>>>      > This change adds the special-case handling to the "fatal" test state
>>>      > handling.
>>>
>>>      Is this the right solution?
>>>
>>>      Is minimum.exe suppose to run and not fail? It would seem easy to make a
>>>      minimum.exe with nothing in it, ie minimal, that seems to pass. It would
>>> make
>>>      great marketing material.
>>>
>>>      What happens if minimum fails? I feel minimum needs to be able to run
>>> and not
>>>      fail to be a valid minimum.
>>>
>>>
>>> It is an empty thread body that doesn't print. I suppose we could add
>>> rtems_shutdown_executive(0) if that helps
>> What if the work to make it small removes something that is needed? Is minimum
>> suppose to be run and if it is how do we know it was successfull?
>>
>> My point is about the purpose of minimum. If we can never tell a run failed
>> should it be run? If we cannot tell then excluding it as a test to run for all
>> BSPs may be a simpler option that this change.
> 
> hello.exe already provides a minimum test case for something that can be
> verified to be functional, so I'd lean toward excluding minimum.exe from test
> runs of all BSPs. If minimum.exe is kept in, it needs its expected test state to
> be set to fatal-error by default for all BSPs. Unfortunately, fatal-error is
> currently bundled in as a more generic failure when being reported.

Or minimum.exe is created as a no-run executable? See the various API link only
tests for an example.

Chris