[PATCH 4/6] testsuite: Add expected-fail to psim

[Chris Johns]

On 10/5/20 10:37 am, Joel Sherrill wrote:
> 
> If it is just to move things from unexpected failure to another category 
> oh, I'm okay with it. I just don't want test written off as expected 
> failure when they haven't been investigated. We have a fair number of those.

Sure, I can add this. We need to be mindful of the number of states we 
have so it does not become too complex to manage and assign tests too. 
More states means more rules and more rules means more possible errors 
and more checking.

> We also have the situation where some bsps run on multiple simulators 
> and real hardware and the results don't always align. The tcfg file 
> doesn't capture that either.

Yes, this is a difficult problem as the possible states grow another 
dimension. I have not wanted to step into that area until we have a 
suitable baseline to work from.

> But at least we aren't putting tests and in a bin where they will be 
> ignored forever. That's a step forward and explainable as a state and a 
> work activity.

Great. I will take a look and post a patch when I can find the time.

Chris