How to Classify Intermittent Test Failures

Joel Sherrill joel at rtems.org
Mon Feb 1 16:42:01 UTC 2021


Hi

On the aarch64 qemu testing, we are seeing some tests which seem to pass
most of the time but fail intermittently. It appears to be based somewhat
on host load but there may be other factors.

There does not appear to be a good test results state for these. Marking
them expected pass or fail means they will get flagged incorrectly
sometimes.

I don't see not running them as a good option. Beyond adding a new state to
reflect this oddity, any suggestions?

--joel
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.rtems.org/pipermail/devel/attachments/20210201/3a888926/attachment.html>


More information about the devel mailing list