[PATCH] tester: Make the SIS time limit user configurable
chrisj at rtems.org
Wed Jul 6 21:55:50 UTC 2022
On 6/7/2022 6:00 pm, Sebastian Huber wrote:
> Yes, if tests go wrong the tester can kill a test execution after the specified
Killing should be taken as a sign something in the test equipment is broken.
> Why do we need this arbitrary SIS -tlim of 400 s?
There was a few values and I select this one based on those. If you have a
better value please say.
I do not recommend removing the time limit option from testing. If you want to
operate that way create a user config with:
> Which problem does this solve?
Repeatable test results across wide ranging hosts and host operating systems.
> Why can't we let the tests run in SIS without a limit just like we do it for
Qemu is painful to make work in a consistent and reliable way.
> Normally, if a test is done, it terminates the SIS execution.
The time outs are for the cases that end abnormally.
> The new performance tests can be used to catch performance regressions.
We need to consider the existing benchmarks.
> In the
> long run I think we need a more modular approach. For example, one component
> which runs tests and reports the test output. Another component which analyses
> the test output. The test outputs can be archived.
The rtems-test command is required to report regressions and as simply as
possible. A user wants to build, test then know what they have matches what we
More information about the devel