crypt01 execution time

[Chris Johns]

On 27/11/2014 12:48 am, Joel Sherrill wrote:
> On 11/26/2014 01:12 AM, Sebastian Huber wrote:
>> On 25/11/14 23:25, Joel Sherrill wrote:
>>> How long is this test supposed to run?
>>>
>>> It takes 4:42 using sis on my computer which is a 2.9 Ghz i7 .
>> SIS is a slow simulator.  On Qemu it runs much faster.
>>
>>> Is there anything to do? Split it?
>> Splitting it up doesn't reduce the overall test time.  The test cases in
>> this test are standard test cases.  I am not so fond of removing them,
>> but in case the test time is too long for you then you can drop the ones
>> with the many rounds.
>>
> Chris.. my old test scripts knew some tests ran
> quickly and some ran long. The BSP settings could
> adjust the timeout period based on the test name.

We need a BSP database in the tester and this can be part of it.

> We shouldn't wait 5 minutes for every test to time
> out when 95+% run in milliseconds.

The time out only happens if the test fails so lets fix all tests. :)

>
> How can we account for this in rtems-tester?
>

I feel a BSP specific time out setting is only part of the answer. We 
have simulators running on hosts of various speeds plus we have tests 
that have different timing needs and this makes the problem a little 
more complex. There is also the ability to change a clock rate on a 
target and users do this for power and other reasons.

I am wondering about the tests telling us what the time out is plus 
adding tags that reset the time out in the tester. I also wonder if we 
create a test that runs first which "calibrates" the performance. The 
test prints a datum time and the tester takes the real time it runs for 
and scales the time out.

Chris