[tools] tester: Remove hard coded time limits for SIS

[Jiri Gaisler]

On 7/5/22 12:31, Chris Johns wrote:
>
>> On 5 Jul 2022, at 6:23 pm, Sebastian Huber <sebastian.huber at embedded-brains.de> wrote:
>>
>> On 05/07/2022 10:21, Chris Johns wrote:
>>>> On 5/7/2022 4:29 pm, Sebastian Huber wrote:
>>>> On 05/07/2022 08:23, Chris Johns wrote:
>>>>> On 5/7/2022 4:02 pm, Sebastian Huber wrote:
>>>>>> On 05/07/2022 07:14, Chris Johns wrote:
>>>>>>> On 5/7/2022 2:58 pm, Sebastian Huber wrote:
>>>>>>>> On 05/07/2022 03:08, Chris Johns wrote:
>>>>>>>>> On 5/7/2022 9:44 am, Joel Sherrill wrote:
>>>>>>>>>> The limit removed in sis and tsim is the simulated cpu time used. If not
>>>>>>>>>> using
>>>>>>>>>> that, the behavior of the tester is to let the simulator run for so much
>>>>>>>>>> real
>>>>>>>>>> processor time.
>>>>>>>>>>
>>>>>>>>>> Replacing these with a command line argument is probably good but just
>>>>>>>>>> removing
>>>>>>>>>> these mean these simulators will just run much longer before being killed.
>>>>>>>>>>
>>>>>>>>>> How best to capture the distinction between target run time and host run
>>>>>>>>>> time?
>>>>>>>>> Thank you for the explanation. I was not sure how the option effected things
>>>>>>>>> and
>>>>>>>>> yes it does matter we have this set correctly.
>>>>>>>>>
>>>>>>>>> Options can be set in the $HOME/.rtemstesterrc is via the --user-config
>>>>>>>>> option.
>>>>>>>>> Maybe this can be used to control the time out for specific user tests?
>>>>>>>> I would not make this more complicated than necessary. We have a --timeout
>>>>>>>> command line option and the default timeout value can be set by *.ini
>>>>>>>> files. The
>>>>>>>> simulator speed is just a detail similar to running a target at 100MHz or
>>>>>>>> 1GHz.
>>>>>>> It is actually simpler to have this option and to measure time against the cpu
>>>>>>> time. The work loads on SMP hosts with qemu shows simulation timeouts are
>>>>>>> difficult to get right.
>>>>>> I don't know what is wrong with the patch. Overruling command line options is
>>>>>> just bad.
>>>>> It does not work that way. When simulating the timeout in the tester is a catch
>>>>> all. It may triggered if the simulator locks up. With real hardware it is the
>>>>> timeout but that is a different use case. A simulator timeout is preferred when
>>>>> available.
>>>> Ok, good. Who will fix this?
>>> I am sorry I am not following. The tests have valid times for the default
>>> optimisation. What is broken?
>> What is broken is that the --timeout command line option doesn't work with SIS because it uses hard coded values.
> The timeout option is correct and your understanding of it’s purpose is wrong. Joining them as you would like would break it.

I think that Sebastian has a point, a hard-coded value in the sis script prevents changing the time-out value from rtems-test.

The timeout value in sis is in simulated CPU time, not host time. I am not sure how it works on the other simulators. If you would prefer host time time-outs, let me know and I will modify sis for this.

Jiri.