[tools] tester: Remove hard coded time limits for SIS

[Chris Johns]

> On 5 Jul 2022, at 6:23 pm, Sebastian Huber <sebastian.huber at embedded-brains.de> wrote:
> 
> On 05/07/2022 10:21, Chris Johns wrote:
>>> On 5/7/2022 4:29 pm, Sebastian Huber wrote:
>>> On 05/07/2022 08:23, Chris Johns wrote:
>>>> On 5/7/2022 4:02 pm, Sebastian Huber wrote:
>>>>> On 05/07/2022 07:14, Chris Johns wrote:
>>>>>> On 5/7/2022 2:58 pm, Sebastian Huber wrote:
>>>>>>> On 05/07/2022 03:08, Chris Johns wrote:
>>>>>>>> On 5/7/2022 9:44 am, Joel Sherrill wrote:
>>>>>>>>> The limit removed in sis and tsim is the simulated cpu time used. If not
>>>>>>>>> using
>>>>>>>>> that, the behavior of the tester is to let the simulator run for so much
>>>>>>>>> real
>>>>>>>>> processor time.
>>>>>>>>> 
>>>>>>>>> Replacing these with a command line argument is probably good but just
>>>>>>>>> removing
>>>>>>>>> these mean these simulators will just run much longer before being killed.
>>>>>>>>> 
>>>>>>>>> How best to capture the distinction between target run time and host run
>>>>>>>>> time?
>>>>>>>> Thank you for the explanation. I was not sure how the option effected things
>>>>>>>> and
>>>>>>>> yes it does matter we have this set correctly.
>>>>>>>> 
>>>>>>>> Options can be set in the $HOME/.rtemstesterrc is via the --user-config
>>>>>>>> option.
>>>>>>>> Maybe this can be used to control the time out for specific user tests?
>>>>>>> I would not make this more complicated than necessary. We have a --timeout
>>>>>>> command line option and the default timeout value can be set by *.ini
>>>>>>> files. The
>>>>>>> simulator speed is just a detail similar to running a target at 100MHz or
>>>>>>> 1GHz.
>>>>>> It is actually simpler to have this option and to measure time against the cpu
>>>>>> time. The work loads on SMP hosts with qemu shows simulation timeouts are
>>>>>> difficult to get right.
>>>>> I don't know what is wrong with the patch. Overruling command line options is
>>>>> just bad.
>>>> It does not work that way. When simulating the timeout in the tester is a catch
>>>> all. It may triggered if the simulator locks up. With real hardware it is the
>>>> timeout but that is a different use case. A simulator timeout is preferred when
>>>> available.
>>> 
>>> Ok, good. Who will fix this?
>> I am sorry I am not following. The tests have valid times for the default
>> optimisation. What is broken?
> 
> What is broken is that the --timeout command line option doesn't work with SIS because it uses hard coded values.

The timeout option is correct and your understanding of it’s purpose is wrong. Joining them as you would like would break it.

Chris