[tools] tester: Remove hard coded time limits for SIS

[Jiri Gaisler]

On 7/5/22 12:45, Sebastian Huber wrote:
>
>
> On 05/07/2022 12:31, Chris Johns wrote:
>>
>>> On 5 Jul 2022, at 6:23 pm, Sebastian Huber<sebastian.huber at embedded-brains.de>  wrote:
>>>
>>> On 05/07/2022 10:21, Chris Johns wrote:
>>>>> On 5/7/2022 4:29 pm, Sebastian Huber wrote:
>>>>> On 05/07/2022 08:23, Chris Johns wrote:
>>>>>> On 5/7/2022 4:02 pm, Sebastian Huber wrote:
>>>>>>> On 05/07/2022 07:14, Chris Johns wrote:
>>>>>>>> On 5/7/2022 2:58 pm, Sebastian Huber wrote:
>>>>>>>>> On 05/07/2022 03:08, Chris Johns wrote:
>>>>>>>>>> On 5/7/2022 9:44 am, Joel Sherrill wrote:
>>>>>>>>>>> The limit removed in sis and tsim is the simulated cpu time used. If not
>>>>>>>>>>> using
>>>>>>>>>>> that, the behavior of the tester is to let the simulator run for so much
>>>>>>>>>>> real
>>>>>>>>>>> processor time.
>>>>>>>>>>>
>>>>>>>>>>> Replacing these with a command line argument is probably good but just
>>>>>>>>>>> removing
>>>>>>>>>>> these mean these simulators will just run much longer before being killed.
>>>>>>>>>>>
>>>>>>>>>>> How best to capture the distinction between target run time and host run
>>>>>>>>>>> time?
>>>>>>>>>> Thank you for the explanation. I was not sure how the option effected things
>>>>>>>>>> and
>>>>>>>>>> yes it does matter we have this set correctly.
>>>>>>>>>>
>>>>>>>>>> Options can be set in the $HOME/.rtemstesterrc is via the --user-config
>>>>>>>>>> option.
>>>>>>>>>> Maybe this can be used to control the time out for specific user tests?
>>>>>>>>> I would not make this more complicated than necessary. We have a --timeout
>>>>>>>>> command line option and the default timeout value can be set by *.ini
>>>>>>>>> files. The
>>>>>>>>> simulator speed is just a detail similar to running a target at 100MHz or
>>>>>>>>> 1GHz.
>>>>>>>> It is actually simpler to have this option and to measure time against the cpu
>>>>>>>> time. The work loads on SMP hosts with qemu shows simulation timeouts are
>>>>>>>> difficult to get right.
>>>>>>> I don't know what is wrong with the patch. Overruling command line options is
>>>>>>> just bad.
>>>>>> It does not work that way. When simulating the timeout in the tester is a catch
>>>>>> all. It may triggered if the simulator locks up. With real hardware it is the
>>>>>> timeout but that is a different use case. A simulator timeout is preferred when
>>>>>> available.
>>>>> Ok, good. Who will fix this?
>>>> I am sorry I am not following. The tests have valid times for the default
>>>> optimisation. What is broken?
>>> What is broken is that the --timeout command line option doesn't work with SIS because it uses hard coded values.
>> The timeout option is correct and your understanding of it’s purpose is wrong. Joining them as you would like would break it.
>
> It would be nice if someone could offer me a way to run tests which exceed the hard coded SIS timeout values?

sis accepts several -tlim options. The last one will be the active one. So rtems-test could add an extra -tlim option at the end of the sis parameters which would override the default one.