[tools] tester: Remove hard coded time limits for SIS

[Chris Johns]

On 5/7/2022 4:02 pm, Sebastian Huber wrote:
> On 05/07/2022 07:14, Chris Johns wrote:
>> On 5/7/2022 2:58 pm, Sebastian Huber wrote:
>>> On 05/07/2022 03:08, Chris Johns wrote:
>>>> On 5/7/2022 9:44 am, Joel Sherrill wrote:
>>>>> The limit removed in sis and tsim is the simulated cpu time used. If not using
>>>>> that, the behavior of the tester is to let the simulator run for so much real
>>>>> processor time.
>>>>>
>>>>> Replacing these with a command line argument is probably good but just
>>>>> removing
>>>>> these mean these simulators will just run much longer before being killed.
>>>>>
>>>>> How best to capture the distinction between target run time and host run time?
>>>>
>>>> Thank you for the explanation. I was not sure how the option effected things
>>>> and
>>>> yes it does matter we have this set correctly.
>>>>
>>>> Options can be set in the $HOME/.rtemstesterrc is via the --user-config option.
>>>> Maybe this can be used to control the time out for specific user tests?
>>>
>>> I would not make this more complicated than necessary. We have a --timeout
>>> command line option and the default timeout value can be set by *.ini files. The
>>> simulator speed is just a detail similar to running a target at 100MHz or 1GHz.
>>
>> It is actually simpler to have this option and to measure time against the cpu
>> time. The work loads on SMP hosts with qemu shows simulation timeouts are
>> difficult to get right.
> 
> I don't know what is wrong with the patch. Overruling command line options is
> just bad.

It does not work that way. When simulating the timeout in the tester is a catch
all. It may triggered if the simulator locks up. With real hardware it is the
timeout but that is a different use case. A simulator timeout is preferred when
available.

>>>> Sebastian, are some of the standard testsuite test's failing because of this
>>>> setting?
>>>
>>> Yes, with -O0 and code coverage enabled the tests run longer than usual.
>>>
>>
>> Looks to me like the timeout may need to be adjusted?
> 
> One of the performance tests needed 14 minutes on a fast computer. I would keep
> the timeouts as is for now. There is more work to do before the gcov coverage is
> generally usable.

I think it should stay and a user config used to control the setting but I will
let Joel or Jiri make the call.

Chtis