Proposal for hardware configuration dependent performance limits

[Chris Johns]

On 26/11/20 7:04 pm, Sebastian Huber wrote:
> On 25/11/2020 21:37, Chris Johns wrote:
> 
>>> Maybe a configuration option for the RTEMS Tester should be added which allows
>>> you to set the performance hash and ignore the hash provided by the test output.
>>> This could be used to compare a custom board with values obtain from an
>>> evaluation board.
>> Why not let the tester take an alternative set of values for the same hash to
>> override the "standard" set?
> 
> Do you mean that the performance limits should be specified with a particular
> hash like this:

Yes and the hash is nothing more than a key. I had not thought much more but
what you have defined looks good and highlights how wrapping this configuration
state key is open. Your progression from values under the hash to a form of
redirection is nice. I see this evolving as we implement solutions and learn more.

Chris

> 
> limits:
>     sparc/gr712rc/XrY7u+Ae7tCTyyK7j1rNww==:
>       DirtyCache:
>         max-upper-bound: 0.000005
>         mean-upper-bound: 0.000005
> 
> In another item (for example the BSP build item) we could add a mapping from
> this hash value to a list of other hash values. For example:
> 
> performance-hash-equivalence-classes:
>   XrY7u+Ae7tCTyyK7j1rNww==:
>   - abc
>   - def
>   - ghi
> 
> We probably need also description:
> 
> performance-hash-description:
>   XrY7u+Ae7tCTyyK7j1rNww==: |
>     Description.
> 
> Taking performance limits into account for test runs requires a bit of
> maintenance. The first step is being able to do it. The next step is to discuss
> if and how this could be done in the scope of the RTEMS Project.
>