[PATCH] membench: Add memory benchmark programs

[Chris Johns]

On 21/7/2023 3:28 pm, Sebastian Huber wrote:
> On 21.07.23 03:27, Chris Johns wrote:
>> On 21/7/2023 3:51 am, Sebastian Huber wrote:
>>> On 20.07.23 18:58, Gedare Bloom wrote:
>>>> On Thu, Jul 20, 2023 at 7:42 AM Sebastian Huber
>>>> <sebastian.huber at embedded-brains.de>  wrote:
>>>>> These memory benchmark programs are not supposed to run.  Instead, they
>>>>> can be analysed on the host system to measure the memory usage of
>>>>> features.  See the membench module of rtems-central.
>>>>>
>>>> This needs some kind of documentation and probably a README inside of
>>>> membench with that information.
>>>
>>> Ok, I can add a README.md.
>>>
>>>>
>>>> This appears to be about benchmarking the program size (static memory
>>>> usage) only? If so, make that clear in the README / log note. I think
>>>> it's in the doxygen already so that's helpful.
>>>
>>> Yes, it measures only the static memory size required for certain operating
>>> system services. See 4.7 Memory Usage Benchmarks in:
>>>
>>> https://ftp.rtems.org/pub/rtems/people/sebh/rtems-6-sparc-gr740-uni-6-scf.pdf
>>
>> Should `static` be part of naming?
> 
> Yes, good idea.
> 
>>
>>>> What happens when the membench gets built, and then someone runs
>>>> $> rtems-test build/${ARCH}/${BSP}/testsuites
>>>>
>>>> Because I don't see anything that is filtering these executables.
>>>
>>> They are filtered out due to the *.norun.* pattern:>
>>> target: testsuites/membench/mem-scheduler-add-cpu.norun.exe
>>>
>>
>> Currently tests with `norun` assume the build fails if there is an issue with a
>> test. This is why we allow these tests and they are tagged `norun`.
> 
> We already have a couple of norun tests in libtests. This filtering is simple
> and works fine why would you want to change it?

I am not asking for that to change. After a build we will have a set of norun
tests and in that set are some that are be used for other purposes, eg memory
analysis, but that information is not available in the project. The norun could
be extended to be .norun.memstatic.exe and so the executables that form a
specific subset can be found and analysed.

The tests have been self contained for a long time and I would like that to
continue. ELF notes has been discussed in the past however we do not yet support
that so we need to find other ways to handling things.

>> Are they suppose to be checked or are they informational? Is something going to
>> be added to the project, for example in rtems-tools.git, to allow these tests to
>> be checked?
> 
> Currently, they are just informational,

I do not understand. What information, for what purpose and for whom?

> All the stuff to analyze this and work with
> the specification is in rtems-central.git. If you think this needs to be
> changed, then I am happy to discuss this.

Lets first understand the role these tests have. Adding them slows the build
down but that is OK if there is value in them for everyone.

> My preference is still to get rid of
> all the separate repositories and move everything back to rtems.git.

The qual work was separated for specific reasons. Those reasons are still valid.

> What is the plan for the CI flows?

I believe it is with Joel and Amar. I am waiting like everyone else.

Chris