[PATCH] membench: Add memory benchmark programs

[Chris Johns]

On 21/7/2023 3:51 am, Sebastian Huber wrote:
> On 20.07.23 18:58, Gedare Bloom wrote:
>> On Thu, Jul 20, 2023 at 7:42 AM Sebastian Huber
>> <sebastian.huber at embedded-brains.de>  wrote:
>>> These memory benchmark programs are not supposed to run.  Instead, they
>>> can be analysed on the host system to measure the memory usage of
>>> features.  See the membench module of rtems-central.
>>>
>> This needs some kind of documentation and probably a README inside of
>> membench with that information.
> 
> Ok, I can add a README.md.
> 
>>
>> This appears to be about benchmarking the program size (static memory
>> usage) only? If so, make that clear in the README / log note. I think
>> it's in the doxygen already so that's helpful.
> 
> Yes, it measures only the static memory size required for certain operating
> system services. See 4.7 Memory Usage Benchmarks in:
> 
> https://ftp.rtems.org/pub/rtems/people/sebh/rtems-6-sparc-gr740-uni-6-scf.pdf

Should `static` be part of naming?

>> What happens when the membench gets built, and then someone runs
>> $> rtems-test build/${ARCH}/${BSP}/testsuites
>>
>> Because I don't see anything that is filtering these executables.
> 
> They are filtered out due to the *.norun.* pattern:>
> target: testsuites/membench/mem-scheduler-add-cpu.norun.exe
> 

Currently tests with `norun` assume the build fails if there is an issue with a
test. This is why we allow these tests and they are tagged `norun`.

I am happy to see these tests however I am not comfortable about the reference
and dependency to rtems-central to understand or analyse them. I have looked at
the test source and I do not understand their purpose. Are they generated?

Are they suppose to be checked or are they informational? Is something going to
be added to the project, for example in rtems-tools.git, to allow these tests to
be checked?

I am sorry, until I understand more about these tests, I have to say no. I would
much prefer to say yes but I will need a hand in understanding them.

My concern is simple. I make a change and it effects something in these tests
yet I have no ability to know if I have triggered a failure.

We will be adding CI flows to improve the quality of what we do and this
approach side steps that effort however I am not sure about this until I
understand more about them.

Chris