[PATCH] membench: Add memory benchmark programs

Thu Jul 20 16:58:34 UTC 2023

On Thu, Jul 20, 2023 at 7:42 AM Sebastian Huber
<sebastian.huber at embedded-brains.de> wrote:
>
> These memory benchmark programs are not supposed to run.  Instead, they
> can be analysed on the host system to measure the memory usage of
> features.  See the membench module of rtems-central.
>
This needs some kind of documentation and probably a README inside of
membench with that information.

This appears to be about benchmarking the program size (static memory
usage) only? If so, make that clear in the README / log note. I think
it's in the doxygen already so that's helpful.

What happens when the membench gets built, and then someone runs
$> rtems-test build/${ARCH}/${BSP}/testsuites

Because I don't see anything that is filtering these executables.

> Update #3716.
> ---
>  cpukit/doxygen/top-level-groups.h             |   8 +
>  spec/build/testsuites/grp.yml                 |   2 +
>  spec/build/testsuites/membench/grp.yml        | 121 +++++++++++++++
>  .../membench/mem-barrier-wait-rel-del.yml     |  19 +++
>  .../membench/mem-barrier-wait-rel.yml         |  19 +++
>  .../testsuites/membench/mem-bsp-clock.yml     |  19 +++
>  .../membench/mem-clock-get-uptime.yml         |  19 +++
>  .../membench/mem-clock-set-get-tod.yml        |  19 +++
>  .../testsuites/membench/mem-clock-set.yml     |  19 +++
>  .../testsuites/membench/mem-event-snd-rcv.yml |  19 +++
>  .../testsuites/membench/mem-fatal-fatal.yml   |  19 +++
>  .../membench/mem-message-bcst-rcv.yml         |  19 +++
>  .../membench/mem-message-snd-rcv-del.yml      |  19 +++
>  .../membench/mem-message-snd-rcv.yml          |  19 +++
>  .../membench/mem-message-ugt-rcv.yml          |  19 +++
>  .../membench/mem-part-get-ret-del.yml         |  19 +++
>  .../testsuites/membench/mem-part-get-ret.yml  |  19 +++
>  .../membench/mem-ratemon-period-del.yml       |  19 +++
>  .../membench/mem-ratemon-period.yml           |  19 +++
>  .../testsuites/membench/mem-rtems-basic.yml   |  19 +++
>  .../testsuites/membench/mem-rtems-smp-1.yml   |  19 +++
>  .../membench/mem-rtems-smp-global-2.yml       |  19 +++
>  .../membench/mem-rtems-smp-global-4.yml       |  19 +++
>  .../membench/mem-rtems-smp-part-2.yml         |  19 +++
>  .../membench/mem-rtems-smp-part-4.yml         |  19 +++
>  .../membench/mem-scheduler-add-cpu.yml        |  19 +++
>  .../membench/mem-scheduler-rm-cpu.yml         |  19 +++
>  .../membench/mem-sem-obt-rel-del.yml          |  19 +++
>  .../testsuites/membench/mem-sem-obt-rel.yml   |  19 +++
>  .../membench/mem-signal-catch-snd.yml         |  19 +++
>  .../testsuites/membench/mem-task-delete.yml   |  19 +++
>  .../testsuites/membench/mem-task-exit.yml     |  19 +++
>  .../membench/mem-task-get-affinity.yml        |  19 +++
>  .../membench/mem-task-get-priority.yml        |  19 +++
>  .../membench/mem-task-get-scheduler.yml       |  19 +++
>  .../testsuites/membench/mem-task-mode.yml     |  19 +++
>  .../testsuites/membench/mem-task-restart.yml  |  19 +++
>  .../membench/mem-task-set-affinity.yml        |  19 +++
>  .../membench/mem-task-set-priority.yml        |  19 +++
>  .../membench/mem-task-set-scheduler.yml       |  19 +++
>  .../testsuites/membench/mem-task-sus-res.yml  |  19 +++
>  .../membench/mem-task-wake-after.yml          |  19 +++
>  .../membench/mem-task-wake-when.yml           |  19 +++
>  .../testsuites/membench/mem-timer-after.yml   |  19 +++
>  .../testsuites/membench/mem-timer-cancel.yml  |  19 +++
>  .../testsuites/membench/mem-timer-delete.yml  |  19 +++
>  .../testsuites/membench/mem-timer-reset.yml   |  19 +++
>  .../membench/mem-timer-srv-after.yml          |  19 +++
>  .../membench/mem-timer-srv-init.yml           |  19 +++
>  .../membench/mem-timer-srv-when.yml           |  19 +++
>  .../testsuites/membench/mem-timer-when.yml    |  19 +++
>  .../membench/mem-userext-create.yml           |  19 +++
>  .../membench/mem-userext-delete.yml           |  19 +++
>  spec/build/testsuites/optmembench.yml         |  16 ++
>  .../membench/mem-barrier-wait-rel-del.c       | 116 +++++++++++++++
>  testsuites/membench/mem-barrier-wait-rel.c    | 114 ++++++++++++++
>  testsuites/membench/mem-bsp-clock.c           | 109 ++++++++++++++
>  testsuites/membench/mem-clock-get-uptime.c    | 108 ++++++++++++++
>  testsuites/membench/mem-clock-set-get-tod.c   | 110 ++++++++++++++
>  testsuites/membench/mem-clock-set.c           | 108 ++++++++++++++
>  testsuites/membench/mem-event-snd-rcv.c       | 110 ++++++++++++++
>  testsuites/membench/mem-fatal-fatal.c         | 108 ++++++++++++++
>  testsuites/membench/mem-message-bcst-rcv.c    | 111 ++++++++++++++
>  testsuites/membench/mem-message-snd-rcv-del.c | 114 ++++++++++++++
>  testsuites/membench/mem-message-snd-rcv.c     | 111 ++++++++++++++
>  testsuites/membench/mem-message-ugt-rcv.c     | 111 ++++++++++++++
>  testsuites/membench/mem-part-get-ret-del.c    | 115 ++++++++++++++
>  testsuites/membench/mem-part-get-ret.c        | 114 ++++++++++++++
>  testsuites/membench/mem-ratemon-period-del.c  | 114 ++++++++++++++
>  testsuites/membench/mem-ratemon-period.c      | 113 ++++++++++++++
>  testsuites/membench/mem-rtems-basic.c         | 111 ++++++++++++++
>  testsuites/membench/mem-rtems-smp-1.c         | 115 ++++++++++++++
>  testsuites/membench/mem-rtems-smp-global-2.c  | 115 ++++++++++++++
>  testsuites/membench/mem-rtems-smp-global-4.c  | 115 ++++++++++++++
>  testsuites/membench/mem-rtems-smp-part-2.c    | 132 +++++++++++++++++
>  testsuites/membench/mem-rtems-smp-part-4.c    | 140 ++++++++++++++++++
>  testsuites/membench/mem-scheduler-add-cpu.c   | 114 ++++++++++++++
>  testsuites/membench/mem-scheduler-rm-cpu.c    | 115 ++++++++++++++
>  testsuites/membench/mem-sem-obt-rel-del.c     | 115 ++++++++++++++
>  testsuites/membench/mem-sem-obt-rel.c         | 114 ++++++++++++++
>  testsuites/membench/mem-signal-catch-snd.c    | 110 ++++++++++++++
>  testsuites/membench/mem-task-delete.c         | 108 ++++++++++++++
>  testsuites/membench/mem-task-exit.c           | 108 ++++++++++++++
>  testsuites/membench/mem-task-get-affinity.c   | 108 ++++++++++++++
>  testsuites/membench/mem-task-get-priority.c   | 108 ++++++++++++++
>  testsuites/membench/mem-task-get-scheduler.c  | 108 ++++++++++++++
>  testsuites/membench/mem-task-mode.c           | 108 ++++++++++++++
>  testsuites/membench/mem-task-restart.c        | 108 ++++++++++++++
>  testsuites/membench/mem-task-set-affinity.c   | 108 ++++++++++++++
>  testsuites/membench/mem-task-set-priority.c   | 108 ++++++++++++++
>  testsuites/membench/mem-task-set-scheduler.c  | 108 ++++++++++++++
>  testsuites/membench/mem-task-sus-res.c        | 110 ++++++++++++++
>  testsuites/membench/mem-task-wake-after.c     | 108 ++++++++++++++
>  testsuites/membench/mem-task-wake-when.c      | 108 ++++++++++++++
>  testsuites/membench/mem-timer-after.c         | 110 ++++++++++++++
>  testsuites/membench/mem-timer-cancel.c        | 110 ++++++++++++++
>  testsuites/membench/mem-timer-delete.c        | 110 ++++++++++++++
>  testsuites/membench/mem-timer-reset.c         | 110 ++++++++++++++
>  testsuites/membench/mem-timer-srv-after.c     | 110 ++++++++++++++
>  testsuites/membench/mem-timer-srv-init.c      | 123 +++++++++++++++
>  testsuites/membench/mem-timer-srv-when.c      | 110 ++++++++++++++
>  testsuites/membench/mem-timer-when.c          | 110 ++++++++++++++
>  testsuites/membench/mem-userext-create.c      | 111 ++++++++++++++
>  testsuites/membench/mem-userext-delete.c      | 112 ++++++++++++++
>  104 files changed, 6701 insertions(+)
>  create mode 100644 spec/build/testsuites/membench/grp.yml
>  create mode 100644 spec/build/testsuites/membench/mem-barrier-wait-rel-del.yml
>  create mode 100644 spec/build/testsuites/membench/mem-barrier-wait-rel.yml
>  create mode 100644 spec/build/testsuites/membench/mem-bsp-clock.yml
>  create mode 100644 spec/build/testsuites/membench/mem-clock-get-uptime.yml
>  create mode 100644 spec/build/testsuites/membench/mem-clock-set-get-tod.yml
>  create mode 100644 spec/build/testsuites/membench/mem-clock-set.yml
>  create mode 100644 spec/build/testsuites/membench/mem-event-snd-rcv.yml
>  create mode 100644 spec/build/testsuites/membench/mem-fatal-fatal.yml
>  create mode 100644 spec/build/testsuites/membench/mem-message-bcst-rcv.yml
>  create mode 100644 spec/build/testsuites/membench/mem-message-snd-rcv-del.yml
>  create mode 100644 spec/build/testsuites/membench/mem-message-snd-rcv.yml
>  create mode 100644 spec/build/testsuites/membench/mem-message-ugt-rcv.yml
>  create mode 100644 spec/build/testsuites/membench/mem-part-get-ret-del.yml
>  create mode 100644 spec/build/testsuites/membench/mem-part-get-ret.yml
>  create mode 100644 spec/build/testsuites/membench/mem-ratemon-period-del.yml
>  create mode 100644 spec/build/testsuites/membench/mem-ratemon-period.yml
>  create mode 100644 spec/build/testsuites/membench/mem-rtems-basic.yml
>  create mode 100644 spec/build/testsuites/membench/mem-rtems-smp-1.yml
>  create mode 100644 spec/build/testsuites/membench/mem-rtems-smp-global-2.yml
>  create mode 100644 spec/build/testsuites/membench/mem-rtems-smp-global-4.yml
>  create mode 100644 spec/build/testsuites/membench/mem-rtems-smp-part-2.yml
>  create mode 100644 spec/build/testsuites/membench/mem-rtems-smp-part-4.yml
>  create mode 100644 spec/build/testsuites/membench/mem-scheduler-add-cpu.yml
>  create mode 100644 spec/build/testsuites/membench/mem-scheduler-rm-cpu.yml
>  create mode 100644 spec/build/testsuites/membench/mem-sem-obt-rel-del.yml
>  create mode 100644 spec/build/testsuites/membench/mem-sem-obt-rel.yml
>  create mode 100644 spec/build/testsuites/membench/mem-signal-catch-snd.yml
>  create mode 100644 spec/build/testsuites/membench/mem-task-delete.yml
>  create mode 100644 spec/build/testsuites/membench/mem-task-exit.yml
>  create mode 100644 spec/build/testsuites/membench/mem-task-get-affinity.yml
>  create mode 100644 spec/build/testsuites/membench/mem-task-get-priority.yml
>  create mode 100644 spec/build/testsuites/membench/mem-task-get-scheduler.yml
>  create mode 100644 spec/build/testsuites/membench/mem-task-mode.yml
>  create mode 100644 spec/build/testsuites/membench/mem-task-restart.yml
>  create mode 100644 spec/build/testsuites/membench/mem-task-set-affinity.yml
>  create mode 100644 spec/build/testsuites/membench/mem-task-set-priority.yml
>  create mode 100644 spec/build/testsuites/membench/mem-task-set-scheduler.yml
>  create mode 100644 spec/build/testsuites/membench/mem-task-sus-res.yml
>  create mode 100644 spec/build/testsuites/membench/mem-task-wake-after.yml
>  create mode 100644 spec/build/testsuites/membench/mem-task-wake-when.yml
>  create mode 100644 spec/build/testsuites/membench/mem-timer-after.yml
>  create mode 100644 spec/build/testsuites/membench/mem-timer-cancel.yml
>  create mode 100644 spec/build/testsuites/membench/mem-timer-delete.yml
>  create mode 100644 spec/build/testsuites/membench/mem-timer-reset.yml
>  create mode 100644 spec/build/testsuites/membench/mem-timer-srv-after.yml
>  create mode 100644 spec/build/testsuites/membench/mem-timer-srv-init.yml
>  create mode 100644 spec/build/testsuites/membench/mem-timer-srv-when.yml
>  create mode 100644 spec/build/testsuites/membench/mem-timer-when.yml
>  create mode 100644 spec/build/testsuites/membench/mem-userext-create.yml
>  create mode 100644 spec/build/testsuites/membench/mem-userext-delete.yml
>  create mode 100644 spec/build/testsuites/optmembench.yml
>  create mode 100644 testsuites/membench/mem-barrier-wait-rel-del.c
>  create mode 100644 testsuites/membench/mem-barrier-wait-rel.c
>  create mode 100644 testsuites/membench/mem-bsp-clock.c
>  create mode 100644 testsuites/membench/mem-clock-get-uptime.c
>  create mode 100644 testsuites/membench/mem-clock-set-get-tod.c
>  create mode 100644 testsuites/membench/mem-clock-set.c
>  create mode 100644 testsuites/membench/mem-event-snd-rcv.c
>  create mode 100644 testsuites/membench/mem-fatal-fatal.c
>  create mode 100644 testsuites/membench/mem-message-bcst-rcv.c
>  create mode 100644 testsuites/membench/mem-message-snd-rcv-del.c
>  create mode 100644 testsuites/membench/mem-message-snd-rcv.c
>  create mode 100644 testsuites/membench/mem-message-ugt-rcv.c
>  create mode 100644 testsuites/membench/mem-part-get-ret-del.c
>  create mode 100644 testsuites/membench/mem-part-get-ret.c
>  create mode 100644 testsuites/membench/mem-ratemon-period-del.c
>  create mode 100644 testsuites/membench/mem-ratemon-period.c
>  create mode 100644 testsuites/membench/mem-rtems-basic.c
>  create mode 100644 testsuites/membench/mem-rtems-smp-1.c
>  create mode 100644 testsuites/membench/mem-rtems-smp-global-2.c
>  create mode 100644 testsuites/membench/mem-rtems-smp-global-4.c
>  create mode 100644 testsuites/membench/mem-rtems-smp-part-2.c
>  create mode 100644 testsuites/membench/mem-rtems-smp-part-4.c
>  create mode 100644 testsuites/membench/mem-scheduler-add-cpu.c
>  create mode 100644 testsuites/membench/mem-scheduler-rm-cpu.c
>  create mode 100644 testsuites/membench/mem-sem-obt-rel-del.c
>  create mode 100644 testsuites/membench/mem-sem-obt-rel.c
>  create mode 100644 testsuites/membench/mem-signal-catch-snd.c
>  create mode 100644 testsuites/membench/mem-task-delete.c
>  create mode 100644 testsuites/membench/mem-task-exit.c
>  create mode 100644 testsuites/membench/mem-task-get-affinity.c
>  create mode 100644 testsuites/membench/mem-task-get-priority.c
>  create mode 100644 testsuites/membench/mem-task-get-scheduler.c
>  create mode 100644 testsuites/membench/mem-task-mode.c
>  create mode 100644 testsuites/membench/mem-task-restart.c
>  create mode 100644 testsuites/membench/mem-task-set-affinity.c
>  create mode 100644 testsuites/membench/mem-task-set-priority.c
>  create mode 100644 testsuites/membench/mem-task-set-scheduler.c
>  create mode 100644 testsuites/membench/mem-task-sus-res.c
>  create mode 100644 testsuites/membench/mem-task-wake-after.c
>  create mode 100644 testsuites/membench/mem-task-wake-when.c
>  create mode 100644 testsuites/membench/mem-timer-after.c
>  create mode 100644 testsuites/membench/mem-timer-cancel.c
>  create mode 100644 testsuites/membench/mem-timer-delete.c
>  create mode 100644 testsuites/membench/mem-timer-reset.c
>  create mode 100644 testsuites/membench/mem-timer-srv-after.c
>  create mode 100644 testsuites/membench/mem-timer-srv-init.c
>  create mode 100644 testsuites/membench/mem-timer-srv-when.c
>  create mode 100644 testsuites/membench/mem-timer-when.c
>  create mode 100644 testsuites/membench/mem-userext-create.c
>  create mode 100644 testsuites/membench/mem-userext-delete.c
>
> diff --git a/cpukit/doxygen/top-level-groups.h b/cpukit/doxygen/top-level-groups.h
> index fd32db347a..8a50d32a4d 100644
> --- a/cpukit/doxygen/top-level-groups.h
> +++ b/cpukit/doxygen/top-level-groups.h
> @@ -81,6 +81,14 @@
>   * @brief This group contains the test suites related to filesystem tests.
>   */
>
> +/**
> + * @defgroup RTEMSTestSuitesMembench Memory Benchmark Test Suites
> + *
> + * @ingroup RTEMSTestSuites
> + *
> + * @brief This group contains the test suites related to memory benchmarks.
Clarify the purpose and scope (static memory usage of features in isolation)

I'm not convinced about the name, as a memory benchmark would imply
measuring dynamic usage too. This is really more of a "layout
benchmark" if I understood rtemsspec/membench.py correctly.

-Gedare