[PATCH 0/8] Add test suite to validate performance requirements

[Gedare Bloom]

I didn't really raise this in your other threads related to
performance, but how are we (RTEMS Project) defining performance
requirements? Are these simply the performance values we get by
running the tests on our release? Do we aim to hit certain targets and
reject patches if they don't? Or do we have to debate when a patch
does change a performance value? Are we going to need to run these
performance tests as part of regular testing before committing?

I'd like to get some clarity on the idea of performance requirements
as a community burden before we bake them in to RTEMS.

Gedare

On Fri, Nov 13, 2020 at 3:08 AM Sebastian Huber
<sebastian.huber at embedded-brains.de> wrote:
>
> This patch set introduces a test suite for runtimer performance
> validation tests and a test case for the Partition Manager.
>
> Sebastian Huber (8):
>   libtest: Make test case allocator configurable
>   libtest: Simplify runtime measurement support
>   libtest: Add primitive test case memory allocator
>   libtest: Rename ValidCache in FullCache
>   libtest: Allow assert checks during test begin
>   libtest: Simplify "Load" environment reporting
>   validation/ts-performance-0: Add test suite
>   validation/ts-performance-0: Add partition tests
>
>  cpukit/Makefile.am                            |   1 -
>  cpukit/include/rtems/test.h                   |  12 +-
>  cpukit/libtest/t-test-rtems-measure.c         |  93 ++--
>  ...{t-test-malloc.c => t-test-rtems-memory.c} | 110 ++---
>  cpukit/libtest/t-test.c                       | 115 ++++-
>  cpukit/libtest/testrun.c                      |   4 +
>  spec/build/cpukit/librtemstest.yml            |   2 +-
>  spec/build/testsuites/validation/grp.yml      |   2 +
>  .../testsuites/validation/performance-0.yml   |  20 +
>  testsuites/libtests/ttest01/init.c            |   3 +
>  testsuites/validation/tc-part-performance.c   | 437 ++++++++++++++++++
>  testsuites/validation/ts-default.h            | 237 ++++++++++
>  testsuites/validation/ts-performance-0.c      |  74 +++
>  testsuites/validation/ts-validation-0.c       | 155 +------
>  14 files changed, 973 insertions(+), 292 deletions(-)
>  rename cpukit/libtest/{t-test-malloc.c => t-test-rtems-memory.c} (51%)
>  create mode 100644 spec/build/testsuites/validation/performance-0.yml
>  create mode 100644 testsuites/validation/tc-part-performance.c
>  create mode 100644 testsuites/validation/ts-default.h
>  create mode 100644 testsuites/validation/ts-performance-0.c
>
> --
> 2.26.2
>
> _______________________________________________
> devel mailing list
> devel at rtems.org
> http://lists.rtems.org/mailman/listinfo/devel