[PATCH] spec: Add QEMU test annotations
Gedare Bloom
gedare at rtems.org
Fri Jul 7 17:04:17 UTC 2023
On Thu, Jul 6, 2023 at 12:48 PM Kinsey Moore <kinsey.moore at oarcorp.com> wrote:
>
> QEMU is known to fail certain tests intermittently due to clock tick
> delivery issues. This defines those tests as intermittent for BSPs
> intended to run on QEMU alone.
>
> Updates #4922
> Updates #4072
> ---
> spec/build/bsps/aarch64/a53/tsta53.yml | 33 +--------
> spec/build/bsps/aarch64/a72/tsta72.yml | 33 +--------
> .../bsps/aarch64/xilinx-versal/tstqemu.yml | 4 +-
> .../bsps/aarch64/xilinx-zynqmp/tstqemu.yml | 33 +--------
> .../arm/realview-pbx-a9/bsprealviewpbxa9.yml | 2 +
> spec/build/bsps/arm/xilinx-zynq/bspqemu.yml | 2 +
> spec/build/bsps/tstqemu.yml | 72 +++++++++++++++++++
> 7 files changed, 88 insertions(+), 91 deletions(-)
> create mode 100644 spec/build/bsps/tstqemu.yml
>
> diff --git a/spec/build/bsps/aarch64/a53/tsta53.yml b/spec/build/bsps/aarch64/a53/tsta53.yml
> index a3d8c9810d..adf6d85ae1 100644
> --- a/spec/build/bsps/aarch64/a53/tsta53.yml
> +++ b/spec/build/bsps/aarch64/a53/tsta53.yml
> @@ -6,40 +6,13 @@ actions:
> state: exclude
> tests:
> - minimum
> -- set-test-state:
> - reason: |
> - Tests that are passing intermittently.
> - state: indeterminate
> - tests:
> - - psx12
> - - psxtimes01
> - - rtmonuse
> - - rtmonusxtimes01
> - - sp04
> - - sp20
> - - sp68
> - - sp69
> - - spcpucounter01
> - - spedfsched02
> - - spedfsched04
> - - sprmsched01
> - - sptimecounter02
> - - sptimecounter04
> - - ttest02
> -- set-test-state:
> - reason: |
> - Tests that pass nominally, but fail under Qemu when the host is under
> - heavy load.
> - state: indeterminate
> - tests:
> - - spintrcritical03
> - - spintrcritical04
> - - spintrcritical05
> build-type: option
> copyrights:
> - Copyright (C) 2020 On-Line Applications Research (OAR)
> default: []
> description: ''
> enabled-by: true
> -links: []
> +links:
> +- role: build-dependency
> + uid: ../../tstqemu
> type: build
> diff --git a/spec/build/bsps/aarch64/a72/tsta72.yml b/spec/build/bsps/aarch64/a72/tsta72.yml
> index a3d8c9810d..adf6d85ae1 100644
> --- a/spec/build/bsps/aarch64/a72/tsta72.yml
> +++ b/spec/build/bsps/aarch64/a72/tsta72.yml
> @@ -6,40 +6,13 @@ actions:
> state: exclude
> tests:
> - minimum
> -- set-test-state:
> - reason: |
> - Tests that are passing intermittently.
> - state: indeterminate
> - tests:
> - - psx12
> - - psxtimes01
> - - rtmonuse
> - - rtmonusxtimes01
> - - sp04
> - - sp20
> - - sp68
> - - sp69
> - - spcpucounter01
> - - spedfsched02
> - - spedfsched04
> - - sprmsched01
> - - sptimecounter02
> - - sptimecounter04
> - - ttest02
> -- set-test-state:
> - reason: |
> - Tests that pass nominally, but fail under Qemu when the host is under
> - heavy load.
> - state: indeterminate
> - tests:
> - - spintrcritical03
> - - spintrcritical04
> - - spintrcritical05
> build-type: option
> copyrights:
> - Copyright (C) 2020 On-Line Applications Research (OAR)
> default: []
> description: ''
> enabled-by: true
> -links: []
> +links:
> +- role: build-dependency
> + uid: ../../tstqemu
> type: build
> diff --git a/spec/build/bsps/aarch64/xilinx-versal/tstqemu.yml b/spec/build/bsps/aarch64/xilinx-versal/tstqemu.yml
> index 0a0ab619af..69b85cadbd 100644
> --- a/spec/build/bsps/aarch64/xilinx-versal/tstqemu.yml
> +++ b/spec/build/bsps/aarch64/xilinx-versal/tstqemu.yml
> @@ -12,5 +12,7 @@ copyrights:
> default: []
> description: ''
> enabled-by: true
> -links: []
> +links:
> +- role: build-dependency
> + uid: ../../tstqemu
> type: build
> diff --git a/spec/build/bsps/aarch64/xilinx-zynqmp/tstqemu.yml b/spec/build/bsps/aarch64/xilinx-zynqmp/tstqemu.yml
> index a3d8c9810d..adf6d85ae1 100644
> --- a/spec/build/bsps/aarch64/xilinx-zynqmp/tstqemu.yml
> +++ b/spec/build/bsps/aarch64/xilinx-zynqmp/tstqemu.yml
> @@ -6,40 +6,13 @@ actions:
> state: exclude
> tests:
> - minimum
> -- set-test-state:
> - reason: |
> - Tests that are passing intermittently.
> - state: indeterminate
> - tests:
> - - psx12
> - - psxtimes01
> - - rtmonuse
> - - rtmonusxtimes01
> - - sp04
> - - sp20
> - - sp68
> - - sp69
> - - spcpucounter01
> - - spedfsched02
> - - spedfsched04
> - - sprmsched01
> - - sptimecounter02
> - - sptimecounter04
> - - ttest02
> -- set-test-state:
> - reason: |
> - Tests that pass nominally, but fail under Qemu when the host is under
> - heavy load.
> - state: indeterminate
> - tests:
> - - spintrcritical03
> - - spintrcritical04
> - - spintrcritical05
> build-type: option
> copyrights:
> - Copyright (C) 2020 On-Line Applications Research (OAR)
> default: []
> description: ''
> enabled-by: true
> -links: []
> +links:
> +- role: build-dependency
> + uid: ../../tstqemu
> type: build
> diff --git a/spec/build/bsps/arm/realview-pbx-a9/bsprealviewpbxa9.yml b/spec/build/bsps/arm/realview-pbx-a9/bsprealviewpbxa9.yml
> index a6e4d5b1a9..4111830ce9 100644
> --- a/spec/build/bsps/arm/realview-pbx-a9/bsprealviewpbxa9.yml
> +++ b/spec/build/bsps/arm/realview-pbx-a9/bsprealviewpbxa9.yml
> @@ -53,6 +53,8 @@ links:
> uid: ../../objmem
> - role: build-dependency
> uid: ../../opto0
> +- role: build-dependency
> + uid: ../../tstqemu
> - role: build-dependency
> uid: ../../bspopts
> source:
> diff --git a/spec/build/bsps/arm/xilinx-zynq/bspqemu.yml b/spec/build/bsps/arm/xilinx-zynq/bspqemu.yml
> index 4780a9d64b..8068284d82 100644
> --- a/spec/build/bsps/arm/xilinx-zynq/bspqemu.yml
> +++ b/spec/build/bsps/arm/xilinx-zynq/bspqemu.yml
> @@ -13,6 +13,8 @@ install: []
> links:
> - role: build-dependency
> uid: ../../opto2
> +- role: build-dependency
> + uid: ../../tstqemu
> - role: build-dependency
> uid: grp
> source: []
> diff --git a/spec/build/bsps/tstqemu.yml b/spec/build/bsps/tstqemu.yml
> new file mode 100644
> index 0000000000..3956cc4828
> --- /dev/null
> +++ b/spec/build/bsps/tstqemu.yml
> @@ -0,0 +1,72 @@
> +SPDX-License-Identifier: CC-BY-SA-4.0 OR BSD-2-Clause
> +actions:
> +- set-test-state:
> + reason: |
> + Tests that are passing intermittently due to #4922
> + state: indeterminate
> + tests:
> + - psx12
> + - psxclock
> + - psxtimes01
> + - psxualarm
> + - rtmonuse
> + - rtmonusxtimes01
> + - smpclock01
> + - smpfatal01
> + - smpfatal03
> + - smpmrsp01
> + - smpmutex01
> + - smppsxmutex01
> + - smpschedaffinity01
> + - smpschedaffinity02
> + - smpschededf01
> + - smpschededf03
> + - smpscheduler04
> + - smpthreadpin01
> + - sp04
> + - sp20
> + - sp68
> + - sp69
> + - sp71
> + - spcpucounter01
> + - spedfsched02
> + - spedfsched04
> + - spintrcritical01
> + - spintrcritical02
> + - spintrcritical03
> + - spintrcritical04
> + - spintrcritical05
> + - spintrcritical06
> + - spintrcritical07
> + - spintrcritical08
> + - spintrcritical09
> + - spintrcritical10
> + - spintrcritical11
> + - spintrcritical12
> + - spintrcritical13
> + - spintrcritical14
> + - spintrcritical15
> + - spintrcritical16
> + - spintrcritical17
> + - spintrcritical18
> + - spintrcritical19
> + - spintrcritical20
> + - spintrcritical21
> + - spintrcritical22
> + - spintrcritical23
> + - spintrcritical24
> + - sprmsched01
> + - sptimecounter01
> + - sptimecounter02
> + - sptimecounter04
> + - ttest02
> + - validation-0
> + - validation-smp-only-0
This looks OK to me, but it looks like you've added a lot more tests
than just the union of the existing ones in the individual BSPs. I
take it that you have found more variance in these tests also?
> +build-type: option
> +copyrights:
> +- Copyright (C) 2023 On-Line Applications Research (OAR)
> +default: []
> +description: ''
> +enabled-by: true
> +links: []
> +type: build
> --
> 2.30.2
>
> _______________________________________________
> devel mailing list
> devel at rtems.org
> http://lists.rtems.org/mailman/listinfo/devel
More information about the devel
mailing list