New validation test suites

[Sebastian Huber]

Hello,

I implemented the new Interrupt Manager directives for the powerpc/qemu 
BSPs. The new validation tests pass now on real hardware (MVME2500).

Passed:        662
Failed:          3
User Input:      5
Expected Fail:   1
Indeterminate:   0
Benchmark:       3
Timeout:         0
Test too long:   0
Invalid:         0
Wrong Version:   0
Wrong Build:     0
Wrong Tools:     0
Wrong Header:    0
------------------
Total:         674
Failures:
  libfdt01.exe.img
  spintrcritical21.exe.img
  minimum.exe.img
User Input:
  top.exe.img
  fileio.exe.img
  capture.exe.img
  monitor.exe.img
  termios.exe.img
Expected Fail:
  psxfenv01.exe.img
Benchmark:
  whetstone.exe.img
  linpack.exe.img
  dhrystone.exe.img
Average test time: 0:00:13.517023
Testing time     : 2:31:50.473792

I fixed the setjmp() and longjmp() for some powerpc multilibs so that 
the new validation tests pass on the powerpc/psim simulator except the 
test for the Interrupt Manager (ts-validation-intr.exe). The 
powerpc/psim BSP uses a legacy interrupt controller support and I don't 
have time to convert this. I simply marked the test as "expected-fail".

A test run for i386/pc686 was not really great using the pc-qemu 
rtems-test configuration:

rtems-test --rtems-bsp=pc-qemu build/i386/pc686/testsuites/validation
[...]
Passed:        12
Failed:         1
User Input:     0
Expected Fail:  0
Indeterminate:  0
Benchmark:      0
Timeout:        5
Test too long:  0
Invalid:        2
Wrong Version:  0
Wrong Build:    0
Wrong Tools:    0
Wrong Header:   0
-----------------
Total:         20
Failures:
  ts-validation-one-cpu-0.exe
Timeouts:
  ts-performance-no-clock-0.exe
  ts-validation-io-kernel.exe
  ts-validation-timecounter-1.exe
  ts-validation-no-clock-0.exe
  ts-validation-intr.exe
Invalid:
  ts-fatal-init-task-construct-failed.exe
  ts-fatal-too-large-tls-size.exe

Some failed tests print "raw_idt_notify has been called" before they 
time out or fail. There is probably an issue with the tm27 support which 
is used to call handlers in interrupt context. I don't have time to fix 
this. I marked all failed, timed out, and invalid tests as "expected-fail":

Passed:        12
Failed:         0
User Input:     0
Expected Fail:  5
Indeterminate:  0
Benchmark:      0
Timeout:        1
Test too long:  0
Invalid:        2
Wrong Version:  0
Wrong Build:    0
Wrong Tools:    0
Wrong Header:   0
-----------------
Total:         20
Expected Fail:
  ts-performance-no-clock-0.exe
  ts-validation-one-cpu-0.exe
  ts-validation-intr.exe
  ts-validation-no-clock-0.exe
  ts-validation-timecounter-1.exe
Timeouts:
  ts-validation-io-kernel.exe
Invalid:
  ts-fatal-too-large-tls-size.exe
  ts-fatal-init-task-construct-failed.exe
Average test time: 0:00:02.357636
Testing time     : 0:00:47.152718

Since the test state reporting is done by the test, this doesn't always 
work. Maybe we should add the test state and some other information as 
ELF annotations. Since rtems-test doesn't use ELF files currently, some 
work would be necessary to use this information.

I will do a test run on a arm/imx7 board tomorrow. The new validation 
tests already passed some time ago on this board.

 From my point of view the new validation tests are ready to get integrated.

-- 
embedded brains GmbH
Herr Sebastian HUBER
Dornierstr. 4
82178 Puchheim
Germany
email: sebastian.huber at embedded-brains.de
phone: +49-89-18 94 741 - 16
fax:   +49-89-18 94 741 - 08

Registergericht: Amtsgericht München
Registernummer: HRB 157899
Vertretungsberechtigte Geschäftsführer: Peter Rasmussen, Thomas Dörfler
Unsere Datenschutzerklärung finden Sie hier:
https://embedded-brains.de/datenschutzerklaerung/