[RTEMS Project] #2963: Add a testsuite top level confguration file that is common to all tests.

[RTEMS trac]

#2963: Add a testsuite top level confguration file that is common to all tests.
-------------------------+------------------------------
 Reporter:  Chris Johns  |       Owner:  joel.sherrill@…
     Type:  defect       |      Status:  assigned
 Priority:  normal       |   Milestone:  4.12.0
Component:  testing      |     Version:  4.12
 Severity:  normal       |  Resolution:
 Keywords:  testing      |
-------------------------+------------------------------

Comment (by Sebastian Huber):

 Replying to [comment:2 Chris Johns]:
 > I have never looked at the gcc testsuite and from what I have read about
 it was not flattering.

 My comment was not about the GCC testsuite in general.

 >
 > I do not think adding 500+ files to state `fileio` is a `user-input`
 test and will never complete is not good. Maybe global is not a great
 word, maybe common is better. We need accurate data to determine the
 results of tests.

 Why 500+ files, its just one:

 {{{
 diff --git a/testsuites/samples/fileio/init.c
 b/testsuites/samples/fileio/init.c
 index 07ec2c6..68942e8 100644
 --- a/testsuites/samples/fileio/init.c
 +++ b/testsuites/samples/fileio/init.c
 @@ -34,6 +34,7 @@
  #include <rtems/nvdisk-sram.h>
  #include <rtems/shell.h>

 +/* FANCY TEST COMMENT: user-input */
  const char rtems_test_name[] = "FILE I/O";

  #if FILEIO_BUILD
 }}}

 >
 > It is similar to the work you have been doing to have a common linkercmd
 file where ever possible. It is the same thing or are you saying we should
 create a separate linker command file for every bsp as well? ;)
 >
 > Look at the results with a work in progress `rtems-test` for
 `erc32-run`:
 >
 > {{{
 > Passed:        546
 > Failed:          1
 > User Input:      4
 > Expected Fail:   0
 > Indeterminate:   0
 > Timeout:         6
 > Invalid:         1
 > ------------------
 > Total:         558
 >
 > Failures:
 >  spcontext01.exe
 > User Input:
 >  fileio.exe
 >  top.exe
 >  termios.exe
 >  monitor.exe
 > Timeouts:
 >  jffs2_fssymlink.exe
 >  mrfs_fserror.exe
 >  dhrystone.exe
 >  fsdosfsformat01.exe
 >  imfs_fsrdwr.exe
 >  whetstone.exe
 > Invalid:
 >  minimum.exe
 > Average test time: 0:00:00.481800
 > Testing time     : 0:04:28.844749
 > }}}
 >
 > Note, the benchmark tests have broken parallel testing because of the
 time they now take.

 On my host these benchmark tests did run less than 3 minutes.

--
Ticket URL: <http://devel.rtems.org/ticket/2963#comment:3>
RTEMS Project <http://www.rtems.org/>
RTEMS Project