[PATCH] testsuite: Add a common test configuration. Fix configure.ac and Makefile.am errors.

Joel Sherrill joel at rtems.org
Mon Apr 3 22:34:58 UTC 2017


Go ahead and push this, I will build all BSPs overnight if you do and
see how it works.

--joel

On Mon, Apr 3, 2017 at 5:24 PM, Chris Johns <chrisj at rtems.org> wrote:

> - Add a top level test configuration file for test states that are common
>   to all BSPs. This saves adding a test configuration (tcfg) file for
>   every BSP.
>
> - Add the test states 'user-input' and 'benchmark'. This
>   lets 'rtems-test' stop the test rather than waiting for a timeout or
>   letting a benchmark run without the user asking for it to run.
>
> - Implement rtems-test-check in Python to make it faster. The shell script
>   had grown to a point it was noticably slowing the build down.
>
> - Fix the configure.ac and Makefile.am files for a number of the
>   test directories. The files are difficiult to keep in sync with the
>   number of tests and mistakes can happen such as tests being left
>   out of the build. The test fsrofs01 is an example. Also a there was
>   a mix of SUBDIRS and _SUBDIRS being used and only _SUBDIRS should be
>   used.
>
> - Fix the test fsrofs01 so it compiles.
>
> Closes #2963.
> ---
>  testsuites/automake/subdirs.am              |  45 +++++++++
>  testsuites/automake/test-subdirs.am         |  42 --------
>  testsuites/benchmarks/Makefile.am           |   8 +-
>  testsuites/benchmarks/configure.ac          |   2 +-
>  testsuites/fstests/Makefile.am              |  31 +++---
>  testsuites/fstests/configure.ac             |  26 ++---
>  testsuites/fstests/fsrofs01/init.c          |   4 +-
>  testsuites/libtests/Makefile.am             | 113 ++++++++++++++-------
>  testsuites/libtests/configure.ac            |  68 ++++++-------
>  testsuites/libtests/dl01/Makefile.am        |  11 +--
>  testsuites/libtests/dl02/Makefile.am        |  11 +--
>  testsuites/libtests/dl04/Makefile.am        |  11 +--
>  testsuites/libtests/dl05/Makefile.am        |  11 +--
>  testsuites/mptests/Makefile.am              |  18 +++-
>  testsuites/psxtests/Makefile.am             | 147
> ++++++++++++++++++++++------
>  testsuites/psxtests/configure.ac            |   4 +-
>  testsuites/psxtmtests/Makefile.am           | 102 +++++++++----------
>  testsuites/rhealstone/Makefile.am           |  13 +--
>  testsuites/rhealstone/configure.ac          |   6 +-
>  testsuites/samples/Makefile.am              |  20 ++--
>  testsuites/samples/configure.ac             |  18 ++--
>  testsuites/smptests/Makefile.am             |  98 +++++++++----------
>  testsuites/smptests/configure.ac            |   8 +-
>  testsuites/sptests/Makefile.am              |   2 +-
>  testsuites/support/include/buffer_test_io.h |  14 ++-
>  testsuites/testdata/rtems.tcfg              |  28 ++++++
>  testsuites/tmtests/Makefile.am              |   2 +-
>  tools/build/rtems-test-check                |  81 ++++++++++++---
>  tools/build/rtems-test-check-py             | 116 ++++++++++++++++++++++
>  tools/build/rtems-testsuite-autostuff       |  49 ++++++++++
>  30 files changed, 746 insertions(+), 363 deletions(-)
>  delete mode 100644 testsuites/automake/test-subdirs.am
>  create mode 100644 testsuites/testdata/rtems.tcfg
>  create mode 100755 tools/build/rtems-test-check-py
>  create mode 100755 tools/build/rtems-testsuite-autostuff
>
> diff --git a/testsuites/automake/subdirs.am b/testsuites/automake/subdirs.
> am
> index e69de29bb2..70a7ecb78c 100644
> --- a/testsuites/automake/subdirs.am
> +++ b/testsuites/automake/subdirs.am
> @@ -0,0 +1,45 @@
> +## Borrowed from automake/subdir.am which borrowed automake-1.4 and
> adapted to RTEMS
> +
> +## NOTE: This is a temporary work-around until automake is removed from
> RTEMS.
> +##   It is a hack within many hacks and is designed to keep the source as
> clean
> +##   as possible.
> +
> +all-local:
> +       @set fnord $(MAKEFLAGS); amf=$$2; \
> +       dot_seen=no; \
> +       target=`echo $@ | sed s/-recursive//`; \
> +       if test "$$target" = "all-local-am"; then \
> +         target="all-am"; \
> +       fi; \
> +       if test "$$target" = "all-local"; then \
> +         target="all"; \
> +       fi; \
> +       tcheck="$(top_srcdir)/../../tools/build/rtems-test-check-py"; \
> +       tdata="$(top_srcdir)/../../c/src/lib/libbsp/$(RTEMS_CPU)/$(
> RTEMS_BSP_FAMILY)/make/custom/$(RTEMS_BSP)-testsuite.tcfg"; \
> +       if test -f "$$tdata"; then \
> +         vtdata="$(RTEMS_CPU)/$(RTEMS_BSP_FAMILY)/make/custom/$(RTEMS_BSP)-testsuite.tcfg";
> \
> +       else \
> +         vtdata="all tests"; \
> +       fi; \
> +       echo "BSP Testsuite Data: $$vtdata"; \
> +       if test -f $$tcheck; then \
> +         list=`$$tcheck exclude $$tdata $(top_srcdir)/.. $(RTEMS_BSP)
> $(_SUBDIRS)`; \
> +       else \
> +         list=$(_SUBDIRS); \
> +       fi; \
> +       for subdir in $$list; do \
> +         echo "Making $$target in $$subdir"; \
> +         if test "$$subdir" != "."; then \
> +           if test -f $$tcheck; then \
> +             test_FLAGS=`$$tcheck flags $$tdata $(top_srcdir)/..
> $(RTEMS_BSP) $$subdir`; \
> +           fi; \
> +           local_target="$$target"; \
> +           if test -z "$$test_FLAGS"; then \
> +               echo "BSP Testsuite Flags: $$subdir: PASS"; \
> +           else \
> +               echo "BSP Testsuite Flags: $$subdir: $$test_FLAGS"; \
> +           fi; \
> +           (cd $$subdir && $(MAKE) $(AM_MAKEFLAGS)
> TEST_FLAGS="$$test_FLAGS" $$local_target) \
> +            || case "$$amf" in *=*) exit 1;; *k*) fail=yes;; *) exit 1;;
> esac; \
> +         fi; \
> +       done; test -z "$$fail"
> diff --git a/testsuites/automake/test-subdirs.am b/testsuites/automake/
> test-subdirs.am
> deleted file mode 100644
> index f83f03541f..0000000000
> --- a/testsuites/automake/test-subdirs.am
> +++ /dev/null
> @@ -1,42 +0,0 @@
> -## Borrowed from automake/subdir.am which borrowed automake-1.4 and
> adapted to RTEMS
> -
> -## NOTE: This is a temporary work-around until automake is removed from
> RTEMS.
> -##   It is a hack within many hacks and is designed to keep the source as
> clean
> -##   as possible.
> -
> -all-local:
> -       @set fnord $(MAKEFLAGS); amf=$$2; \
> -       dot_seen=no; \
> -       target=`echo $@ | sed s/-recursive//`; \
> -       if test "$$target" = "all-local-am"; then \
> -         target="all-am"; \
> -       fi; \
> -       if test "$$target" = "all-local"; then \
> -         target="all"; \
> -       fi; \
> -       tdata="$(top_srcdir)/../../c/src/lib/libbsp/$(RTEMS_CPU)/$(
> RTEMS_BSP_FAMILY)/make/custom/$(RTEMS_BSP)-testsuite.tcfg"; \
> -       if test -f "$$tdata"; then \
> -         vtdata="$(RTEMS_CPU)/$(RTEMS_BSP_FAMILY)/make/custom/$(RTEMS_BSP)-testsuite.tcfg";
> \
> -        else \
> -         vtdata="all tests"; \
> -       fi; \
> -       echo "BSP Testsuite Data: $$vtdata"; \
> -       list=`$(top_srcdir)/../../tools/build/rtems-test-check \
> -                exclude $$tdata $(top_srcdir)/.. $(RTEMS_BSP)
> $(_SUBDIRS)`; \
> -       for subdir in $$list; do \
> -         echo "Making $$target in $$subdir"; \
> -         if test "$$subdir" != "."; then \
> -            test_FLAGS=`$(top_srcdir)/../../tools/build/rtems-test-check
> \
> -                        flags $$tdata $(top_srcdir)/.. $(RTEMS_BSP)
> $$subdir`; \
> -           local_target="$$target"; \
> -           if test -z "$$test_FLAGS"; then \
> -               echo "BSP Testsuite Flags: $$subdir: PASS"; \
> -           else \
> -               echo "BSP Testsuite Flags: $$subdir: $$test_FLAGS"; \
> -           fi; \
> -           (cd $$subdir && $(MAKE) $(AM_MAKEFLAGS)
> TEST_FLAGS="$$test_FLAGS" $$local_target) \
> -            || case "$$amf" in *=*) exit 1;; *k*) fail=yes;; *) exit 1;;
> esac; \
> -         fi; \
> -       done; test -z "$$fail"
> -
> -include $(top_srcdir)/../automake/subdirs.am
> diff --git a/testsuites/benchmarks/Makefile.am b/testsuites/benchmarks/
> Makefile.am
> index 3d95ecdb9f..6c5fe7fb27 100644
> --- a/testsuites/benchmarks/Makefile.am
> +++ b/testsuites/benchmarks/Makefile.am
> @@ -1,9 +1,9 @@
>  ACLOCAL_AMFLAGS = -I ../aclocal
>
> -SUBDIRS =
> -SUBDIRS += linpack
> -SUBDIRS += whetstone
> -SUBDIRS += dhrystone
> +_SUBDIRS =
> +_SUBDIRS += dhrystone
> +_SUBDIRS += linpack
> +_SUBDIRS += whetstone
>
>  include $(top_srcdir)/../automake/subdirs.am
>  include $(top_srcdir)/../automake/local.am
> diff --git a/testsuites/benchmarks/configure.ac b/testsuites/benchmarks/
> configure.ac
> index 18af749fb9..f28fc79162 100644
> --- a/testsuites/benchmarks/configure.ac
> +++ b/testsuites/benchmarks/configure.ac
> @@ -22,8 +22,8 @@ RTEMS_CHECK_CUSTOM_BSP(RTEMS_BSP)
>
>  # Explicitly list all Makefiles here
>  AC_CONFIG_FILES([Makefile
> +dhrystone/Makefile
>  linpack/Makefile
>  whetstone/Makefile
> -dhrystone/Makefile
>  ])
>  AC_OUTPUT
> diff --git a/testsuites/fstests/Makefile.am b/testsuites/fstests/Makefile.
> am
> index 514b60912c..1302fe009a 100644
> --- a/testsuites/fstests/Makefile.am
> +++ b/testsuites/fstests/Makefile.am
> @@ -1,16 +1,21 @@
>  ACLOCAL_AMFLAGS = -I ../aclocal
>
> -_SUBDIRS =
> -_SUBDIRS += fsjffs2gc01
> -_SUBDIRS += fsimfsconfig03
> -_SUBDIRS += fsimfsconfig02
> -_SUBDIRS += fsimfsconfig01
> +_SUBDIRS  =
> +_SUBDIRS += fsbdpart01
> +_SUBDIRS += fsdosfsformat01
>  _SUBDIRS += fsdosfsname01
>  _SUBDIRS += fsdosfsname02
> +_SUBDIRS += fsdosfssync01
>  _SUBDIRS += fsdosfswrite01
> -_SUBDIRS += fsdosfsformat01
>  _SUBDIRS += fsfseeko01
> -_SUBDIRS += fsdosfssync01
> +_SUBDIRS += fsimfsconfig01
> +_SUBDIRS += fsimfsconfig02
> +_SUBDIRS += fsimfsconfig03
> +_SUBDIRS += fsimfsgeneric01
> +_SUBDIRS += fsjffs2gc01
> +_SUBDIRS += fsnofs01
> +_SUBDIRS += fsrfsbitmap01
> +_SUBDIRS += fsrofs01
>  _SUBDIRS += imfs_fserror
>  _SUBDIRS += imfs_fslink
>  _SUBDIRS += imfs_fspatheval
> @@ -30,19 +35,20 @@ _SUBDIRS += jffs2_fstime
>  _SUBDIRS += mdosfs_fserror
>  _SUBDIRS += mdosfs_fspatheval
>  _SUBDIRS += mdosfs_fsrdwr
> -_SUBDIRS += mdosfs_fsstatvfs
>  _SUBDIRS += mdosfs_fsscandir01
> +_SUBDIRS += mdosfs_fsstatvfs
>  _SUBDIRS += mdosfs_fstime
>  _SUBDIRS += mimfs_fserror
>  _SUBDIRS += mimfs_fslink
>  _SUBDIRS += mimfs_fspatheval
>  _SUBDIRS += mimfs_fspermission
>  _SUBDIRS += mimfs_fsrdwr
> +_SUBDIRS += mimfs_fsrename
>  _SUBDIRS += mimfs_fsscandir01
>  _SUBDIRS += mimfs_fssymlink
>  _SUBDIRS += mimfs_fstime
> -_SUBDIRS += mimfs_fsrename
>  _SUBDIRS += mrfs_fserror
> +_SUBDIRS += mrfs_fsfpathconf
>  _SUBDIRS += mrfs_fslink
>  _SUBDIRS += mrfs_fspatheval
>  _SUBDIRS += mrfs_fspermission
> @@ -50,11 +56,6 @@ _SUBDIRS += mrfs_fsrdwr
>  _SUBDIRS += mrfs_fsscandir01
>  _SUBDIRS += mrfs_fssymlink
>  _SUBDIRS += mrfs_fstime
> -_SUBDIRS += mrfs_fsfpathconf
> -_SUBDIRS += fsrfsbitmap01
> -_SUBDIRS += fsnofs01
> -_SUBDIRS += fsimfsgeneric01
> -_SUBDIRS += fsbdpart01
>
>  EXTRA_DIST =
>  EXTRA_DIST += support/ramdisk_support.c
> @@ -63,5 +64,5 @@ EXTRA_DIST += support/fstest_support.c
>  EXTRA_DIST += support/fstest_support.h
>  EXTRA_DIST += support/fstest.h
>
> -include $(top_srcdir)/../automake/test-subdirs.am
> +include $(top_srcdir)/../automake/subdirs.am
>  include $(top_srcdir)/../automake/local.am
> diff --git a/testsuites/fstests/configure.ac b/testsuites/fstests/
> configure.ac
> index f9daa814e7..e5dc840ba8 100644
> --- a/testsuites/fstests/configure.ac
> +++ b/testsuites/fstests/configure.ac
> @@ -77,16 +77,21 @@ AC_CHECK_SIZEOF([blkcnt_t])
>
>  # Explicitly list all Makefiles here
>  AC_CONFIG_FILES([Makefile
> -fsjffs2gc01/Makefile
> -fsimfsconfig03/Makefile
> -fsimfsconfig02/Makefile
> -fsimfsconfig01/Makefile
> +fsbdpart01/Makefile
> +fsdosfsformat01/Makefile
>  fsdosfsname01/Makefile
>  fsdosfsname02/Makefile
> +fsdosfssync01/Makefile
>  fsdosfswrite01/Makefile
> -fsdosfsformat01/Makefile
>  fsfseeko01/Makefile
> -fsdosfssync01/Makefile
> +fsimfsconfig01/Makefile
> +fsimfsconfig02/Makefile
> +fsimfsconfig03/Makefile
> +fsimfsgeneric01/Makefile
> +fsjffs2gc01/Makefile
> +fsnofs01/Makefile
> +fsrfsbitmap01/Makefile
> +fsrofs01/Makefile
>  imfs_fserror/Makefile
>  imfs_fslink/Makefile
>  imfs_fspatheval/Makefile
> @@ -114,11 +119,12 @@ mimfs_fslink/Makefile
>  mimfs_fspatheval/Makefile
>  mimfs_fspermission/Makefile
>  mimfs_fsrdwr/Makefile
> +mimfs_fsrename/Makefile
>  mimfs_fsscandir01/Makefile
>  mimfs_fssymlink/Makefile
>  mimfs_fstime/Makefile
> -mimfs_fsrename/Makefile
>  mrfs_fserror/Makefile
> +mrfs_fsfpathconf/Makefile
>  mrfs_fslink/Makefile
>  mrfs_fspatheval/Makefile
>  mrfs_fspermission/Makefile
> @@ -126,11 +132,5 @@ mrfs_fsrdwr/Makefile
>  mrfs_fsscandir01/Makefile
>  mrfs_fssymlink/Makefile
>  mrfs_fstime/Makefile
> -mrfs_fsfpathconf/Makefile
> -fsrfsbitmap01/Makefile
> -fsnofs01/Makefile
> -fsimfsgeneric01/Makefile
> -fsbdpart01/Makefile
> -
>  ])
>  AC_OUTPUT
> diff --git a/testsuites/fstests/fsrofs01/init.c
> b/testsuites/fstests/fsrofs01/init.c
> index cd1dcc4a29..45aae88df9 100644
> --- a/testsuites/fstests/fsrofs01/init.c
> +++ b/testsuites/fstests/fsrofs01/init.c
> @@ -153,12 +153,12 @@ static void test_rofs(void)
>
>  static void Init(rtems_task_argument arg)
>  {
> -  TEST_BEGIN():
> +  TEST_BEGIN();
>
>    test_create_file_system();
>    test_rofs();
>
> -  TEST_END():
> +  TEST_END();
>    rtems_test_exit(0);
>  }
>
> diff --git a/testsuites/libtests/Makefile.am b/testsuites/libtests/
> Makefile.am
> index 286875d143..e72ca1bbaa 100644
> --- a/testsuites/libtests/Makefile.am
> +++ b/testsuites/libtests/Makefile.am
> @@ -1,42 +1,85 @@
>  ACLOCAL_AMFLAGS = -I ../aclocal
>
> -_SUBDIRS = POSIX
> -_SUBDIRS += termios09
> -_SUBDIRS += libfdt01
> +_SUBDIRS  =
> +_SUBDIRS += POSIX
> +_SUBDIRS += block01
> +_SUBDIRS += block02
> +_SUBDIRS += block03
> +_SUBDIRS += block04
> +_SUBDIRS += block05
> +_SUBDIRS += block06
> +_SUBDIRS += block07
> +_SUBDIRS += block08
> +_SUBDIRS += block09
> +_SUBDIRS += block10
> +_SUBDIRS += block11
> +_SUBDIRS += block12
> +_SUBDIRS += block13
> +_SUBDIRS += block14
> +_SUBDIRS += block15
> +_SUBDIRS += block16
> +_SUBDIRS += block17
> +_SUBDIRS += bspcmdline01
> +_SUBDIRS += capture01
> +_SUBDIRS += complex
> +_SUBDIRS += cpuuse
> +_SUBDIRS += crypt01
>  _SUBDIRS += defaultconfig01
> -_SUBDIRS += pwdgrp02
> -_SUBDIRS += shell01
> +_SUBDIRS += devfs01
> +_SUBDIRS += devfs02
> +_SUBDIRS += devfs03
> +_SUBDIRS += devfs04
> +_SUBDIRS += deviceio01
> +_SUBDIRS += devnullfatal01
> +_SUBDIRS += dumpbuf01
> +_SUBDIRS += exit01
> +_SUBDIRS += exit02
> +_SUBDIRS += flashdisk01
> +_SUBDIRS += gxx01
> +_SUBDIRS += heapwalk
> +_SUBDIRS += i2c01
> +_SUBDIRS += libfdt01
> +_SUBDIRS += malloc02
> +_SUBDIRS += malloc03
> +_SUBDIRS += malloc04
> +_SUBDIRS += malloctest
> +_SUBDIRS += math
> +_SUBDIRS += mathf
> +_SUBDIRS += mathl
> +_SUBDIRS += md501
> +_SUBDIRS += monitor
> +_SUBDIRS += monitor02
> +_SUBDIRS += mouse01
> +_SUBDIRS += newlib01
> +_SUBDIRS += putenvtest
>  _SUBDIRS += pwdgrp01
> -_SUBDIRS += crypt01
> +_SUBDIRS += pwdgrp02
> +_SUBDIRS += rbheap01
> +_SUBDIRS += rtmonuse
>  _SUBDIRS += sha
> -_SUBDIRS += i2c01
> +_SUBDIRS += shell01
> +_SUBDIRS += sparsedisk01
>  _SUBDIRS += spi01
> -_SUBDIRS += newlib01
> -_SUBDIRS += block17
> -_SUBDIRS += exit02
> -_SUBDIRS += exit01
> +_SUBDIRS += stackchk
> +_SUBDIRS += stackchk01
> +_SUBDIRS += stringto01
> +_SUBDIRS += tar01
> +_SUBDIRS += tar02
> +_SUBDIRS += tar03
> +_SUBDIRS += termios
> +_SUBDIRS += termios01
> +_SUBDIRS += termios02
> +_SUBDIRS += termios03
> +_SUBDIRS += termios04
> +_SUBDIRS += termios05
> +_SUBDIRS += termios06
> +_SUBDIRS += termios07
> +_SUBDIRS += termios08
> +_SUBDIRS += termios09
> +_SUBDIRS += top
> +_SUBDIRS += tztest
> +_SUBDIRS += uid01
>  _SUBDIRS += utf8proc01
> -_SUBDIRS += md501
> -_SUBDIRS += sparsedisk01
> -_SUBDIRS += block16
> -_SUBDIRS += block15
> -_SUBDIRS += block14
> -_SUBDIRS += block13
> -_SUBDIRS += rbheap01
> -_SUBDIRS += flashdisk01
> -_SUBDIRS += capture01
> -
> -_SUBDIRS += bspcmdline01 cpuuse devfs01 devfs02 devfs03 devfs04 \
> -    deviceio01 devnullfatal01 dumpbuf01 gxx01 top\
> -    malloctest malloc02 malloc03 malloc04 heapwalk \
> -    putenvtest monitor monitor02 rtmonuse stackchk stackchk01 \
> -    termios termios01 termios02 termios03 termios04 termios05 \
> -    termios06 termios07 termios08 \
> -    tztest block01 block02 block03 block04 block05 block06 block07 \
> -    block08 block09 block10 block11 block12 stringto01 \
> -    tar01 tar02 tar03 \
> -    math mathf mathl complex \
> -    mouse01 uid01
>
>  if NETTESTS
>  if HAS_POSIX
> @@ -47,6 +90,10 @@ _SUBDIRS += networking01
>  _SUBDIRS += syscall01
>  endif
>
> +##
> +## The libdl use use AM's BUILT_SOURCE and this has a weird side effect of
> +## restarting the subdir loop. No idea why.
> +##
>  if DLTESTS
>  _SUBDIRS += dl01 dl02 dl03
>  if HAS_CXX
> @@ -58,5 +105,5 @@ if DEBUGGERTESTS
>  _SUBDIRS += debugger01
>  endif
>
> -include $(top_srcdir)/../automake/test-subdirs.am
> +include $(top_srcdir)/../automake/subdirs.am
>  include $(top_srcdir)/../automake/local.am
> diff --git a/testsuites/libtests/configure.ac b/testsuites/libtests/
> configure.ac
> index 4e82caff18..bbdf4df9d1 100644
> --- a/testsuites/libtests/configure.ac
> +++ b/testsuites/libtests/configure.ac
> @@ -93,31 +93,7 @@ AM_CONDITIONAL(DEBUGGERTESTS,[test
> x"$TEST_LIBDEBUGGER" = x"yes"])
>
>  # Explicitly list all Makefiles here
>  AC_CONFIG_FILES([Makefile
> -networking01/Makefile
> -libfdt01/Makefile
> -defaultconfig01/Makefile
> -pwdgrp02/Makefile
> -shell01/Makefile
> -pwdgrp01/Makefile
> -crypt01/Makefile
> -sha/Makefile
> -i2c01/Makefile
> -spi01/Makefile
> -newlib01/Makefile
> -block17/Makefile
> -exit02/Makefile
> -exit01/Makefile
> -utf8proc01/Makefile
> -md501/Makefile
> -sparsedisk01/Makefile
> -block16/Makefile
> -mghttpd01/Makefile
> -block15/Makefile
> -block14/Makefile
> -block13/Makefile
> -rbheap01/Makefile
> -syscall01/Makefile
> -flashdisk01/Makefile
> +POSIX/Makefile
>  block01/Makefile
>  block02/Makefile
>  block03/Makefile
> @@ -130,8 +106,18 @@ block09/Makefile
>  block10/Makefile
>  block11/Makefile
>  block12/Makefile
> +block13/Makefile
> +block14/Makefile
> +block15/Makefile
> +block16/Makefile
> +block17/Makefile
>  bspcmdline01/Makefile
> +capture01/Makefile
> +complex/Makefile
>  cpuuse/Makefile
> +crypt01/Makefile
> +debugger01/Makefile
> +defaultconfig01/Makefile
>  devfs01/Makefile
>  devfs02/Makefile
>  devfs03/Makefile
> @@ -144,22 +130,41 @@ dl03/Makefile
>  dl04/Makefile
>  dl05/Makefile
>  dumpbuf01/Makefile
> +exit01/Makefile
> +exit02/Makefile
> +flashdisk01/Makefile
>  ftp01/Makefile
>  gxx01/Makefile
>  heapwalk/Makefile
> -malloctest/Makefile
> +i2c01/Makefile
> +libfdt01/Makefile
>  malloc02/Makefile
>  malloc03/Makefile
>  malloc04/Makefile
> +malloctest/Makefile
> +math/Makefile
> +mathf/Makefile
> +mathl/Makefile
> +md501/Makefile
> +mghttpd01/Makefile
>  monitor/Makefile
>  monitor02/Makefile
>  mouse01/Makefile
> -uid01/Makefile
> +networking01/Makefile
> +newlib01/Makefile
>  putenvtest/Makefile
> +pwdgrp01/Makefile
> +pwdgrp02/Makefile
> +rbheap01/Makefile
>  rtmonuse/Makefile
> +sha/Makefile
> +shell01/Makefile
> +sparsedisk01/Makefile
> +spi01/Makefile
>  stackchk/Makefile
>  stackchk01/Makefile
>  stringto01/Makefile
> +syscall01/Makefile
>  tar01/Makefile
>  tar02/Makefile
>  tar03/Makefile
> @@ -175,12 +180,7 @@ termios08/Makefile
>  termios09/Makefile
>  top/Makefile
>  tztest/Makefile
> -capture01/Makefile
> -POSIX/Makefile
> -math/Makefile
> -mathf/Makefile
> -mathl/Makefile
> -complex/Makefile
> -debugger01/Makefile
> +uid01/Makefile
> +utf8proc01/Makefile
>  ])
>  AC_OUTPUT
> diff --git a/testsuites/libtests/dl01/Makefile.am
> b/testsuites/libtests/dl01/Makefile.am
> index d7c9a0d507..07460694e2 100644
> --- a/testsuites/libtests/dl01/Makefile.am
> +++ b/testsuites/libtests/dl01/Makefile.am
> @@ -29,14 +29,9 @@ dl-tar.h: dl.tar
>         $(BIN2C) -H $< $@
>  CLEANFILES += dl-tar.h
>
> -dl01.pre$(EXEEXT): $(dl01_OBJECTS) $(dl01_DEPENDENCIES)
> -       @rm -f dl01.pre$(EXEEXT)
> -       $(make-exe)
> -       rm -f dl01.pre.ralf
> -
> -dl01.pre: dl01.pre$(EXEEXT)
> -       mv $< $@
> -CLEANFILES += dl01.pre
> +dl01.pre: $(dl01_OBJECTS) $(dl01_DEPENDENCIES)
> +       @rm -f dl01.pre
> +       $(LINK_APP)
>
>  dl-sym.o: dl01.pre
>         rtems-syms -e -c "$(CFLAGS)" -o $@ $<
> diff --git a/testsuites/libtests/dl02/Makefile.am
> b/testsuites/libtests/dl02/Makefile.am
> index 39a8efe82a..52ef98dc8a 100644
> --- a/testsuites/libtests/dl02/Makefile.am
> +++ b/testsuites/libtests/dl02/Makefile.am
> @@ -31,14 +31,9 @@ dl-tar.h: dl.tar
>         $(BIN2C) -H $< $@
>  CLEANFILES += dl-tar.h
>
> -dl02.pre$(EXEEXT): $(dl02_OBJECTS) $(dl02_DEPENDENCIES)
> -       @rm -f dl02.pre$(EXEEXT)
> -       $(make-exe)
> -       rm -f dl02.pre.ralf
> -
> -dl02.pre: dl02.pre$(EXEEXT)
> -       mv $< $@
> -CLEANFILES += dl02.pre
> +dl02.pre: $(dl02_OBJECTS) $(dl02_DEPENDENCIES)
> +       @rm -f dl02.pre
> +       $(LINK_APP)
>
>  dl-sym.o: dl02.pre
>         rtems-syms -e -c "$(CFLAGS)" -o $@ $<
> diff --git a/testsuites/libtests/dl04/Makefile.am
> b/testsuites/libtests/dl04/Makefile.am
> index d45bc87c4d..a888434770 100644
> --- a/testsuites/libtests/dl04/Makefile.am
> +++ b/testsuites/libtests/dl04/Makefile.am
> @@ -30,14 +30,9 @@ dl-tar.h: dl.tar
>         $(BIN2C) -H $< $@
>  CLEANFILES += dl-tar.h
>
> -dl04.pre$(EXEEXT): $(dl04_OBJECTS) $(dl04_DEPENDENCIES)
> -       @rm -f dl04.pre$(EXEEXT)
> -       $(make-exe)
> -       rm -f dl04.pre.ralf
> -
> -dl04.pre: dl04.pre$(EXEEXT)
> -       mv $< $@
> -CLEANFILES += dl04.pre
> +dl04.pre: $(dl04_OBJECTS) $(dl04_DEPENDENCIES)
> +       @rm -f dl04.pre
> +       $(LINK_APP)
>
>  dl-sym.o: dl04.pre
>         rtems-syms -e -c "$(CFLAGS)" -o $@ $<
> diff --git a/testsuites/libtests/dl05/Makefile.am
> b/testsuites/libtests/dl05/Makefile.am
> index d63f355c76..15608cd3c6 100644
> --- a/testsuites/libtests/dl05/Makefile.am
> +++ b/testsuites/libtests/dl05/Makefile.am
> @@ -29,15 +29,10 @@ dl-tar.h: dl.tar
>         $(BIN2C) -H $< $@
>  CLEANFILES += dl-tar.h
>
> -dl05.pre$(EXEEXT): $(dl05_OBJECTS) $(dl05_DEPENDENCIES)
> -       @rm -f dl05.pre$(EXEEXT)
> +dl05.pre: $(dl05_OBJECTS) $(dl05_DEPENDENCIES)
> +       @rm -f dl05.pre
>         $(LINK.cc) $(CPU_CFLAGS) $(AM_CFLAGS) $(AM_LDFLAGS) \
> -                   -o $(basename $@)$(EXEEXT) $(LINK_OBJS) $(LINK_LIBS)
> -       rm -f dl05.pre.ralf
> -
> -dl05.pre: dl05.pre$(EXEEXT)
> -       mv $< $@
> -CLEANFILES += dl05.pre
> +                   -o $@ $(LINK_OBJS) $(LINK_LIBS)
>
>  dl-sym.o: dl05.pre
>         rtems-syms -e -c "$(CFLAGS)" -o $@ $<
> diff --git a/testsuites/mptests/Makefile.am b/testsuites/mptests/Makefile.
> am
> index 45889f1835..cfb80e5a23 100644
> --- a/testsuites/mptests/Makefile.am
> +++ b/testsuites/mptests/Makefile.am
> @@ -1,7 +1,19 @@
>  ACLOCAL_AMFLAGS = -I ../aclocal
>
> -_SUBDIRS = mp01      mp03 mp04 mp05 mp06 mp07 mp08 mp09 mp10 mp11 mp12
> mp13 \
> -    mp14
> +_SUBDIRS  =
> +_SUBDIRS += mp01
> +_SUBDIRS += mp03
> +_SUBDIRS += mp04
> +_SUBDIRS += mp05
> +_SUBDIRS += mp06
> +_SUBDIRS += mp07
> +_SUBDIRS += mp08
> +_SUBDIRS += mp09
> +_SUBDIRS += mp10
> +_SUBDIRS += mp11
> +_SUBDIRS += mp12
> +_SUBDIRS += mp13
> +_SUBDIRS += mp14
>
> -include $(top_srcdir)/../automake/test-subdirs.am
> +include $(top_srcdir)/../automake/subdirs.am
>  include $(top_srcdir)/../automake/local.am
> diff --git a/testsuites/psxtests/Makefile.am b/testsuites/psxtests/
> Makefile.am
> index a00aea40d5..584e8c5753 100644
> --- a/testsuites/psxtests/Makefile.am
> +++ b/testsuites/psxtests/Makefile.am
> @@ -1,48 +1,131 @@
>  ACLOCAL_AMFLAGS = -I ../aclocal
>
> -_SUBDIRS = psxclock
> -_SUBDIRS += psxthreadname01
> +## Some of the tests need other POSIX features to work while some do not.
> +
> +_SUBDIRS  =
>  if HAS_POSIX
> -_SUBDIRS += psxhdrs psx01 psx02 psx03 psx04 psx05 psx06 psx07 psx08 psx09
> \
> -    psx10 psx11 psx12 psx13 psx14 psx15 psx16 \
> -    psxaio01 psxaio02 psxaio03 \
> -    psxalarm01 psxautoinit01 psxautoinit02 psxbarrier01 \
> -    psxcancel psxcancel01 psxclassic01 psxcleanup psxcleanup01 \
> -    psxconcurrency01 psxcond01 psxcond02 psxconfig01 psxenosys \
> -    psxitimer psxmsgq01 psxmsgq02 psxmsgq03 psxmsgq04 \
> -    psxmutexattr01 psxobj01 psxrwlock01 psxsem01 psxshm01 psxshm02 \
> -    psxsignal01 psxsignal02 psxsignal03 psxsignal04 psxsignal05
> psxsignal06 \
> -    psxspin01 psxstrsignal01 psxsysconf \
> -    psxtime psxtimer01 psxtimer02 psxualarm psxusleep psxfatal01
> psxfatal02 \
> -    psxintrcritical01 psxstack01 psxstack02 \
> -    psxeintr_join psxgetattrnp01 psxclock01
> +_SUBDIRS += psx01
> +_SUBDIRS += psx02
> +_SUBDIRS += psx03
> +_SUBDIRS += psx04
> +_SUBDIRS += psx05
> +_SUBDIRS += psx06
> +_SUBDIRS += psx07
> +_SUBDIRS += psx08
> +_SUBDIRS += psx09
> +_SUBDIRS += psx10
> +_SUBDIRS += psx11
> +_SUBDIRS += psx12
> +_SUBDIRS += psx14
> +_SUBDIRS += psx15
> +_SUBDIRS += psx16
> +_SUBDIRS += psxaio01
> +_SUBDIRS += psxaio02
> +_SUBDIRS += psxaio03
> +_SUBDIRS += psxalarm01
> +_SUBDIRS += psxautoinit01
> +_SUBDIRS += psxautoinit02
> +_SUBDIRS += psxbarrier01
> +_SUBDIRS += psxcancel
> +_SUBDIRS += psxcancel01
> +_SUBDIRS += psxclassic01
> +_SUBDIRS += psxcleanup
> +_SUBDIRS += psxcleanup01
>  _SUBDIRS += psxcleanup02
> +endif
> +_SUBDIRS += psxclock
> +if HAS_POSIX
> +_SUBDIRS += psxclock01
> +_SUBDIRS += psxconcurrency01
> +_SUBDIRS += psxcond01
> +_SUBDIRS += psxcond02
> +_SUBDIRS += psxconfig01
> +endif
> +_SUBDIRS += psxdevctl01
> +if HAS_POSIX
> +_SUBDIRS += psxeintr_join
> +_SUBDIRS += psxenosys
> +_SUBDIRS += psxfatal01
> +_SUBDIRS += psxfatal02
> +endif
> +if HAS_POSIX
>  if HAS_CPLUSPLUS
>  _SUBDIRS += psxglobalcon01
>  _SUBDIRS += psxglobalcon02
>  endif
> +_SUBDIRS += psxhdrs
> +_SUBDIRS += psxintrcritical01
> +_SUBDIRS += psxitimer
>  endif
> -
> -## File IO tests
> -_SUBDIRS += psxfile01 psxfile02 psxfilelock01 psxgetrusage01 psxid01 \
> -    psximfs01 psximfs02 psxreaddir psxstat psxmount psx13 psxchroot01 \
> -    psxpasswd01 psxpasswd02 psxpipe01 psxtimes01 psxfchx01
> -
> -## POSIX Devctl tests
> -_SUBDIRS += psxdevctl01
> -
> -## POSIX Keys are always available
> -_SUBDIRS += psxkey01 psxkey02 psxkey03 psxkey04 \
> -    psxkey05 psxkey06 psxkey08 psxkey09 psxkey10
> -## But some of the tests need other POSIX features
> +_SUBDIRS += psxkey01
> +_SUBDIRS += psxkey02
> +_SUBDIRS += psxkey03
> +_SUBDIRS += psxkey04
> +_SUBDIRS += psxkey05
> +_SUBDIRS += psxkey06
>  if HAS_POSIX
>  _SUBDIRS += psxkey07
>  endif
> -
> -## POSIX Once is always available
> +_SUBDIRS += psxkey08
> +_SUBDIRS += psxkey09
> +_SUBDIRS += psxkey10
> +if HAS_POSIX
> +_SUBDIRS += psxmsgq01
> +_SUBDIRS += psxmsgq02
> +_SUBDIRS += psxmsgq03
> +_SUBDIRS += psxmsgq04
> +_SUBDIRS += psxmutexattr01
> +_SUBDIRS += psxobj01
> +endif
>  _SUBDIRS += psxonce01
> -
>  _SUBDIRS += psxrdwrv
> +if HAS_POSIX
> +_SUBDIRS += psxrwlock01
> +_SUBDIRS += psxsem01
> +_SUBDIRS += psxshm01
> +_SUBDIRS += psxshm02
> +_SUBDIRS += psxsignal01
> +_SUBDIRS += psxsignal02
> +_SUBDIRS += psxsignal03
> +_SUBDIRS += psxsignal04
> +_SUBDIRS += psxsignal05
> +_SUBDIRS += psxsignal06
> +_SUBDIRS += psxspin01
> +_SUBDIRS += psxstack01
> +_SUBDIRS += psxstack02
> +_SUBDIRS += psxstrsignal01
> +_SUBDIRS += psxsysconf
> +endif
> +_SUBDIRS += psxthreadname01
> +if HAS_POSIX
> +_SUBDIRS += psxtime
> +_SUBDIRS += psxtimer01
> +_SUBDIRS += psxtimer02
> +_SUBDIRS += psxualarm
> +_SUBDIRS += psxusleep
> +endif
> +
> +
> +
> +## File IO tests
> +_SUBDIRS += psx13
> +_SUBDIRS += psxchroot01
> +_SUBDIRS += psxfile01
> +_SUBDIRS += psxfile02
> +_SUBDIRS += psxfilelock01
> +_SUBDIRS += psxgetattrnp01
> +_SUBDIRS += psxgetrusage01
> +_SUBDIRS += psxid01
> +_SUBDIRS += psximfs01
> +_SUBDIRS += psximfs02
> +_SUBDIRS += psxmount
> +_SUBDIRS += psxpasswd01
> +_SUBDIRS += psxpasswd02
> +_SUBDIRS += psxpipe01
> +_SUBDIRS += psxreaddir
> +_SUBDIRS += psxstat
> +_SUBDIRS += psxtimes01
> +_SUBDIRS += psxfchx01
>
> -include $(top_srcdir)/../automake/test-subdirs.am
> +include $(top_srcdir)/../automake/subdirs.am
>  include $(top_srcdir)/../automake/local.am
> diff --git a/testsuites/psxtests/configure.ac b/testsuites/psxtests/
> configure.ac
> index 3211a8d380..1de1312984 100644
> --- a/testsuites/psxtests/configure.ac
> +++ b/testsuites/psxtests/configure.ac
> @@ -184,8 +184,8 @@ psxonce01/Makefile
>  psxpasswd01/Makefile
>  psxpasswd02/Makefile
>  psxpipe01/Makefile
> -psxreaddir/Makefile
>  psxrdwrv/Makefile
> +psxreaddir/Makefile
>  psxrwlock01/Makefile
>  psxsem01/Makefile
>  psxshm01/Makefile
> @@ -202,11 +202,11 @@ psxstack02/Makefile
>  psxstat/Makefile
>  psxstrsignal01/Makefile
>  psxsysconf/Makefile
> +psxthreadname01/Makefile
>  psxtime/Makefile
>  psxtimer01/Makefile
>  psxtimer02/Makefile
>  psxtimes01/Makefile
> -psxthreadname01/Makefile
>  psxualarm/Makefile
>  psxusleep/Makefile
>  ])
> diff --git a/testsuites/psxtmtests/Makefile.am b/testsuites/psxtmtests/
> Makefile.am
> index e7e079aff8..bcb11dc4f9 100644
> --- a/testsuites/psxtmtests/Makefile.am
> +++ b/testsuites/psxtmtests/Makefile.am
> @@ -1,60 +1,60 @@
>  ACLOCAL_AMFLAGS = -I ../aclocal
>
> -SUBDIRS =
> +_SUBDIRS =
>
>  if HAS_POSIX
> -SUBDIRS += psxtmbarrier01
> -SUBDIRS += psxtmbarrier02
> -SUBDIRS += psxtmbarrier03
> -SUBDIRS += psxtmbarrier04
> -SUBDIRS += psxtmclocknanosleep01
> -SUBDIRS += psxtmclocknanosleep02
> -SUBDIRS += psxtmclocknanosleep03
> -SUBDIRS += psxtmcond01
> -SUBDIRS += psxtmcond02
> -SUBDIRS += psxtmcond03
> -SUBDIRS += psxtmcond04
> -SUBDIRS += psxtmcond05
> -SUBDIRS += psxtmcond06
> -SUBDIRS += psxtmcond07
> -SUBDIRS += psxtmcond08
> -SUBDIRS += psxtmcond09
> -SUBDIRS += psxtmcond10
> -SUBDIRS += psxtmkey01
> -SUBDIRS += psxtmkey02
> -SUBDIRS += psxtmmq01
> -SUBDIRS += psxtmmutex01
> -SUBDIRS += psxtmmutex02
> -SUBDIRS += psxtmmutex03
> -SUBDIRS += psxtmmutex04
> -SUBDIRS += psxtmmutex05
> -SUBDIRS += psxtmmutex06
> -SUBDIRS += psxtmmutex07
> -SUBDIRS += psxtmnanosleep01
> -SUBDIRS += psxtmnanosleep02
> -SUBDIRS += psxtmrwlock01
> -SUBDIRS += psxtmrwlock02
> -SUBDIRS += psxtmrwlock03
> -SUBDIRS += psxtmrwlock04
> -SUBDIRS += psxtmrwlock05
> -SUBDIRS += psxtmrwlock06
> -SUBDIRS += psxtmrwlock07
> -SUBDIRS += psxtmsem01
> -SUBDIRS += psxtmsem02
> -SUBDIRS += psxtmsem03
> -SUBDIRS += psxtmsem04
> -SUBDIRS += psxtmsem05
> -SUBDIRS += psxtmsleep01
> -SUBDIRS += psxtmsleep02
> -SUBDIRS += psxtmthread01
> -SUBDIRS += psxtmthread02
> -SUBDIRS += psxtmthread03
> -SUBDIRS += psxtmthread04
> -SUBDIRS += psxtmthread05
> -SUBDIRS += psxtmthread06
> +_SUBDIRS += psxtmbarrier01
> +_SUBDIRS += psxtmbarrier02
> +_SUBDIRS += psxtmbarrier03
> +_SUBDIRS += psxtmbarrier04
> +_SUBDIRS += psxtmclocknanosleep01
> +_SUBDIRS += psxtmclocknanosleep02
> +_SUBDIRS += psxtmclocknanosleep03
> +_SUBDIRS += psxtmcond01
> +_SUBDIRS += psxtmcond02
> +_SUBDIRS += psxtmcond03
> +_SUBDIRS += psxtmcond04
> +_SUBDIRS += psxtmcond05
> +_SUBDIRS += psxtmcond06
> +_SUBDIRS += psxtmcond07
> +_SUBDIRS += psxtmcond08
> +_SUBDIRS += psxtmcond09
> +_SUBDIRS += psxtmcond10
> +_SUBDIRS += psxtmkey01
> +_SUBDIRS += psxtmkey02
> +_SUBDIRS += psxtmmq01
> +_SUBDIRS += psxtmmutex01
> +_SUBDIRS += psxtmmutex02
> +_SUBDIRS += psxtmmutex03
> +_SUBDIRS += psxtmmutex04
> +_SUBDIRS += psxtmmutex05
> +_SUBDIRS += psxtmmutex06
> +_SUBDIRS += psxtmmutex07
> +_SUBDIRS += psxtmnanosleep01
> +_SUBDIRS += psxtmnanosleep02
> +_SUBDIRS += psxtmrwlock01
> +_SUBDIRS += psxtmrwlock02
> +_SUBDIRS += psxtmrwlock03
> +_SUBDIRS += psxtmrwlock04
> +_SUBDIRS += psxtmrwlock05
> +_SUBDIRS += psxtmrwlock06
> +_SUBDIRS += psxtmrwlock07
> +_SUBDIRS += psxtmsem01
> +_SUBDIRS += psxtmsem02
> +_SUBDIRS += psxtmsem03
> +_SUBDIRS += psxtmsem04
> +_SUBDIRS += psxtmsem05
> +_SUBDIRS += psxtmsleep01
> +_SUBDIRS += psxtmsleep02
> +_SUBDIRS += psxtmthread01
> +_SUBDIRS += psxtmthread02
> +_SUBDIRS += psxtmthread03
> +_SUBDIRS += psxtmthread04
> +_SUBDIRS += psxtmthread05
> +_SUBDIRS += psxtmthread06
>  endif
>
> -DIST_SUBDIRS = $(SUBDIRS)
> +DIST_SUBDIRS = $(_SUBDIRS)
>
>  include $(top_srcdir)/../automake/subdirs.am
>  include $(top_srcdir)/../automake/local.am
> diff --git a/testsuites/rhealstone/Makefile.am b/testsuites/rhealstone/
> Makefile.am
> index d22c6004f1..e2dcf694da 100644
> --- a/testsuites/rhealstone/Makefile.am
> +++ b/testsuites/rhealstone/Makefile.am
> @@ -1,11 +1,12 @@
>  ACLOCAL_AMFLAGS = -I ../aclocal
>
> -SUBDIRS  = rhtaskswitch
> -SUBDIRS += rhtaskpreempt
> -SUBDIRS += rhilatency
> -SUBDIRS += rhmlatency
> -SUBDIRS += rhsemshuffle
> -SUBDIRS += rhdeadlockbrk
> +_SUBDIRS  =
> +_SUBDIRS += rhdeadlockbrk
> +_SUBDIRS += rhilatency
> +_SUBDIRS += rhmlatency
> +_SUBDIRS += rhsemshuffle
> +_SUBDIRS += rhtaskpreempt
> +_SUBDIRS += rhtaskswitch
>
>  include $(top_srcdir)/../automake/subdirs.am
>  include $(top_srcdir)/../automake/local.am
> diff --git a/testsuites/rhealstone/configure.ac b/testsuites/rhealstone/
> configure.ac
> index bcc6592600..af35f6819d 100644
> --- a/testsuites/rhealstone/configure.ac
> +++ b/testsuites/rhealstone/configure.ac
> @@ -25,11 +25,11 @@ RTEMS_CHECK_CUSTOM_BSP(RTEMS_BSP)
>
>  # Explicitly list all Makefiles here
>  AC_CONFIG_FILES([Makefile
> -rhtaskswitch/Makefile
> -rhtaskpreempt/Makefile
> +rhdeadlockbrk/Makefile
>  rhilatency/Makefile
>  rhmlatency/Makefile
>  rhsemshuffle/Makefile
> -rhdeadlockbrk/Makefile
> +rhtaskpreempt/Makefile
> +rhtaskswitch/Makefile
>  ])
>  AC_OUTPUT
> diff --git a/testsuites/samples/Makefile.am b/testsuites/samples/Makefile.
> am
> index 374617b7e4..85e89a50af 100644
> --- a/testsuites/samples/Makefile.am
> +++ b/testsuites/samples/Makefile.am
> @@ -1,18 +1,24 @@
>  ACLOCAL_AMFLAGS = -I ../aclocal
>
> -_SUBDIRS = hello capture ticker base_sp unlimited minimum fileio
> +_SUBDIRS  =
> +_SUBDIRS += base_sp
> +_SUBDIRS += capture
> +_SUBDIRS += fileio
> +_SUBDIRS += hello
> +_SUBDIRS += minimum
> +_SUBDIRS += nsecs
> +_SUBDIRS += paranoia
> +_SUBDIRS += ticker
> +_SUBDIRS += unlimited
>
>  if MPTESTS
>  ## base_mp is a sample multiprocessing test
>  _SUBDIRS += base_mp
>  endif
>
> -_SUBDIRS += paranoia
> -
> -_SUBDIRS += nsecs
> -
>  if CXXTESTS
> -_SUBDIRS += iostream cdtest
> +_SUBDIRS += cdtest
> +_SUBDIRS += iostream
>  endif
>
>  if NETTESTS
> @@ -24,5 +30,5 @@ _SUBDIRS += pppd
>  endif
>  endif
>
> -include $(top_srcdir)/../automake/test-subdirs.am
> +include $(top_srcdir)/../automake/subdirs.am
>  include $(top_srcdir)/../automake/local.am
> diff --git a/testsuites/samples/configure.ac b/testsuites/samples/
> configure.ac
> index 91a36612df..ffe1bad2c2 100644
> --- a/testsuites/samples/configure.ac
> +++ b/testsuites/samples/configure.ac
> @@ -37,7 +37,7 @@ AS_IF([test $HAS_CPLUSPLUS = yes],[
>    [rtems_cv_HAVE_IOSTREAM_STD_NAMESPACE],
>    [AC_COMPILE_IFELSE(
>      [AC_LANG_PROGRAM(
> -      [[#include <iostream>]],
> +      [[#include <iostream>]],
>        [[std::cout << "hello" << std::endl;]])
>      ],
>      [rtems_cv_HAVE_IOSTREAM_STD_NAMESPACE=yes],
> @@ -60,21 +60,21 @@ AC_CHECK_SIZEOF([time_t])
>
>  # Explicitly list all Makefiles here
>  AC_CONFIG_FILES([Makefile
> +base_mp/Makefile
> +base_mp/node1/Makefile
> +base_mp/node2/Makefile
>  base_sp/Makefile
> +capture/Makefile
> +cdtest/Makefile
> +fileio/Makefile
>  hello/Makefile
> +iostream/Makefile
>  loopback/Makefile
>  minimum/Makefile
> -fileio/Makefile
>  nsecs/Makefile
>  paranoia/Makefile
> +pppd/Makefile
>  ticker/Makefile
>  unlimited/Makefile
> -base_mp/Makefile
> -base_mp/node1/Makefile
> -base_mp/node2/Makefile
> -iostream/Makefile
> -cdtest/Makefile
> -pppd/Makefile
> -capture/Makefile
>  ])
>  AC_OUTPUT
> diff --git a/testsuites/smptests/Makefile.am b/testsuites/smptests/
> Makefile.am
> index ba5b35f973..76c02e72d8 100644
> --- a/testsuites/smptests/Makefile.am
> +++ b/testsuites/smptests/Makefile.am
> @@ -1,56 +1,56 @@
>  ACLOCAL_AMFLAGS = -I ../aclocal
>
> -SUBDIRS =
> +_SUBDIRS  =
>  if SMPTESTS
> -SUBDIRS += smp01
> -SUBDIRS += smp02
> -SUBDIRS += smp03
> -SUBDIRS += smp05
> -SUBDIRS += smp07
> -SUBDIRS += smp08
> -SUBDIRS += smp09
> -SUBDIRS += smpaffinity01
> -SUBDIRS += smpatomic01
> -SUBDIRS += smpcache01
> -SUBDIRS += smpcapture01
> -SUBDIRS += smpcapture02
> -SUBDIRS += smpclock01
> -SUBDIRS += smpfatal01
> -SUBDIRS += smpfatal02
> -SUBDIRS += smpfatal03
> -SUBDIRS += smpfatal04
> -SUBDIRS += smpfatal05
> -SUBDIRS += smpfatal06
> -SUBDIRS += smpfatal08
> -SUBDIRS += smpipi01
> -SUBDIRS += smpload01
> -SUBDIRS += smplock01
> -SUBDIRS += smpmigration01
> -SUBDIRS += smpmigration02
> -SUBDIRS += smpmrsp01
> -SUBDIRS += smpmutex01
> -SUBDIRS += smpmutex02
> -SUBDIRS += smpschedaffinity01
> -SUBDIRS += smpschedaffinity02
> -SUBDIRS += smpschedaffinity03
> -SUBDIRS += smpschedaffinity04
> -SUBDIRS += smpschedaffinity05
> -SUBDIRS += smpscheduler01
> -SUBDIRS += smpscheduler02
> -SUBDIRS += smpscheduler03
> -SUBDIRS += smpscheduler04
> -SUBDIRS += smpschedsem01
> -SUBDIRS += smpsignal01
> -SUBDIRS += smpstrongapa01
> -SUBDIRS += smpswitchextension01
> -SUBDIRS += smpthreadlife01
> -SUBDIRS += smpunsupported01
> -SUBDIRS += smpwakeafter01
> +_SUBDIRS += smp01
> +_SUBDIRS += smp02
> +_SUBDIRS += smp03
> +_SUBDIRS += smp05
> +_SUBDIRS += smp07
> +_SUBDIRS += smp08
> +_SUBDIRS += smp09
> +_SUBDIRS += smpaffinity01
> +_SUBDIRS += smpatomic01
> +_SUBDIRS += smpcache01
> +_SUBDIRS += smpcapture01
> +_SUBDIRS += smpcapture02
> +_SUBDIRS += smpclock01
> +_SUBDIRS += smpfatal01
> +_SUBDIRS += smpfatal02
> +_SUBDIRS += smpfatal03
> +_SUBDIRS += smpfatal04
> +_SUBDIRS += smpfatal05
> +_SUBDIRS += smpfatal06
> +_SUBDIRS += smpfatal08
> +_SUBDIRS += smpipi01
> +_SUBDIRS += smpload01
> +_SUBDIRS += smplock01
> +_SUBDIRS += smpmigration01
> +_SUBDIRS += smpmigration02
> +_SUBDIRS += smpmrsp01
> +_SUBDIRS += smpmutex01
> +_SUBDIRS += smpmutex02
> +_SUBDIRS += smppsxaffinity01
> +_SUBDIRS += smppsxaffinity02
> +_SUBDIRS += smpschedaffinity03
> +_SUBDIRS += smpschedaffinity04
> +_SUBDIRS += smpschedaffinity05
> +_SUBDIRS += smpschedsem01
> +_SUBDIRS += smpscheduler01
> +_SUBDIRS += smpscheduler02
> +_SUBDIRS += smpscheduler03
> +_SUBDIRS += smpscheduler04
> +_SUBDIRS += smpsignal01
> +_SUBDIRS += smpstrongapa01
> +_SUBDIRS += smpswitchextension01
> +_SUBDIRS += smpthreadlife01
> +_SUBDIRS += smpunsupported01
> +_SUBDIRS += smpwakeafter01
>  if HAS_POSIX
> -SUBDIRS += smppsxaffinity01
> -SUBDIRS += smppsxaffinity02
> -SUBDIRS += smppsxmutex01
> -SUBDIRS += smppsxsignal01
> +_SUBDIRS += smppsxaffinity01
> +_SUBDIRS += smppsxaffinity02
> +_SUBDIRS += smppsxmutex01
> +_SUBDIRS += smppsxsignal01
>  endif
>  endif
>
> diff --git a/testsuites/smptests/configure.ac b/testsuites/smptests/
> configure.ac
> index f242b0b264..434fe8dad7 100644
> --- a/testsuites/smptests/configure.ac
> +++ b/testsuites/smptests/configure.ac
> @@ -53,9 +53,6 @@ AC_CHECK_DECLS([pthread_getattr_np],[],[],[[
>
>  # Explicitly list all Makefiles here
>  AC_CONFIG_FILES([Makefile
> -smpmutex02/Makefile
> -smppsxmutex01/Makefile
> -smpstrongapa01/Makefile
>  smp01/Makefile
>  smp02/Makefile
>  smp03/Makefile
> @@ -83,20 +80,23 @@ smpmigration01/Makefile
>  smpmigration02/Makefile
>  smpmrsp01/Makefile
>  smpmutex01/Makefile
> +smpmutex02/Makefile
>  smppsxaffinity01/Makefile
>  smppsxaffinity02/Makefile
> +smppsxmutex01/Makefile
>  smppsxsignal01/Makefile
>  smpschedaffinity01/Makefile
>  smpschedaffinity02/Makefile
>  smpschedaffinity03/Makefile
>  smpschedaffinity04/Makefile
>  smpschedaffinity05/Makefile
> +smpschedsem01/Makefile
>  smpscheduler01/Makefile
>  smpscheduler02/Makefile
>  smpscheduler03/Makefile
>  smpscheduler04/Makefile
> -smpschedsem01/Makefile
>  smpsignal01/Makefile
> +smpstrongapa01/Makefile
>  smpswitchextension01/Makefile
>  smpthreadlife01/Makefile
>  smpunsupported01/Makefile
> diff --git a/testsuites/sptests/Makefile.am b/testsuites/sptests/Makefile.
> am
> index e019055d16..c50e7575bc 100644
> --- a/testsuites/sptests/Makefile.am
> +++ b/testsuites/sptests/Makefile.am
> @@ -80,5 +80,5 @@ _SUBDIRS += sptimerserver01
>  _SUBDIRS += spclock_err02
>  _SUBDIRS += spcpuset01
>
> -include $(top_srcdir)/../automake/test-subdirs.am
> +include $(top_srcdir)/../automake/subdirs.am
>  include $(top_srcdir)/../automake/local.am
> diff --git a/testsuites/support/include/buffer_test_io.h
> b/testsuites/support/include/buffer_test_io.h
> index 1e396b7dcf..c2a92058f5 100644
> --- a/testsuites/support/include/buffer_test_io.h
> +++ b/testsuites/support/include/buffer_test_io.h
> @@ -23,15 +23,23 @@ extern "C" {
>  /*
>   * Test states. No state string is an expected pass.
>   */
> -#if TEST_STATE_EXPECTED_FAIL && \
> -    TEST_INDETERMINATE_FAIL
> +#if (TEST_STATE_EXPECTED_FAIL && TEST_STATE_USER_INPUT) || \
> +    (TEST_STATE_EXPECTED_FAIL && TEST_STATE_INDETERMINATE) || \
> +    (TEST_STATE_EXPECTED_FAIL && TEST_STATE_BENCHMARK) || \
> +    (TEST_STATE_USER_INPUT    && TEST_STATE_INDETERMINATE) || \
> +    (TEST_STATE_USER_INPUT    && TEST_STATE_BENCHMARK) || \
> +    (TEST_STATE_INDETERMINATE && TEST_STATE_BENCHMARK)
>    #error Test states must be unique
>  #endif
>
>  #if TEST_STATE_EXPECTED_FAIL
>    #define TEST_STATE_STRING "*** TEST STATE: EXPECTED-FAIL\n"
> -#elif TEST_INDETERMINATE_FAIL
> +#elif TEST_STATE_USER_INPUT
> +  #define TEST_STATE_STRING "*** TEST STATE: USER_INPUT\n"
> +#elif TEST_STATE_INDETERMINATE
>    #define TEST_STATE_STRING "*** TEST STATE: INDETERMINATE\n"
> +#elif TEST_STATE_BENCHMARK
> +  #define TEST_STATE_STRING "*** TEST STATE: BENCHMARK\n"
>  #endif
>
>  /*
> diff --git a/testsuites/testdata/rtems.tcfg b/testsuites/testdata/rtems.
> tcfg
> new file mode 100644
> index 0000000000..255a5e1e86
> --- /dev/null
> +++ b/testsuites/testdata/rtems.tcfg
> @@ -0,0 +1,28 @@
> +#
> +# RTEMS Test Database common defines
> +#
> +# Format is one line per test with a directive. The directives are:
> +#
> +#       include: Include the test configuration file after this file
> +#                has been processed.
> +#       exclude: Exclude the test from being build and so run.
> +#    user-input: The test requires user input to work.
> +# indeterminate: The test result cannot be determined, it may pass or
> +#                it may not pass.
> +#
> +
> +#
> +# User input tests.
> +#
> +user-input: capture
> +user-input: fileio
> +user-input: monitor
> +user-input: termios
> +user-input: top
> +
> +#
> +# Benchmarks
> +#
> +benchmark: dhrystone
> +benchmark: linpack
> +benchmark: whetstone
> diff --git a/testsuites/tmtests/Makefile.am b/testsuites/tmtests/Makefile.
> am
> index b259269ac9..8326abe329 100644
> --- a/testsuites/tmtests/Makefile.am
> +++ b/testsuites/tmtests/Makefile.am
> @@ -7,5 +7,5 @@ _SUBDIRS += tmtimer01
>  _SUBDIRS += tmcontext01
>  _SUBDIRS += tmfine01
>
> -include $(top_srcdir)/../automake/test-subdirs.am
> +include $(top_srcdir)/../automake/subdirs.am
>  include $(top_srcdir)/../automake/local.am
> diff --git a/tools/build/rtems-test-check b/tools/build/rtems-test-check
> index 923af49306..988556d59e 100755
> --- a/tools/build/rtems-test-check
> +++ b/tools/build/rtems-test-check
> @@ -53,17 +53,29 @@ case ${mode} in
>  esac
>
>  #
> +# Read the common settings first.
> +#
> +if [ -f $includepath/testdata/rtems.tcfg ]; then
> + testdata="$includepath/testdata/rtems.tcfg $testdata"
> +fi
> +
> +#
>  # If there is no testdata all tests are valid and must pass.
>  #
>
> -if test -f $testdata; then
> +if [ ! -z "$testdata" ]; then
>    excluded_tests=""
>    expected_fails=""
> +  user_inputs=""
>    indeterminates=""
> +  benchmarks=""
>    while [ ! -z "$testdata" ];
>    do
>      for td in $testdata;
>      do
> +      if [ ! -f $td ]; then
> +        continue
> +      fi
>        ntd=""
>        exec 3<& 0
>        exec 0<$td
> @@ -87,10 +99,18 @@ if test -f $testdata; then
>                line=$(echo $line | sed -e 's/expected-fail://g;s/[[:
> blank:]]//g')
>                expected_fails="${expected_fails} $line"
>                ;;
> +            user-input)
> +              line=$(echo $line | sed -e 's/user-input://g;s/[[:blank:]
> ]//g')
> +              user_inputs="${user_inputs} $line"
> +              ;;
>              indeterminate)
>                line=$(echo $line | sed -e 's/indeterminate://g;s/[[:
> blank:]]//g')
>                indeterminates="${indeterminates} $line"
>                ;;
> +            benchmark)
> +              line=$(echo $line | sed -e 's/benchmark://g;s/[[:blank:]]
> //g')
> +              benchmarks="${benchmarks} $line"
> +              ;;
>              *)
>                echo "error: invalid test state: $state in $td" 1>&2
>                echo "INVALID-TEST-DATA"
> @@ -119,26 +139,55 @@ if test -f $testdata; then
>          fi
>          ;;
>        flags)
> -        allow="no"
> -        for et in ${expected_fails};
> +        allow="yes"
> +        for et in ${excluded_tests};
>          do
>            if test ${t} = ${et}; then
> -            allow="yes"
> +            allow="no"
>            fi
>          done
> -        if test ${allow} = yes; then
> -          output="-DTEST_STATE_EXPECTED_FAIL=1"
> -        fi
> -        allow="no"
> -        for it in ${indeterminates};
> -        do
> -          if test ${t} = ${it}; then
> -            allow="yes"
> +       if test ${allow} = yes; then
> +          allow="no"
> +          for et in ${expected_fails};
> +          do
> +            if test ${t} = ${et}; then
> +              allow="yes"
> +            fi
> +          done
> +          if test ${allow} = yes; then
> +            output="-DTEST_STATE_EXPECTED_FAIL=1"
>            fi
> -        done
> -        if test ${allow} = yes; then
> -          output="${output} -DTEST_STATE_INDETERMINATE=1"
> -        fi
> +          allow="no"
> +          for ut in ${user_inputs};
> +          do
> +            if test ${t} = ${ut}; then
> +              allow="yes"
> +            fi
> +          done
> +          if test ${allow} = yes; then
> +            output="-DTEST_STATE_USER_INPUT=1"
> +          fi
> +          allow="no"
> +          for it in ${indeterminates};
> +          do
> +            if test ${t} = ${it}; then
> +              allow="yes"
> +            fi
> +          done
> +          if test ${allow} = yes; then
> +            output="${output} -DTEST_STATE_INDETERMINATE=1"
> +          fi
> +          allow="no"
> +          for bt in ${benchmarks};
> +          do
> +            if test ${t} = ${bt}; then
> +              allow="yes"
> +            fi
> +          done
> +          if test ${allow} = yes; then
> +            output="${output} -DTEST_STATE_BENCHMARK=1"
> +          fi
> +       fi
>          ;;
>        *)
>          echo "error: invalid mode" 1>&2
> diff --git a/tools/build/rtems-test-check-py b/tools/build/rtems-test-
> check-py
> new file mode 100755
> index 0000000000..e6bf29b7cf
> --- /dev/null
> +++ b/tools/build/rtems-test-check-py
> @@ -0,0 +1,116 @@
> +#! /usr/bin/env python
> +#
> +# Copyright 2017 Chris Johns <chrisj at rtems.org>
> +# All rights reserved
> +#
> +
> +#
> +# Python version the rtems-test-check script.
> +#
> +
> +from __future__ import print_function
> +import os.path
> +import sys
> +
> +def eprint(*args, **kwargs):
> +    print(*args, file=sys.stderr, **kwargs)
> +
> +#
> +# Arguments. Keep it simple.
> +#
> +if len(sys.argv) < 4:
> +    eprint('error: invalid command line')
> +    print('INVALID-TEST-DATA')
> +    sys.exit(2)
> +
> +mode = sys.argv[1]
> +testconfig = [sys.argv[2]]
> +includepath = sys.argv[3]
> +bsp = sys.argv[4]
> +tests = sys.argv[5:]
> +
> +#
> +# Handle the modes.
> +#
> +if mode == 'exclude':
> +    pass
> +elif mode == 'flags':
> +    if len(tests) != 1:
> +        eprint('error: test count not 1 for mode: %s' % (mode))
> +        print('INVALID-TEST-DATA')
> +        sys.exit(1)
> +else:
> +    eprint('error: invalid mode: %s' % (mode))
> +    print('INVALID-TEST-DATA')
> +    sys.exit(1)
> +
> +#
> +# Common RTEMS testsuite configuration. Load first.
> +#
> +rtems_testdata = os.path.join(includepath, 'testdata', 'rtems.tcfg')
> +if os.path.exists(rtems_testdata):
> +    testconfig.insert(0, rtems_testdata)
> +
> +states = ['exclude',
> +          'expected-fail',
> +          'user-input',
> +          'indeterminate',
> +          'benchmark']
> +defines = { 'expected-fail' : '-DTEST_STATE_EXPECTED_FAIL=1',
> +            'user-input'    : '-DTEST_STATE_USER_INPUT=1',
> +            'indeterminate' : '-DTEST_STATE_INDETERMINATE=1',
> +            'benchmark'     : '-DTEST_STATE_BENCHMARK=1' }
> +output = []
> +testdata = {}
> +
> +def clean(line):
> +    line = line[0:-1]
> +    b = line.find('#')
> +    if b >= 0:
> +        line = line[1:b]
> +    return line.strip()
> +
> +#
> +# Load the test data.
> +#
> +for tc in range(0, len(testconfig)):
> +    if not os.path.exists(testconfig[tc]):
> +        continue
> +    with open(testconfig[tc]) as f:
> +        tdata = [clean(l) for l in f.readlines()]
> +    lc = 0
> +    for line in tdata:
> +        lc += 1
> +        ls = [s.strip() for s in line.split(':')]
> +        if len(line) == 0:
> +            continue
> +        if len(ls) != 2:
> +            eprint('error: syntax error: %s:%d' % (tc, lc))
> +            print('INVALID-TEST-DATA')
> +            sys.exit(1)
> +        state = ls[0]
> +        test = ls[1]
> +        if state == 'include':
> +            testconfig.insert(td, test)
> +        elif state in states:
> +            if state not in testdata:
> +                testdata[state] = [test]
> +            else:
> +                testdata[state] += [test]
> +        else:
> +            eprint('error: invalid test state: %s in %s:%d' % (state, tc,
> lc))
> +            print('INVALID-TEST-DATA')
> +            sys.exit(1)
> +
> +for test in tests:
> +    if mode == 'exclude':
> +        if 'exclude' not in testdata or test not in testdata['exclude']:
> +            output += [test]
> +    elif mode == 'flags':
> +        for state in states:
> +            if state != 'exclude' and state in testdata and test in
> testdata[state]:
> +                output += [defines[state]]
> +
> +print(' '.join(sorted(set(output))))
> +
> +sys.exit(0)
> diff --git a/tools/build/rtems-testsuite-autostuff
> b/tools/build/rtems-testsuite-autostuff
> new file mode 100755
> index 0000000000..8f298bac3c
> --- /dev/null
> +++ b/tools/build/rtems-testsuite-autostuff
> @@ -0,0 +1,49 @@
> +#! /usr/bin/env python
> +
> +#
> +# Copyright 2017 Chris Johns <chrisj at rtems.org>
> +# All rights reserved
> +#
> +
> +#
> +# Create the testsuite's configure.am and Makefile.am from the
> directories
> +# found. This does not handle any conditional functionality that may be
> needed.
> +#
> +
> +from __future__ import print_function
> +import os
> +import os.path
> +import sys
> +
> +def eprint(*args, **kwargs):
> +    print(*args, file = sys.stderr, **kwargs)
> +
> +def die(*args, **kwargs):
> +    print(*args, file = sys.stderr, **kwargs)
> +    sys.exit(1)
> +
> +if len(sys.argv) != 2:
> +    die('error: just provide the path to the test directory')
> +
> +testdir = sys.argv[1]
> +
> +if not os.path.exists(testdir):
> +    die('error: not found: %s' % (testdir))
> +if not os.path.isdir(testdir):
> +    die('error: not a directory: %s' % (testdir))
> +
> +excludes = ['autom4te.cache']
> +
> +tests = sorted([t for t in os.listdir(testdir)
> +                if os.path.isdir(os.path.join(testdir, t)) \
> +                and t not in excludes \
> +                and os.path.exists(os.path.join(testdir, t,
> 'Makefile.am'))])
> +
> +configure = ['AC_CONFIG_FILES([Makefile'] + ['%s/Makefile' % (t) for t in
> tests] + ['])']
> +makefile  = ['_SUBDIRS  ='] + ['_SUBDIRS += %s' % (t) for t in tests]
> +
> +print(os.linesep.join(configure))
> +print()
> +print(os.linesep.join(makefile))
> +
> +sys.exit(0)
> --
> 2.11.0
>
> _______________________________________________
> devel mailing list
> devel at rtems.org
> http://lists.rtems.org/mailman/listinfo/devel
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.rtems.org/pipermail/devel/attachments/20170403/17cce517/attachment-0002.html>


More information about the devel mailing list