[PATCH v3] User Manual: Added rtems-tester in the docs
Tanu Hari Dixit
tokencolour at gmail.com
Thu Jun 1 23:07:43 UTC 2017
Thank you, Chris, for the review. I have made the changes and sent
again as version 4 patches. I have split the commits into 2 parts (one
not containing the ini bits for v4.12 release and the other one
containing it). Kindly review them.
Thank you for your time.
Tanu.
On Thu, Jun 1, 2017 at 7:10 AM, Chris Johns <chrisj at rtems.org> wrote:
> Thanks for patch. Comments in the email.
>
> Note, I seem to have broken the section headings and will fix soon.
>
> On 31/05/2017 23:33, Tanu Hari Dixit wrote:
>> Adds as part of Tools
>> ---
>> user/tools/index.rst | 1 +
>> user/tools/rtems-tester.rst | 636 ++++++++++++++++++++++++++++++++++++++++++++
>> 2 files changed, 637 insertions(+)
>> create mode 100644 user/tools/rtems-tester.rst
>>
>> diff --git a/user/tools/index.rst b/user/tools/index.rst
>> index d77a7ca..9e2ca89 100644
>> --- a/user/tools/index.rst
>> +++ b/user/tools/index.rst
>> @@ -21,3 +21,4 @@ executables.
>> .. include:: exeinfo.rst
>> .. include:: trace-linker.rst
>> .. include:: bsp-builder.rst
>> +.. include:: rtems-tester.rst
>
> Please call the file 'tester.rst'. This makes the naming consistent with the
> other commands (hint: git mv ...).
>
>> diff --git a/user/tools/rtems-tester.rst b/user/tools/rtems-tester.rst
>> new file mode 100644
>> index 0000000..425a1fb
>> --- /dev/null
>> +++ b/user/tools/rtems-tester.rst
>> @@ -0,0 +1,636 @@
>> +.. comment SPDX-License-Identifier: CC-BY-SA-4.0
>> +
>> +.. comment: Copyright (c) 2017 Chris Johns <chrisj at rtems.org>
>> +.. comment: All rights reserved.
>> +
>> +RTEMS Tester
>> +============
>> +
>> +.. index:: Tools, rtems-tester
>> +
>> +The RTEMS Tester is a test framework. It includes a command line interface to
>> +run tests on supported targets. The framework provides back-end support for
>> +common simulators and debuggers. The board support package (BSP) configurations
>> +for RTEMS are provided and can be used to run all the tests provided with
>> +RTEMS. The framework is not specific to RTEMS and can be configured to run any
>> +suitable application.
>> +
>> +RTEMS is an embedded operating system and is cross-compiled on a range of host
>> +machines. The executables run on the target hardware and this can vary widely
>> +from open source simulators, commercial simulators, debuggers with simulators,
>> +to debuggers with hardware specific pods and devices. Testing RTEMS requires
>> +the cross-compiled test executable is transferred to the target hardware,
>> +executed and the output returned to the host where it is analyzed to determine
>> +the test result. The RTEMS Tester provides a framework to do this.
>> +
>> +Running all the RTEMS tests on your target is very important. It provides you
>> +with a traceable record showing that your RTEMS version and its tools are
>> +working at the level the RTEMS development team expect when releasing
>> +RTEMS. Being able to easily run the tests and verify the results is critical in
>> +maintaining a high standard.
>> +
>> +The RTEMS Tester contains:
>> +
>> +* Command line tool (``rtems-test``)
>
> Please use ":program:`rtems-test`". There are other places that need updating.
>
>> +* BSP Configuration scripts
>> +* Back-end Configuration scripts
>> +* Back-end Python classes
>> +* Python based framework
>> +
>> +This document is for users of the RTEMS Tester as well as those wanting to
>> +develop, extend and investigate the RTEMS Tester.
>
> I think "The RTEMS Tester contains:" ... here can be removed.
>
>> +
>> +License
>> +-------
>> +
>> +The RTEMS Tester is part of the RTEMS Tools Project. The code is released under
>> +the OSI approved "The BSD 2-Clause License". It is free to use and we encourage
>> +this, including on operating systems other than RTEMS.
>> +
>> +The code and command line tools must retain the same names and always reference
>> +the RTEMS Tools Project.
>> +
>
> This section can be removed.
>
>> +Quick Start
>> +-----------
>> +
>> +The quick start will show you how to run the test suite for a BSP. It will
>> +explain how to get the RTEMS Tester, set it up and run the tests for the erc32
>> +BSP. It assumes you have a valid SPARC tool chain and have built the erc32 BSP
>> +version of RTEMS 4.12.
>> +
>> +Setup
>> +-----
>> +
>> +Set up a development work space:
>> +
>> +.. code-block:: shell
>> +
>> + $ cd
>> + $ mkdir -p development/rtems/test
>> + $ cd development/rtems/test
>> +
>> +First fetch the RTEMS tester from the RTEMS Tools repository:
>> +
>> +.. code-block:: shell
>> +
>> + $ git clone git://git.rtems.org/rtems-tools.git rtems-tools.git
>> + $ cd rtems-tools.git/tester
>> +
>
> This section is not needed anymore. The User Manual should have covered this
> already.
>
>> +Available BSP testers
>> +---------------------
>> +
>> +You can list the available BSP testers with:
>> +
>> +.. code-block:: shell
>> +
>> + $ ./rtems-test --list-bsps
>
> $ rtems-test --list-bsps
>
>> + arm920
>> + beagleboardxm
>> + jmr3904-run
>> + jmr3904
>> + mcf5235
>> + psim-run
>> + psim
>> + realview_pbx_a9_qemu
>> + sis-run
>> + sis
>> + xilinx_zynq_a9_qemu
>> + xilinx_zynq_a9_qemu_smp
>> + xilinx_zynq_zc706
>> + xilinx_zynq_zc706_qemu
>> +
>> +.. note:: The list is growing all the time and if your BSP is not supported we
>> + encourage you to add it and submit the configuration back to the project.
>
> Over 80 columns. Please make sure all text is wraps at 80 columns.
>
> .. note:: The list is growing all the time and if your BSP is not supported we
> encourage you to add it and submit the configuration back to the
> project.
>
>> +
>> +Some of the BSPs may appear more than once in the list. These are aliased BSP
>> +configurations that may use a different back-end. An example is the SPARC
>> +Instruction Simulator (SIS) BSP. There is the 'sis' tester which uses the GDB
>> +back-end and the 'sis-run' tester which uses the command line version of the SIS
>> +simulator. We will show how to use ``rtems-test`` command with the SIS BSP
>> +because it is easy to build an use.
>> +
>> +Adding support for a BSP to the tester
>> +--------------------------------------
>> +
>> +To add support for a BSP, one should add a configuration file in
>> +``rtems-tools.git/tester/rtems/testing/bsps``. The configuration files
>> +should be specified in an INI file. The configuration files provide a
>> +means to the backend of `rtems-tester` to automate the running of tests.
>> +There is the ``<bsp>`` tester which uses the GDB back-end and the ``<bsp>-run``
>> +tester which uses the command line version of the respective simulator.
>> +The general format for specifying an INI configuration file is as follows:
>> +
>> +<bsp>.ini:
>
> Should this be ``<bsp>.ini``: ?
>
>> +
>> +.. code-block:: shell
>> +
>> + [<bsp_name>]
>> + bsp = '<bsp_name>'
>> + gdb = '%{_rtscripts}/gdb.cfg'
>> + arch = '<arch_name>'
>> +
>> + [gdb_script]
>> + gdb_script ='<bsp>_gdb_script'
>> + <bsp>_gdb_script = '<Instructions for gdb
>> + ...
>> + ...>'
>> +
>> +<bsp>-run.ini:
>
> Same?
>
>> +
>> +.. code-block:: shell
>> +
>> + [<bsp_name>]
>> + bsp = '<bsp_name>'
>> +
>> + [run]
>> + run = '%{_rtscripts}/run.cfg'
>> + arch = '<arch_name>'
>> + bsp_run_cmd = '%{rtems_tools}/%{bsp_arch}-rtems%{rtems_version}-run'
>> + bsp_run_opts = '<options>'
>> +
>> +Here is an example:
>> +
>> +arm7tdmi.ini:
>
> And again?
>
>> +
>> +.. code-block:: shell
>> +
>> + [arm7tdmi]
>> + bsp = 'arm7tdmi'
>> + gdb = '%{_rtscripts}/gdb.cfg'
>> + arch = 'arm'
>> +
>> + [gdb_script]
>> + gdb_script ='arm7tdmi_gdb_script'
>> + arm7tdmi_gdb_script = 'target sim
>> + load
>> + run'
>> +
>> +arm7tdmi-run.ini:
>> +
>> +.. code-block:: shell
>> +
>> + [arm7tdmi]
>> + bsp = 'arm7tdmi'
>> +
>> + [run]
>> + run = '%{_rtscripts}/run.cfg'
>> + arch = 'arm'
>> + bsp_run_cmd = '%{rtems_tools}/%{bsp_arch}-rtems%{rtems_version}-run'
>> + bsp_run_opts = '-a -nouartrx'
>> +
>> +If the tester should run the tests on qemu for a bsp, the given pattern can
>> +be followed:
>> +
>> +.. code-block:: shell
>> +
>> + [<bsp_name>]
>> + bsp = '<bsp_name>'
>> +
>> + [qemu-script]
>> + run = '%{_rtscripts}/qemu.cfg'
>> + arch = '<arch_name>'
>> + opts = '%{qemu_opts_base} %{qemu_opts_no_net} -m 32M'
>> +
>> +For instance,
>> +
>> +generic_or1k.ini:
>> +
>> +.. code-block:: shell
>> +
>> + [generic_or1k]
>> + bsp = 'generic_or1k'
>> +
>> + [qemu-script]
>> + run = '%{_rtscripts}/qemu.cfg'
>> + arch = 'or32'
>> + opts = '%{qemu_opts_base} %{qemu_opts_no_net} -m 32M'
>> +
>> +User can also provide a "settings.ini" file in case a configuration file needs
>> +a path to, for instance, a first stage bootloader (fsbl) that is placed
>> +somewhere in the host machine. The settings.ini file can be passed with
>> +``rtems-test --with-settings = /path/to/settings/file`` option. The
>> +<bsp_name>.ini can contain defaults and the settings file can contain the
>> +user-specified configuration.
>> +
>> +
>> +xilinx_zynq_zc706.ini:
>> +
>> +.. code-block:: shell
>> +
>> + [<xilinx_zynq_zc706>]
>
> Is the <> needed here?
>
>> + bsp = 'xilinx_zynq_zc706'
>
> Is the extra whitespace needed ater the '=' ?
>
>> + jobs = '1'
>> +
>> + [gdb-script]
>> + gdb = '%{_rtscripts}/gdb.cfg'
>> + arch = 'arm'
>> + bsp_tty_dev = ${settings:bsp_tty_dev}
>> + gdb_script = 'xilinx_zynq_zc706_gdb_script'
>> + xilinx_zynq_zc706_gdb_script = 'target remote kaka:3333
>> + mon load_image ${settings:path_to_fsbl} 0 elf
>> + mon resume 0
>> + mon sleep 4000
>> + mon halt
>> + load
>> + b bsp_reset
>> + continue'
>> +
>> +settings.ini:
>> +
>> +.. code-block:: shell
>> +
>> + [settings]
>> + bsp_tty_dev = '/dev/cu.SLAB_USBtoUART'
>> + path_to_fsbl = '/home/chris/development/si/work/hydra/boot/xilinx-zynq-fsbl/build/arm-rtems4.11-xilinx_zynq_zc706/hydra-fsbl.elf'
>
> Please remove this. We should not document the hard coded paths. This can be
> added later once we have figure this out.
>
>> +
>> +Building RTEMS Tests
>> +--------------------
>> +
>> +Build RTEMS with a configuration command line something similar to:
>> +
>> +.. note:: The following assumes a Unix-type host and that the tools have been
>> + built with a prefix of ``$HOME/development/rtems/4.12``.
>> +
>> +.. code-block:: shell
>> +
>> + $ cd
>> + $ export PATH=$HOME/development/rtems/4.12/bin:$PATH
>> + $ mkdir -p development/rtems/kernel
>> + $ cd development/rtems/kernel
>> + $ git clone git://git.rtems.org/rtems.git rtems.git
>> + $ cd rtems.git
>> + $ ./bootstrap -c && ./bootstrap -p && ./bootstrap
>> + $ cd ..
>> + $ mkdir -p builds/erc32
>> + $ cd builds/erc32
>> + $ ../../rtems.git/configure --target=sparc-rtems4.12 \
>> + --enable-tests --enable-rtemsbsp=erc32
>> + $ make
>> +
>> +Add the -j option to the make command with the number of cores to run the
>> +build parallel where it can.
>
> I think this can he made simpler by reducing it to documenting the
> ``--enable-tests`` option. A reference to the kernel building sections of this
> manual would help.
>
>> +
>> +Building all the tests takes time and it uses more disk so be patient. When
>> +finished all the tests will be built and ready to run. Before running all the
>> +tests it is a good idea to run the ``hello`` test. The ``hello`` test is an
>> +RTEMS version of the classic "Hello World" example and running it shows you
>> +have a working tool chain and build of RTEMS ready to run the tests. Using the
>> +run command:
>> +
>> +.. code-block:: shell
>> +
>> + $ sparc-rtems4.12-run sparc-rtems4.12/c/erc32/testsuites/samples/hello/hello.exe
>> +
>> + *** BEGIN OF TEST HELLO WORLD ***
>> + Hello World
>> + *** END OF TEST HELLO WORLD ***
>> +
>> +The run command is the GDB simulator without the GDB part.
>> +
>> +Running the example using GDB:
>> +
>> +.. code-block:: shell
>> +
>> + $ sparc-rtems4.12-gdb sparc-rtems4.12/c/erc32/testsuites/samples/hello/hello.exe
>> + GNU gdb (GDB) 7.12
>> + Copyright (C) 2016 Free Software Foundation, Inc.
>> + License GPLv3+: GNU GPL version 3 or later <http://gnu.org/licenses/gpl.html>
>> + This is free software: you are free to change and redistribute it.
>> + There is NO WARRANTY, to the extent permitted by law. Type "show copying"
>> + and "show warranty" for details.
>> + This GDB was configured as "--host=x86_64-linux-gnu --target=sparc-rtems4.12".
>> + Type "show configuration" for configuration details.
>> + For bug reporting instructions, please see:
>> + <http://www.gnu.org/software/gdb/bugs/>.
>> + Find the GDB manual and other documentation resources online at:
>> + <http://www.gnu.org/software/gdb/documentation/>.
>> + For help, type "help".
>> + Type "apropos word" to search for commands related to "word"...
>> + Reading symbols from
>> + sparc-rtems4.12/c/erc32/testsuites/samples/hello/hello.exe...done.
>> + (gdb) target sim
>> + Connected to the simulator.
>> + (gdb) load
>> + (gdb) r
>> + Starting program: sparc-rtems4.12/c/erc32/testsuites/samples/hello/hello.exe
>> +
>> +
>> + *** BEGIN OF TEST HELLO WORLD ***
>> + Hello World
>> + *** END OF TEST HELLO WORLD ***
>> + [Inferior 1 (process 42000) exited normally]
>> + (gdb) q
>> +
>> +The command r is used to debug set break points before issuing the GDB run
>> +command.
>
> The command ``r`` ... ``run`` ..
>> +
>> +There are currently close to 500 separate tests and you can run them all with a
>> +single RTEMS Tester command.
>> +
>> +.. note:: Use a recent RSB version to build a SPARC tool chain. A recent patch
>> + fixes issues with the GDB simulator and it improves the test results.
>> + It is also recommended you use a recent RTEMS 4.11 version as the
>> + tests have been cleaned up to improve the results.
>
> Please remove the note.
>
>> +
>> +Running the Tests
>> +-----------------
>> +
>> +The ``rtems-test`` command line accepts a range of options. These are discussed
>> +later in the manual. Any command line argument without a `--` prefix is a test
>> +executable. You can pass more than one executable on the command line. If the
>> +executable is a path to a directory the directories under that path are
>> +searched for any file with a ``.exe`` extension. This is the default extension
>> +for RTEMS executables built within RTEMS.
>> +
>> +To run the erc32 tests enter the following command from the top of the erc32 BSP
>> +build tree:
>> +
>> +.. code-block:: shell
>> +
>> + $ ~/development/rtems/test/rtems-tools.git/tester/rtems-test \
>> + --log=log_erc32_run \
>> + --rtems-bsp=erc32-run \
>> + --rtems-tools=$HOME/development/rtems/4.12 \
>> + sparc-rtems4.12/c/erc32/testsuites/samples
>> + RTEMS Testing - Tester, 4.12.not_released
>> + [ 1/13] p:0 f:0 u:0 e:0 I:0 B:0 t:0 i:0 | sparc/erc32: base_sp.exe
>> + [ 2/13] p:0 f:0 u:0 e:0 I:0 B:0 t:0 i:0 | sparc/erc32: capture.exe
>> + [ 3/13] p:0 f:0 u:0 e:0 I:0 B:0 t:0 i:0 | sparc/erc32: cdtest.exe
>> + [ 4/13] p:0 f:0 u:0 e:0 I:0 B:0 t:0 i:0 | sparc/erc32: fileio.exe
>> + [ 5/13] p:2 f:0 u:0 e:0 I:0 B:0 t:0 i:0 | sparc/erc32: hello.exe
>> + [ 6/13] p:2 f:0 u:0 e:0 I:0 B:0 t:0 i:0 | sparc/erc32: cxx_iostream.exe
>> + [ 8/13] p:2 f:0 u:0 e:0 I:0 B:0 t:2 i:0 | sparc/erc32: minimum.exe
>> + [ 7/13] p:2 f:0 u:0 e:0 I:0 B:0 t:2 i:0 | sparc/erc32: loopback.exe
>> + [ 9/13] p:3 f:0 u:0 e:0 I:0 B:0 t:3 i:0 | sparc/erc32: nsecs.exe
>> + [10/13] p:3 f:0 u:0 e:0 I:0 B:0 t:3 i:0 | sparc/erc32: paranoia.exe
>> + [11/13] p:4 f:0 u:0 e:0 I:0 B:0 t:3 i:0 | sparc/erc32: pppd.exe
>> + [12/13] p:6 f:0 u:0 e:0 I:0 B:0 t:3 i:0 | sparc/erc32: ticker.exe
>> + [13/13] p:6 f:0 u:0 e:0 I:0 B:0 t:3 i:0 | sparc/erc32: unlimited.exe
>> + Passed: 7
>> + Failed: 0
>> + User Input: 0
>> + Expected Fail: 0
>> + Indeterminate: 0
>> + Benchmark: 0
>> + Timeout: 5
>> + Invalid: 1
>> + Total: 13
>> + Average test time: 0:00:27.963000
>> + Testing time : 0:06:03.519012
>> +
>> +
>> +
>> +* The RTEMS Tester's test command. In this example we are using an absolute
>> + path.
>> +* The ``--log`` option sends the output to a log file. By default only failed
>> + tests log the complete output.
>> +* Select the erc32 BSP and use GDB.
>> +* Path to the RTEMS tools so GDB can be found.
>> +* Path to the erc32 BSP built with all tests to run. If you add subdirectories
>> + to the path specific tests can be run.
>> +* The output has been shortened so it fits nicely here.
>> +* The test results shows passes, fails, timeouts, and invalid results. In
>> + this run 13 tests passed and 5 tests timed out and 1 is invalid. The
>> + timeouts are probably due to the tests not having enough execute time to
>> + complete. The default timeout is 180 seconds and some of the interrupt tests
>> + need longer. The amount of time depends on the performance of your host CPU
>> + running the simulations.
>> +* The output shows the average time per test and the total time taken to run
>> + all the tests.
>> +* If the path to the testsuites was put to
>> + ``sparc-rtems4.12/c/erc32/testsuites`` instead of
>> + ``sparc-rtems4.12/c/erc32/testsuites/samples`` then all the executables
>> + would have been tested and not just those in samples.
>> +
>> +This BSP requires the ``--rtems-tools`` option because the SPARC GDB is the
>> +``sparc-rtems4.11-gdb`` command that is part of the RTEMS tools. Not every BSP
>> +will require this option so you will need to check the specifics of the BSP
>> +configuration to determine if it is needed.
>> +
>> +The output you see is each test starting to run. The ``rtems-test`` command by
>> +default runs multiple tests in parallel so you will see a number start quickly
>> +and then new tests start as others finish. The output shown here is from an
>> +8 core processor so the first 8 are started in parallel and the status shows
>> +the order in which they actually started, which is not 1 to 8.
>> +
>> +The test start line shows the current status of the tests. The status reported
>> +is when the test starts and not the result of that test. A fail, timeout or
>> +invalid count changing means a test running before this test started failed,
>> +not the starting test. The status here has 7 tests passed, no failures, 5
>> +timeouts and 1 invalid test.
>> +
>> +.. code-block:: shell
>> +
>> + [ 5/13] p:2 f:0 u:0 e:0 I:0 B:0 t:0 i:0 | sparc/erc32: hello.exe
>> +
>> +* [ 5/13] indicates the test number, in this case test 5 of 13 tests.
>> +* p is the passed test count (2 in this case)
>
> * ``p`` is the ...
>
> and for the others below ..
>
>> +* f is the failed test count (0 in this case)
>> +* u is the count for test marked as "user-input" as they expect input from user
>> +* e is the expected-fail count (tests that are expected to fail)
>> +* I is the count for tests the results of which are indeterminate
>> +* B is the count for benchmarked tests
>> +* t is the timeout test count
>> +* i is the invalid test count.
>> +* sparc/erc32 is the architecture and BSP names.
>
> ``sparc/erc32`` ..
>
>> +* hello.exe is the executable name.
>
> ``hello.exe`` ...
>
>> +
>> +The test log records all the tests and results. The reporting mode by default
>> +only provides the output history if a test fails, times out, or is invalid. The
>> +time taken by each test is also recorded.
>> +
>> +The tests must complete in a specified time or the test is marked as timed
>> +out. The default timeout is 3 minutes and can be globally changed using the
>> +``--timeout`` command line option. The time required to complete a test can
>> +vary. When simulators are run in parallel the time taken depends on the
>> +specifics of the host machine being used. A test per core is the most stable
>> +method even though more tests can be run than available cores. If your machine
>> +needs longer or you are using a VM you may need to lengthen the timeout.
>> +
>> +Test Status
>> +-----------
>> +
>> +Tests can be marked with one of the following:
>> +
>> +* Pass
>> +* Fail
>> +* Timeout
>> +* Invalid
>
> This needs updating.
>
>> +
>> +The RTEMS console or stdout output from the test is needed to determine the
>
> ... ``staout`` ...
>
>> +result of the test.
>> +
>> +Pass
>> +^^^^
>> +A test passes if the start and end markers are seen in the test output. The
>> +start marker is ``***`` and the end mark is ``*** END OF TEST``. All tests in
>> +the RTEMS test suite have these markers.
>> +
>> +Fail
>> +^^^^
>> +A test fails if the start marker is seen and there is no end marker.
>> +
>> +Timeout
>> +^^^^^^^
>> +If the test does not complete within the timeout setting the test is marked as
>> +having timed out.
>> +
>> +Invalid
>> +^^^^^^^
>> +If no start marker is seen the test is marked as invalid. If you are testing on
>> +real target hardware things can sometimes go wrong and the target may not
>> +initialize or respond to the debugger in an expected way.
>> +
>> +Reporting
>> +---------
>> +
>> +The report written to the log has the following modes:
>> +
>> +* All (``all``)
>> +* Failures (``failures``)
>> +* None (``none``)
>> +
>> +The mode is controlled using the command line option ``--report-mode`` using
>> +the values listed above.
>> +
>> +All
>> +^^^
>> +The output of all tests is written to the log.
>> +
>> +Failures
>> +^^^^^^^^
>> +The output of the all tests that do not pass is written to the log.
>> +
>> +None
>> +^^^^
>> +No output is written to the log.
>> +
>> +The output is tagged so you can determine where it comes from. The following is
>> +the complete output for the In Memory File System test ``imfs_fslink.exe``
>> +running on a Coldfire MCF5235 using GDB and a BDM pod:
>> +
>> +.. code-block:: shell
>> +
>> + [ 11/472] p:9 f:0 t:0 i:1 | m68k/mcf5235: imfs_fslink.exe
>> + > gdb: ..../bin/m68k-rtems4.11-gdb -i=mi --nx --quiet ..../imfs_fslink.exe
>> + > Reading symbols from ..../fstests/imfs_fslink/imfs_fslink.exe...
>> + > done.
>> + > target remote | m68k-bdm-gdbserver pipe 003-005
>> + > Remote debugging using | m68k-bdm-gdbserver pipe 003-005
>> + > m68k-bdm: debug module version 0
>> + > m68k-bdm: detected MCF5235
>> + > m68k-bdm: architecture CF5235 connected to 003-005
>> + > m68k-bdm: Coldfire debug module version is 0 (5206(e)/5235/5272/5282)
>> + > Process 003-005 created; pid = 0
>> + > 0x00006200 in ?? ()
>> + > thb *0xffe254c0
>> + > Hardware assisted breakpoint 1 at 0xffe254c0
>> + > continue
>> + > Continuing.
>> + ]
>> + ]
>> + ] External Reset
>> + ]
>> + ] ColdFire MCF5235 on the BCC
>> + ] Firmware v3b.1a.1a (Built on Jul 21 2004 17:31:28)
>> + ] Copyright 1995-2004 Freescale Semiconductor, Inc. All Rights Reserved.
>> + ]
>> + ] Enter 'help' for help.
>> + ]
>> + > Temporary breakpoint
>> + > 1, 0xffe254c0 in ?? ()
>> + > load
>> + > Loading section .text, size 0x147e0 lma 0x40000
>> + > Loading section .data, size 0x5d0 lma 0x547e0
>> + > Start address 0x40414, load size 85424
>> + > Transfer rate: 10 KB/sec, 1898 bytes/write.
>> + > b bsp_reset
>> + > Breakpoint 2 at 0x41274: file ..../shared/bspreset_loop.c, line 14.
>> + > continue
>> + > Continuing.
>> + ] dBUG>
>> + ]
>> + ] *** FILE SYSTEM TEST ( IMFS ) ***
>> + ] Initializing filesystem IMFS
>> + ]
>> + ]
>> + ] *** LINK TEST ***
>> + ] link creates hardlinks
>> + ] test if the stat is the same
>> + ] chmod and chown
>> + ] unlink then stat the file
>> + ] *** END OF LINK TEST ***
>> + ]
>> + ]
>> + ] Shutting down filesystem IMFS
>> + ] *** END OF FILE SYSTEM TEST ( IMFS ) ***
>> + > Breakpoint
>> + > 2, bsp_reset () at ..../m68k/mcf5235/../../shared/bspreset_loop.c:14
>> + > 14 {
>> + Result: passed Time: 0:00:10.045447
>> +
>> +* GDB command line (Note: paths with \'....' have been shortened)
>> +* Lines starting with ``>`` are from GDB's console.
>> +* Line starting with ``]`` are from the target's console.
>> +* The result with the test time.
>> +
>> +Running Tests in Parallel
>> +-------------------------
>> +
>> +The RTEMS Tester supports parallel execution of tests by default. This only
>> +makes sense if the test back-end can run in parallel without resulting in
>> +resource contention. Simulators are an example of back-ends that can run in
>> +parallel. A hardware debug tool like a BDM or JTAG pod can manage only a
>> +single test at once so the tests need to be run one at a time.
>> +
>> +The test framework manages the test jobs and orders the output in the report
>> +log in test order. Output is held for completed tests until the next test to be
>> +reported has finished.
>> +
>> +Command Line Help
>> +-----------------
>> +
>> +The ``rtems-test`` command line accepts a range of options. You can review the
>> +available option by the ``--help`` option:
>> +
>> +.. code-block:: shell
>> +
>> + RTEMS Tools Project (c) 2012-2014 Chris Johns
>> + Options and arguments:
>> + --always-clean : Always clean the build tree, even with an error
>> + --debug-trace : Debug trace based on specific flags
>> + --dry-run : Do everything but actually run the build
>> + --force : Force the build to proceed
>> + --jobs=[0..n,none,half,full] : Run with specified number of jobs, default: num CPUs.
>> + --keep-going : Do not stop on an error.
>> + --list-bsps : List the supported BSPs
>> + --log file : Log file where all build output is written to
>> + --macros file[,file] : Macro format files to load after the defaults
>> + --no-clean : Do not clean up the build tree
>> + --quiet : Quiet output (not used)
>> + --report-mode : Reporting modes, failures (default),all,none
>> + --rtems-bsp : The RTEMS BSP to run the test on
>> + --rtems-tools : The path to the RTEMS tools
>> + --target : Set the target triplet
>> + --timeout : Set the test timeout in seconds (default 180 seconds)
>> + --trace : Trace the execution
>> + --warn-all : Generate warnings
>> +
>
> This should really be updated to the command formatting for the other commands.
> It can wait will we agree it has to be done.
>
>> +Development
>> +-----------
>> +
>> +The RTEMS Tester framework and command line tool is under active
>> +development. These are changing, being fixed, broken and generally improved. If
>> +you want to help please see the Wiki page for open items.
>> +
>
> Remove this section.
>
>> +History
>> +-------
>> +
>> +The RTEMS Tester is based on a refactored base of Python code used in the RTEMS
>> +Source Builder. This code provided a working tested base that has been extended
>> +and expanded to meet the requirements for the RTEMS Tester. The tester uses the
>> +specifics found in the various scripts and configurations in the
>> +rtems-testing.git repo that has been accumulated over many years. The shell
>> +script implementation is restricted in what it can do and, per BSP script, is a
>> +maintenance burden. For example the command lines and options vary between each
>> +script.
>
> This section can also go.
>
> Thanks
> Chris
More information about the devel
mailing list