[PATCH v7 2/2] Adding Trace Documentation
Vidushi Vashishth
reachvidu at gmail.com
Thu Jun 14 14:17:58 UTC 2018
- Updates #3454
- This commit adds Tracing Framework Chapter in the RTEMS User Manual
- It comprises of subchapters on RTEMS Trace Linker, Capture Engine, Trace
generation techniques explaining trace generation using Trace Buffering and
Printk generators and sample demonstrations.
---
user/index.rst | 2 +
user/tracing/captureengine.rst | 169 ++++++++++++++
user/tracing/examples.rst | 199 +++++++++++++++++
user/tracing/index.rst | 29 +++
user/tracing/introduction.rst | 103 +++++++++
user/tracing/tracelinker.rst | 494 +++++++++++++++++++++++++++++++++++++++++
6 files changed, 996 insertions(+)
create mode 100644 user/tracing/captureengine.rst
create mode 100644 user/tracing/examples.rst
create mode 100644 user/tracing/index.rst
create mode 100644 user/tracing/introduction.rst
create mode 100644 user/tracing/tracelinker.rst
diff --git a/user/index.rst b/user/index.rst
index 8cbcd1b..a764fe8 100644
--- a/user/index.rst
+++ b/user/index.rst
@@ -52,6 +52,8 @@ to the Community Project hosted at http://www.rtems.org/.
tools/index
+ tracing/index
+
support/index
glossary/index
diff --git a/user/tracing/captureengine.rst b/user/tracing/captureengine.rst
new file mode 100644
index 0000000..8eaed3b
--- /dev/null
+++ b/user/tracing/captureengine.rst
@@ -0,0 +1,169 @@
+.. comment SPDX-License-Identifier: CC-BY-SA-4.0
+
+.. comment: Copyright (c) 2018 Vidushi Vashishth <vidushivashishth96 at gmail.com>
+.. comment: All rights reserved.
+
+.. _capturengine:
+
+Capture Engine
+**************
+
+Capture Engine is a trace tool built inside the RTEMS operating system. Capture
+Engine is designed to cause the lowest load on the system when operating. Hence
+it does not effect RTEMS when operating or when disabled. It binds to RTEMS at
+runtime and does not require RTEMS or your application to be rebuilt in order
+to use it.
+
+The Capture Engine's sample testcase for the `sparc/erc32` is available in
+build directory created when building RTEMS in the path
+file: `sparc-rtems5/c/erc32/testsuites/samples`. In order to access the capture
+testcase perform the following set of operations inside the RTEMS build
+directory.
+
+.. code-block:: shell
+
+ $ cd /sparc-rtems5/c/erc32/testsuites/samples
+ $ sparc-rtems5-run ./capture.exe
+
+
+ *** BEGIN OF TEST CAPTURE ENGINE ***
+ *** TEST VERSION: 5.0.0.de9b7d712bf5da6593386fd4fbca0d5f8b8431d8
+ *** TEST STATE: USER_INPUT
+ *** TEST BUILD: RTEMS_NETWORKING RTEMS_POSIX_API
+ *** TEST TOOLS: 7.3.0 20180125 (RTEMS 5, RSB a3a6c34c150a357e57769a26a460c475e188438f, Newlib 3.0.0)
+ Press any key to start capture engine (20s remaining)
+ Press any key to start capture engine (19s remaining)
+ Press any key to start capture engine (18s remaining)
+
+ Monitor ready, press enter to login.
+
+ 1-rtems $
+
+Capture Engine comes with a set of commands to perform various actions.
+
+Capture Engine Commands
+-----------------------
+
+1) ``copen <buffer-size>``: Used to initialize the Capture Engine with the
+ trace buffer size in bytes. By default the Capture Engine is not initialized
+ and not running.
+
+2) ``cwceil <priority-value>``: Capture Engine filter used to put an upper
+ limit on the event priority to be captured.
+
+3) ``cwfloor <priority-value>``: Capture Engine filter used to put a lower
+ limit on the event priority to be captured.
+
+4) ``cwglob <on/off>``: Enable or disable the global watch.
+
+5) ``cenable``: Enables the Capture Engine. Capture Engine is by default
+ disabled after being opened.
+
+6) ``cdisable``: Disables the Capture Engine.
+
+7) ``ctlist``: Lists the watch and trigger configurations.
+
+8) ``ctrace``: Dumps the recorded traces. By default this command displays 24
+ trace records. Repeated use of this command will display all the recorded
+ traces.
+
+9) ``cwadd <task-name>``: Add watch on a particular task.
+
+10) ``cwtctl <task-name> <on/off>``: Enable or disable watch on a particular
+ task.
+
+11) ``ctset``: Used to set a trigger. The general form of the command is:
+
+``ctset [-?] type [to name/id] [from] [from name/id]``
+
+`type` in the above command refers to the type of trigger needed. The types of
+triggers that currently exist are:
+
+- switch : a context switch from one task to another task
+- create : the executing task creates a task
+- start : the executing task starts a task
+- restart : the executing task restarts a task
+- delete : the executing task deletes a task
+- begin : a task is beginning
+- exitted : a task is exitting
+
+Example
+-------
+
+The following is a sample run of the capture testsuite. The `test1` command on
+the Capture Engine Command Line Interface (CLI) makes the `RMON` task invoke a
+call to the `capture_test_1()` command. This function (in the `test1.c` source
+code) creates and starts three tasks : `CT1a`, `CT1b` and `CT1c`. These tasks
+are passed the object id of a semaphore as a task argument. This run through
+traces the context switches between these tasks. ``cwceil`` and ``cwfloor`` are
+set to a narrow range of task priorities to avoid creating noise from a large
+number of context switches between tasks we are not interested in.
+
+.. code:: shell
+
+ *** BEGIN OF TEST CAPTURE ENGINE ***
+ *** TEST VERSION: 5.0.0.de9b7d712bf5da6593386fd4fbca0d5f8b8431d8
+ *** TEST STATE: USER_INPUT
+ *** TEST BUILD: RTEMS_NETWORKING RTEMS_POSIX_API
+ *** TEST TOOLS: 7.3.0 20180125 (RTEMS 5, RSB a3a6c34c150a357e57769a26a460c475e188438f, Newlib 3.0.0)
+ Press any key to start capture engine (20s remaining)
+ Press any key to start capture engine (19s remaining)
+ Press any key to start capture engine (18s remaining)
+ Press any key to start capture engine (17s remaining)
+
+ Monitor ready, press enter to login.
+
+ 1-rtems $ copen 50000
+ capture engine opened.
+ 1-rtems $ cwceil 100
+ watch ceiling is 100.
+ 1-rtems $ cwfloor 102
+ watch floor is 102.
+ 1-rtems $ cwglob on
+ global watch enabled.
+ 1-rtems $ ctset RMON
+ trigger set.
+ 1-rtems $ cenable
+ capture engine enabled.
+ 1-rtems $ test1
+ 1-rtems $ cdisable
+ capture engine disabled.
+ 1-rtems $ ctrace
+ 0 0:18:17.462314124 0a010003 CT1a 102 102 102 4096 TASK_RECORD
+ 0 0:18:17.462398963 0 0a010003 CT1a 102 102 CREATED
+ 0 0:18:17.462647987 249024 0a010003 CT1a 102 102 STARTED
+ 0 0:18:17.462904334 256347 0a010003 CT1a 102 102 SWITCHED_IN
+ 0 0:18:17.463069129 164795 0a010003 CT1a 102 102 BEGIN
+ 0 0:18:17.463335853 266724 0a010003 CT1a 102 102 SWITCHED_OUT
+ 0 0:18:18.461348547 0a010004 CT1b 101 101 101 4096 TASK_RECORD
+ 0 0:18:18.461433997 998098144 0a010004 CT1b 101 101 CREATED
+ 0 0:18:18.461683631 249634 0a010004 CT1b 101 101 STARTED
+ 0 0:18:18.461934485 250854 0a010004 CT1b 101 101 SWITCHED_IN
+ 0 0:18:18.462099891 165406 0a010004 CT1b 101 101 BEGIN
+ 0 0:18:19.460935339 998835448 0a010004 CT1b 101 101 SWITCHED_OUT
+ 0 0:18:19.461431555 0a010005 CT1c 100 100 100 4096 TASK_RECORD
+ 0 0:18:19.461516394 581055 0a010005 CT1c 100 100 CREATED
+ 0 0:18:19.461765418 249024 0a010005 CT1c 100 100 STARTED
+ 0 0:18:19.462019324 253906 0a010005 CT1c 100 100 SWITCHED_IN
+ 0 0:18:19.462184119 164795 0a010005 CT1c 100 100 BEGIN
+ 0 0:18:19.462475257 291138 0a010005 CT1c 100 100 SWITCHED_OUT
+ 0 0:18:19.462551551 76294 0a010004 CT1b 101 101 SWITCHED_IN
+ 0 0:18:19.960935645 498384094 0a010004 CT1b 101 101 SWITCHED_OUT
+ 0 0:18:19.961012549 76904 0a010003 CT1a 102 100 SWITCHED_IN
+ 0 0:18:19.961341528 328979 0a010003 CT1a 102 102 SWITCHED_OUT
+ 1-rtems $ ctrace
+ 0 0:18:19.961418433 0 0a010005 CT1c 100 100 SWITCHED_IN
+ 0 0:18:19.961672339 253906 0a010005 CT1c 100 100 SWITCHED_OUT
+ 0 0:18:19.961749854 77515 0a010004 CT1b 101 101 SWITCHED_IN
+ 0 0:18:20.460967077 499217223 0a010004 CT1b 101 101 SWITCHED_OUT
+ 0 0:18:20.461219763 252686 0a010005 CT1c 100 100 SWITCHED_IN
+ 0 0:18:20.461424231 204468 0a010005 CT1c 100 100 TERMINATED
+ 0 0:18:20.461747107 322876 0a010005 CT1c 100 100 SWITCHED_OUT
+ 0 0:18:20.461824011 76904 0a010004 CT1b 101 101 SWITCHED_IN
+ 0 0:18:20.462015052 191041 0a010004 CT1b 101 101 TERMINATED
+ 0 0:18:20.462336707 321655 0a010004 CT1b 101 101 SWITCHED_OUT
+ 0 0:18:20.462414222 77515 0a010003 CT1a 102 102 SWITCHED_IN
+ 0 0:18:20.462608924 194702 0a010003 CT1a 102 102 TERMINATED
+ 0 0:18:20.462933021 324097 0a010003 CT1a 102 102 SWITCHED_OUT
+ 1-rtems $ ctrace
+ 1-rtems $
diff --git a/user/tracing/examples.rst b/user/tracing/examples.rst
new file mode 100644
index 0000000..a694bff
--- /dev/null
+++ b/user/tracing/examples.rst
@@ -0,0 +1,199 @@
+.. comment SPDX-License-Identifier: CC-BY-SA-4.0
+
+.. comment: Copyright (c) 2018 Vidushi Vashishth <vidushivashishth96 at gmail.com>
+.. comment: All rights reserved.
+
+.. _examples:
+
+Tracing Examples
+****************
+
+The following example executes RTEMS trace using trace buffering for the
+`fileio` sample testcase.
+
+Features
+--------
+
+Tracing using trace buffering consists of the following sets of features:
+
+- Individual entry and exit records.
+- Task details such as CPU, current priority, real priority, task state and
+ interrupt state.
+- Nano-second timestamp.
+- Interrupt safe buffer management.
+- Function argument capture.
+- Return value capture.
+- Shell command support to report to the console, save a buffer, assess status
+ of tracing, or view buffers between specified index ranges.
+
+Prerequisites
+-------------
+
+1. Setup RTEMS for the `sparc/erc32` architecture-bsp pair to run the
+ following example.
+2. Download the fileio `configuration file <https://devel.rtems.org/attachment
+ /wiki/Developer/Tracing/Trace_Buffering/fileio-trace.ini>`_ and store it on
+ the top of the installed BSP's directory.
+3. Change the value of the keys: `rtems-path` and `prefix` according to your
+ rtems installation. The `rtems-path` is the path to the bsp installation
+ and `prefix` is the path to the tools used to build rtems. Also set the
+ value of the `rtems-bsp` key to `sparc/erc32`.
+
+Demonstration
+-------------
+
+Inside the RTEMS build directory (the directory where the fileio configuration
+has been stored) run the following commands to generate traces:
+
+BSP is configured with the following command -
+
+.. code:: shell
+
+ ../rtems/configure --target=sparc-rtems5 --prefix=/development/rtems/5 \
+ --enable-networking --enable-tests --enable-rtemsbsp=erc32 --enable-cxx
+
+The next two commands are used to link the fileio executable.The `-B` option
+signifies the use of the complete path to the required directory or file. Write
+the full path instead of the path file: `sparc-rtems5/erc32/lib/` in the
+following commands according to your installation. Also confirm the path of the
+fileio’s executable and object files in the last line of the command according
+to your installation.
+
+.. code:: shell
+
+ sparc-rtems5-gcc -Bsparc-rtems5/erc32/lib/ \
+ -specs bsp_specs -qrtems -mcpu=cypress -O2 -g -ffunction-sections \
+ -fdata-sections -Wall -Wmissing-prototypes -Wimplicit-function-declaration \
+ -Wstrict-prototypes -Wnested-externs -Wl,--gc-sections -mcpu=cypress \
+ -o sparc-rtems5/c/erc32/testsuites/samples/fileio.exe sparc-rtems5/c/erc32/\
+ testsuites/samples/fileio/fileio-init.o
+
+This is the trace linker command to generate and compile the wrapper c file for
+the application. The link command follows the escape sequence "--". "-C" option
+denotes the name of the user configuration file and "-W" specifies the name of
+the wrapper c file.
+
+.. code:: shell
+
+ rtems-tld -C fileio-trace.ini -W fileio-wrapper -- -Bsparc-rtems5/erc32/lib/ \
+ -specs bsp_specs -qrtems -mcpu=cypress -O2 -g -ffunction-sections \
+ -fdata-sections -Wall -Wmissing-prototypes -Wimplicit-function-declaration \
+ -Wstrict-prototypes -Wnested-externs -Wl,--gc-sections -mcpu=cypress \
+ -o sparc-rtems5/c/erc32/testsuites/samples/fileio.exe sparc-rtems5/c/erc32/\
+ testsuites/samples/fileio/fileio-init.o
+
+The following command is used to run the application. Hit enter key quickly and
+type "s" and "root" and "pwd" to run the rtems shell. Use the `rtrace status`,
+`rtrace trace` and `rtrace save` commands to know the status of the tracing,
+display the contents of the trace buffer and save the buffer to disk in the form
+of binary files. Use `rtrace -l` to list the availalble options for commands
+with `rtrace`.
+
+.. code:: shell
+
+ sparc-rtems5-run sparc-rtems5/c/erc32/testsuites/samples/fileio.exe
+
+The output from the above commands will be as follows:
+
+.. code:: shell
+
+ *** BEGIN OF TEST FILE I/O ***
+ *** TEST VERSION: 5.0.0.de9b7d712bf5da6593386fd4fbca0d5f8b8431d8
+ *** TEST STATE: USER_INPUT
+ *** TEST BUILD: RTEMS_NETWORKING RTEMS_POSIX_API
+ *** TEST TOOLS: 7.3.0 20180125 (RTEMS 5, RSB a3a6c34c150a357e57769a26a460c475e188438f, Newlib 3.0.0)
+ Press any key to start file I/O sample (20s remaining)
+ Press any key to start file I/O sample (19s remaining)
+ Press any key to start file I/O sample (18s remaining)
+ Press any key to start file I/O sample (17s remaining)
+ Press any key to start file I/O sample (16s remaining)
+ Press any key to start file I/O sample (15s remaining)
+ Press any key to start file I/O sample (14s remaining)
+ =========================
+ RTEMS FILE I/O Test Menu
+ =========================
+ p -> part_table_initialize
+ f -> mount all disks in fs_table
+ l -> list file
+ r -> read file
+ w -> write file
+ s -> start shell
+ Enter your selection ==>s
+ Creating /etc/passwd and group with four useable accounts:
+ root/pwd
+ test/pwd
+ rtems/NO PASSWORD
+ chroot/NO PASSWORD
+ Only the root user has access to all available commands.
+ =========================
+ starting shell
+ =========================
+
+ Welcome to rtems-5.0.0 (SPARC/w/FPU/erc32)
+ COPYRIGHT (c) 1989-2008.
+ On-Line Applications Research Corporation (OAR).
+
+ Login into RTEMS
+ /dev/foobar login: root
+ Password:
+
+ RTEMS Shell on /dev/foobar. Use 'help' to list commands.
+ SHLL [/] # rtrace status
+ RTEMS Trace Bufferring: status
+ Running: yes
+ Triggered: yes
+ Level: 0%
+ Traces: 25
+ SHLL [/] # rtrace stop
+ RTEMS Trace Bufferring: stop
+ SHLL [/] # rtrace trace
+ RTEMS Trace Bufferring: trace
+ Trace buffer: 0x20921d8
+ Words traced: 1487
+ Traces: 25
+ 0:00:40.983197010 2081910 0a010002 [ 2/ 2] > malloc((size_t) 00000130)
+ 0:00:40.983333119 136109 0a010002 [ 2/ 2] < malloc => (void*) 0x219bb88
+ 0:00:40.983471669 138550 0a010002 [ 2/ 2] > malloc((size_t) 00000006)
+ 0:00:40.983606557 134888 0a010002 [ 2/ 2] < malloc => (void*) 0x219bcc0
+ 0:00:40.983684682 78125 0a010002 [ 2/ 2] > malloc((size_t) 00000007)
+ 0:00:40.983819569 134887 0a010002 [ 2/ 2] < malloc => (void*) 0x219bcd0
+ 0:00:40.983909901 90332 0a010002 [ 2/ 2] > malloc((size_t) 000003fc)
+ 0:00:40.984046620 136719 0a010002 [ 2/ 2] < malloc => (void*) 0x219bce0
+ 0:00:40.986624137 2577517 0a010003 [200/200] > malloc((size_t) 00000080)
+ 0:00:40.986767569 143432 0a010003 [200/200] < malloc => (void*) 0x219bce0
+ 0:00:40.987531119 763550 0a010003 [200/200] > calloc((size_t) 00000001, (size_t) 0000005d)
+ 0:00:40.987603751 72632 0a010003 [200/200] > malloc((size_t) 0000005d)
+ 0:00:40.987744743 140992 0a010003 [200/200] < malloc => (void*) 0x219bce0
+ 0:00:40.987824699 79956 0a010003 [200/200] < calloc => (void*) 0x219bce0
+ 0:00:40.988302604 477905 0a010003 [200/200] > malloc((size_t) 00000080)
+ 0:00:40.988446647 144043 0a010003 [200/200] < malloc => (void*) 0x219bd48
+ 0:00:40.988667595 220948 0a010003 [200/200] > calloc((size_t) 00000001, (size_t) 00000080)
+ 0:00:40.988740837 73242 0a010003 [200/200] > malloc((size_t) 00000080)
+ 0:00:40.988884880 144043 0a010003 [200/200] < malloc => (void*) 0x219bdd0
+ 0:00:40.988964836 79956 0a010003 [200/200] < calloc => (void*) 0x219bdd0
+ 0:00:40.989042961 78125 0a010003 [200/200] > calloc((size_t) 00000001, (size_t) 00000080)
+ 0:00:40.989110100 67139 0a010003 [200/200] > malloc((size_t) 00000080)
+ 0:00:40.989254143 144043 0a010003 [200/200] < malloc => (void*) 0x219be58
+ 0:00:40.989334099 79956 0a010003 [200/200] < calloc => (void*) 0x219be58
+ 0:00:40.990118401 784302 0a010003 [200/200] > calloc((size_t) 00000001, (size_t) 00000061)
+ 0:00:40.990176995 58594 0a010003 [200/200] > malloc((size_t) 00000061)
+ 0:00:40.990309441 132446 0a010003 [200/200] < malloc => (void*) 0x219bd48
+ 0:00:40.990384515 75074 0a010003 [200/200] < calloc => (void*) 0x219bd48
+ 0:00:40.990870355 485840 0a010003 [200/200] > malloc((size_t) 00000080)
+ 0:00:40.991011346 140991 0a010003 [200/200] < malloc => (void*) 0x219bee0
+ 0:00:40.991227411 216065 0a010003 [200/200] > calloc((size_t) 00000001, (size_t) 00000080)
+ 0:00:40.991296380 68969 0a010003 [200/200] > malloc((size_t) 00000080)
+ 0:00:40.991438593 142213 0a010003 [200/200] < malloc => (void*) 0x219bf68
+ 0:00:40.991514276 75683 0a010003 [200/200] < calloc => (void*) 0x219bf68
+ 0:00:40.991589349 75073 0a010003 [200/200] > calloc((size_t) 00000001, (size_t) 00000080)
+ 0:00:40.991653437 64088 0a010003 [200/200] > malloc((size_t) 00000080)
+ 0:00:40.991794428 140991 0a010003 [200/200] < malloc => (void*) 0x219bff0
+ 0:00:40.991871332 76904 0a010003 [200/200] < calloc => (void*) 0x219bff0
+ 0:00:40.992283320 411988 0a010003 [200/200] > malloc((size_t) 00000008)
+ SHLL [/] # rtrace save fileio-trace.bin
+ RTEMS Trace Bufferring: trace
+ Trace File: fileio-trace.bin
+ Trace buffer: 0x20921d8
+ Words traced: 1487
+ Traces: 25
+ SHLL [/] #
diff --git a/user/tracing/index.rst b/user/tracing/index.rst
new file mode 100644
index 0000000..0d714bc
--- /dev/null
+++ b/user/tracing/index.rst
@@ -0,0 +1,29 @@
+.. comment SPDX-License-Identifier: CC-BY-SA-4.0
+
+.. comment: Copyright (c) 2016 Chris Johns <chrisj at rtems.org>
+.. comment: All rights reserved.
+
+.. _tracing-framework:
+
+RTEMS Tracing Framework
+***********************
+.. index:: Tracing Framework
+
+RTEMS Tracing Framework is an on-target software based system which helps track
+the ongoings inside the operation of applications, 3rd party packages, and the
+kernel in real time.
+
+Software based tracing is a complex process which requires components on both
+the target and the host to work together. However its portability across all
+architectures and board support packages makes it a useful asset. A key
+requirement in RTEMS trace process is to take existing code in compiled format
+(ELF) and instrument it in order to log various events and records in real time.
+However instrumenting of the code for tracing should happen without rebuilding
+the code from the source and without annotating the source with trace code.
+
+.. toctree::
+
+ introduction
+ examples
+ captureengine
+ tracelinker
diff --git a/user/tracing/introduction.rst b/user/tracing/introduction.rst
new file mode 100644
index 0000000..27de441
--- /dev/null
+++ b/user/tracing/introduction.rst
@@ -0,0 +1,103 @@
+.. comment SPDX-License-Identifier: CC-BY-SA-4.0
+
+.. comment: Copyright (c) 2016 Chris Johns <chrisj at rtems.org>
+.. comment: All rights reserved.
+
+.. _introduction:
+
+Introduction to Tracing
+***********************
+
+Tracing is an important function which has several applications including
+identification of complex threading, detection of deadlocks, tracing
+functions along with their argument values, and return values through
+progression of several function calls and audit the performance of an
+application according to required specifications.
+
+RTEMS tracing framework is under development and welcomes contribution by users.
+
+RTEMS has the following trace components:
+
+- RTEMS :ref:`tracelinker`
+- RTEMS :ref:`capturengine`
+- Common Trace Format Integration
+
+RTEMS trace framework can currently function using the following methods. Both
+of the methods make use of the :ref:`tracelinker` :
+
+.. _tracebuffering:
+
+RTEMS Trace Using Trace Buffering
+=================================
+
+This scheme of tracing goes through the flow of events described in a
+subsequent flowchart:
+
+Step 1: The user creates an application and user configuration file. The
+configuration file specifies the use of the trace buffer generator and other
+standard initializations. The user then configures their BSP and invokes the
+trace linker using a command to link the application executable. The trace
+linker uses the application files in compiled format (ELF) and the libraries
+used to build the application for performing this link.
+
+Step 2: The RTEMS Trace Linker reads the user’s configuration file and that
+results in it reading the standard Trace Buffering Configuration files
+installed with the RTEMS Trace Linker. The trace linker uses the target
+compiler and linker to create the trace enabled application executable. It
+wraps the functions defined in the user’s configuration with code that captures
+trace records into the statically allocated buffer. The trace wrapper code is
+compiled with the target compiler and the resulting ELF object file is added to
+the standard link command line used to link the application and the application
+is re-linked using the wrapping option of the GNU linker.
+
+Step 3: The trace linker creates an executable which is capable of running on
+the target hardware or simulator.
+
+Step 4: RTEMS shell provides the “rtrace” command to display and save trace
+buffers.
+
+.. comment: taken from https://devel.rtems.org/wiki/Developer/Tracing
+.. figure:: ../../images/user/rtems-trace-buffering.png
+ :align: center
+ :width: 75%
+
+.. _printk:
+
+RTEMS Trace Using Printk
+========================
+
+This scheme of tracing goes through the flow of events described in a subsequent
+flowchart:
+
+Step 1: The user creates an RTEMS application in the normal manner as well as a
+Trace Linker configuration file. The configuration file specifies using the
+Printk trace mode and the functions to trace. The user invokes the Trace Linker
+with the configuration and the normal link command line used to the link the
+application executable. The application ELF object files and libraries,
+including the RTEMS libraries are standard and do not need to be built
+specially.
+
+Step 2: The RTEMS Trace Linker reads the user's configuration file and that
+results in it reading the standard Printk Trace Configuration files installed
+with the RTEMS Trace Linker. The trace linker uses the target compiler and
+linker to create the trace enabled application executable. It wraps the
+functions defined in the user's configuration with code that prints the entry
+with arguments and exit and return value if any. The trace wrapper code is
+compiled with the target compiler and the resulting ELF object file is added to
+the standard link command line used to link the application and the application
+is relinked using the wrapping option of the GNU linker.
+
+Step 3: The trace linker creates and RTEMS ELF executable that can be run on the
+target hardware or simulator.
+
+Step 4: The application is run in the hardware directly or using a debugger. The
+printk() output appears on the target console and the user can save that to a
+file.
+
+.. comment: taken from https://devel.rtems.org/wiki/Developer/Tracing
+.. figure:: ../../images/user/rtems-trace-printk.png
+ :align: center
+ :width: 75%
+
+The :ref:`examples` section describes generation of traces using Trace Buffering
+technique for the `fileio` testsuite available with RTEMS installation.
diff --git a/user/tracing/tracelinker.rst b/user/tracing/tracelinker.rst
new file mode 100644
index 0000000..ab42de6
--- /dev/null
+++ b/user/tracing/tracelinker.rst
@@ -0,0 +1,494 @@
+.. comment SPDX-License-Identifier: CC-BY-SA-4.0
+
+.. comment: Copyright (c) 2016 Chris Johns <chrisj at rtems.org>
+.. comment: All rights reserved.
+
+.. _tracelinker:
+
+Trace Linker
+************
+
+RTEMS trace linker is a post link tool central to the RTEMS trace framework. It
+is installed as a part of the RTEMS Tool Project. The RTEMS Trace Linker is a
+post link tool that performs a re-link of your application to produce a trace
+executable. A trace executable has been instrumented by the RTEMS Trace Linker
+with additional code that implements software tracing. A key requirement of the
+trace process in RTEMS is to take existing code in a compiled format (ELF) and
+instrument it without rebuilding that code from source and without annotating
+that source with trace code.
+
+Command Line
+============
+
+A typical command to invoke the trace linker consists of two parts separated by
+``--``. The first part controls the trace linker and provides the various
+options it needs and the second part is a standard linker command line you would
+use to link an RTEMS application. The current command line for trace linker
+consists of:
+
+.. code-block:: shell
+
+ $ rtems-tld -h
+ rtems-trace-ld [options] objects
+ Options and arguments:
+ -h : help (also --help)
+ -V : print linker version number and exit (also --version)
+ -v : verbose (trace import parts), can supply multiple times
+ to increase verbosity (also --verbose)
+ -w : generate warnings (also --warn)
+ -k : keep temporary files (also --keep)
+ -c compiler : target compiler is not standard (also --compiler)
+ -l linker : target linker is not standard (also --linker)
+ -E prefix : the RTEMS tool prefix (also --exec-prefix)
+ -f cflags : C compiler flags (also --cflags)
+ -r path : RTEMS path (also --rtems)
+ -B bsp : RTEMS arch/bsp (also --rtems-bsp)
+ -W wrapper : wrapper file name without ext (also --wrapper)
+ -C ini : user configuration INI file (also --config)
+ -P path : user configuration INI file search path (also --path)
+
+The trace linker generates code that needs to be compiled and linked to the
+application executable so it needs to know the target compiler and `CFLAGS`.
+There are a couple of ways to do this. The simplest is to provide the path to
+RTEMS using the `-r` option and the architecture and BSP name in the standard
+RTEMS format of arch/bsp. The trace linker will extract the compiler and flags
+used to build RTEMS and will use them. If you require specific options you can
+use the `-f`, `-c`, `-l`, and `-E` options to provide them. If the functions you
+are tracing use types from your code then add the include path to the `CFLAGS`.
+
+The trace linker requires you to provide a user configuration file using the
+`-C` or ``--config`` option. This is an INI format file detailed in the
+Configuration section. You can also provide an INI file search path using the
+`-P` option.
+
+If you are working with new configuration files and you want to view the files
+the trace linker generates, add the `-k` option to keep the temporary files, and
+`-W` to specify an explicit wrapper C file name. If you set the
+``dump-on-error`` option in the configuration options section you will get a
+dump of the configuration on an error.
+
+Configuration (INI) files
+=========================
+
+The Trace Linker is controlled using configuration files. Configuration files
+are categorized into 3 types:
+
+- User Configuration: These are specific to the user application to be traced.
+ This file initializes the values of the trace generator, triggers, enables,
+ and traces.
+
+- Tracer Configuration: These are like a library of common or base trace
+ functions that can be referenced by an application. These files tend to hold
+ the details needed to wrap a specific set of functions. Examples provided with
+ the RTEMS Linker are the RTEMS API and Libc.
+
+- Generator Configuration: This is used to encapsulate a specific method of
+ tracing. RTEMS currently provides generators for trace buffering, printk, and
+ printf.
+
+The configuration files are in the *INI file format* which is composed of
+`sections`. Each section has a section name and set of *keys* which consist of
+*names* and *values*. A typical key is of the form ``name=value``. Keys can be
+used to include other INI files using the include key name. This is shown in the
+following example where the values indicate rtems and rtld-base configuration
+files:
+
+.. code-block:: shell
+
+ include = rtems.ini, rtld-base.ini
+
+The trace linker also uses values in keys to specify other sections. In this
+example the functions name lists `test-trace-funcs` and that section contains a
+headers key that further references a section called `test-headers`:
+
+.. code-block:: shell
+
+ functions = test-trace-funcs, rtems-api
+
+ [test-trace-funcs]
+ ; Parsed via the 'function-set', not parse as a 'trace'.
+ headers = test-headers
+
+ [test-headers]
+ header = '#include "test-trace-1.h"'
+
+The format of a configuration file is explained next. Snippets of the file:
+`test-trace.ini` have been used for explicit understanding. This file can
+be found in the rtems-tools directory of the rtems installation.
+
+Tracer Section
+--------------
+
+The topmost level section is the ``tracer`` section. It can contains the
+following keys:
+
+- ``name``: The name of trace being linked.
+
+- ``options``: A list of option sections.
+
+- ``defines``: A list of sections containing defines or define record.
+
+- ``define``: A list of define string that are single or double quoted.
+
+- ``enables``: The list of sections containing enabled functions to trace.
+
+- ``triggers``: The list of sections containing enabled functions to trigger
+ trace on.
+
+- ``traces``: The list of sections containing function lists to trace.
+
+- ``functions``: The list of sections containing function details.
+
+- ``include``: The list of files to include.
+
+The tracer section of the file:`test-trace.ini` is shown below with explanatory
+comments.
+
+.. code-block:: shell
+
+ ;
+ ; RTEMS Trace Linker Test Configuration.
+ ;
+ ; We must provide a top level trace section.
+ ;
+ [tracer]
+ ;
+ ; Name of the trace.
+ ;
+ name = RTEMS Trace Linker Test
+ ;
+ ; The BSP.
+ ;
+ bsp = sparc/sis
+ ;
+ ; Functions to trace.
+ ;
+ traces = test-trace, test-trace-funcs, rtems-api-task
+ ;
+ ; Specify the options.
+ ;
+ options = test-options
+ ;
+ ; Define the function sets. These are the function's that can be
+ ; added to the trace lists.
+ ;
+ functions = test-trace-funcs, rtems-api
+ ;
+ ; Include RTEMS Trace support.
+ ;
+ include = rtems.ini, rtld-base.ini
+
+Options section
+---------------
+
+The options section in the fileio-trace.ini is called the `fileio-options`. A
+general options section can contain following sets of keys:
+
+- ``dump-on-error``: Dump the parsed configuration data on error. The value can
+ be true or false.
+
+- ``verbose``: Set the verbose level. The value can be true or a number value.
+
+- ``prefix``: The prefix for the tools and an install RTEMS if rtems-path is not
+ set.
+
+- ``cc``: The compiler used to compile the generated wrapper code. Overrides the
+ BSP configuration value if a BSP is specified.
+
+- ``ld``: The linker used to link the application. The default is the cc value
+ as read from the BSP configuration if specified. If your application contains
+ C++ code use this setting to the change the linker to g++.
+
+- ``cflags``: Set the CFLAGS used to compiler the wrapper. These flags are
+ pre-pended to the BSP read flags if a BSP is specified. This option is used
+ to provide extra include paths to header files in your application that
+ contain types referenced by functions being traced.
+
+- ``rtems-path``: The path to an install RTEMS if not installed under the
+ prefix.
+
+- ``rtems-bsp``: The BSP we are building the trace executable for. The is an
+ arch and bsp pair. For example sparc/erc32.
+
+The options section of the file: `test-trace.ini` uses two of the aforementioned
+keys as shown below:
+
+.. code-block:: shell
+
+ ;
+ ; Options can be defined here or on the command line.
+ ;
+ [test-options]
+ prefix = /development/rtems/5
+ verbose = true
+
+Trace Section
+-------------
+
+A trace section defines how trace wrapper functions are built. To build a trace
+function that wraps an existing function in an ELF object file or library
+archive we need to have the function's signature. A signature is the function's
+declaration with any types used. The signature has specific types and we need
+access to those types which means the wrapper code needs to include header files
+that define those types. There may also be specific defines needed to access
+those types. A trace section can contain the following keys:
+
+- ``generator``: The generator defines the type of tracing being used.
+
+- ``headers``: List of sections that contain header file's keys.
+
+- ``header``: A header key. Typically the include code.
+
+- ``defines``: List of sections that contain defines.
+
+- ``define``: A define key. Typically the define code.
+
+- ``signatures``: List of function signature sections.
+
+- ``trace``: Functions that are instrumented with trace code.
+
+The trace section of the file: `test-trace.ini` is shown below. A trace section
+can reference other trace sections of a specific type. This allows a trace
+sections to build on other trace sections.
+
+.. code:: shell
+
+ ; User application trace example.
+ ;
+ [test-trace]
+ generator = printf-generator
+ ; Just here for testing.
+ trace = test_trace_3
+
+ [test-trace-funcs]
+ ; Parsed via the 'function-set', not parse as a 'trace'.
+ headers = test-headers
+ header = '#include "test-trace-2.h"'
+ defines = test-defines
+ define = "#define TEST_TRACE_2 2"
+ signatures = test-signatures
+ ; Parsed via the 'trace', not parsed as a function-set
+ trace = test_trace_1, test_trace_2
+
+ [test-headers]
+ header = '#include "test-trace-1.h"'
+
+ [test-defines]
+ define = "#define TEST_TRACE_1 1"
+
+ [test-signatures]
+ test_trace_1 = void, int
+ test_trace_2 = test_type_2, test_type_1
+ test_trace_3 = float, float*
+
+Function Section
+----------------
+
+Function sections define functions that can be traced. Defining a function so it
+can be traced does not mean it is traced. The function must be added to a trace
+list to be traced. Function sections provide any required defines, header files,
+and the function signatures.
+
+A function signature is the function's declaration. It is the name of the
+function, the return value, and the arguments. Tracing using function wrappers
+requires that we have accurate function signatures and ideally we would like to
+determine the function signature from the data held in ELF files. ELF files can
+contain DWARF data, the ELF debugging data format. In time the trace project
+would like to support libdwarf so the DWARF data can be accessed and used to
+determine a function's signature. This work is planned but not scheduled to be
+done and so in the meantime we explicitly define the function signatures in the
+configuration files.
+
+A function section can consist of the following keys:
+
+- ``headers``: A list of sections containing headers or header records.
+- ``header``: A list of include string that are single or double quoted.
+- ``defines``: A list of sections containing defines or define record.
+- ``defines``: A list of define string that are single or double quoted.
+- ``signatures``: A list of section names of function signatures.
+- ``includes``: A list of files to include.
+
+Function signatures are specified with the function name being the key's name
+and the key's value being the return value and a list of function arguments. You
+need to provide void if the function uses void. Variable argument list are
+currently not supported. There is no way to determine statically a variable
+argument list. The function section in the file: `test-trace.ini` has been
+labeled as `test-trace-funcs`. This can be seen in the file snippet of the
+previous section.
+
+Generators
+----------
+
+The trace linker's major role is to wrap functions in the existing executable
+with trace code. The directions on how to wrap application functions is provided
+by the generator configuration. The wrapping function uses a GNU linker option
+called --wrap=symbol. The GNU Ld manual states:
+
+"Use a wrapper function for symbol. Any undefined reference to symbol will be
+resolved to __wrap_symbol. Any undefined reference to __real_symbol will be
+resolved to symbol."
+
+Generator sections specify how to generate trace wrapping code. The trace
+linker and generator section must match to work. The trace linker expects a some
+things to be present when wrapping functions. The section's name specifies the
+generator and can be listed in a generator key in a tracer or trace section. If
+the generator is not interested in a specific phase it does not need to define
+it. Nothing will be generated in regard to this phase. For example code to
+profile specific functions may only provide the entry-trace and exit-trace code
+where a nano-second time stamp is taken.
+
+The generate code will create an entry and exit call and the generator code
+block can be used to allocate buffer space for each with the lock held. The
+entry call and argument copy is performed with the lock released. The buffer
+space having been allocated will cause the trace events to be in order. The same
+goes for the exit call. Space is allocated in separate buffer allocate calls so
+the blocking calls will have the exit event appear in the correct location in
+the buffer.
+
+The following keys can be a part of the generator configuration:
+
+- ``headers``: A list of sections containing headers or header records.
+- ``header``: A list of include string that are single or double quoted.
+- ``defines``: A list of sections containing defines or define record.
+- ``define``: A list of define string that are single or double quoted.
+- ``entry-trace``: The wrapper call made on a function's entry. Returns bool
+ where true is the function is being traced. This call is made without the lock
+ being held if a lock is defined.
+- ``arg-trace``: The wrapper call made for each argument to the trace function
+ if the function is being traced. This call is made without the lock being held
+ if a lock is defined.
+- ``exit-trace``: The wrapper call made after a function's exit. Returns bool
+ where true is the function is being traced. This call is made without the lock
+ being held if a lock is defined.
+- ``ret-trace``: The wrapper call made to log the return value if the function
+ is being traced. This call is made without the lock being held if a lock is
+ defined.
+- ``lock-local``: The wrapper code to declare a local lock variable.
+- ``lock-acquire``: The wrapper code to acquire the lock.
+- ``lock-release``: The wrapper code to release the lock.
+- ``buffer-local``: The wrapper code to declare a buffer index local variable.
+- ``buffer-alloc``: The wrapper call made with a lock held if defined to
+ allocate buffer space to hold the trace data. A suitable 32bit buffer index is
+ returned. If there is no space an invalid index is returned. The generator
+ must handle any overhead space needed. The generator needs to make sure the
+ space is available before making the alloc all.
+- ``code-blocks``: A list of code block section names.
+- ``code``: A code block in <<CODE --- CODE (without the single quote).
+- ``includes``: A list of files to include.
+
+The following macros can be used in wrapper calls:
+
+- ``@FUNC_NAME@``: The trace function name as a quote C string.
+- ``@FUNC_INDEX@``: The trace function index as a held in the sorted list of
+ trace functions by the trace linker. It can be used to index the names,
+ enables, and triggers data.
+- ``@FUNC_LABEL@``: The trace function name as a C label that can be referenced.
+ You can take the address of the label.
+- ``@FUNC_DATA_SIZE@``: The size of the data in bytes.
+- ``@FUNC_DATA_ENTRY_SIZE@``: The size of the entry data in bytes.
+- ``@FUNC_DATA_RET_SIZE@``: The size of the return data in bytes.
+- ``@ARG_NUM@``: The argument number to the trace function.
+- ``@ARG_TYPE@``: The type of the argument as a C string.
+- ``@ARG_SIZE@``: The size of the type of the argument in bytes.
+- ``@ARG_LABEL@``: The argument as a C label that can be referenced.
+- ``@RET_TYPE@``: The type of the return value as a C string.
+- ``@RET_SIZE@``: The size of the type of the return value in bytes.
+- ``@RET_LABEL@``: The return value as a C label that can be referenced.
+
+The `buffer-alloc`, `entry-trace`, and `exit-trace` can be transformed using the
+following macros:
+
+- ``@FUNC_NAME@``
+- ``@FUNC_INDEX@``
+- ``@FUNC_LABEL@``
+- ``@FUNC_DATA_SZIE@``
+- ``@FUNC_DATA_ENTRY_SZIE@``
+- ``@FUNC_DATA_EXIT_SZIE@``
+
+The `arg-trace` can be transformed using the following macros:
+
+- ``@ARG_NUM@``
+- ``@ARG_TYPE@``
+- ``@ARG_SIZE@``
+- ``@ARG_LABEL@``
+
+The `ret-trace` can be transformed using the following macros:
+
+- ``@RET_TYPE@``
+- ``@RET_SIZE@``
+- ``@RET_LABEL@``
+
+The file: `test-trace.ini` specifies ``printf-generator`` as its generator. This
+section can be found in the file: `rtld-print.ini` in the rtems-tools directory
+and is shown below:
+
+.. code:: shell
+
+ ;
+ ; A printf generator prints to stdout the trace functions.
+ ;
+ [printf-generator]
+ headers = printf-generator-headers
+ entry-trace = "rtld_pg_printf_entry(@FUNC_NAME@, (void*) &@FUNC_LABEL@);"
+ arg-trace = "rtld_pg_printf_arg(@ARG_NUM@, @ARG_TYPE@, @ARG_SIZE@, (void*) &@ARG_LABEL@);"
+ exit-trace = "rtld_pg_printf_exit(@FUNC_NAME@, (void*) &@FUNC_LABEL@);"
+ ret-trace = "rtld_pg_printf_ret(@RET_TYPE@, @RET_SIZE@, (void*) &@RET_LABEL@);"
+ code = <<<CODE
+ static inline void rtld_pg_printf_entry(const char* func_name,
+ void* func_addr)
+ {
+ printf (">>> %s (0x%08x)\n", func_name, func_addr);
+ }
+ static inline void rtld_pg_printf_arg(int arg_num,
+ const char* arg_type,
+ int arg_size,
+ void* arg)
+ {
+ const unsigned char* p = arg;
+ int i;
+ printf (" %2d] %s(%d) = ", arg_num, arg_type, arg_size);
+ for (i = 0; i < arg_size; ++i, ++p) printf ("%02x", (unsigned int) *p);
+ printf ("\n");
+ }
+ static inline void rtld_pg_printf_exit(const char* func_name,
+ void* func_addr)
+ {
+ printf ("<<< %s (0x%08x)\n", func_name, func_addr);
+ }
+ static inline void rtld_pg_printf_ret(const char* ret_type,
+ int ret_size,
+ void* ret)
+ {
+ const unsigned char* p = ret;
+ int i;
+ printf (" rt] %s(%d) = ", ret_type, ret_size);
+ for (i = 0; i < ret_size; ++i, ++p) printf ("%02x", (unsigned int) *p);
+ printf ("\n");
+ }
+ CODE
+
+ [printf-generator-headers]
+ header = "#include <stdio.h>"
+
+The trace linker generates C code with a wrapper for each function to be
+instrumented. The trace code generated is driven by the configuration INI files.
+
+Development
+===========
+
+The Trace Linker is part of the RTEMS tools git repository available at :
+https://git.rtems.org/rtems-tools
+The RTEMS tools project utilizes the waf build system. Use the following
+commands in the topmost build directory to build the tools project:
+
+First we configure using:
+
+.. code-block:: shell
+
+ $./waf configure --prefix=$HOME/development/rtems/5
+
+Then we build and install using:
+
+.. code-block:: shell
+
+ $./waf build install
--
2.7.4
More information about the devel
mailing list