[RTEMS Project] #2961: Run-Time Tracing

[RTEMS trac]

#2961: Run-Time Tracing
---------------------------+------------------------
 Reporter:  vivek kukreja  |      Owner:  (none)
     Type:  project        |     Status:  new
 Priority:  normal         |  Milestone:  Indefinite
Component:  General        |    Version:  4.11
 Severity:  normal         |   Keywords:
---------------------------+------------------------
 = RTEMS Trace Tool =



 [[TOC(Developer/Projects/Open/TraceTool, depth=2)]]

 RTEMS Trace Tool - trace your application in RTEMS

 Chris Johns is the primary contact on this. Please check
 [wiki:Developer/Tracing Tracing] for updated information.

 The RTEMS trace project develops the RTEMS Trace framework. The framework
 provides a consistent user interface to instrumenting an application and
 tracing it. The framework has multiple parts.

  * Instrumenting the application using the RTEMS Trace Linker (rtems-tld).
  * Target execution and capture of data.
  * Control of event triggering and logging.
  * Integration to 3rd party trace toolkits.

 There are a number of uses cases for tracing and these effect the
 requirements for the trace framework. They are:

  * Audit an RTEMS application for conformance of input/output parameters
 and/or timing
  * Debug an application.
  * Realtime visualization.

 '''Acknowledgements'''

  *  Chris Johns is the father of this effort and has mentored two Google
 Summer of Code students.
  *  Cocan Lucian Marius worked on this project for Google Summer of Code
 2009
  *  Zeng (Alana) Reng worked on this project for Google Summer of Code
 2008.

 = Introduction =

 The following are the open tasks for the Trace project:

 1. RTEMS Trace linker (rtems-tld) Symbol Signatures
 1. Target Integration of the Capture Engine with rtems-tld
 1. CTF integration.
 1. Tracing tools improvements.

 '''Note:''' The rtems-tld tool is written in C++ and to work on this  tool
 you need good C++ coding skills and some knowledge of using the   STL. The
 tools has a tight coding standard that is not documented but plenty of
 code exists you can see and follow. Help is available so please ask if you
 need help and are willing to learn.

 == RTEMS Trace linker (rtems-tld) Symbol Signatures ==

 Update the elftools component and add libdwarf support to rtems-tools.
 Provide a C++ framework in a similar manner to the ELF framework to wrap
 libdwarf. The current use case for libdwarf is to get a function signature
 of a library symbol from the debug info and with this create a suitable
 function signature for the trace linker. Currently this is manually done
 by adding the signatures to the generator INI files. The manual approach
 is a stop gap measure and is not maintainable.

 This task requires good C++ skills.

 == Target Integration of the Capture Engine with rtems-tld ==

 The Capture Engine is a module in RTEMS for the target that provides low
 level services for buffers, triggers and captures for the target. This
 task integrates the Capture Engine to the RTEMS Trace Linker to allow
 rtems-tld linked application to use the Capture Engine for the capture,
 buffering and transport of the trace data. The task also includes looking
 at adding other transports to the Capture Engine other than a command line
 and console output. A role of the Capture Engine is to provide an API
 users can access to allow custom transports. This is important with custom
 hardware with restricted external interfaces.

 == CTF integration ==

 [http://www.efficios.com/ctf CTF] is really neat and brings to RTEMS a
 range of features we really need. The format is good and applicable to
 RTEMS and the post processing and real-time monitoring tools are valuable.

 The CTF has a range of tools and parts and this task is to bring those
 together and into the RTEMS Trace Linker. The goal is to encapsulate what
 is needed to use CTF and what it provides with the functionality of the
 RTEMS Trace Linker to build an application for a user that uses CTF. The
 `rtems-tld` tool knows about RTEMS and function signatures (see task 1)
 and it can be taught to internally invoke the CTF tools to create a
 required configuration files and target code to integrate with the CTF
 post processing tools. Being able to use the Capture Engine is considered
 an advantage.

 A key part is [https://github.com/efficios/barectf barectf]. This accepts
 a YAML configuration and generates suitable C code for integration into
 the application. This tool can be wrapped inside the `rtems-tld` and used
 transparently to the user. The `rtems-tld` can build any `barectf`
 generated code for a target because it already knows the compiler and
 flags required by the target. The user's code is wrapped by the GNU ld and
 that creates the dependencies to the `rtems-tld` generated code and in
 turn to the `barectf` generated code. No user code is touched at the
 source or even the object file level and this is an important requirement.

 What follow is mostly guess work on my part and needs to be clarified,
 corrected and cleaned up as we learn more. I base what is written on the
 current wrapping of trace calls is controlled by `rtems-tld` and so it can
 generate the `barectf` configuration file. The `barectf` generated code
 takes in the trace data and returns CTF format records suitable for
 buffering. The Capture Engine is used because it provides safe concurrent
 buffering of data. The project is bring all this together.

 The task can be divided up into:

 1. Investigate how `barectf` works and what code is generated.
 2. Determine how the trace wrapped calls created by `rtems-tld` integrate
 to the `barectf` trace support. I suspect this is a matter of assigning
 event ids in the `barectf` configuration file to match the `rtems-tld`
 generated ids.
 3. Create an `rtems-tld` generator to generate C code to bind to the
 `barectf` calls.
 4. Integrate the `barectf` output support to the Capture Engine so it
 performs the buffering. This assumes the `barectf` code generated is
 sequential and event processing is sequential and contains no code to
 update global variables. The concurrent activity is the buffering and this
 is handled by the Capture Engine.
 5. Determine how to add extra special features such as CPU number.

 The final part of this task is transport to the host machine. How this
 happens has not been determined yet and is open to suggestions and
 innovation.

 == Tracing Improvements ==

 This is an opened ended task and not one to be initially focused on but is
 here for completeness.

 The rtems-tld uses the binutils symbol mapping feature in GNU ld and I
 have observed some interesting behaviour with it. It would be good to
 characterise this behaviour and if possible improve the support in GNU ld.
 I however feel there are limitations, eg tracing static C functions
 internally optimised by the compiler, C++ and templates, and more. Long
 term there maybe the need to add passes to gcc to allow even better trace
 control. I can see us using the [https://fedorahosted.org/gcc-python-
 plugin/ GCC Python Plug-In] tool.

 For this to happen I would like to see a separate GCC Plug-In project
 within RTEMS for RTEMS started and trace support being just one part of
 it. This would allow other interesting but yet to be defined things
 requiring GCC plug-in passes to be collected in one place. A separate
 project for special gcc passes would allow auditing and controls to be
 added so those users in our community who need to know everything about
 the code running in their target can control what happens.

--
Ticket URL: <http://devel.rtems.org/ticket/2961>
RTEMS Project <http://www.rtems.org/>
RTEMS Project