student for gsoc2008

[Reng Zeng]

Hi Chris,

I am also interested to apply GSoc with run-time tracing. Your explanation
below helps a lot!

My understanding and questions for this project is as following, Your
comments to confirm it or any correction would be greatly appreciated.

   - Addition of trace points to generate log entries
      - Currently the Capture engine only hooks RTEMS user extension
      and trace those events including Task creation, Task initiation, Task
      reinitiation,  Task deletion,  Task context switch,  Post task context
      switch, Task begin,  Task exits,  Fatal error detection, which
is defined by

      typedef struct {
        /** This field is the thread creation handler. */
        User_extensions_thread_create_extension       thread_create;
        /** This field is the thread starting handler. */
        User_extensions_thread_start_extension        thread_start;
        /** This field is the thread restarting handler. */
        User_extensions_thread_restart_extension      thread_restart;
        /** This field is the thread deleting handler */
        User_extensions_thread_delete_extension       thread_delete;
        /** This field is thread context switch handler. */
        User_extensions_thread_switch_extension       thread_switch;
        /** This field is the thread beginning handler. */
        User_extensions_thread_begin_extension        thread_begin;
        /** This field is thread exiting handler. */
        User_extensions_thread_exitted_extension      thread_exitted;
        /** This field is the fatal error extension. */
        User_extensions_fatal_extension               fatal;
      }   User_extensions_Table;
      - Current capture engine hook the user extension table as
      prototype to test performance impact by introducing the filtering,
      triggering, and storage of events. So this part of project is to
add tracing
      points by using "Wrap capability" of GNU ld.
      - For first stage, there are two lists, one is the function list
      specified by user, another is the function list extracted from
the library.
      The task is, for every function matched, generated __Wrap version.
      - For second stage, integrate the new __Wrap functions with the
      existing capture engine to enable filtering and triggering.
      - This way could make it possible to add a bulk of tracing point
      without changing any code in the libraries to be traced.
      - Transmission of logged data to host
      - The first challenge of this part is to make the system and the
      log survive from high traffic.
      - The second challenge is to implement it in TCP while providing
      the flexibility for other transports in future.
      - The third challenge is to trade off minimizing the data to be
      sent across the network against the over-head to compress data.
      - Receipt of logged data on host
      - Use Python to write a CLI to receive data into a file.
      - Decoding of logged data on host
      - From the compressed data received, decode it for high-level
      application to display as tracing data.
      - GUI Timeline Visualization tool
      - Use GNUplot to integrate the decoded data into visualization
      format. What kind of visualization is expected?

Regarding to the schedule, I have no idea how long it need to figure out the
set of user generated list of trace points.


Thanks,
Alan

------------------------------

   - *Date*: Wed, 19 Mar 2008 12:24:09 +1100
   - *From*: chrisj at rtems.org (Chris Johns)
   - *Subject*: student for gsoc2008

------------------------------

Maria Soler wrote:
> Hi,
>
>> Just start discussing it on the list.  That way the entire
>>  community can provide answers and we can enhance the
>>  ideas page to answer questions.  Even if you are
>>  trying to decide between multiple ideas, it will capture
>>  information for when the others get tackled.
>>
>>  I definitely want to help everyone on their project proposal.
>>
>
> The project that catches more my attention is the run-time tracing. It
> is said that some prototype is already in the code. I've been taking a
> look at the code.. but it is difficult to follow it at the first look.
> Can you point out any specific parts to look at for understanding?
>

The Wiki page in Open Projects talks about multiple parts. The prototype code
refers to 2 specific parts. The first is the Capture Engine in the RTEMS
source tree. This code is under cpukit/libmisc/capture. I have a tutorial on
how to use the Capture engine with the pc BSP and qemu. This can be found in
my ftp area:

  ftp://www.rtems.org/pub/rtems/people/chrisj/capture

The Capture engine code currently only hooks the RTEMS user extensions and
traces those events. This API is documented in the C User Manual. The
interface allowed easy access to a readily available stream of events from
RTEMS without extensive RTEMS changes as the focus of the work was on
filtering and triggering in timely manner.

The capture code has 2 parts. The capture engine (capture.[ch]) and a command
line user interface (capture-cli.[ch]). An application wishing to use the
capture engine via the CLI calls 'rtems_capture_cli_init'. The separation
allows for the addition of other interfaces such as TCP. How well this
separation is currently implemented will only be determined when something
like TCP is added. I am sure some refactoring will be needed.

The capture engine code has a few parts. The first is a collection of utility
type functions such as 'rtems_capture_match_names' and
'rtems_capture_match_ids'. Then some useful functions such as
'rtems_capture_create_control', 'rtems_capture_create_capture_task' and
'rtems_capture_record'. The last one handles the filtering and the placing of
the record in the event buffer. The other important function in this group is
the 'rtems_capture_trigger' function. This handles the triggering. This code
requires an understanding of how the various trigger types are mapped to the
capture control structure. Understanding the terms is important as well as the
the correct perspective. For example a "from trigger" on the task being
"switched to" requires you look at the "to" task's "from" list for the "from"
task. A "to" trigger on a "from" task is the opposite. Yeap it does get a
little twisted and takes a little time to understand so do not rush this part.
The tutorial helps here as it shows the type of triggers at a high level. The
next group of functions pick up the events from the context switch user
extension. The final group is the API to the capture engine to control and
configure it. The difficult one here is the 'rtems_capture_set_trigger'
function again because of the how you look and name triggers.

The second part of the prototype code relates to how Joel and I see RTEMS
feeding events to the Capture engine. Joel found the linker had a --wrap
option and this is a really nice feature of the GNU linker. The man page for
ld has the details. It allows you to write a function called __wrap_malloc and
in it call __real_malloc. If you then pass to the linker the option '--wrap
malloc' and the linker will point all 'malloc' calls to __wrap_malloc and pull
in malloc from the C library and rename it __real_malloc. No code changes in
the library and no code is recompiled with special defines. The prototype code
I have takes a library or object file and attempts to extract the function
signatures from that library. Joel has code that takes function signatures and
creates the wrap code that is linked to the application. This means we take a
user created a list of functions we wish to trace and the first tool obtains
the function signatures from the library, the second tool creates a C file of
wrapped functions and final phase is to relink the application.

The final piece of the puzzle is to integrate this into the existing capture
engine. The interesting issues to resolve are the management of the filters
and triggers as there could be a large number and event number allocation and
tracking for each entry and exit of the traced function. A further issue to
consider is the capture of arguments and return values.

I hope this helps.


Regards
Chris
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.rtems.org/pipermail/users/attachments/20080321/13809f9b/attachment-0001.html>