[PATCH] Add low level event recording support

[Sebastian Huber]

On 23/01/2019 07:49, Chris Johns wrote:
> On 23/1/19 12:34 am, Sebastian Huber wrote:
>> Hello Chris,
>>
>> On 20/12/2018 07:46, Sebastian Huber wrote:
>>>> Sorry but I have no time to review this and consider it until next year.
>>> No problem, take your time. I work on this since April this year from time to
>>> time, so it can wait a couple of more weeks.
>> had you time to look at this?
>>
> Sorry I have not.
>
> I understand there is a need for high performance tracing and welcome this work.
> I have tended to use custom code to handle this, an example is
> https://git.rtems.org/rtems/tree/cpukit/dev/i2c/xilinx-axi-i2c.c#n120.

You could use the record support of this patch set for this purpose.

>
>> The focus of this new stuff is the recording of high frequency events on
>> multiple processors. It doesn't deal with filtering or event generation.
> Like the idea but I am confused on how what is being offered fits into what we
> have. It seems to provide better performance but it also overlaps some of what
> we have and is missing some other things. I cannot tell from my brief look if it
> sits as a component in the existing RTEMS trace framework or it is completely
> separate.

It closes a gap in the existing stuff, recording of high frequency 
events on a multiprocessor platform. For example in the output of the 
previous e-mail you see:

6769.471385216:5:TCP_OUTPUT:7dd21000
6769.471385232:4:TCP_INPUT:7dd77100

These two events on different processors are separated by 16ns. It is 
impossible to get this independence with a recording facility which uses 
locks (atomic read-modify-write) or a global time source. You need 
per-processor data structures with per-processor synchronization and a 
per-processor time source for this. It is not complicated, it is just a 
ring buffer per processor.

>
> I am OK with something new and better but we need to make sure what we offer is
> consistent and makes sense to users. I am concerned users will become confused
> if we have multiple approaches with separate code, set up, post processing and
> documentation. I am fine if what we have is replaced and removed if your new
> approach is to become a complete framework.

There is no one size fits all. This stuff just solves one problem and 
tries to do it well. You need other components that work together to get 
a good tracing solution for RTEMS. This is a major piece of work. 
Providing a documentation set along is a lot of work. Maybe we can use 
GSoC projects in this area.

>
> Filtering and triggering as implemented is important in some cases to make sure
> the overhead and transport processes does not become unstable but it costs at
> run-time. If done property the overhead could be small. Manually inserting trace
> points is problematic for some users and wrapping has limitations. I am not sure
> we can have a single approach for all use cases.
>
>> Attached is a very simple example program to get the records via a TCP stream
>> from the target. It outputs text like this:
> How have you inserted the trace points on the target in this example? Again I am
> missing the high level view. Is this in the detail of the other patch?

I tried to use the GNU ld --wrap, but this didn't work well since it 
wraps only undefined references. I had also a look at extending GNU ld 
to support also defined references, but this turned out to be quite 
difficult to me.

Some events are generated by user extensions. In the network stack I 
used code patches. The focus of this recording stuff is not the event 
generation, the filtering, the transport from A to B, visualization, 
etc. The focus is event recording of high frequency events on 
multiprocessor platforms. It is just one tiny building block that can be 
used in a tracing infrastructure.

>
> A common transport approach would be nice for all pieces of the trace puzzle.

I think that transformation to standard formats should be done on a host 
computer and not the target.

-- 
Sebastian Huber, embedded brains GmbH

Address : Dornierstr. 4, D-82178 Puchheim, Germany
Phone   : +49 89 189 47 41-16
Fax     : +49 89 189 47 41-09
E-Mail  : sebastian.huber at embedded-brains.de
PGP     : Public key available on request.

Diese Nachricht ist keine geschäftliche Mitteilung im Sinne des EHUG.