Interrupt Latency on PPC (was Re: Interrupt latency problems on MPC860)

[Till Straumann]

OK, let me summarize:

I am using a hardware timer (on the OpenPIC) to measure
the time from asserting an IRQ until my ISR is invoked.
(I.e. the test measures vector hunting, ISR dispatching and
IRQ disabled sections [the latter only on a statistical basis,
of course]).

RTEMS BSP implementation details:

  1) BSP re-enables IRQs during user ISR call [both for
      external interrupts and for the timer tick (decrementer)].

  2) external interrupts are all at priority 8, my timer uses
      priority 13 which is higher [and hence should nest into
      any other ISR].

Hardware: MVME2306 (PPC 604 @ 300MHz, OpenPIC compatible)

Measurement results:

  Latencies are (test run during ~10min):

     Average:   ~3us
     Maximal:  ~15us (idle system or some network traffic)
     Maximal:  ~40us (system loaded with network traffic and serial console
output)

I also recorded the IRQ nesting level (which is available on the
powerpc/shared
BSPs in SPRG0):

    The maximal nesting level that ever occurred was 3 (with a latency of
    only 2us!!). Note that the maximal latency that I measured occurred at a
    nesting level of only 2.

Conclusion:

The maximal latency observed is much worse than the average. Furthermore,
there was a case of high nesting level with a quite low latency. Because the
'vector-hunt' and dispatching code is executed for every single interrupt, it

shows up in the average figure which is quite fast.
Hence, neither the PPC hardware nor the interrupt vector-hunting/dispatching
code can be responsible for the worst case delays.

There _must_ be significant code sections with IRQs disabled!!!

Since heavy console use seems to increase the maximal latency, there must
be IRQ disabled code in the console driver, termios etc.

Note for the hunter: RTEMS core does _not_ call the rtems_interrupt_disable()
but
uses _ISR_Disable(). Furthermore, some BSPs and CPU ports define their own
macros/inlines etc. :-(

If we could bring down the worst case to, let's say < 10us, RTEMS would be
quite a bit faster than vxWorks!

-- Till.