Kind of strange interrupt behavior

[gregory.menke at gsfc.nasa.gov]

Hi,

I've been doing some benchmarking of interrupts under RTEMS on our
MIPS R3000 board, and have been seeing something a bit odd.  I was
wondering if anyone else has seen it- either way, it would be helpful
information while we try to track it down.

In our benchmarks, every 20 minutes or so, for a couple minutes,
interrupt latency increases by a "substantial" amount.  "Substantial"
appears to be proportional to the number of tasks being scheduled and
does not appear to correspond to networking being enabled.  In the
most pathological cases, the latencies went up by 2 orders of
magnitude.

On our board, the normal interrupt latency is about 100 to 120us.
When only a single thread is being scheduled, the spikes are only 50us
or so.  In a heavily loaded system, latencies often exceed 1ms.

I have some initial results on 

http://osgroup.gsfc.nasa.gov:8000/interrupt.html


I somewhat suspect our bsp, as it has not had time to mature and be
deeply debugged.  Even so, the 20 minute periodicity is very
suspicious.  I've run a couple different programs, measuring the int
latency experienced by each, and have seen identical behavior.


We are coming under the heavy hand of sweeping IT Policy changes at
Goddard- imposed from on high with no negiotiation or consultation.
As a consequence, our site may or may not be available to addresses
outside Goddard.  We're going thru official channels, such as they
are, trying to maintain public access to the site.  If it appears we
cannot do so, I'll provide Joel with our info as we develop it and
maybe he can mirror it someplace.

Gregm