Multiple dec2114x units on ppc

[Till Straumann]

gregory.menke at gsfc.nasa.gov wrote:
> Joel Sherrill writes:
>  > 
>  > 
>  > gregory.menke at gsfc.nasa.gov wrote:
>  > > 
>  > > Till Straumann writes:
>  > >  > gregory.menke at gsfc.nasa.gov wrote:
>  > >  > > Till Straumann writes:
>  > >  > >  > Gregory.
>  > 
>  > Sorry .. deleted some comment about irq API on/off.  I believe
>  > these are support routines that should be able to be provided 
>  > by the BSP for a specific DEC NIC interface.  Just because it is
>  > a particular NIC doesn't mean it has the usual interrupt setup.
>  > The "normal" way could be provided as support routines which the
>  > BSP could configure without providing them itself.
>  > 
>  > This is simply part of the glue that a BSP may have to provide or
>  > use defaults.
> 
> The issue we're grappling with is cpci interrupts in general.  Any
> cpci board that can issue an interrupt must face this issue.
> 

BTW: it's not only CPCI. The same applies for PMC modules (at least
on some boards).

>  
>  > >  > NOTE: since the BSP doesn't implement true IRQ sharing,
>  > >  >        you run into trouble as soon as different PCI
>  > >  >        devices (i.e. a dec2114x and some other device)
>  > >  >        share a common IRQ line.
>  > > 
>  > > It looks like managing the cpci vectoring in some kind of organized
>  > > way will be quite a pain.  It makes me almost want to defer it to the
>  > > user- if they want cpci devices, they must supply INTERRUPT_PIN and
>  > > INTERRUPT_LINE(for translation to interrupt name) for each device
>  > > they're going to use.  Its not pretty, but it would keep things
>  > > simpler.
>  > 
>  > I don't know the exact interrupt structure the baord has but 
>  > we found that a PowerPC BSP almost has to support being able to
>  > have an "ISR handler" that is in fact a second level vectoring
>  > routine.  THe case I am thinking of is a PMC module that has about
>  > 15 interrupt sources on it.  It is mapped to a single IRQ to the
>  > baseboard which is in turn off of an IRQ.  You get one interrupt
>  > that says "something on the PMC wants attention" and it is up to 
>  > "PMC specific vectoring routine" to figure out which one.  
> 
> This is my understanding; The PPC board has a 16 channel interrupt
> controller- MPIC, supported by the openpic* suite of functions.
> Various built-in peripherals wire their interrupts to particular (and
> known) channels on the controller- these are no problem, one vector
> per interrupt (with the exception being the ISA interrupts, which I
> don't really understand yet).  The cpci chassis wires each of the 4
> PCI interrupt wires onto the controller as well, also on known
> channels.  The problem is the cpci boards cannot supply a vector when
> asserting an interrupt- all the interrupt controller sees is one (or
> more) of the 4 cpci interrupt lines is asserting an interrupt.  Since
> each line has its own vector, the PPC vectoring routines can at most
> distinguish between 4 sets of interrupt sources.  If you have n >= 2
> boards that both assert INT A, then something in the interrupt service
> routines has to check each of the boards to see which is asserting the
> interrupt (and it could be 1, some or all of them).  The 4 interrupt
> pins are entirely independent, so the easy case would be to assign
> each of 4 cpci boards their own pin (and as a consequence each has
> their own vector)- which is fine until you have more than 4 boards.
> The PPC firmware doesn't configure the cpci or pmc slots on startup,
> which is why this issue is so wide open.
> 
> This leaves 2 alternatives;
> 
> 1.  Have the user supply the interrupt line and pin for each cpci
>     device.  If vectors can be chained, then compliant drivers can do
>     a low-cost check if its their board asserting and service it if so
>     or allow the system to keep checking if not.
> 
> 2.  Make a fancier bsp-level interrupt management layer that abstracts
>     all the issues- this will still require vector chaining.
> 

 > Frankly, I like #1.

I don't agree. There are actually 2 (orthogonal) issues here.
For one we have interrupt routing which is a PCI configuration
issue and should be treated as such.
It is an essential FEATURE of PCI. The BSP just needs to correctly
setup the INTERRUPT_LINE PCI config register.

The second issue is interrupt sharing. The IRQ API must support
sharing interrupts to correctly support PCI devices (as you say
yourself).

Just consider the mess you get when you have a couple of
(could even be multifunction) boards in your crate. You really
believe having the user hardcode the interrupt routing
is a good idea? What happens if it occurs to someone to
swap boards - I don't know too much about CPCI but I know
that PMC carriers usually rotate interrupt lines, i.e. the
slot matters...

Such behaviour is not acceptible e.g. at a large accelerator
facility with a long lifetime. I start sweating when I think
of some technician repairing a crate in a couple of years
from now who accidentially swaps slots.

I repeat my mantra: it's a BSP's responsibility to fixup the
routing information. Some boards (e.g. the Synergy VGMs) have
firmware who correctly configures PCI already.

> It requires less labor to make it work-

Not true. All the driver writer has to do is read PCI_INTERRUPT_LINE
and use a BSP macro to translate it into a 'name'.

Only the BSP writer (supporting boards with incomplete/buggy
PCI configuration) has a little more work.

Last but not least: PCI is a standard and RTEMS should follow it.

-- Till

> really
> its a driver architecture issue- they must be chainable and have a
> low-cost interrupt assertion test and thats pretty much it.  An
> abstract interface is nice, but it buries a lot of complexity that I
> think is unneeded in most cases.  I think keeping the PIC servicing
> buried in the BSP is a good idea, the drivers should only have to know
> about how to service their devices.  It seems to me we could define an
> "interrupt vector" struct which the user fills in and that takes on
> appropriate meanings for each bsp using it, sort of like how struct
> sockaddr can be interpreted for different address families.
> 
> Gregm
>