PPC IRQ routing tutorial (was Re: Bugs in the PPC/shared interrupt code + dec21140)

[till]

gregory.menke at gsfc.nasa.gov wrote:
-- snip --

>
>
>It seems that RTEMS simply uses the default content of the
>PCI_INTERRUPT_LINE register, blindly using it as the vector.  It would
>be useful to include something in the revised irq api to provide some
>means of allocating vectors rather than simply picking arbitrary ones.
>Would traversing the vector list to find unused entries be enough?
>
>
>
> > 
> > 
> > >  --> fix to this would require the dec21140 driver to
> > >      a) add the correct PIC_IRQ_LOWEST_OFFSET to the
> > >         PCI_INTERRUPT_LINE
> > >      b) bootloader would have to do some fixup on the
> > >         PCI_INTERRUPT_LINE register to replace the
> > >         ISA interrupt lines with the OpenPIC ones.
> > > --> submitted as PR#361
> > > 
> > > Interesting
> > 
>
>I'm completely overhauling the dec21140 driver right now to put it
>into line with the existing vectoring api
>
What do you mean by 'the existing vectoring api'?

Before you completely overhaul the dec21140 driver, _please_ make sure you
understand what's going on.  Note the difference between an interrupt LINE
and an interrupt VECTOR: (the explanation is slightly geared towards the
PowerPC architecture example)

- an interrupt LINE is a a hardware resource used by the interrupter 
("device")
   to signal an event (IRQ) to the CPU. You can think of the interrupt 
line as of
   a 'wire', usually it is one wire amongst others on a bus.
   Many computers provide  more interupt lines than the CPU itself has. They
   usually incorporate PICs (interrupt controllers) to manage and multiplex
   multiple interrupt lines into a single output who is connected to the CPU
   (or even a cascaded PIC).

   E.g. the PowerPC has one 'external exception (EE)' input pin which is 
used to
   interrupt the CPU. An 'OpenPIC' interrupt controller is connected to 
EE. The
   OpenPIC has sixteen 'external interrupt' inputs. On some platforms, 
one of
   these inputs is used to cascade a second, legacy ISA PIC (which usually
   itself is a cascade of two PICs)

-  An interrupt VECTOR is a software resource, a mere 'number'  generated
   by the interrupter and read by the CPU during an interrupt 
acknowledge cycle.
   (Originally, the term 'vector' was introduced on hardware which supports
   reading the vector during an interrupt acknowledge cycle and directly 
branch to
   a vector-specific ISR)
   This is useful to distinguish amongst different interrupters. Here 
are two
   examples:

     a) VME: The VME bus supports 7 interrupt lines (AKA 'levels') AND 
reading
          a vector from the bus as part of the IACK (interrupt 
acknowledge) cycle.
          A VME device 'knows' its vector number (hardcoded or 
programmed into
         a device register during setup by the driver) and puts it on 
the bus during
         IACK. --> The CPU, upon reception of an interrupt on one of the 
7 levels
         does IACK and identifies the interrupter based on the vector.

     b) PowerPC/PCI/OpenPIC. The PCI bus has 4 interrupt lines (INTA..INTD)
         and NO concept of an interrupt vector. If several devices share 
a PCI
         interrupt line, software has to query all devices to identify 
the interrupter.
         The OpenPIC, as already mentioned, has 16 interrupt input pins. 
It also
         supports 'vectors' in hardware. This means that each of the 16 
inputs
         can be associated with a number (vector) which is produced by the
         OpenPIC as a result of an interupt acknowledge command. It's simply
         a fast way of identifying the currently active input pin (with 
the highest priority).
         Hence, in this case, the vector is not generated by the 
interrupting device
         but by the OpenPIC itself.

After this lengthy introduction, I hope it becomes clear that

1) OpenPIC 'vectors' are (transparently) handled by the BSP. They are used
     internally to identify the interrupting pin on the OpenPIC (the 
vectors happen
     to be identical to the OpenPIC interrupt line number + 16, i.e. the 
interrupt
     "NAME").

2) The motorola Powerpc boards have ~32 interrupt lines. 15 ISA interrupts
     and 15 OpenPIC interrupts. (the ~ is due to the fact that some 
lines are
     used for cascading the PICs and are not usable by devices). The 
interrupt
     'NAME' is a number defined by the BSP's IRQ API. Names are an alias
     for interrupt lines starting with the ISA interrupts. E.g. the 2nd 
input of the
     OpenPIC has 'name' 17, the 5th ISA interrupt has the 'name' 4
     (I start numbering pins with 1, names start with 0 - unusable lines
      [cascade] still happen to have a 'name').

3) RTEMS has NO WAY of knowing how interrupt wires/lines are routed.
     The board designer decided to which OpenPIC input he/her wanted to
     connect the DEC21140 device. Likewise for PMC/PCI interrupts 
INTA..INTD.
     These four are connected to four different OpenPIC interrupt lines only
     known to the board designer (and the BSP developer).

4) Usually, the OpenPIC is used for PCI interrupts. Firmware should set
     the PCI INTERRUPT_LINE config register to the correct line used by
     the respective device on the particular board. Hence, it makes sense
     for a driver to use that information. When using the
     BSP_install_rtems_irq_handler() API, it needs to be translated into
     an irq NAME, however (by adding BSP_PCI_INTERRUPT_LOWEST_OFFSET).
     Makes sense, doesn't it? Read a PCI_INTERRUPT_LINE, but the
     API requires a 'name' --> conversion necessary.

5) Unfortunately: motorola/ppc boards connect a couple of devices to 
_multiple_
     interrupt lines (i.e. ISA _and_ OpenPIC inputs) and for legacy
     reasons set the PCI_INTERRUPT_LINE register to the ISA line.
     Since the RTEMS driver (if it wasn't buggy) assumes the DEC21140
     to be a PCI device and hence hooked to the OpenPIC, it uses the
     wrong name. Note that if the PCI API was unified across x86/ppc,
     ISR installation could well be identical (since the BSPs share the
     IRQ API).

It has been suggested that the BSP simply define 'symbolic names' for
interrupts which should then be used by the driver. The driver would then
not use PCI_INTERRUPT_LINE but a symbolic constant such as
BSP_IRQ_NAME_DEC21140. Since the BSP developer knows the physical
interrupt routing, this sounds like a good solution, doesn't it?

NO, IT DOESN'T. It's OK for non-PCI on-board devices, but for PCI it is BAD.
(It is one of the reasons why PCI was invented).
Imagine, you want to support multiple instances or PMC cards. How are you
going to know what slot it will be plugged into? You will have to pass
something like BSP_IRQ_PCIINTA to driver initialization. And recompile.
What if the PMC module is swapped into another slot after a couple of
months? Oh  - forgot to modify the application (now using BSP_IRQ_PCIINTB)
and recompile.

IMO, the only solution to this is having fixup code in the BSP startup on
boards with buggy/incomplete PCI initialization firmware.

Sorry for the long message

-- Till

> and to support multipleu
>units.  Right now the built-in dec21140 still works (whew...), and the
>2nd unit is addressing OK but having interrupt problems.  I'll give
>the above a try.
>
>I REALLY wish somebody would tell me where to go to find and/or submit
>PRs.  www.rtems.com has been inaccessible to me all weekend.
>
>Gregm
>
>