PCI API

[Till Straumann]

Kate Feng wrote:

>Conclusion :
>
>Done!  Eveyone will be  happy.   Thanks to all  for clearing  the
>confusion.  I will  add :
>
>#define  MaxPCIbridgePerBandwidth  8      /* Did not figure out a better name yet */
>  
>
what should that be needed for?

>to the  bsp.h of mvme5500.  Thus I  can use the original PCI API.
>
>
>"chris at chriscaudle.org" wrote:
>
>  
>
>>>Imagine that one application has  five devices on PCI0.  One day, it requires to
>>>hot adding   two more  PCI devices while the system has
>>>been serving  for three months.  However, the application can not
>>>disturb any other devices on PCI1 - at least for the prebuilt 1GHZ
>>>network.  What is the new bus number for the two added devices
>>>based on the original proposal  ?
>>>      
>>>
>>Maybe I don't understand exactly what you are asking, but it is as simple as seems, then the two added devices have the same bus number as the other five devices on the PCI0 bridge.  All devices on a single bus segment have the same bus number.
>>If one of the cards installed has a bridge, then the secondary devices on that card have a higher bus number.  If there are no other busses in the system, all devices on PCI0 would probably have bus number 0, all devices on PCI1 would have bus number 1, and if you added a device that contained a bridge onto the bus segment from PCI0, it would be assigned bus number 2.
>>    
>>
>
>Maybe a better example is to look at Peter Default's drawing for his system :
>(P.S. : Thanks for the drawing)
>
>MVME5500 PCI diagram
>PCI.0(0/0/0)-+-IPMC(0/1/0)
>             |     11,-,-,-
>             |
>             +-PMCSLOT1(0/6/0)
>             |          8,9,10,11
>             |
>             +-PMCSLOT1 IDSEL B(0/7/0)
>             |
>             +-PCI6154BR(0/10/0)-+-PCI-VMEBR(1/0/0)
>             |                   |           12,13,14,15
>             |                   |
>             |                   +-DEC21150BR(1/4/0)-+-PMCSPAN1(2/2/0)
>             |                                       |          14,15,12,13
>             |                                       |
>             |                                       +-PMCSPAN2(2/3/0)
>             |                                       |          15,12,13,14
>             |                                       |
>             |                                       +-PMCSPAN3(2/4/0)
>             |                                       |          12,13,14,15
>             |                                       |
>             |                                       +-PMCSPAN4(2/5/0)
>             |                                                  13,14,15,12
>             +-GT64260BR(0/21/0)
>PCI.1(0/0/0)-+-PMCSLOT2(0/6/0)
>             |
>             +-PMCSLOT2 IDSEL B(0/7/0)
>             |
>             +-10/100/1000 Ether(0/10/0)  /* 1GHZ network */
>             |
>             +-GT64260BR(0/21/0)
>
>Till suggested :
>
>  
>
>>pci_read_config_byte(unchar bus, unchar dev, unchar func, unchar offset)
>>{
>>int peer;
>>if ( bus>= ucMaxPCIBus)
>>  return -1;
>>for (peer=0; peer_subordinates[peer] < bus; peer++)
>> bus -= peer_subordinates[peer];
>>return
>>BSP_pci_config_access[peer].read_config_byte(peer,bus,dev,func,offset);
>>}
>>    
>>
>
>Based on this proposal,  if the DEC21150BR is not presented :
>
>the ucMaxPCIBus should return 2.
>
No, 3, the two segments right off the discovery plus the segment behind 
the PCI6154

>The bus  number for the 1 GHZ network will be 1.
>
no, 2.

>
>Later on , a hot added DEC21150BR is needed, what bus number will it be ?
>
I don't think 'hot adding' is needed or in any way easy to do. AFAIK you 
rely on MOTLoad [or whatever it's called]
to perform the PCI configuration and the firmware is not going to 
support any 'hot adding' [hot == while system
is powered-on / running].

OTOH, if a DEC21150 is plugged into the [cold] system and powered up, 
MOTLoad will configure PCI
and you'll end up with 4 segments in total, the 1GHz ending up on segment 3.

If a PMC card with a local bridge is [cold] plugged into the 21150, 
after power-up, MOTLoad configures
PCI and there will now be 4 bus segments off 'hose A' [that's linux 
terminology corresponding to your PCI #]
and still 1 segment off 'hose B'. Total bus number should be 5, the 1GHz 
now ending up on bus # 4.

All you need to do is count the number of segments off hose A during 
init (using the generic pci_Initialize()
algorithm) and use that number to know whether to talk to hose A or B 
during a config space access
as outlined earlier.

>Based on the propsal, the 1GHZ network  should be  reconfigured to be 3.
>However, let's say the 1GHZ network is a device the application does not
>want to disturb during the hot change.   Thus  Gregory  suggested :
>
>  
>
>>Assign a range of bus numbers to each tree so the bridges can be
>>reconfigured as necessary.
>>    
>>
>
>OK.  I  decided that eight seems to be enough for my architecture to
>allow the reconfiguration of the PCI bridges.
>
>However, in your   previous E-mail, you wrote :
>
>  
>
>>That may be acceptable for one particular current hardware implementation, but is completely unacceptable for
>>general use.
>>Once you transition to PCI-X it is common to have one bus segment per connector so that you can run at the
>>maximum data rate (133M transfers/s, or 266M if you use the dual rate variant).  PCI Express systems will have
>>similar issues, because that architecture is inherently point to point and relies on switches.  The switches
>>create multiple bus segments and look to the software like multiple PCI-PCI bridges.
>>Some of the large servers I work on have 64 or more bus segments.  Those systems typically run a general purpose
>>OS rather than a real time OS, but it seems unnecessarily limiting to design an API into the RTOS or even into
>>the BSP which is known to be far below the architectural limitations of the bus.
>>    
>>
>
>Thus, you can tune your own value of the  MaxPCIbridgePerBandwidth
>in your bsp.h.
>
>
>How does that sound ?
>
Apart from my not understanding what MaxPCIbridgePerBandwidth should be 
needed for, good.

T.

>
>
>Thanks to all  including Peter Siddons,
>Kate
>
>
>  
>