PowerPC Burst PCI access

[Peter Dufault]

On Apr 7, 2005, at 9:39 PM, Till Straumann wrote:

> Peter Dufault wrote:
>
>> These comments relate to the MVME5500 VME powerPC port and a PCI data 
>> acquisition card.
>>
>> In my control application I noticed a while ago that reading back the 
>> data from the PCI data acquisition board is extremely slow.  The 
>> board (acromag PMC341) collects the data into a pre-fetchable memory 
>> buffer that it says will transfer data at 40 MByte/s using burst 
>> reads.  I'm seeing about 4 MByte/s using naive reads and I eventually 
>> need to solve this problem.
>
> That seems awfully slow. However, I can confirm that this matches what 
> I observe when I do PIO
> to my PMC card mentioned below (I can only do 16-bit PIO, though).

I'm toggling a TTL line around the copy and looking at a scope so that 
is the rate.  The time doesn't change when I go from 32 to 64 bit 
transfers but that is to be expected, the pmc341 only does 32 bit 
transfers.  I'm pretty sure it did change when I went to the 128 bit 
transfers but then I got the FP errors.  I didn't look long, though, my 
program went into emergency stop which took my attention away from the 
scope, so maybe I imagined it was faster.

> When I use DMA the throughput is significantly better. I observe 
> latencies of 25us avg. and 45us worst case
> for transferring 512 bytes [latency measures setting-up DMA, actual 
> data xfer, dispatching of 'dma done' ISR].
>
>> With a 20KHz control loop I'm losing about 1/4 of the CPU using naive 
>> reads.
>
> So the memory is on the PMC card - it's not that the PMC can DMA data 
> over to memory on the MVME5500,
> right?

Correct.

>
>>
>>
>> As I understand it, if the memory region that the PowerPC fetches 
>> from is cacheable then the PowerPC will attempt to burst cache-line 
>> size reads.   I think the mapping from PowerPC to DAQ card is first 
>> through the PowerPC BAT registers and then a PCI window.
>
> Hmm - I suspect the PCI memory is mapped non-cacheable. Don't know if 
> it's safe to make it cachable (does the PCI bus + the MVME5500's PCI 
> bridge implement coherency?). You have to be careful to avoid having 
> stale data in the cache.

The bridge is the GT64260 bridge, the same one that the system ethernet 
is through, so I assume it does.
Then I think it is through a BAT to the system.  I'll have to double 
check.

>
> Anyways, probably the best solution would be using DMA (don't know if 
> the MVME5500 has a DMA controller).

It might, but if it does it's on the NDA GT64260.
>
> Otherwise, you could try to use e.g., a page table entry (or a spare 
> BAT) to map that special region cacheable
> and then use 'dcbi' prior to each new transfer.

That's what I was thinking.  I have spare BATs since there are 8 on the 
7450.

>
> Sidenote: look at the PMC16AI64SS by generalstandards.com -- 64 x 
> 16bit @ 100kHz on a PMC module
> with a fifo + DMA controller. A really nice card and less $/channel, 
> it seems.
>
>>
>>
>> Is my understanding about right?  Is there some code I should look at 
>> as an example?
>>
>> I also tried fetching the memory as a "long double" but I started 
>> getting exceptions in my tasks. I assume my ISR started using 
>> floating point resources during the copy which caused this, so I also 
>> need a pointer to a method of doing cache-line size memory fetches 
>> without affecting the floating point state.
>
> Bad - you *must not* use the FPU from an ISR. Results in FPreg 
> corruption - I just filed a PR today
> regarding this...

As I said, I wasn't sure the fetches would use the FPU.  I already keep 
my ISR in a separate file to generate errors by undefining sem_wait, 
printf, etc at compile time, it would be nice to have a "generate error 
on FPU access" compiler switch instead of a "soft FPU" compiler switch.
>
>
>
Peter Dufault
HD Associates, Inc.