NFS Performance

[Steven Johnson]

Joel, Thanks for the pointer. setvbuf worked like a treat.  So, we don't
have to hack stdio.h, thanks for the suggestion.

Following are some benchmarks we got with different block sizes.  MPC862
Processor (PowerPC) at 100Mhz, with 50Mhz SDRAM main memory and using
the FEC (Fast Ethernet Controller @ 100Mbit/s, full duplex), its a
custom design and doesn't use a standard BSP, but isn't dissimilar to
the standard MPC862 BSP:

Times to transfer the 7.4M test file timed using ticks_since_boot, (1
tick is 10ms on our target):
Buffer Size @ Ticks to transfer = ~Mbit/s

256 @ 4265  = ~1.4Mbit/s
512 @ 2221  = ~2.7Mbit/s
1024 @ 1239 = ~4.8Mbit/s
2048 @ 712 = ~8.4Mbit/s
4096 @ 439 = ~13.7Mbit/s
8192 @ 298 = ~20.2Mbit/s
16384 @ 299 =~20.2Mbit/s
32768 @ 298 =~20.2Mbit/s

So, as Till said:

> The UDP packet size limit imposes a
> limit of 8k per RPC call

And this is demonstrated by the above figures.

I understand we may be able to get it faster with some sort of
multi-threaded reader, but for our purposes ~20Mbit/s is more than fast
enough.  Obviously the mileage will vary on different platforms, but
it's worth noting and trying if you are using NFS and need a bit more
speed. 

Not that I've timed it, but I would expect the latency of bigger buffers
would be greater than the small one (it physically takes longer to
receive 8K than 1K) so the trade off between buffer size and average
latency would be different from application to application. 

I also don't think this needs to be incorporated into the NFS code
anywhere, but it's worth noting in the README, and maybe a Wiki page on
it wouldn't hurt.  I'll put Till's comments on getting further
performance, details on setvbuf and my results in the Wiki for posterity
in the next few days.

Steven J



Joel Sherrill wrote:

> Steven Johnson wrote:
>
>> Hi,
>>
>> We are using the NFS Client stack as found in the RTEMS CVS.  We
>> experienced the slow performance that the readme talks about.  We were
>> getting around 3.5Mbit/s on a 100MBit link.  Our Processor is a 100Mhz
>> MPC862, and we are using the FEC.
>>
>> This isn't fast enough for us, as we need around 6-7 Mbit/s so we can
>> read media files and play them in real time from the NFS Share.
>>
>> So we investigated the causes.
>>
>> The single biggest thing we found is BUFSIZ in stdio.h.
>>
>> It was defined as 1024, which gave us ~3.5Mbit/s
>> We Changed it to 8192, and NFS Performance on our board jumped to
>> ~19.53MBit/s
>> We also tried 32768, which gave us ~19.99MBit/s
>>
>> (This is all Read performance, we don't know how fast write is, because
>> we don't write with our application).
>>
>> Also, the readme for the NFS Client indicates performance sucks due to a
>> lack of read ahead buffering.  Changing BUFSIZ to 8192 seems to provide
>> read ahead buffering, because if we read 512 bytes, 8192 bytes are
>> transferred from the file, and another request for 8192 bytes only
>> occurs when we have exhausted that data.
>>
>> Anyway, I'm just reporting this so that anyone else who needs NFS to
>> perform better can know what we did to achieve dramatic speed increases.
>>
>>   
>
> Interesting.  Since hacking stdio.h is generally not a good thing. 
> Can you try using
> one of the setbuf methods on the file descriptor and seeing if that
> accomplishes the
> same thing.
>
> If it does, then perhaps an open on a NFS client should implicitly
> increase the buffer
> size.
>
> This would limit the impact of the change.
>
> 20 Mbit/s is pretty nice performance from any computer but a 100 Mhz
> PowerPC
> makes it impressive.
>
> --joel
>
>> Steven J
>>   
>
>
>