RFC: Bdbuf transfer error handling

Thomas Doerfler Thomas.Doerfler at embedded-brains.de
Sat Nov 21 09:11:44 UTC 2009


Chris,

Chris Johns wrote:
> Sebastian Huber wrote:
>>
>> R3. Read Ahead Request and No User
>>
>> We discard the buffer.  This is the current approach.
>>
> 
> As you know I wish to move the read ahead logic out of the cache into
> the file systems. I propose to change the API to have a
> rtems_chain_control passed in for gets and reads and the buffers
> returned linked the chain. This means a file system can determine the
> number of buffers it wants and the cache will attempt to do this. If it
> cannot it does what it can which could be 0 buffers returned because of
> read errors. It is up to the file system to manage this, typically with
> a EIO. Note, a resource issue in the cache would block the requester
> until it can be completed. The way to return ENXIO when the device is
> not available is something I need to figure out.

I fear that this would make the filesystems code more complicated,
because then they are responsible for keeping track which read-ahead
buffers they have requested.

Example:

- You open a bg file and read the first 1024 bytes.
- the filesystem will request to read ahead from bdbuf.
- therefore, the bdbuf read call will only return to the FS code, when
ALL sectors are available.
- then you process the 1024 bytes and it takes a VERY long time to do so
(e.g. because you transfer them over a slow network connection or do
complicated math, or send them to a slow output device or....)
-> since these read-ahead blocks are requested and occupied from the
file system, these block are not available for other caching.

In this scenario, a lot of buffer space gets eaten up.

If you change this scenario slightly, because the read data is processed
quickly, you get a performance gain since the read-ahead requires less
transactions between bdbuf and the storage hardware.

So my point is that it would make sense to have the "read-ahead" buffers
marked as "reusable".
----------------

I agree that only the file system knows, if and how much read ahead
really makes sense. OTOT, the switch from sector based bdbuf to block
(cluster?) based bdbuf also reads bigger chunks, which partially solves
the read-ahead requriements.

Would it make sense that the filesystems code simply passes an
additional "hint" parameter to each read/get call and the bdbuf layer is
again responsible to do the read-ahead (or not) and to keep track of the
available buffers?

I have already discussed with Sebastian, I think it would make sense to
define some usage scenarios for the file systems/bdbuf/blockdev area, so
we can discuss the pros and cons of the different architectures from a
common basis.

And before you get the wrong impression: I really appreciate the great
improvement you are doing in that area, it is just that we have
different use cases in mind when doing certain design decisions and
therefore we tend to different paths.

With kind regards,

Thomas.


-- 

--------------------------------------------
embedded brains GmbH
Thomas Doerfler        Obere Lagerstrasse 30
D-82178 Puchheim       Germany
email: Thomas.Doerfler at embedded-brains.de
Phone: +49-89-18908079-2
Fax:   +49-89-18908079-9
PGP:   Public key available on request.

Diese Nachricht ist keine geschäftliche Mitteilung im Sinne des EHUG.



More information about the users mailing list