Writing to absolute sectors on SD Card

Robert S. Grimes rsg at alum.mit.edu
Wed Oct 29 12:28:35 UTC 2008


Sorry for the delay - I was off-line for a few days...  See below for 
comments.

Chris Johns wrote:
> Robert S. Grimes wrote:
>   
> If you write less than a block size the file system will have to read the 
> block to update the data in it. There is not concept of the block having never 
> been used before. If you write as single bytes quickly there should still be 
> only one block read and one block write. The remaining updates of the block 
> occur at the cache level.
>   
I think I understand you here, and it explains the performance I've been 
seeing.  I'm just a bit surprised that the driver doesn't buffer small 
writes; if it did, my test program would likely not have shown the poor 
performance for the 256 byte writes.  See, my test simply calls write() 
in a loop with no delays - arguably, not very realistic, but rather, 
just a first step.  Anyway, what is happening is that the block is being 
read and written twice for the 256 byte case, and once for all the 
others (which are all multiples of full blocks).

When I say "a bit surprised", it's because I don't really understand 
_exactly_ what the various parameters specify, and how exactly they 
affect the block driver's performance.  Here are my settings:

    #define CONFIGURE_BDBUF_BUFFER_COUNT            1024
    #define CONFIGURE_BDBUF_MAX_READ_AHEAD_BLOCKS   64
    #define CONFIGURE_BDBUF_MAX_WRITE_BLOCKS        32
    #define CONFIGURE_SWAPOUT_SWAP_PERIOD           250
    #define CONFIGURE_SWAPOUT_BLOCK_HOLD            1000


I should stress that, for my application, I can readily ensure that all 
writes are block-sized, so this is not terribly critical to me.  Of 
course, that assumes my assumption that files all start on a block 
boundary - that is true, isn't it?  It certainly seems so...

>   
>> Then, I decided to see if my write() calls were causing the problem back 
>> on the MS-DOS test program.  My original test was writing 256 bytes 
>> (half an SD card sector, by the way); I was assuming the driver would 
>> attempt to coalesce writes to sector writes.
>>     
>
> It should do this but it needs to read the block first because it does not 
> know if the 2nd half of the block will be written.
>
> The cache will read ahead a configured number of blocks so you can use the 
> multi-block reading feature of some devices. This means you should see a burst 
> of blocks read then a number of writes.
>   
Makes sense to me - understood.

>> To test this assumption, I 
>> changed my write() sizes to 512, 1024, 2048, 4096, and 8192 bytes.  
>> Interestingly, the write throughput for size 512 writes doubled (~40 
>> kbytes/sec), but so did the other larger sizes.
>>     
>
> What if you write 768 bytes ?
>   
I would guess it will be like the 256 byte writes, right?  I can't test 
that right now, but eventually...
>   
>>  I didn't look "under 
>> the hood" at this point, because I was running out of time, and I could 
>> easily use larger write calls anyway.  But someone else may be interested.
>>     
>
> It would be nice to know if all the timing figures add up and the code is 
> working as it should.
>   
Well, it (the observed performance) seems to make sense...

>   
>> Finally, I noticed a blurb in the SanDisk manual that said writing to a 
>> sector that has already been erased was significantly faster, as the 
>> erase that is normally done during a "normal write" is skipped.  This 
>> triggered my memory - isn't it true that writing is much faster than 
>> erasing for most flash?
>>     
>
> Where can I find this manual ?
>   
Couldn't find a link or where I got it from, so I'm sending it to you 
separately.

>   
>> So, is there a way to pre-erase sectors? Simple enough on the 
>> low-level, but how could that fit into the picture when MS-DOS is involved?
>>     
>
> I do not think it does. The flash file system has the same problem. Let me try 
> and explain with another question. How do you know a block is not being used 
> any more by the file system and could be erased ? Below the file system at the 
> cache and lower you do not. The file system may toggle a bit somewhere and 
> perform a write but it does not erase the block. The idea of erasing a block 
> on a real disk means nothing. There is no point. In the flash driver the block 
> sits with data until it is written again and at that point you know the 
> current block is not being used.
>
> My point which I suspect I am not making in a clear way is pre-erasing only 
> helps once and after that it does not. The way the disk is used at the block 
> level become complex and depends on the file and directories created and erased.
>   
Yes, this is all clear; probably I'm the one not being clear.  What I'm 
envisioning is specific to my application, but could well be used by 
others. The SD disk is primarily a data acquisition storage medium, and 
the typical scenario is to start with an empty disk, which gets filled 
over time.  At some point, the disk is removed and the data processed 
off-line; a replacement disk takes it place in the target.  Therefore, 
if the disk itself takes advantage of pre-erased sectors, then I can 
easily "arrange" for better write performance.  That said, this 
obviously won't help more general usage patterns, as you have noted; 
also clear is that it's not likely that the software could _reasonably_ 
implement any special "write-no-erase" functionality.  So it seems we're 
done wrt RTEMS; if I get an answer on whether the card can use 
pre-erased sectors to increase performance, I'll let you know.

>   
>> One thing I didn't see was how the SD card distinguishes a "normal 
>> write" from one that doesn't require a pre-format.  If this is handled 
>> by the card itself, then whatever we do in software will automatically 
>> see the speed-up if the sectors are pre-erased, but still work correctly 
>> if the erase cycle is needed.  If not, then things become much more 
>> difficult on the software side, I suppose.
>>
>> If the card recognizes erased sectors, then we could simply prepare the 
>> disk at some user-selected point by erasing all sectors, then formatting 
>> the disk.  This would likely be acceptable for a good number of cases, 
>> as it would be for me.
>>     
>
> The file system, cache and drivers currently have no way to pass the info a 
> block is not in use any more and could be queued to be erased. Maybe it should.
>   
Or maybe not.  Some measurements of increased write performance would 
need to be made to assess whether the speed-up is worth the effort, and 
of course, increased complexity.  I am certainly interested, but I can't 
justify asking for it!

> I hope this helps.
>   
Yes, it does!  Thanks!
-Bob

> Regards
> Chris
>
>
>   




More information about the users mailing list