Formatting DOSFS volume
Bogdan Vacaliuc
bvacaliuc at ngit.com
Mon Oct 25 16:51:24 UTC 2004
Hi Etienne,
I'm sorry if there was any misunderstanding.
I wasn't planning to add any code in the RTEMS CVS anytime soon (I am working locally with 4.6.1 + patch by Jay for CSB350). In my
last note, I realized that updating RDCF2 (the original author's source) for FAT16+FORMAT was going to take more time than I
anticpated. I had considered that step to be a prerequisite before any RTEMS work with the code from RDCF2.
In terms of the code for RDCF2, I announced a compilable, tested implementation here:
http://www.rtems.com/ml/rtems-users/2004/october/msg00139.html
I had hoped to provide an update to bring this up-to-date with FAT16 and the format code, but had to stop. So I sent the format
code snippets to you so you could see what would need to be done and compare with mkdosfs, etc.
So I think you have my code, right? But you have to understand a little about RDCF2, I think.
I would be happy to review and test the code you implement for RTEMS, I just don't think it would be fair to the group if I tried to
lead the development on this right now.
Thanks,
-bogdan
On Monday, October 25, 2004 11:48 AM, Etienne Fortin wrote:
> Hi Bogdan,
> Where are you in the implementation of formating dosfs for RTEMS? You
> are talking in your last email about including the code you already
> have. But if it isn't "RTEMS ready" yet, I guess you didn't put it in
> the RTEMS CVS? I don't see it in the CVS tree anyway.
>
> If I can get your code, I'll look at it and, as you said, will be
> able to extrapolate with the mkdosfs code. I know it's a quite
> involving implementation, but I need it anyway so I must do it :)
>
> Etienne Fortin
> Sensio
>
>
>
> -----Message d'origine-----
> De : Bogdan Vacaliuc [mailto:bvacaliuc at ngit.com]
> Envoyé : 18 octobre, 2004 23:39
> À : 'Etienne Fortin'
> Cc : rtems-users at rtems.com; 'Eugeny S. Mints';
> joel.sherrill at OARcorp.com Objet : RE: RE : RE : Formatting DOSFS
> volume
>
>
> Etienne,
>
> Well, in merging over the FAT16 and format updates from my codebase,
> I find it to be a rather painful and time consuming merge. The FAT16
> is done now, but not the format. There really were significant
> architectural changes vs. Philip's RDCF2 (especially in the
> cache/device
> interface) that would take a lot more time to merge. Without them,
> the bad sector operations in the format code are rather meaningless.
>
> I'm in danger of losing sight of the goal of all this: Implement
> format for RTEMS.
>
> With that in mind, I'm including the code for the two functions
> necessary for implementing format from my codebase. This is based on
> the RDCF2-ism of computing a "format table" which is passed-in to
> "format". You can extrapolate this to the mkdosfs-like
> implementation that you may be implementing.
>
> I looked briefly at the current Debian source of mkdosfs (from
> dosfstools-2.8). It's a pretty big, involved implementation.
>
> I see the following:
>
> 1) it only does a 'read' test to see if a block is usable.
> 2) it doesn't deal with bad blocks before the data area:
>
> if (currently_testing < start_data_block)
> die ("bad blocks before data-area: cannot make fs");
>
> I should think that a 'write' test would be more conclusive (I was
> doing a write/read/compare, but it takes forever. In the current
> version, I just do erase_cluster() which calls the device driver's
> IOCTL that performs an ERASE_SECTOR command. This is faster than
> write/read/compare.)
>
> Also, it is not a-typical for NAND flash parts to fail in blocks
> below the start of data, since those blocks are accessed with much
> greater frequency than DATA section. In the project that this code
> comes from, we have found it useful to have this capability.
>
>
> So I hope this can provide you with some inspiration as you implement
> the format code. Let me know if there is anything I can help with.
>
> Best Regards,
>
> -bogdan
>
>
> --- Code snippets follow ---
>
> // compute BPB format info based on partition size and bytes per
> sector. // *fmt points to a buffer that will be filled with
> contents of the // FAT BPB starting at BPB_BytsPerSec, which is
> at offset 11 in the BPB. // This code is from the M$ document,
> with the FAT32 stuff taken out static int
> rdcf_compute_format_table(struct rdcf_format *fmt, uint32
> DskSize, uint32 sector_bytes ) { uint32 spc; uint8 BPB_SecPerClus;
> // computed below uint32 RootDirSectors; // working
> variable uint32 TmpVal1, TmpVal2, FATSz; // working variables
> uint16 BPB_FATSz16; // computed below
>
> // hardcoded assumptions to make M$ calculation work
> #define BPB_BytsPerSec 512
> #define BPB_ResvdSecCnt 1
> #define BPB_NumFATs (uint32)2
> #define BPB_RootEntCnt 512
>
> #define LOWORD(A) (uint16)(A & 0xFFFF)
> #define HIWORD(A) (uint16)((A >> 16) & 0xFFFF)
>
> // BPB_SecPerClus from the FAT16 table above. (valid is 2^[0..7],
> BPS * SPC <= 32K) // Note: we dont have to worry about spc not
> being valid, because the last // entry in the table has a value
> for DiskSize that will guarantee a break. for(spc=0;
> spc<(sizeof(DskTableFAT16)/sizeof(struct
> DSKSZTOSECPERCLUS)); spc++)
> {
> if( DskSize <= DskTableFAT16[spc].DiskSize ) break;
> }
> BPB_SecPerClus = DskTableFAT16[spc].SecPerClusVal;
>
> // verify parameters
> if( BPB_BytsPerSec != _2plog2n( sector_bytes ) )
> {
> LOG_NOTICE( "BPB_BytsPerSec != _2plog2n( sector_bytes )" );
> return 1;
> }
> if( (BPB_BytsPerSec * BPB_SecPerClus) > (32*1024) )
> {
> LOG_NOTICE( "(BPB_BytsPerSec * BPB_SecPerClus) > (32*1024)" );
> return 1;
> }
>
> // compute BPB_FATSz16 so as to generate a FAT16 filesystem
> // see pp. 13-14 in Microsoft "FAT: General Overview of On-Disk
> Format" //
> // NOTE: We are operating under the following assumptions:
> // 1) FAT16 is forced
> // 2) BPB_BytsPerSec = 512
> // 3) BPB_SecPerClus is computed according to the M$ table above.
> // 4) BPB_ResvdSecCnt = 1
> // 5) BPB_NumFATs = 2
> // 6) BPB_RootEntCnt = 512
>
> RootDirSectors = ((BPB_RootEntCnt * 32) + (BPB_BytsPerSec - 1)) /
> BPB_BytsPerSec; TmpVal1 = DskSize - (BPB_ResvdSecCnt +
> RootDirSectors); TmpVal2 = (256 * BPB_SecPerClus) + BPB_NumFATs;
> //if(FATType == FAT32) TmpVal2 = TmpVal2 / 2;
> FATSz = (TmpVal1 + (TmpVal2 - 1)) / TmpVal2;
> //if(FATType == FAT32)
> //{
> // BPB_FATSz16 = 0;
> // BPB_FATSz32 = FATSz;
> //} else
> //{
> BPB_FATSz16 = LOWORD(FATSz);
> /* there is no BPB_FATSz32 in a FAT16 BPB */
> //}
>
> //
> // Initialize the Boot Paramater Block Table
> //
>
> // BPB_BytsPerSec can be 512 (recommended), 1024, 2048, 4096
> // however, we use M$ tables below, which only work for 512 byte
> sectors. fmt->x[0] = BPB_BytsPerSec & 0xFF;
> fmt->x[1] = BPB_BytsPerSec >> 8;
>
> // BPB_SecPerClus from the FAT16 table above. (valid is 2^[0..7],
> BPS * SPC <= 32K) fmt->x[2] = BPB_SecPerClus;
>
> // BPB_ResvdSecCnt must be 1
> fmt->x[3] = BPB_ResvdSecCnt & 0xFF;
> fmt->x[4] = BPB_ResvdSecCnt >> 8;
>
> // BPB_NumFATs should be 2 (for flash can make 1, but check
> compatibility with extern OS) fmt->x[5] = BPB_NumFATs;
>
> // BPB_RootEntCnt must be 512 for FAT16; (FAT32:0, FAT16:512,
> FAT12:(RDIR*32)%BPS = 0) fmt->x[6] = BPB_RootEntCnt & 0xFF;
> fmt->x[7] = BPB_RootEntCnt >> 8;
>
> // BPB_TotSec16 or BPB_TotSec32 according to the total
> addressable LBNs on the device if( DskSize < 0x10000 )
> {
> fmt->x[8] = (uint8)(DskSize & 0xFF); // NOTE: M$ comes
> up
> with +1 vs. us on this...
> fmt->x[9] = (uint8)((DskSize >> 8) & 0xFF);
> memset( &fmt->x[21], 0, 4 );
> }
> else
> {
> memset( &fmt->x[8], 0, 2 );
> fmt->x[21] = (uint8)(DskSize & 0xFF);
> fmt->x[22] = (uint8)((DskSize >> 8) & 0xFF);
> fmt->x[23] = (uint8)((DskSize >> 16) & 0xFF);
> fmt->x[24] = (uint8)((DskSize >> 24) & 0xFF);
> }
>
> // BPB_Media, 0xF0, 0xF8-0xFF are valid
> // NOTE: must match value in FAT[0]
> fmt->x[10] = 0xF0; // 0xF0 typically used for removable media
>
> // BPB_FATSz16 which is forced
> fmt->x[11] = (uint8)(BPB_FATSz16 & 0xFF);
> fmt->x[12] = (uint8)((BPB_FATSz16 >> 8) & 0xFF);
>
> // number of sectors per track
> fmt->x[13] = 63; // per M$ empirical
> fmt->x[14] = 0;
>
> // number of sides
> fmt->x[15] = 255; // per M$ empirical
> fmt->x[16] = 0;
>
> // BPB_HiddSec; always 0 on unpartitioned media
> fmt->x[17] = 0;
> fmt->x[18] = 0;
> fmt->x[19] = 0;
> fmt->x[20] = 0;
>
> return 0;
> }
>
> int rdcf_format(struct rdcf *f,
> #ifdef RDCF_MULTIPLE_DRIVE
> char *spec,
> #endif
> struct rdcf_format *format)
> {
> static const char bootstrap_header[11] =
> {
> (char)235,(char)254,(char)144,'R','D','C','F','2','0',' ',' '
> };
> static const char bootstrap_trailer[26] = // for
> offset[36]
> {
> 0,0,0, //
> drv,resv1,bootsig
> 4,3,2,1, // 32bit S/N
> 'N','O',' ','N','A','M','E',' ',' ',' ',' ', // volume
> label 'F','A','T','1','6',' ',' ',' ' //
> filsystype };
> unsigned char *buffer = f->buffer;
> unsigned char media_descriptor;
> rdcf_sector_t sector, bad_sector_limit;
> rdcf_cluster_t cluster;
> rdcf_cluster_t bad_clusters;
>
> /* setup for failure exit */
> if ((f->result=setjmp(f->error)) != 0) return f->result;
> #ifdef RDCF_MULTIPLE_DRIVE
> if (*get_drive(f, spec) != 0) error_exit(f, RDCF_INVALID_SPEC);
> #endif
>
> /* verify that the BPB sector is viable [this is the first sector
> of the volume] */ test_sector(f, f->bpb_sector, 1);
>
> /* 03/23/04: compute limit to # of bad sectors we accept
> dynamically (by parsing format table) */
> bad_sector_limit = format->x[8]; // BPB_TotSec16...
> bad_sector_limit |= format->x[9]<<8; // ...
> if( bad_sector_limit == 0 )
> {
> bad_sector_limit = format->x[21]; // BPB_TotSec32...
> bad_sector_limit |= format->x[22]<<8; // ...
> bad_sector_limit |= format->x[23]<<16; // ...
> bad_sector_limit |= format->x[24]<<24; // ...
> }
> bad_sector_limit /= 100;
> bad_sector_limit *= BAD_SECTOR_RESERVE; // percentage, [1..100]
>
> /* 11/19/03: handle failures in the FAT and directory tables */
> if( (f->result=setjmp(f->error)) != 0 )
> {
> unsigned int bad = f->drive_error;
>
> /* Handle failures in test_sector() by updating the reserved
> sector count */ bad &= ~31; // align to 32 LBN
> boundary (presuming NAND
> driver)
> bad += 32; // increment to next block
>
> /* check against limits computed above */
> if( bad < bad_sector_limit )
> {
> format->x[3] = bad & 0xFF; // BPB_RsvdSecCnt...
> format->x[4] = (bad>>8) & 0xFF; // ...
> }
> else return f->result;
> }
>
> /* Initialize the BPB sector */
> memset(buffer, 0, SECTOR_SIZE);
> memcpy(buffer, bootstrap_header, 11); // always exactly 11
> bytes
> memcpy(buffer + 11, format, 25); // always exactly 25
> bytes
> memcpy(buffer + 36, bootstrap_trailer, 26); // at least 26 bytes
> buffer[510 /*SECTOR_SIZE-2*/] = 0x55; // per M$; the
> FSI_TrailSig
> buffer[511 /*SECTOR_SIZE-1*/] = 0xAA; // per M$; NOTE: NOT
> the
> end of the sector
> write_sector(f, 0, buffer);
> f->buffer_status = CLEAN;
> f->sector_in_buffer = 0;
>
> /* (re)load filesystem info based on current BPB data */
> read_file_system_information(f);
> media_descriptor = MEDIA_DESCRIPTOR;
>
> /* Initialize FAT tables (FAT[1..N]) */
> memset(buffer, 0, SECTOR_SIZE);
> sector = f->first_FAT_sector;
> while (sector < f->first_directory_sector)
> {
> rdcf_sector_t count = f->sectors_per_FAT;
> test_sector(f, sector, 1); // NOTE: uses
> error_exit()/longjmp()
> buffer[0] = media_descriptor;
> //buffer[1] = buffer[2] = 0xFF; // NOTE: This gave
> 0xFFFFMD, wrong
> buffer[1] = 0xFF; // per M$; only 0xFFMD
> is required
> write_sector(f, sector++, buffer);
> buffer[0] = buffer[1] = buffer[2] = 0;
> while (--count != 0)
> write_sector(f, sector++, buffer);
> }
>
> /* Initialize the ROOT Directory table */
> /* TODO: see about handling the ClnShutBitMask and HrdErrBitMask
> */ buffer[0] = buffer[1] = buffer[2] = 0;
> while (sector < f->first_data_sector)
> {
> test_sector(f, sector, 1); // NOTE: uses
> error_exit()/longjmp()
> write_sector(f, sector++, buffer);
> }
>
> /* Initialize the DATA area */
> bad_clusters = 0;
> if( ENABLE_TEST_DATA_CLUSTERS )
> {
> //
> // Perform a cluster check of all data clusters and populate
> the FAT // tables with the bad cluster information.
> //
> for ( cluster = 2; cluster <= f->maximum_cluster_number;
> cluster++ )
> {
> #ifndef NDEBUG
> if( !(cluster&0x7F) ) // 1:128
> {
> printf("\rrdcf_format(): verifying %d/%d...\r", cluster,
> f->maximum_cluster_number);
> }
> #endif // NDEBUG
> if( test_cluster( f, cluster ) )
> {
> set_FAT_entry ( f, cluster, BAD_CLUSTER );
> bad_clusters++;
> }
> else if( ENABLE_ERASE_DATA_CLUSTERS )
> {
> (void)erase_cluster( f, cluster );
> }
> }
> #ifndef NDEBUG
> printf("\rrdcf_format(): verified %d/%d \n", cluster-1,
> f->maximum_cluster_number);
> #endif // NDEBUG
> }
> else if( ENABLE_ERASE_DATA_CLUSTERS )
> {
> for ( cluster = 2; cluster <= f->maximum_cluster_number;
> cluster++ )
> {
> #ifndef NDEBUG
> if( !(cluster&0x7F) ) // 1:128
> {
> printf("\rrdcf_format(): erasing %d/%d...\r", cluster,
> f->maximum_cluster_number);
> }
> #endif // NDEBUG
> if( erase_cluster( f, cluster ) )
> {
> set_FAT_entry ( f, cluster, BAD_CLUSTER );
> bad_clusters++;
> }
> }
> #ifndef NDEBUG
> printf("\rrdcf_format(): erased %d/%d \n", cluster-1,
> f->maximum_cluster_number);
> #endif // NDEBUG
> }
> else
> {
> cluster = f->maximum_cluster_number;
> }
>
> /* Cleanup and Exit */
>
> // NOTE: test_cluster() has called error_exit() to trap low level
> I/O // errors during its work. It would be invalid to do any
> more work // that could possibly throw another exception. If you
> needed to // for some reason, you would have to call setjmp()
> again: //
> //if ((f->result=setjmp(f->error)) != 0) return f->result;
>
> return 0;
> }
>
>
>
>
> On Thursday, October 14, 2004 11:24 AM, Bogdan Vacaliuc wrote:
>
>> Hi Etienne,
>>
>> Thanks. I won't use it right now; let me concentrate on the patch
>> using the working/tested code. I have to be careful to strip any
>> proprietary modifications, etc, before releasing back to the public
>> domain.
>>
>> Afterwards, I will take a look.
>>
>> -bogdan
>>
>>
>> On Thursday, October 14, 2004 11:19 AM, Etienne Fortin wrote:
>>
>>> I have the source code of mkdosfs for I don't remember what OS. Its
>>> old, like years old, but it may be usefull for inspiration. Do you
>>> want it?
>>>
>>> Etienne Fortin
>>> Sensio
More information about the users
mailing list