Formatting DOSFS volume
Bogdan Vacaliuc
bvacaliuc at ngit.com
Tue Oct 19 03:38:58 UTC 2004
Etienne,
Well, in merging over the FAT16 and format updates from my codebase, I find it to be a rather painful and time consuming merge. The
FAT16 is done now, but not the format. There really were significant architectural changes vs. Philip's RDCF2 (especially in the
cache/device interface) that would take a lot more time to merge. Without them, the bad sector operations in the format code are
rather meaningless.
I'm in danger of losing sight of the goal of all this: Implement format for RTEMS.
With that in mind, I'm including the code for the two functions necessary for implementing format from my codebase. This is based
on the RDCF2-ism of computing a "format table" which is passed-in to "format". You can extrapolate this to the mkdosfs-like
implementation that you may be implementing.
I looked briefly at the current Debian source of mkdosfs (from dosfstools-2.8). It's a pretty big, involved implementation.
I see the following:
1) it only does a 'read' test to see if a block is usable.
2) it doesn't deal with bad blocks before the data area:
if (currently_testing < start_data_block)
die ("bad blocks before data-area: cannot make fs");
I should think that a 'write' test would be more conclusive (I was doing a write/read/compare, but it takes forever. In the current
version, I just do erase_cluster() which calls the device driver's IOCTL that performs an ERASE_SECTOR command. This is faster than
write/read/compare.)
Also, it is not a-typical for NAND flash parts to fail in blocks below the start of data, since those blocks are accessed with much
greater frequency than DATA section. In the project that this code comes from, we have found it useful to have this capability.
So I hope this can provide you with some inspiration as you implement the format code. Let me know if there is anything I can help
with.
Best Regards,
-bogdan
--- Code snippets follow ---
// compute BPB format info based on partition size and bytes per sector.
// *fmt points to a buffer that will be filled with contents of the
// FAT BPB starting at BPB_BytsPerSec, which is at offset 11 in the BPB.
// This code is from the M$ document, with the FAT32 stuff taken out
static int rdcf_compute_format_table(struct rdcf_format *fmt, uint32 DskSize, uint32 sector_bytes )
{
uint32 spc;
uint8 BPB_SecPerClus; // computed below
uint32 RootDirSectors; // working variable
uint32 TmpVal1, TmpVal2, FATSz; // working variables
uint16 BPB_FATSz16; // computed below
// hardcoded assumptions to make M$ calculation work
#define BPB_BytsPerSec 512
#define BPB_ResvdSecCnt 1
#define BPB_NumFATs (uint32)2
#define BPB_RootEntCnt 512
#define LOWORD(A) (uint16)(A & 0xFFFF)
#define HIWORD(A) (uint16)((A >> 16) & 0xFFFF)
// BPB_SecPerClus from the FAT16 table above. (valid is 2^[0..7], BPS * SPC <= 32K)
// Note: we dont have to worry about spc not being valid, because the last
// entry in the table has a value for DiskSize that will guarantee a break.
for(spc=0; spc<(sizeof(DskTableFAT16)/sizeof(struct DSKSZTOSECPERCLUS)); spc++)
{
if( DskSize <= DskTableFAT16[spc].DiskSize ) break;
}
BPB_SecPerClus = DskTableFAT16[spc].SecPerClusVal;
// verify parameters
if( BPB_BytsPerSec != _2plog2n( sector_bytes ) )
{
LOG_NOTICE( "BPB_BytsPerSec != _2plog2n( sector_bytes )" );
return 1;
}
if( (BPB_BytsPerSec * BPB_SecPerClus) > (32*1024) )
{
LOG_NOTICE( "(BPB_BytsPerSec * BPB_SecPerClus) > (32*1024)" );
return 1;
}
// compute BPB_FATSz16 so as to generate a FAT16 filesystem
// see pp. 13-14 in Microsoft "FAT: General Overview of On-Disk Format"
//
// NOTE: We are operating under the following assumptions:
// 1) FAT16 is forced
// 2) BPB_BytsPerSec = 512
// 3) BPB_SecPerClus is computed according to the M$ table above.
// 4) BPB_ResvdSecCnt = 1
// 5) BPB_NumFATs = 2
// 6) BPB_RootEntCnt = 512
RootDirSectors = ((BPB_RootEntCnt * 32) + (BPB_BytsPerSec - 1)) / BPB_BytsPerSec;
TmpVal1 = DskSize - (BPB_ResvdSecCnt + RootDirSectors);
TmpVal2 = (256 * BPB_SecPerClus) + BPB_NumFATs;
//if(FATType == FAT32) TmpVal2 = TmpVal2 / 2;
FATSz = (TmpVal1 + (TmpVal2 - 1)) / TmpVal2;
//if(FATType == FAT32)
//{
// BPB_FATSz16 = 0;
// BPB_FATSz32 = FATSz;
//} else
//{
BPB_FATSz16 = LOWORD(FATSz);
/* there is no BPB_FATSz32 in a FAT16 BPB */
//}
//
// Initialize the Boot Paramater Block Table
//
// BPB_BytsPerSec can be 512 (recommended), 1024, 2048, 4096
// however, we use M$ tables below, which only work for 512 byte sectors.
fmt->x[0] = BPB_BytsPerSec & 0xFF;
fmt->x[1] = BPB_BytsPerSec >> 8;
// BPB_SecPerClus from the FAT16 table above. (valid is 2^[0..7], BPS * SPC <= 32K)
fmt->x[2] = BPB_SecPerClus;
// BPB_ResvdSecCnt must be 1
fmt->x[3] = BPB_ResvdSecCnt & 0xFF;
fmt->x[4] = BPB_ResvdSecCnt >> 8;
// BPB_NumFATs should be 2 (for flash can make 1, but check compatibility with extern OS)
fmt->x[5] = BPB_NumFATs;
// BPB_RootEntCnt must be 512 for FAT16; (FAT32:0, FAT16:512, FAT12:(RDIR*32)%BPS = 0)
fmt->x[6] = BPB_RootEntCnt & 0xFF;
fmt->x[7] = BPB_RootEntCnt >> 8;
// BPB_TotSec16 or BPB_TotSec32 according to the total addressable LBNs on the device
if( DskSize < 0x10000 )
{
fmt->x[8] = (uint8)(DskSize & 0xFF); // NOTE: M$ comes up with +1 vs. us on this...
fmt->x[9] = (uint8)((DskSize >> 8) & 0xFF);
memset( &fmt->x[21], 0, 4 );
}
else
{
memset( &fmt->x[8], 0, 2 );
fmt->x[21] = (uint8)(DskSize & 0xFF);
fmt->x[22] = (uint8)((DskSize >> 8) & 0xFF);
fmt->x[23] = (uint8)((DskSize >> 16) & 0xFF);
fmt->x[24] = (uint8)((DskSize >> 24) & 0xFF);
}
// BPB_Media, 0xF0, 0xF8-0xFF are valid
// NOTE: must match value in FAT[0]
fmt->x[10] = 0xF0; // 0xF0 typically used for removable media
// BPB_FATSz16 which is forced
fmt->x[11] = (uint8)(BPB_FATSz16 & 0xFF);
fmt->x[12] = (uint8)((BPB_FATSz16 >> 8) & 0xFF);
// number of sectors per track
fmt->x[13] = 63; // per M$ empirical
fmt->x[14] = 0;
// number of sides
fmt->x[15] = 255; // per M$ empirical
fmt->x[16] = 0;
// BPB_HiddSec; always 0 on unpartitioned media
fmt->x[17] = 0;
fmt->x[18] = 0;
fmt->x[19] = 0;
fmt->x[20] = 0;
return 0;
}
int rdcf_format(struct rdcf *f,
#ifdef RDCF_MULTIPLE_DRIVE
char *spec,
#endif
struct rdcf_format *format)
{
static const char bootstrap_header[11] =
{
(char)235,(char)254,(char)144,'R','D','C','F','2','0',' ',' '
};
static const char bootstrap_trailer[26] = // for offset[36]
{
0,0,0, // drv,resv1,bootsig
4,3,2,1, // 32bit S/N
'N','O',' ','N','A','M','E',' ',' ',' ',' ', // volume label
'F','A','T','1','6',' ',' ',' ' // filsystype
};
unsigned char *buffer = f->buffer;
unsigned char media_descriptor;
rdcf_sector_t sector, bad_sector_limit;
rdcf_cluster_t cluster;
rdcf_cluster_t bad_clusters;
/* setup for failure exit */
if ((f->result=setjmp(f->error)) != 0) return f->result;
#ifdef RDCF_MULTIPLE_DRIVE
if (*get_drive(f, spec) != 0) error_exit(f, RDCF_INVALID_SPEC);
#endif
/* verify that the BPB sector is viable [this is the first sector of the volume] */
test_sector(f, f->bpb_sector, 1);
/* 03/23/04: compute limit to # of bad sectors we accept dynamically
(by parsing format table) */
bad_sector_limit = format->x[8]; // BPB_TotSec16...
bad_sector_limit |= format->x[9]<<8; // ...
if( bad_sector_limit == 0 )
{
bad_sector_limit = format->x[21]; // BPB_TotSec32...
bad_sector_limit |= format->x[22]<<8; // ...
bad_sector_limit |= format->x[23]<<16; // ...
bad_sector_limit |= format->x[24]<<24; // ...
}
bad_sector_limit /= 100;
bad_sector_limit *= BAD_SECTOR_RESERVE; // percentage, [1..100]
/* 11/19/03: handle failures in the FAT and directory tables */
if( (f->result=setjmp(f->error)) != 0 )
{
unsigned int bad = f->drive_error;
/* Handle failures in test_sector() by updating the reserved sector count */
bad &= ~31; // align to 32 LBN boundary (presuming NAND driver)
bad += 32; // increment to next block
/* check against limits computed above */
if( bad < bad_sector_limit )
{
format->x[3] = bad & 0xFF; // BPB_RsvdSecCnt...
format->x[4] = (bad>>8) & 0xFF; // ...
}
else return f->result;
}
/* Initialize the BPB sector */
memset(buffer, 0, SECTOR_SIZE);
memcpy(buffer, bootstrap_header, 11); // always exactly 11 bytes
memcpy(buffer + 11, format, 25); // always exactly 25 bytes
memcpy(buffer + 36, bootstrap_trailer, 26); // at least 26 bytes
buffer[510 /*SECTOR_SIZE-2*/] = 0x55; // per M$; the FSI_TrailSig
buffer[511 /*SECTOR_SIZE-1*/] = 0xAA; // per M$; NOTE: NOT the end of the sector
write_sector(f, 0, buffer);
f->buffer_status = CLEAN;
f->sector_in_buffer = 0;
/* (re)load filesystem info based on current BPB data */
read_file_system_information(f);
media_descriptor = MEDIA_DESCRIPTOR;
/* Initialize FAT tables (FAT[1..N]) */
memset(buffer, 0, SECTOR_SIZE);
sector = f->first_FAT_sector;
while (sector < f->first_directory_sector)
{
rdcf_sector_t count = f->sectors_per_FAT;
test_sector(f, sector, 1); // NOTE: uses error_exit()/longjmp()
buffer[0] = media_descriptor;
//buffer[1] = buffer[2] = 0xFF; // NOTE: This gave 0xFFFFMD, wrong
buffer[1] = 0xFF; // per M$; only 0xFFMD is required
write_sector(f, sector++, buffer);
buffer[0] = buffer[1] = buffer[2] = 0;
while (--count != 0)
write_sector(f, sector++, buffer);
}
/* Initialize the ROOT Directory table */
/* TODO: see about handling the ClnShutBitMask and HrdErrBitMask */
buffer[0] = buffer[1] = buffer[2] = 0;
while (sector < f->first_data_sector)
{
test_sector(f, sector, 1); // NOTE: uses error_exit()/longjmp()
write_sector(f, sector++, buffer);
}
/* Initialize the DATA area */
bad_clusters = 0;
if( ENABLE_TEST_DATA_CLUSTERS )
{
//
// Perform a cluster check of all data clusters and populate the FAT
// tables with the bad cluster information.
//
for ( cluster = 2; cluster <= f->maximum_cluster_number; cluster++ )
{
#ifndef NDEBUG
if( !(cluster&0x7F) ) // 1:128
{
printf("\rrdcf_format(): verifying %d/%d...\r", cluster, f->maximum_cluster_number);
}
#endif // NDEBUG
if( test_cluster( f, cluster ) )
{
set_FAT_entry ( f, cluster, BAD_CLUSTER );
bad_clusters++;
}
else if( ENABLE_ERASE_DATA_CLUSTERS )
{
(void)erase_cluster( f, cluster );
}
}
#ifndef NDEBUG
printf("\rrdcf_format(): verified %d/%d \n", cluster-1, f->maximum_cluster_number);
#endif // NDEBUG
}
else if( ENABLE_ERASE_DATA_CLUSTERS )
{
for ( cluster = 2; cluster <= f->maximum_cluster_number; cluster++ )
{
#ifndef NDEBUG
if( !(cluster&0x7F) ) // 1:128
{
printf("\rrdcf_format(): erasing %d/%d...\r", cluster, f->maximum_cluster_number);
}
#endif // NDEBUG
if( erase_cluster( f, cluster ) )
{
set_FAT_entry ( f, cluster, BAD_CLUSTER );
bad_clusters++;
}
}
#ifndef NDEBUG
printf("\rrdcf_format(): erased %d/%d \n", cluster-1, f->maximum_cluster_number);
#endif // NDEBUG
}
else
{
cluster = f->maximum_cluster_number;
}
/* Cleanup and Exit */
// NOTE: test_cluster() has called error_exit() to trap low level I/O
// errors during its work. It would be invalid to do any more work
// that could possibly throw another exception. If you needed to
// for some reason, you would have to call setjmp() again:
//
//if ((f->result=setjmp(f->error)) != 0) return f->result;
return 0;
}
On Thursday, October 14, 2004 11:24 AM, Bogdan Vacaliuc wrote:
> Hi Etienne,
>
> Thanks. I won't use it right now; let me concentrate on the patch
> using the working/tested code. I have to be careful to strip any
> proprietary modifications, etc, before releasing back to the public
> domain.
>
> Afterwards, I will take a look.
>
> -bogdan
>
>
> On Thursday, October 14, 2004 11:19 AM, Etienne Fortin wrote:
>
>> I have the source code of mkdosfs for I don't remember what OS. Its
>> old, like years old, but it may be usefull for inspiration. Do you
>> want it?
>>
>> Etienne Fortin
>> Sensio
More information about the users
mailing list