Formatting DOSFS volume
Etienne Fortin
etienne.fortin at sensio.tv
Mon Oct 25 15:47:47 UTC 2004
Hi Bogdan,
Where are you in the implementation of formating dosfs for RTEMS? You
are talking in your last email about including the code you already
have. But if it isn't "RTEMS ready" yet, I guess you didn't put it in
the RTEMS CVS? I don't see it in the CVS tree anyway.
If I can get your code, I'll look at it and, as you said, will be able
to extrapolate with the mkdosfs code. I know it's a quite involving
implementation, but I need it anyway so I must do it :)
Etienne Fortin
Sensio
-----Message d'origine-----
De : Bogdan Vacaliuc [mailto:bvacaliuc at ngit.com]
Envoyé : 18 octobre, 2004 23:39
À : 'Etienne Fortin'
Cc : rtems-users at rtems.com; 'Eugeny S. Mints'; joel.sherrill at OARcorp.com
Objet : RE: RE : RE : Formatting DOSFS volume
Etienne,
Well, in merging over the FAT16 and format updates from my codebase, I
find it to be a rather painful and time consuming merge. The FAT16 is
done now, but not the format. There really were significant
architectural changes vs. Philip's RDCF2 (especially in the cache/device
interface) that would take a lot more time to merge. Without them, the
bad sector operations in the format code are rather meaningless.
I'm in danger of losing sight of the goal of all this: Implement format
for RTEMS.
With that in mind, I'm including the code for the two functions
necessary for implementing format from my codebase. This is based on
the RDCF2-ism of computing a "format table" which is passed-in to
"format". You can extrapolate this to the mkdosfs-like implementation
that you may be implementing.
I looked briefly at the current Debian source of mkdosfs (from
dosfstools-2.8). It's a pretty big, involved implementation.
I see the following:
1) it only does a 'read' test to see if a block is usable.
2) it doesn't deal with bad blocks before the data area:
if (currently_testing < start_data_block)
die ("bad blocks before data-area: cannot make fs");
I should think that a 'write' test would be more conclusive (I was doing
a write/read/compare, but it takes forever. In the current version, I
just do erase_cluster() which calls the device driver's IOCTL that
performs an ERASE_SECTOR command. This is faster than
write/read/compare.)
Also, it is not a-typical for NAND flash parts to fail in blocks below
the start of data, since those blocks are accessed with much greater
frequency than DATA section. In the project that this code comes from,
we have found it useful to have this capability.
So I hope this can provide you with some inspiration as you implement
the format code. Let me know if there is anything I can help with.
Best Regards,
-bogdan
--- Code snippets follow ---
// compute BPB format info based on partition size and bytes per sector.
// *fmt points to a buffer that will be filled with contents of the //
FAT BPB starting at BPB_BytsPerSec, which is at offset 11 in the BPB. //
This code is from the M$ document, with the FAT32 stuff taken out static
int rdcf_compute_format_table(struct rdcf_format *fmt, uint32 DskSize,
uint32 sector_bytes ) {
uint32 spc;
uint8 BPB_SecPerClus; // computed below
uint32 RootDirSectors; // working variable
uint32 TmpVal1, TmpVal2, FATSz; // working variables
uint16 BPB_FATSz16; // computed below
// hardcoded assumptions to make M$ calculation work
#define BPB_BytsPerSec 512
#define BPB_ResvdSecCnt 1
#define BPB_NumFATs (uint32)2
#define BPB_RootEntCnt 512
#define LOWORD(A) (uint16)(A & 0xFFFF)
#define HIWORD(A) (uint16)((A >> 16) & 0xFFFF)
// BPB_SecPerClus from the FAT16 table above. (valid is 2^[0..7],
BPS * SPC <= 32K)
// Note: we dont have to worry about spc not being valid, because
the last
// entry in the table has a value for DiskSize that will guarantee a
break.
for(spc=0; spc<(sizeof(DskTableFAT16)/sizeof(struct
DSKSZTOSECPERCLUS)); spc++)
{
if( DskSize <= DskTableFAT16[spc].DiskSize ) break;
}
BPB_SecPerClus = DskTableFAT16[spc].SecPerClusVal;
// verify parameters
if( BPB_BytsPerSec != _2plog2n( sector_bytes ) )
{
LOG_NOTICE( "BPB_BytsPerSec != _2plog2n( sector_bytes )" );
return 1;
}
if( (BPB_BytsPerSec * BPB_SecPerClus) > (32*1024) )
{
LOG_NOTICE( "(BPB_BytsPerSec * BPB_SecPerClus) > (32*1024)" );
return 1;
}
// compute BPB_FATSz16 so as to generate a FAT16 filesystem
// see pp. 13-14 in Microsoft "FAT: General Overview of On-Disk
Format"
//
// NOTE: We are operating under the following assumptions:
// 1) FAT16 is forced
// 2) BPB_BytsPerSec = 512
// 3) BPB_SecPerClus is computed according to the M$ table above.
// 4) BPB_ResvdSecCnt = 1
// 5) BPB_NumFATs = 2
// 6) BPB_RootEntCnt = 512
RootDirSectors = ((BPB_RootEntCnt * 32) + (BPB_BytsPerSec - 1)) /
BPB_BytsPerSec;
TmpVal1 = DskSize - (BPB_ResvdSecCnt + RootDirSectors);
TmpVal2 = (256 * BPB_SecPerClus) + BPB_NumFATs;
//if(FATType == FAT32) TmpVal2 = TmpVal2 / 2;
FATSz = (TmpVal1 + (TmpVal2 - 1)) / TmpVal2;
//if(FATType == FAT32)
//{
// BPB_FATSz16 = 0;
// BPB_FATSz32 = FATSz;
//} else
//{
BPB_FATSz16 = LOWORD(FATSz);
/* there is no BPB_FATSz32 in a FAT16 BPB */
//}
//
// Initialize the Boot Paramater Block Table
//
// BPB_BytsPerSec can be 512 (recommended), 1024, 2048, 4096
// however, we use M$ tables below, which only work for 512 byte
sectors.
fmt->x[0] = BPB_BytsPerSec & 0xFF;
fmt->x[1] = BPB_BytsPerSec >> 8;
// BPB_SecPerClus from the FAT16 table above. (valid is 2^[0..7],
BPS * SPC <= 32K)
fmt->x[2] = BPB_SecPerClus;
// BPB_ResvdSecCnt must be 1
fmt->x[3] = BPB_ResvdSecCnt & 0xFF;
fmt->x[4] = BPB_ResvdSecCnt >> 8;
// BPB_NumFATs should be 2 (for flash can make 1, but check
compatibility with extern OS)
fmt->x[5] = BPB_NumFATs;
// BPB_RootEntCnt must be 512 for FAT16; (FAT32:0, FAT16:512,
FAT12:(RDIR*32)%BPS = 0)
fmt->x[6] = BPB_RootEntCnt & 0xFF;
fmt->x[7] = BPB_RootEntCnt >> 8;
// BPB_TotSec16 or BPB_TotSec32 according to the total addressable
LBNs on the device
if( DskSize < 0x10000 )
{
fmt->x[8] = (uint8)(DskSize & 0xFF); // NOTE: M$ comes up
with +1 vs. us on this...
fmt->x[9] = (uint8)((DskSize >> 8) & 0xFF);
memset( &fmt->x[21], 0, 4 );
}
else
{
memset( &fmt->x[8], 0, 2 );
fmt->x[21] = (uint8)(DskSize & 0xFF);
fmt->x[22] = (uint8)((DskSize >> 8) & 0xFF);
fmt->x[23] = (uint8)((DskSize >> 16) & 0xFF);
fmt->x[24] = (uint8)((DskSize >> 24) & 0xFF);
}
// BPB_Media, 0xF0, 0xF8-0xFF are valid
// NOTE: must match value in FAT[0]
fmt->x[10] = 0xF0; // 0xF0 typically used for removable media
// BPB_FATSz16 which is forced
fmt->x[11] = (uint8)(BPB_FATSz16 & 0xFF);
fmt->x[12] = (uint8)((BPB_FATSz16 >> 8) & 0xFF);
// number of sectors per track
fmt->x[13] = 63; // per M$ empirical
fmt->x[14] = 0;
// number of sides
fmt->x[15] = 255; // per M$ empirical
fmt->x[16] = 0;
// BPB_HiddSec; always 0 on unpartitioned media
fmt->x[17] = 0;
fmt->x[18] = 0;
fmt->x[19] = 0;
fmt->x[20] = 0;
return 0;
}
int rdcf_format(struct rdcf *f,
#ifdef RDCF_MULTIPLE_DRIVE
char *spec,
#endif
struct rdcf_format *format)
{
static const char bootstrap_header[11] =
{
(char)235,(char)254,(char)144,'R','D','C','F','2','0',' ',' '
};
static const char bootstrap_trailer[26] = // for
offset[36]
{
0,0,0, //
drv,resv1,bootsig
4,3,2,1, // 32bit S/N
'N','O',' ','N','A','M','E',' ',' ',' ',' ', // volume label
'F','A','T','1','6',' ',' ',' ' // filsystype
};
unsigned char *buffer = f->buffer;
unsigned char media_descriptor;
rdcf_sector_t sector, bad_sector_limit;
rdcf_cluster_t cluster;
rdcf_cluster_t bad_clusters;
/* setup for failure exit */
if ((f->result=setjmp(f->error)) != 0) return f->result;
#ifdef RDCF_MULTIPLE_DRIVE
if (*get_drive(f, spec) != 0) error_exit(f, RDCF_INVALID_SPEC);
#endif
/* verify that the BPB sector is viable [this is the first sector of
the volume] */
test_sector(f, f->bpb_sector, 1);
/* 03/23/04: compute limit to # of bad sectors we accept dynamically
(by parsing format table) */
bad_sector_limit = format->x[8]; // BPB_TotSec16...
bad_sector_limit |= format->x[9]<<8; // ...
if( bad_sector_limit == 0 )
{
bad_sector_limit = format->x[21]; // BPB_TotSec32...
bad_sector_limit |= format->x[22]<<8; // ...
bad_sector_limit |= format->x[23]<<16; // ...
bad_sector_limit |= format->x[24]<<24; // ...
}
bad_sector_limit /= 100;
bad_sector_limit *= BAD_SECTOR_RESERVE; // percentage, [1..100]
/* 11/19/03: handle failures in the FAT and directory tables */
if( (f->result=setjmp(f->error)) != 0 )
{
unsigned int bad = f->drive_error;
/* Handle failures in test_sector() by updating the reserved
sector count */
bad &= ~31; // align to 32 LBN boundary (presuming NAND
driver)
bad += 32; // increment to next block
/* check against limits computed above */
if( bad < bad_sector_limit )
{
format->x[3] = bad & 0xFF; // BPB_RsvdSecCnt...
format->x[4] = (bad>>8) & 0xFF; // ...
}
else return f->result;
}
/* Initialize the BPB sector */
memset(buffer, 0, SECTOR_SIZE);
memcpy(buffer, bootstrap_header, 11); // always exactly 11
bytes
memcpy(buffer + 11, format, 25); // always exactly 25
bytes
memcpy(buffer + 36, bootstrap_trailer, 26); // at least 26 bytes
buffer[510 /*SECTOR_SIZE-2*/] = 0x55; // per M$; the
FSI_TrailSig
buffer[511 /*SECTOR_SIZE-1*/] = 0xAA; // per M$; NOTE: NOT the
end of the sector
write_sector(f, 0, buffer);
f->buffer_status = CLEAN;
f->sector_in_buffer = 0;
/* (re)load filesystem info based on current BPB data */
read_file_system_information(f);
media_descriptor = MEDIA_DESCRIPTOR;
/* Initialize FAT tables (FAT[1..N]) */
memset(buffer, 0, SECTOR_SIZE);
sector = f->first_FAT_sector;
while (sector < f->first_directory_sector)
{
rdcf_sector_t count = f->sectors_per_FAT;
test_sector(f, sector, 1); // NOTE: uses
error_exit()/longjmp()
buffer[0] = media_descriptor;
//buffer[1] = buffer[2] = 0xFF; // NOTE: This gave
0xFFFFMD, wrong
buffer[1] = 0xFF; // per M$; only 0xFFMD
is required
write_sector(f, sector++, buffer);
buffer[0] = buffer[1] = buffer[2] = 0;
while (--count != 0)
write_sector(f, sector++, buffer);
}
/* Initialize the ROOT Directory table */
/* TODO: see about handling the ClnShutBitMask and HrdErrBitMask */
buffer[0] = buffer[1] = buffer[2] = 0;
while (sector < f->first_data_sector)
{
test_sector(f, sector, 1); // NOTE: uses
error_exit()/longjmp()
write_sector(f, sector++, buffer);
}
/* Initialize the DATA area */
bad_clusters = 0;
if( ENABLE_TEST_DATA_CLUSTERS )
{
//
// Perform a cluster check of all data clusters and populate the
FAT
// tables with the bad cluster information.
//
for ( cluster = 2; cluster <= f->maximum_cluster_number; cluster++
)
{
#ifndef NDEBUG
if( !(cluster&0x7F) ) // 1:128
{
printf("\rrdcf_format(): verifying %d/%d...\r", cluster,
f->maximum_cluster_number);
}
#endif // NDEBUG
if( test_cluster( f, cluster ) )
{
set_FAT_entry ( f, cluster, BAD_CLUSTER );
bad_clusters++;
}
else if( ENABLE_ERASE_DATA_CLUSTERS )
{
(void)erase_cluster( f, cluster );
}
}
#ifndef NDEBUG
printf("\rrdcf_format(): verified %d/%d \n", cluster-1,
f->maximum_cluster_number);
#endif // NDEBUG
}
else if( ENABLE_ERASE_DATA_CLUSTERS )
{
for ( cluster = 2; cluster <= f->maximum_cluster_number; cluster++
)
{
#ifndef NDEBUG
if( !(cluster&0x7F) ) // 1:128
{
printf("\rrdcf_format(): erasing %d/%d...\r", cluster,
f->maximum_cluster_number);
}
#endif // NDEBUG
if( erase_cluster( f, cluster ) )
{
set_FAT_entry ( f, cluster, BAD_CLUSTER );
bad_clusters++;
}
}
#ifndef NDEBUG
printf("\rrdcf_format(): erased %d/%d \n", cluster-1,
f->maximum_cluster_number);
#endif // NDEBUG
}
else
{
cluster = f->maximum_cluster_number;
}
/* Cleanup and Exit */
// NOTE: test_cluster() has called error_exit() to trap low level
I/O
// errors during its work. It would be invalid to do any more work
// that could possibly throw another exception. If you needed to
// for some reason, you would have to call setjmp() again:
//
//if ((f->result=setjmp(f->error)) != 0) return f->result;
return 0;
}
On Thursday, October 14, 2004 11:24 AM, Bogdan Vacaliuc wrote:
> Hi Etienne,
>
> Thanks. I won't use it right now; let me concentrate on the patch
> using the working/tested code. I have to be careful to strip any
> proprietary modifications, etc, before releasing back to the public
> domain.
>
> Afterwards, I will take a look.
>
> -bogdan
>
>
> On Thursday, October 14, 2004 11:19 AM, Etienne Fortin wrote:
>
>> I have the source code of mkdosfs for I don't remember what OS. Its
>> old, like years old, but it may be usefull for inspiration. Do you
>> want it?
>>
>> Etienne Fortin
>> Sensio
More information about the users
mailing list