How to sync a dos file system externally

Wed Oct 3 16:18:51 UTC 2018

Thanks for the advice.

I looked at JFFS2 but I hope to use this file system on a removable SD card and
JFFS2 does not appear to be widely supported by other PCs.

fsync() is exactly what I was looking for.  I call it after a 3-second pause in
the write stream.  The user needs to stop writing 3 seconds before a power down.

But there is something more subtle going on with the RTEMS file system that I do
not understand.  It is at a higher level than the bdbuf caching.

After writing files via FTP to this file system, when the FTP daemon times out
after several minutes, it closes the connection and issues chdir("/").  That
causes a write operation on my mounted flash device.  The stack is:

    fat_sector_write

    fat_file_write_first_cluster_num

    fat_file_update

    fat_file_close

    msdos_free_node_info

    rtems_filesystem_location_free

    release_with_count

    deferred_release

    rtems_filesystem_global_locataion_obtain

    set_startloc

    rtems_filesystem_eval_path_start_with_root_and_current

    rtems_filesystem_eval_path_start

    rtems_filesystem_eval_path_extrace_currentloc

    chdir

    session at ftpd.c

Sometimes reading the directory locally or from FTP using opendir() causes a
similar write operation on the flash, but not always.  The stack is similar:

    fat_sector_write

    fat_file_write_first_cluster_num

    fat_file_update

    fat_file_close

    msdos_free_node_info

    rtems_filesystem_location_free

    release_with_count

    deferred_release

    rtems_filesystem_global_locataion_obtain

    set_startloc

    rtems_filesystem_eval_path_start_with_root_and_current

    rtems_filesystem_eval_path_start

    do_open

    opendir

I would not expect chdir or opening a directory for read to cause a write to the
device.  I do not understand why msdos_free_node_info is calling fat_file_update
that changes the first cluster num, file size, or time and date.

My question is:  How to I force the "deferred_release" or the fat_file_update to
happen when I sync?  I don't really understand the inner workings of
rtems_filesystem_*.

I tried changing msdos_sync to include fat_file_update, like msdos_file_sync,
but that did not help.  I imagine the pathinfo is not setup to be meaningful
when msdos_sync is called.

If there is not a better method, I may wind restoring/saving the current path
around all ftpd commands and always do chdir("/").  That seems like a very
inefficient hack.

BTW, I am working with the RTEMS 4.11 version dosfs.

Thanks for your help,

Cliff

-----Original Message-----
From: Chris Johns
Sent: Tuesday, October 02, 2018 6:32 PM
To: Cliff Geschke; 

Subject: Re: How to sync a dos file system externally

On 03/10/2018 04:56, Cliff Geschke wrote:

> I have implemented a dos file system (msdos_*) on a flash device.

Is the flash device a chip you have direct access too? The reason I ask is if

possible using JFFS2 (a journaling file system) or even YAFFS (commercial

license maybe needed) is a better solution.

> Because it is possible for the user to power down the system unexpectedly, I

> want to sync the file system to the flash device after a 3 second idle time. 

> How do I externally force a sync on msdos from another thread?

The following test ...

 https://git.rtems.org/rtems/tree/testsuites/fstests/fsdosfssync01/init.c

shows how to purge a disk at the block layer. Wrap something like this is the

way to purge the cache.

> A related problem is that I use FTP (ftpd.c) to externally read/write files on

> the msdos formatted flash.  I have the idle timeout set to 3 minutes for the
FTP

> connection.  After 3 minutes, ftpd issues a chdir("/") which eventually calls

> msdos_free_node_info() that calls fat_file_close() and may try to write out
data

> to the flash.  This is a problem because if the power is turned off at that
time

> the flash is corrupted.

> 

> So whatever method I use to sync msdos needs to update and write out the fat
so

> that the subsequent msdos_free_node_info() does nothing.

I should point out there is a finite chance the disk can still become corrupted.

How small the window becomes depends on the system design, for example how often

the disk is updated, power supply power down storage vs media write time,

non-bricking read-only partitions, etc.

I suspect triggering a timer to purge the cache as shown above is no different

to lowering the `swapout` task's period. It has been a while since I looked over

this code. The libblock cache implements separate threads to "sync" the cache to

the media on a periodic basis. The cache's configuration lets you set the

period. The values are documented here:

https://docs.rtems.org/doxygen/branches/master/group__rtems__bdbuf.html#define-m
embers

This however raises a difficult question which is documented in the code in a

comment:

 https://git.rtems.org/rtems/tree/cpukit/libblock/src/bdbuf.c#n1013

It is difficult when using a timer to know if the purge is 100% safe. If you can

arrange the purge call to happen after you know the update has finished it

lowers the chance the disk maybe become corrupted.

Chris

-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.rtems.org/pipermail/users/attachments/20181003/2ab142cf/attachment.html>