[RTEMS File System] - Flash Memories for Space Applications

[Chris Johns]

Michele Fabiano wrote:
> Hi, everybody!
> 
> I'm Michele Fabiano, PhD Student in Politecnico di Torino.
> 
> With my group, we are working on Flash Memory for Space Applications 
> and, now we are facing some problems (as wondered... :D).
> 

You are not alone in doing this.

> Our final goal should be to get a Non Volatile Storing Device (let's 
> simply say an Hard Disk) based on Flash Memory Technology; a lot of 
> requirements need to be met, but do not take care of them! ;)
> 
> However we know that the OBC (On-Board Computer) that is going to be 
> sent in the space will have the RTEMS OS on it and so we were 
> investigating first of all *_WHAT to do_*, i.e., all the possibilities 
> that we can use in our design, crossing them with RTEMS capabilities.
> 
> First of all we would like to choose the "best" suitable File-System for 
> Space Applications: I read on your nice Wiki pages that, actually, in 
> the MSDOS FS there is the "Block Device" supporting "Flash disk", mainly 
> accomplishing a block-device emulation, creating the illusion that the 
> OS and the applications are accessing a "classic" Hard Disk (e.g., FAT32 
> formatted) but in reality there's a hidden mapping between a 
> block-device (just existing and that the OS can just use) and the Flash 
> device. (correct me if I'm wrong)

There are a few block devices that are part of the RTEMS source tree including 
a flash block driver. All these drivers live under the libblock cache.

The RAM disk is designed to take a single block of continuous memory and 
create blocks from it. The blocks have no checksum and it is assumed you have 
directly mapped memory to it. The NV disk driver is designed for EEPROM type 
memory that is in use in some space applications. Here each block has a 
checksum held in a checksum table. The design assumes byte or word 
write/erasable memory. The NV disk can also access windowed type hardware 
where access is via is device window. Here the driver would set registers to 
access the window of the memory.

The flash disk maps the blocks of the libblock cache currently to NOR type 
flash devices. The top of cpukit/libblock/src/flashdisk.c holds more details 
about the code and how it works. I suppose it works more like the inside of a 
USB flash stick not that I have ever looked into these devices. The flash is a 
collection of blocks (called pages in the driver) scattered over the 
device(s). During initialisation the state of all pages is determined and 
queues based on the state of segments (or sectors) in the flash devices are 
created. A write to a block will result in a new copy of the block in the 
flash being made and the old block being marked as used. Once all the blocks 
or pages in a segment (or sector) of the flash are used the segment is queued 
to be erased or is erased. If available space becomes tight the driver 
compacts blocks into segments to allow segments to be erased.

Please note the libblock cache handles the issue of small writes to a page and 
you can control the period a block is held in the cache before being written 
to disk. You need to review this driver with the effects of the cache in mind. 
They work together.

The flash driver currently does not support NAND type device but it could be 
added (that is the comment I made in the code when the driver was written) and 
it does not manage wear levelling. This could also be added with a little more 
meta-data about the state of use of segments. The driver does rotate segments 
for use so a simple type of wear levelling is present.

The flash driver is not a flash file system like JFFS or YAFFS. In these cases 
the file system is designed to work with the flash devices at a higher level 
but also with the low level device model in mind. I did not follow the file 
system model because they already exists, and did not port them because of 
there age and there are also reports of problems (JFFS). For example JFFS is 
slow to initialise and the size of memory being managed can be a problem. You 
will observe around the net many comments about existing file systems do not 
map well to flash devices because of wear, performance or other factors. I 
questioned these assumptions based on the huge number of products which appear 
as a block based disk drive such as compact flash, USB sticks, and now days 
SSD drives. I think all these issues will have been considered. My point is 
these issues can be resolved at different layers and not exclusively one way. 
For example the RTEMS flash driver, the flash file systems and these products 
will have some form of compaction, have lists of empty, used and full sectors.

The flash disk is new in RTEMS and it needs more rigorous testing. I have 
performed a range of tests but I would suggest more testing needs to be performed.

> 
> I point out once more that we are investigating on WHAT to do: that's 
> why I'm thinking a lot about it. In fact:
> 
>  *:)* -> on one side for us there's the chance to choose the "simplest" 
> way to go on, choosing a FS just supported by your RTEMS;
> 
>  *:(* -> on the other side there's the fact that actually there's no 
> support for Journaling, and so we need to manage power failures by 
> ourselves (and for sure we need to do it in Space Applications, and also 
> _/power consumption/ is a big issue concerning Space Applications_).

This is an important issue and I feel can effect the whole system. Journaling 
on a file system is valuable tool that can help here but you need to look into 
all possible issues the specific file system may have. You can avoid needing 
it. I have worked on systems where the power supply is designed to allow an 
application halt and data sync and a file system sync to complete once power 
fails. You also need to consider the amount of time you have to bring the file 
system online. Each solution will have a certain initialisation period.

Power consumption is something I cannot help with. There are too many factors 
to put in a post here.

> 
> So we're wondering also about choosing the "hardest" way to go on, 
> developing or just re-using a native FS which supports Journaling (e.g., 
> YAFFS) and could be more "suitable" for our goals.

If a file system like JFFS or YAFFS meets all your needs I would use it. I 
would not invent a new file system. How it compares with the RTEMS block flash 
driver is something you need to evaluate.

> 
> So, my questions are:
> 
>    1. in your experience, what do you think is the _*"best" way to
>       proceed*_? If you were me, what would you be wondering and taking
>       care of?

The flash disk driver is in the source tree so it is not a big effort to try 
and see how it works, find its limitations and advantages, find any bugs and 
provide any patches. I think one of the RTEMS target simulators has flash 
device support so may be it can be tried on it.

>    2. depending in your previous answer, could you please suggest me
>       some stuff to "study" to accomplish your idea? Or some tips to not
>       lose too much time developing something that someone just did
>       better than me? (Let's say, some _*tips to not reinvent the
>       wheel*_! :D )

I think you need to take the time to look at all the possible solutions about 
and determine what fits for you. I suspect it will effect a number of parts of 
your system. You need to match the type of use the application makes of the 
file system with the various solutions, ie data write rate, number of files, 
size of files etc.

> 
> 
> We do not have too much experience with RTEMS, a little bit more 
> experienced with Flash Memories and that's why I am here asking you 
> those questions: _we do not know which are the actual capabilities 
> generating from the crossing of these 2 "strange" animals_, i.e., RTEMS 
> and Flash Memories, and if there is a one-size-fits-all solution or 
> whatever else...we simply do not know! :D
> 
> Sorry for the lenght, but I tried to be the most accurate I could.
> 
> Any help/suggestion/remark will be really appreciated! Thanks in advance 
> for everything!
> 

I cannot answer in a way that gives you a ready solution but I hope this all 
helps. Please feel free to contact me directly with specific technical issues 
about your application and I am on IRC.

Regards
Chris

> Have a nice day!
> 
> Michele
> 
> PS: just in case, if someone is interested, I could send him my academic 
> mail to share more details! See you! ;)
> 
> ---------------------------------------------------------
> 
>  Michele FABIANO
>  
>  Politecnico di Torino
>  Dip. di Automatica e Informatica
>  Corso Duca degli Abruzzi 24
>  I-10129 Torino TO
>  Italy
> 
>  "Hard•ware n. The part of a computer system that can be kicked."
> 
> 
> ------------------------------------------------------------------------
> 
> _______________________________________________
> rtems-users mailing list
> rtems-users at rtems.org
> http://www.rtems.org/mailman/listinfo/rtems-users