IMFS on ram-disk

Tue Jul 24 17:05:49 UTC 2018

On Tue, Jul 24, 2018 at 11:34 AM, Gedare Bloom <gedare at rtems.org> wrote:

> On Tue, Jul 24, 2018 at 10:46 AM, Joel Sherrill <joel at rtems.org> wrote:
> >
> >
> > On Tue, Jul 24, 2018 at 9:24 AM, Udit agarwal <dev.madaari at gmail.com>
> wrote:
> >>
> >>
> >>
> >> On Jul 24, 2018 7:45 PM, "Gedare Bloom" <gedare at rtems.org> wrote:
> >>
> >> On Tue, Jul 24, 2018 at 10:12 AM, Gedare Bloom <gedare at rtems.org>
> wrote:
> >> > On Tue, Jul 24, 2018 at 4:48 AM, Udit agarwal <dev.madaari at gmail.com>
> >> > wrote:
> >> >> Hi all,
> >> >> I've been trying to use IMFS with a ramdisk on BBB. Here's what i
> did:
> >> >>
> >> >> After creating a ram disk (block size:512, block count:262144), i
> used
> >> >>
> >> >> mount_and_make_target_path("/dev/rda","/mnt",RTEMS_
> FILESYSTEM_TYPE_IMFS,RTEMS_FILESYSTEM_READ_WRITE,NULL)
> >> >>
> >> >> There is no error during bootup, but when allocating a file(in the
> /mnt
> >> >> directory) of size 10MB(or even of 1MB) it gives a file too large
> >> >> error.
> >> >> I've used similar config for RFS too and that did worked, so probably
> >> >> there's no issue while setting up the RAM disk.
> >> >>
> >> >> I've also tried testsuite/fstests/imfs_support setup, but looks like
> >> >> similar
> >> >> method doesn't work with ram-disks
> >> >
> >> > What did you get back from mount_and_make_target_path()? Did it
> >> > succeed? I'm pretty sure this method should not be used for imfs, see
> >> > fstests/imfs_support/fs_support.c
> >> >
> >>
> >> Oh, I take that back. Sebastian's comment is accurate. You can
> >> probably make your approach work by replacing "/dev/rda" with NULL I
> >> would guess. The IMFS doesn't need a source, since it is backed by the
> >> heap.
> >>
> >> Yes, Sebastian's approach worked. I'm able to generate some benchmarking
> >> stats on IMFS too.
> >
> >
> > I would expect the tuning parameter CONFIGURE_IMFS_MEMFILE_BYTES_
> PER_BLOCK
> > to have a significant impact on the performance of the IMFS.  The default
> > size is 128.
> > There is a block comment in cpukit/libfs/src/imfs/imfs.h which shows the
> > possible values.
> > Since the file blocks are malloc'ed, the blocks default small-ish to
> avoid
> > wasting
> > memory for small files.
> >
> > The IMFS uses an inode structure based on that of the original UNIX
> > filesystem.
> > So as the block size goes up, you can have more and more blocks in the
> > maximum file size.
> >
> > *  The data structure for the in-memory "memfiles" is based on classic
> UNIX.
> >  *
> >  *  block_ptr is a pointer to a block of IMFS_MEMFILE_BYTES_PER_BLOCK in
> >  *  length which could be data or a table of pointers to blocks.
> >  *
> >  *  Setting IMFS_MEMFILE_BYTES_PER_BLOCK to different values has a
> > significant
> >  *  impact on the maximum file size supported as well as the amount of
> >  *  memory wasted due to internal file fragmentation.  The following
> >  *  is a list of maximum file sizes based on various settings
> >  *
> >  *  @code
> >  *    max_filesize with blocks of   16 is         1,328
> >  *    max_filesize with blocks of   32 is        18,656
> >  *    max_filesize with blocks of   64 is       279,488
> >  *    max_filesize with blocks of  128 is     4,329,344
> >  *    max_filesize with blocks of  256 is    68,173,568
> >  *    max_filesize with blocks of  512 is 1,082,195,456
> >
> >
> > If a system has lots of memory and wants to store larger files with lower
> > malloc
> > overhead, then the default size should be increased.
> >
> > NOTE: The IMFS silently ignores requests which are not power of
> > 2 and are < 16 or greater than 512. See IMFS_determine_bytes_per_block()
> > and its use in imfs_initsupp.c.
> >
> > For the purposes of benchmarking and since the 512 upper
> > block size (plus implied max file size) was determined so long
> > ago that a 1G RAM file seemed impossible, it probably makes sense
> > to let people configure up to 4K or 8K as the IMFS bytes per block.
> >
> A 1 GiB RAM is still not that practical in an IMFS.
>
> What would be the maximum file sizes if we did increase it? From the
> above, the trend appears to be an increase by a factor of 2^4 each
> time, so I'd guess the max file size with blocks of 1024 would be
> about 16 GB, 2048 about 256 GB, and 4096 about 4 TB.
>
> There is for sure no practical reason to consider an IMFS to support
> file sizes larger than 16 GB for the near future...
>

The math is in macros in imfs.h.

The "inode" has direct, doubly indirect, and triply indirect blocks and
there are (block size/ sizeof(void *)) blocks pointed to. .This is how
the original unix filesystem was done. The RFS should be similar
but I don't recall the details. This is shown here.

https://cs.nyu.edu/courses/spring09/V22.0202-002/lectures/lecture-24.html

We could follow the macros and compute it but I would let a computer
calculate the maximum size. Probably easier to either fix the code
and then use a small program to print the maximum size. Or hack
the macros out into a simple test program. :)

And hacking the macros into a simple test program is what I did. :)
See if I got the attached program right. It has one hack to account
for 32-bits for a pointer and not 64-bits like a native program.
64-bit pointers significantly reduce the maximum size which might
be a good reason to have larger blocks.

16 ==> 1328 or < 1MB
32 ==> 18656 or < 1MB
64 ==> 279488 or < 1MB
128 ==> 4329344 or 4MB
256 ==> 68173568 or 65MB
512 ==> 1082195456 or 1032MB
1024 ==> 17247239168 or 16448MB
2048 ==> 275415824384 or 262656MB

for 32-bit pointers.  For 64-bit pointers:

16 ==> 208 or < 1MB
32 ==> 2656 or < 1MB
64 ==> 37312 or < 1MB
128 ==> 558976 or < 1MB
256 ==> 8658688 or 8MB
512 ==> 136347136 or 130MB
1024 ==> 2164390912 or 2064MB
2048 ==> 34494478336 or 32896MB

>
> > And we should consider whether mis-configuring the bytes per block
> > in the IMFS should result in a silent defaulting or a fatal error. I know
> > I was surprised the last time I ran into this.
> >
> I filed a ticket for both.
>
> >>
> >>
> >> >> --
> >> >> Regards,
> >> >> Udit kumar agarwal
> >> >> http://uditagarwal.in/
> >> >>
> >> >> _______________________________________________
> >> >> users mailing list
> >> >> users at rtems.org
> >> >> http://lists.rtems.org/mailman/listinfo/users
> >>
> >>
> >>
> >> _______________________________________________
> >> users mailing list
> >> users at rtems.org
> >> http://lists.rtems.org/mailman/listinfo/users
> >
> >
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.rtems.org/pipermail/users/attachments/20180724/f8357b6a/attachment.html>
-------------- next part --------------
A non-text attachment was scrubbed...
Name: imfs_size.c
Type: application/octet-stream
Size: 1270 bytes
Desc: not available
URL: <http://lists.rtems.org/pipermail/users/attachments/20180724/f8357b6a/attachment.obj>