IMFS - Impact of 32 vs 64 bit Pointers

[Gedare Bloom]

On Wed, Jul 25, 2018 at 6:54 PM, Joel Sherrill <joel at rtems.org> wrote:
> Hi
>
> In some recent discussions, the maximum file size supported
> by the IMFS came up. I proposed increasing the allowed block
> size from the maximum of 512. When asked what the impact
> on maximum file size would be, I decided to hack the imfs.h
> macros into a simple test program. When I ran that program
> natively, it didn't produce the same maximum file sizes in
> imfs.h. After some investigation, it occurred to me that the
> native pointer was 64-bits and the maximum file sizes were
> computed based on 32-bit pointers to the malloc'ed blocks.
>
> $ ./imfs_size    # source attached
>
> =========== Pointer = 4 bytes ===========
> 16 ==> 1328 or < 1MB
> 32 ==> 18656 or < 1MB
> 64 ==> 279488 or < 1MB
> 128 ==> 4329344 or 4MB
> 256 ==> 68173568 or 65MB
> 512 ==> 1082195456 or 1032MB
> 1024 ==> 17247239168 or 16448MB
> 2048 ==> 275415824384 or 262656MB
>
> =========== Pointer = 8 bytes ===========
> 16 ==> 208 or < 1MB
> 32 ==> 2656 or < 1MB
> 64 ==> 37312 or < 1MB
> 128 ==> 558976 or < 1MB
> 256 ==> 8658688 or 8MB
> 512 ==> 136347136 or 130MB
> 1024 ==> 2164390912 or 2064MB
> 2048 ==> 34494478336 or 32896MB
>
> What this means is that on target architectures with 64-bit pointers
> the maximum file size is much smaller.
>
> I am proposing two things:
>
> (1) that the default block size (128 now) be dependent on
> whether __SIZEOF_POINTER__ is 4 or 8.
>

I don't think this should be done. It is not necessary. If an
application needs larger IMFS file size than supported, it should set
this explicitly. We should provide a sane default that works well in
the common case.

> (2) that the maximum block size be increased from 512 to at
> least 2048. On 32-bit pointer targets, this is ridiculously large and
> on 64-bit targets, it should be large enough for any foreseeable
> use case.
>

That's fine with me.

> At the risk of increased complexity, the maximum block size could
> be conditional on pointer size also. 512 for 32-bit pointers and
> 2K for 64-bit pointers. But I think 2K blocks has efficiency improvements
> because it reduces the number of allocations and per allocation
> overhead so I don't want to do this.
>

But it also causes internal fragmentation.

> I am prone to even allow 4K blocks based on allocation efficiency.
> The maximum filesize is likely larger than available RAM on many
> boards anyway.
>
> Thanks.
>
> --joel
>
> _______________________________________________
> devel mailing list
> devel at rtems.org
> http://lists.rtems.org/mailman/listinfo/devel