libio preparation for file systems

[Joel Sherrill]

On Thu, 26 Feb 1998, Robin Kirkham wrote:

> 
> On Wed, 25 Feb 1998, Joel Sherrill wrote:
> 
> > I have nearly reached the conclusion that enhancing the current libio by
> > adding directories and mounts will provide nearly all the functionality
> > required. 
> 
> I still think it is cleaner to have a "new" libio which deals only in the
> filesystem objects (mounts, etc), the current directory, file descriptors
> and file table (or equivalent), and which provides the front end of all the
> POSIX functions we mentioned.
>
> At the same time, one would provide a simple, separate "memory" filesystem,
> which implements a hierarchical namespace (i.e., directories), and device
> nodes (which would support the current major/minor number system).

OK.  I had not really thought into the detail that far yet.  But you are
right in that having a 2 layer system  right from the start makes things
easier to debug.
 
> In simple RTEMS systems, the new libio would manage just one of these 
> filesystems, which would be the root filesystem. The two together would 
> provide the current libio functionality, but with huge potential for
> expansion. In more complicated systems, this memory filesystem provides
> points to mount other filesystems on, even other invocations of itself.

Are you proposing that there be a "device management" file system which
according to current (and UNIX) convention is mounted onto /dev?

I think it is important for the memory file system to allow mounts of
itself.  I don't think this is particularly a problem just something to
make sure works. :)

> I think what you propose above (enhancing the current libio by adding
> directories and mounts) rolls the two together, and I think this will limit
> flexibility in the long term. The reasons I stress this separation are:
> 
>    1. You can expand or adapt the memory filesystem to do other more
>       interesting things (e.g., memory "files") and leave the new libio
>       alone.

I had thought about testing using a memory file.

>    2. One day you might not want to use it as a root filesystem,
>       because you want to use a real disk or a flash filesystem.
>       You don't want to have to split libio to do that.

Definitely not.  At the end of this procedure, I want the proposed FFS to
simple plug in.

>    3. It gives you an example filesystem that you can use to test
>       the new libio with, and which you can use as a skeleton to commence
>       implementation of other filesystem types from.

How to test this is an important thing I had not thought about yet.  You
are right that having an example file system which is really used all the
time will ease debug.

>    4. The data structures that implement the memory filesystem can be
>       constrained into a defined block of memory, which you can arrange
>       to be in a battery-backed-up RAM. Result--instant persistent
>       namespace (much more useful if you implement files as in 1).

This will be cool.

> > > Wouldn't worry me if it were 0% UNIX.
> > 
> > We all want some "unix" -- just having a good C library gives you an
> > amazingly high degree of compatibility.  But I don't want to go overboard
> > to the point of destroying the basic performance characteristics.
> 
> My flippant remark was supposed to mean it wouldn't worry me of 0% of
> the *implementation* were UNIX. What's essential is that 100% of the
> *semantics* are POSIX. We can deviate considerably from traditional
> implementation practices, and probably should.

OK.  We are on the same page.

> > It is not the multi-threaded semantics that are the problem.  They are
> > pretty clear in the POSIX spec.  The problem is that much of POSIX assumes
> > multiple processes.  There are oodles of references to process ids, group
> > ids, process groups, fork/exec/wait, etc.  Some of this is hard to decide
> > what it means (if anything) in a single process environment.
> 
> One way through it is to pretend RTEMS is a process on a multiprocess system
> where the process table is full. getpid() returns a constant, of course,
> fork() always fails (no more processes), wait() fails (no such child process),
> etc. exec() is a tricky one.

I think that for the process oriented routines which do exist in the POSIX
API, this is what we did.  We have discussed viewing an RTEMS
multiprocessor system as a multiple process POSIX system but never made
any effort to make that work.   

> This does remind me: for NFS (a very useful, and fairly easily implemented
> filesystem, once someone ports RPC) does imply a basically working
> getuid(), getgrpid(), getgroups() set-up.

Some of these routines are already implemented but tend to return
variables which may be set via the set*() routines.

> Yes, but we are talking about *system calls* which are not really part of a
> C library (and yes, the system call functions are in libc, but in a Unix
> systems they are just a front end to the kernel trap instruction). I assume
> many of the newlib "system call" functions are either just do-nothing
> functions, or pass things on (like open(), read(), write(), do to the
> current libio).

But to make it work, those have to be supported.  I am getting there.  I
need to do some write up and spend some time on a white-board. :)

> > specific stuff to at least validate the remaining path name.
> 
> Yes, it would use the filesystem's stat(). There will be other cases like
> this. They need to be thought through. It's quite possible just the standard
> set of operations will be sufficient, but if not, some "helpers" could be
> defined.

OK.

> BTW in vxWorks you can chdir() to anything you like, but if it's wrong you
> don't find out until you open("./something")! Another problem I'd like to
> see addressed from the start.

This will not be a problem.  This is a problem that should not have been
introduced. We are pretty paranoid about catching errors as early as
possible.

--joel