libio preparation for file systems

[Robin Kirkham]

On Wed, 25 Feb 1998, Joel Sherrill wrote:

> I have nearly reached the conclusion that enhancing the current libio by
> adding directories and mounts will provide nearly all the functionality
> required. 

I still think it is cleaner to have a "new" libio which deals only in the
filesystem objects (mounts, etc), the current directory, file descriptors
and file table (or equivalent), and which provides the front end of all the
POSIX functions we mentioned.

At the same time, one would provide a simple, separate "memory" filesystem,
which implements a hierarchical namespace (i.e., directories), and device
nodes (which would support the current major/minor number system).

In simple RTEMS systems, the new libio would manage just one of these 
filesystems, which would be the root filesystem. The two together would 
provide the current libio functionality, but with huge potential for
expansion. In more complicated systems, this memory filesystem provides
points to mount other filesystems on, even other invocations of itself.

I think what you propose above (enhancing the current libio by adding
directories and mounts) rolls the two together, and I think this will limit
flexibility in the long term. The reasons I stress this separation are:

   1. You can expand or adapt the memory filesystem to do other more
      interesting things (e.g., memory "files") and leave the new libio
      alone.

   2. One day you might not want to use it as a root filesystem,
      because you want to use a real disk or a flash filesystem.
      You don't want to have to split libio to do that.

   3. It gives you an example filesystem that you can use to test
      the new libio with, and which you can use as a skeleton to commence
      implementation of other filesystem types from.

   4. The data structures that implement the memory filesystem can be
      constrained into a defined block of memory, which you can arrange
      to be in a battery-backed-up RAM. Result--instant persistent
      namespace (much more useful if you implement files as in 1).

 
> > Wouldn't worry me if it were 0% UNIX.
> 
> We all want some "unix" -- just having a good C library gives you an
> amazingly high degree of compatibility.  But I don't want to go overboard
> to the point of destroying the basic performance characteristics.

My flippant remark was supposed to mean it wouldn't worry me of 0% of
the *implementation* were UNIX. What's essential is that 100% of the
*semantics* are POSIX. We can deviate considerably from traditional
implementation practices, and probably should.


> > I don't think it's that tough--multithreaded UNIX is of course like that,
> > as is vxWorks. The task extension hooks are there for applications to make
> > this a per-task property if they really want (but getcwd() had better be
> > fast!)
> 
> It is not the multi-threaded semantics that are the problem.  They are
> pretty clear in the POSIX spec.  The problem is that much of POSIX assumes
> multiple processes.  There are oodles of references to process ids, group
> ids, process groups, fork/exec/wait, etc.  Some of this is hard to decide
> what it means (if anything) in a single process environment.

One way through it is to pretend RTEMS is a process on a multiprocess system
where the process table is full. getpid() returns a constant, of course,
fork() always fails (no more processes), wait() fails (no such child process),
etc. exec() is a tricky one.

This does remind me: for NFS (a very useful, and fairly easily implemented
filesystem, once someone ports RPC) does imply a basically working
getuid(), getgrpid(), getgroups() set-up.


> > I think the filesystem interface is already essentially defined by the set
> > of POSIX operations already mentioned. Practically, you would have a
> > ...
> 
> I think so too.  That is why I have already started discussions with the
> newlib maintainers to see what all has to be provided to support more of
> newlib.  Rather than trashing the current scheme and starting over, it is
> just time to use more of the interface with newlib.  

Yes, but we are talking about *system calls* which are not really part of a
C library (and yes, the system call functions are in libc, but in a Unix
systems they are just a front end to the kernel trap instruction). I assume
many of the newlib "system call" functions are either just do-nothing
functions, or pass things on (like open(), read(), write(), do to the
current libio).


> And questioning .. wouldn't libio need help from file system code to do a
> "chdir"?  THe organization of each directory is a function of the file
> system.  I guess you would at least need some "helper" functions from the
> file system.
> ...
> OK.  Then you agree that chdir() would have to do some file system
> specific stuff to at least validate the remaining path name.

Yes, it would use the filesystem's stat(). There will be other cases like
this. They need to be thought through. It's quite possible just the standard
set of operations will be sufficient, but if not, some "helpers" could be
defined.

BTW in vxWorks you can chdir() to anything you like, but if it's wrong you
don't find out until you open("./something")! Another problem I'd like to
see addressed from the start.

Cheers,

Robin