possible RFS lock inversion bug

[Radu Margarint]

Hello,

I'm hitting a deadlock in our RTEMS 4.10 based system, in what seems
to be a lock inversion issue with the RFS file system driver. Or at
least so it seems from my initial investigation, coming here for
clarification.

Scenario is as follows:

We have two tasks in our system doing file i/o operations, to a single
RFS partition. One task is creating, writing and closing a new file
once a second or so. The other is doing constant looping scan of the
entire file system and stat'ing every file encountered. (This is a
somewhat contrived scenario, but this scenario allowed us to get to a
constantly reproducible issue from a random occurrence).

Symptoms are as follows:

- Within a very short time, the system will dead lock in the file i/o layer.
- The scanning task is locked in a callpath starting at
rtems_filesystem_eval_path_start ending in rtems_bdbuf_anonymous_wait
semaphore-wait on access_waiters.
- The low freq file creation task is locked a callpath starting at
close() through rtems_rfs_rtems_file_close waiting to obtain the
rtems_rfs_rtems_lock semaphore.
- the bdbuff swapout task is just at the top waiting for the event to
wake it up and do something

Digging and tracing through the code what seems to be happening is:

- the file creator task creates files and starts writing to them,
acquiring access locks on the bdbuf buffers.
- the enumeration tasks scans and starts seeing files created, and
wants to stat them
- as part of stat, the filesystem lock handler is called, acquires the RFS lock
- then as part of stat, it wants to load the RFS inode, tries to
acquire the block from bdbuf for read
- the block might be currently the one locked by the other task, so it waits.
- the other task gets to close()'ing the file it has open, which
*WOULD* release its lock on the buffers, and wake up the swapout task
which would them flush and release all the other waiter tasks
- *BUT* the rfs close implementation never gets that far because it's
trying to acquire the global RFS lock which is held by the other task.

So, wondering if my understanding of what might be going on sounds
legitimate. If so, if there are any mitigation strategies we could
employ work around it (aside from don't do file i/o from multiple
tasks).

Thanks
--radu