RTEMS | Backport Chris's libio fix to 6 branch (#5311)

[Sebastian Huber (@sebhub)]

Sebastian Huber commented: https://gitlab.rtems.org/rtems/rtos/rtems/-/issues/5311#note_134878


There are more issues associated with the reference counting change.

1. There is an API change on a release branch (`rtems_libio_fcntl_flags()` was renamed).
2. There are ABI changes (`LIBIO_FLAGS_FREE` and related flags).
3. With reference counting you must never set the count to a specific value. The only valid operations are increments and decrements. This leads to problems:

   ```
   void rtems_libio_free_iop(
     rtems_libio_t *iop
   )
   {
     size_t zero;
   
     rtems_libio_lock();
   
     if ( !rtems_libio_iop_is_free( iop ) ) {
   <- Lets assume someone calls rtems_libio_iop_hold() here and gets a reference count of 1.
       /*
        * Clear the flags. All references should have been dropped.
        */
       _Atomic_Store_uint( &iop->flags, LIBIO_FLAGS_FREE, ATOMIC_ORDER_RELAXED );
   <- This sets the reference count to zero. Lets assume the caller of rtems_libio_iop_hold() notices that the file descriptor is bad. It calls rtems_libio_iop_drop(). The reference count is now -1.
   
       rtems_filesystem_location_free( &iop->pathinfo );
   ```
4. Calling `rtems_filesystem_location_free( &iop->pathinfo )` while owning the libio lock (`rtems_libio_lock()`) may cause lock order reversal issues (deadlock).
5. You should not use snapshots of the flags, actions should be based on state changes. For example:

   ```
   @@ -226,7 +226,10 @@ static int vfcntl(
    
      if (ret >= 0) {
        int err = (*iop->pathinfo.handlers->fcntl_h)( iop, cmd );
   -    if (err) {
   
   <-- Here you started with a valid open file descriptor. The higher level code cannot decide the outcome of the fcntl_h handler.
   
   +    if (err == 0 && !rtems_libio_iop_is_open( iop ) ) {
   +      err = EBADF;
   +    }
   +    if (err != 0) {
          errno = err;
          ret = -1;
        }
   ```
6. I think there is an issue with the recent fixes.
7. My patch doesn't work also.

-- 
View it on GitLab: https://gitlab.rtems.org/rtems/rtos/rtems/-/issues/5311#note_134878
You're receiving this email because of your account on gitlab.rtems.org.


-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.rtems.org/pipermail/bugs/attachments/20251013/d19841b7/attachment.htm>