Possible thread synchronisation bug in rtems networking

[Graham Darnell]

Hi,

I am concerned that there is a thread synchronisation problem in the 
rtems networking code.

All calls to rtems socket functions are protected by a single mutex, but 
in order to prevent the networking stack from seizing up in the presence 
of long-running calls, there are two mechanisms in place. One of these 
deals with blocking reads, the other deals with functions like connect, 
accept, and close (in the presence of SO_LINGER), which need to wait for 
a connection or disconnection.

It is this second mechanism that concerns me.

It consists of two functions:

soconnsleep: this function is called in connect, accept, and close in 
the context of a client thread of arbitrary priority in order to wait 
for the connection state to change.
soconnwakeup: this function is called in the context of a high priority 
networking thread when the connection state changes

The idea is that soconnsleep releases the global networking mutex then 
waits on an rtems event which is produced some time later when 
soconnwakeup is called. soconnsleep then reacquires the global 
networking mutex and carries on as normal.

The source code is reproduced below.

My concern is with the call to rtems_event_receive() marked with the 
comment 'Soak up any pending events'. And my fear is that this will, 
under unfavourable thread scheduling conditions, soak up not only the 
pending events but also the actual connection/disconnection event that 
we are waiting for. The result being that we never return from 
soconnsleep and the application thread is largely toast.

What I am really seeking is some confirmation that my fears are correct 
from someone who knows this code better than me.

I do have a couple of proposals for a solution, but I guess that can 
wait until my fears have been verified or refuted.

Best regards
Graham Darnell

/*
 * Wait for a connection/disconnection event.
 */
*int*
soconnsleep (*struct* socket *so)
{
        rtems_event_set events;
        rtems_id tid;
        rtems_status_code sc;

        /*
         * Soak up any pending events.
         * The sleep/wakeup synchronization in the FreeBSD
         * kernel has no memory.
         */
        rtems_event_receive (SOSLEEP_EVENT, RTEMS_EVENT_ANY | 
RTEMS_NO_WAIT, RTEMS_NO_TIMEOUT, &events);

        /*
         * Set this task as the target of the wakeup operation.
         */
       * if* (so->so_pgid)
                rtems_panic ("Another task is already sleeping on that 
socket");
        rtems_task_ident (RTEMS_SELF, 0, &tid);
        so->so_pgid = tid;

        /*
         * Wait for the wakeup event.
         */
        sc = rtems_bsdnet_event_receive (SOSLEEP_EVENT, RTEMS_EVENT_ANY 
| RTEMS_WAIT, so->so_rcv.sb_timeo, &events);

        /*
         * Relinquish ownership of the socket.
         */
        so->so_pgid = 0;

       * switch* (sc) {
       * case* RTEMS_SUCCESSFUL: * return* 0;
       * case* RTEMS_TIMEOUT:    * return* EWOULDBLOCK;
       * default*:               * return* ENXIO;
        }
}

/*
 * Wake up a task waiting for a connection/disconnection to complete.
 */
*void*
soconnwakeup (*struct* socket *so)
{
       * if* (so->so_pgid)
                rtems_event_send (so->so_pgid, SOSLEEP_EVENT);
}