RTEMS | score: A watchdog service routine can run on a freed object (#5741)
Sebastian Huber (@sebhub)
gitlab at rtems.org
Thu Sep 3 02:01:37 UTC 2026
Issue created by Sebastian Huber: https://gitlab.rtems.org/rtems/rtos/rtems/-/work_items/5741
Assignee: Sebastian Huber
## Summary
`_Watchdog_Do_tickle()` marks a watchdog inactive and releases the lock of the
header before it calls the service routine. A party which frees the object of
that watchdog in this window leaves the routine on freed memory. The watchdog
has no state which tells a caller that a service routine runs. No remove path
and no delete directive can therefore wait for it.
Five object classes of the operating system own a watchdog and can be freed.
Four of them are open to this order.
## Affected versions
The window and the set of watchdog states are the same in 5.1, 5.2, 5.3, 6.1,
6.2 and main. The line numbers below are those of main at commit 3fc18b536e.
## The window
`cpukit/score/src/watchdogtick.c:56`
```c
do {
if ( first->expire <= now ) {
Watchdog_Service_routine_entry routine;
_Watchdog_Next_first( header, first );
_RBTree_Extract( &header->Watchdogs, &first->Node.RBTree );
_Watchdog_Set_state( first, WATCHDOG_INACTIVE );
routine = first->routine;
_ISR_lock_Release_and_ISR_enable( lock, lock_context );
( *routine )( first );
_ISR_lock_ISR_disable_and_acquire( lock, lock_context );
```
The lock of the header is free from the release to the reacquire. Another
processor runs in that span. The state of the watchdog is `WATCHDOG_INACTIVE`
throughout it.
## No caller can see the routine
`cpukit/include/rtems/score/watchdogimpl.h:63`
```c
typedef enum {
WATCHDOG_SCHEDULED_BLACK,
WATCHDOG_SCHEDULED_RED,
WATCHDOG_INACTIVE,
WATCHDOG_PENDING
} Watchdog_State;
```
There is no state for a watchdog under service. A caller which reads
`WATCHDOG_INACTIVE` cannot tell a watchdog which never ran from one whose
routine is in flight.
`_Timer_Cancel()` at `cpukit/rtems/src/timercreate.c:180` shows the pattern. It
removes the watchdog when the state is scheduled. It takes the watchdog off the
chain of the timer server when the state is pending. It does nothing when the
state is inactive. `rtems_timer_delete()` frees the object right after that
call.
## The affected objects
| Owner | Freed by | Service routine | Open |
| ----- | -------- | --------------- | ---- |
| `Timer_Control.Ticker` | `rtems_timer_delete()` | `_Timer_Routine_adaptor()` | yes |
| `POSIX_Timer_Control.Timer` | `timer_delete()` | `_POSIX_Timer_TSR()` | yes |
| `Thread_Control` timer | `_Thread_Free()` | `_Thread_Timeout()` | yes |
| `POSIX_API_Control.Sporadic.Timer` | `_Thread_Free()` | `_POSIX_Threads_Sporadic_timer()` | yes |
| `Rate_monotonic_Control.Timer` | `rtems_rate_monotonic_delete()` | `_Rate_monotonic_Timeout()` | yes |
| `Record_Control.Watchdog` | never, one per processor | none | no |
## What each routine does to the freed object
### rtems_timer_delete()
`cpukit/rtems/src/timerdelete.c:57` cancels the timer and line 59 frees the
object. `_Timer_Routine_adaptor()` at `cpukit/rtems/src/timercreate.c:62`
writes `the_timer->stop_time` and then runs line 71.
```c
( *the_timer->routine )( the_timer->Object.id, the_timer->user_data );
```
It reads the function pointer and both arguments out of the freed object. A
`Timer_Control` which the allocator gave away carries the routine of another
timer.
### timer_delete()
`cpukit/posix/src/psxtimerdelete.c:74` removes the watchdog and line 79 frees
the object. `_POSIX_Timer_TSR()` at `cpukit/posix/src/timersettime.c:74` writes
`ptimer->overrun`. A periodic timer reaches `_POSIX_Timer_Insert()` at line 92,
which puts the node of the freed object back into the tree of the processor.
Line 105 then runs.
```c
if ( pthread_kill( ptimer->thread_id, ptimer->inf.sigev_signo ) ) {
```
Both arguments come out of the freed object.
### Task delete
`_Thread_Make_zombie()` calls `_Thread_Timer_remove_and_continue()` at
`cpukit/score/src/threadrestart.c:135`. That call meets the window and removes
nothing. `_Thread_Kill_zombies()` frees the thread at line 184. It waits at
line 155 for the thread to leave its processor. It waits for no service
routine. `_Thread_Timeout()` then reads and writes the wait flags of a freed
thread and can unblock it.
The timer of a thread carries every timed wait of the operating system, so this
case has the widest reach. The sporadic server timer of a POSIX thread sits in
the same storage. See `cpukit/include/rtems/posix/threadsup.h:77`.
### rtems_rate_monotonic_delete()
`cpukit/rtems/src/ratemondelete.c:56` cancels the period and line 57 frees the
object. The directive tests no owner, so every task may call it.
`_Rate_monotonic_Timeout()` reads the owner of the period out of the freed
object, takes the lock of that object and can end a period wait of the thread
which it names.
## Reproducer
The test builds the order for the rate monotonic period on one
processor. The link wraps `_Rate_monotonic_Timeout()`. The wrapper keeps the
watchdog, wakes a master task and returns without a call of the real routine,
which leaves the state of the window as it is. The master then carries out the
work of the window in task context and calls the real routine itself.
1. Task A creates a period and calls `rtems_rate_monotonic_period()` once. The
call arms the timer and returns, so the window opens without a wait of A.
2. The clock tick interrupt reaches the wrapper.
3. The master calls `rtems_rate_monotonic_delete()`.
4. Task B calls `rtems_rate_monotonic_create()`. The configuration holds one
period, so the allocator gives B the object which the delete freed. B enters
its period and waits on it.
5. The master calls the real routine.
The test reports eight rounds out of eight in which the object address of B
matched the object address of A, and eight rounds in which the routine ended
the period wait of B. It reproduces on `sparc/gr740` with and without
`RTEMS_SMP` and with and without `RTEMS_DEBUG`.
A real system reaches the order on an SMP configuration. The tickle runs in the
clock tick interrupt, and every delete directive above obtains the object
allocator mutex, which no interrupt handler may do. The delete therefore runs
on another processor while the routine is in flight.
## Suggested repair
A per object counter does not close this. Every one of the routines above would
read the counter out of the object which it must not touch.
The repair belongs to the watchdog. `_Watchdog_Do_tickle()` publishes the
watchdog which it is about to service. A caller which is about to free the
owner waits until that publication clears. All five delete paths run in task
context under the object allocator lock, so they can afford to wait.
The ordinary remove paths must not wait. `_Thread_Timer_remove()` runs from
interrupt context, and an interrupt which preempted the routine on its own
processor would spin without an end. A separate call for the free paths keeps
the two apart.
## A second defect of the same window
The same window produces a defect which does not need a free. A party which
ends the wait of a thread in the window leaves the routine to act on the next
wait of that thread. The thread then reports `STATUS_TIMEOUT` for a wait which
did not time out, or the watchdog of a new timed wait stays in the tree and the
next insert of that node breaks the red-black tree of the processor. This
report does not cover that defect. See also #5733.
--
View it on GitLab: https://gitlab.rtems.org/rtems/rtos/rtems/-/work_items/5741
You're receiving this email because of your account on gitlab.rtems.org. Unsubscribe from this thread: https://gitlab.rtems.org/-/sent_notifications/5-97n8yiebb9w8eu8wtx99z8fcy-1d/unsubscribe | Manage all notifications: https://gitlab.rtems.org/-/profile/notifications | Help: https://gitlab.rtems.org/help
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.rtems.org/pipermail/bugs/attachments/20260903/bf7462a8/attachment-0001.htm>
More information about the bugs
mailing list