RTEMS | record: no uptime anchors in secondary processor rings with CLOCK_DRIVER_USE_ONLY_BOOT_PROCESSOR (#5671)

[Zebulun Barnett (@zbarnett)]

Issue created by Zebulun Barnett: https://gitlab.rtems.org/rtems/rtos/rtems/-/work_items/5671



Disclaimer: I got Claude to write this out so I didn't have to type so much but I did read through and everything looks correct to me.

## Summary

In SMP configurations built with `CLOCK_DRIVER_USE_ONLY_BOOT_PROCESSOR`, the record subsystem produces all `RTEMS_RECORD_UPTIME_LOW`/`RTEMS_RECORD_UPTIME_HIGH` pairs into the boot processor's ring. The rings of secondary processors never receive uptime anchors. Tools that rely on per-ring anchors to associate the 22-bit event timestamps with absolute time (rtems-record-client and everything downstream) therefore cannot resolve timestamps for events of secondary processors: the client's hold-back state for those rings never clears, and captures come out with unusable timestamps for CPUs \> 0. As far as I can tell from the history (below), this combination has never worked.

## Mechanism

`_Record_Watchdog()` in `cpukit/libtrace/record/record-sysinit.c` re-arms its watchdog on the _owning_ processor but produces the uptime events on the _executing_ processor:

```c
_Watchdog_Per_CPU_insert_ticks(
  watchdog,
  _Watchdog_Get_CPU( watchdog ),                  /* owner CPU */
  _Record_Tick_interval
);
now = _Timecounter_Sbinuptime();
rtems_record_prepare_critical( &context, _Per_CPU_Get() );  /* executing CPU */
```

With `CLOCK_DRIVER_USE_ONLY_BOOT_PROCESSOR`, `Clock_driver_timecounter_tick()` in `bsps/shared/dev/clock/clockimpl.h` has the boot processor service the watchdog headers of all online processors:

```c
} else if ( _Processor_mask_Is_set( _SMP_Get_online_processors(), cpu_index ) ) {
  _Watchdog_Tick( cpu );
}
```

So for every secondary processor's record watchdog, executor != owner: the routine runs on the boot processor, `_Per_CPU_Get()` yields the boot processor, and the anchors land in ring 0. The owner-based re-arm keeps ownership intact, so this persists for the lifetime of the system. This is the only cross-processor `_Watchdog_Tick()` call site in the tree; in the default configuration each processor runs `_Timecounter_Tick()` itself, executor == owner always holds, and the record subsystem behaves as intended.

Setting `CLOCK_DRIVER_USE_ONLY_BOOT_PROCESSOR = False` restores per-ring anchors and correct client output.

## History

* b61d5cac7c (2016-06-14) "bsps: Add CLOCK_DRIVER_USE_ONLY_BOOT_PROCESSOR" — clock tick processing on the boot processor on behalf of all other processors, introduced as a workaround for a QEMU shortcoming on ARM (#2737). It has since become a regular build option and is hardwired by some BSPs.
* dca618404e (2018-04-28) "Add low level event recording support" — `_Record_Watchdog()` already has its current owner/executor shape in this commit. The incompatibility has existed since then.

## Possible fix

When `_Per_CPU_Get() != _Watchdog_Get_CPU( watchdog )`, defer the production to the owner via `_Per_CPU_Submit_job()`: a `Per_CPU_Job` embedded in `Record_Control`, re-initialized each period, with submission skipped if the previous job has not completed. Cost is one IPI per secondary processor per anchor interval, which seems negligible. Alternatives look worse: producing into the remote ring violates the single-producer invariant; adding the processor index to the uptime events churns the protocol and clients and is only sound where the counter is system-global; handling it purely in the client cannot restore the per-ring at-least-one-item-per-2^22-cycles property that bounds timestamp wrap tracking.

@sebhub 

<!--Pre-set options
- milestone-->

-- 
View it on GitLab: https://gitlab.rtems.org/rtems/rtos/rtems/-/work_items/5671
You're receiving this email because of your account on gitlab.rtems.org. Unsubscribe from this thread: https://gitlab.rtems.org/-/sent_notifications/4-7dm7rjt5qt2nyz5i9dzykgl8b-1d/unsubscribe | Manage all notifications: https://gitlab.rtems.org/-/profile/notifications | Help: https://gitlab.rtems.org/help


-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.rtems.org/pipermail/bugs/attachments/20260729/4de6eb0e/attachment-0001.htm>