RTEMS | cpukit/libdebugger: Prevent hang on memory access (!523)

[Kinsey Moore (@opticron)]

Kinsey Moore created a merge request: https://gitlab.rtems.org/rtems/rtos/rtems/-/merge_requests/523

Project:Branches: opticron/rtems:brmain-debugger-bad-mem-exception to rtems/rtos/rtems:main
Author:   Kinsey Moore
Assignee: Kinsey Moore



## Summary

cpukit/libdebugger: Prevent hang on memory access

When memory is accessed by the remote debugging client, the access is
sandboxed with setjmp/longjmp and appropriate exception handlers to
prevent the attempted access from causing a failure of the debugger or
otherwise altering execution. The existing implementation works as
expected when the context executing the memory access and the exception
context resulting from a failed access do not share a stack.

In the case of AArch64, a failed access when the debugger is already in
exception context causes a re-entry into exception context where the
machine state is pushed onto the same stack that was in use where the
exception occurred. When setjmp is called inside a stack frame and the
exception occurs outside that stack frame, the stack frame is unwound
before the exception occurs and the exception entry overwrites the area
previously occupied by the stack frame housing the setjmp and corrupting
the link register that is stored there. After restoration of state using
longjmp(), this corrupted link register information is loaded from the
stack frame and undesired behavior occurs.

In the instance of this bug that was encountered, the corrupted link
register contained an unaligned pointer which caused an unending cascade
of prefetch abort exceptions presenting as a hard hang.


<!-- Default settings, if it is a dropdown it will set after submission -->

-- 
View it on GitLab: https://gitlab.rtems.org/rtems/rtos/rtems/-/merge_requests/523
You're receiving this email because of your account on gitlab.rtems.org.


-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.rtems.org/pipermail/bugs/attachments/20250618/e8d2ea9b/attachment.htm>