[RTEMS Project] #3449: Workspace corruption due to nested acquisition of API Mutex

[RTEMS trac]

#3449: Workspace corruption due to nested acquisition of API Mutex
-----------------------+--------------------
  Reporter:  strauman  |      Owner:  (none)
      Type:  defect    |     Status:  new
  Priority:  high      |  Milestone:  4.10.3
 Component:  score     |    Version:  4.10
  Severity:  major     |   Keywords:
Blocked By:            |   Blocking:
-----------------------+--------------------
 rtems_task_delete is does most of its work holding the RTEMS allocator
 mutex. This mutex (_RTEMS_Allocator_Mutex) is created by
 _API_Mutex_Allocate() in CORE_MUTEX_NESTING_IS_ERROR mode.

 Note that _API_Mutex_Lock() does *not* check the return status of the
 underlying _Core_mutex_Seize()!

 I found that the allocator mutex *is* actually acquired recursively at
 least from one place: during task deletion when the RTEMS API extension is
 deleted this walks through all task variables and invokes their deletor.

 In general it is unknown and therefore not guaranteed that such a deletor
 might not acquire the allocator mutex -- indeed I found that 'free' which
 is probably the most common deletor protects the
 heap with _RTEMS_Lock_allocator() (i.e., the very allocator mutex)

 Because _API_Mutex_Lock() fails to detect the unsuccessful nested  attempt
 the code proceeds and releases the API mutex prematurely (upon return from
 free(), when the allocator mutex is unlocked).

 The resulting lack of protection during the later steps of
 rtems_task_delete() opens the door for corruption.

--
Ticket URL: <http://devel.rtems.org/ticket/3449>
RTEMS Project <http://www.rtems.org/>
RTEMS Project