More heap corruption / allocator lock problems

[Paul Neelands]

-----Original Message-----
From: Till Straumann [mailto:strauman at SLAC.Stanford.EDU]
Sent: Thursday, October 16, 2003 6:29 PM
To: pauln at spectranet.ca
Cc: 'rtems-users'
Subject: Re: More heap corruption / allocator lock problems


>Paul Neelands wrote:
>> Regarding solution b)
>>
>> If the special "gc" thread approach is used, how does one insure that it
is
>> run at an appropriate time?
>-----------
>What do you mean by "appropriate time"? The 'gc' thread approach guarantees
>that 'free' is executed from a regular thread context with
>_Thread_Dispatch_disable_level == 0 and interrupts enabled. Thus, no heap
>corruption can occur and provided that enough CPU time is available for
>scheduling the 'gc' task, it is also ensured that the memory is eventually
>released.
---------------

Insufficient cpu time is precisely the problem that moving it to the
malloc/free chain avoids.
Moving it to the idle task is equally problematical. At least one other RTOS
requires
that the idle task always get scheduling time to do precisely this.
(See below)
-----------------------------
>> i.e. It may not free memory more quickly unless
>> the "gc" thread is highest priority,

>If you are concerned abouth how quickly memory is effectively released
>(due to the delay that must be introduced because the allocator lock
>must not be taken from a dispatch-disabled code section):

>  a) you are right that this is affected by the priority of the 'gc' thread
>     (for most applications this priority is probably very low - any spare
>     CPU cycles are good enough)

>> which ends up being effectively the
>> same as the list approach but done via a thread and it's overhead.

>>  b) this is actually not true. Note that the 'real' free() always must
>>     be called from a thread context. Joel's solution delays the 'real'
>>     free until some other thread uses malloc/free which may result in
>>     the memory being released even later than when using a 'gc' thread
>>     at the lowest priority (but it shouldn't matter because noone needs
>>     it anyways).

>The only objection I have to Joel's proposal is that it imposes
>a burden on an 'innocent' task - something I have strong feelings
>about in the context of a real-time OS. Consider the following scenario:

>Task A has a high priority and is willing to incur the the deterministic
>latency of executing one call to 'free()'

>Task B destroys 10000 task variables - they all end up on the 'gc' list.

>Task A is scheduled to run and executes what the designer believed to
>be one call to 'malloc()' - however, it will end up executing 10000 times
>'parasitic' free()s from the 'gc' list.

>> Presumably the malloc/free list approach solves this problem and
eliminates
>> the additional interface complexity of solution a)
>>

>There is no interface complexity. The interface doesn't change at all.


-- Till
---------------------

A.	 The interface complexity I was referring to was the special alloc/free
	mentioned in:
	>Till:
	> a) memory that is going to be released by a task-variable
	>    destructor must be allocated from the workspace.
	>Joel:
	>I like this solution and it is where I was heading.  I was thinking of
	>adding some
	>"name wrapper macros" that are like rtems_extension_malloc and
	>rtems_extension_free
	>which are specifically to be used ONLY from the task create and delete
	>extensions.
	from an earlier message. (The perils of not quoting.)

B.	I agree that your pathological case is a problem. On the other hand,
	a case where say several tasks high priority tasks among themselves are
	saturating the cpu has the opposite pathology. The 'freeing task' is
	never run.
	In effect a higher priority 'freeing task' must steal some cpu cycles
	from some task to free the space, thus leading to at least some
	indeterminacy. In this case, presumably task 'A' of your example will
	be delayed while the 1000 parasitic free's occur. If task 'A' on the
	other hand is higher priority than the 'freeing task', it could well
	run out of memory	during the malloc.

C.	In this case, one could presumably have a task do the freeing, but allow
	malloc to do some freeing if it runs out of room. (Additional complexity)

D.	I agree with your later comments about dtor's.

Paul Neelands

-------------------------
> Paul Neelands
>
> -----Original Message-----
> From: Joel Sherrill [mailto:joel.sherrill at OARcorp.com]
> ...
>
>>b) free could be enhanced:
>>
>>   if 'free' is called while _Thread_Dispatch_disable_level > 0,
>>   it sends its pointer argument to a special "gc" thread which
>>   will eventually execute the real 'free'. The synchronization
>>   mechanism *must* be nonblocking, of course, to avoid the
>>   same kind of problem...
>>
>>While I feel that b) is 'hackish', it could certainly provide
>>backwards compatibility...
>
>
> Another simpler approach to this might be to have a list like you suggest
> but check that it is non-empty on each malloc or free when
> _Thread_Dispatch_disable_level
> is == 0.  It doesn't free memory as quickly as a gc thread would but it
> would get it into
> the heap when it matters.
>
>
>>RFC
>>
>>-- Till
>
>
>
>
>
>
>