[GSoC]The overheads caculation of POSIX Key
Ashi
ashi08104 at gmail.com
Tue Aug 28 09:03:05 UTC 2012
2012/8/23 Joel Sherrill <joel.sherrill at oarcorp.com>
> On 08/22/2012 10:20 PM, Ashi wrote:
>
>
>
> 2012/8/22 Joel Sherrill <joel.sherrill at oarcorp.com>
>
>> On 08/21/2012 07:15 PM, Gedare Bloom wrote:
>>
>>
>>
>> On Mon, Aug 20, 2012 at 3:01 AM, Ashi <ashi08104 at gmail.com> wrote:
>>
>>> Hi, all. This post is about the overheads calculation of POSIX Key. We
>>> haven't reach a resolution about what to do with this. Thanks for any
>>> advice!
>>>
>>> One problem of old POSIX Key is: extra memory is reserved in keys for
>>> each thread or task, which can be heavy memory overhead. Now in current
>>> one-rbtree implementation of POSIX Key, each key's data isn't allocated
>>> until one thread(or task) sets its key data. But RTEMS system still need
>>> reserve enough workspace memory for all possible key's data. For example,
>>> there are 32 threads and 1 key in system. And only 1 thread need key data,
>>> then other 31 thread data is not used and reserved. I'm not sure whether it
>>> is necessary to provide user a option to set the upper bound of the key
>>> data number in system. If it is necessary, I think we can add something
>>> like CONFIGURE_MAXIMUM_THREAD_KEY_DATA, it can be from 1 to #max key * #max
>>> thread. If we reach the upper bound, we can return ENOMEM in the key
>>> setspecific function as the POSIX standard[0] allows.
>>>
>>> I think it is a good idea to configure the number of keys independent
>> of the number of threads.
>>
>> I agree. There are two factors:
>>
>> + number of keys
>> + number of threads/tasks using keys
>>
>> For the first, if you don't _Workspace_Allocate as part of creating a
>> key, then it is
>> covered by the sizeof(POSIX_Keys_Control). If you simply changed just
>> that, the
>> existing confdefs.h code does not need to change.
>>
>> Each key calling get or set will have a structure allocated via
>> _Workspace_Allocate.
>> This has to be accounted for.
>>
>> The "per thread using key" is a number which could be < the number of
>> tasks+threads
>> or as much as (number of keys * (tasks + threads)) in the case where
>> EVERY task/thread
>> uses every key.
>>
>> I'm a little confused by the the "number of threads/tasks using keys".
> For an concrete example:
> there are 3 Keys, 5 threads in system. And 2 threads use Keys, let's say
> they are thread1, thread2. Thread1 uses 3 keys, thread2 only uses 1 key.
> Then if we use "number of threads using keys", there are 2 threads using
> keys, and we get the 2 * 3 key instances in system. But actually, there
> are only 3 + 1 key instances in system.
> So I think the "number of threads using keys" can't be used to compute the
> memory overhead. Or maybe I made a mistake..
>
>
> In an earlier thread, I used a phrase like "key/thread pairs"
>
> It occurred to me to ask ... do you allocate all of these during
> _POSIX_Key_Manager_initialization? You could then have an
> RTEMS chain consisting of all of the available "pair instances".
> You would _Workspace_Allocate and then call _Chain_Initialize.
> Allocation at set would be a _Chain_Get. Deallocation would
> be a _Chain_Append. This will have a few advantages:
>
> + one Workspace allocation during system initialization (GOOD!)
> + Slightly easier to compute size in confdefs.h
> + much faster and WAY more deterministic to allocate
> and deallocate than a heap operation.
> + One memory allocation helps reduce fragmentation versus
> multiple allocate/free operations
>
> And finally, doing one allocation of a pool of instances is more
> in the "RTEMS style" of doing things to stay predictable and favor
> operations at system init over object creation over during an
> "normal" operation's run-time
>
> Hi, all, I have tried to implement this pre-allocation idea. It results a
good improvement of keysetspecific operation:). And I have pushed code to
github. However, there are some problems:
1. I haven't figured out how to define correct POSIX KEY configuration in
confdefs.h, and the code can't pass the key test 07 and key test 08, which
are all about unlimited thread/task test. When I run the test, they return
as below respectively:
bootcard: work space too big for work area: 80010722 >= 3DA290
bootcard: work space too big for work area: 8000B612 >= 3DB3B0
And I changed the POSIX Key related configuration as below, and other
changes can be find in this commit(
https://github.com/ashi08104/rtems/commit/96ba51d0269d0d23708e014f6783669d61fc2962).
#ifndef CONFIGURE_MAXIMUM_POSIX_KEYS
#define CONFIGURE_MAXIMUM_POSIX_KEYS 0
#define CONFIGURE_MAXIMUM_POSIX_KEY_PAIRS 0
#define CONFIGURE_MEMORY_FOR_POSIX_KEYS(_keys) 0
#else
#ifndef CONFIGURE_MAXIMUM_POSIX_KEY_PAIRS
#define CONFIGURE_MAXIMUM_POSIX_KEY_PAIRS \
CONFIGURE_MAXIMUM_POSIX_KEYS \
* (CONFIGURE_MAXIMUM_POSIX_THREADS + CONFIGURE_MAXIMUM_TASKS)
#endif
#define CONFIGURE_MEMORY_FOR_POSIX_KEYS(_keys) \
(_Configure_Object_RAM(_keys, sizeof(POSIX_Keys_Control) ) \
+ CONFIGURE_MAXIMUM_POSIX_KEY_PAIRS \
* _Configure_From_workspace(sizeof(POSIX_Keys_Rbtree_node)))
#endif
2. What shall I do when key deleted or thread deleted? Is just appending
the freed key node to the pre-allocation chain enough?
3. Which version of chain operation should I use during the key nodes pool
initialization in _POSIX_Keys_Manager_initialization(), protected chain
operation or unprotected chain operation?
>
>
>>
>>> Another problem: I find there is only maximum keys
>>> configuration:CONFIGURE_MAXIMUM_POSIX_KEYS, but how and where does RTEMS
>>> determine the proper workspace size for all POSIX Key data?
>>>
>>> Probably in cpukit/sapi/include/confdefs.h Check for how the macro is
>> used / propagates in that file.
>>
>>
>> Specifically at this line of code:
>>
>> http://git.rtems.org/rtems/tree/cpukit/sapi/include/confdefs.h#n1703
>>
>> Note that currently it allocates a POSIX_Keys_Control PLUS the table
>> of key values per API.
>>
>> I see.
>
>> Based on our discussion, you would be removing that second factor and
>> adding another CONFIGURE_xxx symbol corresponding to the number of
>> tasks/threads using key instances as discussed above.
>>
>
>> links:
>>> [0]
>>> http://pubs.opengroup.org/onlinepubs/009604499/functions/pthread_setspecific.html
>>> --
>>> Best wishes!
>>> Zhongwei Yao
>>>
>>>
>>
>>
>> --
>> Joel Sherrill, Ph.D. Director of Research& Developmentjoel.sherrill at OARcorp.com On-Line Applications Research
>> Ask me about RTEMS: a free RTOS Huntsville AL 35805
>> Support Available (256) 722-9985
>>
>>
>
>
> --
> Best wishes!
> Zhongwei Yao
>
>
>
> --
> Joel Sherrill, Ph.D. Director of Research& Developmentjoel.sherrill at OARcorp.com On-Line Applications Research
> Ask me about RTEMS: a free RTOS Huntsville AL 35805
> Support Available (256) 722-9985
>
>
--
Best wishes!
Zhongwei Yao
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.rtems.org/pipermail/devel/attachments/20120828/8f9d705e/attachment-0001.html>
More information about the devel
mailing list