<div dir="ltr"><div><div>For every mutex held by a thread it is not feasible to have reference to CORE_mutex_order_list of all mutex that it holds.<b> I feel that current solution with CORE_mutex_order_list as argument is the best choice to have as it is O(1) operation and needs no extra bookkeeping for the thread.<br></b><br>In JPF model we have it as follows:<br> <span class="pl-k">public</span> <span class="pl-k">List<<span class="pl-smi">Mutex</span>></span> mutexOrderList; <span class="pl-c">//it is a linkedList which stores acquired mutex objects in LIFO order.<br><br></span></div><span class="pl-c">and to extract the index of mutex we call :<br></span><br><span class="pl-k">public</span> <span class="pl-k">int</span> <span class="pl-en">getMutexIndex</span>(<span class="pl-smi">Mutex</span> <span class="pl-v">obj</span>)<br>{<br> <span class="pl-k">return</span> mutexOrderList<span class="pl-k">.</span>indexOf(obj);<br>}<br><br></div><div>Lets see it through example:<br><br></div><div>Holder thread: H <br>Mutex being acquired by executing thread which is acquired by holder thread(H): M</div><div>Executing thread: T<br><br></div><div>From rtems we have following information:<br><br></div><div>1. In TCB:<br></div><div>Every thread maintains a list of mutex acquired by it through out its course in doubly circular linked list fashion :<br></div><div><br>#ifdef __RTEMS_STRICT_ORDER_MUTEX__<br> /** This field is the head of queue of priority inheritance mutex<br> * held by the thread.<br> */<br> Chain_Control lock_mutex;<br>#endif<br><br>thread->lock_mutex<br><br></div><div>2. In Mutex Control Block:<br><br></div><div>typedef struct{<br> /** This field is a chian of locked mutex by a thread,new mutex will<br> * be added to the head of queue, and the mutex which will be released<br> * must be the head of queue.<br> */<br> Chain_Node lock_queue;<br> /** This field is the priority of thread before locking this mutex<br> *<br> */<br> Priority_Control priority_before;<br> } CORE_mutex_order_list;<br></div><div><br>Mutex->queue <CORE_mutex_order_list><br><br></div><div>When a thread acquires a mutex the mutex->queue.lock_queue node is prepended to the holder's H->lock_mutex.<br></div><div>This way we can traverse through all mutex's acquired by holder thread. <br><br></div><div>Now when executing thread tries to acquire the mutex M which is acquired by the holder thread(H) it wants to raise the recorded priority before of all mutex's acquired by the holder after acquiring mutex M. So the best way to do it in O(1) operation and without storing any reference we do it in following manner in new solution: <br><br>while(check!=head)<br> {<br> queue = RTEMS_CONTAINER_OF(check, CORE_mutex_order_list, lock_queue);<br> if(!(queue->priority_before > new_priority))<br> {<br> return PRIORITY_STATUS_NOT_CHANGED;<br> }<br> queue->priority_before = new_priority;<br> check = check->previous;<br> }<br><br></div><div>This is the most efficient way we can do that. We don't need to change any design for this nor need to do any bookkeeping thing. <br></div></div><div class="gmail_extra"><br clear="all"><div><div class="gmail_signature"><div dir="ltr">Thanks,<div><br></div><div>Saurabh Gadia</div></div></div></div>
<br><div class="gmail_quote">On Sun, Aug 16, 2015 at 6:48 PM, Gedare Bloom <span dir="ltr"><<a href="mailto:gedare@rtems.org" target="_blank">gedare@rtems.org</a>></span> wrote:<br><blockquote class="gmail_quote" style="margin:0 0 0 .8ex;border-left:1px #ccc solid;padding-left:1ex">We will rely on the linker should not pull the function in if unused.<br>
So if it is only referenced by test code, this solution may be fine.<br>
<span class="HOEnZb"><font color="#888888"><br>
Gedare<br>
</font></span><div class="HOEnZb"><div class="h5"><br>
On Sun, Aug 16, 2015 at 7:50 PM, Cyrille Artho <<a href="mailto:cyrille.artho@gmail.com">cyrille.artho@gmail.com</a>> wrote:<br>
> Hi all,<br>
> As I wrote earlier, I think it makes sense to include validate but<br>
> compile it conditionally. We want to use it for regression testing but<br>
> not in deployed code.<br>
> Which macros do you use for this purpose?<br>
><br>
> On Sat, Aug 15, 2015 at 2:50 PM, Cyrille Artho <<a href="mailto:cyrille.artho@gmail.com">cyrille.artho@gmail.com</a>> wrote:<br>
>> I will look at the code in detail on Monday, but we should keep in<br>
>> mind that validate is only needed for testing. It does not have to be<br>
>> compiled into the final code otherwise.<br>
>> So we do not have to be overly concerned with its performance but<br>
>> instead we have to make sure that validate itself is not subject to<br>
>> data races.<br>
>><br>
>> On Sat, Aug 15, 2015 at 5:05 AM, Saurabh Gadia <<a href="mailto:gadia@usc.edu">gadia@usc.edu</a>> wrote:<br>
>>> Hi,<br>
>>><br>
>>> I have implemented the validate method. following are the links for it:<br>
>>> github: <a href="https://github.com/saurabhgadia4/rtems/tree/nested-mutex" rel="noreferrer" target="_blank">https://github.com/saurabhgadia4/rtems/tree/nested-mutex</a><br>
>>> commit:<br>
>>> <a href="https://github.com/saurabhgadia4/rtems/commit/e7f0f169c056076c46ef5ea17b0c38efe33fe576" rel="noreferrer" target="_blank">https://github.com/saurabhgadia4/rtems/commit/e7f0f169c056076c46ef5ea17b0c38efe33fe576</a><br>
>>><br>
>>> I am waiting on the decision of how to integrate call to this<br>
>>> _Thread_Validate_Priority.<br>
>>><br>
>>> Thanks,<br>
>>><br>
>>> Saurabh Gadia<br>
>>><br>
>>> On Fri, Aug 14, 2015 at 10:06 AM, Saurabh Gadia <<a href="mailto:gadia@usc.edu">gadia@usc.edu</a>> wrote:<br>
>>>><br>
>>>> And in respect of efficiency, we have to traverse through all the mutex<br>
>>>> held my the thread and do the checking. There is no other way to confirm<br>
>>>> that priority inversion has occurred or not. One way we can do is have<br>
>>>> default argument variable for core_mutex_surrender but then there is change<br>
>>>> in API call semantics. So kind of stuck between which way to take.<br>
>>>><br>
>>>> One way is that we can do lazy evaluation. following should be the calling<br>
>>>> pair:<br>
>>>> task1()<br>
>>>> {<br>
>>>> ...<br>
>>>> ...<br>
>>>> rtems_semaphore_release()<br>
>>>> validate()<br>
>>>> }<br>
>>>> these pairs should be intact so even if the task1 thread gets preempted<br>
>>>> after calling rtems_semaphore_release(), then other thread will get control<br>
>>>> of processor. When this task1 thread get the control back then next call it<br>
>>>> will do is validate() which is no harm to us as only task 1 thread can<br>
>>>> release rest of its resources of we have owner release binding. But there<br>
>>>> can be one problem that is Till the task 1 thread get the control back its<br>
>>>> priority may be promoted by other task thread. So it won't be 100% accurate<br>
>>>> validate test.<br>
>>>><br>
>>>> Main problem still exists is: For uniprocessor(for this project scope)<br>
>>>> implementation how can we make sure that only after validate method task1<br>
>>>> can be preempted ( To acheive this behavior I guess we will need to make<br>
>>>> change to core_mutex_surrender).<br>
>>>><br>
>>>> Thanks,<br>
>>>><br>
>>>> Saurabh Gadia<br>
>>>><br>
>>>> On Fri, Aug 14, 2015 at 9:43 AM, Saurabh Gadia <<a href="mailto:gadia@usc.edu">gadia@usc.edu</a>> wrote:<br>
>>>>><br>
>>>>> When a thread releases a semaphore/mutex we call this validate method to<br>
>>>>> make sure that there does not exists any priority inversion. Following is<br>
>>>>> the course of action that needs to be performed:<br>
>>>>><br>
>>>>> 1. Validate method needs to be called within mutex_surrender method<br>
>>>>> because after releasing a mutex a new holder thread can get scheduled and<br>
>>>>> then we can't call validate method. We need to do call validate before we<br>
>>>>> enable interrupts in uniprocessor or dispatching of threads.<br>
>>>>><br>
>>>>> 2. Functioning of validate method: input param - executing thread<br>
>>>>> (thread which releases the mutex)<br>
>>>>><br>
>>>>> 2.a) Go through the list of Mutex Chain i.e( Chain_Control lock_mutex;)<br>
>>>>> in TCB.<br>
>>>>> 2.b) Extract mutex from the chain node linked list. This gives us<br>
>>>>> (the_mutex->queue.lock_queue) from which we will extract mutex by using<br>
>>>>> CONTAINER_OF() method.<br>
>>>>><br>
>>>>> 2.c) Now access the thread wait queue of this mutex<br>
>>>>> i.e(the_mutex->Wait_queue) to get the information about the first waiting<br>
>>>>> thread in this waitqueue by calling<br>
>>>>> _Thread_queue_First_locked(&the_mutex->Wait_queue)<br>
>>>>><br>
>>>>> 2.d) Now check whether<br>
>>>>> assert(releasingThread.current_priority<=first_locked_thread.currentPriority).<br>
>>>>> If assertion fails then there is priority inversion problem and we can break<br>
>>>>> out of loop and return error status.<br>
>>>>><br>
>>>>> 2.e) Repeat from 2.a till all acquired resources are visited present in<br>
>>>>> Chain_Control of releasing thread.<br>
>>>>><br>
>>>>> So I am sceptical about how can I include this validate method in the<br>
>>>>> _CORE_mutex_Surrender(). We can have it as a separate call from user level<br>
>>>>> but then we need to make sure that the thread dispatch mechanism is<br>
>>>>> disabled. If not then whether including this validate method in<br>
>>>>> _CORE_mutex_Surrender for only strict_order_mutex and Priority inheritance<br>
>>>>> attribute is feasible or not.<br>
>>>>><br>
>>>>> Please guide me on this.<br>
>>>>><br>
>>>>><br>
>>>>> Thanks,<br>
>>>>><br>
>>>>> Saurabh Gadia<br>
>>>>><br>
>>>>> On Thu, Aug 13, 2015 at 9:10 AM, Saurabh Gadia <<a href="mailto:gadia@usc.edu">gadia@usc.edu</a>> wrote:<br>
>>>>>><br>
>>>>>> That is how we were doing in JPF. The validate method was triggered<br>
>>>>>> after every release of mutex by any thread and we would check for all the<br>
>>>>>> waiting threads on mutex's held by the holder. And if it finds a thread with<br>
>>>>>> higher priority blocked then it would panic or give assertion failure.<br>
>>>>>><br>
>>>>>> Thanks,<br>
>>>>>><br>
>>>>>> Saurabh Gadia<br>
>>>>>><br>
>>>>>> On Thu, Aug 13, 2015 at 9:08 AM, Gedare Bloom <<a href="mailto:gedare@rtems.org">gedare@rtems.org</a>> wrote:<br>
>>>>>>><br>
>>>>>>> Thanks. Would it be possible for you to turn the failure cases into<br>
>>>>>>> real test failures? In other words, add some logic to detect the<br>
>>>>>>> priority inversion and abort the test?<br>
>>>>>>><br>
>>>>>>> Gedare<br>
>>>>>>><br>
>>>>>>> On Thu, Aug 13, 2015 at 12:05 PM, Saurabh Gadia <<a href="mailto:gadia@usc.edu">gadia@usc.edu</a>> wrote:<br>
>>>>>>> > Hi,<br>
>>>>>>> ><br>
>>>>>>> > Following is the result of spsem04 test without the patch:<br>
>>>>>>> ><br>
>>>>>>> > *** BEGIN OF TEST SPSEM 4 ***<br>
>>>>>>> ><br>
>>>>>>> > init: S0 created<br>
>>>>>>> ><br>
>>>>>>> > init: S1 created<br>
>>>>>>> ><br>
>>>>>>> > init: TA01 created with priority 36<br>
>>>>>>> ><br>
>>>>>>> > init: TA02 created with priority 34<br>
>>>>>>> ><br>
>>>>>>> > init: TA03 created with priority 32<br>
>>>>>>> ><br>
>>>>>>> > TA01: started with priority 36<br>
>>>>>>> ><br>
>>>>>>> > TA01: priority 36, holding S0<br>
>>>>>>> ><br>
>>>>>>> > TA02: started with priority 34<br>
>>>>>>> ><br>
>>>>>>> > TA02: priority 34, holding S1<br>
>>>>>>> ><br>
>>>>>>> > TA01: priority 34, holding S0 and now creating TA03<br>
>>>>>>> ><br>
>>>>>>> > TA03: started with priority 32<br>
>>>>>>> ><br>
>>>>>>> > TA01: priority 34 Releasing s0 (This is priority inheritance problem<br>
>>>>>>> > as TA02<br>
>>>>>>> > waits on mutex held by TA01 and has higher priority than TA01. TA03<br>
>>>>>>> > just<br>
>>>>>>> > promotes the priority TA02.)<br>
>>>>>>> ><br>
>>>>>>> > TA02: priority 32, holding S1,S0<br>
>>>>>>> ><br>
>>>>>>> > TA02: priority 32 before releasing S0<br>
>>>>>>> ><br>
>>>>>>> > TA02: priority 32 after releasing S0<br>
>>>>>>> ><br>
>>>>>>> > TA02: priority 32 before releasing S1<br>
>>>>>>> ><br>
>>>>>>> > TA03: priority 32, holding S1<br>
>>>>>>> ><br>
>>>>>>> > TA03: priority 32<br>
>>>>>>> ><br>
>>>>>>> > TA03: suspending<br>
>>>>>>> ><br>
>>>>>>> > TA02: priority 34 after releasing S1<br>
>>>>>>> ><br>
>>>>>>> > TA02: suspending<br>
>>>>>>> ><br>
>>>>>>> > TA01: priority 36<br>
>>>>>>> ><br>
>>>>>>> > TA01: exiting<br>
>>>>>>> ><br>
>>>>>>> > *** END OF TEST SPSEM 4 ***<br>
>>>>>>> ><br>
>>>>>>> > You can see the difference in highlighted portions of both outputs.<br>
>>>>>>> ><br>
>>>>>>> > Thanks,<br>
>>>>>>> ><br>
>>>>>>> > Saurabh Gadia<br>
>>>>>>> ><br>
>>>>>>> > On Thu, Aug 13, 2015 at 8:17 AM, Saurabh Gadia <<a href="mailto:gadia@usc.edu">gadia@usc.edu</a>> wrote:<br>
>>>>>>> >><br>
>>>>>>> >> Ok. I will mail you back soon.<br>
>>>>>>> >><br>
>>>>>>> >><br>
>>>>>>> >> On Thursday, August 13, 2015, Gedare Bloom <<a href="mailto:gedare@rtems.org">gedare@rtems.org</a>> wrote:<br>
>>>>>>> >>><br>
>>>>>>> >>> Saurabh,<br>
>>>>>>> >>><br>
>>>>>>> >>> Please pull from rtems.git again, create a new branch from<br>
>>>>>>> >>> 'master',<br>
>>>>>>> >>> and apply your changes to the branch. It's too hard to review code<br>
>>>>>>> >>> that is not all by itself on a branch.<br>
>>>>>>> >>><br>
>>>>>>> >>> Gedare<br>
>>>>>>> >>><br>
>>>>>>> >>> On Thu, Aug 13, 2015 at 5:28 AM, Saurabh Gadia <<a href="mailto:gadia@usc.edu">gadia@usc.edu</a>><br>
>>>>>>> >>> wrote:<br>
>>>>>>> >>> > Hi,<br>
>>>>>>> >>> ><br>
>>>>>>> >>> > I have implemented uniprocessor model of nested mutex problem in<br>
>>>>>>> >>> > rtems.<br>
>>>>>>> >>> > its<br>
>>>>>>> >>> > still in basic form. I tried to multiplex it with the existing<br>
>>>>>>> >>> > solution<br>
>>>>>>> >>> > but<br>
>>>>>>> >>> > was finding hard time. To push ahead, I decided to have separate<br>
>>>>>>> >>> > functions<br>
>>>>>>> >>> > for uniprocessor and SMP(kept default behavior) and with your<br>
>>>>>>> >>> > review<br>
>>>>>>> >>> > comments will know what to do. Following is the link for the git<br>
>>>>>>> >>> > repo:<br>
>>>>>>> >>> > <a href="https://github.com/saurabhgadia4/rtems/commits/master" rel="noreferrer" target="_blank">https://github.com/saurabhgadia4/rtems/commits/master</a> and its JPF<br>
>>>>>>> >>> > branch:<br>
>>>>>>> >>> ><br>
>>>>>>> >>> ><br>
>>>>>>> >>> > <a href="https://github.com/saurabhgadia4/lock-model/blob/uniproc-new1/rtems/Mutex.java" rel="noreferrer" target="_blank">https://github.com/saurabhgadia4/lock-model/blob/uniproc-new1/rtems/Mutex.java</a><br>
>>>>>>> >>> ><br>
>>>>>>> >>> > I have also tested spsem01, 02, 03 test cases. Following are the<br>
>>>>>>> >>> > links<br>
>>>>>>> >>> > for<br>
>>>>>>> >>> > the test case results which states output before solution and<br>
>>>>>>> >>> > after<br>
>>>>>>> >>> > applying<br>
>>>>>>> >>> > the solution. I am still not getting whether my code is passing<br>
>>>>>>> >>> > spsem03<br>
>>>>>>> >>> > test<br>
>>>>>>> >>> > or not. How can I verify that?<br>
>>>>>>> >>> ><br>
>>>>>>> >>> > Test Case Link:<br>
>>>>>>> >>> ><br>
>>>>>>> >>> ><br>
>>>>>>> >>> > <a href="https://drive.google.com/folderview?id=0B44HRKVuGCkFfnFDVmxqQzZZUzljNUg4YmVPZmEybEp2Q0NNclpvS2FvemZ4Tm5Xa19nemM&usp=sharing" rel="noreferrer" target="_blank">https://drive.google.com/folderview?id=0B44HRKVuGCkFfnFDVmxqQzZZUzljNUg4YmVPZmEybEp2Q0NNclpvS2FvemZ4Tm5Xa19nemM&usp=sharing</a><br>
>>>>>>> >>> ><br>
>>>>>>> >>> > Thanks,<br>
>>>>>>> >>> ><br>
>>>>>>> >>> > Saurabh Gadia<br>
>>>>>>> >><br>
>>>>>>> >><br>
>>>>>>> >><br>
>>>>>>> >> --<br>
>>>>>>> >> Thanks,<br>
>>>>>>> >><br>
>>>>>>> >> Saurabh Gadia<br>
>>>>>>> >><br>
>>>>>>> ><br>
>>>>>><br>
>>>>>><br>
>>>>><br>
>>>><br>
>>><br>
>><br>
>><br>
>><br>
>> --<br>
>> Regards,<br>
>> Cyrille Artho<br>
><br>
><br>
><br>
> --<br>
> Regards,<br>
> Cyrille Artho<br>
</div></div></blockquote></div><br></div>