Proposed Affinity Changes

[Joel Sherrill]

Hi

Although I think there are only a few code paths to
address for affinity added to the Priority SMP Scheduler,
the modifications appear to be very subtle and I want
to get feedback given the potential impact on other
schedulers.

+ set_affinity
+ _Scheduler_SMP_Enqueue_ordered()
+ _Scheduler_SMP_Extract()
+ _Scheduler_SMP_Schedule()

== set_affinity
set_affinity scheduler support is being discussed in Sebastian's
change priority patch thread. But basically use the pattern
he proposed for _Scheduler_Change_priority() but for affinity.

== _Scheduler_SMP_Extract()
_Scheduler_SMP_Extract() doesn't appear to need anything
done.

==_Scheduler_SMP_Enqueue_ordered()
For the enqueue ordered path, this is what I think needs to be
done. The changes appear to be minor but the interweaving of
indirect calls and rapid changes means I need some feedback.

When a node is in the air and we look for highest ready  in
_Scheduler_SMP_Enqueue_ordered, the highest ready should
have an affinity for this core.  It is does, then it gets returned
by (*get_highest_ready)(). If not, we can return NULL is
returned since we couldn't impact the thread allocated to this
node.

   How do you think this filter should be inserted into the framework
   of indirect calls leading here?

Similarly, if a node is not in the air, we look for lowest scheduled.
I am thinking that the hard-coded call to
_Scheduler_SMP_Get_lowest_scheduled() needs to be an indirect
call so an affinity aware version which scans scheduled to
find lowest with affinity for this node.

And is it safe to assume that node == current processor ID?
So there don't need to be arguments added to the calls?
Just check affinity against current processor number.

==_Scheduler_SMP_Schedule()
This is only called as a side-effect of _Thread_Change_priority().
Sebastian.. is this entry point needed after your changes?

If needed, then...

_Scheduler_SMP_Schedule_highest_ready() already is passed
get_highest_ready().  If we go with what was discussed for
_Scheduler_SMP_Enqueue_ordered() and use the same
get_highest_ready() implementation, then (*get_highest_ready)()
will return a NULL if the highest priority thread does not have
affinity for the core we are scheduling for.  Returning NULL
from get_highest_priority appears to be a bad idea for this
case.

Plus we need a thread with affinity for the core/node that
the priority was changed on so we can't assume that
the processor ID is easily available.

So there are two hard things I don't see an obvious answer to.

I think this path has a decision.

-- 
Joel Sherrill, Ph.D.             Director of Research & Development
joel.sherrill at OARcorp.com        On-Line Applications Research
Ask me about RTEMS: a free RTOS  Huntsville AL 35805
Support Available                (256) 722-9985