Scheduler Set Affinity Design Thoughts

[Sebastian Huber]

On 2014-05-16 19:48, Joel Sherrill wrote:
> Hi
>
> Questions first:
>
> + The affinity mask must be non-empty. This means a thread must
> be able to be scheduled on some processor.  With cluster scheduling,
> a scheduler implementation would only be associated with a subset
> of processors. The affinity can't be set such that the thread can't be
> scheduled by this scheduler instance.

Ok.

> How can a scheduler instance
> know which processors it is associated with?

You can store this information in the scheduler context if it needs this 
information quickly or you have iterate though the scheduler assignments:

RTEMS_INLINE_ROUTINE bool _Scheduler_default_Set_affinity_body(
   const Scheduler_Control *scheduler,
   Thread_Control          *the_thread,
   size_t                   cpusetsize,
   const cpu_set_t         *cpuset
)
{
   size_t   cpu_max   = _CPU_set_Maximum_CPU_count( cpusetsize );
   uint32_t cpu_count = _SMP_Get_processor_count();
   uint32_t cpu_index;
   bool     ok = true;

   for ( cpu_index = 0 ; cpu_index < cpu_count ; ++cpu_index ) {
#if defined(RTEMS_SMP)
     const Scheduler_Control *scheduler_of_cpu =
       _Scheduler_Get_by_CPU_index( cpu_index );

     ok = ok
       && ( ( CPU_ISSET_S( (int) cpu_index, cpusetsize, cpuset )
           && scheduler == scheduler_of_cpu )
         || ( !CPU_ISSET_S( (int) cpu_index, cpusetsize, cpuset )
           && scheduler != scheduler_of_cpu ) );
#else
     (void) scheduler;

     ok = ok && CPU_ISSET_S( (int) cpu_index, cpusetsize, cpuset );
#endif
   }

   for ( ; cpu_index < cpu_max ; ++cpu_index ) {
     ok = ok && !CPU_ISSET_S( (int) cpu_index, cpusetsize, cpuset );
   }

   _Scheduler_Set( scheduler, the_thread );

   return ok;
}


>
> + Similarly, changing scheduler for a thread dynamically should not
> allow the above to be violated. The thread must have affinity for one
> or more of the cores. How can this be done?

See _Scheduler_default_Set_affinity_body().

>
> Design thoughts:
>
> + Change priority is very careful to avoid rescheduling an executing
> thread. Is this optimization worth the trouble for affinity? I really have
> trouble seeing it be worth the trouble. Changing priority is fairly
> common thanks to priority inheritance and ceiling. Changing affinity
> should be pretty uncommon since I think it is an application design
> parameter.

I think we have two use cases for the set affinity operation.

1. To select the scheduler instance.

2. To define the affinity map inside a scheduler instance.

The second operation may be used more frequently.  If this can be done 
efficiently depends on the overall implementation of the arbitrary thread 
processor affinity support.

>
> + If this optimization is not worth the trouble, then
> _Scheduler_Set_affinity()
> should be able to do something like this like when you change the
> scheduler in _Scheduler_Set():
>
>      validate new affinity     // includes can run on subset for this
> scheduler
>      if ! changed return
>        _Thread_Set_state( the_thread, STATES_MIGRATING );
>          update affinity
>      _Thread_Clear_state( the_thread, STATES_MIGRATING );
>
> Comments?
>

Yes, this is how I would do it.  Should interrupt service routines be able to 
change the scheduler or the affinity set?

-- 
Sebastian Huber, embedded brains GmbH

Address : Dornierstr. 4, D-82178 Puchheim, Germany
Phone   : +49 89 189 47 41-16
Fax     : +49 89 189 47 41-09
E-Mail  : sebastian.huber at embedded-brains.de
PGP     : Public key available on request.

Diese Nachricht ist keine geschäftliche Mitteilung im Sinne des EHUG.