Partitioned/clustered scheduling

[Sebastian Huber]

On 2013-11-19 18:36, Gedare Bloom wrote:
> On Tue, Nov 19, 2013 at 8:39 AM, Sebastian Huber
[...]
>> >Scheduler and processor configuration example:
>> >
>> >  RTEMS_SCHED_DEFINE_FP_SMP(sched_fp0, 256);
>> >  RTEMS_SCHED_DEFINE_FP_SMP(sched_fp1, 64);
>> >  RTEMS_SCHED_DEFINE_EDF_SMP(sched_edf0);
>> >
>> >  const rtems_cpu_config rtems_cpu_config_table[] = {
>> >    RTEMS_CPU_CONFIG_INIT(RTEMS_SCHED_REF_FP_SMP(sched_fp0)),
>> >    RTEMS_CPU_CONFIG_INIT(RTEMS_SCHED_REF_FP_SMP(sched_fp1)),
>> >    RTEMS_CPU_CONFIG_INIT(RTEMS_SCHED_REF_FP_SMP(sched_fp1)),
>> >    RTEMS_CPU_CONFIG_INIT(RTEMS_SCHED_REF_FP_SMP(sched_fp1)),
>> >    RTEMS_CPU_CONFIG_INIT(NULL),
>> >    RTEMS_CPU_CONFIG_INIT(NULL),
>> >    RTEMS_CPU_CONFIG_INIT(RTEMS_SCHED_REF_EDF_SMP(sched_edf0)),
>> >    RTEMS_CPU_CONFIG_INIT(RTEMS_SCHED_REF_EDF_SMP(sched_edf0)
>> >  };
>> >
>> >  const size_t rtems_cpu_config_count =
>> >    RTEMS_ARRAY_SIZE(rtems_cpu_config_table);
>> >
> This looks good to me. I guess the user must define the table. We
> should offer some logical/safe defaults, especially for
> single-processor.

Yes, the user must provide this table.  I am not sure how to do the 
single-processor configuration.

One option might be to use something like this:

#ifndef RTEMS_SMP
#define RTEMS_SCHED_DEFINE_FP(name, prio_count)
   static Scheduler_FP_Control name = {
	...
   };
   void _Scheduler_Block(Thread_Control *thread)
   {
     _Scheduler_FP_Block(&name, thread);
   }
   ...

This would remove the function pointer overhead on single-processor configurations.

>
>> >An alternative to the processor configuration table would be to specify in
>> >the
>> >scheduler instance which processors are owned by the instance.  This would
>> >require a static initialization of CPU sets which is difficult.  Also the
>> >schedulers have to be registered somewhere, so some sort of table is needed
>> >anyway.  Since a processor can be owned by at most one scheduler instance
>> >this
>> >configuration approach enables an additional error source which is avoided
>> >by
>> >the processor configuration table.
>> >
>> >==== Scheduler Implementation Changes ====
>> >
>> >Currently the scheduler operations have no control context and use global
>> >variables instead.  Thus the scheduler operations signatures must change to
>> >use
>> >a scheduler control context as the first parameter, e.g.
>> >
>> >  typedef struct Scheduler_Control Scheduler_Control;
>> >
>> >  typedef struct {
>> >    [...]
>> >    void ( *set_affinity )(
>> >      Scheduler_Control *self,
>> >      Thread_Control *thread,
>> >      size_t affinity_set_size,
>> >      const cpu_set_t *affinity_set
>> >    );
>> >    [...]
>> >  } Scheduler_Operations;
>> >
>> >  /**
>> >   * @brief General scheduler control.
>> >   */
>> >  struct Scheduler_Control {
>> >    /**
>> >     * @brief The scheduler operations.
>> >     */
>> >    Scheduler_Operations Operations;
>> >
>> >    /**
>> >     * @brief Size of the owned processor set in bytes.
>> >     */
>> >    size_t owned_cpu_set_size
>> >
>> >    /**
>> >     * @brief Reference to the owned processor set.
>> >     *
>> >     * A set bit means this processor is owned by this scheduler instance, an
>> >     * unset bit means the opposite.
>> >     */
>> >    cpu_set_t *owned_cpu_set;
>> >  };
>> >
> For uniprocessor we don't need the owned processor set fields?

Yes.

>
>> >Single processor configurations benefit also from this change since it makes
>> >all dependencies explicit and easier to access (allows more efficient
>> >machine
>> >code).
>> >
> Good. This should eliminate the _Scheduler variable that is the
> uniprocessor global scheduler handle? I never liked it.
>
>> >==== RTEMS API Changes ====
>> >
>> >Each thread needs a processor affinity set in the RTEMS SMP configuration.
>> >The
>> >rtems_task_create() function will use the processor affinity set of the
>> >executing thread to initialize the processor affinity set of the created
>> >task.  This enables backward compatibility for existing software.
>> >
> Good. What will be the default affinity of the init task?

This is a good question.  I think it should use the affinity set of the 
scheduler of the initialization processor.

>
>> >Two new functions should be added to alter and retrieve the processor
>> >affinity
>> >sets of threads.
>> >
>> >  /**
>> >   * @brief Sets the processor affinity set of a task.
>> >   *
>> >   * @param[in] task_id Identifier of the task.  Use @ref RTEMS_SELF to
>> >select
>> >   * the executing task.
>> >   * @param[in] affinity_set_size Size of the specified affinity set in
>> >bytes.
>> >   * This value must be positive.
>> >   * @param[in] affinity_set The processor affinity set for the task.  This
>> >   * pointer must not be @c NULL.  A set bit in the affinity set means that
>> >the
>> >   * task can execute on this processor and an unset bit means the opposite.
>> >   *
>> >   * @retval RTEMS_SUCCESSFUL Successful operation.
>> >   * @retval RTEMS_INVALID_ID Invalid task identifier.
>> >   * @retval RTEMS_INVALID_CPU_SET Invalid processor affinity set.
>> >   */
>> >  rtems_status_code rtems_task_set_affinity(
>> >    rtems_id task_id,
>> >    size_t affinity_set_size,
>> >    const cpu_set_t *affinity_set
>> >  );
>> >
>> >  /**
>> >   * @brief Gets the processor affinity set of a task.
>> >   *
>> >   * @param[in] task_id Identifier of the task.  Use @ref RTEMS_SELF to
>> >select
>> >   * the executing task.
>> >   * @param[in] affinity_set_size Size of the specified affinity set in
>> >bytes.
>> >   * This value must be positive.
>> >   * @param[out] affinity_set The processor affinity set of the task.  This
>> >   * pointer must not be @c NULL.  A set bit in the affinity set means that
>> >the
>> >   * task can execute on this processor and an unset bit means the opposite.
>> >   *
>> >   * @retval RTEMS_SUCCESSFUL Successful operation.
>> >   * @retval RTEMS_INVALID_ID Invalid task identifier.
>> >   * @retval RTEMS_INVALID_CPU_SET The affinity set is too small for the
>> >   * processor affinity set of the task.
>> >   */
>> >  rtems_status_code rtems_task_get_affinity(
>> >    rtems_id task_id,
>> >    size_t affinity_set_size,
>> >    cpu_set_t *affinity_set
>> >  );
>> >
> These classic API set/get affinity functions seem reasonable. They
> mimic existing affinity interfaces nicely. Now users should do
> rtems_task_create(), rtems_task_set_affinity(), rtems_task_start() in
> order to specify the affinity of their tasks, correct?

Yes, after reading it again, should we use the naming of the Linux man page

http://man7.org/linux/man-pages/man3/pthread_setaffinity_np.3.html

affinity_set_size -> cpusetsize
affinity_set -> cpuset

?

-- 
Sebastian Huber, embedded brains GmbH

Address : Dornierstr. 4, D-82178 Puchheim, Germany
Phone   : +49 89 189 47 41-16
Fax     : +49 89 189 47 41-09
E-Mail  : sebastian.huber at embedded-brains.de
PGP     : Public key available on request.

Diese Nachricht ist keine geschäftliche Mitteilung im Sinne des EHUG.