[PATCH 2/2] score: Implement forced thread migration
Gedare Bloom
gedare at rtems.org
Fri May 2 17:16:05 UTC 2014
On Fri, May 2, 2014 at 12:56 PM, Sebastian Huber
<sebastian.huber at embedded-brains.de> wrote:
> On 05/02/2014 06:23 PM, Gedare Bloom wrote:
>>
>> On Fri, May 2, 2014 at 7:39 AM, Sebastian Huber
>> <sebastian.huber at embedded-brains.de> wrote:
>>>
>>> >The current implementation of task migration in RTEMS has some
>>> >implications with respect to the interrupt latency. It is crucial to
>>> >preserve the system invariant that a task can execute on at most one
>>> >processor in the system at a time. This is accomplished with a boolean
>>> >indicator in the task context. The processor architecture specific
>>> >low-level task context switch code will mark that a task context is no
>>> >longer executing and waits that the heir context stopped execution
>>> >before it restores the heir context and resumes execution of the heir
>>> >task. So there is one point in time in which a processor is without a
>>> >task. This is essential to avoid cyclic dependencies in case multiple
>>> >tasks migrate at once. Otherwise some supervising entity is necessary to
>>> >prevent life-locks. Such a global supervisor would lead to scalability
>>> >problems so this approach is not used. Currently the thread dispatch is
>>> >performed with interrupts disabled. So in case the heir task is
>>> >currently executing on another processor then this prolongs the time of
>>> >disabled interrupts since one processor has to wait for another
>>> >processor to make progress.
>>> >
>>> >It is difficult to avoid this issue with the interrupt latency since
>>> >interrupts normally store the context of the interrupted task on its
>>> >stack. In case a task is marked as not executing we must not use its
>>> >task stack to store such an interrupt context. We cannot use the heir
>>> >stack before it stopped execution on another processor. So if we enable
>>> >interrupts during this transition we have to provide an alternative task
>>> >independent stack for this time frame. This issue needs further
>>> >investigation.
>>
>> Thanks for the good explanation. Is the problem of using the
>> non-executing task's stack that the task might get resumed on some
>> other processor whlie the interrupt is running?
>
>
> Yes, as soon as you set is_executing to false it may start execution on
> another processor.
>
>
>> When you say "cannot
>> use the heir stack" do you mean cannot use it on the "idle" processor
>> because the heir is executing on some oher processor where the heir
>> should be getting blocked soon?
>
>
> Yes, exactly, the heir may currently execute on another processor. On this
> other processor it must have an heir thread and a thread dispatch must be
> pending.
>
> For example lets have a look at _Scheduler_Set() and assume that the_thread
> is the currently executing thread:
>
>
> RTEMS_INLINE_ROUTINE void _Scheduler_Set(
> const Scheduler_Control *scheduler,
> Thread_Control *the_thread
> )
> {
> const Scheduler_Control *current_scheduler = _Scheduler_Get( the_thread );
>
> if ( current_scheduler != scheduler ) {
> _Thread_Set_state( the_thread, STATES_MIGRATING );
>
> <-- here we are still executing on this processor (thread dispatching is
> disabled), but we have an heir for this processor
>
>
> _Scheduler_Free( _Scheduler_Get( the_thread ), the_thread );
> the_thread->scheduler = scheduler;
> _Scheduler_Allocate( scheduler, the_thread );
> _Scheduler_Update( scheduler, the_thread );
> _Thread_Clear_state( the_thread, STATES_MIGRATING );
>
> <-- here we are still executing on this processor (thread dispatching is
> still disabled) and we may be an heir on another processor which just
> received an inter-processor interrupt
>
> }
> }
>
> <-- high level code will eventually call _Thread_Dispatch() and we finish
> the migration
>
>
>>
>>> >---
>>> > c/src/lib/libbsp/sparc/shared/irq_asm.S | 18 ++
>>> > c/src/lib/libcpu/powerpc/new-exceptions/cpu.c | 4 +
>>> > c/src/lib/libcpu/powerpc/new-exceptions/cpu_asm.S | 18 ++
>>> > cpukit/libmisc/cpuuse/cpuusagereport.c | 2 +-
>>> > cpukit/rtems/include/rtems/rtems/tasks.h | 2 -
>>> > cpukit/rtems/src/tasksetscheduler.c | 5 +-
>>> > cpukit/score/cpu/arm/cpu.c | 12 +
>>> > cpukit/score/cpu/arm/cpu_asm.S | 18 ++
>>> > cpukit/score/cpu/arm/rtems/score/cpu.h | 16 ++
>>> > cpukit/score/cpu/i386/cpu.c | 18 ++
>>> > cpukit/score/cpu/i386/cpu_asm.S | 29 ++-
>>> > cpukit/score/cpu/i386/rtems/score/cpu.h | 27 ++
>>> > cpukit/score/cpu/no_cpu/rtems/score/cpu.h | 18 ++
>>> > cpukit/score/cpu/powerpc/cpu.c | 4 +
>>> > cpukit/score/cpu/powerpc/rtems/score/cpu.h | 14 +-
>>> > cpukit/score/cpu/sparc/cpu.c | 8 +
>>> > cpukit/score/cpu/sparc/rtems/score/cpu.h | 13 +
>>> > cpukit/score/include/rtems/score/percpu.h | 11 +-
>>> > cpukit/score/include/rtems/score/scheduler.h | 2 +-
>>> > cpukit/score/include/rtems/score/schedulerimpl.h | 24 +--
>>> > cpukit/score/include/rtems/score/schedulersmp.h | 4 +-
>>> > .../score/include/rtems/score/schedulersmpimpl.h | 33 ++-
>>> > cpukit/score/include/rtems/score/statesimpl.h | 2 +
>>> > cpukit/score/include/rtems/score/thread.h | 11 +
>>> > cpukit/score/include/rtems/score/threadimpl.h | 35 +++-
>>> > cpukit/score/src/smp.c | 5 +-
>>> > cpukit/score/src/threaddispatch.c | 31 +--
>>> > cpukit/score/src/threadhandler.c | 4 +-
>>> > cpukit/score/src/threadinitialize.c | 6 +-
>>> > cpukit/score/src/threadrestart.c | 2 +-
>>> > cpukit/score/src/threadstartmultitasking.c | 18 +-
>>> > doc/user/smp.t | 46 ++++
>>> > testsuites/smptests/Makefile.am | 1 +
>>> > testsuites/smptests/configure.ac | 1 +
>>> > testsuites/smptests/smpmigration02/Makefile.am | 19 ++
>>> > testsuites/smptests/smpmigration02/init.c | 255
>>> > ++++++++++++++++++++
>>> > .../smptests/smpmigration02/smpmigration02.doc | 12 +
>>> > .../smptests/smpmigration02/smpmigration02.scn | 7 +
>>> > testsuites/smptests/smpscheduler02/init.c | 4 +-
>>> > 39 files changed, 679 insertions(+), 80 deletions(-)
>>> > create mode 100644 testsuites/smptests/smpmigration02/Makefile.am
>>> > create mode 100644 testsuites/smptests/smpmigration02/init.c
>>> > create mode 100644
>>> > testsuites/smptests/smpmigration02/smpmigration02.doc
>>> > create mode 100644
>>> > testsuites/smptests/smpmigration02/smpmigration02.scn
>>> >
>>> >diff --git a/c/src/lib/libbsp/sparc/shared/irq_asm.S
>>> > b/c/src/lib/libbsp/sparc/shared/irq_asm.S
>>> >index 3a86ad5..cdaaf6d 100644
>>> >--- a/c/src/lib/libbsp/sparc/shared/irq_asm.S
>>> >+++ b/c/src/lib/libbsp/sparc/shared/irq_asm.S
>>> >@@ -163,6 +163,21 @@ done_flushing:
>>> > nop
>>> > nop
>>> >
>>> >+#if defined(RTEMS_SMP)
>>> >+ ! Indicate that this context is no longer executing
>>> >+ stb %g0, [%o0 + SPARC_CONTEXT_CONTROL_IS_EXECUTING_OFFSET]
>>> >+
>>
>> The register usage should be documented at the declaration of
>> CPU_Context_restore_heir above. Probably in the other architectures it
>> should be documented as well.
>>
>
> %g0 is the zero register on SPARC and %o0 is the executing context, so I
> don't think there is something to document here.
>
I meant it should be stated o0 is the executing context, where it is
stated above o1 is the context to restore (i.e. the heir). It just
helps make reading the context switch code a bit easier to know what
the registers are used for.
> --
> Sebastian Huber, embedded brains GmbH
>
> Address : Dornierstr. 4, D-82178 Puchheim, Germany
> Phone : +49 89 189 47 41-16
> Fax : +49 89 189 47 41-09
> E-Mail : sebastian.huber at embedded-brains.de
> PGP : Public key available on request.
>
> Diese Nachricht ist keine geschäftliche Mitteilung im Sinne des EHUG.
>
More information about the devel
mailing list