SMP boot sequence
joel.sherrill at OARcorp.com
Thu Oct 24 13:52:17 UTC 2013
On 10/24/2013 8:42 AM, Daniel Hellstrom wrote:
> I can see from the running on the LEON SMP hardware and analysing with GRMON that the init task is started and begins its execution on CPU1 before CPU0 (the booting CPU) has finished the RTEMS
> initialization in boot_card(). As I understand the boot procedure of RTEMS global CPU interrupt is not enabled before the first task is actually scheduled which works fine on the single-core. However
> on a SMP machine the secondary CPUs have enabled global CPU interrupt in order to receive IPIs during the boot sequence. At some point in the initialization sequence an IPI is sent to a secondary CPU,
> after the secondary CPU IPI has been handled the ISR exists into thread dispatch which switches in the Init task, and this is before CPU0 has completed boot_card(). I'm not sure how this should be
> fixed, perhaps secondary CPUs should be waked later in the initialization or the init task should not be enabled until the complete OS has been initialized.
> What is the point in waking secondary CPUs before the can be given work to do? Of course initialization sequence would be easier to always have them on like in the current case... it seems that in
> other SMP OSes the initialization order have been split up in two parts, the first being single-core only, and the other to be multi-core "safe", I guess in the long run that is perhaps a good
> strategy for RTEMS as well where multiple CPUs could speed up the final initialization.
In the original implementation, the release of of the secondary cores
was not done until some point in the first user task to run. It was
for precisely this reason. RTEMS initialization needed to reach the
desired state of completion.
The IPI's used before this point in shouldn't be unblocking any threads.
As you and I have discussed, there was an issue in that EVERY core was
stepped up and all had to come to the same state. I know that we want
to be able to run on a subset of CPU cores so this is undesirably
rigid. But the idea that the secondary cores that ARE available do
not execute threads until we are ready is still needed.
Basically, the initialization needs to be single threaded until we
want to use the other cores.
> Driver initialization triggered from boot_card() may create READY tasks that might also be executed is my guess if more than two CPUs are present.
> It seems to me that this is a platform independent problem, or how have PPC, Intel and ARM solved this?
> Any insights on this?
> Daniel Hellstrom
Joel Sherrill, Ph.D. Director of Research & Development
joel.sherrill at OARcorp.com On-Line Applications Research
Ask me about RTEMS: a free RTOS Huntsville AL 35805
Support Available (256) 722-9985
More information about the devel