SMP boot sequence

[Daniel Hellstrom]

On 10/24/2013 03:52 PM, Joel Sherrill wrote:
> On 10/24/2013 8:42 AM, Daniel Hellstrom wrote:
>> Hello,
>>
>> I can see from the running on the LEON SMP hardware and analysing with GRMON that the init task is started and begins its execution on CPU1 before CPU0 (the booting CPU) has finished the RTEMS
>> initialization in boot_card(). As I understand the boot procedure of RTEMS global CPU interrupt is not enabled before the first task is actually scheduled which works fine on the single-core. However
>> on a SMP machine the secondary CPUs have enabled global CPU interrupt in order to receive IPIs during the boot sequence. At some point in the initialization sequence an IPI is sent to a secondary CPU,
>> after the secondary CPU IPI has been handled the ISR exists into thread dispatch which switches in the Init task, and this is before CPU0 has completed boot_card(). I'm not sure how this should be
>> fixed, perhaps secondary CPUs should be waked later in the initialization or the init task should not be enabled until the complete OS has been initialized.
>>
>> What is the point in waking secondary CPUs before the can be given work to do? Of course initialization sequence would be easier to always have them on like in the current case... it seems that in
>> other SMP OSes the initialization order have been split up in two parts, the first being single-core only, and the other to be multi-core "safe", I guess in the long run that is perhaps a good
>> strategy for RTEMS as well where multiple CPUs could speed up the final initialization.
> In the original implementation, the release of of the secondary cores
> was not done until some point in the first user task to run. It was
> for precisely this reason. RTEMS initialization needed to reach the
> desired state of completion.
>
> The IPI's used before this point in shouldn't be unblocking any threads.
Ok.

>
> As you and I have discussed, there was an issue in that EVERY core was
> stepped up and all had to come to the same state. I know that we want
> to be able to run on a subset of CPU cores so this is undesirably
> rigid. But the idea that the secondary cores that ARE available do
> not execute threads until we are ready is still needed.
>
> Basically, the initialization needs to be single threaded until we
> want to use the other cores.
Ok.

I can see that CPU1 executes the Init task when CPU0 is executing bsp_post_driver_hook().

Daniel

>
>> Driver initialization triggered from boot_card() may create READY tasks that might also be executed is my guess if more than two CPUs are present.
>>
>> It seems to me that this is a platform independent problem, or how have PPC, Intel and ARM solved this?
>>
>> Any insights on this?
>>
>> Regards,
>> Daniel Hellstrom
>>
>
>