[GSoC - x86_64] Using FreeBSD's UEFI loader for RTEMS static binaries

Thu Jun 14 16:08:19 UTC 2018

Thanks for your input, everyone! I appreciate it! :)

On Thu, Jun 14, 2018 at 11:25 AM, Chris Johns <chrisj at rtems.org> wrote:
> On 14/06/2018 05:33, Joel Sherrill wrote:
>> On Wed, Jun 13, 2018, 6:57 PM Amaan Cheval <amaan.cheval at gmail.com
>> <mailto:amaan.cheval at gmail.com>> wrote:
>>
>>     On Wed, Jun 13, 2018 at 9:35 PM, Gedare Bloom <gedare at rtems.org
>>     <mailto:gedare at rtems.org>> wrote:
>>     > On Wed, Jun 13, 2018 at 11:33 AM, Amaan Cheval <amaan.cheval at gmail.com
>>     <mailto:amaan.cheval at gmail.com>> wrote:
>>     >> Hi!
>>     >>
>>     >> As we discussed in the last thread on the topic[1], I'm trying to use
>>     >> FreeBSD's loader.efi directly with RTEMS' generated static binaries
>>     >> (since FreeBSD's loader.efi has an ELF loader).
>>     >>
>>     >> In brief, I did this by:
>>     >> - Installing FreeBSD in QEMU with UEFI firmware
>>     >> - Confirming that FreeBSD's loader.efi is in fact used
>>     >> - Replacing FreeBSD's ELF kernel with a "custom" kernel[2] with an RTEMS ELF
>>     >> - Verifying that the code running after FreeBSD's loader.efi is in
>>     >> fact the "RTEMS ELF" by attaching gdb to QEMU (the rtems ELF is simply
>>     >> a while(1) loop compiled with RTEMS' tools - see later on why I can't
>>     >> do something more elaborate)
>>     >>
>>     >> Some more details of the process I followed for testing this:
>>     >> https://gist.github.com/AmaanC/42faa131ee97a1d6c4c7c25c29f0fde9z
>>     >>
>>     >> I think this method is superior to the PIC RTEMS method because:
>>     >> - FreeBSD uses it
>>     >> - RTEMS retains static ELF binaries, which can likely easily be
>>     >> combined with a Multiboot header + protect mode starter code
>>     >> - FreeBSD has methods to provide ACPI related hints to their ELF
>>     >> kernel - this might make our implementation with regards to ACPI
>>     >> simpler too
>
> I agree this is the best approach. In time we can host on our file server a
> package of FreeBSD binaries that boot an RTEMS kernel.
>
>>     >>
>>     >> Regarding some concerns Chris had with linker options and whatnot,
>>     >> here's what FreeBSD uses:
>>     >> https://www.freebsd.org/doc/en/books/arch-handbook/boot-kernel.html
>>     >>
>>     >> Here's what I used (with the code being a simple while(1) loop):
>>     >>   x86_64-rtems5-gcc ktest.c -c -nostdlib
>>     >>   x86_64-rtems5-ld ktest.o -e main -o kernel
>>     >>
>
> Nice, this looks fine. It is normal for a bare metal piece of C code.
>
>>     >>
>>     -------------------------------------------------------------------------------------
>>     >>
>>     >> What I need input on:
>>     >> - Right now, we use the following RTEMS code for testing:
>>     >>
>>     >> int main() {
>>     >>   while(1) {}
>>     >> }
>>     >>
>>     >
>>     > It's not really an RTEMS code, it is a C program (ktest.c) compiled
>>     > with the RTEMS-flavored toolchain, right?
>>
>>     Yeah, for now that's right. I'm going to conduct the same gdb based
>>     debug-stepping style test for RTEMS setting boot_card as the entry
>>     point soon - for now, it crashes QEMU with:
>>
>>     qemu: fatal: Trying to execute code outside RAM or ROM at 0x00000000000b0000
>>
>>     RAX=00000000006004c0 RBX=00000000006003d8 RCX=0000000037f36000
>>     RDX=0000000000400000
>>     RSI=0000000004000000 RDI=0000000000000180 RBP=00000000006003d8
>>     RSP=000000003c589fb8
>>     ...
>>
>>     I see that it reaches that stage even from some code it ought not to
>>     be executing, so I'll look into what that may be about.

It was quite simple, really - my stub doesn't define
_CPU_Context_restore yet - rtems_initialize_executive calls that
function expecting it to never return, but when it does, we lose
control and just start running code from virtual address 0 (or
possibly whatever happens to be on the stack as the return instruction
pointer).

What we _do_ know is a positive sign, though - an actual RTEMS static
binary does seem to be loaded just fine, and starts executing too,
until we call _CPU_Context_restore and lose control.

Next up: I'll work on the context-switching code to move past this,
and then we can follow the original plan in my proposal
(context-switching, basic IRQ, idle thread based clock driver,
printk/console support - I'd like to get to the console driver as soon
as is viable - I could work on it directly outside of the RTEMS static
binary, using the "ktest" style kernel I mentioned earlier, but I
think we'd rather make progress directly on the BSP first).

>
> Hmm.
>
>>
>>     >
>>     > It would be nice to get an RTEMS x86-64 BSP to start, at least to
>>     > confirm that you reach _start, and then even you can try to make it to
>>     > the "boot_card" startup sequence.
>>
>>     Right, I'll aim to have that working soon (using boot_card as the
>>     entry, since "_start" usually does the bootloader stuff that we're now
>>     offloading to FreeBSD, and then calls boot_card anyway).
>>
>>
>> To be consistent with other BSPs, I have a start.c on the Deos BSPs. It fetches
>> the boot arguments which are passed to boot_card() and does some other setup
>> specific to Deos.
>>
>> No need to do this now but there is a good reason to follow the pattern. Start
>> doesn't have to be in assembly.

Noted for the future, thanks!

>>
>>
>>     >
>>     >> That's literally it, because we have no access to standard libraries,
>>     >> and loader.efi calls ExitBootServices, after which we can't just
>>     >> easily directly access video memory (at 0xb8000 for eg.) to print to
>>     >> the screen. The way FreeBSD handles this is by initializing the
>>     >> console and printing to that - I haven't been able to easily port that
>>     >> yet.
>>     >>
>>     >> The question is - should I start with that effort (i.e. bringing
>>     >> printk console functionality to RTEMS) the way FreeBSD does? This way,
>>     >> we skip the bootloader for now by simply using the one built on the
>>     >> real FreeBSD - if the console prints and more elaborate linking tests
>>     >> work fine, we can be certain that this works. If _not_, I believe the
>>     >> console initialization code will likely still remain the same since
>>     >> we'll want to do it similar to how FreeBSD does it.
>>     >>
>>     >
>>     > I think this approach to getting a console to work may be reasonable,
>>     > assuming the FreeBSD console is not much more complicated than what
>>     > RTEMS needs. ...
>>
>>     I can't say about this yet, but I'll look into it (and perhaps
>>     simplifying it as we port it if it _is_ too complicated).
>>
>
> It has been a couple of years but I think FreeBSD contains some of the Intel
> code to interface to UEFI and via this you can get to the UEFI console. This
> should be easy but it comes with a side effect.
>
> UEFI boots in graphics mode and so it's console on a PC is a slow scroll one. On
> boards like a Minnow using the UEFI console has the advantage of being able to
> support any redirection UEFI has enabled such as a serial port. The disadvantage
> of this is performance and overhead. In time this may be a boot option.
>
> What I am not sure is the boundary between UEFI and the kernel and what is
> enabled or available when the kernel is loaded.

That's good information, thank you! I'll look into it as I can - for
now, can we settle on these for next steps?

- We're using FreeBSD's loader.efi - to do so, we just need our BSP to
generate static ELFs, so nothing needs to go in the source tree
- I'll focus on the context-switching code for the BSP next, aiming to
get it to actually reach bsp_start - once that's done, we can focus on
the console output (this means that until then, verifying the progress
will likely still be done through emulators and debuggers).

Let me know!

>
>>     >
>>     >> What do you think?
>>     >>
>
> Awesome work.
> Thanks
> Chris