[GSoC - x86_64] Using FreeBSD's UEFI loader for RTEMS static binaries

[Amaan Cheval]

On Thu, Jun 14, 2018 at 11:25 AM, Chris Johns <chrisj at rtems.org> wrote:
> On 14/06/2018 05:33, Joel Sherrill wrote:
>> On Wed, Jun 13, 2018, 6:57 PM Amaan Cheval <amaan.cheval at gmail.com
>> <mailto:amaan.cheval at gmail.com>> wrote:
>>
>>     On Wed, Jun 13, 2018 at 9:35 PM, Gedare Bloom <gedare at rtems.org
>>     <mailto:gedare at rtems.org>> wrote:
>>     > On Wed, Jun 13, 2018 at 11:33 AM, Amaan Cheval <amaan.cheval at gmail.com
>>     <mailto:amaan.cheval at gmail.com>> wrote:
>>     >> Hi!
>>     >>
>>     >> As we discussed in the last thread on the topic[1], I'm trying to use
>>     >> FreeBSD's loader.efi directly with RTEMS' generated static binaries
>>     >> (since FreeBSD's loader.efi has an ELF loader).
>>     >>
>>     >> In brief, I did this by:
>>     >> - Installing FreeBSD in QEMU with UEFI firmware
>>     >> - Confirming that FreeBSD's loader.efi is in fact used
>>     >> - Replacing FreeBSD's ELF kernel with a "custom" kernel[2] with an RTEMS ELF
>>     >> - Verifying that the code running after FreeBSD's loader.efi is in
>>     >> fact the "RTEMS ELF" by attaching gdb to QEMU (the rtems ELF is simply
>>     >> a while(1) loop compiled with RTEMS' tools - see later on why I can't
>>     >> do something more elaborate)
>>     >>
>>     >> Some more details of the process I followed for testing this:
>>     >> https://gist.github.com/AmaanC/42faa131ee97a1d6c4c7c25c29f0fde9z
>>     >>
>>     >> I think this method is superior to the PIC RTEMS method because:
>>     >> - FreeBSD uses it
>>     >> - RTEMS retains static ELF binaries, which can likely easily be
>>     >> combined with a Multiboot header + protect mode starter code
>>     >> - FreeBSD has methods to provide ACPI related hints to their ELF
>>     >> kernel - this might make our implementation with regards to ACPI
>>     >> simpler too
>
> I agree this is the best approach. In time we can host on our file server a
> package of FreeBSD binaries that boot an RTEMS kernel.

Something worth noting here is that most Linux kernels (at least
Ubuntu and Debian, from my tests) would need to be recompiled to
enable write-support for UFS filesystems, which FreeBSD uses.
https://unix.stackexchange.com/questions/24589/mounting-ufs-partition-with-read-write-permissions-on-ubuntu-10-04

What this means is that it's likely best to provide to users a disk
image _including_ the FreeBSD kernel too, not just the bootloader,
since it can then be used to edit the image itself in QEMU to replace
the kernel with the RTEMS static binary.

Procedure would look like:
- Download freebsd.img
- Run qemu ... freebsd.img
- Edit filesystem to replace /boot/kernel/kernel with RTEMS binary
(leaving old FreeBSD kernel as a backup in /boot/kernel.old/)

I just thought I'd bring this caveat up. We can write documentation up
on the exact steps needed to accomplish this on the wiki soon too.

>
>>     >>
>>     >> Regarding some concerns Chris had with linker options and whatnot,
>>     >> here's what FreeBSD uses:
>>     >> https://www.freebsd.org/doc/en/books/arch-handbook/boot-kernel.html
>>     >>
>>     >> Here's what I used (with the code being a simple while(1) loop):
>>     >>   x86_64-rtems5-gcc ktest.c -c -nostdlib
>>     >>   x86_64-rtems5-ld ktest.o -e main -o kernel
>>     >>
>
> Nice, this looks fine. It is normal for a bare metal piece of C code.
>
>>     >>
>>     -------------------------------------------------------------------------------------
>>     >>
>>     >> What I need input on:
>>     >> - Right now, we use the following RTEMS code for testing:
>>     >>
>>     >> int main() {
>>     >>   while(1) {}
>>     >> }
>>     >>
>>     >
>>     > It's not really an RTEMS code, it is a C program (ktest.c) compiled
>>     > with the RTEMS-flavored toolchain, right?
>>
>>     Yeah, for now that's right. I'm going to conduct the same gdb based
>>     debug-stepping style test for RTEMS setting boot_card as the entry
>>     point soon - for now, it crashes QEMU with:
>>
>>     qemu: fatal: Trying to execute code outside RAM or ROM at 0x00000000000b0000
>>
>>     RAX=00000000006004c0 RBX=00000000006003d8 RCX=0000000037f36000
>>     RDX=0000000000400000
>>     RSI=0000000004000000 RDI=0000000000000180 RBP=00000000006003d8
>>     RSP=000000003c589fb8
>>     ...
>>
>>     I see that it reaches that stage even from some code it ought not to
>>     be executing, so I'll look into what that may be about.
>
> Hmm.
>
>>
>>     >
>>     > It would be nice to get an RTEMS x86-64 BSP to start, at least to
>>     > confirm that you reach _start, and then even you can try to make it to
>>     > the "boot_card" startup sequence.
>>
>>     Right, I'll aim to have that working soon (using boot_card as the
>>     entry, since "_start" usually does the bootloader stuff that we're now
>>     offloading to FreeBSD, and then calls boot_card anyway).
>>
>>
>> To be consistent with other BSPs, I have a start.c on the Deos BSPs. It fetches
>> the boot arguments which are passed to boot_card() and does some other setup
>> specific to Deos.
>>
>> No need to do this now but there is a good reason to follow the pattern. Start
>> doesn't have to be in assembly.
>>
>>
>>     >
>>     >> That's literally it, because we have no access to standard libraries,
>>     >> and loader.efi calls ExitBootServices, after which we can't just
>>     >> easily directly access video memory (at 0xb8000 for eg.) to print to
>>     >> the screen. The way FreeBSD handles this is by initializing the
>>     >> console and printing to that - I haven't been able to easily port that
>>     >> yet.
>>     >>
>>     >> The question is - should I start with that effort (i.e. bringing
>>     >> printk console functionality to RTEMS) the way FreeBSD does? This way,
>>     >> we skip the bootloader for now by simply using the one built on the
>>     >> real FreeBSD - if the console prints and more elaborate linking tests
>>     >> work fine, we can be certain that this works. If _not_, I believe the
>>     >> console initialization code will likely still remain the same since
>>     >> we'll want to do it similar to how FreeBSD does it.
>>     >>
>>     >
>>     > I think this approach to getting a console to work may be reasonable,
>>     > assuming the FreeBSD console is not much more complicated than what
>>     > RTEMS needs. ...
>>
>>     I can't say about this yet, but I'll look into it (and perhaps
>>     simplifying it as we port it if it _is_ too complicated).
>>
>
> It has been a couple of years but I think FreeBSD contains some of the Intel
> code to interface to UEFI and via this you can get to the UEFI console. This
> should be easy but it comes with a side effect.
>
> UEFI boots in graphics mode and so it's console on a PC is a slow scroll one. On
> boards like a Minnow using the UEFI console has the advantage of being able to
> support any redirection UEFI has enabled such as a serial port. The disadvantage
> of this is performance and overhead. In time this may be a boot option.
>
> What I am not sure is the boundary between UEFI and the kernel and what is
> enabled or available when the kernel is loaded.
>
>>     >
>>     >> What do you think?
>>     >>
>
> Awesome work.
> Thanks
> Chris