Progress notes on hacking on RPi2 with new firmware

[Pavel Pisa]

Hello all,

I have borrowed Raspberry Pi board from my colleague
and thought that I can make it work in reasonable time.
Result is that things are getting more complicated
after two full days of hacking.

I know cause why loading and running RTEMS application
by new firmware fails. New firmware starts application
in ARM Hypervisor (HYP) mode. Linux kernel is adapted
for that and even latest U-boot can be run in this
mode and starts loaded image in hypervisor mode.

The reason for this change is that Linux provides
KVM virtualization on ARM systems now. When chip
rom/low level firmware drops hypervisor mode then
it cannot be gained easily. It is possible to
left doors open by startup code when it setups
hypervisor vector table base to the area which
is provided as writable to loaded operating system.
But because address has to be know, can clash with
starting kernel address ... , the decision of community
has been to start loader payload in HYP mode that
manages necessary steps setup HYP vector table with
support to update it and control and then switches
to SVC mode. Background information can be found
in Supporting KVM on the ARM architecture LWN
article

  https://lwn.net/Articles/557132/

One option is to modify common RTEMS ARM startup code
and if option is set that start can occur in HYP
the compile there conditional hook to jump to piece
of the code which drops HYP and setups CPU to allow
transparency for SVC to access all HW and interrupts
and then continues in execution.

I have started from the other side to find where
are problems and implemented code which switches from HYP
mode in the bsp_start_hook_0 with keeping stack and execution
in C environment

https://github.com/ppisa/rtems/blob/bc41df2ec4d3b0a455f2cfd499dd18f8c63c1892/c/src/lib/libbsp/arm/raspberrypi/startup/bspstarthooks.c

Significant problem is that IRQ, FIQ etc stacks setup in common
start code is ignored in this case so I need to substitute/duplicate
that in the BSP startup. Other option is to drop HYP if it is detected
and jump again to start. But that is hacky as well and can lead to double
initialization.

The code on my rtems-rpi-devel GitHub branch can be used for experimentations.
VideoCore support and accesses are disable in it because there
are problems with VideoCore access in this scenario. It seems to be caused
by use arm_cp15_data_cache_clean_and_invalidate() which seems to be not
work on RPi2 Cortex-A. It works OK on RPi1. May it be that some set of 
synchronization functions stay disabled because incomplete pass-though
configuration on HYP level.

Generally, changes are learning/experimentation material but not for mainline.

This minimal part/patch is in the shape. Rest are only hacks.

https://github.com/ppisa/rtems/commit/258bc323084a8e2aa6f9b87c73fe7acbb34cb465

We can expect that on many other ARM Cortex-A targets same issue
with booting switched to HYP appears soon. So it worth to be solved
on more generic level. I am not sure when I get to that.

I have used low level print hacks and printk for debugging
as well as I have managed OpenOCD with our FTDI pod to work
with target.
Only the first core is attached (rest is commented out) because
recent switch of boot loaders to HYP mode cause OpenOCD to cope
badly with CPUs  waiting in this mode. Even debugged CPU has to
be switched first to SVC mode, then enable JTAG and waits
for GDB. This way RTEMS and othe bare metal code can be debugged.

I am not sure how my time allows to work on this so if somebody
starts get me a note to coordinate work.

I have used actual version of firmware from Raspberry GitHub
for testing. I have got ticker (with lot of debugging messages)
running when loaded from SD card. I have tried actual U-boot.
It works with TFTP and loads and starts application but switching
from HYP to SVC mode leads to some state where CPU breaks
short after switch and print of some lines. So more testing
and analyzing what is going on is required still.

Best wishes,

                Pavel
-- 
                Pavel Pisa
    e-mail:     pisa at cmp.felk.cvut.cz
    www:        http://cmp.felk.cvut.cz/~pisa
    university: http://dce.fel.cvut.cz/
    company:    http://www.pikron.com/