Problem running RTEMS on raspberrypi3
Christian Mauderer
list at c-mauderer.de
Mon Dec 16 14:19:12 UTC 2019
On 16/12/2019 12:43, Niteesh wrote:
> On Mon, Dec 16, 2019 at 2:36 AM Christian Mauderer <list at c-mauderer.de
> <mailto:list at c-mauderer.de>> wrote:
>
>
>
> On 15/12/2019 21:29, Niteesh wrote:
> >
> >
> > On Mon, Dec 16, 2019 at 12:53 AM Christian Mauderer
> <list at c-mauderer.de <mailto:list at c-mauderer.de>
> > <mailto:list at c-mauderer.de <mailto:list at c-mauderer.de>>> wrote:
> >
> > On 15/12/2019 19:46, Niteesh wrote:
> > >
> > >
> > > On Sun, Dec 15, 2019 at 10:15 PM Christian Mauderer
> > <list at c-mauderer.de <mailto:list at c-mauderer.de>
> <mailto:list at c-mauderer.de <mailto:list at c-mauderer.de>>
> > > <mailto:list at c-mauderer.de <mailto:list at c-mauderer.de>
> <mailto:list at c-mauderer.de <mailto:list at c-mauderer.de>>>> wrote:
> > >
> > > Hello Niteesh,
> > >
> > > On 15/12/2019 09:05, Niteesh wrote:
> > > > I am trying to get RTEMS examples running on the RPI3, the
> > RPI3 is
> > > > similar to RPI2 so the examples built for RPI2 should
> > technically
> > > run on
> > > > the RPi3.But they don't :(, I am really sure of what
> is causing
> > > the problem.
> > >
> > > Note that there are at least two different versions of the
> > RPi3 which
> > > use different chips. The original RPi3 which uses a BCM2837
> > (same like
> > > later versions of RPi2) and the RPi3+ which uses a
> BCM2837B0.
> > Broadcom
> > > is always quite sparse with documentation so it's not
> easy to
> > tell the
> > > differences. Which one do you have?
> > >
> > > I have Rpi3 model b v1.2 which uses BCM2837 SOC, in my
> bare-metal
> > > programming I used the
> > > 2835 doc as a reference because the only major difference these
> > two SOC
> > > is the peripheral base address
> > > offset. But this is arm cpu is also capable of executing in
> 64bit
> > mode.
> >
> > OK. Did you check, whether the offset is correct? In the
> raspberrypi.h
> > in RTEMS there is the following define:
> >
> > #if (BSP_IS_RPI2 == 1)
> > #define RPI_PERIPHERAL_BASE 0x3F000000
> > #else
> > #define RPI_PERIPHERAL_BASE 0x20000000
> > #endif
> >
> > The offsets are right.
>
> Good.
>
> >
> > >
> > > > I followed the steps
> > > >
> > >
> >
> from http://alanstechnotes.blogspot.com/2013/03/running-your-first-rtems-program-on.html (modified
> > > > commands to use rtems5) to build the kernel img.
> > >
> > > It's a bit odd that the Bootloader doesn't use some image
> > format like
> > > U-Boot but if that's the case for Raspberry, that's OK.
> > >
> > > Do you want me to try U-Boot, I was planning to use it for my
> > bare-metal
> > > stuff because copying the kernel
> > > to SD-card was a real pain. Will it even work with RTEMS?
> >
> > The manual that you linked uses the default Raspberry
> bootloader. I'm
> > not sure whether it's an U-Boot. If you skip the bootloader
> entirely,
> > your SDRAM might isn't initialized.
> >
> > The manual uses the default bootloader. I don't think we have to worry
> > about the SDRAM initialization
> > because all of that is taken care of by the GPU.
>
> Sounds OK.
>
> > When using Uboot, the
> > GPU will load the uboot image and
> > pass the control to the CPU. And then the uboot continue's it's
> execution.
> >
>
> I don't wanted to suggest to use an extra U-Boot. I was just not sure
> whether the stage 3 loader is an U-Boot. Your approach sounds fine
> so far.
>
> >
> >
> > PS: You answered that further below. You are using the stage 3
> loader.
> >
> > >
> > > > I did try running it on
> > > > Qemu but it doesn't always work, sometimes it gives
> > weird output.
> > >
> > > How did you run it on Qemu? Did you build some image for
> that too?
> > >
> > > qemu-system-arm -M raspi2 -m 1G -kernel hello.exe -serial
> mon:stdio
> > > -nographic
> > > *
> > > *
> > > *
> > > qemu-system-aarch64: GLib: g_mapped_file_unref: assertion
> 'file !=
> > NULL'
> > > failed *I get this error
> > > while trying to emulate raspi3.
> >
> > That sounds like a problem with Qemu. Is there some official
> test image
> > for rpi3 on qemu? Note that this isn't really relevant for
> your current
> > problem. So if you don't have some manual just ignore the
> question.
> >
> > >
> > > I ran qemu along with GDB to find what was causing the wrong
> output. I
> > > am really not sure if this is right,
> > > I still have a lot to learn, but my assumption's using GDB
> are as
> > follows.
> > > There are 4 active thread which run the same code.
> > >
> > > (gdb) info thread
> > > Id Target Id Frame
> > > * 1 Thread 1.1 (CPU#0 [running]) _start () at
> > >
> >
> ../../../../../../../../rtems/c/src/lib/libbsp/arm/raspberrypi/../../../../../../bsps/arm/shared/start/start.S:153
> > > 2 Thread 1.2 (CPU#1 [running]) _start () at
> > >
> >
> ../../../../../../../../rtems/c/src/lib/libbsp/arm/raspberrypi/../../../../../../bsps/arm/shared/start/start.S:153
> > > 3 Thread 1.3 (CPU#2 [running]) _start () at
> > >
> >
> ../../../../../../../../rtems/c/src/lib/libbsp/arm/raspberrypi/../../../../../../bsps/arm/shared/start/start.S:153
> > > 4 Thread 1.4 (CPU#3 [running]) _start () at
> > >
> >
> ../../../../../../../../rtems/c/src/lib/libbsp/arm/raspberrypi/../../../../../../bsps/arm/shared/start/start.S:153
> >
> > In this case that are not threads but it's the CPU cores. GDB
> shows them
> > as threads. Most likely it wouldn't be able to detect the
> RTEMS threads.
> >
> > It's a bit odd that they are all pointing to start.S:153.
> That's the
> > entry point for the program. It looks like not even one
> instruction has
> > been executed yet.
> >
> > I took this output before executing the program, that the reason
> why not
> > even a single instruction has been
> > executed yet.
>
> OK.
>
> >
> > >
> > > After some time one of the thread call's the BSP reset function
> > this is
> > > when the program crashes, the other threads complain "*executing
> > thread
> > > is NULL*"
> >
> > I would rather assume that one core tries to do the
> initialization while
> > the others hang in a endless loop till they are needed. The
> one core
> > doing the initialization work hits an exception somewhere and
> calls the
> > exception handler which calls the bsp reset function.
> >
> > The executing thread is NULL is a sign that it happens
> somewhere during
> > initialization when the RTEMS threading hasn't been started yet.
> >
> > The PC has an odd value. The linker command file tells that
> there is a
> > RAM_MMU at 0x00100000. It only puts a bsp_translation_table
> there but
> > there shouldn't be any code. So I don't know what the
> processor is doing
> > there. You could try to set a breakpoint on the address
> 0x00100fc4 and
> > take a look at why the processor is there with a "bt" (backtrace).
> >
> > When I re-run it again, it now stops at a different address. As
> you said
> > that the other cores are put
> > in an endless loop, I don't think that's is happening. I single
> stepped
> > the instruction and later at some point checked the threads
> >
> > (gdb) info threads
>
> >
>
> >
> > Target Id Frame
> > 1 Thread 1.1 (CPU#0 [running]) arm_ccsidr_get_line_power
> > (ccsidr=<optimized out>)
> > at
> >
> /home/niteesh/development/rtems/kernel/rtems/cpukit/score/cpu/arm/include/libcpu/arm-cp15.h:850
> > 2 Thread 1.2 (CPU#1 [running])
> arm_cp15_cache_invalidate_level
> > (inst_data_fl=0, level=1)
> > at
> >
> /home/niteesh/development/rtems/kernel/rtems/cpukit/score/cpu/arm/include/libcpu/arm-cp15.h:1162
> > 3 Thread 1.3 (CPU#2 [running]) arm_ccsidr_get_line_power
> > (ccsidr=<optimized out>)
> > at
> >
> /home/niteesh/development/rtems/kernel/rtems/cpukit/score/cpu/arm/include/libcpu/arm-cp15.h:850
> > * 4 Thread 1.4 (CPU#3 [running])
> > arm_cp15_get_cache_size_id_for_level (level_and_inst_dat=0)
> > at
> >
> /home/niteesh/development/rtems/kernel/rtems/cpukit/score/cpu/arm/include/libcpu/arm-cp15.h:936
> > (gdb)
> >
> > They all are executing different instructions at the same time.
>
> Some of the initialization is done on all cores. Some isn't. I took a
> look at the initialization and it seems that I was wrong: There is no
> wait loop. All processors are running through the initialization
> process. Some just skip parts. The part where they really start to
> differ is in bsp_start_hook_0.
>
> > I> googled about just running one thread or CPU as you said at a
> time and
> > used "*set scheduler-locking on" *on doing this I always get the right
> > output.
> >
> > (gdb) info threads
> > Id Target Id Frame
> > * 1 Thread 1.1 (CPU#0 [running]) bsp_reset ()
> > at
> >
> ../../../../../../../../rtems/c/src/lib/libbsp/arm/raspberrypi/../../../../../../bsps/arm/raspberrypi/start/bspreset.c:18
> > 2 Thread 1.2 (CPU#1 [running]) _start ()
> > at
> >
> ../../../../../../../../rtems/c/src/lib/libbsp/arm/raspberrypi/../../../../../../bsps/arm/shared/start/start.S:153
> > 3 Thread 1.3 (CPU#2 [running]) _start ()
> > at
> >
> ../../../../../../../../rtems/c/src/lib/libbsp/arm/raspberrypi/../../../../../../bsps/arm/shared/start/start.S:153
> > 4 Thread 1.4 (CPU#3 [running]) _start ()
> > at
> >
> ../../../../../../../../rtems/c/src/lib/libbsp/arm/raspberrypi/../../../../../../bsps/arm/shared/start/start.S:153
> > (gdb)
> >
> > The above command allow's only a single thread to run.
>
> Maybe there is a timing difference between the simulator and the real
> hardware. I'm not sure how well tested the SMP code is on the Raspberry.
> There can be a hidden bug.
>
> Just a guess: If there is a bug it could be possible that you hit it
> with your rpi3 too. Maybe it would be good to try a single core version
> of the BSP. I assume you have configured with "--enable-smp"? Can you
> try to build without it?
>
> I built 2 versions with SMP enabled and disabled, the one we are talking
> about is the SMP disabled version, I ran
> the example with SMP enabled, still, the error's are similar, I only
> difference is, in the disabled one, there are only 4 or less panic's
> (maybe corresponding to 4 cpu's) but the other one has a higher number
> of panics.
So for the SMP disabled version all four cores do something? That sounds
like a big bug. Only one core should be active in that case.
>
> > Won't it be a good idea to make a separate BSP for rpi3?
>
> As soon as it is necessary: Sure. But from what you told me it seems
> that the hardware is very similar so that we won't hit this point soon.
> Or do you already see differences that would make it necessary.
>
> I haven't had a look at the details but it could also be possible to
> unify the BSPs and entirely remove the rpi2 variant if the information
> from the flattened device tree are used.
>
> Can you explain how FDT work in RTEMS. Can you mention some BSP's which
> use FDT so I can use them as a reference to learn.
We have three types of BSPs in regard to FDT:
1. BSPs that don't use a FDT.
2. BSPs that use the FDT only for libbsd (Beagle is one example for this)
3. BSPs that use the FDT for the RTEMS drivers too. I think currently
that is:
- arm/imx
- riscv/riscv
- powerpc/qoriq
- arm/altera-cyclone-v
arm/imx is most likely one of the best examples because it uses FDT even
for drivers like I2C.
> I previously took a look at the beagle FDT project (#3784), you
> mentioned about hardcoded values and initialization functions, can
> you explain more about what exactly do the initialization functions do?
The drivers in the beagle BSP currently use fixed base addresses quite
similar to the raspberry BSP. For example in the I2C driver you can find
the following:
err = am335x_i2c_bus_register(BBB_I2C_0_BUS_PATH,
AM335X_I2C0_BASE,
I2C_BUS_CLOCK_DEFAULT,
BBB_I2C0_IRQ);
There is a fixed AM335X_I2C0_BASE address. Or take a look at the
am335x_i2c_fill_registers function. It uses some fixed registers for the
pin initialization.
You can find similar things in other drivers of the beagle BSP. A sane
approach would be to have a pinmux initialization based on the
information from the FDT. The same is true for the drivers: They should
be initialized based on the information of the FDT (base register,
interrupt, clock rate for I2C or SPI, ...).
> Do they assign a function to a particular pin, like in raspi
> the pins are multiplexed for various functions, so do the initialization
> functions assign those pins to a particular function?
>
> And also please explain how does the initialization of the system happen
> from the DT file.
The device tree files contain information about which pins are used for
what and which modules are active. Take a look at the files for Beagle
in FreeBSD:
https://github.com/freebsd/freebsd/blob/master/sys/gnu/dts/arm/am335x-boneblack.dts
and especially the included ones:
https://github.com/freebsd/freebsd/blob/master/sys/gnu/dts/arm/am335x-bone-common.dtsi
You can just add overlays to enable or disable further modules, change
pin functions and so on. That is useful if you want to write an
application that - for example - can work with the same binary on a
beagle with an HDMI or a parallel display. Depending on the FDT another
hardware is initialized.
>
> >
> > > *** FATAL ***
> > > fatal source: 9 (RTEMS_FATAL_SOURCE_EXCEPTION)
> > >
> > > R0 = 0x400005e6 R8 = 0x00000000
> > > R1 = 0x00000001 R9 = 0x00000000
> > > R2 = 0xbffffa1a R10 = 0x00000000
> > > R3 = 0x00000000 R11 = 0x00000000
> > > R4 = 0x002001db R12 = 0x00000000
> > > R5 = 0x00000000 SP = 0x00300bd0
> > > R6 = 0x00000000 LR = 0x00100fc4
> > > R7 = 0x00000000 PC = 0x00100fc4
> > > CPSR = 0x000001d3 VEC = 0x00000002
> > > FPEXC = 0x40000000
> > > FPSCR = 0x00000000
> > > D00 = 0x0000000000000000
> > > D01 = 0x0000000000000000
> > > D02 = 0x0000000000000000
> > > D03 = 0x0000000000000000
> > > D04 = 0x0000000000000000
> > > D05 = 0x0000000000000000
> > > D06 = 0x0000000000000000
> > > D07 = 0x0000000000000000
> > > D08 = 0x0000000000000000
> > > D09 = 0x0000000000000000
> > > D10 = 0x0000000000000000
> > > D11 = 0x0000000000000000
> > > D12 = 0x0000000000000000
> > > D13 = 0x0000000000000000
> > > D14 = 0x0000000000000000
> > > D15 = 0x0000000000000000
> > > D16 = 0x0000000000000000
> > > D17 = 0x0000000000000010
> > > D18 = 0x0000000000000000
> > > D19 = 0x0000000000000000
> > > D20 = 0x0000000000000000
> > > D21 = 0x0000000000000000
> > > D22 = 0x0000000000000000
> > > D23 = 0x0000000000000000
> > > D24 = 0x0000000000000000
> > > D25 = 0x0000000000000000
> > > D26 = 0x0000000000000000
> > > D27 = 0x0000000000000000
> > > D28 = 0x0000000000000000
> > > D29 = 0x0000000000000000
> > > D30 = 0x0000000000000000
> > > D31 = 0x0000000000000000
> > > RTEMS version:
> > 5.0.0.c6d8589bb00a9d2a5a094c68c90290df1dc44807-modified
> > > RTEMS tools: 7.5.0 20191114 (RTEMS 5, RSB
> > > 83fa79314dd87c0a8c78fd642b2cea3138be8dd6, Newlib 3e24fbf6f)
> > > executing thread is NULL
> > >
> > > > The steps that I followed are:
> > > > 1. Created a bootable SD card using raspbian.
> > > > 2. Replaced the kernel.img file with RTEMS kernel.img
> file and
> > > modified
> > > > the config.txt to boot from the RTEMs kernel (boots in
> > aarch32 bit
> > > mode).
> > > > I am still not able to wrap my head around the RPI bsp
> build
> > process.
> > > > This is what I understood as of now, correct me if I
> am wrong.
> > > > Both RPi and Rpi2 are based on the same BSP, they just
> > differ in the
> > > > peripheral offsets, hardcoded checks are used to
> select the
> > right
> > > offset
> > > > at the time of compiling
> > >
> > > >From what I know of the Raspberry BSPs that is correct.
> > >
> > > > and the linkercmd file is responsible for
> > > > building the final executable file.
> > >
> > > The linkercmd file is - like for all programs - responsible
> > where the
> > > memory regions are that can be used for code or data. So you
> > could more
> > > or less explain it like you did.
> > >
> > > > I looked at the linker script, it seem's to have the start
> > section at
> > > > address 0x200000, I also loaded it in GDB and the start
> > address is
> > > > *Start address 0x200080,*
> > >
> > > I agree with that. The different start in GDB is most likely
> > because
> > > there is a vector table in front (at least if the
> Broadcom chip is
> > > similar to a lot of other processors that I have
> encountered).
> > >
> > > Does that mean that you have a debugger connected to the
> > raspberry? Can
> > > you load code with it? If yes: Is the bootloader executed
> > before you
> > > load your code? Otherwise the SDRAM might isn't
> initialized yet.
> > >
> > > I don't have a debugger connected to it. I from what I have
> SDRAM is
> > > initialized by the 3 stage bootloader(start.elf).
> >
> > That should be OK and it answers my question above.
> >
> > >
> > > > I did some bare metal programming on RPI3
> > > > there I had the start section at address 0x8000 Is
> this causing
> > > the problem?
> > >
> > > I assume that you used some internal RAM when you did
> bare metal
> > > programming. You maybe even skipped one or two bootloader
> > stages. From a
> > > quick look Raspberry has a quite complex boot process
> with at
> > least
> > > three bootloaders:
> > http://lions-wing.net/maker/raspberry-1/boot.html
> > >
> > > I don't think I have skipped any stages. The boot process is
> > exactly the
> > > same as how it boot's a normal raspbian or any other linux
> > > distro, I just to replace the linux kernel with my own kernel.
> >
> > Sounds reasonable. Does the bootloader print anything where it
> puts the
> > kernel image? Maybe the start address changed during the raspberry
> > versions.
> >
> > the default kernel load address is 0x8000 in 32bit mode and 0x80000 in
> > 64bit mode I have no idea about the raspberry 1,
> > but the load address is same for rpi2 and 3.
>
> That sounds odd. Do you have a memory map somewhere? From the linker
> command file it seems quite clear that RTEMS is build for a 0x200000.
>
> >
> > >
> > >
> > > > I have no idea on how to debug this, any suggestion on how
> > to start
> > > > would be really helpfull.
> > > >
> > >
> >
>
More information about the devel
mailing list