Problem running RTEMS on raspberrypi3
Christian Mauderer
list at c-mauderer.de
Mon Dec 16 14:21:05 UTC 2019
I think I have guided you to a wrong path here. I mentioned U-Boot
because it is often used on a lot of evaluation boards. In the raspberry
case it seems that the stage 3 loader is something different. But
everything should work with that stage 3 loader. I don't think that
U-Boot is necessary.
On 16/12/2019 14:01, Niteesh wrote:
> I got uboot running on my raspi3. But I can't figure out to load and run
> a custom kernel. Can you explain the steps or point me to some
> reference.
> On Mon, Dec 16, 2019 at 5:13 PM Niteesh <gsnb.gn at gmail.com
> <mailto:gsnb.gn at gmail.com>> wrote:
>
> On Mon, Dec 16, 2019 at 2:36 AM Christian Mauderer
> <list at c-mauderer.de <mailto:list at c-mauderer.de>> wrote:
>
>
>
> On 15/12/2019 21:29, Niteesh wrote:
> >
> >
> > On Mon, Dec 16, 2019 at 12:53 AM Christian Mauderer
> <list at c-mauderer.de <mailto:list at c-mauderer.de>
> > <mailto:list at c-mauderer.de <mailto:list at c-mauderer.de>>> wrote:
> >
> > On 15/12/2019 19:46, Niteesh wrote:
> > >
> > >
> > > On Sun, Dec 15, 2019 at 10:15 PM Christian Mauderer
> > <list at c-mauderer.de <mailto:list at c-mauderer.de>
> <mailto:list at c-mauderer.de <mailto:list at c-mauderer.de>>
> > > <mailto:list at c-mauderer.de <mailto:list at c-mauderer.de>
> <mailto:list at c-mauderer.de <mailto:list at c-mauderer.de>>>> wrote:
> > >
> > > Hello Niteesh,
> > >
> > > On 15/12/2019 09:05, Niteesh wrote:
> > > > I am trying to get RTEMS examples running on the
> RPI3, the
> > RPI3 is
> > > > similar to RPI2 so the examples built for RPI2 should
> > technically
> > > run on
> > > > the RPi3.But they don't :(, I am really sure of
> what is causing
> > > the problem.
> > >
> > > Note that there are at least two different versions
> of the
> > RPi3 which
> > > use different chips. The original RPi3 which uses a
> BCM2837
> > (same like
> > > later versions of RPi2) and the RPi3+ which uses a
> BCM2837B0.
> > Broadcom
> > > is always quite sparse with documentation so it's
> not easy to
> > tell the
> > > differences. Which one do you have?
> > >
> > > I have Rpi3 model b v1.2 which uses BCM2837 SOC, in my
> bare-metal
> > > programming I used the
> > > 2835 doc as a reference because the only major
> difference these
> > two SOC
> > > is the peripheral base address
> > > offset. But this is arm cpu is also capable of
> executing in 64bit
> > mode.
> >
> > OK. Did you check, whether the offset is correct? In the
> raspberrypi.h
> > in RTEMS there is the following define:
> >
> > #if (BSP_IS_RPI2 == 1)
> > #define RPI_PERIPHERAL_BASE 0x3F000000
> > #else
> > #define RPI_PERIPHERAL_BASE 0x20000000
> > #endif
> >
> > The offsets are right.
>
> Good.
>
> >
> > >
> > > > I followed the steps
> > > >
> > >
> >
> from http://alanstechnotes.blogspot.com/2013/03/running-your-first-rtems-program-on.html (modified
> > > > commands to use rtems5) to build the kernel img.
> > >
> > > It's a bit odd that the Bootloader doesn't use some
> image
> > format like
> > > U-Boot but if that's the case for Raspberry, that's OK.
> > >
> > > Do you want me to try U-Boot, I was planning to use it
> for my
> > bare-metal
> > > stuff because copying the kernel
> > > to SD-card was a real pain. Will it even work with RTEMS?
> >
> > The manual that you linked uses the default Raspberry
> bootloader. I'm
> > not sure whether it's an U-Boot. If you skip the
> bootloader entirely,
> > your SDRAM might isn't initialized.
> >
> > The manual uses the default bootloader. I don't think we have
> to worry
> > about the SDRAM initialization
> > because all of that is taken care of by the GPU.
>
> Sounds OK.
>
> > When using Uboot, the
> > GPU will load the uboot image and
> > pass the control to the CPU. And then the uboot continue's
> it's execution.
> >
>
> I don't wanted to suggest to use an extra U-Boot. I was just not
> sure
> whether the stage 3 loader is an U-Boot. Your approach sounds
> fine so far.
>
> >
> >
> > PS: You answered that further below. You are using the
> stage 3 loader.
> >
> > >
> > > > I did try running it on
> > > > Qemu but it doesn't always work, sometimes it gives
> > weird output.
> > >
> > > How did you run it on Qemu? Did you build some image
> for that too?
> > >
> > > qemu-system-arm -M raspi2 -m 1G -kernel hello.exe
> -serial mon:stdio
> > > -nographic
> > > *
> > > *
> > > *
> > > qemu-system-aarch64: GLib: g_mapped_file_unref:
> assertion 'file !=
> > NULL'
> > > failed *I get this error
> > > while trying to emulate raspi3.
> >
> > That sounds like a problem with Qemu. Is there some
> official test image
> > for rpi3 on qemu? Note that this isn't really relevant for
> your current
> > problem. So if you don't have some manual just ignore the
> question.
> >
> > >
> > > I ran qemu along with GDB to find what was causing the
> wrong output. I
> > > am really not sure if this is right,
> > > I still have a lot to learn, but my assumption's using
> GDB are as
> > follows.
> > > There are 4 active thread which run the same code.
> > >
> > > (gdb) info thread
> > > Id Target Id Frame
> > > * 1 Thread 1.1 (CPU#0 [running]) _start () at
> > >
> >
> ../../../../../../../../rtems/c/src/lib/libbsp/arm/raspberrypi/../../../../../../bsps/arm/shared/start/start.S:153
> > > 2 Thread 1.2 (CPU#1 [running]) _start () at
> > >
> >
> ../../../../../../../../rtems/c/src/lib/libbsp/arm/raspberrypi/../../../../../../bsps/arm/shared/start/start.S:153
> > > 3 Thread 1.3 (CPU#2 [running]) _start () at
> > >
> >
> ../../../../../../../../rtems/c/src/lib/libbsp/arm/raspberrypi/../../../../../../bsps/arm/shared/start/start.S:153
> > > 4 Thread 1.4 (CPU#3 [running]) _start () at
> > >
> >
> ../../../../../../../../rtems/c/src/lib/libbsp/arm/raspberrypi/../../../../../../bsps/arm/shared/start/start.S:153
> >
> > In this case that are not threads but it's the CPU cores.
> GDB shows them
> > as threads. Most likely it wouldn't be able to detect the
> RTEMS threads.
> >
> > It's a bit odd that they are all pointing to start.S:153.
> That's the
> > entry point for the program. It looks like not even one
> instruction has
> > been executed yet.
> >
> > I took this output before executing the program, that the
> reason why not
> > even a single instruction has been
> > executed yet.
>
> OK.
>
> >
> > >
> > > After some time one of the thread call's the BSP reset
> function
> > this is
> > > when the program crashes, the other threads complain
> "*executing
> > thread
> > > is NULL*"
> >
> > I would rather assume that one core tries to do the
> initialization while
> > the others hang in a endless loop till they are needed.
> The one core
> > doing the initialization work hits an exception somewhere
> and calls the
> > exception handler which calls the bsp reset function.
> >
> > The executing thread is NULL is a sign that it happens
> somewhere during
> > initialization when the RTEMS threading hasn't been
> started yet.
> >
> > The PC has an odd value. The linker command file tells
> that there is a
> > RAM_MMU at 0x00100000. It only puts a
> bsp_translation_table there but
> > there shouldn't be any code. So I don't know what the
> processor is doing
> > there. You could try to set a breakpoint on the address
> 0x00100fc4 and
> > take a look at why the processor is there with a "bt"
> (backtrace).
> >
> > When I re-run it again, it now stops at a different address.
> As you said
> > that the other cores are put
> > in an endless loop, I don't think that's is happening. I
> single stepped
> > the instruction and later at some point checked the threads
> >
> > (gdb) info threads
>
> >
>
> >
> > Target Id Frame
> > 1 Thread 1.1 (CPU#0 [running]) arm_ccsidr_get_line_power
> > (ccsidr=<optimized out>)
> > at
> >
> /home/niteesh/development/rtems/kernel/rtems/cpukit/score/cpu/arm/include/libcpu/arm-cp15.h:850
> > 2 Thread 1.2 (CPU#1 [running])
> arm_cp15_cache_invalidate_level
> > (inst_data_fl=0, level=1)
> > at
> >
> /home/niteesh/development/rtems/kernel/rtems/cpukit/score/cpu/arm/include/libcpu/arm-cp15.h:1162
> > 3 Thread 1.3 (CPU#2 [running]) arm_ccsidr_get_line_power
> > (ccsidr=<optimized out>)
> > at
> >
> /home/niteesh/development/rtems/kernel/rtems/cpukit/score/cpu/arm/include/libcpu/arm-cp15.h:850
> > * 4 Thread 1.4 (CPU#3 [running])
> > arm_cp15_get_cache_size_id_for_level (level_and_inst_dat=0)
> > at
> >
> /home/niteesh/development/rtems/kernel/rtems/cpukit/score/cpu/arm/include/libcpu/arm-cp15.h:936
> > (gdb)
> >
> > They all are executing different instructions at the same time.
>
> Some of the initialization is done on all cores. Some isn't. I
> took a
> look at the initialization and it seems that I was wrong: There
> is no
> wait loop. All processors are running through the initialization
> process. Some just skip parts. The part where they really start to
> differ is in bsp_start_hook_0.
>
> > I> googled about just running one thread or CPU as you said at
> a time and
> > used "*set scheduler-locking on" *on doing this I always get
> the right
> > output.
> >
> > (gdb) info threads
> > Id Target Id Frame
> > * 1 Thread 1.1 (CPU#0 [running]) bsp_reset ()
> > at
> >
> ../../../../../../../../rtems/c/src/lib/libbsp/arm/raspberrypi/../../../../../../bsps/arm/raspberrypi/start/bspreset.c:18
> > 2 Thread 1.2 (CPU#1 [running]) _start ()
> > at
> >
> ../../../../../../../../rtems/c/src/lib/libbsp/arm/raspberrypi/../../../../../../bsps/arm/shared/start/start.S:153
> > 3 Thread 1.3 (CPU#2 [running]) _start ()
> > at
> >
> ../../../../../../../../rtems/c/src/lib/libbsp/arm/raspberrypi/../../../../../../bsps/arm/shared/start/start.S:153
> > 4 Thread 1.4 (CPU#3 [running]) _start ()
> > at
> >
> ../../../../../../../../rtems/c/src/lib/libbsp/arm/raspberrypi/../../../../../../bsps/arm/shared/start/start.S:153
> > (gdb)
> >
> > The above command allow's only a single thread to run.
>
> Maybe there is a timing difference between the simulator and the
> real
> hardware. I'm not sure how well tested the SMP code is on the
> Raspberry.
> There can be a hidden bug.
>
> Just a guess: If there is a bug it could be possible that you hit it
> with your rpi3 too. Maybe it would be good to try a single core
> version
> of the BSP. I assume you have configured with "--enable-smp"?
> Can you
> try to build without it?
>
> I built 2 versions with SMP enabled and disabled, the one we are
> talking about is the SMP disabled version, I ran
> the example with SMP enabled, still, the error's are similar, I only
> difference is, in the disabled one, there are only 4 or less panic's
> (maybe corresponding to 4 cpu's) but the other one has a higher
> number of panics.
>
> > Won't it be a good idea to make a separate BSP for rpi3?
>
> As soon as it is necessary: Sure. But from what you told me it seems
> that the hardware is very similar so that we won't hit this
> point soon.
> Or do you already see differences that would make it necessary.
>
> I haven't had a look at the details but it could also be possible to
> unify the BSPs and entirely remove the rpi2 variant if the
> information
> from the flattened device tree are used.
>
> Can you explain how FDT work in RTEMS. Can you mention some BSP's
> which use FDT so I can use them as a reference to learn.
> I previously took a look at the beagle FDT project (#3784), you
> mentioned about hardcoded values and initialization functions, can
> you explain more about what exactly do the initialization functions
> do? Do they assign a function to a particular pin, like in raspi
> the pins are multiplexed for various functions, so do the
> initialization functions assign those pins to a particular function?
>
> And also please explain how does the initialization of the system
> happen from the DT file.
>
> >
> > > *** FATAL ***
> > > fatal source: 9 (RTEMS_FATAL_SOURCE_EXCEPTION)
> > >
> > > R0 = 0x400005e6 R8 = 0x00000000
> > > R1 = 0x00000001 R9 = 0x00000000
> > > R2 = 0xbffffa1a R10 = 0x00000000
> > > R3 = 0x00000000 R11 = 0x00000000
> > > R4 = 0x002001db R12 = 0x00000000
> > > R5 = 0x00000000 SP = 0x00300bd0
> > > R6 = 0x00000000 LR = 0x00100fc4
> > > R7 = 0x00000000 PC = 0x00100fc4
> > > CPSR = 0x000001d3 VEC = 0x00000002
> > > FPEXC = 0x40000000
> > > FPSCR = 0x00000000
> > > D00 = 0x0000000000000000
> > > D01 = 0x0000000000000000
> > > D02 = 0x0000000000000000
> > > D03 = 0x0000000000000000
> > > D04 = 0x0000000000000000
> > > D05 = 0x0000000000000000
> > > D06 = 0x0000000000000000
> > > D07 = 0x0000000000000000
> > > D08 = 0x0000000000000000
> > > D09 = 0x0000000000000000
> > > D10 = 0x0000000000000000
> > > D11 = 0x0000000000000000
> > > D12 = 0x0000000000000000
> > > D13 = 0x0000000000000000
> > > D14 = 0x0000000000000000
> > > D15 = 0x0000000000000000
> > > D16 = 0x0000000000000000
> > > D17 = 0x0000000000000010
> > > D18 = 0x0000000000000000
> > > D19 = 0x0000000000000000
> > > D20 = 0x0000000000000000
> > > D21 = 0x0000000000000000
> > > D22 = 0x0000000000000000
> > > D23 = 0x0000000000000000
> > > D24 = 0x0000000000000000
> > > D25 = 0x0000000000000000
> > > D26 = 0x0000000000000000
> > > D27 = 0x0000000000000000
> > > D28 = 0x0000000000000000
> > > D29 = 0x0000000000000000
> > > D30 = 0x0000000000000000
> > > D31 = 0x0000000000000000
> > > RTEMS version:
> > 5.0.0.c6d8589bb00a9d2a5a094c68c90290df1dc44807-modified
> > > RTEMS tools: 7.5.0 20191114 (RTEMS 5, RSB
> > > 83fa79314dd87c0a8c78fd642b2cea3138be8dd6, Newlib
> 3e24fbf6f)
> > > executing thread is NULL
> > >
> > > > The steps that I followed are:
> > > > 1. Created a bootable SD card using raspbian.
> > > > 2. Replaced the kernel.img file with RTEMS
> kernel.img file and
> > > modified
> > > > the config.txt to boot from the RTEMs kernel (boots in
> > aarch32 bit
> > > mode).
> > > > I am still not able to wrap my head around the RPI
> bsp build
> > process.
> > > > This is what I understood as of now, correct me if
> I am wrong.
> > > > Both RPi and Rpi2 are based on the same BSP, they just
> > differ in the
> > > > peripheral offsets, hardcoded checks are used to
> select the
> > right
> > > offset
> > > > at the time of compiling
> > >
> > > >From what I know of the Raspberry BSPs that is correct.
> > >
> > > > and the linkercmd file is responsible for
> > > > building the final executable file.
> > >
> > > The linkercmd file is - like for all programs -
> responsible
> > where the
> > > memory regions are that can be used for code or
> data. So you
> > could more
> > > or less explain it like you did.
> > >
> > > > I looked at the linker script, it seem's to have
> the start
> > section at
> > > > address 0x200000, I also loaded it in GDB and the
> start
> > address is
> > > > *Start address 0x200080,*
> > >
> > > I agree with that. The different start in GDB is
> most likely
> > because
> > > there is a vector table in front (at least if the
> Broadcom chip is
> > > similar to a lot of other processors that I have
> encountered).
> > >
> > > Does that mean that you have a debugger connected to the
> > raspberry? Can
> > > you load code with it? If yes: Is the bootloader
> executed
> > before you
> > > load your code? Otherwise the SDRAM might isn't
> initialized yet.
> > >
> > > I don't have a debugger connected to it. I from what I
> have SDRAM is
> > > initialized by the 3 stage bootloader(start.elf).
> >
> > That should be OK and it answers my question above.
> >
> > >
> > > > I did some bare metal programming on RPI3
> > > > there I had the start section at address 0x8000 Is
> this causing
> > > the problem?
> > >
> > > I assume that you used some internal RAM when you
> did bare metal
> > > programming. You maybe even skipped one or two
> bootloader
> > stages. From a
> > > quick look Raspberry has a quite complex boot
> process with at
> > least
> > > three bootloaders:
> > http://lions-wing.net/maker/raspberry-1/boot.html
> > >
> > > I don't think I have skipped any stages. The boot process is
> > exactly the
> > > same as how it boot's a normal raspbian or any other linux
> > > distro, I just to replace the linux kernel with my own
> kernel.
> >
> > Sounds reasonable. Does the bootloader print anything
> where it puts the
> > kernel image? Maybe the start address changed during the
> raspberry
> > versions.
> >
> > the default kernel load address is 0x8000 in 32bit mode and
> 0x80000 in
> > 64bit mode I have no idea about the raspberry 1,
> > but the load address is same for rpi2 and 3.
>
> That sounds odd. Do you have a memory map somewhere? From the linker
> command file it seems quite clear that RTEMS is build for a
> 0x200000.
>
> >
> > >
> > >
> > > > I have no idea on how to debug this, any
> suggestion on how
> > to start
> > > > would be really helpfull.
> > > >
> > >
> >
>
More information about the devel
mailing list