Problem running RTEMS on raspberrypi3

Niteesh gsnb.gn at gmail.com
Mon Dec 16 13:01:15 UTC 2019


I got uboot running on my raspi3. But I can't figure out to load and run a
custom kernel. Can you explain the steps or point me to some
reference.
On Mon, Dec 16, 2019 at 5:13 PM Niteesh <gsnb.gn at gmail.com> wrote:

> On Mon, Dec 16, 2019 at 2:36 AM Christian Mauderer <list at c-mauderer.de>
> wrote:
>
>>
>>
>> On 15/12/2019 21:29, Niteesh wrote:
>> >
>> >
>> > On Mon, Dec 16, 2019 at 12:53 AM Christian Mauderer <list at c-mauderer.de
>> > <mailto:list at c-mauderer.de>> wrote:
>> >
>> >     On 15/12/2019 19:46, Niteesh wrote:
>> >     >
>> >     >
>> >     > On Sun, Dec 15, 2019 at 10:15 PM Christian Mauderer
>> >     <list at c-mauderer.de <mailto:list at c-mauderer.de>
>> >     > <mailto:list at c-mauderer.de <mailto:list at c-mauderer.de>>> wrote:
>> >     >
>> >     >     Hello Niteesh,
>> >     >
>> >     >     On 15/12/2019 09:05, Niteesh wrote:
>> >     >     > I am trying to get RTEMS examples running on the RPI3, the
>> >     RPI3 is
>> >     >     > similar to RPI2 so the examples built for RPI2 should
>> >     technically
>> >     >     run on
>> >     >     > the RPi3.But they don't :(, I am really sure of what is
>> causing
>> >     >     the problem.
>> >     >
>> >     >     Note that there are at least two different versions of the
>> >     RPi3 which
>> >     >     use different chips. The original RPi3 which uses a BCM2837
>> >     (same like
>> >     >     later versions of RPi2) and the RPi3+ which uses a BCM2837B0.
>> >     Broadcom
>> >     >     is always quite sparse with documentation so it's not easy to
>> >     tell the
>> >     >     differences. Which one do you have?
>> >     >
>> >     > I have Rpi3 model b v1.2 which uses BCM2837 SOC, in my bare-metal
>> >     > programming I used the
>> >     > 2835 doc as a reference because the only major difference these
>> >     two SOC
>> >     > is the peripheral base address
>> >     > offset. But this is arm cpu is also capable of executing in 64bit
>> >     mode.
>> >
>> >     OK. Did you check, whether the offset is correct? In the
>> raspberrypi.h
>> >     in RTEMS there is the following define:
>> >
>> >     #if (BSP_IS_RPI2 == 1)
>> >        #define RPI_PERIPHERAL_BASE      0x3F000000
>> >     #else
>> >        #define RPI_PERIPHERAL_BASE      0x20000000
>> >     #endif
>> >
>> > The offsets are right.
>>
>> Good.
>>
>> >
>> >     >
>> >     >     > I followed the steps
>> >     >     >
>> >     >
>> >      from
>> http://alanstechnotes.blogspot.com/2013/03/running-your-first-rtems-program-on.html
>>  (modified
>> >     >     > commands to use rtems5) to build the kernel img.
>> >     >
>> >     >     It's a bit odd that the Bootloader doesn't use some image
>> >     format like
>> >     >     U-Boot but if that's the case for Raspberry, that's OK.
>> >     >
>> >     > Do you want me to try U-Boot, I was planning to use it for my
>> >     bare-metal
>> >     > stuff because copying the kernel
>> >     > to SD-card was a real pain. Will it even work with RTEMS?
>> >
>> >     The manual that you linked uses the default Raspberry bootloader.
>> I'm
>> >     not sure whether it's an U-Boot. If you skip the bootloader
>> entirely,
>> >     your SDRAM might isn't initialized.
>> >
>> > The manual uses the default bootloader. I don't think we have to worry
>> > about the SDRAM initialization
>> > because all of that is taken care of by the GPU.
>>
>> Sounds OK.
>>
>> > When using Uboot, the
>> > GPU will load the uboot image and
>> > pass the control to the CPU. And then the uboot continue's it's
>> execution.
>> >
>>
>> I don't wanted to suggest to use an extra U-Boot. I was just not sure
>> whether the stage 3 loader is an U-Boot. Your approach sounds fine so far.
>>
>> >
>> >
>> >     PS: You answered that further below. You are using the stage 3
>> loader.
>> >
>> >     >
>> >     >     > I did try running it on
>> >     >     > Qemu but it doesn't always work, sometimes it gives
>> >     weird output.
>> >     >
>> >     >     How did you run it on Qemu? Did you build some image for that
>> too?
>> >     >
>> >     > qemu-system-arm -M raspi2 -m 1G -kernel hello.exe -serial
>> mon:stdio
>> >     > -nographic
>> >     > *
>> >     > *
>> >     > *
>> >     > qemu-system-aarch64: GLib: g_mapped_file_unref: assertion 'file !=
>> >     NULL'
>> >     > failed *I get this error
>> >     > while trying to emulate raspi3.
>> >
>> >     That sounds like a problem with Qemu. Is there some official test
>> image
>> >     for rpi3 on qemu? Note that this isn't really relevant for your
>> current
>> >     problem. So if you don't have some manual just ignore the question.
>> >
>> >     >
>> >     > I ran qemu along with GDB to find what was causing the wrong
>> output. I
>> >     > am really not sure if this is right,
>> >     > I still have a lot to learn, but my assumption's using GDB are as
>> >     follows.
>> >     > There are 4 active thread which run the same code.
>> >     >
>> >     >     (gdb) info thread
>> >     >       Id   Target Id                    Frame
>> >     >     * 1    Thread 1.1 (CPU#0 [running]) _start () at
>> >     >
>> >
>>   ../../../../../../../../rtems/c/src/lib/libbsp/arm/raspberrypi/../../../../../../bsps/arm/shared/start/start.S:153
>> >     >       2    Thread 1.2 (CPU#1 [running]) _start () at
>> >     >
>> >
>>   ../../../../../../../../rtems/c/src/lib/libbsp/arm/raspberrypi/../../../../../../bsps/arm/shared/start/start.S:153
>> >     >       3    Thread 1.3 (CPU#2 [running]) _start () at
>> >     >
>> >
>>   ../../../../../../../../rtems/c/src/lib/libbsp/arm/raspberrypi/../../../../../../bsps/arm/shared/start/start.S:153
>> >     >       4    Thread 1.4 (CPU#3 [running]) _start () at
>> >     >
>> >
>>   ../../../../../../../../rtems/c/src/lib/libbsp/arm/raspberrypi/../../../../../../bsps/arm/shared/start/start.S:153
>> >
>> >     In this case that are not threads but it's the CPU cores. GDB shows
>> them
>> >     as threads. Most likely it wouldn't be able to detect the RTEMS
>> threads.
>> >
>> >     It's a bit odd that they are all pointing to start.S:153. That's the
>> >     entry point for the program. It looks like not even one instruction
>> has
>> >     been executed yet.
>> >
>> > I took this output before executing the program, that the reason why not
>> > even a single instruction has been
>> > executed yet.
>>
>> OK.
>>
>> >
>> >     >
>> >     > After some time one of the thread call's the BSP reset function
>> >     this is
>> >     > when the program crashes, the other threads complain "*executing
>> >     thread
>> >     > is NULL*"
>> >
>> >     I would rather assume that one core tries to do the initialization
>> while
>> >     the others hang in a endless loop till they are needed. The one core
>> >     doing the initialization work hits an exception somewhere and calls
>> the
>> >     exception handler which calls the bsp reset function.
>> >
>> >     The executing thread is NULL is a sign that it happens somewhere
>> during
>> >     initialization when the RTEMS threading hasn't been started yet.
>> >
>> >     The PC has an odd value. The linker command file tells that there
>> is a
>> >     RAM_MMU at 0x00100000. It only puts a bsp_translation_table there
>> but
>> >     there shouldn't be any code. So I don't know what the processor is
>> doing
>> >     there. You could try to set a breakpoint on the address 0x00100fc4
>> and
>> >     take a look at why the processor is there with a "bt" (backtrace).
>> >
>> > When I re-run it again, it now stops at a different address. As you said
>> > that the other cores are put
>> > in an endless loop, I don't think that's is happening. I single stepped
>> > the instruction and later at some point checked the threads
>> >
>> >     (gdb) info threads
>> >
>> >
>> >         Target Id                    Frame
>> >       1    Thread 1.1 (CPU#0 [running]) arm_ccsidr_get_line_power
>> >     (ccsidr=<optimized out>)
>> >         at
>> >
>>  /home/niteesh/development/rtems/kernel/rtems/cpukit/score/cpu/arm/include/libcpu/arm-cp15.h:850
>> >       2    Thread 1.2 (CPU#1 [running]) arm_cp15_cache_invalidate_level
>> >     (inst_data_fl=0, level=1)
>> >        at
>> >
>>  /home/niteesh/development/rtems/kernel/rtems/cpukit/score/cpu/arm/include/libcpu/arm-cp15.h:1162
>> >      3    Thread 1.3 (CPU#2 [running]) arm_ccsidr_get_line_power
>> >     (ccsidr=<optimized out>)
>> >        at
>> >
>>  /home/niteesh/development/rtems/kernel/rtems/cpukit/score/cpu/arm/include/libcpu/arm-cp15.h:850
>> >     * 4    Thread 1.4 (CPU#3 [running])
>> >     arm_cp15_get_cache_size_id_for_level (level_and_inst_dat=0)
>> >         at
>> >
>>  /home/niteesh/development/rtems/kernel/rtems/cpukit/score/cpu/arm/include/libcpu/arm-cp15.h:936
>> >     (gdb)
>> >
>> > They all are executing different instructions at the same time.
>>
>> Some of the initialization is done on all cores. Some isn't. I took a
>> look at the initialization and it seems that I was wrong: There is no
>> wait loop. All processors are running through the initialization
>> process. Some just skip parts. The part where they really start to
>> differ is in bsp_start_hook_0.
>>
>> > I> googled about just running one thread or CPU as you said at a time
>> and
>> > used "*set scheduler-locking on" *on doing this I always get the right
>> > output.
>> >
>> >     (gdb) info threads
>> >       Id   Target Id                    Frame
>> >     * 1    Thread 1.1 (CPU#0 [running]) bsp_reset ()
>> >         at
>> >
>>  ../../../../../../../../rtems/c/src/lib/libbsp/arm/raspberrypi/../../../../../../bsps/arm/raspberrypi/start/bspreset.c:18
>> >       2    Thread 1.2 (CPU#1 [running]) _start ()
>> >         at
>> >
>>  ../../../../../../../../rtems/c/src/lib/libbsp/arm/raspberrypi/../../../../../../bsps/arm/shared/start/start.S:153
>> >       3    Thread 1.3 (CPU#2 [running]) _start ()
>> >         at
>> >
>>  ../../../../../../../../rtems/c/src/lib/libbsp/arm/raspberrypi/../../../../../../bsps/arm/shared/start/start.S:153
>> >       4    Thread 1.4 (CPU#3 [running]) _start ()
>> >         at
>> >
>>  ../../../../../../../../rtems/c/src/lib/libbsp/arm/raspberrypi/../../../../../../bsps/arm/shared/start/start.S:153
>> >     (gdb)
>> >
>> > The above command allow's only a single thread to run.
>>
>> Maybe there is a timing difference between the simulator and the real
>> hardware. I'm not sure how well tested the SMP code is on the Raspberry.
>> There can be a hidden bug.
>>
>> Just a guess: If there is a bug it could be possible that you hit it
>> with your rpi3 too. Maybe it would be good to try a single core version
>> of the BSP. I assume you have configured with "--enable-smp"? Can you
>> try to build without it?
>>
> I built 2 versions with SMP enabled and disabled, the one we are talking
> about is the SMP disabled version, I ran
> the example with SMP enabled, still, the error's are similar, I only
> difference is, in the disabled one, there are only 4 or less panic's
> (maybe corresponding to 4 cpu's) but the other one has a higher number of
> panics.
>
> > Won't it be a good idea to make a separate BSP for rpi3?
>>
>> As soon as it is necessary: Sure. But from what you told me it seems
>> that the hardware is very similar so that we won't hit this point soon.
>> Or do you already see differences that would make it necessary.
>>
>> I haven't had a look at the details but it could also be possible to
>> unify the BSPs and entirely remove the rpi2 variant if the information
>> from the flattened device tree are used.
>>
> Can you explain how FDT work in RTEMS. Can you mention some BSP's which
> use FDT so I can use them as a reference to learn.
> I previously took a look at the beagle FDT project (#3784), you mentioned
> about hardcoded values and initialization functions, can
> you explain more about what exactly do the initialization functions do? Do
> they assign a function to a particular pin, like in raspi
> the pins are multiplexed for various functions, so do the initialization
> functions assign those pins to a particular function?
>
> And also please explain how does the initialization of the system happen
> from the DT file.
>
>> >
>> >     >     *** FATAL ***
>> >     >     fatal source: 9 (RTEMS_FATAL_SOURCE_EXCEPTION)
>> >     >
>> >     >     R0   = 0x400005e6 R8  = 0x00000000
>> >     >     R1   = 0x00000001 R9  = 0x00000000
>> >     >     R2   = 0xbffffa1a R10 = 0x00000000
>> >     >     R3   = 0x00000000 R11 = 0x00000000
>> >     >     R4   = 0x002001db R12 = 0x00000000
>> >     >     R5   = 0x00000000 SP  = 0x00300bd0
>> >     >     R6   = 0x00000000 LR  = 0x00100fc4
>> >     >     R7   = 0x00000000 PC  = 0x00100fc4
>> >     >     CPSR = 0x000001d3 VEC = 0x00000002
>> >     >     FPEXC = 0x40000000
>> >     >     FPSCR = 0x00000000
>> >     >     D00 = 0x0000000000000000
>> >     >     D01 = 0x0000000000000000
>> >     >     D02 = 0x0000000000000000
>> >     >     D03 = 0x0000000000000000
>> >     >     D04 = 0x0000000000000000
>> >     >     D05 = 0x0000000000000000
>> >     >     D06 = 0x0000000000000000
>> >     >     D07 = 0x0000000000000000
>> >     >     D08 = 0x0000000000000000
>> >     >     D09 = 0x0000000000000000
>> >     >     D10 = 0x0000000000000000
>> >     >     D11 = 0x0000000000000000
>> >     >     D12 = 0x0000000000000000
>> >     >     D13 = 0x0000000000000000
>> >     >     D14 = 0x0000000000000000
>> >     >     D15 = 0x0000000000000000
>> >     >     D16 = 0x0000000000000000
>> >     >     D17 = 0x0000000000000010
>> >     >     D18 = 0x0000000000000000
>> >     >     D19 = 0x0000000000000000
>> >     >     D20 = 0x0000000000000000
>> >     >     D21 = 0x0000000000000000
>> >     >     D22 = 0x0000000000000000
>> >     >     D23 = 0x0000000000000000
>> >     >     D24 = 0x0000000000000000
>> >     >     D25 = 0x0000000000000000
>> >     >     D26 = 0x0000000000000000
>> >     >     D27 = 0x0000000000000000
>> >     >     D28 = 0x0000000000000000
>> >     >     D29 = 0x0000000000000000
>> >     >     D30 = 0x0000000000000000
>> >     >     D31 = 0x0000000000000000
>> >     >     RTEMS version:
>> >     5.0.0.c6d8589bb00a9d2a5a094c68c90290df1dc44807-modified
>> >     >     RTEMS tools: 7.5.0 20191114 (RTEMS 5, RSB
>> >     >     83fa79314dd87c0a8c78fd642b2cea3138be8dd6, Newlib 3e24fbf6f)
>> >     >     executing thread is NULL
>> >     >
>> >     >     > The steps that I followed are:
>> >     >     > 1. Created a bootable SD card using raspbian.
>> >     >     > 2. Replaced the kernel.img file with RTEMS kernel.img file
>> and
>> >     >     modified
>> >     >     > the config.txt to boot from the RTEMs kernel (boots in
>> >     aarch32 bit
>> >     >     mode).
>> >     >     > I am still not able to wrap my head around the RPI bsp build
>> >     process.
>> >     >     > This is what I understood as of now, correct me if I am
>> wrong.
>> >     >     > Both RPi and Rpi2 are based on the same BSP, they just
>> >     differ in the
>> >     >     > peripheral offsets, hardcoded checks are used to select the
>> >     right
>> >     >     offset
>> >     >     > at the time of compiling
>> >     >
>> >     >     >From what I know of the Raspberry BSPs that is correct.
>> >     >
>> >     >     > and the linkercmd file is responsible for
>> >     >     > building the final executable file.
>> >     >
>> >     >     The linkercmd file is - like for all programs - responsible
>> >     where the
>> >     >     memory regions are that can be used for code or data. So you
>> >     could more
>> >     >     or less explain it like you did.
>> >     >
>> >     >     > I looked at the linker script, it seem's to have the start
>> >     section at
>> >     >     > address 0x200000, I also loaded it in GDB and the start
>> >     address is
>> >     >     > *Start address 0x200080,*
>> >     >
>> >     >     I agree with that. The different start in GDB is most likely
>> >     because
>> >     >     there is a vector table in front (at least if the Broadcom
>> chip is
>> >     >     similar to a lot of other processors that I have encountered).
>> >     >
>> >     >     Does that mean that you have a debugger connected to the
>> >     raspberry? Can
>> >     >     you load code with it? If yes: Is the bootloader executed
>> >     before you
>> >     >     load your code? Otherwise the SDRAM might isn't initialized
>> yet.
>> >     >
>> >     > I don't have a debugger connected to it. I from what I have SDRAM
>> is
>> >     > initialized by the 3 stage bootloader(start.elf).
>> >
>> >     That should be OK and it answers my question above.
>> >
>> >     >
>> >     >     > I did some bare metal programming on RPI3
>> >     >     > there I had the start section at address 0x8000 Is this
>> causing
>> >     >     the problem?
>> >     >
>> >     >     I assume that you used some internal RAM when you did bare
>> metal
>> >     >     programming. You maybe even skipped one or two bootloader
>> >     stages. From a
>> >     >     quick look Raspberry has a quite complex boot process with at
>> >     least
>> >     >     three bootloaders:
>> >     http://lions-wing.net/maker/raspberry-1/boot.html
>> >     >
>> >     > I don't think I have skipped any stages. The boot process is
>> >     exactly the
>> >     > same as how it boot's a normal raspbian or any other linux
>> >     > distro, I just to replace the linux kernel with my own kernel.
>> >
>> >     Sounds reasonable. Does the bootloader print anything where it puts
>> the
>> >     kernel image? Maybe the start address changed during the raspberry
>> >     versions.
>> >
>> > the default kernel load address is 0x8000 in 32bit mode and 0x80000 in
>> > 64bit mode I have no idea about the raspberry 1,
>> > but the load address is same for rpi2 and 3.
>>
>> That sounds odd. Do you have a memory map somewhere? From the linker
>> command file it seems quite clear that RTEMS is build for a 0x200000.
>>
>> >
>> >     >
>> >     >
>> >     >     > I have no idea on how to debug this, any suggestion on how
>> >     to start
>> >     >     > would be really helpfull.
>> >     >     >
>> >     >
>> >
>>
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.rtems.org/pipermail/devel/attachments/20191216/ebcad074/attachment-0001.html>


More information about the devel mailing list