RTEMS_FATAL_SOURCE_EXCEPTION in RTEMS

Fri May 3 07:35:27 UTC 2019

On Thu, May 2, 2019 at 3:35 PM Christian Mauderer <
christian.mauderer at embedded-brains.de> wrote:

> ----- Ursprüngliche Mail -----
> > Von: "Amarnath MB" <amarnath.mb at mistralsolutions.com>
> > An: "Christian Mauderer" <christian.mauderer at embedded-brains.de>
> > CC: "RTEMS Users" <users at rtems.org>, "Ravi G Patil" <
> ravigp at mistralsolutions.com>, "Shekhar Suman Singh"
> > <shekhar.s at mistralsolutions.com>
> > Gesendet: Donnerstag, 2. Mai 2019 11:18:48
> > Betreff: Re: RTEMS_FATAL_SOURCE_EXCEPTION in RTEMS
>
> > Hi Christian,
> >
> > As per your suggestion, i tried disabling  global interrupt before my
> erase
> > call and enabling it afterwards. With this setup the test routine is
> > passing without giving exception. So i think, interrupt was the issue
> here.
> >
>
> Hello Amarnath,
>
> good. Than you have at least a hint toward the problem. I wouldn't see it
> as the solution. Like already discussed, depending on your application,
> this can lead to missed interrupts. If you have a very specific case for
> writing the flash that isn't during your normal operation (for example
> firmware updates) you are most likely fine with that solution. But if you
> write for example log files, you might get a quite unpredictable behavior
> later.
> Hi Christian,
>
> Thank you for the detailed explanation. Even though we are using NOR
> flash for firmware storage, there is a chance that it may be used for log
> files later.
>

> > Flash controller is implemented using the AHB to external SRAM interface
> in
> > the Cortex-M System Design Kit and there are no pins shared between RAM
> and
> > NOR Flash.  We can't doubt that flash is blocking bus accesses for the
> RAM,
> > because the test executes fine in the uboot running from RAM.
>
> So it's a custom FPGA based design? Most likely your RAM and Flash
> controller are both connected to the same system bus (AHB in that case). I
> don't know the modules you mentioned but you might want to take a look at
> the documentation whether they lock the AHB or not.
>
>  Yes its a custom FPGA based design. Sure, I will go through the
> documentation.

U-Boot most likely doesn't use interrupts. I assume you are waiting for a
> flash operation to be finished in some busy wait loop there (polling a flag
> or similar). That loop can be fully put into the processor cache (if you
> have one which is quite likely). So in your U-Boot test case, the external
> RAM most likely isn't accessed during the flash operations. If you have a problem
> with conflicting bus accesses you maybe only see them there if you disable
> all caches.
>
> You are correct, we are waiting in a loop polling the busy status of NOR
> flash. Currently, the caches are enabled in both uboot and RTEMS.
>
> In RTEMS on the other hand you are running with enabled interrupts and
> task switches. So RAM access are more or less guaranteed during a longer
> flash operation if you don't lock the interrupts.
>
> Okay, got it.
>
> >
> > You had mentioned there are a lot of possible reasons for a
> > RTEMS_FATAL_SOURCE_EXCEPTION, is it somewhere documented what all reasons
> > can cause this exception. Any references would be very much helpful.
>
> Basically for an ARM system, you have that source:
>
> cpukit/score/cpu/arm/arm-exception-default.c:24:  rtems_fatal(
> RTEMS_FATAL_SOURCE_EXCEPTION, (rtems_fatal_code) frame );
>
> That is default handler for all exceptions where no extra handler is
> installed. So basically every exception that is listed in the ARM manual
> can be the source.
>
Thank you that was helpful.
>

> >
> > Thank you once again for your prompt response, it really saved lot of our
> > time.
> >
> > Below is the exception frame we got,
> > FYI, 0x0 - 0x007FFFFF is NOR Flash and 0x40000000 to 0x7FFFFFFF the RAM.
>
> Yust a note: Putting memory at 0 might hides some NULL-pointer accesses.
> If you can do that in your design you might want to think about putting
> some boot ROM there that isn't accessible by the application (for example
> locked via MPU).
>
> I got it. We have kept uboot at 0x0 and RTEMS at 0x40000 offset of flash,
> we have restricted access from application to these uboot and RTEMS
> sectors.
>
> Of course there can be worse types of memory Flash (which should be mostly
> a read only memory). I had a controller with some RAM there once. Finding
> NULL-pointer accesses on such a system is really nasty.
>
> >
> > *** FATAL ***
> > fatal source: 9 (RTEMS_FATAL_SOURCE_EXCEPTION)
> >
> > R0   = 0x00000040 R8  = 0x00100000
> > R1   = 0x00080000 R9  = 0x00000001
> > R2   = 0x00000048 R10 = 0x4010a3e8
> > R3   = 0x00000048 R11 = 0x4011737c
> > R4   = 0x00000a00 R12 = 0x00000000
> > R5   = 0x00000500 SP  = 0x40100c40
> > R6   = 0x00000055 LR  = 0x4000a36c
> > R7   = 0x000000aa PC  = 0x003fde80
> > CPSR = 0x200000d2 VEC = 0x00000001
> > RTEMS version: 5.0.0.
> > RTEMS tools: 7.4.0 20181206 (RTEMS 5, RSB
> > 40ae056f12e1cbe530f76a3ebd1e2ac745a888ef, Newlib
> > dc6e94551f09d3a983afd571478d63a09d6f66fa)
> > executing thread ID: 0x08a010002
> > executing thread name: Alpi
>
> Your program counter (PC) points to a Flash address here. Are you sure
> that your application runs entirely from RAM?
>
> We are sure that RTEMS is executing entirely from RAM, we are debugging on
> this issue.
>
> Best regards
>
> Christian
>
> >
> > *Thank you & Regards,*
> > *Amarnath MB*
> >
> >
> >
> > On Thu, May 2, 2019 at 12:49 AM Christian Mauderer <
> > christian.mauderer at embedded-brains.de> wrote:
> >
> >>
> >> ----- Ursprüngliche Mail -----
> >> > Von: "Amarnath MB" <amarnath.mb at mistralsolutions.com>
> >> > An: "Christian Mauderer" <christian.mauderer at embedded-brains.de>
> >> > CC: "RTEMS Users" <users at rtems.org>, "Ravi G Patil" <
> >> ravigp at mistralsolutions.com>, "Shekhar Suman Singh"
> >> > <shekhar.s at mistralsolutions.com>
> >> > Gesendet: Mittwoch, 1. Mai 2019 17:07:13
> >> > Betreff: Re: RTEMS_FATAL_SOURCE_EXCEPTION in RTEMS
> >>
> >> > Hi Christian,
> >> >
> >> > Thanks for your quick response.
> >> > Our boot process is like, first u-boot will be loaded from the NOR
> flash
> >> > and then the uboot copies RTEMS application from NOR flash to the RAM.
> >> > After that RTEMS executes entirely from RAM.
> >> >
> >> > How can i issue a global interrupt disable? Is it using *
> >> > rtems_interrupt_disable(0)*?
> >> > One more doubt, if i issue a global interrupt disable then will there
> be
> >> > chance that i can miss few clock ticks?
> >> >
> >> > *Thank you & Regards,*
> >> > *Amarnath MB*
> >> >
> >>
> >> Hello Amarnath,
> >>
> >> with a global interrupt lock, all kinds of events can be missed
> including
> >> clock ticks. Most should be processed just a little late but some
> >> interfaces might loose packets or data (depending on the data rate /
> flash
> >> access times). So if not necessary it's not a good solution.
> >>
> >> The rtems_interrupt_disable() function should be called with a level
> >> argument. See the example at
> >>
> https://docs.rtems.org/branches/master/c-user/interrupt_manager.html#rtems-interrupt-disable
> .
> >> Note that for SMP configurations, you might have to use other functions.
> >>
> >> But if your application runs entirely from RAM than the bus access
> >> shouldn't be the problem. So the interrupt might isn't the right guess.
> >>
> >> Do you have any more information on the exception and where it happens?
> >> Some output or a stack trace from a debugger?
> >>
> >> What kind of flash controller is used? Can it block bus accesses for the
> >> RAM? Does your RAM share pins with the flash?
> >>
> >> With kind regards
> >>
> >> Christian
> >>
> >> >
> >> >
> >> > On Wed, May 1, 2019 at 8:16 PM Christian Mauderer <
> >> > christian.mauderer at embedded-brains.de> wrote:
> >> >
> >> >> ----- Ursprüngliche Mail -----
> >> >> > Von: "Amarnath MB" <amarnath.mb at mistralsolutions.com>
> >> >> > An: "RTEMS Users" <users at rtems.org>
> >> >> > CC: "Ravi G Patil" <ravigp at mistralsolutions.com>, "Shekhar Suman
> >> Singh"
> >> >> <shekhar.s at mistralsolutions.com>
> >> >> > Gesendet: Mittwoch, 1. Mai 2019 16:28:23
> >> >> > Betreff: RTEMS_FATAL_SOURCE_EXCEPTION in RTEMS
> >> >>
> >> >> > Hi All,
> >> >> >
> >> >> > I'm developing RTEMS 5.00 BSP and device drivers for a custom
> >> ARM926EJ-S
> >> >> > core. I was successful in porting and building BSP for the ARM
> core.
> >> >> >
> >> >> > We are facing a strange issue with the generic NOR flash driver we
> >> have
> >> >> > developed. Our device drivers are designed such that it can be used
> >> with
> >> >> > RTEMS as well as the bare metal programs.
> >> >> > Our driver is working fine in bare metal program (erase, read,
> write
> >> >> > everything), but when we use the driver with RTEMS application and
> >> issue
> >> >> an
> >> >> > erase call, then the application gives
> RTEMS_FATAL_SOURCE_EXCEPTION.
> >> >> >
> >> >> > For testing driver in RTEMS, we have added each test routine as a
> >> custom
> >> >> > shell command using rtems_shell_add_cmd().
> >> >> >
> >> >> > For your info we also tested the same driver with u-boot and it
> works
> >> >> > fines.
> >> >> > Can anyone guide me on this issue?
> >> >> >
> >> >> > *Thank you & Regards,*
> >> >> > *Amarnath MB*
> >> >> >
> >> >>
> >> >> Hello Amarnath,
> >> >>
> >> >> it's a little hard with the given information to tell you a concrete
> >> >> problem. There are a lot of possible reasons for a
> >> >> RTEMS_FATAL_SOURCE_EXCEPTION.
> >> >>
> >> >> From your description, my guess would be some problem with an
> interrupt.
> >> >> Most U-Boot code avoids interrupts. I don't know about your bare
> metal
> >> >> application but maybe in some test application, you don't have too
> much
> >> >> interrupts too. In RTEMS you have most likely at least a periodic
> tick
> >> >> interrupt.
> >> >>
> >> >> Do you execute your code from the same flash or do you keep some
> data in
> >> >> the flash? In that case it could be an access error during a flash
> >> erase or
> >> >> write (for example caused by an interrupt). In that case you might
> have
> >> to
> >> >> do a global interrupt disable before you enter certain routines.
> >> >>
> >> >> Best regards
> >> >>
> >> >> Christian
> >>
>
> --
> --------------------------------------------
> embedded brains GmbH
> Christian Mauderer
> Dornierstr. 4
> D-82178 Puchheim
> Germany
> email: christian.mauderer at embedded-brains.de
> Phone: +49-89-18 94 741 - 18
> Fax:   +49-89-18 94 741 - 08
> PGP: Public key available on request.
>
> Diese Nachricht ist keine geschäftliche Mitteilung im Sinne des EHUG.
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.rtems.org/pipermail/users/attachments/20190503/85e50a62/attachment-0002.html>