<div dir="ltr"><div class="gmail_default" style="font-size:small">Hi Till, all:</div><div class="gmail_default" style="font-size:small"><br></div><div class="gmail_default" style="font-size:small">Last week we recommissioned one of our telescope systems using RTEMS and EPICS on the MVME2307 BSP. After nearly two days of uptime (~42 hours), we get various unrecoverable exceptions (shown below). This occurs suddenly when the system is "idle" and not being used for many hours.</div><div class="gmail_default" style="font-size:small"><br></div><div class="gmail_default" style="font-size:small">Any suggestions tracking down this stack trace, finding the offending application code, or getting this to fail-faster? </div><div class="gmail_default" style="font-size:small">We've produced an object dump available at the following link: <a href="https://github.com/rcardenes/crcs-info">https://github.com/rcardenes/crcs-info</a></div><div class="gmail_default" style="font-size:small"><br></div><div class="gmail_default" style="font-size:small">Has anyone else used the MVME2307 BSP?</div><div class="gmail_default" style="font-size:small"><br></div><div class="gmail_default" style="font-size:small">Thanks,</div><div class="gmail_default" style="font-size:small">-Matt</div><div class="gmail_default" style="font-size:small"><br></div><div class="gmail_default" style="font-size:small">System details:</div><div class="gmail_default" style="font-size:small">RTEMS: 4.10.2</div><div class="gmail_default" style="font-size:small">EPICS: 3.14.12.4. </div><div class="gmail_default" style="font-size:small">Hardware: MVME-2700 (using mvme2307 alias)</div><div class="gmail_default" style="font-size:small">VME Boards: </div><div class="gmail_default" style="font-size:small"><ol><li>Bancomm 635 Time Board<br></li><li>PMAC 1</li><li>Xycom-240</li></ol></div><div class="gmail_default" style="font-size:small"><br></div><div class="gmail_default" style="font-size:small"><br></div><div class="gmail_default" style="font-size:small"><b>Event #1:</b></div><div class="gmail_default" style="font-size:small"><blockquote class="gmail_quote" style="margin:0px 0px 0px 0.8ex;border-left:1px solid rgb(204,204,204);padding-left:1ex"><span style="font-family:monospace"><span style="color:rgb(0,0,0)">Jun 23 15:19:43 E) PORT: crcs, MSG: </span><span style="color:rgb(255,255,255);background-color:rgb(0,0,0)">Exception</span><span style="color:rgb(0,0,0)"> handler called for exception 7 (0x7) <br></span></span><span style="font-family:monospace">Jun 23 15:19:43 E) PORT: crcs, MSG: #011 Next PC or Address of fault = 011B010C <br></span><span style="font-family:monospace">Jun 23 15:19:43 E) PORT: crcs, MSG: #011 Saved MSR = 0008B032 <br></span><span style="font-family:monospace">Jun 23 15:19:43 E) PORT: crcs, MSG: #011 Context: Task ID 0x0A01004B <br></span><span style="font-family:monospace">Jun 23 15:19:43 E) PORT: crcs, MSG: #011 R0 = 01010101 R1 = 00AF5A40 R2 = 00000000 R3 = 007F8014 <br></span><span style="font-family:monospace">Jun 23 15:19:43 E) PORT: crcs, MSG: #011 R4 = 00000014 R5 = 007AD148 R6 = 00AF62BC R7 = 00000000 <br></span><span style="font-family:monospace">Jun 23 15:19:43 E) PORT: crcs, MSG: #011 R8 = 000031A8 R9 = 011B0101 R10 = 00AF62B8 R11 = 007AD148 <br></span><span style="font-family:monospace">Jun 23 15:19:44 E) PORT: crcs, MSG: #011 R12 = 22000022 R13 = 001E4B90 R14 = AEFE7BBF R15 = 77FF7FBB <br></span><span style="font-family:monospace">Jun 23 15:19:44 E) PORT: crcs, MSG: #011 R16 = AA7FBFFC R17 = FF7FF5FF R18 = 001E0000 R19 = 00000000 <br></span><span style="font-family:monospace">Jun 23 15:19:44 E) PORT: crcs, MSG: #011 R20 = 000634A0 R21 = 001929E8 R22 = 001A2284 R23 = 00191098 <br></span><span style="font-family:monospace">Jun 23 15:19:44 E) PORT: crcs, MSG: #011 R24 = 00000000 R25 = 00000001 R26 = 00000001 R27 = 001A85C4 <br></span><span style="font-family:monospace">Jun 23 15:19:44 E) PORT: crcs, MSG: #011 R28 = 00AF62BC R29 = 00000028 R30 = 00419260 R31 = 007AD148 <br></span><span style="font-family:monospace">Jun 23 15:19:44 E) PORT: crcs, MSG: #011 CR = 22042028 <br></span><span style="font-family:monospace">Jun 23 15:19:44 E) PORT: crcs, MSG: #011 CTR = 011B00FF <br></span><span style="font-family:monospace">Jun 23 15:19:44 E) PORT: crcs, MSG: #011 XER = 20000000 <br></span><span style="font-family:monospace">Jun 23 15:19:44 E) PORT: crcs, MSG: #011 LR = 0006371C <br></span><span style="font-family:monospace">Jun 23 15:19:44 E) PORT: crcs, MSG: #011 DAR = 00000000 <br></span><span style="font-family:monospace">Jun 23 15:19:44 E) PORT: crcs, MSG: Stack Trace: <br></span><span style="font-family:monospace">Jun 23 15:19:44 E) PORT: crcs, MSG: IP: 0x011B010C, LR: 0x0006371C <br></span><span style="font-family:monospace">Jun 23 15:19:44 E) PORT: crcs, MSG: --^ 0x0006371C--^ 0x00082FDC--^ 0x000E7094--^ 0x00136048--^ 0x00135F6C <br></span><span style="font-family:monospace">Jun 23 15:19:44 E) PORT: crcs, MSG: Suspending faulting task (0x0A01004B) </span></blockquote></div><div class="gmail_default" style="font-size:small">...</div><div class="gmail_default" style="font-size:small"><br></div><div class="gmail_default" style="font-size:small">Using the memory map for the Exception 7 we reach the following trace:<br></div><div class="gmail_default"><div class="gmail_default"><br></div></div><blockquote style="margin:0px 0px 0px 40px;border:none;padding:0px"><div class="gmail_default"><blockquote class="gmail_quote" style="margin:0px 0px 0px 0.8ex;border-left:1px solid rgb(204,204,204);padding-left:1ex">0x00135f6c _Thread_Handler</blockquote></div><div class="gmail_default"><blockquote class="gmail_quote" style="margin:0px 0px 0px 0.8ex;border-left:1px solid rgb(204,204,204);padding-left:1ex">/gem_sw/targetOS/RTEMS/source/rtems/rtems-4.10.2/cpukit/score/src/threadhandler.c:80</blockquote></div><div class="gmail_default"><blockquote class="gmail_quote" style="margin:0px 0px 0px 0.8ex;border-left:1px solid rgb(204,204,204);padding-left:1ex">0x00136048</blockquote></div><div class="gmail_default"><blockquote class="gmail_quote" style="margin:0px 0px 0px 0.8ex;border-left:1px solid rgb(204,204,204);padding-left:1ex">/gem_sw/targetOS/RTEMS/source/rtems/rtems-4.10.2/cpukit/score/src/threadhandler.c:145</blockquote></div><div class="gmail_default"><blockquote class="gmail_quote" style="margin:0px 0px 0px 0.8ex;border-left:1px solid rgb(204,204,204);padding-left:1ex">0x000e7094</blockquote></div><div class="gmail_default"><blockquote class="gmail_quote" style="margin:0px 0px 0px 0.8ex;border-left:1px solid rgb(204,204,204);padding-left:1ex">/gem_sw/epics/R3.14.12.4/base/src/libCom/osi/os/RTEMS/osdThread.c:169</blockquote></div><div class="gmail_default"><blockquote class="gmail_quote" style="margin:0px 0px 0px 0.8ex;border-left:1px solid rgb(204,204,204);padding-left:1ex">0x00082fdc</blockquote></div><div class="gmail_default"><blockquote class="gmail_quote" style="margin:0px 0px 0px 0.8ex;border-left:1px solid rgb(204,204,204);padding-left:1ex">/gem_sw/epics/R3.14.12.4/base/src/db/dbEvent.c:883</blockquote></div><div class="gmail_default"><blockquote class="gmail_quote" style="margin:0px 0px 0px 0.8ex;border-left:1px solid rgb(204,204,204);padding-left:1ex">0x0006371c</blockquote></div><div class="gmail_default"><blockquote class="gmail_quote" style="margin:0px 0px 0px 0.8ex;border-left:1px solid rgb(204,204,204);padding-left:1ex">/gem_sw/targetOS/RTEMS/source/tools/gcc-4.4.7/newlib/libc/stdio/vfscanf.c:270</blockquote></div></blockquote><div class="gmail_default" style="font-size:small"><br></div><div class="gmail_default" style="font-size:small">Power Cycle</div><div class="gmail_default" style="font-size:small"><br></div><div class="gmail_default" style="font-size:small"><b>Event #2:</b></div><blockquote style="margin:0px 0px 0px 0.8ex;border-left:1px solid rgb(204,204,204);padding-left:1ex" class="gmail_quote"><span style="font-family:monospace"><span style="color:rgb(0,0,0)">Jun 23 18:32:20 E) PORT: crcs, MSG: </span><span style="color:rgb(255,255,255);background-color:rgb(0,0,0)">Exception</span><span style="color:rgb(0,0,0)"> handler called for exception 3 (0x3) </span><br>Jun 23 18:32:20 E) PORT: crcs, MSG: #011 Next PC or Address of fault = 001541E4 <br>Jun 23 18:32:20 E) PORT: crcs, MSG: #011 Saved MSR = 00009032 <br>Jun 23 18:32:20 E) PORT: crcs, MSG: #011 Context: Task ID 0x09010001 <br>Jun 23 18:32:20 E) PORT: crcs, MSG: #011 R0 = FFF5A4DA R1 = 00366720 R2 = 00000000 R3 = FFF0A442 <br>Jun 23 18:32:20 E) PORT: crcs, MSG: #011 R4 = 00352498 R5 = 00000008 R6 = 00000000 R7 = FFF0A442 <br>Jun 23 18:32:20 E) PORT: crcs, MSG: #011 R8 = 00352498 R9 = 00000002 R10 = 00000000 R11 = 00000001 <br>Jun 23 18:32:20 E) PORT: crcs, MSG: #011 R12 = 003BC0E0 R13 = 001E4B90 R14 = 00000000 R15 = 00000000 <br>Jun 23 18:32:21 E) PORT: crcs, MSG: #011 R16 = 00000000 R17 = 00000000 R18 = 00000000 R19 = 00000000 <br>Jun 23 18:32:21 E) PORT: crcs, MSG: #011 R20 = 00000000 R21 = 00000000 R22 = 00000000 R23 = 00000000 <br>Jun 23 18:32:21 E) PORT: crcs, MSG: #011 R24 = 00000000 R25 = 00000000 R26 = 00000000 R27 = 00000000 <br>Jun 23 18:32:21 E) PORT: crcs, MSG: #011 R28 = FFF0A340 R29 = 00000000 R30 = FFF0A341 R31 = 0034D5B0 <br>Jun 23 18:32:21 E) PORT: crcs, MSG: #011 CR = 40000004 <br>Jun 23 18:32:21 E) PORT: crcs, MSG: #011 CTR = 0012E3AC <br>Jun 23 18:32:21 E) PORT: crcs, MSG: #011 XER = 00000000 <br>Jun 23 18:32:21 E) PORT: crcs, MSG: #011 LR = 000F2CDC <br>Jun 23 18:32:21 E) PORT: crcs, MSG: #011 DAR = FFF0A442 <br>Jun 23 18:32:21 E) PORT: crcs, MSG: Stack Trace: <br>Jun 23 18:32:21 E) PORT: crcs, MSG: IP: 0x001541E4, LR: 0x000F2CDC <br>Jun 23 18:32:21 E) PORT: crcs, MSG: --^ 0x00135F6C <br>Jun 23 18:32:21 E) PORT: crcs, MSG: Suspending faulting task (0x09010001)</span></blockquote><div><br></div><div class="gmail_default" style="font-size:small"><b>Event #3:</b></div><div class="gmail_default" style="font-size:small"><blockquote class="gmail_quote" style="margin:0px 0px 0px 0.8ex;border-left:1px solid rgb(204,204,204);padding-left:1ex"><span style="font-family:monospace"><span style="color:rgb(0,0,0)">Jun 24 07:37:50 E) PORT: crcs, MSG: *Exception handler called for </span><span style="color:rgb(255,255,255);background-color:rgb(0,0,0)">exception 8</span><span style="color:rgb(0,0,0)"> (0x8) <br></span></span><span style="font-family:monospace">Jun 24 07:37:50 E) PORT: crcs, MSG: #011 Next PC or Address of fault = 0015BB08 <br></span><span style="font-family:monospace">Jun 24 07:37:50 E) PORT: crcs, MSG: #011 Saved MSR = 00009032 <br></span><span style="font-family:monospace">Jun 24 07:37:50 E) PORT: crcs, MSG: #011 Context: Task ID 0x09010001 <br></span><span style="font-family:monospace">Jun 24 07:37:50 E) PORT: crcs, MSG: #011 R0 = 00157754 R1 = 003664B8 R2 = 00000000 R3 = 00366758 <br></span><span style="font-family:monospace">Jun 24 07:37:50 E) PORT: crcs, MSG: #011 R4 = 0036661C R5 = 001B4C68 R6 = 00366610 R7 = 00000000 <br></span><span style="font-family:monospace">Jun 24 07:37:50 E) PORT: crcs, MSG: #011 R8 = 005F5370 R9 = 00642450 R10 = 0034C940 R11 = 00366688 <br></span><span style="font-family:monospace">Jun 24 07:37:50 E) PORT: crcs, MSG: #011 R12 = 40000048 R13 = 001E4B90 R14 = 001B4C68 R15 = 00000000 <br></span><span style="font-family:monospace">Jun 24 07:37:51 E) PORT: crcs, MSG: #011 R16 = 00000000 R17 = 00000000 R18 = 00000000 R19 = 00366610 <br></span><span style="font-family:monospace">Jun 24 07:37:51 E) PORT: crcs, MSG: #011 R20 = 00000000 R21 = 00000000 R22 = 00366758 R23 = 001B4CC0 <br></span><span style="font-family:monospace">Jun 24 07:37:51 E) PORT: crcs, MSG: #011 R24 = 0036661C R25 = 00377E40 R26 = 00000000 R27 = 0000002F <br></span><span style="font-family:monospace">Jun 24 07:37:51 E) PORT: crcs, MSG: #011 R28 = 00377E40 R29 = 001E49B0 R30 = 00377E40 R31 = 00000000 <br></span><span style="font-family:monospace">Jun 24 07:37:51 E) PORT: crcs, MSG: #011 CR = 40000048 <br></span><span style="font-family:monospace">Jun 24 07:37:51 E) PORT: crcs, MSG: #011 CTR = 00000000 <br></span><span style="font-family:monospace">Jun 24 07:37:51 E) PORT: crcs, MSG: #011 XER = 00000000 <br></span><span style="font-family:monospace">Jun 24 07:37:51 E) PORT: crcs, MSG: #011 LR = 00157754 <br></span><span style="font-family:monospace">Jun 24 07:37:51 E) PORT: crcs, MSG: #011 DAR = 00000000 <br></span><span style="font-family:monospace">Jun 24 07:37:51 E) PORT: crcs, MSG: Stack Trace: <br></span><span style="font-family:monospace">Jun 24 07:37:51 E) PORT: crcs, MSG: IP: 0x0015BB08, LR: 0x00157754 <br></span><span style="font-family:monospace">Jun 24 07:37:51 E) PORT: crcs, MSG: --^ 0x00157754--^ 0x000F2820--^ 0x000F2DA0--^ 0x001E49BD--^ 0x00135F6C <br></span><span style="font-family:monospace">Jun 24 07:37:51 E) PORT: crcs, MSG: Suspending faulting task (0x09010001) </span></blockquote><span style="font-family:monospace">
<br></span></div><div class="gmail_default" style="font-size:small"><span style="font-family:monospace"><br></span></div><div class="gmail_default" style="font-size:small"><br></div><blockquote style="margin:0px 0px 0px 0.8ex;border-left:1px solid rgb(204,204,204);padding-left:1ex" class="gmail_quote"> <span style="font-family:monospace"><br></span></blockquote></div>