<html><head><meta http-equiv="content-type" content="text/html; charset=ISO-8859-1"><style>body { line-height: 1.5; }blockquote { margin-top: 0px; margin-bottom: 0px; margin-left: 0.5em; }body { font-size: 10.5pt; font-family: ????; color: rgb(0, 0, 0); line-height: 1.5; }</style></head><body>
<div><span></span>Thanks very much for you reply. I get many useful information from your suggestion.</div><div>Our BSP has a 8MB memory for rtems and a 32MB flash device. So it may not spend too much time for dumping the core. </div><div>The question is that our application running with rtems is very complex. There are several threads and many async processes.</div><div>When the fatal error happened, it is possible that the backtrace of current thread is not enough.</div><div>We need analyse other data structures to debug, such as all items in a queue or some global variable.</div><div>What's more, when our product sold to consumer and some bugs trigger a crash. We could not connect to user's environment to debug.</div><div>So, it will be a better option if a core dump is generated.</div><div><br></div><div>However, if the core dump policy is not possible, the method you provided is feasible.</div><div>I will study it and check how it will be integrated to our system.</div>
<div><br></div><hr style="width: 210px; height: 1px;" color="#b5c4df" size="1" align="left">
<div><span><div style="MARGIN: 10px; FONT-FAMILY: verdana; FONT-SIZE: 10pt"><div>smallphd@aliyun.com</div></div></span></div>
<blockquote style="margin-top: 0px; margin-bottom: 0px; margin-left: 0.5em;"><div> </div><div style="border:none;border-top:solid #B5C4DF 1.0pt;padding:3.0pt 0cm 0cm 0cm"><div style="PADDING-RIGHT: 8px; PADDING-LEFT: 8px; FONT-SIZE: 12px;FONT-FAMILY:tahoma;COLOR:#000000; BACKGROUND: #efefef; PADDING-BOTTOM: 8px; PADDING-TOP: 8px"><div><b>From:</b> <a href="mailto:chrisj@rtems.org">Chris Johns</a></div><div><b>Date:</b> 2020-09-16 08:33</div><div><b>To:</b> <a href="mailto:smallphd@aliyun.com">smallphd@aliyun.com</a>; <a href="mailto:devel@rtems.org">devel</a></div><div><b>Subject:</b> Re: does rtems 5.1 support create a core dump file when accessing a invalid address or other fatal errors?</div></div></div><div><div>On 15/9/20 8:58 pm, smallphd@aliyun.com wrote:</div>
<div>> I am developing applications in rtems 5.1. As we know, my application and rtems</div>
<div>> kernel are both in the same address space.</div>
<div>> So if my application access an invalid address or encounter other fatal errors,</div>
<div>> I want the kernel not just being hunging, but create a core dump file.</div>
<div>> This file contains the whole contents of memory and I could use a debuger to</div>
<div>> analyse the file to handle the bug.</div>
<div>> The question arise because I do not want always debug rtems in the bsp.</div>
<div> </div>
<div>This is an interesting question. For production units I think capturing and</div>
<div>reporting an error is important but a full core is not worth the effort.</div>
<div> </div>
<div>Core images can be saved with a single address space OS. I remember Cisco's</div>
<div>single address space OS for routers from 20 years ago could capture a complete</div>
<div>core that could be loaded by gdb. Those devices had a Compact flash card</div>
<div>installed to capture the core and I suppose their users did not mind the wait</div>
<div>while the core was saved.</div>
<div> </div>
<div>As others have explained capturing the full address space and saving it so gdb</div>
<div>could be taught to load it is difficult. You need to put aside some memory to</div>
<div>construct the core image as you save it and you need to have small stand alone</div>
<div>drivers and what ever else to get the image off the target and saved. RTEMS</div>
<div>cannot be used. Where this approach gets hard is when you start to consider</div>
<div>hardware failure type issues.</div>
<div> </div>
<div>My preferred solution is to add a small storage area away from the RTEMS memory</div>
<div>map called the Run Time Error (RTE) store. This is a piece of RAM that can</div>
<div>survive a reset or reboot and is not part of the RTEMS memory map. Internal SoC</div>
<div>memory can often be enough. The memory cannot be cleared or corrupted during</div>
<div>reset. The struct is something like:</div>
<div> </div>
<div>typedef struct</div>
<div>{</div>
<div> uint32_t type; /* The type of error in this trace buffer. */</div>
<div> uint32_t count; /* The number of times we have had an error. */</div>
<div> uint64_t uptime; /* The period of time we have been up. */</div>
<div> union</div>
<div> {</div>
<div> error_trace_fatal fatal; /* A fatal error. */</div>
<div> error_trace_assert assert; /* An assert error. */</div>
<div> error_trace_error error; /* An error code. */</div>
<div> } error;</div>
<div> uint32_t crc; /* Checksum */</div>
<div>} error_trace;</div>
<div> </div>
<div>You provide a struct for a fatal error, an assert or an error. It is a matter of</div>
<div>hooking the error handlers and saving the data. The fatal error is something like:</div>
<div> </div>
<div>typedef struct</div>
<div>{</div>
<div> rtems_fatal_source source;</div>
<div> uint32_t internal;</div>
<div> rtems_fatal_code code;</div>
<div> CPU_Exception_frame frame;</div>
<div> uint32_t stack[ET_STACK_SIZE];</div>
<div>} error_trace_fatal;</div>
<div> </div>
<div>Catch the fatal error handler and fill in the fields including the crc then</div>
<div>reset the board. Limit the code you call before reset.</div>
<div> </div>
<div>When RTEMS starts get your application to check the RTE and if the checksum is</div>
<div>valid check the error count. If the error count is not zero you have captured an</div>
<div>error. You now have a working RTEMS that can be used to process and save the</div>
<div>error. I have production systems that save errors to a JFFS2 disk and a web</div>
<div>interface can be used to download it. I also have systems that send the data to</div>
<div>a syslog type server when the devices are networked. In those systems it is</div>
<div>really important to capture _every_ reset.</div>
<div> </div>
<div>Finding the error in the code is a matter of getting the PC address and the ELF</div>
<div>executable image with the DWARF debug information and using `objdump -d</div>
<div>--source`. Disassemble the exe with source and search for the PC address. The</div>
<div>report points to the location and the dumped registers will help you see what</div>
<div>the issue is. Most of the time the issue can be found or investigated further</div>
<div>and resolved but sometimes it cannot be found directly. In those cases you need</div>
<div>to stress the system in a lab to expose the crash and then investigate. The key</div>
<div>information is the crash happened and where.</div>
<div> </div>
<div>Chris</div>
</div></blockquote>
</body></html>