rtems-addr2line not working on ARM?

[Chris Johns]

On 9/9/19 5:12 pm, Sebastian Huber wrote:
> On 09/09/2019 07:31, Sebastian Huber wrote:
>> On 07/09/2019 16:25, Sebastian Huber wrote:
>>> ----- Am 6. Sep 2019 um 18:06 schrieb Sebastian Huber
>>> sebastian.huber at embedded-brains.de:
>>>
>>>> ----- Am 6. Sep 2019 um 11:09 schrieb Sebastian Huber
>>>> sebastian.huber at embedded-brains.de:
>>>>
>>>>> On 06/09/2019 09:26, Sebastian Huber wrote:
>>>>>> On 06/09/2019 09:01, Chris Johns wrote:
>>>>>>> On 6/9/19 4:20 pm, Sebastian Huber wrote:
>>>>>>>> Hello,
>>>>>>>>
>>>>>>>> I tried the rtems-addr2line on ARM and SPARC. On SPARC it seems to
>>>>>>>> work fine,
>>>>>>>> however, on ARM I get this:
>>>>>>>>
>>>>>>>> rtems-addr2line -e
>>>>>>>> build/arm-rtems5-xilinx_zynq_a9_qemu-everything/media01.exe
>>>>>>>> 0x0000135a
>>>>>>>> /home/EB/sebastian_h/git-rtems-5/c/src/lib/libbsp/arm/xilinx-zynq/../../../../../../bsps/arm/shared/irq/irq-gic.c:264
>>>>>>>>
>>>>>>>>
>>>>>>>>
>>>>>>>>
>>>>>>>> addr2line -e
>>>>>>>> build/arm-rtems5-xilinx_zynq_a9_qemu-everything/media01.exe 0000135a
>>>>>>>> /scratch/git-rtems-libbsd/build/arm-rtems5-xilinx_zynq_a9_qemu-everything/../../testsuite/include/rtems/bsd/test/default-network-init.h:179
>>>>>>>>
>>>>>>>>
>>>>>>>>
>>>>>>>>
>>>>>>>> The GNU tool is right. It is also considerably faster.
>>>>>>> I would hope it is faster and exact. It has had many years of work on it,
>>>>>>> written in C and not a means to test a C++ framework so we can grow an
>>>>>>> ecosystem. I have stated its purpose before. I am perplexed by the
>>>>>>> intent of
>>>>>>> this statement.
>>>>>>>
>>>>>>> If you want to compare performance I suggest you try the elftools one?
>>>>>>> There is
>>>>>>> one. It is not built because GNU provides one and good one.
>>>>>>>
>>>>>>> Also be-careful as the exec call is not as fast as Linux on all the
>>>>>>> hosts we
>>>>>>> support.
>>>>>>
>>>>>> That the GNU tool is faster was just an observation.
>>>>>>
>>>>>> Do we have a working library solution to get from an address to the
>>>>>> source line/function? I don't have time to debug the DWARF support code
>>>>>> at the moment.
>>>>>>
>>>>>> I have a working solution using "addr2line" and rld::process::execute().
>>>>>
>>>>> Thanks for the hint to the elftoolchain:
>>>>>
>>>>> https://github.com/elftoolchain/elftoolchain/blob/trunk/addr2line/addr2line.c
>>>>>
>>>>> To me it looks pretty complicated. It should be possible to refactor
>>>>> this code to use it as a library, e.g. call translate() for each address
>>>>> you need the source information.
>>>>>
>>>>> I tried to debug the rld get_source() problem for more than two hours
>>>>> and the elftoolchain code looks similar. I am not sure what to do.
>>>>
>>>> Another option is to use the LLVM infrastructure:
>>>>
>>>> https://github.com/llvm-mirror/llvm/blob/master/tools/llvm-symbolizer/llvm-symbolizer.cpp
>>>>
>>>>
>>>>  From a first glance, this looks like the way to go.
>>>
>>> Attached is a patch which uses the LLVM library to get an addr2line
>>> functionality. The code to resolve an address is very easy (item.data is the
>>> address):
>>>
>>>      auto res_or_err = symbolizer_.symbolizeCode(elf_file_, item.data);
>>>
>>>      if (res_or_err) {
>>>        auto info = res_or_err.get();
>>>        std::string fn = info.FunctionName;
>>>        std::string str;
>>>
>>>        if (fn != "<invalid>") {
>>>          str += fn;
>>>          str += " at ";
>>>        }
>>>
>>>        str += llvm::sys::path::filename(info.FileName);
>>>        str += ":";
>>>        str += std::to_string(info.Line);
>>>
>>> LLVM has also support for command line arguments, file handling, etc.
>>
>> The LLVM solution is also the fastest so far:
>>
>> time ../build/trace/rtems-record-lttng -e media01.exe records.bin
>>
>> real    0m0.425s
>> user    0m0.293s
>> sys     0m0.132s
>>
>> addr2line based solution on Linux:
>>
>> real    0m11.944s
>> user    0m5.334s
>> sys     0m6.593s
> 
> Compiling this on Windows using mingw64 was a bit difficult. I had to install:
> 
> mingw-w64-x86_64-llvm
> mingw-w64-x86_64-clang
> 
> I used clang to build the tool by hand. I had to remove the use of the POSIX
> open/connect/socket/etc. and replace it with fopen().
> 
> The CTF stream files generated on Linux and Windows were identical using LLVM.
> 

I think LLVM is a great idea for us moving forward but it has some hurdles we
need to overcome. I like the idea of a maintained code base that is fast and has
working 64bit support, something that is not tested or working in the
rtemstoolkit code.

Does LLVM as used in trace have support for all of the current RTEMS
architectures? If it does not what do we do?

We have been avoiding depending on packages on hosts because of our past
experiences ..

1. getting a stable consistent version across all hosts can be difficult and
with environments like cygwin and msys having a floating head and no release
things age and break which makes long term support harder.

2. A more complicated set up process for users.

3. Not all hosts have a packaging system, ie MacOS. Using macports or homebrew
in our experience can result in collateral issues when building our tools.

We need to consider how we transition from where we are to an LLVM based
solution, it doubt it can happening in a few simple steps. As stated in another
email I wonder if focusing on clang and LLVM as a tool set for RTEMS is a valid
path. This is a large task and I suspect not available for every architecture
but something worth giving real consideration too as it will happen in time.

Chris