dynamic loading in 4.11

[Chris Johns]

On 2/12/11 7:42 PM, Oleg Moroz wrote:
> Hello everyone. I want to try announced dynamical code loading. How can
> i do that? I've checkout source code from CSV, but there is no dlfcn.h
> file.

The projects current status can be download from here:

  http://www.rtems.org/ftp/pub/rtems/people/chrisj/rtl/

Please download the latest version.

> Is this functionality in main branch?

No. I develop outside the main source tree and integrate once completed. 
I do this to speed up the development edit->compile cycle.

There are 2 parts to the project. The target side (rtl-<date>.tar.bz2) 
and host side (rtl-host-<date>.tar.bz2). You need both parts.

The current status is:

1. The target side will load sparc, i386 and m68k and arm support is 
provided but I have not tested it. There is stubs for the NIOS target 
but it will not load.

2. The target side needs support to handle object files loaded with 
unresolved externals. This can occur if 2 object modules depend on each 
other. One needs to be loaded with the error and resolved once the other 
module is loaded. This means there needs to some memory put aside for 
the unresolved symbols and that complicates the management of the 
memory. I need to add support for a memory pool for the loader and to 
move all allocations to the pool. Memory is a critical issue to make 
sure the heap does not fragment.

3. The target code needs to have the allocator hooks added so custom 
allocators can be used when loading. This will allow for specific 
locations to be used when loading and MMU support to write protect the 
code if this is a requirement. That functionality is outside the loader.

4. The host code is a linker called rtems-ld. It is used to create 
either a script file that lists the object modules to load or create an 
archive or object modules. The archive is a direct copy of the object 
files in the libraries and contain the debugging data. The archive needs 
to be stripped.

5. The rtems-ld linker now support the elftoolchain libelf package. This 
is used in FreeBSD and other projects and is a high quality active 
libelf package. It contains a strip tool that can strip archives and 
therefore can strip the output of the RTEMS linker. I have ported the 
package to Darwin and Windows and I am currently preparing patches to 
submit back to the project. The rtl-host package I have contain the 
libelf source and is self contained. I have tested Linux, MacOS, Windows 
and FreeBSD.

6. Debugging support for code loaded needs to be figured out.

Building.

The target and host code uses waf as a build system. You can find waf 
here http://code.google.com/p/waf/. The host build is:

  $ tar jxf rtl-host-20111124.tar.bz2
  $ cd rtl-host-20111124
  $ waf configure build

The target code is:

  $ tar jxf rtl-20111005.tar.bz2
  $ cd rtl-20111005
  $ waf configure \
    --rtems=/Users/chris/Development/rtems/build/4.11 \
    --rtems-tools=/Users/chris/Development/rtems/4.11,\
       /Users/chris/Development/rtems/nios/gnu/nios2-rtems-11.0

where '--rtems=' is the path to installed board support packages such as 
SIS and '--rtems-tools=' is a list of paths to tools. I have NIOS tools 
installed and a built NIOS BSP. This builds an application you can run 
and it will have 'dlo' and other commands.

The target code uses the pkgconfig files RTEMS creates. The package 
config support in RTEMS is experimental support.

The linker is invoked with something like:

$ ./build-darwin/rtems-ld -v 
/Users/chris/Development/rtems/src/apps/rtl/build/m68k-rtems4.11-mcf5235/x-long-name-to-create-gnu-extension-in-archive.c.2.o 
-L /Users/chris/Development/rtems/build/4.11/m68k-rtems4.11/mcf5235/lib 
-lrtemsbsp -lrtemscpu -lm -b 
/Users/chris/Development/rtems/src/apps/rtl/build/m68k-rtems4.11-mcf5235/rtld 

RTEMS Linker 1.0.0
Finding libraries:
found: 
/Users/chris/Development/rtems/build/4.11/m68k-rtems4.11/mcf5235/lib/librtemsbsp.a
found: 
/Users/chris/Development/rtems/build/4.11/m68k-rtems4.11/mcf5235/lib/librtemscpu.a
found: 
/Users/chris/Development/rtems/4.11/bin/../lib/gcc/m68k-rtems4.11/4.6.1/../../../../m68k-rtems4.11/lib/libm.a
gcc::stdlib: libgcc.a
gcc::stdlib: libssp.a
gcc::stdlib: libc.a
resolver:resolving: top: 
x-long-name-to-create-gnu-extension-in-archive.c.2.o
resolver:resolving: 
x-long-name-to-create-gnu-extension-in-archive.c.2.o, unresolved: 4
resolver:resolved :   exit -> rtld
resolver:resolved :   printf -> rtld
resolver:resolved :   puts -> rtld
resolver:resolved :   sin -> libm.a:lib_a-s_sin.o at 805916
resolver:resolving:  libm.a:lib_a-s_sin.o at 805916, unresolved: 3
resolver:resolved :    __ieee754_rem_pio2 -> 
libm.a:lib_a-e_rem_pio2.o at 304536
resolver:resolved :    __kernel_cos -> libm.a:lib_a-k_cos.o at 511700
resolver:resolved :    __kernel_sin -> libm.a:lib_a-k_sin.o at 524656
resolver:resolving:   libm.a:lib_a-e_rem_pio2.o at 304536, unresolved: 2
resolver:resolved :     __kernel_rem_pio2 -> 
libm.a:lib_a-k_rem_pio2.o at 516224
resolver:resolved :     fabs -> libm.a:lib_a-s_fabs.o at 655520
resolver:resolving:    libm.a:lib_a-k_rem_pio2.o at 516224, unresolved: 2
resolver:resolved :      floor -> libm.a:lib_a-s_floor.o at 665352
resolver:resolved :      scalbn -> libm.a:lib_a-s_scalbn.o at 795192
resolver:resolving:     libm.a:lib_a-s_floor.o at 665352, unresolved: 0 
(resolved)
resolver:resolving:     libm.a:lib_a-s_scalbn.o at 795192, unresolved: 1
resolver:resolved :       copysign -> libm.a:lib_a-s_copysign.o at 625924
resolver:resolving:      libm.a:lib_a-s_copysign.o at 625924, unresolved: 0
resolver:resolving:    libm.a:lib_a-s_fabs.o at 655520, unresolved: 0
resolver:resolving:   libm.a:lib_a-k_cos.o at 511700, unresolved: 0
resolver:resolving:   libm.a:lib_a-k_sin.o at 524656, unresolved: 0
outputter:archive: a.out

You could now load a.out using the 'dlo' command and then call functions 
in the loaded code.

All this code is experimental so please be warned.

Chris