How to build librtemscpu.a and librtemsbsp.a in the future?

Wed Mar 28 22:16:52 UTC 2018

On 28/03/2018 16:30, Sebastian Huber wrote:
> On 28/03/18 02:35, Chris Johns wrote:
>> On 27/03/2018 17:17, Sebastian Huber wrote:
>>> On 27/03/18 01:15, Chris Johns wrote:
>>>> On 26/03/2018 17:32, Sebastian Huber wrote:
>>>>> On 25/03/18 05:20, Chris Johns wrote:
>>>>>> On 24/3/18 1:27 am, Sebastian Huber wrote:
>>>>>>> Hello,
>>>>>>>
>>>>>>> I tried to consolidate the cpukit Makefile.am. This ended up in the
>>>>>>> following
>>>>>>> problem. The content of librtemscpu.a depends on the CPU port. This is
>>>>>>> currently
>>>>>>> done via a non-standard
>>>>>>>
>>>>>>> _SUBDIRS = ../@RTEMS_CPU@
>>>>>>>
>>>>>>> construct.
>>>>>> Where is this? I cannot find it.
>>>>> https://git.rtems.org/rtems/tree/cpukit/score/cpu/Makefile.am#n1
>>>>>
>>>> I am confused, the link's version does not have '../'?
>>> Sorry, I should have used copy and paste.
>>>
>>>> [...]
>>>>> I am not sure how we want to proceed with the build system stuff.
>>>> It is becoming more and more important this work starts to happen. I am
>>>> happy to
>>>> make the change to waf if we can attract the funding to do it.
>>>>
>>>> I have pushed the existing build system as far as I want to without major
>>>> refactoring. I would like to test a recent autoconf and automake to see if we
>>>> can move to a recent version. The recent perl issues highlights the problems we
>>>> can expect to face if we do not change.
>>>>
>>>>> Should we first clean up the existing build system first?
>>>> I have no interesting in a "clean up" or a refactor. I have played with it
>>>> enough in recent times with the parallel build and pre-install work to know it
>>>> is a deep pit with much gnashing of teeth, rolling of eyes, and terrible roars.
>>> I think there are some issues left which need a clean up:
>>>
>>> * Use of RTEMS_RELLDFLAGS'
>> I have not looked into this stuff. What is happening with these flags and these
>> *.rel files?
>>
>> Is this some sort of component system using incremental objects ?
> 
> I think it is a hack to enable the use of LIBADD. It was used mainly by the libcpu.

Some BSPs also have this?

> 
>>
>>> * Command line defines
>>>
>> Lets leave that to the ticket on this topic.
>>
>>> * Host tools should be removed or moved to rtems-tools
>> Agreed.
>>
>>>> The major issue we have is the time configure takes. I you play with the
>>>> rtems-bsp-builder and the --jobs option you will get an idea. If you look at
>>>> Joel's latest build:
>>>>
>>>> https://lists.rtems.org/pipermail/build/2018-March/000554.html
>>>>
>>>> jobs is '--jobs=6/12' which means run 6 parallel RTEMS BSP builds and use 12
>>>> cores per build. We can do this because of the time configure takes using a
>>>> single core. I think Joel has 24 cores on this box. This setting builds a BSP
>>>> every 18 seconds on average which is impressive but it is a hack.
>>> Yes, I think we need only two configure runs. One in the top-level and one in
>>> the BSP (for the BSP options).
>>>
>> That would be nice. I suggest we resolve how the tree looks in the other thread
>> before we proceed. It would provide clarity for this work and a solid direction.
> 
> The host tools are a blocking point for me to do anything with the configure.ac
> files.
> 
> Another issue are the complicated test program Makefile.am.

What issues are you seeing?

> We have the optional
> BSP-specific post-link command which transforms an .exe file (ELF) into a .ralf
> file. 

I see printing the size as wasted build time and we should remove that step. You
can run size on any static ELF executable and get the sizes if you are
interested and it is easy to script something to print all test executable sizes.

I am not sure about the post-link parts, I can see they are useful even as an
example and I can also see the case for RTEMS stopping at a static ELF file and
simplifying the BSP configs.

The current way the post-link is done is specific to building the samples and
tests and any Makefile.inc support, after that the details of what needs to
happen are not accessible. Mkaefile.inc support is to be remove so this leave
the samples and tests. The rtems_waf tool cannot perform a post-link operation
because nothing is captured in the pkgconfig file and adding it from the current
Makefile fragment would be a difficult task. This is why the .pc work stalled.

> The RTEMS tester duplicates this feature somehow.

Yes it does. It has to because of the last point, the details are not openly
available from a built or installed BSP. The RTEMS Tester's support is effective
but not optimal.

> How would waf deal with this post-link step?

I do not know, it is something I have not considered.

The problem I see with what we currently have is not the operation itself it is
access to the details outside of the RTEMS build system. If that can be solved
at the BSP level as an exported controlled interface we can build eco-system
support around it.

For waf I would look at adding a per BSP configuration in a format that can be
exported easily. Once that information exists I would have the RTEMS tester use
it as well. In waf you could see if a post-link operation is configured and add
a build job based on a rule, ie a more advanced version of the current RTEMS
tester's `target_pretest_command`. How and what requires looking at all the
existing post-link commands to see what can be done.

Just wondering aloud, if the BSP configuration was a pkgconfig file and not a
makefile fragment and the RTEMS build system parsed it would that help? It could
then be installed or an extended version with build specific details capturing
any details we want. Pkg-config supports variables so we could have something
like (hand made and cleaned up version installed by RTEMS):

$ cat /tmp/arm-rtems5-xilinx_zynq_zedboard.pc
#
# pkg-config support file for RTEMS BSP xilinx_zynq_zedboard
#
# Warning: This stuff is experimental and may be changed at any time.
#
rtems_major=5
arch=arm
bsp=xilinx_zynq_zedboard

arch_bsp=${arch}/${bsp}
arch_prefix=${arch}-rtems${rtems_major}

prefix=/opt/work/rtems/bsps/5
exec_prefix=/opt/work/rtems/bsps/5/${arch_prefix}
libdir=${exec_prefix}/${bsp}/lib
includedir=${exec_prefix}/${bsp}/lib/include

CFLAGS=-march=armv7-a -mthumb -mfpu=neon -mfloat-abi=hard -mtune=cortex-a9 -O2
-g -ffunction-sections -fdata-sections

Name: arm-rtems5-xilinx_zynq_zedboard
Version: 5.0.0
Description: RTEMS BSP xilinx_zynq_zedboard
Libs:
Cflags: -qrtems -B${exec_prefix}/lib/ -B${libdir}/ --specs bsp_specs ${CFLAGS}

start_addr=0x00104000
entry_addr=0x00104000

bin=${arch_prefix}-objdump -R -S --strip-debug -O binary @EXE@ @EXE at .bin && \
    cat @EXE at .bin | gzip -9 > @EXE at .gz && \
    mkimage -A arm -O rtems -T kernel -a ${start_addr} -e ${entry_addr} \
    -n "RTEMS" -d @EXE at .gz @EXE at .img

Asking for the variable returns:

$ PKG_CONFIG_PATH=/tmp pkg-config --variable=bin arm-rtems5-xilinx_zynq_zedboard
arm-rtems5-objdump -R -S --strip-debug -O binary @EXE@ @EXE at .bin && cat
@EXE at .bin | gzip -9 > @EXE at .gz && mkimage -A arm -O rtems -T kernel -a
0x00104000 -e 0x00104000 -n "RTEMS" -d @EXE at .gz @EXE at .img

(I am not sure the gzip pipe and redirect would work but it shows the idea)

We could define and document a set of standard variables with known functions.

BSPs could provide more, for example DFT, BSP options, etc and the BSP
documentation would list those.

Detecting a variable could be checking `--print-variables` or the empty result
pkg-config gives.

It would really improve rtems_waf having this detail. I could remove the
'tweaks' support I need to have for some BSPs.

Any build system that supports pkg-config could be used.

> Is there a standard support for a post-link step in Automake?

I do not know.

> 
> The "make dist" is broken?
> 

Is it used? I do not use it to make a release.

Chris