How to build librtemscpu.a and librtemsbsp.a in the future?

Fri Mar 30 18:07:58 UTC 2018

----- Am 29. Mrz 2018 um 0:16 schrieb Chris Johns chrisj at rtems.org:

> On 28/03/2018 16:30, Sebastian Huber wrote:
>> On 28/03/18 02:35, Chris Johns wrote:
>>> On 27/03/2018 17:17, Sebastian Huber wrote:
>>>> On 27/03/18 01:15, Chris Johns wrote:
>>>>> On 26/03/2018 17:32, Sebastian Huber wrote:
>>>>>> On 25/03/18 05:20, Chris Johns wrote:
>>>>>>> On 24/3/18 1:27 am, Sebastian Huber wrote:
>>>>>>>> Hello,
>>>>>>>>
>>>>>>>> I tried to consolidate the cpukit Makefile.am. This ended up in the
>>>>>>>> following
>>>>>>>> problem. The content of librtemscpu.a depends on the CPU port. This is
>>>>>>>> currently
>>>>>>>> done via a non-standard
>>>>>>>>
>>>>>>>> _SUBDIRS = ../@RTEMS_CPU@
>>>>>>>>
>>>>>>>> construct.
>>>>>>> Where is this? I cannot find it.
>>>>>> https://git.rtems.org/rtems/tree/cpukit/score/cpu/Makefile.am#n1
>>>>>>
>>>>> I am confused, the link's version does not have '../'?
>>>> Sorry, I should have used copy and paste.
>>>>
>>>>> [...]
>>>>>> I am not sure how we want to proceed with the build system stuff.
>>>>> It is becoming more and more important this work starts to happen. I am
>>>>> happy to
>>>>> make the change to waf if we can attract the funding to do it.
>>>>>
>>>>> I have pushed the existing build system as far as I want to without major
>>>>> refactoring. I would like to test a recent autoconf and automake to see if we
>>>>> can move to a recent version. The recent perl issues highlights the problems we
>>>>> can expect to face if we do not change.
>>>>>
>>>>>> Should we first clean up the existing build system first?
>>>>> I have no interesting in a "clean up" or a refactor. I have played with it
>>>>> enough in recent times with the parallel build and pre-install work to know it
>>>>> is a deep pit with much gnashing of teeth, rolling of eyes, and terrible roars.
>>>> I think there are some issues left which need a clean up:
>>>>
>>>> * Use of RTEMS_RELLDFLAGS'
>>> I have not looked into this stuff. What is happening with these flags and these
>>> *.rel files?
>>>
>>> Is this some sort of component system using incremental objects ?
>> 
>> I think it is a hack to enable the use of LIBADD. It was used mainly by the
>> libcpu.
> 
> Some BSPs also have this?

There are not that many spots left to clean up:

grep -r --include='Makefile.am' RTEMS_RELLDFLAGS                                                                                                  
c/src/lib/libbsp/powerpc/beatnik/Makefile.am:network_support_rel_LDFLAGS = $(RTEMS_RELLDFLAGS)
c/src/lib/libbsp/powerpc/beatnik/Makefile.am:network_if_mve_tmp_rel_LDFLAGS  = $(RTEMS_RELLDFLAGS)
c/src/lib/libbsp/powerpc/beatnik/Makefile.am:network_if_gfe_tmp_rel_LDFLAGS = $(RTEMS_RELLDFLAGS)
c/src/lib/libbsp/powerpc/beatnik/Makefile.am:network_if_em_tmp_rel_LDFLAGS = $(RTEMS_RELLDFLAGS)
c/src/lib/libbsp/powerpc/mvme5500/Makefile.am:network_rel_LDFLAGS = $(RTEMS_RELLDFLAGS)
c/src/lib/libbsp/powerpc/mvme5500/Makefile.am:mvme5500start___OBJEXT__LDFLAGS = $(RTEMS_RELLDFLAGS)
c/src/lib/libbsp/powerpc/motorola_powerpc/Makefile.am:polledIO_rel_LDFLAGS = $(RTEMS_RELLDFLAGS)
cpukit/libfs/src/nfsclient/Makefile.am:dirutils_rel_LDFLAGS = $(RTEMS_RELLDFLAGS)
cpukit/libfs/src/nfsclient/Makefile.am:nfs_rel_LDFLAGS = $(RTEMS_RELLDFLAGS)
cpukit/libfs/src/nfsclient/Makefile.am:rpcio_rel_LDFLAGS = $(RTEMS_RELLDFLAGS)

> 
>> 
>>>
>>>> * Command line defines
>>>>
>>> Lets leave that to the ticket on this topic.
>>>
>>>> * Host tools should be removed or moved to rtems-tools
>>> Agreed.
>>>
>>>>> The major issue we have is the time configure takes. I you play with the
>>>>> rtems-bsp-builder and the --jobs option you will get an idea. If you look at
>>>>> Joel's latest build:
>>>>>
>>>>> https://lists.rtems.org/pipermail/build/2018-March/000554.html
>>>>>
>>>>> jobs is '--jobs=6/12' which means run 6 parallel RTEMS BSP builds and use 12
>>>>> cores per build. We can do this because of the time configure takes using a
>>>>> single core. I think Joel has 24 cores on this box. This setting builds a BSP
>>>>> every 18 seconds on average which is impressive but it is a hack.
>>>> Yes, I think we need only two configure runs. One in the top-level and one in
>>>> the BSP (for the BSP options).
>>>>
>>> That would be nice. I suggest we resolve how the tree looks in the other thread
>>> before we proceed. It would provide clarity for this work and a solid direction.
>> 
>> The host tools are a blocking point for me to do anything with the configure.ac
>> files.
>> 
>> Another issue are the complicated test program Makefile.am.
> 
> What issues are you seeing?

It is a mixture of an Automake program and the classic RTEMS makefile support.

> 
>> We have the optional
>> BSP-specific post-link command which transforms an .exe file (ELF) into a .ralf
>> file.
> 
> I see printing the size as wasted build time and we should remove that step. 

Yes, we definitely should not print this stuff.

> You
> can run size on any static ELF executable and get the sizes if you are
> interested and it is easy to script something to print all test executable
> sizes.
> 
> I am not sure about the post-link parts, I can see they are useful even as an
> example and I can also see the case for RTEMS stopping at a static ELF file and
> simplifying the BSP configs.
> 
> The current way the post-link is done is specific to building the samples and
> tests and any Makefile.inc support, after that the details of what needs to
> happen are not accessible. Mkaefile.inc support is to be remove so this leave
> the samples and tests. The rtems_waf tool cannot perform a post-link operation
> because nothing is captured in the pkgconfig file and adding it from the current
> Makefile fragment would be a difficult task. This is why the .pc work stalled.

Normally, an ELF file is all you need. A post link step is quite common for embedded targets, but from a host operating system point of view it is quite exotic.

> 
>> The RTEMS tester duplicates this feature somehow.
> 
> Yes it does. It has to because of the last point, the details are not openly
> available from a built or installed BSP. The RTEMS Tester's support is effective
> but not optimal.
> 
>> How would waf deal with this post-link step?
> 
> I do not know, it is something I have not considered.
> 
> The problem I see with what we currently have is not the operation itself it is
> access to the details outside of the RTEMS build system. If that can be solved
> at the BSP level as an exported controlled interface we can build eco-system
> support around it.
> 
> For waf I would look at adding a per BSP configuration in a format that can be
> exported easily. Once that information exists I would have the RTEMS tester use
> it as well. In waf you could see if a post-link operation is configured and add
> a build job based on a rule, ie a more advanced version of the current RTEMS
> tester's `target_pretest_command`. How and what requires looking at all the
> existing post-link commands to see what can be done.

Ok, so per se waf supports a post link step.

> 
> Just wondering aloud, if the BSP configuration was a pkgconfig file and not a
> makefile fragment and the RTEMS build system parsed it would that help? It could
> then be installed or an extended version with build specific details capturing
> any details we want. Pkg-config supports variables so we could have something
> like (hand made and cleaned up version installed by RTEMS):
> 
> $ cat /tmp/arm-rtems5-xilinx_zynq_zedboard.pc
> #
> # pkg-config support file for RTEMS BSP xilinx_zynq_zedboard
> #
> # Warning: This stuff is experimental and may be changed at any time.
> #
> rtems_major=5
> arch=arm
> bsp=xilinx_zynq_zedboard
> 
> arch_bsp=${arch}/${bsp}
> arch_prefix=${arch}-rtems${rtems_major}
> 
> prefix=/opt/work/rtems/bsps/5
> exec_prefix=/opt/work/rtems/bsps/5/${arch_prefix}
> libdir=${exec_prefix}/${bsp}/lib
> includedir=${exec_prefix}/${bsp}/lib/include
> 
> CFLAGS=-march=armv7-a -mthumb -mfpu=neon -mfloat-abi=hard -mtune=cortex-a9 -O2
> -g -ffunction-sections -fdata-sections
> 
> Name: arm-rtems5-xilinx_zynq_zedboard
> Version: 5.0.0
> Description: RTEMS BSP xilinx_zynq_zedboard
> Libs:
> Cflags: -qrtems -B${exec_prefix}/lib/ -B${libdir}/ --specs bsp_specs ${CFLAGS}
> 
> start_addr=0x00104000
> entry_addr=0x00104000
> 
> bin=${arch_prefix}-objdump -R -S --strip-debug -O binary @EXE@ @EXE at .bin && \
>    cat @EXE at .bin | gzip -9 > @EXE at .gz && \
>    mkimage -A arm -O rtems -T kernel -a ${start_addr} -e ${entry_addr} \
>    -n "RTEMS" -d @EXE at .gz @EXE at .img
> 
> Asking for the variable returns:
> 
> $ PKG_CONFIG_PATH=/tmp pkg-config --variable=bin arm-rtems5-xilinx_zynq_zedboard
> arm-rtems5-objdump -R -S --strip-debug -O binary @EXE@ @EXE at .bin && cat
> @EXE at .bin | gzip -9 > @EXE at .gz && mkimage -A arm -O rtems -T kernel -a
> 0x00104000 -e 0x00104000 -n "RTEMS" -d @EXE at .gz @EXE at .img
> 
> (I am not sure the gzip pipe and redirect would work but it shows the idea)
> 
> We could define and document a set of standard variables with known functions.
> 
> BSPs could provide more, for example DFT, BSP options, etc and the BSP
> documentation would list those.
> 
> Detecting a variable could be checking `--print-variables` or the empty result
> pkg-config gives.
> 
> It would really improve rtems_waf having this detail. I could remove the
> 'tweaks' support I need to have for some BSPs.
> 
> Any build system that supports pkg-config could be used.

Ok, this sounds like a good direction. I think it is acceptable if we break the existing post link support during the build system conversion if we have a suitable replacement and a proof of concept. Maintained BSPs can then easily resurrect this feature.

> 
>> Is there a standard support for a post-link step in Automake?
> 
> I do not know.
> 
>> 
>> The "make dist" is broken?
>> 
> 
> Is it used? I do not use it to make a release.

Good, then I don't have to care about this. Was it ever used? There is a lot of stuff in the Makefile.am for the make dist.