[PATCH] wscript: Add post-build processing support

[Chris Johns]

On 5/3/21 6:29 am, Sebastian Huber wrote:
> On 04/03/2021 20:15, Kinsey Moore wrote:
> 
>> -----Original Message-----
>> From: Sebastian Huber <sebastian.huber at embedded-brains.de>
>> Sent: Thursday, March 4, 2021 12:57
>> To: Kinsey Moore <kinsey.moore at oarcorp.com>; devel at rtems.org
>> Subject: Re: [PATCH] wscript: Add post-build processing support
>>
>>> On 04/03/2021 17:18, Kinsey Moore wrote:
>>>
>>>> This allows packaging of the compiled binaries to occur automatically
>>>> and be bundled with other scripts and build items as convenient.
>>> This is not how a build system should work. When an item changes it is
>>> the job of the build system to automatically rebuild all its dependents.
>>>
>>> I think this issue was already discussed with Joel and Chris. The
>>> discussed approach is to add an optional post-build script and run this
>>> script in a post-link step.
>> I thought this might be the case. I'll try to find some time to rework it
>> into a more proper patch.
> I discussed this with Chis in November last year.

We did. Seems like it was yesterday. Arrrrrrrr....

>  Unfortunately we didn't add a ticket for this. 

I recently raise a ticket as a GSoC Open project ...

https://devel.rtems.org/ticket/4272

> The general idea was to let BSPs add an optional post-link
> script which is installed as <bsp-install>/bin/post-link. For the test
> executables, please have a look at:
> 
> https://waf.io/blog/2016/09/waf-strip-binaries.html

Please do not add any post processing to the rtems.git build. Let me explain ...

1. The previous approach in the autotool build system performed a range of post
link steps and most of the builds done by developers never ran so that whole
step was a nop. The step consumed an ever growing amount of time and disk space
and as a result the post link step was turned off.

2. The autotool build system post-link solution was available to only those
users who built with make and used the Makefile.inc support. Today we have a
number of build systems so this is no longer a viable solution. A command line
tool can be integrated into any build system and we need to maintain it as a
public user interface across releases.

3. The post link steps are the domain of users and their set up. It is not
possible for RTEMS to capture and included everything users need to do. For
example the previous post-link processing generated a map file and I have no
idea why when we have an ELF executable image at hand. There are a number of
tools that can create a map file from it.

4. The rtems-test command has the `target_pretest_command` hook that is used to
convert the ELF executable into a suitable format for loading on to the target.
We should be educating and encouraging our user to on-demand convert.

5. The post-processing can only depend on the RTEMS ecosystem support provided.
For example `mkimage` is a Linux and u-boot command not available on all hosts.
We do have the `mkimage.py` script so that could be used. Yes this creates some
sort term pain but the long term benefit is important.

What I would like to see happen is:

a) A script added to each BSP that needs post processing.

b) A reference added to the BSP's YAML spec support to indicate there is a
post-processing script.

c) Have waf build the script into the BSP build tree applying some form of yet
to be defined filtering. For example a base address or some other setting the
script and BSP depend on. I have no idea how this could be implemented and if it
is possible but the few places we handle some of these settings the better. For
example if a BSP config changes a setting the build and a script uses we need to
see the change in the script as well.

d) Have waf install the script into the BSP specific area under the
$PREFIX/$ARCH-rtems$RTEMS_VERSION/$BSP/.... path.

e) Provide and install a command to $PREFIX/bin with a formal name such
rtems-post-link that finds and runs the BSP installed script. This implies an
interface between the BSP script and this command. The BSP scripts can provide
extensions but there needs to be a core level of functionality provided. An
example of an extension is creating a second image type such as the bootloader
for the motorola_powerpc BSP.

f) Support all major hosts we support.

Again please do not load up the build of rtems with post processing that is not
used most of the time. Post process on demand when you need it.

Chris