New Build System Status

[Sebastian Huber]

----- Am 22. Nov 2019 um 23:31 schrieb Chris Johns chrisj at rtems.org:

> On 23/11/19 1:49 am, Sebastian Huber wrote:
>> Hello,
>> 
>> I converted all BSPs to the new build system. I was able to build the tests for
>> all BSPs without POSIX and networking (my system was busy for approx. 8h). I
>> will do build runs with POSIX and networking enabled next week.
> 
> Nice and well done.
> 
>> I think the build system structure is quite good. With the script items you can
>> also do complicated special case build steps, e.g.
>> 
>> https://git.rtems.org/sebh/rtems.git/tree/spec/build/bsps/powerpc/motorola_powerpc/RTEMS-BUILD-BSP-POWERPC-MOTOROLAPOWERPC-BOOT.yml?h=build
> 
> Interesting and useful.
> 
> In relation to the `cflags` as shown in this fragment are the flags ever
> separated into types, that is debug, optimise, machine, warnings etc?

Yes:

./waf bsp_defaults --rtems-bsps=sparc/erc32 | grep FLAGS
ARFLAGS = crD
CC_WARNING_FLAGS = -Wmissing-prototypes -Wimplicit-function-declaration -Wstrict-prototypes -Wnested-externs
CXX_WARNING_FLAGS = 
LDFLAGS = -Wl,--gc-sections
WARNING_FLAGS = -Wall
OPTIMIZATION_FLAGS = -O2 -g -fdata-sections -ffunction-sections
ABI_FLAGS = -mcpu=cypress

> 
> I think it is important to have the separation so we can export the flags as
> types in a pkgconfig file. A user can combine them in ways that suite them, for
> example some 3rd party packages cannot be built with the warning flags RTEMS
> has.

Yes, it is important to have the flags available in useful groups and not as one big chunk. For the pkg-config support see:

https://git.rtems.org/sebh/rtems.git/tree/spec/build/bsps/RTEMS-BUILD-BSP-PKGCONFIG.yml?h=build

> 
>> For 99% of the jobs the standard items are fine.
>> 
>> Open issues:
>> 
>> * Convert tests which use pax, see latest patches sent to mailing list:
>> 
>> https://lists.rtems.org/pipermail/devel/2019-November/056197.html
> 
> I reviewed the patch but I must have missed how this resolves the pax issue. How
> is that done?

The pax issue with waf was that waf doesn't like directories as nodes. This is understandable from a dependency tracking point of view. The IMFS untar support lacked the ability to create parent directories. So I unified the untar support. Having two untar implementations in RTEMS makes little sense.

> 
>> With these patches I think I am able to convert all C/C++ tests.
>> 
>> * Ada tests
>> 
>> * User manual documentation
>> 
>> * Licensing of *.yml files
>> 
>> * Generation of the old Makefile support
>> 
>> * Generation of pkg-config files
>>> https://lists.rtems.org/pipermail/devel/2019-November/056209.html
>> 
>> For the latest documentation proposals see:
>> 
>> https://ftp.rtems.org/pub/rtems/people/sebh/eng.pdf
>> 
>> https://ftp.rtems.org/pub/rtems/people/sebh/user.pdf
> 
> I would like to see if we can fix the latex generation on your machine if that
> is OK?

I have too much to do before my holidays. The document content is fine, only the branding is missing.

> 
>> The RTEMS Software Engineering parts are ready to commit from my point of view.
> 
> I have not reviewed these, I hope someone else does.
>> In the User Manual the quick start chapter is ready to commit (there was not
>> much to do). I added a new chapter "Build System". Please check if the chapter
>> placement is all right. I will add the content in the next week or so.
> 
> Looks good, so much simpler.
> 
> Should there be a note or something about waf needing python and we recommend
> python3? Plus waf needs a `python` installed and not just `python2` or
> `python3`?

I think this belongs to the Host Computer section. The quick start uses the RSB, so if you managed to build the tools, you must have a working Python. The RSB uses Python and the RTEMS Tools use waf.

> 
> How would a user adjust a BSP setting, for example the optimisation to -O1 to
> debug? A simple example would be nice. I see cannot see how as there is nothing
> in the INI file except building the tests.

This is too detailed for the quick start from my point of view. This is why I would like to add a Build System chapter.

> 
>> We have to decide how we continue with the integration. I would merge everything
>> in one patch into the RTEMS sources. This patch is too big to review.
> 
> Does this mean all the specs are added in that same patch?

Yes, the spec directory, waf, wscript, yaml, and gccdeps.py.

> 
>> Then I
>> would add a configuration option to the old configure script (e.g.
>> "--I-only-want-to-compare-results-with-the-new-build-system"). This basically
>> disables the normal use. The new build system should be used, fixed and
>> improved. In a three month period we keep the old build system in the sources.
>> Afterwards we remove it completely.
> 
> I am fine with this. We could also remove the autotools building from the RSB
> (yippy) when the old build system is removed.

Yes, after the three months we should remove everything related to the old build system. For a git bisect, you just have to build the corresponding tools with the RSB.