New Build System Ready for Integration

[Sebastian Huber]

On 02/12/2019 01:00, Chris Johns wrote:
> 
> 
> On 28/11/19 7:10 pm, Sebastian Huber wrote:
>> Hello,
>>
>> the new build system is now ready for integration. You find the latest
>> documentation here:
>>
>> https://ftp.rtems.org/pub/rtems/people/sebh/user.pdf
>>
>> https://ftp.rtems.org/pub/rtems/people/sebh/eng.pdf
>>
>> You can review the new build system for RTEMS BSPs here:
>>
>> git clone git://git.rtems.org/sebh/rtems.git
>> cd rtems
>> git checkout --track origin/build
>>
>> The new build system builds all BSPs and all tests (including the Ada tests). I
>> tested the build on OpenSUSE 15.1, MinGW64, MSYS2, and FreeBSD 12.1.
>>
>> The build specification consists of
>>
>> find spec -name '*.yml' -and -not -name .doorstop.yml | wc
>>     2011    2011  136959
>>
>> YAML files. From these only 18 have hand-crafted content:
>>
>> grep -r 'build-type: script' spec/ | sort
>> spec/build/bsps/powerpc/motorola_powerpc/RTEMS-BUILD-BSP-POWERPC-MOTOROLAPOWERPC-BOOT.yml:build-type:
>> script
>> spec/build/bsps/powerpc/motorola_powerpc/RTEMS-BUILD-BSP-POWERPC-MOTOROLAPOWERPC-QEMUFAKEROM.yml:build-type:
>> script
>> spec/build/bsps/powerpc/mvme5500/RTEMS-BUILD-BSP-POWERPC-MVME5500-START.yml:build-type:
>> script
>> spec/build/bsps/powerpc/RTEMS-BUILD-BSP-POWERPC-MOTLD.yml:build-type: script
>> spec/build/bsps/RTEMS-BUILD-BSP-LINKCMDS.yml:build-type: script
>> spec/build/cpukit/RTEMS-BUILD-CPUKIT-VCKEY.yml:build-type: script
>> spec/build/testsuites/libtests/RTEMS-BUILD-TEST-LIB-DL01.yml:build-type: script
>> spec/build/testsuites/libtests/RTEMS-BUILD-TEST-LIB-DL02.yml:build-type: script
>> spec/build/testsuites/libtests/RTEMS-BUILD-TEST-LIB-DL04.yml:build-type: script
>> spec/build/testsuites/libtests/RTEMS-BUILD-TEST-LIB-DL05.yml:build-type: script
>> spec/build/testsuites/libtests/RTEMS-BUILD-TEST-LIB-DL06.yml:build-type: script
>> spec/build/testsuites/libtests/RTEMS-BUILD-TEST-LIB-DL07.yml:build-type: script
>> spec/build/testsuites/libtests/RTEMS-BUILD-TEST-LIB-DL08.yml:build-type: script
>> spec/build/testsuites/libtests/RTEMS-BUILD-TEST-LIB-DL09.yml:build-type: script
>> spec/build/testsuites/libtests/RTEMS-BUILD-TEST-LIB-DL10.yml:build-type: script
>> spec/build/testsuites/libtests/RTEMS-BUILD-TEST-LIB-MGHTTPD01.yml:build-type:
>> script
>> spec/build/testsuites/libtests/RTEMS-BUILD-TEST-LIB-TAR01.yml:build-type: script
>> spec/build/testsuites/libtests/RTEMS-BUILD-TEST-LIB-TAR02.yml:build-type: script
>>
>> So, more than 99% of the build specification is done through standard items. The
>> attached script can be used to load the content of all items into a standard
>> Python data structure (dictionaries, lists, strings, integers). The next person
>> doing a new build system will have a much easier task, no more parsing of
>> Makefile.am, configure.ac, *.cfg, and *.tcfg files.
>>
>> My proposed next steps are:
>>
>> 1. I push the source and documentation patches next Monday to the main
>> repositories.
> 
> What is reason for this needing to happen right now?

At some point it should be good enough. I don't want to wait until it is 
perfect. This Monday was probably a bit too optimistic. There are three 
open issues from my point of view:

1. The license and copyright information of the specification items.

https://lists.rtems.org/pipermail/devel/2019-November/056209.html

2. The pkg-config support:

https://lists.rtems.org/pipermail/devel/2019-November/056268.html

3. The rtems_test_pause():

https://lists.rtems.org/pipermail/devel/2019-December/056292.html

It needed a bit of debugging from my side to figure out why some tests 
ended up in an endless loop.

> 
> Has the other thread between you and Joel been resolved? I am not sure if it
> has. I saw you posted somethings we need to make lists of or document. Has these
> been made and agreed too?
> 
> I would like to see these things resolved before we make any permanent changes
> by pushing this code. 

The new build system is not read-only once it is integrated.

> I would also like to have the Windows gccdeps issue
> resolved. I am attempting to look into this now.

The Windows gccdeps issue is not a blocking issue from my point of view. 
It is only loaded in case we are not on a Windows host as a workaround.

> 
>> 2. I make it harder to use the existing build system. I add a
>> --I-only-want-to-compare-results-with-the-new-build-system" to the top-level
>> configure script. If this option is not present, then the build stops with an
>> error. This basically disables the normal use.
>>
>> 3. I create a new ticket with a build system conversion checklist.
> 
> Would it be better if this is done before pushing any code.

I started with the check list:

https://devel.rtems.org/ticket/3828

> 
>> 4. We announce a four month period in which we keep the old build system in the
>> sources. Everyone is encouraged to test the new build system with the help of
>> the checklist. The new build system should be used, fixed and improved.
> 
> I will try and push a new release snapshot this week. I have been away and it is
> not yet automated.
> 
>> 5. If the new build system shows satisfactory results after the testing period,
>> we remove the old one.
> 
> What does satisfactory mean?

Satisfactory means, no open build system related bugs reported on the 
mailing list and the ticket system.

-- 
Sebastian Huber, embedded brains GmbH

Address : Dornierstr. 4, D-82178 Puchheim, Germany
Phone   : +49 89 189 47 41-16
Fax     : +49 89 189 47 41-09
E-Mail  : sebastian.huber at embedded-brains.de
PGP     : Public key available on request.

Diese Nachricht ist keine geschäftliche Mitteilung im Sinne des EHUG.