New Build System Status

[Chris Johns]

On 23/11/19 9:15 pm, Sebastian Huber wrote:
> ----- Am 22. Nov 2019 um 23:31 schrieb Chris Johns chrisj at rtems.org:
> 
>> On 23/11/19 1:49 am, Sebastian Huber wrote:
>>> Hello,
>>>
>>> I converted all BSPs to the new build system. I was able to build the tests for
>>> all BSPs without POSIX and networking (my system was busy for approx. 8h). I
>>> will do build runs with POSIX and networking enabled next week.
>>
>> Nice and well done.
>>
>>> I think the build system structure is quite good. With the script items you can
>>> also do complicated special case build steps, e.g.
>>>
>>> https://git.rtems.org/sebh/rtems.git/tree/spec/build/bsps/powerpc/motorola_powerpc/RTEMS-BUILD-BSP-POWERPC-MOTOROLAPOWERPC-BOOT.yml?h=build
>>
>> Interesting and useful.
>>
>> In relation to the `cflags` as shown in this fragment are the flags ever
>> separated into types, that is debug, optimise, machine, warnings etc?
> 
> Yes:
> 
> ./waf bsp_defaults --rtems-bsps=sparc/erc32 | grep FLAGS
> ARFLAGS = crD
> CC_WARNING_FLAGS = -Wmissing-prototypes -Wimplicit-function-declaration -Wstrict-prototypes -Wnested-externs
> CXX_WARNING_FLAGS = 
> LDFLAGS = -Wl,--gc-sections
> WARNING_FLAGS = -Wall
> OPTIMIZATION_FLAGS = -O2 -g -fdata-sections -ffunction-sections
> ABI_FLAGS = -mcpu=cypress
> 

Ah OK. The spec file in the link has only a single cflags and that confused me.

>> I think it is important to have the separation so we can export the flags as
>> types in a pkgconfig file. A user can combine them in ways that suite them, for
>> example some 3rd party packages cannot be built with the warning flags RTEMS
>> has.
> 
> Yes, it is important to have the flags available in useful groups and not as one big chunk. For the pkg-config support see:
> 
> https://git.rtems.org/sebh/rtems.git/tree/spec/build/bsps/RTEMS-BUILD-BSP-PKGCONFIG.yml?h=build

Excellent and thanks.

>>> For 99% of the jobs the standard items are fine.
>>>
>>> Open issues:
>>>
>>> * Convert tests which use pax, see latest patches sent to mailing list:
>>>
>>> https://lists.rtems.org/pipermail/devel/2019-November/056197.html
>>
>> I reviewed the patch but I must have missed how this resolves the pax issue. How
>> is that done?
> 
> The pax issue with waf was that waf doesn't like directories as nodes. This is understandable from a dependency tracking point of view. The IMFS untar support lacked the ability to create parent directories. So I unified the untar support. Having two untar implementations in RTEMS makes little sense.

Oh OK I see.

>>> With these patches I think I am able to convert all C/C++ tests.
>>>
>>> * Ada tests
>>>
>>> * User manual documentation
>>>
>>> * Licensing of *.yml files
>>>
>>> * Generation of the old Makefile support
>>>
>>> * Generation of pkg-config files
>>>> https://lists.rtems.org/pipermail/devel/2019-November/056209.html
>>>
>>> For the latest documentation proposals see:
>>>
>>> https://ftp.rtems.org/pub/rtems/people/sebh/eng.pdf
>>>
>>> https://ftp.rtems.org/pub/rtems/people/sebh/user.pdf
>>
>> I would like to see if we can fix the latex generation on your machine if that
>> is OK?
> 
> I have too much to do before my holidays. The document content is fine, only the branding is missing.

When you are ready please ping me.

>>> The RTEMS Software Engineering parts are ready to commit from my point of view.
>>
>> I have not reviewed these, I hope someone else does.
>>> In the User Manual the quick start chapter is ready to commit (there was not
>>> much to do). I added a new chapter "Build System". Please check if the chapter
>>> placement is all right. I will add the content in the next week or so.
>>
>> Looks good, so much simpler.
>>
>> Should there be a note or something about waf needing python and we recommend
>> python3? Plus waf needs a `python` installed and not just `python2` or
>> `python3`?
> 
> I think this belongs to the Host Computer section. The quick start uses the RSB, so if you managed to build the tools, you must have a working Python. The RSB uses Python and the RTEMS Tools use waf.

The RSB can use python2 or python3 without a python. What about a note to say
... "Waf uses python and you need to make this command available on your system".

>> How would a user adjust a BSP setting, for example the optimisation to -O1 to
>> debug? A simple example would be nice. I see cannot see how as there is nothing
>> in the INI file except building the tests.
> 
> This is too detailed for the quick start from my point of view. This is why I would like to add a Build System chapter.

Sure this makes sense however as a new user working through the Quick Start a
link to how configuration is handled is important, it provides a clear hint at a
possible next step and a new user does not have to had read the entire document.

>>> Then I
>>> would add a configuration option to the old configure script (e.g.
>>> "--I-only-want-to-compare-results-with-the-new-build-system"). This basically
>>> disables the normal use. The new build system should be used, fixed and
>>> improved. In a three month period we keep the old build system in the sources.
>>> Afterwards we remove it completely.
>>
>> I am fine with this. We could also remove the autotools building from the RSB
>> (yippy) when the old build system is removed.
> 
> Yes, after the three months we should remove everything related to the old build system. For a git bisect, you just have to build the corresponding tools with the RSB.

OK.

Chris