All installed host tools are removed from the RTEMS sources - next steps?

[Chris Johns]

On 18/06/2018 17:03, Sebastian Huber wrote:
> On 18/06/18 08:39, Chris Johns wrote:
>>> The BSP sources and header files were moved some weeks ago to "bsps/*", but the
>>> build files (configure.ac and Makefile.am) are still in "c". The next goal is to
>>> move also the build files to "bsps/*". The long term goal is the introduction of
>>> a new build system based on waf. Each change in the current build system should
>>> make it easier to introduce the new build system.
>> With all these changes I suspect a new build system is going to distill down to
>> the configuration details and the management of that data.
>>
>> An example Amar created is:
>>
>> https://git.rtems.org/amar/waf.git/tree/py/waf/defaults/bsp/arm.py?h=archive/waf/2015-06
>>
>> https://git.rtems.org/amar/waf.git/tree/py/waf/defaults/options.py?h=archive/waf/2015-06
>>
>>
>> I am not sure a like a single file for the whole of RTEMS, it could becomes a
>> merge hot spot plus I do not like having it in the build system source tree. I
>> see the build system files and the data that drives it as separate. You should
>> be able to roll out build system v2, v3 etc and the data stays consistent.
> 
> Yes, BSP-specific stuff should definitely be not in a separated repository.

Oh of course not. Should these files be in the BSP tree or a config tree?

> Why is the build system in a separate repository?

Sorry I was meaning a directory for the build source in rtems.git. The links I
provided are to a clone of the RTEMS kernel repo and all the build code and
config data is in the same tree.

> I don't think that a single global file would lead to a merge hot spot, we don't
> add that many BSPs per day. I am still more in favour of per-BSP configuration
> files (no scattered BSP-specifics throughout the code base).

Great.

> It would be nice to have the configuration data in a simple format (e.g.
> Makefile fragment, INI, pkg-config) and not as a Python source file.

Agreed. I like INI because Python does not need any external packages. I also
like YAML but that needs 3rd party packages and that complicates things.

> What about the post-link steps?

This info should be captured where it can however I suggest we leave it for
later. If it falls out as part of the work then add it otherwise we standardize
on ELF. The tester has already made a successful steps in the post-link
direction so maybe we can review and merge what exists there. The issue with
adding it into the rtems.git repo is the expected dependence the post link tools
make on the kernel build system. Getting these tools to work on all hosts is
time consuming and a distraction.

Also the post-link processing uses more disk space and slows the build down. The
demand driven rtems-test solution is simple and cleans up after.

The post-link config can be captured as pkg-config variables (or something like
that) and reported with a query.

> Should we also consider the  BSP configuration options? This area is a lot more
> complicated and time consuming to change. I would like to post pone this and
> concentrate on build system issues first, e.g. location and format of compiler
> flags.

That is what the second link has. That code was generated from the BSPOPTS. The
config support Amar created has range checking and more. The data could be read
from a INI file but the checking kept etc is neat.

>>
>> I do not know if we want to localize the data to the functionality being
>> configured, for example shell configuration in the shell source. I think is
>> makes sense for a BSP. I think a hierarchy is a good thing so there can be reuse
>> of common configuration data.
>>
>>> I think one result of previous discussions is to use pkg-config format for
>>> BSP-specific build system input as a replacement of the bsps/*/*/config/*.cfg
>>> files.
>>>
>>> https://en.wikipedia.org/wiki/Pkg-config
>>>
>> Yes I agree with using this format. The difficult part of this is capturing the
>> details needed to create a suitable set of flags. The current config files are
>> make format meaning there is no standard pattern and each BSP needs to be
>> reviewed. We can also consider using variables and a variable query to add extra
>> detail if we need it.
>>
>>> We could temporarily add a Makefile rule to generate the first version of the
>>> new BSP-specific file, e.g. bsps/arch/bsp/config/some-bsp.pc. Then build all
>>> BSPs to get the initial content for the RTEMS sources.
>> Why replace the existing one, why not make it better? The rtems_waf support for
>> applications is using the existing file now. It is not perfect but it does work.
>>
> 
> Do you want to use the pkc-config files internally or only export them to the
> installed BSP? The *.cfg files are Makefile fragments and not pkc-config files.

I was thinking internally. It is a simple way to make sure they work. The .cfg
files will go away in time. Initially I though about creating the .pc files to
export and checking them with rtems_waf and the examples and then switching to
internally use them. I do not know if this can be BSP by BSP.

I have some python and C++ code that handles pkg-config and I seem to remember
FreeBSD had a simple piece of code to do the same thing. This way we can avoid
the problematic pkg-config where it is not easily supported, ie darwin, windows
(cmd.exe).

Chris