Separation of RTEMS sources and tool chain patches

[Chris Johns]

On 26/09/2014 7:40 pm, Sebastian Huber wrote:
> On 18/09/14 00:58, Chris Johns wrote:
>> On 17/09/2014 6:49 pm, Sebastian Huber wrote:
>>>
>>> currently the RTEMS sources contain no reference to the intended tool
>>> chain versions (Binutils, Newlib, GCC, GDB) and patches for the tools.
>>> This is specified elsewhere, for example in the RTEMS tools repository.
>>>
>>
>> The RSB has the ability to report its configuration as an INI file. If
>> you:
>>
>>   $ cd rtems-source-builder/rtems
>>   $ ../source-builder/sb-reports --format=ini 4.11/rtems-all
>>
>> you will get a '4.11-rtems-all.ini' file. It contains the exact source
>> configuration including the MD5 hashes for all the architectures and
>> is human
>> readable.
>>
>> Just looking at the output now I see I did not place the RSB git hash
>> in a
>> section and I think it should be. It is in a comment. I will make the
>> change.
>> If you have the git hash you can request that specific RSB version and
>> so build
>> those tools.
>
> This is really nice.  I think the default output is even better since it
> includes also the shell scripts for the tool builds.  Maybe we should
> add also some sort of XML output since this is easier to parse.
>

I can add the scripts to INI file format. I feel XML is too heavy a 
requirement for parsing. There is a single C++ file that does it and 
Python handles the format easily. I also think it is easier to read.

FYI I have never tested the results of the scripts outside of the RSB. I 
have attempted to make sure everything is captured however only testing 
will prove this is ok. I do not have the time to do this. The scripts do 
not handle the unpacking or patching. That is outside the scope of what 
they are suppose to do.

>>> Since the RTEMS version is tightly coupled to a particular tool chain
>>> version (mostly do to interaction with Newlib) it would be beneficial to
>>> add a human and machine readable tool chain description to the RTEMS
>>> sources (for example base version plus patches).  One consumer of this
>>> description could be the RSB.  This would enable automatic builds of a
>>> compatible tool chain as part of a Git bisect process to find changes
>>> causing a regression.
>>
>> This is a nice idea and I fully support us doing this. There are a
>> couple of
>> assumptions that need to be handled.
>>
>> The first is a mutual dependence between the tools and RTEMS so
>> someone needs
>> to build the tools and then build RTEMS then take the INI file and add
>> it to
>> RTEMS so in effect tagging the RTEMS version with the tools.
>>
>> The second assumption is the build hosts are not moving too fast.
>> There will be
>> a historical limit to which we can regression test based on the
>> specific host's
>> ability to build older tool sets. For example breakage tends to
>> happens when a
>> host picks up a new version of a compiler that moves the default standard
>> supported.
>>
>> Doing this means a user should be able to get an RTEMS source tree and
>> run a
>> top level command that builds the tools and then RTEMS. I like this
>> because it
>> is logical for a user to do and removes a major step that has proved
>> to be
>> error prone and problematic for new users.
>>
>> The RSB built tools embed the RSB hash in the GCC version string so
>> the RTEMS
>> build system could verify the tools match if not in maintainer mode.
>
> The RSB uses the rtems-tools repository to get some patches.  Uses it
> always the latest versions or a particular commit?

The patches are specific by the file name and should not change unless 
there is an error in the importing into the rtems-tools repo. I have 
started referencing the upstream versions these days via patchworks and 
this is proving much simpler for me in terms of maintenance. As the 
patch is merged upstream the reference is removed and we do not have the 
clutter left in rtems-tools.

>
> Why do we not move the tool chain patches back to the RTEMS sources?  I
> think it would be nice if you can check out a particular RTEMS version
> and then use the RSB and say: build me the tool chain for this version.
>

I am ok with the INI file that defines the tools for a release being 
held in RTEMS and the build system checking the tools however at this 
point in time I do not think is it a good idea to move everything into 
RTEMS. I have not given the idea much consideration but I see issues 
with the validation process. For example the tools and RTEMS can move 
independently of each other and on head this is a little more difficult 
however with a release things are much more stable. This means a dot 
point release of RTEMS does not require a dot point version of tools.

My view may change as what we have moved too settles and maybe I am just 
hesitant to move too quickly here.

Chris