[PATCH 1/3] build: Format build items

[Chris Johns]

On 20/1/2023 6:01 am, Amar Takhar wrote:
> On 2023-01-19 08:21 +0100, Sebastian Huber wrote:
>>
>> In rtems-central.git there are Python modules and scripts which generate 
>> source, header, and documentation files from specification items. This 
>> repository contains the pre-qualification support for RTEMS. By 
>> accident, a part of the build system uses also specification items. So, 
>> if you want to change the format of the items, you can use the tooling 
>> available in rtems-central.git to do this. An example is the recent 
>> merge of the "default" and "default-by-variant" attributes.
> 
> Why are these in 'rtems-central' and not the RTEMS repository?  Why are we using 
> tools with unpinned versioning generating files in RTEMS?

My view is below.

>> The changes were made by a 10 liner throw away script. I could have done 
>> the changes also manually it would be just more work.
> 
> This does not alter my above point.  I understand that in this case it was a 
> script but the rest wasn't generated by a throw-away script.

It is good to know the amount of python is small. It should be easy to add :)

>> There is nothing to regenerate. The patch set contains simple format 
>> changes and a transformation of attributes.
>>
>> However, your concern is still valid for the generated files, for example
>>
>> https://github.com/RTEMS/rtems/blob/master/cpukit/include/rtems.h
>>
>> which is generated from
>>
>> https://github.com/RTEMS/rtems-central/blob/master/spec/rtems/if/header.yml
> 
> Yes I do not like this at all both the software and header.yml should be sitting 
> in our main repository.

I have been OK with the headers and tests being generated this way because the
agreement is files in rtems.git can be manually edited and rtems-central has to
track those changes. The sources and code to generate the files need to be
located together.

>> Most of the Classic API header files are generated from specification 
>> items. If nobody takes care that rtems-central.git is used and stays up 
>> to date the specification will get out of synchronization with rtems.git.
> 
> Okay but why is this sitting in rtems-central.git and not in rtems.git.  You've 
> just made the argument that it's very important we don't want anything to be out 
> of sync.

The specification maintenance is a separate responsibility to the function and
role of rtems.git and related repos. The bar to make changes in rtems.git is at
the source level because it is the norm for open source projects and those
working with the code. The separation also helps make it clear how funding of
the qual related work happens. If we bring those specifications items down into
rtems.git the responsibility moves to that repo and that raises the bar on being
able to make a change. Our view on this may change as we learn now the pre-qual
effort effects what we do.

We have YAML files to run our build system and it is awesome. It also allows
automation so I think it makes sense to provide an interface to read and write
these files to aid automation rather than a disjointed set of personal scripts,
tools etc that may break or clash.

>>> I'm not against the purpose of the tool or the work has been done but this is
>>> not the correct way to handle generated files within a repository if we want to
>>> maintain the ability to make changes or debug years down the line.
>>
>> I would use a single repository for RTEMS just like the FreeBSD base 
>> system, but this is another discussion.
> 
> FreeBSD also uses a vendor system which brings all this code under a central 
> repository.  They aren't maintaining code outside and then requiring FreeBSD to 
> be built with it.  They even bring the compiler in this has an astounding impact 
> on the usability of FreeBSD.
> 
> Recently I took a FreeBSD 3.1 image and rebuilt it for a customer who needed a 
> change it was used in an appliance.  It just worked.
> 
> If I want to make a chance to rtems.h it's now generated from a header.yml with 
> no version pinning how am I going to find the first header.yml to make a single 
> change in 20 years?
> 
> rtems.h is a small file where we can get rid of it and move to another tool.  
> It's integral and anything that works with it is integral this is arguably 
> different.

The FreeBSD single repo is about the kernel and base runtime. The ports are not
part of this so the analogy breaks down. Bringing the RSB, RTEMS tools and
documentation into a single repo creates new complexities I am not sure we are
prepared enough to handle those. Maybe patch review and CI would help here and
how they effect a single repo is something I have not given much consideration too.

Chris