New source layout.

[Chris Johns]

On 11/03/2015 12:42 pm, Amar Takhar wrote:
> On 2015-03-11 00:36 +0100, Pavel Pisa wrote:
>> Hello Amar,
>>
>> thanks for reply.
>>
>> On Tuesday 10 of March 2015 19:00:03 Amar Takhar wrote:
>>> On 2015-03-10 11:03 +0100, Pavel Pisa wrote:
>>>> I think that includes really belong near to the implementation and that
>>>> new architecture could be (in optimal case) introduced by adding single
>>>> directory in "arch" subtree.
>>>
>>> The includes are near the implementation when they're local headers.
>>> Anything that is consumable by another part of RTEMS is moved.
>>
>> OK, if you think about it from perspective of public API it has
>> sense. But it is at some flexibility lost price. May it be it worth
>> to be paid.
>
> That's all relative and depends on what your definition of flexibility is.  I
> believe the method you are proposing is *less* flexible -- that is my opinion
> though.
>
> We all have different workflows.  What we're trying to do is to have a layout
> that makes RTEMS easy to test and helps us keep our interface(s) clean
> design-wise by easily having an overview of what we're doing.
>

I am attempting to dig out a web available link to the 27th July 1998 
posts about the pre-installing of header files for RTEMS. In some 
respects RTEMS is coming full circle. There were many claims made about 
pre-installing headers and over the last 16 years of supporting 
pre-installing some have happened and others have not and that is not 
important any more however the lesson we have learned with hindsight are 
worth discussing:

1. The number of headers has increased to a point it is not 
maintainable. If you step back and look it is kind of silly to have 
every build of RTEMS taking files from one location on your disk and 
putting then in another location to make exactly the same tree of files 
for every user who builds RTEMS. Why not just check out that structure ? 
The concept of modularity that says you can have a custom kernel with 
various options and so a custom include tree should not be allowed in 
RTEMS and the reason is simple, who tests all those combinations ? The 
RTEMS Project does not have the resources to test the few configure 
options we have now yet the RTEMS Project signs off on saying it is 
tested and ready for release. Personally I am not comfortable with 
"might work". Ideally RTEMS as released should be built to be a single 
repeatable configuration and structure with no variation. Of course this 
is an ideal situation and we are not at that point but we should be 
moving to it. There is variation across architectures and BSPs but this 
should be the limit of supported configuration options. We also have BSP 
options which under the autotools build system is messy and clumsy. 
RTEMS has evolved to better manage how we configure the runtime and BSP 
components and we have a long way to go. Having a single include tree 
helps improve auditing RTEMS.

2. The complexity it added to the build system was much more than 
expected and it has never been robust and stable. There are a number of 
ways a change in a header file does not result in a pre-install update 
and a correct build. I am sure everyone who has played with RTEMS code 
has edited the installed header because a compile error brings it into 
the editor only to have it overwritten at a later stage when pre-install 
kicks in.

3. Finally a key issue is leaking APIs. The experience we have had with 
headers in a module's directory has resulted in leaking API headers and 
this is partially the reason we have so many header files being 
installed. When designing and building a module it is difficult to force 
yourself to create an API and stick to it and test you have done it 
cleanly. The simple method we have all used to overcome this is to add 
the needed headers to the pre-install list "hey it is only a couple of 
extra headers and it saves me having to re-factor the code". I feel 
RTEMS has too many internal header exposed and this is directly related 
to the localised module approach for development.

>
>> The native system one should be strictly avoided, Linux kernel
>> uses -nostdinc. Required cross-GCC headers and newlib can be reintroduced
>> by adding explicitly directories reported by GCC as right multiarch and base
>> includes.
>
> We have an entirely different scenario from the Linux kernel:
>
>   * Multiple compilers
>   * Multiple platforms
>
> The waf build system supports any compiler it has been 100% unhooked from GCC.
> Once a BSP compilers CLang can be dropped in and used to build some BSPs.
>
> This solution also removes the possibility of RTEMS compiling under several
> compilers native to Windows.
>

This is important in terms of the waf build system but any compiler 
should be able to work if we use a standard set up for driving it. 
Currently we need to use specs files and that is limiting us to gcc.

> The key difference between what is being done above and what I'm proposing is
> there is only one include on the compiler, ever -I/path/to/include and nothing
> more.  This is for *both* public API linkage and internal to RTEMS.  Software
> that links different whether it's building itself or being linked against is
> confusing.

This is the subject of another thread so we should discuss it there.

>
> Symlinks won't work on Windows.
>

I will not support the use of symlinks.

>>> Another benefit of having them all in the same place is it is far easier to
>>> see what is going on when working on RTEMS.  The preinclude step hides far
>>> too much, so do multiple -I lines.
>>
>> Yes, removing automake and preinclude for RTEMS is usesfull goal.
>> Automake does not play nice in RTEMS case, it takes horrible time
>> to bootsrap tree. Much longer then actual build in my case.
>
> Please see this page:
>
>    https://devel.rtems.org/wiki/waf/Timing
>
>  From 'git co' you can have RTEMS built in 12 seconds *including* all tests.  On
> the average machine it takes ~25s including all tests.  This is not an
> exaggeration please try it for yourself.
>

Performance is important but the overall architecture needs careful 
consideration.

Chris