Draft for moving network headers from RTEMS to newlib

[Chris Johns]

On 26/04/2016 17:31, Sebastian Huber wrote:
> On 26/04/16 07:51, Chris Johns wrote:
>> On 26/04/2016 01:06, Christian Mauderer wrote:
>>> currently we try to remove the network specific POSIX headers from
>>> RTEMS. Instead, we add current headers from FreeBSD to newlib. This will
>>> simplify the build process of some libraries that depend on the network
>>> (like LibreSSL).
>>
>> What does this work flow offer over building and installing an RTEMS
>> kernel for a BSP and adding that path to the packages include paths?
>
> You don't build per BSP in this scenario. You build per multilib and
> have only one set of header files installed.

Hmm multilib, this is what I thought was happening. I see this exploding 
the libraries if packages are built this way. Do we manage this or not 
bother and just accept all variants have to be built?

> You are able to use the
> network header files during GCC build, which makes it easier to build
> the run-time libraries of Ada and Go.

Nice. Maybe limiting what is added to just building languages would be a 
nice first starting point.

>
>>
>> How do you get the flags for the compiler to build the package?
>
> See attached script which builds for example libpng for all multilibs.

Multilibs has not had a very successful history with RTEMS (outside of 
gcc). They are not very user friendly and are a source of questions and 
issues.

I would like to see something more than a script posted before I get 
excited and accepting of this path. I would like to understand the work 
flow for the project, the developers and the users?

> You don't need BSPs installed to do this. The libpng is just an add-on
> to the tool chain.

Sure, I understand this and it is nice. I hope users do not forget to 
build the BSP. ;)

>
>>
>> How are any tests present in the package built and linked?
>
> Tests are executables, so they need a BSP.
>

Yes, so how do these packages get tested?

>>
>> Do we risk limited the functionality of a package by restricting the
>> headers exposed to only standards based headers? There are headers
>> which some packages use that will not be present.
>
> In case a common header file is missing for a particular package, then
> we can add this file to Newlib.
>

If a package changes or something happens we would need a new version of 
tools. I am concerned about the management of the detail here and 
finding a suitable set of headers for all packages across all 
architecture may become complicated if there are incompatibilities 
between architectures and/or packages. With standards based headers we 
have the standard to fall back on and we can get the package fixed. When 
we move away from standards based headers we open up a range potential 
issue. Adding the headers is easy, removing would be hard if not 
impossible. Once we head down this path will be difficult to turn back.

It is a massive task to get the tools to build for supported 
architectures _now_ and I hope this path does not increase the work load 
here.

Do you have limits or boundaries for suitable headers? Who administers 
these boundaries?

What is the procedure for a user to add a header?

If a header is missing does this means the user cannot build a package? 
In other words can we mix how we currently build packages with this way 
of building packages?

>>
>>> Further it will be another step into the direction of
>>> extracting the old RTEMS network stack and build it as an independent
>>> package.
>>
>> Is this just a specialised version of the generic vertical integration
>> problem being discussed in the civetweb thread?
>>
>> I am not against standards based headers like the ones being discussed
>> here being move to newlib however I currently do not see what the
>> advantage is and how value is being adding over a specific build order
>> of packages.
>
> Building libraries against a particular BSP using actually BSP
> independent header files is not really a great overall setup.

Correct each BSP is independent and that gives me the ability to vary 
and change things in RTEMS without needing to repeat tools builds or 
create separate sandboxes at the tools level. The whole point of the 
cpukit was to multilib build the generic code and avoid the duplication 
in the BSPs however in the end it became clear a single specific BSP is 
all most users need and overhead of that specific build and install even 
with per BSP headers is faster and simpler than the whole multilib path.

Having a single tree of headers is fine if they are stable and not 
changing. I do not use a single prefix and prefer separate prefixes so I 
can mix and match and play. Having more and more header become part of 
the tools removes an important part of this process.

It comes down to the type of header and how often they change.

>
>>
>> I still see all the normal issues of CFLAGS, LDFLAGS, 3rd party
>> package dependence still being present once this work is completed.
>
> For libraries, the LDFLAGS are irrelevant.

What about testing? This needs to be resolved.

> One advantage is that the
> built-in search paths are sufficient if you build against a multilib
> (except the machine flags), see attached script.

This assumes you install everything under the one prefix. Unless you are 
on a production build I do not recommend this. If you are building and 
installing different libraries under different prefixes you are back at 
the same issue of handling include paths so include paths will always 
need to work.

Please understand I am not against this, I would just like the all 
detail exposed and understood.

Chris