question about building framework

[Chris Johns]

On 05/01/2017 10:09, Denis Obrezkov wrote:
> Hello,

Welcome.

> I am trying to understand a building framework architecture.

Ok.

> And I have a lot of questions:

No problem, I have created a long and detailed answer, maybe too long. :)

> what is the future of autotools and waf framework regarding rtems kernel
> and tools?

The autotools build frame work in the RTEMS kernel source is being 
patched and no major changes or work is being done on it. We will be 
moving to a waf based build system when the time and funds become available.

-----
To the whole community, we would welcome any funding, big or small to 
help this effort. It is unrealistic to think it can be done unfunded by 
core developers. Please directly contact Joel or me if you can help.
-----

The current waf build tree has 2 parts, the addition of waf and moving 
headers away from the pre-install mess we have to single include tree in 
the source. Both parts are important and needed. I am currently 
evaluating separating these into 2 stages, adding waf first then a 
header file move. I feel we can do each stage but together the effort is 
too big without a lot of work.

The tools are separate and not related. The build systems the tools use 
are contained in those packages and do not effect RTEMS or any RTEMS 
application build systems.

> what is the motivation to use wav framework?

I can list the reasons I can think of however build framework 
discussions often descend into arguments based on personal preferences 
and I would like to avoid that.

Waf is being used because:

- It is built on the proven popular and widely available programming 
language, Python. It is not based around a language specially created as 
a collection features each designed to solve a specific build problem 
that may not scale or interact cleanly with each other. With a proper 
programming language at it's base waf lets the build system have an 
architecture and designed based on proven software design methodologies 
like any other part of the code in the system. You could even have unit 
tests for the build system. I often think it is funny how we spend a lot 
of time reviewing and auditing code and a build system is often never 
reviewed yet it creates the binary that runs on the target and a wrong 
flag or option can cause subtle hard to find issues.

- It is fast on all hosts including Windows. It evaluates all targets 
before starting a build and automatically runs all cores on multicore hosts.

- It is self contained, no need to have an external package installed to 
work. Waf is included as source in the source tree of the package being 
built.

> will it replace RSB?

No, the RSB is a separate tool and not unrelated to build systems.

> will it be possible to use makefiles?

I assume you are talking about building applications, yes you can use 
Makefiles to build applications.

There is no relationship between a user's build system for an 
application and what is used to build the RTEMS kernel. The RTEMS 
Project encourages user to select a build system that suites their 
project, system or company needs.

Historically RTEMS has documented applications be built with 
Makefile.inc which implied a relationship. The unfortunate side effect 
of this is the impression there is a direct relationship. The installing 
of some of the RTEMS build support files to aid building a specific 
niche of applications has created this misunderstanding.

> what is the preferable way of building custom application? makefile or
> wscript?

As stated before there is no preferred method because everyone is 
different. People have personal favourites, localised system demands, 
integration to larger systems and many more reasons.

The critical factor in an application build system is making sure you 
use the same or compatible compiler and linker flags when building your 
application for all the code linked into your application. RTEMS is a 
statically linked executable so all code needs to conform to the same 
ABI and that ABI needs to be supported by your target hardware. The BSP 
defines this.

How you handle the compile flags, linker flags and libraries depends on 
your system requirements and there is no right or wrong way so when 
reviewing comments I suggest you look at posts and comments from other 
users as their experiences rather than rules and pick out what works for 
you. For example you may hard code the compiler, cflags, ldflags and 
libs into your build system because you have a single BSP you need to 
support. A more complex case is a single application that needs to run 
on a number of different BSPs so you may hard code a number of flags or 
extract the information from RTEMS somehow.

RTEMS application build systems can be view as having 3 basic parts:

1. An application build system. This is how code is built and is no 
different to any application or package on any OS.

2. Cross-compiling. Do not under estimate what this means.

3. BSP compiler flags, linker flags and libraries.

All build system systems will handle each of these in different ways and 
with varying degrees of success. I will avoid going into the strengths 
and weaknesses of each system because that is off topic here and I do 
not want to get flamed. :)

The important item is 3, BSP compiler and linker flags. There is a 
missed place view that all RTEMS applications need to be based on 
Makefiles because you need to include Makefile.inc to get the compiler 
and linker flags (disclamer: my dislike of Makefile.inc is well known 
and over 15 years old).  Let me explain the issue in detail.

Installing and exporting of various pieces of the RTEMS build system to 
aid building applications helps a specific niche of users however it has 
created a maintenance issue the project can no longer sustain. RTEMS 
currently exports to users a makefile fragment from deep within the BSP 
with no rules or user interface that is only accessible by the make 
program. It is very difficult, almost impossible to parse without 
running make due to macros and make variables which means non-make build 
systems are excluded. On top of this BSP developers have used a wide and 
varying set of make features, eg check the i386/pc686's code base 
address of 0x0010000. It is very difficult for the RTEMS Project to 
control this.

As a result a future version of RTEMS will *not* export any application 
makefile support, ie Makefile.inc, so I recommend users start to 
consider this now.

The question remains, how does a user get the compiler and linker flags 
for a BSP? We are adding a tools called 'rtems-config' which is simlar 
to pkgconfig. You ask it a question about your BSP, eg cflags, ldflags, 
libs, and it returns the answer. The tool will be formally supported and 
its user interface will be separated from how it works and what happens 
inside the RTEMS kernel and source code.

Finally the rtems_waf.git repo in my personal git area provides an easy 
to use way to integrate waf and RTEMS for applications. The examples-v2 
and libbsd use it.

The rtems_waf.git package uses pkgconfig and some local tweaks to get 
the data from installed BSP .pc files. This works but only to a limited 
extent. The BSP makefile fragments are too varied to make generating 
correct .pc files possible from within RTEMS when building it and the 
effort was abandoned and has not and will not progress any further.

Chris