[PATCH v2 1/2] waf: Support building from libbsd.py directly from waf.

Wed Apr 4 05:28:00 UTC 2018

On 04/04/2018 00:43, Christian Mauderer wrote:
> Am 27.03.2018 um 09:56 schrieb Christian Mauderer:
>> Am 27.03.2018 um 09:00 schrieb Christian Mauderer:
>>> Am 26.03.2018 um 14:59 schrieb Christian Mauderer:
>>>> Hello Chris,
>>>>
>>>> I took the liberty to rebase the patches to the latest master. See the
>>>> two following v3 mails. I'll take a look at the configuration based on
>>>> these patches.
>>>>
>>>> I think that you wanted to make i386 tests. After that it might would be
>>>> a good idea to commit them. What do you think?
>>>>
>>>> Best regards
>>>>
>>>> Christian
>>>>
>>>
>>> Hello Chris,
>>>
>>> just noted: The freebsd-to-rtems.py (with -R) doesn't work anymore. I'll
>>> fix that.
>>>
>>> Maybe it would be a good idea to develop that (and the config options)
>>> on a branch (on a user specific repo on git.rtems.org) and post it for
>>> review? At least it's a bigger change.
>>>
>>> Best regards
>>>
>>> Christian
>>>
>>
>> Most likely there will be some more changes till the configuration is
>> implemented. So I created a branch called ticket-3351 for review here:
>> https://git.rtems.org/christianm/rtems-libbsd.git/log/?h=ticket-3351
>>
>> I added a fix for the freebsd-to-rtems.py to that branch.
>>
> 
> Hello Chris,
> 
> may I ask you about some points:
> 
> 1. I've put some thoughts into the configuration file format. My first
> idea to use ini files for it had a big disadvantage: It's hard to
> include one ini file into another.

Yes, the rtemstoolkit's configuration.py supports includes and why I highlighted
this in https://devel.rtems.org/ticket/3351#comment:4. An example INI file using
includes is:

 https://git.rtems.org/rtems-tools/tree/tester/rtems/rtems-bsps.ini#n98

> So I have tried to use a python
> script as a configuration file instead. What do you think of that
> approach? My current implementation is mainly in this commit on the
> ticket-branch:
> https://git.rtems.org/christianm/rtems-libbsd.git/commit/?h=ticket-3351&id=0504219600d6a6cac8775a8e95ebe56f473a9991

I am OK with python but I personally do not think it is widely suitable for a
configuration file format as you have in the repo. I feel you are asking too
much of users having them learn python and then learn about the indenting
syntax, dictionaries, functions, and then importing to specialise, that is
tough. Also there are other better and more established design patterns for
using a language like python as a bare configuration format, for example sphinx
with:

 # If true, '()' will be appended to :func: etc. cross-reference text.
 #add_function_parentheses = True

but this can become complicated quickly.

I suspect INI will be simpler and cleaner. Please have a look at coping down
configuration.py from the rtemstoolkit.

Internally the dictionary looks fine.

> 
> Note that also the config can be read, this branch currently doesn't
> build (see 2.).
> 
> 2. I'm struggling to build two versions of libbsd in one waf call.
> Currently the two configurations have increased the number of files that
> have to be build by the factor two. Beneath the point that waf has
> problems building the same file twice, I'm not really happy with that
> because it increases the build time a lot. I would prefer some solution
> that only builds files twice that have different options. Would you
> agree with that or would you go for building each file for each config?

This is one way, another below which I think will be easier and better.

The build system handles builds for a range of arch/bsp pairs here:

https://git.rtems.org/chrisj/rtems_waf.git/tree/rtems.py#n182

I would play with the `bsp_configure` option to `configure()`:

 https://git.rtems.org/chrisj/rtems_waf.git/tree/rtems.py#n269

and see what happens if you play with:

  env = conf.env.derive()

and:

  conf.setenv(ab + '/ipv6', env)

setting the env back at the end to:

  conf.setenv(ab, env)

I have no idea if you stack env's.

Maybe you can create a new conf environment for +1 config variants setting the
derived env with the path to the INI file. If you only have one config to build
use the conf env passed to `bsp_configure()`.

If you look into this file and the waf examples etc you will see the
`build(bld)` does not have any loops.

> 
> 3. That depends a little on the answer to two:
> 
> 3a) If I build everything for every config, I could try to put the
> objects for every config into its own subdirectory. That seems to be
> possible somehow: We already have for example a
> "build/arm-rtems5-atsamv" directory. If I find out how that is created,

The other approach is the list of `builders` as you started to implement and
passing into `builder.build(bld, 'ipv4-only')` a configuration label, something
short and simple, that you prepend to `bld` `targets` making them unique. You
need to update the `uses` so a config build correctly track their build objects.
If you did this waf would figure out the build details.

The down side is needing to manage the library name because it is a single build
directory. I suppose the install can rename to libbsd and use config paths. I am
not sure which way is best, what do you think?

Reviewing the file:

https://git.rtems.org/christianm/rtems-libbsd.git/tree/wscript?h=ticket-3351&id=0504219600d6a6cac8775a8e95ebe56f473a9991#n69

I would generate a waf fatal error if not None. Or initialise builders to `[]`
and raise a fatal error if not 0.

https://git.rtems.org/christianm/rtems-libbsd.git/tree/wscript?h=ticket-3351&id=0504219600d6a6cac8775a8e95ebe56f473a9991#n72

Please do not `print()`, please use:

https://git.rtems.org/chrisj/rtems_waf.git/tree/rtems.py#n148

> I can just add a build variant suffix. Do you happen to know where that
> directory comes from? 

I suspect the env for the confi. I cannot remember.

> It seems that the name is created in the
> rtems_waf/rtems.py. But I'm not totally sure:
> 
> https://git.rtems.org/chrisj/rtems_waf.git/tree/rtems.py#n124
> 
> 3b) If I try to build only variants of files I somehow have to add a
> suffix (or prefix) to the generated object files. I haven't found out
> yet how I could do that. Do you know (without searching too much)
> whether waf has such an option? If you don't know it, I'll dig deeper
> into waf myself.

I think this is the target name. If a build target is unique waf needs to manage
the object file names. It has too. You can see this with a build of the same
source with different defines or cflags.

Chris