[PATCH v2 1/2] waf: Support building from libbsd.py directly from waf.

Wed Apr 4 05:51:43 UTC 2018

Am 04.04.2018 um 07:28 schrieb Chris Johns:
> On 04/04/2018 00:43, Christian Mauderer wrote:
>> Am 27.03.2018 um 09:56 schrieb Christian Mauderer:
>>> Am 27.03.2018 um 09:00 schrieb Christian Mauderer:
>>>> Am 26.03.2018 um 14:59 schrieb Christian Mauderer:
>>>>> Hello Chris,
>>>>>
>>>>> I took the liberty to rebase the patches to the latest master. See the
>>>>> two following v3 mails. I'll take a look at the configuration based on
>>>>> these patches.
>>>>>
>>>>> I think that you wanted to make i386 tests. After that it might would be
>>>>> a good idea to commit them. What do you think?
>>>>>
>>>>> Best regards
>>>>>
>>>>> Christian
>>>>>
>>>>
>>>> Hello Chris,
>>>>
>>>> just noted: The freebsd-to-rtems.py (with -R) doesn't work anymore. I'll
>>>> fix that.
>>>>
>>>> Maybe it would be a good idea to develop that (and the config options)
>>>> on a branch (on a user specific repo on git.rtems.org) and post it for
>>>> review? At least it's a bigger change.
>>>>
>>>> Best regards
>>>>
>>>> Christian
>>>>
>>>
>>> Most likely there will be some more changes till the configuration is
>>> implemented. So I created a branch called ticket-3351 for review here:
>>> https://git.rtems.org/christianm/rtems-libbsd.git/log/?h=ticket-3351
>>>
>>> I added a fix for the freebsd-to-rtems.py to that branch.
>>>
>>
>> Hello Chris,
>>
>> may I ask you about some points:
>>
>> 1. I've put some thoughts into the configuration file format. My first
>> idea to use ini files for it had a big disadvantage: It's hard to
>> include one ini file into another.
> 
> Yes, the rtemstoolkit's configuration.py supports includes and why I highlighted
> this in https://devel.rtems.org/ticket/3351#comment:4. An example INI file using
> includes is:
> 
>  https://git.rtems.org/rtems-tools/tree/tester/rtems/rtems-bsps.ini#n98
> 
>> So I have tried to use a python
>> script as a configuration file instead. What do you think of that
>> approach? My current implementation is mainly in this commit on the
>> ticket-branch:
>> https://git.rtems.org/christianm/rtems-libbsd.git/commit/?h=ticket-3351&id=0504219600d6a6cac8775a8e95ebe56f473a9991
> 
> I am OK with python but I personally do not think it is widely suitable for a
> configuration file format as you have in the repo. I feel you are asking too
> much of users having them learn python and then learn about the indenting
> syntax, dictionaries, functions, and then importing to specialise, that is
> tough. Also there are other better and more established design patterns for
> using a language like python as a bare configuration format, for example sphinx
> with:
> 
>  # If true, '()' will be appended to :func: etc. cross-reference text.
>  #add_function_parentheses = True
> 
> but this can become complicated quickly.
> 
> I suspect INI will be simpler and cleaner. Please have a look at coping down
> configuration.py from the rtemstoolkit.
> 
> Internally the dictionary looks fine.
> 

Thanks for the hint. I missed that feature with the include in the ini
file. I agree that with this additional point, ini is the better format.

>>
>> Note that also the config can be read, this branch currently doesn't
>> build (see 2.).
>>
>> 2. I'm struggling to build two versions of libbsd in one waf call.
>> Currently the two configurations have increased the number of files that
>> have to be build by the factor two. Beneath the point that waf has
>> problems building the same file twice, I'm not really happy with that
>> because it increases the build time a lot. I would prefer some solution
>> that only builds files twice that have different options. Would you
>> agree with that or would you go for building each file for each config?
> 
> This is one way, another below which I think will be easier and better.
> 
> The build system handles builds for a range of arch/bsp pairs here:
> 
> https://git.rtems.org/chrisj/rtems_waf.git/tree/rtems.py#n182
> 
> I would play with the `bsp_configure` option to `configure()`:
> 
>  https://git.rtems.org/chrisj/rtems_waf.git/tree/rtems.py#n269
> 
> and see what happens if you play with:
> 
>   env = conf.env.derive()
> 
> and:
> 
>   conf.setenv(ab + '/ipv6', env)
> 
> setting the env back at the end to:
> 
>   conf.setenv(ab, env)
> 
> I have no idea if you stack env's.
> 
> Maybe you can create a new conf environment for +1 config variants setting the
> derived env with the path to the INI file. If you only have one config to build
> use the conf env passed to `bsp_configure()`.
> 
> If you look into this file and the waf examples etc you will see the
> `build(bld)` does not have any loops.
> 

Thanks for the pointers. I'll take a look at them. But from what I
currently read, that might will lead to the variant where every file is
build for every config.

To be honest: I never tried to build multiple BSPs. I'll try that with
BSPs for the same architecture and take a look at how the files are
generated. Most likely it would be the best to use the same approach.

>>
>> 3. That depends a little on the answer to two:
>>
>> 3a) If I build everything for every config, I could try to put the
>> objects for every config into its own subdirectory. That seems to be
>> possible somehow: We already have for example a
>> "build/arm-rtems5-atsamv" directory. If I find out how that is created,
> 
> The other approach is the list of `builders` as you started to implement and
> passing into `builder.build(bld, 'ipv4-only')` a configuration label, something
> short and simple, that you prepend to `bld` `targets` making them unique. You
> need to update the `uses` so a config build correctly track their build objects.
> If you did this waf would figure out the build details.
> 
> The down side is needing to manage the library name because it is a single build
> directory. I suppose the install can rename to libbsd and use config paths. I am
> not sure which way is best, what do you think?

Like I said above, I would prefere to exclude some modules from building
multiple times. For example OpenSSL has around 700 files but it
shouldn't depend on IPv4 or IPv6 or on WiFi support.

Disadvantage of that is that with this approach every module would have
to know which options are relevant for it and which are not. So I
wouldn't change the defines directly but instead create a list of
features that can be enabled or disabled. Every module then would get
this list and would have to decide for itself which defines are
necessary in that case.

> 
> Reviewing the file:

Please don't put too much time into review of these first patches. They
are still work in progress. I mostly pushed them so I can get some
feedback regarding the config. Of course I'll clean up everything and
create only very few clean ones at the end.

> 
> https://git.rtems.org/christianm/rtems-libbsd.git/tree/wscript?h=ticket-3351&id=0504219600d6a6cac8775a8e95ebe56f473a9991#n69
> 
> I would generate a waf fatal error if not None. Or initialise builders to `[]`
> and raise a fatal error if not 0.

OK.

> 
> https://git.rtems.org/christianm/rtems-libbsd.git/tree/wscript?h=ticket-3351&id=0504219600d6a6cac8775a8e95ebe56f473a9991#n72
> 
> Please do not `print()`, please use:
> 
> https://git.rtems.org/chrisj/rtems_waf.git/tree/rtems.py#n148

I wasn't aware of that. I'll change it.
> 
>> I can just add a build variant suffix. Do you happen to know where that
>> directory comes from? 
> 
> I suspect the env for the confi. I cannot remember.

OK. Thanks.

> 
>> It seems that the name is created in the
>> rtems_waf/rtems.py. But I'm not totally sure:
>>
>> https://git.rtems.org/chrisj/rtems_waf.git/tree/rtems.py#n124
>>
>> 3b) If I try to build only variants of files I somehow have to add a
>> suffix (or prefix) to the generated object files. I haven't found out
>> yet how I could do that. Do you know (without searching too much)
>> whether waf has such an option? If you don't know it, I'll dig deeper
>> into waf myself.
> 
> I think this is the target name. If a build target is unique waf needs to manage
> the object file names. It has too. You can see this with a build of the same
> source with different defines or cflags.

Odd. I had problems even with different target names. I'll try to dig
deeper into that.

> 
> Chris
> 

-- 
--------------------------------------------
embedded brains GmbH
Herr Christian Mauderer
Dornierstr. 4
D-82178 Puchheim
Germany
email: christian.mauderer at embedded-brains.de
Phone: +49-89-18 94 741 - 18
Fax:   +49-89-18 94 741 - 08
PGP: Public key available on request.

Diese Nachricht ist keine geschäftliche Mitteilung im Sinne des EHUG.