New Build System Status

[Chris Johns]

On 2/12/19 5:42 pm, Sebastian Huber wrote:
> On 01/12/2019 23:53, Chris Johns wrote:
>> On 27/11/19 11:26 pm, Sebastian Huber wrote:
>>> On 27/11/2019 13:17, Hesham Almatary wrote:
>>>> On Wed, 27 Nov 2019 at 11:59, Sebastian Huber
>>>> <sebastian.huber at embedded-brains.de> wrote:
>>>>> On 27/11/2019 12:49, Hesham Almatary wrote:
>>>>>> Hi Sebastian,
>>>>>>
>>>>>> Thanks for that great effort. I'd aim to use this build system for my
>>>>>> RISC-V development.
>>>>>>
>>>>>> I followed the user manual trying to build RISC-V targets and RTEMS
>>>>>> (aaf7f8b84) and here are a few comments:
>>>>>>
>>>>>> * .waf bsp_defaults doesn't give an error when mistyping the BSP name,
>>>>>> but just outputs an empty .ini file.
>>>>>
>>>>> If there is no matching BSP, then you get nothing. I think this is the
>>>>> right thing to do.
>>
>> If the user does not enter the correct BSP name we should provide an relevant
>> error message on stderr and return a non-zero error code. I believe we should
>> not silently move past an error without a clear indication there is a problem.
> 
> We are talking here about the "./waf bsp_defaults" command. I think it is
> perfectly fine to output nothing if there is nothing to output. This is the
> normal UNIX style. What happens if you grep for something which doesn't exits?
> Anyway, I changed the command like this:
> 
> $ ./waf bsp_defaults --rtems-specs=spec/build/cpukit
> # The build specification contains no BSPs
> 
> $ ./waf bsp_defaults --rtems-bsps=foobar,perfection
> # No BSP matches with the specified patterns: 'foobar', 'perfection'

Unix commands often cannot change because of the things users have built up
around them. It does not mean what is provided is the right way. What we are
discussing is going to be around a long time and we need to examine the detail
so we can avoid breaking users with changes.

In the case of grep it's default if no file names are provided is to read
string. I do not think it relates.

As I said before it makes scripting and tooling around the command problematic
and complicated.

>>>>> Doing a
>>>>>
>>>>> ./waf bsp_defaults --rtems-bsps=riscv/rv64imac_medany  > bsps.ini
>>>>>
>>>>> is not a recommended use case. You should only set the necessary minimum
>>>>> of options.
>>
>> If this documented?
> 
> Yes, but apparently not good enough. I will try to fix this.

Great.

>>>> Not sure, I only wanted to build a specific BSP variant without having
>>>> to build all variants. Is that not recommended?
>>>
>>> This is fine, but I would not dump all options into the config.ini, maybe just:
>>>
>>> [riscv/rv64imac_medany]
>>> COMPILER = clang
>>> BUILD_TESTS = True
>>
>> How do you get a list of valid options and values to know what to set and not
>> set?
> 
> You get the options with default values via "./waf bsp_defaults". You should
> only set options for which you don't like the default value. For example, some
> BSPs allow you to set the initial value of some registers, e.g.
> 
> # initial BUCSR value
> QORIQ_INITIAL_BUCSR = 0x01400201
> 
> Lets suppose a chip errata appears and it says you have to set the undocumented
> bit 31 to activate an errata workaround. You change the value in the build
> specification to 0x81400201 in the RTEMS sources. Everyone who did a
> 
> ./waf bsp_defaults --rtems-bsps=xxx  > config.ini
> 
> will not profit from this bug fix.

I understand this however my next question was ...

>> If you do not providing a value do you get your current version's default? Does
>> this make updating defaults in rtems.git more complicated?

Often a BSP is based on the developers set up and a specific project. This is a
fair thing to do and I fine with that happening. A number of "defaults" are
actually "settings" for a project. In the case of the example you give above if
that register already has a local setting for another reason that RTEMS has not
adopted you have a conflict that needs to be exposed and resolved. We will never
achieve a unified set of defaults.

The issue I am attempting to address is present no matter what solution you
provide. I am wondering if in time we need to add some sort of diff and conflict
resolution tool so a user can see how they stand against the defaults in any
release we make? We cannot solve the problem however we can provide a standard
way our users can.

>>>>>> * ./waf complains about not finding gcc if I don't give it the
>>>>>> --prefix, even though it's in my PATH
>>>>>
>>>>> Yes, this is currently the intentional behaviour. Should this be changed to
>>>>>
>>>>> conf.find_program(XYZ, path_list=tools +
>>>>> environ.get('PATH','').split(os.pathsep))
>>>>>
>>>>> ?
>>>> I think that makes sense (also for default prefix?), but it would
>>>> still be useful to be able to override that with --prefix to use
>>>> different installed toolchains if needed.
>>>
>>> Chris, what is your opinion on this?
>>
>> A rigorous approach would ignore a user's path because paths are something we
>> are not good at maintaining and this can lead to subtle issues in a project that
>> are hard to figure out.
>>
>> The rtems_waf behavior currently is:
>>
>> 1. --rtems-tools is the first path searched
>> 2. --prefix if no --rtems-tools
>> 3. $PATH if no --rtems-tools and no --prefix
>>
>> I am fine with rtems.git's waf doing the same thing.
> 
> Ok, I changed it like this:
> 
> def get_path_list(conf):
>     path_list = []
>     tools = conf.options.rtems_tools
>     if tools is not None:
>         for t in tools.split(","):
>             path_list.extend([t + "/bin", t])
>     path_list.append(conf.env.PREFIX + "/bin")
>     path_list.extend(os.environ.get("PATH", "").split(os.pathsep))
>     return path_list

Great and thanks. I used the `--rtems-tools` option with the Windows build. It
is simple when the prefix and tools are split and it means rtems_waf and
rtems.git have the same option. I think this is really important.

Chris