waf default number of jobs

[Joel Sherrill]

Hi

What was the rationale behind the choice of the default for the number of
jobs for the waf build system?

I know there isn't a good default because there is a huge difference
between being able to completely use a computer or just do build more or
less in the background.

Historically, the number needed to be 1.5-2x the number of cores. This was
usually recommended because you could overlap compilation with reading
source and writing objects. With modern faster disk subsystems, it seems
the ratio needs to be on the order of 3x

On the 8-core Xeon I have been testing on, evn -j24 with waf only sustains
~80% CPU usage. The default of 8 leaves a much lower utilization. Based on
my builds so far, I think -j24 is giving me build times around 1/3 of -j8.
For 100s of BSP configurations, this is substantial and the machine is not
swamped.

GNU make has a -l for a load average option. Something like this might be
worth considering. This is from the man page:

====================
   When the system is heavily loaded, you will probably want to run
fewer jobs than when it is lightly loaded.  You can use the `-l' option
to tell `make' to limit the number of jobs to run at once, based on the
load average.  The `-l' or `--max-load' option is followed by a
floating-point number.  For example,

     -l 2.5

will not let `make' start more than one job if the load average is
above 2.5.  The `-l' option with no following number removes the load
limit, if one was given with a previous `-l' option.

  ====================

Anyway, for faster builds, the current default -j is too low since it
doesn't allow for overlappying IO and computation.

--joel
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.rtems.org/pipermail/devel/attachments/20201001/2e85e1dd/attachment-0001.html>