waf default number of jobs
chrisj at rtems.org
Sat Oct 3 06:37:26 UTC 2020
On 2/10/20 5:53 am, Joel Sherrill wrote:
> On Thu, Oct 1, 2020 at 2:39 PM Sebastian Huber
> <sebastian.huber at embedded-brains.de <mailto:sebastian.huber at embedded-brains.de>>
> On 01/10/2020 20:09, Joel Sherrill wrote:
> > What was the rationale behind the choice of the default for the number
> > of jobs for the waf build system?
> I don't know. It is the default behaviour of waf. I noticed also
> scalability problems but had no time to track them down. It could be
> also a limitation of the Python multiprocessing capabilities. I am also
> not a waf expert, I may have produced some bottlenecks.
> I don't think you produced any bottlenecks. Here are a couple of fragments
> from my build. I have attached the ini file so you can double check that it
> does match. Please let me know if the ini matches the configure command
> or not. If it doesn't, then I have a lot of bogus builds. :(
> ../rtems/configure --target=arm-rtems6 --enable-rtemsbsp=gumstix
> --prefix=/home/joel/rtems-cron-6/tools/6/bsp-install --disable-networking
> --enable-posix --disable-smp --disable-multiprocessing --enable-rtems-debug
> --enable-profiling --enable-tests --enable-cxx --enable-maintainer-mode
> + make -j24
> real 1m53.064s
> user 7m14.675s
> sys 1m45.913s
> + ./waf -j 24
> real 1m1.404s
> user 0m9.601s
> sys 0m1.547s
> Notice how the build time for waf is 50 seconds faster but that doesn't
> match the user and system time. Perhaps the Python jobs model doesn't
> keep them as children the same way so time doesn't get to count them.
> I honestly can't explain that.
Which host OS?
I am running a waf build of 5 BSPs with tests on FreeBSD 12.1 and I ran `systat
-vmstat 1` at the same time and it shows idle is around 0%, 25% is in sys and
75% is in user. The machine has with 8 cores. The powerpc/mvme5500 took 58
seconds. Python is 3.7.6.
More information about the devel