pkgsrc bulk build profile
wbh at conducive.org
Sun Feb 22 21:29:24 PST 2009
Justin C. Sherrill wrote:
Something interesting I noticed while doing an initial bulk pkgsrc build
Build start: 2009-02-13 04:26
Build end: 2009-02-20 18:46
This machine has 4 2.4G cpus, but I think this is the same time I'd get on
single-CPU pkgbox under good circumstances. My handwaving guess is that
either the pbulk process combined with the BGL is I/O bound, not CPU
I recall Joerg had mentioned using VMWare to put multiple hosts on a
single machine, and then using pbulk's internal distributed build method
to spread work between those hosts; that may be possible on this system
if/when Qemu works on DragonFly?
Other ideas welcome; I'd like to use bulk pkgsrc builds as a sort of
performance profile, plus anything that makes them faster is good.
I don't think you need to wait for Qemu on DragonFly.
Seems to me that vkernel would do a better job, and it is already there.
OTOH - I'm not at all au fait with how well DFLY is doing SMP these days...
And yes - storage I/O bound - or 'contentious' - is almost a certainty.
Put an microphone on the HDD's and listen...
I'd look to separate the work over multiple HDD and controllers more
than, or at least as much as, separate CPU.
For dependency reasons, each store would need 100% of the sources, libs,
and toolchain, even if only 1/4 of the 'finals' were being asked to be
built on that portion.
More information about the Kernel