pkgsrc bulk build profile

[Bill Hacker]

Justin C. Sherrill wrote:
Something interesting I noticed while doing an initial bulk pkgsrc build
on df.v12.su:
Build start: 2009-02-13 04:26
Build end:   2009-02-20 18:46
This machine has 4 2.4G cpus, but I think this is the same time I'd get on
single-CPU pkgbox under good circumstances.  My handwaving guess is that
either the pbulk process combined with the BGL is I/O bound, not CPU
bound.
I recall Joerg had mentioned using VMWare to put multiple hosts on a
single machine, and then using pbulk's internal distributed build method
to spread work between those hosts; that may be possible on this system
if/when Qemu works on DragonFly?
Other ideas welcome; I'd like to use bulk pkgsrc builds as a sort of
performance profile, plus anything that makes them faster is good.
I don't think you need to wait for Qemu on DragonFly.

Seems to me that vkernel would do a better job, and it is already there.

OTOH - I'm not at all au fait with how well DFLY is doing SMP these days...

And yes - storage I/O bound  - or 'contentious' - is almost a certainty.

Put an microphone on the HDD's and listen...

;-)

I'd look to separate the work over multiple HDD and controllers more 
than, or at least as much as, separate CPU.

For dependency reasons, each store would need 100% of the sources, libs, 
and toolchain, even if only 1/4 of the 'finals' were being asked to be 
built on that portion.

Bill