Parallel compression not using all available CPU
PeerCorps Trust Fund
ipc at peercorpstrust.org
Sat Dec 10 13:14:26 PST 2016
On both systems HAMMER was used. One small correction concerning the 2c/2t machine, both compression programs did effectively utilize that CPU which had an idle % of 0.0. It is the bigger machine, 16c/32t where the CPU isn't effectively maxed out. I'll continue to try and investigate why and report back if I find anything.
On 12/10/2016 10:26 PM, Justin Sherrill wrote:
> On the two DragonFly systems, was it Hammer or UFS? I would be
> surprised if that made a difference, but it might?
> On Sat, Dec 10, 2016 at 6:19 AM, PeerCorps Trust Fund
> <ipc at peercorpstrust.org> wrote:
>> I've observed that parallel compression tools such as pixz and lbzip2 do not
>> make use of all of the available CPU under Dragonfly. On other OSes, it
>> When testing on a 50 gb file, using top I've observed that CPU idle
>> percentages consistently hover around the 90% range for pixz and ~70% for
>> lbzip2. These values under FreeBSD and Linux are typically ~0.0% idle until
>> compression is complete. Correspondingly, compression takes significantly
>> longer under Dragonfly, so the CPU is really being under utilized in this
>> case as opposed to erroneous reporting by top.
>> This was tested on two systems, one 16c/32t and a 2c/2t system on a recent
>> master DragonFly v188.8.131.523.g8d7da-DEVELOPMENT #2: Wed Dec 7 11:44:04 EET
>> Has anyone else possibly observed this?
More information about the Users