git: kernel - Update AMD topology detection, scheduler NUMA work (TR2)
dillon at crater.dragonflybsd.org
Wed Aug 22 18:46:23 PDT 2018
Author: Matthew Dillon <dillon at apollo.backplane.com>
Date: Wed Aug 22 18:34:32 2018 -0700
kernel - Update AMD topology detection, scheduler NUMA work (TR2)
* Update AMD topology detection to use the correct cpuid. It
now properly detects the Threadripper 2990WX as having four nodes
with 8 cores and 2 threads per core, per node. It previously detected
the chip as one node with 32 cores and 2 threads per core.
* Report the basic detected topology without requiring bootverbose.
* Record information about how much memory is attached to each node.
We previously just assumed that it was symmetric. This will be
used by the scheduler.
* Fix instability in the scheduler when running on a large number
of cores. Flag 0x08 (on by default) is needed to actively
schedule overloaded threads onto other cores, but this operation
was being executed on all cores simultaneously which throws the
uload/ucount metrics into an unstable state, causing threads to
bounce around longer the necessary.
Fix by round-robining the operation based on something similar to
sched_ticks % cpuid.
This significantly improves heavy multi-tasking performance on systems
with many cores.
* Add memory-on-node weighting to the scheduler. This detects asymetric
NUMA configurations for situations where not all DIMM slots have been
populated, and for CPUs which are naturally assymetric such as the
2990WX which only has memory directly connected to two of its four
This change will preferentially schedule threads onto nodes with
greater amounts of attached memory under light loads, and dig into
the less desirable cpu nodes as the load increases.
Summary of changes:
sys/kern/subr_cpu_topology.c | 43 +++++--
sys/kern/usched_dfly.c | 210 +++++++++++++++++++++++++---------
sys/platform/pc64/x86_64/mp_machdep.c | 112 ++++++++++++++++--
sys/sys/cpu_topology.h | 32 ++++--
sys/sys/msgbuf.h | 2 +-
sys/vm/vm_page.c | 43 +++++++
6 files changed, 351 insertions(+), 91 deletions(-)
DragonFly BSD source repository
More information about the Commits