git: kernel - Break up scheduler and loadavg callout
Matthew Dillon
dillon at crater.dragonflybsd.org
Sat Aug 12 12:34:30 PDT 2017
commit 586c43085fc900273732f99de6c9ef43f73ded76
Author: Matthew Dillon <dillon at apollo.backplane.com>
Date: Sat Aug 12 12:24:16 2017 -0700
kernel - Break up scheduler and loadavg callout
* Change the scheduler and loadavg callouts from cpu 0 to all cpus, and
adjust the allproc_scan() and alllwp_scan() to segment the hash table
when asked.
Every cpu is now tasked with handling the nominal scheduler recalc and
nominal load calculation for a portion of the process list. The portion
is unrelated to which cpu(s) the processes are actually scheduled on,
it is strictly a way to spread the work around, split up by hash range.
* Significantly reduces cpu 0 stalls when a large number of user processes
or threads are present (that is, in the tens of thousands or more). In
the test below, before this change, cpu 0 was straining under 40%+
interupt load (from the callout). After this change the load is spread
across all cpus, approximately 1.5% per cpu.
* Tested with 400,000 running user processes on a 32-thread dual-socket
xeon (yes, these numbers are real):
12:27PM up 8 mins, 3 users, load avg: 395143.28, 270541.13, 132638.33
12:33PM up 14 mins, 3 users, load avg: 399496.57, 361405.54, 225669.14
* NOTE: There are still a number of other non-segmented allproc scans in
the system, particularly related to paging and swapping.
* NOTE: Further spreading-out of the work may be needed, by using a more
frequent callout and smaller hash index range for each.
Summary of changes:
sys/kern/imgact_elf.c | 2 +-
sys/kern/init_main.c | 2 +-
sys/kern/kern_descrip.c | 4 +--
sys/kern/kern_ktrace.c | 4 +--
sys/kern/kern_proc.c | 30 ++++++++++++++++---
sys/kern/kern_resource.c | 8 ++---
sys/kern/kern_sig.c | 2 +-
sys/kern/kern_synch.c | 70 +++++++++++++++++++++++++++++++------------
sys/kern/vfs_syscalls.c | 4 +--
sys/sys/globaldata.h | 8 ++++-
sys/sys/proc.h | 6 ++--
sys/vfs/procfs/procfs_vnops.c | 2 +-
sys/vm/vm_glue.c | 7 +++--
sys/vm/vm_meter.c | 2 +-
sys/vm/vm_object.c | 2 +-
sys/vm/vm_pageout.c | 4 +--
16 files changed, 111 insertions(+), 46 deletions(-)
http://gitweb.dragonflybsd.org/dragonfly.git/commitdiff/586c43085fc900273732f99de6c9ef43f73ded76
--
DragonFly BSD source repository
More information about the Commits
mailing list