git: kernel - Implement segment pmap optimizations for x86-64
dillon at crater.dragonflybsd.org
Wed Sep 12 18:34:20 PDT 2012
Author: Matthew Dillon <dillon at apollo.backplane.com>
Date: Wed Sep 12 18:25:19 2012 -0700
kernel - Implement segment pmap optimizations for x86-64
* Implement 2MB segment optimizations for x86-64. Any shared read-only
or read-write VM object mapped into memory, including physical objects
(so both sysv_shm and mmap), which is a multiple of the segment size
and segment-aligned can be optimized.
* Enable with sysctl machdep.pmap_mmu_optimize=1
Default is off for now. This is an experimental feature.
* It works as follows: A VM object which is large enough will, when VM
faults are generated, store a truncated pmap (PD, PT, and PTEs) in the
VM object itself.
VM faults whos vm_map_entry's can be optimized will cause the PTE, PT,
and also the PD (for now) to be stored in a pmap embedded in the VM_OBJECT,
instead of in the process pmap.
The process pmap then creates PT entry in the PD page table that points
to the PT page table page stored in the VM_OBJECT's pmap.
* This removes nearly all page table overhead from fork()'d processes or
even unrelated process which massively share data via mmap() or sysv_shm.
We still recommend using sysctl kern.ipc.shm_use_phys=1 (which is now
the default), which also removes the PV entries associated with the
shared pmap. However, with this optimization PV entries are no longer
a big issue since they will not be replicated in each process, only in
the common pmap stored in the VM_OBJECT.
* Features of this optimization:
* Number of PV entries is reduced to approximately the number of live
pages and no longer multiplied by the number of processes separately
mapping the shared memory.
* One process faulting in a page naturally makes the PTE available to
all other processes mapping the same shared memory. The other processes
do not have to fault that same page in.
* Page tables survive process exit and restart.
* Once page tables are populated and cached, any new process that maps
the shared memory will take far fewer faults because each fault will
bring in an ENTIRE page table. Postgres w/ 64-clients, VM fault rate
was observed to drop from 1M faults/sec to less than 500 at startup,
and during the run the fault rates dropped from a steady decline into
the hundreds of thousands into an instant decline to virtually zero
* We no longer have to depend on sysv_shm to optimize the MMU.
* CPU caches will do a better job caching page tables since most of
them are now themselves shared. Even when we invltlb, more of the
page tables will be in the L1, L2, and L3 caches.
Summary of changes:
sys/kern/kern_slaballoc.c | 3 +-
sys/platform/pc32/acpica5/acpi_wakeup.c | 6 +-
sys/platform/pc32/i386/pmap.c | 38 +--
sys/platform/pc32/include/pmap.h | 3 +
sys/platform/pc64/include/pmap.h | 30 ++-
sys/platform/pc64/include/pmap_inval.h | 1 +
sys/platform/pc64/x86_64/pmap.c | 646 ++++++++++++++++++++++++++-----
sys/platform/pc64/x86_64/pmap_inval.c | 6 +
sys/platform/vkernel/include/pmap.h | 3 +
sys/platform/vkernel/platform/pmap.c | 37 +-
sys/platform/vkernel64/include/pmap.h | 3 +
sys/platform/vkernel64/platform/pmap.c | 37 +-
sys/vm/pmap.h | 11 +-
sys/vm/vm_fault.c | 18 +-
sys/vm/vm_object.c | 6 +
sys/vm/vm_object.h | 6 +
16 files changed, 671 insertions(+), 183 deletions(-)
DragonFly BSD source repository
More information about the Commits