git: kernel - Reduce/refactor nbuf and maxvnodes calculations.

[Matthew Dillon]

commit c9678a7ec9d5f155e1ac1c2f35e3932720940f37
Author: Matthew Dillon <dillon at apollo.backplane.com>
Date:   Tue May 21 20:12:34 2019 -0700

    kernel - Reduce/refactor nbuf and maxvnodes calculations.
    
    * The prime motivation for this commit is to target about 1/20
      (5%) of physical memory for use by the kernel.  These changes
      significantly reduce kernel memory usage on systems with less
      than 4GB of ram (and more specific for systems with less
      than 1TB of ram), and also emplace more reasonable caps on
      systems with 128GB+ of ram.
    
      These changes return 100-200MB of ram to userland on systems
      with 1GB of ram, and return around 6.5GB of ram on systems
      with 128G of ram.
    
    * The nbuf calculation and related code documentation was a bit
      crufty, still somewhat designed for an earlier era and was
      calculating about twice the stated 5% target.  For systems with
      128GB of ram or less the calculation was simply creating too many
      filesystem buffers, allowing as much as 10% of physical memory to
      be locked up by the buffer cache.
    
      Particularly on small systems, this 10% plus other kernel overheads
      left a lot less memory available for user programs than we would
      have liked.  This work gets us closer to the 5% target.
    
    * Change the base calculation from 1/10 of physical memory to 1/20
      of physical memory, cutting the number of buffers in half on
      most systems.  The code documentation stated 1/20 but was actually
      calculating 1/10.
    
    * On large memory systems > 100GB the number of buffers is now capped
      at around 400000 or so (allowing the buffer cache to use around
      6.5 GBytes).  This cap was previously based on a relatively
      disconnected parameter relating to available memory in early boot,
      and when triggered it actually miscalculating nbufs to be double
      the intended number.
    
      The new cap is based on a fixed maximum of 500MB worth of
      struct bufs, roughly similar to the original intention.  This
      change reduces the number of buffers reserved on system with
      more than around 100GB of ram from around 12GB worth of data
      down to 6.5GB.
    
    * With the BKVABIO work eliminating most SMP invltlbs on buffer
      recyclement, there is no real reason to need a huge buffer
      cache.  Just make sure its big enough on large-memory machines
      to fully cache the likely live datasets for things like bulk
      compiles and such.
    
    * For kern.maxvnodes (which can be changed at run-time if you
      desire), the base calcualtion on systems with less than 1GB
      of ram has been cut in half (~60K vnodes to ~30K vnodes).  It
      will ramp up more slowly until it roughly matches the prior
      calculation at 4GB of system memory.  On systems with enough
      memory, maxvnodes is now explicitly capped at 4M.
    
      There generally is no need to allow an excessive number of vnodes
      to be cached.
    
      For HAMMER1 you can set vfs.hammer.double_buffer=1 to cause it
      to cache data from the underlying device, allowing it to utilize
      all available free(ish) memory regardless of the maxvnodes setting.
    
      HAMMER2 caches disk blocks in the underlying device by default.
      The vnode-based vm_object caches decompressed data, so we want
      to have enough vnodes for nominal heavily parallel bulk operations
      to avoid unnecessary re-lookups of the vnode as well as avoid having
      to decompress the same thing over and over again.
    
      In both cases an excessively high kern.maxvnodes actually wastes
      memory on both HAMMER1 and HAMMER2... or at least makes the pageout
      daemon's job more difficult.
    
    * Remove vfs.maxmallocbufspace.  It is no longer connected to
      anything.

Summary of changes:
 sys/kern/vfs_bio.c                       | 13 +--------
 sys/kern/vfs_subr.c                      | 49 ++++++++++++++++++--------------
 sys/platform/pc64/x86_64/machdep.c       | 33 +++++++++++----------
 sys/platform/vkernel64/x86_64/autoconf.c | 39 +++++++++++++++----------
 4 files changed, 71 insertions(+), 63 deletions(-)

http://gitweb.dragonflybsd.org/dragonfly.git/commitdiff/c9678a7ec9d5f155e1ac1c2f35e3932720940f37


-- 
DragonFly BSD source repository