git: kernel - Refactor lockmgr()

[Matthew Dillon]

commit 3b6a19b26fb9c0e7918cf2d1c7c0ade7d5a4f3de
Author: Matthew Dillon <dillon at apollo.backplane.com>
Date:   Mon Oct 23 18:39:16 2017 -0700

    kernel - Refactor lockmgr()
    
    * Seriously refactor lockmgr() so we can use atomic_fetchadd_*() for
      shared locks and reduce unnecessary atomic ops and atomic op loops.
    
      The main win here is being able to use atomic_fetchadd_*() when
      acquiring and releasing shared locks.  A simple fstat() loop (which
      utilizes a LK_SHARED lockmgr lock on the vnode) improves from 191ns
      to around 110ns per loop with 32 concurrent threads (on a 16-core/
      32-thread xeon).
    
    * To accomplish this, the 32-bit lk_count field becomes 64-bits.  The
      shared count is separated into the high 32-bits, allowing it to be
      manipulated for both blocking shared requests and the shared lock
      count field.  The low count bits are used for exclusive locks.
      Control bits are adjusted to manage lockmgr features.
    
      LKC_SHARED	Indicates shared lock count is active, else excl lock
    		count.  Can predispose the lock when the related count
    		is 0 (does not have to be cleared, for example).
    
      LKC_UPREQ	Queued upgrade request.  Automatically granted by
    		releasing entity (UPREQ -> ~SHARED|1).
    
      LKC_EXREQ	Queued exclusive request (only when lock held shared).
    		Automatically granted by releasing entity
    		(EXREQ -> ~SHARED|1).
    
      LKC_EXREQ2	Aggregated exclusive request.  When EXREQ cannot be
    		obtained due to the lock being held exclusively or
    		EXREQ already being queued, EXREQ2 is flagged for
    		wakeup/retries.
    
      LKC_CANCEL	Cancel API support
    
      LKC_SMASK	Shared lock count mask (LKC_SCOUNT increments).
    
      LKC_XMASK	Exclusive lock count mask (+1 increments)
    
      The 'no lock' condition occurs when LKC_XMASK is 0 and LKC_SMASK is
      0, regardless of the state of LKC_SHARED.
    
    * Lockmgr still supports exclusive priority over shared locks.  The
      semantics have slightly changed.  The priority mechanism only applies
      to the EXREQ holder.  Once an exclusive lock is obtained, any blocking
      shared or exclusive locks will have equal priority until the exclusive
      lock is released.  Once released, shared locks can squeeze in, but
      then the next pending exclusive lock will assert its priority over
      any new shared locks when it wakes up and loops.
    
      This isn't quite what I wanted, but it seems to work quite well.  I
      had to make a trade-off in the EXREQ lock-grant mechanism to improve
      performance.
    
    * In addition, we use atomic_fcmpset_long() instead of
      atomic_cmpset_long() to reduce cache line flip flopping at least
      a little.
    
    * Remove lockcount() and lockcountnb(), which tried to count lock refs.
      Replace with lockinuse(), which simply tells the caller whether the
      lock is referenced or not.
    
    * Expand some of the copyright notices (years and authors) for major
      rewrites.  Really there are a lot more and I have to pay more attention
      to adjustments.

Summary of changes:
 sys/dev/disk/dm/dm_table.c           |    2 +-
 sys/dev/drm/include/linux/mutex.h    |    2 +-
 sys/dev/drm/include/linux/spinlock.h |    2 +-
 sys/kern/kern_lock.c                 | 1271 ++++++++++++++++++++++------------
 sys/kern/kern_shutdown.c             |    4 +-
 sys/kern/vfs_bio.c                   |   28 +-
 sys/kern/vfs_lock.c                  |    2 +-
 sys/kern/vfs_subr.c                  |    4 +-
 sys/sys/buf2.h                       |   12 +-
 sys/sys/lock.h                       |   47 +-
 sys/vfs/nfs/nfs_subs.c               |    2 +-
 sys/vfs/nfs/nfs_vnops.c              |    2 +-
 sys/vfs/tmpfs/tmpfs_subr.c           |    2 +-
 sys/vfs/ufs/ffs_softdep.c            |   24 +-
 14 files changed, 880 insertions(+), 524 deletions(-)

http://gitweb.dragonflybsd.org/dragonfly.git/commitdiff/3b6a19b26fb9c0e7918cf2d1c7c0ade7d5a4f3de


-- 
DragonFly BSD source repository