git: hammer2 - Implement error processing and free reserve enforcement

[Matthew Dillon]

commit 65cacacfb9005fe8dd541b0e830c2466c0ab8453
Author: Matthew Dillon <dillon at apollo.backplane.com>
Date:   Wed Sep 6 19:56:24 2017 -0700

    hammer2 - Implement error processing and free reserve enforcement
    
    * newfs_hammer2 calculates the correct amount of reserved space.  We
      have to reserve 4MB per 1GB, not 4MB per 2GB, due to a snafu.  This
      is still only 0.4% of the storage.
    
    * Flesh out HAMMER2_ERROR_* codes and make most hammer2 functions return
      a proper error code.
    
    * Add error handling to nearly all code that can dirty a chain, in
      particular to handle ENOSPC issues.  Any dirty buffers that cannot be
      flushed will incur a write error (which in DragonFly typically causes
      the buffer to be retries later).  Any dirty chain that cannot be
      flushed will remain in the topology and can be completed in a later
      flush if space has been freed up.
    
      We try to avoid allowing the filesystem to get into this situation in
      the first place, but if it does, it should be possible to flush these
      asynchronous modifying chains and buffers once space is freed up via
      bulkfree.
    
    * Relax class match requirements in the freemap allocator when the freemap
      gets close to full.  This will allow e.g. inodes to be allocated out of
      DATA bitmaps and vise versa, and so forth.  This fixes edge conditions
      where there is enough free space available but it has all been earmarked
      for the wrong data class.
    
    * Try to fix a bug in live_count tracking when destroying an indirect
      block chain or inode chain that has not yet been blockmapped due to
      a drop.  This situation only occurs when chains cannot be flushed due
      to I/O errors or disk full conditions, and are then later destroyed
      (e.g. such as when the governing file is removed).
    
      This should fix a live_count assertion that can occur under these
      circumstances.  See hammer2_chain_lastdrop().
    
    * Enforce the free reserve requirement for all modifying VOP calls.
      Root users can nominally fill the file system to 97.5%, non-root
      users to 95%.  At 90%, write()s will enforce bawrite() verses bdwrite()
      to try to avoid buffer cache flushes from actually running the
      filesystem out of space.
    
      This is needed because we do not actually know how much disk space is
      going to be needed at write() time.  Deduplication and compression
      occurs later, at buffer-flush time.
    
    * Do NOT flush the volume header when a vfs sync is unable to completely
      flush a device due to errors.  This ensures that the underlying media
      does not become corrupt.
    
    * Fix an issue where bref.check.freemap.bigmask was not being properly
      reset to -1 when bulkfree is able to free an element.  This bug
      prevented the allocator from recognizing that free space was available
      in that bitmap.
    
    * Modify bulkfree operation to use the live topology when flushing and
      snapshot operations fail due to errors, allowing bulkfree to run.
    
    * Nominal bulkfree operations now runs on the snapshot without a
      transaction (more testing is needed).  This theoretically should allow
      bulkfree to run concurrent with just about any operation including
      flushes.
    
    * Add a freespace tracking heuristic to reduce the overhead that modifying
      VOP calls incur in checking the free reserve requirement.
    
    * hammer2 show dumps additional info for freemap nodes.

Summary of changes:
 sbin/hammer2/cmd_debug.c           |   5 +
 sbin/newfs_hammer2/newfs_hammer2.c |   9 +-
 sys/vfs/hammer2/TODO               |   3 -
 sys/vfs/hammer2/hammer2.h          |  78 ++++++++++++---
 sys/vfs/hammer2/hammer2_admin.c    |  20 ++--
 sys/vfs/hammer2/hammer2_bulkfree.c |  35 +++++--
 sys/vfs/hammer2/hammer2_chain.c    | 186 ++++++++++++++++++++++++++---------
 sys/vfs/hammer2/hammer2_cluster.c  |  10 +-
 sys/vfs/hammer2/hammer2_flush.c    | 192 +++++++++++++++++++++++--------------
 sys/vfs/hammer2/hammer2_freemap.c  | 142 +++++++++++++++++++--------
 sys/vfs/hammer2/hammer2_inode.c    | 106 +++++++++++---------
 sys/vfs/hammer2/hammer2_iocom.c    |   1 -
 sys/vfs/hammer2/hammer2_ioctl.c    |  59 +++++++++---
 sys/vfs/hammer2/hammer2_strategy.c |  93 +++++++++---------
 sys/vfs/hammer2/hammer2_subr.c     |  59 ++++++++----
 sys/vfs/hammer2/hammer2_synchro.c  |  52 ++++++----
 sys/vfs/hammer2/hammer2_vfsops.c   |  84 +++++++++++++++-
 sys/vfs/hammer2/hammer2_vnops.c    |  90 +++++++++++++----
 sys/vfs/hammer2/hammer2_xops.c     | 172 +++++++++++++++++----------------
 19 files changed, 954 insertions(+), 442 deletions(-)

http://gitweb.dragonflybsd.org/dragonfly.git/commitdiff/65cacacfb9005fe8dd541b0e830c2466c0ab8453


-- 
DragonFly BSD source repository