git: hammer2 - Implement error processing and free reserve enforcement
dillon at crater.dragonflybsd.org
Wed Sep 6 20:13:34 PDT 2017
Author: Matthew Dillon <dillon at apollo.backplane.com>
Date: Wed Sep 6 19:56:24 2017 -0700
hammer2 - Implement error processing and free reserve enforcement
* newfs_hammer2 calculates the correct amount of reserved space. We
have to reserve 4MB per 1GB, not 4MB per 2GB, due to a snafu. This
is still only 0.4% of the storage.
* Flesh out HAMMER2_ERROR_* codes and make most hammer2 functions return
a proper error code.
* Add error handling to nearly all code that can dirty a chain, in
particular to handle ENOSPC issues. Any dirty buffers that cannot be
flushed will incur a write error (which in DragonFly typically causes
the buffer to be retries later). Any dirty chain that cannot be
flushed will remain in the topology and can be completed in a later
flush if space has been freed up.
We try to avoid allowing the filesystem to get into this situation in
the first place, but if it does, it should be possible to flush these
asynchronous modifying chains and buffers once space is freed up via
* Relax class match requirements in the freemap allocator when the freemap
gets close to full. This will allow e.g. inodes to be allocated out of
DATA bitmaps and vise versa, and so forth. This fixes edge conditions
where there is enough free space available but it has all been earmarked
for the wrong data class.
* Try to fix a bug in live_count tracking when destroying an indirect
block chain or inode chain that has not yet been blockmapped due to
a drop. This situation only occurs when chains cannot be flushed due
to I/O errors or disk full conditions, and are then later destroyed
(e.g. such as when the governing file is removed).
This should fix a live_count assertion that can occur under these
circumstances. See hammer2_chain_lastdrop().
* Enforce the free reserve requirement for all modifying VOP calls.
Root users can nominally fill the file system to 97.5%, non-root
users to 95%. At 90%, write()s will enforce bawrite() verses bdwrite()
to try to avoid buffer cache flushes from actually running the
filesystem out of space.
This is needed because we do not actually know how much disk space is
going to be needed at write() time. Deduplication and compression
occurs later, at buffer-flush time.
* Do NOT flush the volume header when a vfs sync is unable to completely
flush a device due to errors. This ensures that the underlying media
does not become corrupt.
* Fix an issue where bref.check.freemap.bigmask was not being properly
reset to -1 when bulkfree is able to free an element. This bug
prevented the allocator from recognizing that free space was available
in that bitmap.
* Modify bulkfree operation to use the live topology when flushing and
snapshot operations fail due to errors, allowing bulkfree to run.
* Nominal bulkfree operations now runs on the snapshot without a
transaction (more testing is needed). This theoretically should allow
bulkfree to run concurrent with just about any operation including
* Add a freespace tracking heuristic to reduce the overhead that modifying
VOP calls incur in checking the free reserve requirement.
* hammer2 show dumps additional info for freemap nodes.
Summary of changes:
sbin/hammer2/cmd_debug.c | 5 +
sbin/newfs_hammer2/newfs_hammer2.c | 9 +-
sys/vfs/hammer2/TODO | 3 -
sys/vfs/hammer2/hammer2.h | 78 ++++++++++++---
sys/vfs/hammer2/hammer2_admin.c | 20 ++--
sys/vfs/hammer2/hammer2_bulkfree.c | 35 +++++--
sys/vfs/hammer2/hammer2_chain.c | 186 ++++++++++++++++++++++++++---------
sys/vfs/hammer2/hammer2_cluster.c | 10 +-
sys/vfs/hammer2/hammer2_flush.c | 192 +++++++++++++++++++++++--------------
sys/vfs/hammer2/hammer2_freemap.c | 142 +++++++++++++++++++--------
sys/vfs/hammer2/hammer2_inode.c | 106 +++++++++++---------
sys/vfs/hammer2/hammer2_iocom.c | 1 -
sys/vfs/hammer2/hammer2_ioctl.c | 59 +++++++++---
sys/vfs/hammer2/hammer2_strategy.c | 93 +++++++++---------
sys/vfs/hammer2/hammer2_subr.c | 59 ++++++++----
sys/vfs/hammer2/hammer2_synchro.c | 52 ++++++----
sys/vfs/hammer2/hammer2_vfsops.c | 84 +++++++++++++++-
sys/vfs/hammer2/hammer2_vnops.c | 90 +++++++++++++----
sys/vfs/hammer2/hammer2_xops.c | 172 +++++++++++++++++----------------
19 files changed, 954 insertions(+), 442 deletions(-)
DragonFly BSD source repository
More information about the Commits