git: DragonFly_RELEASE_6_2 hammer2 - Fix excess chain structure allocations during bulkfree
dillon at crater.dragonflybsd.org
Fri Jan 7 16:51:06 PST 2022
Author: Matthew Dillon <dillon at apollo.backplane.com>
Date: Fri Jan 7 16:41:35 2022 -0800
hammer2 - Fix excess chain structure allocations during bulkfree
* On H2 filesystems with a very large number of inodes, such as
those used for backups or which contain many snapshots (e.g.
tens of millions of inodes), the bulkfree process may attempt
to track more hammer2_chain structures than the kmalloc subsystem
* This issue might also have contributed to reported lockups during
bulkfree (if the bulkfree wound up eating too much non-pagable kernel
memory), though the more typical outcome is a kernel panic when
the kmalloc limit is exceeded for the pool.
* During bulkfree, H2 must control recursion depth during the scan.
It does this by deferring deep nodes in the topology by recording
a pointer to a referenced hammer2_chain structure on a list.
Once the recursion backs out, H2 then processes these saved chains.
Topologies greater than 32 nodes deep can wind up being deferred
more than once to cover the entire depth.
It is possible for an excessive number of chains to accumulate on the
deferral list during the scan, each structure burning up kernel memory.
* Emplace a limit on the number of chains which can be deferred,
vfs.hammer2.limit_saved_chains, with a reasonable default. When
the limit is reached, the bulkfree simply records a chain higher-up
in the recursion instead of going deeper, thus limiting the number
of chains that will be deferred.
This has the effect is capping memory use during the bulkfree scan.
The cost is slightly less scan efficiency, but it should mostly be in
Summary of changes:
sys/vfs/hammer2/hammer2.h | 1 +
sys/vfs/hammer2/hammer2_bulkfree.c | 28 +++++++++++++++++++++++++++-
sys/vfs/hammer2/hammer2_vfsops.c | 5 +++++
3 files changed, 33 insertions(+), 1 deletion(-)
DragonFly BSD source repository
More information about the Commits