HAMMER update 06-Feb-2008
dillon at apollo.backplane.com
Wed Feb 6 17:44:04 PST 2008
:This sounds quite like LFS now. LFS however split the volume in smaller =
:blocks which could be "used", "empty" or "open", IIRC. Their background =
:cleaner then could push remaining data from used blocks to a currently=20
:open one, marking the block "empty" after that, allowing the FS to write =
:to the blocks again.
:They always had problems with the cleaner, I think.
It's the 'pushing data around' action that causes the most problems
with recovery and cleaning, or more specifically, recovery after you
crash while cleaning. It can get messy really fast, especially if you
are trying to manage bitmaps.
With the cluster idea I got stuck the moment a transaction spanned
more the one cluster. I suddenly had two recovery domains related
to one transaction, and I couldn't find a solution to it.
A filesystem-wide FIFO abstraction solves all three problems by
providing exactly one synchronization point (the circular fifo's indices)
for the entire filesystem.
I really do think that the performance issues with cleaning HAMMER
will be addressable through the use of a very-large-block blockmap,
or by implementing a set of named very-large blocks. I have some
other ideas as well, such as treating the data associated with records
as an out-of-band entity that is not made part of the FIFO itself.
One last thing that must be considered is real time mirroring. There's
something to be said for having a single logical FIFO for the entire
filesystem in a clustered environment. It makes non-queued mirroring
ridiculously easy to implement.
More information about the Kernel