HAMMER update 06-Feb-2008
dillon at apollo.backplane.com
Wed Feb 6 17:26:44 PST 2008
:How about "Generational Garbarge Collection"? Assuming that there are
:some files that will never be deleted this could give slighly better
:Keep a "copy count" (a copy occurs if the cleaner has to copy data from
:the left end to the right end of the FIFO). If that increases over, say
:3, copy it into the old generation FIFO.
:One problem of course is how to dimension each generation, and how many
:I think that's basically how LFS works.
The idea I have is turn the physical layout of the disk into a
logical linear layout. That is, instead of actually laying the FIFO
down linearly on the physical disk, instead create a very large backing
blockmap that allows large (say 16MB) chunks of the disk to be shifted
around simply by adjusting the block map.
Now the FIFO copying can simply shift the underlying block map to move
a 16MB block from the beginning of the FIFO to the end of the FIFO
if no garbage collection is otherwise required.
One can also have the idea of 'named very large blocks', where each one
represents a separate recovery zone. This is like a blockmap allocation
mechanism that allows you to move the block from the beginning of the
FIFO to the end without having to rewire the references to elements
within that block.
I think the initial implementation has to just work with the disk. I
am making sure my design is condusive to these sorts of optimizations
but I am not going to try to implement a blockmap immediately.
What is becoming very clear to me now that I'm actually coding this stuff,
is that my original clustering idea was simply too highly integrated into
the general filesystem operations, requiring complex code all over the
place. The new idea is far, FAR less complex and yet as we see there are
plenty of layer-based solutions that can solve the copy case issues
without requiring the filesystem core to have much knowledge about the
<dillon at backplane.com>
More information about the Kernel