HAMMER Update - 28 June 2008

[Matthew Dillon]

    Here's another update on the state of HAMMER for the release!


			    Large File Testing

    I've added a large-file test to my testing rigimen.  In addition to
    running blogbench, buildworld -j 8, and fsx all at once I am now also
    doing an unbounded "dd" to a test file until the disk fills up.  The
    file grows to approximately the size of the filesystem, 1.9TB. 
    I then 'rm' the file in full history mode, and later run hammer prune.

    This sequence located a number of issues with the buffer cache, in
    particular HAMMER was trying to delete all 28 million records in a
    single flush cycle, which deadlocked the buffer cache.

    In addition to that, retries due to deadlocks caused the delete scan
    to start from offset 0, instead of picking up where it left off,
    causing the delete operation to take several hours instead of 
    being sub-1-minute.

    I am still testing but I believe I have fixed most of the issues.

    FUN ASIDE: With my little 3-ware raid stripe HAMMER can maintain
    full platter speed on writes on the dd when nothing else is running,
    and still does a good job when all the other tests are running at
    the same time.  This is about 109MB/sec.  It takes 5 hours for
    the dd to actually fill up the 1.9TB test filesystem.  And then
    I get to 'rm' it and do a 'hammer prune' pass.

				    Mirroring

    I have three mirroring-related items left to do for the release.

    The first two are safety-oriented features, basically to disallow
    user modification of the mirror-write target, and to allow a named
    label to be associated with each mirroring target and require that
    it matches the mirroring source.   The safeties are needed to prevent
    users from shooting themselves in the foot too easily as doing a
    mirror-write to the wrong FS has a very good chance of destroying it.

    The last item is to finish some B-Tree updates that have to occur
    on the mirror master to make incremental and streaming mirroring work
    properly.

				Buffer cache work

    HAMMER uses mixed block sizes and under heavy load testing this revealed
    a couple of issues with the kernel's buffer cache which should now be
    fixed.  I may be doing a little more work in this area, though.