HAMMER Update 13-June-2008 - HEADS UP: MEDIA CHANGED AGAIN

[Matthew Dillon]

    Commit 55 to the filesystem and utilities changes the media format
    again.  A new blockmap has been added as a stop-gap while I 
    research storage model issues more.  Please note:

	* You must recompile the filesystem AND its utilities (hammer and
	  newfs_hammer).

	* You must re-newfs_hammer your test filesystems.

    This commit also fixes a panic and *might* also have fixed the reported
    machine lockup.  It does not fix the memory exhaustion issue yet.

    There is a known issue (that's been around for a while) where the
    HAMMER flusher threads can become cpu bound, creating large noticeable
    pauses in machine operation.  That is related to B-Tree overheads and
    has not yet been dealt with.

    I have made progress on the blogbench performance drop issue when run
    on the same directory over and over again.  Blogbench does its best
    to really mess up B-Trees :-).  I have narrowed the issue down to
    directory/namespace lookups in the face of massive directory rewriting.
    blogbench does that in spades by creating .tmp files and renaming them
    over the originals.

    Unfortunately, the only way to really fix it is to introduce true
    locality of reference on disk on a file-by-file and directory-by-
    directory basis, kinda like how UFS does with cylinder groups.
    In HAMMER's case this means adding a field to the blockmap's
    layer2 structure (which represents an 8MB chunk of disk space) to
    allow incremental appends anywhere on the disk.  This is probably what
    I will be working on for the rest of this month.

						-Matt
						Matthew Dillon 
						<dillon at backplane.com>