HAMMER filesystem update - design document

[Bill Hacker]

Matthew Dillon wrote:
:Awesome!
:
:Tells me: "ZFS, bend over, grab your ankles and kiss your an(atomy) 'Goodbye'"
:
: From the amount of work that has HAD to go into this, it also tells me you are:
:
:A) probably single, or soon will be and
    Alas.

:B) don't sleep much anyway!
:...
:Bill Hacker
    I get a good 8 hours of sleep.  As I get older I find myself unable to
    pull all-nighters any more without really screwing up the entire next
    day.
As *I* have gotten older 'day' has become virtualized anyway. Partly 'coz I 
spend half the year on Washington DC's timezone and half on Hong Kong's 
timezone and work nights anyway 'coz that's when the cable modems are at their 
best..

    --

    ZFS serves a different purpose and I think it is cool, but as time
    has progressed I find myself liking ZFS's design methodology less and
    less, and I am very glad I decided against trying to port it.  I do
    not think it is a good idea to put all one's marbles in a single copy
    of a filesystem, no matter how redundant its storage model is, and there
    isn't much point having that level of redundancy if the intent is
    to operate in a replicated environment.
    The problem ZFS has is that it is TOO redundant.  You just don't need
    that scale of redundancy if you intend to operate in a multi-master
    replicated environment because you not only have wholely independant
    (logical) copies of the filesystem, they can also all be live and online
    at the same time.
Dunno about Slowlaris, but from what is passing by on 7-CURRENT, despite Pawel's 
excellent work, it 'seems to me' that ZFS is more fragile at the OS & RAM 
resource level than even yesteryear's storage media was/is at the hardware level.

I can't put a lot of faith in a fs that knocks the whole box offline ev'ry now 
and then...

    HAMMER's approach to redundancy is logical replication of the entire
    filesystem.  That is, wholely independant copies operating on different
    machines in different locations.
Basically, I'm a believer in getting any layer between hardware and 
'logical-whatever' as simple and robust as can be.

ZFS seem to me to have put too much in the way of DB features into the fs.

HAMMER - at first glance - seems to have a simpler and more elegant approach to 
the 'basics'.

Just enough of the 'DB' in it (as all fs are in one way or another)
to provide the flexibility, scaleability, and resilience, not so much as to 
negate its advantages with fragile, complex structures.

Or excessive overhead and devil-take-the-other-daemons resource thirst.

    Ultimately HAMMER's mirroring features will be used to further our
    clustering goals.  The major goal of this project is transparent
    clustering and a major requirement for that is to have a multi-master
    replicated environment.  That is the role HAMMER will eventually fill.
    We wont have multi-master in 2.0, but there's a good chance we will
    have it by the end of next year.

						-Matt

Given the recent trend toward more affordable SAN hardware and links, it should 
find a niche at about the right time.

Perhaps even the 'killer app' (along with kernel virtualization) that DragonFly 
could use..

It has had me take it off the back-burner again, anyway... off to dld an new iso..

Best,

Bill