Filesystem Journaling Update II - July 2005

[Matthew Dillon]

    Softlinks and hardlinks are now supported and I cleaned up a few
    more bugs.  I can now run a buildworld loop and the mirror generated
    by the journal stays in synch (diff -r reports no differences if I
    idle the buildworld, wait a second or two for the journal to catch up,
    and run it).  I'm still not sure about modifications made via mmap()
    but everything else seems to work.

    In my buildworld timing tests a journal with a 1 megabyte memory buffer
    mirroring to the same physical disk is able to run the buildworld in
    2250 seconds.  Without the journal it takes 2100 seconds.  This is about
    a 7% performance penalty, which is quite reasonable considering what
    you get for the trouble.

    We are very close to being able to use this for near real-time backups.
    With a forward-running journal the idea would be to keep a base mirror
    of the filesystem and use any remaining space to hold the journal files.
    When the backup filesystem fills up, you run the earliest journaling
    file into the mirror and remove it.  Thus one can use 100% of the 
    available backup media.  The disadvantage is that to restore the
    filesystem the base mirror must be copied back and then all the
    journaling files run to bring it up to date.

    Once we get reverse-journaling (UNDO records) working, we can do it
    the other way.. the mirror would contain the completely up to date
    filesystem and the journal files would be used to 'revert' it to an
    earlier state if necessary.  This is safer because you can run the
    filesystem closer to 100% full (when it fills up you just remove the
    oldest journaling file), and you can restore a backup from the live
    mirror without having to run through any journaling files.

    To make the backup regimen work properly the originating machine needs
    a buffering stage capable of using swap for robustness and needs the
    two-way protocol to allow reconnects after connection failures at any
    point in the pipe line.  I am going to work on a userland buffering
    stage and the (userland and kernel) two-way protocol next.

					-Matt
					Matthew Dillon 
					<dillon at xxxxxxxxxxxxx>