nohistory'd directories on Hammer

[Matthew Dillon]

:
:I have nohistory'd one of my directories and stufed a lot of data into
:it. ls- lao shows the flag correctly. But when I delete files from it,
:there is no space that gets freed, I have tried a prune and still no
:more free space. Isn't space supposed to be available right after
:delete something that is nohistory'd?
:
:-- 
:Gergo Szakal MD <bastyaelvtars at gmail.com>
:University Of Szeged, HU

    It should be able to recover some of the space, it won't be able to
    recover all of it.

    In particular, if you are deleting large files, most of their space
    should be recovered.  If you copied a lot of small files, though,
    only some of the space can be recovered without reblocking.

    Here's an example with small files:

test12# mkdir /usr/obj/testdir
test12# chflags nohistory /usr/obj/testdir
test12# df /usr/obj
Filesystem 1K-blocks       Used      Avail Capacity  Mounted on
TEST       1695047680 136159232 1558888448     8%    /usr/obj

test12# cpdup / /usr/obj/testdir/root
test12# sync
test12# sync
test12# sync
test12# df /usr/obj
test12# Filesystem 1K-blocks       Used      Avail Capacity  Mounted on
TEST       1695047680 136314880 1558732800     8%    /usr/obj

test12# rm -rf /usr/obj/testdir
test12# df /usr/obj
test12# sync
test12# sync
test12# sync
Filesystem 1K-blocks       Used      Avail Capacity  Mounted on
TEST       1695047680 136216576 1558831104     8%    /usr/obj

136159232
136314880	+155648 (155MB)
136216576	-98304  (98MB)	     <<< only some of the space was recovered

    Here's an example with a large file:

test12# mkdir testdir
test12# chflags nohistory testdir
test12# cd testdir
test12# df /usr/obj
ilesystem 1K-blocks       Used      Avail Capacity  Mounted on
TEST       1695047680 136216576 1558831104     8%    /usr/obj

test12# dd if=/dev/zero of=test bs=32k count=10240
10240+0 records in
10240+0 records out
335544320 bytes transferred in 2.652017 secs (126524202 bytes/sec)
test12# sync
test12# sync
test12# df /usr/obj
Filesystem 1K-blocks       Used      Avail Capacity  Mounted on
TEST       1695047680 136544256 1558503424     8%    /usr/obj

test12# rm test
test12# sync; sleep 1
test12# sync; sleep 1
test12# sync; sleep 1
test12# df /usr/obj
Filesystem 1K-blocks       Used      Avail Capacity  Mounted on
TEST       1695047680 136216576 1558831104     8%    /usr/obj
test12# 

The three Used numbers:

136216576
136544256	+327680K
136216576	-327680K	<<< most/all of the space was recovered

    I won't be able to make that work any better for this release.  HAMMER
    really needs to be reblocked incrementally.  Eventually I think it
    will be possible to recover the space more efficiently at 'rm' time
    but it isn't a priority right now.

    Generally speaking these are the characteristics we want.  If someone
    is messing around with really big files and wants to recover the space,
    nohistory will do it.  If someone is messing around with lots of tiny
    files, they won't get instant gratification.

    Remember, HAMMER is designed for big storage media, where nobody
    fills up the media in an hour, or a day, or a week, or even a month
    unless they are doing it on purpose with a 'dd' :-).  The idea is
    for the filesystem to have enough runway such that incremental pruning
    and reblocking by an overnight cron job keeps it in shape over the
    long haul.

    If it came down to it I could implement a more fine-grained storage
    allocation layer, like I had originally (and which had horrible 
    performance).  I'd rather not.  I think long-term reblocking and pruning
    is the way to go.

					-Matt
					Matthew Dillon 
					<dillon at backplane.com>