HAMMER & HAMMER2: volumes of different sizes, dedupe, mirroring, and stability

[Michael Neumann]

On Sat, Jan 08, 2022 at 01:39:01AM +0000, Xianwen Chen (陈贤文) wrote:
> 
> 
> Hi!

Hey!

> 
> I got quite a few questions regarding HAMMER & HAMMER 2 fs.
> 
> I haven't been able to find out whether HAMMER & HAMMER2 fs could add
> volumes of different sizes. Let's assume that there are 4 hard drives: 200
> GB, 300 GB, 400 GB, and 500 GB. Could I put all these 4 hard drives under
> HAMMER or HAMMER 2 fs and have a total of 1400 GB storage? Or will there be
> just 800 GB like ZFS?

A HAMMER1 filesystem can natively span multiple volumes (see `hammer
volume-add / volume-del` or just add multiple devices at `newfs_hammer`
time). To do the same with HAMMER2, use LVM. LVM allows to construct
*one* big logical volume from multiple devices onto which you can then
put a HAMMER2 filesystem.  But be aware of the risks related to the
choosen RAID level.

Keep in mind that ZFS is a volume manager *and* a file system in one,
while HAMMER1 or HAMMER2 are "just" file systems. 

> 
> Do I understand it correctly that HAMMER fs only supports offline dedupe?

Yes, HAMMER1 has "background" deduplication (`hammer dedup`). You can
still access your HAMMER1 filesystem while deduplication is in progress.

> Does HAMMER 2 fs support online dedupe?

Yes. HAMMER2 only supports online dedupe, not "background/offine"
deduplication. That is, when you reboot, your deduplication caches will
be empty and copying duplicate files to it won't recognize them as
duplicates unless you read the duplicate files again.

> It seems that mirroring is the way to have data redundancy in HAMMER and
> HAMMER 2. Continuing with the example of 4 hard drives (200 GB, 300 GB, 400
> GB, and 500 GB), will I be able to have 3 mirrors, in addition to the
> ``original'' one where mirrors copy data from?

HAMMER1 is pretty cool in that it allows to stream any changes to one or
more remote mirrors (`hammer mirror-stream / hammer mirror-copy`) in
"real-time". Of course you can also mirror to local disks. Keep in mind
that HAMMER1 is less recommend these days.

HAMMER2 has support for SLAVE PFSes (mirroring a master PFS across a
cluster), though I am not sure how stable this feature is. My
understanding is that any of the cluster features of HAMMER2 are highly
experimental (please correct me if I am wrong).

But you can achive the same by creating snapshots and `cpdup` or `rsync`
them to your mirror(s) every now and then using a cronjob.  Or you use a
proper backup solution like borg etc. and do incremental backups.

> Is there any risk of data loss when I add and remove volumes?

Yes, there is always risk in loosing data if you don't have backups. One
wrong command and all your data is gone!

I'd be very careful when adding / removing volumes to a HAMMER1
filesystem as it was never tested and used widely to my knowledge.
Always back your data up if your data is important to you. Same applies
to LVM of course.

That beeing said, HAMMER1 and HAMMER2 turned out to be rock solid. But
when your disk goes very wrong, it cannot do wonders!

Hope this helps!

Regards,

  Michael