HAMMER2 questions

[Matthew Dillon]

On Mon, Jan 27, 2020 at 11:53 PM Chuck Musser <cmusser at sonic.net> wrote:

> I've run Dfly for several years now, but never paused to understand
> HAMMER2 (or for that matter, it's predecessor) very deeply. I've used
> undo(1) a few times, but haven't dug into concepts like clustering or
> even what exactly PFSes are used for. I have a feeling I'm missing out
> on interesting things. So I'm finally getting around to asking a
> scattershot bunch of questions. I'm using my simple one-disk setup as a
> starting point. Its df -h output is below:
>
> Filesystem                     Size   Used  Avail Capacity  Mounted on
> serno/5QE57G6D.s1d             193G  1054M   192G     1%    /
> devfs                         1024B  1024B     0B   100%    /dev
> /dev/serno/5QE57G6D.s1a       1022M   187M   753M    20%    /boot
> /dev/serno/5QE57G6D.s1e at DATA  18.5G   123M  18.4G     1%    /build
> /build/usr.obj                18.5G   123M  18.4G     1%    /usr/obj
> /build/var.crash              18.5G   123M  18.4G     1%    /var/crash
> /build/var.cache              18.5G   123M  18.4G     1%    /var/cache
> /build/var.spool              18.5G   123M  18.4G     1%    /var/spool
> /build/var.log                18.5G   123M  18.4G     1%    /var/log
> /build/var.tmp                18.5G   123M  18.4G     1%    /var/tmp
> tmpfs                          935M     0B   935M     0%    /tmp
> procfs                        4096B  4096B     0B   100%    /proc
> tmpfs                          935M     0B   935M     0%    /var/run/shm
>
> 1. What are are the three entries that begin with "serno" and why is the
> root slice not prefixed with /dev/ like the other two? What does
> the nomenclature mean, specifically "5QE57G6D", "s1a" and "@DATA", for
> instance).
>

All attached storage devices have a software-readable serial number and can
be addressed by that serial number instead of by the raw device.  This
allows the raw device numbers to probe in random order or for the drive to
be moved without having to make adjustments to /etc/fstab.


> 2. Are the subdirectories under /build each their own PFS? What do those
> do and why are they there? As an administrator, when would I want to
> create my own PFSes? I saw someone's example of creating a set of jails
> on a dedicated PFS, but I don't understand the implications of doing
> that.
>

The installer used to create a PFS for each major subdirectory but no
longer does.  Instead it just separates everything into three partitions.
The 'a' partition is a small UFS filesystem containing /boot.  We use UFS
for /boot because its tiny and deletions immediately free up media space.
The 'd' partition contains the root mount and anything that one would
normally want to backup.  For example, /home is on 'd'.   The 'e' partition
contains the /build mount and has all the stuff one would normally NOT want
to backup, such as /var/crash, /usr/obj, /var/log, and so forth.  I put
/var/spool in there too but its debatable.  Mostly I put /var/spool in
there too because it might see significant activity and I wanted the root
mount to be as unassuming as possible in crash situations.


> 3. What are null mounts? It seems like that's how you mount PFSes, at
> least judging by the output of the mount(1) command.
>

A NULL mount mounts one directory that already exists in the filesystem
onto another part of the filesystem.  Both places reference the same
directory.  Also note that these NULL mounts are not PFSs.  The installer
no longer creates a separate PFS for each directory, it just creates three
partitions ('a' for boot, 'd' for root, and 'e' for /build) and then uses
NULL mounts to arrange the subdirectories that we want to physically be on
/build.

4. What are clusters, in HAMMER2? Can I use them to create a big
> filesystem from storage on separate systems?
>

Clusters are not implemented yet, but that's the idea.


> 5. I think I recall hearing that the remote mirroring feature in HAMMER
> is not yet supported in HAMMER2. Is that status still accurate?
>

This is correct.  My recommendation for now is to just use a regular rsync
or cpdup, and then use hammer2's snapshot feature to create a frozen
snapshot of the backup.  Ultimately mirroring will be built into the
clustering.  Not yet though.


> 6. Although I have a feeling that RAID configuration is a lower level
> construct than HAMMER, what are the options? I did see that the
> natacontrol(8) is able to manage RAIDs, but If I happened to get my
> hands on NVMe storage, those might call for a different utility? There
> is a nvmectl, but it seems like it might be for a different purpose.
>
> Thanks,
>
> Chuck
>

Honestly I don't recommend using NATA or natacontrol in any manner.  The
ATA code is extremely crufty, old, and just not reliable.  Use AHCI mode
(which attaches as /dev/da0, /dev/da1, etc...).  The AHCI driver is fully
supported and reliable.  If the BIOS does not support an AHCI-based
softraid then your best raid option with DragonFly is some sort of hardware
raid solution that is underneath the filesystem.

It is possible to use DM (man dm) and the device mapper to do some things.
Crypted volumes are typical.  There is a mirroring feature in there too but
I don't know how good it is because nobody uses it.  There are no RAID
features ported.  We also have 'vinum' for RAID support but it is also very
old and basically unsupported, though I have received reports that it still
works.

Still, my recommendation is that for anything that fits on one drive no
mirroring or RAID should be used.  Make discrete backups to another drive
on a regular schedule instead.  RAIDs are not actually any more reliable
than non-RAID on small systems in terms of machine uptime.  For larger
many-drive arrays HAMMER2 just isn't the right solution (not yet) and I
would recommend running ZFS on FreeBSD instead.  But for any single-drive
solution (even a large one), HAMMER2 gives premium performance and has a
number of extremely useful features built-in such as automatic
de-duplication (when copying a large file or tree), and compression.  I use
HAMMER2 on a bunch of 4TB HDDs and SSDs myself and it works flawlessly.

-Matt
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.dragonflybsd.org/pipermail/users/attachments/20200128/70265041/attachment.htm>