<!DOCTYPE HTML PUBLIC "-//W3C//DTD HTML 4.01 Transitional//EN" "http://www.w3.org/TR/html4/loose.dtd"><html>
<head>
<meta name="Generator" content="Zarafa WebAccess v7.1.4-41394">
<meta http-equiv="Content-Type" content="text/html; charset=utf-8">
<title>RE: A few questions about HAMMER master/slave PFS</title>
<style type="text/css">
body
{
font-family: Arial, Verdana, Sans-Serif ! important;
font-size: 12px;
padding: 5px 5px 5px 5px;
margin: 0px;
border-style: none;
background-color: #ffffff;
}
p, ul, li
{
margin-top: 0px;
margin-bottom: 0px;
}
</style>
</head>
<body>
<p>Hello Michael,</p><p> </p><p>Thank you kindly for your thorough reply, it helped me to make sense of what was going on with the snapshots, indeed, once I reduced the daily retention time to 7d for snapshots, I was able to get rid of those old ones with hammer cleanup !</p><p> </p><p>I also learned about being able to have different retention time for the daily PFS slave snapshot which is neat.</p><p> </p><p>Apologies for the borked output regarding hammer info. As it stands now after running prune-everything on all PFS's, I still have a noticable size difference, I'll try to investigate some more and maybe leave some time to hammer mirror-stream to even things out.</p><p> </p><p>Cheers,</p><p> </p><p>Laurent</p><blockquote style="border-left: 2px solid #325FBA; padding-left: 5px;margin-left:5px;">-----Original message-----<br /><strong>From:</strong> Michael Neumann <mneumann@ntecs.de><br /><strong>Sent:</strong> Mon 08-27-2018 10:19 pm<br /><strong>Subject:</strong> Re: A few questions about HAMMER master/slave PFS<br /><strong>To:</strong> Laurent Vivier <laurent@lamzi.com>; <br /><strong>CC:</strong> users@dragonflybsd.org; <br />On Mon, Aug 27, 2018 at 06:08:10PM +0200, Laurent Vivier wrote:<br />> Hello DFlyers,<br />> <br />> I am running DragonFly 5.2.2 as an NFS server with 2x 2TB LUKS-backed HDD's with HAMMER1 v7 as FS in a PFS master/slave mirror-stream setup and it's been working great so far :)<br />> <br />> The setup looks like this :<br />> <br />> Disk1 -> LUKS -> HAMMER1_2TB -> PFS# 0 (root) <br />> Disk2 -> LUKS -> HAMMER_SLAVE -> PFS# 0 (root) + PFS 1 (slave to HAMMER1_2TB / PFS# 0)<br />> <br />> Now that I am using the system for a little while, I have a few questions regarding its behavior :<br />> <br />> 1) I realized that the HAMMER slave PFS has several snapshots (not created by me) that seems seemingly impossible to remove e.g<br /><br />HAMMER1 uses fine-grained snapshots, which means that it basically<br />automatically creates an "unnamed" snapshot whenever it flushes<br />something to disk (roughly every 30 seconds). Usually, you don't want to<br />keep all these fine-grained snapshots and instead keep one snapshot per<br />day (or one per week...). This is what "hammer cleanup" does. You can<br />configure it's history retention policy by running "hammer viconfig".<br />From the man page of "hammer cleanup":<br /><br /> snapshots 1d 60d # 0d 0d for PFS /tmp, /var/tmp, /usr/obj<br /> prune 1d 5m<br /> rebalance 1d 5m<br /> #dedup 1d 5m # not enabled by default<br /> reblock 1d 5m<br /> recopy 30d 10m<br /><br />This means, when you run "hammer cleanup", it takes one snapshot every<br />day, and retains the last 60 daily snapshots. hammer cleanup performs<br />other tasks, for instance pruning (1d = every day, for 5 minutes).<br />Pruning deletes all the intermediate fine-grained snapshots between the<br />"named" daily snapshots. It also rebalances the B-tree, dedups, reblocks<br />and recopies. These are all operations to optimize performance. Dedup is<br />to save space. <br /><br />If you want to delete snapshots, just change "snapshots 1d 60d" to, for<br />instance, "snapshots 1d 7d" and run "hammer cleanup". If you want to<br />delete all historical data, you might use "hammer prune-everything", but<br />be careful and read the man page!!!<br /><br />One nice feature of HAMMER1 is that the master PFS and slave PFS can<br />have different history retention policies in place.<br /><br />> <br />> Hikaeme# hammer info /HAMMER_SLAVE<br />> Volume identification<br />> ?????? Label???????????????????????????? hammer1_secure_slave<br />> ?????? No. Volumes???????????????? 1<br />> ?????? HAMMER Volumes?????????? /dev/mapper/knox2<br />> ?????? Root Volume???????????????? /dev/mapper/knox2<br />> ?????? FSID?????????????????????????????? 0198767f-7139-11e8-9608-6d626d258b95<br />> ?????? HAMMER Version?????????? 7<br />> Big-block information<br />> ?????? Total?????????????????? 238335<br />> ?????? Used???????????????????? 192009 (80.56%)<br />> ?????? Reserved???????????????????? 32 (0.01%)<br />> ?????? Free?????????????????????? 46294 (19.42%)<br />> Space information<br />> ?????? No. Inodes?????????? 35668<br />> ?????? Total size???????????? 1.8T (1999298887680 bytes)<br />> ?????? Used???????????????????????? 1.5T (80.56%)<br />> ?????? Reserved???????????????? 256M (0.01%)<br />> ?????? Free???????????????????????? 362G (19.42%)<br />> PFS information<br />> ?????? ?? PFS#?? Mode?????? Snaps<br />> ?????? ???????? 0?? MASTER?????????? 0 (root PFS)<br />> ?????? ???????? 1?? SLAVE???????????? 3<br />> Hikaeme# hammer snapls /HAMMER_SLAVE/pfs/hanma<br />> Snapshots on /HAMMER_SLAVE/pfs/hanma?????? PFS#1<br />> Transaction ID?????? ?????? Timestamp?????? ?????? Note<br />> 0x00000001034045c0?????? 2018-07-04 18:19:42 CEST?????? -<br />> 0x00000001034406c0?????? 2018-07-09 19:28:04 CEST?????? -<br />> 0x000000010383bc30?????? 2018-08-12 10:51:07 CEST?????? -<br />> Hikaeme# hammer snaprm 0x00000001034045c0<br />> hammer: hammer snaprm 0x00000001034045c0: Operation not supported<br /><br />Have you tried hammer snaprm /HAMMER_SLAVE/pfs/hanma@@0x00000001034045c0<br /><br />> My question here is should I worry about it/is that an intended behavior ? <br />> <br />> 2) When executing hammer info and looking at the used space between master and slave PFS, I have quite a big difference (22GB, even after running hammer cleanup)<br />> <br />> Hikaeme# hammer info<br />> Volume identification<br />> ?????? Label???????????????????????????? HAMMER1_2TB<br />> ?????? No. Volumes???????????????? 1<br />> ?????? HAMMER Volumes?????????? /dev/mapper/knox<br />> ?????? Root Volume???????????????? /dev/mapper/knox<br />> ?????? FSID?????????????????????????????? 81e9d5eb-6be7-11e8-802d-6d626d258b95<br />> ?????? HAMMER Version?????????? 7<br />> Big-block information<br />> ?????? Total?????????????????? 238335<br />> ?????? Used???????????????????? 194636 (81.66%)<br />> ?????? Reserved???????????????????? 32 (0.01%)<br />> ?????? Free?????????????????????? 43667 (18.32%)<br />> Space information<br />> ?????? No. Inodes?????????? 35665<br />> ?????? Total size???????????? 1.8T (1999298887680 bytes)<br />> ?????? Used???????????????????????? 1.5T (81.66%)<br />> ?????? Reserved???????????????? 256M (0.01%)<br />> ?????? Free???????????????????????? 341G (18.32%)<br />> PFS information<br />> ?????? ?? PFS#?? Mode?????? Snaps<br />> ?????? ???????? 0?? MASTER?????????? 0 (root PFS)<br />> <br />> Volume identification<br />> ?????? Label???????????????????????????? hammer1_secure_slave<br />> ?????? No. Volumes???????????????? 1<br />> ?????? HAMMER Volumes?????????? /dev/mapper/knox2<br />> ?????? Root Volume???????????????? /dev/mapper/knox2<br />> ?????? FSID?????????????????????????????? 0198767f-7139-11e8-9608-6d626d258b95<br />> ?????? HAMMER Version?????????? 7<br />> Big-block information<br />> ?????? Total?????????????????? 238335<br />> ?????? Used???????????????????? 191903 (80.52%)<br />> ?????? Reserved???????????????????? 32 (0.01%)<br />> ?????? Free?????????????????????? 46400 (19.47%)<br />> Space information<br />> ?????? No. Inodes?????????? 35668<br />> ?????? Total size???????????? 1.8T (1999298887680 bytes)<br />> ?????? Used???????????????????????? 1.5T (80.52%)<br />> ?????? Reserved???????????????? 256M (0.01%)<br />> ?????? Free???????????????????????? 363G (19.47%)<br />> PFS information<br />> ?????? ?? PFS#?? Mode?????? Snaps<br />> ?????? ???????? 0?? MASTER?????????? 0 (root PFS)<br />> ?????? ???????? 1?? SLAVE???????????? 3<br />> <br />> Is that something I should be worried about too ? As far as I can tell the replication of new files from master and slave works great, I can see the new files on the slave PFS fairly quickly.<br /><br />It's hard to read your above output, as there are too many "???". Note<br />that HAMMER1 mirroring does not operate on the block level, but on the<br />logical level. So both master and slave PFSs are separate file systems<br />and their underlying disk blocks are differently organized.<br /><br />If you have different history retention policies on slave and master,<br />this might lead to different file system sizes. This is not an issue.<br />Just configure it using "hammer viconfig" and run "hammer cleanup".<br /><br />Best regards,<br /><br /> Michael<br /><br /></blockquote>
</body>
</html>