PFS mirror problems

[Tomohiro Kusumi]

I tried to reproduce it, but doesn't reproduce with below.
A pfs in / gets copied to a pfs in /HAMMER.

# cat -n ./pfsd2.sh
     1  #!/usr/local/bin/bash
     2
     3  HAMMER_DEVICE=/dev/da1
     4
     5  cd
     6  umount /HAMMER
     7  newfs_hammer -L TEST ${HAMMER_DEVICE} > /dev/null
     8  mount_hammer ${HAMMER_DEVICE} /HAMMER
     9  cd /HAMMER
    10
    11  hammer pfs-master a
    12  hammer pfs-downgrade a
    13  hammer pfs-update a `hammer pfs-status /pfs/tmp | grep shared-uuid`
    14
    15  hammer mirror-copy /pfs/tmp a
    16
    17  du -sh /HAMMER/a/
    18  cd
    19  umount /HAMMER
    20  mount_hammer ${HAMMER_DEVICE} /HAMMER
    21  du -sh /HAMMER/a/


Doing something like below may end up not copying a file, but not sure
if this is technically what you're seeing.
This happens in older versions (3.x) too.

# cat -n ./pfsd1.sh
     1  #!/usr/local/bin/bash
     2
     3  HAMMER_DEVICE=/dev/da1
     4
     5  cd
     6  umount /HAMMER
     7  newfs_hammer -L TEST ${HAMMER_DEVICE} > /dev/null
     8  mount_hammer ${HAMMER_DEVICE} /HAMMER
     9  cd /HAMMER
    10
    11  hammer pfs-master a
    12  hammer pfs-slave b `hammer pfs-status a | grep shared-uuid`
    13
    14  dd if=/dev/urandom of=./a/xxx bs=1024 count=1
    15  #sync; sync; sync; sync; sync
    16  hammer mirror-copy a b
    17
    18  cd
    19  umount /HAMMER
    20  mount_hammer ${HAMMER_DEVICE} /HAMMER
    21  ls /HAMMER/*


2016-08-25 19:35 GMT+09:00 Tomohiro Kusumi <kusumi.tomohiro at gmail.com>:
> I'll see if I can reproduce it within a few days and try to fix it if
> that's a bug,
> since I'm currently trying to fix lots of PFS related bugs.
>
> 2016-08-25 5:31 GMT+09:00 Lanir <lanir at cisns.net>:
>> I'm on 4.6-RELEASE.
>>
>> When I unmounted and remounted the base hammer filesystem it showed a
>> matching size. But when I stop the mirror-stream and mount the
>> former-slave PFS, I have no data to show for it. After a reboot the size
>> is off again.
>>
>> I'm a little confused about what's actually going on with it at the
>> moment. Data appears to be there but inaccessible. I think that bug is
>> related to what I'm seeing.
>>
>>
>>
>>
>> On 08/24/16 09:37, Tomohiro Kusumi wrote:
>>>> This might be related to the bug I reported:
>>>>
>>>> http://bugs.dragonflybsd.org/issues/2915
>>>
>>> Lanir,
>>> Based on Tim's report on DragonFly 4.5, what's your uname -r ?
>>>
>>>
>>>
>>> 2016-08-24 22:23 GMT+09:00 Tomohiro Kusumi <kusumi.tomohiro at gmail.com>:
>>>> BTW what I mean by unmounting is unmounting hammer, but not PFS (if
>>>> you've null mounted them somewhere or your dfly installer has done
>>>> it).
>>>>
>>>> 2016-08-24 22:14 GMT+09:00 Tomohiro Kusumi <kusumi.tomohiro at gmail.com>:
>>>>> I first saw this in 2014 when I first used DragonFly (I think 3.8).
>>>>> It happened without upgrade or downgrade of PFS.
>>>>>
>>>>> Unmounting once and then mounting again made the slave PFS in sync with master.
>>>>> It may not be possible in your case as yours seem to be / fs.
>>>>>
>>>>>
>>>>>
>>>>> 2016-08-24 12:53 GMT+09:00 Lanir <lanir at cisns.net>:
>>>>>> Hi,
>>>>>>
>>>>>> I think I must have setup the mirroring incorrectly because nothing is
>>>>>> happening. I tried following the guide "how to implement hammer pseudo
>>>>>> file system( pfs ) slave mirroring from pfs master" on the website but
>>>>>> something appears to have gone wrong.
>>>>>>
>>>>>> I have two 7.3tb hammer filesystems spread over two physical disks each
>>>>>> on the same system. I had problems creating the slave PFS so created it
>>>>>> as a master, demoted it, then altered the shared-uuid to match before
>>>>>> trying to mirror it. When I do a mirror-copy I get this result:
>>>>>>
>>>>>> # hammer mirror-copy /usr/stor/backups /usr/backup-stor/pfs/backups
>>>>>> Mirror-read /usr/stor/backups succeeded
>>>>>> #
>>>>>>
>>>>>> The data is obviously not being copied over:
>>>>>>
>>>>>> # df -h /usr/stor /usr/backup-stor
>>>>>> Filesystem                       Size   Used  Avail Capacity  Mounted on
>>>>>> stor1                            7.3T    44G   7.2T     1%    /usr/stor
>>>>>> backup-stor1                     7.3T   1.0G   7.3T     0%
>>>>>> /usr/backup-stor
>>>>>>
>>>>>> As far as I can tell they're setup correctly:
>>>>>>
>>>>>> # hammer pfs-status /usr/stor/backups /usr/backup-stor/backups/ | egrep
>>>>>> '(shared-uuid|operating)'
>>>>>>     shared-uuid=4138d697-3355-11e5-99b1-e13f494aab78
>>>>>>     operating as a MASTER
>>>>>>     shared-uuid=4138d697-3355-11e5-99b1-e13f494aab78
>>>>>>     operating as a SLAVE
>>>>>>
>>>>>>
>>>>>> Can someone help point me towards the problem? I feel like I may have
>>>>>> missed something.
>>>>>>
>>>>>>
>>>>>> Thank you!
>>>>>>
>>>>>>
>>
>>