taking advantage of hammer

[patric conant]

Bob,

The biggest boon to workflows I've seen, is being able to run invasive
reports against a stale database, as the impact of running the reports on
the live database is a larger painpoint than using recent, but not current
data. In the case of a dbms, some level of coordination telling it to
become consistent on disk at the moment of the snapshot may be necessary.
The pfs in question was offloaded to another machine for the operation,
although I suspect that nice could get you what you wanted running on the
same machine.

On Wed, Aug 8, 2018 at 1:40 PM Bob Obrien <bobo90210 at protonmail.com> wrote:

> Thanks, I appreciate that. I understand and have been known to take UFS
> drives offline or at least go single user in order to backup in the past.
>
> I guess what I am truly looking for is a use for PFS. They feel like a
> technology right on the cusp of something I could use to make a big
> improvement to my workflow, but I cannot figure out how!
>
>
> Sent with ProtonMail <https://protonmail.com> Secure Email.
>
> ‐‐‐‐‐‐‐ Original Message ‐‐‐‐‐‐‐
> On August 8, 2018 11:08 AM, patric conant <mirage.computing at gmail.com>
> wrote:
>
> Bryan,
>
> Without snapshots filesystem operations are not atomic, when you Backup,
> copy, sync, rsync, you are not getting a cohesive operation, but the
> various states of individual files at the time they are accessed. So, no,
> you can't back up UFS, with any sort of guaranteed consistency, yes each
> file can be consistent, but for example, the logs for services, and the
> file states of those services are out of sync, now when you restore you
> have logs that are not for point of time that the files are for.
> Snapshotting is pretty much the cornerstone of modern storage.  Next
> checksumming, which I'm not aware of UFS support in. Without checksumming
> the only source of validity of a file is the file itself. There's no
> guarantee or even expectation of file consistency, in traditional file
> systems, just that we trust the hardware because we have to. RAID mitigates
> this, to the tune of reducing it 50-90%, but an 8TB live data set simply is
> not exactly what was laid down on the storage medium, and there's no
> expectation of such. That's some of the nitty gritty of the modern storage
> layer, but as we all know and experience, systems run for years without
> these problems enjoying any sort of remedy outside of best effort, there's
> probably machines with massive data-sets, and decades of history, whom have
> been subject to these problems, being restored from file-level backups many
> times, and having the total of the writes be in the multiple petabytes
> range, and the applications running on top seeming no worse for wear. This
> speaks to the resiliency to some level of corruption of most applications.
>
> On Tue, Aug 7, 2018 at 6:57 PM Bob Obrien <bobo90210 at protonmail.com>
> wrote:
>
>> Hello there,
>> I am enjoying Dragon Fly BSD so far, nice and snappy, seems sensibly
>> organized.
>>
>> I like the idea of HAMMER but for my relatively basic uses I am not
>> seeing a great benefits. Most notably, PFS seem to be a huge part of the
>> design and what makes it tick. OK, cool, I guess you can snapshot via PFS.
>> But wait, now you can snapshot ANY directory? mount_null also seems core to
>> a basic DF setup but that is not limited to hammer at all. Backups, copies,
>> syncs, yes it seems a neat organizational tool but at the same time I copy,
>> rsync, etc etc with all my UFS drives and it works great. Assuming small
>> enough files and drives that limitations are not being reached, of course.
>>
>> Maybe if I was a more advanced sysadmin type I would have more specific
>> demands or use cases that would take advantage?
>>
>> Or maybe I simply need someone to explain how these benefit me and I will
>> see the light, and improve my workflows. Thanks!
>>
>>
>> Sent with ProtonMail <https://protonmail.com> Secure Email.
>>
>>
>
> --
> Patric Conant
> Mirage Computing Lead Consultant
> *@MirageComputing <https://twitter.com/MirageComputing>*on twitter
> https://m.facebook.com/MirageComputing/
> 316 409 2424
>
>
>

-- 
Patric Conant
Mirage Computing Lead Consultant
@MirageComputing <https://twitter.com/MirageComputing>on twitter
https://m.facebook.com/MirageComputing/
316 409 2424
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.dragonflybsd.org/pipermail/users/attachments/20180808/bbc0ec2c/attachment.htm>