Questions on HAMMER2 current details and implementation

[HolyTaint]

On Mon, 19 Sep 2022 21:39:21 -0400 Justin Sherrill <justin at shiningsilence.com> wrote:
> That PFS you have linked is not the PFS you think it is.  PFS in this case
> stands for pseudo file system, if I remember correctly.  The design
> document may help you:
> 
> http://apollo.backplane.com/DFlyMisc/hammer2.txt
> 
> Or the man page:
> 
> https://man.dragonflybsd.org/?command=hammer2&section=ANY
> 
> 
> On Mon, Sep 19, 2022 at 8:00 PM HolyTaint <holytaint at disroot.org> wrote:
> 
> > As far as I understood, HAMMER2 is a composite filesystem and files are
> > organized in separated PFS filesystems which differ a bit from the original
> > implementation described in
> > https://en.wikipedia.org/wiki/Professional_File_System
> > The only modification I grasped is the inode tree being modified for
> > accomodating segments of the main radix tree, but I still miss some
> > informations.
> > I tried to infer some theories on these questions:
> >
> > * How are snapshots made? Is it safe to assume that snapshots are actually
> > a collection of links to PFS filesystems and that's the reason why they are
> > modifiable?
> > * How does HAMMER2 avoid the need of a journal and support an undo
> > feature? Does it use CoW on EVERYTHING?
> >
> > I totally have no clue on these questions instead:
> >
> > * Do the individual PFS file systems have fixed spaces or do they increase
> > according to the load? Does their fixed/maximum space occupied on the disk
> > coincide with the original implementation referred in the wikipedia article?
> > * How does HAMMER2 achieve deduplication and compression across chunks on
> > the multiple PFS volumes present while ALSO mantaining the premise of
> > separating them and allowing async operations on them? How much are the PFS
> > volumes isolated between themselves? Do they have an individual freemap or
> > do they share a collective freemap for every storage device?
> >

This is my (probably false) current understanding of HAMMER2, given docs reading and chats with Dillon (I may have misunderstood lots of stuff)

HAMMER2 has a quadruple superblock containing inodes and freemaps radix trees roots, which are extended in segments around the disk of 2 MB each,
containing a fat inode structure (1 kb) and a freemap keeping tracks of single blocks down to 1kb~ (weights another kb I think); these structures are repeated 4 times too

Each block starts with a 128 bytes blockref containing a reserved 64 bytes hash field, and given that each modification is CoW and HAMMER2 keeps an history of modifications, whenever a block is modified the changed data is shifted to a new block inside the segment while the previous block is marked for elimination

HAMMER2 supports modifiable snapshots, probably snapshots are collections of these 2MB segments, and diverging from these snapshots means creating copies of these segments with some data changes. Although, since HAMMER2 supports dedup on copy, most file inodes will actually be refs to the previous segments chunks

Finally, Pseudo File Systems are collections of a number of these segments

Please fix any misunderstanding of anything said

Assuming said words as true, I can't fathom why HAMMER2 emergency overwrite mode doesn't ensure blocks coherency verification, since there is already an hash field for each block