git: hammer2 - Add xxhash to H2 and throw in debug stuff for performance testing.

[Matthew Dillon]

commit 7d565a4f16015b664e90560061ad7574c3e68737
Author: Matthew Dillon <dillon at apollo.backplane.com>
Date:   Wed Jun 8 16:06:51 2016 -0700

    hammer2 - Add xxhash to H2 and throw in debug stuff for performance testing.
    
    * Add the xxhash.  This is a high-speed non-cryptographic hash code
      algorithm.  Sam pointed me at the site, the code is available on
      github and is BSD licensed:
    
          git://github.com/Cyan4973/xxHash.git
    
      This hash has good distribution and is very fast.
    
    * Change HAMMER2 to default to using xxhash64 instead of iscsi_crc32().
      xxhash can process data at several GBytes/sec where as even the
      multi-table iscsi_crc32() can only do around 500 MBytes/sec, which
      is too slow for today's modern storage subsystems (NVME can nominally
      do 1.5-2.5 GBytes/sec, and high-end cards can do 5GBytes/sec).
    
    * There are four major paths that eat tons of CPU in H2:
    
      - The XIO path does a ton of allocation/deallocation and synchronous
        messaging.  This has not yet been fixed.
    
      - The check code (when it was iscsi_crc32()) slowed everything down.
        This is fixed, the default check code is now xxhash64.
    
      - The check code was being called over and over again for the same cached
        buffer due to the hammer2_chain_t structure being thrown away.
    
        Currently a hack involving a mask stored in the underlying DIO is being
        used to indicate that the check code was previously valid.  This is
        strictly temporary.  The actual mask will have to be stored in the
        device buffer cache buffer and a second one in the chain structure.
    
        The chain structure must be made persistent as well (not yet done).
    
      - The DEDUP code was also calling iscsi_crc32() redundantly (at least for
        reads).
    
        The read path has been fixed.  The write path is doable but requires more
        coding (not yet fixed).
    
      - The logical file cluster_read() in the kernel was not doing any read-ahead
        due to H2 not implementing BMAP, creating long synchronous latencies.
    
        The kernel code for cluster_read() and cluster_readcb() has been fixed
        to do read-ahead whether a logical BMAP is implemented or not.  H2 will
        now pipeline reads.
    
    Suggested-by: Samuel J. Greear <sjg at thesjg.com>	(xxhash)

Summary of changes:
 sbin/hammer2/Makefile                              |   3 +
 sbin/hammer2/cmd_debug.c                           |  13 +-
 sbin/hammer2/cmd_info.c                            |  10 +-
 sbin/hammer2/cmd_setcheck.c                        |   4 +-
 sbin/hammer2/hammer2.h                             |   1 +
 sbin/newfs_hammer2/Makefile                        |   5 +-
 sbin/newfs_hammer2/newfs_hammer2.c                 |  19 +-
 sys/vfs/hammer2/Makefile                           |   4 +-
 sys/vfs/hammer2/hammer2.h                          |  27 +-
 sys/vfs/hammer2/hammer2_chain.c                    |  71 +-
 sys/vfs/hammer2/hammer2_disk.h                     |   6 +-
 sys/vfs/hammer2/hammer2_flush.c                    |   4 +-
 sys/vfs/hammer2/hammer2_inode.c                    |  30 +-
 sys/vfs/hammer2/hammer2_io.c                       | 103 ++-
 sys/vfs/hammer2/hammer2_ioctl.c                    |   2 +-
 sys/vfs/hammer2/hammer2_strategy.c                 |  75 +-
 sys/vfs/hammer2/hammer2_thread.c                   |  33 +-
 sys/vfs/hammer2/hammer2_vfsops.c                   |  27 +-
 sys/vfs/hammer2/hammer2_vnops.c                    |   5 +-
 .../hammer2/hammer2_xxhash.h}                      |  11 +-
 sys/vfs/hammer2/xxhash/xxhash.c                    | 863 +++++++++++++++++++++
 sys/vfs/hammer2/xxhash/xxhash.h                    | 275 +++++++
 22 files changed, 1522 insertions(+), 69 deletions(-)
 copy sys/{bus/gpio/gpio_acpi/gpio_acpivar.h => vfs/hammer2/hammer2_xxhash.h} (90%)
 create mode 100644 sys/vfs/hammer2/xxhash/xxhash.c
 create mode 100644 sys/vfs/hammer2/xxhash/xxhash.h

http://gitweb.dragonflybsd.org/dragonfly.git/commitdiff/7d565a4f16015b664e90560061ad7574c3e68737


-- 
DragonFly BSD source repository