Vinum troubleshooting

Sun Sep 18 14:42:45 PDT 2016

Hi all,

I'm having a rough time trying to get vinum going. After a few failed starts I got my
disks in a format that appeared to work, but vinum won't bring the volume up after
creation.

loki# vinum lv
V storage               State: down	Plexes:       1	Size:       9315 GB

Broadly - I have six disks of 2Tb or more. I want to bundle them all up and be able to
survive a single drive failure. RAID-5 with vinum seems my best option; I gave up on LVM2
when I got lost trying to set up the disks in a way it approved of. Open to other
solutions than vinum if they exist though.

Any guidance gratefully received.

(PS, man vinum(8) states "vinum maintains a log file, by default /var/tmp/vinum_history,",
it's actually in /var/log/vinum_history)

- Andrew

Diagnostics:

loki# uname -v
DragonFly v4.6.0-RELEASE #0: Mon Aug  1 12:46:25 EDT 2016     root at www.shiningsilence.com:/usr/obj/build/sys/X86_64_GENERIC 

loki# vinum info -V
Flags: 0x80004
Can't get information: Invalid argument

loki# cat /etc/vinum.conf 
drive d1 device /dev/da0s1a
drive d2 device /dev/da1s1a
drive d3 device /dev/da3s1a
drive d4 device /dev/da4s1a
drive d5 device /dev/da5s1a
drive d6 device /dev/da6s1a
volume storage
 plex org raid5 512k
   sd length 1863G drive d1
   sd length 1863G drive d2
   sd length 1863G drive d3
   sd length 1863G drive d4
   sd length 1863G drive d5
   sd length 1863G drive d6

loki# vinum l
6 drives:
D d1                    State: up	Device /dev/da0s1a	Avail: 13/1907725 MB (0%)
D d2                    State: up	Device /dev/da1s1a	Avail: 189438/2097150 MB (9%)
D d3                    State: up	Device /dev/da3s1a	Avail: 189438/2097150 MB (9%)
D d4                    State: up	Device /dev/da4s1a	Avail: 13/1907725 MB (0%)
D d5                    State: up	Device /dev/da5s1a	Avail: 13/1907725 MB (0%)
D d6                    State: up	Device /dev/da6s1a	Avail: 13/1907725 MB (0%)

1 volumes:
V storage               State: down	Plexes:       1	Size:       9315 GB

1 plexes:
P storage.p0         R5 State: init	Subdisks:     6	Size:       9315 GB

6 subdisks:
S storage.p0.s0         State: empty	PO:        0  B Size:       1863 GB
S storage.p0.s1         State: empty	PO:      512 kB Size:       1863 GB
S storage.p0.s2         State: empty	PO:     1024 kB Size:       1863 GB
S storage.p0.s3         State: empty	PO:     1536 kB Size:       1863 GB
S storage.p0.s4         State: empty	PO:     2048 kB Size:       1863 GB
S storage.p0.s5         State: empty	PO:     2560 kB Size:       1863 GB

loki# vinum start
** no drives found: No such file or directory
Warning: defective objects

V storage               State: down	Plexes:       1	Size:       9315 GB
P storage.p0         R5 State: init	Subdisks:     6	Size:       9315 GB
S storage.p0.s0         State: empty	PO:        0  B Size:       1863 GB
S storage.p0.s1         State: empty	PO:      512 kB Size:       1863 GB
S storage.p0.s2         State: empty	PO:     1024 kB Size:       1863 GB
S storage.p0.s3         State: empty	PO:     1536 kB Size:       1863 GB
S storage.p0.s4         State: empty	PO:     2048 kB Size:       1863 GB
S storage.p0.s5         State: empty	PO:     2560 kB Size:       1863 GB

da0 through to da6 (excluding da2) all have identical results on the below (aside from 1 & 3 being a little bigger):

loki# fdisk /dev/da0
******* Working on device /dev/da0 *******
parameters extracted from device are:
cylinders=242251 heads=256 sectors/track=63 (16128 blks/cyl)

Figures below won't work with BIOS for partitions not in cyl 1
parameters to be used for BIOS calculations are:
cylinders=242251 heads=256 sectors/track=63 (16128 blks/cyl)

Media sector size is 512
Warning: BIOS sector numbering starts with sector 1
Information from DOS bootblock is:
The data for partition 1 is:
sysid 165,(DragonFly/FreeBSD/NetBSD/386BSD)
    start 63, size 3907024065 (1907726 Meg), flag 80 (active)
	beg: cyl 0/ head 1/ sector 1;
	end: cyl 1023/ head 255/ sector 63
The data for partition 2 is:
<UNUSED>
The data for partition 3 is:
<UNUSED>
The data for partition 4 is:
<UNUSED>

loki# gpt show /dev/da0
       start        size  index  contents
           0           1      -  MBR
           1          62      -  
          63  3907024065      0  MBR part 165
  3907024128        5007      -  
  3907029135          32      -  Sec GPT table
  3907029167           1      -  Sec GPT header

loki# disklabel64 /dev/da0s1
# /dev/da0s1:
#
# Informational fields calculated from the above
# All byte equivalent offsets must be aligned
#
# boot space:    1044992 bytes
# data space: 1953511003 blocks	# 1907725.59 MB (2000395267584 bytes)
#
# NOTE: If the partition data base looks odd it may be
#       physically aligned instead of slice-aligned
#
diskid: f413c18e-7d78-11e6-ae3b-010000000000
label: 
boot2 data base:      0x000000001000
partitions data base: 0x000000100200
partitions data stop: 0x01d1c0e97000
backup label:         0x01d1c0e97000
total size:           0x01d1c0e98200	# 1907726.59 MB
alignment: 4096
display block size: 1024	# for partition display only

16 partitions:
#          size     offset    fstype   fsuuid
  a: 1953511000          0     vinum	# 1907725.586MB
  a-stor_uuid: 0a51ab7d-7d79-11e6-ae3b-010000000000

What I'm seeing in /var/log/messages (trimmed, but representative):

Sep 19 06:47:09 loki kernel: vinum: CONFIGURATION OBLITERATED
Sep 19 06:47:09 loki kernel: vinumioctl: invalid ioctl from process 4489 (vinum): 40906468
Sep 19 06:47:34 loki kernel: vinumioctl: invalid ioctl from process 4491 (vinum): 40906468
Sep 19 06:47:34 loki kernel: vinumioctl: invalid ioctl from process 4492 (vinum): 40906468
Sep 19 06:47:34 loki kernel: vinum: drive d1 is up
Sep 19 06:47:34 loki kernel: vinum: drive d2 is up
Sep 19 06:47:34 loki kernel: vinum: drive d3 is up
Sep 19 06:47:34 loki kernel: vinum: drive d4 is up
Sep 19 06:47:34 loki kernel: vinum: drive d5 is up
Sep 19 06:47:34 loki kernel: vinum: drive d6 is up
Sep 19 06:54:41 loki kernel: Warning: cache_resolve: ncp 'vol' was unlinked