/var is read-only

[Pierre Abbat]

I just noticed that the usual periodic email is not coming. I checked /var/
log/maillog and found that it's getting "Read-only filesystem" errors from 
maildrop several times a second. 

The computer has two disks. /boot, swap, and / are on one; /boot1, /var, and /
crypt are on the other. /boot1 and /crypt are read-write (I created a file on 
each, then deleted it). Only /var is read-only (and /var/crash and /var/tmp 
are null-mounts from /, which is on the other disk). It is somehow writing 
huge reams of "Read-only filesystem" errors to a read-only filesystem.

Attached is the snippet of /var/log/messages when it put /var/ into read-only 
mode. What should I do?

Pierre
-- 
gau do li'i co'e kei do
-------------- next part --------------
Dec 27 03:01:21 zyxomma kernel: ahci0.0: TFES slot 16 ci_saved = fffd96c4
Dec 27 03:01:33 zyxomma kernel: ahci0.0: read NCQ error page slot=16
Dec 27 03:01:33 zyxomma kernel: ahci0.0: DONE log page target 0 err_slot=16
Dec 27 03:01:33 zyxomma kernel: ahci0.0: disk_rw: error fiscmd=0x60 @off=0x0000000cf1500000, 16384
Dec 27 03:01:33 zyxomma kernel: ahci0.0: Restart fffc96c4
Dec 27 03:01:33 zyxomma kernel: (da0:ahci0:0:0:0): READ(10). CDB: 28 0 6 78 a8 0 0 0 20 0 
Dec 27 03:01:33 zyxomma kernel: (da0:ahci0:0:0:0): CAM Status: SCSI Status Error
Dec 27 03:01:33 zyxomma kernel: (da0:ahci0:0:0:0): SCSI Status: Check Condition
Dec 27 03:01:33 zyxomma kernel: (da0:ahci0:0:0:0): MEDIUM ERROR asc:0,0
Dec 27 03:01:33 zyxomma kernel: (da0:ahci0:0:0:0): No additional sense information
Dec 27 03:01:33 zyxomma kernel: (da0:ahci0:0:0:0): Retrying Command (per Sense Data)
Dec 27 03:01:33 zyxomma kernel: ahci0.0: TFES slot 27 ci_saved = 0ffdfec5
Dec 27 03:01:33 zyxomma kernel: ahci0.0: read NCQ error page slot=0
Dec 27 03:01:33 zyxomma kernel: ahci0.0: DONE log page target 0 err_slot=0
Dec 27 03:01:33 zyxomma kernel: ahci0.0: disk_rw: error fiscmd=0x60 @off=0x0000000cf1500000, 16384
Dec 27 03:01:33 zyxomma kernel: ahci0.0: Restart 0ffdfec4
Dec 27 03:01:33 zyxomma kernel: (da0:ahci0:0:0:0): READ(10). CDB: 28 0 6 78 a8 0 0 0 20 0 
Dec 27 03:01:33 zyxomma kernel: (da0:ahci0:0:0:0): CAM Status: SCSI Status Error
Dec 27 03:01:33 zyxomma kernel: (da0:ahci0:0:0:0): SCSI Status: Check Condition
Dec 27 03:01:33 zyxomma kernel: (da0:ahci0:0:0:0): MEDIUM ERROR asc:0,0
Dec 27 03:01:33 zyxomma kernel: (da0:ahci0:0:0:0): No additional sense information
Dec 27 03:01:33 zyxomma kernel: (da0:ahci0:0:0:0): Retrying Command (per Sense Data)
Dec 27 03:01:33 zyxomma kernel: ahci0.0: TFES slot 0 ci_saved = ffff8139
Dec 27 03:01:33 zyxomma kernel: ahci0.0: read NCQ error page slot=28
Dec 27 03:01:33 zyxomma kernel: ahci0.0: DONE log page target 0 err_slot=28
Dec 27 03:01:33 zyxomma kernel: ahci0.0: disk_rw: error fiscmd=0x60 @off=0x0000000cf1500000, 16384
Dec 27 03:01:33 zyxomma kernel: ahci0.0: Restart efff8139
Dec 27 03:01:33 zyxomma kernel: (da0:ahci0:0:0:0): READ(10). CDB: 28 0 6 78 a8 0 0 0 20 0 
Dec 27 03:01:33 zyxomma kernel: (da0:ahci0:0:0:0): CAM Status: SCSI Status Error
Dec 27 03:01:33 zyxomma kernel: (da0:ahci0:0:0:0): SCSI Status: Check Condition
Dec 27 03:01:33 zyxomma kernel: (da0:ahci0:0:0:0): MEDIUM ERROR asc:0,0
Dec 27 03:01:33 zyxomma kernel: (da0:ahci0:0:0:0): No additional sense information
Dec 27 03:01:33 zyxomma kernel: (da0:ahci0:0:0:0): Retrying Command (per Sense Data)
Dec 27 03:01:33 zyxomma kernel: ahci0.0: TFES slot 10 ci_saved = efff8ec4
Dec 27 03:01:33 zyxomma kernel: ahci0.0: read NCQ error page slot=2
Dec 27 03:01:33 zyxomma kernel: ahci0.0: DONE log page target 0 err_slot=2
Dec 27 03:01:33 zyxomma kernel: ahci0.0: disk_rw: error fiscmd=0x60 @off=0x0000000cf1500000, 16384
Dec 27 03:01:33 zyxomma kernel: ahci0.0: Restart efff8ec0
Dec 27 03:01:33 zyxomma kernel: (da0:ahci0:0:0:0): READ(10). CDB: 28 0 6 78 a8 0 0 0 20 0 
Dec 27 03:01:33 zyxomma kernel: (da0:ahci0:0:0:0): CAM Status: SCSI Status Error
Dec 27 03:01:33 zyxomma kernel: (da0:ahci0:0:0:0): SCSI Status: Check Condition
Dec 27 03:01:33 zyxomma kernel: (da0:ahci0:0:0:0): MEDIUM ERROR asc:0,0
Dec 27 03:01:33 zyxomma kernel: (da0:ahci0:0:0:0): No additional sense information
Dec 27 03:01:33 zyxomma kernel: (da0:ahci0:0:0:0): Retrying Command (per Sense Data)
Dec 27 03:01:33 zyxomma kernel: ahci0.0: TFES slot 26 ci_saved = 07ff9ff4
Dec 27 03:01:33 zyxomma kernel: ahci0.0: read NCQ error page slot=12
Dec 27 03:01:33 zyxomma kernel: ahci0.0: DONE log page target 0 err_slot=12
Dec 27 03:01:33 zyxomma kernel: ahci0.0: disk_rw: error fiscmd=0x60 @off=0x0000000cf1500000, 16384
Dec 27 03:01:33 zyxomma kernel: ahci0.0: Restart 07ff8ff4
Dec 27 03:01:33 zyxomma kernel: (da0:ahci0:0:0:0): READ(10). CDB: 28 0 6 78 a8 0 0 0 20 0 
Dec 27 03:01:33 zyxomma kernel: (da0:ahci0:0:0:0): CAM Status: SCSI Status Error
Dec 27 03:01:33 zyxomma kernel: (da0:ahci0:0:0:0): SCSI Status: Check Condition
Dec 27 03:01:33 zyxomma kernel: (da0:ahci0:0:0:0): MEDIUM ERROR asc:0,0
Dec 27 03:01:33 zyxomma kernel: (da0:ahci0:0:0:0): No additional sense information
Dec 27 03:01:33 zyxomma kernel: (da0:ahci0:0:0:0): Retries Exhausted
Dec 27 03:01:33 zyxomma kernel: HAMMER(stem) Critical error inode=-1 error=5 while flushing meta-data
Dec 27 03:01:33 zyxomma kernel: HAMMER(stem) Forcing read-only mode
Dec 27 03:01:33 zyxomma kernel: HAMMER(stem) Critical error inode=-1 error=5 while flushing meta-data
Dec 27 03:01:33 zyxomma last message repeated 15 times