It happened earlier and I rebooted after not being able to get courier-IMAP to respond.
Hrm, I can't SSH in.
So I check the console, and I have I/O errors, lost blocks, whatever.
I can't reboot, but the kernel isn't dead... just spinning.
So I power off, boot up, and I get initrd only.
My boot device is a 5-drive mirror. It's not up? WTF. I assemble it and mark it readwrite. No rebuild required. WHAT IS YOUR MAJOR MALFUNCTION?
My 5-drive raid6/datavg also assembles happily. Again, WTF.
My 5-drive raid6/rootvg says it's dirty and degraded. "Try --force".
Okay, I try --force --run, and it just gives me an I/O error.
DD shows that all 5 devices are JUST FUCKING FINE!
mdadm -D shows /dev/sdb2 is missing.
Why is it missing? I force assembled you into the array!
No, that is not good enough. I have to --re-add the device into the array.
WTF. This is stupid.
Anyway, now, I can mark it readwrite, and it is resyncing the array at a whopping 23428K/sec (all 5 drives hang off of a SATA-1 port multiplier).
So here is my complaint -- Being a 5 drive RAID6,
I should be able to force run on THREE OF FIVE DRIVES.
A) Why did I get an I/O error for only one drive being kicked out?
B) Why did mdadmd not email me about this problem?
C) Why did ONE drive get kicked and STAY kicked, and a second drive stay dirty
Seriously! This is why I run RAID6. It is not acceptable to kick a drive out, but then say you won't start without it, even though I told you to start with it!
This is just stupid MDADM is apparently there, not to actually provide any operational resiliency, but to shut every fucking thing down at the slightest sneeze.
So, should I expect anything better on the *BSD variants?
Should I go with FreeBSD because of the userbase and ZFS, or would one of the others be better?
etc etc etc. Input is appreciated.