Josh-Daniel S. Davis (joshdavis) wrote,
Josh-Daniel S. Davis

udev and multipath-tools are not friends.

ENV: Debian testing, unstable, experimental
  • Linux ns1 2.6.14-2-686-smp #2 SMP Fri Dec 9 10:20:41 UTC 2005 i686 GNU/Linux
  • sda, sdb and sdd are 9.1g WD 7200RPM
  • sdc is 173g Seagate
  • the others are 50g seagate w/o domain validation support.
  • Everything's on a dual channel adaptec U160 card.

PROBLEM: non-multipath SCSI devices other than the boot device become
unavailable if multipath-tools is installed as show by this output:
/bin/bash# mount
/dev/sda1 on / type ext3 (rw,user_xattr,errors=remount-ro)
/dev/sda6 on /usr type ext3 (rw,user_xattr,errors=remount-ro)
/dev/sda7 on /var type ext3 (rw,user_xattr,errors=remount-ro)
/dev/sda5 on /tmp type ext3 (rw)
proc on /proc type proc (rw)
sysfs on /sys type sysfs (rw)
devpts on /dev/pts type devpts (rw,gid=5,mode=620)
tmpfs on /dev/shm type tmpfs (rw)
usbfs on /proc/bus/usb type usbfs (rw)
tmpfs on /dev type tmpfs (rw,size=10M,mode=0755)
/dev/sdc1 on /storage type ext3 (rw,user_xattr)
/bin/bash# mount -a
mount: /dev/sdb2 already mounted or /home busy
mount: mount point /home/xaminmo does not exist
mount: /dev/sde1 already mounted or /storage/music busy
mount: /dev/sdf1 already mounted or /storage/other busy
mount: /dev/sdg1 already mounted or /storage/programs busy
mount: /dev/sdh1 already mounted or /storage/uploads busy
/bin/bash# ls /var/cache/apt/archives/{mul*,udev*}
-rw-r-----  1 root root 127860 2005-09-30 06:17 multipath-tools_0.4.5-3_i386.deb
-rw-r-----  1 root root 277114 2005-12-03 12:47 udev_0.076-4_i386.deb
-rw-r-----  1 root root 278126 2005-12-11 10:47 udev_0.076-6_i386.deb
/bin/bash# uname -a
NOTE: Debian still, after a month, does not have multipath-tools 4.6. Only 4.5.
NOTE: Debian's 2.6 kernel packages are a little laggy too

ACTION TAKEN: I came current on testing, installed udev.
I rebooted and nothing but sda* would mount.
I went to unstable to get the new udev, rebooted, no change.
I went to experimental to get the new udev, rebooted.
/storage would mount (/dev/sdc1), but still not the others.

Lots of places online wanted kernel 2.6.14.
I got this from unstable.
When I ran lilo, it complained that dm-0 wasn't correct anymore.


udevtest called the disk dm-0.
dm-0 was renamed to '' because udev says drivemapper makes it's own entries.


I commented out the config line, and udevtest showed that it would make the right thing.
I did udevsynthesize, and the dm entries still didn't show up.
Even worse, drivemapper devices don't build partition entries in /dev.
You have to use the whole LUN as one device.


Uninstalling multipath-tools and rebooting fixed the problem.

SUMMARY: I'm leaving udev on there for now. It SEEMS to be working well.
Negative items
multipath-tools expects all of your scsi disks to be multi-path.
multipath-tools REQUIRES udev, but doesn't provide sane rules.
udev man pages mostly all link back to the same page with only a basic usage statement which is inadequate for actually making things work, and omits many crucial commands.
udev isn't really designed to regenerate all devices should a problem ocur, rather it listens for hotplug events. If it misses something, or the daemon dies, you have to use undocumented commands to recover.

Positive items
/dev is rebuilt on every reboot as a tmpfs.
Now I have /dev/drive/by-label/* entries.
There's also ID, uuid, and serial directories.
Assuming I never had to plug in a disk with a duplicate label,
this could be great for disks which might otherwise
change where they were on the bus.
Examples include usb, hotswap, or anything else which renumbers.

after-the-fact update
I found which happens to have a quick walkthrough on setting up multipath-tools, so I'll have to pass the blame on to the Debian package maintainer for now. I'm still not up for retrying the install since I'm not multipathed ATM.
Tags: evil, linux, storage
  • Post a new comment


    Anonymous comments are disabled in this journal

    default userpic

    Your reply will be screened

    Your IP address will be recorded