Critical Issues with Multipath losing Config on reboot

Issues related to applications and software problems
Post Reply
h1tchiker
Posts: 14
Joined: 2017/08/17 07:40:24

Re: Urgent Issues with Multipath losing Config on reboot

Post by h1tchiker » 2017/10/12 11:42:07

Is there any one who has enough experience of Multipath who can tell me why my partitions disappear on reboot please?
Last edited by h1tchiker on 2017/10/13 19:44:28, edited 2 times in total.
Top
h1tchiker
Posts: 14
Joined: 2017/08/17 07:40:24

Re: Urgent Issues with Multipath losing Config on reboot

Post by h1tchiker » 2017/10/13 10:29:02

Good Morning,

Perhaps some one needs more info? OK here is my config:

# cd /dev/mapper
# ll
lrwxrwxrwx. 1 root root 7 Oct 12 10:09 mpatha -> ../dm-3
lrwxrwxrwx. 1 root root 7 Oct 12 10:09 mpatha1 -> ../dm-7
lrwxrwxrwx. 1 root root 8 Oct 12 10:24 mpatha1p1 -> ../dm-12
lrwxrwxrwx. 1 root root 8 Oct 12 10:24 mpatha1p1p1 -> ../dm-13
lrwxrwxrwx. 1 root root 7 Oct 12 10:09 mpathb -> ../dm-4
lrwxrwxrwx. 1 root root 7 Oct 12 10:09 mpathb1 -> ../dm-8
lrwxrwxrwx. 1 root root 8 Oct 12 10:24 mpathb1p1 -> ../dm-15
lrwxrwxrwx. 1 root root 8 Oct 12 10:09 mpathb2 -> ../dm-10
lrwxrwxrwx. 1 root root 7 Oct 12 10:09 mpathc -> ../dm-2
lrwxrwxrwx. 1 root root 7 Oct 12 10:09 mpathc1 -> ../dm-6
lrwxrwxrwx. 1 root root 8 Oct 12 10:25 mpathc1p1 -> ../dm-17
lrwxrwxrwx. 1 root root 8 Oct 12 10:25 mpathc1p1p1 -> ../dm-18
lrwxrwxrwx. 1 root root 7 Oct 12 10:09 mpathd -> ../dm-5
lrwxrwxrwx. 1 root root 7 Oct 12 10:09 mpathd1 -> ../dm-9
lrwxrwxrwx. 1 root root 8 Oct 12 10:25 mpathd1p1 -> ../dm-20
lrwxrwxrwx. 1 root root 8 Oct 12 10:25 mpathd1p1p1 -> ../dm-21
lrwxrwxrwx. 1 root root 8 Oct 12 10:24 vg_file01_vdp-lv_file01 -> ../dm-16
lrwxrwxrwx. 1 root root 8 Oct 12 10:25 vg_iscsi_storage-lv_iscsi_storage -> ../dm-19
lrwxrwxrwx. 1 root root 8 Oct 12 10:25 vg_iscsi_vdp-lv_vdp -> ../dm-22
lrwxrwxrwx. 1 root root 8 Oct 12 10:24 vg_iscsi_vms-lv_iscsi_vm -> ../dm-14

OK so I rightly or wrongly read that to get multipath's you have to create the partitions on the /dev/mapper/mpath*1 so that is what I did, as you can see from the output above I had a few issues creating these (perhaps because this was the wrong thing to do?) So what I read just said to run the command again and you will just end up with another partition, for example:

lrwxrwxrwx. 1 root root 8 Oct 12 10:25 mpathc1p1p1 -> ../dm-18

While the system is up after using kpartx to hotadd the filesystems all works OK.

Now looking at the default config for multipath.conf "dm-" is blacklisted, so I have done the following changes to the multipath.conf:

blacklist_exceptions {
device {
vendor "HP"
product "P2000 G3 FC"
}
devnode "dm-"
}

## Use user friendly names, instead of using WWIDs as names.
defaults {
user_friendly_names yes
find_multipaths yes
path_grouping_policy multibus
path_checker readsector0
path_selector "round-robin 0"
failback "5"
rr_weight priorities
no_path_retry "5"
}

#Multipaths
multipaths {
#----------------- START VG=vg_file01_vdp LV=lv_file01 VM=mmgb-d02v-vdp02 ----------------------
multipath {
wwid 3600c0ff00012e03163ea655901000000
alias mpathb
}
#----------------- END VG=vg_file01_vdp LV=lv_file01 VM=mmgb-d02v-vdp02 ----------------------
#----------------- START VG=vg_iscsi_vms LV=lv_iscsi_vm USE=Additional VM storage ----------------------
multipath {
wwid 3600c0ff000dadd07d959635901000000
alias mpatha
}
#----------------- END VG=vg_iscsi_vms LV=lv_iscsi_vm USE=Additional VM storage ----------------------
#----------------- START VG=vg_iscsi_storage LV=lv_iscsi_storage USE=Additional storage ----------------------
multipath {
wwid 3600c0ff00014ebf5c757635901000000
alias mpathc
}
#----------------- END VG=vg_iscsi_storage LV=lv_iscsi_storage USE=Additional storage ----------------------
#----------------- START VG=vg_iscsi_vdp LV=lv_vdp VM=mmgb-d02v-vdp01 ----------------------
multipath {
wwid 3600c0ff00012e031efe9655901000000
alias mpathd
}
#----------------- START VG=vg_iscsi_vdp LV=lv_vdp VM=mmgb-d02v-vdp01 ----------------------
}

devices {
device {
vendor "HP"
product "P2000 G3 FC"
path_grouping_policy multibus
path_checker readsector0
path_selector "round-robin 0"
hardware_handler "0"
failback 15
rr_weight priorities
no_path_retry queue
}
device {
vendor "HP"
product "P2000 G3 FC"
path_grouping_policy multibus
}
}

blacklist {
devnode "^(sda|ram|raw|loop|fd|md|sr|scd|st|cl-home|cl-swap|cl-root)[0-9]*"
}

Now in the multipath.conf, there is a section for configuring multipaths, however the only relevant entry that I could see that I could possibly make here in the example is alias:

#multipaths {
# multipath {
# wwid 3600508b4000156d700012000000b0000
# alias yellow

I have added:

multipaths {
#----------------- START VG=REFERENCE ----------------------
multipath {
wwid 3600c0ff00012e03163ea655901000000
alias mpathb

}
#----------------- END VG=REFERENCE ----------------------
#----------------- START VG=REFERENCE ----------------------
multipath {
wwid 3600c0ff000dadd07d959635901000000
alias mpatha
}
#----------------- END VG=REFERENCE ----------------------
#----------------- START VG=REFERENCE ----------------------
multipath {
wwid 3600c0ff00014ebf5c757635901000000
alias mpathc
}
#----------------- END VG=REFERENCE ----------------------
#----------------- START VG=reference ----------------------
multipath {
wwid 3600c0ff00012e031efe9655901000000
alias mpathd
}
#----------------- END VG=REFERENCE ----------------------
}

Do i need to change the alias to one of my multipaths for example, such as /dev/mapper/mpathc1p1p1 for example? But this doesn't really make a lot of sense to me.

As far as I can tell/understand there is not an entry in the system to state that these paths shown here, should exist during the bootup process:

# pvs && vgs && lvs
PV VG Fmt Attr PSize PFree
/dev/mapper/mpatha1p1p1 vg_iscsi_vms lvm2 a-- <2.73t 0
/dev/mapper/mpathb1p1 vg_file01_vdp lvm2 a-- <7.26t 0
/dev/mapper/mpathc1p1p1 vg_iscsi_storage lvm2 a-- <9.09t 0
/dev/mapper/mpathd1p1p1 vg_iscsi_vdp lvm2 a-- 12.73t 0
/dev/sda2 cl lvm2 a-- <135.70g 0

VG #PV #LV #SN Attr VSize VFree
vg_file01_vdp 1 1 0 wz--n- <7.26t 0
vg_iscsi_storage 1 1 0 wz--n- <9.09t 0
vg_iscsi_vdp 1 1 0 wz--n- 12.73t 0
vg_iscsi_vms 1 1 0 wz--n- <2.73t 0

LV VG Attr LSize Pool Origin Data% Meta% Move Log Cpy%Sync Convert
lv_file01 vg_file01_vdp -wi-ao---- <7.26t
lv_iscsi_storage vg_iscsi_storage -wi-ao---- <9.09t
lv_vdp vg_iscsi_vdp -wi-ao---- 12.73t
lv_iscsi_vm vg_iscsi_vms -wi-ao---- <2.73t

This turned my attention to the lvm.conf file where the RedHat docco states that if you are using multipath you should add the following entry to the lvm.conf under devices:

filter = [ "r/block/", "r/disk/", "r/sd.*/", "a/.*/" ]

I did this and updated the initramfs and it caused the system not to boot. Once I removed this from the initramfs the system booted :(

if I show the blkid:

# blkid
/dev/sda1: UUID="887de158-d44f-4365-aee5-f16930f5a947" TYPE="xfs"
/dev/sda2: UUID="595V2O-D2xc-blZc-iMsc-aKDf-gVwk-wMkD1I" TYPE="LVM2_member"
/dev/mapper/cl-root: UUID="cf0bb0fa-de76-4e4e-b2df-a0acaf1a380b" TYPE="xfs"
/dev/mapper/cl-swap: UUID="eabf4b79-78d9-41c0-a99c-fb03ca7ba919" TYPE="swap"
/dev/mapper/cl-home: UUID="88a919f3-ab0e-4176-9a2f-d26a1f23393f" TYPE="xfs"
/dev/mapper/mpatha1p1p1: UUID="eSRWwg-A0qQ-CKZv-XWCp-sDib-JZUP-22LwR9" TYPE="LVM2_member" PARTLABEL="primary" PARTUUID="8ef397db-c6d8-4dc7-9470-d3c079afc010"
/dev/mapper/mpathb1p1: UUID="lp8I4T-4bcf-P2TE-2iyv-p1wJ-csik-4vjX0r" TYPE="LVM2_member" PARTLABEL="primary" PARTUUID="0e906c88-2d19-40f9-af91-ac3ab36b9f9e"
/dev/mapper/mpathc1p1p1: UUID="8LIbJ3-0r9U-ZuRq-F2kj-MysM-nryN-yM1JUN" TYPE="LVM2_member" PARTLABEL="primary" PARTUUID="634786a1-578d-4827-9ab0-ec7c797b2532"
/dev/mapper/mpathd1p1p1: UUID="SpXMnR-3fNu-LjQz-a32y-sEO6-jLkH-M9XvNw" TYPE="LVM2_member" PARTLABEL="primary" PARTUUID="ec6dfc62-55c3-41bd-bb1c-54a954d3e7bc"
/dev/sdb1: PTTYPE="gpt" PARTLABEL="primary" PARTUUID="82a53134-64e9-4918-8e8e-de166145f50f"
/dev/sdc1: PTTYPE="gpt" PARTUUID="5b90fa45-698b-467d-96f2-e5110478f193"
/dev/sdd1: PTTYPE="gpt" PARTUUID="5b90fa45-698b-467d-96f2-e5110478f193"
/dev/sde1: PTTYPE="gpt" PARTLABEL="primary" PARTUUID="82a53134-64e9-4918-8e8e-de166145f50f"
/dev/sdf1: PTTYPE="gpt" PARTUUID="4b647143-af50-4e82-ae05-7ad9d694b6cb"
/dev/sdg1: PTTYPE="gpt" PARTLABEL="primary" PARTUUID="b542f6ec-fbdf-437e-9d2f-baa496e0d1e4"
/dev/sdg2: PARTLABEL="primary" PARTUUID="5ca89dab-e827-4301-ab66-10929162b772"
/dev/sdh1: PTTYPE="gpt" PARTLABEL="primary" PARTUUID="b542f6ec-fbdf-437e-9d2f-baa496e0d1e4"
/dev/sdh2: PARTLABEL="primary" PARTUUID="5ca89dab-e827-4301-ab66-10929162b772"
/dev/sdi1: PTTYPE="gpt" PARTUUID="4b647143-af50-4e82-ae05-7ad9d694b6cb"
/dev/mapper/mpathc: PTTYPE="gpt"
/dev/mapper/mpatha: PTTYPE="gpt"
/dev/mapper/mpathb: PTTYPE="gpt"
/dev/mapper/mpathd: PTTYPE="gpt"
/dev/mapper/mpathc1: PTTYPE="gpt" PARTLABEL="primary" PARTUUID="82a53134-64e9-4918-8e8e-de166145f50f"
/dev/mapper/mpatha1: PTTYPE="gpt" PARTUUID="4b647143-af50-4e82-ae05-7ad9d694b6cb"
/dev/mapper/mpathb1: PTTYPE="gpt" PARTLABEL="primary" PARTUUID="b542f6ec-fbdf-437e-9d2f-baa496e0d1e4"
/dev/mapper/mpathd1: PTTYPE="gpt" PARTUUID="5b90fa45-698b-467d-96f2-e5110478f193"
/dev/mapper/mpathb2: PARTLABEL="primary" PARTUUID="5ca89dab-e827-4301-ab66-10929162b772"
/dev/mapper/mpatha1p1: PTTYPE="gpt" PARTLABEL="primary" PARTUUID="7e1262c8-a1db-4cab-be14-06b2d9a5bab3"
/dev/mapper/vg_iscsi_vms-lv_iscsi_vm: PTTYPE="gpt"
/dev/mapper/vg_file01_vdp-lv_file01: PTTYPE="gpt"
/dev/mapper/mpathc1p1: PTTYPE="gpt" PARTLABEL="primary" PARTUUID="94adcc42-771d-4a26-bd23-a8ab944148e1"
/dev/mapper/vg_iscsi_storage-lv_iscsi_storage: PTTYPE="gpt"
/dev/mapper/mpathd1p1: PTTYPE="gpt" PARTLABEL="primary" PARTUUID="ece1e107-6ae7-42db-9c22-a7153d50b6c8"
/dev/mapper/vg_iscsi_vdp-lv_vdp: PTTYPE="gpt"

It shows entries from /dev/sdb1 through to /dev/sdi1 and also my paths, which looks messy due to using kpartx to bring them online. I then turned my attention to trying fstab but did not want to cause system instability so tried using mount first and as I suspected nothing mounts correctly as it is already mounted. For reference here is the output from lvdisplay:

# lvdisplay
--- Logical volume ---
LV Path /dev/vg_iscsi_vdp/lv_vdp
LV Name lv_vdp
VG Name vg_iscsi_vdp
LV UUID SDT2sZ-RMzK-7NRk-0f1Y-OQma-5hQC-kCYVUp
LV Write Access read/write
LV Creation host, time HOSTNAME, 2017-07-24 09:42:05 +0100
LV Status available
# open 1
LV Size 12.73 TiB
Current LE 3337858
Segments 1
Allocation inherit
Read ahead sectors auto
- currently set to 256
Block device 253:22

--- Logical volume ---
LV Path /dev/vg_iscsi_vms/lv_iscsi_vm
LV Name lv_iscsi_vm
VG Name vg_iscsi_vms
LV UUID OtB1Q1-3jiK-UVQF-dqSU-uJ6z-Zsy1-O4cx46
LV Write Access read/write
LV Creation host, time HOSTNAME, 2017-07-19 12:10:09 +0100
LV Status available
# open 1
LV Size <2.73 TiB
Current LE 714539
Segments 1
Allocation inherit
Read ahead sectors auto
- currently set to 256
Block device 253:14

--- Logical volume ---
LV Path /dev/vg_iscsi_storage/lv_iscsi_storage
LV Name lv_iscsi_storage
VG Name vg_iscsi_storage
LV UUID cOjc8O-04DC-Exbw-3acC-2J3Q-cFW8-aV6HkE
LV Write Access read/write
LV Creation host, time HOSTNAME, 2017-07-21 12:55:08 +0100
LV Status available
# open 1
LV Size <9.09 TiB
Current LE 2382255
Segments 1
Allocation inherit
Read ahead sectors auto
- currently set to 256
Block device 253:19

--- Logical volume ---
LV Path /dev/vg_file01_vdp/lv_file01
LV Name lv_file01
VG Name vg_file01_vdp
LV UUID QpPn6z-1zlp-H3so-XNF3-BwtL-JxID-XikQeI
LV Write Access read/write
LV Creation host, time HOSTNAME, 2017-07-19 11:04:10 +0100
LV Status available
# open 1
LV Size <7.26 TiB
Current LE 1903103
Segments 1
Allocation inherit
Read ahead sectors auto
- currently set to 256
Block device 253:16

multipath -v2 command displays nothing, multipath -v3 displays:

Oct 13 11:13:55 | loading /lib64/multipath/libcheckdirectio.so checker
Oct 13 11:13:55 | loading /lib64/multipath/libprioconst.so prioritizer
Oct 13 11:13:55 | sda: not found in pathvec
Oct 13 11:13:55 | sda: device node name blacklisted
Oct 13 11:13:55 | sdf: not found in pathvec
Oct 13 11:13:55 | sdf: mask = 0x3f
Oct 13 11:13:55 | sdf: dev_t = 8:80
Oct 13 11:13:55 | sdf: size = 5854593280
Oct 13 11:13:55 | sdf: vendor = HP
Oct 13 11:13:55 | sdf: product = P2000 G3 FC
Oct 13 11:13:55 | sdf: rev = T250
Oct 13 11:13:55 | sdf: h:b:t:l = 4:0:0:0
Oct 13 11:13:55 | SCSI target 4:0:0 -> FC rport 4:0-0
Oct 13 11:13:55 | sdf: tgt_node_name = 0x208000c0ffdad7a8
Oct 13 11:13:55 | (null): (HP :P2000 G3 FC ) vendor/product whitelisted
Oct 13 11:13:55 | sdf: path state = running

Oct 13 11:13:55 | sdf: 65535 cyl, 255 heads, 63 sectors/track, start at 0
Oct 13 11:13:55 | sdf: serial = 00c0ffdadd070000d959635901000000
Oct 13 11:13:55 | sdf: get_state
Oct 13 11:13:55 | sdf: detect_checker = 1 (config file default)
Oct 13 11:13:55 | loading /lib64/multipath/libchecktur.so checker
Oct 13 11:13:55 | sdf: path checker = tur (controller setting)
Oct 13 11:13:55 | sdf: checker timeout = 30000 ms (sysfs setting)
Oct 13 11:13:55 | sdf: state = up
Oct 13 11:13:55 | sdf: uid_attribute = ID_SERIAL (internal default)
Oct 13 11:13:55 | sdf: uid = 3600c0ff000dadd07d959635901000000 (udev)
Oct 13 11:13:55 | sdf: detect_prio = 1 (config file default)
Oct 13 11:13:55 | loading /lib64/multipath/libprioalua.so prioritizer
Oct 13 11:13:55 | sdf: prio = alua (controller setting)
Oct 13 11:13:55 | sdf: prio args = (null) (controller setting)
Oct 13 11:13:55 | reported target port group is 1
Oct 13 11:13:55 | aas = 01 [active/non-optimized]
Oct 13 11:13:55 | sdf: alua prio = 10
Oct 13 11:13:55 | sdg: not found in pathvec
Oct 13 11:13:55 | sdg: mask = 0x3f
Oct 13 11:13:55 | sdg: dev_t = 8:96
Oct 13 11:13:55 | sdg: size = 15590232192
Oct 13 11:13:55 | sdg: vendor = HP
Oct 13 11:13:55 | sdg: product = P2000 G3 FC
Oct 13 11:13:55 | sdg: rev = T250
Oct 13 11:13:55 | sdg: h:b:t:l = 4:0:1:0
Oct 13 11:13:55 | SCSI target 4:0:1 -> FC rport 4:0-1
Oct 13 11:13:55 | sdg: tgt_node_name = 0x208000c0ff1138a8
Oct 13 11:13:55 | (null): (HP :P2000 G3 FC ) vendor/product whitelisted
Oct 13 11:13:55 | sdg: path state = running

Oct 13 11:13:55 | sdg: 65535 cyl, 255 heads, 63 sectors/track, start at 0
Oct 13 11:13:55 | sdg: serial = 00c0ff12e031000063ea655901000000
Oct 13 11:13:55 | sdg: get_state
Oct 13 11:13:55 | sdg: detect_checker = 1 (config file default)
Oct 13 11:13:55 | sdg: path checker = tur (controller setting)
Oct 13 11:13:55 | sdg: checker timeout = 30000 ms (sysfs setting)
Oct 13 11:13:55 | sdg: state = up
Oct 13 11:13:55 | sdg: uid_attribute = ID_SERIAL (internal default)
Oct 13 11:13:55 | sdg: uid = 3600c0ff00012e03163ea655901000000 (udev)
Oct 13 11:13:55 | sdg: detect_prio = 1 (config file default)
Oct 13 11:13:55 | sdg: prio = alua (controller setting)
Oct 13 11:13:55 | sdg: prio args = (null) (controller setting)
Oct 13 11:13:55 | reported target port group is 0
Oct 13 11:13:55 | aas = 80 [active/optimized] [preferred]
Oct 13 11:13:55 | sdg: alua prio = 50
Oct 13 11:13:55 | sdh: not found in pathvec
Oct 13 11:13:55 | sdh: mask = 0x3f
Oct 13 11:13:55 | sdh: dev_t = 8:112
Oct 13 11:13:55 | sdh: size = 15590232192
Oct 13 11:13:55 | sdh: vendor = HP
Oct 13 11:13:55 | sdh: product = P2000 G3 FC
Oct 13 11:13:55 | sdh: rev = T250
Oct 13 11:13:55 | sdh: h:b:t:l = 4:0:2:0
Oct 13 11:13:55 | SCSI target 4:0:2 -> FC rport 4:0-2
Oct 13 11:13:55 | sdh: tgt_node_name = 0x208000c0ff1138a8
Oct 13 11:13:55 | (null): (HP :P2000 G3 FC ) vendor/product whitelisted
Oct 13 11:13:55 | sdh: path state = running

Oct 13 11:13:55 | sdh: 65535 cyl, 255 heads, 63 sectors/track, start at 0
Oct 13 11:13:55 | sdh: serial = 00c0ff12e031000063ea655901000000
Oct 13 11:13:55 | sdh: get_state
Oct 13 11:13:55 | sdh: detect_checker = 1 (config file default)
Oct 13 11:13:55 | sdh: path checker = tur (controller setting)
Oct 13 11:13:55 | sdh: checker timeout = 30000 ms (sysfs setting)
Oct 13 11:13:55 | sdh: state = up
Oct 13 11:13:55 | sdh: uid_attribute = ID_SERIAL (internal default)
Oct 13 11:13:55 | sdh: uid = 3600c0ff00012e03163ea655901000000 (udev)
Oct 13 11:13:55 | sdh: detect_prio = 1 (config file default)
Oct 13 11:13:55 | sdh: prio = alua (controller setting)
Oct 13 11:13:55 | sdh: prio args = (null) (controller setting)
Oct 13 11:13:55 | reported target port group is 1
Oct 13 11:13:55 | aas = 01 [active/non-optimized]
Oct 13 11:13:55 | sdh: alua prio = 10
Oct 13 11:13:55 | sdi: not found in pathvec
Oct 13 11:13:55 | sdi: mask = 0x3f
Oct 13 11:13:55 | sdi: dev_t = 8:128
Oct 13 11:13:55 | sdi: size = 5854593280
Oct 13 11:13:55 | sdi: vendor = HP
Oct 13 11:13:55 | sdi: product = P2000 G3 FC
Oct 13 11:13:55 | sdi: rev = T250
Oct 13 11:13:55 | sdi: h:b:t:l = 4:0:3:0
Oct 13 11:13:55 | SCSI target 4:0:3 -> FC rport 4:0-3
Oct 13 11:13:55 | sdi: tgt_node_name = 0x208000c0ffdad7a8
Oct 13 11:13:55 | (null): (HP :P2000 G3 FC ) vendor/product whitelisted
Oct 13 11:13:55 | sdi: path state = running

Oct 13 11:13:55 | sdi: 65535 cyl, 255 heads, 63 sectors/track, start at 0
Oct 13 11:13:55 | sdi: serial = 00c0ffdadd070000d959635901000000
Oct 13 11:13:55 | sdi: get_state
Oct 13 11:13:55 | sdi: detect_checker = 1 (config file default)
Oct 13 11:13:55 | sdi: path checker = tur (controller setting)
Oct 13 11:13:55 | sdi: checker timeout = 30000 ms (sysfs setting)
Oct 13 11:13:55 | sdi: state = up
Oct 13 11:13:55 | sdi: uid_attribute = ID_SERIAL (internal default)
Oct 13 11:13:55 | sdi: uid = 3600c0ff000dadd07d959635901000000 (udev)
Oct 13 11:13:55 | sdi: detect_prio = 1 (config file default)
Oct 13 11:13:55 | sdi: prio = alua (controller setting)
Oct 13 11:13:55 | sdi: prio args = (null) (controller setting)
Oct 13 11:13:55 | reported target port group is 0
Oct 13 11:13:55 | aas = 80 [active/optimized] [preferred]
Oct 13 11:13:55 | sdi: alua prio = 50
Oct 13 11:13:55 | sdb: not found in pathvec
Oct 13 11:13:55 | sdb: mask = 0x3f
Oct 13 11:13:55 | sdb: dev_t = 8:16
Oct 13 11:13:55 | sdb: size = 19515446400
Oct 13 11:13:55 | sdb: vendor = HP
Oct 13 11:13:55 | sdb: product = P2000 G3 FC
Oct 13 11:13:55 | sdb: rev = T250
Oct 13 11:13:55 | sdb: h:b:t:l = 1:0:0:0
Oct 13 11:13:55 | SCSI target 1:0:0 -> FC rport 1:0-0
Oct 13 11:13:55 | sdb: tgt_node_name = 0x208000c0ffdad7a8
Oct 13 11:13:55 | (null): (HP :P2000 G3 FC ) vendor/product whitelisted
Oct 13 11:13:55 | sdb: path state = running

Oct 13 11:13:55 | sdb: 65535 cyl, 255 heads, 63 sectors/track, start at 0
Oct 13 11:13:55 | sdb: serial = 00c0ff14ebf50000c757635901000000
Oct 13 11:13:55 | sdb: get_state
Oct 13 11:13:55 | sdb: detect_checker = 1 (config file default)
Oct 13 11:13:55 | sdb: path checker = tur (controller setting)
Oct 13 11:13:55 | sdb: checker timeout = 30000 ms (sysfs setting)
Oct 13 11:13:55 | sdb: state = up
Oct 13 11:13:55 | sdb: uid_attribute = ID_SERIAL (internal default)
Oct 13 11:13:55 | sdb: uid = 3600c0ff00014ebf5c757635901000000 (udev)
Oct 13 11:13:55 | sdb: detect_prio = 1 (config file default)
Oct 13 11:13:55 | sdb: prio = alua (controller setting)
Oct 13 11:13:55 | sdb: prio args = (null) (controller setting)
Oct 13 11:13:55 | reported target port group is 1
Oct 13 11:13:55 | aas = 80 [active/optimized] [preferred]
Oct 13 11:13:55 | sdb: alua prio = 50
Oct 13 11:13:55 | sdc: not found in pathvec
Oct 13 11:13:55 | sdc: mask = 0x3f
Oct 13 11:13:55 | sdc: dev_t = 8:32
Oct 13 11:13:55 | sdc: size = 27343749888
Oct 13 11:13:55 | sdc: vendor = HP
Oct 13 11:13:55 | sdc: product = P2000 G3 FC
Oct 13 11:13:55 | sdc: rev = T250
Oct 13 11:13:55 | sdc: h:b:t:l = 1:0:1:0
Oct 13 11:13:55 | SCSI target 1:0:1 -> FC rport 1:0-1
Oct 13 11:13:55 | sdc: tgt_node_name = 0x208000c0ff1138a8
Oct 13 11:13:55 | (null): (HP :P2000 G3 FC ) vendor/product whitelisted
Oct 13 11:13:55 | sdc: path state = running

Oct 13 11:13:55 | sdc: 65535 cyl, 255 heads, 63 sectors/track, start at 0
Oct 13 11:13:55 | sdc: serial = 00c0ff12e0310000efe9655901000000
Oct 13 11:13:55 | sdc: get_state
Oct 13 11:13:55 | sdc: detect_checker = 1 (config file default)
Oct 13 11:13:55 | sdc: path checker = tur (controller setting)
Oct 13 11:13:55 | sdc: checker timeout = 30000 ms (sysfs setting)
Oct 13 11:13:55 | sdc: state = up
Oct 13 11:13:55 | sdc: uid_attribute = ID_SERIAL (internal default)
Oct 13 11:13:55 | sdc: uid = 3600c0ff00012e031efe9655901000000 (udev)
Oct 13 11:13:55 | sdc: detect_prio = 1 (config file default)
Oct 13 11:13:55 | sdc: prio = alua (controller setting)
Oct 13 11:13:55 | sdc: prio args = (null) (controller setting)
Oct 13 11:13:55 | reported target port group is 0
Oct 13 11:13:55 | aas = 80 [active/optimized] [preferred]
Oct 13 11:13:55 | sdc: alua prio = 50
Oct 13 11:13:55 | sdd: not found in pathvec
Oct 13 11:13:55 | sdd: mask = 0x3f
Oct 13 11:13:55 | sdd: dev_t = 8:48
Oct 13 11:13:55 | sdd: size = 27343749888
Oct 13 11:13:55 | sdd: vendor = HP
Oct 13 11:13:55 | sdd: product = P2000 G3 FC
Oct 13 11:13:55 | sdd: rev = T250
Oct 13 11:13:55 | sdd: h:b:t:l = 1:0:2:0
Oct 13 11:13:55 | SCSI target 1:0:2 -> FC rport 1:0-2
Oct 13 11:13:55 | sdd: tgt_node_name = 0x208000c0ff1138a8
Oct 13 11:13:55 | (null): (HP :P2000 G3 FC ) vendor/product whitelisted
Oct 13 11:13:55 | sdd: path state = running

Oct 13 11:13:55 | sdd: 65535 cyl, 255 heads, 63 sectors/track, start at 0
Oct 13 11:13:55 | sdd: serial = 00c0ff12e0310000efe9655901000000
Oct 13 11:13:55 | sdd: get_state
Oct 13 11:13:55 | sdd: detect_checker = 1 (config file default)
Oct 13 11:13:55 | sdd: path checker = tur (controller setting)
Oct 13 11:13:55 | sdd: checker timeout = 30000 ms (sysfs setting)
Oct 13 11:13:55 | sdd: state = up
Oct 13 11:13:55 | sdd: uid_attribute = ID_SERIAL (internal default)
Oct 13 11:13:55 | sdd: uid = 3600c0ff00012e031efe9655901000000 (udev)
Oct 13 11:13:55 | sdd: detect_prio = 1 (config file default)
Oct 13 11:13:55 | sdd: prio = alua (controller setting)
Oct 13 11:13:55 | sdd: prio args = (null) (controller setting)
Oct 13 11:13:55 | reported target port group is 1
Oct 13 11:13:55 | aas = 01 [active/non-optimized]
Oct 13 11:13:55 | sdd: alua prio = 10
Oct 13 11:13:55 | sde: not found in pathvec
Oct 13 11:13:55 | sde: mask = 0x3f
Oct 13 11:13:55 | sde: dev_t = 8:64
Oct 13 11:13:55 | sde: size = 19515446400
Oct 13 11:13:55 | sde: vendor = HP
Oct 13 11:13:55 | sde: product = P2000 G3 FC
Oct 13 11:13:55 | sde: rev = T250
Oct 13 11:13:55 | sde: h:b:t:l = 1:0:3:0
Oct 13 11:13:55 | SCSI target 1:0:3 -> FC rport 1:0-3
Oct 13 11:13:55 | sde: tgt_node_name = 0x208000c0ffdad7a8
Oct 13 11:13:55 | (null): (HP :P2000 G3 FC ) vendor/product whitelisted
Oct 13 11:13:55 | sde: path state = running

Oct 13 11:13:55 | sde: 65535 cyl, 255 heads, 63 sectors/track, start at 0
Oct 13 11:13:55 | sde: serial = 00c0ff14ebf50000c757635901000000
Oct 13 11:13:55 | sde: get_state
Oct 13 11:13:55 | sde: detect_checker = 1 (config file default)
Oct 13 11:13:55 | sde: path checker = tur (controller setting)
Oct 13 11:13:55 | sde: checker timeout = 30000 ms (sysfs setting)
Oct 13 11:13:55 | sde: state = up
Oct 13 11:13:55 | sde: uid_attribute = ID_SERIAL (internal default)
Oct 13 11:13:55 | sde: uid = 3600c0ff00014ebf5c757635901000000 (udev)
Oct 13 11:13:55 | sde: detect_prio = 1 (config file default)
Oct 13 11:13:55 | sde: prio = alua (controller setting)
Oct 13 11:13:55 | sde: prio args = (null) (controller setting)
Oct 13 11:13:55 | reported target port group is 0
Oct 13 11:13:55 | aas = 01 [active/non-optimized]
Oct 13 11:13:55 | sde: alua prio = 10
Oct 13 11:13:55 | sr0: not found in pathvec
Oct 13 11:13:55 | sr0: device node name blacklisted
Oct 13 11:13:55 | dm-0: not found in pathvec
Oct 13 11:13:55 | dm-0: device node name whitelisted
Oct 13 11:13:55 | dm-0: mask = 0x3f
Oct 13 11:13:55 | dm-0: dev_t = 253:0
Oct 13 11:13:55 | dm-0: size = 104857600
Oct 13 11:13:55 | dm-0: 0 cyl, 0 heads, 0 sectors/track, start at 0
Oct 13 11:13:55 | dm-0: get_state
Oct 13 11:13:55 | dm-0: detect_checker = 1 (config file default)
Oct 13 11:13:55 | loading /lib64/multipath/libcheckreadsector0.so checker
Oct 13 11:13:55 | dm-0: path checker = readsector0 (config file default)
Oct 13 11:13:55 | dm-0: checker timeout = 300000 ms (internal default)
Oct 13 11:13:55 | dm-0: state = down
Oct 13 11:13:55 | dm-0: checker msg is "readsector0 checker reports path is down"
Oct 13 11:13:55 | dm-0: uid_attribute = ID_SERIAL (internal default)
Oct 13 11:13:55 | dm-0: no ID_SERIAL attribute
Oct 13 11:13:55 | dm-0: uid = <empty> (udev)
Oct 13 11:13:55 | dm-1: not found in pathvec
Oct 13 11:13:55 | dm-1: device node name whitelisted
Oct 13 11:13:55 | dm-1: mask = 0x3f
Oct 13 11:13:55 | dm-1: dev_t = 253:1
Oct 13 11:13:55 | dm-1: size = 28672000
Oct 13 11:13:55 | dm-1: 0 cyl, 0 heads, 0 sectors/track, start at 0
Oct 13 11:13:55 | dm-1: get_state
Oct 13 11:13:55 | dm-1: detect_checker = 1 (config file default)
Oct 13 11:13:55 | dm-1: path checker = readsector0 (config file default)
Oct 13 11:13:55 | dm-1: checker timeout = 300000 ms (internal default)
Oct 13 11:13:55 | dm-1: state = down
Oct 13 11:13:55 | dm-1: checker msg is "readsector0 checker reports path is down"
Oct 13 11:13:55 | dm-1: uid_attribute = ID_SERIAL (internal default)
Oct 13 11:13:55 | dm-1: no ID_SERIAL attribute
Oct 13 11:13:55 | dm-1: uid = <empty> (udev)
Oct 13 11:13:55 | dm-10: not found in pathvec
Oct 13 11:13:55 | dm-10: device node name whitelisted
Oct 13 11:13:55 | dm-10: mask = 0x3f
Oct 13 11:13:55 | dm-10: dev_t = 253:10
Oct 13 11:13:55 | dm-10: size = 2014
Oct 13 11:13:55 | dm-10: 0 cyl, 0 heads, 0 sectors/track, start at 0
Oct 13 11:13:55 | dm-10: get_state
Oct 13 11:13:55 | dm-10: detect_checker = 1 (config file default)
Oct 13 11:13:55 | dm-10: path checker = readsector0 (config file default)
Oct 13 11:13:55 | dm-10: checker timeout = 300000 ms (internal default)
Oct 13 11:13:55 | dm-10: state = up
Oct 13 11:13:55 | dm-10: uid_attribute = ID_SERIAL (internal default)
Oct 13 11:13:55 | dm-10: no ID_SERIAL attribute
Oct 13 11:13:55 | dm-10: uid = <empty> (udev)
Oct 13 11:13:55 | dm-11: not found in pathvec
Oct 13 11:13:55 | dm-11: device node name whitelisted
Oct 13 11:13:55 | dm-11: mask = 0x3f
Oct 13 11:13:55 | dm-11: dev_t = 253:11
Oct 13 11:13:55 | dm-11: size = 151044096
Oct 13 11:13:55 | dm-11: 0 cyl, 0 heads, 0 sectors/track, start at 0
Oct 13 11:13:55 | dm-11: get_state
Oct 13 11:13:55 | dm-11: detect_checker = 1 (config file default)
Oct 13 11:13:55 | dm-11: path checker = readsector0 (config file default)
Oct 13 11:13:55 | dm-11: checker timeout = 300000 ms (internal default)
Oct 13 11:13:55 | dm-11: state = down
Oct 13 11:13:55 | dm-11: checker msg is "readsector0 checker reports path is down"
Oct 13 11:13:55 | dm-11: uid_attribute = ID_SERIAL (internal default)
Oct 13 11:13:55 | dm-11: no ID_SERIAL attribute
Oct 13 11:13:55 | dm-11: uid = <empty> (udev)
Oct 13 11:13:55 | dm-12: not found in pathvec
Oct 13 11:13:55 | dm-12: device node name whitelisted
Oct 13 11:13:55 | dm-12: mask = 0x3f
Oct 13 11:13:55 | dm-12: dev_t = 253:12
Oct 13 11:13:55 | dm-12: size = 5853511680
Oct 13 11:13:55 | dm-12: 0 cyl, 0 heads, 0 sectors/track, start at 0
Oct 13 11:13:55 | dm-12: get_state
Oct 13 11:13:55 | dm-12: detect_checker = 1 (config file default)
Oct 13 11:13:55 | dm-12: path checker = readsector0 (config file default)
Oct 13 11:13:55 | dm-12: checker timeout = 300000 ms (internal default)
Oct 13 11:13:55 | dm-12: state = up
Oct 13 11:13:55 | dm-12: uid_attribute = ID_SERIAL (internal default)
Oct 13 11:13:55 | dm-12: no ID_SERIAL attribute
Oct 13 11:13:55 | dm-12: uid = <empty> (udev)
Oct 13 11:13:55 | dm-13: not found in pathvec
Oct 13 11:13:55 | dm-13: device node name whitelisted
Oct 13 11:13:55 | dm-13: mask = 0x3f
Oct 13 11:13:55 | dm-13: dev_t = 253:13
Oct 13 11:13:55 | dm-13: size = 5853507584
Oct 13 11:13:55 | dm-13: 0 cyl, 0 heads, 0 sectors/track, start at 0
Oct 13 11:13:55 | dm-13: get_state
Oct 13 11:13:55 | dm-13: detect_checker = 1 (config file default)
Oct 13 11:13:55 | dm-13: path checker = readsector0 (config file default)
Oct 13 11:13:55 | dm-13: checker timeout = 300000 ms (internal default)
Oct 13 11:13:55 | dm-13: state = up
Oct 13 11:13:55 | dm-13: uid_attribute = ID_SERIAL (internal default)
Oct 13 11:13:55 | dm-13: no ID_SERIAL attribute
Oct 13 11:13:55 | dm-13: uid = <empty> (udev)
Oct 13 11:13:55 | dm-14: not found in pathvec
Oct 13 11:13:55 | dm-14: device node name whitelisted
Oct 13 11:13:55 | dm-14: mask = 0x3f
Oct 13 11:13:55 | dm-14: dev_t = 253:14
Oct 13 11:13:55 | dm-14: size = 5853503488
Oct 13 11:13:55 | dm-14: 0 cyl, 0 heads, 0 sectors/track, start at 0
Oct 13 11:13:55 | dm-14: get_state
Oct 13 11:13:55 | dm-14: detect_checker = 1 (config file default)
Oct 13 11:13:55 | dm-14: path checker = readsector0 (config file default)
Oct 13 11:13:55 | dm-14: checker timeout = 300000 ms (internal default)
Oct 13 11:13:55 | dm-14: state = up
Oct 13 11:13:55 | dm-14: uid_attribute = ID_SERIAL (internal default)
Oct 13 11:13:55 | dm-14: no ID_SERIAL attribute
Oct 13 11:13:55 | dm-14: uid = <empty> (udev)
Oct 13 11:13:55 | dm-15: not found in pathvec
Oct 13 11:13:55 | dm-15: device node name whitelisted
Oct 13 11:13:55 | dm-15: mask = 0x3f
Oct 13 11:13:55 | dm-15: dev_t = 253:15
Oct 13 11:13:55 | dm-15: size = 15590225920
Oct 13 11:13:55 | dm-15: 0 cyl, 0 heads, 0 sectors/track, start at 0
Oct 13 11:13:55 | dm-15: get_state
Oct 13 11:13:55 | dm-15: detect_checker = 1 (config file default)
Oct 13 11:13:55 | dm-15: path checker = readsector0 (config file default)
Oct 13 11:13:55 | dm-15: checker timeout = 300000 ms (internal default)
Oct 13 11:13:55 | dm-15: state = up
Oct 13 11:13:55 | dm-15: uid_attribute = ID_SERIAL (internal default)
Oct 13 11:13:55 | dm-15: no ID_SERIAL attribute
Oct 13 11:13:55 | dm-15: uid = <empty> (udev)
Oct 13 11:13:55 | dm-16: not found in pathvec
Oct 13 11:13:55 | dm-16: device node name whitelisted
Oct 13 11:13:55 | dm-16: mask = 0x3f
Oct 13 11:13:55 | dm-16: dev_t = 253:16
Oct 13 11:13:55 | dm-16: size = 15590219776
Oct 13 11:13:55 | dm-16: 0 cyl, 0 heads, 0 sectors/track, start at 0
Oct 13 11:13:55 | dm-16: get_state
Oct 13 11:13:55 | dm-16: detect_checker = 1 (config file default)
Oct 13 11:13:55 | dm-16: path checker = readsector0 (config file default)
Oct 13 11:13:55 | dm-16: checker timeout = 300000 ms (internal default)
Oct 13 11:13:55 | dm-16: state = up
Oct 13 11:13:55 | dm-16: uid_attribute = ID_SERIAL (internal default)
Oct 13 11:13:55 | dm-16: no ID_SERIAL attribute
Oct 13 11:13:55 | dm-16: uid = <empty> (udev)
Oct 13 11:13:55 | dm-17: not found in pathvec
Oct 13 11:13:55 | dm-17: device node name whitelisted
Oct 13 11:13:55 | dm-17: mask = 0x3f
Oct 13 11:13:55 | dm-17: dev_t = 253:17
Oct 13 11:13:55 | dm-17: size = 19515439104
Oct 13 11:13:55 | dm-17: 0 cyl, 0 heads, 0 sectors/track, start at 0
Oct 13 11:13:55 | dm-17: get_state
Oct 13 11:13:55 | dm-17: detect_checker = 1 (config file default)
Oct 13 11:13:55 | dm-17: path checker = readsector0 (config file default)
Oct 13 11:13:55 | dm-17: checker timeout = 300000 ms (internal default)
Oct 13 11:13:55 | dm-17: state = up
Oct 13 11:13:55 | dm-17: uid_attribute = ID_SERIAL (internal default)
Oct 13 11:13:55 | dm-17: no ID_SERIAL attribute
Oct 13 11:13:55 | dm-17: uid = <empty> (udev)
Oct 13 11:13:55 | dm-18: not found in pathvec
Oct 13 11:13:55 | dm-18: device node name whitelisted
Oct 13 11:13:55 | dm-18: mask = 0x3f
Oct 13 11:13:55 | dm-18: dev_t = 253:18
Oct 13 11:13:55 | dm-18: size = 19515435008
Oct 13 11:13:55 | dm-18: 0 cyl, 0 heads, 0 sectors/track, start at 0
Oct 13 11:13:55 | dm-18: get_state
Oct 13 11:13:55 | dm-18: detect_checker = 1 (config file default)
Oct 13 11:13:55 | dm-18: path checker = readsector0 (config file default)
Oct 13 11:13:55 | dm-18: checker timeout = 300000 ms (internal default)
Oct 13 11:13:55 | dm-18: state = up
Oct 13 11:13:55 | dm-18: uid_attribute = ID_SERIAL (internal default)
Oct 13 11:13:55 | dm-18: no ID_SERIAL attribute
Oct 13 11:13:55 | dm-18: uid = <empty> (udev)
Oct 13 11:13:55 | dm-19: not found in pathvec
Oct 13 11:13:55 | dm-19: device node name whitelisted
Oct 13 11:13:55 | dm-19: mask = 0x3f
Oct 13 11:13:55 | dm-19: dev_t = 253:19
Oct 13 11:13:55 | dm-19: size = 19515432960
Oct 13 11:13:55 | dm-19: 0 cyl, 0 heads, 0 sectors/track, start at 0
Oct 13 11:13:55 | dm-19: get_state
Oct 13 11:13:55 | dm-19: detect_checker = 1 (config file default)
Oct 13 11:13:55 | dm-19: path checker = readsector0 (config file default)
Oct 13 11:13:55 | dm-19: checker timeout = 300000 ms (internal default)
Oct 13 11:13:55 | dm-19: state = up
Oct 13 11:13:55 | dm-19: uid_attribute = ID_SERIAL (internal default)
Oct 13 11:13:55 | dm-19: no ID_SERIAL attribute
Oct 13 11:13:55 | dm-19: uid = <empty> (udev)
Oct 13 11:13:55 | dm-2: not found in pathvec
Oct 13 11:13:55 | dm-2: device node name whitelisted
Oct 13 11:13:55 | dm-2: mask = 0x3f
Oct 13 11:13:55 | dm-2: dev_t = 253:2
Oct 13 11:13:55 | dm-2: size = 19515446400
Oct 13 11:13:55 | dm-2: 65535 cyl, 255 heads, 63 sectors/track, start at 0
Oct 13 11:13:55 | dm-2: get_state
Oct 13 11:13:55 | dm-2: detect_checker = 1 (config file default)
Oct 13 11:13:55 | dm-2: path checker = readsector0 (config file default)
Oct 13 11:13:55 | dm-2: checker timeout = 300000 ms (internal default)
Oct 13 11:13:55 | dm-2: state = up
Oct 13 11:13:55 | dm-2: uid_attribute = ID_SERIAL (internal default)
Oct 13 11:13:55 | dm-2: no ID_SERIAL attribute
Oct 13 11:13:55 | dm-2: uid = <empty> (udev)
Oct 13 11:13:55 | dm-20: not found in pathvec
Oct 13 11:13:55 | dm-20: device node name whitelisted
Oct 13 11:13:55 | dm-20: mask = 0x3f
Oct 13 11:13:55 | dm-20: dev_t = 253:20
Oct 13 11:13:55 | dm-20: size = 27343742976
Oct 13 11:13:55 | dm-20: 0 cyl, 0 heads, 0 sectors/track, start at 0
Oct 13 11:13:55 | dm-20: get_state
Oct 13 11:13:55 | dm-20: detect_checker = 1 (config file default)
Oct 13 11:13:55 | dm-20: path checker = readsector0 (config file default)
Oct 13 11:13:55 | dm-20: checker timeout = 300000 ms (internal default)
Oct 13 11:13:55 | dm-20: state = up
Oct 13 11:13:55 | dm-20: uid_attribute = ID_SERIAL (internal default)
Oct 13 11:13:55 | dm-20: no ID_SERIAL attribute
Oct 13 11:13:55 | dm-20: uid = <empty> (udev)
Oct 13 11:13:55 | dm-21: not found in pathvec
Oct 13 11:13:55 | dm-21: device node name whitelisted
Oct 13 11:13:55 | dm-21: mask = 0x3f
Oct 13 11:13:55 | dm-21: dev_t = 253:21
Oct 13 11:13:55 | dm-21: size = 27343738880
Oct 13 11:13:55 | dm-21: 0 cyl, 0 heads, 0 sectors/track, start at 0
Oct 13 11:13:55 | dm-21: get_state
Oct 13 11:13:55 | dm-21: detect_checker = 1 (config file default)
Oct 13 11:13:55 | dm-21: path checker = readsector0 (config file default)
Oct 13 11:13:55 | dm-21: checker timeout = 300000 ms (internal default)
Oct 13 11:13:55 | dm-21: state = up
Oct 13 11:13:55 | dm-21: uid_attribute = ID_SERIAL (internal default)
Oct 13 11:13:55 | dm-21: no ID_SERIAL attribute
Oct 13 11:13:55 | dm-21: uid = <empty> (udev)
Oct 13 11:13:55 | dm-22: not found in pathvec
Oct 13 11:13:55 | dm-22: device node name whitelisted
Oct 13 11:13:55 | dm-22: mask = 0x3f
Oct 13 11:13:55 | dm-22: dev_t = 253:22
Oct 13 11:13:55 | dm-22: size = 27343732736
Oct 13 11:13:55 | dm-22: 0 cyl, 0 heads, 0 sectors/track, start at 0
Oct 13 11:13:55 | dm-22: get_state
Oct 13 11:13:55 | dm-22: detect_checker = 1 (config file default)
Oct 13 11:13:55 | dm-22: path checker = readsector0 (config file default)
Oct 13 11:13:55 | dm-22: checker timeout = 300000 ms (internal default)
Oct 13 11:13:55 | dm-22: state = up
Oct 13 11:13:55 | dm-22: uid_attribute = ID_SERIAL (internal default)
Oct 13 11:13:55 | dm-22: no ID_SERIAL attribute
Oct 13 11:13:55 | dm-22: uid = <empty> (udev)
Oct 13 11:13:55 | dm-3: not found in pathvec
Oct 13 11:13:55 | dm-3: device node name whitelisted
Oct 13 11:13:55 | dm-3: mask = 0x3f
Oct 13 11:13:55 | dm-3: dev_t = 253:3
Oct 13 11:13:55 | dm-3: size = 5854593280
Oct 13 11:13:55 | dm-3: 65535 cyl, 255 heads, 63 sectors/track, start at 0
Oct 13 11:13:55 | dm-3: get_state
Oct 13 11:13:55 | dm-3: detect_checker = 1 (config file default)
Oct 13 11:13:55 | dm-3: path checker = readsector0 (config file default)
Oct 13 11:13:55 | dm-3: checker timeout = 300000 ms (internal default)
Oct 13 11:13:55 | dm-3: state = up
Oct 13 11:13:55 | dm-3: uid_attribute = ID_SERIAL (internal default)
Oct 13 11:13:55 | dm-3: no ID_SERIAL attribute
Oct 13 11:13:55 | dm-3: uid = <empty> (udev)
Oct 13 11:13:55 | dm-4: not found in pathvec
Oct 13 11:13:55 | dm-4: device node name whitelisted
Oct 13 11:13:55 | dm-4: mask = 0x3f
Oct 13 11:13:55 | dm-4: dev_t = 253:4
Oct 13 11:13:55 | dm-4: size = 15590232192
Oct 13 11:13:55 | dm-4: 65535 cyl, 255 heads, 63 sectors/track, start at 0
Oct 13 11:13:55 | dm-4: get_state
Oct 13 11:13:55 | dm-4: detect_checker = 1 (config file default)
Oct 13 11:13:55 | dm-4: path checker = readsector0 (config file default)
Oct 13 11:13:55 | dm-4: checker timeout = 300000 ms (internal default)
Oct 13 11:13:55 | dm-4: state = up
Oct 13 11:13:55 | dm-4: uid_attribute = ID_SERIAL (internal default)
Oct 13 11:13:55 | dm-4: no ID_SERIAL attribute
Oct 13 11:13:55 | dm-4: uid = <empty> (udev)
Oct 13 11:13:55 | dm-5: not found in pathvec
Oct 13 11:13:55 | dm-5: device node name whitelisted
Oct 13 11:13:55 | dm-5: mask = 0x3f
Oct 13 11:13:55 | dm-5: dev_t = 253:5
Oct 13 11:13:55 | dm-5: size = 27343749888
Oct 13 11:13:55 | dm-5: 65535 cyl, 255 heads, 63 sectors/track, start at 0
Oct 13 11:13:55 | dm-5: get_state
Oct 13 11:13:55 | dm-5: detect_checker = 1 (config file default)
Oct 13 11:13:55 | dm-5: path checker = readsector0 (config file default)
Oct 13 11:13:55 | dm-5: checker timeout = 300000 ms (internal default)
Oct 13 11:13:55 | dm-5: state = up
Oct 13 11:13:55 | dm-5: uid_attribute = ID_SERIAL (internal default)
Oct 13 11:13:55 | dm-5: no ID_SERIAL attribute
Oct 13 11:13:55 | dm-5: uid = <empty> (udev)
Oct 13 11:13:55 | dm-6: not found in pathvec
Oct 13 11:13:55 | dm-6: device node name whitelisted
Oct 13 11:13:55 | dm-6: mask = 0x3f
Oct 13 11:13:55 | dm-6: dev_t = 253:6
Oct 13 11:13:55 | dm-6: size = 19515443200
Oct 13 11:13:55 | dm-6: 0 cyl, 0 heads, 0 sectors/track, start at 0
Oct 13 11:13:55 | dm-6: get_state
Oct 13 11:13:55 | dm-6: detect_checker = 1 (config file default)
Oct 13 11:13:55 | dm-6: path checker = readsector0 (config file default)
Oct 13 11:13:55 | dm-6: checker timeout = 300000 ms (internal default)
Oct 13 11:13:55 | dm-6: state = up
Oct 13 11:13:55 | dm-6: uid_attribute = ID_SERIAL (internal default)
Oct 13 11:13:55 | dm-6: no ID_SERIAL attribute
Oct 13 11:13:55 | dm-6: uid = <empty> (udev)
Oct 13 11:13:55 | dm-7: not found in pathvec
Oct 13 11:13:55 | dm-7: device node name whitelisted
Oct 13 11:13:55 | dm-7: mask = 0x3f
Oct 13 11:13:55 | dm-7: dev_t = 253:7
Oct 13 11:13:55 | dm-7: size = 5853515776
Oct 13 11:13:55 | dm-7: 0 cyl, 0 heads, 0 sectors/track, start at 0
Oct 13 11:13:55 | dm-7: get_state
Oct 13 11:13:55 | dm-7: detect_checker = 1 (config file default)
Oct 13 11:13:55 | dm-7: path checker = readsector0 (config file default)
Oct 13 11:13:55 | dm-7: checker timeout = 300000 ms (internal default)
Oct 13 11:13:55 | dm-7: state = up
Oct 13 11:13:55 | dm-7: uid_attribute = ID_SERIAL (internal default)
Oct 13 11:13:55 | dm-7: no ID_SERIAL attribute
Oct 13 11:13:55 | dm-7: uid = <empty> (udev)
Oct 13 11:13:55 | dm-8: not found in pathvec
Oct 13 11:13:55 | dm-8: device node name whitelisted
Oct 13 11:13:55 | dm-8: mask = 0x3f
Oct 13 11:13:55 | dm-8: dev_t = 253:8
Oct 13 11:13:55 | dm-8: size = 15590230016
Oct 13 11:13:55 | dm-8: 0 cyl, 0 heads, 0 sectors/track, start at 0
Oct 13 11:13:55 | dm-8: get_state
Oct 13 11:13:55 | dm-8: detect_checker = 1 (config file default)
Oct 13 11:13:55 | dm-8: path checker = readsector0 (config file default)
Oct 13 11:13:55 | dm-8: checker timeout = 300000 ms (internal default)
Oct 13 11:13:55 | dm-8: state = up
Oct 13 11:13:55 | dm-8: uid_attribute = ID_SERIAL (internal default)
Oct 13 11:13:55 | dm-8: no ID_SERIAL attribute
Oct 13 11:13:55 | dm-8: uid = <empty> (udev)
Oct 13 11:13:55 | dm-9: not found in pathvec
Oct 13 11:13:55 | dm-9: device node name whitelisted
Oct 13 11:13:55 | dm-9: mask = 0x3f
Oct 13 11:13:55 | dm-9: dev_t = 253:9
Oct 13 11:13:55 | dm-9: size = 27343747807
Oct 13 11:13:55 | dm-9: 0 cyl, 0 heads, 0 sectors/track, start at 0
Oct 13 11:13:55 | dm-9: get_state
Oct 13 11:13:55 | dm-9: detect_checker = 1 (config file default)
Oct 13 11:13:55 | dm-9: path checker = readsector0 (config file default)
Oct 13 11:13:55 | dm-9: checker timeout = 300000 ms (internal default)
Oct 13 11:13:55 | dm-9: state = up
Oct 13 11:13:55 | dm-9: uid_attribute = ID_SERIAL (internal default)
Oct 13 11:13:55 | dm-9: no ID_SERIAL attribute
Oct 13 11:13:55 | dm-9: uid = <empty> (udev)
===== paths list =====
uuid hcil dev dev_t pri dm_st chk_st vend/p
3600c0ff000dadd07d959635901000000 4:0:0:0 sdf 8:80 10 undef undef HP
3600c0ff00012e03163ea655901000000 4:0:1:0 sdg 8:96 50 undef undef HP
3600c0ff00012e03163ea655901000000 4:0:2:0 sdh 8:112 10 undef undef HP
3600c0ff000dadd07d959635901000000 4:0:3:0 sdi 8:128 50 undef undef HP
3600c0ff00014ebf5c757635901000000 1:0:0:0 sdb 8:16 50 undef undef HP
3600c0ff00012e031efe9655901000000 1:0:1:0 sdc 8:32 50 undef undef HP
3600c0ff00012e031efe9655901000000 1:0:2:0 sdd 8:48 10 undef undef HP
3600c0ff00014ebf5c757635901000000 1:0:3:0 sde 8:64 10 undef undef HP
#:#:#:# dm-0 253:0 -1 undef undef ,
#:#:#:# dm-1 253:1 -1 undef undef ,
#:#:#:# dm-10 253:10 -1 undef undef ,
#:#:#:# dm-11 253:11 -1 undef undef ,
#:#:#:# dm-12 253:12 -1 undef undef ,
#:#:#:# dm-13 253:13 -1 undef undef ,
#:#:#:# dm-14 253:14 -1 undef undef ,
#:#:#:# dm-15 253:15 -1 undef undef ,
#:#:#:# dm-16 253:16 -1 undef undef ,
#:#:#:# dm-17 253:17 -1 undef undef ,
#:#:#:# dm-18 253:18 -1 undef undef ,
#:#:#:# dm-19 253:19 -1 undef undef ,
#:#:#:# dm-2 253:2 -1 undef undef ,
#:#:#:# dm-20 253:20 -1 undef undef ,
#:#:#:# dm-21 253:21 -1 undef undef ,
#:#:#:# dm-22 253:22 -1 undef undef ,
#:#:#:# dm-3 253:3 -1 undef undef ,
#:#:#:# dm-4 253:4 -1 undef undef ,
#:#:#:# dm-5 253:5 -1 undef undef ,
#:#:#:# dm-6 253:6 -1 undef undef ,
#:#:#:# dm-7 253:7 -1 undef undef ,
#:#:#:# dm-8 253:8 -1 undef undef ,
#:#:#:# dm-9 253:9 -1 undef undef ,
Oct 13 11:13:55 | params = 1 queue_if_no_path 0 1 1 round-robin 0 2 1 8:32 1 8:48 1
Oct 13 11:13:55 | status = 2 0 0 0 1 1 A 0 2 0 8:32 A 0 8:48 A 0
Oct 13 11:13:55 | mpathd: disassemble map [1 queue_if_no_path 0 1 1 round-robin 0 2 1 8:32 1 8:48 1 ]
Oct 13 11:13:55 | mpathd: disassemble status [2 0 0 0 1 1 A 0 2 0 8:32 A 0 8:48 A 0 ]
Oct 13 11:13:55 | params = 1 queue_if_no_path 0 1 1 round-robin 0 2 1 8:16 1 8:64 1
Oct 13 11:13:55 | status = 2 0 0 0 1 1 A 0 2 0 8:16 A 0 8:64 A 0
Oct 13 11:13:55 | mpathc: disassemble map [1 queue_if_no_path 0 1 1 round-robin 0 2 1 8:16 1 8:64 1 ]
Oct 13 11:13:55 | mpathc: disassemble status [2 0 0 0 1 1 A 0 2 0 8:16 A 0 8:64 A 0 ]
Oct 13 11:13:55 | params = 1 queue_if_no_path 0 1 1 round-robin 0 2 1 8:96 1 8:112 1
Oct 13 11:13:55 | status = 2 0 0 0 1 1 A 0 2 0 8:96 A 0 8:112 A 0
Oct 13 11:13:55 | mpathb: disassemble map [1 queue_if_no_path 0 1 1 round-robin 0 2 1 8:96 1 8:112 1 ]
Oct 13 11:13:55 | mpathb: disassemble status [2 0 0 0 1 1 A 0 2 0 8:96 A 0 8:112 A 0 ]
Oct 13 11:13:55 | params = 1 queue_if_no_path 0 1 1 round-robin 0 2 1 8:80 1 8:128 1
Oct 13 11:13:55 | status = 2 0 0 0 1 1 A 0 2 0 8:80 A 0 8:128 A 0
Oct 13 11:13:55 | mpatha: disassemble map [1 queue_if_no_path 0 1 1 round-robin 0 2 1 8:80 1 8:128 1 ]
Oct 13 11:13:55 | mpatha: disassemble status [2 0 0 0 1 1 A 0 2 0 8:80 A 0 8:128 A 0 ]
Oct 13 11:13:55 | readsector0 checker refcount 23
Oct 13 11:13:55 | readsector0 checker refcount 22
Oct 13 11:13:55 | readsector0 checker refcount 21
Oct 13 11:13:55 | readsector0 checker refcount 20
Oct 13 11:13:55 | readsector0 checker refcount 19
Oct 13 11:13:55 | readsector0 checker refcount 18
Oct 13 11:13:55 | readsector0 checker refcount 17
Oct 13 11:13:55 | readsector0 checker refcount 16
Oct 13 11:13:55 | readsector0 checker refcount 15
Oct 13 11:13:55 | readsector0 checker refcount 14
Oct 13 11:13:55 | readsector0 checker refcount 13
Oct 13 11:13:55 | readsector0 checker refcount 12
Oct 13 11:13:55 | readsector0 checker refcount 11
Oct 13 11:13:55 | readsector0 checker refcount 10
Oct 13 11:13:55 | readsector0 checker refcount 9
Oct 13 11:13:55 | readsector0 checker refcount 8
Oct 13 11:13:55 | readsector0 checker refcount 7
Oct 13 11:13:55 | readsector0 checker refcount 6
Oct 13 11:13:55 | readsector0 checker refcount 5
Oct 13 11:13:55 | readsector0 checker refcount 4
Oct 13 11:13:55 | readsector0 checker refcount 3
Oct 13 11:13:55 | readsector0 checker refcount 2
Oct 13 11:13:55 | readsector0 checker refcount 1
Oct 13 11:13:55 | tur checker refcount 8
Oct 13 11:13:55 | alua prioritizer refcount 8
Oct 13 11:13:55 | tur checker refcount 7
Oct 13 11:13:55 | alua prioritizer refcount 7
Oct 13 11:13:55 | tur checker refcount 6
Oct 13 11:13:55 | alua prioritizer refcount 6
Oct 13 11:13:55 | tur checker refcount 5
Oct 13 11:13:55 | alua prioritizer refcount 5
Oct 13 11:13:55 | tur checker refcount 4
Oct 13 11:13:55 | alua prioritizer refcount 4
Oct 13 11:13:55 | tur checker refcount 3
Oct 13 11:13:55 | alua prioritizer refcount 3
Oct 13 11:13:55 | tur checker refcount 2
Oct 13 11:13:55 | alua prioritizer refcount 2
Oct 13 11:13:55 | tur checker refcount 1
Oct 13 11:13:55 | alua prioritizer refcount 1
Oct 13 11:13:55 | unloading alua prioritizer
Oct 13 11:13:55 | unloading const prioritizer
Oct 13 11:13:55 | unloading readsector0 checker
Oct 13 11:13:55 | unloading tur checker
Oct 13 11:13:55 | unloading directio checker

Interestingly if I go into multipathd -k and look at the paths:

multipathd> show paths
hcil dev dev_t pri dm_st chk_st dev_st next_check
4:0:0:0 sdf 8:80 10 active ready running XXXXXX.... 13/20
4:0:1:0 sdg 8:96 50 active ready running XXXXXXX... 14/20
4:0:2:0 sdh 8:112 10 active ready running XXXXXX.... 13/20
4:0:3:0 sdi 8:128 50 active ready running XXXXXX.... 13/20
1:0:0:0 sdb 8:16 50 active ready running XXXXXXX... 14/20
1:0:1:0 sdc 8:32 50 active ready running XXXXXX.... 13/20
1:0:2:0 sdd 8:48 10 active ready running XXXXXXX... 14/20
1:0:3:0 sde 8:64 10 active ready running XXXXXX.... 13/20
#:#:#:# dm-0 253:0 -1 undef undef unknown orphan
#:#:#:# dm-1 253:1 -1 undef undef unknown orphan
#:#:#:# dm-10 253:10 -1 undef undef unknown orphan
#:#:#:# dm-11 253:11 -1 undef undef unknown orphan
#:#:#:# dm-12 253:12 -1 undef undef unknown orphan
#:#:#:# dm-13 253:13 -1 undef undef unknown orphan
#:#:#:# dm-14 253:14 -1 undef undef unknown orphan
#:#:#:# dm-15 253:15 -1 undef undef unknown orphan
#:#:#:# dm-16 253:16 -1 undef undef unknown orphan
#:#:#:# dm-17 253:17 -1 undef undef unknown orphan
#:#:#:# dm-18 253:18 -1 undef undef unknown orphan
#:#:#:# dm-19 253:19 -1 undef undef unknown orphan
#:#:#:# dm-2 253:2 -1 undef undef unknown orphan
#:#:#:# dm-20 253:20 -1 undef undef unknown orphan
#:#:#:# dm-21 253:21 -1 undef undef unknown orphan
#:#:#:# dm-22 253:22 -1 undef undef unknown orphan
#:#:#:# dm-3 253:3 -1 undef undef unknown orphan
#:#:#:# dm-4 253:4 -1 undef undef unknown orphan
#:#:#:# dm-5 253:5 -1 undef undef unknown orphan
#:#:#:# dm-6 253:6 -1 undef undef unknown orphan
#:#:#:# dm-7 253:7 -1 undef undef unknown orphan
#:#:#:# dm-8 253:8 -1 undef undef unknown orphan
#:#:#:# dm-9 253:9 -1 undef undef unknown orphan

it shows that the /dev/sdb* paths are OK.

I have updated my initramfs with:

dracut --force --add multipath --include /etc/multipath

What am I missing, everything I have read doesnt state that I have to do any more than I have done but yet when I reboot none of my drives are there?????????

Please help?
Top
h1tchiker
Posts: 14
Joined: 2017/08/17 07:40:24

Re: Critical Issues with Multipath losing Config on reboot

Post by h1tchiker » 2017/10/23 14:11:41

OK so no response :(

Are there any developers who look at these forums?

I have taken a look at the journalctl to try and figure this out wth the following results:

I can see the disks get found during the bootup process:

Oct 12 10:09:47 HOSTNAME kernel: sd 1:0:0:0: [sdb] 19515446400 512-byte logical blocks: (9.99 TB/9.08 TiB)
Oct 12 10:09:47 HOSTNAME kernel: sd 1:0:0:0: [sdb] Write Protect is off
Oct 12 10:09:47 HOSTNAME kernel: sd 1:0:0:0: [sdb] Mode Sense: d7 00 00 08
Oct 12 10:09:47 HOSTNAME kernel: sd 1:0:0:0: [sdb] Write cache: enabled, read cache: enabled, doesn't support DPO or FUA
Oct 12 10:09:47 HOSTNAME kernel: sdb: sdb1
Oct 12 10:09:47 HOSTNAME kernel: sd 1:0:0:0: [sdb] Attached SCSI disk
Oct 12 10:09:47 HOSTNAME kernel: scsi 1:0:1:0: Direct-Access HP P2000 G3 FC T250 PQ: 0 ANSI: 5
Oct 12 10:09:47 HOSTNAME kernel: scsi 1:0:1:0: alua: supports implicit TPGS
Oct 12 10:09:47 HOSTNAME kernel: scsi 1:0:1:0: alua: port group 00 rel port 01
Oct 12 10:09:47 HOSTNAME kernel: scsi 1:0:1:0: alua: port group 00 state A preferred supports tOlusNA
Oct 12 10:09:47 HOSTNAME kernel: scsi 1:0:1:0: alua: Attached
Oct 12 10:09:47 HOSTNAME kernel: sd 1:0:1:0: [sdc] 27343749888 512-byte logical blocks: (13.9 TB/12.7 TiB)
Oct 12 10:09:47 HOSTNAME kernel: sd 1:0:1:0: [sdc] Write Protect is off
Oct 12 10:09:47 HOSTNAME kernel: sd 1:0:1:0: [sdc] Mode Sense: d7 00 00 08
Oct 12 10:09:47 HOSTNAME kernel: sd 1:0:1:0: [sdc] Write cache: enabled, read cache: enabled, doesn't support DPO or FUA
Oct 12 10:09:47 HOSTNAME kernel: scsi 1:0:2:0: Direct-Access HP P2000 G3 FC T250 PQ: 0 ANSI: 5
Oct 12 10:09:47 HOSTNAME kernel: sdc: sdc1
Oct 12 10:09:47 HOSTNAME kernel: sr 2:0:0:0: [sr0] scsi3-mmc drive: 24x/24x writer dvd-ram cd/rw xa/form2 cdda tray
Oct 12 10:09:47 HOSTNAME kernel: cdrom: Uniform CD-ROM driver Revision: 3.20
Oct 12 10:09:47 HOSTNAME kernel: sr 2:0:0:0: Attached scsi CD-ROM sr0
Oct 12 10:09:47 HOSTNAME kernel: sd 1:0:1:0: [sdc] Attached SCSI disk
Oct 12 10:09:47 HOSTNAME kernel: scsi 1:0:2:0: alua: supports implicit TPGS
Oct 12 10:09:47 HOSTNAME kernel: scsi 1:0:2:0: alua: port group 01 rel port 05
Oct 12 10:09:47 HOSTNAME kernel: scsi 1:0:2:0: alua: port group 01 state N non-preferred supports tOlusNA
Oct 12 10:09:47 HOSTNAME kernel: scsi 1:0:2:0: alua: Attached
Oct 12 10:09:47 HOSTNAME kernel: sd 1:0:2:0: [sdd] 27343749888 512-byte logical blocks: (13.9 TB/12.7 TiB)
Oct 12 10:09:47 HOSTNAME kernel: sd 1:0:2:0: [sdd] Write Protect is off
Oct 12 10:09:47 HOSTNAME kernel: sd 1:0:2:0: [sdd] Mode Sense: d7 00 00 08
Oct 12 10:09:47 HOSTNAME kernel: sd 1:0:2:0: [sdd] Write cache: enabled, read cache: enabled, doesn't support DPO or FUA
Oct 12 10:09:47 HOSTNAME kernel: scsi 1:0:3:0: Direct-Access HP P2000 G3 FC T250 PQ: 0 ANSI: 5
Oct 12 10:09:47 HOSTNAME kernel: sdd: sdd1
Oct 12 10:09:47 HOSTNAME kernel: sd 1:0:2:0: [sdd] Attached SCSI disk
Oct 12 10:09:47 HOSTNAME kernel: scsi 1:0:3:0: alua: supports implicit TPGS
Oct 12 10:09:47 HOSTNAME kernel: scsi 1:0:3:0: alua: port group 00 rel port 01
Oct 12 10:09:47 HOSTNAME kernel: scsi 1:0:3:0: alua: port group 00 state N non-preferred supports tOlusNA
Oct 12 10:09:47 HOSTNAME kernel: scsi 1:0:3:0: alua: Attached
Oct 12 10:09:47 HOSTNAME kernel: sd 1:0:3:0: [sde] 19515446400 512-byte logical blocks: (9.99 TB/9.08 TiB)
Oct 12 10:09:47 HOSTNAME kernel: sd 1:0:3:0: [sde] Write Protect is off
Oct 12 10:09:47 HOSTNAME kernel: sd 1:0:3:0: [sde] Mode Sense: d7 00 00 08
Oct 12 10:09:47 HOSTNAME kernel: sd 1:0:3:0: [sde] Write cache: enabled, read cache: enabled, doesn't support DPO or FUA
Oct 12 10:09:47 HOSTNAME kernel: sde: sde1
Oct 12 10:09:47 HOSTNAME kernel: sd 1:0:3:0: [sde] Attached SCSI disk
Oct 12 10:09:47 HOSTNAME kernel: scsi 4:0:0:0: Direct-Access HP P2000 G3 FC T250 PQ: 0 ANSI: 5
Oct 12 10:09:47 HOSTNAME kernel: scsi 4:0:0:0: alua: supports implicit TPGS
Oct 12 10:09:47 HOSTNAME kernel: scsi 4:0:0:0: alua: port group 01 rel port 06
Oct 12 10:09:47 HOSTNAME kernel: scsi 4:0:0:0: alua: port group 01 state N non-preferred supports tOlusNA
Oct 12 10:09:47 HOSTNAME kernel: scsi 4:0:0:0: alua: Attached
Oct 12 10:09:47 HOSTNAME kernel: sd 4:0:0:0: [sdf] 5854593280 512-byte logical blocks: (2.99 TB/2.72 TiB)
Oct 12 10:09:47 HOSTNAME kernel: sd 4:0:0:0: [sdf] Write Protect is off
Oct 12 10:09:47 HOSTNAME kernel: sd 4:0:0:0: [sdf] Mode Sense: d7 00 00 08
Oct 12 10:09:47 HOSTNAME kernel: sd 4:0:0:0: [sdf] Write cache: enabled, read cache: enabled, doesn't support DPO or FUA
Oct 12 10:09:47 HOSTNAME kernel: scsi 4:0:1:0: Direct-Access HP P2000 G3 FC T250 PQ: 0 ANSI: 5
Oct 12 10:09:47 HOSTNAME kernel: sdf: sdf1
Oct 12 10:09:47 HOSTNAME kernel: sd 4:0:0:0: [sdf] Attached SCSI disk
Oct 12 10:09:47 HOSTNAME kernel: scsi 4:0:1:0: alua: supports implicit TPGS
Oct 12 10:09:47 HOSTNAME kernel: scsi 4:0:1:0: alua: port group 00 rel port 02
Oct 12 10:09:47 HOSTNAME kernel: scsi 4:0:1:0: alua: port group 00 state A preferred supports tOlusNA
Oct 12 10:09:47 HOSTNAME kernel: scsi 4:0:1:0: alua: Attached
Oct 12 10:09:47 HOSTNAME kernel: sd 4:0:1:0: [sdg] 15590232192 512-byte logical blocks: (7.98 TB/7.25 TiB)
Oct 12 10:09:47 HOSTNAME kernel: sd 4:0:1:0: [sdg] Write Protect is off
Oct 12 10:09:47 HOSTNAME kernel: sd 4:0:1:0: [sdg] Mode Sense: d7 00 00 08
Oct 12 10:09:47 HOSTNAME kernel: sd 4:0:1:0: [sdg] Write cache: enabled, read cache: enabled, doesn't support DPO or FUA
Oct 12 10:09:47 HOSTNAME kernel: scsi 4:0:2:0: Direct-Access HP P2000 G3 FC T250 PQ: 0 ANSI: 5
Oct 12 10:09:47 HOSTNAME kernel: sdg: sdg1 sdg2
Oct 12 10:09:47 HOSTNAME kernel: sd 4:0:1:0: [sdg] Attached SCSI disk
Oct 12 10:09:47 HOSTNAME kernel: scsi 4:0:2:0: alua: supports implicit TPGS
Oct 12 10:09:47 HOSTNAME kernel: scsi 4:0:2:0: alua: port group 01 rel port 06
Oct 12 10:09:47 HOSTNAME kernel: scsi 4:0:2:0: alua: port group 01 state N non-preferred supports tOlusNA
Oct 12 10:09:47 HOSTNAME kernel: scsi 4:0:2:0: alua: Attached
Oct 12 10:09:47 HOSTNAME kernel: sd 4:0:2:0: [sdh] 15590232192 512-byte logical blocks: (7.98 TB/7.25 TiB)
Oct 12 10:09:47 HOSTNAME kernel: sd 4:0:2:0: [sdh] Write Protect is off
Oct 12 10:09:47 HOSTNAME kernel: sd 4:0:2:0: [sdh] Mode Sense: d7 00 00 08
Oct 12 10:09:47 HOSTNAME kernel: sd 4:0:2:0: [sdh] Write cache: enabled, read cache: enabled, doesn't support DPO or FUA
Oct 12 10:09:47 HOSTNAME kernel: scsi 4:0:3:0: Direct-Access HP P2000 G3 FC T250 PQ: 0 ANSI: 5
Oct 12 10:09:47 HOSTNAME kernel: sdh: sdh1 sdh2
Oct 12 10:09:47 HOSTNAME kernel: sd 4:0:2:0: [sdh] Attached SCSI disk
Oct 12 10:09:47 HOSTNAME kernel: scsi 4:0:3:0: alua: supports implicit TPGS
Oct 12 10:09:47 HOSTNAME kernel: scsi 4:0:3:0: alua: port group 00 rel port 02
Oct 12 10:09:47 HOSTNAME kernel: scsi 4:0:3:0: alua: port group 00 state A preferred supports tOlusNA
Oct 12 10:09:47 HOSTNAME kernel: scsi 4:0:3:0: alua: Attached
Oct 12 10:09:47 HOSTNAME kernel: sd 4:0:3:0: [sdi] 5854593280 512-byte logical blocks: (2.99 TB/2.72 TiB)
Oct 12 10:09:47 HOSTNAME kernel: sd 4:0:3:0: [sdi] Write Protect is off
Oct 12 10:09:47 HOSTNAME kernel: sd 4:0:3:0: [sdi] Mode Sense: d7 00 00 08
Oct 12 10:09:47 HOSTNAME kernel: sd 4:0:3:0: [sdi] Write cache: enabled, read cache: enabled, doesn't support DPO or FUA
Oct 12 10:09:47 HOSTNAME kernel: random: crng init done
Oct 12 10:09:47 HOSTNAME kernel: sdi: sdi1
Oct 12 10:09:47 HOSTNAME kernel: sd 4:0:3:0: [sdi] Attached SCSI disk

Then multipathd joins the party:

Oct 12 10:09:50 HOSTNAME systemd[1]: Started Device-Mapper Multipath Device Controller.
Oct 12 10:09:50 HOSTNAME multipathd[697]: path checkers start up

Oct 12 10:09:52 HOSTNAME multipathd[697]: mpathc: load table [0 19515446400 multipath 1 queue_if_no_path 0 1 1 round-robin 0 1 1 8:16 1]
Oct 12 10:09:52 HOSTNAME multipathd[697]: mpathc: event checker started
Oct 12 10:09:52 HOSTNAME multipathd[697]: sdb [8:16]: path added to devmap mpathc
Oct 12 10:09:52 HOSTNAME multipathd[697]: sdi: add path (uevent)
Oct 12 10:09:52 HOSTNAME multipathd[697]: sdi: spurious uevent, path already in pathvec
Oct 12 10:09:52 HOSTNAME multipathd[697]: mpatha: load table [0 5854593280 multipath 1 queue_if_no_path 0 1 1 round-robin 0 1 1 8:128 1]
Oct 12 10:09:52 HOSTNAME multipathd[697]: mpatha: event checker started
Oct 12 10:09:52 HOSTNAME multipathd[697]: sdi [8:128]: path added to devmap mpatha
Oct 12 10:09:52 HOSTNAME multipathd[697]: sdh: add path (uevent)
Oct 12 10:09:52 HOSTNAME multipathd[697]: sdh: spurious uevent, path already in pathvec
Oct 12 10:09:52 HOSTNAME multipathd[697]: mpathb: load table [0 15590232192 multipath 1 queue_if_no_path 0 1 1 round-robin 0 1 1 8:112 1]
Oct 12 10:09:52 HOSTNAME multipathd[697]: mpathb: event checker started
Oct 12 10:09:52 HOSTNAME multipathd[697]: sdh [8:112]: path added to devmap mpathb
Oct 12 10:09:52 HOSTNAME multipathd[697]: sde: add path (uevent)
Oct 12 10:09:52 HOSTNAME multipathd[697]: sde: spurious uevent, path already in pathvec
Oct 12 10:09:52 HOSTNAME multipathd[697]: sde : delaying path addition until mpathc is fully initialized
Oct 12 10:09:52 HOSTNAME multipathd[697]: sdg: add path (uevent)
Oct 12 10:09:52 HOSTNAME multipathd[697]: sdg: spurious uevent, path already in pathvec
Oct 12 10:09:52 HOSTNAME multipathd[697]: sdg : delaying path addition until mpathb is fully initialized
Oct 12 10:09:52 HOSTNAME multipathd[697]: sdd: add path (uevent)
Oct 12 10:09:52 HOSTNAME multipathd[697]: sdd: spurious uevent, path already in pathvec
Oct 12 10:09:52 HOSTNAME multipathd[697]: mpathd: load table [0 27343749888 multipath 1 queue_if_no_path 0 1 1 round-robin 0 1 1 8:48 1]
Oct 12 10:09:52 HOSTNAME multipathd[697]: mpathd: event checker started
Oct 12 10:09:52 HOSTNAME multipathd[697]: sdd [8:48]: path added to devmap mpathd
Oct 12 10:09:52 HOSTNAME multipathd[697]: sdf: add path (uevent)
Oct 12 10:09:52 HOSTNAME multipathd[697]: sdf: spurious uevent, path already in pathvec
Oct 12 10:09:52 HOSTNAME multipathd[697]: sdf : delaying path addition until mpatha is fully initialized
Oct 12 10:09:52 HOSTNAME multipathd[697]: sdc: add path (uevent)
Oct 12 10:09:52 HOSTNAME multipathd[697]: sdc: spurious uevent, path already in pathvec
Oct 12 10:09:52 HOSTNAME multipathd[697]: sdc : delaying path addition until mpathd is fully initialized
Oct 12 10:09:52 HOSTNAME kernel: ipmi_si 0000:01:04.6: Found new BMC (man_id: 0x00000b, prod_id: 0x2000, dev_id: 0x11)
Oct 12 10:09:52 HOSTNAME kernel: ipmi_si 0000:01:04.6: IPMI kcs interface initialized
Oct 12 10:09:52 HOSTNAME kernel: IPMI SSIF Interface driver
Oct 12 10:09:52 HOSTNAME multipathd[697]: sda: add path (uevent)
Oct 12 10:09:52 HOSTNAME multipathd[697]: sda: spurious uevent, path already in pathvec
Oct 12 10:09:52 HOSTNAME systemd[1]: Created slice system-lvm2\x2dpvscan.slice.
Oct 12 10:09:52 HOSTNAME systemd[1]: Starting system-lvm2\x2dpvscan.slice.
Oct 12 10:09:52 HOSTNAME systemd[1]: Starting LVM2 PV scan on device 8:2...
Oct 12 10:09:52 HOSTNAME lvm[870]: WARNING: lvmetad is being updated, retrying (setup) for 10 more seconds.
Oct 12 10:09:52 HOSTNAME multipathd[697]: mpathc: performing delayed actions
Oct 12 10:09:52 HOSTNAME multipathd[697]: mpathc: load table [0 19515446400 multipath 1 queue_if_no_path 0 2 1 round-robin 0 1 1 8:16 1 round-robin 0 1 1 8:64 1]
Oct 12 10:09:52 HOSTNAME multipathd[697]: mpatha: performing delayed actions
Oct 12 10:09:52 HOSTNAME multipathd[697]: mpatha: load table [0 5854593280 multipath 1 queue_if_no_path 0 2 1 round-robin 0 1 1 8:128 1 round-robin 0 1 1 8:80 1]
Oct 12 10:09:52 HOSTNAME multipathd[697]: mpathd: performing delayed actions
Oct 12 10:09:52 HOSTNAME multipathd[697]: mpathd: load table [0 27343749888 multipath 1 queue_if_no_path 0 2 1 round-robin 0 1 1 8:32 1 round-robin 0 1 1 8:48 1]
Oct 12 10:09:52 HOSTNAME multipathd[697]: mpathb: performing delayed actions
Oct 12 10:09:52 HOSTNAME multipathd[697]: mpathb: load table [0 15590232192 multipath 1 queue_if_no_path 0 2 1 round-robin 0 1 1 8:96 1 round-robin 0 1 1 8:112 1]

So, what ever ‘spurious uevent, path already in pathvec’ means appears to be the issue as the partitions never load, any ideas?
Top
h1tchiker
Posts: 14
Joined: 2017/08/17 07:40:24

Re: Critical Issues with Multipath losing Config on reboot

Post by h1tchiker » 2017/10/24 12:00:11

Aren't there any developers who monitor these forums? Have I posted in the wrong forum or some thing, I mean it respectfully because I understand how these forum's work, but the lack of response from the CentOS team is baffling to me!

Yet another day of talking to myself about this issue :(

I appreciate any response on this. I checked the journalctl and found the following:

#journalctl --boot

Oct 24 09:10:51 HOSTNAME kernel: sd 1:0:0:0: [sdb] 19515446400 512-byte logical blocks: (9.99 TB/9.08 TiB)
Oct 24 09:10:51 HOSTNAME kernel: scsi 1:0:1:0: Direct-Access HP P2000 G3 FC T250 PQ: 0 ANSI: 5
Oct 24 09:10:51 HOSTNAME kernel: sd 1:0:0:0: [sdb] Write Protect is off
Oct 24 09:10:51 HOSTNAME kernel: sd 1:0:0:0: [sdb] Mode Sense: d7 00 00 08
Oct 24 09:10:51 HOSTNAME kernel: sd 1:0:0:0: [sdb] Write cache: enabled, read cache: enabled, doesn't support DPO or FUA
Oct 24 09:10:51 HOSTNAME kernel: scsi 1:0:1:0: alua: supports implicit TPGS
Oct 24 09:10:51 HOSTNAME kernel: scsi 1:0:1:0: alua: port group 00 rel port 01
Oct 24 09:10:51 HOSTNAME kernel: scsi 1:0:1:0: alua: port group 00 state A preferred supports tOlusNA
Oct 24 09:10:51 HOSTNAME kernel: scsi 1:0:1:0: alua: Attached
Oct 24 09:10:51 HOSTNAME kernel: sd 1:0:1:0: [sdc] 27343749888 512-byte logical blocks: (13.9 TB/12.7 TiB)
Oct 24 09:10:51 HOSTNAME kernel: sd 1:0:1:0: [sdc] Write Protect is off
Oct 24 09:10:51 HOSTNAME kernel: sd 1:0:1:0: [sdc] Mode Sense: d7 00 00 08
Oct 24 09:10:51 HOSTNAME kernel: sd 1:0:1:0: [sdc] Write cache: enabled, read cache: enabled, doesn't support DPO or FUA
Oct 24 09:10:51 HOSTNAME kernel: sdc: sdc1
Oct 24 09:10:51 HOSTNAME kernel: scsi 1:0:2:0: Direct-Access HP P2000 G3 FC T250 PQ: 0 ANSI: 5
Oct 24 09:10:51 HOSTNAME kernel: sd 1:0:1:0: [sdc] Attached SCSI disk
Oct 24 09:10:51 HOSTNAME kernel: sr 2:0:0:0: [sr0] scsi3-mmc drive: 24x/24x writer dvd-ram cd/rw xa/form2 cdda tray
Oct 24 09:10:51 HOSTNAME kernel: cdrom: Uniform CD-ROM driver Revision: 3.20
Oct 24 09:10:51 HOSTNAME kernel: sr 2:0:0:0: Attached scsi CD-ROM sr0
Oct 24 09:10:51 HOSTNAME kernel: sdb: sdb1
Oct 24 09:10:51 HOSTNAME kernel: sd 1:0:0:0: [sdb] Attached SCSI disk
Oct 24 09:10:51 HOSTNAME multipathd[389]: sdc: add path (uevent)
Oct 24 09:10:51 HOSTNAME systemd-udevd[346]: inotify_add_watch(6, /dev/sdc1, 10) failed: No such file or directory
Oct 24 09:10:51 HOSTNAME systemd-udevd[344]: inotify_add_watch(6, /dev/sdb1, 10) failed: No such file or directory
Oct 24 09:10:51 HOSTNAME kernel: device-mapper: multipath service-time: version 0.3.0 loaded
Oct 24 09:10:51 HOSTNAME multipathd[389]: mpathd: load table [0 27343749888 multipath 1 queue_if_no_path 0 1 1 service-time 0 1 1 8:32 1]
Oct 24 09:10:51 HOSTNAME multipathd[389]: mpathd: event checker started
Oct 24 09:10:51 HOSTNAME multipathd[389]: sdc [8:32]: path added to devmap mpathd
Oct 24 09:10:51 HOSTNAME multipathd[389]: sdb: add path (uevent)
Oct 24 09:10:51 HOSTNAME multipathd[389]: mpathc: load table [0 19515446400 multipath 1 queue_if_no_path 0 1 1 service-time 0 1 1 8:16 1]
Oct 24 09:10:51 HOSTNAME multipathd[389]: mpathc: event checker started
Oct 24 09:10:51 HOSTNAME multipathd[389]: sdb [8:16]: path added to devmap mpathc
Oct 24 09:10:51 HOSTNAME kernel: scsi 1:0:2:0: alua: supports implicit TPGS
Oct 24 09:10:51 HOSTNAME kernel: scsi 1:0:2:0: alua: port group 01 rel port 05
Oct 24 09:10:51 HOSTNAME kernel: scsi 1:0:2:0: alua: port group 01 state N non-preferred supports tOlusNA
Oct 24 09:10:51 HOSTNAME kernel: scsi 1:0:2:0: alua: Attached
Oct 24 09:10:51 HOSTNAME kernel: sd 1:0:2:0: [sdd] 27343749888 512-byte logical blocks: (13.9 TB/12.7 TiB)
Oct 24 09:10:51 HOSTNAME kernel: sd 1:0:2:0: [sdd] Write Protect is off
Oct 24 09:10:51 HOSTNAME kernel: sd 1:0:2:0: [sdd] Mode Sense: d7 00 00 08
Oct 24 09:10:51 HOSTNAME kernel: sd 1:0:2:0: [sdd] Write cache: enabled, read cache: enabled, doesn't support DPO or FUA
Oct 24 09:10:51 HOSTNAME kernel: scsi 1:0:3:0: Direct-Access HP P2000 G3 FC T250 PQ: 0 ANSI: 5
Oct 24 09:10:51 HOSTNAME kernel: sdd: sdd1
Oct 24 09:10:51 HOSTNAME kernel: sd 1:0:2:0: [sdd] Attached SCSI disk
Oct 24 09:10:51 HOSTNAME kernel: scsi 1:0:3:0: alua: supports implicit TPGS
Oct 24 09:10:51 HOSTNAME kernel: scsi 1:0:3:0: alua: port group 00 rel port 01
Oct 24 09:10:51 HOSTNAME kernel: scsi 1:0:3:0: alua: port group 00 state N non-preferred supports tOlusNA
Oct 24 09:10:51 HOSTNAME kernel: scsi 1:0:3:0: alua: Attached
Oct 24 09:10:51 HOSTNAME kernel: sd 1:0:3:0: [sde] 19515446400 512-byte logical blocks: (9.99 TB/9.08 TiB)
Oct 24 09:10:51 HOSTNAME multipathd[389]: sdd: add path (uevent)
Oct 24 09:10:51 HOSTNAME systemd-udevd[348]: inotify_add_watch(6, /dev/sdd1, 10) failed: No such file or directory
Oct 24 09:10:51 HOSTNAME kernel: sd 1:0:3:0: [sde] Write Protect is off
Oct 24 09:10:51 HOSTNAME kernel: sd 1:0:3:0: [sde] Mode Sense: d7 00 00 08
Oct 24 09:10:51 HOSTNAME kernel: sd 1:0:3:0: [sde] Write cache: enabled, read cache: enabled, doesn't support DPO or FUA
Oct 24 09:10:51 HOSTNAME multipathd[389]: mpathd: load table [0 27343749888 multipath 1 queue_if_no_path 0 1 1 service-time 0 2 1 8:32 1 8:48 1]
Oct 24 09:10:51 HOSTNAME multipathd[389]: sdd [8:48]: path added to devmap mpathd
Oct 24 09:10:51 HOSTNAME kernel: sde: sde1
Oct 24 09:10:51 HOSTNAME kernel: sd 1:0:3:0: [sde] Attached SCSI disk
Oct 24 09:10:51 HOSTNAME kernel: random: crng init done
Oct 24 09:10:51 HOSTNAME kernel: scsi 4:0:0:0: Direct-Access HP P2000 G3 FC T250 PQ: 0 ANSI: 5
Oct 24 09:10:51 HOSTNAME kernel: scsi 4:0:0:0: alua: supports implicit TPGS
Oct 24 09:10:51 HOSTNAME kernel: scsi 4:0:0:0: alua: port group 01 rel port 06
Oct 24 09:10:51 HOSTNAME kernel: scsi 4:0:0:0: alua: port group 01 state N non-preferred supports tOlusNA
Oct 24 09:10:51 HOSTNAME kernel: scsi 4:0:0:0: alua: Attached
Oct 24 09:10:51 HOSTNAME multipathd[389]: sde: add path (uevent)
Oct 24 09:10:51 HOSTNAME systemd-udevd[344]: inotify_add_watch(6, /dev/sde1, 10) failed: No such file or directory
Oct 24 09:10:51 HOSTNAME kernel: sd 4:0:0:0: [sdf] 5854593280 512-byte logical blocks: (2.99 TB/2.72 TiB)
Oct 24 09:10:51 HOSTNAME kernel: sd 4:0:0:0: [sdf] Write Protect is off
Oct 24 09:10:51 HOSTNAME kernel: sd 4:0:0:0: [sdf] Mode Sense: d7 00 00 08
Oct 24 09:10:51 HOSTNAME kernel: sd 4:0:0:0: [sdf] Write cache: enabled, read cache: enabled, doesn't support DPO or FUA
Oct 24 09:10:51 HOSTNAME multipathd[389]: mpathc: load table [0 19515446400 multipath 1 queue_if_no_path 0 1 1 service-time 0 2 1 8:16 1 8:64 1]
Oct 24 09:10:51 HOSTNAME multipathd[389]: sde [8:64]: path added to devmap mpathc
Oct 24 09:10:51 HOSTNAME kernel: scsi 4:0:1:0: Direct-Access HP P2000 G3 FC T250 PQ: 0 ANSI: 5
Oct 24 09:10:51 HOSTNAME kernel: sdf: sdf1
Oct 24 09:10:51 HOSTNAME kernel: sd 4:0:0:0: [sdf] Attached SCSI disk
Oct 24 09:10:51 HOSTNAME kernel: scsi 4:0:1:0: alua: supports implicit TPGS
Oct 24 09:10:51 HOSTNAME kernel: scsi 4:0:1:0: alua: port group 00 rel port 02
Oct 24 09:10:51 HOSTNAME kernel: scsi 4:0:1:0: alua: port group 00 state A preferred supports tOlusNA
Oct 24 09:10:51 HOSTNAME kernel: scsi 4:0:1:0: alua: Attached
Oct 24 09:10:51 HOSTNAME kernel: sd 4:0:1:0: [sdg] 15590232192 512-byte logical blocks: (7.98 TB/7.25 TiB)
Oct 24 09:10:51 HOSTNAME kernel: sd 4:0:1:0: [sdg] Write Protect is off
Oct 24 09:10:51 HOSTNAME kernel: sd 4:0:1:0: [sdg] Mode Sense: d7 00 00 08
Oct 24 09:10:51 HOSTNAME kernel: sd 4:0:1:0: [sdg] Write cache: enabled, read cache: enabled, doesn't support DPO or FUA
Oct 24 09:10:51 HOSTNAME kernel: scsi 4:0:2:0: Direct-Access HP P2000 G3 FC T250 PQ: 0 ANSI: 5
Oct 24 09:10:51 HOSTNAME kernel: sdg: sdg1 sdg2
Oct 24 09:10:51 HOSTNAME kernel: sd 4:0:1:0: [sdg] Attached SCSI disk
Oct 24 09:10:51 HOSTNAME multipathd[389]: sdg: add path (uevent)
Oct 24 09:10:51 HOSTNAME systemd-udevd[348]: inotify_add_watch(6, /dev/sdg1, 10) failed: No such file or directory
Oct 24 09:10:51 HOSTNAME kernel: scsi 4:0:2:0: alua: supports implicit TPGS
Oct 24 09:10:51 HOSTNAME kernel: scsi 4:0:2:0: alua: port group 01 rel port 06
Oct 24 09:10:51 HOSTNAME kernel: scsi 4:0:2:0: alua: port group 01 state N non-preferred supports tOlusNA
Oct 24 09:10:51 HOSTNAME kernel: scsi 4:0:2:0: alua: Attached
Oct 24 09:10:51 HOSTNAME kernel: sd 4:0:2:0: [sdh] 15590232192 512-byte logical blocks: (7.98 TB/7.25 TiB)
Oct 24 09:10:51 HOSTNAME systemd-udevd[349]: inotify_add_watch(6, /dev/sdg2, 10) failed: No such file or directory
Oct 24 09:10:51 HOSTNAME systemd-udevd[344]: inotify_add_watch(6, /dev/sdf1, 10) failed: No such file or directory
Oct 24 09:10:51 HOSTNAME kernel: sd 4:0:2:0: [sdh] Write Protect is off
Oct 24 09:10:51 HOSTNAME kernel: sd 4:0:2:0: [sdh] Mode Sense: d7 00 00 08
Oct 24 09:10:51 HOSTNAME kernel: sd 4:0:2:0: [sdh] Write cache: enabled, read cache: enabled, doesn't support DPO or FUA
Oct 24 09:10:51 HOSTNAME multipathd[389]: mpathb: load table [0 15590232192 multipath 1 queue_if_no_path 0 1 1 service-time 0 1 1 8:96 1]
Oct 24 09:10:51 HOSTNAME multipathd[389]: mpathb: event checker started
Oct 24 09:10:51 HOSTNAME multipathd[389]: sdg [8:96]: path added to devmap mpathb
Oct 24 09:10:51 HOSTNAME multipathd[389]: sdf: add path (uevent)
Oct 24 09:10:51 HOSTNAME kernel: scsi 4:0:3:0: Direct-Access HP P2000 G3 FC T250 PQ: 0 ANSI: 5
Oct 24 09:10:51 HOSTNAME kernel: sdh: sdh1 sdh2
Oct 24 09:10:51 HOSTNAME kernel: sd 4:0:2:0: [sdh] Attached SCSI disk
Oct 24 09:10:51 HOSTNAME multipathd[389]: mpatha: load table [0 5854593280 multipath 1 queue_if_no_path 0 1 1 service-time 0 1 1 8:80 1]
Oct 24 09:10:51 HOSTNAME multipathd[389]: mpatha: event checker started
Oct 24 09:10:51 HOSTNAME multipathd[389]: sdf [8:80]: path added to devmap mpatha
Oct 24 09:10:51 HOSTNAME multipathd[389]: sdh: add path (uevent)
Oct 24 09:10:51 HOSTNAME kernel: scsi 4:0:3:0: alua: supports implicit TPGS
Oct 24 09:10:51 HOSTNAME kernel: scsi 4:0:3:0: alua: port group 00 rel port 02
Oct 24 09:10:51 HOSTNAME kernel: scsi 4:0:3:0: alua: port group 00 state A preferred supports tOlusNA
Oct 24 09:10:51 HOSTNAME kernel: scsi 4:0:3:0: alua: Attached
Oct 24 09:10:51 HOSTNAME systemd-udevd[346]: inotify_add_watch(6, /dev/sdh1, 10) failed: No such file or directory
Oct 24 09:10:51 HOSTNAME systemd-udevd[352]: inotify_add_watch(6, /dev/sdh2, 10) failed: No such file or directory
Oct 24 09:10:51 HOSTNAME multipathd[389]: sdh : delaying path addition until mpathb is fully initialized
Oct 24 09:10:51 HOSTNAME multipathd[389]: mpathb: performing delayed actions
Oct 24 09:10:51 HOSTNAME kernel: sd 4:0:3:0: [sdi] 5854593280 512-byte logical blocks: (2.99 TB/2.72 TiB)
Oct 24 09:10:51 HOSTNAME multipathd[389]: mpathb: load table [0 15590232192 multipath 1 queue_if_no_path 0 1 1 service-time 0 2 1 8:96 1 8:112 1]
Oct 24 09:10:51 HOSTNAME kernel: sd 4:0:3:0: [sdi] Write Protect is off
Oct 24 09:10:51 HOSTNAME kernel: sd 4:0:3:0: [sdi] Mode Sense: d7 00 00 08
Oct 24 09:10:51 HOSTNAME kernel: sd 4:0:3:0: [sdi] Write cache: enabled, read cache: enabled, doesn't support DPO or FUA
Oct 24 09:10:51 HOSTNAME kernel: sdi: sdi1
Oct 24 09:10:51 HOSTNAME kernel: sd 4:0:3:0: [sdi] Attached SCSI disk
Oct 24 09:10:51 HOSTNAME multipathd[389]: sdi: add path (uevent)
Oct 24 09:10:51 HOSTNAME systemd-udevd[345]: inotify_add_watch(6, /dev/sdi1, 10) failed: No such file or directory
Oct 24 09:10:51 HOSTNAME multipathd[389]: mpatha: load table [0 5854593280 multipath 1 queue_if_no_path 0 1 1 service-time 0 2 1 8:80 1 8:128 1]
Oct 24 09:10:51 HOSTNAME multipathd[389]: sdi [8:128]: path added to devmap mpatha

#journalctl -u multipathd

Oct 12 10:09:52 HOSTNAME multipathd[697]: mpathc: load table [0 19515446400 multipath 1 queue_if_no_path 0 1 1 round-robin 0 1 1 8:16 1]
Oct 12 10:09:52 HOSTNAME multipathd[697]: mpathc: event checker started
Oct 12 10:09:52 HOSTNAME multipathd[697]: sdb [8:16]: path added to devmap mpathc
Oct 12 10:09:52 HOSTNAME multipathd[697]: sdi: add path (uevent)
Oct 12 10:09:52 HOSTNAME multipathd[697]: sdi: spurious uevent, path already in pathvec
Oct 12 10:09:52 HOSTNAME multipathd[697]: mpatha: load table [0 5854593280 multipath 1 queue_if_no_path 0 1 1 round-robin 0 1 1 8:128 1]
Oct 12 10:09:52 HOSTNAME multipathd[697]: mpatha: event checker started
Oct 12 10:09:52 HOSTNAME multipathd[697]: sdi [8:128]: path added to devmap mpatha
Oct 12 10:09:52 HOSTNAME multipathd[697]: sdh: add path (uevent)
Oct 12 10:09:52 HOSTNAME multipathd[697]: sdh: spurious uevent, path already in pathvec
Oct 12 10:09:52 HOSTNAME multipathd[697]: mpathb: load table [0 15590232192 multipath 1 queue_if_no_path 0 1 1 round-robin 0 1 1 8:112 1]
Oct 12 10:09:52 HOSTNAME multipathd[697]: mpathb: event checker started
Oct 12 10:09:52 HOSTNAME multipathd[697]: sdh [8:112]: path added to devmap mpathb
Oct 12 10:09:52 HOSTNAME multipathd[697]: sde: add path (uevent)
Oct 12 10:09:52 HOSTNAME multipathd[697]: sde: spurious uevent, path already in pathvec
Oct 12 10:09:52 HOSTNAME multipathd[697]: sde : delaying path addition until mpathc is fully initialized
Oct 12 10:09:52 HOSTNAME multipathd[697]: sdg: add path (uevent)
Oct 12 10:09:52 HOSTNAME multipathd[697]: sdg: spurious uevent, path already in pathvec
Oct 12 10:09:52 HOSTNAME multipathd[697]: sdg : delaying path addition until mpathb is fully initialized
Oct 12 10:09:52 HOSTNAME multipathd[697]: sdd: add path (uevent)
Oct 12 10:09:52 HOSTNAME multipathd[697]: sdd: spurious uevent, path already in pathvec
Oct 12 10:09:52 HOSTNAME multipathd[697]: mpathd: load table [0 27343749888 multipath 1 queue_if_no_path 0 1 1 round-robin 0 1 1 8:48 1]
Oct 12 10:09:52 HOSTNAME multipathd[697]: mpathd: event checker started
Oct 12 10:09:52 HOSTNAME multipathd[697]: sdd [8:48]: path added to devmap mpathd
Oct 12 10:09:52 HOSTNAME multipathd[697]: sdf: add path (uevent)
Oct 12 10:09:52 HOSTNAME multipathd[697]: sdf: spurious uevent, path already in pathvec
Oct 12 10:09:52 HOSTNAME multipathd[697]: sdf : delaying path addition until mpatha is fully initialized
Oct 12 10:09:52 HOSTNAME multipathd[697]: sdc: add path (uevent)
Oct 12 10:09:52 HOSTNAME multipathd[697]: sdc: spurious uevent, path already in pathvec
Oct 12 10:09:52 HOSTNAME multipathd[697]: sdc : delaying path addition until mpathd is fully initialized
Oct 12 10:09:52 HOSTNAME kernel: ipmi_si 0000:01:04.6: Found new BMC (man_id: 0x00000b, prod_id: 0x2000, dev_id: 0x11)
Oct 12 10:09:52 HOSTNAME kernel: ipmi_si 0000:01:04.6: IPMI kcs interface initialized
Oct 12 10:09:52 HOSTNAME kernel: IPMI SSIF Interface driver
Oct 12 10:09:52 HOSTNAME multipathd[697]: sda: add path (uevent)
Oct 12 10:09:52 HOSTNAME multipathd[697]: sda: spurious uevent, path already in pathvec
Oct 12 10:09:52 HOSTNAME systemd[1]: Created slice system-lvm2\x2dpvscan.slice.
Oct 12 10:09:52 HOSTNAME systemd[1]: Starting system-lvm2\x2dpvscan.slice.
Oct 12 10:09:52 HOSTNAME systemd[1]: Starting LVM2 PV scan on device 8:2...
Oct 12 10:09:52 HOSTNAME lvm[870]: WARNING: lvmetad is being updated, retrying (setup) for 10 more seconds.
Oct 12 10:09:52 HOSTNAME multipathd[697]: mpathc: performing delayed actions
Oct 12 10:09:52 HOSTNAME multipathd[697]: mpathc: load table [0 19515446400 multipath 1 queue_if_no_path 0 2 1 round-robin 0 1 1 8:16 1 round-robin 0 1 1 8:64 1]
Oct 12 10:09:52 HOSTNAME multipathd[697]: mpatha: performing delayed actions
Oct 12 10:09:52 HOSTNAME multipathd[697]: mpatha: load table [0 5854593280 multipath 1 queue_if_no_path 0 2 1 round-robin 0 1 1 8:128 1 round-robin 0 1 1 8:80 1]
Oct 12 10:09:52 HOSTNAME multipathd[697]: mpathd: performing delayed actions
Oct 12 10:09:52 HOSTNAME multipathd[697]: mpathd: load table [0 27343749888 multipath 1 queue_if_no_path 0 2 1 round-robin 0 1 1 8:32 1 round-robin 0 1 1 8:48 1]
Oct 12 10:09:52 HOSTNAME multipathd[697]: mpathb: performing delayed actions
Oct 12 10:09:52 HOSTNAME multipathd[697]: mpathb: load table [0 15590232192 multipath 1 queue_if_no_path 0 2 1 round-robin 0 1 1 8:96 1 round-robin 0 1 1 8:112 1]

During the boot process the multipaths are being picked up on sd* and then seeing:

'spurious uevent, path already in pathvec'

It lead me to think that there was some thing wrong with the config, so I have stumbled across a RH KB which has the same eror as such and talks about the blacklisting not being correct:

https://access.redhat.com/solutions/2360781

I tried blacklisting sd* and also the sdb through to sdi which meant none of my /dev/mapper/mpath* showed up which was worse :( I even tried disabling smartd which is why the output above may look a little different. I have enabled this since my testing.

So I had to fall back to my work around. I am now completely lost as to what the problem is. I dont think my lvm.conf is wrong because as soon as I hot add the partitions the VGs and LVs appear, I am out of ideas at this point :(
Top
User avatar
TrevorH
Site Admin
Posts: 33218
Joined: 2009/09/24 10:40:56
Location: Brighton, UK

Re: Critical Issues with Multipath losing Config on reboot

Post by TrevorH » 2017/10/24 12:10:18

Aren't there any developers who monitor these forums? Have I posted in the wrong forum or some thing, I mean it respectfully because I understand how these forum's work, but the lack of response from the CentOS team is baffling to me!
CentOS is a rebuild of the same source used to build RHEL. The CentOS "developers" do almost no development as such except that required to remove Redhat branding and trademarks and replace it with CentOS. Thus having access to CentOS developers will not assist you in the slightest.

If you need support for problems then you should be paying Redhat for a RHEL license with a support subscription.
The future appears to be RHEL or Debian. I think I'm going Debian.
Info for USB installs on http://wiki.centos.org/HowTos/InstallFromUSBkey
CentOS 5 and 6 are deadest, do not use them.
Use the FAQ Luke
Top
h1tchiker
Posts: 14
Joined: 2017/08/17 07:40:24

Re: Critical Issues with Multipath losing Config on reboot

Post by h1tchiker » 2017/10/24 12:14:31

Wow! Thanks for that info!

First reply in over a week and its some on having a go :(

If i could have got the budget for a distro I would of :(

Just not seen a lack of response like this before other, distro's fall over backwards to help.
Top
User avatar
TrevorH
Site Admin
Posts: 33218
Joined: 2009/09/24 10:40:56
Location: Brighton, UK

Re: Critical Issues with Multipath losing Config on reboot

Post by TrevorH » 2017/10/24 13:11:00

Everyone on this forum is an unpaid volunteer.
The future appears to be RHEL or Debian. I think I'm going Debian.
Info for USB installs on http://wiki.centos.org/HowTos/InstallFromUSBkey
CentOS 5 and 6 are deadest, do not use them.
Use the FAQ Luke
Top
h1tchiker
Posts: 14
Joined: 2017/08/17 07:40:24

Re: Critical Issues with Multipath losing Config on reboot

Post by h1tchiker » 2017/10/24 13:23:11

Does any one please have any ideas about the multipath issue ?
Top
h1tchiker
Posts: 14
Joined: 2017/08/17 07:40:24

Re: Critical Issues with Multipath losing Config on reboot

Post by h1tchiker » 2017/10/25 08:47:09

Hi

OK, I have since removed the alias's altogether from multipath.conf as they were not really necessary. However when they are in place /dev/mapper/* changes from having the default name such as /dev/mapper/mpatha and /dev/mapper/mpatha1 to the alias in my case as suggested /dev/mapper/hp-mpatha. This had no effect on the partitions being available at boot hence I removed them and now I just have the standard config for this as can be seen in /etc/multipath/bindings:

mpatha 3600c0ff000dadd07d959635901000000
mpathb 3600c0ff00012e03163ea655901000000
mpathc 3600c0ff00014ebf5c757635901000000
mpathd 3600c0ff00012e031efe9655901000000

I have removed the blacklist exceptions for the device and devnode "dm-" and restarted multipathd, journalctl shows:

Oct 25 08:01:20 HOSTNAME systemd[1]: Stopping Device-Mapper Multipath Device Controller...
Oct 25 08:01:20 HOSTNAME multipathd[882]: mpatha: stop event checker thread (139940414228224)
Oct 25 08:01:20 HOSTNAME multipathd[882]: mpathb: stop event checker thread (139940414195456)
Oct 25 08:01:20 HOSTNAME multipathd[882]: mpathc: stop event checker thread (139940414162688)
Oct 25 08:01:20 HOSTNAME multipathd[882]: mpathd: stop event checker thread (139940414129920)
Oct 25 08:01:20 HOSTNAME multipathd[882]: --------shut down-------
Oct 25 08:01:20 HOSTNAME systemd[1]: Starting Device-Mapper Multipath Device Controller...
Oct 25 08:01:21 HOSTNAME systemd[1]: Started Device-Mapper Multipath Device Controller.
Oct 25 08:01:21 HOSTNAME multipathd[16256]: mpatha: load table [0 5854593280 multipath 1 queue_if_no_path 0 1 1 service-time 0 2 1 8:80 1 8:128 1]
Oct 25 08:01:21 HOSTNAME multipathd[16256]: mpathb: load table [0 15590232192 multipath 1 queue_if_no_path 0 1 1 service-time 0 2 1 8:96 1 8:112 1]
Oct 25 08:01:21 HOSTNAME multipathd[16256]: mpathc: load table [0 19515446400 multipath 1 queue_if_no_path 0 1 1 service-time 0 2 1 8:16 1 8:64 1]
Oct 25 08:01:21 HOSTNAME multipathd[16256]: mpathd: load table [0 27343749888 multipath 1 queue_if_no_path 0 1 1 service-time 0 2 1 8:32 1 8:48 1]
Oct 25 08:01:21 HOSTNAME multipathd[16256]: mpatha: event checker started
Oct 25 08:01:21 HOSTNAME multipathd[16256]: mpathb: event checker started
Oct 25 08:01:21 HOSTNAME multipathd[16256]: mpathc: event checker started
Oct 25 08:01:21 HOSTNAME multipathd[16256]: mpathd: event checker started
Oct 25 08:01:21 HOSTNAME multipathd[16256]: path checkers start up
Oct 25 08:03:25 HOSTNAME kernel: sdb: sdb1
Oct 25 08:03:26 HOSTNAME kernel: sdc: sdc1
Oct 25 08:03:26 HOSTNAME kernel: sdd: sdd1
Oct 25 08:03:26 HOSTNAME kernel: sde: sde1
Oct 25 08:03:26 HOSTNAME kernel: sdf: sdf1
Oct 25 08:03:26 HOSTNAME kernel: sdg: sdg1 sdg2
Oct 25 08:03:26 HOSTNAME kernel: sdh: sdh1 sdh2
Oct 25 08:03:26 HOSTNAME kernel: sdi: sdi1

Which tells me that my partitions are being picked up on /dev/sd* instead of /dev/mapper/mpath* using multipath -v3 confirms this:

===== paths list =====
uuid hcil dev dev_t pri dm_st chk_st vend/prod
3600508b1001c029881c94e4b91967594 0:1:0:0 sda 8:0 1 undef undef HP ,
3600c0ff000dadd07d959635901000000 4:0:0:0 sdf 8:80 10 undef undef HP ,
3600c0ff00012e03163ea655901000000 4:0:1:0 sdg 8:96 50 undef undef HP ,
3600c0ff00012e03163ea655901000000 4:0:2:0 sdh 8:112 10 undef undef HP ,
3600c0ff000dadd07d959635901000000 4:0:3:0 sdi 8:128 50 undef undef HP ,
3600c0ff00014ebf5c757635901000000 1:0:0:0 sdb 8:16 50 undef undef HP ,
3600c0ff00012e031efe9655901000000 1:0:1:0 sdc 8:32 50 undef undef HP ,
3600c0ff00012e031efe9655901000000 1:0:2:0 sdd 8:48 10 undef undef HP ,
3600c0ff00014ebf5c757635901000000 1:0:3:0 sde 8:64 10 undef undef HP ,

It's almost like multipathd cannot see the partitions that I created on /dev/mapper/mpath*

FYI: I used parted to create these partitions as they are very large with gpt disks.

The rest of my multipath.conf that sits in /etc/ and not in /etc/multipath/ is as follows:

## Use user friendly names, instead of using WWIDs as names.
defaults {
user_friendly_names yes
find_multipaths yes

}

devices {
device {
vendor "HP"
product "P2000 G3 FC"
path_grouping_policy multibus
path_checker readsector0
path_selector "round-robin 0"
hardware_handler "0"
failback 15
rr_weight priorities
no_path_retry queue
}
device {
vendor "HP"
product "P2000 G3 FC"
path_grouping_policy multibus
}
}

blacklist {
devnode "^(sda|ram|raw|loop|fd|md|sr|scd|st|cl-home|cl-swap|cl-root)[0-9]*"
}

I have also tried setting this back to the default configuration with just the blacklist. I have tried changing the 'path checker' option to all of the available options:

path_checker - Specifies the default method used to determine the state of the paths. Possible values include:

readsector0: Read the first sector of the device.
tur: Issue a TEST UNIT READY to the device.
emc_clariion: Query the EMC Clariion specific EVPD page 0xC0 to determine the path.
hp_sw: Check the path state for HP storage arrays with Active/Standby firmware.
rdac: Check the path status for LSI/Engenio RDAC storage controller.
directio: Read the first sector with direct I/O.

Then restarted multipathd with the same results. Now here is the bit I do not get, there is no mention of the paths in /dev/mapper/ unless I run the command:

#dmsetup ls

mpatha1p1p1 (253:13)
cl-swap (253:10)
cl-root (253:9)
3600c0ff00012e031efe9655901000000p1 (253:2)
mpatha1p1 (253:12)
3600c0ff000dadd07d959635901000000p1 (253:8)
mpathc1p1p1 (253:18)
vg_file01_vdp-lv_file01 (253:16)
3600c0ff00012e03163ea655901000000 (253:4)
mpathd1p1p1 (253:21)
vg_iscsi_vms-lv_iscsi_vm (253:14)
3600c0ff000dadd07d959635901000000 (253:5)
mpathd1p1 (253:20)
3600c0ff00014ebf5c757635901000000 (253:1)
vg_iscsi_vdp-lv_vdp (253:22)
vg_iscsi_storage-lv_iscsi_storage (253:19)
3600c0ff00012e03163ea655901000000p2 (253:7)
3600c0ff00012e03163ea655901000000p1 (253:6)
mpathc1p1 (253:17)
3600c0ff00014ebf5c757635901000000p1 (253:3)
cl-home (253:11)
3600c0ff00012e031efe9655901000000 (253:0)
mpathb1p1 (253:15)

Which is the only place I can find a mention of my partitions and also looks incorrect to me.

So I guess the question really is, why doesn't multipath see my partitions on /dev/mapper/mpath*?

I am happy to reconfigure the iSCSI to work around the problem and subsequently use /dev/sd* but even this would defeat the object of multipath because if I used say /dev/sdb and this path failed the connection would be dropped via iSCSI :(

NOTE: I have NOT made any changes to /etc/lvm/lvm.conf as I keep reading conflicting information and the last time I did so (following RHELs guide) the system failed to boot so I had to recover the initramfs to get back to a working state, which turns out is not much fun.

Is it possible my lack of entry to the lvm.conf is my issue?

In regard to the LVM config once my partitions are present by using kpartx these show up as follows:

# pvs && vgs && lvs
PV VG Fmt Attr PSize PFree
/dev/mapper/mpatha1p1p1 vg_iscsi_vms lvm2 a-- <2.73t 0
/dev/mapper/mpathb1p1 vg_file01_vdp lvm2 a-- <7.26t 0
/dev/mapper/mpathc1p1p1 vg_iscsi_storage lvm2 a-- <9.09t 0
/dev/mapper/mpathd1p1p1 vg_iscsi_vdp lvm2 a-- 12.73t 0
/dev/sda2 cl lvm2 a-- <135.70g 0
VG #PV #LV #SN Attr VSize VFree
cl 1 3 0 wz--n- <135.70g 0
vg_file01_vdp 1 1 0 wz--n- <7.26t 0
vg_iscsi_storage 1 1 0 wz--n- <9.09t 0
vg_iscsi_vdp 1 1 0 wz--n- 12.73t 0
vg_iscsi_vms 1 1 0 wz--n- <2.73t 0

I hope this information helps
Top
Post Reply

Return to “CentOS 7 - Software Support”