Reservation Conflict Error - 2 Multipath Paths Failed

[theocharotas]

Hello,

I think I need your help on a strange case , I have to deal with. I have a two-node cluster (CentOS Linux release 7.3.1611 (Core)), that is configured to use two different common storage devices, exported from 3PAR 7200 storage. ISCSi protocol is used for the connection between virtual machines and the virtual disks exported by 3PAR and 4 different paths are configured for multipathing. I'm using the common configuration with DLM/CLMVD processes in clone configuration for both nodes.

I'm using the following common devices: (/dev/mapper/mvas-com2, /dev/mapper/mvas-ipsmgw1)

1. When Node-1 has common devices as open and mounted, it can write on them without any issue.
2. When Node-2 has common devices as open and mounted, it can write on /dev/mapper/mvas-ipsmgw1 without any problem, but when it writes on /dev/mapper/mvas-com2, it receives always "reservation conflict error" for two 2 specific paths between 4.

mvas-ipsmgw (360002ac0000000000000003b00014981) dm-5 3PARdata,VV
size=50G features='1 queue_if_no_path' hwhandler='1 alua' wp=rw
`-+- policy='round-robin 0' prio=50 status=active
|- 3:0:0:14 sdh 8:112 active ready running
|- 5:0:0:14 sde 8:64 active ready running
|- 4:0:0:14 sdi 8:128 active ready running
`- 6:0:0:14 sdc 8:32 active ready running
mvas-com (360002ac0000000000000003500014981) dm-6 3PARdata,VV
size=30G features='1 queue_if_no_path' hwhandler='1 alua' wp=rw
`-+- policy='round-robin 0' prio=50 status=active
|- 3:0:0:11 sdf 8:80 active ready running
|- 5:0:0:11 sdd 8:48 active ready running
|- 4:0:0:11 sdg 8:96 active ready running
`- 6:0:0:11 sdb 8:16 active ready running

Oct 17 09:56:49 nm-mvas-srv02 kernel: sd 3:0:0:11: [sdf] FAILED Result: hostbyte=DID_OK driverbyte=DRIVER_OK
Oct 17 09:56:49 nm-mvas-srv02 kernel: sd 3:0:0:11: [sdf] CDB: Write(10) 2a 00 00 08 80 00 00 00 80 00
Oct 17 09:56:49 nm-mvas-srv02 kernel: blk_update_request: 7 callbacks suppressed
Oct 17 09:56:49 nm-mvas-srv02 kernel: blk_update_request: critical nexus error, dev sdf, sector 557056
Oct 17 09:56:49 nm-mvas-srv02 kernel: device-mapper: multipath: Failing path 8:80.
Oct 17 09:56:49 nm-mvas-srv02 kernel: sd 3:0:0:11: reservation conflict
Oct 17 09:56:49 nm-mvas-srv02 kernel: sd 3:0:0:11: [sdf] FAILED Result: hostbyte=DID_OK driverbyte=DRIVER_OK
Oct 17 09:56:49 nm-mvas-srv02 kernel: sd 3:0:0:11: [sdf] CDB: Write(10) 2a 00 00 db 91 00 00 02 00 00
Oct 17 09:56:49 nm-mvas-srv02 kernel: blk_update_request: critical nexus error, dev sdf, sector 14389504

I have two clarify that 4 paths used for the storage devices, are common for both of them. I have checked the configuration of disk devices and I can see tha on parameter is different for only this 2 path (FUA=Force Unit Access).

[root@nm-swap-mdt-srv02 scsi_disk]# cat [34]:0:0:11/FUA
0
0
[root@nm-swap-mdt-srv02 scsi_disk]# cat [56]:0:0:1[14]/FUA
1
1
1
1

As you can see FUA is disabled for these paths. For the second node, I have deleted iscsi discovery results and I have unmapped the device from the storage, in order to discover the device from the beginning. But the result was the same. Node-1 has FUA enabled for all paths and devices, and Node-2 has FUA disabled only for this divice and only for these two paths. Any help?

I' m sending you attached also the multipath configuration. It's common for both nodes.

[theocharotas]

Could anyone help me please ? I have not found yet any solution about these two paths out of 4.