mount fails to use DFS secondary referrals

[MHCent]

Code: Select all

 $ umount -a:
Linux server01 3.10.0-957.1.3.el7.x86_64 #1 SMP Thu Nov 29 14:49:43 UTC 2018 x86_64 x86_64 x86_64 GNU/Linux

I am able to successfully mount a DFS file share using the following command:

Code: Select all

$ sudo mount.cifs \\\\domain.com\\share /mnt/share -o username=user

Code: Select all

$ mount | grep domain
\\domain.com\share on /mnt/share type cifs (rw,relatime,vers=default,cache=strict,username=user,domain=,uid=0,noforceuid,gid=0,noforcegid,addr=10.0.0.10,file_mode=0755,dir_mode=0755,soft,nounix,serverino,mapposix,rsize=1048576,wsize=1048576,echo_interval=60,actimeo=1)

here's an output from DFSutil.exe on windows showing the valid referrals. Here DFS-02 is the primary from the above mount output 10.0.0.10:

Code: Select all

 C:/> dfsutil /pktinfo
Entry: \domain.com\share
ShortEntry: \domain.com\share
Expires in 260 seconds
UseCount: 0 Type:0x8081 ( REFERRAL_SVC DFS FAILBACK_ENABLED )
   0:[\DFS-02\Share] AccessStatus: 0 ( ACTIVE TARGETSET )
   1:[\DFS-01\Share] ( TARGETSET )

If I simulate a failure of the primary server (DFS-02), I am not able to mount the share, even though the second referral is valid. I've included the verbose command output:

Code: Select all

sudo iptables -I OUTPUT -j DROP -d DFS-02.domain.com
$ sudo mount.cifs \\\\domain.com\\share /mnt/share -o username=user --verbose
Password for user@\domain.com\share:  *****
mount.cifs kernel mount options: ip=10.0.0.4,unc=\\domain.com\share,user=user,pass=********
mount error(115): Operation now in progress
Refer to the mount.cifs(8) manual page (e.g. man mount.cifs)
$ sudo tail /var/log/messages
May 30 11:50:52 server01 kernel: CIFS VFS: BAD_NETWORK_NAME: \\domain.com\share

10.0.0.4 is my DC. I can manually connect to either DFS-01 or DFS-02 by IP or by re-prioritizing DFS referrals on the windows side, but the point of our DFS here is to automatically roll over to a valid server. The windows clients are able to do this by default. We recently had one of the DFS servers go down, causing more issues than we expected.

Is there a configuration I missed or a package for cifs\samba that this works correctly with?

[TrevorH]

As you are 6 months out of date, your first step is to run yum update and then reboot.

[MHCent]

I'm now up to date, and verified the behavior is the same on the current kernel.

Code: Select all

$ uname -a
Linux Server01 3.10.0-957.21.2.el7.x86_64 #1 SMP Wed Jun 5 14:26:44 UTC 2019 x86_64 x86_64 x86_64 GNU/Linux

I've noticed I get the following message whether the mount is successful or not

Code: Select all

Jun 11 11:15:49 Server01 kernel: CIFS VFS: BAD_NETWORK_NAME: \\domain.com\share

I've enabled cifs debugging

Code: Select all

echo 7 > /proc/fs/cifs/cifsFYI

It seems like if cifs fails to connect to the first server, it kills the whole process instead of trying the other referrals - here's the full output for a simulated failure:

Code: Select all

[ 5300.910720] Status code returned 0xc0000016 STATUS_MORE_PROCESSING_REQUIRED
[ 5300.922708] Status code returned 0xc00000cc STATUS_BAD_NETWORK_NAME
[ 5300.922714] CIFS VFS: BAD_NETWORK_NAME: \\domain.com\share
[ 5311.137301] CIFS VFS: Error connecting to socket. Aborting operation.
[ 5311.138088] CIFS VFS: cifs_mount failed w/return code = -115

[reenberg]

MHCent, did you ever sort out this issue?

I recently hit somewhat the same, when one of our multiple DFS servers suddenly wasn't reachable on port 445 by our Linux servers.

It was even worse by the fact that the DC seems to return a "random" DFS server from its list (from my tcpdump investigation), each time you do a mount operation, which resulted in mounts failing each time that DFS server was returned.

It sure would have been nice if CIFS asked the DC for a new server and did some retries.
However the error would most likely have gone unnoticed in that case.

[MHCent]

Unfortunately, no. It doesn't seem like the cifs team have added the ability to do anything other than connect to the first server in the referral list from the DC.

I was able to mitigate the randomness by configuring my "primary" DFS host to always have the highest priority for my particular folder/namespace. I also am able to do a kind-of fail over by re-ordering the referrals manually and letting linux remount in the case of maintenance etc.

This could probably be submitted to the cifs email group, but they are a little over my head. The following DFS protocol references might be helpful if anyone wants to dive in:

Expected client behavior description: DFS client target failover
DFS protocol definition for handling this case - basically cifs should have been configured to select another target from the list or fail with a better error: 3.1.5.2 I/O Operation to Target Fails with an Error Other Than STATUS_PATH_NOT_COVERED