Trouble with mounting a kerberized NFS4 mount from a CentOS 7 client

[knifex]

Hello,

I've been struggling for a few days now with mounting an NFS4 share from a CentOS 7 workstation. For this thread, let's say the client is at nfsclient.domain, and the server (running CentOS 6) is at nfsserver.domain. nfsserver.domain is also the primary Kerberos server for this network, and Kerberos has been working between these two machines for over a year now with no issues.

I have been trying to mount the NFS share using the following command:

Code: Select all

# mount -t nfs4 -o sec=krb5p nfsserver.domain:/ /mnt/nfs

but I always get the error message:

Code: Select all

mount.nfs4: access denied by server while mounting nfsserver.domain:/

I have no trouble listing the mounts:

Code: Select all

# showmount -e nfsserver.domain
Export list for nfsserver.domain:
/export/home *
/export      *

If I disable the Kerberos security on the server, then I can mount the share from the workstation with no issue. It is only when Kerberos is added that things go wrong.

Going down the list of possible Kerberos issues, I've ensured:

• The keytab on nfsserver.domain holds keys for nfs/nfsserver.domain and host/nfsserver.domain.
• The keytab on nfsclient.domain holds keys for nfs/nfsclient.domain and host/nfsclient.domain.
• nfsclient.domain can obtain keys from the KDC on nfsserver.domain.
• nfsclient.domain and nfsserver.domain resolve to the correct IP addresses.
• NTP is set up correctly on both machines.

As I said, Kerberos authentication for everything but NFS has been working on these two machines for over a year now, so I'm not sure what is going on.

Just to make sure this wasn't a weird firewall thing, I flushed all of the iptables chains. Same error message. Again, NFS and Kerberos work fine independently with my current firewall configuration.

Finally, I've tried to track down the problem by looking in the system logs. Right after trying to mount the share on nfsclient.domain, this shows up in /var/log/messages:

Code: Select all

Jun 22 18:27:13 nfsclient gssproxy: gssproxy[769]: (OID: { 1 2 840 113554 1 2 2 }) Unspecified GSS failure.  Minor code may provide more information, No credentials cache found
Jun 22 18:27:13 nfsclient gssproxy: gssproxy[769]: (OID: { 1 2 840 113554 1 2 2 }) Unspecified GSS failure.  Minor code may provide more information, No credentials cache found
Jun 22 18:27:13 nfsclient gssproxy: gssproxy[769]: (OID: { 1 2 840 113554 1 2 2 }) Unspecified GSS failure.  Minor code may provide more information, No credentials cache found
Jun 22 18:27:13 nfsclient gssproxy: gssproxy[769]: (OID: { 1 2 840 113554 1 2 2 }) Unspecified GSS failure.  Minor code may provide more information, No credentials cache found
Jun 22 18:27:13 nfsclient gssproxy: gssproxy[769]: (OID: { 1 2 840 113554 1 2 2 }) Unspecified GSS failure.  Minor code may provide more information, No credentials cache found
Jun 22 18:27:13 nfsclient gssproxy: gssproxy[769]: (OID: { 1 2 840 113554 1 2 2 }) Unspecified GSS failure.  Minor code may provide more information, No credentials cache found

On the other hand, nothing shows up in the logs on nfsserver.domain. Running klist as root on nfsclient.domain shows that I have a credentials cache at /tmp/krb5cc_0 with default principal host/nfsclient.domain. So maybe there is something wrong with my gssproxy service?

Here is my /etc/gssproxy/gssproxy.conf:

Code: Select all

[gssproxy]

[service/HTTP]
  mechs = krb5
  cred_store = keytab:/etc/gssproxy/http.keytab
  cred_store = ccache:/var/lib/gssproxy/clients/krb5cc_%U
  euid = 48

[service/nfs-server]
  mechs = krb5
  socket = /run/gssproxy.sock
  cred_store = keytab:/etc/krb5.keytab
  trusted = yes
  kernel_nfsd = yes
  euid = 0

[service/nfs-client]
  mechs = krb5
  cred_store = keytab:/etc/krb5.keytab
  cred_store = ccache:FILE:/var/lib/gssproxy/clients/krb5cc_%U
  cred_store = client_keytab:/var/lib/gssproxy/clients/%U.keytab
  cred_usage = initiate
  allow_any_uid = yes
  trusted = yes
  euid = 0

So if I'm understanding this right, then gssproxy is looking for the credentials cache in a file under /var/lib/gssproxy/clients. This directory is empty on my workstation, so I get the feeling this is related to my issue. Is there something I'm missing here? I'm out of ideas at the moment.

Thanks for any help!

EDIT: Figured it out. I pointed gss-proxy to /tmp for its credentials caches, which made it start talking to the server (but still with an access denied message). Then I had to set SECURE_NFS=yes in /etc/sysconfig/nfs on the server so that rpc.svcgssd would start (apparently this was the real problem - the client couldn't authenticate via the GSS protocol because the server GSS daemon wasn't starting).

[chtsalid]

Hi,

I came across the same problem. Could you please provide some more information on how you fixed it?

Many thanks!

Here are some details

My topology is the following

freeipa server - 192.168.122.1
nfs server - 192.168.122.2
nfs client - 192.168.122.3

[root@rh3 ~]# mount -o sec=krb5p rh2.lab.local:/srv/nfsexport /mnt/securenfs/
mount.nfs: access denied by server while mounting rh2.lab.local:/srv/nfsexport


[root@rh3 ~]# cat /var/log/messages
Feb 19 00:21:51 rh3 gssproxy: gssproxy[20313]: (OID: { 1 2 840 113554 1 2 2 }) Unspecified GSS failure. Minor code may provide more information, No credentials cache found
Feb 19 00:21:51 rh3 gssproxy: gssproxy[20313]: (OID: { 1 2 840 113554 1 2 2 }) Unspecified GSS failure. Minor code may provide more information, No credentials cache found
Feb 19 00:21:51 rh3 gssproxy: gssproxy[20313]: (OID: { 1 2 840 113554 1 2 2 }) Unspecified GSS failure. Minor code may provide more information, No credentials cache found
Feb 19 00:21:51 rh3 gssproxy: gssproxy[20313]: (OID: { 1 2 840 113554 1 2 2 }) Unspecified GSS failure. Minor code may provide more information, No credentials cache found
Feb 19 00:21:51 rh3 gssproxy: gssproxy[20313]: (OID: { 1 2 840 113554 1 2 2 }) Unspecified GSS failure. Minor code may provide more information, No credentials cache found
Feb 19 00:21:51 rh3 gssproxy: gssproxy[20313]: (OID: { 1 2 840 113554 1 2 2 }) Unspecified GSS failure. Minor code may provide more information, No credentials cache found



[root@rh3 ~]# cat /etc/gssproxy/gssproxy.conf
[gssproxy]

[service/HTTP]
mechs = krb5
cred_store = keytab:/etc/gssproxy/http.keytab
cred_store = ccache:/var/lib/gssproxy/clients/krb5cc_%U
euid = 48

[service/nfs-server]
mechs = krb5
socket = /run/gssproxy.sock
cred_store = keytab:/etc/krb5.keytab
trusted = yes
kernel_nfsd = yes
euid = 0

[service/nfs-client]
mechs = krb5
cred_store = keytab:/etc/krb5.keytab
cred_store = ccache:FILE:/var/lib/gssproxy/clients/krb5cc_%U
cred_store = client_keytab:/var/lib/gssproxy/clients/%U.keytab
cred_usage = initiate
allow_any_uid = yes
trusted = yes
euid = 0

[chtsalid]

I don't know exactly, how it happened, but after a system reboot
it works fine. I always do a restart of services, after .conf files
is modified.
So, I am now not sure why it now works.