Error with NFSv4 when changing the GSSProxy/Kerberos ccache path

Issues related to applications and software problems
Post Reply
pmadrid
Posts: 4
Joined: 2018/08/09 06:49:25

Error with NFSv4 when changing the GSSProxy/Kerberos ccache path

Post by pmadrid » 2018/08/09 10:48:16

Hello everyone.

I'm struggling to make NFS4 work with Kerberos but I'm finding certain behaviours I don't understand.

I have two CentOS 7.5 servers: a NFS/DNS/Kerberos server (nfs01.nfs.com), and a NFS client (nfs02.nfs.com), but I'm unable to make it work correctly. Depending on the path of the ccache they fail (or not).
  • If the ccache resides in the Keyring I can mount the shares correctly.
  • If the ccache resides in /var/lib/gssproxy/clients/{UID} I can mount the shares.
  • If the ccache resides in /tmp I can't mount the shares ("Matching credential not found (filename: /tmp/krb5cc_0)").
The server export file is the following:

Code: Select all

[root@nfs01 ~]# cat /etc/exports
/exports  *(rw,root_squash,fsid=0,no_subtree_check,sync,sec=krb5)
/exports/prueba01  *(rw,sync,root_squash,insecure,nohide,no_subtree_check,anonuid=65000,anongid=65000,sec=krb5)
/exports/prueba02  *(rw,sync,root_squash,insecure,nohide,no_subtree_check,anonuid=65000,anongid=65000,sec=krb5)

[root@nfs01 ~]# ls -la /exports/
total 0
drwxr-xr-x.  4 root root  38 jul 25 10:27 .
dr-xr-xr-x. 19 root root 252 jul 25 10:26 ..
drwxr-xr-x.  2 root root   6 jul 25 10:27 prueba01
drwxr-x---.  2 paco paco   6 jul 25 10:27 prueba02
Moreover the client seems to detect correctly the exports:

Code: Select all

[root@nfs02 ~]# showmount -e nfs01.nfs.com
Export list for nfs01.nfs.com:
/exports/prueba02 *
/exports/prueba01 *
/exports          *
The kerberos config file "/etc/krb5.conf" is as follows:

Code: Select all

# Configuration snippets may be placed in this directory as well
includedir /etc/krb5.conf.d/

[logging]
 default = FILE:/var/log/krb5libs.log
 kdc = FILE:/var/log/krb5kdc.log
 admin_server = FILE:/var/log/kadmind.log

[libdefaults]
 dns_lookup_realm = false
 ticket_lifetime = 24h
 renew_lifetime = 7d
 forwardable = true
 rdns = false
 default_realm = NFS.COM
default_ccache_name = KEYRING:persistent:%{uid}

[realms]
 NFS.COM = {
  kdc = nfs01.nfs.com
  admin_server = nfs01.nfs.com
  default_domain = nfs.com
 }

[domain_realm]
 .nfs.com = NFS.COM
 nfs.com = NFS.COM
The only line I change when testing is the one that defines "default_ccache_name", which I change (depending on the test) to:
  • default_ccache_name = FILE:/var/lib/gssproxy/clients/%{uid}/krb5cc_%{uid}
  • default_ccache_name = FILE:/tmp/krb5cc_%{uid}
The GSSproxy config file "/etc/gssproxy/99-nfs-client.conf" is as follows:

Code: Select all

[service/nfs-client]
  mechs = krb5
  cred_store = keytab:/etc/krb5.keytab
  cred_store = ccache:KEYRING:persistent:%U
  cred_store = client_keytab:/var/lib/gssproxy/clients/%U/%U.keytab
  cred_usage = initiate
  allow_any_uid = yes
  trusted = yes
  euid = 0
The only line I change when testing is the one that sais "cred_store = ccache:", which I change (depending on the test) to:
  • cred_store = client_keytab:/var/lib/gssproxy/clients/%U/%U.keytab
  • cred_store = ccache:FILE:/tmp/krb5cc_%U
The keytab should be ok because it works on certain configurations:

Code: Select all

[root@nfs02 ~]# klist -kt /etc/krb5.keytab 
Keytab name: FILE:/etc/krb5.keytab
KVNO Timestamp         Principal
---- ----------------- --------------------------------------------------------
   4 30/07/18 13:26:20 nfs/nfs02.nfs.com@NFS.COM
   4 30/07/18 13:26:20 nfs/nfs02.nfs.com@NFS.COM
   4 30/07/18 13:26:20 nfs/nfs02.nfs.com@NFS.COM
   4 30/07/18 13:26:20 nfs/nfs02.nfs.com@NFS.COM
   4 30/07/18 13:26:20 nfs/nfs02.nfs.com@NFS.COM
   4 30/07/18 13:26:20 nfs/nfs02.nfs.com@NFS.COM
   4 30/07/18 13:26:20 nfs/nfs02.nfs.com@NFS.COM
   4 30/07/18 13:26:20 nfs/nfs02.nfs.com@NFS.COM
   4 30/07/18 13:26:21 host/nfs02.nfs.com@NFS.COM
   4 30/07/18 13:26:21 host/nfs02.nfs.com@NFS.COM
   4 30/07/18 13:26:21 host/nfs02.nfs.com@NFS.COM
   4 30/07/18 13:26:21 host/nfs02.nfs.com@NFS.COM
   4 30/07/18 13:26:21 host/nfs02.nfs.com@NFS.COM
   4 30/07/18 13:26:21 host/nfs02.nfs.com@NFS.COM
   4 30/07/18 13:26:21 host/nfs02.nfs.com@NFS.COM
   4 30/07/18 13:26:21 host/nfs02.nfs.com@NFS.COM
The thing is that when I configure all the pieces so that they store and use the ccache in /tmp they all fail, but when I use other paths they seem to work (¿?). I have tons of logs but I don't know which of them

None of them have IPtables or SElinux enabled (to minimize potential error they might cause). DNS resolution works correctly (tested it) and there are no connectivity problems (they are both VMs in my laptop).

Code: Select all

[root@nfs02 ~]# cat /etc/redhat-release 
CentOS Linux release 7.5.1804 (Core) 

[root@nfs02 ~]# rpm -qa | grep release
centos-release-7-5.1804.1.el7.centos.x86_64

[root@nfs02 ~]# getenforce
Disabled

[root@nfs02 ~]# iptables -L
Chain INPUT (policy ACCEPT)
target     prot opt source               destination         

Chain FORWARD (policy ACCEPT)
target     prot opt source               destination         

Chain OUTPUT (policy ACCEPT)
target     prot opt source               destination         

[root@nfs02 ~]# rpm -qa | grep nfs
libnfsidmap-0.25-19.el7.x86_64
nfs-utils-1.3.0-0.54.el7.x86_64

[root@nfs02 ~]# rpm -qa | grep gss
gssproxy-0.7.0-17.el7.x86_64
I attach both the logs when it's configured with the /tmp directory and when it uses /var/lib/gssproxy/clients.

Could someone please help me to understand how does it work? Thank you very much!
Attachments
logs.tar.gz
(5.8 KiB) Downloaded 63 times
Francisco Madrid-S.

hunter86_bg
Posts: 2019
Joined: 2015/02/17 15:14:33
Location: Bulgaria
Contact:

Re: Error with NFSv4 when changing the GSSProxy/Kerberos ccache path

Post by hunter86_bg » 2018/08/13 03:55:32

When you set the kerberos cache to '/tmp' , does

Code: Select all

kdestroy && klist
point to the '/tmp' cache ?
Also do you update the 'cred_store = client_keytab:/var/lib/'..... in file '/etc/gssproxy/99-nfs-client.conf' to match the '/etc/krb5.conf' settings?

I'm not sure if multiple 'cred_store' stanzas are allowed. Have you checked the documentation for examples?

pmadrid
Posts: 4
Joined: 2018/08/09 06:49:25

Re: Error with NFSv4 when changing the GSSProxy/Kerberos ccache path

Post by pmadrid » 2018/08/14 09:55:03

Hello hunter86, thx for your reply.

Prior to mount the share, the cache is empty but references the /tmp cache:

Code: Select all

[root@nfs02 ~]# klist
klist: No credentials cache found (filename: /tmp/krb5cc_0)
When I try to mount the share (and fails), my cache is the following:

Code: Select all

[root@nfs02 ~]# klist
Ticket cache: FILE:/tmp/krb5cc_0
Default principal: nfs/nfs02.nfs.com@NFS.COM

Valid starting     Expires            Service principal
01/01/70 01:00:00  01/01/70 01:00:00  Encrypted/Credentials/v1@X-GSSPROXY:
When I change the configuration, I change both files accordingly. Moreover, I've followed the official documentation and several tutorials, but what drives me crazy is that depending on the path it does work or not.

Thank you very much!

Best regards
Francisco Madrid-S.

pmadrid
Posts: 4
Joined: 2018/08/09 06:49:25

Re: Error with NFSv4 when changing the GSSProxy/Kerberos ccache path

Post by pmadrid » 2018/10/15 15:43:19

Hello everyone.

Seems as if it was my fault from the beginning. Kerberos cache and GSSProxy cache *must* not be placed in the same file.

GSSproxy will read the kerberos cache (by reading its path from "/etc/krb5.conf") to populate its own cache (stored elsewhere), so if you configure them to use the same file things will start up going weird. I guess the recommended configuration would be to configure Kerberos cache in the kernel keyring (more secure), and the GSSproxy cache in an arbitrary path.

And by the way, if you are thinking about using GSSproxy you *must* use keytabs: no kinit, no interactive password, just keytabs.

Regards,
Francisco Madrid-S.

Post Reply