I'm looking for some guidance on how to troubleshoot/debug an issue with (SELinux) labels over NFS that we've been having as a result of a kernel upgrade CentOS7.3->CentOS7.4 - description below.
I looked around on http://linux-nfs.org and on the NFS-devel mailing list but as this is a direct result of upgrading CentOS I thought I'd check here as well.
So, does anyone know how to get to the bottom of this? Or alternatively where I should go for help instead?
I’d also be interested if anyone has labelled NFSv4.2 working on the latest CentOS kernel.
Thank you very much,
Alexander Hermes
----
## Summary
In upgrading our servers from CentOS 7.3 to 7.4 we upgraded the kernel from 3.10.0-327.36.2.el7.x86_64 to 3.10.0-693.2.2.el7.x86_64. As a result, NFS v4.2 mounts mounted via /etc/fstab at boot do not have proper SElinux label support - attempting to change labels on a mounted file leads to "Operation Not Supported". `ls -lZ` shows the incorrect labels. Upon rebooting to the earlier kernel version the issue goes away.
## Background
As part of our gitlab HA deployment we use NFS to host data on a back end server that is then mounted by application servers (cf. https://docs.gitlab.com/ce/administrati ... y/nfs.html). To do this we have a fairly typical setup where the server (in this example "enfigitback2-devel") exports a bunch of mounts via /etc/exports which are then mounted on a couple of application servers (“enfigitfront1-devel” / “enfigitfront2-devel”).
## The issue
On the new kernel I am not able to change or view the SELinux labels of files / directories mounted on the client:
Code: Select all
[root@enfigitfront2-devel ~]# chcon --recursive --type ssh_home_t /var/opt/gitlab/.ssh/authorized_keys
chcon: failed to change context of ‘/var/opt/gitlab/.ssh/authorized_keys’ to ‘system_u:object_r:ssh_home_t:s0’: Operation not supported [root@enfigitfront2-devel ~]# uname -r
3.10.0-693.2.2.el7.x86_64
Code: Select all
[root@enfigitfront1-devel ~]# chcon --recursive --type ssh_home_t /var/opt/gitlab/.ssh/authorized_keys
[root@enfigitfront1-devel ~]# uname -r
3.10.0-327.36.2.el7.x86_64
We can't keep using the old kernel forever so I'd like to get to the bottom of this - what could this be due to? How can I debug this further to understand where the "Operation not supported" is coming from?
## Server details
(see also full details for the working client (https://pastebin.com/aw97Dxp9), the non-working one (https://pastebin.com/61uvsz7N) and the server (https://pastebin.com/mCMCfAEx ))
Distro: CentOS 7.4
Kernel (`uname -r`):
* 3.10.0-693.5.2.el7 (server)
* 3.10.0-693.2.2.el7.x86_64 (client - new)
* 3.10.0-327.36.2.el7.x86_64 (client - old)
nfs-utils: RPM package version 1.3.0
Server mount option example:
Code: Select all
/export/.ssh 172.18.10.148(rw,sync,no_root_squash,security_label) 172.18.10.151(rw,sync,no_root_squash,security_label)
Code: Select all
enfigitback2-devel.datcon.co.uk:/export/.ssh /var/opt/gitlab/.ssh nfs defaults,soft,v4.2,rsize=1048576,wsize=1048576,noatime,_netdev,lookupcache=none 0 0
### Mounting by hand
I tried to mount one of the exported mounts "by hand" using `mount` and found the following:
* mounting the same export on a different mount point using the same options as in /etc/fstab yields a mount that has the same issue
* mounting with `nosharecache` results in a mount that *does not* have the issue.
So this suggests some issues related to the cache.