NFSv4, NetApp, CentOS 6.8 : your thought ?

Issues related to configuring your network
Post Reply
NdFeB
Posts: 9
Joined: 2017/02/03 16:39:04

NFSv4, NetApp, CentOS 6.8 : your thought ?

Post by NdFeB » 2017/02/03 17:19:34

Hi guys !

I'd like to hear some experience reports, today in 2017 !

I am actually trying to share a folder to both Windows and Linux systems using a NetApp as the server. The actual situation is that : NFSv3 is enabled on the NetApp, and NFSv4 is disabled. The thing is, the NetApp's feature carrying Unix-like ACLs over NFS is only available for NFSv4 (and I need to manage ACLs on Linux because it is integrated to an Active Directory domain, long story).

I suggested to my colleague "hey, maybe we should enable NFSv4 on the NetApp filer so we can manage our ACLs on Linux and solve this problem". He answered : "Yeah, no, i don't know... NFSv4 is too recent to me, and not used enough to be 100% trustworthy". This surprised me. He is 12 years older than me, and I'm 23... I don't know what to think about this...

So, what's your personal opinion / analyze about NFSv4 ? What about its use on CentOS 6.8 as a client ? As a server ?
For those who know NetApp servers, can both NFSv3 and NFSv4 be enabled on a same tree / aggregate ? Then can you choose what protocol to use for each share ? (the NetApp is in production, i can't test it blindly :( )

(Our CentOS boxes : 6.8, kernel 2.6.32. Our NetApp licences : basics : NFS, CiFS, iSCSI, OnCommand)

Thank you guys for sharing ! ;)

Whoever
Posts: 1357
Joined: 2013/09/06 03:12:10

Re: NFSv4, NetApp, CentOS 6.8 : your thought ?

Post by Whoever » 2017/02/05 19:19:05

Enable NFS4 on the server, but set the clients to mount using NFS3, except for one client on which you test out NFS4, including performance.

NdFeB
Posts: 9
Joined: 2017/02/03 16:39:04

Re: NFSv4, NetApp, CentOS 6.8 : your thought ?

Post by NdFeB » 2017/02/06 13:23:15

Hi,

thanks for the information. I saw that Windows does not support NFSv4, and shares have to be explicitly mounted as NFSv4 in /etc/fstab, so enabling NFSv4 on the NetApp shouldn't impact any client without doing any configuration.

I'll try to manipulate a test share with NFSv4 and see if i can manage ACLs. I can't do it right now, but stay tuned if you want to know how it works !

Don't forget that this thread is open to any comment about NFSv4, even out of NetApp context !

Regards.

User avatar
jlehtone
Posts: 4523
Joined: 2007/12/11 08:17:33
Location: Finland

Re: NFSv4, NetApp, CentOS 6.8 : your thought ?

Post by jlehtone » 2017/02/07 12:24:27

I must confess that I'm curious about the NetApp.

Lets say I have one. One that shares with NFS. It knows one LDAP server, so it can resolve ids for NFSv4 too. Perhaps it has a CIFS service too, but is not a member of any AD and thus cannot share via CIFS. Now an AD could be added. LDAP users, AD users, NFS permissions, CIFS ACLs, oh my .. or an auto-disaster? Should RTFM, I suppose.


Anyway. There was a volume in NetApp. It was mounted with NFSv3. No LDAP/AD was required. Some files were written. Old files. Then the same volume was mounted with NFSv4. One could not see all files. Anything with non-ASCII-7 characters in their names with wrong coding. The old files had a mixture of cp437, cp1250, utf-8, etc symbols. The NetApp volume is created with codepage. Cannot change later. NFSv4 requires every incomer to comply. Totally freaked with the "illegal aliens" that had sneaked in via NFSv3.


That implies that I have tried NFSv4. Yes, more than that; it has been in use for a while as primary. CentOS 5 NFSv4 clients; CentOS 6 NFSv4 servers and clients; CentOS6 and 7 NFSv4.1 clients; CentOS 7 and NetApp as NFSv4.1 servers. (The NetApp has pNFS on by default; had to disable -- unroutable subnets.)

A beauty of NFSv4 is that client connects to the NFS-port. NFSv3 client does query portmapper of the server for various sub-services. Many ports to open on server's firewall. NFSv4 server does all that via one port. The NFSv4 has annoying callback from server to client, but NFSv4.1 apparently does not.

The harder part is idmapping. While NFSv3 sends numeric ids, the NFSv4 sends strings that server's idmapper must resolve to some ids. In other words, NFSv3 server does not need to know about any users.

NFSv4 introduces GSS security; the traffic can be encrypted and authentication can be more than "sudo my name is fubar". That has been backported to NFSv3, at least by Red Hat. Good, if you cannot trust your users. Tried that too, but had some issues.

Yes, there have been kinks and oddballs, but overall the NFSv4 in CentOS has been quite solid.


Note: autofs (I like it better than static /etc/fstab. However, in CentOS 7 an entry in fstab can autogenerate an systemd automount.service.) defaults to NFSv4 and falls back to NFSv3. IIRC, default was 3 up to CentOS 5, but changed to 4 in CentOS 6. Might apply to fstab mounts too, but as said, I use autofs with explicit maps (in LDAP).

Scratch that:
CentOS 6# man 5 nfs
The fstype field contains "nfs". Use of the "nfs4" fstype in /etc/fstab is deprecated.
# man 5 nfsmount.conf
Leads you to where the default is set.

NdFeB
Posts: 9
Joined: 2017/02/03 16:39:04

Re: NFSv4, NetApp, CentOS 6.8 : your thought ?

Post by NdFeB » 2017/02/08 18:17:50

Hi !

Thank you for those informations, and the feedback about NFSv4. I guess I just learnt that I better never mount a volume previously mounted with NFSv3. At least from a NetApp. Cool ! Some tests to do :D
Yeah, i know idmapping is a pain in the ass. I'm still struggling with it : i got the integration to work with a basic idmap configuration, but it was incrementative, and I need the Linux boxes to map SIDs to precise UID/GID to be able to share ACLs on both OS, and on all servers individually integrated. But this is an other story.

I never used autofs, only /etc/fstab. Will check it out.

I can't really do any test at the moment, as we are working on other priorities this week, and I'm going to school for the next 2 weeks. This thread will look like dying, but please don't lock it !

Regards.

NdFeB
Posts: 9
Joined: 2017/02/03 16:39:04

Re: NFSv4, NetApp, CentOS 6.8 : your thought ?

Post by NdFeB » 2017/04/15 10:32:12

Hi guys !

Sorry for the late feedback. So, I tested NFSv4 on the NetApp. It works well, but was not adapted to our needs. I also ended up to configure rid idmap backend for my winbind service, and it works well : all AD users/groups are mapped to the same UID/GID on all Linux boxes.

At first, to anyone who has a NetApp : check on clients that the default version for mount.nfs is v3, then activate NFSv4. If you don't check this, you will have protocol failures when disabling it ! Sounds obvious, but I got suckered lol.

At first, I wanted to use NFS to share a volume from the filer to a single host, ans this host would share it to the whole network. This way we can manage ACL with Samba tools from NT clients, = centralized administration. But the delegation of ACL control works really bad : I could't get the samba server to make authority on the ACL, I got many "denied permissions" while this server was supposed to have root access to the volume. I also tried many different configurations on the filer (security flavor, initial protocol, etc.), but it didn't work better. I think I will sniff some network to understand better why it fails with this setup.

I came to this conclusion : if you want a heterogeneous share, physically stored on a NetApp filer (or any other SAN i guess), and want a centralized ACL management (it works from a Windows, I didnt try to set acl / xattr from a Linux client), the best way is to dedicate a LUN to this share, and setup an iSCSI link between this LUN and the samba server. This way, you format it as ext4, and mount it as a local HDD.

Now the Linux samba server has an absolute control on the share : no matter what happens, the filer will NEVER be involved in permission checking, only writing raw data on physical disks.

I am actually writing a complete tutorial on how to setup such a share with, optionally, an iSCSI link setup with a NetApp. If you guys are interested, I can post it on this forum, because I have no idea where to post it to share it with a good visibility.

A last word : why an intermediary samba server ? Because ACL management on a simple NetApp heterogeneous share is a pain in the ass, or just not possible at all (ACL applying only on Windows, or only on Linux, permission denied, wrong charset, etc.).

Thanks for your time, and your experience sharing about NFSv4 ! See you soon if you want this same setup !

aks
Posts: 3073
Joined: 2014/09/20 11:22:14

Re: NFSv4, NetApp, CentOS 6.8 : your thought ?

Post by aks » 2017/04/18 17:51:43

Not read all posts ('cause life is so short), but a tupence worth. But actually NFSv4 removed passing authentication as POSIX attributes (UID/GID) and replaced them with M$ authentication strings (user@domain). Apart from that, every NFSv4 file access is 100% "faster" than NFSv3 as NFSv3 actually must do two access for every file, while NFSv4 does one. Also you'll be in a better place for pNFS (parallel NFS - some products are available but am not sure if it's standardised yet). Also NFSv4 is more firewall friendly ....

Post Reply