I recently turned up a new RAID array and plopped an XFS filesystem down on it. I didn’t bother setting any specific tunings when I created the filesystem. However I couldn’t for the life of me export any subdirectories from the volume over NFS. Local access was fine and I could export via netatalk and samba.
On the server I saw messages like this in the logs:
Feb 14 13:08:43 monolith rpc.mountd[3092]: authenticated mount request from 192.168.1.50:1003 for /mnt/music (/mnt/music) Feb 14 13:08:57 monolith rpc.mountd[3092]: authenticated mount request from 192.168.1.50:1002 for /opt/music (/opt/music) Feb 14 13:15:19 monolith rpc.mountd[3092]: authenticated mount request from 192.168.1.20:717 for /mnt/music (/mnt/music) Feb 14 13:15:20 monolith rpc.mountd[3092]: authenticated mount request from 192.168.1.20:1001 for /mnt/music (/mnt/music) Feb 14 13:15:22 monolith rpc.mountd[3092]: authenticated mount request from 192.168.1.20:1002 for /mnt/music (/mnt/music) Feb 14 13:15:26 monolith rpc.mountd[3092]: authenticated mount request from 192.168.1.20:801 for /mnt/music (/mnt/music) Feb 14 13:15:34 monolith rpc.mountd[3092]: authenticated mount request from 192.168.1.20:967 for /mnt/music (/mnt/music) Feb 14 13:15:44 monolith rpc.mountd[3092]: authenticated mount request from 192.168.1.20:794 for /mnt/music (/mnt/music) Feb 14 13:15:54 monolith rpc.mountd[3092]: authenticated mount request from 192.168.1.20:855 for /mnt/music (/mnt/music) Feb 14 13:16:04 monolith rpc.mountd[3092]: authenticated mount request from 192.168.1.20:863 for /mnt/music (/mnt/music) Feb 14 13:16:14 monolith rpc.mountd[3092]: authenticated mount request from 192.168.1.20:932 for /mnt/music (/mnt/music) Feb 14 13:16:24 monolith rpc.mountd[3092]: authenticated mount request from 192.168.1.20:830 for /mnt/music (/mnt/music)
On the client I would get two different behaviours, depending on whether it was NFSv4 or NFSv3 that was being used. With NFSv4 it would mount the directory, but any attempt to read from it would give a ‘Stale NFS handle’ error:
root:~# mount -t nfs -v 192.168.1.10:/mnt/music /mnt/ mount.nfs: timeout set for Fri Feb 14 16:49:39 2014 mount.nfs: trying text-based options 'vers=4,addr=192.168.1.10,clientaddr=192.168.1.20' root:~# ls /mnt/ ls: cannot open directory /mnt/: Stale NFS file handle
With NFSv3 it would just time out, although if I used the -v switch I could see that it was timing out because it was getting ‘Stale NFS handle’ errors during the attempt to mount the directory:
root:~# mount -t nfs -o nfsvers=3 -v 192.168.1.10:/mnt/music /mnt/ mount.nfs: timeout set for Fri Feb 14 16:51:29 2014 mount.nfs: trying text-based options 'nfsvers=3,addr=192.168.1.10' mount.nfs: prog 100003, trying vers=3, prot=6 mount.nfs: trying 192.168.1.10 prog 100003 vers 3 prot TCP port 2049 mount.nfs: prog 100005, trying vers=3, prot=17 <SNIP> mount.nfs: prog 100003, trying vers=3, prot=6 mount.nfs: trying 192.168.1.10 prog 100003 vers 3 prot TCP port 2049 mount.nfs: prog 100005, trying vers=3, prot=17 mount.nfs: trying 192.168.1.10 prog 100005 vers 3 prot UDP port 60925 mount.nfs: mount(2): Stale NFS file handle mount.nfs: Connection timed out
After much frustrated googling I finally stumbled across the answer: inode64. By default NFS still exports 32bit inodes, and XFS by default uses 64bit inodes. There are a few workarounds that worked for me:
1) Export the root of the filesystem and not subdirectories. If you’re in a high trust environment this might be a reasonable solution. However you might not want to export everything on your filesystem, in which case you should look at option 2.
2) You can also manually set the fsid option for each exported subdirectory like this:
/mnt/music 192.168.1.0/24(rw,async,wdelay,insecure,no_root_squash,no_subtree_check,fsid=1) /mnt/pictures 192.168.1.0/24(rw,async,wdelay,insecure,no_root_squash,no_subtree_check,fsid=2)
I’ve tested both of the above with the following clients: Ubuntu 13.10, FreeBSD 9.2, and Mac OS X 10.9. I don’t actually have any older clients to test with at the moment.