Puppet runs stall with offline hard NFS mounts in Puppet Enterprise

If your Puppet Enterprise deployment includes offline hard NFS mounts that don’t have an active timeout or retransmission limit, your Puppet runs on nodes can stall and never complete.

Error messages and logs

When you run Puppet in debug mode on a node, Facter fails to complete. The run becomes unresponsive on a message similar to the following:

}.
Debug: Facter: resolving timezone facts.
Debug: Facter: fact "timezone" has resolved to "CDT".
Debug: Facter: resolving file system facts.

Version and installation information

PE version: All supported

Solution

Offline hard NFS mounts can prevent servers from rebooting. If NFS mounts don’t have an active timeout or retransmission limit, any NFS action prevents runs from completing. This is an expected and intentional behavior for offline NFS hard mounts in your OS.

To fix the issue, set a timeout or retransmission rate (soft mount) for your NFS mounts, for example:

'tcp,nolock,rsize=32768,wsize=32768,soft,noatime,actimeo=3,retrans=1',

If you have issues changing this setting, ask your OS vendor Support for help. If you continue to have issues after changing it, please open a ticket with us.