Regenerating a primary server certificate in Puppet Enterprise fails

Your primary server certificate generated by the Puppet CA expired. When you use steps in our documentation to regenerate it with a plan, the plan fails.

Error messages and logs

When you run Puppet:

Error: The certificate 'CN=puppet.primary.com has expired, verify time is synchronized

Error: Could not run: The certificate 'CN=puppet.primary.com has expired, verify time is synchronized

In the PuppetDB log (at /var/log/puppetlabs/puppetdb/puppetdb.log):

2022-04-04T16:45:15.489-07:00 ERROR [p.p.s.pe-routing] Error getting RBAC status information

javax.net.ssl.SSLHandshakeException: Received fatal alert: certificate_unknown

When you run the regenerate_master_certificate plan:

puppet infrastructure run regenerate_master_certificate force=true

Master_cert_regen: {action => run_task, object => enterprise_tasks::clean, result_set => [{"target":"localhost","action":"task","object":"enterprise_tasks::clean","status":"failure","value":{"_error":{"kind":"puppetlabs.certregen/clean-failed","msg":"Unable to clean host certificate for primary.puppet.com from CA puppet.primary.com","details":{"output":"Fatal error when running action 'clean'\n  Error: Failed connecting to https://puppet.primary.com:8140/puppet-ca/v1/certificate_status/\n  Root cause: Failed to open TCP connection to puppet.primary.com:8140 (Connection refused - connect(2) for \"puppet.primary.com\" port 8140)\n"}}}}]}

run_task 'enterprise_tasks::clean' failed on 1 target

Version and installation information

PE version: All supported

Solution

If you can’t complete the steps in the documentation to regenerate your primary certificate, run the steps in the plan manually. Complete all steps on the primary server. If you have issues with any of the steps, please open a ticket and let our team know.

1. Stop agent and PXP agent services. Run:

   puppet resource service puppet ensure=stopped puppet resource service pxp-agent ensure=stopped

2. Back up the directory containing SSL certs. Run:

   cp -r /etc/puppetlabs/puppet/ssl/ /etc/puppetlabs/puppet/ssl_$(date -Is)/

3. Remove the cached catalog. Run:

   rm -f /opt/puppetlabs/puppet/cache/client_data/catalog/$(puppet config print certname).json

4. Revoke the primary server certificate. Run:

   /opt/puppetlabs/bin/puppetserver ca clean --certname $(puppet config print certname)

5. Delete the old certificates:

   find /etc/puppetlabs/puppet/ssl -name $(puppet config print certname).pem -delete

6. Rebuild certificates. Run:

   puppet infrastructure configure --no-recover

   You might get the following error, if you do, it’s okay to continue to the next step.

   Error: Failed to initialize SSL: The private key is missing from '/etc/puppetlabs/puppet/ssl/private_keys/primary.puppet.com' Error: Run puppet agent -t`

7. To complete the process, run puppet agent -t