Compilers converted to run PuppetDB relying on high-latency connections experience slowdowns in Puppet Enterprise 2019.8

Compilers converted to run PuppetDB that rely on high-latency connections might experience significant slowdowns. You might be affected by this issue if, for example, your primary server and compilers are distributed among multiple data centers connected by high-latency links or congested network segments. If so, use the steps below to remove PuppetDB from your compilers to improve performance.

After completing the steps, when you upgrade your compiler using puppet infra upgrade compiler, you will get a warning telling you to convert your legacy compiler (with no PuppetDB service) to use PuppetDB. That warning might be updated in a future release if there’s a continued need for legacy compilers.

Version and installation information

PE version: 2019.8

Solution

Remove PuppetDB from your compilers using these steps.

1. On each compiler, stop the pe-puppetdb and the puppet service.

   puppet resource service puppet ensure=stopped
    puppet resource service pe-puppetdb ensure=stopped enable=false

2. Pin compiler nodes to the PE Master node group. In the console, click Node Groups (in earlier versions of 2019.8, click Classification). In the PE Infrastructure group, select the PE Master group. On the Rules tab, in the pinned nodes section, enter each compiler certname, click Pin node, and then commit changes.

3. On each compiler, in /etc/puppetlabs/puppet/csr_attributes.yaml remove the pp_auth_role setting by removing the following line under extension_requests:

   pp_auth_role: pe_compiler

4. Regenerate compiler certificates using the puppet infra command or manually. Using the command is easier and less error-prone.

   To regenerate compiler certificates with the puppet infra command:

   You can use the puppet infra command to regenerate compiler certificates. Run the following on the primary server:

   puppet infra run regenerate_agent_certificate agent=<COMPILER CERTNAME> node_type=pe_compiler dns_alt_names=<DNS_ALT_NAMES>

   To regenerate compiler certificates manually

   A. If your compilers use a DNS alternative names, make sure dns_alt_names are set on each compiler in /etc/puppetlabs/puppet/puppet.conf

   B. On each compiler, clean certificates.

   puppet ssl clean

   C. On the primary server, clean compiler certificates.

   puppetserver ca clean --certname <COMPILER CERTNAME>

   D. On each compiler, submit a new certificate request.

   puppet ssl submit_request

   E. On the primary server, sign the new compiler certificates.

   puppetserver ca sign --certname <COMPILER CERTNAME>

   F. On each compiler, run puppet agent.

   puppet agent -t

5. On each compiler, run puppet.

   puppet agent -t

6. If you have an external PostgreSQL node, run puppet on the PostgreSQL node.

   puppet agent -t

7. On the primary server and infrastructure nodes (compilers and replicas), run puppet.

   puppet agent -t

8. On each compiler, remove the pe-puppetdb package.

   puppet resource package pe-puppetdb ensure=purged

9. On each compiler, remove the pe-puppetdb user.

   puppet resource user pe-puppetdb ensure=absent

10. On each compiler, remove pe-puppetdb directories.

    rm -rf /etc/puppetlabs/puppetdb
    rm -rf /var/log/puppetlabs/puppetdb
    rm -rf /opt/puppetlabs/server/data/puppetdb

11. Verify that pe-puppetdb service was removed from the compilers.

    A. On each compiler, run the following. If pe-puppetdb was removed, it won’t be in the output.

    puppet resource service | grep pe-

    B. On the primary server, run the following, if pe-puppetdb was removed, the compilers won’t have pe-puppetdb service.

    puppet infra status

12. On each compiler, start the puppet service.

    puppet resource service puppet ensure=running