Change the log level for PE services in Puppet Enterprise 2016.1 and later – Puppet Support

Support might ask you to increase the log level from info (the default) to debug to gather additional information when troubleshooting a problem with PE services. For example, increasing the log level for pe-puppetserver is helpful for troubleshooting Code Manager and file sync issues.

Version and installation information

PE version: 2016.1 and later (manual version), 2017.3 and later (tasks versions)
OS: Any
Installation type: Any

Solution

Change the log level by editing the service's logback.xmlroot-level setting and restarting the service. You can do this manually, or you can install a module that uses a task to complete the steps.

Note: When you finish gathering information, it is imperative that the log level is changed back to info to avoid exposing sensitive data. Before you send debug level logs to us, review them for sensitive information, such as hostnames and IP addresses. In addition, debug-level logs fill up disk space quickly.

We changed our terminology, the primary server was called the master in versions of PE earlier than 2019.8.

To complete the steps manually:

The following command, uses puppet apply with an augeas resource to edit the service’s logback.xmlroot-level setting and restart the service.

Set the value for FACTER_level to: debug to increase the log level, or info to decrease the log level to the default level. Set the value for FACTER_service to the affected service: puppetserver, puppetdb, console-services, or orchestration-services

FACTER_level="<DEBUG OR INFO>" FACTER_service="<SERVICE NAME>" puppet apply -e "augeas {'toggle logging level': incl => \"/etc/puppetlabs/$::service/logback.xml\", lens => 'Xml.lns', context => \"/files/etc/puppetlabs/$::service/logback.xml/configuration/root/#attribute\", changes => \"set level \'$::level\'\"}~> service {\"pe-$::service\": ensure => running }"

Examples

Set pe-puppetserver to debug-level logging:

# FACTER_level="debug" FACTER_service="puppetserver" puppet apply -e "augeas {'toggle logging level': incl => \"/etc/puppetlabs/$::service/logback.xml\", lens => 'Xml.lns', context => \"/files/etc/puppetlabs/$::service/logback.xml/configuration/root/#attribute\", changes => \"set level \'$::level\'\"}~> service {\"pe-$::service\": ensure => running }"

Notice: Compiled catalog for pe-201642-server.puppetdebug.vlan in environment production in 0.31 seconds Notice: /Stage[main]/Main/Augeas[toggle logging level]/returns: executed successfully Notice: /Stage[main]/Main/Service[pe-puppetserver]: Triggered 'refresh' from 1 events Notice: Applied catalog in 35.43 seconds # tail -3 /var/log/puppetlabs/puppetserver/puppetserver.log 2016-12-15 16:11:23,076 INFO [async-dispatch-2] [p.e.s.m.master-service] Puppet Server has successfully started and is now ready to handle requests 2016-12-15 16:11:23,077 DEBUG [main] [p.t.internal] Registering SIGHUP handler for restarting TK apps 2016-12-15 16:11:23,079 DEBUG [async-dispatch-2] [p.t.internal] Lifecycle worker completed :boot lifecycle task; awaiting next task.

Set puppetserver to info-level logging:

# FACTER_level="info" FACTER_service="puppetserver" puppet apply -e "augeas {'toggle logging level': incl => \"/etc/puppetlabs/$::service/logback.xml\", lens => 'Xml.lns', context => \"/files/etc/puppetlabs/$::service/logback.xml/configuration/root/#attribute\", changes => \"set level \'$::level\'\"}~> service {\"pe-$::service\": ensure => running }"

Notice: Compiled catalog for pe-201642-server.puppetdebug.vlan in environment production in 0.32 seconds Notice: /Stage[main]/Main/Augeas[toggle logging level]/returns: executed successfully Notice: /Stage[main]/Main/Service[pe-puppetserver]: Triggered 'refresh' from 1 events Notice: Applied catalog in 41.88 seconds # tail -3 /var/log/puppetlabs/puppetserver/puppetserver.log 2016-12-15 16:13:56,983 INFO [async-dispatch-2] [p.e.s.a.analytics-service] Puppet Server Analytics has successfully started and will run in the background 2016-12-15 16:13:56,986 INFO [async-dispatch-2] [p.s.l.legacy-routes-service] The legacy routing service has successfully started and is now ready to handle requests 2016-12-15 16:13:56,988 INFO [async-dispatch-2] [p.e.s.m.master-service] Puppet Server has successfully started and is now ready to handle requests

To use the task:

Note: You can only complete the task-based steps in PE 2017.3 and later.

You can run the task from the command line either by using the puppet task command or by using Bolt. If you’re using Bolt with the default SSH transport (and not the PCP protocol), you will avoid getting an error when Puppet services restart. However, either method will set the log level and restart the service.

To run the task that changes the PE service log levels, you must download and install the puppetlabs-support_tasks module, which includes the task for this solution. If you’re using a version of the puppetlabs-support_tasks module older than 1.1.1 (2020 17 January), please update the module to use these steps. We renumbered all the tasks in the module at that time.

Run the task on the command line:

On the primary server, run the task against the certname of the target infrastructure node. Change the value for loglevel to debug to increase the log level, or info to decrease the log level to the default level and change the value for service to the affected service: puppetserver, puppetdb, console-services, or orchestration-services

puppet task run support_tasks::st0009_change_pe_service_loglevel loglevel="<DEBUG OR INFO>" service="<SERVICE NAME>" -n $(puppet config print certname)

Note: The task restarts the target Puppet service. For console-services and orchestration-services, this restart will cause a connection error. You can safely ignore the error while the task continues to run in the background.

Example

To set the debug level for the puppetserver service, on the primary server run the following:

puppet task run support_tasks::st0009_change_pe_service_loglevel loglevel="debug" service="puppetserver" -n $(puppet config print certname)

Run the task using Bolt and SSH

To avoid errors when services are restarted during the task, use Bolt with the default SSH transport (and not the PCP protocol).

bolt task run support_tasks::st0009_change_pe_service_loglevel loglevel="<DEBUG OR INFO>" service="<SERVICE NAME>" -n $(puppet config print certname) --modulepath="/etc/puppetlabs/code/environments/production/modules"