After upgrading Continuous Delivery for Puppet Enterprise (PE), you try to log in to Puppet Application Manager (PAM), but it doesn’t load. In the command line you get an
Insufficient cpu error from Kubernetes.
Warning FailedScheduling 99m (x1 over 100m) default-scheduler 0/1 nodes are available: 1 Insufficient cpu.
During an upgrade, Kubernetes starts up another application instance on your control plane node to migrate data from one copy of the application to the other. You get this message when there are not enough CPU resources to schedule both application instances.
Version and installation information
Product: Continuous Delivery for Puppet Enterprise
To fix the issue, you can either add more CPUs or remove optional PAM components.
Adding more CPUs is a long-term solution that helps prevent future issues. If the time it takes to add resources causes too much downtime, roll back to a pre-upgrade system snapshot before adding more cores (CPUs).
Remove optional PAM components
Removing an optional PAM component to free up resources is a quick solution but might not resolve the problem the next time you upgrade.
The best components to remove in this case are Prometheus and Grafana, which are used for metrics. Since you are most likely trying to resolve this issue during an upgrade, keep Velero and Restic, the optional components used for snapshots.
To remove Prometheus and Grafana, run
kubectl delete ns/monitoring
If removing Prometheus and Grafana doesn’t work, you can add more CPUs to resolve the issue.
How can we improve this article?
Please sign in to leave a comment.