How to fix KubeletNotReady in Kubernetes
KubeletNotReady occurs when the kubelet service on a worker node becomes unhealthy or stops communicating with the control plane. This is often caused by container runtime failures, certificate issues, or systemd service problems.
Quick fix
Fixes KubeletNotReady
kubectl get nodes
kubectl describe node <node-name>- Check node status and conditions
- SSH into the node and check kubelet service status
- Verify container runtime is healthy
- Disable swap and fix cgroup driver mismatches
kubectl get nodeskubectl describe node <node-name>KubeletNotReady
What this error means
The KubeletNotReady condition signals that the kubelet—the Kubernetes agent running on each node responsible for managing pods and containers—is either down, misconfigured, or unable to communicate with the API server. When this occurs, the control plane marks the node as 'NotReady' and stops scheduling new pods to that node. This error appears in the node status conditions and typically means the kubelet process is not running, the systemd service has failed, or the kubelet cannot establish a healthy connection to the control plane due to certificate validation failures, container runtime issues, or network problems. Underlying causes range from transient issues (restarting the kubelet fixes them immediately) to persistent configuration problems (cgroup driver mismatches, missing container runtime endpoints) and infrastructure issues (disk full, swap enabled, network isolation, certificate expiration).
Typical symptoms
- Node shows "NotReady" status when running "kubectl get nodes"
- All node conditions (Ready, MemoryPressure, DiskPressure, PIDPressure) show "Unknown" in kubectl describe node
- Pod events show "Unknown" node status preventing scheduling
- Existing pods on the node are evicted and rescheduled to other nodes after a grace period
- Kubelet process is not running or systemctl shows "Unit entered failed state"
- Connection timeout errors when kubelet tries to reach the API server (port 6443)
Common causes
- Container runtime (Docker, containerd) is not running or its endpoint is misconfigured
- Kubelet systemd service crashed or failed to start due to misconfiguration (swap enabled, missing config file, cgroup driver mismatch)
- TLS certificate validation failures—kubelet client certificate expired or signed by wrong CA
- CNI plugin not initialized or missing—node cannot establish pod networking
- Resource exhaustion on the node—insufficient disk space, memory pressure, or PID pressure
- Network connectivity issues—node isolated from cluster network or firewall blocking ports
- Kubelet configuration file missing or invalid
How to fix "KubeletNotReady"
Verify the node is actually NotReady and identify which conditions are reporting 'Unknown':
kubectl get nodes
kubectl describe node <node-name>Look at the 'Conditions' section. If all conditions show 'Unknown', the kubelet is down. If specific conditions (MemoryPressure, DiskPressure) show True, you have a resource exhaustion issue. The 'Status' field and 'Last Heartbeat' time tell you when the kubelet last communicated.
Connect directly to the affected worker node and inspect the kubelet systemd service:
ssh <node-ip>
sudo systemctl status kubelet
sudo journalctl -xu kubelet.service --no-pager | tail -50The journalctl output shows the actual error preventing kubelet from starting. Common errors include 'running with swap on is not supported', 'container runtime endpoint address was not specified', or 'misconfiguration: kubelet cgroup driver'.
Check if the container runtime (Docker or containerd) is running:
# For containerd
sudo systemctl status containerd
sudo systemctl restart containerd
# For Docker
sudo systemctl status docker
sudo systemctl restart dockerEnsure the kubelet configuration points to the correct runtime endpoint. If using containerd:
# Check kubelet extra args
cat /etc/systemd/system/kubelet.service.d/0-containerd.conf
# Should contain:
# Environment="KUBELET_EXTRA_ARGS=--container-runtime=remote --container-runtime-endpoint=unix:///run/containerd/containerd.sock"Swap must be disabled for kubelet to function. Permanently disable it:
# Immediately disable
sudo swapoff -a
# Permanently disable by editing fstab
sudo nano /etc/fstab
# Comment out any line containing 'swap'If kubelet and container runtime have mismatched cgroup drivers, edit the kubelet config:
sudo nano /etc/systemd/system/kubelet.service.d/10-kubeadm.conf
# Add: Environment="KUBELET_CGROUP_ARGS=--cgroup-driver=systemd"
# Then reload and restart
sudo systemctl daemon-reload
sudo systemctl restart kubeletIf the error mentions 'NetworkPluginNotReady' or '/etc/cni/net.d' is empty, install a CNI plugin:
# First check if CNI config exists
ls /etc/cni/net.d/
# If empty, apply a network plugin (example: Weave)
kubectl apply -f https://github.com/weaveworks/weave/releases/download/v2.8.1/weave-daemonset-k8s.yaml
# Or for Flannel
kubectl apply -f https://raw.githubusercontent.com/coreos/flannel/master/Documentation/kube-flannel.yml
# Restart kubelet and wait for CNI pod to initialize
sudo systemctl restart kubeletCheck that kubelet client certificates are valid and trusted by the API server:
# Check certificate expiration
sudo openssl x509 -in /var/lib/kubelet/pki/kubelet-client-current.pem -text -noout | grep -A2 'Validity'If certificates are expired or invalid, regenerate them (on control plane):
sudo kubeadm certs renew allThen restart kubelet:
sudo systemctl daemon-reload
sudo systemctl restart kubelet
# Monitor startup
sudo journalctl -xu kubelet.service -fAdvanced notes
Kubelet operates as a systemd service with configuration split across multiple files: the base unit file (/etc/systemd/system/kubelet.service), kubeadm drop-in configs (/etc/systemd/system/kubelet.service.d/), and the kubelet config file (/var/lib/kubelet/config.yaml). Changes to drop-in files require systemctl daemon-reload before taking effect.
The kubelet communicates with the API server using mutual TLS—the client certificate in /var/lib/kubelet/pki/ must be valid and signed by the cluster CA, while the kubeconfig at /etc/kubernetes/kubelet.conf must point to the correct CA file.
Cgroup driver selection (systemd vs cgroupfs) must match the container runtime's choice, typically configured during kubeadm init with --cgroup-driver=systemd. On systems with cgroup v2, additional kernel parameters may be required.
The --fail-swap-on=false flag can temporarily bypass swap checks for troubleshooting, but swap should be permanently disabled before production use.
Frequently asked questions
- What does "KubeletNotReady" mean?
- The KubeletNotReady condition signals that the kubelet—the Kubernetes agent running on each node responsible for managing pods and containers—is either down, misconfigured, or unable to communicate with the API server. When this occurs, the control plane marks the node as 'NotReady' and stops scheduling new pods to that node. This error appears in the node status conditions and typically means the kubelet process is not running, the systemd service has failed, or the kubelet cannot establish a healthy connection to the control plane due to certificate validation failures, container runtime issues, or network problems.
- How do I fix "KubeletNotReady"?
- KubeletNotReady occurs when the kubelet service on a worker node becomes unhealthy or stops communicating with the control plane. This is often caused by container runtime failures, certificate issues, or systemd service problems.
- What causes "KubeletNotReady"?
- Container runtime (Docker, containerd) is not running or its endpoint is misconfigured Kubelet systemd service crashed or failed to start due to misconfiguration (swap enabled, missing config file, cgroup driver mismatch) TLS certificate validation failures—kubelet client certificate expired or signed by wrong CA
- Is the "KubeletNotReady" error serious?
- Yes. This is a critical-severity error that usually blocks work until it is fixed, but the steps above resolve it.
Official resources & further reading
- 📚Kubernetes Official Documentation
- 📖Kubernetes: Kubelet Configuration(Official Docs)
- 📖Kubernetes: Nodes(Official Docs)
- 💬Stack Overflow: KubeletNotReady(Stack Overflow)