How to fix node not found in Kubernetes
The "node not found" error occurs when kubelet fails to register with the Kubernetes API server during cluster initialization or after a node restart. This prevents the node from joining the cluster and scheduling pods.
Quick fix
Fixes node not found
ssh <node-ip>
sudo systemctl status kubelet- Check kubelet status on the node
- Restart kubelet to force re-registration
- Verify hostname matches configuration
- Verify API server is running and accessible
ssh <node-ip>sudo systemctl status kubeletnode not found
What this error means
When you see "Error getting node err=node not found" in kubelet logs, it means the kubelet on a worker node cannot find itself registered in the API server. This typically happens during cluster initialization when kubelet tries to register before the API server is ready, or after a reboot when node registration fails. Kubelet registers itself with the API server by creating a Node object. If this fails, the node cannot join the cluster and cannot schedule pods. The error appears in kubelet logs, not in kubectl commands.
Typical symptoms
- Kubelet logs show "Error getting node err=node not found" repeatedly
- kubectl get nodes doesn't display the node (master or worker)
- Container runtime reports "network not ready" errors
- kubeadm init or kubeadm join hangs or times out
- Worker nodes fail to join the cluster during initialization
- After server reboot, kubelet cannot find the node in API server state
- Kubelet continuously fails to register with API server during startup
Common causes
- Kubelet service crashes or fails to start due to misconfiguration
- API server not running on port 6443 or firewall blocking TCP connections
- Hostname mismatches (KUBELET_HOSTNAME in kubelet config doesn't match system hostname)
- Docker or container runtime incompatibility or not running
- CNI (pod network) plugin not installed, preventing kubelet startup
- Node registration timing—kubelet tries to register before API server is ready
- Container runtime (Docker, CRI-O, containerd) not properly initialized
How to fix "node not found"
SSH to the affected node and verify kubelet is running:
ssh <node-ip>
sudo systemctl status kubeletIf not running, check why:
sudo journalctl -xeu kubelet # Detailed logs with explanations
sudo journalctl -u kubelet -n 100 # Last 100 kubelet log linesLook for errors about API server, configuration, or runtime connectivity.
Kubelet should automatically re-register with the API server after restart:
sudo systemctl restart kubeletMonitor registration from the control plane:
kubectl get nodes -w # Watch for the node to appearWatch logs from control plane:
kubectl logs -n kube-system <api-server-pod> # Check for registration eventsGive it 30-60 seconds to register.
Check the system hostname:
hostnamectl
hostname -fVerify it matches kubeadm configuration or kubelet flags. If using kubeadm, check:
cat /etc/kubernetes/kubelet.conf | grep hostnameIf there's a mismatch, either:
- Change the hostname: sudo hostnamectl set-hostname <new-name>
- Update kubelet config to match the actual hostname
Ensure master and worker nodes have different hostnames.
From the node, test API server connectivity:
kubectl cluster-info # From control plane
kubectl get nodes # From control plane
kubectl --kubeconfig=/etc/kubernetes/kubelet.conf get nodes # Using kubelet credentialsTest port 6443:
netstat -tlnp | grep 6443 # On control plane
curl -k https://<control-plane-ip>:6443/api/v1 # From nodeIf API server isn't listening or responding, it must be fixed first.
Check Docker or other container runtime:
sudo systemctl status docker # For Docker
sudo systemctl status containerd # For containerd
sudo systemctl status crio # For CRI-OIf not running, start it:
sudo systemctl start dockerTest container runtime connectivity:
sudo docker version
sudo ctr version # For containerdKubelet may fail startup if pod network (CNI) is missing. Check from control plane:
kubectl get pods -n kube-system | grep -E "flannel|calico|weave" # Check for CNI podsIf missing, install the network plugin:
# Flannel example:
kubectl apply -f https://raw.githubusercontent.com/coreos/flannel/master/Documentation/kube-flannel.ymlThen restart kubelet on the node:
sudo systemctl restart kubeletVerify the node can reach the API server on port 6443:
sudo firewall-cmd --list-all # If using firewalld
sudo iptables -L -n # If using iptablesOpen port if needed:
sudo firewall-cmd --permanent --add-rich-rule='rule family="ipv4" destination address="<control-plane-ip>" port protocol="tcp" port="6443" accept'
sudo firewall-cmd --reloadFor testing, temporarily disable firewall:
sudo systemctl stop firewalldSome Kubernetes versions have compatibility issues with specific Docker versions. Check:
docker version
kubectl version # From control planeDocker 18.6-18.9 have known issues with Kubernetes. If incompatible, upgrade Docker:
sudo apt-get update
sudo apt-get install docker-ce=<version> # Specific versionThen restart kubelet.
When joining workers to a new cluster, control plane must be fully ready first:
# On control plane, wait for this to succeed:
kubectl get nodes
# Only then run on worker:
kubeadm join <control-plane-ip>:6443 --token <token> --discovery-token-ca-cert-hash sha256:<hash>Wait 2-3 minutes for control plane to stabilize before joining workers. Monitor kubelet logs during join:
sudo journalctl -u kubelet -fAdvanced notes
Node registration happens automatically—kubelet polls the API server with backoff until successful. If logs show repeated registration attempts failing, one of the underlying issues (API server down, hostname mismatch, firewall) must be fixed. For kubeadm-managed clusters, the kubeadm join command handles most configuration automatically. For self-managed clusters, ensure kubelet systemd unit or kubelet startup script properly configures API server address. On cloud providers (AWS, GCP, Azure), node registration typically succeeds automatically unless there are IAM or network issues at the cloud layer. WSL2-based Kubernetes may have temporary registration delays during VM startup—give it extra time. In air-gapped environments, ensure all required images and binaries are pre-downloaded before starting cluster initialization.
Frequently asked questions
- What does "node not found" mean?
- When you see "Error getting node err=node not found" in kubelet logs, it means the kubelet on a worker node cannot find itself registered in the API server. This typically happens during cluster initialization when kubelet tries to register before the API server is ready, or after a reboot when node registration fails. Kubelet registers itself with the API server by creating a Node object.
- How do I fix "node not found"?
- The "node not found" error occurs when kubelet fails to register with the Kubernetes API server during cluster initialization or after a node restart. This prevents the node from joining the cluster and scheduling pods.
- What causes "node not found"?
- Kubelet service crashes or fails to start due to misconfiguration API server not running on port 6443 or firewall blocking TCP connections Hostname mismatches (KUBELET_HOSTNAME in kubelet config doesn't match system hostname)
- Is the "node not found" error serious?
- Yes. This is a high-severity error that usually blocks work until it is fixed, but the steps above resolve it.
Official resources & further reading
- 📚Kubernetes Official Documentation
- 📖K8s: Nodes(Official Docs)
- 🐙K8s GitHub Issues(GitHub)
- 💬SO: K8s Nodes(Stack Overflow)