The "node not found" error occurs when kubelet fails to register with the Kubernetes API server during cluster initialization or after a node restart. This prevents the node from joining the cluster and scheduling pods.
When you see "Error getting node err=node not found" in kubelet logs, it means the kubelet on a worker node cannot find itself registered in the API server. This typically happens during cluster initialization when kubelet tries to register before the API server is ready, or after a reboot when node registration fails. Kubelet registers itself with the API server by creating a Node object. If this fails, the node cannot join the cluster and cannot schedule pods. The error appears in kubelet logs, not in kubectl commands.
SSH to the affected node and verify kubelet is running:
ssh <node-ip>
sudo systemctl status kubeletIf not running, check why:
sudo journalctl -xeu kubelet # Detailed logs with explanations
sudo journalctl -u kubelet -n 100 # Last 100 kubelet log linesLook for errors about API server, configuration, or runtime connectivity.
Kubelet should automatically re-register with the API server after restart:
sudo systemctl restart kubeletMonitor registration from the control plane:
kubectl get nodes -w # Watch for the node to appearWatch logs from control plane:
kubectl logs -n kube-system <api-server-pod> # Check for registration eventsGive it 30-60 seconds to register.
Check the system hostname:
hostnamectl
hostname -fVerify it matches kubeadm configuration or kubelet flags. If using kubeadm, check:
cat /etc/kubernetes/kubelet.conf | grep hostnameIf there's a mismatch, either:
- Change the hostname: sudo hostnamectl set-hostname <new-name>
- Update kubelet config to match the actual hostname
Ensure master and worker nodes have different hostnames.
From the node, test API server connectivity:
kubectl cluster-info # From control plane
kubectl get nodes # From control plane
kubectl --kubeconfig=/etc/kubernetes/kubelet.conf get nodes # Using kubelet credentialsTest port 6443:
netstat -tlnp | grep 6443 # On control plane
curl -k https://<control-plane-ip>:6443/api/v1 # From nodeIf API server isn't listening or responding, it must be fixed first.
Check Docker or other container runtime:
sudo systemctl status docker # For Docker
sudo systemctl status containerd # For containerd
sudo systemctl status crio # For CRI-OIf not running, start it:
sudo systemctl start dockerTest container runtime connectivity:
sudo docker version
sudo ctr version # For containerdKubelet may fail startup if pod network (CNI) is missing. Check from control plane:
kubectl get pods -n kube-system | grep -E "flannel|calico|weave" # Check for CNI podsIf missing, install the network plugin:
# Flannel example:
kubectl apply -f https://raw.githubusercontent.com/coreos/flannel/master/Documentation/kube-flannel.ymlThen restart kubelet on the node:
sudo systemctl restart kubeletVerify the node can reach the API server on port 6443:
sudo firewall-cmd --list-all # If using firewalld
sudo iptables -L -n # If using iptablesOpen port if needed:
sudo firewall-cmd --permanent --add-rich-rule='rule family="ipv4" destination address="<control-plane-ip>" port protocol="tcp" port="6443" accept'
sudo firewall-cmd --reloadFor testing, temporarily disable firewall:
sudo systemctl stop firewalldSome Kubernetes versions have compatibility issues with specific Docker versions. Check:
docker version
kubectl version # From control planeDocker 18.6-18.9 have known issues with Kubernetes. If incompatible, upgrade Docker:
sudo apt-get update
sudo apt-get install docker-ce=<version> # Specific versionThen restart kubelet.
When joining workers to a new cluster, control plane must be fully ready first:
# On control plane, wait for this to succeed:
kubectl get nodes
# Only then run on worker:
kubeadm join <control-plane-ip>:6443 --token <token> --discovery-token-ca-cert-hash sha256:<hash>Wait 2-3 minutes for control plane to stabilize before joining workers. Monitor kubelet logs during join:
sudo journalctl -u kubelet -fNode registration happens automatically—kubelet polls the API server with backoff until successful. If logs show repeated registration attempts failing, one of the underlying issues (API server down, hostname mismatch, firewall) must be fixed. For kubeadm-managed clusters, the kubeadm join command handles most configuration automatically. For self-managed clusters, ensure kubelet systemd unit or kubelet startup script properly configures API server address. On cloud providers (AWS, GCP, Azure), node registration typically succeeds automatically unless there are IAM or network issues at the cloud layer. WSL2-based Kubernetes may have temporary registration delays during VM startup—give it extra time. In air-gapped environments, ensure all required images and binaries are pre-downloaded before starting cluster initialization.
Failed to connect to server: connection refused (HTTP/2)
How to fix "HTTP/2 connection refused" error in Kubernetes
missing request for cpu in container
How to fix "missing request for cpu in container" in Kubernetes HPA
error: invalid configuration
How to fix "error: invalid configuration" in Kubernetes
etcdserver: cluster ID mismatch
How to fix "etcdserver: cluster ID mismatch" in Kubernetes
running with swap on is not supported
How to fix "running with swap on is not supported" in kubeadm