How to fix "etcdserver: no leader"

1Immediately verify control plane node status

Check which control plane nodes are running:

bash

kubectl get nodes -l node-role.kubernetes.io/control-plane

For self-managed clusters, verify physically or via cloud provider:

bash

# AWS:
aws ec2 describe-instances --filter Name=tag:karpenter.sh/capacity-type,Values=master

# Azure:
az vm list --output table

Count healthy vs down. Need majority healthy for quorum.

2Verify etcd cluster membership

From a healthy node, check etcd status:

bash

ETCDCTL_API=3 etcdctl \
  --cacert=/etc/kubernetes/pki/etcd/ca.crt \
  --cert=/etc/kubernetes/pki/etcd/server.crt \
  --key=/etc/kubernetes/pki/etcd/server.key \
  --endpoints=https://127.0.0.1:2379 \
  member list

Note which members are reachable. For 3-member cluster, need 2+ healthy. For 5-member, need 3+.

3Bring down and restart unreachable nodes

If majority is unreachable, restart them:

bash

# For cloud VMs:
aws ec2 reboot-instances --instance-ids <instance-id>
az vm restart --ids <vm-id> --resource-group <group>

# Or for on-prem:
power cycle the server

Wait 2-3 minutes for nodes to rejoin cluster. Monitor:

bash

watch "ETCDCTL_API=3 etcdctl --cacert=/etc/kubernetes/pki/etcd/ca.crt --cert=/etc/kubernetes/pki/etcd/server.crt --key=/etc/kubernetes/pki/etcd/server.key --endpoints=https://127.0.0.1:2379 member list"

4Remove stale members from cluster

If nodes are permanently gone, remove them:

bash

ETCDCTL_API=3 etcdctl \
  --cacert=/etc/kubernetes/pki/etcd/ca.crt \
  --cert=/etc/kubernetes/pki/etcd/server.crt \
  --key=/etc/kubernetes/pki/etcd/server.key \
  --endpoints=https://127.0.0.1:2379 \
  member remove <member-id>

This reduces quorum requirement. Example: 3-member to 2-member cluster still needs 2 healthy (no loss in tolerance).

5Test connectivity between nodes

Network problems may be isolating members:

bash

# Test from each control node:
for node in <node1> <node2> <node3>; do
  echo "Testing $node:"
  ping -c 1 $node
  telnet $node 2379  # etcd client port
  telnet $node 2380  # etcd peer port
done

If telnet fails, firewall is blocking. Open ports:

bash

sudo iptables -A INPUT -p tcp --dport 2379 -j ACCEPT
sudo iptables -A INPUT -p tcp --dport 2380 -j ACCEPT

6Check and restart etcd on healthy nodes

Restart etcd to force rejoin cluster:

bash

sudo systemctl restart etcd
# or for containerized:
sudo docker restart <etcd-container>

Monitor recovery:

bash

watch "ETCDCTL_API=3 etcdctl --cacert=/etc/kubernetes/pki/etcd/ca.crt --cert=/etc/kubernetes/pki/etcd/server.crt --key=/etc/kubernetes/pki/etcd/server.key --endpoints=https://127.0.0.1:2379 endpoint health"

Wait 10-30 seconds for leader election.

7For kubeadm clusters, rebuild control plane

If majority of nodes are lost, rebuild:

bash

# On one healthy node:
kubeadm init phase certs all
kubeadm init phase kubeconfig all
kubeadm init phase control-plane all

# Then join other nodes:
kubeadm join <control-plane-endpoint> \
  --token <token> \
  --discovery-token-ca-cert-hash sha256:<hash> \
  --control-plane

8Verify etcd data integrity

After recovery, verify data wasn't corrupted:

bash

ETCDCTL_API=3 etcdctl \
  --cacert=/etc/kubernetes/pki/etcd/ca.crt \
  --cert=/etc/kubernetes/pki/etcd/server.crt \
  --key=/etc/kubernetes/pki/etcd/server.key \
  --endpoints=https://127.0.0.1:2379 \
  check perf

Also verify API server can communicate:

bash

kubectl get nodes
kubectl get pods --all-namespaces

9Review and prevent future quorum loss

Implement safeguards:

bash

# Monitoring (Prometheus + alerting):
alert if etcd_has_leader == 0
alert if etcd_members_unhealthy > 0

# Recommended cluster size: odd number
# 3-member: tolerate 1 failure
# 5-member: tolerate 2 failures
# Avoid even numbers

# Backup etcd regularly:
ETCDCTL_API=3 etcdctl \
  --cacert=/etc/kubernetes/pki/etcd/ca.crt \
  --cert=/etc/kubernetes/pki/etcd/server.crt \
  --key=/etc/kubernetes/pki/etcd/server.key \
  --endpoints=https://127.0.0.1:2379 \
  snapshot save backup.db

How to fix etcdserver: no leader in Kubernetes | DevErrors

How to fix etcdserver: no leader in Kubernetes

What this error means

Typical symptoms

Common causes

How to fix "etcdserver: no leader"

Advanced notes

Related errors

Official resources & further reading