How to fix Failed to read from server connection=&discovery.Typha in Kubernetes
Typha synchronization errors occur when calico-node cannot connect to Typha service. Common causes include service discovery failures, TLS certificate issues, and network connectivity problems.
Quick fix
Fixes Failed to read from server connection=&discovery.Typha
kubectl get deployment -n calico-system calico-typha
kubectl get endpoints -n calico-system calico-typha
kubectl describe svc calico-typha -n calico-system- Check Typha deployment status
- Check Typha pod logs for errors
- Verify network connectivity to Typha
- Check for network policies blocking Typha access
kubectl get deployment -n calico-system calico-typhakubectl get endpoints -n calico-system calico-typhakubectl describe svc calico-typha -n calico-systemFailed to read from server connection=&discovery.Typha
What this error means
Typha is Calico's intermediate data store that syncs API server state to all calico-node instances. When calico-node cannot connect to Typha, it cannot receive policy and network configuration updates, leaving the node unable to properly enforce network policies and route traffic.
Typical symptoms
- Error message "Failed to read from server connection=&discovery.Typha"
- Calico node pod shows "Kubernetes service missing IP or port"
- calico-node pods not ready or failing readiness probe
- Connection timeout errors to Typha on port 5473
- TLS certificate verification failures in logs
- All calico-node pods down when all Typha pods are down
- Network policies not updating on nodes
- Service discovery failures with timeout exceptions
Common causes
- Typha service not properly configured or unavailable
- Network connectivity blocked between nodes and Typha (firewall, NetworkPolicy)
- TLS certificate issues or certificate mismatch
- Kubernetes API server connectivity timeout
- Typha replicas scaled down or not running
- Incorrect Typha service endpoint configuration
- High latency on network causing timeouts
- DNS resolution failing for Typha service
How to fix "Failed to read from server connection=&discovery.Typha"
Verify Typha is running and has endpoints:
kubectl get deployment -n calico-system calico-typha
kubectl get endpoints -n calico-system calico-typha
kubectl describe svc calico-typha -n calico-systemReview Typha logs for configuration or startup issues:
kubectl logs -n calico-system -l app=calico-typha --tail=50Look for certificate errors, API server connectivity issues, or port conflicts.
From a calico-node pod, test connectivity to Typha:
kubectl exec -it -n calico-system ds/calico-node -- \
nc -zv calico-typha.calico-system.svc.cluster.local 5473If connection fails, check firewall and network policies.
Ensure network policies allow calico-node to reach Typha:
calicoctl get networkpolicy -o yaml | grep -A 10 "to:"Allow port 5473 from kube-system namespace to calico-system.
Check certificate-related secrets and ConfigMaps:
kubectl get secret -n calico-system | grep -i tls
kubectl get configmap -n calico-system calico-config -o yamlVerify certificate paths and CA bundles are correct.
Typha needs to query the API server. Verify connectivity:
kubectl exec -it -n calico-system deployment/calico-typha -- \
curl -k https://kubernetes.default.svc.cluster.local:443Ensure Typha has enough replicas for redundancy:
kubectl scale deployment -n calico-system calico-typha --replicas=3
kubectl rollout status deployment/calico-typha -n calico-systemForce full restart to reconnect:
kubectl rollout restart deployment/calico-typha -n calico-system
kubectl rollout restart daemonset/calico-node -n calico-system
kubectl rollout status daemonset/calico-node -n calico-systemAdvanced notes
For production, run Typha with multiple replicas (minimum 3) and ensure they are spread across different nodes. Monitor Typha pod status continuously. For large clusters (100+ nodes), Typha reduces load on the API server significantly. Ensure TLS certificates used by Typha are not in rotation workflows that could cause immediate disconnections. Set connection timeout thresholds appropriately for your network latency characteristics.
Frequently asked questions
- What does "Failed to read from server connection=&discovery.Typha" mean?
- Typha is Calico's intermediate data store that syncs API server state to all calico-node instances. When calico-node cannot connect to Typha, it cannot receive policy and network configuration updates, leaving the node unable to properly enforce network policies and route traffic.
- How do I fix "Failed to read from server connection=&discovery.Typha"?
- Typha synchronization errors occur when calico-node cannot connect to Typha service. Common causes include service discovery failures, TLS certificate issues, and network connectivity problems.
- What causes "Failed to read from server connection=&discovery.Typha"?
- Typha service not properly configured or unavailable Network connectivity blocked between nodes and Typha (firewall, NetworkPolicy) TLS certificate issues or certificate mismatch
- Is the "Failed to read from server connection=&discovery.Typha" error serious?
- Yes. This is a high-severity error that usually blocks work until it is fixed, but the steps above resolve it.
Official resources & further reading
- ๐Kubernetes Official Documentation
- ๐Typha Configuration(Official Docs)
- ๐Calico Typha Error on K8s 1.29(GitHub)
- ๐Failed to Read from Typha(GitHub)