Network packets cannot reach the destination host because network policies block traffic, IP forwarding is disabled on nodes, or pods are isolated by CNI configuration. Unlike connection refused, the packet never reaches the application. Fix by reviewing network policies, enabling IP forwarding, or configuring CNI correctly.
This error indicates the network cannot route packets to the destination. In Kubernetes, it commonly occurs when: network policies deny inter-pod traffic, nodes have IP forwarding disabled, pod CIDR ranges conflict, or the CNI plugin is misconfigured. The packets are dropped at the network layer before reaching the pod.
List all network policies:
kubectl get networkpolicies -A
kubectl describe networkpolicy <name> -n <namespace>Network policies default to DENY if namespace is in deny-all mode. To allow all traffic:
apiVersion: networking.k8s.io/v1
kind: NetworkPolicy
metadata:
name: allow-all
spec:
podSelector: {}
policyTypes:
- Ingress
- Egress
ingress:
- {}
egress:
- {}SSH to node and check IP forwarding:
sudo sysctl net.ipv4.ip_forward
# Should return: net.ipv4.ip_forward = 1If 0, enable it:
sudo sysctl -w net.ipv4.ip_forward=1
# Make persistent:
sudo echo "net.ipv4.ip_forward = 1" >> /etc/sysctl.conf
sudo sysctl -pVerify CNI pods are running on all nodes:
kubectl get daemonset -n kube-system
kubectl get pods -n kube-system -o wide | grep -E "(weave|flannel|calico|cilium)"Ensure DaemonSet has desired = current = ready count for all nodes. If not, check node taints and tolerations.
Check cluster CIDR and node pod CIDR allocations:
kubectl cluster-info dump | grep -i cidr
kubectl get nodes -o wide
# Check node allocatable IPs:
kubectl describe node <node-name> | grep -A2 "Allocated resources"Ensure pod CIDR doesn't overlap with node network or other services.
SSH into one node and test reachability:
ssh node1
ping <pod-ip-on-node2>
ping <service-ip>If pings fail, the network layer has issues. Check iptables and routes on nodes.
On cloud platforms (AWS, Azure, GCP), check security groups:
# AWS
aws ec2 describe-security-groups --query 'SecurityGroups[?GroupName=='default']'
# Azure
az network nsg list --query "[].{Name:name, ResourceGroup:resourceGroup}"Ensure ingress/egress allow pod and service CIDR ranges.
Capture packets on source and destination nodes:
sudo tcpdump -i any -nn src <pod-ip-1> or dst <pod-ip-1> -w capture.pcap
# Then analyze with Wireshark or tcpdump
sudo tcpdump -r capture.pcapLook for packets arriving at destination node interface.
This error is network-layer (layer 3), unlike "connection refused" which is application-layer. Cross-node communication is critical in Kubernetes; most CNI plugins (Flannel, Weave, Calico) handle routing automatically. In on-premises clusters, ensure all nodes are on same L2 network or routing is configured. For multus (multiple network interfaces), ensure routes are correct for secondary networks. Cilium offers advanced networking debugging: cilium connectivity test.
Failed to connect to server: connection refused (HTTP/2)
How to fix "HTTP/2 connection refused" error in Kubernetes
missing request for cpu in container
How to fix "missing request for cpu in container" in Kubernetes HPA
error: invalid configuration
How to fix "error: invalid configuration" in Kubernetes
etcdserver: cluster ID mismatch
How to fix "etcdserver: cluster ID mismatch" in Kubernetes
running with swap on is not supported
How to fix "running with swap on is not supported" in kubeadm