How to fix connect: connection timed out in Kubernetes

KubernetesINTERMEDIATEHIGHUpdated Dec 5, 2025· 300 views

This error occurs when network connectivity fails between pods, services, or external endpoints. Fix it by verifying IP forwarding settings, checking DNS resolution, reviewing network policies, and ensuring CNI plugins are functioning correctly.

Quick fix

Fixes connect: connection timed out

# SSH to affected node and check
sysctl net.ipv4.ip_forward
# Output: 0 means disabled (problematic)
  1. Verify and enable IP forwarding on nodes
  2. Enable bridge netfilter settings
  3. Debug DNS resolution from a pod
  4. Verify CoreDNS status and scale if needed
kubernetes — error
# SSH to affected node and check
sysctl net.ipv4.ip_forward
# Output: 0 means disabled (problematic)
connect: connection timed out

What this error means

The "connect: connection timed out" error indicates that a network connection attempt failed to establish within the expected time limit. In Kubernetes, this typically means a pod cannot reach another pod, service, or external endpoint due to networking issues. This error can occur at multiple layers: DNS resolution failures prevent hostname lookup, IP forwarding disabled on nodes blocks inter-pod traffic, network policies restrict communication, or CNI plugin misconfiguration prevents pod networking entirely. The timeout behavior distinguishes this from "connection refused" (target reachable but not listening) — with timeouts, packets never receive a response, suggesting routing, firewall, or fundamental connectivity issues.

Typical symptoms

  • curl requests to services timeout after 30-60 seconds
  • nslookup returns "connection timed out; no servers could be reached"
  • Pods cannot resolve Kubernetes DNS names (service.namespace.svc.cluster.local)
  • Direct pod IP works but service IP times out
  • Intermittent timeouts on certain nodes only
  • Readiness/liveness probes fail with timeout errors

Common causes

  1. IP forwarding disabled on nodes (net.ipv4.ip_forward=0)
  2. Bridge netfilter not enabled (net.bridge.bridge-nf-call-iptables=0)
  3. CoreDNS pods unreachable or unavailable
  4. Network policies blocking egress or ingress traffic
  5. CNI plugin not initialized or misconfigured

How to fix "connect: connection timed out"

1Verify and enable IP forwarding on nodes

Check if IP forwarding is disabled (common cause):

bash
# SSH to affected node and check
sysctl net.ipv4.ip_forward
# Output: 0 means disabled (problematic)

Enable IP forwarding:

bash
# Enable immediately
sudo sysctl -w net.ipv4.ip_forward=1

# Persist across reboots
echo "net.ipv4.ip_forward=1" | sudo tee /etc/sysctl.d/99-ipforward.conf
sudo sysctl -p /etc/sysctl.d/99-ipforward.conf
2Enable bridge netfilter settings

Bridge netfilter is required for iptables rules on Linux bridges:

bash
# Enable on each node
sudo sysctl -w net.bridge.bridge-nf-call-iptables=1
sudo sysctl -w net.bridge.bridge-nf-call-ip6tables=1

# Persist across reboots
cat <<EOF | sudo tee /etc/sysctl.d/99-bridge-nf.conf
net.bridge.bridge-nf-call-iptables=1
net.bridge.bridge-nf-call-ip6tables=1
EOF
sudo sysctl -p /etc/sysctl.d/99-bridge-nf.conf
3Debug DNS resolution from a pod

Create a debug pod with diagnostic tools:

bash
kubectl run -it --rm debug --image=nicolaka/netshoot --restart=Never -- bash

# Inside pod, test DNS resolution
nslookup kubernetes.default
# Should show IP, not "connection timed out"

# Check resolv.conf
cat /etc/resolv.conf
# Should include nameserver pointing to CoreDNS

If DNS fails, check CoreDNS directly:

bash
# Get CoreDNS pod IP
kubectl get pods -n kube-system -l k8s-app=kube-dns -o wide

# Query CoreDNS directly from debug pod
nslookup kubernetes.default <COREDNS_POD_IP>
4Verify CoreDNS status and scale if needed

Check CoreDNS pod status:

bash
kubectl get pods -n kube-system -l k8s-app=kube-dns
kubectl logs -n kube-system -l k8s-app=kube-dns --tail=50

If CoreDNS is overwhelmed, scale it up:

bash
kubectl scale --replicas=3 -n kube-system deployment/coredns

Verify CoreDNS is receiving queries:

bash
kubectl logs -n kube-system -l k8s-app=kube-dns -f
5Test service connectivity directly

From a debug pod, test both DNS and direct IP:

bash
# Test by service name (uses DNS)
curl -v --connect-timeout 5 http://my-service:8080/health

# Get service cluster IP
kubectl get service my-service

# Test by cluster IP directly (bypasses DNS)
curl -v --connect-timeout 5 http://10.100.225.223:8080/health

Interpretation:
- Direct IP works but service name fails → DNS issue
- Both fail → network policy or service not running

6Check network policies blocking traffic

List network policies affecting traffic:

bash
kubectl get networkpolicies --all-namespaces
kubectl describe networkpolicy <policy-name> -n <namespace>

If policies exist, ensure DNS egress is allowed:

yaml
apiVersion: networking.k8s.io/v1
kind: NetworkPolicy
metadata:
  name: allow-dns
spec:
  podSelector: {}
  policyTypes:
  - Egress
  egress:
  - to:
    - namespaceSelector:
        matchLabels:
          kubernetes.io/metadata.name: kube-system
    ports:
    - protocol: UDP
      port: 53

Use tcpdump on worker node to verify traffic:

bash
# On node, capture traffic to pod IP
sudo tcpdump -i any -n host 10.244.1.25
# SYN without ACK = traffic blocked or port not listening

Advanced notes

CNI Plugin Troubleshooting:
- Flannel: Check routes and IP forwarding; issues often stem from misconfigured routes
- Calico: Verify calico-node pods are running; check calicoctl node status
- AWS VPC CNI: Requires IAM permissions and available subnet IPs

Pod-to-Pod vs Pod-to-Service:
- Pod-to-pod uses CNI plugin directly (overlay network)
- Pod-to-service goes through kube-proxy + iptables + CoreDNS
- If pod-to-pod works but pod-to-service times out → DNS or kube-proxy issue
- If both fail → CNI plugin or IP forwarding issue

Performance Tuning:
- Install Node Local DNS Cache to reduce latency
- Set appropriate ndots value in pod resolv.conf (default 5 causes extra DNS queries)
- Consider scaling CoreDNS based on cluster size

Frequently asked questions

What does "connect: connection timed out" mean?
The "connect: connection timed out" error indicates that a network connection attempt failed to establish within the expected time limit. In Kubernetes, this typically means a pod cannot reach another pod, service, or external endpoint due to networking issues. This error can occur at multiple layers: DNS resolution failures prevent hostname lookup, IP forwarding disabled on nodes blocks inter-pod traffic, network policies restrict communication, or CNI plugin misconfiguration prevents pod networking entirely.
How do I fix "connect: connection timed out"?
This error occurs when network connectivity fails between pods, services, or external endpoints. Fix it by verifying IP forwarding settings, checking DNS resolution, reviewing network policies, and ensuring CNI plugins are functioning correctly.
What causes "connect: connection timed out"?
IP forwarding disabled on nodes (net.ipv4.ip_forward=0) Bridge netfilter not enabled (net.bridge.bridge-nf-call-iptables=0) CoreDNS pods unreachable or unavailable
Is the "connect: connection timed out" error serious?
Yes. This is a high-severity error that usually blocks work until it is fixed, but the steps above resolve it.

Official resources & further reading