How to fix CoreDNS loop detected in Kubernetes
CoreDNS detected a DNS query loop, usually caused by misconfigured forwarders or upstreams. This causes DNS to fail. Disable loop detection, fix forwarder configuration, or review Corefile settings.
Quick fix
Fixes LOOP_DETECTED
cat /etc/resolv.conf | head -5- Check if systemd-resolved is causing the issue
- Identify the real resolv.conf file
- Configure kubelet with the correct resolv.conf
- Restart kubelet to apply changes
cat /etc/resolv.conf | head -5CoreDNS loop detected
What this error means
DNS loops occur when CoreDNS forwards queries in a circle, eventually returning to itself. This typically happens when CoreDNS forwards to an upstream that sends queries back to CoreDNS, or when systemd-resolved or other resolvers create circular dependencies. The loop detection plugin drops these queries, breaking DNS resolution.
Typical symptoms
- CoreDNS logs show "loop detected" warnings
- DNS resolution fails intermittently or completely
- nslookup commands timeout or fail
- Pod networking broken due to DNS failures
- External domain lookups fail while internal work
- CoreDNS Pod restarts repeatedly
- Query latency increases due to dropped packets
Common causes
- CoreDNS forwards to upstream that is itself (127.0.0.1 or ::1)
- Systemd-resolved running on host, creating loop
- Stub domain configuration points back to CoreDNS
- Multiple DNS servers pointing to each other
- Docker host DNS misconfiguration
- Kubernetes Node localhost DNS misconfiguration
- Search domain recursion issue
How to fix "CoreDNS loop detected"
First, verify if systemd-resolved is the root cause:
cat /etc/resolv.conf | head -5Look for nameserver entries like 127.0.0.53. If present, systemd-resolved is involved.
Find the actual resolv.conf file with real upstream nameservers:
# On systemd-resolved systems, find the real config:
systemctl status systemd-resolved
ls -la /etc/resolv.conf
# Check if it's a symlink (likely on systemd-resolved systems):
cat /run/systemd/resolve/resolv.confThis file usually contains the actual nameservers (e.g., 8.8.8.8, 1.1.1.1).
Edit the kubelet configuration file (typically at /etc/kubernetes/kubelet/kubelet-config.yaml or /etc/sysconfig/kubelet):
resolvConf: /run/systemd/resolve/resolv.confOr if using kubeadm, you can use the command line flag:
--resolv-conf=/run/systemd/resolve/resolv.confUse the real resolv.conf path you found in step 2.
Restart the kubelet service to pick up the new configuration:
sudo systemctl restart kubelet
# Verify it's running:
sudo systemctl status kubeletWait 30-60 seconds for CoreDNS pods to restart.
Check CoreDNS pod status:
kubectl get pods -n kube-system -l k8s-app=kube-dns
# Or for newer versions:
kubectl get pods -n kube-system -l k8s-app=kube-dns -o wideAll CoreDNS pods should be in Running status.
Check CoreDNS logs for the loop error:
kubectl logs -n kube-system -l k8s-app=kube-dns | tail -20Verify DNS works by launching a test pod:
kubectl run -it --rm debug --image=nicolaka/netshoot --restart=Never -- sh
# Inside the pod:
nslookup kubernetes.default
nslookup google.comIf both resolve successfully, the fix is complete.
If you initialized the cluster with kubeadm and still have issues:
# Check kubeadm detected systemd-resolved:
grep resolv /etc/kubernetes/kubelet.conf
# Manually fix all nodes:
sudo kubeadm init phase etcd init
# Restart kubelets on all nodeskubeadm version 1.10+ automatically detects and configures systemd-resolved.
Test full DNS chain from multiple pods:
# Test internal DNS:
kubectl run test -it --rm --image=alpine -- nslookup kubernetes.default
# Test external DNS:
kubectl run test -it --rm --image=alpine -- nslookup google.com
# Check CoreDNS endpoints:
kubectl get endpoints -n kube-system kube-dnsAll services should resolve without errors.
Advanced notes
### systemd-resolved Details
systemd-resolved is Linux's modern DNS caching system. It puts its resolver at 127.0.0.53:53 and manages /etc/resolv.conf. This conflicts with Kubernetes because:
1. CoreDNS reads /etc/resolv.conf as upstream servers
2. If that file points to 127.0.0.53, CoreDNS becomes its own upstream
3. Every query bounces between CoreDNS and itself infinitely
### Alternative: Disable systemd-resolved
For testing only (not recommended in production):
sudo systemctl disable systemd-resolved
sudo systemctl stop systemd-resolvedThen restore /etc/resolv.conf with real nameservers.
### kubeadm Automatic Detection
kubeadm 1.10+ automatically:
- Detects systemd-resolved
- Sets --resolv-conf=/run/systemd/resolve/resolv.conf automatically
- Applies this to all nodes during initialization
### Containerd vs Docker
This issue affects any container runtime. Both containerd and Docker will inherit the kubelet's resolv.conf configuration.
### Multi-node Cluster Fixes
For control plane and worker nodes:
1. Update kubelet config on all nodes
2. Restart kubelet on all nodes
3. CoreDNS will auto-restart on updated nodes
4. Monitor CoreDNS pod restart timeline across nodes
Frequently asked questions
- What does "LOOP_DETECTED" mean?
- DNS loops occur when CoreDNS forwards queries in a circle, eventually returning to itself. This typically happens when CoreDNS forwards to an upstream that sends queries back to CoreDNS, or when systemd-resolved or other resolvers create circular dependencies. The loop detection plugin drops these queries, breaking DNS resolution.
- How do I fix "LOOP_DETECTED"?
- CoreDNS detected a DNS query loop, usually caused by misconfigured forwarders or upstreams. This causes DNS to fail. Disable loop detection, fix forwarder configuration, or review Corefile settings.
- What causes "LOOP_DETECTED"?
- CoreDNS forwards to upstream that is itself (127.0.0.1 or ::1) Systemd-resolved running on host, creating loop Stub domain configuration points back to CoreDNS
- Is the "LOOP_DETECTED" error serious?
- It is a medium-severity error โ more of a blocker than a risk. The steps above resolve it.
Official resources & further reading
- ๐Kubernetes Official Documentation
- ๐Debugging DNS Resolution - Kubernetes Docs(Official Docs)
- ๐Troubleshooting kubeadm - Kubernetes Docs(Official Docs)
- ๐
- ๐ฌCoreDNS CrashLoopBackOff in Kubernetes - Stack Overflow(Stack Overflow)