How to fix Descheduler error in Kubernetes
The Descheduler fails to evict Pods based on scheduling policies. This occurs when eviction rules conflict with Pod Disruption Budgets, node affinity requirements, or resource constraints. Resolve by reviewing descheduler strategies, PDB settings, and cluster topology.
Quick fix
Fixes Descheduler error
kubectl get pod -n kube-system -l app=descheduler
kubectl logs -n kube-system -l app=descheduler --tail=100- Check descheduler Pod status
- Review descheduler configuration
- Check Pod Disruption Budgets
- Review Pod constraints
kubectl get pod -n kube-system -l app=deschedulerkubectl logs -n kube-system -l app=descheduler --tail=100Descheduler error
What this error means
The Descheduler is a Kubernetes component that identifies and evicts Pods from nodes based on policies to balance the cluster. When it encounters errors, it may fail to evict underutilized Pods, leading to suboptimal resource distribution. Errors can stem from permission issues, conflicting constraints, or misconfigured policies.
Typical symptoms
- Descheduler Pod logs show eviction failure errors
- Pods remain on overutilized nodes despite descheduler running
- Descheduler Pod restarts repeatedly
- Metrics show no evictions happening when expected
- Events show "FailedEviction" with policy violations
- Descheduler creates eviction requests but kubelet rejects them
- Application Pods are not migrating to balance load
Common causes
- Pod has PodDisruptionBudget (PDB) that prevents eviction
- Pod has local storage (emptyDir) that cannot be migrated
- Pod is owned by DaemonSet or StaticPod that cannot be evicted
- Node affinity or topology spread constraints conflict with rebalancing
- Descheduler RBAC permissions are insufficient
- Descheduler policies are too conservative or misconfigured
- Cluster topology changed, making current placement impossible
How to fix "Descheduler error"
Verify the descheduler is running and check its logs:
kubectl get pod -n kube-system -l app=descheduler
kubectl logs -n kube-system -l app=descheduler --tail=100Check the descheduler policy and strategies:
kubectl get configmap -n kube-system descheduler-policy -o yaml
kubectl get deschedulerpolicy -o yamlEnsure policies are not too restrictive or conflicting.
List PDBs that may prevent eviction:
kubectl get pdb -A
kubectl describe pdb <name> -n <namespace>Verify minAvailable/maxUnavailable settings allow some disruption.
Check target Pods for constraints preventing eviction:
kubectl get pod <pod-name> -o yaml | grep -E "affinity|topologySpreadConstraints|nodeSelector"Loosen constraints if they conflict with rebalancing goals.
Check if descheduler has eviction permissions:
kubectl get clusterrole descheduler -o yaml
kubectl get clusterrolebinding descheduler -o yamlEnsure the role includes "evict" verb on Pods.
Assess current cluster balance:
kubectl top nodes
kubectl top pods -A
kubectl describe nodes | grep -A5 "Allocated resources"Update descheduler ConfigMap or DeschedulerPolicy:
apiVersion: descheduler.x-k8s.io/v1alpha1
kind: DeschedulerPolicy
strategies:
- name: "RemovePodsViolatingNodeAffinity"
params:
nodeAffinityType:
- requiredDuringSchedulingIgnoredDuringExecution
- name: "RemovePodsViolatingInterPodAntiAffinity"Re-deploy descheduler after changes.
Watch descheduler metrics to verify fixes:
kubectl logs -n kube-system -l app=descheduler -f
grep -i "evict" /var/log/pods/kube-system_descheduler-*/descheduler.logVerify evictions complete successfully.
Advanced notes
Descheduler works with the kube-scheduler, not against it. If you see persistent eviction failures, check both components. The eviction process respects graceful termination periods (terminationGracePeriodSeconds), so very large values may cause timeouts. In production clusters, use multiple descheduler replicas with leader election to avoid conflicts. Monitor descheduler CPU/memory usage as it scans all Pods and Nodes during each run.
Frequently asked questions
- What does "Descheduler error" mean?
- The Descheduler is a Kubernetes component that identifies and evicts Pods from nodes based on policies to balance the cluster. When it encounters errors, it may fail to evict underutilized Pods, leading to suboptimal resource distribution. Errors can stem from permission issues, conflicting constraints, or misconfigured policies.
- How do I fix "Descheduler error"?
- The Descheduler fails to evict Pods based on scheduling policies. This occurs when eviction rules conflict with Pod Disruption Budgets, node affinity requirements, or resource constraints. Resolve by reviewing descheduler strategies, PDB settings, and cluster topology.
- What causes "Descheduler error"?
- Pod has PodDisruptionBudget (PDB) that prevents eviction Pod has local storage (emptyDir) that cannot be migrated Pod is owned by DaemonSet or StaticPod that cannot be evicted
- Is the "Descheduler error" error serious?
- It is a medium-severity error โ more of a blocker than a risk. The steps above resolve it.
Official resources & further reading
- ๐Kubernetes Official Documentation
- ๐Kubernetes: Descheduler(Official Docs)
- ๐GitHub: Descheduler Project(GitHub)
- ๐Kubernetes: Pod Disruption Budgets(Official Docs)
- ๐ฌStackOverflow: Descheduler questions(Stack Overflow)
- ๐