LoadBalancer services stuck in "pending" state cannot assign external IPs. This blocks external traffic and prevents service exposure. Usually caused by cloud provider delays, quota issues, or controller problems.
When you create a LoadBalancer service, Kubernetes: 1. Requests external IP from cloud provider 2. Cloud provider allocates IP and creates load balancer 3. Service status updates with EXTERNAL-IP If status stays "pending": - Cloud provider has not allocated IP - Load balancer creation is in progress (normal for 1-2 minutes) - Load balancer creation failed (will stay pending indefinitely) - Cloud provider controller is not running - Quota or IAM permissions are insufficient
Get service creation time:
kubectl get svc <service-name> -o wide
kubectl get svc <service-name> -o jsonpath='{.metadata.creationTimestamp}'Normal behavior:
- Cloud provider load balancer creation takes 1-5 minutes
- Small additional delay for health checks
If pending for > 10 minutes, something is wrong.
Check for recent events:
kubectl describe svc <service-name>Look at the Events section:
- Recent events show latest errors
- If no events, controller may not be running
- If events are old (> 5 minutes), controller stuck
Check if cloud provider's load balancer controller exists:
AWS:
kubectl get deployment -n kube-system aws-load-balancer-controller
kubectl get pods -n kube-system -l app.kubernetes.io/name=aws-load-balancer-controller
kubectl logs -n kube-system -l app.kubernetes.io/name=aws-load-balancer-controllerGCP/GKE:
kubectl get pods -n kube-system -l k8s-app=gke-metrics-agent
kubectl logs -n kube-system -l k8s-app=gke-metrics-agentAzure/AKS:
kubectl get pods -n kube-system -l app=cloud-node-manager
kubectl logs -n kube-system -l app=cloud-provider-azureIf controller is not running or crashing (CrashLoopBackOff):
kubectl describe pod <controller-pod> -n kube-system
kubectl logs <controller-pod> -n kube-system --previousReinstall if needed (varies by provider).
Get detailed error information:
AWS:
kubectl logs -n kube-system deployment/aws-load-balancer-controller | tail -100
kubectl logs -n kube-system deployment/aws-load-balancer-controller | grep -i <service-name>
kubectl logs -n kube-system deployment/aws-load-balancer-controller | grep -i errorGCP:
kubectl logs -n kube-system -l k8s-app=gke-metrics-agent | grep -i error
gcloud logging read "resource.type=k8s_cluster and jsonPayload.message=~"load.*balancer"" --limit 50Azure:
kubectl logs -n kube-system -l app=cloud-provider-azure | grep -i error
az aks get-credentials --resource-group <rg> --name <cluster>
kubectl logs -n kube-system -l component=cloud-node-manager | grep -i errorCommon errors:
- "quota exceeded"
- "unauthorized"
- "subnet exhausted"
- "failed to create"
- "rate limited"
Verify quotas are not exceeded:
AWS:
aws ec2 describe-account-attributes
aws elb describe-load-balancers | grep "LoadBalancerName" | wc -l
aws service-quotas get-service-quota \
--service-code elasticloadbalancing \
--quota-code L-E9E9831D # Load Balancers quotaGCP:
gcloud compute resource-settings list
gcloud compute forwarding-rules list
gcloud compute addresses listAzure:
az network lb list --output table
az vm list-usage --location <region>
az account get-access-tokenIf quota exceeded, request increase from cloud provider (AWS/GCP/Azure dashboards).
Ensure the cluster has permissions to create load balancers:
AWS IAM:
# Get node role
aws ec2 describe-instances --instance-ids <instance-id> \
--query "Instances[0].IamInstanceProfile.Arn"
# Check role policies
aws iam list-attached-role-policies --role-name <role>
# Check for required policy
aws iam get-role-policy --role-name <role> --policy-name <policy>Required permissions:
{
"Effect": "Allow",
"Action": [
"elasticloadbalancing:CreateLoadBalancer",
"elasticloadbalancing:DescribeLoadBalancers",
"ec2:AllocateAddress",
"ec2:DescribeSubnets"
],
"Resource": "*"
}GCP Service Account:
kubectl get sa -n kube-system
gcloud projects get-iam-policy <project>Azure Identity:
kubectl get pod -n kube-system -o jsonpath='{.items[*].spec.containers[*].env[?(@.name=="MSI_ID")].value}'
az role assignment list --assignee <principal-id>Verify subnets have available IPs:
AWS VPC Subnets:
aws ec2 describe-subnets --filters "Name=vpc-id,Values=<vpc-id>" \
--query "Subnets[*].[SubnetId,AvailableIpAddressCount,CidrBlock]"If AvailableIpAddressCount is low (< 10), expand CIDR or create new subnet.
GCP Subnets:
gcloud compute networks subnets list --network=<network> \
--format="table(name,ipCidrRange,privateIpAddress)"Azure Subnets:
az network vnet subnet show -g <rg> -n <subnet> --vnet-name <vnet>
az network vnet subnet list-available-ips -g <rg> -n <subnet> --vnet-name <vnet>If subnets exhausted, create new subnets with larger CIDR ranges.
Review service annotations:
kubectl get svc <service-name> -o yaml | grep -A20 "metadata:"Common annotations:
AWS:
metadata:
annotations:
service.beta.kubernetes.io/aws-load-balancer-type: "nlb" # Must be nlb or classic
service.beta.kubernetes.io/aws-load-balancer-subnets: "subnet-12345,subnet-67890"
spec:
type: LoadBalancerGCP:
metadata:
annotations:
cloud.google.com/load-balancer-type: "External" # Or "Internal"Azure:
metadata:
annotations:
service.beta.kubernetes.io/azure-load-balancer-internal: "false" # Public LBIncorrect or missing annotations may prevent load balancer creation. Verify against cloud provider documentation.
If controller is running and configuration is correct, sometimes it just needs time:
# Monitor status
kubectl get svc <service-name> -wIf it allocates IP within 5-10 minutes, normal behavior.
If still pending after 10 minutes, force a refresh:
Delete and recreate the service:
kubectl delete svc <service-name>
kubectl apply -f service.yaml # Wait a few minutesOr trigger controller reconciliation:
# Restart the load balancer controller
kubectl rollout restart deployment/aws-load-balancer-controller -n kube-system
# Annotate service to trigger update
kubectl annotate service <service-name> version="$(date +%s)" --overwriteMonitor controller logs during reconciliation:
kubectl logs -n kube-system deployment/aws-load-balancer-controller -fLoadBalancer IP allocation is eventually consistent; expecting immediate IP assignment (within 1 minute) is usually safe but not guaranteed. Cloud provider degradation (rare) can delay allocation by hours. For mission-critical services, implement readiness probes that check EXTERNAL-IP assignment before marking service ready. Network complexity (multiple VPCs, cross-region) may require manual configuration—consult cloud provider networking docs. Service mesh (Istio, Linkerd) can manage load balancing independently of cloud provider, useful for multi-cloud deployments. For performance-critical applications, use cloud provider's native load balancer directly instead of Kubernetes LoadBalancer abstraction. Monitoring external IP assignment with Prometheus helps catch quota exhaustion early. Cost of LoadBalancer services adds up—implement cost tracking and use Ingress where possible.
No subnets found for EKS cluster
How to fix "eks subnet not found" in Kubernetes
unable to compute replica count
How to fix "unable to compute replica count" in Kubernetes HPA
error: context not found
How to fix "error: context not found" in Kubernetes
default backend - 404
How to fix "default backend - 404" in Kubernetes Ingress
serviceaccount cannot list resource
How to fix "serviceaccount cannot list resource" in Kubernetes