How to fix 504 Gateway Timeout in Kubernetes

KubernetesINTERMEDIATEHIGHUpdated Dec 6, 2025· 362 views

A 504 Gateway Timeout in Kubernetes Ingress occurs when the NGINX controller cannot receive a response from backend services before the timeout expires. This commonly happens with slow applications, database operations, or when timeout settings don't match your workload requirements.

Quick fix

Fixes 504 Gateway Timeout

proxy-read-timeout: "300"
proxy-connect-timeout: "10"
proxy-send-timeout: "300"
upstream-fail-timeout: "10s"
  1. Check your Ingress controller logs
  2. Verify backend pods are running and healthy
  3. Increase NGINX Ingress global timeout
  4. Set per-Ingress timeout annotations
kubernetes — error
proxy-read-timeout: "300"
proxy-connect-timeout: "10"
proxy-send-timeout: "300"
upstream-fail-timeout: "10s"
504 Gateway Timeout

What this error means

A 504 Gateway Timeout error indicates that your Kubernetes Ingress controller (typically NGINX) successfully contacted a backend pod, but the pod failed to respond within the configured timeout window (default 60 seconds). This is different from a 502 Bad Gateway, which means no backend pods are available at all. The error occurs at the load balancer layer when the upstream application takes too long to process a request. The Ingress controller gives up waiting and returns a 504 error to the client, even if the backend pod is still working on the request.

Typical symptoms

  • Requests succeed initially but fail with 504 after approximately 60 seconds
  • Long-running operations like database migrations, file uploads, or batch processing consistently timeout
  • Intermittent 504 errors when traffic spikes cause pod slowdowns
  • Specific application endpoints timeout while others work fine
  • 504 errors occur at predictable time intervals matching your load balancer idle timeout
  • Load balancer health checks succeed but API requests through Ingress fail

Common causes

  1. Default NGINX Ingress timeout of 60 seconds is too short for your application workload
  2. Backend pods responding slowly due to heavy processing, complex database queries, or external API calls
  3. Load balancer idle timeout (AWS ELB/ALB default is 60 seconds) drops connections before the backend responds
  4. Readiness probes failing, causing Kubernetes to remove pod IPs from the service endpoint list
  5. Service configuration errors with incorrect serviceName or servicePort in Ingress resource
  6. kube-proxy issues causing premature connection termination without waiting for full timeout

How to fix "504 Gateway Timeout"

1Check your Ingress controller logs

Run kubectl logs -n ingress-nginx <controller-pod-name> to see if timeouts are the issue. Look for "upstream timed out" messages. Replace <controller-pod-name> with the actual NGINX Ingress controller pod running in the ingress-nginx namespace.

2Verify backend pods are running and healthy

Run kubectl get pods -o wide to see all pods and their status. Check readiness probes: kubectl describe pod <pod-name>. Test direct pod connectivity with kubectl exec -it <pod-name> -- curl http://localhost:<port>/<path> to ensure the pod responds properly.

3Increase NGINX Ingress global timeout

Edit the Ingress NGINX ConfigMap: kubectl edit configmap nginx-configuration -n ingress-nginx. Add or update these values (in seconds, no "s" suffix):

bash
proxy-read-timeout: "300"
proxy-connect-timeout: "10"
proxy-send-timeout: "300"
upstream-fail-timeout: "10s"

Save and the controller will automatically reload. Test with kubectl rollout restart deployment -n ingress-nginx.

4Set per-Ingress timeout annotations

For specific Ingress resources, add annotations without touching the global config:

yaml
apiVersion: networking.k8s.io/v1
kind: Ingress
metadata:
  annotations:
    nginx.ingress.kubernetes.io/proxy-read-timeout: "300"
    nginx.ingress.kubernetes.io/proxy-connect-timeout: "10"
    nginx.ingress.kubernetes.io/proxy-send-timeout: "300"
spec:
  # ... rest of Ingress config

Apply with kubectl apply -f ingress.yaml.

5Increase AWS load balancer idle timeout (if using ELB/ALB)

For Classic Load Balancer, add to Service annotation:

yaml
metadata:
  annotations:
    service.beta.kubernetes.io/aws-load-balancer-connection-idle-timeout: "300"

For Application Load Balancer (ALB), add to Ingress:

yaml
metadata:
  annotations:
    alb.ingress.kubernetes.io/load-balancer-attributes: idle_timeout.timeout_seconds=300

Apply changes with kubectl apply -f and verify with AWS console.

6Adjust readiness/liveness probes to match application timing

If your application has slow startup, increase probe timeouts: kubectl edit deployment <deployment-name>. Update the probe configuration:

yaml
readinessProbe:
  httpGet:
    path: /health
    port: 8080
  initialDelaySeconds: 30
  timeoutSeconds: 5
  periodSeconds: 10
  failureThreshold: 3

Increase initialDelaySeconds and periodSeconds to allow proper pod startup detection.

7Monitor application response times

Use Prometheus metrics or application logs to identify slow endpoints. Run kubectl exec <pod> -- curl -w "@curl-format.txt" http://localhost:8080/endpoint to measure response times. Optimize your application code or database queries if endpoints consistently approach timeout limits.

Advanced notes

For long-running operations like file uploads or batch processing, consider implementing an asynchronous pattern with webhooks instead of expecting synchronous responses within strict timeout windows. On AWS EKS, the load balancer timeout is separate from Ingress timeout—both must be configured. For Azure AKS with Traefik, configure forwardAuth.responsHeaderTimeout and transport.respondingTimeouts. In CI/CD pipelines (GitHub Actions, GitLab Runner), check if orchestration platform timeouts conflict with Kubernetes timeouts. WSL2 environments require timeout settings to propagate from Linux layer. Consider using Kubernetes NetworkPolicy to reduce latency between Ingress controller and backend pods.

Frequently asked questions

What does "504 Gateway Timeout" mean?
A 504 Gateway Timeout error indicates that your Kubernetes Ingress controller (typically NGINX) successfully contacted a backend pod, but the pod failed to respond within the configured timeout window (default 60 seconds). This is different from a 502 Bad Gateway, which means no backend pods are available at all. The error occurs at the load balancer layer when the upstream application takes too long to process a request.
How do I fix "504 Gateway Timeout"?
A 504 Gateway Timeout in Kubernetes Ingress occurs when the NGINX controller cannot receive a response from backend services before the timeout expires. This commonly happens with slow applications, database operations, or when timeout settings don't match your workload requirements.
What causes "504 Gateway Timeout"?
Default NGINX Ingress timeout of 60 seconds is too short for your application workload Backend pods responding slowly due to heavy processing, complex database queries, or external API calls Load balancer idle timeout (AWS ELB/ALB default is 60 seconds) drops connections before the backend responds
Is the "504 Gateway Timeout" error serious?
Yes. This is a high-severity error that usually blocks work until it is fixed, but the steps above resolve it.

Official resources & further reading