When Terraform Cloud or Enterprise fails to queue a run with "Failed to request run", the issue typically stems from API connectivity problems, network timeouts, or authentication failures. Troubleshooting requires verifying API access and identifying the underlying cause.
The "Failed to request run" error occurs when Terraform Cloud (TFC) or Terraform Enterprise (TFE) cannot successfully create or queue a new run. This usually happens during the API call that initiates a plan or apply operation. The error indicates a communication breakdown between your Terraform client and the TFC/TFE API server, which could be caused by network issues, authentication problems, server saturation, or API timeouts. When this error occurs, the run is never created in the system, so no logs or execution state are available on the server side.
Ensure you have a valid API token configured correctly.
For Terraform CLI, verify your credentials in ~/.terraform.d/credentials.tfrc.json:
{
"credentials": {
"app.terraform.io": {
"token": "your-api-token-here"
}
}
}For API-driven workflows, verify the Authorization header includes a valid bearer token:
curl -H "Authorization: Bearer YOUR_API_TOKEN" \
https://app.terraform.io/api/v2/account/detailsIf the curl command returns a 401 error, your token is invalid or expired. Generate a new API token from your TFC/TFE account settings.
Verify that your network can reach the TFC/TFE API endpoint without timeouts or blockages.
Test basic connectivity:
# For Terraform Cloud
ping app.terraform.io
curl -I https://app.terraform.io/api/v2/ping
# For Terraform Enterprise (replace with your domain)
ping terraform.example.com
curl -I https://terraform.example.com/api/v2/pingIf you're behind a proxy or firewall, ensure:
- Port 443 (HTTPS) is open to the TFC/TFE domain
- Proxy authentication is configured if required
- No man-in-the-middle proxies are intercepting the connection
Check firewall and proxy settings:
# Test DNS resolution
nslookup app.terraform.io
getent hosts app.terraform.io
# Test with explicit proxy if needed
export HTTP_PROXY=http://proxy.example.com:8080
export HTTPS_PROXY=http://proxy.example.com:8080
terraform planEnable verbose logging to see what's happening during the API request.
Set Terraform debug logging:
export TF_LOG=DEBUG
export TF_LOG_PATH=/tmp/terraform-debug.log
terraform planCheck the logs for detailed information about the failed request:
cat /tmp/terraform-debug.log | grep -i "request\|timeout\|connection\|error"Look for specific error patterns:
- "connection refused" - API endpoint unreachable
- "context deadline exceeded" - Request timeout, increase timeout setting
- "no such host" - DNS resolution failure
- "certificate verify failed" - TLS/SSL certificate issue
- "401 Unauthorized" - Authentication token problem
If you're experiencing timeouts on slow networks, increase the request timeout.
In your Terraform configuration, add a timeout to the remote backend block:
terraform {
cloud {
organization = "my-org"
workspaces {
name = "my-workspace"
}
# Note: The Terraform CLI uses a default timeout of 30 seconds
# This cannot be directly overridden in the configuration
}
}For API-driven workflows, implement client-side timeout handling:
# Example with curl - increase timeout to 120 seconds
curl --connect-timeout 30 --max-time 120 \
-H "Authorization: Bearer YOUR_TOKEN" \
-X POST https://app.terraform.io/api/v2/runs \
-H "Content-Type: application/vnd.api+json" \
-d @run-payload.jsonIf using Terraform CLI with persistent timeouts, use the TF_LOG_CORE environment variable to identify bottlenecks:
export TF_LOG_CORE=TRACE
terraform planEnsure the TFC/TFE service is healthy and your workspace is properly configured.
Verify server status for Terraform Cloud:
- Visit https://status.hashicorp.com/ to check for ongoing incidents
- Check the TFC web UI to confirm the workspace exists and is accessible
For Terraform Enterprise, contact your admin to check:
- Service health and logs
- Puma worker pool saturation (symptoms: slow requests, high latency)
- Database connectivity and performance
Verify your workspace backend configuration:
terraform init # Reinitialize to validate configuration
terraform workspace list # Confirm workspace exists
terraform validate # Check configuration syntaxIf using API-driven workflows, verify the workspace and organization exist:
# Get workspace ID
curl -H "Authorization: Bearer YOUR_TOKEN" \
https://app.terraform.io/api/v2/organizations/my-org/workspaces/my-workspaceIf the error is intermittent, retry the operation with delays to handle transient failures.
For manual testing:
# Simple retry with backoff
for attempt in 1 2 3; do
echo "Attempt $attempt..."
terraform plan && break
sleep $((2 ** attempt)) # Exponential backoff: 2, 4, 8 seconds
doneFor scripted/CI workflows, use a retry mechanism:
#!/bin/bash
max_attempts=5
attempt=1
while [ $attempt -le $max_attempts ]; do
terraform plan && exit 0
if [ $attempt -lt $max_attempts ]; then
sleep_time=$((2 ** attempt))
echo "Plan failed. Retrying in ${sleep_time}s..."
sleep $sleep_time
fi
((attempt++))
done
echo "All retry attempts exhausted."
exit 1Note: Transient failures are often caused by temporary API saturation and typically resolve on retry.
Network Architecture Considerations:
- If using Terraform Enterprise behind a corporate firewall, ensure the firewall whitelist includes the TFE domain and all required ports.
- For Terraform Cloud, requests must reach app.terraform.io on port 443 (HTTPS).
- If using a corporate proxy, Terraform CLI does not automatically read proxy settings from environment. You may need to configure proxy settings at the OS level or use a .terraformrc file.
Rate Limiting:
- HCP Terraform enforces rate limits of 30 requests per second per API token. High-concurrency CI/CD pipelines may trigger rate limiting, causing failed requests. Implement backoff and retry logic in automated workflows.
TLS/Certificate Issues:
- If you see certificate verification errors, verify that your system's CA certificate bundle is up to date. On Linux, update your system packages. On macOS, consider using the system Python with Terraform. For Terraform Enterprise, ensure self-signed certificates are trusted by adding them to your system's trust store.
Terraform Enterprise Specific:
- For on-premises TFE installations, high request volume can saturate the Puma application server. Scale the Puma worker pool or load balance requests across multiple TFE instances if available.
- Check TFE logs at /var/log/terraform-enterprise/ (or your configured log directory) for detailed error information.
Error: Error installing helm release: cannot re-use a name that is still in use
How to fix "release name in use" error in Terraform with Helm
Error: Error creating GKE Cluster: BadRequest
BadRequest error creating GKE cluster in Terraform
Error: External program failed to produce valid JSON
External program failed to produce valid JSON
Error: Unsupported argument in child module call
How to fix "Unsupported argument in child module call" in Terraform
Error: network is unreachable
How to fix "network is unreachable" in Terraform