Terraform state write failures occur when Terraform cannot persist state changes to your backend storage due to permission issues, backend misconfiguration, or serial number conflicts. This prevents infrastructure changes from being recorded and can leave your state in an inconsistent state.
After Terraform creates or modifies infrastructure, it must write the updated state file back to its configured backend (local filesystem, S3, Azure Storage, Terraform Cloud, etc.). A "Failed to write state" error means this write operation failed. Terraform will save the state to an "errored.tfstate" file for recovery, but resources remain created in your cloud provider while state is out of sync. This is dangerous because Terraform loses track of what infrastructure exists, potentially leading to orphaned resources or duplicate creation attempts.
When Terraform fails to write state, it saves the attempted state to errored.tfstate in your working directory:
ls -la errored.tfstate
cat errored.tfstate | jq . | head -50This file contains the infrastructure changes Terraform was trying to persist. Keep this file safe—it's your recovery tool. Do NOT delete it yet.
Re-run the failed terraform command with debug logging to see the exact backend error:
TF_LOG=DEBUG terraform apply -out=tfplan 2>&1 | tail -100Common errors to look for:
- "AccessDenied: Access Denied" → IAM permission issue
- "invalid character 'R' looking for beginning of value" → Backend returned invalid JSON
- "cannot serialize resource instance" → State corruption or provider issue
- "Error acquiring the state lock" → State is locked by another operation
Note the exact error message—it tells you whether the issue is permissions, connectivity, or state format.
S3 backend failures are usually permission-related. Check your AWS credentials:
aws sts get-caller-identityVerify the returned user/role has these permissions:
{
"Version": "2012-10-17",
"Statement": [
{
"Effect": "Allow",
"Action": [
"s3:GetObject",
"s3:PutObject",
"s3:ListBucket"
],
"Resource": [
"arn:aws:s3:::my-terraform-bucket",
"arn:aws:s3:::my-terraform-bucket/*"
]
},
{
"Effect": "Allow",
"Action": [
"dynamodb:GetItem",
"dynamodb:PutItem",
"dynamodb:DeleteItem",
"dynamodb:DescribeTable"
],
"Resource": "arn:aws:dynamodb:*:ACCOUNT_ID:table/terraform-state-lock"
}
]
}If permissions are missing, contact your AWS admin to update the IAM policy. Then retry:
terraform apply -out=tfplanAzure backend errors usually indicate authentication or access issues:
az account show
az storage account show --name mystorageaccountVerify your storage account key or SAS token is current:
az storage account keys list --name mystorageaccount --resource-group myresourcegroupIf keys are old, rotate them:
az storage account keys renew --name mystorageaccount --resource-group myresourcegroup --key primaryUpdate your .env file or terraform backend config with the new key, then retry:
terraform init
terraform apply -out=tfplanTerraform Cloud write failures happen when your user lacks workspace permissions:
terraform loginWhen prompted, generate a new API token at https://app.terraform.io/app/settings/tokens
Verify your user has proper permissions for the workspace:
- Go to your workspace in Terraform Cloud
- Click Settings > Team access
- Ensure your team has "Manage all workspaces" or "Write" permission
Also verify the workspace exists and you're targeting the right organization:
terraform workspace list
terraform workspace select productionOnce you've fixed the underlying issue (permissions, credentials, connectivity), push the errored state to the backend:
terraform state push errored.tfstateTerraform will verify the serial number matches and push the state. If you get a serial number conflict error:
jq .serial errored.tfstate
jq .serial terraform.tfstateIf errored.tfstate has a lower serial, manually increment it:
jq '.serial += 1' errored.tfstate > errored.tfstate.new && mv errored.tfstate.new errored.tfstate
terraform state push errored.tfstateAfter successful push, verify the state is now consistent:
terraform state list
terraform plan # Should show no changesState serial number conflicts occur when your local state is older than the remote state. This happens if someone else applied changes while you had state write failures.
Check the conflict:
terraform state pull > remote.tfstate
jq .serial remote.tfstate
jq .serial errored.tfstateIf errored.tfstate serial is lower, update it to be higher:
REMOTE_SERIAL=$(jq .serial remote.tfstate)
NEW_SERIAL=$((REMOTE_SERIAL + 1))
jq --arg serial "$NEW_SERIAL" '.serial = ($serial | tonumber)' errored.tfstate > errored.tfstate.new
mv errored.tfstate.new errored.tfstate
terraform state push errored.tfstateThis tells Terraform that your local state is the newer version, allowing the push to succeed.
After recovering state, verify your infrastructure matches what Terraform knows about:
terraform planIf plan shows:
- No changes → State is now consistent, you're done
- Changes needed → Terraform found drift between state and real infrastructure
- Resources to destroy → Possible zombie resources created during failed operation
For zombie resources (created but not in state), decide whether to:
Option A: Import them into state
terraform import aws_instance.web i-1234567890abcdef0Option B: Destroy them manually (in AWS/Azure/GCP console) and document the orphan
Option C: Use terraform apply to recreate/update resources matching the recovered state
Once plan shows no unexpected changes, apply the fix:
terraform applyState write failures are arguably the scariest Terraform errors because they leave infrastructure untracked. Prevention is critical:
Pre-flight permission checks: Before running apply, test backend connectivity:
terraform init -upgrade
terraform validate
terraform refresh # Syncs state with real infrastructureState snapshots: Regularly back up your state file (especially before major changes):
terraform state pull > state-backup-$(date +%Y%m%d-%H%M%S).jsonLock tables for S3: If using S3 backends without a DynamoDB lock table, add one:
terraform {
backend "s3" {
bucket = "my-terraform-state"
key = "prod/terraform.tfstate"
region = "us-east-1"
dynamodb_table = "terraform-state-lock"
}
}CI/CD safety: Never run terraform apply without protecting against concurrent operations. Use GitHub Actions concurrency, GitLab resource_groups, or Jenkins mutex locks.
Manual state edits (rare): If you must manually edit state files, always work on a copy and increment the serial number. Better yet, use terraform state commands instead of editing JSON directly.
Error: Error rendering template: template not found
How to fix "template not found" error in Terraform
Error: Error generating private key
How to fix 'Error generating private key' in Terraform
Error creating Kubernetes Service: field is immutable
How to fix "field is immutable" errors in Terraform
Error: Error creating local file: open: permission denied
How to fix "Error creating local file: permission denied" in Terraform
Error: line endings have changed from CRLF to LF
Line endings have changed from CRLF to LF in Terraform