How to fix Error: Failed to write state in Terraform

When Terraform fails to write state, it saves the attempted state to errored.tfstate in your working directory:

ls -la errored.tfstate
cat errored.tfstate | jq . | head -50

This file contains the infrastructure changes Terraform was trying to persist. Keep this file safe—it's your recovery tool. Do NOT delete it yet.

Re-run the failed terraform command with debug logging to see the exact backend error:

TF_LOG=DEBUG terraform apply -out=tfplan 2>&1 | tail -100

Common errors to look for:
- "AccessDenied: Access Denied" → IAM permission issue
- "invalid character 'R' looking for beginning of value" → Backend returned invalid JSON
- "cannot serialize resource instance" → State corruption or provider issue
- "Error acquiring the state lock" → State is locked by another operation

Note the exact error message—it tells you whether the issue is permissions, connectivity, or state format.

S3 backend failures are usually permission-related. Check your AWS credentials:

aws sts get-caller-identity

Verify the returned user/role has these permissions:

{
  "Version": "2012-10-17",
  "Statement": [
    {
      "Effect": "Allow",
      "Action": [
        "s3:GetObject",
        "s3:PutObject",
        "s3:ListBucket"
      ],
      "Resource": [
        "arn:aws:s3:::my-terraform-bucket",
        "arn:aws:s3:::my-terraform-bucket/*"
      ]
    },
    {
      "Effect": "Allow",
      "Action": [
        "dynamodb:GetItem",
        "dynamodb:PutItem",
        "dynamodb:DeleteItem",
        "dynamodb:DescribeTable"
      ],
      "Resource": "arn:aws:dynamodb:*:ACCOUNT_ID:table/terraform-state-lock"
    }
  ]
}

If permissions are missing, contact your AWS admin to update the IAM policy. Then retry:

terraform apply -out=tfplan

Azure backend errors usually indicate authentication or access issues:

az account show
az storage account show --name mystorageaccount

Verify your storage account key or SAS token is current:

az storage account keys list --name mystorageaccount --resource-group myresourcegroup

If keys are old, rotate them:

az storage account keys renew --name mystorageaccount --resource-group myresourcegroup --key primary

Update your .env file or terraform backend config with the new key, then retry:

terraform init
terraform apply -out=tfplan

Terraform Cloud write failures happen when your user lacks workspace permissions:

terraform login

When prompted, generate a new API token at https://app.terraform.io/app/settings/tokens

Verify your user has proper permissions for the workspace:
- Go to your workspace in Terraform Cloud
- Click Settings > Team access
- Ensure your team has "Manage all workspaces" or "Write" permission

Also verify the workspace exists and you're targeting the right organization:

terraform workspace list
terraform workspace select production

Once you've fixed the underlying issue (permissions, credentials, connectivity), push the errored state to the backend:

terraform state push errored.tfstate

Terraform will verify the serial number matches and push the state. If you get a serial number conflict error:

jq .serial errored.tfstate
jq .serial terraform.tfstate

If errored.tfstate has a lower serial, manually increment it:

jq '.serial += 1' errored.tfstate > errored.tfstate.new && mv errored.tfstate.new errored.tfstate
terraform state push errored.tfstate

After successful push, verify the state is now consistent:

terraform state list
terraform plan  # Should show no changes

State serial number conflicts occur when your local state is older than the remote state. This happens if someone else applied changes while you had state write failures.

Check the conflict:

terraform state pull > remote.tfstate
jq .serial remote.tfstate
jq .serial errored.tfstate

If errored.tfstate serial is lower, update it to be higher:

REMOTE_SERIAL=$(jq .serial remote.tfstate)
NEW_SERIAL=$((REMOTE_SERIAL + 1))
jq --arg serial "$NEW_SERIAL" '.serial = ($serial | tonumber)' errored.tfstate > errored.tfstate.new
mv errored.tfstate.new errored.tfstate
terraform state push errored.tfstate

This tells Terraform that your local state is the newer version, allowing the push to succeed.

After recovering state, verify your infrastructure matches what Terraform knows about:

terraform plan

If plan shows:
- No changes → State is now consistent, you're done
- Changes needed → Terraform found drift between state and real infrastructure
- Resources to destroy → Possible zombie resources created during failed operation

For zombie resources (created but not in state), decide whether to:

Option A: Import them into state

terraform import aws_instance.web i-1234567890abcdef0

Option B: Destroy them manually (in AWS/Azure/GCP console) and document the orphan

Option C: Use terraform apply to recreate/update resources matching the recovered state

Once plan shows no unexpected changes, apply the fix:

terraform apply