State migration fails when Terraform cannot automatically transfer state from an old backend to a new one. This typically occurs due to version mismatches, lock conflicts, permissions issues, or backend connectivity problems during terraform init -migrate-state.
When you change your Terraform backend configuration and run terraform init -migrate-state, Terraform attempts to copy your existing state from the old backend to the new one. If this operation fails, it indicates a problem transferring the state file. This is a critical issue because it blocks you from initializing your infrastructure on the new backend and can result in state becoming out of sync between the old and new locations.
Before attempting migration, ensure both backends are accessible. Test your target backend credentials:
# For S3 backend
aws s3 ls s3://your-bucket-name/ --region us-east-1
# For Azure blob storage
az storage blob list --container-name tfstate --account-name youraccount
# For Google Cloud Storage
gsutil ls gs://your-terraform-state-bucket/If any of these commands fail, fix the credential or connectivity issue before attempting migration again. Ensure your IAM user/service account has read, write, and delete permissions on the target backend.
State locks prevent concurrent access during migrations. Force unlock both backends if necessary:
# Check for locks
terraform state list
# Force unlock if a stale lock exists
terraform force-unlock <lock-id>You can also manually remove lock files if available (e.g., .tflock files in Azure Blob Storage). Only force unlock if you're certain no one else is currently modifying the infrastructure.
If migration continues to fail, switch to -reconfigure mode instead. This tells Terraform to reinitialize without copying state:
terraform init -reconfigureAfter this, you can manually migrate state using terraform state pull/push:
terraform state pull > /tmp/backup.tfstate
terraform state push /tmp/backup.tfstateThis two-step approach gives you more control and lets you verify each step. Always keep a backup like we did above.
If automatic migration fails, perform a manual state migration. First, initialize with the old backend configuration:
terraform init # Uses old backend config
terraform state pull > /tmp/terraform.tfstate.backupThen switch backend configuration and migrate manually:
terraform init -reconfigure # Switch to new backend
terraform state push /tmp/terraform.tfstate.backupVerify the state transferred correctly:
terraform state list
terraform state showKeep the backup file indefinitely as a recovery point.
Large state files can timeout during migration. Retry with increased wait time:
terraform init -migrate-state -lock-timeout=15mThis tells Terraform to wait up to 15 minutes for locks to be released. If the state file is very large (over 100MB), split it by migrating resources in batches:
terraform state rm 'module.large_module[0]'
terraform init -migrate-state
terraform state push previous-backup.tfstate
terraform import module.large_module[0].aws_instance.main i-12345After successful migration, verify the state wasn't corrupted:
terraform validate
terraform plan -out=tfplanReview the plan output to ensure it accurately reflects your infrastructure. If the plan shows unexpected resource replacements or deletions, the migration may have introduced inconsistencies. In that case:
terraform state show <resource>
terraform state list | wc -l # Count resources
terraform refreshIf problems persist, restore from the backup and contact HashiCorp support with the error logs.
State Lineage: Terraform tracks state lineage (version history) to prevent accidental overwrites. If migrating between drastically different backends, you may need to force override lineage checks by editing the backup tfstate JSON file's lineage field to match the target.
DynamoDB Lock Migration: If migrating from DynamoDB state locking to native S3 locking, run terraform init -reconfigure first to avoid migration conflicts.
Large State Files: State files over 100MB can experience timeout issues. Consider splitting your configuration into multiple modules or using separate workspaces to reduce state size.
Automation: In CI/CD, use -force-copy with -migrate-state to automate migration without prompts:
terraform init -migrate-state -force-copyAlways run this in a protected CI/CD pipeline with state backup enabled.
Cross-Cloud Migration: Migrating between AWS S3, Azure Blob, and GCS requires careful credential management. Use separate terraform configs with -backend-config flags to avoid mixing credentials.
Error: Error installing helm release: cannot re-use a name that is still in use
How to fix "release name in use" error in Terraform with Helm
Error: Error creating GKE Cluster: BadRequest
BadRequest error creating GKE cluster in Terraform
Error: External program failed to produce valid JSON
External program failed to produce valid JSON
Error: Unsupported argument in child module call
How to fix "Unsupported argument in child module call" in Terraform
Error: network is unreachable
How to fix "network is unreachable" in Terraform