How to fix failed detecting s3 prefix: could not list objects in Terraform

TerraformINTERMEDIATECRITICAL

The "failed detecting s3 prefix" error occurs when Terraform Enterprise cannot list objects in an S3 bucket during state detection. This typically indicates missing IAM permissions, incorrect credentials, or network connectivity issues with your S3 backend.

What this error means

This error occurs when Terraform Enterprise tries to detect existing workspaces by listing objects in the configured S3 bucket. The "could not list objects" portion indicates that the S3 ListObjects (or ListObjectsV2) operation has failed. Terraform needs to list bucket contents to discover existing workspace state files. When this operation fails, it prevents Terraform Enterprise from initializing properly. The error can stem from authentication issues, permission problems, network failures, or S3 configuration issues.

How to fix "failed detecting s3 prefix: could not list objects"

1Verify AWS credentials are configured

Check that Terraform Enterprise has valid AWS credentials. This can be provided via:

1. Instance profile (IAM role attached to the EC2/ECS instance)
2. Environment variables: AWS_ACCESS_KEY_ID and AWS_SECRET_ACCESS_KEY
3. ~/.aws/credentials file

To verify instance profile:

bash

# On the TFE instance
curl http://169.254.169.254/latest/meta-data/iam/info

If using environment variables or credentials file:

bash

aws sts get-caller-identity

This should return your AWS account ID and IAM principal.

2Ensure IAM permissions include s3:ListBucket

Terraform Enterprise needs the s3:ListBucket permission on the bucket itself (not just objects).

Required minimal IAM policy:

json

{
  "Version": "2012-10-17",
  "Statement": [
    {
      "Effect": "Allow",
      "Action": [
        "s3:ListBucket"
      ],
      "Resource": "arn:aws:s3:::your-state-bucket"
    },
    {
      "Effect": "Allow",
      "Action": [
        "s3:GetObject",
        "s3:PutObject",
        "s3:DeleteObject"
      ],
      "Resource": "arn:aws:s3:::your-state-bucket/*"
    }
  ]
}

If using a prefix restriction:

json

{
  "Effect": "Allow",
  "Action": ["s3:ListBucket"],
  "Resource": "arn:aws:s3:::your-state-bucket",
  "Condition": {
    "StringLike": {
      "s3:prefix": ["your-prefix/*"]
    }
  }
}

3Verify the S3 bucket exists and region is correct

Confirm the bucket name and region match your Terraform configuration:

bash

# List buckets
aws s3 ls

# Check bucket region
aws s3api get-bucket-location --bucket your-state-bucket

# Verify the bucket is accessible
aws s3 ls s3://your-state-bucket/

The bucket must exist in the region specified in your Terraform backend configuration.

4Check custom S3 endpoint configuration (if used)

If using a custom S3 endpoint (MinIO, LocalStack, or S3-compatible service):

bash

# Verify endpoint URL is correct and includes protocol
# Example: https://minio.example.com:9000 (not just minio.example.com)

# Test connectivity
curl -v https://your-s3-endpoint/

For Terraform Enterprise Replicated:

bash

# Check the s3_endpoint setting
replicatedctl app-config show s3_endpoint

# Should include protocol: https://...
# If missing protocol, update it
replicatedctl app-config set s3_endpoint --value https://your-endpoint:9000

5Verify TLS certificate validity (for custom endpoints)

For custom S3 endpoints, check certificate validity:

bash

# Check certificate expiration
openssl s_client -connect your-s3-endpoint:443 -servername your-s3-endpoint </dev/null 2>/dev/null | openssl x509 -noout -dates

# Verify certificate chain is complete
openssl s_client -connect your-s3-endpoint:443 -showcerts </dev/null 2>/dev/null

If using self-signed certificates, ensure TFE has them in its trusted CA store.

6Check for EC2 instance profile or assume role issues

If using IAM assume role:

bash

# Verify the instance profile exists
aws iam get-instance-profile --instance-profile-name tfe-instance-profile

# Verify the role has S3 permissions
aws iam get-role-policy --role-name tfe-role --policy-name s3-policy

# Check if role assumption is working
aws sts assume-role --role-arn arn:aws:iam::ACCOUNT:role/tfe-role --role-session-name test

For EKS/Kubernetes deployments, verify the ServiceAccount has the correct IAM role annotation (IRSA).

7Check Terraform Enterprise version compatibility

Newer Terraform Enterprise versions (1.6.0+) have different credential handling. If upgrading:

1. For prefix-scoped ListBucket permissions, update the IAM policy to allow ListBucket without prefix conditions
2. Use TFE_OBJECT_STORAGE_S3_USE_INSTANCE_PROFILE=true if using instance profiles
3. Verify the upgrade completed successfully and services restarted

Check your TFE version:

bash

# In Replicated TFE
replicatedctl app-config show release_sequence

8Enable debug logging to identify the exact error

Increase logging verbosity:

For Replicated TFE:

bash

# Check application logs
replicatedctl logs app | grep -i "list"

# Or view logs in real-time
tail -f /var/log/replicated/replicated.log | grep -i s3

For FDO (Flexible Deployment Options):

bash

# Check pod logs
kubectl logs -n tfe -l app=tfe --tail=100 | grep -i s3

Look for the specific S3 error (AccessDenied, NoSuchBucket, InvalidRegion, etc.) to pinpoint the issue.

How to fix failed detecting s3 prefix: could not list objects in Terraform

What this error means

Typical symptoms

Common causes

How to fix "failed detecting s3 prefix: could not list objects"

Advanced notes

Related errors

Official resources & further reading