How to fix failed detecting s3 prefix: could not list objects in Terraform

TerraformINTERMEDIATECRITICALUpdated Dec 30, 2025· 129 views

The "failed detecting s3 prefix" error occurs when Terraform Enterprise cannot list objects in an S3 bucket during state detection. This typically indicates missing IAM permissions, incorrect credentials, or network connectivity issues with your S3 backend.

Quick fix

Fixes failed detecting s3 prefix: could not list objects

# On the TFE instance
curl http://169.254.169.254/latest/meta-data/iam/info
  1. Verify AWS credentials are configured
  2. Ensure IAM permissions include s3:ListBucket
  3. Verify the S3 bucket exists and region is correct
  4. Check custom S3 endpoint configuration (if used)
terraform — error
# On the TFE instance
curl http://169.254.169.254/latest/meta-data/iam/info
failed detecting s3 prefix: could not list objects

What this error means

This error occurs when Terraform Enterprise tries to detect existing workspaces by listing objects in the configured S3 bucket. The "could not list objects" portion indicates that the S3 ListObjects (or ListObjectsV2) operation has failed. Terraform needs to list bucket contents to discover existing workspace state files. When this operation fails, it prevents Terraform Enterprise from initializing properly. The error can stem from authentication issues, permission problems, network failures, or S3 configuration issues.

Typical symptoms

  • Terraform Enterprise fails to start or initialize
  • Error appears during "terraform init" or workspace detection
  • Error mentions "operation error S3: ListObjectsV2"
  • State migration or backend initialization fails
  • Works in some environments but not others
  • Error began after upgrading Terraform Enterprise version
  • Custom S3 endpoints or VPC endpoints involved

Common causes

  1. IAM permissions missing s3:ListBucket action on the bucket
  2. AWS credentials not configured or expired
  3. S3 bucket policy restricting the principal
  4. Incorrect S3 bucket name or region mismatch
  5. Instance profile not attached to EC2/ECS task
  6. Custom S3 endpoint misconfigured or unreachable
  7. TLS/SSL certificate issues with S3 endpoint
  8. Terraform Enterprise version incompatible with credential method
  9. Network firewall or security group blocking S3 access

How to fix "failed detecting s3 prefix: could not list objects"

1Verify AWS credentials are configured

Check that Terraform Enterprise has valid AWS credentials. This can be provided via:

1. Instance profile (IAM role attached to the EC2/ECS instance)
2. Environment variables: AWS_ACCESS_KEY_ID and AWS_SECRET_ACCESS_KEY
3. ~/.aws/credentials file

To verify instance profile:

bash
# On the TFE instance
curl http://169.254.169.254/latest/meta-data/iam/info

If using environment variables or credentials file:

bash
aws sts get-caller-identity

This should return your AWS account ID and IAM principal.

2Ensure IAM permissions include s3:ListBucket

Terraform Enterprise needs the s3:ListBucket permission on the bucket itself (not just objects).

Required minimal IAM policy:

json
{
  "Version": "2012-10-17",
  "Statement": [
    {
      "Effect": "Allow",
      "Action": [
        "s3:ListBucket"
      ],
      "Resource": "arn:aws:s3:::your-state-bucket"
    },
    {
      "Effect": "Allow",
      "Action": [
        "s3:GetObject",
        "s3:PutObject",
        "s3:DeleteObject"
      ],
      "Resource": "arn:aws:s3:::your-state-bucket/*"
    }
  ]
}

If using a prefix restriction:

json
{
  "Effect": "Allow",
  "Action": ["s3:ListBucket"],
  "Resource": "arn:aws:s3:::your-state-bucket",
  "Condition": {
    "StringLike": {
      "s3:prefix": ["your-prefix/*"]
    }
  }
}
3Verify the S3 bucket exists and region is correct

Confirm the bucket name and region match your Terraform configuration:

bash
# List buckets
aws s3 ls

# Check bucket region
aws s3api get-bucket-location --bucket your-state-bucket

# Verify the bucket is accessible
aws s3 ls s3://your-state-bucket/

The bucket must exist in the region specified in your Terraform backend configuration.

4Check custom S3 endpoint configuration (if used)

If using a custom S3 endpoint (MinIO, LocalStack, or S3-compatible service):

bash
# Verify endpoint URL is correct and includes protocol
# Example: https://minio.example.com:9000 (not just minio.example.com)

# Test connectivity
curl -v https://your-s3-endpoint/

For Terraform Enterprise Replicated:

bash
# Check the s3_endpoint setting
replicatedctl app-config show s3_endpoint

# Should include protocol: https://...
# If missing protocol, update it
replicatedctl app-config set s3_endpoint --value https://your-endpoint:9000
5Verify TLS certificate validity (for custom endpoints)

For custom S3 endpoints, check certificate validity:

bash
# Check certificate expiration
openssl s_client -connect your-s3-endpoint:443 -servername your-s3-endpoint </dev/null 2>/dev/null | openssl x509 -noout -dates

# Verify certificate chain is complete
openssl s_client -connect your-s3-endpoint:443 -showcerts </dev/null 2>/dev/null

If using self-signed certificates, ensure TFE has them in its trusted CA store.

6Check for EC2 instance profile or assume role issues

If using IAM assume role:

bash
# Verify the instance profile exists
aws iam get-instance-profile --instance-profile-name tfe-instance-profile

# Verify the role has S3 permissions
aws iam get-role-policy --role-name tfe-role --policy-name s3-policy

# Check if role assumption is working
aws sts assume-role --role-arn arn:aws:iam::ACCOUNT:role/tfe-role --role-session-name test

For EKS/Kubernetes deployments, verify the ServiceAccount has the correct IAM role annotation (IRSA).

7Check Terraform Enterprise version compatibility

Newer Terraform Enterprise versions (1.6.0+) have different credential handling. If upgrading:

1. For prefix-scoped ListBucket permissions, update the IAM policy to allow ListBucket without prefix conditions
2. Use TFE_OBJECT_STORAGE_S3_USE_INSTANCE_PROFILE=true if using instance profiles
3. Verify the upgrade completed successfully and services restarted

Check your TFE version:

bash
# In Replicated TFE
replicatedctl app-config show release_sequence
8Enable debug logging to identify the exact error

Increase logging verbosity:

For Replicated TFE:

bash
# Check application logs
replicatedctl logs app | grep -i "list"

# Or view logs in real-time
tail -f /var/log/replicated/replicated.log | grep -i s3

For FDO (Flexible Deployment Options):

bash
# Check pod logs
kubectl logs -n tfe -l app=tfe --tail=100 | grep -i s3

Look for the specific S3 error (AccessDenied, NoSuchBucket, InvalidRegion, etc.) to pinpoint the issue.

Advanced notes

For Terraform Enterprise 202309-1 and later with Consolidated Services enabled, custom S3 endpoints must include the protocol prefix (https://). If upgrading from an older version, update endpoint URLs explicitly.

For EKS deployments (Terraform Enterprise FDO), ensure HttpPutResponseHopLimit is set to 2 in the launch template, especially on EKS 1.30+, to allow proper IMDS credential retrieval.

For on-premises S3-compatible solutions (MinIO, Wasabi), verify TLS certificates are in the system trust store and that the endpoint supports ListObjectsV2 API calls. Some older S3-compatible implementations may only support ListObjects (v1).

In air-gapped or private network deployments, ensure network routes exist to reach the S3 endpoint, and no firewall rules are blocking egress to the S3 service ports.

Frequently asked questions

What does "failed detecting s3 prefix: could not list objects" mean?
This error occurs when Terraform Enterprise tries to detect existing workspaces by listing objects in the configured S3 bucket. The "could not list objects" portion indicates that the S3 ListObjects (or ListObjectsV2) operation has failed. Terraform needs to list bucket contents to discover existing workspace state files.
How do I fix "failed detecting s3 prefix: could not list objects"?
The "failed detecting s3 prefix" error occurs when Terraform Enterprise cannot list objects in an S3 bucket during state detection. This typically indicates missing IAM permissions, incorrect credentials, or network connectivity issues with your S3 backend.
What causes "failed detecting s3 prefix: could not list objects"?
IAM permissions missing s3:ListBucket action on the bucket AWS credentials not configured or expired S3 bucket policy restricting the principal
Is the "failed detecting s3 prefix: could not list objects" error serious?
Yes. This is a critical-severity error that usually blocks work until it is fixed, but the steps above resolve it.

Official resources & further reading