How to fix Error: Error creating Kinesis Stream: LimitExceededException in Terraform

First, verify how many streams are currently in the CREATING state using the AWS CLI:

aws kinesis list-streams --region us-east-1
aws kinesis describe-stream --stream-name your-stream-name --region us-east-1

Look for streams with status "CREATING". If more than 4 are in this state, wait for them to reach ACTIVE status before creating new streams.

Use the DescribeLimits API to check your current shard quota:

aws kinesis describe-limits --region us-east-1

This shows OpenShardCount (current shards) and ShardLimit (your quota). If you're at or near the limit, you need to request a quota increase.

If you've reached your shard limit, request an increase through AWS Service Quotas:

# Open Service Quotas console and navigate to Kinesis
# Search for "Kinesis Data Streams"
# Select "Shard quota"
# Click "Request quota increase"
# Enter your desired shard count
# Submit (usually approved within hours)

Alternatively, contact AWS Support for faster processing of quota increases.

Terraform by default parallelizes resource creation. Since Kinesis has a 5 calls/second limit for CreateStream, reduce parallelism:

terraform apply -parallelism=1

Or permanently set it in your Terraform code:

terraform {
  required_version = ">= 1.0"
}

Then apply with explicit parallelism or use the command flag.

Add retry configuration to your AWS Terraform provider to automatically retry rate-limited requests:

provider "aws" {
  region       = "us-east-1"
  retry_mode   = "standard"
  max_retries  = 10
}

The standard retry mode includes exponential backoff and will retry rate-limited requests automatically.

Instead of provisioned mode, use On-Demand for variable workloads:

resource "aws_kinesis_stream" "example" {
  name             = "my-stream"
  stream_mode_details {
    stream_mode = "ON_DEMAND"
  }
}

On-Demand mode automatically scales and reduces quota conflicts. It starts with 4 shards and scales automatically, supporting up to 2 GB/s write throughput.