The remote-exec provisioner failed to execute scripts on a remote resource. This typically occurs due to connectivity issues, authentication failures, or timeout problems when Terraform tries to run commands over SSH or WinRM.
Terraform's remote-exec provisioner is designed to run commands on a remote machine (usually after creation). When this error appears, it means Terraform couldn't successfully execute one or more commands on the target instance. This could be because the connection itself failed, authentication didn't work, the instance wasn't ready yet, or the command itself encountered an error. The provisioner logs may not always provide clear details about what went wrong, making troubleshooting challenging.
Ensure your security group or firewall allows inbound traffic on port 22 (SSH) or 5986 (WinRM):
resource "aws_security_group" "example" {
name = "allow_ssh"
ingress {
from_port = 22
to_port = 22
protocol = "tcp"
cidr_blocks = ["0.0.0.0/0"] # Restrict this to your IP in production
}
egress {
from_port = 0
to_port = 0
protocol = "-1"
cidr_blocks = ["0.0.0.0/0"]
}
}Test manually first:
# Test SSH connectivity
ssh -i /path/to/key.pem ec2-user@<instance-ip>
# Test WinRM connectivity (Windows)
winrm id -r:https://<instance-ip>:5986Add a dependency or delay to let the instance fully boot and SSH service start:
resource "aws_instance" "example" {
ami = "ami-0c55b159cbfafe1f0"
instance_type = "t2.micro"
}
# Add a small delay to allow SSH to be ready
resource "time_sleep" "wait_for_instance" {
depends_on = [aws_instance.example]
create_duration = "30s"
}
resource "null_resource" "remote_exec" {
depends_on = [time_sleep.wait_for_instance]
provisioner "remote-exec" {
inline = ["echo 'Instance ready'"]
connection {
type = "ssh"
user = "ec2-user"
private_key = file("~/.ssh/id_rsa")
host = aws_instance.example.public_ip
}
}
}Alternatively, add a user_data script to cloud-init and wait for it to complete.
Ensure your connection block has the correct type, user, authentication, and host:
resource "null_resource" "example" {
provisioner "remote-exec" {
inline = [
"sudo yum install -y httpd",
"sudo systemctl start httpd"
]
connection {
type = "ssh"
user = "ec2-user" # Varies by AMI: ubuntu, admin, ec2-user, root, etc.
private_key = file("~/.ssh/my-key.pem")
host = aws_instance.example.public_ip
timeout = "5m"
}
}
depends_on = [aws_instance.example, aws_security_group.example]
}Check your AMI documentation for the correct username:
- Amazon Linux: ec2-user
- Ubuntu: ubuntu
- Windows: Administrator (with WinRM)
- RHEL: ec2-user or root
If provisioner times out, increase the timeout and enable Terraform debug logging:
provisioner "remote-exec" {
connection {
type = "ssh"
host = aws_instance.example.public_ip
user = "ec2-user"
private_key = file("~/.ssh/id_rsa")
timeout = "10m" # Increased from default 5m
}
inline = ["echo 'Provisioning...'"]
}Run Terraform with debug output:
TF_LOG=DEBUG terraform apply 2>&1 | tee terraform_debug.logLook for connection timeout details in the logs to understand where it's failing.
Some hardened images block script execution from /tmp. Specify an alternative:
provisioner "remote-exec" {
connection {
type = "ssh"
host = aws_instance.example.public_ip
user = "ec2-user"
private_key = file("~/.ssh/id_rsa")
script_path = "/home/ec2-user/terraform-script.sh" # Use home directory instead
}
inline = ["echo 'Testing from home directory'"]
}The script_path parameter controls where Terraform copies remote-exec scripts on the remote machine.
SSH to the instance and test your commands manually to identify script errors:
ssh -i ~/.ssh/id_rsa ec2-user@<instance-ip>
# Once connected, run your commands
sudo yum install -y httpd
sudo systemctl start httpdIf manual execution works but provisioner fails, the issue is likely connectivity or permissions. If it fails manually, fix the commands and retry.
HashiCorp recommends using remote-exec as a last resort. Better alternatives include:
User Data Script (for simple bootstrap):
resource "aws_instance" "example" {
ami = "ami-0c55b159cbfafe1f0"
instance_type = "t2.micro"
user_data = base64encode(file("${path.module}/init-script.sh"))
}Cloud-Init (more robust):
user_data = base64gzip(templatefile("${path.module}/cloud-init.yaml", {
packages : ["httpd"]
}))Configuration Management (Ansible, Chef, Puppet):
More powerful for complex provisioning and easier to test outside Terraform.
These approaches are more reliable, idempotent, and avoid the complexity of maintaining SSH credentials in Terraform state.
Why remote-exec provisioners are problematic:
1. Non-idempotent: Running provisioners multiple times can cause unexpected behavior
2. Not tracked in state: Terraform cannot detect if provisioned changes drift
3. Hidden dependencies: Scripts have implicit dependencies on remote state that Terraform doesn't track
4. Fragile: Network failures, timing issues, and host-level problems are difficult to debug
5. Security risk: Requires storing credentials in Terraform state
SSH Troubleshooting checklist:
- Verify security group/firewall allows port 22 inbound
- Check that instance has public IP (or bastion access)
- Confirm private key permissions: chmod 600 ~/.ssh/id_rsa
- Verify correct username for the AMI/OS
- Test manually: ssh -i key.pem user@host should work
- Check instance system logs for SSH service startup errors
- For Ubuntu/Debian, ensure openssh-server package is installed
- On hardened images, /tmp execution may be blocked - use script_path parameter
WinRM (Windows) specific:
- Ensure Windows instance has WinRM enabled (check security group port 5986)
- Use https connection type, not ssh
- Password authentication is more common than key-based on Windows
- EC2Launch (Amazon's tool) handles WinRM setup automatically on Windows AMIs
Performance considerations:
- Each remote-exec provisioner creates a new connection - group commands where possible
- Large scripts should use file provisioner first, then remote-exec to run them
- Consider cloud-init user_data for better performance (runs at instance launch)
Error: Error installing helm release: cannot re-use a name that is still in use
How to fix "release name in use" error in Terraform with Helm
Error: Error creating GKE Cluster: BadRequest
BadRequest error creating GKE cluster in Terraform
Error: External program failed to produce valid JSON
External program failed to produce valid JSON
Error: Unsupported argument in child module call
How to fix "Unsupported argument in child module call" in Terraform
Error: network is unreachable
How to fix "network is unreachable" in Terraform