How to fix Error: External program failed to produce valid JSON in Terraform

TerraformINTERMEDIATEHIGHUpdated Dec 31, 2025· 109 views

This error occurs when a Terraform external data source invokes a program that doesn't return properly formatted JSON. The external program must output a flat JSON object with string keys and string values to stdout.

Quick fix

Fixes Error: External program failed to produce valid JSON

# Run your external program and pipe to a JSON validator
your_program | jq empty
# If jq fails, your JSON is invalid
  1. Verify JSON syntax with a validator
  2. Ensure your program outputs valid flat JSON
  3. Use jq for shell-based programs
  4. Remove debug output and ensure clean stdout
terraform — error
# Run your external program and pipe to a JSON validator
your_program | jq empty
# If jq fails, your JSON is invalid
Error: External program failed to produce valid JSON

What this error means

The external data source in Terraform runs an external program and expects it to return valid JSON on stdout. This error indicates the program's output doesn't conform to Terraform's strict JSON requirements: it must be a flat object (no nested objects or arrays), all values must be strings, and the syntax must be valid JSON with proper quoting.

Typical symptoms

  • Terraform apply or plan fails with 'External program failed to produce valid JSON' error
  • External data source references show invalid data or null values
  • Python scripts using single quotes instead of double quotes produce this error
  • Programs outputting nested JSON objects or arrays fail validation

Common causes

  1. External program uses single quotes instead of double quotes in JSON output (e.g., Python dict representation)
  2. Program returns nested JSON objects or arrays instead of a flat string-to-string map
  3. JSON syntax error such as trailing commas, missing quotes, or unclosed brackets
  4. Program outputs additional text to stdout besides the JSON (like debug prints or log messages)
  5. Program returns non-string values (numbers, booleans, objects) instead of stringified values

How to fix "Error: External program failed to produce valid JSON"

1Verify JSON syntax with a validator

Use an online JSON validator (like JSONLint) or a command-line tool to check your program's output:

bash
# Run your external program and pipe to a JSON validator
your_program | jq empty
# If jq fails, your JSON is invalid

This will immediately show you syntax errors like trailing commas, missing quotes, or unclosed brackets.

2Ensure your program outputs valid flat JSON

Modify your external program to output a flat JSON object with only string values:

python
# WRONG - returns Python dict with single quotes
result = {"ip": "10.190.170.37", "network": "10.190.170.0"}
print(result)

# CORRECT - proper JSON with double quotes
import json
result = {"ip": "10.190.170.37", "network": "10.190.170.0"}
print(json.dumps(result))

Ensure:
- All keys and string values use double quotes
- No nested objects or arrays
- All values are strings (convert numbers/booleans to strings if needed)

3Use jq for shell-based programs

If writing shell scripts, use jq to safely construct JSON:

bash
#!/bin/bash
ip="10.190.170.37"
network="10.190.170.0"

# Safe JSON construction with jq
jq -n --arg ip "$ip" --arg network "$network" \
  '{"ip": $ip, "network": $network}'

This prevents accidental JSON syntax errors from string interpolation.

4Remove debug output and ensure clean stdout

Check that your program only outputs JSON to stdout:

python
import json
import sys

# DON'T DO THIS - debug output breaks JSON
print("Starting script...") # This breaks the JSON!
result = {"key": "value"}
print(json.dumps(result))

# DO THIS - only JSON to stdout
result = {"key": "value"}
print(json.dumps(result))
# Use stderr for debug output instead
print("Debug message", file=sys.stderr)

All debug prints must go to stderr, not stdout.

5Verify exit code is zero

Ensure your program exits with status code 0 on success:

bash
# In bash
exit 0  # Success

# In Python
import sys
sys.exit(0)  # Success
sys.exit(1)  # Failure

Non-zero exit codes cause Terraform to ignore the output and treat it as a program error.

6Test your external program independently

Run your program outside of Terraform to verify it works:

bash
# Test with the same inputs Terraform would use
echo '{"arg1": "value1"}' | your_program | jq .

# Check exit code
echo $?

This isolates whether the problem is in your program or in how Terraform invokes it.

Advanced notes

The external data source protocol has strict limitations by design: Terraform requires all values to be strings to maintain consistency with variable types. If you need to return complex data structures, serialize them as JSON strings within the flat object (e.g., {"data": "{\"nested\": \"value\"}"}), then parse them in your Terraform configuration with jsondecode(). For debugging, enable Terraform logging with TF_LOG=TRACE to see the exact program output and error details.

Frequently asked questions

What does "Error: External program failed to produce valid JSON" mean?
The external data source in Terraform runs an external program and expects it to return valid JSON on stdout. This error indicates the program's output doesn't conform to Terraform's strict JSON requirements: it must be a flat object (no nested objects or arrays), all values must be strings, and the syntax must be valid JSON with proper quoting.
How do I fix "Error: External program failed to produce valid JSON"?
This error occurs when a Terraform external data source invokes a program that doesn't return properly formatted JSON. The external program must output a flat JSON object with string keys and string values to stdout.
What causes "Error: External program failed to produce valid JSON"?
External program uses single quotes instead of double quotes in JSON output (e.g., Python dict representation) Program returns nested JSON objects or arrays instead of a flat string-to-string map JSON syntax error such as trailing commas, missing quotes, or unclosed brackets
Is the "Error: External program failed to produce valid JSON" error serious?
Yes. This is a high-severity error that usually blocks work until it is fixed, but the steps above resolve it.

Official resources & further reading