This error occurs when Elasticsearch nodes cannot discover or elect a master node, preventing cluster formation. Usually caused by discovery misconfiguration, network issues, or insufficient master-eligible nodes to form a quorum.
The MasterNotDiscoveredException indicates that Elasticsearch nodes in your cluster are unable to discover or communicate with the master node, or cannot elect a new master node when needed. The master node is critical in Elasticsearch as it manages cluster-wide operations like creating/deleting indices, tracking cluster members, and allocating shards. This exception typically appears when a cluster cannot form because nodes cannot find each other during the discovery process, or when an existing cluster loses its master node and cannot elect a replacement. The discovery process relies on nodes communicating over the network (default port 9300) and having proper configuration for `discovery.seed_hosts` and `cluster.initial_master_nodes`. In production environments, this error prevents all cluster operations from functioning, as every Elasticsearch operation requires a functioning master node to coordinate activities across the cluster.
First, verify that your Elasticsearch nodes are running and can communicate:
# Check if Elasticsearch is running
curl -X GET "localhost:9200/_cluster/health?pretty"
# Check which nodes are visible
curl -X GET "localhost:9200/_cat/nodes?v"
# Check master node status
curl -X GET "localhost:9200/_cat/master?v"If you get connection errors or the cluster health endpoint is unreachable, verify that:
- Elasticsearch process is running (ps aux | grep elasticsearch)
- Port 9200 (HTTP) and 9300 (transport) are listening (netstat -tlnp | grep 9[23]00)
- No firewall rules are blocking these ports
Review the Elasticsearch logs for specific error messages:
# Location varies by installation method
tail -f /var/log/elasticsearch/elasticsearch.log
# or
journalctl -u elasticsearch -fThe discovery.seed_hosts setting tells each node where to find other master-eligible nodes. Edit elasticsearch.yml on each node:
# elasticsearch.yml
cluster.name: my-cluster
# List all master-eligible nodes by IP or hostname
discovery.seed_hosts:
- 192.168.1.10:9300
- 192.168.1.11:9300
- 192.168.1.12:9300
# Or use hostnames
discovery.seed_hosts:
- es-master-1
- es-master-2
- es-master-3Important considerations:
- Include at least one master-eligible node that will be available
- Use consistent cluster names across all nodes
- Port 9300 is the default transport port (can be omitted if using default)
- Ensure DNS resolution works if using hostnames
Restart the node after configuration changes:
sudo systemctl restart elasticsearchWhen starting a brand-new cluster for the first time, you must specify which nodes can participate in the initial master election. Add this to elasticsearch.yml on all master-eligible nodes:
# elasticsearch.yml
cluster.initial_master_nodes:
- es-master-1
- es-master-2
- es-master-3Use the node.name values (not IP addresses) that match each node's configuration:
# On first node (es-master-1)
node.name: es-master-1
# On second node (es-master-2)
node.name: es-master-2
# On third node (es-master-3)
node.name: es-master-3Critical: After the cluster successfully forms for the first time, remove cluster.initial_master_nodes from all node configurations and restart. This setting should only exist during initial bootstrap.
# After cluster is running, edit elasticsearch.yml and remove
# cluster.initial_master_nodes, then restart
sudo systemctl restart elasticsearchEnsure all nodes can communicate on port 9300 (Elasticsearch transport protocol):
# From each node, test connectivity to other nodes
telnet es-master-2 9300
# or
nc -zv es-master-2 9300
# Test all master-eligible nodes
for host in es-master-1 es-master-2 es-master-3; do
echo "Testing $host..."
nc -zv $host 9300
doneCheck firewall rules:
# On Ubuntu/Debian with ufw
sudo ufw status
sudo ufw allow 9300/tcp
# On RHEL/CentOS with firewalld
sudo firewall-cmd --list-all
sudo firewall-cmd --permanent --add-port=9300/tcp
sudo firewall-cmd --reload
# Check iptables rules
sudo iptables -L -n | grep 9300For cloud environments (AWS, GCP, Azure), verify security group rules allow TCP port 9300 between Elasticsearch instances.
Elasticsearch requires a majority (quorum) of master-eligible nodes to elect a master. For a cluster with N master-eligible nodes, you need at least (N/2) + 1 available.
Recommended configurations:
- 3 master-eligible nodes: Need 2 available (tolerates 1 failure)
- 5 master-eligible nodes: Need 3 available (tolerates 2 failures)
- 7 master-eligible nodes: Need 4 available (tolerates 3 failures)
Check how many master-eligible nodes are configured:
# elasticsearch.yml - ensure nodes have master role
node.roles: [ master, data ]
# or dedicated master nodes
node.roles: [ master ]Verify master-eligible nodes in the cluster:
curl -X GET "localhost:9200/_cat/nodes?v&h=name,node.role,master"If you've lost too many master nodes, you may need to:
1. Start additional master-eligible nodes to restore quorum
2. Or temporarily reconfigure with fewer required nodes (not recommended for production)
If configuration changes don't resolve the issue, perform a full cluster restart:
For production environments, use a rolling restart:
# 1. Disable shard allocation
curl -X PUT "localhost:9200/_cluster/settings" -H 'Content-Type: application/json' -d'
{
"persistent": {
"cluster.routing.allocation.enable": "primaries"
}
}'
# 2. Stop one node at a time, update config, restart
sudo systemctl stop elasticsearch
# Edit elasticsearch.yml with correct discovery settings
sudo systemctl start elasticsearch
# 3. Wait for node to rejoin, then repeat for next node
# 4. Re-enable shard allocation after all nodes are back
curl -X PUT "localhost:9200/_cluster/settings" -H 'Content-Type: application/json' -d'
{
"persistent": {
"cluster.routing.allocation.enable": "all"
}
}'For new/test clusters that won't form, start fresh:
# 1. Stop all Elasticsearch nodes
sudo systemctl stop elasticsearch
# 2. Configure discovery settings on all nodes
# 3. Start master-eligible nodes FIRST, one at a time
sudo systemctl start elasticsearch
# 4. Watch logs to confirm cluster forms
journalctl -u elasticsearch -f
# 5. Once master is elected, start remaining nodesSplit-brain prevention: Elasticsearch 7.0+ uses a voting configuration that automatically adjusts when nodes leave, eliminating classic split-brain scenarios. The discovery.zen.minimum_master_nodes setting from earlier versions is deprecated and should not be used.
Dedicated master nodes: For large production clusters (50+ data nodes), use dedicated master nodes that don't handle data or client requests. This prevents resource contention from affecting cluster stability:
# Dedicated master node
node.roles: [ master ]Cloud/container deployments: When running Elasticsearch in Docker, Kubernetes, or cloud environments, ensure:
- Container networking allows inter-pod/container communication on port 9300
- For Kubernetes, use StatefulSets with headless services for stable network identities
- Set network.host appropriately (often 0.0.0.0 in containers)
- Use network.publish_host to advertise the correct external IP if nodes are behind NAT
Auto-bootstrap considerations: Single-node clusters automatically bootstrap without cluster.initial_master_nodes. However, if you later add nodes, you must configure discovery properly.
Time synchronization: Ensure all nodes use NTP or similar time synchronization. Clock skew beyond 500ms can cause master election issues.
DNS caching: If using hostnames in discovery.seed_hosts, be aware of DNS TTL and caching. Some Java versions cache DNS indefinitely by default. Consider setting networkaddress.cache.ttl in JVM options.
QueryShardException: No mapping found for [field] in order to sort on
How to fix "QueryShardException: No mapping found for field in order to sort on" in Elasticsearch
IllegalStateException: There are no ingest nodes in this cluster, unable to forward request to an ingest node
How to fix "There are no ingest nodes in this cluster" in Elasticsearch
IndexNotFoundException: no such index [index_name]
How to fix "IndexNotFoundException: no such index [index_name]" in Elasticsearch
DocumentMissingException: [index][type][id]: document missing
DocumentMissingException: Document missing
ParsingException: Unknown key for a START_OBJECT in [query]
How to fix "ParsingException: Unknown key for a START_OBJECT in [query]" in Elasticsearch