How to fix container runtime error in Kubernetes

KubernetesADVANCEDCRITICALUpdated Dec 6, 2025· 277 views

Container runtime errors indicate the kubelet cannot communicate with Docker, containerd, or another CRI runtime. Pods fail to start and nodes become degraded when the runtime is unavailable or misconfigured.

Quick fix

Fixes container runtime error

kubectl get nodes -o wide  # Check CONTAINER-RUNTIME column
  1. Verify container runtime is running
  2. Check container runtime logs
  3. Verify runtime socket file exists
  4. Verify disk space for container runtime
kubernetes — error
kubectl get nodes -o wide  # Check CONTAINER-RUNTIME column
container runtime error

What this error means

The kubelet relies on a container runtime (Docker, containerd, CRI-O) to: 1. Create and run containers 2. Manage container lifecycle (start, stop, restart) 3. Handle image pulls 4. Configure container networking When the runtime fails, the kubelet cannot execute any pods. Common causes: - Runtime daemon crashed or is not responding - Runtime socket file missing or permissions wrong - Container runtime image pull failures - Incompatible kubelet and runtime versions - Node disk full, preventing runtime operations

Typical symptoms

  • Pods fail with "container runtime error" or "rpc error: code = Unavailable"
  • All pods on a node fail to start
  • Pod status shows "SandboxNotReady" or "PodInitializing"
  • kubelet logs show "failed to get runtime service"
  • Node events mention "failed to create sandbox" or "runtime unavailable"
  • Docker/containerd daemon is not running or unresponsive
  • kubectl describe pod shows "failed to start container"

Common causes

  1. Container runtime daemon (dockerd, containerd) crashed or stopped
  2. Runtime socket file missing (e.g., /run/containerd/containerd.sock)
  3. Container runtime out of disk space
  4. Kubelet cannot authenticate with runtime (permission/socket issues)
  5. Runtime image pull failures blocking pod startup
  6. Incompatible CRI API version between kubelet and runtime
  7. Container storage (overlayfs) corruption or misconfiguration

How to fix "container runtime error"

1Verify container runtime is running

Check which runtime your cluster uses:

bash
kubectl get nodes -o wide  # Check CONTAINER-RUNTIME column

SSH into the affected node and verify runtime status:

For Docker:

bash
sudo systemctl status docker
sudo systemctl start docker  # If stopped
sudo docker ps  # Test connectivity

For containerd:

bash
sudo systemctl status containerd
sudo systemctl start containerd
sudo ctr -a /run/containerd/containerd.sock version

For CRI-O:

bash
sudo systemctl status crio
sudo systemctl start crio

If the daemon is running, proceed to next step.

2Check container runtime logs

Review runtime logs for errors:

Docker:

bash
sudo journalctl -u docker -f
sudo tail -f /var/log/docker.log

containerd:

bash
sudo journalctl -u containerd -f
sudo tail -f /var/log/containerd/containerd.log

CRI-O:

bash
sudo journalctl -u crio -f

Look for errors like:
- "failed to create container"
- "out of memory"
- "no space left on device"
- "permission denied"
- "connection refused"

3Verify runtime socket file exists

Check that the runtime socket exists and is accessible:

Docker:

bash
ls -la /var/run/docker.sock
# Should output: srw-rw---- root docker /var/run/docker.sock

containerd:

bash
ls -la /run/containerd/containerd.sock
# Should output: srw-rw-rw- root root /run/containerd/containerd.sock

CRI-O:

bash
ls -la /var/run/crio/crio.sock

If socket file is missing:
1. Restart the runtime daemon
2. Check if the runtime directory exists
3. Verify mount points if using unusual storage

If permissions are wrong:

bash
sudo chmod 660 /var/run/docker.sock
sudo chown root:docker /var/run/docker.sock
4Verify disk space for container runtime

Container runtime typically needs space in:
- /var/lib/docker (Docker)
- /var/lib/containerd (containerd)
- /var/lib/crio (CRI-O)

Check disk usage:

bash
df -h /var/lib/docker
df -h /var/lib/containerd
du -sh /var/lib/docker/*  # Breakdown by component

If disk is full (> 90%):

bash
# Clean up unused images
sudo docker rmi $(docker images -q -f "dangling=true")
sudo ctr -n k8s.io i rm $(ctr -n k8s.io i ls -q | tail -20)

# Remove unused containers
sudo docker container prune

# Clear build cache
sudo docker builder prune

For persistent space issues, increase the volume or move storage:

bash
sudo docker info | grep "Storage Driver"
5Check kubelet configuration for runtime socket

Verify kubelet is configured to use the correct runtime socket:

bash
ps aux | grep kubelet | grep -E "(container-runtime|container-runtime-endpoint)"

For containerd (Kubernetes 1.24+):

bash
kubectl describe node <node> | grep -i "container"

Edit kubelet config:

For kubeadm clusters:

bash
sudo nano /etc/sysconfig/kubelet
# or
sudo nano /etc/kubernetes/kubelet.env

Ensure it includes:

bash
--container-runtime=remote
--container-runtime-endpoint=unix:///var/run/containerd/containerd.sock

Or for Docker:

bash
--container-runtime=docker

Restart kubelet:

bash
sudo systemctl restart kubelet
sudo journalctl -u kubelet -f
6Inspect container runtime API errors

Test runtime API connectivity:

containerd:

bash
sudo ctr -a /run/containerd/containerd.sock version
sudo ctr -a /run/containerd/containerd.sock images list

Docker:

bash
sudo docker version
sudo docker ps

CRI-O:

bash
sudo crictl version
sudo crictl images

If API errors appear:
- Check socket permissions (see Step 3)
- Restart runtime daemon (Step 1)
- Check for stale connections:

bash
sudo lsof | grep containerd.sock
sudo kill <stale-pid>
7Drain node and restart container runtime

As a last resort, restart the entire runtime:

bash
# Cordon the node (prevent new pods)
kubectl cordon <node-name>

# Drain existing pods
kubectl drain <node-name> --ignore-daemonsets --delete-emptydir-data

# SSH into node and restart runtime
sudo systemctl stop kubelet
sudo systemctl restart docker  # or containerd, crio
sudo systemctl start kubelet

# Monitor recovery
sudo journalctl -u kubelet -f
kubectl uncordon <node-name>
kubectl get nodes -w

Pods will be rescheduled to other nodes. Monitor their status:

bash
kubectl get pods -A -w | grep <node-name>
8Upgrade container runtime if version mismatch

Check for version compatibility:

bash
kubectl version  # Kubernetes version
sudo docker --version  # Docker version
sudo containerd --version  # containerd version

Ref: https://kubernetes.io/docs/setup/production-environment/container-runtimes/

If versions are incompatible, upgrade the runtime:

Docker (Ubuntu/Debian):

bash
sudo apt-get update
sudo apt-get install -y docker-ce=<version>
sudo systemctl restart docker

containerd (Ubuntu/Debian):

bash
sudo apt-get install -y containerd.io=<version>
sudo systemctl restart containerd

CRI-O:

bash
sudo apt-get install -y cri-o=<version>
sudo systemctl restart crio

Always test in a staging environment first.

Advanced notes

Container runtime errors at scale (many nodes) suggest a cluster-wide issue: perhaps a container registry outage, storage backend failure, or network partition. For managed services (EKS, GKE, AKS), runtime issues are usually handled by the platform—report via support. Kubernetes 1.24 deprecated Docker in favor of containerd, which is lighter and faster. For development, Docker Desktop includes a runtime; production should use dedicated runtime (containerd preferred). Runtime errors can cascade: if one pod fails to start, its retry storms can consume kubelet resources and destabilize the node. Use exponential backoff in pod restart policies. Monitoring runtime health is critical: Prometheus metrics from kubelet expose container creation/destruction errors. For stateless workloads, automatic node replacement (managed scaling) is safer than manual recovery. Custom OCI runtimes (kata, gVisor) have their own troubleshooting paths—consult runtime-specific docs.

Frequently asked questions

What does "container runtime error" mean?
The kubelet relies on a container runtime (Docker, containerd, CRI-O) to: 1. Create and run containers 2. Manage container lifecycle (start, stop, restart) 3.
How do I fix "container runtime error"?
Container runtime errors indicate the kubelet cannot communicate with Docker, containerd, or another CRI runtime. Pods fail to start and nodes become degraded when the runtime is unavailable or misconfigured.
What causes "container runtime error"?
Container runtime daemon (dockerd, containerd) crashed or stopped Runtime socket file missing (e.g., /run/containerd/containerd.sock) Container runtime out of disk space
Is the "container runtime error" error serious?
Yes. This is a critical-severity error that usually blocks work until it is fixed, but the steps above resolve it.

Official resources & further reading