How to fix Error response from daemon: Unknown runtime specified nvidia in Docker

DockerINTERMEDIATEMEDIUM

This error occurs when Docker cannot find the NVIDIA container runtime. It typically means the nvidia-container-toolkit is not installed, Docker's daemon.json is not configured correctly, or Docker needs to be restarted after installation.

What this error means

When you try to run a Docker container with `--runtime=nvidia` or use GPU acceleration, Docker looks for the NVIDIA runtime in its configuration. This runtime acts as a bridge between Docker and your NVIDIA GPU drivers, allowing containers to access GPU resources. The "Unknown runtime specified nvidia" error indicates that Docker doesn't recognize "nvidia" as a valid runtime. This happens because: 1. The NVIDIA Container Toolkit (which includes `nvidia-container-runtime`) is not installed 2. Docker's daemon.json configuration doesn't include the nvidia runtime entry 3. Docker daemon hasn't been restarted after installing the toolkit 4. The runtime binary path is incorrect in the configuration

How to fix "Error response from daemon: Unknown runtime specified nvidia"

1Check if NVIDIA runtime is available

First, verify what runtimes Docker currently knows about:

bash

docker info | grep -i runtime

If you see only Runtimes: runc (without nvidia), the NVIDIA runtime is not configured. If nvidia is listed but still failing, the configuration may be incorrect.

2Install the NVIDIA Container Toolkit

Install the NVIDIA Container Toolkit which provides the nvidia-container-runtime:

For Ubuntu/Debian:

bash

# Add the NVIDIA GPG key
curl -fsSL https://nvidia.github.io/libnvidia-container/gpgkey | sudo gpg --dearmor -o /usr/share/keyrings/nvidia-container-toolkit-keyring.gpg

# Add the repository
curl -s -L https://nvidia.github.io/libnvidia-container/stable/deb/nvidia-container-toolkit.list | \
  sed 's#deb https://#deb [signed-by=/usr/share/keyrings/nvidia-container-toolkit-keyring.gpg] https://#g' | \
  sudo tee /etc/apt/sources.list.d/nvidia-container-toolkit.list

# Install the toolkit
sudo apt-get update
sudo apt-get install -y nvidia-container-toolkit

For RHEL/CentOS:

bash

curl -s -L https://nvidia.github.io/libnvidia-container/stable/rpm/nvidia-container-toolkit.repo | \
  sudo tee /etc/yum.repos.d/nvidia-container-toolkit.repo

sudo yum install -y nvidia-container-toolkit

3Configure Docker to use the NVIDIA runtime

Use the nvidia-ctk tool to automatically configure Docker:

bash

sudo nvidia-ctk runtime configure --runtime=docker

This command modifies /etc/docker/daemon.json to include the nvidia runtime. You can verify the configuration:

bash

cat /etc/docker/daemon.json

You should see something like:

json

{
  "runtimes": {
    "nvidia": {
      "path": "/usr/bin/nvidia-container-runtime",
      "runtimeArgs": []
    }
  }
}

4Restart the Docker daemon

Apply the configuration changes by restarting Docker:

bash

sudo systemctl restart docker

Alternatively, you can send a SIGHUP signal:

bash

sudo pkill -SIGHUP dockerd

Verify the runtime is now available:

bash

docker info | grep -i runtime
# Should show: Runtimes: nvidia runc

5Test GPU access in a container

Verify everything works by running a test container:

bash

# Using the new --gpus flag (recommended for Docker 19.03+)
docker run --rm --gpus all nvidia/cuda:12.0-base nvidia-smi

# Or using the legacy --runtime=nvidia flag
docker run --rm --runtime=nvidia nvidia/cuda:12.0-base nvidia-smi

You should see the output of nvidia-smi showing your GPU(s).

How to fix Error response from daemon: Unknown runtime specified nvidia in Docker

What this error means

Typical symptoms

Common causes

How to fix "Error response from daemon: Unknown runtime specified nvidia"

Advanced notes

Related errors

Official resources & further reading