How to fix devmapper: Thin Pool has 0 free data blocks in Docker

DockerADVANCEDCRITICALUpdated Dec 6, 2025· 164 views

This critical error occurs when Docker's devicemapper thin pool is completely exhausted with zero free blocks remaining. The Docker daemon becomes unresponsive and cannot create containers. Immediate action is required to recover storage space or extend the thin pool.

Quick fix

Fixes devmapper: Thin Pool has 0 free data blocks

# Try stopping all containers (may hang if daemon is unresponsive)
docker stop $(docker ps -q) 2>/dev/null || true

# Check if Docker is responding at all
timeout 10 docker info
  1. Assess the situation and stop non-critical containers
  2. Check thin pool status with LVM tools
  3. Reclaim space using fstrim (if possible)
  4. Extend thin pool if volume group has space
docker — error
# Try stopping all containers (may hang if daemon is unresponsive)
docker stop $(docker ps -q) 2>/dev/null || true
# Check if Docker is responding at all
timeout 10 docker info
devmapper: Thin Pool has 0 free data blocks

What this error means

The "Thin Pool has 0 free data blocks" error is the most severe form of thin pool exhaustion in Docker's devicemapper storage driver. Unlike warnings about "less than minimum required" blocks, having exactly 0 free blocks means the thin pool is completely full and Docker operations are entirely blocked. When the thin pool reaches zero free data blocks, the Docker daemon enters a critical state. It cannot allocate any new storage for containers, images, or volumes. Existing containers may continue running but cannot write new data. The daemon itself may become unresponsive, making normal cleanup commands like `docker system prune` fail or hang indefinitely. This error typically occurs on systems using the legacy devicemapper storage driver, which was common on RHEL/CentOS 6 and 7, older Amazon Linux AMIs, and some cloud deployments. The devicemapper driver was deprecated in Docker 25.0 (January 2024) in favor of overlay2, but many legacy systems still encounter this critical failure mode.

Typical symptoms

  • Docker daemon becomes completely unresponsive
  • All docker commands hang or fail with 'Thin Pool has 0 free data blocks'
  • CannotCreateContainerError in container orchestrators (ECS, Kubernetes)
  • Existing running containers cannot write new data
  • docker system prune hangs or fails to free space
  • CI/CD pipelines fail at any Docker operation
  • AWS ECS tasks fail with API error (500): devmapper
  • docker info command shows 100% Data Space Used

Common causes

  1. Thin pool completely filled by accumulated images, containers, and volumes
  2. No autoextend configuration causing pool to hit hard limit
  3. Large image pulls or builds consuming remaining space suddenly
  4. Orphaned layers from deleted images not being reclaimed
  5. Loopback mode with fixed-size backing file reaching capacity
  6. Volume group has no free space to extend thin pool
  7. Container writing excessive logs or data exhausting pool
  8. Multiple concurrent builds or deployments racing to fill pool

How to fix "devmapper: Thin Pool has 0 free data blocks"

1Assess the situation and stop non-critical containers

When the thin pool has 0 free blocks, the Docker daemon may be unresponsive. First, try to stop containers to free any reclaimable space:

bash
# Try stopping all containers (may hang if daemon is unresponsive)
docker stop $(docker ps -q) 2>/dev/null || true

# Check if Docker is responding at all
timeout 10 docker info

If Docker is completely unresponsive, you may need to kill the daemon:

bash
# Kill the Docker daemon (last resort)
sudo systemctl kill -s SIGKILL docker

Warning: Killing the daemon can leave containers in an inconsistent state.

2Check thin pool status with LVM tools

Since Docker may be unresponsive, use LVM tools directly to assess the situation:

bash
# View all logical volumes
sudo lvs -a

# Get detailed thin pool status
sudo lvs -a -o +devices,lv_health_status

# Check volume group free space
sudo vgs

Look for the thin pool (usually named docker-pool, docker/thinpool, or similar) and note:
- Data%: Should show 100% if completely full
- VG Free: Available space in the volume group for extension

Example output when full:

bash
LV         VG     Attr       LSize   Pool Origin Data%  Meta%
docker-pool docker twi-a-t--- 100.00g             100.00 50.00
3Reclaim space using fstrim (if possible)

If any containers are still running, try to reclaim unused blocks:

bash
# Run fstrim on all running containers
sudo sh -c "docker ps -q | xargs docker inspect --format='{{ .State.Pid }}' | xargs -IZ fstrim /proc/Z/root/" 2>/dev/null

This command:
1. Gets PIDs of running containers
2. Runs fstrim on each container's root filesystem
3. Discards unused blocks back to the thin pool

Note: This only works if Docker is still somewhat responsive and containers are running.

4Extend thin pool if volume group has space

If sudo vgs shows free space in the volume group, extend the thin pool immediately:

bash
# Find exact thin pool name
sudo lvs -a | grep -i pool

# Extend thin pool data volume (adjust path to match your setup)
sudo lvextend -l+100%FREE docker/thinpool

# Or extend by a specific size
sudo lvextend -L+50G docker/thinpool

After extending:

bash
# Restart Docker to recognize new space
sudo systemctl restart docker

# Verify space is available
docker info | grep "Data Space"

Common thin pool paths:
- /dev/docker/thinpool
- /dev/mapper/docker-pool
- /dev/mapper/docker-thinpool

5Add physical volume if volume group is full

If the volume group itself has no free space, add a new disk or partition:

bash
# Create physical volume on new disk (DESTROYS ALL DATA on disk)
sudo pvcreate /dev/sdX

# Add to the docker volume group
sudo vgextend docker /dev/sdX

# Now extend the thin pool
sudo lvextend -l+100%FREE docker/thinpool

# Restart Docker
sudo systemctl restart docker

For AWS/cloud instances, you may need to:
1. Attach a new EBS volume
2. Create partition: sudo fdisk /dev/xvdf
3. Create PV: sudo pvcreate /dev/xvdf1
4. Extend VG: sudo vgextend docker /dev/xvdf1

6Delete Docker data directory (emergency recovery)

If extension is not possible and Docker is completely stuck, you may need to delete Docker's data directory:

Warning: This removes ALL images, containers, and non-persistent volumes. Back up any important data first.

bash
# Stop Docker (force if necessary)
sudo systemctl stop docker || sudo systemctl kill docker

# Back up important volumes if accessible
sudo cp -r /var/lib/docker/volumes/important_volume /backup/

# Remove Docker data directory
sudo rm -rf /var/lib/docker

# Restart Docker (will recreate directory structure)
sudo systemctl start docker

# Verify clean state
docker info

You will need to re-pull all images and recreate containers after this operation.

7Clean up Docker resources once recovered

Once Docker is responsive again, aggressively clean up to prevent recurrence:

bash
# Remove all stopped containers
docker container prune -f

# Remove all unused images (including tagged ones)
docker image prune -a -f

# Remove all unused volumes (careful - may delete data!)
docker volume prune -f

# Remove all unused networks
docker network prune -f

# Nuclear option: remove everything unused
docker system prune -a --volumes -f

Check space recovered:

bash
docker system df
docker info | grep "Data Space"
8Configure autoextend to prevent future issues

Set up automatic thin pool extension to avoid reaching 0 blocks again:

bash
# Create or edit LVM profile
sudo vi /etc/lvm/profile/docker-thinpool.profile

Add the following configuration:

bash
activation {
  thin_pool_autoextend_threshold=70
  thin_pool_autoextend_percent=20
}

Apply the profile:

bash
# Find your thin pool logical volume path
sudo lvs -a | grep pool

# Apply profile (adjust path as needed)
sudo lvchange --metadataprofile docker-thinpool docker/thinpool

This automatically extends the pool by 20% when usage reaches 70%.

9Migrate to overlay2 storage driver (recommended long-term fix)

The devicemapper driver is deprecated. Migrate to overlay2 to eliminate thin pool issues entirely:

Warning: This removes all existing images and containers.

bash
# Stop Docker
sudo systemctl stop docker

# Back up Docker directory (optional but recommended)
sudo cp -au /var/lib/docker /var/lib/docker.backup

# Remove Docker data
sudo rm -rf /var/lib/docker

# Configure overlay2
sudo vi /etc/docker/daemon.json

Set contents to:

json
{
  "storage-driver": "overlay2"
}

Start Docker:

bash
sudo systemctl start docker
docker info | grep "Storage Driver"

Should show: Storage Driver: overlay2

Re-pull required images after migration.

Advanced notes

Why 0 free blocks is worse than "minimum required" warnings: The devicemapper driver has a safety threshold (configurable via dm.min_free_space) that triggers warnings before exhaustion. Having 0 blocks means you've bypassed or ignored these warnings, or the pool filled suddenly (e.g., during a large build). At 0 blocks, the kernel's dm-thinp module cannot allocate any new blocks, causing all write operations to fail.

Docker daemon unresponsiveness: When the thin pool is completely exhausted, the Docker daemon itself may hang on internal operations. This is because the daemon needs to write state changes to disk. Commands that should be read-only (like docker ps) may still hang because they trigger internal state updates.

Recovery order of operations: The most reliable recovery sequence is:
1. Kill the Docker daemon if unresponsive
2. Use LVM tools to extend thin pool (if possible)
3. Restart Docker
4. Clean up resources
5. Configure prevention measures

Loopback vs direct-lvm considerations: Loopback mode uses sparse files with a maximum size limit. Even if your disk has space, the loopback file cannot grow beyond its configured limit. Direct-lvm mode can use entire block devices and is easier to extend, but requires more initial setup.

AWS ECS/Elastic Beanstalk specific recovery: AWS environments using the default Amazon Linux AMI with devicemapper may need:
1. Terminate and relaunch instances with larger storage volumes
2. Or attach additional EBS volumes and extend the thin pool
The default 12GB xvdcz volume fills quickly with multiple container deployments.

Monitoring recommendations: After recovery, implement monitoring:

bash
# Add to monitoring/alerting system
USAGE=$(sudo lvs --noheadings -o data_percent docker/thinpool | tr -d ' ')
if (( $(echo "$USAGE > 80" | bc -l) )); then
  echo "ALERT: Docker thin pool at $USAGE%"
fi

Kernel-level recovery: In extreme cases where LVM commands also hang, you may need to:
1. Reboot the system
2. Boot into rescue mode
3. Manually deactivate the thin pool: lvchange -an docker/thinpool
4. Delete Docker data: rm -rf /var/lib/docker
5. Reactivate volumes: vgchange -ay
6. Reboot normally

Frequently asked questions

What does "devmapper: Thin Pool has 0 free data blocks" mean?
The "Thin Pool has 0 free data blocks" error is the most severe form of thin pool exhaustion in Docker's devicemapper storage driver. Unlike warnings about "less than minimum required" blocks, having exactly 0 free blocks means the thin pool is completely full and Docker operations are entirely blocked. When the thin pool reaches zero free data blocks, the Docker daemon enters a critical state.
How do I fix "devmapper: Thin Pool has 0 free data blocks"?
This critical error occurs when Docker's devicemapper thin pool is completely exhausted with zero free blocks remaining. The Docker daemon becomes unresponsive and cannot create containers. Immediate action is required to recover storage space or extend the thin pool.
What causes "devmapper: Thin Pool has 0 free data blocks"?
Thin pool completely filled by accumulated images, containers, and volumes No autoextend configuration causing pool to hit hard limit Large image pulls or builds consuming remaining space suddenly
Is the "devmapper: Thin Pool has 0 free data blocks" error serious?
Yes. This is a critical-severity error that usually blocks work until it is fixed, but the steps above resolve it.

Official resources & further reading