How to fix devmapper: Thin Pool has 1234 free data blocks which is less than minimum required in Docker

First, assess the current state of your thin pool to understand how severe the situation is:

# Check Docker storage driver info
docker info | grep -A 20 "Storage Driver"

Key values to look for:
- Data Space Used / Data Space Total: Shows how much of the thin pool is consumed
- Thin Pool Minimum Free Space: The threshold that triggers this error (default 10%)

Also check the thin pool directly with LVM tools:

# View logical volumes and usage percentage
sudo lvs

# Get more detailed information
sudo lvs -a -o +devices

Example output showing a nearly full pool:

LV          VG     Attr       LSize   Pool Origin Data%  Meta%
docker-pool docker twi-a-t--- 100.00g             92.50  45.00

Remove stopped containers, unused images, and orphaned volumes to free space:

# Remove stopped containers
docker container prune -f

# Remove unused images (dangling only)
docker image prune -f

# Remove ALL unused images (including tagged ones not in use)
docker image prune -a -f

# Remove unused volumes (careful - may delete data!)
docker volume prune -f

# Remove unused networks
docker network prune -f

# Or clean everything at once (recommended)
docker system prune -a --volumes -f

Check how much space was recovered:

docker system df
docker info | grep "Data Space"

Note: With devicemapper, pruning may not immediately free thin pool blocks. Restart Docker to force block reclamation:

sudo systemctl restart docker

The devicemapper driver may not automatically reclaim blocks from deleted files. Force reclamation using fstrim:

# Run fstrim on all running containers to discard unused blocks
sudo sh -c "docker ps -q | xargs docker inspect --format='{{ .State.Pid }}' | xargs -IZ fstrim /proc/Z/root/"

This command:
1. Gets the PID of each running container
2. Runs fstrim on the container's root filesystem
3. Discards unused blocks back to the thin pool

For a more aggressive cleanup when Docker is responsive:

# Run fstrim on the host filesystem too
sudo fstrim -av

If pruning doesn't recover enough space due to orphaned or fragmented layers, consolidate images by saving and reloading them:

# List images to identify what to keep
docker images

# Save important images to tar files
docker save -o myapp.tar myapp:latest
docker save -o nginx.tar nginx:alpine

# Remove all images
docker rmi $(docker images -q) -f

# Reload the images you need
docker load -i myapp.tar
docker load -i nginx.tar

# Clean up tar files
rm *.tar

This creates fresh, consolidated image layers without orphaned intermediate data.

Check if your volume group has free space to extend the thin pool:

# Check volume group free space
sudo vgs

If there's free space (VFree column), extend the thin pool:

# Find your thin pool name
sudo lvs -a | grep -i pool

# Extend the thin pool to use all free space
sudo lvextend -l+100%FREE /dev/docker/thinpool

# Or extend by a specific amount
sudo lvextend -L+20G /dev/docker/thinpool

After extending, restart Docker and verify:

sudo systemctl restart docker
docker info | grep "Data Space"

Common thin pool paths vary by setup:
- /dev/docker/thinpool
- /dev/mapper/docker-pool
- /dev/mapper/docker-thinpool

If the volume group itself is full, add a new disk or partition:

# Create physical volume on new disk (WARNING: destroys all data on disk)
sudo pvcreate /dev/sdX

# Add the physical volume to the docker volume group
sudo vgextend docker /dev/sdX

# Now extend the thin pool
sudo lvextend -l+100%FREE /dev/docker/thinpool

# Restart Docker
sudo systemctl restart docker

For AWS/cloud environments:
1. Attach a new EBS volume in the AWS console
2. Find the new device: lsblk
3. Create physical volume: sudo pvcreate /dev/xvdf
4. Extend volume group: sudo vgextend docker /dev/xvdf
5. Extend thin pool: sudo lvextend -l+100%FREE docker/thinpool

Enable automatic thin pool extension to prevent this error in the future:

# Create or edit the LVM profile for Docker
sudo vi /etc/lvm/profile/docker-thinpool.profile

Add the following configuration:

activation {
  thin_pool_autoextend_threshold=80
  thin_pool_autoextend_percent=20
}

This automatically extends the pool by 20% when usage reaches 80%.

Apply the profile to your thin pool:

# Apply the profile (adjust path to match your thin pool)
sudo lvchange --metadataprofile docker-thinpool docker/thinpool

# Verify the profile is applied
sudo lvs -o+lv_profile

Note: Autoextend only works if the volume group has free space available.

As a temporary measure, you can lower Docker's minimum free space threshold to allow operations to continue:

# Stop Docker
sudo systemctl stop docker

Edit or create /etc/docker/daemon.json:

{
  "storage-opts": [
    "dm.min_free_space=5%"
  ]
}

Restart Docker:

sudo systemctl start docker

Warning: Lowering this threshold increases the risk of thin pool exhaustion and potential data corruption. Use this only as a temporary measure while implementing a proper fix (cleanup, extension, or migration).

The devicemapper driver is deprecated. Migrating to overlay2 eliminates thin pool issues entirely and provides better performance:

Warning: This process removes all existing images and containers. Back up important data first.

# Stop Docker
sudo systemctl stop docker

# Back up Docker data (optional but recommended)
sudo cp -au /var/lib/docker /var/lib/docker.backup

# Remove Docker data directory
sudo rm -rf /var/lib/docker

# Configure overlay2 storage driver
sudo vi /etc/docker/daemon.json

Set the storage driver:

{
  "storage-driver": "overlay2"
}

Start Docker and verify:

sudo systemctl start docker
docker info | grep "Storage Driver"

Should show: Storage Driver: overlay2

You'll need to re-pull all images after migration.

Requirements: overlay2 requires:
- Linux kernel 4.0+ (kernel 3.10.0-693+ on RHEL/CentOS 7)
- d_type=true on the filesystem (check with xfs_info /var/lib/docker)