This error occurs when Docker's BTRFS storage driver runs out of allocatable chunk space, even though disk space may appear available. The BTRFS filesystem cannot create new chunks for container operations. Rebalancing the filesystem or freeing chunk space is required to recover.
The "BTRFS error: unable to allocate chunk" error indicates that the BTRFS filesystem backing Docker cannot allocate new storage chunks, preventing container creation, image pulls, or write operations. This is a filesystem-level error that occurs specifically when using Docker with the BTRFS storage driver. BTRFS allocates storage in chunks (typically 1GB for data and 256MB for metadata). When Docker performs operations like creating containers, building images, or writing data, BTRFS needs to allocate new chunks. If the filesystem cannot allocate chunksโeven if `df` shows available spaceโthis error occurs. The deceptive aspect of this error is that standard Unix tools like `df -h` will often show free space, but BTRFS manages space differently through its chunk allocation system. The filesystem may have fragmented or poorly utilized chunks, leaving no contiguous unallocated space for new chunk creation. Docker's pattern of starting/stopping many containers with numerous small writes is particularly prone to creating this condition.
Standard Unix tools like df don't accurately report BTRFS space. Use BTRFS-native commands:
# Show BTRFS filesystem status
sudo btrfs filesystem show /var/lib/docker
# Get detailed usage breakdown
sudo btrfs filesystem usage /var/lib/docker
# Check device statistics
sudo btrfs device stats /var/lib/dockerLook at the output for:
- Device unallocated: Space available for new chunks
- Data ratio/Metadata ratio: How efficiently space is used
- Used/Total: Actual data stored vs allocated
Example problematic output:
Overall:
Device size: 100.00GiB
Device allocated: 99.00GiB
Device unallocated: 1.00GiBIf "Device unallocated" is very low or zero, you need to rebalance or free space.
Rebalancing relocates data from underutilized chunks to free up unallocated space. Start conservatively:
# Balance chunks with less than 5% usage (safest, minimal impact)
sudo btrfs balance start -dusage=5 /var/lib/docker
# Check balance progress
sudo btrfs balance status /var/lib/dockerIf no blocks were relocated, gradually increase the usage threshold:
# Try 10% usage threshold
sudo btrfs balance start -dusage=10 /var/lib/docker
# Then 20% if needed
sudo btrfs balance start -dusage=20 /var/lib/docker
# More aggressive - 50%
sudo btrfs balance start -dusage=50 /var/lib/dockerNote: Balance operations can take hours on large filesystems and may impact performance. Consider running during off-peak hours.
If even the conservative balance fails with ENOSPC, try cleaning up completely empty chunks first:
# Clean up completely unused chunks (requires no extra space)
sudo btrfs balance start -dusage=0 /var/lib/dockerThis operation should always succeed because it only moves data from completely empty chunks, requiring no additional allocation.
If this succeeds, follow up with progressively higher usage values:
sudo btrfs balance start -dusage=1 /var/lib/docker
sudo btrfs balance start -dusage=5 /var/lib/docker
sudo btrfs balance start -dusage=10 /var/lib/dockerRemove unused Docker resources to reduce chunk usage:
# Remove stopped containers
docker container prune -f
# Remove unused images
docker image prune -a -f
# Remove unused volumes (careful - may delete data)
docker volume prune -f
# Remove build cache
docker builder prune -a -f
# Nuclear option - remove everything unused
docker system prune -a --volumes -f
# Check space reclaimed
docker system dfAfter cleanup, run another balance:
sudo btrfs balance start -dusage=10 /var/lib/dockerIf balance operations fail due to ENOSPC, temporarily add another device to the filesystem:
# Create a sparse file as temporary device (adjust size as needed)
sudo truncate -s 10G /tmp/btrfs-temp.img
# Add it to the BTRFS filesystem
sudo btrfs device add /tmp/btrfs-temp.img /var/lib/docker
# Now run balance (should succeed with extra space)
sudo btrfs balance start /var/lib/docker
# After balance completes, remove the temporary device
sudo btrfs device remove /tmp/btrfs-temp.img /var/lib/docker
# Clean up the temporary file
sudo rm /tmp/btrfs-temp.imgAlternative: Use a USB drive or spare disk partition instead of a sparse file for better performance.
# Using a USB drive
sudo btrfs device add /dev/sdb1 /var/lib/docker
sudo btrfs balance start /var/lib/docker
sudo btrfs device remove /dev/sdb1 /var/lib/dockerSometimes metadata chunks are exhausted while data chunks have space. Balance them separately:
# Check metadata vs data usage
sudo btrfs filesystem usage /var/lib/docker
# Balance metadata chunks (use cautiously)
sudo btrfs balance start -musage=50 /var/lib/docker
# Balance data chunks
sudo btrfs balance start -dusage=50 /var/lib/dockerWarning: Metadata balancing can sometimes worsen ENOSPC issues. Only use if data balancing alone doesn't resolve the problem.
For a combined balance with filters:
# Balance both data and metadata with usage filters
sudo btrfs balance start -dusage=50 -musage=50 /var/lib/dockerDocker containers running databases generate many small writes that fragment BTRFS. Disable COW for such workloads:
# For existing directories (doesn't affect existing files)
sudo chattr +C /var/lib/docker/volumes/database_volume
# For new database containers, mount a nodatacow volume
docker run -v /nodatacow/mysql:/var/lib/mysql mysqlCreate a dedicated nodatacow directory:
sudo mkdir /nodatacow
sudo chattr +C /nodatacow
# Mount volumes from here for database containersIn Docker Compose:
services:
database:
image: postgres
volumes:
- /nodatacow/postgres:/var/lib/postgresql/dataIf BTRFS problems persist, consider using a different storage driver:
# Check current storage driver
docker info | grep "Storage Driver"
# Stop Docker
sudo systemctl stop docker
# Back up Docker data (optional)
sudo cp -au /var/lib/docker /var/lib/docker.backup
# Remove Docker data directory
sudo rm -rf /var/lib/docker
# Configure overlay2 (most recommended)
sudo mkdir -p /etc/docker
echo '{"storage-driver": "overlay2"}' | sudo tee /etc/docker/daemon.json
# Start Docker
sudo systemctl start docker
# Verify new driver
docker info | grep "Storage Driver"Note: This removes all existing images and containers. Re-pull images after switching.
If your root filesystem is BTRFS but you want overlay2, mount a separate ext4/xfs filesystem:
# Create ext4 filesystem on spare partition
sudo mkfs.ext4 /dev/sdb1
# Mount for Docker
sudo mount /dev/sdb1 /var/lib/docker
# Add to /etc/fstab for persistence
echo '/dev/sdb1 /var/lib/docker ext4 defaults 0 0' | sudo tee -a /etc/fstabPrevent future chunk allocation issues with regular maintenance:
# Create maintenance script
sudo tee /usr/local/bin/docker-btrfs-maintenance.sh << 'EOF'
#!/bin/bash
# Docker BTRFS maintenance script
# Balance data chunks with low usage
btrfs balance start -dusage=10 /var/lib/docker
# Scrub filesystem for errors
btrfs scrub start /var/lib/docker
# Clean up Docker
docker system prune -f
EOF
sudo chmod +x /usr/local/bin/docker-btrfs-maintenance.sh
# Add cron job for weekly maintenance
echo "0 3 * * 0 root /usr/local/bin/docker-btrfs-maintenance.sh" | sudo tee /etc/cron.d/docker-btrfs-maintenanceMonitor BTRFS health:
# Add to monitoring system
UNALLOCATED=$(sudo btrfs filesystem usage /var/lib/docker | grep "Device unallocated" | awk '{print $3}')
echo "BTRFS unallocated space: $UNALLOCATED"Why df lies on BTRFS: Standard df reports total block device size minus used blocks. BTRFS allocates space in chunks, and unallocated space (not assigned to any chunk) is what matters for new allocations. A filesystem can show 50% free via df but have 0% unallocated space because all space is assigned to underutilized chunks.
Docker's problematic write pattern: Docker creates and destroys many small subvolumes (for container layers). Each container layer is a BTRFS snapshot. Starting and stopping containers rapidly creates fragmented, underutilized chunks that don't get automatically consolidated.
Balance operation internals: btrfs balance reads chunks matching the filter (e.g., -dusage=10 means chunks less than 10% full), rewrites their data to other chunks, and frees the original chunks. This consolidates sparse data and creates unallocated space.
The metadata chunk trap: BTRFS has separate chunks for data and metadata. Docker's many small files create disproportionate metadata usage. You might have plenty of data chunk space but exhausted metadata chunks. Use btrfs filesystem usage to identify which type is exhausted.
Single vs RAID profiles: On single-device BTRFS, use -dusage for data and -musage for metadata. On RAID configurations, you may need to specify -dprofiles and -mprofiles to balance specific RAID levels.
BTRFS quotas and Docker: If BTRFS quotas (qgroups) are enabled, they can cause additional allocation failures. Check with btrfs qgroup show /var/lib/docker. Consider disabling quotas if they're causing issues: btrfs quota disable /var/lib/docker.
Snapshot-aware defragmentation: Running btrfs filesystem defragment on Docker's BTRFS can actually increase space usage by breaking shared extents between snapshots. Avoid defragmentation on Docker's storage.
Emergency recovery without extra device: If you cannot add a device and balance fails:
1. Export critical container data: docker export container_name > backup.tar
2. Note running container images: docker ps --format '{{.Image}}'
3. Stop Docker and delete /var/lib/docker
4. Restart Docker and re-pull images
5. Restore containers from backups
Kernel version considerations: BTRFS improvements vary significantly by kernel version. Kernel 5.x+ has better space management than 4.x. Consider upgrading your kernel if running an older version with chronic BTRFS issues.
unable to configure the Docker daemon with file /etc/docker/daemon.json
How to fix 'unable to configure the Docker daemon with file daemon.json' in Docker
docker: Error response from daemon: OCI runtime create failed: container_linux.go: starting container process caused: exec: "/docker-entrypoint.sh": stat /docker-entrypoint.sh: no such file or directory
How to fix 'exec: entrypoint.sh: no such file or directory' in Docker
image operating system "linux" cannot be used on this platform
How to fix 'image operating system linux cannot be used on this platform' in Docker
dockerfile parse error line 5: unknown instruction: RRUN
How to fix 'unknown instruction' Dockerfile parse error in Docker
manifest unknown: manifest unknown
How to fix 'manifest unknown' in Docker