How to fix fatal: shallow file has changed since we read it in Git
This error occurs when multiple Git operations attempt to modify the shallow file simultaneously, typically in CI/CD environments with concurrent jobs. The fix involves isolating build directories or removing stale lock files.
Quick fix
Fixes fatal: shallow file has changed since we read it
variables:
GIT_CLONE_PATH: '$CI_BUILDS_DIR/$CI_PROJECT_NAME/$CI_PIPELINE_ID'- Use unique build directories per pipeline
- Enable custom build directories in runner config
- Remove stale shallow.lock files
- Check for duplicate runners
variables:GIT_CLONE_PATH: '$CI_BUILDS_DIR/$CI_PROJECT_NAME/$CI_PIPELINE_ID'fatal: shallow file has changed since we read it
What this error means
The "fatal: shallow file has changed since we read it" error indicates a race condition where multiple Git processes are trying to access or modify the `.git/shallow` file at the same time. When you create a shallow clone using `git clone --depth N`, Git stores information about the truncated history in the `.git/shallow` file. This file tracks which commits are considered "root" commits in your shallow repository, even though they have parents in the full repository. The error typically occurs in CI/CD environments (like GitLab CI, GitHub Actions, or Jenkins) when multiple concurrent jobs or pipelines share the same build directory. When two jobs simultaneously try to fetch or clone into the same location, one process reads the shallow file while another modifies it, causing this conflict.
Typical symptoms
- CI/CD pipeline jobs fail intermittently with this error
- Error appears randomly when running concurrent builds
- Git fetch or clone operations fail in shared build environments
- Multiple Docker containers or runners accessing the same build directory
- Jobs that were previously working start failing without code changes
- The error disappears when re-running the failed job
Common causes
- Multiple CI/CD jobs sharing the same build directory path
- Concurrent pipelines running on the same GitLab runner or CI host
- Duplicate runners with the same name accessing shared storage
- A shallow.lock file left behind from a cancelled or crashed job
- Docker containers with shared volume mounts for build directories
- Race condition when fetching updates to a shallow clone
How to fix "fatal: shallow file has changed since we read it"
Configure your CI/CD to use unique paths for each pipeline or job. In GitLab CI, add this to your .gitlab-ci.yml:
variables:
GIT_CLONE_PATH: '$CI_BUILDS_DIR/$CI_PROJECT_NAME/$CI_PIPELINE_ID'For per-job isolation (when jobs within the same pipeline conflict):
variables:
GIT_CLONE_PATH: '$CI_BUILDS_DIR/$CI_PROJECT_NAME/job_$CI_JOB_ID'For GitLab runners, ensure custom build directories are enabled in your config.toml:
[[runners]]
[runners.custom_build_dir]
enabled = trueThis allows the GIT_CLONE_PATH variable to take effect. Without this setting, GitLab ignores custom clone paths.
If cancelled jobs leave behind lock files, add a pre-clone cleanup script. In your GitLab runner config.toml:
[[runners]]
pre_clone_script = "rm -f /builds/*/*/.git/shallow.lock"
[runners.custom_build_dir]
enabled = trueFor GitHub Actions or other CI systems, add a cleanup step at the start of your workflow:
- name: Clean stale locks
run: find . -name "shallow.lock" -delete 2>/dev/null || trueVerify you don't have multiple runners registered with the same name or sharing build directories:
# List all registered runners
gitlab-runner list
# Check runner configuration
cat /etc/gitlab-runner/config.tomlEach runner should have a unique name and, ideally, separate build directories. If runners on different machines share network storage, ensure they use different base paths.
If shallow clones continue causing problems, consider using a full clone:
# GitLab CI - disable shallow cloning
variables:
GIT_DEPTH: 0# GitHub Actions - full clone
- uses: actions/checkout@v4
with:
fetch-depth: 0This increases clone time but eliminates shallow-related race conditions entirely.
If you're working with an existing shallow clone locally, you can unshallow it:
# Convert shallow clone to full clone
git fetch --unshallow
# Or deepen the history incrementally
git fetch --depth=100Before running these commands, ensure no other Git processes are accessing the repository.
Advanced notes
### Understanding the Shallow File
The .git/shallow file is a simple text file containing commit hashes that represent the "boundary" of your shallow clone. Git treats these commits as if they have no parents, even though they do in the full repository. When this file changes while Git is reading it, the operation fails to maintain data integrity.
### Docker and Containerized Builds
When using Docker executors in CI, each container may mount the same host directory. Solutions include:
- Using Docker volumes instead of bind mounts
- Configuring runners to use separate build directories
- Using ephemeral containers that don't share state
### Lock File Mechanism
Git uses .git/shallow.lock as a mutex to prevent concurrent writes. However, if a process crashes or is killed (common in CI cancellations), the lock file may remain, blocking subsequent operations. The pre-clone cleanup script addresses this, but you can also manually remove it:
rm -f .git/shallow.lock### GitLab-Specific Considerations
GitLab runners with the Docker executor can be particularly prone to this issue when:
- Multiple runners share the same host
- The concurrent setting is greater than 1
- Runners use shared network storage
Setting GIT_CLONE_PATH with pipeline or job IDs ensures each build gets its own isolated workspace.
Frequently asked questions
- What does "fatal: shallow file has changed since we read it" mean?
- git/shallow file at the same time. git/shallow file. This file tracks which commits are considered "root" commits in your shallow repository, even though they have parents in the full repository.
- How do I fix "fatal: shallow file has changed since we read it"?
- This error occurs when multiple Git operations attempt to modify the shallow file simultaneously, typically in CI/CD environments with concurrent jobs. The fix involves isolating build directories or removing stale lock files.
- What causes "fatal: shallow file has changed since we read it"?
- Multiple CI/CD jobs sharing the same build directory path Concurrent pipelines running on the same GitLab runner or CI host Duplicate runners with the same name accessing shared storage
- Is the "fatal: shallow file has changed since we read it" error serious?
- It is a medium-severity error โ more of a blocker than a risk. The steps above resolve it.
Official resources & further reading
- ๐Git Official Documentation
- ๐ฌ
- ๐
- ๐
- ๐