This error occurs when multiple Git operations attempt to modify the shallow file simultaneously, typically in CI/CD environments with concurrent jobs. The fix involves isolating build directories or removing stale lock files.
The "fatal: shallow file has changed since we read it" error indicates a race condition where multiple Git processes are trying to access or modify the `.git/shallow` file at the same time. When you create a shallow clone using `git clone --depth N`, Git stores information about the truncated history in the `.git/shallow` file. This file tracks which commits are considered "root" commits in your shallow repository, even though they have parents in the full repository. The error typically occurs in CI/CD environments (like GitLab CI, GitHub Actions, or Jenkins) when multiple concurrent jobs or pipelines share the same build directory. When two jobs simultaneously try to fetch or clone into the same location, one process reads the shallow file while another modifies it, causing this conflict.
Configure your CI/CD to use unique paths for each pipeline or job. In GitLab CI, add this to your .gitlab-ci.yml:
variables:
GIT_CLONE_PATH: '$CI_BUILDS_DIR/$CI_PROJECT_NAME/$CI_PIPELINE_ID'For per-job isolation (when jobs within the same pipeline conflict):
variables:
GIT_CLONE_PATH: '$CI_BUILDS_DIR/$CI_PROJECT_NAME/job_$CI_JOB_ID'For GitLab runners, ensure custom build directories are enabled in your config.toml:
[[runners]]
[runners.custom_build_dir]
enabled = trueThis allows the GIT_CLONE_PATH variable to take effect. Without this setting, GitLab ignores custom clone paths.
If cancelled jobs leave behind lock files, add a pre-clone cleanup script. In your GitLab runner config.toml:
[[runners]]
pre_clone_script = "rm -f /builds/*/*/.git/shallow.lock"
[runners.custom_build_dir]
enabled = trueFor GitHub Actions or other CI systems, add a cleanup step at the start of your workflow:
- name: Clean stale locks
run: find . -name "shallow.lock" -delete 2>/dev/null || trueVerify you don't have multiple runners registered with the same name or sharing build directories:
# List all registered runners
gitlab-runner list
# Check runner configuration
cat /etc/gitlab-runner/config.tomlEach runner should have a unique name and, ideally, separate build directories. If runners on different machines share network storage, ensure they use different base paths.
If shallow clones continue causing problems, consider using a full clone:
# GitLab CI - disable shallow cloning
variables:
GIT_DEPTH: 0# GitHub Actions - full clone
- uses: actions/checkout@v4
with:
fetch-depth: 0This increases clone time but eliminates shallow-related race conditions entirely.
If you're working with an existing shallow clone locally, you can unshallow it:
# Convert shallow clone to full clone
git fetch --unshallow
# Or deepen the history incrementally
git fetch --depth=100Before running these commands, ensure no other Git processes are accessing the repository.
### Understanding the Shallow File
The .git/shallow file is a simple text file containing commit hashes that represent the "boundary" of your shallow clone. Git treats these commits as if they have no parents, even though they do in the full repository. When this file changes while Git is reading it, the operation fails to maintain data integrity.
### Docker and Containerized Builds
When using Docker executors in CI, each container may mount the same host directory. Solutions include:
- Using Docker volumes instead of bind mounts
- Configuring runners to use separate build directories
- Using ephemeral containers that don't share state
### Lock File Mechanism
Git uses .git/shallow.lock as a mutex to prevent concurrent writes. However, if a process crashes or is killed (common in CI cancellations), the lock file may remain, blocking subsequent operations. The pre-clone cleanup script addresses this, but you can also manually remove it:
rm -f .git/shallow.lock### GitLab-Specific Considerations
GitLab runners with the Docker executor can be particularly prone to this issue when:
- Multiple runners share the same host
- The concurrent setting is greater than 1
- Runners use shared network storage
Setting GIT_CLONE_PATH with pipeline or job IDs ensures each build gets its own isolated workspace.
kex_exchange_identification: Connection closed by remote host
Connection closed by remote host when connecting to Git server
fatal: unable to access: Proxy auto-configuration failed
How to fix 'Proxy auto-configuration failed' in Git
fatal: unable to access: Authentication failed (proxy requires basic auth)
How to fix 'Authentication failed (proxy requires basic auth)' in Git
fatal: unable to access: no_proxy configuration not working
How to fix 'no_proxy configuration not working' in Git
fatal: unable to read tree object in treeless clone
How to fix 'unable to read tree object in treeless clone' in Git