How to fix Job completions is immutable in Kubernetes

KubernetesINTERMEDIATEMEDIUMUpdated Dec 6, 2025· 132 views

The "Job completions is immutable" error occurs when you attempt to modify the `.spec.completions` field on an already-created Kubernetes Job. This field specifies required pod completions and cannot be changed after creation.

Quick fix

Fixes Job completions is immutable

kubectl delete job <job-name>
  1. Understand which Job fields are immutable
  2. Delete the existing Job
  3. Apply the updated Job manifest
  4. Plan Job changes before creation
kubernetes — error
kubectl delete job <job-name>
Job completions is immutable

What this error means

Kubernetes prevents modifying the `.spec.completions` field (and several other Job fields) after a Job is created. The `completions` field defines how many pod completions are needed for the Job to succeed. Once set at creation time, this field is locked to prevent mid-execution changes that could cause unpredictable behavior. This is an intentional design decision: if you could change the completions mid-run, pods that already completed would contradict new requirements, and the Job controller would face ambiguous completion state. Therefore, Kubernetes enforces immutability on critical Job fields.

Typical symptoms

  • kubectl apply fails with error: "spec.completions: Invalid value... field is immutable"
  • kubectl patch fails with immutability error
  • CI/CD pipelines fail during Job updates with validation errors
  • Helm upgrades error when Job configurations drift from running Job state
  • GitOps tools (ArgoCD, Flux) report validation failures for Job resources
  • Users see validation errors when attempting to update Job specifications

Common causes

  1. Attempting to change `.spec.completions` on an existing Job
  2. Also attempting to change `.spec.template` (pod template specification)
  3. Using `kubectl apply` or `kubectl patch` to modify immutable Job fields
  4. GitOps or Helm configurations drifting from current Job state
  5. Deleting a Job but then reapplying with different completions value without waiting for deletion

How to fix "Job completions is immutable"

1Understand which Job fields are immutable

Immutable Job fields (cannot be changed after creation):
- .spec.completions - number of required completions
- .spec.template - pod template specification
- .spec.selector - label selector for pod tracking
- .spec.parallelism - may be restricted based on completion mode

Mutable fields:
- .spec.parallelism - number of parallel pods (in certain conditions)
- .spec.activeDeadlineSeconds - deadline to stop running Job

If you need to change immutable fields, delete and recreate.

2Delete the existing Job

Remove the Job to allow recreation:

bash
kubectl delete job <job-name>

Wait for deletion to complete:

bash
kubectl delete job <job-name> --wait=true

Verify it's gone:

bash
kubectl get job <job-name>
# Should return: No resources found
3Apply the updated Job manifest

Once deleted, apply the new Job definition:

bash
kubectl apply -f updated-job.yaml

Verify creation:

bash
kubectl get job <job-name>
kubectl describe job <job-name>

Watch completion status:

bash
kubectl get jobs --watch
4Plan Job changes before creation

To avoid this issue, finalize Job specifications before initial creation:

yaml
apiVersion: batch/v1
kind: Job
metadata:
  name: myapp-job
spec:
  completions: 3  # Set this value correctly before creating
  parallelism: 1
  template:
    metadata:
      labels:
        app: myapp-job
    spec:
      containers:
      - name: job-container
        image: myapp:latest
      restartPolicy: Never

Test in development first before applying to production.

5Use delete-then-apply in CI/CD pipelines

For automated deployments, script the delete-and-recreate pattern:

bash
#!/bin/bash
JOB_NAME="myapp-job"
NAMESPACE="default"

# Delete if exists
kubectl delete job $JOB_NAME -n $NAMESPACE --ignore-not-found=true

# Wait for deletion
sleep 5

# Apply new Job
kubectl apply -f job.yaml -n $NAMESPACE

This ensures clean state before Job creation.

6For GitOps tools, handle Job updates carefully

In ArgoCD or Flux configurations, Jobs must be handled specially since they're immutable:

Option 1: Manual delete before update
Add a pre-sync hook in ArgoCD:

yaml
apiVersion: argoproj.io/v1alpha1
kind: Application
metadata:
  name: myapp
spec:
  # ...
  syncPolicy:
    syncOptions:
    - CreateNamespace=true
    - RespectIgnoreDifferences=true
sync:
  - kind: Job
    operation: delete  # Delete before sync

Option 2: Use CronJob instead
For recurring jobs, use CronJob which handles cleanup automatically:

yaml
apiVersion: batch/v1
kind: CronJob
metadata:
  name: myapp-cronjob
spec:
  schedule: "0 2 * * *"  # Daily at 2 AM
  jobTemplate:
    spec:
      completions: 3
      template:
        # ... pod template
7Configure Job deletion policy

When deleting a Job, control what happens to pods:

bash
# Delete Job and cascading pods immediately
kubectl delete job <job-name>

# Delete Job but keep pods running (use for cleanup inspection)
kubectl delete job <job-name> --cascade=orphan

# Force delete (use with caution)
kubectl delete job <job-name> --grace-period=0 --force

For Kubernetes manifests, specify deletion policy:

yaml
metadata:
  ownerReferences:
  - apiVersion: batch/v1
    kind: Job
    name: myapp-job
    uid: <uid>
    blockOwnerDeletion: true
    controller: true
8Use CronJob for recurring workloads

If your Job runs regularly, use CronJob which automatically handles cleanup and recreation:

yaml
apiVersion: batch/v1
kind: CronJob
metadata:
  name: myapp-cronjob
spec:
  schedule: "0 2 * * *"  # Cron schedule
  successfulJobsHistoryLimit: 3  # Keep last 3 successful Jobs
  failedJobsHistoryLimit: 1       # Keep last 1 failed Job
  jobTemplate:
    spec:
      completions: 3
      parallelism: 1
      template:
        spec:
          containers:
          - name: job
            image: myapp:latest
          restartPolicy: OnFailure

CronJob automatically creates new Job instances, bypassing immutability issues.

Advanced notes

Job immutability is by design to prevent race conditions and state inconsistencies. The Job controller relies on spec fields not changing during execution. For dynamic workloads requiring variable parallelism or completions, prefer CronJob for recurring tasks or custom controllers for complex requirements. When debugging Job immutability issues in GitOps workflows, check if ArgoCD/Flux is trying to update the Job in-place rather than replacing it. The --cascade flag controls whether pods are deleted when Job is deleted—orphaned pods must be manually cleaned up. For batch processing pipelines, consider using Argo Workflows or Tekton which handle pod templates more flexibly than native Jobs.

Frequently asked questions

What does "Job completions is immutable" mean?
completions field (and several other Job fields) after a Job is created. The completions field defines how many pod completions are needed for the Job to succeed. Once set at creation time, this field is locked to prevent mid-execution changes that could cause unpredictable behavior.
How do I fix "Job completions is immutable"?
The "Job completions is immutable" error occurs when you attempt to modify the .spec.completions field on an already-created Kubernetes Job. This field specifies required pod completions and cannot be changed after creation.
What causes "Job completions is immutable"?
Attempting to change .spec.completions on an existing Job Also attempting to change .spec.template (pod template specification) Using kubectl apply or kubectl patch to modify immutable Job fields
Is the "Job completions is immutable" error serious?
It is a medium-severity error — more of a blocker than a risk. The steps above resolve it.

Official resources & further reading