How to fix Job completions is immutable in Kubernetes

KubernetesINTERMEDIATEMEDIUM

The "Job completions is immutable" error occurs when you attempt to modify the `.spec.completions` field on an already-created Kubernetes Job. This field specifies required pod completions and cannot be changed after creation.

What this error means

Kubernetes prevents modifying the `.spec.completions` field (and several other Job fields) after a Job is created. The `completions` field defines how many pod completions are needed for the Job to succeed. Once set at creation time, this field is locked to prevent mid-execution changes that could cause unpredictable behavior. This is an intentional design decision: if you could change the completions mid-run, pods that already completed would contradict new requirements, and the Job controller would face ambiguous completion state. Therefore, Kubernetes enforces immutability on critical Job fields.

How to fix "Job completions is immutable"

1Understand which Job fields are immutable

Immutable Job fields (cannot be changed after creation):
- .spec.completions - number of required completions
- .spec.template - pod template specification
- .spec.selector - label selector for pod tracking
- .spec.parallelism - may be restricted based on completion mode

Mutable fields:
- .spec.parallelism - number of parallel pods (in certain conditions)
- .spec.activeDeadlineSeconds - deadline to stop running Job

If you need to change immutable fields, delete and recreate.

2Delete the existing Job

Remove the Job to allow recreation:

bash

kubectl delete job <job-name>

Wait for deletion to complete:

bash

kubectl delete job <job-name> --wait=true

Verify it's gone:

bash

kubectl get job <job-name>
# Should return: No resources found

3Apply the updated Job manifest

Once deleted, apply the new Job definition:

bash

kubectl apply -f updated-job.yaml

Verify creation:

bash

kubectl get job <job-name>
kubectl describe job <job-name>

Watch completion status:

bash

kubectl get jobs --watch

4Plan Job changes before creation

To avoid this issue, finalize Job specifications before initial creation:

yaml

apiVersion: batch/v1
kind: Job
metadata:
  name: myapp-job
spec:
  completions: 3  # Set this value correctly before creating
  parallelism: 1
  template:
    metadata:
      labels:
        app: myapp-job
    spec:
      containers:
      - name: job-container
        image: myapp:latest
      restartPolicy: Never

Test in development first before applying to production.

5Use delete-then-apply in CI/CD pipelines

For automated deployments, script the delete-and-recreate pattern:

bash

#!/bin/bash
JOB_NAME="myapp-job"
NAMESPACE="default"

# Delete if exists
kubectl delete job $JOB_NAME -n $NAMESPACE --ignore-not-found=true

# Wait for deletion
sleep 5

# Apply new Job
kubectl apply -f job.yaml -n $NAMESPACE

This ensures clean state before Job creation.

6For GitOps tools, handle Job updates carefully

In ArgoCD or Flux configurations, Jobs must be handled specially since they're immutable:

Option 1: Manual delete before update
Add a pre-sync hook in ArgoCD:

yaml

apiVersion: argoproj.io/v1alpha1
kind: Application
metadata:
  name: myapp
spec:
  # ...
  syncPolicy:
    syncOptions:
    - CreateNamespace=true
    - RespectIgnoreDifferences=true
sync:
  - kind: Job
    operation: delete  # Delete before sync

Option 2: Use CronJob instead
For recurring jobs, use CronJob which handles cleanup automatically:

yaml

apiVersion: batch/v1
kind: CronJob
metadata:
  name: myapp-cronjob
spec:
  schedule: "0 2 * * *"  # Daily at 2 AM
  jobTemplate:
    spec:
      completions: 3
      template:
        # ... pod template

7Configure Job deletion policy

When deleting a Job, control what happens to pods:

bash

# Delete Job and cascading pods immediately
kubectl delete job <job-name>

# Delete Job but keep pods running (use for cleanup inspection)
kubectl delete job <job-name> --cascade=orphan

# Force delete (use with caution)
kubectl delete job <job-name> --grace-period=0 --force

For Kubernetes manifests, specify deletion policy:

yaml

metadata:
  ownerReferences:
  - apiVersion: batch/v1
    kind: Job
    name: myapp-job
    uid: <uid>
    blockOwnerDeletion: true
    controller: true

8Use CronJob for recurring workloads

If your Job runs regularly, use CronJob which automatically handles cleanup and recreation:

yaml

apiVersion: batch/v1
kind: CronJob
metadata:
  name: myapp-cronjob
spec:
  schedule: "0 2 * * *"  # Cron schedule
  successfulJobsHistoryLimit: 3  # Keep last 3 successful Jobs
  failedJobsHistoryLimit: 1       # Keep last 1 failed Job
  jobTemplate:
    spec:
      completions: 3
      parallelism: 1
      template:
        spec:
          containers:
          - name: job
            image: myapp:latest
          restartPolicy: OnFailure

CronJob automatically creates new Job instances, bypassing immutability issues.

How to fix Job completions is immutable in Kubernetes

What this error means

Typical symptoms

Common causes

How to fix "Job completions is immutable"

Advanced notes

Related errors

Official resources & further reading