This error occurs when a CronJob has missed more than 100 scheduled job runs, typically after cluster downtime or when jobs run longer than the schedule interval. The CronJob controller stops scheduling to prevent cascade failures.
This error occurs when the Kubernetes CronJob controller detects that more than 100 scheduled job runs have been missed without execution. The controller enters a safety state and stops attempting to schedule new jobs to prevent cascading failures. A job is counted as missed if it failed to start at its scheduled time—either because the cluster was down, the controller was unavailable, or the concurrencyPolicy (Forbid) prevented execution due to a long-running previous job. The controller uses a calculation window: if startingDeadlineSeconds is not set, it counts from the last successful schedule until now; if set, it counts only within that deadline window. This is a protective mechanism to prevent resource exhaustion. Without this limit, a cluster recovering from a multi-hour outage could attempt to create hundreds of overdue jobs simultaneously, overwhelming the scheduler and API server.
Confirm the CronJob is in a missed-deadline state:
# Check the CronJob status
kubectl describe cronjob my-cronjob -n my-namespace
# Look for 'Active=False' and the 'TooManyMissedStartTimes' reason
kubectl get cronjobs my-cronjob -n my-namespace -o jsonpath='{.status.conditions}' | jq .Note the timestamp when missed schedules first occurred—this helps identify if cluster downtime was the trigger.
Set startingDeadlineSeconds to limit the missed schedule counting window:
apiVersion: batch/v1
kind: CronJob
metadata:
name: my-cronjob
spec:
schedule: "0 * * * *" # Every hour
startingDeadlineSeconds: 3600 # Count missed schedules only in last hour
jobTemplate:
spec:
template:
spec:
containers:
- name: job-container
image: myimage:latest
restartPolicy: OnFailureFor frequent jobs (every 5 minutes), use a proportional deadline:
schedule: "*/5 * * * *"
startingDeadlineSeconds: 600 # 10 minutes—enough for 2 missed schedulesIf your jobs are long-running and using Forbid, consider different policies:
- Allow (default): Permits concurrent job runs
- Replace: Terminates the old job and starts a new one
- Forbid with longer deadlines: Keep Forbid but ensure jobs complete within the schedule interval
concurrencyPolicy: Replace
startingDeadlineSeconds: 7200
jobTemplate:
spec:
activeDeadlineSeconds: 1800 # Job must complete within 30 minutesVerify that the cluster controller and nodes have synchronized time:
# Check for time drift by examining node status timestamps
kubectl get nodes -o json | jq '.items[] | {name: .metadata.name, conditions: .status.conditions[].lastHeartbeatTime}'
# Check node health
kubectl get nodes -o custom-columns=NAME:.metadata.name,READY:.status.conditions[?(@.type=="Ready")].statusIf clock skew is found, restart NTP/chronyd on affected nodes or force node restart in managed clusters.
If the above steps don't resume scheduling, delete and recreate the CronJob:
# Save the current CronJob definition
kubectl get cronjob my-cronjob -n my-namespace -o yaml > cronjob-backup.yaml
# Delete the CronJob
kubectl delete cronjob my-cronjob -n my-namespace --cascade=orphan
# Recreate with updated configuration
kubectl apply -f cronjob-backup.yaml
# Verify it's now active
kubectl describe cronjob my-cronjob -n my-namespace | grep -A5 'Conditions\|Last Schedule'After applying fixes, ensure the CronJob resumes scheduling:
# Verify CronJob is active again
kubectl get cronjobs my-cronjob -n my-namespace -o jsonpath='{.status.conditions}'
# Check that Jobs are being created on schedule
kubectl get jobs -n my-namespace | grep my-cronjobSet up Prometheus alerts to catch this early:
- alert: CronJobTooManyMissedSchedules
expr: kube_cronjob_status_active == 0
for: 5m
annotations:
summary: "CronJob {{ $labels.cronjob }} has no active schedule"The Kubernetes CronJob controller checks its schedule calculation every 10 seconds. If startingDeadlineSeconds is set to less than 10 seconds, the CronJob may not be scheduled reliably.
The 100-missed-schedule threshold is hardcoded in the controller and is a safety limit to prevent resource exhaustion during cluster recovery.
When using concurrencyPolicy: Forbid, long-running jobs are a common cause of cascade failures—each schedule is marked as missed if a previous job hasn't completed. Use activeDeadlineSeconds within the Job spec to enforce maximum job duration.
For critical jobs, consider combining Allow with external deduplication logic in your job code rather than relying on the CronJob controller to enforce concurrency.
Clock skew between nodes (especially in multi-zone clusters) is harder to detect and can silently cause misaligned schedule calculations; always enable NTP and monitor time synchronization.
Failed to connect to server: connection refused (HTTP/2)
How to fix "HTTP/2 connection refused" error in Kubernetes
missing request for cpu in container
How to fix "missing request for cpu in container" in Kubernetes HPA
error: invalid configuration
How to fix "error: invalid configuration" in Kubernetes
etcdserver: cluster ID mismatch
How to fix "etcdserver: cluster ID mismatch" in Kubernetes
running with swap on is not supported
How to fix "running with swap on is not supported" in kubeadm