This error occurs when a Docker container receives the SIGKILL signal (signal 9), causing immediate termination without graceful shutdown. Common causes include OOM killer, docker stop timeouts, manual kill commands, or orchestration systems forcefully terminating containers.
The "Container was killed with SIGKILL" message indicates that your Docker container received the SIGKILL signal (signal number 9), which causes immediate and unconditional termination. Unlike SIGTERM, SIGKILL cannot be caught, blocked, or ignored by the process - the kernel forcefully terminates it. When a container exits due to SIGKILL, it will have exit code 137 (calculated as 128 + 9, where 9 is the signal number). This exit code is the key indicator that distinguishes a SIGKILL termination from other types of container failures. SIGKILL is typically a last resort signal used when: 1. A process doesn't respond to SIGTERM within a grace period 2. The system's OOM (Out of Memory) killer needs to free memory urgently 3. An administrator or orchestration system needs to force-stop a container 4. Docker's stop command times out waiting for graceful shutdown The abrupt nature of SIGKILL means the application has no opportunity to perform cleanup operations like closing database connections, flushing buffers, or saving state.
First, determine why your container received SIGKILL:
# Check exit code (137 = SIGKILL)
docker inspect --format='{{.State.ExitCode}}' <container_name>
# Check if OOM killed
docker inspect --format='{{.State.OOMKilled}}' <container_name>
# View container state details
docker inspect --format='{{json .State}}' <container_name> | jq
# Check system logs for OOM events
dmesg | grep -i "killed process"
dmesg | grep -i "out of memory"If OOMKilled is true, see the memory-related solutions. If false, the container was killed for another reason (timeout, manual kill, etc.).
Docker sends SIGTERM first, then SIGKILL after timeout. Make sure your application handles SIGTERM:
Node.js example:
process.on('SIGTERM', () => {
console.log('Received SIGTERM, shutting down gracefully...');
server.close(() => {
console.log('Server closed');
process.exit(0);
});
// Force exit after 10 seconds if graceful shutdown fails
setTimeout(() => {
console.error('Graceful shutdown timed out, forcing exit');
process.exit(1);
}, 10000);
});Python example:
import signal
import sys
def handle_sigterm(signum, frame):
print("Received SIGTERM, shutting down...")
# Cleanup code here
sys.exit(0)
signal.signal(signal.SIGTERM, handle_sigterm)Go example:
sigChan := make(chan os.Signal, 1)
signal.Notify(sigChan, syscall.SIGTERM)
go func() {
<-sigChan
log.Println("Received SIGTERM, shutting down...")
// Cleanup and shutdown
os.Exit(0)
}()Shell form prevents your application from receiving signals directly:
Wrong (shell form):
# Runs as: /bin/sh -c "node server.js"
# PID 1 is sh, not node - signals don't reach your app
CMD node server.js
ENTRYPOINT python app.pyCorrect (exec form):
# Runs directly, your app is PID 1
CMD ["node", "server.js"]
ENTRYPOINT ["python", "app.py"]If you must use a shell script as entrypoint, use exec:
#!/bin/bash
# entrypoint.sh
# Setup code here...
# Use exec to replace shell with your app
exec node server.jsThis ensures your application becomes PID 1 and receives signals directly.
If your application needs more time to shut down gracefully, increase the timeout:
Docker run:
# Give 30 seconds instead of default 10
docker stop --time=30 <container_name>
# Or wait indefinitely for graceful shutdown
docker stop --time=-1 <container_name>Docker Compose:
services:
myapp:
image: myimage:latest
stop_grace_period: 30sDockerfile (set default):
# Set default stop signal and timeout
STOPSIGNAL SIGTERM
# Note: STOPSIGNAL only changes the signal, not the timeoutKubernetes:
spec:
terminationGracePeriodSeconds: 30If your application can't be modified to handle signals, use a lightweight init system:
Using tini (recommended):
# Install tini
RUN apt-get update && apt-get install -y tini
# Use tini as entrypoint
ENTRYPOINT ["/usr/bin/tini", "--"]
CMD ["node", "server.js"]Or with Alpine:
FROM alpine:latest
RUN apk add --no-cache tini
ENTRYPOINT ["/sbin/tini", "--"]
CMD ["./myapp"]Using dumb-init:
RUN apt-get update && apt-get install -y dumb-init
ENTRYPOINT ["/usr/bin/dumb-init", "--"]
CMD ["python", "app.py"]Using Docker's built-in init:
docker run --init myimageOr in Compose:
services:
myapp:
image: myimage
init: trueThese init systems properly forward signals to child processes and reap zombie processes.
If the container is being killed by the OOM killer:
Increase memory limits:
docker run --memory=2g --memory-swap=2g myimageMonitor memory usage:
docker stats <container_name>Configure runtime memory limits:
For Node.js:
docker run --memory=1g -e NODE_OPTIONS="--max-old-space-size=768" myappFor Java:
docker run --memory=2g -e JAVA_OPTS="-XX:MaxRAMPercentage=75.0" java-appCheck for memory leaks in your application if memory grows unbounded over time.
See the related "Docker OOM Killed" article for comprehensive memory troubleshooting.
Set up monitoring to detect and prevent SIGKILL issues:
Monitor Docker events:
# Watch for container kill events in real-time
docker events --filter 'event=kill'
docker events --filter 'event=oom'Check container exit reasons:
# List recently exited containers with exit codes
docker ps -a --filter "status=exited" --format "{{.Names}}: {{.Status}}"Log when signals are received:
Add logging to your signal handlers so you can see in container logs whether SIGTERM was received and how long shutdown took.
Set up alerts:
In production, configure alerts for:
- Containers exiting with code 137
- OOM events
- Containers that restart frequently
Understanding exit code 137: Exit code 137 specifically indicates SIGKILL (128 + signal number). Common signal-related exit codes:
- 137 = SIGKILL (128 + 9) - Forced termination
- 143 = SIGTERM (128 + 15) - Graceful termination request
- 130 = SIGINT (128 + 2) - Interrupt (Ctrl+C)
- 139 = SIGSEGV (128 + 11) - Segmentation fault
PID 1 special behavior: In Linux, PID 1 has special signal handling. It only receives signals for which it has explicitly installed handlers. This is why shell form CMD doesn't forward SIGTERM - the shell (PID 1) has no handler, and child processes don't receive it.
Docker stop sequence:
1. Docker sends SIGTERM (or configured STOPSIGNAL) to PID 1
2. Docker waits for stop timeout (default 10s)
3. If container still running, Docker sends SIGKILL
Kubernetes termination sequence:
1. Pod marked as "Terminating"
2. PreStop hook executed (if defined)
3. SIGTERM sent to containers
4. terminationGracePeriodSeconds countdown begins
5. SIGKILL sent if pod still running
Debugging signal issues:
# See what process is PID 1 in container
docker exec <container> ps aux
# Check if your app is actually PID 1
docker exec <container> cat /proc/1/cmdlineCustom stop signals: Some applications expect different signals. Configure in Dockerfile:
STOPSIGNAL SIGQUIT # For nginx graceful shutdownOr at runtime:
docker run --stop-signal=SIGQUIT nginxDocker Compose v2 behavior: Docker Compose v2 uses a different stop behavior than v1. Ensure your stop_grace_period is set explicitly if migrating.
unable to configure the Docker daemon with file /etc/docker/daemon.json
How to fix 'unable to configure the Docker daemon with file daemon.json' in Docker
docker: Error response from daemon: OCI runtime create failed: container_linux.go: starting container process caused: exec: "/docker-entrypoint.sh": stat /docker-entrypoint.sh: no such file or directory
How to fix 'exec: entrypoint.sh: no such file or directory' in Docker
image operating system "linux" cannot be used on this platform
How to fix 'image operating system linux cannot be used on this platform' in Docker
dockerfile parse error line 5: unknown instruction: RRUN
How to fix 'unknown instruction' Dockerfile parse error in Docker
manifest unknown: manifest unknown
How to fix 'manifest unknown' in Docker