docker-errors-runtime
Quick Reference
Exit Code Reference
| Exit Code |
Signal |
Meaning |
Common Cause |
| 0 |
— |
Success |
Container completed normally |
| 1 |
— |
Application error |
Uncaught exception, failed assertion, general error |
| 125 |
— |
Docker daemon error |
Container failed to start (invalid config, missing image) |
| 126 |
— |
Command not executable |
Permission denied on entrypoint/cmd binary |
| 127 |
— |
Command not found |
Binary missing in image, wrong PATH, typo in CMD |
| 137 |
SIGKILL (9) |
Killed |
OOM killer, docker kill, or docker stop timeout |
| 139 |
SIGSEGV (11) |
Segmentation fault |
Native library crash, memory corruption |
| 143 |
SIGTERM (15) |
Graceful termination |
docker stop (process handled SIGTERM) |
Critical Warnings
NEVER ignore exit code 137 — it ALWAYS indicates the container was forcefully killed. Check OOM events with docker inspect and dmesg before increasing memory limits blindly.
NEVER use --oom-kill-disable without setting a memory limit (-m) — the container can consume ALL host memory and crash the entire system.
NEVER assume a container that exits with code 0 is healthy — it may have completed a one-shot command instead of running as a long-lived service. ALWAYS verify the process runs in the foreground.
ALWAYS check docker logs before any other debugging step — 90% of runtime issues are explained in the application output.
ALWAYS use docker inspect --format='{{.State.ExitCode}}' to get the exact exit code — docker ps -a truncates status information.
Debugging Workflow
Step 1: Check Logs
# Last 100 lines
docker logs --tail 100 <container>
# Follow live output with timestamps
docker logs -f -t <container>
# Logs from last 5 minutes
docker logs --since 5m <container>
Step 2: Inspect Container State
# Exit code and error message
docker inspect --format='{{.State.ExitCode}}' <container>
docker inspect --format='{{.State.Error}}' <container>
# OOM killed?
docker inspect --format='{{.State.OOMKilled}}' <container>
# Full state as JSON
docker inspect --format='{{json .State}}' <container> | jq .
Step 3: Exec Into Running Container
# Interactive shell (if container is still running)
docker exec -it <container> sh
docker exec -it <container> bash
# Check filesystem, processes, network
docker exec <container> ps aux
docker exec <container> df -h
docker exec <container> cat /etc/resolv.conf
Step 4: Check System Events
# Events for specific container in last 10 minutes
docker events --since 10m --filter container=<container>
# OOM events specifically
docker events --filter event=oom --since 1h
# All die events
docker events --filter event=die --since 1h
Step 5: Resource Usage
# Live resource stats
docker stats <container>
# Single snapshot
docker stats --no-stream <container>
# System-wide disk usage
docker system df -v
Runtime Error Diagnostic Table
Container Exits Immediately (Exit Code 0 or 1)
| Symptom |
Cause |
Fix |
| Container exits with code 0 instantly |
Main process runs in background (daemonizes) |
ALWAYS run the process in foreground mode. For nginx: CMD ["nginx", "-g", "daemon off;"] |
| Container exits with code 0 instantly |
CMD is a shell command that completes |
Use a long-running process. For shell scripts: end with exec or tail -f /dev/null for debugging |
| Container exits with code 1 |
Application startup failure |
Check docker logs. Fix config, missing env vars, or dependency issues |
| Container exits with code 1 |
Missing environment variables |
ALWAYS pass required env vars: docker run -e DB_HOST=db -e DB_PORT=5432 |
OOM Killed (Exit Code 137)
| Symptom |
Cause |
Fix |
OOMKilled: true in inspect output |
Container exceeded memory limit |
Increase limit: docker run -m 1g. Profile actual usage with docker stats first |
Exit 137 but OOMKilled: false |
docker stop timeout exceeded (SIGKILL after grace period) |
Increase stop timeout: docker stop -t 30. Or fix application to handle SIGTERM faster |
Exit 137 but OOMKilled: false |
Manual docker kill |
Check who/what killed the container via docker events |
| Host OOM killer triggers |
No memory limit set, host runs out of RAM |
ALWAYS set memory limits in production: -m 512m |
Permission Denied
| Symptom |
Cause |
Fix |
Permission denied on volume files |
UID/GID mismatch between host and container |
Match UIDs: docker run -u $(id -u):$(id -g). Or chown in Dockerfile |
Permission denied executing entrypoint |
Script lacks execute permission |
Add in Dockerfile: RUN chmod +x /entrypoint.sh |
Permission denied binding to port < 1024 |
Non-root user cannot bind privileged ports |
Use port > 1024, or add --cap-add NET_BIND_SERVICE |
Operation not permitted on system call |
Missing Linux capability |
Add specific capability: --cap-add SYS_PTRACE for debugging. NEVER use --privileged |
Port Already in Use
| Symptom |
Cause |
Fix |
port is already allocated |
Another container using the same host port |
Find it: docker ps --format "{{.Names}}: {{.Ports}}". Stop or remap |
bind: address already in use |
Host process using the port |
Find process: lsof -i :PORT or ss -tlnp | grep PORT. Stop it or use different port |
| Port conflict after restart |
Old container not removed |
Use --rm flag, or docker rm -f <old-container> before starting |
Exec Format Error
| Symptom |
Cause |
Fix |
exec format error |
Architecture mismatch (e.g., ARM image on x86) |
Build for correct platform: docker buildx build --platform linux/amd64. Or pull correct image: docker pull --platform linux/amd64 nginx |
exec format error on shell script |
Missing shebang (#!/bin/sh) in entrypoint script |
ALWAYS add shebang as first line of entrypoint scripts |
exec user process caused: no such file or directory |
CRLF line endings in shell script |
Convert to LF: RUN sed -i 's/\r$//' /entrypoint.sh or use dos2unix. ALWAYS use LF in Dockerfiles and scripts |
exec user process caused: no such file or directory |
Dynamically linked binary in scratch/distroless image |
Build with CGO_ENABLED=0 for static linking, or use alpine base |
Read-Only Filesystem
| Symptom |
Cause |
Fix |
Read-only file system write error |
Container started with --read-only |
Add tmpfs for writable paths: --tmpfs /tmp --tmpfs /run. Or mount a volume for data directories |
| Application fails to write temp files |
Read-only root FS without tmpfs |
Map writable paths: --read-only --tmpfs /tmp:size=64m --mount type=volume,src=data,dst=/app/data |
| Log file write failure |
Read-only FS, app writes to file instead of stdout |
Redirect logs to stdout, or mount a volume for log directory |
PID Limit and Resource Exhaustion
| Symptom |
Cause |
Fix |
cannot allocate memory inside container |
Memory limit reached |
Increase -m limit or optimize application memory usage |
fork: Resource temporarily unavailable |
PID limit exceeded |
Increase --pids-limit. Default is unlimited; set to 200-500 for most apps |
no space left on device |
Container writable layer full, or host disk full |
Check docker system df. Prune unused resources: docker system prune. Write data to volumes, not container layer |
| Container extremely slow |
CPU throttling |
Check docker stats for CPU%. Increase --cpus limit |
docker inspect for Debugging
Essential Inspect Commands
# Full state overview
docker inspect --format='{{json .State}}' <container> | jq .
# Why did it stop?
docker inspect --format='ExitCode={{.State.ExitCode}} OOM={{.State.OOMKilled}} Error={{.State.Error}}' <container>
# What command is it running?
docker inspect --format='Entrypoint={{.Config.Entrypoint}} Cmd={{.Config.Cmd}}' <container>
# Environment variables
docker inspect --format='{{range .Config.Env}}{{println .}}{{end}}' <container>
# Mount points
docker inspect --format='{{range .Mounts}}{{.Type}}: {{.Source}} -> {{.Destination}} ({{if .RW}}rw{{else}}ro{{end}}){{println}}{{end}}' <container>
# Network settings
docker inspect --format='{{range $net, $config := .NetworkSettings.Networks}}{{$net}}: {{$config.IPAddress}}{{println}}{{end}}' <container>
# Resource limits
docker inspect --format='Memory={{.HostConfig.Memory}} CPUs={{.HostConfig.NanoCpus}} PidsLimit={{.HostConfig.PidsLimit}}' <container>
# Health check status
docker inspect --format='{{.State.Health.Status}}' <container>
docker inspect --format='{{json .State.Health}}' <container> | jq .
# Restart count
docker inspect --format='RestartCount={{.RestartCount}}' <container>
Decision Trees
Container Won't Start
Container won't start
├─ Exit 125 → Docker daemon error
│ ├─ "invalid reference format" → Fix image name/tag
│ ├─ "no such image" → Pull image first: docker pull <image>
│ └─ "invalid mount config" → Fix volume/mount syntax
├─ Exit 126 → Command not executable
│ ├─ Check file permissions → chmod +x
│ └─ Check binary format → file <binary>
├─ Exit 127 → Command not found
│ ├─ Typo in CMD/ENTRYPOINT → Fix spelling
│ ├─ Binary not in PATH → Use absolute path
│ └─ Binary not installed → Add to Dockerfile
└─ Exit 0/1 instantly → See "Container Exits Immediately" table
Container Crashes After Running
Container was running, then died
├─ Exit 137 → Killed
│ ├─ OOMKilled=true → Memory limit too low (see OOM section)
│ ├─ OOMKilled=false, after docker stop → Stop timeout too short
│ └─ OOMKilled=false, unexpected → Check docker events + dmesg
├─ Exit 139 → Segfault
│ ├─ Native library issue → Check library compatibility
│ └─ Memory corruption → Debug with --cap-add SYS_PTRACE
├─ Exit 143 → Graceful SIGTERM
│ └─ Expected from docker stop → Normal shutdown
└─ Exit 1 → Application error
└─ Check docker logs → Fix application bug
Reference Links
Official Sources
