The Docker Architecture Stack
Docker is a developer UX layer over a Linux container runtime stack where containers are ultimately just isolated Linux processes started by runc, managed by containerd, and orchestrated by dockerd. Now that you understand the primitives, let’s trace what happens when you run docker run redis on your machine.
The Layered Architecture
┌─────────────────────────────────────────┐
│ docker CLI / compose │ ← What you interact with
├─────────────────────────────────────────┤
│ dockerd (Docker Engine/Daemon) │ ← The "brain" - API server
├─────────────────────────────────────────┤
│ containerd │ ← Container lifecycle manager
├─────────────────────────────────────────┤
│ containerd-shim │ ← Per-container babysitter
├─────────────────────────────────────────┤
│ runc (OCI runtime) │ ← Actually creates container
├─────────────────────────────────────────┤
│ container (Linux process) │ ← Your app running in isolation
└─────────────────────────────────────────┘
Let’s examine each layer.
Layer 1: docker CLI and compose
This is what you interact with daily. The docker command is just an HTTP client that sends REST API calls to dockerd.
# What you type:
docker run -d -p 6379:6379 redis
# What actually happens:
# 1. CLI parses your command
# 2. CLI sends HTTP POST to dockerd's API
# 3. dockerd does all the work
# 4. CLI displays the result
docker-compose (or docker compose in newer versions) does the same thing, but orchestrates multiple containers from a YAML file.
Layer 2: dockerd (Docker Daemon)
The long-running background service. This is “Docker” in the traditional sense. It handles:
- Image management: Pulling from registries, building from Dockerfiles
- Networking setup: Creating bridge networks, port mapping
- Volume management: Bind mounts, named volumes
- REST API: The endpoint that docker CLI talks to
- Orchestration: Docker Swarm (if you use it)
- Security scanning: Docker Scout for vulnerability detection
Importantly, dockerd doesn’t actually run containers itself. It delegates downward to containerd.
Layer 3: containerd
Here’s where Docker’s architecture gets interesting.
containerd is a separate daemon that manages container lifecycle: create, start, stop, delete. Docker extracted it from their codebase in 2017 and donated it to the Cloud Native Computing Foundation (CNCF).
Why the separation?
This modularity means the container runtime is decoupled from Docker’s UX layer. Kubernetes can use containerd directly without needing dockerd. Cloud providers can use containerd without licensing Docker. The container ecosystem doesn’t depend on one company.
containerd handles:
- Image pull and unpack
- Storage management (snapshots)
- Container execution (via OCI runtime)
- Networking namespace setup
- Task management
Layer 4: containerd-shim
The Unsung Hero — If containerd directly started containers, killing containerd would kill all containers. Solution: each container gets its own shim process. One shim process exists per running container. Its jobs:
- Keeps STDIN/STDOUT/STDERR open — so
docker logsanddocker attachwork even if containerd restarts - Reports exit codes back to containerd
- Enables daemon restarts without killing containers (the “live-restore” feature)
This is why your containers can survive a Docker Desktop update. The shim keeps them alive while the daemons restart.
Layer 5: runc (OCI Runtime)
The tool that actually creates the container. runc is the reference implementation of the OCI (Open Container Initiative) Runtime Specification.
runc does exactly these things:
- Reads a
config.jsonfile (OCI bundle specification) - Creates Linux namespaces for the process
- Sets up cgroups for resource limits
- Configures the root filesystem
- Calls
fork/execto spawn the container process - Exits immediately — runc is not a daemon
That last point is crucial. runc’s job is to birth the container and then disappear. It’s a CLI tool, not a service.
Layer 6: The Container Itself
At the end of all these layers, a container is just a regular Linux process with:
- Its own PID namespace (thinks it’s PID 1)
- Its own network namespace (has its own eth0, localhost)
- Its own filesystem view (via overlay mounts)
- Resource limits enforced by cgroups
That’s it. No virtual machine. No hypervisor. Just a process with a restricted view of the system.
The Complete Flow
macOS (including Apple Silicon / M1) as well as Windows, dockerd and containerd do exist and do run, but not on macOS/Windows itself. They run inside Docker Desktop’s Linux virtual machine. Containers rely on Linux kernel features, because macOS has a Darwin kernel and no namespaces, cgroups, or seccomp capabilities.
You type: docker run -d redis
┌─────────────────────────────────────────────────────────────────┐
│ docker CLI │
│ "Parse command, send to dockerd" │
│ │ │
│ │ HTTP POST /containers/create │
│ ▼ │
├─────────────────────────────────────────────────────────────────┤
│ dockerd │
│ "Check if redis image exists, pull if needed" │
│ "Set up networking, volumes" │
│ "Delegate container creation to containerd" │
│ │ │
│ │ gRPC call │
│ ▼ │
├─────────────────────────────────────────────────────────────────┤
│ containerd │
│ "Unpack image layers into overlay filesystem" │
│ "Create OCI bundle (config.json + rootfs)" │
│ "Spawn containerd-shim" │
│ │ │
│ ▼ │
├─────────────────────────────────────────────────────────────────┤
│ containerd-shim │
│ "I'll manage this container's lifecycle" │
│ "Call runc to start it" │
│ │ │
│ ▼ │
├─────────────────────────────────────────────────────────────────┤
│ runc │
│ "Read config.json" │
│ "Create namespaces (pid, net, mnt, etc.)" │
│ "Set up cgroups" │
│ "fork/exec redis-server" │
│ "Exit — my job is done" │
│ │ │
│ ▼ │
├─────────────────────────────────────────────────────────────────┤
│ redis-server (container process) │
│ "I'm PID 1 in my own little world" │
│ "I have my own filesystem, network, everything" │
└─────────────────────────────────────────────────────────────────┘
The Clean Mental Model (Use This)
Think in terms of verbs:
| Component | Verb |
|---|---|
| Docker CLI | requests |
| dockerd | orchestrates |
| containerd | manages |
| runc | executes |
| Linux kernel | enforces |
Key Insights
- Docker CLI is just an HTTP client - It’s not magic, just REST API calls
- dockerd is the orchestrator - It coordinates but doesn’t run containers
- containerd is the actual runtime manager - This is what Kubernetes uses
- containerd-shim enables daemon restarts - Containers survive Docker updates
- runc is ephemeral - It creates the container and exits
- The container is just a process - Everything else is abstraction
Comments