Docker Architecture & Engine Internals
Many developers view Docker as a single monolithic executable. Under the hood, Docker Engine is a decoupled, multi-tiered distributed system comprising the Docker CLI, the dockerd management daemon, the containerd runtime supervisor, and the low-level runc OCI container executor.
1. The Multi-Tiered Runtime Architecture
When Docker originally launched in 2013, the dockerd daemon executed everything. Today, following the Open Container Initiative (OCI) standardization, the architecture is cleanly modularized:
| Component | Layer | Responsibility | Process Life Cycle |
|---|---|---|---|
Docker CLI (docker) | User Interface | Translates user terminal commands into HTTP/REST API calls. Stateless client. | Runs per command and terminates. |
Docker Daemon (dockerd) | Orchestration & API | Exposes the REST API, manages storage volumes, bridge networks, image builds, and authentication. | Persistent systemd background service. |
| containerd | Supervisor Runtime | Manages image layer distribution (pull/push), snapshot storage, and supervises container execution. | Persistent daemon (also used directly by Kubernetes via CRI). |
| containerd-shim | Process Decoupler | Keeps container STDIN/STDOUT FIFO file descriptors open and reports exit codes without daemon dependency. | One tiny process per active running container. |
| runc | OCI Low-Level Runtime | Directly talks to the Linux kernel to create namespaces, mount root filesystems, configure cgroups, and call execve. | Ephemeral: spawns the container process and exits immediately. |
2. Why containerd-shim is a Game-Changer
In legacy Docker versions, if the dockerd daemon crashed or required a software update, all running containers on that host server instantly died.
Modern Docker Engine solves this via containerd-shim. When you launch a container:
containerdrequestsruncto construct the container process.containerd-shimattaches to the container's standard I/O pipes.runchands over ownership of the process tocontainerd-shimand exits.
containerd-shim rather thandockerd, you can enable "live-restore": true in /etc/docker/daemon.json. This allows you to upgrade, restart, or crash the Docker daemon without dropping a single active HTTP connection or restarting containers!3. The Docker Socket: /var/run/docker.sock
Communication between the Docker CLI and the daemon occurs over a local UNIX domain socket located at/var/run/docker.sock. The CLI sends standard HTTP 1.1 JSON requests over this socket:
# Query the Docker REST API directly using curl over the UNIX socket:
curl --unix-socket /var/run/docker.sock http://localhost/version
# List running containers directly via raw HTTP REST API:
curl --unix-socket /var/run/docker.sock http://localhost/containers/json/var/run/docker.sock inside an untrusted container is equivalent to granting unrestricted root access to the entire host machine. A malicious process inside the container can instruct the daemon to mount the host root directory (/) and compromise the server.4. Storage Driver: The Overlay2 Filesystem
Docker constructs a container's unified root filesystem using the Overlay2 storage driver. Overlay2 merges multiple directory layers into a single combined view using four key directories inside/var/lib/docker/overlay2/:
Image Layers
Read-only base layers containing operating system binaries and dependencies.
Container Layer
Writable layer holding all file modifications, creations, and deletions.
Staging Workdir
Internal scratchpad used by Linux kernel for atomic file operations.
Mount Point
The unified view exposed to the container process as root (/).
Next Up: Essential Docker CLI Commands
Now that you understand the underlying engine architecture, learn how to manage container lifecycles using essential CLI operations.
Chapter 04: Essential Docker Commands →
