4. Docker Images

Images are the unit of distribution in Docker: everything a container needs to run is packaged into an image, shipped through a registry, and unpacked on the target machine. This chapter looks at what an image actually is (layers, copy-on-write, tags, digests, base images), how to manage images on a host (tagging, removing, inspecting, moving them without a registry), and how registries store and serve them. Understanding images well is the foundation for writing good Dockerfiles in chapter 5 and for securing your supply chain in chapter 10.

<a id="41-image-fundamentals"></a>

4.1 Image Fundamentals

This section explains how an image is structured and identified, and how it gets onto your machine.

Image Layers

Why layers exist. Imagine shipping ten applications that all run on the same Debian base. If each image were one big monolithic archive, every host would download and store ten copies of Debian. Layers solve this: an image is split into a stack of independent filesystem archives, so the Debian part is stored and transferred only once and reused by every image built on it. Layers also make builds incremental, because a step that has not changed can reuse its previous layer.

Analogy. Think of an image as a lasagna. Each layer is added on top of the previous one and only contains what that step added (pasta, sauce, cheese). Two lasagnas with the same bottom three layers can literally share them, as long as nobody modifies a shared layer, which is guaranteed because layers are read-only.

What a layer is. A layer is a tar archive containing the differences a single build step made to the filesystem: files added, files modified, and whiteout markers for files deleted (see Union File Systems). In a Dockerfile, only instructions that change files (RUN, COPY, ADD) create layers; instructions such as ENV, CMD, EXPOSE, or LABEL only change the image config and produce a zero-byte history entry.

 Dockerfile                          Resulting image
 ---------------------------         ------------------------------------------
 FROM python:3.13-slim        --->   layers 1..4: inherited from python:3.13-slim
 WORKDIR /app                 --->   layer 5: empty /app directory
 COPY requirements.txt .      --->   layer 6: requirements.txt            (1 KB)
 RUN pip install -r ...       --->   layer 7: site-packages               (45 MB)
 COPY . .                     --->   layer 8: application source          (300 KB)
 ENV PORT=8000                --->   config only (no layer)
 CMD ["python", "app.py"]     --->   config only (no layer)

 Stack seen by a container (bottom = oldest):

   +-----------------------------+
   | 8  app source               |  changes on every commit
   | 7  pip packages             |  changes when requirements change
   | 6  requirements.txt         |
   | 5  /app                     |
   | 1-4 python + debian slim    |  shared with every python:3.13-slim image
   +-----------------------------+

Three identifiers per layer. The same layer is referred to in three ways, which explains why different commands show different hashes:

Identifier	Hash of	Where you see it
Layer digest	The compressed tar blob	Image manifest, registry, docker pull output (short form)
DiffID	The uncompressed tar	Image config (rootfs.diff_ids), docker image inspect (.RootFS.Layers)
ChainID	A layer plus every layer beneath it	Docker's local storage; identifies a stack, not a single diff

The ChainID matters because a layer is only meaningful on top of its parents: the same diff applied on a different base gives a different filesystem.

Practical consequences

• Image size is the sum of all layers. Deleting a file in a later layer hides it but does not remove it from the earlier layer, so the image does not shrink.
• Ordering matters: put rarely-changing steps (dependencies) before frequently-changing ones (source code) so that rebuilds and pulls reuse more layers. This is developed in Layer Ordering.
• There is a practical limit on stack depth (about 127 layers with overlay2), rarely reached by sensible Dockerfiles.

Example

docker pull python:3.13-slim

# DiffIDs: one entry per layer, base first
docker image inspect -f '{{range .RootFS.Layers}}{{println .}}{{end}}' python:3.13-slim
# sha256:2f2d7b1e...   <- debian base
# sha256:9a7c4e0b...
# sha256:e1f3a6d2...
# sha256:5c0d8b7f...   <- python itself

# Pull another image built on the same base and watch layers being reused
docker pull python:3.12-slim
# 2d429b9e73a6: Already exists        <- the debian layer, shared
# 1a3f5c7e9b20: Pull complete
# ...

# Per-step view: which instruction created which layer and how big it is
docker image history python:3.13-slim

Key takeaways

• An image is an ordered stack of read-only filesystem diffs plus a config.
• Only filesystem-changing instructions (RUN, COPY, ADD) create layers.
• Layers are content-addressed, so identical layers are stored and downloaded once.
• Deleted files still occupy space in the layer where they were added.
• Layer digest (compressed), DiffID (uncompressed), and ChainID (stack) are different hashes for related things.

🧪 Practice

1. (Beginner) Pull nginx:alpine and count its layers with docker image inspect. Compare that number with the rows in docker image history and explain why they differ.
2. (Intermediate) Pull node:22-slim and python:3.13-slim. Using the DiffIDs from docker image inspect, find out whether they share a base layer, and confirm your answer with the "Already exists" lines in the pull output.
3. Interview question: "Why does reordering two lines in a Dockerfile sometimes make CI builds and deployments several times faster?" Hint: think about which layers are invalidated on each commit and how many bytes the deployment target has to download.

Copy-on-Write

Why copy-on-write. Read-only layers are perfect for sharing, but a running container needs to write: log files, PID files, caches, temporary data. Copying the whole image for each container would be slow and wasteful. Copy-on-write (CoW) gives every container the illusion of a private, writable filesystem while physically sharing the image layers, and pays the copying cost only for files that actually change.

Analogy. A library lends the same reference book to everyone in the reading room, but nobody may write in it. If you want to annotate a page, you photocopy that page and write on the copy. Everyone else still reads the original, and only the pages you annotated cost extra paper.

How it works for containers

1. When a container is created, Docker adds a thin, empty writable layer (the container layer) on top of the image's read-only layers.
2. Reads go down the stack and return the first (topmost) version of the file found.
3. First write to an existing file triggers a copy-up: the whole file is copied from the image layer into the container layer, and the change is applied to the copy.
4. New files are written directly to the container layer.
5. Deletes create a whiteout in the container layer; the file stays in the image layer.

     container A               container B               container C
   +--------------+          +--------------+          +--------------+
   | writable (A) |          | writable (B) |          | writable (C) |
   | nginx.conf*  |          | (empty)      |          | /tmp/upload  |
   +------+-------+          +------+-------+          +------+-------+
          |                         |                         |
          +-------------------------+-------------------------+
                                    |
                 +------------------v-------------------+
                 |  image nginx:alpine (read-only)      |  stored once on disk,
                 |  layers 1..N, incl. nginx.conf       |  and cached once in RAM
                 +--------------------------------------+
   * nginx.conf was copied up into A's layer when A modified it

Consequences you will run into

Situation	Effect	What to do
Starting many containers from one image	Near-instant start, almost no extra disk	Nothing; this is the benefit
Modifying a large file from the image (e.g. a 2 GB database file)	The first write copies the entire file, even for one changed byte	Keep such data in a volume
Write-heavy workloads in the container layer	Extra overhead of the union filesystem	Use volumes or bind mounts, which bypass CoW
Removing the container	The writable layer and all its data are deleted	Persist important data in volumes
Same image layers in many containers	The kernel page cache is shared, so memory is saved too	Nothing; another benefit

Two sizes per container. docker ps -s reports SIZE, the data in the writable layer, and virtual, which also counts the shared read-only image layers. Only the first number is per-container cost.

Example: watching a copy-up happen

# Build an image containing one 200 MiB file (Dockerfile read from stdin, no build context)
docker build -t cow-demo - <<'EOF'
FROM alpine:3.20
RUN dd if=/dev/zero of=/big.bin bs=1M count=200
EOF

# Container 1 only reads the file: no copy-up
docker run --name reader cow-demo sh -c 'wc -c /big.bin'

# Container 2 appends a single byte: the whole file is copied up first
docker run --name writer cow-demo sh -c 'echo x >> /big.bin'

docker ps -a -s --filter name=reader --filter name=writer --format '{{.Names}}\t{{.Size}}'
# reader   0B (virtual 218MB)
# writer   210MB (virtual 428MB)      <- one byte written, 210 MB consumed

docker diff writer
# C /big.bin                          <- changed: lives in the writable layer now

docker rm reader writer

Key takeaways

• Every container gets its own thin writable layer on top of shared read-only image layers.
• A file is copied into the writable layer only when it is first modified (copy-up), and the whole file is copied.
• docker ps -s shows a container's own size and its virtual size including the image.
• Volumes bypass copy-on-write; use them for databases, uploads, and any write-heavy or persistent data.

🧪 Practice

1. (Beginner) Start three containers from nginx:alpine, modify a file in one of them with docker exec, and compare their sizes with docker ps -s. Explain each number.
2. (Intermediate) Repeat the copy-up example, but time the first and second echo x >> /big.bin executed with docker exec in the same running container. Explain why the first one is slower.
3. Interview question: "A team runs PostgreSQL in a container without a volume, and write latency spikes after each restart. What could be going on?" Hint: consider where the data files live, what happens on their first modification, and what happens to them when the container is recreated.

Image Tags

Why tags exist. Image IDs and digests are 64-character hashes, impossible to remember or type. Humans need readable names such as postgres:16 or myapp:2.3.1, and teams need a way to say "the current stable build". Tags are these human-friendly labels.

What a tag is. A tag is a mutable pointer from a name to an image, much like a Git branch name pointing to a commit. The full reference has this shape:

   [registry[:port]/][namespace/]repository[:tag][@digest]

   registry.example.com:5000/team/api:2.3.1
   ^^^^^^^^^^^^^^^^^^^^^^^^^ ^^^^ ^^^ ^^^^^
   registry (host:port)      ns   repo tag

   Defaults:  registry -> docker.io     namespace -> library (Docker Hub only)
              tag      -> latest

Rules. A tag can contain letters, digits, _, ., and -, cannot start with . or -, and is at most 128 characters long. Repository names must be lowercase.

latest is not "newest". latest is simply the tag used when you do not specify one. It is not updated automatically when a newer image is pushed under another tag, and in your own repositories it only points wherever the last docker push ...:latest left it. Relying on it makes deployments unpredictable.

How publishers use tags. One image usually carries several tags at once, from very specific to very general. A moving tag such as 16 is re-pointed each time a patch release is published:

   postgres:16.4-bookworm   --+
   postgres:16.4             --+
   postgres:16-bookworm      --+-->  sha256:4f1c...   (one image)
   postgres:16               --+
   postgres:latest           --+

   ... a month later, 16.5 is released:
   postgres:16, postgres:latest  -->  sha256:9b2e...  (moved)
   postgres:16.4                 -->  sha256:4f1c...  (unchanged, usually)

Common tagging conventions

Convention	Example	Mutable?	Good for
Semantic version	2.3.1	Should not be	Releases
Floating major/minor	2, 2.3	Yes, by design	Picking up patches automatically
Git commit SHA	git-3f9a1c2	No	Traceability from image to code
Variant suffix	2.3.1-alpine, -slim	Same as the version part	Choosing the base flavor
Environment or channel	staging, stable	Yes	Promotion workflows (use with care)
latest	latest	Yes	Local experiments only

Tagging strategies for CI are covered in Image Tagging Strategies.

Example

docker pull redis                       # same as docker.io/library/redis:latest
docker pull redis:7.4-alpine            # specific version and variant

# Several tags, one image ID
docker image ls redis
# REPOSITORY   TAG          IMAGE ID       CREATED       SIZE
# redis        7.4-alpine   8a5c3e7d2b1f   2 weeks ago   41.4MB
# redis        latest       1c9f0e4a7b62   2 weeks ago   117MB

# List remote tags without pulling (Docker Hub API)
curl -s "https://hub.docker.com/v2/repositories/library/redis/tags?page_size=10" \
  | jq -r '.results[].name'

Key takeaways

• A tag is a mutable, human-readable pointer to an image; it can be moved to another image at any time.
• Omitting the tag means latest, which is a convention, not a guarantee of recency.
• One image commonly has many tags (full version, minor, major, variant).
• Use specific, immutable-by-policy tags (version, commit SHA) for anything you deploy.

🧪 Practice

1. (Beginner) Write the fully expanded reference (registry, namespace, repository, tag) for ubuntu, bitnami/nginx:1.27, and localhost:5000/app.
2. (Beginner) Pull alpine:3, alpine:3.20, and alpine:latest. Use docker image ls to determine which of them currently point to the same image.
3. Interview question: "Your production cluster suddenly runs a different version of a dependency even though nobody changed the deployment manifest. How can tags explain this?" Hint: think about floating tags, image pull policies, and nodes pulling at different times.

Image Digests

Why digests exist. A tag tells you a name; it does not guarantee content. If myapp:1.0 is overwritten, two servers that pulled at different times can run different code under the same name. A digest is a cryptographic fingerprint of the content itself, so a reference by digest always means exactly the same bytes, and any tampering is detectable.

What a digest is. A digest is algorithm:hex, almost always sha256: followed by 64 hex characters, computed over a JSON document (manifest or index). Because the manifest lists the digests of the config and every layer, the digest of the manifest transitively pins the entire image, the way a Git commit hash pins the full source tree.

   index digest  sha256:aaa...   <- what "docker pull nginx:1.27" prints as "Digest:"
        |
        +-- linux/amd64 manifest  sha256:bbb...
        |       +-- config  sha256:ccc...
        |       +-- layers  sha256:ddd..., sha256:eee...
        +-- linux/arm64 manifest  sha256:fff...
                +-- ...

   Change one byte in any layer -> its digest changes -> the manifest changes
   -> the manifest digest changes -> the index digest changes.

IDs and digests compared

Value	What it hashes	Scope	Command
Repo digest	Manifest or index, as stored in the registry	Global, the same everywhere	docker image inspect -f '{{.RepoDigests}}'
Image ID	Config (classic store) or manifest/index (containerd store)	Local; may differ between engines	docker image ls, .Id
Platform manifest digest	One platform's manifest	Global	docker buildx imagetools inspect

Referencing by digest. Add @sha256:... to the name. If a tag is also present (nginx:1.27@sha256:...), the digest wins and the tag is only documentation, a common and readable way to pin base images in Dockerfiles. Automated tools such as Renovate or Dependabot can update pinned digests through pull requests, so pinning does not mean never patching.

Example

docker pull nginx:1.27-alpine
# ...
# Digest: sha256:2140dad2...                      <- index digest (shortened here)
# Status: Downloaded newer image for nginx:1.27-alpine

# Digest of a local image (empty for images that were never pushed or pulled)
docker image inspect -f '{{index .RepoDigests 0}}' nginx:1.27-alpine
# nginx@sha256:2140dad2...

# Remote digest without pulling: index digest and per-platform manifests
docker buildx imagetools inspect nginx:1.27-alpine
# Name:      docker.io/library/nginx:1.27-alpine
# MediaType: application/vnd.oci.image.index.v1+json
# Digest:    sha256:2140dad2...
# Manifests:
#   Name:      docker.io/library/nginx:1.27-alpine@sha256:7a1b...
#   Platform:  linux/amd64
#   ...

# Pull and run by digest: always the exact same content
REF=$(docker image inspect -f '{{index .RepoDigests 0}}' nginx:1.27-alpine)
docker run --rm "$REF" nginx -v

# Readable and reproducible: the tag documents intent, the digest is what is used
FROM node:22-alpine@sha256:<digest-from-imagetools-inspect>

Key takeaways

• A digest is a content hash; the same digest always means the same image.
• The manifest digest transitively covers the config and every layer.
• Multi-platform images have an index digest plus one manifest digest per platform.
• Pin by digest where reproducibility matters (base images, production deploys) and automate digest updates.

🧪 Practice

1. (Beginner) Pull alpine:3.20, note the digest, and then pull it again using only the digest. Confirm with docker image ls --digests that both references point to the same image.
2. (Intermediate) Use docker buildx imagetools inspect to list the per-platform manifest digests of alpine:3.20. Explain why docker image inspect on an amd64 machine and on an arm64 machine can show the same repo digest but different layers.
3. Interview question: "If tags are mutable, what stops someone from pushing a malicious image under a digest you already trust?" Hint: think about what the digest is computed from and what a client verifies after downloading.

Base Images

Why base images matter. Almost no one builds an image from an empty filesystem. Your image starts FROM some other image that provides a C library, a shell, a package manager, or a language runtime. That choice decides a large part of your final image's size, attack surface, available tools, and how often you will need to patch it. It is one of the most consequential lines in a Dockerfile.

Terminology. Strictly, a base image is one built FROM scratch (an empty filesystem), and the image named in your FROM line is your parent image. In everyday use, "base image" means the parent image, and this guide follows that usage.

Families of base images

Family	Examples	Approx. size	Shell / package manager	Notes
Empty	scratch	0 B	No / No	For static binaries (Go, Rust); you supply everything
Minimal distro	alpine:3.20	~8 MB	sh / apk	Uses musl libc instead of glibc; some binaries and wheels behave differently
Slim distro	debian:12-slim, ubuntu:24.04	~75-80 MB	bash / apt	glibc, broad compatibility, good default
Full distro	debian:12, buildpack-deps	120 MB to 1 GB	Yes / Yes	Many build tools included; best for build stages
Language runtime	python:3.13-slim, node:22-alpine, eclipse-temurin:21-jre	50-250 MB	Depends on variant	Runtime preinstalled and maintained upstream
Distroless / hardened	gcr.io/distroless/static, Chainguard images, Docker Hardened Images	2-50 MB	Usually no / No	Minimal attack surface; debugging is harder

Sizes are uncompressed, rough, and change between releases.

Selection criteria

• Compatibility: does your application need glibc, specific system libraries, or a shell at runtime?
• Size and attack surface: fewer packages mean fewer vulnerabilities to patch and faster pulls.
• Maintenance: is the image rebuilt regularly with security fixes? Official and verified images usually are (see Official and Verified Images).
• Debuggability: images without a shell cannot be entered with docker exec sh; plan for tools like docker debug.
• Build versus run: a large image with compilers for building and a tiny image for running is the multi-stage pattern in 5.4 Multi-Stage Builds.

Minimal choices are explored further in Choosing Minimal Base Images and Distroless and Scratch Images.

Example

# Compare a few bases side by side
for img in alpine:3.20 debian:12-slim ubuntu:24.04 python:3.13-slim python:3.13; do
  docker pull -q "$img" >/dev/null
done
docker image ls --format 'table {{.Repository}}:{{.Tag}}\t{{.Size}}' | grep -E 'alpine|debian|ubuntu|python'

# Which C library and shell does each one have?
docker run --rm alpine:3.20     sh -c 'ldd --version 2>&1 | head -n1'   # musl libc
docker run --rm debian:12-slim  sh -c 'ldd --version | head -n1'        # ldd (Debian GLIBC 2.36-9...) 2.36

# Distroless has no shell at all
docker run --rm gcr.io/distroless/static-debian12 sh
# docker: Error response from daemon: ... exec: "sh": executable file not found in $PATH

# Typical choices
FROM python:3.13-slim          # general Python service: glibc, small, maintained
# FROM alpine:3.20             # tiny utilities where musl is fine
# FROM gcr.io/distroless/static-debian12   # statically linked Go binary in production

Key takeaways

• The parent image determines much of your image's size, vulnerabilities, and tooling.
• Alpine is tiny but uses musl; slim Debian/Ubuntu variants are a safe general default.
• Distroless and hardened images minimize attack surface at the cost of debuggability.
• Prefer maintained, regularly rebuilt images, and separate build-time and run-time bases with multi-stage builds.

🧪 Practice

1. (Beginner) Run cat /etc/os-release in alpine:3.20, debian:12-slim, and python:3.13-slim. Which distribution is the Python image built on?
2. (Intermediate) Install the same package (for example curl) in an Alpine-based and a Debian-slim-based container. Compare the commands, the time taken, and the resulting writable layer size (docker ps -s).
3. Interview question: "A Python service is 1.2 GB as an image. How would you choose a better base, and what risks does switching to Alpine introduce?" Hint: consider the -slim variants, compiled dependencies (wheels built for glibc vs musl), and multi-stage builds.

Pulling and Listing Images

Why it matters. Before a container can run, its image must exist in the local image store. Pulling is how it gets there; listing is how you find out what is already there, how big it is, and which versions you have. Both commands have options that become important as soon as you deal with multiple architectures, many images, or scripts.

What happens during a pull

 docker pull nginx:1.27-alpine
   1. resolve   docker.io/library/nginx:1.27-alpine -> GET manifest/index from registry
   2. select    index lists platforms -> choose the one matching the host (e.g. linux/amd64)
   3. compare   for each layer digest in the manifest: already in local store?
                   yes -> "Already exists"
                   no  -> download compressed blob ("Downloading"), verify digest
   4. unpack    decompress layers into the storage driver ("Extracting", "Pull complete")
   5. record    store config, set the tag, print "Digest:" and "Status:"

Layers are downloaded in parallel and verified against their digests, so a corrupted or tampered download is rejected.

Useful pull options

Option	Effect
--platform linux/arm64	Pull a specific architecture instead of the host's
-a, --all-tags	Pull every tag of a repository (use with care)
-q, --quiet	Print only the final reference
docker run --pull always	Check the registry for a newer image before every run (missing is the default, never forbids pulling)

Listing images. docker image ls (alias docker images) shows one row per tag. SIZE is the uncompressed size on disk, including shared layers, which is why it is larger than the download size shown on the registry. Images with no tag appear as <none>:<none>; these are dangling images, typically left behind when a tag is moved to a newer build.

Filter / option	Example	Shows
Positional repository	docker image ls nginx	Only that repository
reference	--filter reference='*/api:*'	Names matching a pattern
dangling	--filter dangling=true	Untagged images
before / since	--filter since=nginx:1.27	Images created before/after another image
label	--filter label=org.opencontainers.image.vendor=Acme	Images with a label
--digests		Adds a DIGEST column
-q		Only image IDs (for scripting)
--format	'{{.Repository}}:{{.Tag}} {{.Size}}'	Custom output

Recent CLI versions also offer docker image ls --tree, which groups the platform variants stored for each multi-platform image.

Example

docker pull nginx:1.27-alpine
# 1.27-alpine: Pulling from library/nginx
# 43c4264eed91: Already exists          <- alpine base layer already local
# 5b19511a843e: Pull complete
# ...
# Digest: sha256:2140dad2...
# Status: Downloaded newer image for nginx:1.27-alpine
# docker.io/library/nginx:1.27-alpine

docker pull nginx:1.27-alpine
# Status: Image is up to date for nginx:1.27-alpine   <- nothing downloaded

# Different architecture (runs under emulation, or for re-pushing / inspection)
docker pull --platform linux/arm64 alpine:3.20
docker image inspect -f '{{.Os}}/{{.Architecture}}' alpine:3.20

# Listing
docker image ls
docker image ls --digests nginx
docker image ls --filter dangling=true
docker image ls --format 'table {{.Repository}}\t{{.Tag}}\t{{.Size}}' | sort -k3 -h

# Script-friendly: IDs of all images from the "myapp" repository
docker image ls -q myapp

Key takeaways

• A pull resolves the reference, picks the matching platform, downloads only missing layers, and verifies every digest.
• --platform selects an architecture; --pull always on docker run refreshes moving tags.
• docker image ls shows uncompressed sizes; shared layers are counted in every image that uses them.
• <none>:<none> entries are dangling images; filter and prune them as needed.

🧪 Practice

1. (Beginner) Pull postgres:16 and postgres:16-alpine. Compare the pull output (number of layers, "Already exists" lines) and the final sizes.
2. (Beginner) Produce a list of all local images sorted by size, showing only repository:tag and size, using --format.
3. (Intermediate) Build an image tagged demo:1, change the Dockerfile, and build it again with the same tag. Find the resulting dangling image with a filter and explain why it exists.

<a id="42-image-management"></a>

4.2 Image Management

This section covers the everyday operations on local images: naming, removing, and inspecting them, moving them between machines without a registry, and creating images from containers.

Tagging Images

Why tag images yourself. Tags are how you give an image a meaningful version, and also where it will be pushed: the registry and repository are part of the name. Before you can push myapp to ghcr.io/acme/myapp:1.2.0, the local image must carry that exact name. Tagging is also how you mark builds for promotion (1.2.0 becomes stable) without rebuilding.

What docker tag does. docker image tag SOURCE TARGET (alias docker tag) adds a new name to an existing image. Nothing is copied: both names point to the same image ID, like a hard link or a second Git branch on the same commit. If TARGET already points to another image, it is moved, and the old image keeps its other tags or becomes dangling.

 Before:   myapp:dev ---------------------> image 7c1d...

 docker tag myapp:dev myapp:1.2.0
 docker tag myapp:dev ghcr.io/acme/myapp:1.2.0

 After:    myapp:dev ------------------+
           myapp:1.2.0 ----------------+--> image 7c1d...   (stored once)
           ghcr.io/acme/myapp:1.2.0 ---+
                     ^
                     registry host in the name decides where "docker push" sends it

Tagging at build time. docker build -t accepts several tags at once, which avoids separate docker tag calls:

docker build -t myapp:1.2.0 -t myapp:1.2 -t ghcr.io/acme/myapp:1.2.0 .

Example

docker pull alpine:3.20

docker tag alpine:3.20 tools/base:1.0                    # local name
docker tag alpine:3.20 localhost:5000/tools/base:1.0     # name for a local registry
docker tag alpine:3.20 tools/base                        # no tag given -> tools/base:latest

docker image ls --filter reference='*/base' --filter reference='localhost:5000/*/*'
# REPOSITORY                  TAG      IMAGE ID       CREATED       SIZE
# tools/base                  1.0      91ef0af61f39   3 weeks ago   7.8MB
# tools/base                  latest   91ef0af61f39   3 weeks ago   7.8MB
# localhost:5000/tools/base   1.0      91ef0af61f39   3 weeks ago   7.8MB   <- same ID everywhere

# Tagging by image ID works too
docker tag 91ef0af61f39 tools/base:pinned

# Invalid names are rejected
docker tag alpine:3.20 Tools/Base:1.0
# Error parsing reference: "Tools/Base:1.0" is not a valid repository/tag: repository name must be lowercase

Key takeaways

• docker tag adds a name to an existing image; no data is copied.
• The name encodes the destination registry and repository for docker push.
• Re-tagging moves the name; the previously tagged image may become dangling.
• docker build can apply several tags in one go with repeated -t.

🧪 Practice

1. (Beginner) Tag nginx:alpine as web:v1, web:v2, and web:latest. Confirm with docker image ls that all three share one image ID.
2. (Intermediate) Build two different images, tag the first as app:current, then re-tag the second as app:current. Show what happened to the first image and how you would find it again.
3. Interview question: "How would you promote an image from staging to production without rebuilding it, and why is not rebuilding important?" Hint: consider re-tagging (or copying by digest) between repositories and what a rebuild could change even from identical source code.

Removing Images

Why removal needs care. Images are shared by tags, by other images (as layers), and by containers. Docker therefore distinguishes between removing a name and deleting image data, and refuses deletions that would break something. Knowing these rules avoids both "why is the disk still full?" and "why can't I delete this?".

How docker image rm decides

 docker image rm <reference>
        |
        +-- reference is a TAG and the image has other tags?
        |        -> "Untagged: ..."  only the name is removed, data stays
        |
        +-- it is the LAST tag (or an ID with one tag)?
        |        -> "Untagged: ..." then "Deleted: sha256:..."
        |           layers are freed unless another image still uses them
        |
        +-- a CONTAINER (running or stopped) uses the image?
        |        -> error "conflict: unable to remove repository reference ..."
        |           -f on a stopped container's image: untags, data kept until container is removed
        |           running container: refused even with -f
        |
        +-- an image ID with SEVERAL tags?
                 -> error "image is referenced in multiple repositories"; use -f or remove each tag

Related commands

Command	Removes
docker image rm IMAGE... (alias docker rmi)	The given tags/images
docker image prune	Dangling (<none>:<none>) images
docker image prune -a	Every image not used by any container
docker image prune -a --filter until=72h	Unused images created more than 72 hours ago
docker system prune	Dangling images plus stopped containers, unused networks, build cache

Bulk pruning is covered in Pruning Unused Resources.

Example

docker pull alpine:3.20
docker tag alpine:3.20 mybase:1

docker image rm mybase:1
# Untagged: mybase:1                              <- alpine:3.20 still holds the data

docker run --name holder alpine:3.20 true         # a stopped container now uses the image
docker image rm alpine:3.20
# Error response from daemon: conflict: unable to remove repository reference "alpine:3.20"
#   (must force) - container 4f2a... is using its referenced image 91ef0af61f39

docker rm holder
docker image rm alpine:3.20
# Untagged: alpine:3.20
# Untagged: alpine@sha256:beefdbd8...
# Deleted: sha256:91ef0af61f39...

# Remove every tag of one repository
docker image ls --format '{{.Repository}}:{{.Tag}}' myapp | xargs -r docker image rm

# Remove dangling images only
docker image prune -f

Key takeaways

• Removing a tag and deleting image data are different; data is deleted only when the last reference is gone.
• Images used by any container, even a stopped one, are protected; remove the container first.
• Shared layers survive as long as at least one image uses them.
• Use docker image prune for dangling images and -a with filters for wider cleanup.

🧪 Practice

1. (Beginner) Pull an image, give it two extra tags, and remove all three names one at a time. Note at which step the Deleted: line appears.
2. (Intermediate) Pull python:3.13-slim and python:3.12-slim, remove one of them, and use docker system df -v before and after to show that shared layers were not freed.
3. Interview question: "docker rmi -f succeeded, but docker system df shows the space was not reclaimed. Why?" Hint: consider what -f does when a stopped container still references the image.

Inspecting Image History

Why look at history. When an image is unexpectedly large, behaves differently than expected, or comes from an unknown source, you need to know how it was built. docker image history lists the build steps that produced each layer, with their sizes, so you can see where the bytes come from and which commands were run.

What it shows. One row per history entry, newest first. Steps that only changed metadata (ENV, CMD, EXPOSE, ...) show 0B. The IMAGE column shows <missing> for layers whose intermediate images do not exist locally, which is normal for every pulled image and for BuildKit builds; it does not mean anything is broken.

Limits of history

• It is self-reported metadata written by the builder. It can be incomplete or edited, so treat it as a hint, not proof. Use provenance attestations (Build Provenance and Attestations) for verifiable build information.
• Multi-stage builds only show the steps of the final stage; build stages leave no trace.
• Commands are recorded verbatim, including build argument values used in RUN. Never pass secrets as build arguments; use secret mounts (Build-Time Secrets).

Example

docker image history nginx:1.27-alpine
# IMAGE          CREATED       CREATED BY                                      SIZE      COMMENT
# 1ff4bb4faebc   2 weeks ago   CMD ["nginx" "-g" "daemon off;"]                0B        buildkit.dockerfile.v0
# <missing>      2 weeks ago   STOPSIGNAL SIGQUIT                              0B        buildkit.dockerfile.v0
# <missing>      2 weeks ago   EXPOSE map[80/tcp:{}]                           0B        buildkit.dockerfile.v0
# <missing>      2 weeks ago   ENTRYPOINT ["/docker-entrypoint.sh"]            0B        buildkit.dockerfile.v0
# <missing>      2 weeks ago   RUN /bin/sh -c set -x && addgroup -g 101 -S…   5.2MB     buildkit.dockerfile.v0
# ...
# <missing>      3 weeks ago   ADD alpine-minirootfs-3.20.3-x86_64.tar.gz /…   7.8MB     buildkit.dockerfile.v0

# Full commands, no truncation, human-readable sizes kept
docker image history --no-trunc --format '{{.Size}}\t{{.CreatedBy}}' nginx:1.27-alpine

# Find the biggest layers
docker image history --format '{{.Size}}\t{{.CreatedBy}}' python:3.13 | sort -h -r | head -n 5

# Demonstrate that build arguments leak into history
docker build -t leaky --build-arg TOKEN=s3cr3t - <<'EOF'
FROM alpine:3.20
ARG TOKEN
RUN echo "using token" > /dev/null
EOF
docker image history --no-trunc leaky | grep TOKEN
# ... RUN |1 TOKEN=s3cr3t /bin/sh -c echo "using token" > /dev/null ...   <- visible to anyone with the image

For an interactive, per-layer view of the files themselves, the third-party tool dive shows what each layer added, changed, or removed.

Key takeaways

• docker image history lists the build steps and the size each one added.
• <missing> in the IMAGE column is normal for pulled images and BuildKit builds.
• History is unverified metadata and omits earlier stages of multi-stage builds.
• Build argument values are recorded in history; never use them for secrets.

🧪 Practice

1. (Beginner) Run docker image history on redis:alpine and identify the step that installs Redis and how much space it takes.
2. (Intermediate) Write a Dockerfile with a RUN step that downloads a large file and a later RUN that deletes it. Use history to show that the image still carries the file's size, then fix the Dockerfile.
3. Interview question: "You are given an image from an unknown vendor. How would you find out what is in it and how it was built?" Hint: combine history, config inspection, filesystem exploration (docker export, dive), labels, SBOMs, and attestations, and consider which of these can be trusted.

Saving and Loading Images

Why save and load. Not every machine can reach a registry: air-gapped networks, factory floors, customer sites, or a CI job that hands an image to the next job as a file. docker save writes one or more images, with all their layers, tags, and metadata, into a tar archive; docker load restores them into another engine's image store exactly as they were.

What is in the archive. Recent Docker versions write an OCI image layout (see OCI Image Specification) plus Docker's legacy manifest.json, so the archive can be read both by docker load and by OCI tools such as skopeo.

 nginx.tar
 ├── oci-layout             OCI layout marker
 ├── index.json             entry point: tags -> manifests
 ├── manifest.json          Docker-format index for older tooling
 ├── repositories           legacy tag file
 └── blobs/sha256/
     ├── 2140dad2...        index / manifest
     ├── 1ff4bb4f...        config
     └── 43c4264e...        layer tars (one per layer)

Example

# Save one or several images (shared layers are written only once)
docker save -o images.tar nginx:1.27-alpine redis:7.4-alpine
ls -lh images.tar

# Compress on the fly; docker load detects gzip, bzip2, xz, and zstd automatically
docker save nginx:1.27-alpine | gzip > nginx.tar.gz

# Look inside
tar -tf images.tar | head

# Restore on another machine (tags and history are preserved)
docker load -i nginx.tar.gz
# Loaded image: nginx:1.27-alpine

# Copy straight to a remote host over SSH, no intermediate file
docker save myapp:1.2.0 | ssh deploy@edge-01 docker load

# Multi-platform images (containerd image store): choose what to export
docker save --platform linux/arm64 -o alpine-arm64.tar alpine:3.20

Key takeaways

• docker save exports images with all layers, tags, and config; docker load restores them unchanged.
• The archive is an OCI layout, readable by other OCI tools.
• Compress with gzip or zstd; docker load decompresses automatically.
• Use it for air-gapped transfers and CI artifacts; use a registry when one is available, because it transfers only missing layers.

🧪 Practice

1. (Beginner) Save alpine:3.20 to a file, remove the image, load it back, and confirm the image ID did not change.
2. (Intermediate) Save python:3.13-slim and python:3.12-slim in separate archives and then together in one archive. Compare the three file sizes and explain the result.
3. Interview question: "How would you deliver container updates to a fleet of machines with no internet access and limited bandwidth?" Hint: compare full docker save archives with a local registry or mirror inside the network, and think about which layers actually change between releases.

Exporting and Importing Containers

Why export exists. Sometimes you do not need an image with its history; you need the filesystem of a container, for example to examine it with ordinary tools, to extract build output, or to produce a single-layer root filesystem for another system. docker export writes a container's merged filesystem as a plain tar; docker import turns such a tar (from Docker or from anywhere else) into a new image with one layer.

Export/import versus save/load

	docker save / docker load	docker export / docker import
Works on	Images	Containers (export), tarballs (import)
Layers	All preserved	Flattened into a single layer
History	Preserved	Lost
Config (CMD, ENV, ENTRYPOINT, ...)	Preserved	Lost; set again with --change on import
Includes container changes	No	Yes, the writable layer is merged in
Includes volume contents	No	No
Typical use	Moving images without a registry	Inspecting/extracting filesystems, creating base images from rootfs tarballs

Example

docker run -d --name web nginx:1.27-alpine
docker exec web sh -c 'echo "<h1>exported</h1>" > /usr/share/nginx/html/index.html'

# Export the merged filesystem (image layers + container changes)
docker export -o web-rootfs.tar web
tar -tf web-rootfs.tar | grep index.html        # usr/share/nginx/html/index.html

# Extract only one directory from an image without running it
docker create --name tmp nginx:1.27-alpine
mkdir -p out
docker export tmp | tar -x -C out etc/nginx      # only etc/nginx is extracted
docker rm tmp

# Import as a new single-layer image: the config is gone ...
docker import web-rootfs.tar web:flat
docker run --rm web:flat
# docker: Error response from daemon: no command specified

# ... so restore what is needed with --change (Dockerfile syntax)
docker import \
  --change 'CMD ["nginx", "-g", "daemon off;"]' \
  --change 'EXPOSE 80' \
  web-rootfs.tar web:flat
docker image history web:flat                   # a single "Imported from -" layer

docker rm -f web

Key takeaways

• docker export dumps a container's merged filesystem; it is a filesystem, not an image.
• docker import creates a single-layer image with no history and no runtime config unless you add --change.
• Neither includes volume data.
• Use save/load to move images; use export/import to work with filesystems.

🧪 Practice

1. (Beginner) Export a debian:12-slim container and list the top-level directories in the archive. Compare the archive size with the image size.
2. (Intermediate) Export a container that has a few modified files, import it, and compare docker image history and docker image inspect -f '{{.Config.Cmd}}' with the original image.
3. Interview question: "Someone suggests using docker export and docker import to shrink images by flattening layers. What do you gain and what do you lose?" Hint: consider layer sharing with other images, cached pulls, image config, history, and why multi-stage builds achieve the same goal better.

Committing Containers to Images

Why commit exists. docker commit takes a container's current state, including all changes in its writable layer, and saves it as a new image layer on top of the original image. It is useful for snapshots when debugging ("save this broken state so I can study it later"), for quick experiments, and for understanding that an image is nothing more than layers plus config. It is not a good way to produce images you ship.

Why not for production. A committed image is a black box: nobody can tell how it was produced, it cannot be rebuilt from source, it often contains leftovers (shell history, caches, temp files, sometimes credentials), and every update requires repeating manual steps. A Dockerfile records each step, is version-controlled and reviewable, and rebuilds the same way every time.

	docker commit	Dockerfile + docker build
Reproducible	No	Yes
Reviewable / versioned	No	Yes, it is source code
Uses the build cache	No	Yes
Risk of leftovers	High	Low (with .dockerignore and multi-stage builds)
Good for	Debug snapshots, experiments, forensics	Everything you build and ship

How it works. By default, the container is paused during commit for consistency (--pause=false disables this). --change applies Dockerfile instructions (CMD, ENTRYPOINT, ENV, EXPOSE, LABEL, USER, VOLUME, WORKDIR, ONBUILD) to the new image's config. Data in volumes is not included.

   nginx:1.27-alpine layers (read-only)       myweb:snapshot
   +-------------------------------+          +-------------------------------+
   | ...                           |          | NEW layer = container's       |
   +-------------------------------+   commit |   writable layer contents     |
   container "web" writable layer  |  ------> +-------------------------------+
   (edited index.html)             |          | nginx:1.27-alpine layers      |
                                              | (reused, not copied)          |
                                              +-------------------------------+

Example

docker run -d --name web nginx:1.27-alpine
docker exec web sh -c 'echo "<h1>Hello from a commit</h1>" > /usr/share/nginx/html/index.html'

docker diff web                                   # what will be committed
# C /usr/share/nginx/html/index.html
# C /var/cache/nginx ...                          <- leftovers get committed too

docker commit \
  -a "Jane Doe <jane@example.com>" \
  -m "Custom index page" \
  -c 'LABEL purpose=debug-snapshot' \
  web myweb:snapshot

docker image history myweb:snapshot | head -n 2
# IMAGE          CREATED          CREATED BY                          SIZE    COMMENT
# 6d0e2f1a9c3b   10 seconds ago   /docker-entrypoint.sh nginx -g 'da…   1.2kB   Custom index page

docker run -d --rm -p 8081:80 --name web2 myweb:snapshot
curl -s localhost:8081                             # <h1>Hello from a commit</h1>

docker rm -f web web2

The equivalent, reproducible version is a two-line Dockerfile:

FROM nginx:1.27-alpine
COPY index.html /usr/share/nginx/html/index.html   # explicit, reviewable, rebuildable

Key takeaways

• docker commit turns a container's writable layer into a new image layer, keeping the parent layers.
• It pauses the container by default and does not include volume data.
• Use it for debugging snapshots and experiments, not for images you deploy.
• Anything committed can be reproduced, and should be, with a Dockerfile.

🧪 Practice

1. (Beginner) Run ubuntu:24.04 interactively, install curl with apt, exit, and commit the container. Check the size of the new layer with docker image history.
2. (Intermediate) Write the Dockerfile that reproduces exercise 1, build it, and compare both images' sizes. Use docker diff on the original container to explain any difference.
3. Interview question: "A colleague's production image was created with docker commit two years ago and nobody knows how to rebuild it. How would you recover a maintainable build?" Hint: inspect history, config, and the filesystem diff against the parent image, then rebuild with a Dockerfile and compare the results.

<a id="43-image-registries"></a>

4.3 Image Registries

This section builds on the overview in Registries and covers the public and private registries you will work with, how pushing works, and how clients authenticate.

Docker Hub

Why Docker Hub matters. Docker Hub is the default registry: whenever an image name has no registry host, Docker looks it up on docker.io. Most base images, databases, and tools you use start there, so its namespaces, limits, and account features directly affect your builds and CI pipelines, even if you push your own images elsewhere.

Structure. Docker Hub hosts repositories under namespaces: library/ for official images (hidden in short names), and one namespace per user or organization. Each repository is public or private and holds tags.

   docker.io
   ├── library/            official images: "nginx" == "docker.io/library/nginx"
   │   ├── nginx
   │   ├── postgres
   │   └── ...
   ├── bitnami/            organization (verified publisher)
   │   └── redis
   └── jdoe/               personal namespace
       ├── hello           public repository
       └── internal-tool   private repository

Main features

Feature	Purpose
Public and private repositories	Distribute images openly or only to your account/organization
Organizations and teams	Grant per-repository read/write/admin permissions to groups
Personal and organization access tokens	Scoped credentials for CLI and CI instead of your password
Webhooks	Notify other systems when a tag is pushed
Image analysis (Docker Scout)	Vulnerability and policy information per tag (Docker Scout)
Search and image pages	Documentation, tag lists, pull counts, trust badges

Pull rate limits. Docker Hub limits how many image pulls (manifest requests) a client may make in a time window. Anonymous pulls are limited per IP address and much more strictly than authenticated ones; paid plans raise or remove the limits. The exact numbers have changed several times, so check Docker's current documentation. Shared IPs (office NAT, CI runners) hit anonymous limits quickly, which shows up as 429 Too Many Requests or toomanyrequests errors. Always authenticate in CI, and consider a pull-through cache (Private Registries).

Example

# Search from the CLI (limited compared to the web UI)
docker search --filter is-official=true --limit 3 postgres
# NAME       DESCRIPTION                                     STARS     OFFICIAL
# postgres   The PostgreSQL object-relational database sy…   14000     [OK]

# List tags through the Hub API
curl -s "https://hub.docker.com/v2/repositories/library/postgres/tags?page_size=5" \
  | jq -r '.results[] | "\(.name)\t\(.last_updated)"'

# Check your current rate limit (HEAD requests do not count as pulls)
TOKEN=$(curl -s "https://auth.docker.io/token?service=registry.docker.io&scope=repository:ratelimitpreview/test:pull" | jq -r .token)
curl -s --head -H "Authorization: Bearer $TOKEN" \
  https://registry-1.docker.io/v2/ratelimitpreview/test/manifests/latest | grep -i ratelimit
# ratelimit-limit: <limit>;w=<window in seconds>
# ratelimit-remaining: <remaining>;w=<window in seconds>

# Log in to raise your limits (opens a browser/device-code flow, or use a token)
docker login

Key takeaways

• Docker Hub (docker.io) is the default registry; library/ holds official images.
• Repositories live in user or organization namespaces and can be public or private.
• Pull rate limits are stricter for anonymous clients; authenticate, especially in CI.
• Use access tokens, not passwords, for CLI and automation.

🧪 Practice

1. (Beginner) Create a free Docker Hub account and a personal access token with read-only scope. Log in with it using docker login -u <user> --password-stdin.
2. (Intermediate) Run the rate-limit check above anonymously and then with a token obtained using your credentials (curl -u user:token "https://auth.docker.io/token?..."). Compare the limits.
3. Interview question: "Several CI jobs started failing with toomanyrequests at the same time of day. How do you diagnose and fix it?" Hint: think about shared egress IPs, anonymous pulls, caching layers locally or in a mirror, and authentication in every job.

Official and Verified Images

Why trust signals matter. Anyone can publish an image on Docker Hub, and an image runs with whatever code its author put in it. Typosquatted names (ngnix, postgress), abandoned images full of known vulnerabilities, and images that mine cryptocurrency have all been found on public registries. Docker Hub's badges help you identify images that are curated or come from a verified source.

The categories

Category	Namespace / badge	Who maintains it	What it guarantees
Docker Official Images	library/ (short names like nginx)	Docker's team together with upstream maintainers	Reviewed Dockerfiles in a public repository, regular rebuilds for security fixes, best-practice conventions, multi-architecture support
Verified Publisher	Vendor namespace with a badge	The commercial vendor (e.g. a database or monitoring company)	The publisher's identity was verified by Docker
Sponsored Open Source	Project namespace with a badge	The open-source project	The project is part of Docker's open-source program
Hardened images	Separate catalog	Docker	Minimal, frequently patched images with SBOMs and provenance
Community	Any user or organization	Anyone	Nothing

How official images are made. Each official image has public source (for example the docker-library/postgres repository on GitHub) and a definition file in the docker-library/official-images repository that maps tags to Git commits. Changes go through public review, and images are rebuilt when their base image or upstream software receives updates.

A checklist for any image

• Does it carry a badge, and does the namespace exactly match the project you expect?
• Is there a linked source repository and Dockerfile (org.opencontainers.image.source label)?
• When was it last updated? Old images accumulate vulnerabilities.
• What does a vulnerability scan show (Vulnerability Scanning)?
• Does it provide SBOM and provenance attestations, or signatures (Image Signing and Verification)?
• Pull counts and stars indicate popularity, not safety.

Example

# Only official images in search results
docker search --filter is-official=true redis

# Labels often point to the source and version
docker image inspect -f '{{json .Config.Labels}}' grafana/grafana:11.2.0 | jq .

# Attestations attached to an image (empty output if the image has none)
docker buildx imagetools inspect nginx:1.27-alpine --format '{{json .Provenance}}' | jq '.[] | .SLSA.buildType' 2>/dev/null
docker buildx imagetools inspect nginx:1.27-alpine --format '{{json .SBOM}}' | head -c 300; echo

Key takeaways

• Docker Official Images are curated, publicly reviewed, and rebuilt regularly; prefer them for base images.
• Verified Publisher and Sponsored Open Source badges confirm who publishes the image, not that it is free of vulnerabilities.
• Check the exact namespace to avoid typosquatting, and verify source, recency, scans, and attestations.
• Popularity is not a security signal.

🧪 Practice

1. (Beginner) Find the GitHub source of the redis official image from its Docker Hub page and locate the Dockerfile for the alpine variant.
2. (Intermediate) Pick an official image and a popular community image providing the same software. Compare their update frequency, labels, size, and scan results (docker scout quickview if available).
3. Interview question: "What policy would you put in place to control which public images developers can use as base images in your company?" Hint: consider allowlists of trusted namespaces, internal mirrors of approved images, digest pinning, admission policies, and scanning in CI.

Pushing Images

Why push. Pushing uploads an image to a registry so that other machines, CI jobs, or orchestrators can pull it. It is the "publish" step that connects building on one machine with running anywhere else.

How a push works. docker push reads the registry and repository from the image name, then uploads only what the registry does not already have:

 docker push ghcr.io/acme/api:1.4.2
   1. auth      get a token with push+pull scope for acme/api
   2. per layer HEAD /v2/acme/api/blobs/<digest>
                  exists                -> "Layer already exists"
                  exists in another repo -> cross-repository mount: "Mounted from acme/base"
                  missing               -> upload blob, registry verifies the digest -> "Pushed"
   3. config    upload the config blob the same way
   4. manifest  PUT /v2/acme/api/manifests/1.4.2   -> tag now points to the new manifest
   5. output    "1.4.2: digest: sha256:... size: ..."

Because layers are content-addressed, pushing a new version of an application usually uploads only the few top layers that changed.

Requirements and common errors

Message	Cause	Fix
denied: requested access to the resource is denied	Not logged in, or no write access to that namespace	docker login, check the namespace (your user or an organization you belong to)
An image does not exist locally with the tag: ...	The image was not tagged with the target name	docker tag first
name unknown / repository does not exist	Some registries (e.g. ECR) require creating the repository first	Create the repository
tag invalid / immutable	The registry enforces immutable tags	Push a new tag instead of overwriting

Pushing many tags and platforms. docker push --all-tags (-a) pushes every local tag of a repository. Multi-platform images are normally built and pushed in one step with docker buildx build --platform ... --push, covered in Multi-Architecture Images.

Example

docker login -u jdoe                                   # Docker Hub; token as password

docker build -t jdoe/hello:1.0 - <<'EOF'
FROM alpine:3.20
CMD ["echo", "hello from Docker Hub"]
EOF

docker push jdoe/hello:1.0
# The push refers to repository [docker.io/jdoe/hello]
# 63ca1fbb43ae: Mounted from library/alpine          <- base layer reused, no upload
# 1.0: digest: sha256:7d2f... size: 528

# Add more tags and push them all
docker tag jdoe/hello:1.0 jdoe/hello:1
docker tag jdoe/hello:1.0 jdoe/hello:latest
docker push --all-tags jdoe/hello
# ... Layer already exists ...                       <- only manifests are written

# Verify from anywhere: remove locally and pull again
docker image rm jdoe/hello:1.0 jdoe/hello:1 jdoe/hello:latest
docker run --rm jdoe/hello:1.0

Key takeaways

• The image name decides the destination; tag the image with the full target name before pushing.
• Only missing blobs are uploaded; existing and cross-repository layers are reused.
• The tag is moved only after all blobs and the manifest are successfully stored.
• Use --all-tags for several tags and buildx --push for multi-platform images.

🧪 Practice

1. (Beginner) Push an image to your Docker Hub namespace, delete it locally, and run it again from the registry.
2. (Intermediate) Change only the CMD of the image and push it as :1.1. Which lines in the push output say Pushed and which say Layer already exists? Explain why.
3. Interview question: "Two CI pipelines push the same tag at the same moment. What ends up in the registry, and how would you prevent the ambiguity?" Hint: consider when the tag is written relative to the blobs, last-write-wins semantics, unique tags per build, and immutable tags.

Private Registries

Why run your own registry. Public registries are not always acceptable: proprietary code must stay inside the company, regulated environments require data residency, air-gapped networks cannot reach the internet, and large fleets benefit from a local cache that avoids rate limits and speeds up pulls. A private registry is any registry you control, whether a single container or a full platform.

Options

Product	Type	Highlights
registry (CNCF Distribution)	Minimal open-source server	The reference implementation; a single container; no UI or user management
Harbor	Open-source platform (CNCF)	Projects, RBAC, vulnerability scanning, replication, signing, quotas
GitLab / Gitea / Forgejo Container Registry	Built into the Git platform	Permissions follow the code repository
Sonatype Nexus, JFrog Artifactory	Commercial / OSS artifact managers	Containers alongside Maven, npm, PyPI, and other formats
Zot	Open-source, OCI-native	Lightweight, OCI-only, built-in extensions

Pieces a production registry needs

   docker CLI / CI / Kubernetes nodes
                |
                |  HTTPS (TLS cert trusted by clients)
                v
   +----------------------------+        +-------------------------+
   | registry                   | -----> | storage backend         |
   |  - authentication (htpasswd|        |  filesystem, S3, Azure, |
   |    / token service / LDAP) |        |  GCS, ...               |
   |  - authorization           |        +-------------------------+
   |  - garbage collection      |
   +----------------------------+

• TLS: Docker refuses plain HTTP to any registry except localhost. Use a certificate trusted by the host, place a private CA in /etc/docker/certs.d/<host>:<port>/ca.crt, or (for labs only) list the host in insecure-registries in daemon.json.
• Authentication: the bare registry image supports htpasswd; larger setups use a token service, LDAP/OIDC, or a platform like Harbor.
• Garbage collection: deleting a tag or manifest does not free blob storage; a garbage-collection run does.
• Pull-through cache: a registry configured as a proxy for Docker Hub stores each image the first time it is pulled and serves later pulls locally.

Example: a registry with TLS and basic authentication

mkdir -p auth certs

# Self-signed certificate for registry.internal (lab use; use your CA in production)
openssl req -x509 -newkey rsa:4096 -nodes -sha256 -days 365 \
  -keyout certs/registry.key -out certs/registry.crt \
  -subj "/CN=registry.internal" -addext "subjectAltName=DNS:registry.internal"

# Bcrypt password file (the registry image no longer ships htpasswd)
docker run --rm --entrypoint htpasswd httpd:2 -Bbn alice 's3cret' > auth/htpasswd

docker run -d --name registry --restart unless-stopped -p 5000:5000 \
  -v "$PWD/auth:/auth:ro" -v "$PWD/certs:/certs:ro" \
  -v registry-data:/var/lib/registry \
  -e REGISTRY_AUTH=htpasswd \
  -e REGISTRY_AUTH_HTPASSWD_REALM="Internal Registry" \
  -e REGISTRY_AUTH_HTPASSWD_PATH=/auth/htpasswd \
  -e REGISTRY_HTTP_TLS_CERTIFICATE=/certs/registry.crt \
  -e REGISTRY_HTTP_TLS_KEY=/certs/registry.key \
  -e REGISTRY_STORAGE_DELETE_ENABLED=true \
  registry:2

# On every client: trust the certificate (directory name must match host:port)
sudo mkdir -p /etc/docker/certs.d/registry.internal:5000
sudo cp certs/registry.crt /etc/docker/certs.d/registry.internal:5000/ca.crt
# (and make "registry.internal" resolve, e.g. via DNS or /etc/hosts)

docker login registry.internal:5000 -u alice
docker tag alpine:3.20 registry.internal:5000/base/alpine:3.20
docker push registry.internal:5000/base/alpine:3.20

# Reclaim space after deleting manifests (run while nothing is pushing)
docker exec registry registry garbage-collect --delete-untagged /etc/docker/registry/config.yml

Example: a Docker Hub pull-through cache

docker run -d --name hub-mirror -p 5001:5000 \
  -e REGISTRY_PROXY_REMOTEURL=https://registry-1.docker.io \
  -e REGISTRY_PROXY_USERNAME=jdoe -e REGISTRY_PROXY_PASSWORD="$HUB_TOKEN" \
  registry:2

{
  "registry-mirrors": ["https://mirror.internal:5001"]
}

With this in /etc/docker/daemon.json (and a daemon restart), pulls of Docker Hub images try the mirror first and fall back to Docker Hub. The mirror must be served over HTTPS (add the TLS variables from the previous example) unless it is listed in insecure-registries. Mirrors configured this way apply only to Docker Hub images.

Key takeaways

• A private registry keeps images in-house, supports air-gapped networks, and avoids public rate limits.
• registry:2 is a minimal server; Harbor and artifact managers add RBAC, scanning, replication, and UI.
• Production registries need TLS, authentication, durable storage, and periodic garbage collection.
• A pull-through cache in front of Docker Hub speeds up pulls and reduces rate-limit problems.

🧪 Practice

1. (Beginner) Run registry:2 on localhost:5000, push two images, and list them through curl http://localhost:5000/v2/_catalog.
2. (Intermediate) Set up the TLS and htpasswd registry above. Verify that pushing without logging in fails, that pushing after docker login succeeds, and that curl -u alice:s3cret --cacert certs/registry.crt https://registry.internal:5000/v2/_catalog lists the repository.
3. Interview question: "Your self-hosted registry's disk keeps growing although old tags are deleted every week. Why, and what would you change?" Hint: think about the difference between deleting references and deleting blobs, garbage collection, untagged manifests, and retention policies.

Cloud Registries (ECR, GCR, ACR, GHCR)

Why use a cloud registry. If your workloads run on a cloud provider or your code lives on GitHub, the provider's registry is usually the simplest choice: no servers to operate, access controlled through the identity system you already use, fast pulls from the same region, and built-in features such as scanning, replication, and retention policies.

Comparison

Registry	Image name format	Log in	Access control	Notes
Amazon ECR	<account>.dkr.ecr.<region>.amazonaws.com/<repo>:<tag>	aws ecr get-login-password | docker login --password-stdin	AWS IAM	Login tokens last 12 hours; repositories are usually created before pushing; lifecycle policies, tag immutability, scanning
Google Artifact Registry (successor to GCR)	<region>-docker.pkg.dev/<project>/<repo>/<image>:<tag>	gcloud auth configure-docker <region>-docker.pkg.dev	Google Cloud IAM	Container Registry (gcr.io) has been shut down in favor of Artifact Registry; gcr.io names can be served by Artifact Registry
Azure Container Registry	<name>.azurecr.io/<repo>:<tag>	az acr login --name <name>	Microsoft Entra ID, tokens	Geo-replication and private endpoints (Premium tier), ACR Tasks for builds
GitHub Container Registry	ghcr.io/<owner>/<image>:<tag>	docker login ghcr.io with a token	GitHub users, organizations, repository permissions	GITHUB_TOKEN in Actions; images can be linked to a repository; free for public images

Common capabilities. Most cloud registries offer: vulnerability scanning on push, tag immutability, lifecycle/retention rules that delete old images automatically, replication across regions, and integration with the provider's container services, which can then pull with machine identities instead of stored passwords.

Example: logging in and pushing

# --- Amazon ECR ---
ACCOUNT=123456789012; REGION=eu-central-1
REGISTRY=$ACCOUNT.dkr.ecr.$REGION.amazonaws.com
aws ecr create-repository --repository-name api --image-tag-mutability IMMUTABLE
aws ecr get-login-password --region $REGION \
  | docker login --username AWS --password-stdin $REGISTRY     # the username is literally "AWS"
docker tag api:1.4.2 $REGISTRY/api:1.4.2
docker push $REGISTRY/api:1.4.2

# --- Google Artifact Registry ---
gcloud artifacts repositories create containers --repository-format=docker --location=europe-west1
gcloud auth configure-docker europe-west1-docker.pkg.dev          # adds a credential helper entry
docker tag api:1.4.2 europe-west1-docker.pkg.dev/my-project/containers/api:1.4.2
docker push europe-west1-docker.pkg.dev/my-project/containers/api:1.4.2

# --- Azure Container Registry ---
az acr login --name acmeregistry
docker tag api:1.4.2 acmeregistry.azurecr.io/api:1.4.2
docker push acmeregistry.azurecr.io/api:1.4.2

# --- GitHub Container Registry ---
echo "$GHCR_TOKEN" | docker login ghcr.io -u <github-user> --password-stdin   # token with write:packages
docker tag api:1.4.2 ghcr.io/acme/api:1.4.2
docker push ghcr.io/acme/api:1.4.2

# GitHub Actions: push to GHCR with the built-in token (no stored secret)
permissions:
  contents: read
  packages: write
steps:
  - uses: docker/login-action@v3
    with:
      registry: ghcr.io
      username: ${{ github.actor }}
      password: ${{ secrets.GITHUB_TOKEN }}

Key takeaways

• Cloud registries integrate with the provider's identity system, compute services, and regions.
• Each has its own image name format, which is part of every tag you push.
• Logins are usually short-lived tokens obtained through the provider's CLI or a credential helper.
• Use tag immutability and lifecycle rules to keep registries trustworthy and costs under control.

🧪 Practice

1. (Beginner) Push an image to GHCR from your machine using a personal access token, then make the package public and pull it without logging in.
2. (Intermediate) In a cloud registry of your choice, configure a lifecycle or cleanup rule that keeps only the 10 most recent tags matching git-*. Push 12 tags and verify the result.
3. Interview question: "How would you let a fleet of cloud VMs pull private images without storing any long-lived registry password on them?" Hint: think about instance or workload identities, short-lived tokens, and credential helpers.

Registry Authentication

Why authentication is subtle. Logging in seems trivial, but the details decide whether credentials end up in plain text on a shared CI runner, whether sudo docker finds them, and why a login that "worked" still yields denied. Knowing the protocol and where credentials are stored makes these problems easy to reason about.

The token flow. Most registries use bearer-token authentication. The client does not send your password to the registry on every request; it exchanges credentials for a short-lived token scoped to a specific repository and action:

  client                                   registry                   auth service
    | GET /v2/library/alpine/manifests/3.20  |                              |
    |--------------------------------------->|                              |
    | 401  WWW-Authenticate: Bearer realm="https://auth.docker.io/token",   |
    |      service="registry.docker.io",scope="repository:library/alpine:pull"
    |<---------------------------------------|                              |
    | GET realm?service=...&scope=...  (with username/token, or anonymous)  |
    |---------------------------------------------------------------------->|
    | 200  {"token": "eyJ..."}   short-lived, limited to that scope         |
    |<----------------------------------------------------------------------|
    | GET manifest  Authorization: Bearer eyJ...                            |
    |--------------------------------------->|                              |
    | 200  manifest                          |                              |

Where credentials are stored. docker login writes to the client's ~/.docker/config.json (or the directory in $DOCKER_CONFIG). Without further configuration, the credentials are stored there base64-encoded, which is not encryption. A credential store or helper keeps them in a secure backend instead:

Setting in config.json	Effect	Examples
auths	Per-registry entries; contain auth (base64 user:password) if no store is used	Plain-file fallback
credsStore	Default store for all registries	desktop, osxkeychain, wincred, secretservice, pass
credHelpers	Helper per registry, overrides credsStore	ecr-login, gcloud, acr-env

A helper named x is the executable docker-credential-x on the PATH. Cloud helpers can generate fresh tokens on demand, so no login step is needed at all.

{
  "auths": {
    "https://index.docker.io/v1/": {},
    "ghcr.io": {}
  },
  "credsStore": "desktop",
  "credHelpers": {
    "123456789012.dkr.ecr.eu-central-1.amazonaws.com": "ecr-login",
    "europe-west1-docker.pkg.dev": "gcloud"
  }
}

Good practices

• Use access tokens with the smallest scope (read-only for pulling), not account passwords.
• Use --password-stdin so the secret does not appear in shell history or ps output.
• On shared CI runners, give each job its own DOCKER_CONFIG directory and run docker logout (or delete the directory) at the end.
• Credentials belong to the client user: sudo docker login writes to root's config, which your normal user does not read.
• Orchestrators need their own pull credentials (for example Kubernetes imagePullSecrets or node identities); your local login does not transfer.

Example

# Non-interactive login without exposing the token in history or ps
echo "$REGISTRY_TOKEN" | docker login ghcr.io -u ci-bot --password-stdin

# What was stored? (without a credential store, the secret is merely base64)
jq '.auths' ~/.docker/config.json
jq -r '.auths["ghcr.io"].auth' ~/.docker/config.json | base64 -d   # ci-bot:<token>  <- readable!

# Isolated credentials for one CI job
export DOCKER_CONFIG=$(mktemp -d)
echo "$REGISTRY_TOKEN" | docker login ghcr.io -u ci-bot --password-stdin
docker push ghcr.io/acme/api:1.4.2
docker logout ghcr.io && rm -rf "$DOCKER_CONFIG"

# Walk through the token flow by hand (anonymous pull scope)
curl -s -o /dev/null -D - https://registry-1.docker.io/v2/library/alpine/manifests/3.20 | grep -i www-authenticate
TOKEN=$(curl -s "https://auth.docker.io/token?service=registry.docker.io&scope=repository:library/alpine:pull" | jq -r .token)
curl -s -H "Authorization: Bearer $TOKEN" \
  -H "Accept: application/vnd.oci.image.index.v1+json" \
  https://registry-1.docker.io/v2/library/alpine/manifests/3.20 | jq '.manifests[].platform'

Key takeaways

• Registries use a challenge/token flow: credentials are exchanged for short-lived, scoped tokens.
• docker login stores credentials in the client's config.json; without a store they are only base64-encoded.
• Use credential stores or helpers, scoped access tokens, and --password-stdin.
• Credentials are per client user and per machine; CI jobs and orchestrators need their own.

🧪 Practice

1. (Beginner) Log in to a registry, inspect ~/.docker/config.json, and determine whether your credentials are in a credential store or in the file. Log out and inspect it again.
2. (Intermediate) Reproduce the token flow with curl for a private repository of your own, passing your username and token to the auth endpoint. Request a pull,push scope and observe what the auth service grants.
3. Interview question: "A developer reports that docker pull of a private image works on their laptop but fails with denied in CI and in Kubernetes. What would you check?" Hint: consider which user and DOCKER_CONFIG the CI job uses, token scopes and expiry, the exact registry hostname in the credentials, and how the cluster nodes obtain pull credentials.

---

<a id="5-building-images-with-dockerfiles"></a>