docker-impl-build-optimization

Quick Reference

Layer Caching Rules

Docker checks each instruction against its cache before executing. If the instruction and its inputs match a cached layer, the cached version is reused.

Critical rule: Once ANY layer's cache is invalidated, ALL subsequent layers MUST rebuild.

Cache Invalidation Triggers

Instruction	Cache Key	Invalidation Trigger
`FROM`	Image reference	Base image tag/digest changed
`RUN`	Command string only	Command text changed (NOT external resources)
`COPY`	File content checksums	File content changed (mtime is NOT checked)
`ADD`	File checksums + URL content	File content or URL content changed
`ENV`	Key=Value pair	Value changed
`ARG`	Name=Value pair	Value changed
`WORKDIR`	Path + `SOURCE_DATE_EPOCH`	Path or epoch changed

Critical Warnings

NEVER separate apt-get update and apt-get install into different RUN instructions -- the cached update layer becomes stale and subsequent installs may fail or use outdated packages.

NEVER use COPY . . before dependency installation -- ANY file change invalidates the COPY layer and forces a full reinstall of all dependencies.

NEVER rely on RUN cache for external resources -- Docker only checks the command string, not what apt-get install or curl fetches. Use --no-cache or --no-cache-filter to force fresh downloads.

ALWAYS include a .dockerignore file -- without it, the entire build context (including .git/, node_modules/, test data) is sent to the builder.

ALWAYS use # syntax=docker/dockerfile:1 at the top of every Dockerfile to enable BuildKit cache mounts and other optimizations.

Instruction Ordering Strategy

Order instructions from LEAST frequently changed to MOST frequently changed:

+--------------------------------------------------+
| FROM base-image                    (rarely changes) |
+--------------------------------------------------+
| RUN install system packages         (rarely changes) |
+--------------------------------------------------+
| COPY package.json / go.mod / *.csproj (dep changes) |
+--------------------------------------------------+
| RUN install dependencies            (dep changes)   |
+--------------------------------------------------+
| COPY . .                            (every commit)  |
+--------------------------------------------------+
| RUN build application               (every commit)  |
+--------------------------------------------------+
| CMD / ENTRYPOINT                    (rarely changes) |
+--------------------------------------------------+
         CACHE FLOWS TOP-DOWN
   First invalidation breaks ALL below

Principle: Expensive, slow-changing operations go at the top. Frequently changing source code goes at the bottom.

.dockerignore Template

ALWAYS create a .dockerignore in the project root:

# Version control
.git
.gitignore
.gitattributes

# Dependencies (rebuilt inside container)
node_modules
vendor
__pycache__
*.pyc
.venv

# Build artifacts
dist
build
target
*.o
*.exe

# IDE and OS files
.vscode
.idea
*.swp
*.swo
.DS_Store
Thumbs.db

# Docker files (not needed in build context)
Dockerfile*
docker-compose*.yml
.dockerignore

# Documentation and non-essential files
*.md
LICENSE
docs/

# Environment and secrets
.env
.env.*
*.pem
*.key
*.cert

# Test and CI files
.github
.gitlab-ci.yml
tests/
coverage/

Negation syntax: Use ! to re-include files excluded by a broader pattern:

*.md
!README.md

Cache Mount Patterns

Cache mounts (--mount=type=cache) persist package manager caches across builds. Even when a layer rebuilds, only new or changed packages are downloaded.

apt-get (Debian/Ubuntu)

RUN --mount=type=cache,target=/var/cache/apt,sharing=locked \
    --mount=type=cache,target=/var/lib/apt,sharing=locked \
    apt-get update && apt-get install -y --no-install-recommends \
    curl git

ALWAYS use sharing=locked for apt -- concurrent access corrupts the cache.

npm

COPY package.json package-lock.json ./
RUN --mount=type=cache,target=/root/.npm \
    npm ci

yarn

COPY package.json yarn.lock ./
RUN --mount=type=cache,target=/usr/local/share/.cache/yarn \
    yarn install --frozen-lockfile

pnpm

COPY package.json pnpm-lock.yaml ./
RUN --mount=type=cache,target=/root/.local/share/pnpm/store \
    pnpm install --frozen-lockfile

pip (Python)

COPY requirements.txt ./
RUN --mount=type=cache,target=/root/.cache/pip \
    pip install -r requirements.txt

Go modules

COPY go.mod go.sum ./
RUN --mount=type=cache,target=/go/pkg/mod \
    go mod download

COPY . .
RUN --mount=type=cache,target=/go/pkg/mod \
    --mount=type=cache,target=/root/.cache/go-build \
    go build -o /app/server ./cmd

Cargo (Rust)

COPY Cargo.toml Cargo.lock ./
RUN --mount=type=cache,target=/app/target/ \
    --mount=type=cache,target=/usr/local/cargo/git/db \
    --mount=type=cache,target=/usr/local/cargo/registry/ \
    cargo build --release

Maven (Java)

COPY pom.xml ./
RUN --mount=type=cache,target=/root/.m2/repository \
    mvn dependency:resolve

COPY src ./src
RUN --mount=type=cache,target=/root/.m2/repository \
    mvn package -DskipTests

NuGet (.NET)

COPY *.csproj ./
RUN --mount=type=cache,target=/root/.nuget/packages \
    dotnet restore

COPY . ./
RUN --mount=type=cache,target=/root/.nuget/packages \
    dotnet publish -c Release -o /app

Bind Mounts for Large Contexts

When source code is only needed to produce an artifact, use bind mounts instead of COPY to avoid persisting source files in any layer:

FROM golang:1.22 AS build
WORKDIR /src
RUN --mount=type=cache,target=/go/pkg/mod \
    --mount=type=bind,source=go.sum,target=go.sum \
    --mount=type=bind,source=go.mod,target=go.mod \
    go mod download

RUN --mount=type=cache,target=/go/pkg/mod \
    --mount=type=cache,target=/root/.cache/go-build \
    --mount=type=bind,target=. \
    go build -o /bin/app ./cmd

Advantages:

Mounted files are NOT persisted in any layer
Only the RUN output is kept in the image
Avoids bloating the build cache with source files
Bind mounts are read-only by default (safe)

CI/CD Cache Backends

Registry Cache (recommended for teams)

docker buildx build --push -t registry/app:latest \
  --cache-to type=registry,ref=registry/app:buildcache,mode=max \
  --cache-from type=registry,ref=registry/app:buildcache .

GitHub Actions Cache

- uses: docker/build-push-action@v7
  with:
    push: true
    tags: user/app:latest
    cache-from: type=gha
    cache-to: type=gha,mode=max

Multi-Branch Cache Strategy

docker buildx build --push -t registry/app:latest \
  --cache-to type=registry,ref=registry/app:cache:$BRANCH \
  --cache-from type=registry,ref=registry/app:cache:$BRANCH \
  --cache-from type=registry,ref=registry/app:cache:main .

ALWAYS fall back to the main branch cache when the feature branch cache misses.

Cache Modes

Mode	Behavior	Use Case
`min` (default)	Caches only exported layers	Smaller cache, faster export
`max`	Caches ALL layers including intermediates	More cache hits, larger storage

ALWAYS use mode=max in CI/CD to maximize cache reuse across builds.

Layer Squashing Considerations

Docker does NOT support true layer squashing natively. Options:

Approach	How	Trade-off
Multi-stage builds	Copy only final artifacts to clean stage	Best approach -- no extra tooling
`--squash` (experimental)	Merge all layers into one	Loses all intermediate cache
`docker export/import`	Flatten to single layer	Loses metadata, CMD, ENV, etc.

ALWAYS prefer multi-stage builds over squashing -- they preserve caching while producing minimal final images.

Forcing Cache Invalidation

# Invalidate ALL cache
docker build --no-cache .

# Invalidate a specific stage only
docker build --no-cache-filter install .

# Pull fresh base images
docker build --pull .

# Clear entire builder cache
docker builder prune

# Clear with size limit
docker builder prune --keep-storage 5GB

Reference Links

references/caching-rules.md -- Complete cache invalidation rules per instruction type
references/examples.md -- Optimized Dockerfiles before/after, .dockerignore patterns
references/anti-patterns.md -- Caching and optimization mistakes with explanations