Container build and hardening

Container Build and Hardening

Overview

A container image is a deployment artifact and an attack surface. Every image must be small, reproducible, scanned, and owned. The Dockerfile is code: reviewed, linted, and versioned like everything else.

Build once, promote everywhere. The exact image (by digest) that passed tests in staging is the one that runs in production. Never rebuild per environment — a rebuild is a different artifact with different risk.

The image is not the place for config or secrets. Environment-specific values arrive at runtime through env vars, mounted config, or a secrets manager.

When to Use

• Writing or reviewing a Dockerfile
• Containerizing an application for the first time
• Setting up docker-compose for local development or small deployments
• Publishing images to a registry (GHCR, GitLab CR, ACR, ECR, Harbor)
• Reducing image size or CVE count
• Reviewing base image choices

The Reference Dockerfile (multi-stage)

# syntax=docker/dockerfile:1

########## Build stage ##########
FROM python:3.13-slim AS builder
WORKDIR /app

# Dependency layer cached separately from source layer:
# copy manifests first so code changes don't invalidate the pip cache
COPY pyproject.toml uv.lock ./
RUN pip install --no-cache-dir --prefix=/install .

COPY src/ ./src/

########## Runtime stage ##########
FROM python:3.13-slim AS runtime

# Non-root user, created explicitly
RUN groupadd --system app && useradd --system --gid app --no-create-home app

WORKDIR /app
COPY --from=builder /install /usr/local
COPY --from=builder /app/src ./src

USER app
EXPOSE 8080

# OCI labels: provenance matters when you're debugging at 3 AM
LABEL org.opencontainers.image.source="https://github.com/org/repo" \
      org.opencontainers.image.licenses="MIT"

HEALTHCHECK --interval=30s --timeout=3s --start-period=10s --retries=3 \
  CMD ["python", "-c", "import urllib.request; urllib.request.urlopen('http://localhost:8080/health')"]

CMD ["python", "-m", "app"]

Why each choice matters:

Choice	Reason
Multi-stage build	Build tools (compilers, pip caches) never reach the runtime image — smaller, fewer CVEs
Pinned base tag (python:3.13-slim, not python:latest)	Reproducibility; latest is a moving target that breaks builds silently
slim / alpine / distroless base	Smaller attack surface; fewer packages, fewer vulnerabilities
Dependency manifests copied before source	Layer caching: code changes don't reinstall dependencies
Non-root USER	A container escape as root is a host compromise; as app it's a much smaller problem
.dockerignore	Small contexts build faster and can't leak .git, .env, or node_modules into layers
HEALTHCHECK / orchestrator probes	The platform can't route around a zombie process it can't detect

The .dockerignore (mandatory)

.git
.gitignore
.env
.env.*
**/__pycache__
**/node_modules
**/*.log
**/target
Dockerfile
docker-compose*.yml

If your build context includes .env, that file's contents are recoverable from image layers (docker history, dive). .dockerignore is a security control, not a convenience.

Build and Scan Workflow

# 1. Lint the Dockerfile before building
hadolint Dockerfile

# 2. Build with BuildKit (default on modern Docker) — cache-friendly, secret-safe
docker buildx build --tag registry.example.com/team/app:1.4.2 \
  --label org.opencontainers.image.revision="$(git rev-parse HEAD)" \
  .

# 3. Inspect what you built
docker images registry.example.com/team/app:1.4.2        # size sanity check
dive registry.example.com/team/app:1.4.2                 # layer-by-layer waste and leaks

# 4. Scan BEFORE pushing — a vulnerable image must never reach the registry
trivy image --severity HIGH,CRITICAL --exit-code 1 registry.example.com/team/app:1.4.2

# 5. Tag immutably and push — version tag, never latest-only
docker push registry.example.com/team/app:1.4.2

# 6. Record the digest — the digest is the deployable identity
docker inspect --format='{{index .RepoDigests 0}}' registry.example.com/team/app:1.4.2

Secrets at build time: never ARG PASSWORD=... (visible in docker history). Use BuildKit secret mounts:

# syntax=docker/dockerfile:1
RUN --mount=type=secret,id=npmrc,target=/root/.npmrc npm ci

docker buildx build --secret id=npmrc,src=$HOME/.npmrc .

Docker Compose (local dev / small deployments)

services:
  app:
    image: registry.example.com/team/app:1.4.2   # pinned version, built by CI
    build: .                                      # local override only
    ports: ["8080:8080"]
    environment:
      APP_ENV: development
    depends_on:
      db:
        condition: service_healthy
    healthcheck:
      test: ["CMD", "curl", "-fsS", "http://localhost:8080/health"]
      interval: 10s
      timeout: 3s
      retries: 5
    read_only: true                               # hardening where possible
    tmpfs: ["/tmp"]
    security_opt: ["no-new-privileges:true"]
    cap_drop: ["ALL"]

  db:
    image: postgres:16
    volumes:
      - dbdata:/var/lib/postgresql/data
    healthcheck:
      test: ["CMD-SHELL", "pg_isready -U postgres"]
      interval: 5s
      retries: 10

volumes:
  dbdata:

Compose rules:

• Healthchecks on every dependency (depends_on: condition: service_healthy), or startup order is a race condition.
• Version-pinned images, never latest.
• Local bind mounts for code in dev; named volumes for data.
• Compose is for dev and small single-host deployments. It is not a production orchestrator — that's Kubernetes territory (see kubernetes-application-delivery).

Registry Discipline

1. One registry of record per organization (GHCR, GitLab CR, ACR, ECR, or Harbor).
2. Immutable tags. A pushed 1.4.2 is never overwritten. If your registry supports tag immutability, enable it.
3. Deploy by digest in production manifests — app@sha256:... — so what you tested is bit-for-bit what runs.
4. Retention policy cleans untagged/old images; storage costs and CVE noise grow otherwise.
5. Access control: CI pushes with a least-privilege identity; humans pull; nobody pushes manually.

Base Image Maintenance

• Base images are dependencies: track and update them deliberately (Renovate/Dependabot can bump Dockerfile FROM lines).
• Updating a base image means rebuilding, re-scanning, and re-testing — treat it like any dependency upgrade, not a blind docker pull in production.
• Know where your base comes from: official Docker Hub images, verified publishers, or your own hardened base built with Packer/Dockerfile. Random community images are a supply-chain risk.

Common Rationalizations

Rationalization	Reality
"Running as root is fine, it's isolated"	Containers share the host kernel. Root in the container + a kernel bug or misconfigured mount = root on the host. USER app is one line.
"latest keeps us up to date automatically"	latest means you don't know what's running, can't roll back, and can't reproduce. Version tags and digests are the only deployable identities.
"The image is big but disk is cheap"	Big images deploy slowly, autoscale slowly, and carry hundreds of unpatched packages. Size is an operational and security property, not aesthetics.
"We'll scan in production later"	The registry and runtime scanners catch what got through; the pipeline scan is what prevents it. Scan before push with --exit-code 1.
"The .env in the image is fine, the repo is private"	Image layers get pulled to laptops, CI runners, and registries you forgot about. Secrets never go into images. Ever.
"Multi-stage is overkill for this app"	The compiler/toolchain you ship is the attacker's post-exploitation toolkit. Multi-stage is five lines of Dockerfile.

Red Flags

• FROM ...:latest or unpinned FROM
• No USER directive (running as root)
• ADD used for remote URLs or where COPY suffices
• .env, credentials, or SSH keys visible in docker history
• No .dockerignore and a 2 GB build context
• Images pushed without a scan gate
• privileged: true, --privileged, or network_mode: host without a documented, justified reason
• Healthchecks missing on every long-running service

Verification

• hadolint Dockerfile passes clean
• Image builds reproducibly from a clean checkout (docker buildx build --no-cache)
• Final image runs as a non-root user: docker run --rm <img> id shows uid ≠ 0
• trivy image --severity HIGH,CRITICAL returns zero findings (or documented, accepted exceptions)
• docker history shows no secrets in any layer
• Image tagged with an immutable version; digest recorded for deployment
• Healthcheck defined and observed working (docker inspect --format='{{json .State.Health}}')
• Same image digest promoted across environments — no per-env rebuilds