Container hardening

Container Hardening

When to use

This skill covers the container-image layer: what's inside, how it runs, and how you prove it checks out. It's the foundation k8s-security builds on (K8s takes these images and adds cluster-level controls).

Activates on:

• A request like "review our Dockerfile", "migrate to distroless", "why does our container run as root", "trivy scan triage", "sign images with cosign".
• A new or modified Dockerfile, Containerfile, docker-compose.yml, .dockerignore, multi-stage build script.
• An image-scan output (trivy/grype/snyk container) that needs triaging.
• A handoff from security-review phase 3 (container in scope) or from k8s-security (PodSecurityContext points to an image-level issue).
• A supply-chain moment: image needs to be signed, attestation published. Together with supply-chain.

When NOT to use (handoff)

• Kubernetes workload spec (PodSecurityContext, NetworkPolicy, RBAC) → k8s-security. The image is the ingredient; K8s is the cook.
• SBOM format and signing-keys setup → supply-chain. This skill calls sigstore; that one explains it.
• Vulnerabilities in packages inside the image → scanner output goes to cve-triage for triage.
• Secrets in image layers → secrets-scanner on image history.
• CI pipeline that runs the build → cicd-hardening.
• Pure code question that just happens to run in a container → secure-coding or the framework skill.

Approach

Six phases. Phases 1–3 are image content, phase 4 is signing, phase 5 is runtime, phase 6 is verification.

1. Base-image choice

The base image is 80% of your attack surface. Choose deliberately.

Options, smallest to largest:

• scratch (0 layers). Only for statically-linked binaries. Go, Rust, sometimes C. Smallest attack surface, no shell, no debug tools. Default for statically linked artifacts.
• Distroless (Google). Minimal runtime for a language (Java, Python, Node), no shell, no package manager. gcr.io/distroless/java21-debian12, gcr.io/distroless/python3-debian12, etc. With a :debug variant for troubleshooting.
• Chainguard Images / Wolfi. Glibc-free, continually rebuilt, SLSA-L3-signed. Comparable to distroless, with a demonstrably smaller CVE window thanks to daily builds.
• Alpine (musl-libc). Small (~5 MB), package manager (apk) present. Caveat: musl can produce subtle differences vs glibc (DNS resolution, thread-locals).
• Debian slim / Ubuntu minimal. Larger footprint but broader compatibility.
• <language>:latest (e.g. node:latest, python:latest). Don't use. latest is unreproducible, and these tags carry far more than needed.

Rules:

• Pin on digest, not on tag. FROM gcr.io/distroless/python3-debian12@sha256:<digest>. A tag can mutate under your feet.
• Document update cadence. Base images get CVEs; you have to rebuild regularly. Renovate or Dependabot for Dockerfile dep bumps.
• One image = one responsibility. Not one mega-image with app + migration + CLI tools; separate images with a shared base.

2. Dockerfile hygiene

The instructions that shape image content. Each of these is a classic foot-gun.

• USER non-root. The default Docker user is root (UID 0). Explicit USER app or USER 10001 on a dedicated UID. Without USER: every process in the container is root in the container, and on container escape it's root on the host.
• Read-only filesystem where possible. Via docker run --read-only or Kubernetes readOnlyRootFilesystem: true. Limit write needs to explicit tmpfs or volume mounts. Exposure: if an attacker gets execution, they can't drop malware.
• Drop capabilities. Default Docker caps (NET_ADMIN, SYS_ADMIN selectively) are too broad. --cap-drop=ALL plus explicit additions for what you need (e.g. --cap-add=NET_BIND_SERVICE for ports < 1024). In Kubernetes: securityContext.capabilities.
• Multi-stage builds. Build stage with compilers and dep installers, runtime stage minimal. Prevents gcc, make, git from ending up in the production image. See Docker docs for syntax.
• .dockerignore. Prevents .env, .git, node_modules, test fixtures from landing in the image via COPY . .. Without .dockerignore your container has the whole source plus any local secrets.
• No secrets in layers. ENV PASSWORD=..., ARG SECRET=..., or COPY .env . commits secrets into image history. Even after a later layer-delete the secret is still there. Use BuildKit secrets (RUN --mount=type=secret,id=...) or runtime injection via K8s secret / Vault.
• Layer order for caching and security. Install dependencies first (rarely change), copy code later (frequently changes). Cache efficiency plus forced rebuild on dep change.
• Apt/apk install cleanup. apt-get install --no-install-recommends plus rm -rf /var/lib/apt/lists/* in the same RUN. Otherwise package lists stay in the layer.
• Add a HEALTHCHECK (when not via K8s liveness/readiness). Lets the orchestrator know the container is hung.
• Document EXPOSE, don't publish. EXPOSE 8080 is documentation. Publishing happens at docker run -p or via a K8s Service.

Anti-pattern:

FROM node:latest                    # latest tag, unreproducible
COPY . /app                         # including .env, .git, tests
RUN npm install --unsafe-perm       # suggests root permission need
USER root                           # explicitly root
CMD ["npm", "start"]

Better variant:

FROM node:20-bookworm-slim@sha256:<digest> AS build
WORKDIR /app
COPY package*.json ./
RUN npm ci --omit=dev
COPY src ./src

FROM gcr.io/distroless/nodejs20-debian12@sha256:<digest>
WORKDIR /app
COPY --from=build /app /app
USER 10001
CMD ["src/index.js"]

3. Build-time scanning

Before the image hits the registry:

• Trivy (Aqua, Apache-2). Scans images for OS and app dep-vulns, secrets, misconfigs. Fastest in adoption.

  trivy image --severity HIGH,CRITICAL --exit-code 1 <image>:<tag>
  

• Grype (Anchore, Apache-2). Vuln scanner combined with Syft for SBOM. Detailed per-package CVEs.
• Clair (Red Hat OSS). Server-side scanning, integrates with Harbor.
• Snyk container / Sysdig Secure (commercial). Enterprise options with broader feed sources and UI.
• Docker Scout (Docker Inc). Default in Docker Desktop since 2023; comparable to Trivy for the use case.

Tie output to cve-triage. All high-severity findings face that skill's blocker criteria (KEV, reachable, exposed). Don't auto-block on every HIGH; you drown in low-impact base-image noise.

Scan configurations too: Trivy has --scanners misconfig for Dockerfile linting (overlapping with phase 2's rules).

4. Image signing and provenance

See supply-chain for the full explanation; here are the container-specific calls.

• Sign at build (not after). Sigstore + cosign, keyless via OIDC.

  cosign sign --yes <registry>/<image>@sha256:<digest>
  

• Attest the SBOM on the same image.

  syft <image>:<tag> -o cyclonedx-json > sbom.json
  cosign attest --yes --predicate sbom.json --type cyclonedx <image>@sha256:<digest>
  

• SLSA provenance via slsa-framework/slsa-github-generator for GitHub Actions builds.
• Verify on pull, not just on push. A Kubernetes admission controller (Kyverno, Sigstore policy-controller) that rejects unsigned images. Reference: k8s-security phase 1.

Registry choice:

• Harbor (OSS) supports signing, vuln scanning, image replication.
• Cloud registries (ECR, GAR, ACR) with built-in scanning.
• GHCR for GitHub-attached builds.

5. Runtime hardening

The image is only half. Runtime policy decides what the container is allowed to do once it runs.

• Seccomp — filter system calls. Docker's default profile is a good starting point; custom profiles for specific workloads (nginx, PostgreSQL) can be sourced from the moby/moby repo or via tools like dockerd-rootless-setuptool.
• AppArmor / SELinux — mandatory access control. Default AppArmor profiles on Ubuntu/Debian, SELinux on RHEL/Fedora. Custom profiles via --security-opt apparmor=profile-name.
• Rootless Docker / Podman — daemon runs as a non-root user; a container escape doesn't reach host-root. Default in Podman, opt-in in Docker.
• No new privileges — --security-opt=no-new-privileges:true prevents setuid escalation inside the container.
• gVisor / Kata Containers — kernel isolation for higher trust levels. Overhead ~5–20%; pays off for multi-tenant workloads or sensitive data.

Ephemeral runtime: containers are stateless. State in volumes, not in writable layers. --rm for one-off runs, emptyDir or persistent-volume in K8s.

6. Verification-loop

Layer 1: scope (every Dockerfile in the repo covered? base images pinned by digest? USER non-root in every stage?), assumptions ("distroless so safe" only with the concrete image hash verified), gaps (both build-time and runtime controls present?), consistency (trivy severity threshold consistent across images?).

Layer 2: image-tag and digest syntax correct, CVE references in scan triage verified against NVD, SLSA-level claim backed by the concrete builder config, no fabricated seccomp-profile names.

Output

Container hardening review — <image(s)>
Dockerfiles in scope: <list>
Base images: <used, pinned by digest: yes/no>

Image content:
  USER non-root:            <yes/no per image>
  Multi-stage build:        <yes/no>
  Read-only compatible:     <yes/no, and which write paths>
  .dockerignore present:    <yes/no>
  Secrets in layers:        <none | found: ...>

Scan output (trivy/grype):
  Blockers (HIGH/CRITICAL): <N, with cve-triage handoff>
  Medium:                   <N>
  Misconfigs:               <list>

Signing + provenance:
  Signed:                   <yes/no, cosign identity>
  SBOM attested:            <yes/no, format>
  SLSA level effective:     <L0–L3>

Runtime policy:
  Seccomp:                  <default | custom | none>
  AppArmor/SELinux:         <profile | none>
  Capabilities dropped:     <ALL + adds | default>
  Rootless runtime:         <yes/no>

Findings (severity-sorted)

Verification-loop: ...

References

• Docker security docs — https://docs.docker.com/engine/security/.
• OCI Image Spec — https://github.com/opencontainers/image-spec. Primary image-format spec.
• NIST SP 800-190 — https://csrc.nist.gov/pubs/sp/800/190/final. Application Container Security Guide.
• CIS Docker Benchmark — https://www.cisecurity.org/benchmark/docker. Checklist form, good for audits.
• Docker Bench for Security — https://github.com/docker/docker-bench-security. Scripted CIS-Docker check.
• Distroless images — https://github.com/GoogleContainerTools/distroless.
• Chainguard / Wolfi — https://edu.chainguard.dev/chainguard/chainguard-images/. Continually rebuilt minimal images.
• Trivy — https://trivy.dev/. Multi-purpose scanner.
• Grype + Syft — https://github.com/anchore/grype. Vuln scanner plus SBOM.
• Sigstore cosign — https://docs.sigstore.dev/cosign/overview/. Image signing without long-lived keys.
• Seccomp profiles — https://docs.docker.com/engine/security/seccomp/.
• OWASP Docker Security Cheat Sheet — https://cheatsheetseries.owasp.org/cheatsheets/Docker_Security_Cheat_Sheet.html.

Categories

• appsec