Capstone: secure a service end to end

The target and the deliverables

Choose a service with the shape of something real: an HTTP API with authentication, a database, at least one secret (an API key or a database password), a container image, a pipeline and a deployed environment you control. The zero-to-production project supplies all of that. Keep a security/ directory in the repository for everything you produce, because the deliverable is as much the record as the configuration.

Phase 1: model, then decide

Run the one-hour threat model from the first module against the service as deployed, not as designed. Draw the data flow diagram with trust boundaries, list assets in rank order, enumerate entry points including the pipeline and the dependency manifest, walk STRIDE, and rate. Take the top eight threats and, for each, write the response and the specific control with a test name. This list is your work plan for the next phases; anything not on it is optional.

T-01 E1 Elevation: read another user's order by id            -> MITIGATE  scoped queries      test_orders_other_user_404
T-02 E1 Spoofing: forged JWT accepted                           -> MITIGATE  pinned alg, aud, exp test_jwt_wrong_alg_rejected
T-03 E4 Tampering: unsigned image deployed                      -> MITIGATE  cosign + admission   test_admission_rejects_unsigned
T-04 E5 Info disclosure: DB password in image layer             -> ELIMINATE runtime secret fetch test_image_has_no_secrets
T-05 E2 Spoofing: forged payment webhook                        -> MITIGATE  HMAC + timestamp     test_webhook_bad_sig_rejected
T-06 --  Elevation: pod escape via root + hostPath              -> MITIGATE  restricted PSS       test_pod_spec_restricted
T-07 --  Info disclosure: DB reachable from CI runners          -> MITIGATE  sg references + NP   test_db_unreachable_from_ci
T-08 --  Repudiation: admin deletes with no record              -> MITIGATE  security events      test_admin_delete_emits_event

Phase 2: harden and prove it

Work down the register. For each control, make the change and write the test that would fail if the control were removed. Application controls get unit or integration tests (tests/security/). Infrastructure controls get policy tests on the Terraform (checkov rules or a small script that parses the plan) and, where possible, a live probe (nc -zv from the wrong place must fail). Kubernetes controls get a conformance test: apply a deliberately bad manifest and assert rejection.

import json
import subprocess

IMAGE = "ghcr.io/acme/orders:test"


def inspect(image: str) -> dict:
    out = subprocess.run(["docker", "image", "inspect", image], capture_output=True, text=True, check=True).stdout
    return json.loads(out)[0]


def test_image_runs_as_non_root():
    user = inspect(IMAGE)["Config"].get("User", "")
    assert user not in ("", "0", "root"), f"image runs as root (User={user!r})"


def test_image_has_no_secret_shaped_env():
    env = inspect(IMAGE)["Config"].get("Env") or []
    bad = [e for e in env if any(k in e.upper() for k in ("PASSWORD", "SECRET", "TOKEN", "AKIA"))]
    assert not bad, f"secret-shaped environment in image: {bad}"


def test_image_layers_contain_no_private_keys():
    r = subprocess.run(["trivy", "image", "--scanners", "secret", "--exit-code", "1", "--quiet", IMAGE],
                       capture_output=True, text=True)
    assert r.returncode == 0, r.stdout

The minimum set for most services: SSO plus phishing-resistant MFA for anything administrative; authorisation scoped at the data layer; security group references and a default-deny NetworkPolicy; a hardened host image with auditd and automatic updates; a distroless non-root image under the restricted profile; secrets fetched at runtime by workload identity; TLS on every hop. Record each as a control with owner and evidence as you go — it costs minutes now and saves days later.

Phase 3: the pipeline as the only road

Extend the service's pipeline until nothing reaches production that has not been checked, inventoried, signed and verified. Two-tier gates on pull requests (secrets, diff-scoped SAST, dependency audit, IaC policy), image scanning by digest at build, SBOM generation and attestation, keyless signing, provenance, and an admission policy in the cluster (or an equivalent check in the deploy step for a non-Kubernetes target) that refuses unsigned images. Pin every action, minimise token permissions, and confirm fork pull requests receive no secrets.

# an image built outside the pipeline, pushed by hand, with no signature
docker build -t ghcr.io/acme/orders:rogue . && docker push ghcr.io/acme/orders:rogue
kubectl -n shop set image deployment/orders app=ghcr.io/acme/orders:rogue
# expected: admission webhook denies the change; the running pods are unchanged
kubectl -n shop get events --field-selector reason=PolicyViolation | tail -3

# the pipeline-built, signed image by digest is accepted
cosign verify --certificate-identity-regexp '^https://github.com/acme/orders/' \
  --certificate-oidc-issuer https://token.actions.githubusercontent.com \
  ghcr.io/acme/orders@sha256:9f86d081884c7d659a2feaa0c55ad015a3bf4f1b2b0b822cd15d6c15b0f00a08 >/dev/null && echo "signed by our workflow"

Phase 4: see it, then break in

Add structured security events to the application for authentication results, authorisation denials, admin actions and data exports. Ship them, the host auth and audit logs, and the cloud audit trail to central storage. Write at least three detections with runbooks: for example, five failed logins then a success from one address; an authorisation denial burst from one user (id enumeration); a shell spawned in an application container; a new IAM user or key created outside the pipeline's identity. Then test them: trigger each with a script or an emulation tool and record whether and how fast the alert arrived.

Now run the incident drill. Have a colleague (or a script you wrote earlier and forgot the details of) do one realistic thing: leak a test credential into a public paste, add an SSH key to the host, or exploit a deliberately re-introduced injection bug in a branch deployed to staging. Respond as the incident module describes: declare, roles, timeline, evidence, scope, contain, eradicate, recover. Time yourself. Write the postmortem with dwell time and time to containment, and the detection or control that would have caught it sooner.

#!/usr/bin/env bash
set -euo pipefail
START=$(date -u +%s)

# behaviour: five failed logins then a success from this machine
for i in 1 2 3 4 5; do curl -s -o /dev/null -X POST https://staging.example.com/login -d 'user=alice&pass=wrong'; done
curl -s -o /dev/null -X POST https://staging.example.com/login -d "user=alice&pass=$ALICE_PASS"

# poll the alert API (adapt to your SIEM) for up to 10 minutes
for _ in $(seq 1 60); do
  if curl -s "$ALERT_API/search?q=rule:brute-force-then-success&since=$START" | grep -q '"count":[1-9]'; then
    echo "alert fired after $(( $(date -u +%s) - START ))s"; exit 0
  fi
  sleep 10
done
echo "NO ALERT within 10 minutes"; exit 1

Phase 5: the evidence pack and the write-up

Finish by making the work legible to someone who was not there. The security/controls/ directory holds one record per control with its evidence script; run the scripts once and commit the dated output for the capstone period. security/README.md is the write-up: the service, the threat model summary, the controls and how each is tested, the pipeline guarantees, the detections and their measured latency, the drill's timeline and postmortem, and an honest list of what is not covered and why. This is the document you link from a CV or hand to an auditor; write it for that reader.

• Every claim in the write-up points at a file, a test or a dated evidence export.
• The threat register shows status for all eight threats, including any you accepted, with reasons.
• The postmortem names dwell time, time to containment and one concrete improvement you then implemented.
• The gaps section is real: nobody believes a service with no residual risk, and stating it shows judgement.

When it is done, you will have exercised every layer of this track on one system and seen how they interact: the threat model chose the controls, the controls produced the evidence, the pipeline enforced the integrity, the detections turned logs into alerts, the drill tested the response, and the record turned all of it into something you can show. That loop — model, harden, verify, detect, respond, record — is the job.