Caddy from the official package source, not from an image: an architecture decision for my Podman platform
Your services platform runs on rootless Podman, and you need a public entry point to terminate TLS and handle ACME. The question is where to put it. I choose to keep Caddy outside Podman, as a host service installed from Caddy’s signed official APT repository.
Caddy in Podman?
No. The boundary is simple: scripts/deploy.sh runs rootless as podman, and it must never acquire root authority over a public TLS service. If Caddy lived in Podman, the rootless application deployment could, by construction, rewrite the configuration of the public entry point. That is exactly the privilege confusion I want to avoid.
Caddy therefore stays outside Podman, outside deploy.sh, and the change procedure is an explicit administrator runbook, not a repository interface. The repository cannot establish host-package provenance, publish a root-owned Caddyfile, or reload a host service from its rootless deployment command. The administrator is the execution authority for this change.
Which Caddy package to install?
The Caddy package in Debian 13 stable is at 2.6.2. Upstream is at 2.11.4. 5 minor versions behind on a service that terminates TLS and handles ACME is too much: upstream fixes arrive late, and ACME keeps evolving.
Backports (2.11.2) improve freshness while keeping Debian packaging, but still lag. I keep them as an explicit fallback, not the default.
Manual binary download weakens provenance, upgrade consistency, systemd integration, rollback, and inventory compared with a signed APT package.
I therefore chose to use Caddy’s official APT repository, stable channel only, with independent verification of the signing key fingerprint before trust. Never testing, never unstable, never an unverified binary.
Automate the procedure?
The initial proposal was ambitious: a separate root-controlled wrapper, a local harness, evidence schemas, a correlation lifecycle, a mutation-testing gate, and a separate root-controlled codebase. All of this to cross the rootless to root boundary in an audited way.
I rejected it as disproportionate for the current phase. The host prerequisites are not available, there is no real runtime evidence to correlate yet, and the implementation and maintenance cost of an automated enforcement system brought nothing concrete now. The administrator runbook is the deliberate KISS alternative: procedural, documented, not automated.
This is not a promise to implement those controls later. It is an acknowledged scope reduction.
What the runbook enforces
Before any change, I explicitly approve 2 things: the exact repository commit containing the Caddyfile, and the exact Caddy package version with its source. Then, step by step, stopping at the first failure:
- Verify the approved Caddyfile from the canonical checkout, never running the checkout as root.
- Confirm the APT origin, key fingerprint, architecture, and exact version.
- Ensure
/etc/caddy/Caddyfileis a regular file (not a symlink), with a timestamped backup. - Stage the candidate as
root:caddy0640, validate withcaddy validate. - Atomic publication where practical (same-filesystem rename), never a blind write through a symlink.
- Re-validate the published file, then
systemctl reload caddy: reload, never an implicit restart. - Validate runtime separately: service state,
:80/:443listeners, DNS/TLS, public proxy, application health.
Unattended upgrades must not implicitly reload Caddy. The previous package and config stay available for an explicit rollback. A failed validation prevents publication. A failed runtime check is reported as a failure, never silently converted to success.
What this does not prove
The runbook is procedural, not automated enforcement. Its command output is not a claim that the host or public runtime has been validated. Until the administrator has performed the procedure and captured acceptable evidence, the runtime status remains BLOCKED. Runtime checks do not turn static inspection into runtime evidence.
What I take from it
The host/rootless boundary is non-negotiable, but enforcement is. A documented runbook, executed by a human who reviews every privileged command, is worth more than a premature automated system whose prerequisites do not yet exist.