Skip to content

cert-manager (ACME / Let's Encrypt)

What is it?

cert-manager is the cluster's ACME client: it issues and automatically renews the TLS certificates that the edge Gateway terminates on. DNS-01 challenges are answered through the Cloudflare API, so issuance is port-mapping-immune (no open 80/8080 HTTP-01 surface on the edge). It is the service that turned the F2 bootstrap self-signed cert into real LE certificates for every public hostname.

How is it mounted?

  • Namespace: cert-manager (Git-declared namespace.yaml, single creator — sync runs with CreateNamespace=false).
  • Version pin: Helm chart cert-manager v1.21.1 from oci://quay.io/jetstack/charts (OCI is the source of truth; the legacy index lags). appVersion v1.21.1 — controller/webhook/cainjector/acmesolver are one released set; no per-image tag overrides. Cluster requirement kubeVersion >= 1.22.0-0 satisfied by RKE2 v1.36.3 (K8s 1.36).
  • Objects (one Argo-owned unit, multi-source Application cert-manager): source 1 = the pinned chart (values from kubernetes/cert-manager/values.yaml via $values; crds.enabled: true so the CRDs are part of the release and owned/pruned by Argo); source 2 = this repo at kubernetes/cert-manager rendering EXACTLY the Namespace, both ClusterIssuers and the staging Certificate (include/exclude filter documented in platform/cert-manager.yaml). The Application is anchored in the root path (kubernetes/platform/cert-manager.yaml) and carries the cascading-deletion finalizer — a git revert of the anchoring merge rolls the full F3 deployment back as a foreground cascade.
  • Issuers: letsencrypt-staging and letsencrypt-prod (ClusterIssuers, DNS-01 Cloudflare solver). Both reference the SAME operator Secret cloudflare-dns-api-token (key api-token) in namespace cert-manager — BY NAME ONLY; the token itself is operator material (0600 env file on ops01, shared with OpenTofu's cloudflare provider), never in Git.
  • Certificates (all in kube-system — the Gateway's namespace, so the listener certificateRefs are same-namespace, no ReferenceGrant): edge-test-tls (staging, the F3 acceptance object), argo-tls, docs-tls, vault-tls (all LE PROD, single-name, no wildcards).

Diagram

flowchart TD
  CF["Cloudflare API (wetopi.net zone: DNS-Edit + Zone-Read)"]
  subgraph CM["namespace cert-manager"]
    CTL["cert-manager controller v1.21.1 (+ webhook, cainjector)"]
    ISP["ClusterIssuer letsencrypt-prod"]
    ISS["ClusterIssuer letsencrypt-staging"]
    TOK["Secret cloudflare-dns-api-token (operator)"]
  end
  subgraph KSYS["kube-system (Gateway namespace)"]
    CE1["Certificate edge-test-tls (staging)"]
    CA["Certificate argo-tls (PROD)"]
    CD["Certificate docs-tls (PROD)"]
    CV["Certificate vault-tls (PROD)"]
    GW["Gateway wetopi-edge https listener certificateRefs"]
  end
  LE["Let's Encrypt ACME (staging + prod)"]
  CF -->|"_acme-challenge TXT"| CTL
  CTL -->|ACME orders| LE
  ISP --> CTL
  ISS --> CTL
  TOK --> ISP
  TOK --> ISS
  CE1 --> CA & CD & CV
  CA & CD & CV --> GW

How is it deployed?

GitOps: Application cert-manager (anchored in kubernetes/platform/cert-manager.yaml, synced by the root app) — prune + selfHeal. One-time bootstrap from ops01 (runbook: kubernetes/cert-manager/README.md): apply namespace.yaml by hand → create the cloudflare-dns-api-token Secret from the ops01 0600 env file (kubectl create secret ... --from-literal, shape-only verification) → the root app creates the Application and syncs. Prod cutover gate: nothing may reference letsencrypt-prod until the F3 staging gate (edge-test certificate issued + full chain verified) is signed off — LE prod rate limits burn real budget. Upgrades = a reviewed PR changing targetRevision + values.yaml together.

Dependencies

Depends on: the Cloudflare API token (Zone - DNS - Edit AND Zone - Zone - Read on wetopi.net) — operator-managed, ops01-only canonical file; the edge Gateway as the certificate consumer (SVC-TRAEFIK-EDGE); DNS records for the issued names (tofu-owned, so the _acme-challenge zone exists); the root Argo Application (SVC-ARGOCD) for lifecycle.

Depended on by: every public hostname on the edge — argo-tls (argo.wetopi.net), docs-tls (docs.wetopi.net), vault-tls (vault.wetopi.net), edge-test-tls (F3 acceptance). Without renewals, the whole https listener degrades at 90-day boundaries.

Operation

  • Health: kubectl -n cert-manager get pods (controller/webhook/ cainjector Ready); kubectl -n kube-system get certificates — all four Ready=True; controller events show ACME order → challenge → issue chain. Expiry: kubectl get certificate <name> -o jsonpath='{.status.notAfter}'.
  • Solver troubleshooting: a zone-discovery failure means the token lacks Zone-Read (the v1.21 solver always resolves the zone first — see README "Solver troubleshooting" + the read-only token probe).
  • Rollback of a name: delete the certificateRef entry from the Gateway (git revert); the Secret stays and keeps renewing unless the Certificate is reverted too.
  • Runbook: infra repo kubernetes/cert-manager/README.md (ownership, token provisioning, prod cutover gate).

Estado honesto (live, 2026-08-31)

DEPLOYED. Live since the F3 apply: controller/webhook/cainjector Running, both issuers configured. Certificates live-verified 2026-08-31 — 4/4 Ready=True: edge-test-tls (staging), argo-tls, docs-tls, vault-tls (LE PROD). Renewals run automatically (renewBefore default 30 days for 90-day ACME certs).

Live inventory (verified 2026-09-06, BookStack mirror — T1.6)

  • 12 certs activos en kube-system, todos Ready (verificación: kubectl -n kube-system get certificates).
  • Certs por host en kube-system (mismo namespace del Gateway, sin ReferenceGrant) — verificación live 2026-09-06.