Skip to content

WAN edge v1 — direct Traefik exposure without L4 layer

Purpose

How the platform publishes services to the Internet in the v1 window: the bounded exception that lets the first public services run without the full L4 edge stack (VRRP VIP + IPVS + DSR) that the master plan requires for production. Decision record: ADR-0006 (accepted 2026-08-30, with the 2026-08-30 UPDATE adding argo.wetopi.net to the service list).

The v1 mechanism

  • Exactly two edge nodes: rke-wan01 (79.137.114.193) and rke-wan02 (79.137.114.194), range 79.137.114.192/27. NIC1 = WAN (vmbr3, eth0), NIC2 = node-LAN (vmbr4, eth1) — the reverse of the single-NIC nodes; the edge_lb Ansible role owns the interface contract. These are also cluster workers: wetopi.io/role=edge (rke2-cluster.md).
  • Traefik directly on the WAN: the bundled rke2-traefik chart (HelmChartConfig rke2-traefik, hostNetwork: true) runs on edge nodes only and binds the chart-native entryPoints 8000 (web) / 8443 (websecure) in the host netns. Public 80/443 is bridged by in-VM nftables REDIRECT 80→8000, 443→8443 (the edge_lb role is the single enforcement point).
  • HA = DNS round-robin over the two public A records (TTL 300, grey-cloud, OpenTofu-owned). Reduced availability by design: there is no L4 health check or automatic failover. A dead node's A record keeps resolving until the operator removes/repoints it (tofu PR or emergency Cloudflare edit); TTL only bounds propagation of that manual remediation. Per-address uptime monitoring is part of the v1 posture.
  • nftables posture: in-VM nftables deny-by-default; the WAN interface exposes 80/443 ONLY (the role asserts the port set is a subset of {80,443}; anything else needs a recorded operator decision). The LAN interface carries only the cluster matrix (UDP 8472, TCP 4244/10250) and rate-limited management SSH. Pod/Service CIDR routes are pinned to the LAN gateway so cluster traffic never uses the WAN default.

Gateway API surface

Gateway API (v1) on the Traefik Gateway controller:

Object Namespace Role
GatewayClass wetopi-edge cluster-scoped controllerName: traefik.io/gateway-controller (chart provider enabled)
Gateway wetopi-edge kube-system listeners web :8000 (HTTP), https :8443 (Terminate); https allowedRoutes.namespaces.from: All to admit cross-namespace routes
HTTPRoute argo (+ argo-http-redirect) kube-system argo.wetopi.net → argocd/argocd-server:80 behind Middleware argo-basic-auth (ReferenceGrant in argocd); port 80 route 308-redirects to https
HTTPRoute docs-site docs-site docs.wetopi.net → docs-site Deployment (basic-auth Middleware), cross-namespace attachment via the from: All widening
HTTPRoute vault vault vault.wetopi.net → Infisical Web UI; no basic-auth Middleware — the UI's own login+TOTP is the gate (operator decision)
Certificates argo-tls, docs-tls, vault-tls kube-system Let's Encrypt PROD, DNS-01 via cert-manager; all four certs (incl. staging edge-test-tls) Ready live 2026-08-31
Secret wetopi-edge-default-cert kube-system bootstrap self-signed fallback cert (superseded per-name by the LE certs)

CertificateRefs are additive and fail-safe ordered: a missing referenced Secret would make the whole https listener ResolvedRefs=False; every cert addition therefore follows "Secret Ready first, Gateway after".

flowchart LR
    I["internet client"] -->|A x2 TTL 300| D["DNS round-robin<br/>79.137.114.193 / .194"]
    D --> W1["rke-wan01<br/>nftables 80/443 REDIRECT → 8000/8443"]
    D --> W2["rke-wan02<br/>nftables 80/443 REDIRECT → 8000/8443"]
    W1 & W2 --> T["Traefik (rke2-traefik, hostNetwork)<br/>entryPoints web :8000 / https :8443"]
    T --> G["Gateway wetopi-edge (kube-system)<br/>https listener :8443 — TLS Terminate"]
    G -->|HTTPRoute argo| M1["Middleware argo-basic-auth"] --> S1["argocd/argocd-server:80"]
    G -->|HTTPRoute docs-site| S2["docs-site Deployment :80"]
    G -->|HTTPRoute vault| S3["Infisical Web UI (vault ns)"]

Service list and its hard bounds

The v1 exception covers exactly docs.wetopi.net + argo.wetopi.net (both recorded operator decisions; each further name is its own decision). vault.wetopi.net was added 2026-08-31 for the in-cluster Infisical Web UI (OBJ-VAULT-001 / ADR-0009) through the same edge machinery — see its ADR for the recorded decision.

The ADR's hard conditions:

  1. Any service with external paying customers requires the v2 edge (VRRP public VIP + IPVS directors + DSR, with the DSR PoC checklist done) — no exceptions. The v2 track is scheduled BEFORE the hosting pilot phase (master plan §17/§33 Fase 16), not an aspiration.
  2. Reduced-availability semantics are documented here and monitored (independent per-address uptime checks).
  3. Cluster-plane identity/traffic of edge nodes is LAN-only (ADR-0006 condition 4): node IP 10.72.0.5x, kubelet, Cilium/VXLAN on eth1.

GitOps ownership

Edge objects are split: the Gateway/GatewayClass are live-verified objects in kubernetes/edge/ (F2 landing; the Gateway carries a strict certificateRefs coordination note), and the per-service routes/certs live in kubernetes/edge-argo/ (Application argo-edge) and the service directories (docs-site/, vault/). See gitops.md for the app inventory.

Invariants

  • WAN ingress = {80, 443} only; asserted at render time by edge_lb.
  • The public 80/443 contract exists ONLY through the nftables REDIRECT; the Gateway never binds 80/443 (entryPoint mismatch would leave it Accepted=False).
  • TLS terminates at the edge Gateway; plaintext 80 routes exist only as 308→https redirects.
  • Cluster CIDRs route over the LAN gateway on edge nodes (never the WAN default).
  • v1 service list is closed: additions require a recorded operator decision; customer hosting requires v2.

Verified state (2026-08-31, live from ops01)

One Gateway wetopi-edge (kube-system) Accepted=True + Programmed=True (plus the wetopi-edge GatewayClass); 3 HTTPRoutes (docs-site, argo, vault) attached; 4 Certificates Ready (docs-tls, argo-tls, vault-tls PROD + edge-test-tls staging). Dated observation — re-read kubectl -n kube-system get gateway,httproute and kubectl get certificate -A before relying on it.

Failure considerations

  • One edge node dead: ~half of new connections fail until operator remediation (no L4 health check) — the documented v1 failure mode; monitoring probes both A records independently.
  • Missing/revoked LE cert: the affected hostname degrades at renewal failure; cert-manager auto-renews (DNS-01, port-mapping-immune).
  • Unhealthy but alive node: not detected (no health check) — same remediation path; the v2 track eliminates this class.

Live inventory (verified 2026-09-06, BookStack mirror — T1.6)

  • Gateway wetopi-edge (kube-system): listener web:8000 HTTP + websecure:8443 HTTPS (nftables edge redirige 80→8000, 443→8443); allowedRoutes from: All en https.
  • TLS Terminate; certRefs a Secrets kube-system (12 activos: default + argo, book, dashboards, docs, grafana, infer, memory, otlp, prometheus, vault, alertmanager … y edge-test self-signed).
  • Fail-safe order: Certificate Ready=True ANTES del certRef (un Secret ausente tumba el listener completo para co-tenants).
  • Allowlist pattern: Middleware ipAllowList twin por ns (.87 + .89), ExtensionRef en la route.
  • Ley 4: el middleware twin va en el MISMO PR/directorio que la route que lo referencia.
  • ExtensionRef unresolvable = fail-closed (404/500 para toda la ruta, Argo verde igual).

Repo: platform-edge.