Skip to content

ARCH-INFRA-PROXMOX: Proxmox cluster wetopi7

The RKE2 platform and all its VMs run on a single Proxmox VE cluster named wetopi7, spread over the 8 OVH dedicated servers (see ovh-and-network.md). OpenTofu owns the VMs and the node VLAN/bridge shape through the Proxmox HTTPS API (ADR-0003 — OpenTofu and Ansible responsibility split).

This pill does not restate the tfvars. The authoritative declarative source is tofu/proxmox/environments/prod/ in the GitOps repo https://github.com/wetopi-platform/infrastructure. The committed terraform.tfvars.example documents the shape; the real terraform.tfvars is gitignored and mode 0600 (never in Git). The table below is a point-in-time restatement of that tfvars (verified 2026-08-31) so a new engineer can read the inventory without the ignored file; the tfvars always wins on any drift.

Cluster nodes

8 Proxmox nodes (= the 8 OVH servers), each with one 802.1Q trunk NIC carrying the vRack VLANs (vmbr3 WAN, vmbr4 node LAN, vmbr2 Ceph). Trunk devices (proxmox_nodes in tfvars, verified live via the Proxmox API):

Node Trunk NIC
ns31826925, ns31826929, ns3235424, ns3243421, ns3249468 enp10s0f1np1
ns3214108, ns3226931, ns3226939 enp2s0f1np1

Base template and datastores

Item Value Source
Base VM template vmssd, VMID 9000 on node ns3243421 template_vm_id = 9000, template_node_name = "ns3243421"
VM datastore vmssd (NVMe SSD) datastore_id per VM
Template download datastore local template_download_datastore_id = "local"
Guest cloud-init identity Ubuntu 24.04, first-boot SSH/network template + ssh_public_key_file

New VMs are cloned from template 9000 and then converged by cloud-init (first boot) and Ansible (guest config) — OpenTofu sets identity/placement and never touches guest OS state.

VM inventory (source: prod tfvars, verified 2026-08-31)

VMIDs 8001-8042. "Datastore" is the VM disk location; LAN IPs are on vmbr4/10.72.0.0/23; WAN IPs (where present) are on vmbr3.

Name VMID Node Role vCPU/RAM/Disk Datastore LAN IP WAN IP
wetopi-ops01 8001 ns3249468 operations (bastion) 2 / 4096 MB / 40 GB vmssd 10.72.0.10 92.222.245.91
rke-cp01 8011 ns3243421 control-plane 4 / 8192 MB / 50 GB local 10.72.0.21 —
rke-cp02 8012 ns3226939 control-plane 4 / 8192 MB / 50 GB local 10.72.0.22 —
rke-cp03 8013 ns3214108 control-plane 4 / 8192 MB / 50 GB local 10.72.0.23 —
rke-platform01 8021 ns31826929 platform worker 4 / 8192 MB / 60 GB vmssd 10.72.0.41 —
rke-platform02 8022 ns3226931 platform worker 4 / 8192 MB / 60 GB vmssd 10.72.0.42 —
rke-platform03 8023 ns31826925 platform worker 4 / 8192 MB / 60 GB vmssd 10.72.0.43 —
wetopi-natgw01 8031 ns3243421 nat-gateway (egress) 2 / 2048 MB / 20 GB vmssd 10.72.0.2 92.222.245.83
rke-wan01 8041 ns3235424 edge (WAN) 2 / 4096 MB / 20 GB vmssd 10.72.0.51 79.137.114.193
rke-wan02 8042 ns3226931 edge (WAN) 2 / 4096 MB / 20 GB vmssd 10.72.0.52 79.137.114.194

Roles and notes. wetopi-ops01 is the SSH bastion (bastion = true) — the only VM with a direct public address other than the edge/natgw; all other VMs are reached through it. The three control planes own the keepalived RKE2 API VIP 10.72.0.20 (ADR-0004). wetopi-natgw01 is the LAN's NAT egress (LAN→WAN forward policy only). rke-wan01/02 are the edge v1 nodes (ADR-0006); they were initially built as wetopi3-wan1/2 and renamed to rke-wan* on 2026-08-30 (VMIDs 8041/8042 unchanged).

Storage NICs (ADR-0007 — verified live 2026-08-31). The three platform workers additionally carry a third NIC on vmbr2/VLAN 2 (the Ceph storage network) with the operator-reserved addresses: rke-platform01=192.168.98.10/20, rke-platform02=192.168.98.11/20, rke-platform03=192.168.98.12/20 — no gateway, no default route (storage traffic is directly connected; see ceph.md). The control planes stay single-NIC in v1. This third NIC was the ADR-0007 mutating apply; it is present in the live VMs and the OpenTofu state (the base table above reflects the tfvars snapshot and lists only the LAN/WAN NICs).

flowchart TB
  subgraph cluster["Proxmox cluster wetopi7 — 8 nodes"]
    direction TB
    subgraph nA["ns3249468"]
      ops01["wetopi-ops01 · 8001 · bastion"]
    end
    subgraph nB["ns3243421"]
      cp01["rke-cp01 · 8011 · control-plane"]
      natgw["wetopi-natgw01 · 8031 · NAT egress"]
      tpl["base template vmssd · VMID 9000"]
    end
    subgraph nC["ns3226939"]
      cp02["rke-cp02 · 8012 · control-plane"]
    end
    subgraph nD["ns3214108"]
      cp03["rke-cp03 · 8013 · control-plane"]
    end
    subgraph nE["ns31826929"]
      p01["rke-platform01 · 8021 · worker (+Ceph NIC)"]
    end
    subgraph nF["ns3226931"]
      p02["rke-platform02 · 8022 · worker (+Ceph NIC)"]
      wan2["rke-wan02 · 8042 · edge"]
    end
    subgraph nG["ns31826925"]
      p03["rke-platform03 · 8023 · worker (+Ceph NIC)"]
    end
    subgraph nH["ns3235424"]
      wan1["rke-wan01 · 8041 · edge"]
    end
  end
  ops01 -.->|"SSH bastion to all VMs"| cp01

Takeover notes

  • Re-verify with tofu plan on tofu/proxmox/environments/prod (no-changes expected) and qemu-pending/qemu-status per VM via the Proxmox API or ops01.
  • The ops01 node ns3249468 also hosts the Proxmox API endpoint used by OpenTofu (PROXMOX_VE_ENDPOINT); see providers.md.
  • VMs must never be renumbered; VMID + node placement is stable identity. Sizing is deliberately minimal and expands after observation (ADR-0002 — Minimal expandable bootstrap sizing).

Live inventory (verified 2026-09-06, BookStack mirror — T1.6)

  • 8 nodes on OVH vRack; VLAN 4 / vmbr4 = node LAN 10.72.0.0/23.
  • Nodes: wetopi-ops01 (bastion 92.222.245.91 WAN / 10.72.0.10), rke-cp01/02/03 (10.72.0.21/.22/.23), rke-platform01/02/03 (.41/.42/.43), rke-wan01/02 (.51/.52 — hostNetwork edge), wetopi-natgw01 (10.72.0.2, WAN .83).