# Public Endpoint, Peer Identity, And Raft Transport ## Status Accepted target architecture. Ordinary peer status checks already support an external HTTPS worker URL. Raft currently uses a separate plaintext listener, static peers, and shared Basic authentication; it does not yet meet this plan. Milestone 1 (worker side) is implemented: the worker configures `PUBLIC_URL` as the canonical advertised origin with the legacy `WORKER_URL` accepted only for a bounded migration, and validates the origin shape at startup and install time. On the wire the worker consumes the accepted endpoint from `wire.WorkerInit` (`public_url`, falling back to the legacy `url` field) and rejects unusable values. Transmitting the proposed URL during registration is the pending RSMon control-plane counterpart: it must populate `public_url` in `WorkerInit` and accept `RegisterRequest.public_url`; until then the worker reads whichever field the control plane sends. ## One Worker, One Public URL Every worker configures one absolute `PUBLIC_URL`, for example: ```text https://worker-1.example.net ``` The worker proposes this URL during registration/configuration. The control plane validates and canonicalizes it, persists the accepted value, and returns it in signed init/config. The URL has scheme and authority only: no userinfo, query, fragment, or ambiguous path. Public production URLs use HTTPS. The control plane owns worker ID, account, region, cluster ID, membership, voter/observer role, peer generation, and topology. A worker cannot gain trust or Raft membership by advertising an endpoint. ## Routes On The Origin - `/healthz`: process liveness; - `/api/peer/status`: authenticated peer/control-plane reachability and Raft summary; - `/raft`: Hashicorp Raft HTTP/1.1 upgrade transport and membership requests; - `/web/*` and `/web/api/*`: operator console. TLS termination belongs to Traefik or nginx. The worker may keep internal HTTP listeners on loopback/private addresses. `PUBLIC_URL` is advertisement, not a request to bind the worker process or provision a public certificate. For `/raft`, proxies must preserve path, query, Host, authorization/client identity, `Connection`, and `Upgrade`; use HTTP/1.1, disable response buffering, and allow long-lived upgraded connections. nginx requires explicit upgrade headers. Tests must exercise this proxy path rather than only direct TCP. ## In-Memory Peer Database Signed control-plane config supplies a monotonic topology generation and peers scoped to one cluster/trust boundary. Each entry includes worker ID, `PUBLIC_URL`, role, region, certificate identity, membership state, and expiry. The worker replaces its in-memory peer database atomically when a newer valid generation arrives. Removed or expired peers stop contributing immediately. Periodic concurrent HTTP checks call each peer's `/api/peer/status`, validate the TLS/client identity and response worker ID, and retain bounded observations. Reachability never grants membership. The response distinguishes process health, control-plane ping duration/result, observation time, Raft role/term/leader, applied/commit indexes, and transport errors. The worker console presents the same bounded per-peer state. ## Bind And Advertise Separation Raft requires separate local bind and advertised addresses. The local listener may be `127.0.0.1:37401`; the advertised endpoint derives from `PUBLIC_URL + /raft`. Hashicorp Raft membership stores the advertised address, never the loopback bind. Outbound Raft transport performs TLS using the public hostname and worker certificate. The current `rafthttp.NewDialTCP` and shared Basic-auth injection must be replaced. Static `WORKER_CLUSTER_PEERS` remains a local lab fallback, not production discovery. ## Certificate Authority Adapt the proven CA shape from `/data/_backup/rbackup-server`, particularly its explicit CA pool, required client-certificate verification, and certificate-to- database identity lookup. Do not copy its tracked keys, ten-year certificates, CN-only identity, inactive CRL, or server-generated private-key delivery. The control plane owns a worker-cluster CA separate from public web certificates. Requirements: - worker-generated key and CSR; - worker and cluster identity in a verified URI SAN; - short-lived leaf certificates with automatic renewal; - encrypted CA private key and documented backup/custody; - issuance and revocation audit; - emergency revocation plus short expiry as the normal revocation bound; - rejection of expired, revoked, wrong-cluster, or SAN-mismatched peers. One-time enrollment tokens authorize a CSR exactly once. They do not become ongoing peer credentials. Browser traffic does not require a client certificate; the reverse proxy applies mTLS policy to peer/Raft routes or forwards a verified client identity through a tightly controlled internal boundary. ## Production Membership Production critical-check clusters require exactly three voters initially. Single-node bootstrap and two-node clusters are lab/bootstrap states and cannot execute customer critical checks. Membership changes are leader-mediated and control-plane-authorized. The control plane is an external witness, never a voter. ## Implementation Order 1. Add `PUBLIC_URL` wire/config fields while accepting legacy `WORKER_URL` only for a bounded migration. Worker side implemented: `PUBLIC_URL` is canonical, `WORKER_URL` is a deprecated fallback with a startup warning, `internal/wire` carries `public_url` on `WorkerInit` (keeping `url` for compatibility), and startup/install enforce the strict origin shape for `PUBLIC_URL` while tolerating legacy `WORKER_URL` shapes. Control-plane counterpart: read `RegisterRequest.public_url` when worker-initiated registration is wired, populate `public_url` (not `url`) in `WorkerInit`, and persist the accepted origin. Until then the worker consumes whichever of `public_url`/`url` the control plane sends. 2. Validate ownership/reachability and return accepted signed configuration. 3. Add scoped, versioned in-memory peer topology and concurrent health probes. 4. Extend status APIs/UI with networking, control-plane RTT, and Raft state. 5. Implement CA storage, CSR enrollment, renewal, revocation, and audit. 6. Split Raft bind/advertise addresses and add outbound TLS dialing. 7. Route `/raft` through the shared public origin and reverse-proxy contract. 8. Replace static production peers with control-plane-managed membership. 9. Run a three-voter TLS-proxy failover/partition campaign. ## Release Gates - A peer cannot impersonate another worker or join another cluster. - An invalid/replayed enrollment token issues no certificate. - Revoked, expired, or SAN-mismatched certificates fail before Raft handling. - Three workers join through external HTTPS origins and elect one leader. - Killing one voter preserves quorum and commits through the proxy path. - The UI reports each peer, transport state, control-plane RTT, role, term, leader, and replication lag without exposing credentials.