docs: plan HTTPS peers and source install

Этот коммит содержится в:
Gleb Tv
2026-08-12 18:40:43 +03:00
родитель bf9253d6fe
Коммит a1ccd50aaf
11 изменённых файлов: 273 добавлений и 25 удалений

Просмотреть файл

@@ -27,6 +27,8 @@ disagree, update both in the same change or mark the discrepancy explicitly.
| [private-workers.md](private-workers.md) | Customer worker isolation, credentials, onboarding | Partial |
| [network-diagnostics.md](network-diagnostics.md) | Confirmation checks and dedicated diagnostic tasks | Partial |
| [critical-check-cluster.md](critical-check-cluster.md) | Raft-backed dispatchless critical checks | Scaffold only |
| [public-endpoint-and-identity.md](public-endpoint-and-identity.md) | One HTTPS origin, peer status, CA/mTLS, managed Raft topology | Planned; peer HTTPS partial |
| [source-installation.md](source-installation.md) | Go SSH source installer and Docker/OpenSSH test matrix | Planned |
| [implementation-roadmap.md](implementation-roadmap.md) | Ordered repository work packages and release gates | Active |
| [source-plan-migration.md](source-plan-migration.md) | Source-to-target conversion ledger and resolved conflicts | Complete mapping |
@@ -39,7 +41,7 @@ This repository owns:
- check and notification execution;
- local web console, local state, inventory collection, and host metrics;
- worker-to-worker transport and Raft state;
- Docker, Compose, systemd, and release-image packaging.
- Docker, Compose, systemd, source installation, and release-image packaging.
The RSMon control-plane repository owns:

Просмотреть файл

@@ -78,8 +78,10 @@ provided by a validated control-plane task.
Raft, and closes SQLite without exceeding the service stop timeout.
- A web-console failure is reported and causes an intentional process policy;
it must not silently leave a partially healthy process.
- Startup rejects malformed `WORKER_URL`, incomplete auth credentials, invalid
cluster settings, and unwritable data directories before accepting work.
- Startup currently rejects malformed `WORKER_URL`; the accepted target is
`PUBLIC_URL` with bounded compatibility migration. Startup also rejects
incomplete auth credentials, invalid cluster settings, and unwritable data
directories before accepting work.
- `GET /healthz` reports process-local liveness. Control-plane reachability is
a separate readiness/selfcheck signal and must not make a healthy container
fail its local liveness probe.

Просмотреть файл

@@ -16,6 +16,9 @@ GET /worker?token=<RSMON_TOKEN>
`/api/worker` and the HTTP jobs/results APIs remain compatibility paths. New
workers use WebSocket task envelopes.
Public endpoint and cluster identity requirements are defined in
[public-endpoint-and-identity.md](public-endpoint-and-identity.md).
## Frame Model
Every frame is a `wire.WorkerMessage` with `kind` and one active content
@@ -43,14 +46,20 @@ without execution or reporting.
## Initialization And Refresh
`wire.WorkerInit` supplies runtime values owned by the control plane:
The worker proposes `PUBLIC_URL` during registration. The control plane
validates and canonicalizes it; `wire.WorkerInit` returns the accepted endpoint
and supplies runtime values owned by the control plane:
- worker ID, region, advertised URL, capabilities, and concurrency;
- allowed notification methods and account IDs;
- optional linked server ID for host metrics;
- LLM endpoints;
- scoped notification credentials and system contacts;
- peer workers for selfcheck/cluster-adjacent behavior.
- signed, cluster-scoped peer topology for selfcheck and Raft behavior.
Worker ID, account, region, cluster, membership, role, topology generation, and
certificate identity are control-plane authority. Local environment or a peer
response cannot override them. Static peer environment remains lab-only.
The worker clamps supplied concurrency to its local maximum. Credentials are
replaced atomically in memory on refresh. Removed credentials must become

Просмотреть файл

@@ -33,7 +33,8 @@ witness. It is never a Raft voter.
## Topology
- Production clusters have 3 or 5 voters, never an even count.
- Initial production clusters require exactly 3 voters. A future measured
five-voter profile may be introduced separately; never use an even count.
- Voters remain within a bounded-latency topology. Remote regions use observer
nodes unless measured RTT supports the configured election timeout.
- Nodes may be `voter`, `observer`, or `voter+observer`.
@@ -44,6 +45,12 @@ Single-voter bootstrap is temporary. The cluster must not execute customer
critical checks until at least three voters are healthy and the signed
observer set is committed.
Membership and mTLS identity are assigned by the control plane as defined in
[public-endpoint-and-identity.md](public-endpoint-and-identity.md). Production
Raft traffic uses `/raft` on each worker's external HTTPS origin through an
HTTP/1.1 upgrade-capable reverse proxy. Static peers and shared Basic auth are
lab compatibility only.
## Raft State
The FSM contains only:
@@ -167,14 +174,16 @@ but cannot commit or fabricate incidents.
1. [x] Remove production exposure of debug config application.
2. [ ] Complete versioned FSM types, command validation, and deterministic tests.
3. [ ] Add mTLS identity and safe one-claim cluster bootstrap.
3. [ ] Adapt the rbackup CA pattern into short-lived SAN-bound mTLS identity,
rotation/revocation, and safe one-claim cluster bootstrap.
4. [ ] Add signed config and observer-set adoption.
5. [ ] Add deterministic scheduler and checkexec bridge in shadow mode.
6. [ ] Implement observation aggregation, incident policy, and idempotency.
7. [ ] Implement encrypted snapshots and restore/migration tests.
8. [ ] Add metadata outbox executor and failover-safe delivery.
9. [ ] Add external witness, replay, metrics, and operational runbooks.
10. [ ] Run synthetic 3/5-node fault campaigns before any customer check.
10. [ ] Run synthetic three-node HTTPS-proxy fault campaigns before any
customer check; define a separate five-node profile before testing it.
## Release Gates

Просмотреть файл

@@ -14,6 +14,9 @@ Worker repository:
- verify amd64 and arm64 image startup, Chromium availability, and version
metadata;
- add a package/install smoke test for Docker and systemd artifacts;
- add the Docker/OpenSSH source-install matrix for Alpine, Ubuntu, and Arch from
[source-installation.md](source-installation.md), using Go 1.26 and branch
`main`;
- document immutable SHA and release tags as production defaults.
Gate: a push publishes `sha-<12>` and `latest` manifests for both platforms,
@@ -27,7 +30,8 @@ Worker repository:
- wire local inventory and metrics collector lifecycle into web server start
and shutdown;
- validate full HTTP config, including `WORKER_URL`, in main startup;
- validate full HTTP config, including the accepted `PUBLIC_URL`, in main
startup;
- [x] reconnect in memory on token rotation without stopping the runner;
- [x] resend bounded check/notification results after websocket reconnect;
- define process policy when the web listener exits unexpectedly;
@@ -70,7 +74,8 @@ Worker repository:
- token handoff and revocation handling;
- stale/expired signed-config behavior;
- clear disabled/revoked UI state;
- optional mTLS client identity.
- mandatory production mTLS client identity and control-plane-owned topology as
defined in [public-endpoint-and-identity.md](public-endpoint-and-identity.md).
Control-plane dependency:
@@ -139,13 +144,13 @@ State: Raft scaffold only.
Worker repository:
- secure bootstrap and mTLS identity;
- secure bootstrap, CA lifecycle, mTLS identity, and one-origin HTTPS transport;
- signed config adoption and observer-set versioning;
- deterministic scheduler and check executor bridge;
- deterministic incident/quorum FSM;
- encrypted snapshots and restore;
- metadata outbox, witness, replay, and metrics;
- 3/5-node fault campaigns.
- three-node external-HTTPS fault campaigns.
Control-plane dependency:

Просмотреть файл

@@ -5,7 +5,8 @@ Docker image) into a running, enabled systemd service on a Linux host. It is
the supported way to deploy the worker: it writes the configuration, the
systemd unit, and the data directory, then starts the service.
> A shell installer that downloads a pre-built binary is planned. Today
> A Go SSH source installer is planned in
> [source-installation.md](source-installation.md). Today
> `install` copies the binary you invoke it from (or pulls the `--image`
> digest), so build first with `make build` and run the resulting
> `./bin/rsmon-worker`.
@@ -53,7 +54,7 @@ values automatically. To override a value, pass the matching flag.
| `RSMON_TOKEN` | yes | none | Worker bearer token. |
| `WORKER_HOST` | no | `127.0.0.1` | Operator-console bind address. |
| `WORKER_PORT` | no | `27401` (primary) | Operator-console port. **Required** for named instances. |
| `WORKER_URL` | no | none | Public URL advertised to the control plane. |
| `PUBLIC_URL` | no | none | Proposed public HTTPS origin; accepted by the control plane. |
| `WORKER_LOGIN` | no | `admin` (generated) | Operator-console basic-auth login. |
| `WORKER_PASSWORD` | no | generated | Operator-console basic-auth password. |
| `WORKER_COMPOSE_ENABLED` | no | feature default (on) | Enable Docker Compose discovery/management. |
@@ -66,6 +67,11 @@ writes it to the env file, and prints it once. Record it; the operator console
requires it for both the browser login and the `/web/api/*` HTTP basic-auth
endpoints.
`PUBLIC_URL` does not bind a listener or terminate TLS. It advertises the one
external origin used for the console, authenticated peer status, and planned
Raft `/raft` transport. See
[public-endpoint-and-identity.md](public-endpoint-and-identity.md).
Values must be systemd-safe: no whitespace, quotes, backslashes, or `$`
interpolation inside a value. This keeps the file unambiguous across systemd
`EnvironmentFile` and `docker --env-file`.

Просмотреть файл

@@ -8,7 +8,7 @@ plane validates identity, applies account scope, reconciles lifecycle, and
persists the inventory projection.
The control-plane inventory entities remain `Server`, `ServerIp`, `Site`,
`Deployment`, `Domain`, `Repo`, and `site_repos`. Their API and rstuff stream
`Deployment`, `Domain`, `Repo`, and `site_repos`. Their API and legacy stream
projection are control-plane concerns. This repository owns only local
discovery and worker-originated reports.
@@ -130,7 +130,7 @@ successful section can mark previously observed entities missing.
- Do not guess a primary address; report interface and route metadata so the
control plane can apply policy.
## Relationship To deploymentd And rstuff
## Relationship To deploymentd And RSLogin
Worker reports and deploymentd ingest are complementary producers of the same
control-plane projection. During migration:
@@ -140,8 +140,9 @@ control-plane projection. During migration:
- every row records source and source report ID;
- reconciliation occurs per source and section to prevent one producer from
deleting another producer's observations;
- the rstuff `rsmon.inventory.v1` stream remains a control-plane output/input,
not a worker protocol.
- the former rstuff stream direction is superseded; RSMon owns inventory and
RSLogin supplies trusted project and SSH-access input through a separate
control-plane integration, never a worker protocol.
## Delivery Phases

123
docs/public-endpoint-and-identity.md Обычный файл
Просмотреть файл

@@ -0,0 +1,123 @@
# Public Endpoint, Peer Identity, And Raft Transport
## Status
Accepted target architecture. Ordinary peer status checks already support an
external HTTPS worker URL. Raft currently uses a separate plaintext listener,
static peers, and shared Basic authentication; it does not yet meet this plan.
## One Worker, One Public URL
Every worker configures one absolute `PUBLIC_URL`, for example:
```text
https://worker-1.example.net
```
The worker proposes this URL during registration/configuration. The control
plane validates and canonicalizes it, persists the accepted value, and returns
it in signed init/config. The URL has scheme and authority only: no userinfo,
query, fragment, or ambiguous path. Public production URLs use HTTPS.
The control plane owns worker ID, account, region, cluster ID, membership,
voter/observer role, peer generation, and topology. A worker cannot gain trust
or Raft membership by advertising an endpoint.
## Routes On The Origin
- `/healthz`: process liveness;
- `/api/peer/status`: authenticated peer/control-plane reachability and Raft
summary;
- `/raft`: Hashicorp Raft HTTP/1.1 upgrade transport and membership requests;
- `/web/*` and `/web/api/*`: operator console.
TLS termination belongs to Traefik or nginx. The worker may keep internal HTTP
listeners on loopback/private addresses. `PUBLIC_URL` is advertisement, not a
request to bind the worker process or provision a public certificate.
For `/raft`, proxies must preserve path, query, Host, authorization/client
identity, `Connection`, and `Upgrade`; use HTTP/1.1, disable response buffering,
and allow long-lived upgraded connections. nginx requires explicit upgrade
headers. Tests must exercise this proxy path rather than only direct TCP.
## In-Memory Peer Database
Signed control-plane config supplies a monotonic topology generation and peers
scoped to one cluster/trust boundary. Each entry includes worker ID,
`PUBLIC_URL`, role, region, certificate identity, membership state, and expiry.
The worker replaces its in-memory peer database atomically when a newer valid
generation arrives. Removed or expired peers stop contributing immediately.
Periodic concurrent HTTP checks call each peer's `/api/peer/status`, validate
the TLS/client identity and response worker ID, and retain bounded observations.
Reachability never grants membership.
The response distinguishes process health, control-plane ping duration/result,
observation time, Raft role/term/leader, applied/commit indexes, and transport
errors. The worker console presents the same bounded per-peer state.
## Bind And Advertise Separation
Raft requires separate local bind and advertised addresses. The local listener
may be `127.0.0.1:37401`; the advertised endpoint derives from
`PUBLIC_URL + /raft`. Hashicorp Raft membership stores the advertised address,
never the loopback bind.
Outbound Raft transport performs TLS using the public hostname and worker
certificate. The current `rafthttp.NewDialTCP` and shared Basic-auth injection
must be replaced. Static `WORKER_CLUSTER_PEERS` remains a local lab fallback,
not production discovery.
## Certificate Authority
Adapt the proven CA shape from `/data/_backup/rbackup-server`, particularly its
explicit CA pool, required client-certificate verification, and certificate-to-
database identity lookup. Do not copy its tracked keys, ten-year certificates,
CN-only identity, inactive CRL, or server-generated private-key delivery.
The control plane owns a worker-cluster CA separate from public web
certificates. Requirements:
- worker-generated key and CSR;
- worker and cluster identity in a verified URI SAN;
- short-lived leaf certificates with automatic renewal;
- encrypted CA private key and documented backup/custody;
- issuance and revocation audit;
- emergency revocation plus short expiry as the normal revocation bound;
- rejection of expired, revoked, wrong-cluster, or SAN-mismatched peers.
One-time enrollment tokens authorize a CSR exactly once. They do not become
ongoing peer credentials. Browser traffic does not require a client certificate;
the reverse proxy applies mTLS policy to peer/Raft routes or forwards a verified
client identity through a tightly controlled internal boundary.
## Production Membership
Production critical-check clusters require exactly three voters initially.
Single-node bootstrap and two-node clusters are lab/bootstrap states and cannot
execute customer critical checks. Membership changes are leader-mediated and
control-plane-authorized. The control plane is an external witness, never a
voter.
## Implementation Order
1. Add `PUBLIC_URL` wire/config fields while accepting legacy `WORKER_URL` only
for a bounded migration.
2. Validate ownership/reachability and return accepted signed configuration.
3. Add scoped, versioned in-memory peer topology and concurrent health probes.
4. Extend status APIs/UI with networking, control-plane RTT, and Raft state.
5. Implement CA storage, CSR enrollment, renewal, revocation, and audit.
6. Split Raft bind/advertise addresses and add outbound TLS dialing.
7. Route `/raft` through the shared public origin and reverse-proxy contract.
8. Replace static production peers with control-plane-managed membership.
9. Run a three-voter TLS-proxy failover/partition campaign.
## Release Gates
- A peer cannot impersonate another worker or join another cluster.
- An invalid/replayed enrollment token issues no certificate.
- Revoked, expired, or SAN-mismatched certificates fail before Raft handling.
- Three workers join through external HTTPS origins and elect one leader.
- Killing one voter preserves quorum and commits through the proxy path.
- The UI reports each peer, transport state, control-plane RTT, role, term,
leader, and replication lag without exposing credentials.

89
docs/source-installation.md Обычный файл
Просмотреть файл

@@ -0,0 +1,89 @@
# SSH Source Installation Plan
## Status
Planned. The current Go installer can upload a binary or deploy an immutable
Docker image over SSH. It does not yet install build prerequisites, download Go,
clone the public repository, or build remotely. Existing tests are unit tests;
there is no live OpenSSH-container installation test.
## Initial Platform Scope
The first source installer supports Linux and is validated on Alpine, Ubuntu,
and Arch Linux containers. CentOS-family support follows after its package and
service differences are implemented. Windows and macOS remain later platform
work despite the worker being written in Go.
Use `reg.rsxx.ru` image mirrors where available. Tests must not depend on Docker
Hub when a local mirror exists.
## Source Install Flow
The Go CLI connects through `golang.org/x/crypto/ssh` using existing key,
passphrase, password, sudo-password, known-hosts, and pinned-fingerprint support.
It then:
1. detects supported OS, architecture, package manager, init system, and
privilege path;
2. installs only required packages (`git`, CA certificates, download/archive
tools); it does not install `build-essential` or a C compiler unless a
detected dependency requires CGO;
3. downloads the pinned Go 1.26 toolchain for the detected architecture and
verifies the published SHA-256;
4. clones `https://rocketgit.ru/rsmon/worker.git` or updates an existing clone;
5. checks out branch `main` and records the resolved commit;
6. builds a reproducible worker binary with the repository build flags;
7. atomically installs the binary, validated environment, data directory, and
service definition;
8. starts the service and verifies process status and `/healthz`.
Repository, branch, Go version, checksum source, build directory, and Go module
proxy may be configurable, but production output records their resolved values.
The default repository is publicly readable and requires no source credential.
## Docker OpenSSH Test Harness
Adapt the real-network pattern from `/data/_swap/sshkeymanager`: start an
OpenSSH container, wait for SSH readiness, connect with the Go installer, and
tear the environment down reliably. Do not mock SSH command execution in the
acceptance test.
Provide images/fixtures for:
- Ubuntu with apt and systemd-compatible service testing where practical;
- Alpine with apk/OpenRC or a clearly separated no-service build/install gate;
- Arch with pacman and its service behavior.
Each clean target begins without Go or the worker source. The test asserts
package installation, verified Go version, clone branch/resolved commit, build,
atomic config permissions, running service where supported, and HTTP liveness.
## Idempotency And Security
- A second run updates/fetches safely and leaves one active service.
- Wrong host fingerprints fail before remote mutation.
- Tokens/passwords come from files or stdin-safe channels and never appear in
command arguments, logs, source checkout, or shell history.
- Remote temporary files are removed on success and failure.
- Failed builds do not replace a working binary or service definition.
- Package-manager and download failures return bounded actionable errors.
- The installer verifies Go tarball checksum before extraction.
## Implementation Work Packages
1. Add reusable Docker/OpenSSH harness and distro fixtures.
2. Add pure distro/toolchain/source-install script planning and unit tests.
3. Execute source installation through the existing SSH transport.
4. Add atomic build/install, idempotency, and failure rollback.
5. Add Alpine, Ubuntu, and Arch network E2E tests to CI.
6. Add CentOS-family support.
7. Plan native Windows service and macOS launchd installers separately.
## Acceptance Gates
- All three initial Linux images install from a clean state through OpenSSH.
- The built worker reports the expected version/commit and serves `/healthz`.
- Re-running the installer succeeds without duplicate services or leaked files.
- Host-key, checksum, clone, build, and service-start failure tests preserve the
previous installation.
- CI uses approved registry mirrors and cleans every test container/network.

Просмотреть файл

@@ -39,10 +39,10 @@ Conversion date: 2026-07-13.
| --- | --- | --- |
| `docs/inventory-management.md` | `inventory.md` | Control-plane projection retained as dependency; worker owns collection only |
| `docs/plans/inventory-management.md` | `inventory.md` | deploymentd and worker reports defined as source-aware complementary producers |
| `docs/parity/rstuff-inventory.md` | `inventory.md` | rstuff enums/stream remain control-plane contracts, not worker protocol |
| `docs/parity/rstuff-inventory.md` | `inventory.md` | Historical only; RSMon owns inventory and RSLogin supplies project/access input |
| `docs/parity/deploymentd.md` | `inventory.md` | deploymentd parity becomes fixture normalization and comparison gate |
| `docs/deploymentd-replay-rollout.md` | `inventory.md` | HTTP nonce rollout remains deploymentd-specific; worker uses authenticated typed reports |
| `docs/integrations/rstuff-inventory-stream.md` | `inventory.md` | Valkey stream remains between control plane and rstuff, never worker transport |
| `docs/integrations/rstuff-inventory-stream.md` | `inventory.md` | Superseded; no rstuff service is planned and Valkey is not the target inventory authority |
| `docs/plans/servers-and-hardware-metrics.md` | `web-console-and-observability.md`, `inventory.md` | Implemented server metric report retained; local hardware extensions staged |
| `docs/server-observability.md` | `web-console-and-observability.md`, `control-plane-protocol.md` | Current wire fields, ownership validation, and TSDB boundary retained |
| `docs/parity/capture.md` | `web-console-and-observability.md`, `implementation-roadmap.md` | Worker-host coverage gaps converted into staged collector work |
@@ -101,9 +101,10 @@ obsolete. Collector lifecycle remains a real implementation gap.
### Inventory
deploymentd HTTP ingest is currently implemented on the control plane. Worker
inventory is local/partial and has no upstream report yet. Future worker and
deploymentd reports are complementary source-scoped inputs; neither may erase
the other's observations.
inventory is local/partial and has no upstream report yet. RSMon is the target
inventory authority; RSLogin supplies trusted project and SSH-access input.
Future worker and deploymentd reports are complementary source-scoped inputs;
neither may erase the other's observations.
### Private Workers And Public Work

Просмотреть файл

@@ -27,8 +27,9 @@ absolute lifetime. State-changing browser requests require CSRF validation.
Authenticated responses use no-store and restrictive security headers.
OAuth/device authorization is not implemented. Public exposure requires a TLS
reverse proxy and network restrictions. Do not claim that `WORKER_URL` enables
TLS; it only advertises the externally reachable URL.
reverse proxy and network restrictions. The accepted target name is
`PUBLIC_URL`; `WORKER_URL` is current legacy configuration during migration.
Neither setting enables TLS; it only advertises the externally reachable URL.
## Known Runtime Gap