# Worker Architecture

## Runtime

`cmd/rsmon-worker/main.go` composes three independently stoppable subsystems:

1. `internal/distworker`: persistent control-plane connection, normal task
   execution, selfcheck, notification delivery, and server metric reports.
2. `internal/webapp`: authenticated local operator console and SQLite-backed
   local state.
3. `internal/workercluster`: optional worker-to-worker Raft transport and FSM.

All three share one cancellation context. The worker can run without the web
console using `--no-web`; the cluster is disabled unless
`WORKER_CLUSTER_ENABLED=true`.

## Two Execution Paths

The two check paths are deliberately separate.

| Path | Work origin | Ownership | Worker implementation |
| --- | --- | --- | --- |
| Normal checks and notifications | Control-plane PostgreSQL `tasks` rows | Exact lease token and expiry held by control plane | `internal/distworker` |
| `distributed_critical` | Signed config adopted by a worker Raft cluster | Raft FSM incident and outbox metadata | `internal/workercluster` plus a planned scheduler bridge |

Normal work always uses control-plane dispatch and leases. The critical path
must never use a normal task row, job assignment, or lease. Adding the critical
path must not change normal protocol behavior.

## Package Boundaries

| Package | Responsibility | Must not do |
| --- | --- | --- |
| `internal/wire` | JSON-compatible protocol types | Network or persistence work |
| `internal/distworker` | Connections, queues, pools, execution orchestration | Direct control-plane database writes |
| `internal/checkexec` | Database-free check dispatch | Scheduling or result persistence |
| `checks/*` | Protocol-specific probes | Worker selection or account policy |
| `internal/webapp` | Local UI, sessions, audit, local snapshots | Become the fleet source of truth |
| `internal/workercluster` | Raft transport, log, FSM, snapshots, membership | Store raw samples or plaintext credentials |
| `internal/sender` | Provider calls shared by worker executors | Own retries or durable task state |

New host discovery should move into `internal/inventory`; the web console may
read it through an interface but must not remain the collector's owner. New
network probes should live in `internal/netdiag` and use an allowlisted target
provided by a validated control-plane task.

## Current Status

| Capability | State | Evidence |
| --- | --- | --- |
| WebSocket normal check execution | Implemented | `internal/distworker/client.go`, `runner.go` |
| Check kinds | Implemented | `internal/checkexec/exec.go`, `checks/*` |
| Delegated notifications | Partial | email, Telegram, webhook, Mattermost implemented; SMS/voice unsupported |
| Host telemetry to control plane | Implemented | `internal/distworker/server_metrics.go` |
| Local web console | Implemented | `internal/webapp/server.go`, `routes.go` |
| Local inventory and status pages | Partial | collectors exist but are not started by `Server.Start` |
| Private-worker scheduler isolation | Implemented in shared selector code | `app/models/worker_node.go`, `task_selector.go`, `check_jobs.go` |
| Private-worker bootstrap and worker-side scope verification | Planned | no one-time bootstrap or signed config path |
| Cross-worker confirmation | Partial | peer selfcheck and shared model logic exist; no rich diagnostic tasks |
| Raft membership and persistence | Partial | `internal/workercluster` |
| Dispatchless critical checks | Planned | no scheduler, signature adoption, quorum incident engine, or outbox executor |

## Data Rules

- Credentials delivered in `init`/`config` stay in memory and are never written
  to the local SQLite store, Raft log, or snapshots.
- Raw check samples and high-volume telemetry never enter Raft.
- Local UI state is not authoritative control-plane state.
- A normal result is not valid without the current exact lease token.
- Any worker-originated account, server, monitor, or target identifier is
  untrusted until the control plane validates it against the authenticated
  worker.
- A worker must not execute a synthetic health check whose target is itself.

## Runtime Completion Criteria

- SIGTERM drains the runner, stops collectors, closes listeners, shuts down
  Raft, and closes SQLite without exceeding the service stop timeout.
- A web-console failure is reported and causes an intentional process policy;
  it must not silently leave a partially healthy process.
- Startup validates the advertised origin: canonical `PUBLIC_URL` is held to
  the strict scheme-and-authority shape (no userinfo, query, fragment, or
  ambiguous path) and plain HTTP on a non-loopback host is rejected in an
  explicitly production environment; the legacy `WORKER_URL` is read as a
  bounded-migration fallback, held only to the tolerant absolute-URL check,
  and logged with a deprecation warning. Both reject a missing hostname, e.g.
  `https://:27401`. Startup also rejects incomplete auth credentials, invalid
  cluster settings, and unwritable data directories before accepting work.
- `GET /healthz` reports process-local liveness. Control-plane reachability is
  a separate readiness/selfcheck signal and must not make a healthy container
  fail its local liveness probe.