fix(worker): harden control-plane lifecycle
Все проверки выполнены успешно
CI / test (push) Successful in 2m32s
Docker / Build and publish worker image (push) Successful in 18m17s

- reconnect safely after token rotation and retry leased results
- reject malformed tasks and remove production cluster debug mutation
- validate environment files and require immutable container images

BREAKING CHANGE: Docker install, deploy, and Compose now require an
immutable repository@sha256 image reference.
Этот коммит содержится в:
Gleb Tv
2026-07-19 23:11:43 +03:00
родитель 6937674449
Коммит e987f24903
38 изменённых файлов: 2203 добавлений и 674 удалений

Просмотреть файл

@@ -55,19 +55,22 @@ Bootstrap replay, expiry, worker-ID mismatch, or account mismatch fails closed.
## Token Rotation
Rotation is a two-token handoff:
The current worker receives a replacement token in memory. In-memory reconnect
rotation is implemented: the control plane
invalidates the old token immediately, then `Runner.RotateToken` closes only
the active control-plane connection and reconnects with the replacement token.
It does not stop the runner or its web, inventory, metrics, or cluster
subsystems. There is no durable token write, overlap, rollback, or rotation
acknowledgement contract yet; persistent replacement-token failures can require
operator action.
1. Control plane issues a replacement token and keeps the old token valid for a
short bounded overlap.
2. Worker atomically writes the replacement secret.
3. Worker closes only the active control-plane connection and reconnects.
4. Successful authentication acknowledges rotation; control plane revokes the
old token.
5. Failure retains the old token until overlap expires and emits an operator
warning.
The current `Runner.RotateToken` terminates the runner by closing its stop
channel. This must be fixed before claiming unattended rotation.
If a websocket write fails after the worker has dequeued a check or notification
result, it resends that envelope after reconnect. Delivery is therefore
at-least-once; the control plane must deduplicate by the leased job and lease
token before applying a resent result. Failed server-metric snapshots are
dropped because they have no idempotency key and the next periodic collection
replaces them. Metric snapshots are bound to the active control connection;
snapshots collected while disconnected or for an older connection are dropped.
## Runtime Config Authentication
@@ -126,7 +129,8 @@ The standalone binary provides two systemd installation paths:
installs a systemd-managed prebuilt image instead;
- `rsmon-worker deploy` verifies an SSH host key, uploads the binary and a
temporary mode-0600 environment file, and invokes `install` remotely;
`--docker` uploads only the environment and unit, then pulls on the target.
`--docker` uploads only the environment and unit, then pulls an explicitly
supplied immutable image digest on the target.
Both default to `https://rsmon.ru` and accept `--token-file` for automation.
Direct secret flags are supported but can be visible in process listings; file
@@ -143,22 +147,25 @@ monitor execution.
## Implementation Work Packages
1. Add signed account/config identity to `internal/wire` and runner state.
2. Validate task account and credential scope locally before dispatch.
3. Implement one-time bootstrap and atomic token storage/rotation.
4. Add worker disable/revoke behavior and visible stale-config state.
5. Add mTLS as an optional first transport, then require it for Raft clusters.
6. Add public-task grant and SSRF-safe executor only after private isolation is
1. [ ] Add signed account/config identity to `internal/wire` and runner state.
2. [ ] Validate task account and credential scope locally before dispatch.
3. [x] Implement in-memory reconnect token rotation without stopping worker
subsystems.
4. [ ] Implement one-time bootstrap, durable token storage, and rotation
acknowledgement.
5. [ ] Add worker disable/revoke behavior and visible stale-config state.
6. [ ] Add mTLS as an optional first transport, then require it for Raft clusters.
7. [ ] Add public-task grant and SSRF-safe executor only after private isolation is
proven.
## Acceptance Tests
- A private worker never leases or executes another account's normal task.
- A forged capability or account field cannot widen scope.
- Invalid, expired, downgraded, or differently scoped signed config is rejected.
- Bootstrap tokens are single-use and absent from disk after exchange.
- Rotation reconnects without stopping web, inventory, metrics, or cluster.
- Credential snapshots contain only the worker account and allowed methods.
- Public execution cannot reach loopback, link-local, RFC1918, metadata, Unix
- [ ] A private worker never leases or executes another account's normal task.
- [ ] A forged capability or account field cannot widen scope.
- [ ] Invalid, expired, downgraded, or differently scoped signed config is rejected.
- [ ] Bootstrap tokens are single-use and absent from disk after exchange.
- [x] Rotation reconnects without stopping web, inventory, metrics, or cluster.
- [ ] Credential snapshots contain only the worker account and allowed methods.
- [ ] Public execution cannot reach loopback, link-local, RFC1918, metadata, Unix
sockets, or a private redirect target.
- Revocation prevents reconnect and clears in-memory credentials.
- [ ] Revocation prevents reconnect and clears in-memory credentials.