docs: migrate worker implementation plans
Все проверки выполнены успешно
CI / test (push) Successful in 30s
Docker / Build and publish worker image (push) Successful in 10m49s
Все проверки выполнены успешно
CI / test (push) Successful in 30s
Docker / Build and publish worker image (push) Successful in 10m49s
Этот коммит содержится в:
147
docs/private-workers.md
Обычный файл
147
docs/private-workers.md
Обычный файл
@@ -0,0 +1,147 @@
|
||||
# Private Workers
|
||||
|
||||
## Model
|
||||
|
||||
Operated and customer-operated workers run the same binary and protocol. The
|
||||
control plane assigns trust and scope. A private worker is untrusted for every
|
||||
account except its own and cannot widen its own scope through capabilities or
|
||||
protocol fields.
|
||||
|
||||
Current shared scheduler code already supports restrictive account selection:
|
||||
|
||||
- `app/models/worker_node.go` derives allowed check and notification accounts;
|
||||
- `app/models/check_jobs.go` filters check leasing;
|
||||
- `app/models/task_selector.go` filters generic task leasing;
|
||||
- a nonempty account list is restrictive; an empty list means platform worker
|
||||
and is never available to a customer worker.
|
||||
|
||||
This is only one isolation layer. The executable still trusts the authenticated
|
||||
control plane's init/task payload and private-worker onboarding is incomplete.
|
||||
|
||||
## Required Invariants
|
||||
|
||||
- A private worker has one immutable `account_id` in signed runtime config.
|
||||
- Every normal task includes `account_id`; the worker rejects mismatch before
|
||||
execution and reports a protocol error without touching the target.
|
||||
- Credentials are filtered by account and method on the control plane, then
|
||||
checked again by exact credential ID in the worker.
|
||||
- Private workers do not receive another account's monitor, contact, LLM, peer,
|
||||
inventory, or notification data.
|
||||
- A private worker cannot become operated by changing local config or reported
|
||||
capabilities.
|
||||
- Results cannot choose their account. The control plane applies them against
|
||||
the leased task and authenticated worker.
|
||||
- Local UI data stays on the worker unless a typed, bounded protocol explicitly
|
||||
permits upload.
|
||||
|
||||
## Onboarding
|
||||
|
||||
The current production path uses a worker token created by an administrator.
|
||||
The customer self-service target is:
|
||||
|
||||
1. Control plane creates a disabled private-worker record and a one-time,
|
||||
15-minute bootstrap token.
|
||||
2. Installer writes control-plane URL and bootstrap token to a mode-0600
|
||||
temporary environment file.
|
||||
3. Worker exchanges it over TLS for a long-lived worker token and signed
|
||||
immutable account identity.
|
||||
4. Worker writes the long-lived token to the configured secret file with mode
|
||||
0600, removes the bootstrap token, and reconnects.
|
||||
5. Control plane activates the worker only after the first authenticated
|
||||
heartbeat and valid capability report.
|
||||
|
||||
The worker must never receive the control-plane registration admin secret.
|
||||
Bootstrap replay, expiry, worker-ID mismatch, or account mismatch fails closed.
|
||||
|
||||
## Token Rotation
|
||||
|
||||
Rotation is a two-token handoff:
|
||||
|
||||
1. Control plane issues a replacement token and keeps the old token valid for a
|
||||
short bounded overlap.
|
||||
2. Worker atomically writes the replacement secret.
|
||||
3. Worker closes only the active control-plane connection and reconnects.
|
||||
4. Successful authentication acknowledges rotation; control plane revokes the
|
||||
old token.
|
||||
5. Failure retains the old token until overlap expires and emits an operator
|
||||
warning.
|
||||
|
||||
The current `Runner.RotateToken` terminates the runner by closing its stop
|
||||
channel. This must be fixed before claiming unattended rotation.
|
||||
|
||||
## Runtime Config Authentication
|
||||
|
||||
Private-worker init/config payloads add:
|
||||
|
||||
- worker ID and immutable account ID;
|
||||
- monotonic config version;
|
||||
- issue and expiry timestamps;
|
||||
- allowed check kinds, notification methods, and public-task policy;
|
||||
- credential-set hash;
|
||||
- Ed25519 signature and key ID.
|
||||
|
||||
The worker pins the control-plane verification key at bootstrap. It rejects
|
||||
signature failure, downgrade, expiry, account change, and unknown critical
|
||||
fields. Credentials remain in memory and are cleared when their signed scope
|
||||
expires.
|
||||
|
||||
## Public Checks
|
||||
|
||||
Cross-account public-check execution is not the same as normal private scope.
|
||||
It requires a separately signed `public_tasks` grant with:
|
||||
|
||||
- explicit opt-in by worker owner;
|
||||
- allowed methods limited to safe public probes, initially HTTP GET/HEAD;
|
||||
- no credentials, custom authorization headers, request bodies, private IP
|
||||
targets, or notification tasks;
|
||||
- independent concurrency and rate limits;
|
||||
- SSRF validation after every DNS resolution and redirect;
|
||||
- auditable reward/usage identity owned by the control plane.
|
||||
|
||||
Do not represent public permission by making the private worker's account list
|
||||
empty or adding arbitrary account IDs.
|
||||
|
||||
## Deployment Modes
|
||||
|
||||
The systemd and Compose packages are both supported. Capabilities, not install
|
||||
type, control host access:
|
||||
|
||||
- base: normal network checks and delegated notifications;
|
||||
- host metrics: procfs and statfs access;
|
||||
- inventory: read-only process/system discovery;
|
||||
- Docker discovery: explicit socket access, treated as root-equivalent;
|
||||
- Compose mutation: separate high-risk capability, disabled by default;
|
||||
- Raft voter: durable fsync-capable cluster data directory and mTLS transport.
|
||||
|
||||
`CAP_NET_RAW` is granted only for ping/traceroute features. The worker remains
|
||||
unprivileged otherwise.
|
||||
|
||||
## Worker Self-Monitoring
|
||||
|
||||
A worker must not run a monitor that represents its own process or host. The
|
||||
control plane excludes it during selection; the worker also rejects a task that
|
||||
names its own worker/host identity once those fields are signed into the task.
|
||||
Peer health checks and control-plane selfcheck are separate from customer
|
||||
monitor execution.
|
||||
|
||||
## Implementation Work Packages
|
||||
|
||||
1. Add signed account/config identity to `internal/wire` and runner state.
|
||||
2. Validate task account and credential scope locally before dispatch.
|
||||
3. Implement one-time bootstrap and atomic token storage/rotation.
|
||||
4. Add worker disable/revoke behavior and visible stale-config state.
|
||||
5. Add mTLS as an optional first transport, then require it for Raft clusters.
|
||||
6. Add public-task grant and SSRF-safe executor only after private isolation is
|
||||
proven.
|
||||
|
||||
## Acceptance Tests
|
||||
|
||||
- A private worker never leases or executes another account's normal task.
|
||||
- A forged capability or account field cannot widen scope.
|
||||
- Invalid, expired, downgraded, or differently scoped signed config is rejected.
|
||||
- Bootstrap tokens are single-use and absent from disk after exchange.
|
||||
- Rotation reconnects without stopping web, inventory, metrics, or cluster.
|
||||
- Credential snapshots contain only the worker account and allowed methods.
|
||||
- Public execution cannot reach loopback, link-local, RFC1918, metadata, Unix
|
||||
sockets, or a private redirect target.
|
||||
- Revocation prevents reconnect and clears in-memory credentials.
|
||||
Ссылка в новой задаче
Block a user