fix(worker): harden control-plane lifecycle
Все проверки выполнены успешно
CI / test (push) Successful in 2m32s
Docker / Build and publish worker image (push) Successful in 18m17s
Все проверки выполнены успешно
CI / test (push) Successful in 2m32s
Docker / Build and publish worker image (push) Successful in 18m17s
- reconnect safely after token rotation and retry leased results - reject malformed tasks and remove production cluster debug mutation - validate environment files and require immutable container images BREAKING CHANGE: Docker install, deploy, and Compose now require an immutable repository@sha256 image reference.
Этот коммит содержится в:
@@ -17,9 +17,11 @@ The implemented path is:
|
||||
|
||||
Implemented check kinds are HTTP, SSL, SSH, FTP, DNS, WHOIS, BSSL, LLM,
|
||||
LLM-HTTP, ping, TCP, and UDP. Control-plane selection uses capabilities. The
|
||||
local dispatcher supports only its explicit switch cases, but strict malformed
|
||||
envelope and unsupported-kind result handling remains work package N1 rather
|
||||
than a complete fail-closed protocol response.
|
||||
runner and local executor share one supported-kind registry. Websocket task
|
||||
envelopes require exactly one payload branch, matching non-empty outer/inner
|
||||
job IDs, and a lease token. Unsupported checks return one terminal failed
|
||||
result without execution. A safely attributable malformed single branch
|
||||
returns a protocol failure; ambiguous and mismatched envelopes fail closed.
|
||||
|
||||
## Queue And Shutdown Rules
|
||||
|
||||
@@ -29,8 +31,9 @@ than a complete fail-closed protocol response.
|
||||
maximum.
|
||||
- A task panic is recovered at the task boundary and reported as a failed
|
||||
attempt; it must not kill the runner.
|
||||
- SIGTERM stops accepting tasks, allows bounded in-flight completion, attempts
|
||||
final result delivery, and then exits.
|
||||
- SIGTERM stops accepting new tasks and waits for active dispatchers before the
|
||||
execution pool closes. A bounded final-result drain across websocket shutdown
|
||||
is not implemented yet.
|
||||
- WebSocket reconnect does not re-run an in-flight or completed task.
|
||||
- HTTP polling remains compatibility-only and must not become a second normal
|
||||
scheduler.
|
||||
@@ -83,26 +86,27 @@ unreachable. They use system contacts and credentials from config, local
|
||||
jitter, and local deduplication. They must not impersonate customer delivery or
|
||||
mutate control-plane message state.
|
||||
|
||||
The local notifications page currently records selfcheck deliveries. Normal
|
||||
delegated delivery attempts should be added to the same bounded view after
|
||||
redaction, with job ID, method, status, duration, and time only.
|
||||
The local notifications page records both selfcheck and delegated delivery
|
||||
attempts. Delegated rows contain job ID, method, status, duration, and time
|
||||
only. Recipient, body, credentials, hook URL, authorization, provider response,
|
||||
and error text are never copied into the bounded ring. Invalid deadlines and
|
||||
recovered executor panics produce static permanent results and one redacted
|
||||
local row.
|
||||
|
||||
## Implementation Work Packages
|
||||
|
||||
### N1: Runner Integration Coverage
|
||||
### N1: Runner Integration Coverage (partial)
|
||||
|
||||
Add strict envelope/inner job-ID validation and structured unsupported-kind
|
||||
results. Add WebSocket integration tests for reconnect, task panic recovery,
|
||||
queue backpressure, result resend, stale lease behavior, malformed envelopes,
|
||||
unsupported kinds, and graceful drain. Target
|
||||
`internal/distworker/runner_protocol_test.go` and a local test WebSocket server.
|
||||
Strict envelope/inner job-ID/lease validation and structured unsupported-kind
|
||||
results are implemented with runner-boundary tests. Live WebSocket reconnect
|
||||
and result-resend tests are implemented. Stale-lease acknowledgement and
|
||||
bounded graceful final-drain tests remain.
|
||||
|
||||
### N2: Complete Local Delivery Audit
|
||||
### N2: Complete Local Delivery Audit (implemented)
|
||||
|
||||
Record delegated notification outcomes in the bounded local notification ring.
|
||||
Never store recipient values, rendered body, credentials, or full provider
|
||||
responses in SQLite. Verify `/notifications` shows both selfcheck and delegated
|
||||
attempts without leaking secrets.
|
||||
Delegated notification outcomes are recorded in the bounded local notification
|
||||
ring. `/notifications` renders selfcheck and delegated attempts without storing
|
||||
recipient values, rendered body, credentials, provider responses, or errors.
|
||||
|
||||
### N3: Per-Account Webhook And Mattermost Credentials
|
||||
|
||||
|
||||
Ссылка в новой задаче
Block a user