docs: migrate worker implementation plans

docs/tasks-and-notifications.md

@@ -0,0 +1,131 @@
+# Normal Tasks And Notifications
+
+## Normal Check Execution
+
+Normal work is centrally scheduled. The control plane owns PostgreSQL task
+rows, selection, leases, retries, deadlines, and dead-letter state. The worker
+owns only one execution attempt and its terminal report.
+
+The implemented path is:
+
+1. `internal/distworker/client.go` opens the authenticated WebSocket.
+2. `internal/distworker/runner.go` selects a recognized populated task branch.
+3. A bounded worker pool executes the task under its deadline.
+4. `internal/checkexec/exec.go` dispatches to a database-free check package.
+5. The runner queues one result and the result writer returns it to the control
+   plane with the exact lease token.
+
+Implemented check kinds are HTTP, SSL, SSH, FTP, DNS, WHOIS, BSSL, LLM,
+LLM-HTTP, ping, TCP, and UDP. Control-plane selection uses capabilities. The
+local dispatcher supports only its explicit switch cases, but strict malformed
+envelope and unsupported-kind result handling remains work package N1 rather
+than a complete fail-closed protocol response.
+
+## Queue And Shutdown Rules
+
+- Input and output queues are bounded. Backpressure must not create unbounded
+  goroutines or memory use.
+- Concurrency is supplied by the control plane but clamped by the local
+  maximum.
+- A task panic is recovered at the task boundary and reported as a failed
+  attempt; it must not kill the runner.
+- SIGTERM stops accepting tasks, allows bounded in-flight completion, attempts
+  final result delivery, and then exits.
+- WebSocket reconnect does not re-run an in-flight or completed task.
+- HTTP polling remains compatibility-only and must not become a second normal
+  scheduler.
+
+## Delegated Notifications
+
+`internal/distworker/notification.go` executes one pre-rendered delivery
+attempt. It does not render alert policy and does not own retry scheduling.
+
+| Method | State | Credential source |
+| --- | --- | --- |
+| Email | Implemented | scoped SMTP list from init/config |
+| Telegram | Implemented | scoped bot list from init/config |
+| Webhook | Implemented | current shared signing configuration |
+| Mattermost | Implemented | current shared defaults/task endpoint |
+| SMS | Explicitly unsupported | no provider selected |
+| Voice | Explicitly unsupported | no provider selected |
+
+SMS and voice return `permanent` with `unsupported_method`; they must never be
+reported as delivered or silently dropped.
+
+Worker result statuses are `delivered`, `retryable`, `permanent`, and
+`partial`. Transport failures, provider rate limits, and provider 5xx responses
+are retryable. Invalid recipients, authentication failures, malformed payloads,
+and unsupported methods are permanent. Positive provider retry-after values
+are included in the result; the control plane owns the actual retry time.
+
+## Credentials
+
+Credentials arrive in `WorkerInit.Credentials` and remain in memory. SMTP and
+Telegram tasks may request a credential ID; the worker resolves only that ID
+from the authorized pushed set. Missing IDs are permanent failures. Do not
+fall back to another account's or platform credential.
+
+The following fields must be redacted from logs and UI buffers: password,
+token, API key, signing secret, hook URL, authorization headers, and fields
+ending in `_secret`. Provider response text is bounded and must not include
+response bodies that could contain secrets.
+
+Webhook and Mattermost need persisted per-account credentials in the control
+plane before they can claim the same isolation guarantees as SMTP and
+Telegram. The worker wire shape already separates method configuration; do not
+introduce environment-global fallback for private workers.
+
+## Selfcheck Notifications
+
+System selfcheck alerts are not normal notification tasks. They allow a worker
+to notify configured system contacts when the control plane itself is
+unreachable. They use system contacts and credentials from config, local
+jitter, and local deduplication. They must not impersonate customer delivery or
+mutate control-plane message state.
+
+The local notifications page currently records selfcheck deliveries. Normal
+delegated delivery attempts should be added to the same bounded view after
+redaction, with job ID, method, status, duration, and time only.
+
+## Implementation Work Packages
+
+### N1: Runner Integration Coverage
+
+Add strict envelope/inner job-ID validation and structured unsupported-kind
+results. Add WebSocket integration tests for reconnect, task panic recovery,
+queue backpressure, result resend, stale lease behavior, malformed envelopes,
+unsupported kinds, and graceful drain. Target
+`internal/distworker/runner_protocol_test.go` and a local test WebSocket server.
+
+### N2: Complete Local Delivery Audit
+
+Record delegated notification outcomes in the bounded local notification ring.
+Never store recipient values, rendered body, credentials, or full provider
+responses in SQLite. Verify `/notifications` shows both selfcheck and delegated
+attempts without leaking secrets.
+
+### N3: Per-Account Webhook And Mattermost Credentials
+
+Once the control plane sends credential IDs and account-scoped records, require
+an exact credential match just like SMTP and Telegram. Remove global fallback
+for private workers. Add cross-account negative tests.
+
+### N4: SMS And Voice Providers
+
+Select a provider and define a credential wire type before adding an executor.
+Implementation must include timeout, retry classification, provider response
+redaction, idempotency support, and contract tests. Until all are present,
+retain the explicit permanent unsupported result.
+
+## Acceptance Tests
+
+- Every supported check kind can execute from a serialized task without a
+  database connection.
+- A stale or missing lease token cannot produce an accepted normal result.
+- One task causes at most one local execution during reconnect and duplicate
+  frame delivery.
+- Private workers cannot resolve a credential outside their authorized set.
+- SMTP 4xx/rate-limit conditions are retryable; invalid authentication is
+  permanent.
+- SMS and voice remain explicit failures until provider tests exist.
+- Logs and local UI contain no pushed credential values.