fd1a010e31
Все проверки выполнены успешно
CI / test (push) Successful in 30s
Docker / Build and publish worker image (push) Successful in 10m49s
128 строки
4.7 KiB
Markdown
128 строки
4.7 KiB
Markdown
# Network Diagnostics
|
|
|
|
## Current Capability
|
|
|
|
Two different features must not be conflated:
|
|
|
|
1. **Cross-worker confirmation** is partially implemented. The control plane
|
|
can assign the same failed check to another operated worker, and peer
|
|
selfcheck/quorum helpers exist in `internal/distworker/peer.go`,
|
|
`consensus.go`, and `selfcheck.go`.
|
|
2. **Rich diagnostic tasks** (`diag_http`, `diag_ssh`, DNS, TCP, traceroute,
|
|
MTR) are not implemented as worker protocol kinds.
|
|
|
|
Confirmation remains a normal centrally leased check and does not require a
|
|
new executor. Rich diagnostics require typed tasks and structured timing/path
|
|
results.
|
|
|
|
## Security Boundary
|
|
|
|
Diagnostics can become a network scanner. Every diagnostic task must be bound
|
|
to a monitor target already authorized and normalized by the control plane.
|
|
The worker receives the normalized target plus a signed target hash and refuses
|
|
runtime overrides.
|
|
|
|
Required restrictions:
|
|
|
|
- private worker account must equal task account;
|
|
- source worker must not equal a target worker represented by the monitor;
|
|
- no arbitrary host, URL, resolver, port range, or shell arguments;
|
|
- resolve every hostname locally and reject prohibited IPs before connecting;
|
|
- revalidate every HTTP redirect target;
|
|
- enforce per-task timeout, output limit, concurrency, and destination rate;
|
|
- never return authorization headers, request bodies, response bodies, SSH
|
|
credentials, or environment data;
|
|
- audit allow and deny decisions on the control plane.
|
|
|
|
## Target Package
|
|
|
|
```text
|
|
internal/netdiag/
|
|
types.go
|
|
validate.go
|
|
http.go
|
|
ssh.go
|
|
tcp.go
|
|
dns.go
|
|
traceroute_linux.go
|
|
mtr.go
|
|
```
|
|
|
|
`internal/netdiag` accepts a validated immutable request and returns a bounded
|
|
result. It does not select workers, query monitors, persist data, or send
|
|
protocol frames. `internal/distworker` owns task dispatch and result transport.
|
|
|
|
## Protocol
|
|
|
|
Extend `wire.TaskEnvelope` with `type=diagnostic` and a dedicated diagnostic
|
|
branch. Do not encode diagnostics as check settings or untyped events.
|
|
|
|
Common request fields:
|
|
|
|
- job ID and exact lease token;
|
|
- account, monitor, source worker, optional target worker, and region IDs;
|
|
- kind and scheduling reason (`periodic`, `on_demand`, `failure_triggered`);
|
|
- normalized target and target hash;
|
|
- timeout and kind-specific bounded options.
|
|
|
|
Common result fields:
|
|
|
|
- job ID, lease token, kind, source and target IDs;
|
|
- start/finish time and total latency;
|
|
- `ok` or `err` status;
|
|
- normalized error class;
|
|
- bounded kind-specific data.
|
|
|
|
Error classes are `timeout`, `dns_error`, `tcp_refused`, `tcp_reset`,
|
|
`tls_handshake`, `tls_cert`, `http_status`, `auth_required`, `auth_failed`,
|
|
`protocol_error`, `target_denied`, `unsupported_kind`, and `internal_error`.
|
|
|
|
## Delivery Phases
|
|
|
|
### D1: HTTP Diagnostics
|
|
|
|
Implement GET/HEAD only. Capture DNS, connect, TLS, first-byte, total timing,
|
|
status, final URL, bounded redirect chain, selected response headers, response
|
|
size, TLS version/cipher, and certificate fingerprints. Do not return body
|
|
content. Clamp total timeout to 30 seconds.
|
|
|
|
### D2: SSH Diagnostics
|
|
|
|
Perform banner, key exchange, and host-key inspection without authentication.
|
|
Return DNS/connect/KEX timing, banner, selected algorithms, and SHA-256 host-key
|
|
fingerprint. Password and private-key authentication are not part of v1.
|
|
|
|
### D3: TCP And DNS
|
|
|
|
TCP performs one destination connect and distinguishes timeout, refused, and
|
|
reset. DNS resolves only the monitor hostname with the system or explicitly
|
|
allowlisted resolver and returns bounded A/AAAA/CNAME answers and timing.
|
|
|
|
### D4: Traceroute And MTR
|
|
|
|
Linux-only, capability-gated, and disabled for private workers until abuse
|
|
review. Use a native bounded implementation or a fixed executable with fixed
|
|
arguments; never pass user strings to a shell. Return a maximum hop count and
|
|
bounded probes per hop.
|
|
|
|
## Relationship To Raft
|
|
|
|
Normal diagnostic tasks remain centrally leased. Do not replicate raw
|
|
diagnostic results in Raft. If duplicate failure-triggered diagnostics become a
|
|
measured problem, the critical cluster may replicate only compact dedup
|
|
metadata; this is not a prerequisite for diagnostic v1.
|
|
|
|
## Acceptance Tests
|
|
|
|
- Existing confirmation excludes the original failing worker and handles
|
|
duplicate results idempotently.
|
|
- Diagnostic tasks cannot target a host or port different from their signed
|
|
monitor target.
|
|
- HTTP redirects to prohibited addresses are rejected.
|
|
- Private workers reject cross-account tasks before opening a socket.
|
|
- HTTP and SSH fixture servers produce deterministic phase timings and bounded
|
|
metadata.
|
|
- Timeouts cancel DNS/connect/TLS/read work and leave no goroutine behind.
|
|
- Unsupported kinds return a structured terminal result.
|
|
- Result serialization contains no credentials or response body.
|