Network Diagnostics

Current Capability

Two different features must not be conflated:

Cross-worker confirmation is partially implemented. The control plane can assign the same failed check to another operated worker, and peer selfcheck/quorum helpers exist in internal/distworker/peer.go, consensus.go, and selfcheck.go.
Rich diagnostic tasks (diag_http, diag_ssh, DNS, TCP, traceroute, MTR) are not implemented as worker protocol kinds.

Confirmation remains a normal centrally leased check and does not require a new executor. Rich diagnostics require typed tasks and structured timing/path results.

Security Boundary

Diagnostics can become a network scanner. Every diagnostic task must be bound to a monitor target already authorized and normalized by the control plane. The worker receives the normalized target plus a signed target hash and refuses runtime overrides.

Required restrictions:

private worker account must equal task account;
source worker must not equal a target worker represented by the monitor;
no arbitrary host, URL, resolver, port range, or shell arguments;
resolve every hostname locally and reject prohibited IPs before connecting;
revalidate every HTTP redirect target;
enforce per-task timeout, output limit, concurrency, and destination rate;
never return authorization headers, request bodies, response bodies, SSH credentials, or environment data;
audit allow and deny decisions on the control plane.

Target Package

internal/netdiag/
  types.go
  validate.go
  http.go
  ssh.go
  tcp.go
  dns.go
  traceroute_linux.go
  mtr.go

internal/netdiag accepts a validated immutable request and returns a bounded result. It does not select workers, query monitors, persist data, or send protocol frames. internal/distworker owns task dispatch and result transport.

Protocol

Extend wire.TaskEnvelope with type=diagnostic and a dedicated diagnostic branch. Do not encode diagnostics as check settings or untyped events.

Common request fields:

job ID and exact lease token;
account, monitor, source worker, optional target worker, and region IDs;
kind and scheduling reason (periodic, on_demand, failure_triggered);
normalized target and target hash;
timeout and kind-specific bounded options.

Common result fields:

job ID, lease token, kind, source and target IDs;
start/finish time and total latency;
ok or err status;
normalized error class;
bounded kind-specific data.

Error classes are timeout, dns_error, tcp_refused, tcp_reset, tls_handshake, tls_cert, http_status, auth_required, auth_failed, protocol_error, target_denied, unsupported_kind, and internal_error.

Delivery Phases

D1: HTTP Diagnostics

Implement GET/HEAD only. Capture DNS, connect, TLS, first-byte, total timing, status, final URL, bounded redirect chain, selected response headers, response size, TLS version/cipher, and certificate fingerprints. Do not return body content. Clamp total timeout to 30 seconds.

D2: SSH Diagnostics

Perform banner, key exchange, and host-key inspection without authentication. Return DNS/connect/KEX timing, banner, selected algorithms, and SHA-256 host-key fingerprint. Password and private-key authentication are not part of v1.

D3: TCP And DNS

TCP performs one destination connect and distinguishes timeout, refused, and reset. DNS resolves only the monitor hostname with the system or explicitly allowlisted resolver and returns bounded A/AAAA/CNAME answers and timing.

D4: Traceroute And MTR

Linux-only, capability-gated, and disabled for private workers until abuse review. Use a native bounded implementation or a fixed executable with fixed arguments; never pass user strings to a shell. Return a maximum hop count and bounded probes per hop.

Relationship To Raft

Normal diagnostic tasks remain centrally leased. Do not replicate raw diagnostic results in Raft. If duplicate failure-triggered diagnostics become a measured problem, the critical cluster may replicate only compact dedup metadata; this is not a prerequisite for diagnostic v1.

Acceptance Tests

Existing confirmation excludes the original failing worker and handles duplicate results idempotently.
Diagnostic tasks cannot target a host or port different from their signed monitor target.
HTTP redirects to prohibited addresses are rejected.
Private workers reject cross-account tasks before opening a socket.
HTTP and SSH fixture servers produce deterministic phase timings and bounded metadata.
Timeouts cancel DNS/connect/TLS/read work and leave no goroutine behind.
Unsupported kinds return a structured terminal result.
Result serialization contains no credentials or response body.

4.7 KiB Исходник Ответственный История