worker/docs/changelog.md

# Changelog

## 2026-08-13

### Source-install E2E in CI (work package 5)

- Added `.github/workflows/test-ssh.yml`, a Gitea Actions workflow that runs
  the Alpine/Ubuntu/Arch Docker/OpenSSH source-install matrix (`make test-ssh`)
  on pushes to `master` and on manual `workflow_dispatch` only, separate from
  the Docker-free unit CI. It does not trigger on `pull_request`: the distro
  fixtures execute the checked-out code inside privileged Docker, so untrusted
  PR code must never run there automatically. It bounds the job with
  `timeout-minutes: 90` (the go-test `-timeout 60m` stays in place), scopes
  concurrency per ref (`test-ssh-${{ gitea.ref }}`), declares
  `permissions: contents: read`, uploads no artifacts, and cleans up on every
  path.
- Pinned the two actions to immutable full commit SHAs (verified against the
  GitHub API): `actions/checkout@v4` ->
  `11d5960a326750d5838078e36cf38b85af677262` and `actions/setup-go@v5` ->
  `40f1582b2485089dde7abd97c1529aa768e1baff`. The repo-wide convention still
  leaves `ci.yml`/`docker.yml` on moving tags (accepted, documented risk); see
  `docs/source-installation.md`.
- Wired `RSMON_TEST_IMAGE_ALPINE` / `RSMON_TEST_IMAGE_UBUNTU` /
  `RSMON_TEST_IMAGE_ARCH` and `RSMON_TEST_DOCKER_DNS` repository variables
  (all empty by default) so CI can pin per-fixture mirror/snapshot images and
  a resolver for flaky CI DNS.
- Added `scripts/ci/test-ssh.sh`: preflights Docker and the harness's
  loopback port-publishing requirement with a tiny `docker run -p
  127.0.0.1::22` probe (fails fast with an actionable message plus
  diagnostics/fix options instead of a 60m timeout on an unsupported runner),
  then runs `make test-ssh` and traps `EXIT` to remove every leftover
  `rsmon-worker-test-*` container/network/image tag. Cleanup filters are
  anchored to the harness's own prefix/repository so they never touch a shared
  base image. The workflow adds an `if: always()` cleanup step as a
  belt-and-suspenders so a killed job never leaves test material on the runner.
- External network is fetched live by design (go.dev toolchain, rocketgit.ru
  source clone, distro repos); operators can pin a resolver via the
  `RSMON_TEST_DOCKER_DNS` repository variable (comma-separated nameservers,
  applied as `docker run --dns ...`) and mirror/snapshot overrides via
  `RSMON_TEST_IMAGE_ALPINE` / `RSMON_TEST_IMAGE_UBUNTU` /
  `RSMON_TEST_IMAGE_ARCH`.
- Fixed a history-dependent test fragility: `TestSourceInstallDirtyCheckoutPreservesStaging`
  and the rollback test's build-failure step dirty the tracked tree by
  appending a marker line to `Makefile` instead of `git checkout master~1 --
  Makefile`, which silently stopped dirtying the tree once the last commit did
  not touch that file.

## 2026-08-12

### Source-install hardening review

- Fail-closed remote scripts: checkout, branch resolution, and build steps now
  run under `set -eu` (and package/record steps chain with `&&`), so a failed
  checkout or fetch can never be masked by a stale `rev-parse` or subsequent
  command. The checkout step additionally refuses (`git diff --quiet` /
  `--cached --quiet`) before the destructive `checkout -B`, because `-B`
  silently discards local changes and would otherwise never fail on a dirty
  tree. A dirty-tree checkout failure surfaces as a `check out branch` error
  before the build runs; the new `TestSourceInstallSSHCheckoutFailureNotMasked`
  unit test and `TestSourceInstallDirtyCheckoutPreservesStaging` Docker test
  prove the previous staging binary and commit record are preserved
  byte-for-byte.
- Atomic toolchain replacement: the Go toolchain is downloaded, SHA-256
  verified, extracted into a same-filesystem staging dir, verified to report
  the target version, and only then swapped into `ToolchainDir` with the prior
  toolchain moved to a sibling `.go-backup` that is restored on swap failure.
  A failed download/verify/extract/swap never destroys the prior Go.
- Record-after-build pairing: `rsmon-worker.commit` is written only after a
  successful build, so the record and the staged binary always correspond to
  the same commit. The build verifies `<stage>.new --version` before an atomic
  `mv -f` over the previous staging binary; `GOMODCACHE` is now set alongside
  `GOCACHE` inside the build dir so reruns reuse both caches.
- Origin verification: an existing checkout's `remote.origin.url` must exactly
  match the configured repository before anything is fetched or built.
- Repository hardening: only `https://` clone URLs without userinfo are
  accepted (`ValidateRepoURL`, enforced before dialing and again when
  planning).
- Explicit charset validation for Go version and architecture overrides
  (`sshinstall.ValidGoVersion` / `ValidGoArch`) before any remote mutation.
- Bounded remote execution: each remote command is capped by
  `--session-timeout` (default 30m) and captured stdout is size-bounded
  alongside the existing stderr bound; deploy's streaming `runRemote` keeps its
  historical no-timeout behavior.
- The source installer now defaults to the remote's default branch (the public
  repo publishes `master`) instead of the plan's stale `main` default, while
  `--branch` still pins an explicit branch that must exist remotely. The
  README quickstart no longer shows the incorrect `--branch main`.
- CLI secret flags keep their compatibility, but docs now explicitly state that
  file options (`-password-file`, etc.) keep secrets out of argv and shell
  history while direct flags expose them through the process list.
- The harness accepts `RSMON_TEST_DOCKER_DNS` (comma-separated) to pin
  `docker run --dns` for fixture containers, so internet-facing installs are
  not at the mercy of a flaky local resolver.

### Remote source-install execution (work package 3)

- Added `installer.SourceInstall` (`internal/installer/sourceinstall.go`):
  executes the source-install flow through the existing SSH transport,
  reusing the `deploy` command's `SSHOptions` (keys, passphrases,
  passwords, sudo passwords, known-hosts, pinned fingerprints) and its
  privilege path. Extracted the shared `SSHOptions` struct and a
  `sudoWrap` helper so deploy and source install cannot diverge.
- Steps implemented: minimal package-prerequisite install per distro
  (`apk`/`apt`/`pacman`/`dnf`, never a compiler), SHA-256-verified Go 1.26
  toolchain download/extraction with an idempotent version-skip and temp-dir
  cleanup, clone-or-update of the public repository (with a bounded 3-attempt
  retry for transient DNS/TLS/proxy failures), resolution of the remote
  default branch (a pinned branch must exist remotely), a resolved branch and
  commit record at `<BuildDir>/rsmon-worker.commit`, and a staging build
  (`CGO_ENABLED=0`, `-trimpath`, repository `-ldflags`) verified via
  `--version`. The running service, config, and data directory are untouched
  (work package 4 boundary).
- Security: every interpolated remote value is single-quoted; branch and
  commit values are strictly validated; no worker token or control-plane
  credential is sent; sudo passwords travel only over session stdin; remote
  errors are bounded (stderr truncated in `runRemoteOutput`).
- Added unit tests for the remote scripts, option validation, branch/commit
  parsing, sudo wrapping, the secrets-absent contract, and an in-process
  real-SSH orchestration flow (with missing-pinned-branch, build-failure, and
  detection-failure paths).
- Added `TestSourceInstallFixtures` to the Docker/OpenSSH harness: each of
  Alpine, Ubuntu, and Arch installs from a clean state through the real
  harness transport (prerequisite install, verified Go 1.26, clone, resolved
  commit, staging build), then a rerun proves idempotency (same branch,
  toolchain reuse, no temp leaks). All three resolved the public repo's
  `master` at `4651deb2...` in the recorded run. `make test-ssh` timeout
  raised to 60m.
- Documented the branch-resolution reality: the public repository currently
  publishes `master`, and the installer records whatever the remote default
  branch resolves to.

### Source-install harness and planning foundations (work packages 1-2)

- Added `internal/installer/harness`: a reusable Docker/OpenSSH test harness
  that builds real OpenSSH containers for Alpine, Ubuntu, and Arch, waits for
  real network SSH readiness, captures the server host key into a temp
  `known_hosts` file, and tears the container, network, per-instance fixture
  image tag, and temp dir down reliably. It uses the `golang.org/x/crypto/ssh`
  library and known_hosts verification semantics the installer's `deploy` path
  relies on (the harness owns its connection code rather than reusing the
  installer functions) and never mocks SSH. A fresh known_hosts file trusts
  the first key (TOFU); the host-key mismatch test proves a different key is
  rejected before any command runs.
- Added distro fixtures under `internal/installer/harness/testdata/fixtures`.
  Alpine defaults to the `reg.rsxx.ru/library/alpine:3` mirror; Ubuntu and Arch
  fall back to Docker Hub refs overridable via `RSMON_TEST_IMAGE_<NAME>`.
  Each fixture starts clean (no Go, no worker source) and authenticates with a
  bundled test key; password auth is disabled. The test key is strictly
  test-only - it grants root only to the disposable fixture containers - and
  must never be used outside the harness.
- Added opt-in integration controls: the Docker tests run only with
  `RSMON_TEST_DOCKER=1` (`make test-ssh`); default `make test` and `go test
  ./...` skip them and never pull or start containers. `make test` also pins
  `RSMON_TEST_DOCKER=0` so an exported opt-in flag cannot leak into the unit
  run.
- Integration tests assert real SSH round trips, clean target state, distro /
  package-manager / init detection per fixture, a full source plan including
  the pinned Go toolchain, host-key mismatch rejection, host-key stability,
  failed-start cleanup, and complete teardown (container, network, fixture
  image tag, and temp dir gone).
- Added `internal/sshinstall`: pure, unit-tested detection and planning for the
  source installer - os-release parsing, distro/package-manager/init
  resolution, `uname -m` to Go archive mapping, pinned Go 1.26 toolchain with
  published SHA-256, and a reviewable ordered plan. No remote execution yet.
- `make test` now includes the new packages; `make test-ssh` runs the live
  fixture matrix.

### Public endpoint configuration (milestone 1 of public-endpoint-and-identity)

- `PUBLIC_URL` is now the canonical advertised public origin; the legacy
  `WORKER_URL` is accepted only for the bounded migration and logs a startup
  deprecation warning. `PUBLIC_URL` wins whenever both are set, and the
  installer drops `WORKER_URL` from freshly written env files when
  `PUBLIC_URL` is present.
- Startup and install validate the origin shape: absolute `http`/`https` URL
  with scheme and authority only; userinfo, query, fragment, and any path
  other than `/` are rejected.
- Plain-HTTP `PUBLIC_URL` on a non-loopback host is rejected in an explicitly
  production environment (`DEPLOY_ENV`, `RSMON_ENV`, or `GO_ENV` =
  `production`); other environments keep the historical warning.
- `internal/wire` adds `public_url` to `WorkerInit` (control plane to worker),
  keeping the legacy `url` field for old control planes; the worker prefers
  `public_url` and rejects unusable values, keeping the previous accepted
  URL. `RegisterRequest.public_url` is the registration contract for the
  pending RSMon counterpart (the worker does not currently transmit the URL
  during registration; it consumes the accepted endpoint from `WorkerInit`).
- The legacy `WORKER_URL` is held only to the tolerant absolute-URL check
  (no newly rejected legacy shapes); `PUBLIC_URL` is held to the strict
  scheme-and-authority origin shape. Both reject a missing hostname, e.g.
  `https://:27401`.

## 2026-07-19

### Standalone installation and deployment

- Added `rsmon-worker install` for installing the current binary, a mode-0600
  environment file, and a root-owned systemd service.
- Added `rsmon-worker deploy` for installing workers over SSH with key or
  password authentication, optional secret files, and host-key verification
  through `known_hosts` or a pinned fingerprint.
- Added `--token-file`, `--url`, `--api-key`, and `--no-start` deployment
  options.
- Added optional `--docker` deployment using a prebuilt image. Docker install
  and deploy now require an immutable `repository@sha256:...` reference before
  any Docker or remote-host mutation; the former mutable `latest` default is no
  longer accepted.
- Simplified the default systemd service to `Type=simple`, `User=root`, and
  `Restart=on-failure`.
- Reworked the legacy `scripts/install-systemd.sh` script as a compatibility
  wrapper around the built-in installer.

### Standalone repository cleanup

- Removed the remaining certificate-bundle fallback under `/data/rsmon` and
  updated its documentation.
- Changed Docker build and runtime bases to public Go and Debian images.
- Documented binary, systemd, Docker, and SSH deployment workflows.
- Added installer and SSH host-verification tests.

### Verification

- Passed `make test`, `make build`, `go vet`, and `go mod verify`.
- Validated systemd units, Compose configuration, Docker image pull and
  execution, and production worker job reporting.
- Published the changes as commit `3256dcd` (`feat: add worker install and
  deploy`) on `master`, triggering the Docker image workflow.

### Task protocol and local audit hardening

- Required one task-envelope branch, matching outer/inner job IDs, and a
  non-empty lease token before local execution.
- Added structured terminal failures for unsupported check kinds and safely
  attributable malformed envelopes.
- Recorded delegated notification outcomes in the bounded `/notifications`
  view using only job ID, method, status, duration, and time.
- Added static permanent handling for invalid deadlines and recovered
  notification executor panics without retaining secret-bearing text.
- Removed the critical-cluster test-config endpoint, CLI flag, environment
  switch, and production helper; hardcoded config application is test-only.
- Made runner token rotation connection-scoped and in-memory: it reconnects
  without stopping web, inventory, metrics, or cluster subsystems. Durable token
  storage, bootstrap exchange, rotation acknowledgement, and revocation remain
  unimplemented.
- Added bounded resend of dequeued check and notification result envelopes after
  websocket reconnect; control-plane application remains at-least-once and must
  deduplicate by leased job and lease token. Failed metric snapshots are dropped
  and replaced by the next periodic tick, not replayed.
- SIGTERM stops new dispatch and waits for active work, but stale-lease
  acknowledgement, bounded graceful final-result drain, and duplicate-frame
  coverage remain open.