wiki-browser continuous delivery — Design

Problem

With sub-project #10 (deployment & git-sync) the deployed wiki-browser participates in a shared master: it pulls document commits to serve them and pushes incorporations back. But the binary itself is still updated by hand — make build-arm64, copy to the Pi, systemctl restart (wrapped as make deploy). Every change to wiki-browser's own Go source needs a human to remember to cut a release. This sub-project removes that step: when a test-passing change to wiki-browser's code lands on master, the Pi rebuilds and redeploys itself.

There is a second, quieter gap. The two of us work trunk-based — commits go straight to master, no pull requests, and no CI. Nothing runs the test suite before code is effectively live. CD is the natural place to introduce that gate, so this spec also stands up the project's first automated test run.

Goals & non-goals

Goals

• A test-passing change to wiki-browser's own Go source on master is rebuilt and redeployed to the Pi with no human action.
• Deploys are crash-safe: an in-flight Agent job (2–3 min) drains to completion rather than being killed.
• A bad deploy never takes the wiki down: a failed build is caught before any binary is touched; a binary that will not come up healthy is auto-rolled-back to the last-known-good.
• The monorepo gets its first automated test gate for wiki-browser code — on every push, deploy or not.
• The running version (full commit + display version + build time) is visible in the UI and at /healthz.
• CD only ever reads the clone (git archive, git cat-file); git-sync stays the sole mutator of the clone's git state.

Non-goals

• High availability. One Pi. The sub-second binary swap plus the restart is the only downtime; that is acceptable.
• Database rollback. CD rolls back the binary only — see the migration warning.
• A staging environment. Tests are the only gate; a passing deploy goes straight to the one production Pi.
• A PR / branch workflow. The collaboration policy stays trunk-based.
• Multi-step rollback history. Exactly one step back (bin/prev/).
• Slack alerts on test failure. A red Action run plus GitHub's own notifications are the test-failure signal. CD's alerts cover deploy-side failures only.

Approach

CD has three moving questions: where tests run, where the binary is built, and how the Pi is triggered.

B is rejected: running the full suite on the Pi spends a couple of minutes of CPU on the serving host for every deploy, gives no real CI surface (no run history, no clean environment), and forces the Pi to diff master itself to decide what changed. C is rejected: shipping a cross-compiled artifact means artifact storage, the Pi authenticating to GitHub to fetch it, and version-matching an artifact back to a commit — too much machinery when the project is pure-Go (CGO_ENABLED=0 already) and a Raspberry Pi 5 builds it natively in seconds.

A is chosen. Tests run free, off-device, in GitHub Actions — which doubles as the CI the project never had. The Pi only ever hears "this tested commit is ready," then rebuilds natively and redeploys. Within A, four sub-decisions, all settled during discovery:

• One GitHub Action, not a native webhook. The WB Action runs on every push, diffs the exact previous-push range, classifies changed paths with the same document-tracking rules as the deployed wiki, and makes at most one request to the Pi — so a code+doc push is one webhook, not two.
• A oneshot, not a daemon. CD has no state worth keeping in memory; between deploys nothing runs, so there is nothing to crash-recover — and a oneshot updates itself for free (see CD self-update).
• systemd .path activation, not a poll. The webhook handler writes a trigger file; a .path unit turns that into a wb-cd run. wb-cd then drains the trigger state before exiting, so a second push during a long build or drain is not lost.
• Auto-rollback, not fail-forward. The bar for shipping CD is "CD cannot take the wiki down"; rollback to last-known-good is the binary-level equivalent of the crash-safety #10 insisted on.

Design

End-to-end flow

Step by step:

1. A push lands on master. The WB Action runs (every push).
2. An internal classifier — a Go helper in the wiki-browser module — diffs github.event.before..github.sha. It checks two sets: deploy build inputs, and tracked document paths — the latter via the same internal/walker matching code the running wiki uses, fed the extensions/exclude values from a committed policy file.
3. If deploy build inputs changed, the Action runs go test ./.... If only tracked documents changed, tests are skipped.
4. If neither set changed, the Action exits without calling the Pi. Otherwise it makes one HMAC-signed POST /api/webhook/ci with {deploy, commit, ref}, where deploy = code-changed && tests-passed and commit is the full 40-character SHA.
5. The endpoint handler verifies the HMAC, triggers a git-sync Sync (doc pull), and — if deploy — writes commit to the trigger file. Responds 204.
6. A systemd .path unit sees the trigger file change and activates wb-cd.service.
7. wb-cd builds the exact commit, swaps the binaries, restarts wiki-browser, health-checks, records success or rolls back, then re-reads the trigger file before exiting and repeats if a newer commit arrived mid-run.

The WB Action

One workflow, committed at the monorepo root — .github/workflows/wiki-browser.yml, not under wiki-browser/, since GitHub only reads workflows from the repo root. It is created as part of this implementation. It fires on every push to master (it must, so doc-only pushes can still reach the Pi), and discriminates internally with a path classifier rather than a workflow-level paths: trigger.

yaml — .github/workflows/wiki-browser.ymlname: wiki-browser
on: { push: { branches: [master] } }

jobs:
  wiki-browser:
    runs-on: ubuntu-latest
    steps:
      - uses: actions/checkout@v4
        with: { fetch-depth: 0 }      # github.event.before must exist
      - uses: actions/setup-go@v5         # before classify — it is a go program
        with: { go-version-file: wiki-browser/go.mod }
      - id: classify
        working-directory: wiki-browser
        run: go run ./cmd/classify-ci-change
          --base "${{ github.event.before }}"
          --head "${{ github.sha }}"
          --policy ci-tracked-paths.yaml
      - if: steps.classify.outputs.code_changed == 'true'
        working-directory: wiki-browser
        run: go test ./...
      - name: notify the pi          # one request if tracked docs or code changed
        if: always() && steps.classify.outputs.notify == 'true'
        run: # HMAC-sign {deploy,commit,ref} and POST it; deploy = code && success

The classifier must use the previous push range, not only HEAD~1..HEAD: base = github.event.before, head = github.sha. For the all-zero first-push sentinel, fall back to the root commit range. This catches multi-commit pushes correctly.

It emits three booleans: code_changed, tracked_changed, and notify. code_changed covers deploy build inputs: wiki-browser/** minus docs/, .claude/, and *.md, plus the workflow and classifier files that control deployment. Templates and static assets (internal/server/templates/, static/) are baked in via embed.FS and are build inputs. tracked_changed covers any changed file the running wiki would serve: repo-relative path under root, extension in extensions, and not matched by either baked-in walker excludes or configured exclude patterns.

tracked_changed must answer exactly what the running wiki would serve, so the classifier does not reimplement that rule. The implementation extracts the path-matching predicate from internal/walker into a small reusable matcher — (repo-relative path, extensions, excludes) → tracked, with the baked-in excludes (.git, node_modules, .worktrees, .obsidian, .claude, tmp-*) living inside that shared code — and both the walker and classify-ci-change call it. The matching logic is thus one implementation and cannot drift. The classifier is a Go helper in the wiki-browser module (cmd/classify-ci-change), run via go run; it needs no filesystem walk, only the git diff --name-only path list.

The one thing still mirrored is the data. The production wiki-browser.yaml holds host paths and secret references and is not committed, so the extensions/exclude values are kept in a committed non-secret file — wiki-browser/ci-tracked-paths.yaml — read by the classifier. Only those two values must stay in step with production config; provisioning verifies that by code review. A classifier error (unreadable policy file, a failed git diff) exits non-zero and fails the Action — surfaced as a red run, never a silently dropped push.

The single webhook endpoint

The #10 Task-9 handler is rewritten. The route is renamed POST /api/webhook/ci and the payload is no longer GitHub's native push event but our own schema:

json — request body{ "deploy": true, "commit": "a1b2c3d4e5f6789012345678901234567890abcd",
  "ref": "refs/heads/master", "delivery_id": "github-run-123456789" }

It stays a public route — mounted outside the OAuth session middleware, like /auth/* — protected by HMAC: read the body under a size cap, compute HMAC-SHA256(body, secret), constant-time compare against X-Hub-Signature-256: sha256=<hex>. The secret file is the existing git.webhook_secret_file from #10 — unchanged on the Pi; only its counterpart moves (from the native-webhook config to a GitHub Actions secret). Handler behavior:

• Bad / missing signature → 401, logged.
• Bad payload — non-master ref, malformed non-40-hex commit, or oversized body → 400, logged.
• delivery_id (the GitHub Actions run id) is recorded in this delivery's log line — correlation only, no behavioral effect.
• Always: call gitSync.RequestSync() — the doc pull. This is the job the native webhook used to do.
• If deploy is true: write the full commit to cd.trigger_file (atomic write — temp file + rename).
• Respond 204 immediately; both the sync and the CD run proceed out of band.

wiki-browser needs no privilege for this — it only writes a file it owns. The systemd .path unit (running as part of systemd) does the activation.

wb-cd — the deploy oneshot

A new binary, cmd/wb-cd, built by make build alongside wiki-browser and wb-agent. It is a Go binary rather than a shell script for two reasons: the rollback logic is correctness-critical and deserves unit tests, and it reuses internal/alert for Slack notifications. It runs once per activation, but it drains trigger state before exit. One target commit loop:

1. Read the target full commit from the trigger file. If it equals the recorded deployed commit → no target is pending. If it equals the recorded poisoned commit → skip (a known-bad commit; do not retry until a new commit arrives).
2. Ensure the clone contains commit — git cat-file -e commit^{commit}, polled with a bounded wait. git-sync's Sync, kicked by the same webhook, delivers it within seconds.
3. git archive commit -- wiki-browser into a temp dir — only the one self-contained Go module, not the rest of the monorepo — and run make -C wiki-browser build COMMIT=<full-sha> VERSION=<short-sha> BUILD_TIME=<ts> there. This builds the exact approved commit, not the live working tree (git-sync keeps that at master HEAD, which may have moved past commit). git archive is a pure read of the object database — no mutation of the live clone, so the sole-mutator invariant holds.
4. Snapshot the current live binaries to bin/prev/.
5. Atomically rename() the three freshly built binaries over the live paths.
6. sudo systemctl restart wiki-browser — this blocks through the graceful drain.
7. Health-check (see below). Healthy → record commit as deployed. Restart failure or unhealthy → roll back.
8. Before exit, re-read cd.trigger_file. If it now names a different un-deployed, non-poisoned commit, start the loop again. This is what makes .path activation safe when a push lands while wb-cd.service is already active.

CD self-update

cmd/wb-cd lives under wiki-browser/, so a change to its own source is a build input like any other. Because wb-cd is a oneshot, keeping it current needs no special machinery: make -C wiki-browser build (step 3 above) compiles all three binaries, and step 5 rename()s the new wb-cd over its own path. The running process keeps its old inode and finishes cleanly; the next trigger runs the new wb-cd. There is no "hot-swap a running process" problem because between triggers nothing is running — the atomic rename() is the one hard requirement.

The residual risk is a new wb-cd that compiles but is behaviorally broken: the old (working) CD will build and install it, and the next trigger runs broken CD. Three things contain this — wb-cd carries unit tests; a systemd OnFailure= hook alerts on any non-zero CD exit; and the failure degrades gracefully — broken CD never touches wiki-browser's health, it just stops deploying, and make deploy remains as the break-glass path.

Health check & rollback

After the restart returns, wb-cd polls GET /healthz for a bounded window (default ~90 s). The check passes only when the response is 200 and the reported commit equals the full commit just built — proving not only that the process came up but that the swap actually took.

• Healthy → write {deployed_commit, deployed_at} to the state file. Done.
• Restart failed or unhealthy → roll back: rename() the bin/prev/ binaries back into place, restart again, re-verify /healthz. Fire a Slack alert (condition, commit, last error). Record commit as poisoned in the state file so a re-fired trigger for the same broken commit is skipped — CD retries only when a new commit lands, i.e. someone pushed a fix.
• Rollback itself fails (both new and previous binary unhealthy) → alert loudly and stop. No thrash loop; a human intervenes.

The state file is small JSON at cd.state_file:

json — cd-state.json{ "deployed_commit": "a1b2c3d4e5f6789012345678901234567890abcd",
  "deployed_at": "2026-05-21T14:30:00Z",
  "poisoned_commit": "" }

Graceful drain

This is a change to wiki-browser, not wb-cd. main.go today gives srv.Shutdown a 5 s budget — both too short and aimed at the wrong thing. srv.Shutdown waits for active HTTP handlers; but an Agent job runs in a background goroutine in the agent service — the POST /api/agent/jobs handler that queued it has already returned. So srv.Shutdown finishing says nothing about whether a 2–3 min job is still running. (Incorporation is fine: it runs synchronously inside its handler, so srv.Shutdown's existing wait already covers it; it is seconds anyway.)

The new SIGTERM sequence:

1. Enter process-wide draining mode. Read-only routes keep serving. New mutating HTTP requests get 503 Service Unavailable with Retry-After unless they are part of the shutdown path itself.
2. Tell the agent service to stop dequeuing new jobs. Queued jobs simply stay in the DB; the next instance drains them on boot.
3. Wait — on a WaitGroup over in-flight job goroutines — for running jobs to finish. HTTP stays up serving reads throughout, so the wiki is browsable during the drain.
4. Once jobs are drained, srv.Shutdown, close the DBs, exit.
5. A 10 min cap on the in-flight job wait — a genuinely hung job cannot strand a redeploy forever; past the cap the drain gives up, that job dies and is swept by collab.Recover on next boot.

Mutating routes include POST /api/agent/jobs, proposal/incorporation routes, edit/write routes, and manual sync/write endpoints. The CI webhook itself is allowed only before shutdown begins; once draining, a concurrent CI request should receive 503 and be retried by the Action rather than extending the old process. wiki-browser.service gets TimeoutStopSec=11m — above the cap, so systemd never SIGKILLs mid-drain. wb-cd's own systemctl restart call blocks for the whole drain; its unit timeout accommodates that.

Versioning & observability

So a human can tell at a glance whether a new version is live, while rollback can compare exact identity. Three values are stamped into the binary at build time via -ldflags -X: the full commit (main.commit), the display short sha (main.version), and a build timestamp (main.buildTime). Build time, not process-start time, is the "deploy time" — it is baked into the binary, so it does not drift when the Pi reboots or the service restarts for an unrelated reason.

makefileCOMMIT     ?= $(shell git rev-parse HEAD 2>/dev/null || echo dev)
VERSION    ?= $(shell git rev-parse --short HEAD 2>/dev/null || echo dev)
BUILD_TIME ?= $(shell date -u +%Y-%m-%dT%H:%MZ)
LDFLAGS     = -s -w -X main.commit=$(COMMIT) -X main.version=$(VERSION) -X main.buildTime=$(BUILD_TIME)

The temp dir from git archive has no .git, so wb-cd passes the values explicitly: make -C wiki-browser build COMMIT=<full-sha> VERSION=<short-sha> BUILD_TIME=<ts>. The defaults above keep a bare make build on a dev machine working.

Two surfaces expose it:

• /healthz — upgraded from plain ok to JSON: {"status":"ok","commit":"a1b2c3d4e5f6789012345678901234567890abcd","version":"a1b2c3d","built":"2026-05-21T14:30Z"}. Still unauthenticated, still registered ahead of the auth middleware. wb-cd compares the full commit, never the short display version.
• UI chrome footer — a subtle, small line in shell.html (a1b2c3d · 2026-05-21 14:30), the display version linked to the GitHub commit.

systemd units

systemd — wb-cd.path[Path]
PathChanged=/srv/wiki-browser/cd-trigger
Unit=wb-cd.service

[Install]
WantedBy=multi-user.target

systemd — wb-cd.service[Service]
Type=oneshot
User=karn
ExecStart=/srv/wiki-browser/bin/wb-cd -config=/srv/wiki-browser/wiki-browser.yaml
TimeoutStartSec=20m
OnFailure=wb-cd-alert.service

wb-cd runs as the unprivileged karn account and needs exactly one root action — restarting wiki-browser. A narrowly scoped sudoers rule grants only that:

sudoers — /etc/sudoers.d/wb-cdkarn ALL=(root) NOPASSWD: /usr/bin/systemctl restart wiki-browser

Configuration

One new optional block in wiki-browser.yaml. Both binaries read the same config file: wiki-browser needs trigger_file (the handler writes it); wb-cd needs all of it, plus it reuses the existing root, listen and alert values.

yaml — wiki-browser.yamlcd:
  bin_dir: "/srv/wiki-browser/bin"          # live binaries + bin/prev/
  trigger_file: "/srv/wiki-browser/cd-trigger"
  state_file: "/srv/wiki-browser/cd-state.json"
  health_poll_timeout: "90s"             # default

Absent a cd: block, wiki-browser behaves exactly as today — the /api/webhook/ci route still mounts for doc-sync if a webhook secret is configured, but a deploy:true payload has nowhere to write and is logged and ignored. The .path unit's watched path is fixed in the unit file; provisioning must keep it in step with cd.trigger_file.

The Action also gets a committed non-secret path-policy file that mirrors the document-tracking part of production config:

yaml — wiki-browser/ci-tracked-paths.yamlextensions: [".md", ".html"]
exclude:
  - "www/**"
  - "marketing/**"

Provisioning

One-time operator runbook, split by where each step runs. The Pi steps need host access the implementation does not have. The GitHub steps are gh-driven and the operator runs them.

Failure modes

Security

• /api/webhook/ci is the single unauthenticated route: constant-time HMAC compare, reject a missing signature, cap the body size, require ref == refs/heads/master, and require a full 40-hex commit. It triggers only a fetch and (conditionally) a trigger-file write — it carries no user-controlled path or command, only a boolean, branch ref, delivery id, and commit.
• wb-cd runs as the unprivileged karn account; the sudoers grant is one exact command (systemctl restart wiki-browser), nothing more.
• The HMAC secret has two halves: a file on the Pi (mode 0600, owned by karn, not in git) and a GitHub Actions secret. Actions secrets are write-only in the GitHub UI and masked in logs.
• The Pi runs go build on repo code on every deploy. This executes nothing beyond the compiler and module downloads (go.sum-verified), and the monorepo is already trusted — the team writes master directly and the Agent runtime already executes against repo content. Building it is within the existing trust boundary. Tests, which do run arbitrary repo code, run in GitHub's isolated runners, not on the Pi.

Resolved decisions

Settled during discovery; recorded so the rationale is not relitigated:

• Build on the Pi, natively. Pure-Go (CGO_ENABLED=0), a Pi 5 builds in seconds — external cross-compile + artifact transport is not worth its machinery.
• Tests run in GitHub Actions, not on the Pi. Free off-device compute, a real CI surface, and the project's first automated test gate.
• A single GitHub Action, native webhook retired. One request per push; a native webhook cannot run the test gate CD needs.
• A oneshot activated by a .path unit, not a daemon or a timer poll. Nothing to crash-recover; self-update is free.
• Graceful drain, not kill-and-recover. A 2–3 min Agent job is finished, not thrown away. Deploy latency was explicitly accepted as a non-problem; the drain cap is 10 min.
• Auto-rollback to one previous version. CD must not be able to take the wiki down.
• git hooks rejected as the trigger. They would couple CD to git-sync's internal git invocations; the Action's webhook is decoupled and precise.

References

• Pi deployment & git-sync — design (#10) — the spec this one revises; source of the webhook handler, the alert package, and the startup sequence.
• Agent runtime — design (#3) — the job model whose in-flight goroutines the drain waits on.
• internal/server/handler_webhook.go — the #10 Task-9 handler rewritten here.
• internal/agent/service.go — gains stop-dequeue + in-flight WaitGroup for the drain.
• cmd/wiki-browser/main.go — the SIGTERM path and the 5 s srv.Shutdown budget replaced here.
• deploy/wiki-browser.service, Makefile — extended for TimeoutStopSec, version -ldflags, and the wb-cd build.