Batched incorporation — Design

Problem

Generating one proposal takes about a minute of Agent latency. The common real workflow on a Source with many open Topics is "5 of these 6 are settled, 1 still needs discussion" — and today the only path to incorporating those 5 is five sequential single-Topic runs, five proposals, five reviews, five commits. The amortised cost is high; the review surface is fragmented; the git history fragments a single human decision across five commits.

The data model in #1 / #2 / #4 assumes one Topic per Proposal per Commit. That assumption is encoded in the composite FK topics.(incorporated_proposal_id, id) → incorporation_proposals(id, topic_id), in the agent_jobs CHECK constraints, in the post-job invariants, in the stale-check, in the wb-agent subcommand surface, and in the wb-incorporate skill prompt. Generalising it touches every one of those layers, which is exactly why #8 deferred batching to its own sub-project rather than folding it into the UI integration pass.

This spec answers: what's the smallest, integrity-preserving shape that lets the user grab N Topics on a Source and ask the Agent to incorporate them in one rewrite, one proposal, one commit?

Goals & non-goals

Goals

• Let the user select 2+ open Topics on a Source and ask the Agent to incorporate them together — one Agent run, one proposal, one commit.
• Model the batch as a parent Topic with children, not as a new entity type. The parent IS a Topic (with a thread, a state machine, presence semantics, sidebar rendering) — the only new thing is the parent ↔ child relationship and a new anchor kind. Existing infrastructure carries the rest.
• Keep N=1 (single-Topic incorporation) flat: no synthetic parent, no UI churn, no backfill. Today's exact code path stays in place for N=1; the N≥2 path is layered on top.
• Preserve every integrity invariant from #1–#4, even where the composite FK has to go: replace it with a write-funnel guard + trigger that enforces the same property in the application layer.
• Generalise the post-job invariant, stale-check, re-anchor pass, and realtime event surface so that N=1 is the degenerate case of the new logic — fewer branches in the skill, not more.
• UI: render parent batches like any other Topic card, with children visually nested under the parent only when the batch is focused. Use the slots #8 already reserved (the hidden card-select checkbox; the "1 Topic" Resolve toolbar label).

Non-goals

• Batch-discard as a primitive. Discarding a batch parent abandons the batch and frees its children to be re-batched or discarded individually. There is no "discard all N Topics in one action"; if the user wants to discard a child after abandoning, they do it individually. Bulk discard is a deferred polish.
• Mutating the child set of an in-flight batch. A batch's Topic set is fixed at launch. Add/remove a child = abandon and re-form. No "edit batch composition" affordance.
• Cross-Source batches. All children must share a source_path. One commit, one Source, one batch.
• Approve / Discard from mobile. #8's mobile policy stands — view + abandon + retry, no Approve. Batch-create from mobile also out: multi-select on touch needs separate UX work.
• A "discarded batches" archive view. Same status as #8's discarded-Topics archive — out of scope, deferred to a future polish project.
• Auto-batching / suggested batches. Manual selection only. The Agent does not propose batch compositions.

Approach

A batch has a clean analogue in the existing schema: a Topic without a region in the doc, with a thread, with a state machine. Modelling a batch as a Topic means presence, sidebar rendering, the realtime event surface, the message-thread primitives, and the proposal-FK pattern all carry over from #1/#2/#4 unchanged; the design surface that's actually new is small.

Three changes carry the whole feature:

1. One new column, one new anchor kind. topics.parent_topic_id nullable self-FK; anchor.kind = "batch" for parents. A parent has no marker in the Source (like global); children keep their existing anchors. The "is this a batch" question becomes a row-local switch on anchor.kind; no joins needed at consumer sites.
2. The proposal points at the parent. For N=1, the proposal points at the lone Topic (unchanged). For N≥2, the proposal points at the parent — and the parent's agent-proposal message holds the explanation that today lives in the lone Topic's thread. The composite FK breaks (a child's incorporated_proposal_id ≠ proposal.topic_id) and is replaced with a write-funnel guard + optional trigger checking the looser invariant: a Topic's incorporated proposal must belong to that Topic or to that Topic's parent.
3. Every consumer extends, no consumer rewrites. wb-agent get-topic returns nested children when the job's Topic is a batch parent; list-open-topics takes a plural --exclude-topics; post-job invariant and stale-check verify N markers instead of 1. The wb-incorporate skill branches on anchor.kind; HTTP gains one new endpoint (POST /api/batches) and the rest of the surface absorbs the batch case naturally.

Design

User-visible state machine

The new states are per-batch; per-Topic states are unchanged from #1/#4.

Data model

A batch parent is a Topic with anchor = {"kind": "batch"}. Children of a batch are Topics whose parent_topic_id references that parent's id; they keep their own anchors (pre-marker or marker) while batched. The parent has no marker in the Source — like a global Topic, no data-orcha-anchor is ever stamped for it. Children are leaf Topics; a batch parent cannot itself be a child. Apart from parent_topic_id and the new anchor kind, the topics table carries the same columns it has carried since #1/#2.

sqlCREATE TABLE topics (
  id                       TEXT PRIMARY KEY,
  source_path              TEXT NOT NULL,
  anchor                   TEXT NOT NULL,         -- JSON; kind ∈ {pre-marker, marker, global, batch}
  parent_topic_id          TEXT NULL REFERENCES topics(id) ON DELETE RESTRICT,
  created_by               TEXT NOT NULL REFERENCES users(id),
  created_at               INTEGER NOT NULL,
  incorporated_proposal_id TEXT NULL REFERENCES incorporation_proposals(id),
  commit_sha               TEXT NULL,
  incorporated_by          TEXT NULL REFERENCES users(id),
  incorporated_at          INTEGER NULL,
  discarded_at             INTEGER NULL,
  discarded_by             TEXT NULL REFERENCES users(id),

  -- all-or-nothing on incorporated columns
  CHECK ((commit_sha IS NULL) = (incorporated_proposal_id IS NULL)),
  CHECK ((commit_sha IS NULL) = (incorporated_by IS NULL)),
  CHECK ((commit_sha IS NULL) = (incorporated_at IS NULL)),
  -- all-or-nothing on discarded columns
  CHECK ((discarded_at IS NULL) = (discarded_by IS NULL)),
  -- a Topic cannot be both incorporated and discarded
  CHECK NOT (commit_sha IS NOT NULL AND discarded_at IS NOT NULL)
);

CREATE INDEX topics_parent_topic_id ON topics(parent_topic_id) WHERE parent_topic_id IS NOT NULL;

Integrity invariants enforced by triggers

Three integrity properties about the relationships between rows can't be expressed as table-level constraints in SQLite — CHECK can't subquery other rows, and the load-bearing relationship for incorporation crosses a join that a foreign key can't span. Each is enforced by a trigger on topics:

• A Topic's incorporated proposal must belong to that Topic or its parent. For a flat (non-batched) Topic, the proposal's topic_id equals the Topic's id — identical to the property a composite foreign key would express. For a batched child, the proposal's topic_id equals the parent's id (because one proposal incorporates the whole batch, and the proposal is anchored on the parent), so the invariant has to admit either branch.
• parent_topic_id must reference a Topic with anchor kind batch. Prevents accidentally attaching a child to a non-batch Topic.
• A batch parent itself has no parent. No nested batches; a batch is a flat collection of leaf Topics.

sqlCREATE TRIGGER topics_incorporated_proposal_match
AFTER UPDATE OF incorporated_proposal_id ON topics
WHEN NEW.incorporated_proposal_id IS NOT NULL
BEGIN
  SELECT CASE WHEN NOT EXISTS (
    SELECT 1 FROM incorporation_proposals p
    WHERE p.id = NEW.incorporated_proposal_id
      AND (p.topic_id = NEW.id OR p.topic_id = NEW.parent_topic_id)
  ) THEN RAISE(ABORT, 'incorporated_proposal_id does not belong to topic or its parent') END;
END;

CREATE TRIGGER topics_parent_must_be_batch
AFTER UPDATE OF parent_topic_id ON topics
WHEN NEW.parent_topic_id IS NOT NULL
BEGIN
  SELECT CASE WHEN NOT EXISTS (
    SELECT 1 FROM topics p
    WHERE p.id = NEW.parent_topic_id
      AND json_extract(p.anchor, '$.kind') = 'batch'
  ) THEN RAISE(ABORT, 'parent_topic_id must reference a topic with anchor kind = batch') END;
END;

CREATE TRIGGER topics_batch_no_grandparent
AFTER UPDATE OF anchor ON topics
WHEN json_extract(NEW.anchor, '$.kind') = 'batch' AND NEW.parent_topic_id IS NOT NULL
BEGIN
  SELECT RAISE(ABORT, 'a batch parent cannot itself be a child');
END;

Triggers fire on UPDATE rather than INSERT because every relevant column transitions only via update: a Topic is inserted with incorporated_proposal_id NULL, parent_topic_id NULL, and an initial anchor.kind ∈ {pre-marker, marker, global, batch} that the application sets at creation. Later transitions — attaching to a parent, recording a proposal on incorporation — are all updates, and that's where the trigger needs to fire.

The application-side write funnel (a single goroutine serialising every write to the collab DB, per #1) also enforces the first invariant before the trigger sees it. The trigger is belt-and-braces: it catches any future code path that bypasses the funnel.

Anchor kind

The anchor column stores JSON; its kind field is validated by the Go internal/collab layer rather than by a SQL constraint. The accepted values are pre-marker, marker, global, and batch. A {"kind": "batch"} anchor carries no payload, matching the shape of {"kind": "global"}; both indicate "no region in the Source, no data-orcha-anchor ever stamped." Extending the enum is a code change only — no schema migration is needed to expand the value set.

Idempotency for in-flight batches

The collab DB already carries a partial unique index agent_jobs_one_inflight_incorporate_topic on agent_jobs(topic_id) WHERE status IN ('queued','running') — at most one in-flight incorporate job per Topic. For a batch, the in-flight job's topic_id is the parent's id, so the same index serves without modification: a parent can have at most one in-flight job, and any rework reuses the existing job_id (returned with 200) instead of enqueueing a duplicate.

Membership uniqueness — "a Topic can't be in two active batches" — is enforced at POST /api/batches time. The handler runs the row insert and the parent_topic_id updates in a single transaction, validating that every requested child has parent_topic_id IS NULL as part of the same write. A concurrent request that tries to claim the same child loses the SQLite write lock; whichever request acquires it first wins, and the other returns 409 child_already_batched.

Migration

The change to the topics table is additive (one nullable column and a new partial index) — no twelve-step rebuild is needed for the schema itself. The supporting triggers are created in the same migration. The composite foreign key from #1's topics schema — topics.(incorporated_proposal_id, id) → incorporation_proposals(id, topic_id) — is replaced by topics_incorporated_proposal_match above. SQLite cannot ALTER TABLE … DROP CONSTRAINT, so dropping the composite FK does require the twelve-step rebuild: the migration runs under the migrate:no-tx directive from #3, toggling foreign_keys itself, copying rows from the old table to a freshly-built one without the composite FK, dropping the old table, renaming the new one in, recreating indexes and triggers, and finishing with PRAGMA foreign_key_check.

No data backfill is required. Existing rows have parent_topic_id IS NULL (the column is added as nullable with no default to populate), and their incorporated_proposal_id values continue to satisfy the new trigger via the first branch of the OR (T.id = P.topic_id) — exactly the property the dropped composite FK previously enforced.

HTTP surface

One new endpoint, the rest of #4's surface absorbs the batch case naturally. All endpoints stay behind auth.RequireCollaborator from #7.

wb-agent surface — extend, don't replace

No new subcommands. Three existing subcommands gain batch awareness:

The --exclude-topic → --exclude-topics flip is the only breaking change in the wire. Migrating the skill body to the plural form is a one-line edit; the singular form is removed.

wb-incorporate rewrite contract — generalisation

The skill body branches once on the response from get-topic, then applies the same rewrite contract from #4 to the resulting Topic set:

markdown# Conceptual flow (the SKILL.md prose stays in user-task voice per #4)

1. Call get-topic --job-id=<id> → response R.
2. Let batchTopicIds = [R.topic.id] ∪ (R.children?.map(c => c.topic.id) ?? []).
3. Call list-open-topics --source-path=R.source_path --exclude-topics=batchTopicIds.join(',').
4. Read the current Source at R.source_path.
5. Apply the discussion from R.topic + R.children[*] to produce the proposed Source. The
   discussion for each child lives in c.messages; the parent's thread (R.messages) carries
   batch-level commentary and the prior proposal's explanation, if any.
6. For each Topic id in batchTopicIds: verify no data-orcha-anchor="<id>" remains in
   the proposed Source. (For N=1 this is the lone Topic; for N≥2 every child marker drops.
   The parent never had a marker.)
7. For each Topic returned by list-open-topics: stamp at least one
   data-orcha-anchor="<id>" in the proposed Source — naturally where it maps, or under the
   ## Other ideas (potentially to discard) parking-lot section.
8. Write a single explanation (1–3 paragraphs) covering every Topic being incorporated.
   For batches the first sentence should function as a TL;DR — the commit subject default
   pulls from this prefix.
9. Call insert-proposal --job-id=<id> --explanation=<text> with the proposed Source on stdin.

Post-job invariant — N markers

The runner's post-exit check generalises from #4's "the incorporated Topic's marker is absent" to a set check:

1. The new incorporation_proposals row is linked to the job via agent_job_id.
2. For every Topic id in batchTopicIds (the parent + every child for batches; the lone Topic for N=1), the substring data-orcha-anchor="<id>" is absent from the proposed Source. Exception: for a batch parent with anchor.kind = "batch", no marker ever existed anyway, so the check is satisfied vacuously — but listing the parent id in the set is harmless.
3. Every Topic returned by list-open-topics at job start has at least one occurrence of data-orcha-anchor="<id>" in the proposed Source.
4. The accompanying agent-proposal message body in the parent's thread (or the lone Topic's thread for N=1) is non-empty.

Failure flips the job to failed with a descriptive error_tail; the proposal row stays in history but is filtered out of approvable proposals (same behaviour as today).

Stale-check — N markers

Identical generalisation. A proposal is stale when any of:

• base_source_sha no longer matches the current Source (stale_reasons includes "source_sha").
• The proposed Source is missing a data-orcha-anchor="<id>" marker for any current open Topic on this Source where the Topic is not in the batch's set and not itself a batch parent (stale_reasons includes "missing_topic_markers", with missing_topic_ids).

"Current open Topic on this Source" excludes incorporated, discarded, batched-as-a-child, and batch-parent Topics — same predicate the Resolve UI uses to render the inline anchors. A new Topic created on this Source mid-job marks the proposal stale (missing that new Topic's marker). A new batch formed on this Source mid-job does not mark the proposal stale on its own, because batch parents have no marker and batched children are excluded from the open-Topic set.

Anchor JSON updates on Incorporation

#4's invariant — "every other open non-global Topic on the Source ends incorporation with anchor kind marker" — generalises to "every Topic returned by list-open-topics at job start." The approval handler computes that set (excluding parent + every child of the batch being incorporated, plus excluding any newer batch parents and their children that appeared mid-flight — they're handled by the stale-check, not the re-anchor) and passes it through CompleteIncorporationInput.ReanchorTopicIDs. The existing UPDATE topics SET anchor = '{"kind":"marker"}' WHERE id IN (…) AND discarded_at IS NULL AND commit_sha IS NULL AND json_extract(anchor, '$.kind') = 'pre-marker' runs unchanged.

Commit subject default for batches

#4's default "Incorporate Topic: <summary>" uses the Topic's first human message, rune-truncated to 60. For batches the default is "Incorporate N Topics: <summary>" where <summary> is the first 60 runes of the parent Topic's agent-proposal message body (the Agent's overarching explanation), Markdown-prefix-stripped, whitespace-collapsed, ellipsised if truncated. The Approve form's subject field still accepts an override — same UX as today.

Why the explanation, not a child summary: the explanation is by construction the batch's TL;DR. Pulling from a single child's first message is arbitrary; concatenating N summaries blows past 60 runes immediately and reads worse than the Agent's framing.

Sidebar — collapsed by default, expanded while batch is focused

The default sidebar renders the parent card alone with a "Batch · N" pill. Children stay hidden until the focused Topic is the parent or any of its children. Once focused, children render directly under the parent, indented ~16px with a 2px accent rail on the left edge connecting them up to the parent card. The focus path covers all entry points: clicking the parent card, clicking a child's anchor in the iframe (children's markers still live in the current Source during the batch's life), clicking a child card after the group is expanded, or deep-linking to a child's URL. Selecting any unrelated Topic, clicking empty iframe space, or pressing Escape collapses the batch back to parent-only.

Batch formation flow — uses #8's reserved slots

#8 reserved a hidden wb-topic-card__select checkbox slot in the top-left of every Topic card. The batch flow unhides it:

1. Clicking any card's checkbox enters multi-select mode. A header bar appears at the top of the right sidebar: "N selected · Batch & Rewrite". The button is disabled when fewer than 2 cards are selected (N=1 falls back to the existing per-Topic Propose Rewrite affordance).
2. Clicking Batch & Rewrite opens a small confirmation: "Rewrite N Topics together. The Agent will produce one proposal covering all selected discussions." Confirming POSTs /api/batches.
3. On success: the parent's Topic card animates into the sidebar's Batches group; children move into the indented position under it (with the rail). The parent's card shows the existing "generating" spinner from #4 while the Agent job runs.
4. Already-batched cards render with their checkboxes disabled + a tooltip ("part of an open batch"). Cross-Source selections are also rejected with a tooltip ("batches can only contain Topics on the same Source").

Resolve view for a batch

Same iframe-slot swap from #8. Differences:

• Toolbar label. #8's "1 Topic" label becomes "N Topics", and clicking it expands an inline panel listing every child with quick-jump links. The label is the affordance the user already sees in the Resolve toolbar; #8 reserved it for exactly this expansion.
• Right sidebar = parent's thread. The thread shown alongside the diff is the parent's, because that's where the Agent's explanation lives. Clicking a child in the expander panel swaps the right sidebar to that child's thread; clicking the parent's card swaps back.
• Diff content. Tier 1 (unified) shows N marker-removal hunks plus the rewrite hunks. Tier 2 (rendered side-by-side) shows the doc with children's anchor highlights vanishing on the right pane. No new diff machinery.
• Approve form. The subject prefill is "Incorporate N Topics: <summary>" as described above. The body field is unchanged.
• Stale banner. The existing banner copy distinguishes source_sha drift from missing_topic_markers; same UI for batches.

Realtime — existing event surface generalises

No new event types. The mapping:

Concurrency & recovery

The batch path inherits #1's two-store atomicity (incorporation_attempts marker → write Source → git commit → SQLite update → complete attempt). The only generalisation: CompleteIncorporationInput now updates parent + every child in the same SQLite transaction, not just the single Topic. collab.Recover at startup is unchanged — the recovery marker still references the parent's id; replaying it transitions the whole batch.

If the Agent job for a batch fails or times out, the partial state on disk is whatever the runner has left (typically nothing — insert-proposal is atomic, no proposal row means no proposed Source landed). Children stay attached to the parent, the parent stays open, the user retries from the UI. No new recovery path.

Mobile

Same constraints from #8: batches are viewable + abandonable on mobile, not approvable. Batch formation is also out on mobile in v1 — multi-select on touch needs separate UX (long-press to enter selection mode? toolbar checkboxes vs. card checkboxes?) and is parking-lot for a future polish project. The mobile shell renders parent cards with their Batch pill; children expand on focus same as desktop.

Open questions

1. Multi-select on touch / mobile batch formation. Parked. Desktop-first for v1; mobile gets view + abandon only. When the polish pass lands, the candidate patterns are long-press to enter selection mode or a per-card toggle in the card's overflow menu.
2. Batch-discard as a primitive. Parked. The current model abandons the batch and leaves the user to discard children individually if they want. A future "discard whole batch" affordance could write the discard reason once and cascade — but it conflates two intents (forget composition vs. throw away discussions) and needs its own UX brainstorm.
3. "Discarded batches" archive view. Parked alongside #8's discarded-Topics archive. Discarded batch parents persist in the DB with their thread intact (including the audit message listing children at abandon time), but the live sidebar filters them out.
4. Auto-batching suggestions. Out of scope. The Agent does not propose batch compositions. A future "5 settled threads detected — batch them?" affordance is conceivable but would need its own settle-detection signal.
5. Nested batches. Explicitly disallowed by the topics_batch_no_grandparent trigger. If we ever want batch-of-batches, the trigger relaxes; for now it's a safety rail against weird states.

References

• Collaborative annotations — domain model (sub-project index)
• Decisions & parking lot — cross-cutting log of decisions across all sub-projects; not load-bearing for this spec but useful for orientation.
• #1 — Document model & persistence (composite FK that this spec replaces with a write-funnel guard + trigger)
• #2 — Topic core: data model + anchoring (anchor kinds; the "batch" kind is the new enum value)
• #3 — Agent runtime & harness invocation (agent_jobs idempotency index; wb-agent scaffold)
• #4 — Topic resolution & incorporation (HTTP surface; wb-agent contract; post-job invariant; stale-check; re-anchor pass)
• #6 — Real-time collaboration mechanics (event surface; ordering pin; REST-is-authoritative refetch)
• #7 — Identity & permissions (auth.RequireCollaborator)
• #8 — Wiki-browser UI integration (forward-compat slots: wb-topic-card__select, Resolve toolbar "1 Topic" label)

Commit message trailers

One Approve action produces one collab.Incorporate call, one Source rewrite, and one commit — #1's "1 Incorporation = 1 commit" holds verbatim, including for batches (the parent's incorporation is the one event; the parent + children transitions are one transaction's worth of side effects). The commit message body emits a Topic-Id: trailer for every Topic transitioned (the parent and each child), so git log --grep <topic-id> finds the commit from any of them.