Note (2026-04-24): After this document was written,
legal_entitywas renamed totenantand the oldtenantwas renamed toorganization. Read references to these terms with the pre-rename meaning.
For agentic workers: REQUIRED SUB-SKILL: Use superpowers:subagent-driven-development (recommended) or superpowers:executing-plans to implement this plan task-by-task. Steps use checkbox (
- [ ]) syntax for tracking.
Goal: Replace per-processor SSE events for diagnostic sections with one atomic diagnostics-recomputed event per recompute cycle, eliminating the HTMX swap race and naturally fixing the Tax Compliance content-loss bug.
Architecture: Application-layer producers (edit-recompute orchestrator, matching worker, reap sweep) fire diagnostics-recomputed via pg_notify. The SSE handler queries authoritative state and emits one message whose body is every diagnostic section rendered as an hx-swap-oob fragment. The initial "stale" transition piggybacks on the edit HTTP response via the same OOB-fragment helper.
Tech Stack: Clojure, next.jdbc, honeysql, Postgres LISTEN/NOTIFY, malli schema, HTMX SSE extension, Hiccup, Integrant, migratus.
Reference spec: docs/superpowers/specs/2026-04-17-atomic-diagnostics-updates-design.md.
Ordering: Phase 1 adds the new machinery; Phase 2 wires up producers; Phase 3 removes old machinery; Phase 4 verifies. At every commit the system remains functional — per-processor events keep firing until the trigger is dropped in Task 12.
:diagnostics-recomputed variant to DocumentEvent schemaFiles:
Modify: src/com/getorcha/app/ingestion.clj (the DocumentEvent multi-schema near line 337)
Test: test/com/getorcha/app/ingestion_test.clj (extend)
Step 1: Write the failing test
Add this case inside document-event-coerces-every-processor-id-test or as a new deftest. Prefer a new deftest so it's explicit about the new event type.
Append to test/com/getorcha/app/ingestion_test.clj:
(deftest document-event-coerces-diagnostics-recomputed-test
(testing "DocumentEvent accepts diagnostics-recomputed events"
(let [payload {:event/type :diagnostics-recomputed
:document/id "019d745a-2881-7028-9646-62533e48de02"
:legal-entity/id "019d745a-2882-7028-9646-62533e48de02"
:tenant/id "019d745a-2883-7028-9646-62533e48de02"}
coerced (m/coerce app.ingestion/DocumentEvent
payload
m.transform/json-transformer)]
(is (m/validate app.ingestion/DocumentEvent coerced))
(is (= :diagnostics-recomputed (:event/type coerced))))))
Run: clj -X:test:silent :nses '[com.getorcha.app.ingestion-test]'
Expected: one FAIL on document-event-coerces-diagnostics-recomputed-test because :diagnostics-recomputed isn't in the multi-dispatch.
In src/com/getorcha/app/ingestion.clj, add a new schema def above DocumentEvent (the (def DocumentEvent …) near line 337):
(def DiagnosticsRecomputedEvent
"Schema for the single-atomic diagnostics-recomputed event.
Fired by the recompute orchestrator (after all phases + persist-derivation),
the matching worker (after matching completes), and the reap sweeper (per
affected document). Carries no slice data — the SSE handler queries the DB
and renders every diagnostic section at current state."
[:map
[:event/type [:= :diagnostics-recomputed]]
[:document/id schema.document/ID]
[:legal-entity/id schema.legal-entity/ID]
[:tenant/id schema.tenant/ID]])
Then extend the :multi dispatch inside DocumentEvent:
(def DocumentEvent
"Union schema for document events (ingestion, export, matching, diagnostic runs, or diagnostics recompute)."
(m/schema
[:multi {:dispatch :event/type}
[:ingestion IngestionEvent]
[:export ExportEvent]
[:matching MatchingEvent]
[:diagnostic-run-started DiagnosticRunStartedEvent]
[:diagnostic-run-completed DiagnosticRunCompletedEvent]
[:diagnostics-recomputed DiagnosticsRecomputedEvent]]))
Run: clj -X:test:silent :nses '[com.getorcha.app.ingestion-test]'
Expected: all tests pass; total assertion count increases by 2.
Run: clj-kondo --lint src test dev
Expected: 0 errors, 0 warnings.
git add src/com/getorcha/app/ingestion.clj test/com/getorcha/app/ingestion_test.clj
git commit -m "feat(ingestion): add :diagnostics-recomputed DocumentEvent variant"
fire-diagnostics-recomputed! helperFiles:
src/com/getorcha/app/ingestion.clj (add public fn near the bottom of the namespace, after the document-events-publisher component defs)test/com/getorcha/app/ingestion_test.clj (extend)Why it lives in app/ingestion.clj: the document-events publisher/listener and the DocumentEvent schema already live there. Event-firing helpers sit naturally alongside.
Append to test/com/getorcha/app/ingestion_test.clj. You will need some fixtures and helpers. Add these requires at the top:
[com.getorcha.db.sql :as db.sql]
[com.getorcha.test.fixtures :as fixtures]
[com.getorcha.test.notification-helpers :as helpers]
[next.jdbc :as jdbc]
Add fixtures:
(use-fixtures :once fixtures/with-running-system)
(use-fixtures :each fixtures/with-db-rollback)
Add the test:
(deftest fire-diagnostics-recomputed-test
(testing "fires pg_notify with correct payload; returns nil"
(let [le-id (helpers/create-legal-entity!)
doc-id (helpers/create-document! le-id)
tenant-id (:legal-entity/tenant-id
(db.sql/execute-one! fixtures/*db*
{:select [:tenant-id]
:from [:legal-entity]
:where [:= :id le-id]}))
calls (atom [])]
(with-redefs [jdbc/execute! (fn [& args] (swap! calls conj args))]
(is (nil? (app.ingestion/fire-diagnostics-recomputed! fixtures/*db* doc-id))))
(is (= 1 (count @calls)))
(let [[_ [sql payload]] (first @calls)
parsed (json/parse-string payload true)]
(is (re-find #"pg_notify\('document_events'" sql))
(is (= "diagnostics-recomputed" (:event/type parsed)))
(is (= (str doc-id) (:document/id parsed)))
(is (= (str le-id) (:legal-entity/id parsed)))
(is (= (str tenant-id) (:tenant/id parsed))))))
(testing "does nothing when document does not exist"
(let [missing-id (java.util.UUID/randomUUID)
calls (atom [])]
(with-redefs [jdbc/execute! (fn [& args] (swap! calls conj args))]
(is (nil? (app.ingestion/fire-diagnostics-recomputed! fixtures/*db* missing-id))))
(is (zero? (count @calls))))))
Run: clj -X:test:silent :nses '[com.getorcha.app.ingestion-test]'
Expected: FAIL — fire-diagnostics-recomputed! is not defined yet.
Add this public defn near the bottom of src/com/getorcha/app/ingestion.clj (after the ig/halt-key! for document-events-publisher):
(defn fire-diagnostics-recomputed!
"Emits a `diagnostics-recomputed` pg_notify on the `document_events`
channel for `document-id`. Looks up the legal-entity and tenant ids
via the document/legal-entity join. No-op if the document does not
exist.
The payload is a minimal routing envelope — the SSE handler queries
the DB for authoritative state when it receives the event."
[db-pool document-id]
(when-let [row (db.sql/execute-one!
db-pool
{:select [:document.legal-entity-id :legal-entity.tenant-id]
:from [:document]
:join [[:legal-entity] [:= :legal-entity.id :document.legal-entity-id]]
:where [:= :document.id document-id]})]
(jdbc/execute!
db-pool
["SELECT pg_notify('document_events', ?::text)"
(json/generate-string
{"event/type" "diagnostics-recomputed"
"document/id" (str document-id)
"legal-entity/id" (str (:document/legal-entity-id row))
"tenant/id" (str (:legal-entity/tenant-id row))})]))
nil)
Run: clj -X:test:silent :nses '[com.getorcha.app.ingestion-test]'
Expected: all tests pass.
Run: clj-kondo --lint src test dev
Expected: 0 warnings.
git add src/com/getorcha/app/ingestion.clj test/com/getorcha/app/ingestion_test.clj
git commit -m "feat(ingestion): add fire-diagnostics-recomputed! pg_notify helper"
render-all-diagnostic-sections helperFiles:
src/com/getorcha/app/http/documents/view/shared.clj (add public fn near where render-diagnostic-section lives, around line 1060)test/com/getorcha/app/http/documents/view/shared_test.clj (new)This helper is the single source of truth for "render every diagnostic section at current DB state, with OOB attributes." Used by both the SSE handler (Task 4) and the edit response (Task 5).
Create test/com/getorcha/app/http/documents/view/shared_test.clj:
(ns com.getorcha.app.http.documents.view.shared-test
(:require [cheshire.core :as json]
[clojure.test :refer [deftest is testing use-fixtures]]
[com.getorcha.app.http.documents.view.shared :as view.shared]
[com.getorcha.db.document-processor-run :as db.run]
[com.getorcha.db.sql :as db.sql]
[com.getorcha.test.fixtures :as fixtures]
[com.getorcha.test.notification-helpers :as helpers]
[hiccup2.core :as hiccup]
[reitit.core :as reitit]))
(use-fixtures :once fixtures/with-running-system)
(use-fixtures :each fixtures/with-db-rollback)
(def ^:private test-router
"Minimal reitit router for URL generation in tests."
(reitit/router
[["/documents/view/:document-id" {:name :com.getorcha.app.http.documents.view/detail}]
["/documents/view/:document-id/matches" {:name :com.getorcha.app.http.documents.view/matches}]]))
(defn ^:private seed-invoice!
[le-id structured-data]
(let [doc-id (helpers/create-document! le-id)]
(db.sql/execute-one! fixtures/*db*
{:update :document
:set {:structured-data [:lift structured-data]
:type (db.sql/->cast "invoice" :document-type)}
:where [:= :id doc-id]})
doc-id))
(deftest render-all-diagnostic-sections-test
(testing "emits OOB fragments for an invoice's diagnostic sections"
(let [le-id (helpers/create-legal-entity!)
doc-id (seed-invoice! le-id {:document-type "invoice"
:issuer {:name "Acme" :country "DE"}
:recipient {:country "DE"}
:total 100.0})
html (str (hiccup/html (view.shared/render-all-diagnostic-sections
fixtures/*db* test-router doc-id #{le-id})))]
(is (re-find #"id=\"diagnostic-section-validations\"" html))
(is (re-find #"id=\"diagnostic-section-fraud-detector\"" html))
(is (re-find #"id=\"diagnostic-section-tax-compliance-analyzer\"" html))
(is (re-find #"id=\"section-matches\"" html))
(is (re-find #"hx-swap-oob=\"outerHTML\"" html))))
(testing "returns empty list when document does not belong to caller's legal entities"
(let [le-id (helpers/create-legal-entity!)
other-le (helpers/create-legal-entity!)
doc-id (seed-invoice! le-id {:document-type "invoice"})
html (str (hiccup/html
(view.shared/render-all-diagnostic-sections
fixtures/*db* test-router doc-id #{other-le})))]
(is (not (re-find #"hx-swap-oob" html)))))
(testing "skips tax-compliance + fraud for non-invoice doc types"
(let [le-id (helpers/create-legal-entity!)
doc-id (helpers/create-document! le-id)
_ (db.sql/execute-one! fixtures/*db*
{:update :document
:set {:structured-data [:lift {:document-type "purchase-order"}]
:type (db.sql/->cast "purchase-order" :document-type)}
:where [:= :id doc-id]})
html (str (hiccup/html
(view.shared/render-all-diagnostic-sections
fixtures/*db* test-router doc-id #{le-id})))]
(is (re-find #"id=\"diagnostic-section-validations\"" html))
(is (not (re-find #"id=\"diagnostic-section-tax-compliance-analyzer\"" html)))
(is (not (re-find #"id=\"diagnostic-section-fraud-detector\"" html))))))
Run: clj -X:test:silent :nses '[com.getorcha.app.http.documents.view.shared-test]'
Expected: FAIL — render-all-diagnostic-sections is not defined.
Add to src/com/getorcha/app/http/documents/view/shared.clj, just above the detail-events fn (around line 1072). The body below is deliberately explicit — the existing render-diagnostic-section dispatcher (line 1060) gets replaced by this helper in a later task.
(defn ^:private oob-wrap
"Adds `hx-swap-oob=\"outerHTML\"` to a section fragment so HTMX
swaps it into its target by id. Returns nil if `fragment` is nil."
[fragment]
(when fragment
(update fragment 1 (fnil assoc {}) :hx-swap-oob "outerHTML")))
(defn render-all-diagnostic-sections
"Renders every diagnostic section for `document-id` at current DB state,
each wrapped with `hx-swap-oob` so HTMX swaps them into their stable ids.
Returns a hiccup `list` ready to embed in a response body — whether an
SSE event payload or the edit HTTP response. Returns an empty list when
the document does not belong to one of `legal-entity-id-set` (defensive
authorisation — callers should already have validated this).
One DB read per source (document, latest processor runs, matches/
reconciliation/cluster-peers, supplier-verification). Section rendering
switches on `:document/type` — contracts get `contract-validation-section`,
invoices get the full panel, POs/GRNs/notices get only the shared
sections."
[db-pool router document-id legal-entity-id-set]
(let [document (db.sql/execute-one!
db-pool
{:select [:*]
:from [:document]
:where [:and
[:= :document.id document-id]
[:in :document.legal-entity-id (vec legal-entity-id-set)]]}
{:builder-fn shared/document-builder-fn})]
(if (nil? document)
(list)
(let [doc-type (some-> (:document/type document) keyword)
structured-data (:document/structured-data document)
diagnostics (:document/diagnostics document)
latest-runs (db.run/latest-runs-per-processor db-pool document-id)
latest-runs-by-id (group-by :document-processor-run/processor-id latest-runs)
matches (db.matching/get-matched-documents db-pool document-id)
reconciliation (db.matching/get-cluster-reconciliation db-pool document-id)
cluster-peers (when reconciliation
(db.matching/get-cluster-peers db-pool document-id))
state-for (fn [processor-id slice-key]
(app.ui.components/diagnostic-section-state
document processor-id slice-key
(first (get latest-runs-by-id processor-id))))
validations-state (state-for "validations" :validations)
tax-state (state-for "tax-compliance-analyzer" :tax-issues)
fraud-state (state-for "fraud-detector" :fraud-flags)
matching-run (first (get latest-runs-by-id "matching"))
matching-status (case (:document-processor-run/status matching-run)
"running" :in-progress
"completed" :succeeded
"failed" :failed
:pending)]
(cond-> (list)
;; Validations (or contract-validation) — all doc types
(contains? #{:invoice :financial-notice :contract :purchase-order :goods-received-note}
doc-type)
(conj (oob-wrap
(if (= :contract doc-type)
(app.ui.components/contract-validation-section
diagnostics structured-data validations-state)
(app.ui.components/validation-results-section
(:validations diagnostics) validations-state))))
;; Fraud — invoices only
(= :invoice doc-type)
(conj (oob-wrap
(app.ui.components/fraud-detection-section
(:fraud-flags diagnostics) fraud-state)))
;; Tax compliance — invoices only; passes structured-data fields through
(= :invoice doc-type)
(conj (oob-wrap
(app.ui.components/tax-compliance-section
{:tax-issues (:tax-issues diagnostics)
:service-category (:service-category structured-data)
:issuer (:issuer structured-data)
:recipient (:recipient structured-data)
:shipping-country (:shipping-country structured-data)}
tax-state)))
;; Matches — doc types with counterparts
(contains? counterpart-types doc-type)
(conj (oob-wrap
(matches-section router document-id doc-type matches
matching-status reconciliation cluster-peers
(get-in diagnostics [:reconciliation :status])))))))))
Note: matches-section is already defined in this namespace (around line 338). counterpart-types is already defined around line 42. shared/document-builder-fn, db.matching/*, app.ui.components/* are already required.
Run: clj -X:test:silent :nses '[com.getorcha.app.http.documents.view.shared-test]'
Expected: all assertions pass.
Run: clj-kondo --lint src test dev
Expected: 0 warnings.
git add src/com/getorcha/app/http/documents/view/shared.clj test/com/getorcha/app/http/documents/view/shared_test.clj
git commit -m "feat(view): add render-all-diagnostic-sections OOB helper"
:diagnostics-recomputed into detail-events SSE handlerFiles:
Modify: src/com/getorcha/app/http/documents/view/shared.clj (add new case in detail-events; add sse-swap listener in detail-area-content)
Step 1: Add the SSE listener on #document-area
In src/com/getorcha/app/http/documents/view/shared.clj:detail-area-content (around line 734), add a small listener div that causes HTMX to process the SSE response. Currently:
[:div#document-area {:hx-ext "sse"
:sse-connect sse-url}
;; SSE swap for full area replacement while processing
(when processing?
[:div {:sse-swap "status-changed"
:hx-target "#document-area"
:hx-swap "outerHTML"}])
(detail-page-content router document opts)])
Change to:
[:div#document-area {:hx-ext "sse"
:sse-connect sse-url}
;; Atomic diagnostics-recomputed listener — processes OOB fragments in the response.
[:div {:sse-swap "diagnostics-recomputed"
:hx-swap "none"}]
;; SSE swap for full area replacement while processing
(when processing?
[:div {:sse-swap "status-changed"
:hx-target "#document-area"
:hx-swap "outerHTML"}])
(detail-page-content router document opts)]
The hx-swap="none" tells HTMX to do nothing with the direct response body; the OOB elements inside get swapped by id via HTMX's OOB machinery.
In the same file, in the detail-events handler's case event-type (around line 1107), add a new branch after :diagnostic-run-completed:
:diagnostics-recomputed
;; Atomic event — render every diagnostic section via OOB.
{:event "diagnostics-recomputed"
:data (hiccup/html
(render-all-diagnostic-sections db-pool router document-id le-id-set))}
The local bindings db-pool, router, document-id, and le-id-set are already in scope in the detail-events handler.
In test/com/getorcha/app/http/documents/view/shared_test.clj, add a simple integration-style test that constructs a fake event, invokes the exec-fn, and verifies the emitted SSE payload contains OOB fragments. Because the exec-fn is a closure, we test by constructing it indirectly: seed a document, simulate the event through the event listener path. The cleanest approach given test constraints is to test render-all-diagnostic-sections (covered in Task 3) + the single case branch logic is trivially wired — the Task 3 test is sufficient for this path.
No new test in this task beyond what Task 3 covers. Sanity check: run the existing test suite.
Run: clj -X:test:silent :nses '[com.getorcha.app.http.documents.view.shared-test com.getorcha.app.ingestion-test]'
Expected: all tests pass.
Run: clj-kondo --lint src test dev
Expected: 0 warnings.
git add src/com/getorcha/app/http/documents/view/shared.clj
git commit -m "feat(view): wire diagnostics-recomputed SSE case with OOB listener"
Files:
Modify: src/com/getorcha/app/http/documents/edits.clj (apply-edit-tx around line 206)
Step 1: Add the render-all call to the success branch
apply-edit-tx currently builds the response via render-fragment with (list (render-success new-sd new-version) (oob-version-swap new-version)). Extend the list with the render-all output. This requires passing router into apply-edit-tx.
First, find every caller of apply-edit-tx in src/com/getorcha/app/http/documents/edits.clj (it's called from scalar-edit-handler, and any other -handler defns in the same file). Update each caller to pass (::reitit/router request) as an additional argument.
Add router to the apply-edit-tx arg vector (after db-pool aws identity-id):
(defn ^:private apply-edit-tx
"...[unchanged docstring]..."
[db-pool aws router identity-id document-id expected-version legal-entity-ids
{:keys [prepare-patch render-success render-conflict
success-status extra-triggers extra-headers]
:or {success-status 200 extra-triggers {} extra-headers {}}}]
...)
Add [com.getorcha.app.http.documents.view.shared :as view.shared] to the namespace :require if not already present.
Find the success render-fragment call in apply-edit-tx (around line 261-266):
(render-fragment
success-status
(list (render-success new-sd new-version)
(oob-version-swap new-version))
(merge {"documentEdited" true} extra-triggers)
extra-headers)
Change the fragment list to include render-all-diagnostic-sections:
(render-fragment
success-status
(list (render-success new-sd new-version)
(oob-version-swap new-version)
(view.shared/render-all-diagnostic-sections
db-pool router document-id (set legal-entity-ids)))
(merge {"documentEdited" true} extra-triggers)
extra-headers)
This runs after the UPDATE to document, inside the transaction. The render sees document.version bumped, while the latest runs are still at the old version — so the state classifier returns :stale for each section. The browser OOB-swaps each section into its stale state.
apply-edit-txFind every call site:
grep -n "apply-edit-tx" src/com/getorcha/app/http/documents/edits.clj
For each handler defn that calls apply-edit-tx, destructure ::reitit/router from the request and pass it. Concretely, in scalar-edit-handler (and any sibling line-item-*-handler, etc.), update the arg destructuring and the call:
(defn ^:private scalar-edit-handler
[{:keys [aws db-pool parameters identity ::reitit/router] :as request} respond _raise]
(let [{:keys [document-id]} (:path parameters)
{:keys [path value expected-version field-type]} (:form parameters)
identity-id (:identity/id identity)
le-ids (documents.shared/legal-entity-ids request)
clj-path (json-patch.path/pointer->clj-path path)]
(respond
(apply-edit-tx
db-pool aws router identity-id document-id expected-version le-ids
{:prepare-patch ...
:render-success ...
:render-conflict ...
...}))))
Every caller does the same transformation: add ::reitit/router to the request destructuring, and insert router as the third argument to apply-edit-tx.
Add to test/com/getorcha/app/http/documents/edits_test.clj (file already exists):
(deftest edit-response-includes-oob-diagnostic-sections-test
(testing "successful edit response appends OOB fragments for diagnostic sections"
(let [le-id (helpers/create-legal-entity!)
doc-id (helpers/create-document! le-id)
_ (db.sql/execute-one! fixtures/*db*
{:update :document
:set {:structured-data [:lift {:document-type "invoice"
:issuer {:name "Acme"}}]
:type (db.sql/->cast "invoice" :document-type)}
:where [:= :id doc-id]})
identity-id (helpers/create-identity!)
request {:db-pool fixtures/*db*
:aws (::aws/state fixtures/*system*)
:parameters {:path {:document-id doc-id}
:form {:path "/issuer/name"
:value "Acme-edited"
:expected-version 1
:field-type "text"}}
:identity {:identity/id identity-id}
::reitit/router (::reitit/router fixtures/*system*)}
resp-atom (atom nil)]
(#'edits/scalar-edit-handler request #(reset! resp-atom %) (fn [_]))
(let [body (str (:body @resp-atom))]
(is (= 200 (:status @resp-atom)))
(is (re-find #"hx-swap-oob=\"outerHTML\"" body))
(is (re-find #"id=\"diagnostic-section-validations\"" body))))))
Add any missing requires to the test ns header (com.getorcha.app.http.documents.edits :as edits, reitit.core :as reitit, etc.). The existing edits_test.clj already has most of them.
If the test harness can't easily supply ::reitit/router (it's constructed via Integrant and not a simple key), fall back to a reitit router like the test-router defined in shared_test.clj. The important assertion is the OOB fragment appearing in the response body.
Run: clj -X:test:silent :nses '[com.getorcha.app.http.documents.edits-test]'
Expected: all tests pass, including the new one.
Run: clj-kondo --lint src test dev
Expected: 0 warnings.
git add src/com/getorcha/app/http/documents/edits.clj test/com/getorcha/app/http/documents/edits_test.clj
git commit -m "feat(edits): append OOB diagnostic sections to edit HTTP response"
try/finallyFiles:
Modify: src/com/getorcha/workers/diagnostics_recompute/orchestrator.clj (recompute-all! at line 44)
Test: test/com/getorcha/workers/diagnostics_recompute_test.clj (extend)
Step 1: Write the failing tests
Append to test/com/getorcha/workers/diagnostics_recompute_test.clj:
(deftest recompute-all-fires-event-on-success-test
(testing "fire-diagnostics-recomputed! called once after successful recompute"
(let [le-id (helpers/create-legal-entity!)
doc-id (helpers/create-document! le-id)
identity-id (helpers/create-identity!)
ingestion-id (helpers/create-ingestion! doc-id identity-id :status :completed)
_ (db.sql/execute-one! fixtures/*db*
{:update :document
:set {:structured-data [:lift {:document-type "invoice"
:issuer {:name "Acme"}}]
:type (db.sql/->cast "invoice" :document-type)
:version [:raw "version + 1"]}
:where [:= :id doc-id]})
_ignored (document-history/insert! fixtures/*db*
{:document-id doc-id
:change-type :ingestion
:ingestion-id ingestion-id
:patch []})
history-row (document-history/insert! fixtures/*db*
{:document-id doc-id
:change-type :edit
:edited-by identity-id
:patch [{"op" "replace" "path" "/issuer/name" "value" "Acme-edited"}]})
fires (atom 0)]
(with-redefs [app.ingestion/fire-diagnostics-recomputed!
(fn [_ _] (swap! fires inc))
matching/->matching
(fn [] (->NoopProcessor :matching))
fraud/->fraud-detector
(fn [] (->NoopProcessor :fraud-detector))
tax-compliance/->tax-compliance
(fn [] (->NoopProcessor :tax-compliance-analyzer))
accounts/->accounts
(fn [] (->NoopProcessor :accounts))
cost-center/->cost-center
(fn [] (->NoopProcessor :cost-center-matcher))
accruals/->accruals
(fn [] (->NoopProcessor :accruals-matcher))
supplier/->supplier-matcher
(fn [] (->NoopProcessor :supplier-matcher))
supplier/->supplier-verifier
(fn [] (->NoopProcessor :supplier-verifier))
financial-validation/->financial-validation-resolver
(fn [] (->NoopProcessor :financial-validation-resolver))
uncertain-validations/->uncertain-validations-resolver
(fn [] (->NoopProcessor :uncertain-validations-resolver))]
(orchestrator/recompute-all!
{:db-pool fixtures/*db* :llm-config {} :search-config {} :notifications {} :aws {}}
{:document-id doc-id
:history-id (:document-history/id history-row)
:document-version 2}))
(is (= 1 @fires)))))
(defrecord ^:private ThrowingProcessor [id]
proc/IProcessor
(-id [_] id)
(-reads [_ _state] [])
(-writes [_ _state] [])
(-diagnostic [_ _state] nil)
(-modes [_] #{:ingestion :edit})
(-always? [_] true)
(-compute [_ _ _] (throw (ex-info "boom" {})))
(-apply-ops [_ _ _] []))
(deftest recompute-all-fires-event-on-processor-failure-test
(testing "fire-diagnostics-recomputed! still called when a processor throws"
(let [le-id (helpers/create-legal-entity!)
doc-id (helpers/create-document! le-id)
identity-id (helpers/create-identity!)
ingestion-id (helpers/create-ingestion! doc-id identity-id :status :completed)
_ (db.sql/execute-one! fixtures/*db*
{:update :document
:set {:structured-data [:lift {:document-type "invoice"
:issuer {:name "Acme"}}]
:type (db.sql/->cast "invoice" :document-type)
:version [:raw "version + 1"]}
:where [:= :id doc-id]})
_ (document-history/insert! fixtures/*db*
{:document-id doc-id
:change-type :ingestion
:ingestion-id ingestion-id
:patch []})
history-row (document-history/insert! fixtures/*db*
{:document-id doc-id
:change-type :edit
:edited-by identity-id
:patch [{"op" "replace" "path" "/issuer/name" "value" "X"}]})
fires (atom 0)]
(with-redefs [app.ingestion/fire-diagnostics-recomputed!
(fn [_ _] (swap! fires inc))
matching/->matching
(fn [] (->NoopProcessor :matching))
fraud/->fraud-detector
(fn [] (->NoopProcessor :fraud-detector))
tax-compliance/->tax-compliance
(fn [] (->ThrowingProcessor :tax-compliance-analyzer))
accounts/->accounts
(fn [] (->NoopProcessor :accounts))
cost-center/->cost-center
(fn [] (->NoopProcessor :cost-center-matcher))
accruals/->accruals
(fn [] (->NoopProcessor :accruals-matcher))
supplier/->supplier-matcher
(fn [] (->NoopProcessor :supplier-matcher))
supplier/->supplier-verifier
(fn [] (->NoopProcessor :supplier-verifier))
financial-validation/->financial-validation-resolver
(fn [] (->NoopProcessor :financial-validation-resolver))
uncertain-validations/->uncertain-validations-resolver
(fn [] (->NoopProcessor :uncertain-validations-resolver))]
(orchestrator/recompute-all!
{:db-pool fixtures/*db* :llm-config {} :search-config {} :notifications {} :aws {}}
{:document-id doc-id
:history-id (:document-history/id history-row)
:document-version 2}))
(is (= 1 @fires)))))
Add [com.getorcha.app.ingestion :as app.ingestion] to the test ns requires.
Run: clj -X:test:silent :nses '[com.getorcha.workers.diagnostics-recompute-test]'
Expected: recompute-all-fires-event-on-success-test FAILS with (= 1 0) — the orchestrator doesn't fire yet. The failure test may already fail or pass unpredictably; doesn't matter at this step.
try/finallyIn src/com/getorcha/workers/diagnostics_recompute/orchestrator.clj:recompute-all! (line 44):
(defn recompute-all!
"Top-level entry-point invoked by the SQS consumer. Builds
edit-mode state and calls the processors engine. Always fires
`diagnostics-recomputed` on exit — success or failure — so the UI
converges on the current DB state even if the engine throws."
[ctx {:keys [document-id history-id _document-version] :as _params}]
(let [db-pool (:db-pool ctx)
doc (fetch-document db-pool document-id)
hist (fetch-history-row db-pool history-id)
document-type (get-in doc [:document/structured-data :document-type])
changed (compute-changed-leaves
(:document-history/patch hist)
document-type
(:document/structured-data doc))
state {:mode :edit
:trigger-kind :edit
:history-id history-id
:edited-by (:document-history/edited-by hist)
:document doc
:structured-data (:document/structured-data doc)
:changed-leaves changed}]
(try
(engine/run-processors!
ctx
state
[[(validations/->validations)]
[(tax-compliance/->tax-compliance)
(matching/->matching)
(accounts/->accounts)
(cost-center/->cost-center)
(accruals/->accruals)
(supplier/->supplier-matcher)
(supplier/->supplier-verifier)
(financial-validation/->financial-validation-resolver)
(uncertain-validations/->uncertain-validations-resolver)]
[(fraud/->fraud-detector)]])
:done
(finally
(try
(app.ingestion/fire-diagnostics-recomputed! db-pool document-id)
(catch Exception e
(log/warn e "Failed to fire diagnostics-recomputed after recompute"
{:document-id document-id})))))))
Add to the ns :require at the top:
[clojure.tools.logging :as log]
[com.getorcha.app.ingestion :as app.ingestion]
Run: clj -X:test:silent :nses '[com.getorcha.workers.diagnostics-recompute-test]'
Expected: both recompute-all-fires-event-on-success-test and recompute-all-fires-event-on-processor-failure-test pass.
Run: clj-kondo --lint src test dev
Expected: 0 warnings.
git add src/com/getorcha/workers/diagnostics_recompute/orchestrator.clj test/com/getorcha/workers/diagnostics_recompute_test.clj
git commit -m "feat(recompute): fire diagnostics-recomputed from orchestrator try/finally"
try/finallyFiles:
Modify: src/com/getorcha/workers/ap/processors/matching/worker.clj (inner try in the orchestrator thread body, around line 221-232)
Test: test/com/getorcha/workers/ap/processors/matching/worker_test.clj (file already exists)
Step 1: Write the failing tests
Append to test/com/getorcha/workers/ap/processors/matching/worker_test.clj:
(deftest process-document-fires-diagnostics-recomputed-on-success-test
(testing "matching worker fires diagnostics-recomputed after successful processing"
(let [le-id (helpers/create-legal-entity!)
doc-id (helpers/create-document! le-id)
_ (db.sql/execute-one! fixtures/*db*
{:update :document
:set {:structured-data [:lift {:document-type "invoice"}]
:type (db.sql/->cast "invoice" :document-type)}
:where [:= :id doc-id]})
fires (atom 0)]
(with-redefs [app.ingestion/fire-diagnostics-recomputed!
(fn [_ _] (swap! fires inc))
matching/->matching
(fn [] (->NoopProcessor :matching))]
(worker/process-document-with-notify!
{:db-pool fixtures/*db* :llm-config {} :aws {}}
doc-id))
(is (= 1 @fires)))))
(deftest process-document-fires-on-failure-test
(testing "matching worker fires diagnostics-recomputed even when processing throws"
(let [le-id (helpers/create-legal-entity!)
doc-id (helpers/create-document! le-id)
;; Don't set structured-data — process-document! will throw ::missing-structured-data
fires (atom 0)]
(with-redefs [app.ingestion/fire-diagnostics-recomputed!
(fn [_ _] (swap! fires inc))]
(try
(worker/process-document-with-notify!
{:db-pool fixtures/*db* :llm-config {} :aws {}}
doc-id)
(catch Exception _)))
(is (= 1 @fires)))))
This references a new public wrapper process-document-with-notify! that Task 7 Step 2 introduces. The wrapper lets the test exercise the try/finally without simulating the full SQS polling loop.
Add test-ns requires as needed (com.getorcha.app.ingestion, com.getorcha.workers.ap.processors.matching.worker, etc.).
Define the Noop processor in this test file (copy from diagnostics_recompute_test.clj), or extract to test/com/getorcha/test/notification_helpers.clj as a reusable helper if you prefer.
Run: clj -X:test:silent :nses '[com.getorcha.workers.ap.processors.matching.worker-test]'
Expected: both new tests fail — process-document-with-notify! is not defined.
process-document! and add the wrapperIn src/com/getorcha/workers/ap/processors/matching/worker.clj, after process-document! (around line 108), add:
(defn process-document-with-notify!
"Runs `process-document!` and fires `diagnostics-recomputed` in a
`finally` block, so the UI converges on current DB state regardless
of success or failure. Exposed publicly so tests can exercise the
try/finally without simulating the SQS polling loop."
[{:keys [db-pool] :as context} document-id]
(try
(process-document! context document-id)
(finally
(try
(app.ingestion/fire-diagnostics-recomputed! db-pool document-id)
(catch Exception e
(log/warn e "Failed to fire diagnostics-recomputed after matching"
{:document-id document-id}))))))
Then in the orchestrator thread body (around line 225), change the call:
(try
(process-document! config document-id)
...)
to:
(try
(process-document-with-notify! config document-id)
...)
Add to ns :require:
[com.getorcha.app.ingestion :as app.ingestion]
Run: clj -X:test:silent :nses '[com.getorcha.workers.ap.processors.matching.worker-test]'
Expected: both new tests pass.
Run: clj-kondo --lint src test dev
Expected: 0 warnings.
git add src/com/getorcha/workers/ap/processors/matching/worker.clj test/com/getorcha/workers/ap/processors/matching/worker_test.clj
git commit -m "feat(matching): fire diagnostics-recomputed from worker try/finally"
reap-stuck-running! to fire events per affected documentFiles:
Modify: src/com/getorcha/db/document_processor_run.clj (reap-stuck-running! at line 119)
Modify: src/com/getorcha/workers/diagnostics_recompute.clj (consumer startup call at line 87)
Test: test/com/getorcha/db/document_processor_run_test.clj (extend)
Step 1: Write the failing test
Append to test/com/getorcha/db/document_processor_run_test.clj:
(deftest reap-stuck-running-returns-document-ids-test
(testing "returns a vector of document ids whose rows were reaped"
(let [le-id (helpers/create-legal-entity!)
doc-id (helpers/create-document! le-id)
identity-id (helpers/create-identity!)
ingestion-id (helpers/create-ingestion! doc-id identity-id :status :in-progress)
run-id (db.run/insert-running!
fixtures/*db*
{:document-id doc-id
:processor-id "validations"
:trigger-kind :ingestion
:ingestion-id ingestion-id
:document-version 1})]
;; Age the row past the threshold
(db.sql/execute-one! fixtures/*db*
{:update :document-processor-run
:set {:started-at [:- [:now] [:raw "interval '1 hour'"]]}
:where [:= :id run-id]})
(let [affected (db.run/reap-stuck-running! fixtures/*db* 30)]
(is (vector? affected))
(is (= [doc-id] affected))))))
Run: clj -X:test:silent :nses '[com.getorcha.db.document-processor-run-test]'
Expected: FAIL — reap-stuck-running! currently returns an integer, not a vector of doc-ids.
reap-stuck-running! to return doc-idsIn src/com/getorcha/db/document_processor_run.clj:119:
(defn reap-stuck-running!
"Transitions any row stuck in status='running' whose `started_at` is older
than `max-age-minutes` to status='failed' with a synthetic error message.
Intended for scheduled / startup use to recover from completion failures
(transient DB errors in complete-run! / fail-run! that left a row orphaned).
Returns a vector of `document-id` values whose rows were reaped
(duplicates removed). Callers typically iterate this to fire a
`diagnostics-recomputed` event per affected document."
[db-pool max-age-minutes]
(let [rows (db.sql/execute!
db-pool
{:update :document-processor-run
:set {:status (db.sql/->cast :failed :processor-run-status)
:ended-at [:now]
:error "reaped: stuck in running state"}
:where [:and
[:= :status (db.sql/->cast :running :processor-run-status)]
[:< :started-at [:raw (str "now() - interval '" (long max-age-minutes) " minutes'")]]]
:returning [:document-id]})]
(into [] (distinct (map :document-processor-run/document-id rows)))))
In src/com/getorcha/workers/diagnostics_recompute.clj:87, change:
(try
(let [reaped (db.run/reap-stuck-running! db-pool stuck-run-max-age-minutes)]
(when (pos? reaped)
(log/warn "Reaped stuck document_processor_run rows on startup"
{:count reaped :max-age-minutes stuck-run-max-age-minutes})))
(catch Exception e
(log/warn e "Failed to reap stuck runs on startup — continuing")))
to:
(try
(let [reaped (db.run/reap-stuck-running! db-pool stuck-run-max-age-minutes)]
(when (seq reaped)
(log/warn "Reaped stuck document_processor_run rows on startup"
{:count (count reaped) :max-age-minutes stuck-run-max-age-minutes})
(doseq [doc-id reaped]
(try
(app.ingestion/fire-diagnostics-recomputed! db-pool doc-id)
(catch Exception e
(log/warn e "Failed to fire diagnostics-recomputed after reap"
{:document-id doc-id}))))))
(catch Exception e
(log/warn e "Failed to reap stuck runs on startup — continuing")))
Add [com.getorcha.app.ingestion :as app.ingestion] to the ns :require.
Run: clj -X:test:silent :nses '[com.getorcha.db.document-processor-run-test com.getorcha.workers.diagnostics-recompute-test]'
Expected: all tests pass.
The consumer-init reap-and-fire loop isn't easily unit-testable in isolation (it runs inside ig/init-key). The Step 1 test validates the API contract; the consumer-level firing is exercised during Task 13's manual smoke test.
Run: clj-kondo --lint src test dev
Expected: 0 warnings.
git add src/com/getorcha/db/document_processor_run.clj src/com/getorcha/workers/diagnostics_recompute.clj test/com/getorcha/db/document_processor_run_test.clj
git commit -m "feat(reap): return doc-ids and fire diagnostics-recomputed per affected document"
sse-swap from diagnostic section componentsNow that the atomic event fires end-to-end, remove the per-event sse-swap subscriptions. Section ids stay (OOB targets).
Files:
Modify: src/com/getorcha/app/ui/components.clj — five section defs.
Step 1: Update validation-results-section
Around line 2926, change:
[:div {:id "diagnostic-section-validations"
:class (diagnostic-section-class state)
:hx-ext "sse"
:sse-swap "diagnostic-run-started:validations,diagnostic-run-completed:validations"
:hx-swap "outerHTML"}
...]
to:
[:div {:id "diagnostic-section-validations"
:class (diagnostic-section-class state)}
...]
contract-validation-sectionAround line 2630, same change — drop :hx-ext, :sse-swap, :hx-swap. Keep :id "diagnostic-section-validations" and :class.
fraud-detection-sectionFind the outer div with :sse-swap "diagnostic-run-started:fraud-detector,…" (around line 2981). Same edit.
tax-compliance-sectionFind the outer div with :sse-swap "diagnostic-run-started:tax-compliance-analyzer,…" (around line 3009). Same edit.
reconciliation-sectionFind the outer div with :sse-swap "diagnostic-run-started:reconciliation,…" (around line 3060). Same edit.
Run: grep -n "diagnostic-run-started\|diagnostic-run-completed" src/com/getorcha/app/ui/components.clj
Expected: no matches. All five sections are updated.
Run: clj -X:test:silent :nses '[com.getorcha.app.http.documents.view.shared-test com.getorcha.workers.diagnostics-recompute-test]'
Expected: all tests pass (sections still render; just without sse-swap attrs).
Run: clj-kondo --lint src test dev
Expected: 0 warnings.
git add src/com/getorcha/app/ui/components.clj
git commit -m "refactor(components): drop sse-swap from diagnostic sections"
matching-complete SSE listener, the :matching handler case, and MatchingEventFiles:
src/com/getorcha/app/http/documents/view/shared.cljsrc/com/getorcha/app/ingestion.cljThe old matching-complete SSE event stopped firing when trigger_document_matching_event was dropped in migration 20260414120000. The sse-swap="matching-complete" listener and the :matching case in detail-events have been dead ever since. export-status-changed in invoice.clj is unrelated and stays.
matching-complete listener in matches-sectionIn src/com/getorcha/app/http/documents/view/shared.clj:matches-section (around line 377-382), remove the whole block that emits:
(when matching-pending?
[:div {:hx-ext "sse"
:sse-connect (app.http.routes/path-for router :com.getorcha.app.http.documents.view/detail-events {:document-id document-id})
:sse-swap "matching-complete"
:hx-target "#section-matches"
:hx-swap "outerHTML"}])
:matching case from detail-eventsIn src/com/getorcha/app/http/documents/view/shared.clj:detail-events (around line 1162), delete the entire :matching branch (the (let [document …] …) block producing matching-complete).
MatchingEvent schema variantIn src/com/getorcha/app/ingestion.clj:
Delete the MatchingEvent def (around line 291).
Remove [:matching MatchingEvent] from the DocumentEvent :multi dispatch.
Step 4: Run relevant tests
Run: clj -X:test:silent :nses '[com.getorcha.app.ingestion-test com.getorcha.app.http.documents.view.shared-test]'
Expected: all tests pass.
Run: clj-kondo --lint src test dev
Expected: 0 warnings.
git add src/com/getorcha/app/http/documents/view/shared.clj src/com/getorcha/app/ingestion.clj
git commit -m "refactor: drop dead matching-complete SSE wiring and MatchingEvent schema"
Everything required for the atomic pipeline is in place; per-processor events can now come out.
Files:
Modify: src/com/getorcha/app/ingestion.clj
Modify: src/com/getorcha/app/http/documents/view/shared.clj
Modify: test/com/getorcha/app/ingestion_test.clj
Modify: test/com/getorcha/workers/diagnostics_recompute_test.clj
Step 1: Remove the event-type schemas
In src/com/getorcha/app/ingestion.clj:
Remove DiagnosticRunStartedEvent def.
Remove DiagnosticRunCompletedEvent def.
Remove [:diagnostic-run-started DiagnosticRunStartedEvent] and [:diagnostic-run-completed DiagnosticRunCompletedEvent] from the DocumentEvent multi.
Step 2: Remove the SSE handler cases
In src/com/getorcha/app/http/documents/view/shared.clj:detail-events (around line 1178-1221), remove the :diagnostic-run-started and :diagnostic-run-completed branches.
processor-id->slice-key map and render-diagnostic-section helperThese live around lines 1052-1070 in shared.clj. Neither is referenced after Step 2.
In test/com/getorcha/app/ingestion_test.clj:
document-event-coerces-every-processor-id-test — it asserted behaviour that no longer exists (every processor id coerces through DiagnosticRunCompletedEvent).all-processors def, the diagnostic-run-payload helper, and the processor-ns requires that only that test used.Keep document-event-coerces-diagnostics-recomputed-test (from Task 1) and the fire-diagnostics-recomputed-test (from Task 2).
end-to-end-edit-recompute assertion setIn test/com/getorcha/workers/diagnostics_recompute_test.clj:end-to-end-edit-recompute — this test currently checks that diagnostics slices are written end to end. With the atomic change, it should also assert the atomic event fired once (see Task 6 tests, which cover this separately). Leave end-to-end-edit-recompute as-is for slice coverage; no change needed here unless it was broken by removing the NoopProcessor ids that matched the old per-processor event types. It should still pass — the engine's internal behaviour is unchanged.
Run it to confirm:
clj -X:test:silent :nses '[com.getorcha.workers.diagnostics-recompute-test]'
Expected: all tests pass.
Run: clj -X:test:silent 2>&1 | grep -A 5 -E "(FAIL in|ERROR in|Execution error|failed because|Ran .* tests)"
Expected: 0 failures, 0 errors.
Run: clj-kondo --lint src test dev
Expected: 0 warnings.
git add src/com/getorcha/app/ingestion.clj src/com/getorcha/app/http/documents/view/shared.clj test/com/getorcha/app/ingestion_test.clj test/com/getorcha/workers/diagnostics_recompute_test.clj
git commit -m "refactor: remove per-processor diagnostic event schemas, handler cases, and dispatcher"
trigger_processor_run_eventFiles:
resources/migrations/20260418100000-drop-processor-run-event-trigger.up.sqlresources/migrations/20260418100000-drop-processor-run-event-trigger.down.sql(Use today's date with a sequence number one higher than the most recent migration; adjust the filename to match.)
.up.sqlDROP TRIGGER IF EXISTS trigger_processor_run_event ON document_processor_run;
--;;
DROP FUNCTION IF EXISTS notify_processor_run_event();
.down.sqlCopy the CREATE OR REPLACE FUNCTION notify_processor_run_event() and CREATE TRIGGER trigger_processor_run_event blocks from the existing migration resources/migrations/20260414120000-add-document-diagnostics.up.sql (lines 207-245). The down migration restores the function/trigger verbatim so a rollback re-enables per-processor events.
Start a Clojure REPL or run the migrate command:
clj -X:migrate
Or equivalent. Check the project's README / bb.edn for the exact migratus invocation.
Expected: migration runs without error.
Run:
psql -h localhost -U postgres -d orcha -c "SELECT tgname FROM pg_trigger WHERE tgrelid = 'document_processor_run'::regclass AND NOT tgisinternal"
Expected: no rows returned (trigger dropped).
Run:
psql -h localhost -U postgres -d orcha -c "\df notify_processor_run_event"
Expected: no such function.
Run: clj -X:test:silent 2>&1 | grep -A 5 -E "(FAIL in|ERROR in|Execution error|failed because|Ran .* tests)"
Expected: 0 failures, 0 errors.
git add resources/migrations/20260418100000-drop-processor-run-event-trigger.up.sql resources/migrations/20260418100000-drop-processor-run-event-trigger.down.sql
git commit -m "chore(db): drop processor-run-event trigger; atomic events replace it"
Files:
None modified. Just verification.
Step 1: Repo-wide lint
Run: clj-kondo --lint src test dev
Expected: linting took Xms, errors: 0, warnings: 0.
Run: clj -X:test:silent 2>&1 | grep -A 5 -E "(FAIL in|ERROR in|Execution error|failed because|Ran .* tests)"
Expected: 0 failures, 0 errors.
(integrant.repl/reset) to pick up all changes.:stale (badge: "Stale") immediately via the edit response OOB swap.diagnostics-recompute).:current atomically when diagnostics-recomputed fires.status-changed.diagnostics-recomputed event.With the REPL running, tail logs/orcha.log. Trigger another edit. Expected:
Log contains fire-diagnostics-recomputed! calls (if you added DEBUG logging, which you shouldn't have).
Log does NOT contain "Invalid Document Event" (the schema now doesn't reject anything real).
Log does NOT contain any reference to :diagnostic-run-started or :diagnostic-run-completed events (the trigger was dropped, so none fire).
Step 6: Confirm the branch is ready
Run:
git log --oneline inline-editing-ui..HEAD
Expected: a clean sequence of commits matching Tasks 1–12.