For Claude: REQUIRED SUB-SKILL: Use superpowers:executing-plans to implement this plan task-by-task.
Goal: Improve invoice-to-contract matching by enriching LLM prompts and adding new evidence signals.
Architecture: Two independent changes: (1) enrich format-document-summary to include line items, deliverables, pricing — giving the LLM enough context to decide; (2) add description-overlap signal and relax date-within-period to boost deterministic scores for invoice<->contract pairs.
Tech Stack: Clojure, clojure.test, clojure.string, clojure.set
Files:
test/com/getorcha/workers/matching/llm_decision_test.cljStep 1: Update existing format-document-summary-test and add new tests
The existing tests check for basic fields (number, name, type). Add assertions for the new fields. Add new test cases for documents with line items, deliverables, and pricing.
;; Replace the existing format-document-summary-test with this expanded version:
(deftest format-document-summary-test
(testing "invoice includes line items, service period, and references"
(let [doc #:document{:type "invoice"
:structured-data {:invoice-number "INV-001"
:invoice-date "2024-02-20"
:issuer {:name "ACME Corp"
:vat-id "DE123"}
:total 15000
:currency "EUR"
:service-period {:start "2024-01-01"
:end "2024-01-31"}
:po-reference "PO-2024-050"
:due-date "2024-03-20"
:line-items [{:description "Consulting services"
:quantity 10
:unit "hours"
:amount 5000}
{:description "Travel expenses"
:quantity 1
:unit "flat"
:amount 2000}]}}
summary (llm-decision/format-document-summary doc)]
(is (str/includes? summary "Invoice"))
(is (str/includes? summary "INV-001"))
(is (str/includes? summary "ACME Corp"))
(is (str/includes? summary "15000"))
(is (str/includes? summary "PO-2024-050"))
(is (str/includes? summary "2024-01-01"))
(is (str/includes? summary "2024-01-31"))
(is (str/includes? summary "2024-03-20"))
(is (str/includes? summary "Consulting services"))
(is (str/includes? summary "Travel expenses"))
(is (str/includes? summary "5000"))
(is (str/includes? summary "2000"))))
(testing "invoice with minimal data uses N/A for missing fields"
(let [doc #:document{:type "invoice"
:structured-data {:invoice-number "INV-002"
:invoice-date "2024-01-01"
:issuer {:name "Test"}
:total 100}}
summary (llm-decision/format-document-summary doc)]
(is (str/includes? summary "INV-002"))
(is (not (str/includes? summary "Line Items")))
(is (not (str/includes? summary "Service Period")))
(is (not (str/includes? summary "PO Reference")))))
(testing "contract includes deliverables, pricing, and PO references"
(let [doc #:document{:type "contract"
:structured-data {:contract-number "C-2024-010"
:title "Transport Management Services"
:contract-type "service"
:counterparty {:name "Legal Partners GmbH"
:tax-id "DE111222333"}
:effective-date "2024-01-01"
:expiration-date "2024-12-31"
:currency "EUR"
:total-value 50000
:base-fee 2000
:deliverables ["Capacity booking" "Nomination processing"]
:variable-components [{:description "Volume fee per MWh"
:formula "0.05 EUR/MWh"}]
:payment-schedule [{:description "Monthly retainer"
:date "2024-01-15"
:amount 2000}]
:po-references ["PO-2024-001" "PO-2024-002"]}}
summary (llm-decision/format-document-summary doc)]
(is (str/includes? summary "Contract"))
(is (str/includes? summary "C-2024-010"))
(is (str/includes? summary "Transport Management Services"))
(is (str/includes? summary "service"))
(is (str/includes? summary "Legal Partners GmbH"))
(is (str/includes? summary "50000"))
(is (str/includes? summary "Capacity booking"))
(is (str/includes? summary "Nomination processing"))
(is (str/includes? summary "Volume fee per MWh"))
(is (str/includes? summary "2000"))
(is (str/includes? summary "PO-2024-001"))))
(testing "contract with minimal data omits empty sections"
(let [doc #:document{:type "contract"
:structured-data {:counterparty {:name "Minimal GmbH"}
:effective-date "2024-01-01"}}
summary (llm-decision/format-document-summary doc)]
(is (str/includes? summary "Minimal GmbH"))
(is (not (str/includes? summary "Deliverables")))
(is (not (str/includes? summary "Pricing")))
(is (not (str/includes? summary "PO References")))))
(testing "purchase-order includes line items and references"
(let [doc #:document{:type "purchase-order"
:structured-data {:po-number "PO-001"
:po-date "2024-02-01"
:supplier {:name "Widgets Inc"
:vat-id "DE999"}
:total-value 20000
:currency "EUR"
:contract-reference "CTR-001"
:requisition-number "REQ-100"
:line-items [{:description "Widget A"
:quantity 100
:unit "pcs"
:amount 10000}]}}
summary (llm-decision/format-document-summary doc)]
(is (str/includes? summary "Purchase Order"))
(is (str/includes? summary "PO-001"))
(is (str/includes? summary "CTR-001"))
(is (str/includes? summary "REQ-100"))
(is (str/includes? summary "Widget A"))
(is (str/includes? summary "10000"))))
(testing "goods-received-note includes line items"
(let [doc #:document{:type "goods-received-note"
:structured-data {:grn-number "GRN-2024-100"
:receipt-date "2024-03-15"
:supplier {:name "Parts Supply AG"}
:po-reference "PO-2024-050"
:line-items [{:description "Part X"
:quantity-received 50
:unit "pcs"}]}}
summary (llm-decision/format-document-summary doc)]
(is (str/includes? summary "Goods Received Note"))
(is (str/includes? summary "GRN-2024-100"))
(is (str/includes? summary "PO-2024-050"))
(is (str/includes? summary "Part X"))
(is (str/includes? summary "50"))))
(testing "unknown type returns type name"
(let [doc {:document/type "unknown-thing"
:document/structured-data {:foo "bar"}}
summary (llm-decision/format-document-summary doc)]
(is (str/includes? summary "unknown-thing")))))
Step 2: Run tests to verify they fail
Run: clj -X:test:silent :nses '[com.getorcha.workers.matching.llm-decision-test]'
Expected: Several assertions FAIL (missing line items, deliverables, etc. in summaries)
Step 3: Commit
git add test/com/getorcha/workers/matching/llm_decision_test.clj
git commit -m "test: add assertions for enriched LLM document summaries"
Files:
src/com/getorcha/workers/matching/llm_decision.cljStep 1: Implement enriched format-document-summary
Replace the existing format-document-summary function. The new version builds sections conditionally — only include a section header if the data exists.
(defn ^:private format-line-items
"Format line items as a readable list for the LLM prompt."
[line-items]
(when (seq line-items)
(->> line-items
(map-indexed
(fn [i {:keys [description quantity unit amount unit-price]}]
(str " " (inc i) ". " (or description "N/A")
(when quantity (str " — " quantity (when unit (str " " unit))))
(when unit-price (str " @ " unit-price))
(when amount (str " = " amount)))))
(str/join "\n"))))
(defn ^:private format-grn-line-items
"Format GRN line items (quantity-received instead of quantity)."
[line-items]
(when (seq line-items)
(->> line-items
(map-indexed
(fn [i {:keys [description quantity-received unit]}]
(str " " (inc i) ". " (or description "N/A")
(when quantity-received (str " — " quantity-received (when unit (str " " unit)))))))
(str/join "\n"))))
(defn format-document-summary
"Format a document's key fields into a human-readable summary for the LLM prompt.
Extracts identifying fields based on document type.
Expects a DB row with `:document/type` (string) and `:document/structured-data`."
[{:document/keys [type structured-data]}]
(case type
"invoice"
(let [{:keys [invoice-number invoice-date total currency
service-period po-reference gr-reference
due-date payment-terms line-items]} structured-data
issuer-name (get-in structured-data [:issuer :name] "N/A")
issuer-vat (get-in structured-data [:issuer :vat-id])
base (str "Type: Invoice"
"\nNumber: " (or invoice-number "N/A")
"\nDate: " (or invoice-date "N/A")
"\nSupplier: " issuer-name (when issuer-vat (str " (VAT: " issuer-vat ")"))
"\nTotal: " (or total "N/A") (when currency (str " " currency)))]
(cond-> base
po-reference (str "\nPO Reference: " po-reference)
gr-reference (str "\nGR Reference: " gr-reference)
service-period (str "\nService Period: " (:start service-period) " to " (:end service-period))
due-date (str "\nDue Date: " due-date)
payment-terms (str "\nPayment Terms: " payment-terms)
(seq line-items)
(str "\nLine Items:\n" (format-line-items line-items))))
"purchase-order"
(let [{:keys [po-number po-date total-value currency
contract-reference requisition-number line-items]} structured-data
supplier-name (get-in structured-data [:supplier :name] "N/A")
supplier-vat (get-in structured-data [:supplier :vat-id])
base (str "Type: Purchase Order"
"\nNumber: " (or po-number "N/A")
"\nDate: " (or po-date "N/A")
"\nSupplier: " supplier-name (when supplier-vat (str " (VAT: " supplier-vat ")"))
"\nTotal: " (or total-value "N/A") (when currency (str " " currency)))]
(cond-> base
contract-reference (str "\nContract Ref: " contract-reference)
requisition-number (str "\nRequisition: " requisition-number)
(seq line-items)
(str "\nLine Items:\n" (format-line-items line-items))))
"contract"
(let [{:keys [contract-number title contract-type
effective-date expiration-date currency total-value
base-fee deliverables variable-components
payment-schedule po-references description]} structured-data
cp-name (get-in structured-data [:counterparty :name] "N/A")
cp-tax (get-in structured-data [:counterparty :tax-id])
base (str "Type: Contract"
(when title (str "\nTitle: " title))
(when contract-type (str "\nContract Type: " contract-type))
"\nNumber: " (or contract-number "N/A")
"\nCounterparty: " cp-name (when cp-tax (str " (VAT: " cp-tax ")"))
"\nEffective: " (or effective-date "N/A") " to " (or expiration-date "open-ended")
(when (or total-value currency)
(str "\nValue: " (or total-value "N/A") (when currency (str " " currency)))))]
(cond-> base
description
(str "\nDescription: " description)
(seq deliverables)
(str "\nDeliverables:\n" (->> deliverables (map #(str " - " %)) (str/join "\n")))
(or base-fee (seq variable-components))
(str "\nPricing:"
(when base-fee (str "\n Base fee: " base-fee))
(when (seq variable-components)
(->> variable-components
(map (fn [{:keys [description formula]}]
(str "\n " description (when formula (str " (" formula ")")))))
(str/join ""))))
(seq payment-schedule)
(str "\nPayment Schedule:\n"
(->> payment-schedule
(map (fn [{:keys [description date amount]}]
(str " - " description
(when date (str " on " date))
(when amount (str ": " amount)))))
(str/join "\n")))
(seq po-references)
(str "\nPO References: " (str/join ", " po-references))))
"goods-received-note"
(let [{:keys [grn-number receipt-date po-reference
delivery-note-number line-items]} structured-data
supplier-name (get-in structured-data [:supplier :name] "N/A")
base (str "Type: Goods Received Note"
"\nNumber: " (or grn-number "N/A")
"\nDate: " (or receipt-date "N/A")
"\nSupplier: " supplier-name
"\nPO Ref: " (or po-reference "N/A"))]
(cond-> base
delivery-note-number (str "\nDelivery Note: " delivery-note-number)
(seq line-items)
(str "\nLine Items:\n" (format-grn-line-items line-items))))
(str "Type: " type)))
Step 2: Run tests to verify they pass
Run: clj -X:test:silent :nses '[com.getorcha.workers.matching.llm-decision-test]'
Expected: ALL PASS
Step 3: Lint
Run: clj-kondo --lint src/com/getorcha/workers/matching/llm_decision.clj --fail-level warning
Expected: No warnings
Step 4: Commit
git add src/com/getorcha/workers/matching/llm_decision.clj
git commit -m "feat(matching): enrich LLM prompt with line items, deliverables, and pricing"
description-overlap Evidence Signal — TestsFiles:
test/com/getorcha/workers/matching/evidence_test.cljStep 1: Add tests for description-overlap signal
Append these tests after the existing delivery-date-match-signal-test:
(deftest description-overlap-signal-test
(testing "fires when invoice line items overlap with contract deliverables"
(let [{:keys [evidence]}
(evidence/compute-score
#:document{:type "invoice"
:structured-data
{:line-items [{:description "Setup Kapazitätsplatform"}
{:description "Kapazitätsbuchung"}
{:description "Setup TSO inkl. Marktkommunikation"}]}}
#:document{:type "contract"
:structured-data
{:deliverables ["Durchführung von Kapazitätsbuchungen"
"Einrichtung und Betrieb der elektronischen Marktkommunikation"]}})]
(is (some #(= :description-overlap (:signal %)) evidence))))
(testing "does not fire when descriptions have no meaningful overlap"
(let [{:keys [evidence]}
(evidence/compute-score
#:document{:type "invoice"
:structured-data
{:line-items [{:description "Office supplies"}
{:description "Printer paper"}]}}
#:document{:type "contract"
:structured-data
{:deliverables ["Software development"
"Cloud hosting services"]}})]
(is (not (some #(= :description-overlap (:signal %)) evidence)))))
(testing "does not fire when one side has no descriptions"
(let [{:keys [evidence]}
(evidence/compute-score
#:document{:type "invoice"
:structured-data {:line-items []}}
#:document{:type "contract"
:structured-data {:deliverables ["Something"]}})]
(is (not (some #(= :description-overlap (:signal %)) evidence)))))
(testing "works for PO line items vs contract deliverables"
(let [{:keys [evidence]}
(evidence/compute-score
#:document{:type "purchase-order"
:structured-data
{:line-items [{:description "Monthly transport management fee"}]}}
#:document{:type "contract"
:structured-data
{:deliverables ["Transport management services"
"Monthly fee for transport operations"]}})]
(is (some #(= :description-overlap (:signal %)) evidence))))
(testing "weight is 25"
(let [{:keys [evidence]}
(evidence/compute-score
#:document{:type "invoice"
:structured-data
{:line-items [{:description "Kapazitätsbuchung services"}]}}
#:document{:type "contract"
:structured-data
{:deliverables ["Durchführung von Kapazitätsbuchungen"]}})]
(is (= 25 (:weight (first (filter #(= :description-overlap (:signal %)) evidence))))))))
Step 2: Run to verify they fail
Run: clj -X:test:silent :nses '[com.getorcha.workers.matching.evidence-test]'
Expected: FAIL — :description-overlap signal never produced
Step 3: Commit
git add test/com/getorcha/workers/matching/evidence_test.clj
git commit -m "test: add description-overlap evidence signal tests"
description-overlap Evidence Signal — ImplementationFiles:
src/com/getorcha/workers/matching/evidence.cljStep 1: Add the signal weight and helper functions
Add :description-overlap to evidence-signals:
;; In evidence-signals map, add:
:description-overlap 25 ; Token overlap between descriptions/deliverables
Add stopwords set and helper functions before collect-signals:
(def ^:private stopwords
"Common German and English stopwords to exclude from description overlap."
#{"der" "die" "das" "den" "dem" "des" "ein" "eine" "einer" "eines" "einem" "einen"
"und" "oder" "aber" "von" "vom" "zu" "zum" "zur" "mit" "für" "auf" "an" "in" "im"
"bei" "nach" "über" "unter" "aus" "als" "wie" "bis" "durch" "um" "gegen"
"the" "a" "an" "and" "or" "but" "of" "to" "with" "for" "on" "at" "in" "by"
"from" "as" "is" "are" "was" "were" "be" "been" "being" "have" "has" "had"
"do" "does" "did" "will" "would" "shall" "should" "may" "might" "can" "could"
"not" "no" "nor" "so" "if" "then" "than" "that" "this" "these" "those"
"it" "its" "they" "their" "them" "we" "our" "you" "your" "he" "she" "his" "her"})
(defn ^:private tokenize
"Tokenize text: lowercase, strip punctuation, remove stopwords."
[text]
(when (seq text)
(->> (str/split (str/lower-case text) #"[\s\p{Punct}]+")
(remove str/blank?)
(remove stopwords))))
(defn ^:private extract-description-tokens
"Extract description tokens from a document's line items or deliverables."
[{:document/keys [type structured-data]}]
(case type
("invoice" "purchase-order")
(->> (:line-items structured-data)
(keep :description)
(mapcat tokenize)
set)
"goods-received-note"
(->> (:line-items structured-data)
(keep :description)
(mapcat tokenize)
set)
"contract"
(->> (:deliverables structured-data)
(mapcat tokenize)
set)
#{}))
Step 2: Add the signal collection to collect-signals
Add this block inside collect-signals, after the currency mismatch check and before (persistent! signals):
;; Description overlap (Jaccard similarity of tokens)
(let [tokens-a (extract-description-tokens doc-a)
tokens-b (extract-description-tokens doc-b)
intersection (set/intersection tokens-a tokens-b)
union (set/union tokens-a tokens-b)]
(when (and (seq union) (> (/ (double (count intersection)) (count union)) 0.15))
(conj! signals {:signal :description-overlap
:value (str/join ", " (sort (take 5 intersection)))
:weight (:description-overlap evidence-signals)})))
Step 3: Run tests
Run: clj -X:test:silent :nses '[com.getorcha.workers.matching.evidence-test]'
Expected: ALL PASS
Step 4: Lint
Run: clj-kondo --lint src/com/getorcha/workers/matching/evidence.clj --fail-level warning
Expected: No warnings
Step 5: Commit
git add src/com/getorcha/workers/matching/evidence.clj
git commit -m "feat(matching): add description-overlap evidence signal"
date-within-period — TestsFiles:
test/com/getorcha/workers/matching/evidence_test.cljStep 1: Update existing date-within-period-signal-test and add new cases
Replace the existing date-within-period-signal-test with:
(deftest date-within-period-signal-test
(testing "fires when invoice service period falls within contract dates"
(let [{:keys [evidence]} (evidence/compute-score
#:document{:type "invoice"
:structured-data {:service-period {:start "2025-10-01"
:end "2025-10-31"}}}
#:document{:type "contract"
:structured-data {:effective-date "2025-01-01"
:expiration-date "2025-12-31"}})]
(is (some #(= :date-within-period (:signal %)) evidence))))
(testing "does not fire when service period extends beyond contract"
(let [{:keys [evidence]} (evidence/compute-score
#:document{:type "invoice"
:structured-data {:service-period {:start "2025-10-01"
:end "2026-03-31"}}}
#:document{:type "contract"
:structured-data {:effective-date "2025-01-01"
:expiration-date "2025-12-31"}})]
(is (not (some #(= :date-within-period (:signal %)) evidence)))))
(testing "fires for open-ended contract when invoice date is after effective date"
(let [{:keys [evidence]} (evidence/compute-score
#:document{:type "invoice"
:structured-data {:invoice-date "2025-12-17"}}
#:document{:type "contract"
:structured-data {:effective-date "2024-01-19"}})]
(is (some #(= :date-within-period (:signal %)) evidence))))
(testing "does not fire for open-ended contract when invoice date is before effective date"
(let [{:keys [evidence]} (evidence/compute-score
#:document{:type "invoice"
:structured-data {:invoice-date "2023-06-01"}}
#:document{:type "contract"
:structured-data {:effective-date "2024-01-19"}})]
(is (not (some #(= :date-within-period (:signal %)) evidence)))))
(testing "fires when invoice date (no service period) falls within contract dates"
(let [{:keys [evidence]} (evidence/compute-score
#:document{:type "invoice"
:structured-data {:invoice-date "2025-06-15"}}
#:document{:type "contract"
:structured-data {:effective-date "2025-01-01"
:expiration-date "2025-12-31"}})]
(is (some #(= :date-within-period (:signal %)) evidence))))
(testing "does not fire when invoice date is outside contract period"
(let [{:keys [evidence]} (evidence/compute-score
#:document{:type "invoice"
:structured-data {:invoice-date "2026-06-15"}}
#:document{:type "contract"
:structured-data {:effective-date "2025-01-01"
:expiration-date "2025-12-31"}})]
(is (not (some #(= :date-within-period (:signal %)) evidence)))))
(testing "does not fire when no dates available at all"
(let [{:keys [evidence]} (evidence/compute-score
#:document{:type "invoice"
:structured-data {}}
#:document{:type "contract"
:structured-data {}})]
(is (not (some #(= :date-within-period (:signal %)) evidence))))))
Step 2: Run to verify failures
Run: clj -X:test:silent :nses '[com.getorcha.workers.matching.evidence-test]'
Expected: New test cases FAIL (open-ended contract, invoice-date fallback)
Step 3: Commit
git add test/com/getorcha/workers/matching/evidence_test.clj
git commit -m "test: add relaxed date-within-period test cases"
date-within-period — ImplementationFiles:
src/com/getorcha/workers/matching/evidence.cljStep 1: Add invoice date extraction helper
Add this function near the existing get-service-period:
(defn ^:private get-invoice-date
"Extract invoice date as a single ISO date string."
[{:document/keys [type structured-data]}]
(when (= type "invoice")
(:invoice-date structured-data)))
Step 2: Replace the date-within-period block in collect-signals
Replace the existing date-within-period logic (the let block with get-service-period and get-contract-dates) with this cascading check:
;; Date within period (cascading: service-period > invoice-date, handles open-ended contracts)
(let [[svc-start svc-end] (or (get-service-period doc-a) (get-service-period doc-b))
[contract-start contract-end] (or (get-contract-dates doc-a) (get-contract-dates doc-b))
inv-date (or (get-invoice-date doc-a) (get-invoice-date doc-b))]
(cond
;; Case 1: Full service period + full contract dates
(and svc-start svc-end contract-start contract-end)
(when (and (>= (compare svc-start contract-start) 0)
(<= (compare svc-end contract-end) 0))
(conj! signals {:signal :date-within-period
:value (str svc-start " to " svc-end " within " contract-start " to " contract-end)
:weight (:date-within-period evidence-signals)}))
;; Case 2: Full service period + open-ended contract
(and svc-start svc-end contract-start (nil? contract-end))
(when (>= (compare svc-start contract-start) 0)
(conj! signals {:signal :date-within-period
:value (str svc-start " to " svc-end " after " contract-start)
:weight (:date-within-period evidence-signals)}))
;; Case 3: Invoice date only + contract dates (with or without expiration)
(and inv-date contract-start)
(when (and (>= (compare inv-date contract-start) 0)
(or (nil? contract-end)
(<= (compare inv-date contract-end) 0)))
(conj! signals {:signal :date-within-period
:value (str inv-date " within " contract-start " to " (or contract-end "open-ended"))
:weight (:date-within-period evidence-signals)}))))
Step 3: Also update get-contract-dates to not require expiration
Change get-contract-dates to return the pair even when expiration is nil:
(defn ^:private get-contract-dates
"Extract effective and expiration dates from contract.
Returns `[effective expiration]` where expiration may be nil for open-ended contracts."
[{:document/keys [type structured-data]}]
(when (= type "contract")
(when-let [effective (:effective-date structured-data)]
[effective (:expiration-date structured-data)])))
Step 4: Run tests
Run: clj -X:test:silent :nses '[com.getorcha.workers.matching.evidence-test]'
Expected: ALL PASS
Step 5: Lint
Run: clj-kondo --lint src/com/getorcha/workers/matching/evidence.clj --fail-level warning
Expected: No warnings
Step 6: Commit
git add src/com/getorcha/workers/matching/evidence.clj
git commit -m "feat(matching): relax date-within-period for open-ended contracts and invoice-date fallback"
Step 1: Run all matching tests
Run: clj -X:test:silent :nses '[com.getorcha.workers.matching.evidence-test com.getorcha.workers.matching.llm-decision-test com.getorcha.workers.matching.core-test com.getorcha.workers.matching.worker-test]' 2>&1 | grep -A 5 -E "(FAIL in|ERROR in|Ran .* tests)"
Expected: ALL PASS, 0 failures
Step 2: Lint all matching source files
Run: clj-kondo --lint src/com/getorcha/workers/matching/ --fail-level warning
Expected: No warnings
Step 3: Verify no untracked files or unstaged changes
Run: git status
Expected: Clean working tree