feat(inbox): match non-invoice documents via prominent amounts (#2048)
* feat(inbox): match non-invoice documents via prominent amounts Bankintyg, bank agreements and other documentKind "other" PDFs carry no invoice-style total, so extraction correctly left totals.total null and the document became structurally unmatchable: findUnderlagCandidates hard-drops items without a comparable amount and the picker lost the 40% amount signal. - extraction: new prominentAmounts[] field (amount + document's own label), populated only when totals.total is null; account/org/phone/reference numbers and zero amounts excluded. totals.total semantics untouched. - matching: bestProminentAmountVariance() tries each printed amount and feeds calculateMatchConfidence at reduced weight (0.3 vs 0.4) in both the agent candidate scorer and TransactionMatchPicker. - UI: inbox rail shows the detected amounts read-only for such documents, list falls back to a single distinct prominent amount, and extraction no longer reads as "found nothing". Co-Authored-By: Claude Fable 5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01Hqm9QgdyNAFaWiz6Ww7pgb * fix(inbox): discount prominent-amount fallback instead of reweighting it Skeptic pass refutations on the first commit: normalized weighting made a reduced amount weight self-defeating. Date + exact fallback amount with no merchant scored (0.25+0.3)/0.55 = 1.0 ("100% sakerhet" on a wrong same-day transaction), and a DISAGREEING fallback amount scored above a disagreeing invoice total (0.67 vs 0.60) because shrinking the weight also shrank the penalty. - score fallbacks at full amount weight, then multiply by a flat FALLBACK_CONFIDENCE_FACTOR (0.85): agreement caps below certainty, disagreement stays at least as damning as for a real total. - agent candidate surface additionally requires the document date within DATE_TOLERANCE_DAYS, so an avtal listing 349 kr no longer matches every future 349 kr charge from the same counterparty. - bestProminentAmountVariance returns which amount matched + its document label, and the match reason names it ("Exakt belopp i dokumentet: 2 500 SEK (Engangspris)"): no more bare "Exakt belopp" reaching the agent while total_amount is null. - prompt: prominentAmounts restricted to non-invoice documentKinds, and never a parking spot for an unreadable invoice total. - fix the stale "deliberately the same list" comment on EXTRACTED_FIELD_ACCESSORS. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01Hqm9QgdyNAFaWiz6Ww7pgb * fix(receipt-hunt): never propose on the prominent-amounts fallback Second skeptic pass: the nightly hunt is a third consumer of scoreUnderlagCandidates and inherited the fallback unaware. A bankintyg whose printed "Insatt belopp" equals a same-day outflow scores 0.85, which clears CERTAIN_CONFIDENCE (0.8) and skips LLM adjudication, on a pairing wrong by construction (the hunt scans outflows only; "Insatt belopp" labels an inflow), with document_amount null in the approval preview. UnderlagCandidate now carries amountSource ('total' | 'prominent') and selectProposals drops fallback-scored candidates. Non-invoice documents stay reachable through the manual picker and the agent candidate surface, both of which have a human reading the amounts. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01Hqm9QgdyNAFaWiz6Ww7pgb * fix(inbox): round fallback confidence via roundOre, not the naive pattern The two confidence discounts (and their test) tripped the naive-ore-round antipattern ratchet (625 vs baseline 622); use the sanctioned helper. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01Hqm9QgdyNAFaWiz6Ww7pgb --------- Co-authored-by: Claude Fable 5 <noreply@anthropic.com>
This commit is contained in:
Mattsson
co-authored by
Claude Fable 5
Claude Fable 5
parent
84ba8323b4
commit
516e8b62ff
@@ -122,6 +122,55 @@ describe('extractInvoiceFields', () => {
|
||||
expect(data.confidence).toBe(1)
|
||||
})
|
||||
|
||||
it('keeps prominentAmounts from a non-invoice document (bankintyg/avtal)', async () => {
|
||||
mockCreate.mockReturnValueOnce(
|
||||
aiResponse({
|
||||
...VALID_RESULT,
|
||||
documentKind: 'other',
|
||||
supplier: { ...VALID_RESULT.supplier, name: 'SEB' },
|
||||
invoice: { ...VALID_RESULT.invoice, invoiceNumber: null, currency: 'SEK' },
|
||||
lineItems: [],
|
||||
totals: { subtotal: null, vatAmount: null, total: null },
|
||||
vatBreakdown: [],
|
||||
prominentAmounts: [{ amount: 2500, label: 'Anslutnings-/Engångspris' }],
|
||||
})
|
||||
)
|
||||
const { data } = await extractInvoiceFields({
|
||||
buffer: Buffer.from('%PDF'),
|
||||
mimeType: 'application/pdf',
|
||||
fileName: 'affarsavtal.pdf',
|
||||
})
|
||||
expect(data.totals.total).toBeNull()
|
||||
expect(data.prominentAmounts).toEqual([
|
||||
{ amount: 2500, label: 'Anslutnings-/Engångspris' },
|
||||
])
|
||||
})
|
||||
|
||||
it('degrades a hallucinated prominentAmounts shape to an empty list, not a parse failure', async () => {
|
||||
mockCreate.mockReturnValueOnce(
|
||||
aiResponse({ ...VALID_RESULT, prominentAmounts: [{ amount: 'tjugofemtusen' }] })
|
||||
)
|
||||
const { data } = await extractInvoiceFields({
|
||||
buffer: Buffer.from('%PDF'),
|
||||
mimeType: 'application/pdf',
|
||||
fileName: 'f.pdf',
|
||||
})
|
||||
// The rest of the document still parses.
|
||||
expect(data.totals.total).toBe(6.25)
|
||||
expect(data.prominentAmounts).toEqual([])
|
||||
})
|
||||
|
||||
it('validates a cached raw output from before prominentAmounts existed', async () => {
|
||||
mockCreate.mockReturnValueOnce(aiResponse(VALID_RESULT))
|
||||
const { data } = await extractInvoiceFields({
|
||||
buffer: Buffer.from('%PDF'),
|
||||
mimeType: 'application/pdf',
|
||||
fileName: 'f.pdf',
|
||||
})
|
||||
expect(data.totals.total).toBe(6.25)
|
||||
expect(data.prominentAmounts).toBeUndefined()
|
||||
})
|
||||
|
||||
it('sends image content for an image upload', async () => {
|
||||
mockCreate.mockReturnValueOnce(aiResponse(VALID_RESULT))
|
||||
await extractInvoiceFields({
|
||||
|
||||
@@ -159,6 +159,20 @@ export const ExtractionSchema = z.object({
|
||||
amount: z.number(),
|
||||
})
|
||||
),
|
||||
// Amounts visible on non-invoice documents (bankintyg, avtal, contracts)
|
||||
// that carry no invoice-style total. Matching hint only; never booked.
|
||||
// .catch([]) so a hallucinated shape degrades to "no amounts" instead of
|
||||
// failing the whole document parse; optional so cached raw outputs from
|
||||
// before the field existed still validate.
|
||||
prominentAmounts: z
|
||||
.array(
|
||||
z.object({
|
||||
amount: z.number(),
|
||||
label: z.string().nullable(),
|
||||
})
|
||||
)
|
||||
.catch([])
|
||||
.optional(),
|
||||
})
|
||||
|
||||
// Agent-supplied extraction: accountSuggestion is preserved instead of forced
|
||||
@@ -222,6 +236,9 @@ Return ONLY a single JSON object that matches this schema exactly. No prose, no
|
||||
},
|
||||
"vatBreakdown": [
|
||||
{ "rate": number, "base": number, "amount": number } // rate as percent integer, e.g. 25 for 25%
|
||||
],
|
||||
"prominentAmounts": [
|
||||
{ "amount": number, "label": string | null } // non-invoice documents only, see rules
|
||||
]
|
||||
}
|
||||
|
||||
@@ -245,7 +262,8 @@ Rules:
|
||||
- Numbers: parse with the document's locale (Swedish "1 234,56" = 1234.56; English "$1,234.56" = 1234.56). Output as plain JSON numbers.
|
||||
- If a field is missing or unreadable, set it to null. Never invent values.
|
||||
- lineItems: include every line. Empty array is fine if the document has no itemised lines.
|
||||
- vatBreakdown: include one entry per distinct VAT rate. Empty array is fine.`
|
||||
- vatBreakdown: include one entry per distinct VAT rate. Empty array is fine.
|
||||
- prominentAmounts: ONLY for documents that are NOT invoices or receipts (documentKind "other" or "government_letter") AND where totals.total is null, when the document still displays clear monetary amounts (a price, fee, deposit or paid-in sum: "Engångspris", "Anslutningspris", "Insatt belopp", "Månadspris", "Pris", "Belopp"). Typical sources: bankintyg, bank/account agreements, contracts, statements. One entry per distinct amount, label = the document's own label for it. NEVER include account numbers, org numbers, phone numbers, OCR/reference numbers, dates, percentages, or zero amounts. ALWAYS an empty array for invoices and receipts, including ones whose total is unreadable: never move an invoice total here.`
|
||||
|
||||
export function emptyResult(): InvoiceExtractionResult {
|
||||
return {
|
||||
@@ -274,6 +292,7 @@ export function emptyResult(): InvoiceExtractionResult {
|
||||
lineItems: [],
|
||||
totals: { subtotal: null, vatAmount: null, total: null, roundingAmount: null },
|
||||
vatBreakdown: [],
|
||||
prominentAmounts: [],
|
||||
confidence: 0,
|
||||
}
|
||||
}
|
||||
@@ -484,6 +503,15 @@ const EXTRACTION_JSON_SCHEMA: Record<string, unknown> = {
|
||||
required: ['rate', 'base', 'amount'],
|
||||
},
|
||||
},
|
||||
prominentAmounts: {
|
||||
type: 'array',
|
||||
items: {
|
||||
type: 'object',
|
||||
additionalProperties: false,
|
||||
properties: { amount: { type: 'number' }, label: nullable('string') },
|
||||
required: ['amount', 'label'],
|
||||
},
|
||||
},
|
||||
},
|
||||
required: [
|
||||
'documentKind',
|
||||
@@ -496,6 +524,7 @@ const EXTRACTION_JSON_SCHEMA: Record<string, unknown> = {
|
||||
'lineItems',
|
||||
'totals',
|
||||
'vatBreakdown',
|
||||
'prominentAmounts',
|
||||
],
|
||||
}
|
||||
|
||||
|
||||
Reference in New Issue
Block a user