feat(inbox): match non-invoice documents via prominent amounts (#2048)
* feat(inbox): match non-invoice documents via prominent amounts Bankintyg, bank agreements and other documentKind "other" PDFs carry no invoice-style total, so extraction correctly left totals.total null and the document became structurally unmatchable: findUnderlagCandidates hard-drops items without a comparable amount and the picker lost the 40% amount signal. - extraction: new prominentAmounts[] field (amount + document's own label), populated only when totals.total is null; account/org/phone/reference numbers and zero amounts excluded. totals.total semantics untouched. - matching: bestProminentAmountVariance() tries each printed amount and feeds calculateMatchConfidence at reduced weight (0.3 vs 0.4) in both the agent candidate scorer and TransactionMatchPicker. - UI: inbox rail shows the detected amounts read-only for such documents, list falls back to a single distinct prominent amount, and extraction no longer reads as "found nothing". Co-Authored-By: Claude Fable 5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01Hqm9QgdyNAFaWiz6Ww7pgb * fix(inbox): discount prominent-amount fallback instead of reweighting it Skeptic pass refutations on the first commit: normalized weighting made a reduced amount weight self-defeating. Date + exact fallback amount with no merchant scored (0.25+0.3)/0.55 = 1.0 ("100% sakerhet" on a wrong same-day transaction), and a DISAGREEING fallback amount scored above a disagreeing invoice total (0.67 vs 0.60) because shrinking the weight also shrank the penalty. - score fallbacks at full amount weight, then multiply by a flat FALLBACK_CONFIDENCE_FACTOR (0.85): agreement caps below certainty, disagreement stays at least as damning as for a real total. - agent candidate surface additionally requires the document date within DATE_TOLERANCE_DAYS, so an avtal listing 349 kr no longer matches every future 349 kr charge from the same counterparty. - bestProminentAmountVariance returns which amount matched + its document label, and the match reason names it ("Exakt belopp i dokumentet: 2 500 SEK (Engangspris)"): no more bare "Exakt belopp" reaching the agent while total_amount is null. - prompt: prominentAmounts restricted to non-invoice documentKinds, and never a parking spot for an unreadable invoice total. - fix the stale "deliberately the same list" comment on EXTRACTED_FIELD_ACCESSORS. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01Hqm9QgdyNAFaWiz6Ww7pgb * fix(receipt-hunt): never propose on the prominent-amounts fallback Second skeptic pass: the nightly hunt is a third consumer of scoreUnderlagCandidates and inherited the fallback unaware. A bankintyg whose printed "Insatt belopp" equals a same-day outflow scores 0.85, which clears CERTAIN_CONFIDENCE (0.8) and skips LLM adjudication, on a pairing wrong by construction (the hunt scans outflows only; "Insatt belopp" labels an inflow), with document_amount null in the approval preview. UnderlagCandidate now carries amountSource ('total' | 'prominent') and selectProposals drops fallback-scored candidates. Non-invoice documents stay reachable through the manual picker and the agent candidate surface, both of which have a human reading the amounts. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01Hqm9QgdyNAFaWiz6Ww7pgb * fix(inbox): round fallback confidence via roundOre, not the naive pattern The two confidence discounts (and their test) tripped the naive-ore-round antipattern ratchet (625 vs baseline 622); use the sanctioned helper. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01Hqm9QgdyNAFaWiz6Ww7pgb --------- Co-authored-by: Claude Fable 5 <noreply@anthropic.com>
This commit is contained in:
co-authored by
Claude Fable 5
parent
84ba8323b4
commit
516e8b62ff
@@ -1,12 +1,15 @@
|
||||
import { describe, it, expect } from 'vitest'
|
||||
import {
|
||||
FALLBACK_CONFIDENCE_FACTOR,
|
||||
levenshteinDistance,
|
||||
normalizeMerchantName,
|
||||
normalizeForMatch,
|
||||
calculateMerchantSimilarity,
|
||||
calculateMatchConfidence,
|
||||
amountVarianceForMatch,
|
||||
bestProminentAmountVariance,
|
||||
} from '../core-receipt-matcher'
|
||||
import { roundOre } from '@/lib/money'
|
||||
|
||||
describe('levenshteinDistance', () => {
|
||||
it('returns 0 for identical strings', () => {
|
||||
@@ -163,6 +166,56 @@ describe('amountVarianceForMatch', () => {
|
||||
})
|
||||
})
|
||||
|
||||
describe('bestProminentAmountVariance', () => {
|
||||
it('picks the closest of several printed amounts and names it', () => {
|
||||
// An agreement listing both a monthly price and a one-off price: the
|
||||
// one-off 2500 matches the -2500 AVGIFT charge exactly.
|
||||
const best = bestProminentAmountVariance(
|
||||
[
|
||||
{ amount: 49, label: 'Månadspris' },
|
||||
{ amount: 2500, label: 'Engångspris' },
|
||||
],
|
||||
'SEK',
|
||||
-2500,
|
||||
'SEK',
|
||||
-2500,
|
||||
)
|
||||
expect(best).toEqual({ variance: 0, amount: 2500, label: 'Engångspris' })
|
||||
})
|
||||
|
||||
it('returns null when nothing is comparable', () => {
|
||||
expect(bestProminentAmountVariance([], 'SEK', -2500, 'SEK', -2500)).toBeNull()
|
||||
// Cross-currency without a rate stays incomparable, like a total would.
|
||||
expect(
|
||||
bestProminentAmountVariance([{ amount: 2500, label: null }], 'EUR', -2500, 'SEK', -2500),
|
||||
).toBeNull()
|
||||
// Zero amounts carry no signal (amountVarianceForMatch drops them).
|
||||
expect(
|
||||
bestProminentAmountVariance([{ amount: 0, label: null }], 'SEK', -2500, 'SEK', -2500),
|
||||
).toBeNull()
|
||||
})
|
||||
|
||||
it('the discount factor keeps fallback agreement below certainty', () => {
|
||||
// Exact date + exact amount + no merchant normalises to 1.0; a fallback
|
||||
// match must not present that as certainty (this exact geometry scored
|
||||
// "100% säkerhet" on a wrong same-day transaction before the factor).
|
||||
const { confidence } = calculateMatchConfidence(0, 0, 0)
|
||||
const discounted = roundOre(confidence * FALLBACK_CONFIDENCE_FACTOR)
|
||||
expect(confidence).toBe(1)
|
||||
expect(discounted).toBeLessThan(1)
|
||||
})
|
||||
|
||||
it('a disagreeing fallback amount scores no better than a disagreeing total', () => {
|
||||
// Renormalized weights made a wrong fallback amount OUTSCORE a wrong
|
||||
// invoice total (0.67 vs 0.60 with exact date + merchant); the factor
|
||||
// approach scores both at full weight and then discounts the fallback.
|
||||
const asTotal = calculateMatchConfidence(0, 1.4, 0.9).confidence
|
||||
const asFallback = roundOre(asTotal * FALLBACK_CONFIDENCE_FACTOR)
|
||||
expect(asFallback).toBeLessThanOrEqual(asTotal)
|
||||
expect(asFallback).toBeLessThan(0.6)
|
||||
})
|
||||
})
|
||||
|
||||
describe('normalizeForMatch', () => {
|
||||
it('leaves the frozen key normalizer alone', () => {
|
||||
// normalizeMerchantName feeds a PERSISTED unique key with a SQL mirror.
|
||||
|
||||
Reference in New Issue
Block a user