diff --git a/DECISIONS.md b/DECISIONS.md index 0f8e10e2..a72c8ad4 100644 --- a/DECISIONS.md +++ b/DECISIONS.md @@ -1364,6 +1364,9 @@ One line per decision: `[YYYY-MM-DD] : `. Appended by agents and [2026-08-30] book_skattekonto_row(s) tier 'medium' + scope 'transactions:write': rule-driven booking with no caller-supplied lines mirrors book_mileage_period (not create_voucher's 'high'); scope follows reconcile_residual (books an outside row). Commit service gates on SKATTEVERKET_ENABLED for HTTP-dispatcher parity, recoverable so the op stays pending. [2026-08-30] Reminder text overrides (company_settings.reminder_text_overrides, level_1..3 x subject/body): the defaults are expressed as placeholder patterns (REMINDER_EMAIL_DEFAULT_TEXTS) and BOTH the stock mail and overrides render through the same substitution pipeline (applyPlaceholders + escape per output variant), so the settings-UI prefill is byte-for-byte the mail that goes out and cannot drift; this differs from the invoice_email_texts precedent, whose hand-written pattern forms can drift from the coded defaults. The level-3 default body is now an explicit inkassovarning (8 days, fordran till inkasso, costs per lag (1981:739)) but the level TITLE stays 'Slutlig paminnelse': the title is reused as the level name in settings labels and subject prefix, and renaming it everywhere is wording churn beyond the ask. An overridden subject owns the whole line (no automatic ' (inkl. drojsmalsranta)' suffix; {belopp} already includes surcharges), the stock subject keeps the suffix byte-identically. No pg test for the migration: a declarative CHECK (jsonb_typeof object) identical in shape to invoice_email_texts (20260703091000), which also shipped without one. The v1 REST/MCP update_company_settings surface was NOT extended: it is a curated field set with staged operations and its own placeholder refinement, a separate parity slice. typecheck/antipattern baselines deliberately not ratcheted in this diff: both one-count drops predate the branch (main drift), gates only fail on increase. [2026-08-30] PR #2021 round 2 (#546): the relayed Peppol buyer restriction now says the customer's org number must not be a personnummer (prepareParty('buyer') in lib/invoices/peppol-bis-billing.ts refuses it with BUYER_PARTICIPANT_IDENTIFIER_UNSUPPORTED, so an enskild firma CUSTOMER is refused, not only an enskild firma sender), Step 4 of the invoicing-rules workflow points at the Peppol section so a top-down reader never reaches the external-provider fallback first, the mark-sent recovery is scoped to the still-draft invoice in every text (INVOICE_MARK_SENT_REPAIR_REQUIRED leaves the invoice sent with the verifikat posted and a second mark-sent returns 409; the reviewer's proposed repair tool gnubok_link_invoice_to_voucher is the PAYMENT link and requires status sent/overdue/partially_paid, so no tool is named and the repair is left to support), and the verifikat parenthetical says "under faktureringsmetoden" (kontantmetod and defer_invoice_booking companies get none at issue). The guard test now also pins the two v1 route descriptions by reading the route source (apiskill:check only detects generated-vs-source drift, not a truth regression). The atom bump was seeded as a THIRD append-only migration (20260830101500, atom v9) rather than consolidating to one: the Supabase preview branch for the PR (xxnqggttsefleehmarjo) has applied both 20260829000100 and 20260829010000 per its schema_migrations, so deleting either would leave a remote with versions absent from the repo, the orphan class the migration rule forbids; all three seeds are idempotent upserts with the version guard, so prod applying them in sequence ends at v9. The generator's max-plus-one name (20260829010001) was renamed to 20260830101500 for the same reason as round 1 (newer than every file on origin/main and every sibling worktree; skills:check hashes content, the pg replay test globs the seed). +[2026-08-30] Non-invoice amount fallback uses a new prominentAmounts extraction field at reduced match weight (0.3 vs 0.4), not a relaxed totals.total: "total = what the buyer pays" keeps invoice/receipt booking paths unregressed, and one-of-several printed figures agreeing is weaker evidence than a total agreeing. Candidate floor (0.6) deliberately unchanged, so a dateless bankintyg still only surfaces via the manual picker. +[2026-08-30] Skeptic pass on PR #2048 replaced the fallback's reduced amount WEIGHT (0.3) with a flat confidence FACTOR (x0.85): normalization made the reduced weight both let date+amount-only fallbacks reach 1.0 and score a DISAGREEING fallback above a disagreeing total (0.67 vs 0.60). Fallback candidates additionally require the document date within DATE_TOLERANCE_DAYS on the agent surface, and the match reason names the matched figure + document label since total_amount stays null. [2026-08-30] Pre-migration inbox marker cutoff derived at query time from max posted source_type='import' entry_date (excl. series M) instead of a stored sie_imports coverage column: no migration/backfill needed and undo/replace self-corrects; series M excluded because the importer's omforingsverifikation is dated at fiscal year end. [2026-08-30] Skeptic round on PR #2047: cutoff armed by a completed sie_imports row (API clients can post source_type='import', which would false-mark never-migrated companies) and omforingsverifikation excluded by description prefix instead of voucher_series 'M' (imported vouchers keep the source file's series; 6 prod companies use series M for real vouchers). Residual known gap: skip-window covered only by the omforing stays unmarked; DuplicateBookingDialog is the backstop. [2026-08-30] Correction to the PR #2047 residual-gap note: the DuplicateBookingDialog backstop only covers a single skipped bank movement within 7 days of the omforing's fiscal-year-end date; mid-year and aggregated-skip variants are unmitigated and accepted on rarity (conjunction of skipped voucher + tail date + synced account + fetched history). Exact closure = persist skipped-voucher max date as coverage_end on sie_imports at import time (skippedDetails already has the dates); filed as follow-up. +[2026-08-30] Receipt-hunt excludes prominent-amounts fallback candidates (amountSource tag on UnderlagCandidate): the nightly hunt scans outflows only and its 0.8 skip-adjudication threshold was calibrated for invoice totals, so a fallback pair (0.85 on date+printed-figure, no merchant) would auto-classify certain on a wrong-by-construction pairing. Fallback docs stay reachable via picker + agent candidates. diff --git a/components/extensions/general/InvoiceInboxWorkspace.tsx b/components/extensions/general/InvoiceInboxWorkspace.tsx index 7a0c740e..22d66fe0 100644 --- a/components/extensions/general/InvoiceInboxWorkspace.tsx +++ b/components/extensions/general/InvoiceInboxWorkspace.tsx @@ -242,7 +242,19 @@ function timeAgo(iso: string): string { } function pickAmount(item: InboxItem): number | null { - return item.extracted_data?.totals?.total ?? null + const total = item.extracted_data?.totals?.total + if (total != null) return total + // Non-invoice documents (bankintyg, avtal) have no total; when exactly one + // distinct amount was read off the document, that is the amount to show. + // Two or more stay ambiguous and render as no amount. + const distinct = [ + ...new Set( + (item.extracted_data?.prominentAmounts ?? []) + .map((a) => a.amount) + .filter((a) => Number.isFinite(a) && a !== 0), + ), + ] + return distinct.length === 1 ? distinct[0] : null } function pickCurrency(item: InboxItem): string { @@ -271,14 +283,18 @@ function hasAnyExtractedField(data: InvoiceExtractionResult | null): boolean { s?.name || s?.orgNumber || s?.vatNumber || s?.bankgiro || s?.plusgiro || inv?.invoiceNumber || inv?.invoiceDate || inv?.dueDate || inv?.paymentReference || t?.subtotal != null || t?.vatAmount != null || t?.total != null || - (data.lineItems?.length ?? 0) > 0 || (data.vatBreakdown?.length ?? 0) > 0 + (data.lineItems?.length ?? 0) > 0 || (data.vatBreakdown?.length ?? 0) > 0 || + (data.prominentAmounts?.length ?? 0) > 0 ) } /** * The fields the extraction is scored against, in the order a person reads - * them. Deliberately the same list hasAnyExtractedField checks, so the summary - * cannot claim a field the "is anything here at all" test does not count. + * them. A subset of what hasAnyExtractedField checks (that test also counts + * lineItems, vatBreakdown and prominentAmounts), so the "fält ifyllda" counter + * never claims a field the "is anything here at all" test does not count: the + * reverse can differ, e.g. a bankintyg with only prominentAmounts has fields + * but counts 0 here. */ const EXTRACTED_FIELD_ACCESSORS: ((d: InvoiceExtractionResult) => unknown)[] = [ (d) => d.supplier?.name, @@ -2893,7 +2909,10 @@ function FieldsRail({ {/* AI classification: what kind of document this is and how it was paid. Read-only context above the editable fields; absent for extractions from before the fields existed. */} - {(data?.documentKind || data?.payment?.method || data?.pages) && ( + {(data?.documentKind || + data?.payment?.method || + data?.pages || + (data?.totals?.total == null && (data?.prominentAmounts?.length ?? 0) > 0)) && (
{data?.documentKind && (
@@ -2911,6 +2930,23 @@ function FieldsRail({
)} + {/* Amounts read off a non-invoice document (bankintyg, avtal): + without this row the empty "Totalt" field reads as if extraction + missed them. */} + {data?.totals?.total == null && (data?.prominentAmounts?.length ?? 0) > 0 && ( +
+ {t('prominent_amounts_label')} + + {(data?.prominentAmounts ?? []) + .map((a) => + a.label + ? `${a.label}: ${formatCurrency(a.amount, data?.invoice?.currency ?? 'SEK')}` + : formatCurrency(a.amount, data?.invoice?.currency ?? 'SEK'), + ) + .join(' · ')} + +
+ )} {data?.pages && (
{t('pages_partial_note', { diff --git a/components/inbox/TransactionMatchPicker.tsx b/components/inbox/TransactionMatchPicker.tsx index 4e72d696..2b089a9f 100644 --- a/components/inbox/TransactionMatchPicker.tsx +++ b/components/inbox/TransactionMatchPicker.tsx @@ -17,11 +17,14 @@ import { cn, formatCurrency, formatDate } from '@/lib/utils' import { Loader2, Search } from 'lucide-react' import { Input } from '@/components/ui/input' import { + FALLBACK_CONFIDENCE_FACTOR, amountVarianceForMatch, + bestProminentAmountVariance, calculateMatchConfidence, calculateMerchantSimilarity, } from '@/lib/documents/core-receipt-matcher' import { resolveSekAmount } from '@/lib/bookkeeping/currency-utils' +import { roundOre } from '@/lib/money' import type { InvoiceExtractionResult } from '@/types' import { getErrorMessage as getUserErrorMessage } from '@/lib/errors/get-error-message' @@ -132,6 +135,17 @@ export default function TransactionMatchPicker({ const total = extractedData?.totals?.total ?? null const receiptCurrency = (extractedData?.invoice?.currency ?? 'SEK').toUpperCase() const supplier = extractedData?.supplier?.name ?? null + // Non-invoice documents (bankintyg, avtal) have no total but often show the + // money amount anyway; those feed a discounted amount fallback below. + const prominentAmounts = useMemo( + () => + total == null + ? (extractedData?.prominentAmounts ?? []).filter( + (a) => Number.isFinite(a.amount) && a.amount !== 0, + ) + : [], + [extractedData, total], + ) // SEK value of the underlag total. For a SEK underlag that's the total // itself; for a foreign one it needs the fetched FX rate. @@ -254,7 +268,7 @@ export default function TransactionMatchPicker({ // Currency-aware variance: null when uncomparable, which makes the // matcher drop the amount signal instead of matching 750 EUR to 750 SEK. - const amountVariance = amountVarianceForMatch( + let amountVariance = amountVarianceForMatch( total, receiptCurrency, receiptSek, @@ -263,17 +277,36 @@ export default function TransactionMatchPicker({ txSek, ) + // No total (bankintyg, avtal): fall back to the closest prominent + // amount, discounted below so a printed figure never presents as the + // certainty a real total gives. + const fallbackMatch = + total == null && prominentAmounts.length > 0 + ? bestProminentAmountVariance( + prominentAmounts, + receiptCurrency, + tx.amount, + txCurrency, + txSek, + ) + : null + if (fallbackMatch) amountVariance = fallbackMatch.variance + const dateVariance = Math.abs( (new Date(tx.date).getTime() - invoiceDate.getTime()) / (1000 * 60 * 60 * 24), ) const merchant = tx.merchant_name || tx.description || '' const similarity = supplier ? calculateMerchantSimilarity(supplier, merchant) : 0 - const { confidence, matchReasons } = calculateMatchConfidence( + const scoredMatch = calculateMatchConfidence( dateVariance, amountVariance, similarity, MATCH_DATE_TOLERANCE_DAYS, ) + const matchReasons = scoredMatch.matchReasons + const confidence = fallbackMatch + ? roundOre(scoredMatch.confidence * FALLBACK_CONFIDENCE_FACTOR) + : scoredMatch.confidence return { id: tx.id, @@ -289,7 +322,7 @@ export default function TransactionMatchPicker({ }) scored.sort((a, b) => b.confidence - a.confidence) return scored - }, [rawRows, invoiceDate, total, receiptCurrency, receiptSek, supplier]) + }, [rawRows, invoiceDate, total, prominentAmounts, receiptCurrency, receiptSek, supplier]) // Instant client-side narrowing while the debounced server query catches up. const filtered = useMemo(() => { @@ -344,7 +377,7 @@ export default function TransactionMatchPicker({ {/* Underlag reference: what we're matching against, so a currency or amount mismatch with a candidate is obvious at a glance. */} - {(total != null || supplier) && ( + {(total != null || prominentAmounts.length > 0 || supplier) && (
Underlag {supplier && {supplier}} @@ -359,6 +392,11 @@ export default function TransactionMatchPicker({ )} )} + {total == null && prominentAmounts.length > 0 && ( + + {prominentAmounts.map((a) => formatCurrency(a.amount, receiptCurrency)).join(' · ')} + + )} {hasInvoiceDate && rawInvoiceDate && ( {formatDate(rawInvoiceDate)} diff --git a/extensions/general/invoice-inbox/__tests__/extract-invoice-fields.test.ts b/extensions/general/invoice-inbox/__tests__/extract-invoice-fields.test.ts index fe2f9e98..ea09720d 100644 --- a/extensions/general/invoice-inbox/__tests__/extract-invoice-fields.test.ts +++ b/extensions/general/invoice-inbox/__tests__/extract-invoice-fields.test.ts @@ -122,6 +122,55 @@ describe('extractInvoiceFields', () => { expect(data.confidence).toBe(1) }) + it('keeps prominentAmounts from a non-invoice document (bankintyg/avtal)', async () => { + mockCreate.mockReturnValueOnce( + aiResponse({ + ...VALID_RESULT, + documentKind: 'other', + supplier: { ...VALID_RESULT.supplier, name: 'SEB' }, + invoice: { ...VALID_RESULT.invoice, invoiceNumber: null, currency: 'SEK' }, + lineItems: [], + totals: { subtotal: null, vatAmount: null, total: null }, + vatBreakdown: [], + prominentAmounts: [{ amount: 2500, label: 'Anslutnings-/Engångspris' }], + }) + ) + const { data } = await extractInvoiceFields({ + buffer: Buffer.from('%PDF'), + mimeType: 'application/pdf', + fileName: 'affarsavtal.pdf', + }) + expect(data.totals.total).toBeNull() + expect(data.prominentAmounts).toEqual([ + { amount: 2500, label: 'Anslutnings-/Engångspris' }, + ]) + }) + + it('degrades a hallucinated prominentAmounts shape to an empty list, not a parse failure', async () => { + mockCreate.mockReturnValueOnce( + aiResponse({ ...VALID_RESULT, prominentAmounts: [{ amount: 'tjugofemtusen' }] }) + ) + const { data } = await extractInvoiceFields({ + buffer: Buffer.from('%PDF'), + mimeType: 'application/pdf', + fileName: 'f.pdf', + }) + // The rest of the document still parses. + expect(data.totals.total).toBe(6.25) + expect(data.prominentAmounts).toEqual([]) + }) + + it('validates a cached raw output from before prominentAmounts existed', async () => { + mockCreate.mockReturnValueOnce(aiResponse(VALID_RESULT)) + const { data } = await extractInvoiceFields({ + buffer: Buffer.from('%PDF'), + mimeType: 'application/pdf', + fileName: 'f.pdf', + }) + expect(data.totals.total).toBe(6.25) + expect(data.prominentAmounts).toBeUndefined() + }) + it('sends image content for an image upload', async () => { mockCreate.mockReturnValueOnce(aiResponse(VALID_RESULT)) await extractInvoiceFields({ diff --git a/extensions/general/invoice-inbox/lib/extract-invoice-fields.ts b/extensions/general/invoice-inbox/lib/extract-invoice-fields.ts index eba7532d..8061b0ff 100644 --- a/extensions/general/invoice-inbox/lib/extract-invoice-fields.ts +++ b/extensions/general/invoice-inbox/lib/extract-invoice-fields.ts @@ -159,6 +159,20 @@ export const ExtractionSchema = z.object({ amount: z.number(), }) ), + // Amounts visible on non-invoice documents (bankintyg, avtal, contracts) + // that carry no invoice-style total. Matching hint only; never booked. + // .catch([]) so a hallucinated shape degrades to "no amounts" instead of + // failing the whole document parse; optional so cached raw outputs from + // before the field existed still validate. + prominentAmounts: z + .array( + z.object({ + amount: z.number(), + label: z.string().nullable(), + }) + ) + .catch([]) + .optional(), }) // Agent-supplied extraction: accountSuggestion is preserved instead of forced @@ -222,6 +236,9 @@ Return ONLY a single JSON object that matches this schema exactly. No prose, no }, "vatBreakdown": [ { "rate": number, "base": number, "amount": number } // rate as percent integer, e.g. 25 for 25% + ], + "prominentAmounts": [ + { "amount": number, "label": string | null } // non-invoice documents only, see rules ] } @@ -245,7 +262,8 @@ Rules: - Numbers: parse with the document's locale (Swedish "1 234,56" = 1234.56; English "$1,234.56" = 1234.56). Output as plain JSON numbers. - If a field is missing or unreadable, set it to null. Never invent values. - lineItems: include every line. Empty array is fine if the document has no itemised lines. -- vatBreakdown: include one entry per distinct VAT rate. Empty array is fine.` +- vatBreakdown: include one entry per distinct VAT rate. Empty array is fine. +- prominentAmounts: ONLY for documents that are NOT invoices or receipts (documentKind "other" or "government_letter") AND where totals.total is null, when the document still displays clear monetary amounts (a price, fee, deposit or paid-in sum: "Engångspris", "Anslutningspris", "Insatt belopp", "Månadspris", "Pris", "Belopp"). Typical sources: bankintyg, bank/account agreements, contracts, statements. One entry per distinct amount, label = the document's own label for it. NEVER include account numbers, org numbers, phone numbers, OCR/reference numbers, dates, percentages, or zero amounts. ALWAYS an empty array for invoices and receipts, including ones whose total is unreadable: never move an invoice total here.` export function emptyResult(): InvoiceExtractionResult { return { @@ -274,6 +292,7 @@ export function emptyResult(): InvoiceExtractionResult { lineItems: [], totals: { subtotal: null, vatAmount: null, total: null, roundingAmount: null }, vatBreakdown: [], + prominentAmounts: [], confidence: 0, } } @@ -484,6 +503,15 @@ const EXTRACTION_JSON_SCHEMA: Record = { required: ['rate', 'base', 'amount'], }, }, + prominentAmounts: { + type: 'array', + items: { + type: 'object', + additionalProperties: false, + properties: { amount: { type: 'number' }, label: nullable('string') }, + required: ['amount', 'label'], + }, + }, }, required: [ 'documentKind', @@ -496,6 +524,7 @@ const EXTRACTION_JSON_SCHEMA: Record = { 'lineItems', 'totals', 'vatBreakdown', + 'prominentAmounts', ], } diff --git a/lib/agent-context/__tests__/underlag-candidates.test.ts b/lib/agent-context/__tests__/underlag-candidates.test.ts index 628efc5b..0a9e96ea 100644 --- a/lib/agent-context/__tests__/underlag-candidates.test.ts +++ b/lib/agent-context/__tests__/underlag-candidates.test.ts @@ -15,6 +15,7 @@ function extraction(partial: { total?: number | null vat?: number | null currency?: string + prominentAmounts?: { amount: number; label: string | null }[] }): InvoiceExtractionResult { return { supplier: { @@ -28,6 +29,7 @@ function extraction(partial: { lineItems: [], totals: { subtotal: null, vatAmount: partial.vat ?? null, total: partial.total ?? null }, vatBreakdown: [], + prominentAmounts: partial.prominentAmounts, confidence: 0.9, } as InvoiceExtractionResult } @@ -71,6 +73,7 @@ describe('scoreUnderlagCandidates', () => { expect(out).toHaveLength(1) expect(out[0].inbox_item_id).toBe('item-1') expect(out[0].confidence).toBeGreaterThanOrEqual(CANDIDATE_MIN_CONFIDENCE) + expect(out[0].amountSource).toBe('total') // and it brings the captured answers along with it expect(out[0].channelContext?.representation?.purpose).toBe('kundmöte') }) @@ -139,6 +142,110 @@ describe('scoreUnderlagCandidates', () => { expect(out[0].inbox_item_id).toBe('strong') }) + it('proposes a non-invoice document via its prominent amounts', () => { + // The Robotministeriet case: an SEB account agreement (documentKind + // "other") has no "Att betala" total, only "Anslutnings-/Engångspris + // 2 500". The bank charges AVGIFT -2500 the same day. Before the + // prominentAmounts fallback this document was structurally unmatchable. + const avgiftTx = { + ...tx, + description: 'AVGIFT', + merchant_name: null, + amount: -2500, + date: '2026-08-26', + } + const out = scoreUnderlagCandidates(avgiftTx, [ + { + id: 'item-avtal', + document_id: 'doc-avtal', + extracted_data: extraction({ + supplier: 'SEB', + date: '2026-08-26', + total: null, + prominentAmounts: [ + { amount: 2500, label: 'Anslutnings-/Engångspris' }, + ], + }), + channel_context: null, + }, + ]) + expect(out).toHaveLength(1) + expect(out[0].inbox_item_id).toBe('item-avtal') + expect(out[0].confidence).toBeGreaterThanOrEqual(CANDIDATE_MIN_CONFIDENCE) + // ...but never as certainty: a printed figure is not an invoice total. + expect(out[0].confidence).toBeLessThan(1) + // Tagged so low-scrutiny consumers (the nightly hunt) can exclude it. + expect(out[0].amountSource).toBe('prominent') + // The reason names WHICH figure matched, since total_amount stays null. + expect(out[0].total_amount).toBeNull() + // toLocaleString('sv-SE') groups with a non-breaking space (U+00A0). + expect(out[0].matchReasons.join(' ')).toContain(`2${' '}500`) + expect(out[0].matchReasons.join(' ')).toContain('Anslutnings-/Engångspris') + }) + + it('does not let a prominent amount alone carry a dateless document over the floor', () => { + // Amount agreement without a date is weaker than a total + date pair; the + // candidate surface trades recall for precision, so this stays in the + // manual picker only. + const out = scoreUnderlagCandidates({ ...tx, amount: -25000 }, [ + { + id: 'item-intyg', + document_id: 'doc-intyg', + extracted_data: extraction({ + supplier: 'SEB', + date: null, + total: null, + prominentAmounts: [{ amount: 25000, label: 'Insatt belopp' }], + }), + channel_context: null, + }, + ]) + expect(out).toEqual([]) + }) + + it('does not match an avtal to a later charge on amount + merchant alone', () => { + // A Telia avtal listing 349 kr must not surface for every future 349 kr + // Telia charge: the fallback requires the document date to agree within + // the normal tolerance. + const out = scoreUnderlagCandidates( + { ...tx, description: 'TELIA SVERIGE AB', merchant_name: 'Telia Sverige AB', amount: -349 }, + [ + { + id: 'item-telia', + document_id: 'doc-telia', + extracted_data: extraction({ + supplier: 'Telia Sverige AB', + date: '2026-01-15', + total: null, + prominentAmounts: [{ amount: 349, label: 'Månadspris' }], + }), + channel_context: null, + }, + ], + ) + expect(out).toEqual([]) + }) + + it('rejects a non-invoice document whose prominent amounts all disagree', () => { + // Same-day, same merchant, but the printed amounts match nothing: the + // discount keeps this under the floor, where a real disagreeing invoice + // total would sit exactly at it. + const out = scoreUnderlagCandidates(tx, [ + { + id: 'item-wrong', + document_id: 'doc-wrong', + extracted_data: extraction({ + supplier: 'Espresso House', + date: '2026-05-12', + total: null, + prominentAmounts: [{ amount: 9999, label: 'Pris' }], + }), + channel_context: null, + }, + ]) + expect(out).toEqual([]) + }) + it('returns nothing for a transaction with no date or amount', () => { expect( scoreUnderlagCandidates({ ...tx, date: null }, [ diff --git a/lib/agent-context/underlag-candidates.ts b/lib/agent-context/underlag-candidates.ts index a8a5d42b..4320964e 100644 --- a/lib/agent-context/underlag-candidates.ts +++ b/lib/agent-context/underlag-candidates.ts @@ -24,11 +24,15 @@ import type { SupabaseClient } from '@supabase/supabase-js' import { CONVERTED_AMOUNT_TOLERANCE_PERCENT, + DATE_TOLERANCE_DAYS, + FALLBACK_CONFIDENCE_FACTOR, amountVarianceForMatch, + bestProminentAmountVariance, calculateMatchConfidence, calculateMerchantSimilarity, } from '@/lib/documents/core-receipt-matcher' import { resolveSekAmount } from '@/lib/bookkeeping/currency-utils' +import { roundOre } from '@/lib/money' import type { InboxChannelContext, InvoiceExtractionResult } from '@/types' /** @@ -62,6 +66,13 @@ export interface UnderlagCandidate { currency: string | null /** 0-1 from the shared receipt matcher. */ confidence: number + /** + * Where the amount signal came from: an invoice-style total, or the + * prominent-amounts fallback for non-invoice documents (bankintyg, avtal). + * Consumers that act with less human scrutiny (the nightly receipt hunt) + * must treat 'prominent' as weaker evidence or exclude it. + */ + amountSource: 'total' | 'prominent' /** Swedish reasons the match scored, for display. */ matchReasons: string[] /** Answers already captured for this item, so they travel with it. */ @@ -103,6 +114,11 @@ function extractionSignals(extracted: InvoiceExtractionResult | null | undefined total: extracted?.totals?.total ?? null, vat: extracted?.totals?.vatAmount ?? null, currency: (extracted?.invoice?.currency || 'SEK').toUpperCase(), + // Non-invoice documents (bankintyg, avtal) carry no total but often show + // the money amount anyway; the extractor lists those here. + prominentAmounts: (extracted?.prominentAmounts ?? []).filter( + (a) => Number.isFinite(a.amount) && a.amount !== 0, + ), } } @@ -128,11 +144,11 @@ export function scoreUnderlagCandidates( for (const item of items) { const sig = extractionSignals(item.extracted_data) - // An extraction with neither a date nor a total carries no signal the + // An extraction with neither a date nor any amount carries no signal the // matcher can use; scoring it returns noise dressed as confidence. - if (!sig.date && sig.total == null) continue + if (!sig.date && sig.total == null && sig.prominentAmounts.length === 0) continue - const amountVariance = amountVarianceForMatch( + let amountVariance = amountVarianceForMatch( sig.total, sig.currency, // A SEK value only when someone resolved a rate for this receipt. @@ -144,6 +160,29 @@ export function scoreUnderlagCandidates( txSek, ) + const dateVariance = sig.date + ? Math.abs((new Date(sig.date).getTime() - txDateMs) / (1000 * 60 * 60 * 24)) + : Number.POSITIVE_INFINITY + + // A document with no invoice-style total (bankintyg, avtal: documentKind + // "other") but visible amounts falls back to the closest prominent + // amount. Two guards keep this precision-first: the document's date must + // agree within the normal tolerance (an avtal listing 349 kr must not + // match every future 349 kr charge from the same counterparty on amount + + // merchant alone), and the confidence is discounted below so a fallback + // can never present as certainty. + const fallbackMatch = + sig.total == null && amountVariance == null && dateVariance <= DATE_TOLERANCE_DAYS + ? bestProminentAmountVariance( + sig.prominentAmounts, + sig.currency, + tx.amount, + txCurrency, + txSek, + ) + : null + if (fallbackMatch) amountVariance = fallbackMatch.variance + // No comparable amount means no candidate. calculateMatchConfidence drops // the amount signal when it cannot normalise the currencies, which leaves // date + merchant carrying the whole normalised score: a same-day receipt @@ -154,21 +193,33 @@ export function scoreUnderlagCandidates( // through the picker; they are just not proposed. if (amountVariance == null) continue - const dateVariance = sig.date - ? Math.abs((new Date(sig.date).getTime() - txDateMs) / (1000 * 60 * 60 * 24)) - : Number.POSITIVE_INFINITY const similarity = sig.supplier ? calculateMerchantSimilarity(sig.supplier, txMerchant) : 0 // A converted total is judged against the wider bar, because the rate // spread is a known error rather than a disagreement about the sum. - const converted = sig.currency !== txCurrency && item.sek_total != null - const { confidence, matchReasons } = calculateMatchConfidence( + const scoredMatch = calculateMatchConfidence( dateVariance, amountVariance, similarity, undefined, - converted ? CONVERTED_AMOUNT_TOLERANCE_PERCENT : undefined, + sig.currency !== txCurrency && item.sek_total != null + ? CONVERTED_AMOUNT_TOLERANCE_PERCENT + : undefined, ) + let confidence = scoredMatch.confidence + let matchReasons = scoredMatch.matchReasons + if (fallbackMatch) { + confidence = roundOre(confidence * FALLBACK_CONFIDENCE_FACTOR) + // Name the figure that matched. A bare "Exakt belopp" would reach the + // agent while total_amount stays null: certainty without a number the + // agent or the user could check against the document. + const label = fallbackMatch.label ? ` (${fallbackMatch.label})` : '' + matchReasons = matchReasons.map((reason) => + reason.startsWith('Exakt belopp') || reason.startsWith('Belopp ±') + ? `${reason} i dokumentet: ${fallbackMatch.amount.toLocaleString('sv-SE')} ${sig.currency}${label}` + : reason, + ) + } if (confidence < CANDIDATE_MIN_CONFIDENCE) continue scored.push({ @@ -180,6 +231,7 @@ export function scoreUnderlagCandidates( vat_amount: sig.vat, currency: sig.currency, confidence, + amountSource: fallbackMatch ? 'prominent' : 'total', matchReasons, channelContext: item.channel_context ?? null, }) diff --git a/lib/documents/__tests__/core-receipt-matcher.test.ts b/lib/documents/__tests__/core-receipt-matcher.test.ts index 36ef0a2a..b02ca2be 100644 --- a/lib/documents/__tests__/core-receipt-matcher.test.ts +++ b/lib/documents/__tests__/core-receipt-matcher.test.ts @@ -1,12 +1,15 @@ import { describe, it, expect } from 'vitest' import { + FALLBACK_CONFIDENCE_FACTOR, levenshteinDistance, normalizeMerchantName, normalizeForMatch, calculateMerchantSimilarity, calculateMatchConfidence, amountVarianceForMatch, + bestProminentAmountVariance, } from '../core-receipt-matcher' +import { roundOre } from '@/lib/money' describe('levenshteinDistance', () => { it('returns 0 for identical strings', () => { @@ -163,6 +166,56 @@ describe('amountVarianceForMatch', () => { }) }) +describe('bestProminentAmountVariance', () => { + it('picks the closest of several printed amounts and names it', () => { + // An agreement listing both a monthly price and a one-off price: the + // one-off 2500 matches the -2500 AVGIFT charge exactly. + const best = bestProminentAmountVariance( + [ + { amount: 49, label: 'Månadspris' }, + { amount: 2500, label: 'Engångspris' }, + ], + 'SEK', + -2500, + 'SEK', + -2500, + ) + expect(best).toEqual({ variance: 0, amount: 2500, label: 'Engångspris' }) + }) + + it('returns null when nothing is comparable', () => { + expect(bestProminentAmountVariance([], 'SEK', -2500, 'SEK', -2500)).toBeNull() + // Cross-currency without a rate stays incomparable, like a total would. + expect( + bestProminentAmountVariance([{ amount: 2500, label: null }], 'EUR', -2500, 'SEK', -2500), + ).toBeNull() + // Zero amounts carry no signal (amountVarianceForMatch drops them). + expect( + bestProminentAmountVariance([{ amount: 0, label: null }], 'SEK', -2500, 'SEK', -2500), + ).toBeNull() + }) + + it('the discount factor keeps fallback agreement below certainty', () => { + // Exact date + exact amount + no merchant normalises to 1.0; a fallback + // match must not present that as certainty (this exact geometry scored + // "100% säkerhet" on a wrong same-day transaction before the factor). + const { confidence } = calculateMatchConfidence(0, 0, 0) + const discounted = roundOre(confidence * FALLBACK_CONFIDENCE_FACTOR) + expect(confidence).toBe(1) + expect(discounted).toBeLessThan(1) + }) + + it('a disagreeing fallback amount scores no better than a disagreeing total', () => { + // Renormalized weights made a wrong fallback amount OUTSCORE a wrong + // invoice total (0.67 vs 0.60 with exact date + merchant); the factor + // approach scores both at full weight and then discounts the fallback. + const asTotal = calculateMatchConfidence(0, 1.4, 0.9).confidence + const asFallback = roundOre(asTotal * FALLBACK_CONFIDENCE_FACTOR) + expect(asFallback).toBeLessThanOrEqual(asTotal) + expect(asFallback).toBeLessThan(0.6) + }) +}) + describe('normalizeForMatch', () => { it('leaves the frozen key normalizer alone', () => { // normalizeMerchantName feeds a PERSISTED unique key with a SQL mirror. diff --git a/lib/documents/core-receipt-matcher.ts b/lib/documents/core-receipt-matcher.ts index 42c3a675..030a020d 100644 --- a/lib/documents/core-receipt-matcher.ts +++ b/lib/documents/core-receipt-matcher.ts @@ -37,6 +37,24 @@ export const AMOUNT_TOLERANCE_PERCENT = 0.05 export const CONVERTED_AMOUNT_TOLERANCE_PERCENT = 0.09 export const MIN_MATCH_CONFIDENCE = 0.4 +/** + * Flat discount on a confidence scored from a prominentAmounts fallback + * (bankintyg, avtal, contracts: no invoice-style total). Such an amount is one + * of possibly several figures printed on the document rather than "what the + * buyer pays", so an agreement is real evidence but must stay weaker than a + * total agreeing. + * + * A discount FACTOR, deliberately not a reduced amount weight inside + * calculateMatchConfidence: the confidence is normalised over the included + * weights, so shrinking the amount weight both let a date+amount-only fallback + * reach 1.0 ((0.25+0.3)/0.55) and, when the amount DISAGREED, shrank the + * penalty so a wrong fallback amount outscored a wrong invoice total + * (0.67 vs 0.60). Scoring at full weight and discounting the result keeps + * agreement capped below certainty and disagreement at least as damning as it + * is for a real total. + */ +export const FALLBACK_CONFIDENCE_FACTOR = 0.85 + /** * Normalize a merchant name for comparison. * Removes special characters, Swedish company suffixes, and extra whitespace. @@ -263,6 +281,55 @@ export function amountVarianceForMatch( return null } +export interface ProminentAmountMatch { + variance: number + /** The printed amount that produced the variance. */ + amount: number + /** The document's own label for it ("Insatt belopp", "Engångspris"). */ + label: string | null +} + +/** + * Fallback amount variance for documents with no invoice-style total but one + * or more prominent amounts (bankintyg "Insatt belopp", an agreement's + * "Engångspris", ...). Tries each amount against the transaction and returns + * the smallest variance, or null when none is comparable. + * + * Same-currency only by construction: prominent amounts never carry a resolved + * SEK value, so a cross-currency pair stays incomparable (receiptSek = null in + * amountVarianceForMatch) exactly like a cross-currency total without a rate. + * Returns the closest amount with its variance and the document's own label. + * + * Callers must multiply the resulting confidence by + * FALLBACK_CONFIDENCE_FACTOR (see its comment for why a factor, not a weight), + * and should surface WHICH amount matched: a bare "Exakt belopp" with no + * number attached is certainty the reader cannot check. + */ +export function bestProminentAmountVariance( + amounts: readonly { amount: number; label: string | null }[], + receiptCurrency: string, + txAmount: number, + txCurrency: string, + txSek: number, +): ProminentAmountMatch | null { + let best: ProminentAmountMatch | null = null + for (const candidate of amounts) { + if (!Number.isFinite(candidate.amount)) continue + const variance = amountVarianceForMatch( + candidate.amount, + receiptCurrency, + null, + txAmount, + txCurrency, + txSek, + ) + if (variance != null && (best == null || variance < best.variance)) { + best = { variance, amount: candidate.amount, label: candidate.label } + } + } + return best +} + /** * Calculate a weighted match confidence score from date, amount, and merchant signals. * Weights: amount 40%, merchant 35%, date 25%. diff --git a/lib/receipt-hunt/__tests__/select.test.ts b/lib/receipt-hunt/__tests__/select.test.ts index 7c0e9eee..74813ed6 100644 --- a/lib/receipt-hunt/__tests__/select.test.ts +++ b/lib/receipt-hunt/__tests__/select.test.ts @@ -68,6 +68,25 @@ describe('selectProposals', () => { expect(selectProposals([tx()], [], noSuppression)).toEqual([]) }) + it('never proposes on the prominent-amounts fallback', () => { + // A bankintyg (documentKind "other", no invoice-style total) whose printed + // "Insatt belopp" happens to equal a same-day outflow scores 0.85 on the + // shared matcher, which would clear CERTAIN_CONFIDENCE and skip + // adjudication, on a pairing that is wrong by construction: the hunt scans + // outflows only, and "Insatt belopp" labels an inflow. Fallback-scored + // candidates are for the picker and the agent, never the nightly hunt. + const bankintyg = item( + { id: 'item-intyg', document_id: 'doc-intyg' }, + { + supplier: { name: null }, + totals: { total: null, vatAmount: null }, + documentKind: 'other', + prominentAmounts: [{ amount: 438.75, label: 'Insatt belopp' }], + }, + ) + expect(selectProposals([tx()], [bankintyg], noSuppression)).toEqual([]) + }) + it('skips a transaction that already has a live proposal', () => { const result = selectProposals([tx()], [item()], { claimedTransactionIds: new Set(['tx-1']), diff --git a/lib/receipt-hunt/select.ts b/lib/receipt-hunt/select.ts index fc04cb6d..c94fb6be 100644 --- a/lib/receipt-hunt/select.ts +++ b/lib/receipt-hunt/select.ts @@ -172,6 +172,15 @@ export function selectProposals( const scored = scoreUnderlagCandidates(tx, pool as never[]).filter( (candidate) => candidate.document_id != null && + // Never propose on the prominent-amounts fallback (non-invoice + // documents: bankintyg, avtal). The hunt's thresholds were calibrated + // for invoice-style totals; a fallback pair can reach 0.85 on date + + // printed-figure alone, which clears CERTAIN_CONFIDENCE and skips + // adjudication, and its preview would show document_amount null. The + // hunt also scans outflows only, so an inflow-labeled figure + // ("Insatt belopp") is guaranteed to pair with the wrong row. Those + // documents stay reachable through the picker and agent candidates. + candidate.amountSource !== 'prominent' && !spentDocumentIds.has(candidate.document_id) && !suppression.claimedDocumentIds.has(candidate.document_id) && !suppression.rejectedPairs.has(pairKey(tx.id, candidate.document_id)), diff --git a/messages/en.json b/messages/en.json index 9b6ffe87..2ae25ac3 100644 --- a/messages/en.json +++ b/messages/en.json @@ -3540,6 +3540,7 @@ "payment_invoice": "Invoiced", "payment_other": "Other", "pages_partial_note": "Extracted from {analyzed} of {total} pages.", + "prominent_amounts_label": "Amounts", "heic_hint": "HEIC images cannot be AI-extracted yet. Upload the receipt as JPEG or PDF, or fill in the fields manually.", "skipped_hint": "AI extraction did not run for this document. You can link the document to a transaction or create a supplier invoice manually.", "retry_overwrite_confirm": "Re-running extraction overwrites the fields, including your own edits. Continue?", diff --git a/messages/sv.json b/messages/sv.json index 656fda43..938be1d8 100644 --- a/messages/sv.json +++ b/messages/sv.json @@ -3540,6 +3540,7 @@ "payment_invoice": "Faktureras", "payment_other": "Annat", "pages_partial_note": "Tolkad från {analyzed} av {total} sidor.", + "prominent_amounts_label": "Belopp", "heic_hint": "HEIC-bilder kan inte AI-tolkas ännu. Ladda upp kvittot som JPEG eller PDF, eller fyll i fälten manuellt.", "skipped_hint": "AI-tolkning kördes inte för det här dokumentet. Du kan koppla dokumentet till en transaktion eller skapa leverantörsfaktura manuellt.", "retry_overwrite_confirm": "Ny tolkning skriver över fälten, även ändringar du gjort själv. Fortsätta?", diff --git a/types/index.ts b/types/index.ts index c99c4f6c..3b519398 100644 --- a/types/index.ts +++ b/types/index.ts @@ -4390,6 +4390,10 @@ export interface InvoiceExtractionResult { roundingAmount?: number | null } vatBreakdown: VatBreakdownItem[] + // Amounts visible on non-invoice documents (bankintyg, avtal, contracts) + // with no invoice-style total. Matching hint only, never booked. Optional: + // extractions from before the field existed lack it. + prominentAmounts?: ProminentAmount[] confidence: number suggestedTemplateId?: string // Set by the caller (not the model) when a long PDF was sliced before @@ -4414,6 +4418,12 @@ export interface VatBreakdownItem { amount: number } +/** One amount printed on a non-invoice document, with the document's own label. */ +export interface ProminentAmount { + amount: number + label: string | null +} + // KPI Report export interface KPIReport { netResult: number // SEK