feat(invoice-inbox): receipt-aware extraction + OCR pipeline fixes (#1331)

Receipts and invoices were extracted through one invoice-shaped prompt
with no document classification. The extractor now also returns
documentKind, payment method (+ card last4), purchaseTime,
merchantCategory and legibility, validated with .catch(null) so a
hallucinated label degrades to unknown instead of sinking the parse.
The FieldsRail shows type and payment method above the editable fields.

Pipeline fixes, all verified against real failure paths:
- PDFs >3 pages: extract from a pdf-lib slice of the first 3 pages
  instead of skipping entirely (issue #553 gate); truncation recorded
  in extracted_data.pages and shown in the UI.
- Oversized images (>4 MB, over Bedrock's 5 MB cap): downscale to
  <=2000px JPEG via sharp before base64, instead of erroring to an
  empty result.
- HEIC/HEIF: attempt sharp transcode to JPEG; when libvips lacks HEIF
  (prebuilt binaries), fall through to today's behavior but show an
  explicit hint instead of silently blank fields.
- Oresavrundning: prompt rule + totals.roundingAmount so receipt totals
  reconcile with subtotal+VAT for exact-amount transaction matching.
- retry-extraction overwrites extracted_data wholesale: a confirm now
  guards against silently destroying manual field edits.

Deliberately NOT added: retry-on-transient-Bedrock-error; the SDK
already retries twice by default (maxRetries=2).

Co-authored-by: Jakob Wennberg <jakob.wennberg@gmail.com>
Co-authored-by: Claude Fable 5 <noreply@anthropic.com>
This commit is contained in:
Jakob Wennberg
2026-08-05 14:20:40 +02:00
committed by GitHub
co-authored by Jakob Wennberg Claude Fable 5
parent b1f71912a2
commit 3b3adf96c7
9 changed files with 466 additions and 26 deletions
+30
View File
@@ -3628,7 +3628,32 @@ export interface IngestResult {
// ── Invoice extraction (used by invoice-inbox extension and core utils) ──
export type ExtractedDocumentKind =
| 'receipt'
| 'supplier_invoice'
| 'government_letter'
| 'other'
export type ExtractedPaymentMethod = 'card' | 'swish' | 'cash' | 'invoice' | 'other'
export type ExtractedMerchantCategory =
| 'restaurant'
| 'cafe'
| 'taxi'
| 'parking'
| 'fuel'
| 'grocery'
| 'hotel'
| 'other'
export type ExtractedLegibility = 'good' | 'partial' | 'unreadable'
export interface InvoiceExtractionResult {
// Classification fields (2026-08): optional because extractions stored
// before they existed lack them. They route UI emphasis and clarifying
// questions only: never bookings.
documentKind?: ExtractedDocumentKind | null
merchantCategory?: ExtractedMerchantCategory | null
legibility?: ExtractedLegibility | null
purchaseTime?: string | null
payment?: { method: ExtractedPaymentMethod | null; cardLast4: string | null } | null
supplier: {
name: string | null
orgNumber: string | null
@@ -3654,10 +3679,15 @@ export interface InvoiceExtractionResult {
subtotal: number | null
vatAmount: number | null
total: number | null
// Öresavrundning line on Swedish receipts; negative when rounded down.
roundingAmount?: number | null
}
vatBreakdown: VatBreakdownItem[]
confidence: number
suggestedTemplateId?: string
// Set by the caller (not the model) when a long PDF was sliced before
// extraction: fields were read from the first `analyzed` of `total` pages.
pages?: { total: number; analyzed: number }
}
export interface ExtractedInvoiceLineItem {