feat(invoice-inbox): receipt-aware extraction + OCR pipeline fixes (#1331)

Receipts and invoices were extracted through one invoice-shaped prompt
with no document classification. The extractor now also returns
documentKind, payment method (+ card last4), purchaseTime,
merchantCategory and legibility, validated with .catch(null) so a
hallucinated label degrades to unknown instead of sinking the parse.
The FieldsRail shows type and payment method above the editable fields.

Pipeline fixes, all verified against real failure paths:
- PDFs >3 pages: extract from a pdf-lib slice of the first 3 pages
  instead of skipping entirely (issue #553 gate); truncation recorded
  in extracted_data.pages and shown in the UI.
- Oversized images (>4 MB, over Bedrock's 5 MB cap): downscale to
  <=2000px JPEG via sharp before base64, instead of erroring to an
  empty result.
- HEIC/HEIF: attempt sharp transcode to JPEG; when libvips lacks HEIF
  (prebuilt binaries), fall through to today's behavior but show an
  explicit hint instead of silently blank fields.
- Oresavrundning: prompt rule + totals.roundingAmount so receipt totals
  reconcile with subtotal+VAT for exact-amount transaction matching.
- retry-extraction overwrites extracted_data wholesale: a confirm now
  guards against silently destroying manual field edits.

Deliberately NOT added: retry-on-transient-Bedrock-error; the SDK
already retries twice by default (maxRetries=2).

Co-authored-by: Jakob Wennberg <jakob.wennberg@gmail.com>
Co-authored-by: Claude Fable 5 <noreply@anthropic.com>
This commit is contained in:
Jakob Wennberg
2026-08-05 14:20:40 +02:00
committed by GitHub
co-authored by Jakob Wennberg Claude Fable 5
parent b1f71912a2
commit 3b3adf96c7
9 changed files with 466 additions and 26 deletions
@@ -1128,6 +1128,7 @@ export default function InvoiceInboxWorkspace(_props: WorkspaceComponentProps) {
{selected ? (
<FieldsRail
item={selected}
docMime={docMime}
accountingMethod={accountingMethod}
onDelete={() => handleDelete(selected.id)}
onBookDirect={() => setBookDirectOpen(true)}
@@ -1741,6 +1742,7 @@ function EmptyPreview({
function FieldsRail({
item,
docMime,
accountingMethod,
onDelete,
onBookDirect,
@@ -1753,6 +1755,7 @@ function FieldsRail({
onRetryRequested,
}: {
item: InboxItem
docMime: string | null
accountingMethod: AccountingMethod
onDelete: () => void
onBookDirect: () => void
@@ -1787,6 +1790,9 @@ function FieldsRail({
!!extractedSupplierName
const handleRetry = async () => {
// Retry overwrites extracted_data wholesale server-side, including any
// manual field edits: make the user opt into that loss explicitly.
if (hasAnyExtractedField(data) && !confirm(t('retry_overwrite_confirm'))) return
setIsRetrying(true)
try {
const res = await fetch(
@@ -1835,6 +1841,38 @@ function FieldsRail({
</div>
)}
{/* AI classification: what kind of document this is and how it was
paid. Read-only context above the editable fields; absent for
extractions from before the fields existed. */}
{(data?.documentKind || data?.payment?.method || data?.pages) && (
<div className="border-b px-4 py-3 text-xs space-y-1">
{data?.documentKind && (
<div className="flex gap-2">
<span className="text-muted-foreground w-14 shrink-0">{t('doc_kind_label')}</span>
<span>{t(`doc_kind_${data.documentKind}`)}</span>
</div>
)}
{data?.payment?.method && (
<div className="flex gap-2">
<span className="text-muted-foreground w-14 shrink-0">{t('payment_label')}</span>
<span>
{t(`payment_${data.payment.method}`)}
{data.payment.cardLast4 ? ` •• ${data.payment.cardLast4}` : ''}
{data.purchaseTime ? ` · ${data.purchaseTime}` : ''}
</span>
</div>
)}
{data?.pages && (
<div className="text-muted-foreground">
{t('pages_partial_note', {
analyzed: data.pages.analyzed,
total: data.pages.total,
})}
</div>
)}
</div>
)}
{item.error_message && (
<div className="border-b border-destructive/30 bg-destructive/5 px-4 py-3 text-xs space-y-2">
<div className="flex items-start gap-2">
@@ -1894,15 +1932,28 @@ function FieldsRail({
)}
{/* Skipped-extraction hint: explains the empty fields and points the
user to the manual paths (transaction link or supplier invoice). */}
user to the manual paths (transaction link or supplier invoice).
Deliberately reason-agnostic: skip covers sandbox, BYO-extraction
and unsliceable PDFs, not just page count anymore. */}
{item.extraction_skipped && !isResolved && (
<div className="border-b bg-muted/30 px-4 py-2 text-xs text-muted-foreground">
AI-tolkning skippades p.g.a. dokumentets storlek (fler än 3 sidor).
Du kan koppla dokumentet till en transaktion eller skapa
leverantörsfaktura manuellt.
{t('skipped_hint')}
</div>
)}
{/* HEIC: extraction ran but Bedrock cannot read the format, so every
field came back empty with no error. Tell the user why instead of
leaving a silently blank rail (iPhone photos default to HEIC). */}
{!item.extraction_skipped &&
!isResolved &&
hasAi &&
(docMime === 'image/heic' || docMime === 'image/heif') &&
!hasAnyExtractedField(data) && (
<div className="border-b bg-muted/30 px-4 py-2 text-xs text-muted-foreground">
{t('heic_hint')}
</div>
)}
{/* Extracted fields */}
<div className="flex-1 overflow-y-auto px-4 py-3">
<h3 className="text-xs uppercase tracking-wide text-muted-foreground font-medium mb-3">