c0ecf2fa3bebd46bdfd0169efd73b89653d1dfed
10
Commits
| Author | SHA1 | Message | Date | |
|---|---|---|---|---|
 MattssonandClaude Fable 5
|
84c8e1ce59 |
fix(inbox): never extract the receiving company as its own supplier (#2080)
* fix(inbox): never extract the receiving company as its own supplier The model sometimes reads the Kund/Kunduppgifter block of bank agreements and similar documents as the issuer, so the inbox offered to create the user's own company as a leverantor. Two layers: a supplier-direction rule in the extraction prompt (issuer, never the customer/recipient block), and a deterministic post-extraction guard that nulls the supplier block when its org number, derived VAT number, or exact name equals the receiving company's own. All AI extraction paths (upload, deferred, retry, attach-document, document-extraction, MCP) pass the company identity. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01Hqm9QgdyNAFaWiz6Ww7pgb * fix(inbox): keep the own-company guard alive for photographed documents Skeptic findings on PR #2080: normalizeImageForExtraction rebuilt the input without ownCompany, so the guard never fired for HEIC/oversized phone photos (the motivating case). Spread the original input instead. Also let a provably different extracted org number outvote a name coincidence, and accept 12-digit personnummer-form own org numbers (19/20 century prefix, enskild firma) alongside the 16-prefixed organisation form. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01Hqm9QgdyNAFaWiz6Ww7pgb * docs(compliance): add RoPA entry for the own-company-identity lookup Compliance swarm (GDPR Art. 30): fetchOwnCompanyIdentity reads companies.name and org_number on every extraction; record the processing activity (transient in-memory comparison, never sent to the model). Co-Authored-By: Claude Fable 5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01Hqm9QgdyNAFaWiz6Ww7pgb * fix(inbox): log own-company identity lookup failures instead of failing silent Compliance swarm (SOC 2 CC7.2): the bare catch in fetchOwnCompanyIdentity made a persistently broken lookup (RLS misconfig, DB outage) disable the guard invisibly. Also pin the VAT-number match across prefixed and bare digit forms in the test matrix. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01Hqm9QgdyNAFaWiz6Ww7pgb * fix(inbox): surface maybeSingle query errors in the own-company lookup CodeRabbit: maybeSingle() reports RLS/query failures in error without throwing, so the failure log added for SOC 2 CC7.2 never fired for exactly those cases. Throw the reported error into the existing catch (still fail open to nulls). Co-Authored-By: Claude Fable 5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01Hqm9QgdyNAFaWiz6Ww7pgb --------- Co-authored-by: Claude Fable 5 <noreply@anthropic.com> |
||
|
MattssonandClaude Fable 5
|
1a3686dd45 |
feat(inbox): promote a single prominent amount into the editable total (#2073)
* feat(inbox): promote a single prominent amount into the editable total Follow-up to #2048 after founder review: the Belopp row was load-bearing for matching but read-only, so a misread amount could not be corrected, and an empty TOTALT still read as "extraction failed". - promoteSingleProminentAmount (extraction post-step, all intake paths): documentKind other/government_letter with no total and exactly one distinct nonzero prominent amount gets it copied into totals.total, stamped totalSource: 'prominent'. Multi-amount documents are left alone: picking one silently would invent a total. - provenance keeps the safety rails: matching demotes a promoted total back through the prominent-amounts fallback (0.85 discount, date guard, amountSource tag), so the nightly receipt-hunt still excludes these documents and confidence never presents as certainty. - the fields-PATCH route clears totalSource when a human edits TOTALT: a user-set amount is a verified total at full weight. - the read-only Belopp row now renders only for multi-amount documents, and filters zero amounts ("Totalt manadspris: 0 kr" noise). Co-Authored-By: Claude Fable 5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01Hqm9QgdyNAFaWiz6Ww7pgb * fix(inbox): review pass: concurrency-safe fields PATCH, zero-amount predicate CodeRabbit findings on #2073: - the fields-PATCH read-merge-write could let a racing autosave restore a stale extracted_data blob (including a totalSource stamp a concurrent TOTALT edit had just cleared). The update is now conditional on the trigger-maintained updated_at; zero rows matched returns 409 and the client's next debounced save re-reads. - hasAnyExtractedField now uses the same meaningful-amount predicate as the Belopp render filter, so a zero-only prominentAmounts list no longer suppresses the retry / upgrade affordances. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01Hqm9QgdyNAFaWiz6Ww7pgb --------- Co-authored-by: Claude Fable 5 <noreply@anthropic.com> |
||
|
MattssonandClaude Fable 5
|
516e8b62ff |
feat(inbox): match non-invoice documents via prominent amounts (#2048)
* feat(inbox): match non-invoice documents via prominent amounts Bankintyg, bank agreements and other documentKind "other" PDFs carry no invoice-style total, so extraction correctly left totals.total null and the document became structurally unmatchable: findUnderlagCandidates hard-drops items without a comparable amount and the picker lost the 40% amount signal. - extraction: new prominentAmounts[] field (amount + document's own label), populated only when totals.total is null; account/org/phone/reference numbers and zero amounts excluded. totals.total semantics untouched. - matching: bestProminentAmountVariance() tries each printed amount and feeds calculateMatchConfidence at reduced weight (0.3 vs 0.4) in both the agent candidate scorer and TransactionMatchPicker. - UI: inbox rail shows the detected amounts read-only for such documents, list falls back to a single distinct prominent amount, and extraction no longer reads as "found nothing". Co-Authored-By: Claude Fable 5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01Hqm9QgdyNAFaWiz6Ww7pgb * fix(inbox): discount prominent-amount fallback instead of reweighting it Skeptic pass refutations on the first commit: normalized weighting made a reduced amount weight self-defeating. Date + exact fallback amount with no merchant scored (0.25+0.3)/0.55 = 1.0 ("100% sakerhet" on a wrong same-day transaction), and a DISAGREEING fallback amount scored above a disagreeing invoice total (0.67 vs 0.60) because shrinking the weight also shrank the penalty. - score fallbacks at full amount weight, then multiply by a flat FALLBACK_CONFIDENCE_FACTOR (0.85): agreement caps below certainty, disagreement stays at least as damning as for a real total. - agent candidate surface additionally requires the document date within DATE_TOLERANCE_DAYS, so an avtal listing 349 kr no longer matches every future 349 kr charge from the same counterparty. - bestProminentAmountVariance returns which amount matched + its document label, and the match reason names it ("Exakt belopp i dokumentet: 2 500 SEK (Engangspris)"): no more bare "Exakt belopp" reaching the agent while total_amount is null. - prompt: prominentAmounts restricted to non-invoice documentKinds, and never a parking spot for an unreadable invoice total. - fix the stale "deliberately the same list" comment on EXTRACTED_FIELD_ACCESSORS. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01Hqm9QgdyNAFaWiz6Ww7pgb * fix(receipt-hunt): never propose on the prominent-amounts fallback Second skeptic pass: the nightly hunt is a third consumer of scoreUnderlagCandidates and inherited the fallback unaware. A bankintyg whose printed "Insatt belopp" equals a same-day outflow scores 0.85, which clears CERTAIN_CONFIDENCE (0.8) and skips LLM adjudication, on a pairing wrong by construction (the hunt scans outflows only; "Insatt belopp" labels an inflow), with document_amount null in the approval preview. UnderlagCandidate now carries amountSource ('total' | 'prominent') and selectProposals drops fallback-scored candidates. Non-invoice documents stay reachable through the manual picker and the agent candidate surface, both of which have a human reading the amounts. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01Hqm9QgdyNAFaWiz6Ww7pgb * fix(inbox): round fallback confidence via roundOre, not the naive pattern The two confidence discounts (and their test) tripped the naive-ore-round antipattern ratchet (625 vs baseline 622); use the sanctioned helper. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01Hqm9QgdyNAFaWiz6Ww7pgb --------- Co-authored-by: Claude Fable 5 <noreply@anthropic.com> |
||
|
MattssonandClaude Fable 5
|
a1cafe495f |
fix(invoice-inbox): read whole PDFs (last-page slice + truncation retry) (#2014)
* fix(invoice-inbox): read whole PDFs (last-page slice + truncation retry) PDF extraction read only part of well-structured PDFs, two confirmed mechanisms (21-day prod window: 49 sliced docs, 29 silent empties): - The auto-extract page budget was 3 (Bedrock-latency legacy, issue #553) and the slice kept only the first pages, so multi-page invoices lost the final page where totals, OCR and 'Att betala' sit. The budget is now 8 on pdf-native backends (Claude reads PDFs directly); the slice always keeps the last page. Rasterizing self-host backends keep the old budget of 3. - A max_tokens-truncated model answer was parsed as-is, failed, and became an all-null extraction with no trace. extractFromDocument now reports stop_reason max_tokens / finish_reason length as truncated; the extractor retries once at double AI_EXTRACTION_MAX_TOKENS and logs ai_extraction_truncated either way. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_012xosyW53HUa9JoFiDayhSk * fix(invoice-inbox): sweep cutoff covers the slower two-call extraction Skeptic finding on #2014: the crash-recovery sweep flipped 'processing' rows to an empty skeleton after 2 minutes, but a deferred extraction can now legitimately run 3-5 minutes (8 native pages plus one truncation retry at a doubled token cap), so the sweep stole the row and the CAS discarded the worker's real result. Cutoff raised to 10 minutes. Also: pages_partial_note made period-agnostic (old rows were extracted from first-pages-only slices, so naming the last page was retroactively wrong for them), and two stale first-pages-only comments updated. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_012xosyW53HUa9JoFiDayhSk * fix(invoice-inbox): keep the first extraction response when the retry throws CodeRabbit finding on #2014: a throttled/failed retry call bubbled to the outer catch before rawText was assigned, discarding a first response whose text may parse fine despite the truncation flag. The retry is now caught locally (logged as ai_extraction_retry_failed) and the first result flows on. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_012xosyW53HUa9JoFiDayhSk --------- Co-authored-by: Claude Fable 5 <noreply@anthropic.com> |
||
|
MattssonandClaude Fable 5
|
4b0a185876 |
fix(invoice-inbox): parse AI extraction output wrapped in markdown fences (#1460)
* fix(invoice-inbox): parse AI extraction output wrapped in markdown fences
Since the Sonnet 5 switch (
|
||
|
|
3b3adf96c7 |
feat(invoice-inbox): receipt-aware extraction + OCR pipeline fixes (#1331)
Receipts and invoices were extracted through one invoice-shaped prompt with no document classification. The extractor now also returns documentKind, payment method (+ card last4), purchaseTime, merchantCategory and legibility, validated with .catch(null) so a hallucinated label degrades to unknown instead of sinking the parse. The FieldsRail shows type and payment method above the editable fields. Pipeline fixes, all verified against real failure paths: - PDFs >3 pages: extract from a pdf-lib slice of the first 3 pages instead of skipping entirely (issue #553 gate); truncation recorded in extracted_data.pages and shown in the UI. - Oversized images (>4 MB, over Bedrock's 5 MB cap): downscale to <=2000px JPEG via sharp before base64, instead of erroring to an empty result. - HEIC/HEIF: attempt sharp transcode to JPEG; when libvips lacks HEIF (prebuilt binaries), fall through to today's behavior but show an explicit hint instead of silently blank fields. - Oresavrundning: prompt rule + totals.roundingAmount so receipt totals reconcile with subtotal+VAT for exact-amount transaction matching. - retry-extraction overwrites extracted_data wholesale: a confirm now guards against silently destroying manual field edits. Deliberately NOT added: retry-on-transient-Bedrock-error; the SDK already retries twice by default (maxRetries=2). Co-authored-by: Jakob Wennberg <jakob.wennberg@gmail.com> Co-authored-by: Claude Fable 5 <noreply@anthropic.com> |
||
|
Jakob WennbergandClaude Opus 4.7
|
8a1c7f0714 |
feat(invoice-inbox): loading state + review fixes + env-driven config (#416)
* feat(invoice-inbox): make Bedrock model + max_tokens env-overridable Read BEDROCK_MODEL_ID and BEDROCK_MAX_TOKENS from process.env so ops can swap models or raise token caps without a code deploy. Defaults align with production: eu.anthropic.claude-sonnet-4-6 in eu-north-1, 8192 tokens (up from 4096 — enough headroom for 20+ line-item invoices that were still occasionally truncating mid-JSON). Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com> * fix(invoice-inbox): guard onFieldsUpdated, tighten env parsing, hoist afterAll Three fixes from Greptile review on #416: 1. Stale-closure bug in onFieldsUpdated. setSelected used an unconditional spread, so a PATCH that completed after the user had navigated to a different item would silently overwrite the new selection's extracted_data with the old item's payload. Guard with prev?.id === selected.id (capturing selected.id outside the updater so it's stable through both setSelected and setItems). 2. BEDROCK_MAX_TOKENS env parser used `Number(...) || 8192`, which silently treats a deliberate `0` the same as unset. Use Number.isFinite(n) && n > 0 so an invalid configuration is not masked by the fallback. 3. Hoist `afterAll` import in the test file to the top alongside other vitest imports. Was working via ES-module hoisting but reads weird to humans and linters. Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com> --------- Co-authored-by: Claude Opus 4.7 (1M context) <noreply@anthropic.com> |
||
|
Jakob WennbergandClaude Opus 4.7
|
9c13586b13 |
feat(invoice-inbox): AI extraction (Bedrock Sonnet 4.6) + editable fields (#415)
* feat(invoice-inbox): AI extraction via Bedrock + editable fields Two big changes that go together: 1. Replace the regex extractor with Claude Sonnet 4.6 via AWS Bedrock. The PDF or image is sent directly to the model — no unpdf, no DOM stubs, no worker bundling. Sonnet handles English receipts (Anthropic, AWS, Stripe), USD/EUR currency symbols, scanned PDFs and image receipts, and Subtotal vs Total disambiguation that the regex layer couldn't. Output is JSON, validated with Zod; anything that doesn't parse falls back to an empty result so the inbox row still lands. 2. Make the extracted fields editable inline in the workspace. Each field is an Input bound to local draft state with debounced auto-save (800ms) to a new PATCH /items/:id/fields route. The route refuses edits once the item is converted to a supplier invoice. The parent updates both the selected-item view and the list rail when a field saves so the summary stays in sync. Drops `unpdf`, adds `@anthropic-ai/bedrock-sdk`. Requires three env vars in Vercel prod (AWS_ACCESS_KEY_ID, AWS_SECRET_ACCESS_KEY, AWS_REGION) — not yet set; when missing, the extractor logs a warning and returns empty so the upload still works. All 43 invoice-inbox unit tests pass. tsc clean. Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com> * feat(invoice-inbox): loading state + #415 review fixes Loading state for AI extraction: - Optimistic placeholder row inserted into the workspace list the moment an upload starts. Shows the file name + a "Tolkar dokument med AI…" spinner instead of timestamp/amount. - Document preview pane shows a centered loader + same caption while the placeholder is selected. - Fields rail shows 6 skeleton inputs + the same caption. - Action buttons are hidden until the real row arrives. - Placeholder is removed on success or failure (toast on error). PR #415 review follow-ups (Greptile + Swedish compliance bot): - Bump max_tokens 1500 → 4096 so multi-line invoices don't silently truncate mid-JSON and land with all-null fields. - Tighten NullableDate to range-checked regex + Date.parse refine — rejects 2026-13-45 / 2026-02-30 with a clean 400 instead of a 500. - Make vatRate representation consistent: percent integer (25, 12, 6, 0) for both lineItems[].vatRate AND vatBreakdown[].rate. Previously the AI was instructed to emit decimals for one and integers for the other. - EditableFieldsList re-seeds drafts when the parent passes a new data snapshot, but only on fields where the local draft still matches the previous server value — so a server-normalised currency upper-cases cleanly without clobbering an in-progress edit. - 409 conflict (item already linked to a supplier invoice) shows the server's specific Swedish message ("Posten är låst") instead of the generic "Kunde inte spara". Out of scope (separate tickets if needed): live updates for email-arrived items, AWS_SESSION_TOKEN plumbing, 6 % livsmedel rate transition validation, server-side totals consistency check. All 43 invoice-inbox unit tests pass. tsc clean. Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com> * fix(invoice-inbox): tighten extraction schema per compliance bot review Three follow-ups from the Swedish compliance bot's second pass on #415: 1. lineItems[].vatRate and vatBreakdown[].rate now refine to 0–100, blocking AI hallucinations like 5000% or negative rates while still accepting non-Swedish rates (UK 20, DE 19) since gnubok stores foreign invoices for reference. The strict Swedish [0, 6, 12, 25] allowlist is not enforced here on purpose — that check belongs in the supplier- invoice-creation step where the data hits the ledger. 2. accountSuggestion is now coerced to null at parse time via .transform, eliminating the brief intermediate window where a hallucinated string could appear in the parsed object before the post-validation .map() nulled it. Removes the redundant .map() afterwards. 3. PATCH /items/:id/fields currency now requires ISO 4217 format (^[A-Z]{3}$). Previously accepted any 3–8 char string, which would flow into supplier-invoice creation and produce a faktura with an invalid currency. Test fixture vatRate updated 0.25 → 25 to match the percent-integer convention introduced in the previous commit. All 43 invoice-inbox unit tests pass. Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com> --------- Co-authored-by: Claude Opus 4.7 (1M context) <noreply@anthropic.com> |
||
|
Jakob WennbergandClaude Opus 4.7
|
8676f10fdd |
fix(invoice-inbox): switch from pdfjs-dist to unpdf (#409)
* fix(invoice-inbox): switch from pdfjs-dist to unpdf for PDF text extraction After three rounds of fighting pdfjs on Vercel (#407 stubbed DOM globals, #408 tried to ship the worker file via outputFileTracingIncludes), text extraction still failed in prod with "Setting up fake worker failed" — Next's tracer can't reliably include pdfjs-dist's worker file when the package is marked as a server external. unpdf is a serverless-first wrapper around pdfjs (by unjs) that ships its own bundled pdfjs build with no canvas/worker dependencies. Drop-in replacement: extractText returns merged page text directly. - Remove DOM stubs, serverExternalPackages, outputFileTracingIncludes - Replace pdfjs-dist with unpdf (no transitive deps) - Update test mock from getDocument → extractText All 45 invoice-inbox unit tests pass. Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com> * fix(invoice-inbox): make unpdf import static, rename pdfjs test labels Per Greptile review on #409: 1. CRITICAL: tryExtractPdfText was still using await import('unpdf'), a dynamic import. CLAUDE.md forbids dynamic imports in extensions precisely because Next.js bundling can't reliably trace them — which is the same class of failure that caused the pdfjs prod bug. unpdf bundles statically (no canvas/worker), so a top-level static import is safe and correct. 2. NIT: two test descriptions still said "pdfjs" after the mock rename. Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com> --------- Co-authored-by: Claude Opus 4.7 (1M context) <noreply@anthropic.com> |
||
|
Mattsson
|
fa7d4075cf |
Supp/invoice bfl errors (#390)
* feat(accounting): update accounting method validation and messaging for aktiebolag and enskild firma * Remove AI subsystem and related code - Deleted AI proposals and requests persistence logic from `lib/ai/proposals/persist.ts`. - Removed re-validation logic for proposals in `lib/ai/proposals/re-validate.ts`. - Cleaned up schemas related to AI flows in `lib/api/schemas.ts`. - Removed AI-related fields from bookkeeping engine in `lib/bookkeeping/engine.ts`. - Eliminated AI event types from `lib/events/types.ts`. - Updated tests to reflect the removal of AI-related functionality in `lib/extensions/__tests__/sectors.test.ts`. - Adjusted initialization logic in `lib/init.ts` to exclude AI proposal handler registration. - Cleaned up transaction ingestion logic in `lib/transactions/ingest.ts` to remove AI flow checks. - Updated helper functions in `tests/helpers.ts` to remove AI-related settings. - Removed AI-related types and interfaces from `types/index.ts`. - Added migration script to drop AI-related tables and settings from the database. * fix(migrations): ensure foreign key constraint is dropped before removing AI tables * feat(invoice-inbox): implement deterministic invoice field extraction and inbox provisioning - Added `extract-invoice-fields.ts` for extracting fields from PDF invoices using regex and pdfjs-dist, replacing the previous AI classifier. - Introduced `inbox-provisioning.ts` to manage company inbox addresses and rotation of inboxes using Supabase RPCs. - Created `resend-inbound.ts` for handling inbound email events and attachments via the Resend API. - Defined the extension manifest for the invoice inbox, specifying required environment variables and descriptions. - Migrated database schema to remove AI-related columns and tighten the status enum in `invoice_inbox_items`. * feat(invoice-inbox): remove AI-specific columns and tighten status enum * fix(skattekonto): remove manual entry creation reference from transaction input * fix(schemas): remove accounting method validation for aktiebolag in UpdateSettingsSchema |