12 Commits

Author	SHA1	Message	Date
Jakob Wennberg	a08bf51ced	feat(reports): log behandlingsregler changes and program versions (BFNAR 2013:2 p. 9.16) (#2097) ... * feat(reports): log behandlingsregler changes and program versions (BFNAR 2013:2 p. 9.16) Part 3 of the behandlingshistorik series (#1787 report, #1790 PDF). BFNAR 2013:2 punkt 9.16 second paragraph requires the behandlingshistorik to record "forandringar i bokforingssystemet som paverkar bokforingsposternas behandling samt nar dessa forandringar infordes", and BFN's commentary names behandlingsregler (automatkonteringar, fasta procentsatser) and new program versions as the examples. Until now both changed without a trace. Audit triggers on the behandlingsregler tables and the import logs: mapping_rules, booking_template_library, categorization_templates, salary_payroll_config, sie_imports, bank_file_imports. categorization_templates learns on every booking (occurrence_count, confidence, last_seen_date), so those telemetry-only updates are excluded by a WHEN clause the same way the api_keys request counters are (20260721115701): only real rule changes are logged. Measured against prod that is roughly 3 800 new audit rows a month against an audit_log already taking 371 688, so about +1 %. app_releases is an append-only log of program versions seen in production, written by the runtime the first time a build answers a request. Vercel exposes no build hook we can trust to write the row, so /api/version records it inside after(): the handler returns synchronously and a floating promise could be frozen before the insert lands, which is how a version log ends up silently empty. The service client is constructed lazily so the constantly polled public probe pays nothing once the module guard is set. Program versions are rolled up per Swedish calendar day in the report. main takes ~570 merges a month, so one event per version would be on the order of 7 000 a fiscal year: enough to trip the PDF's own 4 000-event guard and bury the ~400 events a real company's year contains. The statutory unit is the date, and the same sentence qualifies the requirement to changes that affect processing, which a deploy list cannot distinguish anyway. app_releases keeps the per-version truth for anyone who needs to go deeper. AuditLogEntry.user_id becomes string | null. The column is nullable and write_audit_log() falls back to auth.uid(), which is NULL for a service-role or global write; the company-less salary_payroll_config rows are the first that routinely hit it, and the read model already coded for it. Also restores the point citations the 2026-07-27 pass removed while the chapter was unverified: it is kapitel 9, not kapitel 8 (which is arkivering), verified against BFN's consolidated text. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01L3P2hr19PhQuCoTSGoegcY * test(pg): fix two fixture bugs in the behandlingshistorik trigger tests pg-real caught both, and neither is in the migration: the inserts fail before the trigger is reached. mapping_rules.rule_type is constrained to mcc_code / merchant_name / description_pattern / amount_threshold / combined; the test used 'merchant'. booking_template_library's btl_insert policy requires current_user_can_write() and company_id = current_active_company_id(), so the authenticated insert needs a company_members row and a user_preferences.active_company_id, the same setup booking-template-hidden.pg.test.ts uses. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01L3P2hr19PhQuCoTSGoegcY * test(pg): assert the booking-template audit row inside the user transaction withUserContext always rolls back, so the audit row the trigger writes is gone before an outside connection can see it. The trigger fires in the same transaction as the write, so the assertion belongs there too. The other cases in this file write on the pool (autocommit) and are unaffected. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01L3P2hr19PhQuCoTSGoegcY * fix(reports): name every build id in the per-day program-version entry Raised by the compliance review on #2097: the roll-up listed five ids and a count, which leaves an auditor unable to reconstruct which versions ran that day. app_releases keeps the full record, but the report is the surface anyone actually reads. A day is bounded by the deploy rate (~19), so the full list stays one readable cell. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01L3P2hr19PhQuCoTSGoegcY --------- Co-authored-by: Jakob Wennberg <311770904+jakobwennberg-oss@users.noreply.github.com> Co-authored-by: Claude Opus 5 <noreply@anthropic.com>	2026-09-01 20:31:10 +02:00
Mattsson	1a3686dd45	feat(inbox): promote a single prominent amount into the editable total (#2073) ... * feat(inbox): promote a single prominent amount into the editable total Follow-up to #2048 after founder review: the Belopp row was load-bearing for matching but read-only, so a misread amount could not be corrected, and an empty TOTALT still read as "extraction failed". - promoteSingleProminentAmount (extraction post-step, all intake paths): documentKind other/government_letter with no total and exactly one distinct nonzero prominent amount gets it copied into totals.total, stamped totalSource: 'prominent'. Multi-amount documents are left alone: picking one silently would invent a total. - provenance keeps the safety rails: matching demotes a promoted total back through the prominent-amounts fallback (0.85 discount, date guard, amountSource tag), so the nightly receipt-hunt still excludes these documents and confidence never presents as certainty. - the fields-PATCH route clears totalSource when a human edits TOTALT: a user-set amount is a verified total at full weight. - the read-only Belopp row now renders only for multi-amount documents, and filters zero amounts ("Totalt manadspris: 0 kr" noise). Co-Authored-By: Claude Fable 5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01Hqm9QgdyNAFaWiz6Ww7pgb * fix(inbox): review pass: concurrency-safe fields PATCH, zero-amount predicate CodeRabbit findings on #2073: - the fields-PATCH read-merge-write could let a racing autosave restore a stale extracted_data blob (including a totalSource stamp a concurrent TOTALT edit had just cleared). The update is now conditional on the trigger-maintained updated_at; zero rows matched returns 409 and the client's next debounced save re-reads. - hasAnyExtractedField now uses the same meaningful-amount predicate as the Belopp render filter, so a zero-only prominentAmounts list no longer suppresses the retry / upgrade affordances. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01Hqm9QgdyNAFaWiz6Ww7pgb --------- Co-authored-by: Claude Fable 5 <noreply@anthropic.com>	2026-08-31 11:40:30 +02:00
Mattsson	516e8b62ff	feat(inbox): match non-invoice documents via prominent amounts (#2048) ... * feat(inbox): match non-invoice documents via prominent amounts Bankintyg, bank agreements and other documentKind "other" PDFs carry no invoice-style total, so extraction correctly left totals.total null and the document became structurally unmatchable: findUnderlagCandidates hard-drops items without a comparable amount and the picker lost the 40% amount signal. - extraction: new prominentAmounts[] field (amount + document's own label), populated only when totals.total is null; account/org/phone/reference numbers and zero amounts excluded. totals.total semantics untouched. - matching: bestProminentAmountVariance() tries each printed amount and feeds calculateMatchConfidence at reduced weight (0.3 vs 0.4) in both the agent candidate scorer and TransactionMatchPicker. - UI: inbox rail shows the detected amounts read-only for such documents, list falls back to a single distinct prominent amount, and extraction no longer reads as "found nothing". Co-Authored-By: Claude Fable 5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01Hqm9QgdyNAFaWiz6Ww7pgb * fix(inbox): discount prominent-amount fallback instead of reweighting it Skeptic pass refutations on the first commit: normalized weighting made a reduced amount weight self-defeating. Date + exact fallback amount with no merchant scored (0.25+0.3)/0.55 = 1.0 ("100% sakerhet" on a wrong same-day transaction), and a DISAGREEING fallback amount scored above a disagreeing invoice total (0.67 vs 0.60) because shrinking the weight also shrank the penalty. - score fallbacks at full amount weight, then multiply by a flat FALLBACK_CONFIDENCE_FACTOR (0.85): agreement caps below certainty, disagreement stays at least as damning as for a real total. - agent candidate surface additionally requires the document date within DATE_TOLERANCE_DAYS, so an avtal listing 349 kr no longer matches every future 349 kr charge from the same counterparty. - bestProminentAmountVariance returns which amount matched + its document label, and the match reason names it ("Exakt belopp i dokumentet: 2 500 SEK (Engangspris)"): no more bare "Exakt belopp" reaching the agent while total_amount is null. - prompt: prominentAmounts restricted to non-invoice documentKinds, and never a parking spot for an unreadable invoice total. - fix the stale "deliberately the same list" comment on EXTRACTED_FIELD_ACCESSORS. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01Hqm9QgdyNAFaWiz6Ww7pgb * fix(receipt-hunt): never propose on the prominent-amounts fallback Second skeptic pass: the nightly hunt is a third consumer of scoreUnderlagCandidates and inherited the fallback unaware. A bankintyg whose printed "Insatt belopp" equals a same-day outflow scores 0.85, which clears CERTAIN_CONFIDENCE (0.8) and skips LLM adjudication, on a pairing wrong by construction (the hunt scans outflows only; "Insatt belopp" labels an inflow), with document_amount null in the approval preview. UnderlagCandidate now carries amountSource ('total' | 'prominent') and selectProposals drops fallback-scored candidates. Non-invoice documents stay reachable through the manual picker and the agent candidate surface, both of which have a human reading the amounts. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01Hqm9QgdyNAFaWiz6Ww7pgb * fix(inbox): round fallback confidence via roundOre, not the naive pattern The two confidence discounts (and their test) tripped the naive-ore-round antipattern ratchet (625 vs baseline 622); use the sanctioned helper. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01Hqm9QgdyNAFaWiz6Ww7pgb --------- Co-authored-by: Claude Fable 5 <noreply@anthropic.com>	2026-08-30 23:52:29 +02:00
Mattsson	3829b6add3	fix(ai): complete plain-key self-hosting path (#1584) ... * feat(ai): resolve the Claude backend from the environment Tier 1 of #1406: a self-hosted deployment can now run every AI feature on a plain ANTHROPIC_API_KEY, with no AWS account. Hosted behaviour is unchanged. lib/ai/provider.ts resolves the backend once, from the environment: AI_PROVIDER explicit override, bedrock|anthropic AWS static key pair Bedrock ANTHROPIC_API_KEY the direct Anthropic API nothing set Bedrock, so the AWS credential provider chain (instance profile, IRSA) still resolves Bedrock deliberately wins when both credential sets are present. EU residency in eu-north-1 is a BFL/GDPR posture rather than a default, so adding an Anthropic key for an experiment must not silently move production inference out of the region. AI_PROVIDER is the way to say you meant it. Model ids are written bare in code and prefixed to eu.anthropic.* only for Bedrock, which needs the cross-region inference profile for on-demand throughput. An operator override that already carries a prefix passes through untouched, so BEDROCK_MODEL_ID and friends keep working as written. Converted call sites: the agent composer, invoice-inbox extraction, the document-extraction model label, and both receipt-hunt clients. The last two are not named in the issue, which predates receipt-hunt landing in main. @anthropic-ai/sdk is declared at 0.95.0, the version @anthropic-ai/bedrock-sdk 0.29.1 already pulled in transitively, so the lockfile dedupes to one copy with no new download. scripts/smoke-bedrock.ts becomes scripts/smoke-ai.ts and grows two steps. Unit tests can only prove which provider and model id get resolved; they cannot prove the resulting request is one the backend accepts. The script now sends real traffic over all three shapes the app uses: a plain create, a streamed turn carrying adaptive thinking, an effort level, an hour-long cache breakpoint and a tool, and document extraction end to end when given a file. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com> Signed-off-by: Bjorn Bergenheim <29535152+bjornbergenheim@users.noreply.github.com> * docs(self-hosting): document the AI smoke test The script added alongside the provider split is what closes the #1406 acceptance criterion ("document extraction and the assistant both work"), so a self-hoster needs to know it exists. Covers both invocations and states that it exits non-zero, which is what makes it usable as a post-deploy check. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com> Signed-off-by: Bjorn Bergenheim <29535152+bjornbergenheim@users.noreply.github.com> * test(ai): split the smoke test's thinking probe from its tool probe The combined probe could not falsify what it claimed to. It asked a question that needs a tool call, so the tool was used and adaptive thinking correctly declined to reason about it: the zero thinking-block count that came back was uninformative rather than a signal. 2a keeps the tool and drops thinking. 2b asks a question with several dependent steps (reverse charge, then a partial deduction, then the affected boxes) so that a model honouring the parameter must reason, and reports the thinking text length as well as the block count, since display:"summarized" can yield blocks with empty text. The cached system prompt is also padded past the 1024-token minimum cacheable prefix. Below that the API caches nothing and reports no error, so the old probe's cache counters read zero whether or not caching worked. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com> Signed-off-by: Bjorn Bergenheim <29535152+bjornbergenheim@users.noreply.github.com> * fix(document-extraction): stop requiring AWS_REGION in the manifest The extension now needs one of two credential sets, AWS static keys or ANTHROPIC_API_KEY, and the manifest schema cannot express "one of". Since requiredEnvVars only drives a build-time warning and never gates anything, listing AWS_REGION told every self-hoster running the direct API to set a variable that has no effect for them. The description was also still promising Sonnet 4.6 via Bedrock specifically, which is no longer what the extension does. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com> Signed-off-by: Bjorn Bergenheim <29535152+bjornbergenheim@users.noreply.github.com> * fix(ai): read documentKind defensively in the smoke test The field arrived with the receipt-aware extraction work, so referencing it directly stops the script compiling against any checkout from before that landed. tsconfig includes **/*.ts and next.config does not disable type checking, so on such a checkout this failed the production build rather than just the script: caught while preparing a test branch for a self-hosted instance that had not synced yet. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com> Signed-off-by: Bjorn Bergenheim <29535152+bjornbergenheim@users.noreply.github.com> * fix(deps): restore the nested @swc/helpers entry in the lockfile Declaring @anthropic-ai/sdk with `npm install --package-lock-only` also pruned node_modules/next-intl/node_modules/@swc/helpers@0.5.23, an optional peer entry the local npm 11 considers redundant and the image's npm 10.9.8 does not. The result passed every local check and failed `npm ci` inside the Docker build, which is the only place the lockfile is actually enforced. The lockfile is now the previous one plus the single root dependency line, verified with `npm ci --dry-run`. @anthropic-ai/sdk needed nothing else: it was already in the tree as a transitive dependency of @anthropic-ai/bedrock-sdk. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com> Signed-off-by: Bjorn Bergenheim <29535152+bjornbergenheim@users.noreply.github.com> * Update DECISIONS.md Co-authored-by: coderabbitai[bot] <136622811+coderabbitai[bot]@users.noreply.github.com> * Update Docker documentation for AI provider credentials Clarify the role of credentials in AI provider selection and document extraction requirements. * Update SELF-HOSTING.md with smoke-ai script details Clarify usage of smoke-ai script for credential checks and document extraction. * Improve error handling and logging in smoke-ai script * fix(ai): complete plain-key self-hosting path Signed-off-by: Emil <emilmattsson14@gmail.com> --------- Signed-off-by: Bjorn Bergenheim <29535152+bjornbergenheim@users.noreply.github.com> Signed-off-by: Emil <emilmattsson14@gmail.com> Co-authored-by: Bjorn Bergenheim <29535152+bjornbergenheim@users.noreply.github.com> Co-authored-by: Claude Opus 5 <noreply@anthropic.com> Co-authored-by: coderabbitai[bot] <136622811+coderabbitai[bot]@users.noreply.github.com>	2026-08-13 15:45:24 +02:00
Jakob Wennberg	845add4573	feat(documents): dedupe intake channels on content, not just provenance (#1528) ... * feat(documents): dedupe intake channels on content, not just provenance Every ingestion path already computes and stores sha256_hash, but only WhatsApp ever read it back: the manual upload, Resend inbound, and mail hunt deduped on provenance keys alone (or not at all), so the same receipt forwarded to two inboxes, re-hunted by a sweep, or uploaded twice became a second archived document and a second inbox item. With the hunt live and three channels feeding one inbox, that is an unbounded duplicate generator (flows plan, prerequisite PR 1). uploadDocument gains an opt-in dedupeByContent flag: before storing, it looks for a current-version document in the same company with the same SHA-256 and returns it (marked deduplicated) instead of archiving a copy. Opt-in because archival callers must store what they produced even when bytes repeat; the SELECT-then-insert race is accepted exactly as in the WhatsApp intake precedent. uploadAndExtract turns the flag on for every inbox channel. On a hit it adopts the oldest inbox item for that document, so callers always receive a real inbox_item_id, and only files a new item (against the EXISTING document) when the content entered the archive outside the inbox. The mail hunt skips outright: its provenance key catches the same message re-hunted, the content check catches the same receipt arriving through another inbox. WhatsApp keeps its own pre-check, which also drives the duplicate reply to the sender. No migration: the hash column and its index have existed since the original archive schema. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com> * fix(documents): review round: fail closed, adopt-or-file in the hunt, audit trail CodeRabbit: both dedupe lookups failed OPEN, so a transient DB error would silently archive the duplicate the feature exists to prevent; both now throw before anything is stored, and a regression test locks it. The ingest test also asserts the dedupeByContent flag in the production call, so removing the flag fails the suite. Swedish compliance review, both findings real: (1) the mail hunt's unconditional skip could swallow a receipt whose content matches a document that never passed the inbox (a manually attached copy), leaving an affärshändelse without underlag routing (BFL 5 kap): the hunt now mirrors the funnel's adopt-or-file semantics, skipping only when an inbox item already carries the document and otherwise filing an item against the EXISTING document. (2) The skip decision now lands in behandlingshistorik as DocumentDuplicateSkipped (BFNAR 2013:2 kap 8), not just the app log. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com> * fix(receipt-hunt): keep the audit payload pseudonymous; lock the skip trail in tests Review round 2. The DocumentDuplicateSkipped payload carried the mailbox address, violating the processing-history contract (pseudonymous IDs only, never emails); the digit-shaped PII validator would not have caught it, which is exactly why the contract must hold at the call site. Which mailbox first delivered the receipt is already on the existing item's channel_context. Tests now assert the audit event lands with the right identifiers and no address, and that a history outage still skips rather than filing a duplicate. Not changed: a duplicate-lookup error still soft-fails the attachment (warn + continue). Aborting the candidate would contradict this function's documented contract (one bad message never costs the night's hunt); fail-closed holds either way, and the next sweep retries since no item was filed. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com> --------- Co-authored-by: Jakob Wennberg <311770904+jakobwennberg-oss@users.noreply.github.com> Co-authored-by: Claude Fable 5 <noreply@anthropic.com>	2026-08-12 15:55:12 +02:00
Jakob Wennberg	555a2a20ae	feat(inbox): Underlag rebuilt to answer what is missing, where to get it, and how it would be booked (#1524) ... * fix(mail): stop Gmail refusing the search, and stop calling that "hittade inget" Pressing Leta produced mails=25, documents=0 on a real two-mailbox run. Nothing was found because nothing was searched: every request came back 429 "Too many concurrent requests for user". Two bugs, and the second is the one that matters. The search fanned out with Promise.all over every message id at once, one Gmail request per message, per connection. Gmail enforces a per-user concurrency ceiling as well as a daily quota, and this sailed past it long before any volume worth worrying about. It now runs through a pool of five per connection, which is comfortably under and still finishes a page of results in a couple of round trips. The catch turned each refusal into an empty array, with a comment saying one mailbox's failure must not become the company's. Right instinct, wrong consequence: an empty array is also what an empty mailbox returns, and the manual hunt loop stops on fetched === 0 because that is its signal for "the mailboxes hold nothing more for what is open". So a rate-limited search told the user their receipts do not exist, and stopped looking. searchFailureCount() now separates "could not look" from "nothing there". The run route reports it, and the loop treats a pass with failures as failed rather than finished, so pressing again is the obvious next move instead of a pointless one. This is the failure this feature exists to catch, happening inside the feature: silence that reads as an answer. Restoring the unbounded fan-out fails one test; removing the failure counter fails three. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com> * feat(inbox): segment filter as a dropdown, not three rows of pills Five filters wrapped to three lines in a 280px column. The counts are what people actually read, so they stay on the trigger and inside the menu rather than being traded away for the space. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com> * feat(inbox): one chip for where underlag come from Three routes in, and the page never said so: the forwarding address sat inline in the header, the mailboxes lived only in Instaellningar, and WhatsApp was invisible here entirely. They are behind one chip now. Which mailbox and when it was last read is what people look up when something seems wrong, not what they read every visit, so it opens rather than occupying the header. A mailbox that has stopped working is the exception, so it surfaces on the chip itself rather than waiting to be found one click in. That silence is the failure this feature exists to catch. Configuration stays in Instaellningar; this only reports. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com> * feat(inbox): the kontering first, the evidence folded Reading order was backwards. Nine extracted values came first and the one thing to approve came last, so every matched item meant scrolling past the evidence to reach the decision. The proposed kontering is now the first thing in the rail. The fields fold behind a summary that carries how many of the twelve the extraction actually filled, so a thin extraction is visible without opening it. They stay open when nothing is matched: with no proposal above them the fields are all there is, and folding the only content on the pane would be a hiding place rather than a hierarchy. The counted list is the same one hasAnyExtractedField checks, so the summary cannot claim a field the 'is anything here' test does not count. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com> * feat(inbox): one dialog that changes the whole verifikat The rail offered three overlapping ways to alter a booking and none said what it covered: an Aendra beside the date, an Aendra kontering at the bottom, and a menu entry that did what the primary button already did. This is the one control, and its scope is the whole verifikat: date, series, description, every line. It opens pre-filled with the proposal when there is one and empty when there is not, so there is no separate book-manually path to pick between. A dialog rather than an inline editor: a 340px rail cannot hold an account picker, two money columns and a delete control per row without clipping something, and the document has to stay readable while the numbers change. Checking a momssats against the paper is the reason to open it at all. TransactionBookingDialog already has this shape for the same reason. The form is JournalEntryForm unchanged. It carries the series picker, per line descriptions, dimensions, currency, the balance check and the confirm step, and it posts through the sanctioned route. Extending BookDirectlyDialog was the alternative and is not viable: three effects seed its lines and fight anything injected, and its FormLine has no room for line text, dimensions or tax codes. Nothing posts without the form's own review step, so a proposal stays a draft the user commits. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com> * fix(inbox): show every unreceipted purchase, and fold the mailboxes Three things. The 100 kr floor was hiding 52 of one real company's 119 unreceipted purchases: the page reported 67 and looked tidier for it. The floor was copied from the receipt hunt, where it earns its place because every candidate costs a mail search and a model read. This list costs a query, and bokforingslagen wants an underlag for the 45 kr purchase exactly as much as for the 4 500 kr one. The hunt keeps its floor; the page has none. Mailboxes fold. When it was last searched is what you look up when a mailbox seems to have gone quiet, not what you read on the way past. The address stays on the row, and a connection that needs reconnecting still says so without opening. Dropped the line telling people to go to Instaellningar. The panel reports where underlag come from; sending them elsewhere was the seam this work set out to close. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com> * feat(inbox): split the portal purchases out, and say what a run found Four things from looking at the real page beside the artifact. Hamta fran portal is its own list again. Twelve of one company's 119 unreceipted purchases have a supplier whose invoices sit behind a login, and that is a different job from the other 107: go there and fetch it, versus ask somebody. Collapsing them into one list with a badge buried the twelve you can settle now among the hundred you cannot. A run now says what it did. Pressing Leta and being told nothing is why the feature read as broken even on the runs where it worked: three underlag landed and the page looked identical afterwards. WhatsApp folds like the mailboxes and shows its number, which is the fact worth having. Describing the channel to someone who already connected it was not. The forwarding address lost its subtitle, and WhatsApp rows carry the brand mark. Emailed documents keep the generic one: nothing records which mailbox fetched them, so claiming a provider would be a guess. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com> * fix(inbox): the WhatsApp number, three wrong portals, and somewhere to drop the file The WhatsApp row read the response in snake_case while the route answers camelCase, so a linked number rendered as a dash and a verified link read as unverified. Reading phoneMasked and verifiedAt fixes both. Anthropic, Vercel and Supabase are out of the portal directory. All three email their invoices to European customers, so listing them told somebody to go and log in for a document already sitting in their inbox: worse than saying nothing, because it sends them away from the answer. The directory's bar is 'does not send the invoice', not 'also has a portal'. The poll it was seeded from asked which portals people log into, and people answered with where an invoice can also be found. The same objection may reach further down the list. A purchase with no underlag now offers somewhere to put one. Telling somebody a document is missing without a place to drop it is half an answer, and the drop zone carries the amount and the date so the right file goes to the right purchase. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com> * fix(portal): the links were never opened, and two of them were wrong The directory shipped with eighteen hand-written paths and none had been clicked. The file said so in its own header and shipped regardless, which is how a founder came to land on a 404 opening Google Workspace. A sweep of every URL found GitHub broken as well. Google Workspace now points at the console root rather than a deep billing path: admin.google.com refuses automated requests, so no deeper path can be verified from here, and a link that lands one click short beats one that lands on an error page. GitHub points at the path that actually answers. Trygg Hansa is removed because neither candidate URL could be reached at all, and an unverifiable link is exactly the promise this file kept warning about. scripts/check-portal-urls.mts sweeps them, so the next wrong URL is found by a script rather than by somebody who trusted the link. A 404 fails it; a host that refuses automation reports as unreachable and does not, because failing on those would train people to ignore the output. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com> * fix(inbox): the drop zone now actually attaches the file to the purchase It did not. The generic upload sends only the file, so a document dropped while a purchase was selected landed in the inbox unmatched, while the pane showed that purchase's amount and date directly under the drop zone. The copy promised a link the code never made, and the user was left to match by hand what they had already told us. Uploading from a selected purchase now matches the new item to that transaction through the endpoint that already exists, and a file dropped anywhere on the page while a purchase is selected counts as that purchase's receipt rather than a loose upload. When the match fails the document is still safely filed, so it says so plainly instead of claiming a link that is not there. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com> * fix(inbox): book the underlag against its transaction, and stop claiming links Two blockers found by review, both on the path that writes to the ledger. "Granska och bokför" never sent transaction_id. JournalEntryForm serialises a fixed set of keys and that is not one of them, and BookInboxItemDirectlySchema is a non-strict z.object, so the source_id carrying it was silently stripped. The verifikat posted standalone, the bank transaction stayed unbooked, and matched_transaction_id was overwritten with null: the match somebody had already made, undone, while the rail said Bokförd over all of it. Fixed in three places because one was not enough. JournalEntryForm takes an extraBody passthrough, the dialog sends transaction_id through it, and the route now falls back to the item's existing match rather than null, so a caller that merely forgets the field cannot undo work. Removing that fallback fails the new test. The hunt banner said "kopplades till ett köp" about pending_operations rows. The hunt stages proposals for approval and books nothing, so the number was real and the word was wrong: a user would read it, believe three purchases were done, and leave. It now says how many förslag await granskning, and links there. Booking also left the rail in its pre-booking state, still offering to post, so the same underlag could be submitted twice. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com> * fix(inbox): no marker on a healthy state, no false empty state, no dropped files Three from review. The sources chip painted a sage dot whenever every mailbox was fine. Convention 12 rules semantic colour out of chrome, and convention 5 rules out a marker on a normal state: a chip every company sees always is a chip that says nothing. What is left is the exception, which is worth an ochre word and an icon. The pre-existing sage on matched rows is untouched; it is not this branch's to change. The empty state asserted "Varje köp har sitt underlag" while the trigger directly above it still showed the unsearched count. Type a term under Att göra, switch to Saknar underlag, and the page told you every purchase was covered while the button beside it read 50. It now says what is true: no matches for that term. A drop of several files onto a selected purchase kept the first and discarded the rest in silence, so a receipt scanned as two images left the purchase looking resolved with half its paperwork gone. They cannot all be one purchase's underlag, so the extras are filed in the inbox and the toast says how many. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com> * fix(inbox): the hunt banner now says a press is not the last word A press fetches a bounded number of receipts, so an empty result usually means not yet rather than nothing there. The banner said 'Inget matchade något köp' and stopped, which reads as final and sends people away from a mailbox that still holds their receipts. It now says how many purchases are left to search for, and to press again. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com> * fix(inbox): a count not a score, an honest failure, full-opacity borders '5 av 12' read as a bad extraction even when a kvitto had given up everything a kvitto has: half those twelve fields only exist on an invoice, so the denominator was measuring the document kind rather than the reading of it. It now says how many fields are filled, and says nothing when none are. The failure banner told people their mailbox had not answered even when the failure was ours, sending them to check a healthy Gmail. It now reads searchFailures and only blames the mailbox when a mailbox actually refused. Opacity-suffixed borders on the sources panel, which design.md forbids on surfaces: the border token is calibrated for full opacity. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com> * feat(inbox): translate the new strings, and name the mailbox that fetched a receipt Both of these were deferred with reasons, and one of the reasons was wrong. 57 keys in inbox_workspace, in both locales, covering every string this branch added. The component already had 27 t() calls, so hardcoding beside them was an inconsistency rather than a convention. The message-keys guard caught an invented journal_form.no_document on the way, which is what it is for. The provider mark claimed nothing recorded which mailbox fetched a document. It does: lib/receipt-hunt/ingest.ts writes mail_provider and mail_mailbox into channel_context on every ingest, and GET /items already selects that column. A hunted receipt now carries the mark of the mailbox it came from; forwarded mail has no connection behind it and keeps the envelope, which is the honest distinction rather than a guess. InboxChannelContext was WhatsApp-shaped and is now a union over the two intakes that write it. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com> * fix(agent-context): keep the clarification channel narrow Widening InboxChannelContext.channel to cover the mail hunt broke this: only WhatsApp asks a human anything, so only WhatsApp produces clarifications. The mail hunt writes the same column with its own shape and never carries answers, so the provenance field stays 'whatsapp' rather than following the union. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com> * fix(inbox): book the transaction we preserved, and date the verifikat by the event Three from PR review, two of them real. Preserving matched_transaction_id without booking it was the worse half of the bug it fixed. The transaction update was still guarded on the caller having sent transaction_id, so an omitted field left the item looking resolved while its bank line stayed open forever. Both the update and the item now use the same resolved id: the one the caller named, or the one the item was already matched to. Reverting the guard fails a test. The verifikat date fell back to today when there was no proposal, which is exactly the unknown-supplier case the dialog exists for. BFL 5 kap 6-7 § asks for datum för affärshändelsen; the day somebody opened a dialog is nobody's business event. It now falls back to the document's own date first, and only then to today. An en dash had crept in as a placeholder glyph, which the repo bans. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com> --------- Co-authored-by: Jakob Wennberg <311770904+jakobwennberg-oss@users.noreply.github.com> Co-authored-by: Claude Opus 5 (1M context) <noreply@anthropic.com>	2026-08-12 14:36:08 +02:00
Jakob Wennberg	f2d9e98af3	fix(mail): stop Gmail refusing the search, and stop calling that "hittade inget" (#1521) ... Pressing Leta produced mails=25, documents=0 on a real two-mailbox run. Nothing was found because nothing was searched: every request came back 429 "Too many concurrent requests for user". Two bugs, and the second is the one that matters. The search fanned out with Promise.all over every message id at once, one Gmail request per message, per connection. Gmail enforces a per-user concurrency ceiling as well as a daily quota, and this sailed past it long before any volume worth worrying about. It now runs through a pool of five per connection, which is comfortably under and still finishes a page of results in a couple of round trips. The catch turned each refusal into an empty array, with a comment saying one mailbox's failure must not become the company's. Right instinct, wrong consequence: an empty array is also what an empty mailbox returns, and the manual hunt loop stops on fetched === 0 because that is its signal for "the mailboxes hold nothing more for what is open". So a rate-limited search told the user their receipts do not exist, and stopped looking. searchFailureCount() now separates "could not look" from "nothing there". The run route reports it, and the loop treats a pass with failures as failed rather than finished, so pressing again is the obvious next move instead of a pointless one. This is the failure this feature exists to catch, happening inside the feature: silence that reads as an answer. Restoring the unbounded fan-out fails one test; removing the failure counter fails three. Co-authored-by: Jakob Wennberg <311770904+jakobwennberg-oss@users.noreply.github.com> Co-authored-by: Claude Opus 5 (1M context) <noreply@anthropic.com>	2026-08-11 14:23:58 +02:00
Jakob Wennberg	ca6b962662	feat(receipt-hunt): a directory of where suppliers keep their invoices (#1506) ... Some vendors never attach anything. They mail "your invoice is ready", or nothing at all, and the invoice waits behind a login. Seeing "OPENAI CHATGPT SUBSCR" on a statement is enough to say which page it is on, and that turns a dead end into a one-click errand. Deliberately not what Presto does. They log in for you across a thousand portals, with a desktop app so credentials never leave the machine. Holding a customer's supplier passwords is the expensive and legally heavy half of that product, and skipping it costs little of the value: knowing *where* is most of it. The lookup runs at read time and nothing is stored against a transaction, so adding an entry fixes every ledger at once, retroactively, with no migration and no backfill. A supplier added today starts answering for purchases made last year. Eighteen entries, ordered by how many companies actually pay them, measured across production ledgers and cross-checked against a customer poll. The two agree closely: Google and OpenAI lead both by a wide margin. Measured coverage: 769 of 10 182 undocumented purchases, which is 7.6% of rows but 120 of 224 companies, so more than half of all companies have at least one purchase this can now answer. Two things it refuses to do. It offers nothing for payments that have no invoice, reusing the rule that already keeps the hunt from searching mail for salary and tax: a link there implies somewhere to go. And longer aliases win, so a Google Cloud bill is not sent to the Workspace console. Vendors whose billing page could not be pinned down were left out rather than guessed at, including some with real volume: Fortnox, Apple, Klarna and Mynt. A wrong URL is worse than a missing one, because it spends the trust the feature runs on. Every entry here still deserves a human clicking it before anyone relies on it. No interface yet: this is the data and the lookup, so the workspace can use it the day it lands. Co-authored-by: Jakob Wennberg <311770904+jakobwennberg-oss@users.noreply.github.com> Co-authored-by: Claude Opus 5 (1M context) <noreply@anthropic.com>	2026-08-11 10:39:06 +02:00
Jakob Wennberg	88140e5774	feat(receipt-hunt): let a model settle the pairs the arithmetic cannot (#1498) ... The weights in the matcher are chosen by hand, and two of them were adjusted last week because they produced a wrong answer on real data. A pair agreeing to within 1%, from a merchant the matcher recognised, still scored 0.62 because a date had drifted. That says more about the constants than about the receipt. So the formula keeps what it is good at and hands over what it is not. Above 0.80 it decides alone: an exact amount from a recognised merchant needs no second opinion, and buying one for every pair would be latency and cost for a verdict nobody doubts. Below 0.60 nothing is asked either, because inviting a model to rescue a pair the evidence does not support is how plausible wrong answers get made. Only the band between goes for adjudication, which on a real ledger was five pairs against one the formula had already settled. The question is a yes or no with a reason, never a score. An earlier design in this feature asked a model to rate its own certainty and it anchored on round numbers, which is what the calibration literature predicts. Judging concrete evidence and explaining the judgement is a different task. A verdict is checked, not trusted: a key nobody asked about is dropped, a pair answered twice keeps its first answer, and a failed call accepts nothing, leaving the run exactly where the arithmetic left it. The formula's own score is stored unflattered next to the verdict, because dressing it up would hide the uncertainty that sent the pair for a second opinion in the first place, and agent_metadata records which instrument decided. Adjudication runs on a dry run too. A provkörning is meant to show what a real run would propose, and skipping it would show a smaller, different answer than the one that lands. Measured on a real ledger: five uncertain pairs asked, two accepted with reasons a human can check ("Samma leverantör och belopp stämmer inom rimlig valutamarginal"), three rejected. Proposals went from one to three. Co-authored-by: Jakob Wennberg <311770904+jakobwennberg-oss@users.noreply.github.com> Co-authored-by: Claude Opus 5 (1M context) <noreply@anthropic.com>	2026-08-10 18:12:14 +02:00
Jakob Wennberg	4c479f84f4	feat(receipt-hunt): convert a foreign receipt instead of refusing to compare it (#1497) ... * feat(receipt-hunt): compare a foreign receipt by converting it, not by refusing A Swedish bank posts a converted figure for a purchase abroad while the receipt states the original: Anthropic bills 180,00 EUR and the statement reads -2 014,32 kr. Neither number appears in the other document, so the matcher refused the pair rather than guess. On a SaaS-heavy ledger that is not an edge case: of 25 receipts fetched from a real company's mailboxes, 14 were in USD or EUR and none could ever pair. The receipt's total is now resolved into kronor with Riksbanken's rate for its own date, and handed to the same matcher, which still wants the merchant and the date to agree. The seam already existed: the scorer passed null where a SEK value would go, with a comment explaining that cross-currency pairs were deliberately incomparable. Surfaces that do not resolve a rate still pass nothing and behave exactly as before. Rates are fetched once per currency and day. Riksbanken answers 429 to a caller that asks per document, and a run holds a dozen receipts from one vendor in one month. A rate that cannot be resolved leaves the receipt exactly as incomparable as it was. Two calibration faults surfaced once the amounts became comparable: A converted total is judged at 9% rather than 5%. Riksbanken publishes a mid rate and a card issuer charges its own, so the two carry a known spread on top of any disagreement about the sum: measured against real statements, 1.2% to 3%. Holding both to one bar treats a rate spread as if it were a discrepancy. The date tolerance moves from 3 days to 10. A card settles days after the purchase, an international one routinely a week later, and a forwarded receipt carries the purchase date while the statement carries the posting. At three days the signal scored zero for ordinary correct pairs and took a quarter of the weight with it: a receipt agreeing to within 1%, from a merchant the matcher recognised, still capped at 0.62. The 27 human- confirmed pairs and the 7 near-misses that must not match all still hold. Measured on that ledger: purchases with any candidate at all go 9 -> 13, and the Anthropic pair proposes at 0.82 where it was previously unscoreable. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com> * fix(receipt-hunt): an undated receipt is not converted at today's rate Raised in review. A foreign receipt with no invoiceDate fell back to the current date, which put a receipt of unknown age into amount matching on the strength of a guess: a rate two years out is how something incomparable acquires confidence it has not earned. No date, no conversion, and the receipt stays exactly as it was. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com> --------- Co-authored-by: Jakob Wennberg <311770904+jakobwennberg-oss@users.noreply.github.com> Co-authored-by: Claude Opus 5 (1M context) <noreply@anthropic.com>	2026-08-10 17:54:22 +02:00
Jakob Wennberg	1d635b0d25	feat(receipt-hunt): look for receipts on request, from the mailbox settings page (#1496) ... * feat(receipt-hunt): a button that looks for receipts on request The nightly cron exists but still does not search mailboxes, and for a good reason: a sweep of one real 172-message mailbox took over 600s, against a scheduled function's 300. Pressing a button is the honest shape for work that big. A bounded pass reports what it found and how much is left, and the person decides whether to press again; a nightly run could only truncate silently. POST /api/receipt-hunt/run searches the mailboxes for eight purchases and fetches at most ten receipts per press. Gated on the AI tier, because reading the amount out of a PDF is what makes a fetched attachment matchable at all: without it the hunt would file documents that can never pair, which is worse than not running. Writes no journal entries; every pairing is still a proposal waiting for approval. The button lives on the mailbox settings page, which already ships, and says what happened in words rather than a spinner that stops: "3 underlag hämtade. 12 köp kvar att söka igenom." huntCompany gains maxReceipts so a manual pass can carry a different budget from a nightly one. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com> * fix(receipt-hunt): stop a manual press spending its budget on the wrong purchases The first real press searched eight purchases, read forty mails and found nothing, which looked like a broken model. It was the ordering. Purchases are searched largest first, which is right for a nightly sweep that eventually covers everything and wrong for a button pressed a few times. On a real ledger the largest rows are the least likely to have a findable receipt: rent already invoiced, bare payment references, direct debits. Those filled the forty-mail cap, so the productive purchases further down the list, the ones whose receipts are actually sitting in the mailbox, were never read at all. The cap was the binding constraint, not the time: eight purchases and forty mails took 43s of the 300 available. A press now searches 25 purchases and reads 100 mails, measured at 85s and finding 7 underlag on the same ledger that returned 0 before. huntCompany gains maxMails alongside maxReceipts, so a manual pass can carry a different budget from a nightly one rather than sharing an environment default with it. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com> * fix(receipt-hunt): one underlag per purchase, and a press that fits its budget Your second press exposed three things, none of which a dry run reaches. It took 5.8 minutes. The 85s I measured was a dry run, which never fetches, uploads or extracts; each fetched receipt costs about another 37s because it is downloaded, stored, and then read by a model that opens the PDF. Seven of them ran past the 300s a serverless function gets, so in production that press would have been killed. Four receipts per press keeps a full pass inside the budget. It fetched seven receipts and proposed nothing. A single mail carries the invoice AND the receipt for one purchase under different names ("Invoice-E19DBF63-0021.pdf" beside "Receipt-2066-0204-8388.pdf"), and the same receipt reaches a second mailbox on a different message. Each was fetched separately, so the pool filled with identical candidates and the matcher refused to propose any of them rather than flip a coin. The per-run key is now the vendor and the total, which is what identifies a purchase; the filename only decides when no amount was read. Nine duplicates already in the pool were removed. And with the duplicates gone it still proposed nothing, for a separate reason: "Utlägg Norwegian" scored 0.18 against "Norwegian Air Shuttle AOC AS". Utlägg is Swedish for an expense reimbursement, bank vocabulary rather than a company, and leaving it in broke the token-subset match, so an exact 1 998 kr pair leaned entirely on a date eight days out and fell under the floor. Stripped, along with överföring, via internet, bg-bet and autogiro, in the comparison path only. normalizeMerchantName is untouched: it is the persisted konteringskarta key with a SQL mirror, and its 22 string pins and the 27-pair golden set still pass. Measured after: the Norwegian pair proposes at 0.72. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com> * perf(mail): read each message once per press, not once per query The third press proposed a pairing, which the second had not, but still ran 5.2 minutes against a function's 300s. Cutting receipts from seven to four had only saved 36s, which said the receipts were never the cost. Every search fetched a full message for every hit, and a press searches many purchases across every connected mailbox. One receipt mail answers several of those queries, so 25 purchases against 2 mailboxes could ask Gmail for well over a thousand messages to end up with a hundred distinct ones. Deduplication happened in the caller, too late to save the work. A mail's content never changes, so it is now read once per mailbox and kept, bounded at a thousand entries and evicting oldest first. Measured on the same ledger: 55 purchases and 100 mails now take 102s, where 25 purchases alone previously cost around 264s before a single receipt was fetched. clearMessageCache exists because tests reuse message ids and production does not. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com> * fix(receipt-hunt): do not refetch a receipt the company already holds The fourth press fetched four documents the company already had: Bolagsverket, Supabase twice, Uber. They came back because I had deleted them as duplicates, and the cross-run check is the message and attachment id, which lives in the rows I removed. That was my mistake, but it exposed a real gap. The vendor-and-total key only deduplicates inside a single pass. Across passes the same purchase still arrives as an invoice in one mail and a receipt in another, with different file keys, and both were fetched: the pool fills with identical candidates and the matcher then refuses to choose between them, which is how a press can fetch four documents and propose nothing. The pass now starts from what the company already holds, so its budget goes on documents that are actually missing. Receipts per press drops to three. Measured on this ledger, a fetched receipt costs about 50s from download to a stored amount, and that is the model reading the PDF rather than the network: seven took 5.8 minutes and four took 5.1, both past the 300s a function gets. Three fits, but it is a stopgap. Doing the fetch inside the request is the wrong shape for work this slow, and the fix is to move it off the request rather than keep shaving this number. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com> * fix(mail): a mail body must not outlive the run that read it Two findings from the review, both correct. The message cache held whole MailCandidate values, and one of those fields is the mail body. The contract says a body is read once to extract fields and discarded, and a process-global cache quietly broke that: bodies of one company's mail sat in memory across requests until eviction or a restart. The MailSearchService contract now has releaseCache, the Gmail adapter clears its messages, and the hunt calls it in a finally so a failed run releases them too. The duplicate key accepted an empty vendor, so two unrelated documents that happened to cost the same collapsed into one candidate. Those now fall back to the file they came from: without a vendor there is nothing to anchor an amount to. The same finding caught something worse that I had introduced one commit earlier. The persistent check derived its key from the stored extraction while the fetch derived one from the reading model, so a document filed as "Norwegian Air Shuttle AOC AS" did not recognise an incoming "Norwegian" and was fetched again. Rather than guess at aliases, which would fold "Google Cloud" into "Google Workspace", the identity is now written onto the row when the receipt is filed and read back verbatim. Rows filed before that fall back to the extraction. receiptIdentity is one exported helper with its own tests, used by both sides, instead of the same expression written twice. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com> * fix(receipt-hunt): a monthly subscription is not a duplicate of last month The review caught that my duplicate key was worse than the problem it solved. Anthropic bills the same amount every month, and keying on vendor and total alone made July look like a duplicate of June: every later receipt from any recurring supplier would have been suppressed forever, silently. Duplicates block one proposal; that would have lost a receipt per month per subscription. The identity now carries the document date. Two documents for one purchase share a date; June and July do not. Two smaller faults in the same key. The amount was serialised as a raw float, so 0.1 + 0.2 read as a different total from 0.3; it is rounded to öre like every other money comparison in this codebase. And a document with no vendor was identified by its filename alone, which collapses two unrelated papers whenever a billing system attaches "invoice.pdf": those now carry the message they came from. The key is versioned so a future change to its shape cannot be mistaken for a match against rows written under the old one. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com> --------- Co-authored-by: Jakob Wennberg <311770904+jakobwennberg-oss@users.noreply.github.com> Co-authored-by: Claude Opus 5 (1M context) <noreply@anthropic.com>	2026-08-10 16:08:06 +02:00
Jakob Wennberg	38f5d9812e	feat(receipt-hunt): find receipts in connected mailboxes and pair them on the amount (#1492) ... * feat(receipt-hunt): nightly matcher pairing unbooked purchases with held receipts Stages an attach_document_to_transaction proposal for every unbooked card purchase whose receipt the company already holds, so the underlag is attached before the transaction is booked and the gap never forms. When the user later books it, categorize-core.ts propagates the document onto the new verifikat through the matched_transaction_id link the executor writes. Deliberately scoped to UNBOOKED transactions. The posted-verifikat backlog is 96% imported history whose originals live in the previous system, so it stays a pull (the verifikat_missing_document worklist) rather than a nightly push. Ranking reuses scoreUnderlagCandidates; the pool is loaded once per company instead of per transaction, which removes both the N+1 and the newest-50 truncation a per-transaction lookup imposes on a deep backlog. Five guards, each mutation-tested: a confidence floor above the shared candidate floor, an ambiguity margin so two equally-good receipts are left to the picker rather than coin-flipped, one-receipt-one-purchase, one live proposal per purchase, and permanent suppression of pairs a human rejected. Suppression is derived from pending_operations history rather than a new table: terminal rows are immutable and a rejection is already the durable "no". Runs 05:30 UTC, after the 05:00 bank sync. Gated on RECEIPT_HUNT_COMPANY_IDS, which hunts nobody when unset so enabling it stays a deliberate act. No migration, no journal writes, no UI: proposals land in the existing Granskning queue. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com> * feat(receipt-hunt): dry-run mode for provkörning against a real ledger Returns the pairings a run would stage without writing any of them, so a company can see tonight's proposals before they reach the granskningskö and so the matcher can be validated against production data without staging an operation. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com> * fix(matching): fold Swedish bank descriptors so receipts reach their purchases calculateMerchantSimilarity compared raw bank descriptors, so a receipt from "Alviks kött och fisk" scored 0.125 against the bank's own row for it, "Alviks koett och fisk K3667 Kortköp/uttag" — an öre-exact pair no threshold could reach. Adds normalizeForMatch, used for similarity only, which folds what the card rails add and never changes identity: the K#### token, Kortköp/uttag verbs, a leading "Kortköp YYMMDD", trailing /YY-MM-DD dates, reference numbers glued to the name, domain wrappers, legal forms, and the three ways banks mangle Swedish letters (ö, transliterated "oe", and ?? mojibake). Processor markers become spaces because the merchant sits before the star in GOOGLE*PLAY and after it in K*IKEA GALLE. Token-subset containment is scored level with substring containment so a receipt's legal name matches the bank's trading name. normalizeMerchantName is left byte-identical and now documents why: it is a transitive input to categorization_templates.counterparty_name, a persisted UNIQUE key with a hand-written SQL mirror the ledger-context RPC recomputes at query time. Changing it would make stored keys stop equalling computed ones, so the konteringskarta join misses and insertOrUpdateTemplate inserts a second row per merchant instead of migrating the occurrence counts. Aggressive folding is safe because it is applied to both sides of every comparison, so an over-eager fold still matches; the risk is collision between different merchants, which the new tests guard. Measured on 27 receipt/transaction pairs humans actually confirmed in production: recall 27/27, and 0/7 false positives on deliberately similar but distinct merchants. Full unit suite unchanged (13,004 passing), including the 22 string pins on the frozen key path. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com> * feat(mail): read-only Gmail connector so receipts are found without forwarding Forwarding was the only way a receipt reached Accounted, and it is both unpopular (97% of companies with the problem have never used their inbox address) and fragile: Arcim's own forward has been off for weeks and nobody noticed. This lets the hunt look in the mailbox instead. Scope is gmail.readonly and nothing else. It can search and download attachment bytes, and it structurally cannot send, modify or delete: the promise the consent screen makes is enforced by the grant, not by our code being careful. The consequence is deliberate: the agent can prepare a forward for a portal-link receipt but can never send one itself. Query-then-classify, never sync. For each unexplained purchase we run a provider-side search in a -3/+10 day window, pull metadata for a handful of hits, and keep nothing. No mailbox is mirrored and no message body is stored, which is what keeps this inside Google's Limited Use terms and GDPR data minimisation. Mail is searched only for purchases Underlag could not already explain, so a receipt we already hold never costs a mailbox read. The query ORs merchant against amount rather than requiring both: demanding both misses every rebrand and reseller (Anthropic bills as Claude), while the amount alone is a strong filter inside two weeks. mail_connections is service-role only with RLS enabled and zero policies, because the row holds a live refresh token and RLS cannot hide a column. Uniqueness is (company, provider, address) so a second mailbox is additive and a reconnect updates in place. Tokens are AES-256-GCM under their own key by preference, since a mail grant reads correspondence rather than backups. Core reaches the extension through a registered service, mirroring lib/email/service.ts, so lib/receipt-hunt never imports from @/extensions and a zero-extension build still compiles. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com> * feat(mail): connect UI and ingest, making the hunt reach into the mailbox Two halves that together make the connector usable. Ingest (lib/receipt-hunt/ingest.ts, core): fetches the attachment, files it as a document and an inbox item with source 'mail_hunt', then stages the pairing. It lives in core because it writes documents and inbox items, and an extension may never import another extension; the mail extension only ever hands over bytes. No re-matching for a hunted receipt: it was fetched WHILE SEARCHING for a specific purchase, so the pairing is known by construction. The search is a deliberately broad OR query, which is exactly why the proposal still goes to a human with the mailbox, sender and subject written on it rather than being linked automatically. Provenance goes in channel_context, never extracted_data, because retrying extraction overwrites extracted_data wholesale and the record of which mailbox a receipt came from has to survive that. A partial unique index on (company_id, channel_context->>'mail_message_id') makes re-runs and the same receipt arriving in two mailboxes idempotent, and a 23505 is treated as success rather than an error. Guards, both mutation-tested: a duplicate message costs no provider call, and an oversized attachment is skipped rather than stored. One unreadable attachment falls through to the next and never aborts a night's hunt. UI: /settings/mail lists connected mailboxes with their health, connects a new one through a user-gesture tab (opened before the await, so popup blockers do not eat it), and disconnects behind a ConfirmDialog that states the outcome up front, including that already-approved receipts stay because they belong to the bookkeeping now. Strings in sv and en; the read-only promise is spelled out on the page rather than buried in a consent screen. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com> * fix(mail): renumber migrations to clear a version collision on main 20260806150000 was already taken by preserve_preset_committed_at, and woocommerce_connections plus enforce_balance_on_posted_insert landed after this branch was cut. Two files sharing a version breaks every fresh database, which only shows up on a clean setup rather than on an already-migrated one. Applied to prod under the new versions (20260807090000 / 20260807090100), so schema_migrations matches these filenames exactly. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com> * fix(receipt-hunt): make the mailbox search actually able to find an underlag A provkörning against a real ledger returned the same seven unrelated messages for every purchase, all reporting no attachments. Three separate causes, each fixed and pinned: 1. `getMessageSummary` asked Gmail for `format=metadata`, which returns headers and omits `payload.parts` entirely. Every message therefore looked attachment-free, `bodyIsReceipt` was always true, and the `found.find(c => c.attachmentIds.length > 0)` guard in the hunt could never select anything: the feature could not file a single receipt. Gmail has no format that returns MIME structure without the body, so the body now comes down the wire; it is read for nothing and stored nowhere. 2. The bank's description is not a merchant name. "Lön Juli Jakob Överföring via internet" searched for "Juli" and matched most of the mailbox. Month names and payment-rail boilerplate are now stopwords. 3. Salary and tax runs are a company's largest outgoing rows, so they consumed the whole search budget hunting receipts that cannot exist. `canHaveEmailReceipt` skips them for the mail leg only. Deliberately narrow: a supplier invoice paid over bankgiro does arrive by mail, and an "Utlägg" reimbursement has a real receipt behind it. Measured on the same ledger: 22 hits, 0 with attachments, 0 ingestable -> 4 hits, all with attachments, 3 of 4 correct (Elgiganten, Sting, Anthropic). The fourth matched a Stockholm billing address, which is why every proposal still waits for a human. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com> * feat(receipt-hunt): let a model resolve merchants and pick the receipt The keyword hunt was failing for reasons regex tuning cannot reach, all measured against a real mailbox rather than assumed: - `from:anthropic.com` returns 0. Receipts arrive here by being forwarded, so the sender is the user, not the vendor. - The exact charged amount returns 0. The bank posts a converted SEK figure that appears nowhere in a USD receipt. - A date window around the purchase returns 0, while the same merchant search without one returns 10+. A forward is stamped when it was forwarded, sometimes months later. So the query now searches merchant names across the whole mailbox, and precision is restored by judgement rather than by syntax. Two model calls per run, both through forced tool use so the reply is a shape and not prose to be parsed: 1. `planMerchantGroups` resolves bank descriptors to merchants and merges repeats. Six Anthropic subscriptions become one search and one decision instead of six of each. 2. `assignReceipts` decides which mail, and which attachment on it, is the receipt for which charge, and says why in a sentence the reviewer reads. The attachment, not the message, is the unit of an underlag: a single forward routinely carries receipts for several purchases ("Fwd: Kvitton februari" has five). Migration 20260807103000 moves the dedupe key from message to message+attachment, with a backfill, because the old index would have silently blocked every receipt after the first in a forward. The model may not produce any number that reaches the ledger. It returns ids, a confidence and a reason; amounts, dates and the write stay in deterministic code. Its answer is validated, not trusted: an unknown message id, an invented filename or a low confidence drops the pairing, and any failed call proposes nothing at all. Every result still waits for a human. Measured on the same ledger: 0 receipts that could ever be filed -> 3 correct pairings (Elgiganten, Sting office invoice, Anthropic), each with a stated reason. The five remaining Anthropic charges are dated after 2026-06-15, when forwarding to the connected mailbox stopped; the model declined them correctly. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com> * refactor(receipt-hunt): amount first, and drop the confidence scoring Three findings from how others build this, applied. Production email search (Superhuman, Haystack 2026) reports that recall comes from loosening retrieval and letting the model filter downstream, not from tightening the query. Retrieval depth per merchant 12 -> 25, and purchases the planner cannot name a merchant for are now searched by amount alone instead of skipped: a line like "1260525758758 Europabetalning" identifies no merchant but is a real supplier payment whose invoice may carry exactly that total. Reconciliation engines weight amount far above date (Midday: 35% vs 5%) because banks post late while amounts do not drift. The Gmail query now leads with the amount and ORs the merchant, rather than dropping the amount whenever a merchant alias exists. Still an OR: a receipt billed in USD never contains the SEK figure the bank charged. The confidence score is gone entirely. Research on verbalised confidence finds it badly calibrated, clustered on round-number anchors and barely better than chance at separating a model's own right answers from its wrong ones. That matched what this ran into: the model anchored on 0.6 / 0.7 / 0.75 / 0.9, and the 0.7 threshold discarded two correct pairings. It is replaced by an observation rather than a self-assessment, whether the charged amount is actually visible in the mail, which is what a reviewer checks first and what sorts the queue. Also fixes a real defect the run exposed: the one-file-one-purchase guard only held within a merchant group, so when the planner split one landlord into "Sting" and "Kontorsplatser" both 15 000 kr charges were assigned the same invoice. A file is now claimed once per run, which is the duplicate underlag BFL forbids. Measured on the same ledger: 3 -> 5 pairings, no duplicate. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com> * refactor(receipt-hunt): harvest receipts, then pair them on the amount Splits the mailbox leg in two along the line of what each side can actually know. The model was being asked which purchase a mail belonged to. Deciding that needs the amount; the amount lives inside the PDF; a Gmail preview essentially never shows it. Measured over a real mailbox, every single pairing came back "belopp ej synligt": it was answering without the deciding evidence, which is why it declined five of six repeat subscriptions and why two correct pairings sat just under a threshold. Now it answers only what a subject, a sender and a preview line support: is this mail an underlag, and which attachment is it. Then the receipt is fetched, the extraction that already runs on document.uploaded reads its amount, date and vendor, and the pairing is the same deterministic amount-and-merchant match every other underlag goes through. Amount becomes decisive for real rather than as an instruction the model could not act on. The load-bearing fix is small: ingest now copies the extraction result onto the inbox item. The pool is read from invoice_inbox_items, so a hunted receipt with no extracted_data could never have matched anything, and the whole mail leg was quietly incapable of producing a pairing on amount. Consequences, all deliberate: - Harvesting runs BEFORE the pool is read, so a receipt found tonight is paired tonight rather than a night later. - One staging path instead of two. Mail-sourced proposals carry the same preview and confidence as every other, plus where they came from. - Deduped on the attachment filename, not on the message: the same invoice arrives as an original, a reminder and two forwards, and the old key filed "Invoice_13041840.pdf" four times over. - Capped at 8 receipts per merchant per run. Measured on the same ledger: 5 pairings attempted from thin evidence -> 16 real documents identified, each waiting on an amount it can be checked against. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com> * refactor(receipt-hunt): the model reads mail, arithmetic does the matching Collapses the mailbox leg to one model call that extracts fields, and hands every judgement back to deterministic code. Gone: resolving bank descriptors to merchant names, deciding which mail belongs to which charge, and the confidence score gating the result. Three prompts and two model calls become one, and mail-intelligence.ts drops from 450 lines to 250. What made this possible was measuring what a mail actually contains. The body was being downloaded and thrown away in favour of a 200-character snippet, and the body is where a forwarded receipt quotes its original sender and its original date. That is the purchase date, the thing whose absence forced the date window off entirely and made the old design miss five of six repeat subscriptions. It was there all along. So the model now answers only what text can support: is this an underlag, from whom, when, and for how much if the mail says so. Fields, not judgements. Everything after is arithmetic: - Retrieval is deterministic. No model decides what to search for. - Fetching is gated by worthFetching(): a stated amount is enough on its own, a vendor needs a plausible date, and a mail found by a purchase's own search is evidence in itself. That last rule is what handles a supplier the bank and the invoice name differently ("Kontorsplatser j BG" against "Stockholm Innovation & Growth AB"), which is what the deleted merchant-resolution call used to buy. - The pairing is the existing scorer, reached the same way as every other underlag: fetch, let the extraction that already runs on upload read the PDF, match on the amount. Amount is decisive in fact rather than as an instruction the model could not act on. Also adds the Swedish thousands-space amount formats to the query. Measured: the Sting invoice is findable as "15 000,00" and "15 000" and by no ungrouped form at all, so every amount search was missing them. Measured on the same ledger: 5 thin pairings -> 8 real documents, each with a vendor and a true purchase date, waiting on the amount in its own PDF. Currency is never converted to make a number agree. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com> * fix(receipt-hunt): trust the bytes, not the mail, when filing an attachment Found by the first live run, which fetched nothing and reported success. Three defects, each invisible to a dry run because a dry run never downloads anything. 1. Gmail declares a forwarded PDF as application/octet-stream, and uploadDocument validates content against the declared type, so the upload was rejected: "Filinnehållet matchar inte den angivna filtypen". Every forwarded receipt with a generic MIME type would have failed this way, silently, since ingest swallows one bad attachment to protect the rest of the run. The type is now sniffed from the magic bytes, then the filename, and only then from what the mail claimed. 2. The filename was re-derived by a second full message fetch inside fetchAttachment, which came back empty and fell back to a generic "underlag.pdf", discarding the real "2332687551.pdf" the search had already reported. The known name now wins. 3. The provkörning script imported lib/init instead of calling ensureInitialized(), so document.uploaded reached no handler and nothing was ever extracted. It also used static imports, which are hoisted and ran before .env.local was read, leaving the extraction extension unable to build a Supabase client. Both are script defects, not product defects: the cron route calls ensureInitialized() at module level as the architecture requires. The script now loads the environment first and imports dynamically. Also makes the per-run fetch cap tunable (RECEIPT_HUNT_MAX_RECEIPTS) so a pilot can be held to a couple of documents, and adds --live to the script, which is the only way it writes anything. Verified end to end against a real ledger, every link exercised for the first time: two attachments fetched from Gmail, stored with their real names and types, extraction run on both, the amount copied onto the inbox item, and the deterministic matcher pairing Elgiganten 21 639,00 kr from the PDF against the -21 639 kr card purchase at 0.85, staged into Granskning as attach_document_to_transaction. The second document, a Bolagsverket filing receipt, carries no total and correctly paired with nothing. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com> * feat(receipt-hunt): sweep a whole mailbox, and stop lending one receipt twice A backfill on a real ledger, 22 documents fetched from 172 messages. Batches the extraction (25 mails per call) so a first run on an existing company can read the whole mailbox instead of the 40 mails one call can carry, and makes the per-run caps tunable (RECEIPT_HUNT_MAX_MAILS, RECEIPT_HUNT_MAX_RECEIPTS) so a pilot can be bounded. The nightly caps stay where they are: they pace the review queue, and a backlog is a different job from a nightly tick. Two defects the backfill exposed, neither reachable from a dry run: The one-receipt-one-purchase rule only held inside a single run. `spentDocumentIds` is per-invocation, so an H&M receipt was proposed against a -358 kr purchase on one pass and a -354 kr purchase on the next, and approving both would have put the same underlag on two verifikat. A live proposal now claims its document across runs, the same way it already claimed its transaction. A document reported with no filename, on a message carrying five attachments, was not an answer but a shrug: the caller fetched attachment number one and hoped. Those are dropped now. A body-only receipt, where there is nothing to choose between, still passes. Measured after the sweep: 21 of 22 documents read correctly, and the binding constraint on this ledger is no longer retrieval but currency. Ten receipts are in SEK and five of those pair on the amount; twelve are in USD or EUR, where the bank charged a converted figure that appears nowhere in the receipt, so no comparison is possible and none is attempted. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com> * feat(mail): show the provider's own mark on the mailbox settings page Someone connecting a mailbox is picking an account at a provider, and the provider's mark is how they recognise which one. A generic envelope glyph said "mail" when the question is "whose". The Google "G" already existed, drawn inline inside GoogleAuthButton for the sign-in flow. It moves to components/ui/provider-marks so there is one definition rather than two, and a Microsoft square joins it for the Graph connector. Both stay inline: no external host is contacted for an icon before anyone has agreed to anything. These are the only coloured glyphs in an achromatic interface, which is deliberate rather than an oversight. A brand mark is identity, not chrome, and Google's terms require its mark unaltered rather than tinted to match a palette. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com> * fix(archive): drop the duplicate mail_connections exclusion left by the rebase Main added the table to ARCHIVE_EXCLUDED_TABLES while this branch was open, so rebasing produced the key twice and the zero-extension build failed to type check. Main's entry stays, in its alphabetical place, and keeps the sentence that answers the retention question: the grants are not räkenskapsinformation, but the receipts they find are archived as documents. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com> * fix(mail): record who disconnected a mailbox, without keeping the token Raised by the compliance review: disconnect() hard-deleted the row with no trace, and which mailboxes feed underlag into the books is a control over how räkenskapsinformation is produced (BFNAR 2013:2 kap 8), so switching one off should be reconstructable years later. Written by hand rather than by the write_audit_log trigger the accounting tables use. That trigger copies the whole row into audit_log, which here would mean copying an encrypted refresh token into a second table and keeping it after the entire point of the delete was to destroy it. The sibling credential table shopify_connections omits the trigger for the same reason. Only the address and provider are recorded, pinned by a test that fails if a credential ever reaches the audit entry. The review's two other flags were checked rather than assumed: nothing purges mail_hunt documents, and categorize-core.ts:403 does carry the attached document onto the verifikat when the transaction is booked. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com> * fix(mail): bound every outbound call, and stop the token widening itself Four findings from the review, each checked against the code first. Neither the Gmail API nor Google's token endpoint had a deadline. Both are awaited inside Promise.all across mailboxes, so one stalled request held the whole company's hunt open until the platform killed the run. Both now carry a 15s AbortSignal, which turns a stall into one mailbox missing from tonight's sweep. `include_granted_scopes: 'true'` let Google fold scopes this app was granted elsewhere into the token issued for a mailbox, so a grant could carry more authority than the consent screen showed. Removed, and pinned by a test asserting the parameter is absent. disconnect() ignored both statement results: a failed delete still wrote an audit entry claiming the mailbox was disconnected while the credential was live, and a failed audit insert passed silently. The delete now throws, so the entry is never written for a delete that did not happen. The audit failure is logged rather than rolled back: the two can now only diverge one way, credential gone and note missing, and recreating a credential to keep them in step would be worse than a missing note. The fifth finding is real and stays open by choice, recorded in DECISIONS.md: the cron still passes searchMail=false. A sweep of one 172-message mailbox took over 600s against a maxDuration of 300, so enabling the mailbox leg nightly would time out mid-run. That flag and RECEIPT_HUNT_COMPANY_IDS get flipped together once the per-company budget is measured. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com> * fix(receipt-hunt): file each attachment under its own identity Four more findings from the review. The first is a real defect. ingestMailCandidate loops over candidate.attachmentIds, but the dedupe key, the mail_attachment_id provenance and the filename were all read from index 0. Storing the second attachment therefore recorded the first one's key and name, which mislabels the row and, because the key is unique, permanently blocks the first attachment from ever landing. Masked today only because the hunt narrows to a single attachment before calling in, so nothing in the current path exercises it. All three now come from the attachment actually being stored, and the duplicate pre-check moved inside the loop so trying a second attachment is not suppressed by the first already being filed. Mutation-tested. The per-run fetch key was the bare filename, which is not an identity: "invoice.pdf" is what half the world's billing systems attach, so a second supplier's invoice would be dropped as a duplicate of the first. Scoped by vendor as well, keeping the behaviour it was written for, one fetch for an invoice that arrives as an original, a reminder and two forwards. Adds tests/pg/mail-hunt-file-dedupe.pg.test.ts for the new unique index: five attachments from one forward all land, the same attachment is refused twice, two companies hold the same file independently, other inbox sources are untouched by the partial predicate, and the message-scoped predecessor is gone. Written against CI's Postgres; there is no local DATABASE_URL here, so CI is what exercises it. --live now refuses unless RECEIPT_HUNT_CONFIRM names the same company. The script writes to whatever .env.local points at, which for this repo is production, and a recalled command should not be able to fire it. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com> * fix(test): cast the jsonb parameter so Postgres can type it pg-real could not determine the type of $3 inside jsonb_build_object. An explicit ::text is what the other pg tests do. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com> --------- Co-authored-by: Jakob Wennberg <311770904+jakobwennberg-oss@users.noreply.github.com> Co-authored-by: Claude Opus 5 (1M context) <noreply@anthropic.com>	2026-08-10 12:23:42 +02:00