2eb34412442ade678cc11b6bb15c48ea4c4bb63f
5
Commits
| Author | SHA1 | Message | Date | |
|---|---|---|---|---|
 MattssonandClaude Fable 5
|
4a9fa5e6c5 |
feat(inbox): staged upload ack, HEIC/HEIF validation, WhatsApp silence fixes (#1605)
* fix(whatsapp): app-side unmute, close silent intake paths, health visibility - add POST /link/unmute and a Reactivate control on the Pausad state - company resolution: transient query errors release the row for sweep retry; genuine zero-options sends M19 instead of parking silently - media from unlinked senders bypasses the hourly greeting throttle (10 min burst window, daily cap kept) - GET /link returns 7-day failed-delivery and parked-inbound counts; sweep summary logs outboundFailed24h Co-Authored-By: Claude Fable 5 <noreply@anthropic.com> * fix(documents): real HEIC/HEIF magic-byte validation, bilingual upload errors - detect ISO-BMFF ftyp brands (heic/heix/heim/heis/hevc/hevx/hevm/hevs, mif1/msf1) instead of exempting image/heic from validation; declared heic/heif accepts either family member (iOS labels vary) - new INBOX_UPLOAD_* structured error codes replace raw English strings on the inbox upload and attach-document routes - registry doc corrected to the real 10 MB cap Co-Authored-By: Claude Fable 5 <noreply@anthropic.com> * feat(inbox): staged upload with instant ack and deferred AI extraction - web uploads insert the inbox item as status processing and respond immediately; Bedrock extraction and supplier match run via after() with a CAS flip to received (email and WhatsApp channels keep the synchronous path) - widen invoice_inbox_items.status CHECK to include processing (migration 20260813180000, pg-real test included) - crash-recovery sweep cron (*/2) flips stale processing rows; bulk-book skips extraction_in_progress items - workspace: processing chip, in-flight rows disable actions, realtime flip, retry-extraction button for empty extractions - picker accept list drops HEIC/HEIF so iOS transcodes library photos to JPEG; server allowlists unchanged (supersedes 2026-08-01 HEIC decision, see DECISIONS.md) Co-Authored-By: Claude Fable 5 <noreply@anthropic.com> * fix(migrations): bump inbox processing-status migration past main's latest Main merged 20260813210000 while this PR was in flight; an inserted version older than the latest applied aborts the prod db push at merge. Renamed 20260813180000 to 20260813213000 and updated references. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com> * docs(decisions): log preview-tracker orphan repair after migration rename Co-Authored-By: Claude Fable 5 <noreply@anthropic.com> --------- Co-authored-by: Claude Fable 5 <noreply@anthropic.com> |
||
|
|
845add4573 |
feat(documents): dedupe intake channels on content, not just provenance (#1528)
* feat(documents): dedupe intake channels on content, not just provenance Every ingestion path already computes and stores sha256_hash, but only WhatsApp ever read it back: the manual upload, Resend inbound, and mail hunt deduped on provenance keys alone (or not at all), so the same receipt forwarded to two inboxes, re-hunted by a sweep, or uploaded twice became a second archived document and a second inbox item. With the hunt live and three channels feeding one inbox, that is an unbounded duplicate generator (flows plan, prerequisite PR 1). uploadDocument gains an opt-in dedupeByContent flag: before storing, it looks for a current-version document in the same company with the same SHA-256 and returns it (marked deduplicated) instead of archiving a copy. Opt-in because archival callers must store what they produced even when bytes repeat; the SELECT-then-insert race is accepted exactly as in the WhatsApp intake precedent. uploadAndExtract turns the flag on for every inbox channel. On a hit it adopts the oldest inbox item for that document, so callers always receive a real inbox_item_id, and only files a new item (against the EXISTING document) when the content entered the archive outside the inbox. The mail hunt skips outright: its provenance key catches the same message re-hunted, the content check catches the same receipt arriving through another inbox. WhatsApp keeps its own pre-check, which also drives the duplicate reply to the sender. No migration: the hash column and its index have existed since the original archive schema. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com> * fix(documents): review round: fail closed, adopt-or-file in the hunt, audit trail CodeRabbit: both dedupe lookups failed OPEN, so a transient DB error would silently archive the duplicate the feature exists to prevent; both now throw before anything is stored, and a regression test locks it. The ingest test also asserts the dedupeByContent flag in the production call, so removing the flag fails the suite. Swedish compliance review, both findings real: (1) the mail hunt's unconditional skip could swallow a receipt whose content matches a document that never passed the inbox (a manually attached copy), leaving an affärshändelse without underlag routing (BFL 5 kap): the hunt now mirrors the funnel's adopt-or-file semantics, skipping only when an inbox item already carries the document and otherwise filing an item against the EXISTING document. (2) The skip decision now lands in behandlingshistorik as DocumentDuplicateSkipped (BFNAR 2013:2 kap 8), not just the app log. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com> * fix(receipt-hunt): keep the audit payload pseudonymous; lock the skip trail in tests Review round 2. The DocumentDuplicateSkipped payload carried the mailbox address, violating the processing-history contract (pseudonymous IDs only, never emails); the digit-shaped PII validator would not have caught it, which is exactly why the contract must hold at the call site. Which mailbox first delivered the receipt is already on the existing item's channel_context. Tests now assert the audit event lands with the right identifiers and no address, and that a history outage still skips rather than filing a duplicate. Not changed: a duplicate-lookup error still soft-fails the attachment (warn + continue). Aborting the candidate would contradict this function's documented contract (one bad message never costs the night's hunt); fail-closed holds either way, and the next sweep retries since no item was filed. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com> --------- Co-authored-by: Jakob Wennberg <311770904+jakobwennberg-oss@users.noreply.github.com> Co-authored-by: Claude Fable 5 <noreply@anthropic.com> |
||
|
MattssonandClaude Fable 5
|
9dbaebcc50 |
fix(invoices): ROT/RUT credit notes; verifikat amount sort, HTML underlag, source chip (#1523)
* feat(invoice-inbox): store HTML mails as underlag, expandable field editor Body-only mails and .html attachments (including forwarded .eml bodies) no longer dead-end as "Fel vid bearbetning": the mail body is wrapped into a self-contained text/html document, stored through the normal upload/extract pipeline, and extracted via a new HTML-to-text Bedrock path, so the mail itself can serve as bookable underlag. Empty mails keep the error row, unsupported types are still rejected, and webhook retries dedupe on resend_email_id. Mail HTML is attacker-controlled, so rendering is fully sandboxed: iframe sandbox in the workspace preview and a CSP sandbox header on /api/documents/:id/inline for text/html. The type is accepted only from the email pipeline (EMAIL_ALLOWED_MIME_TYPES), never from manual upload. The "Extraherade falt" rail gains an expand button opening a centered dialog with the same autosaving field editor at a readable size (two columns), which also gives every failed or skipped extraction a manual fallback. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com> * feat(bookkeeping): sortable verifikat list headers with amount sort - clickable sort toggles on the verifikat list headers (asc -> desc -> default) - total_amount computed column + sort_by total/description on the list route - failed list loads render an error card with retry, never the empty-ledger state Co-Authored-By: Claude Fable 5 <noreply@anthropic.com> * fix(import): decode bank CSV as Windows-1252 fallback in column mapping The client read the uploaded file with file.text(), which is UTF-8-only, so Windows-1252 exports (e.g. Handelsbanken) rendered and re-parsed with U+FFFD in place of Swedish characters. Decode from bytes with the shared decodeFileContent() helper, matching what the server parse route does. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com> * feat(bookkeeping): stackable sort keys on verifikat list headers - shift-click adds a column as secondary/tertiary sort key (max 3), plain click keeps the single-key tri-state cycle - sort_by accepts a comma-separated priority list; single tokens stay valid - voucher tiebreak follows the last key's direction (#972 parity) - priority numbers on stacked headers; hint text in the filter dialog Co-Authored-By: Claude Fable 5 <noreply@anthropic.com> * fix(invoices): keep ROT/RUT deduction fields positive on credit notes Crediting an invoice with a ROT/RUT deduction failed 100% of the time: the credit-note path negated deduction_total (and per-item deduction_amount) like the other amounts, but both columns carry CHECK (>= 0), so Postgres rejected the insert and the user only saw 'Kunde inte skapa kreditfaktura'. Store the deduction fields as positive magnitudes, matching the convention everywhere else. The stored sign is inert on credit notes: the reversing verifikat recomputes the ROT/RUT split from the items, and the PDF and amount-to-pay logic skip deductions on credit notes. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com> * refactor(transactions): share the source chip across inbox and history modes Move SourceFilter to transaction-types.ts (widened with 'bank:other' and 'acct:<id>'), render the one toolbar ContextPicker in both view modes, and drop the narrower duplicate chip inside TransactionHistoryList. The history list now applies the acct:/bank:other narrowing itself and hides skattekonto rows under any bank-side selection. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com> * chore(deps): bump js-yaml to 4.3.1 Co-Authored-By: Claude Fable 5 <noreply@anthropic.com> * test(schema): recognize PostgREST computed columns in the migration parser The verifikat amount sort orders by total_amount, a PostgREST computed column (a function on the journal_entries row type, migration 20260811100000). The schema guard only modeled real columns, so no-phantom-columns flagged the order as a phantom. Teach the parser that a function whose only argument is a table's row type joins that table's column set, with DROP FUNCTION retraction when the signature names the row type. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com> * fix: resolve PR #1523 review findings - journal-entries route: append the globally unique id tiebreak to every direct-query sort; voucher series+number repeat across fiscal years, so the all-years scope could duplicate or drop rows at page boundaries. Existing order assertions updated, new all-years tiebreak test. - documents inline route: CSP source policy on HTML previews; sandbox alone still loads remote resources, letting a tracking pixel notify the sender on open. New route test asserts the full header. - JournalEntryList: catch rejected list requests so loading cannot stick forever, and gate every post-await state write behind a request generation so a slow earlier request cannot overwrite the current sort. - TransactionHistoryList: pagination follows the selected source scope (reachable with zero matches on the current page, hidden for the skattekonto scope it cannot affect). - transactions page: bank:other picker availability derives from history rows too, not only the pending inbox dataset. - DECISIONS.md: mark the superseded single-sort decision; record the credit-note deduction positive-magnitude invariant and its verified reader inventory (Swedish review flag). Co-Authored-By: Claude Fable 5 <noreply@anthropic.com> * fix: guard metadata refetches behind the list request generation fetchAttachmentCounts and fetchRattelseFlags write state after their own awaits; a stale list request's late completion could overwrite attachment counts and rattelse flags for rows a newer request just rendered, showing false missing-underlag warnings. Both helpers now take the caller's generation guard and discard stale completions, including the attachment-counts loaded flag. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com> --------- Co-authored-by: Claude Fable 5 <noreply@anthropic.com> |
||
|
|
398c734b93 |
feat(whatsapp-inbox): intake extension with webhook, phone linking and receipt ack (#1338)
Webhook lifecycle: GET hub.challenge handshake (constant-time verify-token compare); POST verifies X-Hub-Signature-256 over the RAW body before any parse, Zod-parses the envelope, persists inbound rows (partial-unique wamid = dedupe against Meta's up-to-7-day redelivery), acks 200 fast and defers media processing via the after() idiom. Rejected and rate-limited content always acks 200 and lands as skipped/error rows, never a retryable status. Linking: the settings panel (Installningar -> WhatsApp) mints AC- one-time codes (sha256 stored, 10 min TTL, single use, ambiguity-free alphabet); the webhook consumes the code, binds phone to user (HMAC-peppered hash + AES-256- GCM at rest) and confirms with M3. Keyword commands stopp/start/hjalp; unknown senders get one throttled M1 greeting (1/h, 3/day) behind the sender-quota RPC, with no media download and no content persistence. Intake worker: atomic claim on the message row (the durable job record), company resolution (default -> sole membership -> M6 fallback, no item), per-company inbox quota (ack-and-drop, M17 once per 10 min per sender), MIME allowlist, 10 MB stream-checked media download, exact sha256 duplicate check, then the shared uploadAndExtract funnel (source 'whatsapp', channel_context caption, whatsapp_message_id) and the M4 ack with extracted merchant/total/date. Failures wrap to 'error' + error_message + one M18. uploadAndExtract widened: source 'whatsapp', optional channelMeta + actorId; email/upload paths behaviorally unchanged. Deferred to PR4: burst debounce + combined ack (M5), in-chat company choice (M6 buttons + 8h pin), clarifying questions M7-M10, interpret-answer LLM call, sweep cron, retention cron. Co-authored-by: Jakob Wennberg <jakob.wennberg@gmail.com> Co-authored-by: Claude Fable 5 <noreply@anthropic.com> |
||
|
|
02e48efe11 |
refactor(invoice-inbox): export uploadAndExtract from lib (#1336)
Pure move of uploadAndExtract and its private helpers (sanitisers, page-count/slice, sandbox check, MIME/size consts) from index.ts into lib/upload-and-extract.ts so a future channel extension can import the shared funnel the way document-extraction already imports extract-invoice-fields. No behavior change; existing tests prove it. Co-authored-by: Jakob Wennberg <jakob.wennberg@gmail.com> Co-authored-by: Claude Fable 5 <noreply@anthropic.com> |