diff --git a/DECISIONS.md b/DECISIONS.md index ae93eb23..7fe97840 100644 --- a/DECISIONS.md +++ b/DECISIONS.md @@ -1533,6 +1533,7 @@ One line per decision: `[YYYY-MM-DD] : `. Appended by agents and [2026-09-03] SCB name search is a picker, never a lookup: the user chooses among SCB's matches and the chosen org number is recorded as a fact with source user before any fetch; one match is shown, not auto-picked, because a trade name is not an identity (Adobe Systems Software resolves to an Irish entity and a Swedish one) [2026-09-03] AGI redovisningsperiod = the payout month (agiReportingPeriod on payment_date), not salary_runs.period_*: Skatteverket files per the month the pay went out (kontantprincipen), so lön i efterskott (August work paid 25 September) is declared in September. The in-period payment-date guard (dashboard PATCH, lib/salary/update-run.ts, v1 PATCH, RunHeader min/max) is lifted rather than widened: its only stated reason was that the AGI keyed on period_*, and any residual month window would bite the next efterskott variant. Existing agi_declarations rows keep their stored period (no backfill): a declaration already filed under the earned month is a real-world correction with Skatteverket, not a re-key. New AGI_PERIOD_CONFLICT (409) refuses to overwrite a live run's declaration for the same payout month, since one month's AGI must cover every payment that month and the generator cannot merge runs. Issue #2191. [2026-09-03] The cursor:// deeplink is its own allowlist provider (cursor_deeplink) rendered "Din egen dator" and never "Verifierad", after the skeptic, CodeRabbit and Superagent all made the same point: a custom scheme can be claimed by any local app (RFC 8252 section 8.4), so it carries loopback trust, not vendor trust, and the consent page must not say otherwise; https://www.cursor.com/... keeps the verified label. Same pass fixed the consent-page CSP for custom schemes: new URL('cursor://...').origin is the string "null", so form-action became `'self' null` and Chromium would have blocked the post-consent 303 (correctness skeptic refutation); the header now uses the scheme-source (`cursor:`) when the origin is opaque. Not done: rejecting a missing code_challenge at /authorize. A code minted without one is unexchangeable (verifyPkce against an empty challenge is always false, now pinned by a test), so it is fail-closed; making it fail earlier is a separate change touching every client. +[2026-09-03] Inbound mail (#2181): a mail that names the same inbox as both +lev and +ver files once with kind_hint null (extraction classifies) and the conflict is recorded on the InboundMailReceived history event, rather than letting +lev win: the sender said two things, and a silent pick would be indistinguishable from the tag being ignored. The per-attachment catch in the webhook now writes an error row instead of only a console line: prod showed a +lev mail Resend accepted at 14:52Z with no inbox row at all, and the reporter's +ver mail lost its second PDF the same way. Dedupe on (email, attachment) became per company (index replaced in 20260904001000) so one mail to two inboxes files once per inbox. The Inkomna mejl panel defaults to 30 days. After the skeptic pass: the InboundMailReceived payload carries inbox_id, the documented tags and an unknown-tag count, never the address or a sender-typed tag (the local part of an enskild firma's inbox is the owner's name, and a numeric tag trips the PII validator so the record would be lost); the DB strip trigger from 20260901110000 now covers the new type; and a catch-path error row is marked transient so a Resend redelivery replaces it instead of reporting a duplicate, keeping the self-heal main had. Review pass on PR #2244: fan-out capped at five processed inboxes per mail with every addressed inbox still resolved and the ones past the cap recorded as fan_out_capped on their own InboundMailReceived event (Superagent asked for the bound, CodeRabbit and the Swedish review for not losing the trace), the history route reports has_more past 200 rows (CodeRabbit), and a replaced transient row is named on the record (Swedish review, BFL 5 kap 5 § trace). Declined: resolving verified custom-domain recipients alongside a shared one (the plan keeps today's precedence, the feature is gated off with zero verified domains on prod, and the existing test pins it), and CONCURRENTLY index ops (Supabase branching runs migrations in a transaction; same call as 20260706120000). [2026-09-03] Jämkning valid_to is required on every write path (web, v1, MCP staging and executors, both Zod schemas) through one shared validator (lib/salary/jamkning-rules.ts), closing #2058: the engine never applies a beslut without an end date, so the previously accepted percentage+valid_from shape stored an inert beslut behind a 200. Declined the alternative of defaulting valid_to to 31 December of the from-year: it matches most beslut but silently changes withholding on rows that today do nothing. Existing incomplete rows are listed by scripts/list-incomplete-jamkning.ts and decided per company (set an end date or clear the beslut), since either changes the next payslip. [2026-09-03] The Lucide Building2 glyph is retired app-wide (founder, from the register walkthrough). Suppliers use Truck, companies and company-scoped things use Briefcase, banks use Landmark; the extension manifest icon name changed with it [2026-09-03] settleInvoicePayment writes the invoice_payments row BEFORE the CAS status update and removes it in both failure branches, instead of inserting after the update: the kontantmetod cut-off reads invoice_payments only, so a paid invoice without a row is the #2019 defect itself; failing closed on the insert (voucher storno + INVOICE_PAID_BOOK_FAILED) keeps GL, sub-ledger and invoice status in step. The #2019 backfill inserts only where exactly one posted payment voucher exists (invoice_paid / invoice_cash_payment with source_id = invoice); zero or several vouchers are reported, never guessed, and every row is tagged backfill:#2019 in notes so one DELETE reverts a run. diff --git a/components/extensions/general/InvoiceInboxWorkspace.tsx b/components/extensions/general/InvoiceInboxWorkspace.tsx index e654c7e0..643004b8 100644 --- a/components/extensions/general/InvoiceInboxWorkspace.tsx +++ b/components/extensions/general/InvoiceInboxWorkspace.tsx @@ -47,7 +47,7 @@ import { Globe, } from 'lucide-react' import Link from 'next/link' -import { cn, formatCurrency, formatDate, formatDateLong } from '@/lib/utils' +import { cn, formatCurrency, formatDate, formatDateLong, formatDateTime } from '@/lib/utils' import { QUIET_LINK_CLASS, CHECKBOX_REVEAL_CLASS } from '@/components/ui/dry-table' import { useRangeSelect } from '@/lib/hooks/use-range-select' import { GoogleMark, MicrosoftMark } from '@/components/ui/provider-marks' @@ -222,6 +222,38 @@ interface InboxAddress { status: string } +// One received mail per inbox, from the InboundMailReceived history event +// the inbound webhook appends (#2181). Sender, subject and address are +// deliberately absent from the event (processing_history is outside the +// erasure path); the route resolves inbox_id to the company's own address +// at read time, and the filed item ids are what the panel links to. +interface InboundMailAttachment { + id: string + outcome: 'filed' | 'duplicate' | 'rejected' | 'failed' + inbox_item_id?: string + reason?: string + mime?: string +} +interface InboundMail { + event_id: string + email_id: string + occurred_at: string + inbox_id: string | null + custom_domain: boolean + tags: string[] + unknown_tag_count: number + inbox_local_part: string | null + inbox_status: string | null + kind_hint: string | null + tag_conflict: boolean + outcome: string + attachment_count: number + inbox_item_id: string | null + attachments: InboundMailAttachment[] +} +// Window for the received-mail panel; the route caps at 365. +const INBOUND_MAIL_DAYS = 30 + // `acme-x7f2@inbox.example` + 'lev' → `acme-x7f2+lev@inbox.example`. The // webhook splits the local part at the first `+` and looks up what is before // it, so the tag never changes which company the mail reaches. @@ -632,6 +664,29 @@ export default function InvoiceInboxWorkspace(_props: WorkspaceComponentProps) { const [mailConnections, setMailConnections] = useState([]) const [whatsapp, setWhatsapp] = useState<{ linked: boolean; phoneMasked?: string; verifiedAt?: string | null } | null>(null) const [sourcesOpen, setSourcesOpen] = useState(false) + // Received-mail history (#2181): read when its panel is first opened, so + // the sources strip costs nothing for people who never look. + const [inboundMails, setInboundMails] = useState(null) + const [inboundMailsFailed, setInboundMailsFailed] = useState(false) + // The route caps the list; when the window held more, say so rather than + // let "every mail" stand over a list that is missing the oldest ones. + const [inboundMailsTruncated, setInboundMailsTruncated] = useState(false) + + const fetchInboundMails = useCallback(async () => { + try { + const res = await fetch( + `/api/extensions/ext/invoice-inbox/inbound-history?days=${INBOUND_MAIL_DAYS}`, + ) + if (!res.ok) throw new Error(`inbound-history ${res.status}`) + const { data } = await res.json() + setInboundMails(Array.isArray(data?.mails) ? data.mails : []) + setInboundMailsTruncated(data?.has_more === true) + setInboundMailsFailed(false) + } catch (err) { + console.error('[invoice-inbox] fetchInboundMails failed:', err) + setInboundMailsFailed(true) + } + }, []) useEffect(() => { void (async () => { @@ -803,6 +858,11 @@ export default function InvoiceInboxWorkspace(_props: WorkspaceComponentProps) { return counts }, [statusFilteredItems]) + // Rows the type menu is hiding right now (#2181): a +lev mail filed as a + // leverantörsfaktura is invisible under Underlag, and the trigger's count + // alone does not say that anything is missing. + const hiddenByKindFilter = kindFilter === 'all' ? 0 : kindCounts.all - kindCounts[kindFilter] + // The type menu only earns its row once something is classified (or the // user has already narrowed): an inbox of unclassified rows has nothing to // split. @@ -1543,6 +1603,61 @@ export default function InvoiceInboxWorkspace(_props: WorkspaceComponentProps) { )} + {/* Every mail that reached the address (#2181), whatever became of + it: filed, duplicate, rejected, failed. This is where "I mailed + it and it is not there" gets an answer instead of a shrug. */} + {inboxAddress && ( +
{ + if (e.currentTarget.open && inboundMails === null && !inboundMailsFailed) { + void fetchInboundMails() + } + }} + > + + + {t('inbound_mail_title')} + {inboundMails !== null && ( + {inboundMails.length} + )} + + +
+

+ {inboundMailsTruncated && inboundMails + ? t('inbound_mail_truncated', { count: inboundMails.length, days: INBOUND_MAIL_DAYS }) + : t('inbound_mail_hint', { days: INBOUND_MAIL_DAYS })} +

+ {inboundMailsFailed ? ( + { void fetchInboundMails() } }} + > + {t('inbound_mail_load_failed')} + + ) : inboundMails === null ? ( + + + {t('inbound_mail_loading')} + + ) : inboundMails.length === 0 ? ( +

{t('inbound_mail_empty', { days: INBOUND_MAIL_DAYS })}

+ ) : ( +
    + {inboundMails.map((mail) => ( + + ))} +
+ )} +
+
+ )} + {mailConnections.map((c) => (
@@ -1878,6 +1993,22 @@ export default function InvoiceInboxWorkspace(_props: WorkspaceComponentProps) { ))} )} + {/* The type menu hides rows without saying so (#2181): a +lev mail + is a leverantörsfaktura and does not show under Underlag. Say + how many, with the one click that brings them back. The empty + state above already says it when nothing is left. */} + {filter !== 'missing' && filter !== 'portal' && filteredItems.length > 0 && hiddenByKindFilter > 0 && ( +
+ {t('hidden_by_kind_filter', { count: hiddenByKindFilter })}{' '} + +
+ )} {/* Document preview (hero). When the inbox is empty there is nothing @@ -2151,6 +2282,78 @@ export default function InvoiceInboxWorkspace(_props: WorkspaceComponentProps) { // mounted. No toast on any path, so nothing here can be evicted by (or evict) // another toast under TOAST_LIMIT = 1. +function InboundMailRow({ + mail, + domain, + onOpenItem, +}: { + mail: InboundMail + /** The shared inbound domain, from the company's own address. */ + domain: string + onOpenItem: (id: string) => void +}) { + const t = useTranslations('inbox_workspace') + // The address is reconstructed from the inbox row, never read from the + // event: one line per tag the mail used, or the bare address. + const tags = mail.tags ?? [] + const address = mail.custom_domain + ? t('inbound_mail_custom_domain') + : mail.inbox_local_part + ? (tags.length > 0 ? tags : [null]) + .map((tag) => `${mail.inbox_local_part}${tag ? `+${tag}` : ''}@${domain}`) + .join(', ') + : t('inbound_mail_former_address') + const counts = { filed: 0, duplicate: 0, rejected: 0, failed: 0 } + for (const a of mail.attachments ?? []) { + if (a.outcome in counts) counts[a.outcome] += 1 + } + const parts: string[] = [] + if (mail.outcome === 'rate_limited') parts.push(t('inbound_outcome_rate_limited')) + else if (mail.outcome === 'no_attachments') parts.push(t('inbound_outcome_empty')) + else if (mail.outcome === 'email_body') parts.push(t('inbound_outcome_body')) + else if (mail.outcome === 'email_body_duplicate') parts.push(t('inbound_outcome_body_duplicate')) + else if (mail.outcome === 'fan_out_capped') parts.push(t('inbound_outcome_fan_out_capped')) + else { + if (counts.filed > 0) parts.push(t('inbound_outcome_filed', { count: counts.filed })) + if (counts.duplicate > 0) parts.push(t('inbound_outcome_duplicate', { count: counts.duplicate })) + if (counts.rejected > 0) parts.push(t('inbound_outcome_rejected', { count: counts.rejected })) + if (counts.failed > 0) parts.push(t('inbound_outcome_failed', { count: counts.failed })) + } + const hasFailure = + mail.outcome === 'rate_limited' || mail.outcome === 'fan_out_capped' || counts.rejected > 0 || counts.failed > 0 + // Every row the mail produced, in attachment order, each a click away. + const openable: string[] = [] + if (mail.inbox_item_id) openable.push(mail.inbox_item_id) + for (const a of mail.attachments ?? []) { + if (a.inbox_item_id && !openable.includes(a.inbox_item_id)) openable.push(a.inbox_item_id) + } + return ( +
  • +
    + {formatDateTime(mail.occurred_at)} + {address} + {(mail.unknown_tag_count ?? 0) > 0 && ( + {t('inbound_unknown_tags', { count: mail.unknown_tag_count })} + )} +
    +
    + {parts.join(', ')} + {openable.map((id, i) => ( + + ))} +
    + {mail.tag_conflict && {t('inbound_tag_conflict')}} +
    • + ) +} + function InboxAddressBar({ address, onRotate, diff --git a/extensions/general/invoice-inbox/__tests__/inbound-history-payloads.test.ts b/extensions/general/invoice-inbox/__tests__/inbound-history-payloads.test.ts index 40897469..b7768f33 100644 --- a/extensions/general/invoice-inbox/__tests__/inbound-history-payloads.test.ts +++ b/extensions/general/invoice-inbox/__tests__/inbound-history-payloads.test.ts @@ -37,6 +37,7 @@ vi.mock('@/extensions/general/invoice-inbox/lib/upload-and-extract', async () => ) return { ...actual, uploadAndExtract: vi.fn() } }) +import { uploadAndExtract } from '@/extensions/general/invoice-inbox/lib/upload-and-extract' vi.mock('@/lib/rate-limits/inbox', () => ({ checkInboxUploadRateLimit: vi.fn(), @@ -181,6 +182,49 @@ describe('POST /inbound behandlingshistorik payloads', () => { expect(JSON.stringify(payload)).not.toContain(SUBJECT) }) + it('records InboundMailReceived with ids and a closed vocabulary only: no sender, subject, address or sender-typed tag', async () => { + const to = [ + 'Anna-Andersson-x7f2+lev@arcim.io', + 'anna-andersson-x7f2+ref-19850101-1234@arcim.io', + ] + vi.mocked(verifyInboundWebhook).mockReturnValue({ + ...mockReceivedEvent(0), + data: { ...mockReceivedEvent(0).data, to }, + } as never) + vi.mocked(fetchReceivingEmail).mockResolvedValue({ ...mockFullEmail(0), to, html: '

    Faktura

    ' } as never) + vi.mocked(checkInboxUploadRateLimit).mockResolvedValue({ ok: true } as never) + vi.mocked(uploadAndExtract).mockResolvedValue({ inbox_item_id: 'item-body-1' } as never) + + const { supabase, enqueue } = createQueuedMockSupabase() + enqueue({ data: { id: 'inbox-1', company_id: 'company-1', status: 'active' } }) + enqueue({ data: { created_by: 'user-owner-1' } }) + enqueue({ data: null }) // body-document dedupe check + vi.mocked(createClient).mockReturnValue(supabase as never) + + const res = await webhookRoute.handler(createMockRequest('/inbound', { method: 'POST', body: {} })) + expect(res.status).toBe(200) + + const payload = historyPayload('InboundMailReceived') + expect(payload).toEqual({ + inbox_id: 'inbox-1', + custom_domain: false, + tags: ['lev'], + unknown_tag_count: 1, + kind_hint: 'supplier_invoice', + tag_conflict: false, + outcome: 'email_body', + attachment_count: 0, + inbox_item_id: 'item-body-1', + attachments: [], + }) + const json = JSON.stringify(payload) + expect(json).not.toContain(SENDER) + expect(json).not.toContain(SUBJECT) + expect(json).not.toContain('@') + expect(json).not.toContain('anna') + expect(json).not.toContain('19850101') + }) + it('keeps the mail traceable through the Resend email id', async () => { vi.mocked(verifyInboundWebhook).mockReturnValue(mockReceivedEvent(1) as never) vi.mocked(fetchReceivingEmail).mockResolvedValue(mockFullEmail(1) as never) diff --git a/extensions/general/invoice-inbox/__tests__/inbound-history-route.test.ts b/extensions/general/invoice-inbox/__tests__/inbound-history-route.test.ts new file mode 100644 index 00000000..d460e794 --- /dev/null +++ b/extensions/general/invoice-inbox/__tests__/inbound-history-route.test.ts @@ -0,0 +1,138 @@ +import { describe, it, expect, vi, beforeEach } from 'vitest' +import { invoiceInboxExtension } from '@/extensions/general/invoice-inbox' +import { createQueuedMockSupabase, createMockRequest, parseJsonResponse } from '@/tests/helpers' +import type { ExtensionContext } from '@/lib/extensions/types' + +vi.mock('@/lib/rate-limits/inbox', () => ({ + checkInboxUploadRateLimit: vi.fn().mockResolvedValue({ ok: true }), +})) + +const route = invoiceInboxExtension.apiRoutes!.find( + (r) => r.method === 'GET' && r.path === '/inbound-history' +)! + +function buildCtx(supabase: unknown): ExtensionContext { + return { + userId: 'user-1', + companyId: 'company-1', + extensionId: 'invoice-inbox', + supabase: supabase as ExtensionContext['supabase'], + emit: vi.fn(), + settings: { get: vi.fn(), set: vi.fn() }, + storage: { from: vi.fn() } as unknown as ExtensionContext['storage'], + log: { info: vi.fn(), warn: vi.fn(), error: vi.fn(), debug: vi.fn() } as unknown as ExtensionContext['log'], + services: {}, + } as unknown as ExtensionContext +} + +const EVENT_ROW = { + event_id: 'evt-1', + correlation_id: '6ec3164f-1f59-40e7-acd2-ebedf9c4d56e', + occurred_at: '2026-09-02T14:21:56Z', + payload: { + inbox_id: 'inbox-1', + custom_domain: false, + tags: ['ver'], + unknown_tag_count: 0, + kind_hint: 'receipt', + tag_conflict: false, + outcome: 'attachments', + attachment_count: 2, + inbox_item_id: null, + attachments: [ + { id: 'att-1', outcome: 'filed', inbox_item_id: 'item-1' }, + { id: 'att-2', outcome: 'failed', inbox_item_id: 'item-2' }, + ], + }, +} + +describe('GET /inbound-history (#2181)', () => { + beforeEach(() => vi.clearAllMocks()) + + it('returns 401 without context', async () => { + const res = await route.handler(createMockRequest('/inbound-history'), undefined) + expect(res.status).toBe(401) + }) + + it.each(['0', '-1', '366', 'abc', '1.5'])('returns 400 for days=%s', async (days) => { + const { supabase } = createQueuedMockSupabase() + const res = await route.handler( + createMockRequest(`/inbound-history?days=${days}`), + buildCtx(supabase) + ) + expect(res.status).toBe(400) + }) + + it('returns the company-scoped InboundMailReceived events, newest first, 30 days by default', async () => { + const { supabase, enqueue, calls } = createQueuedMockSupabase() + enqueue({ data: [EVENT_ROW] }) + enqueue({ data: [{ id: 'inbox-1', local_part: 'acme-ab-x7f2', status: 'active' }] }) // company's inboxes + const res = await route.handler(createMockRequest('/inbound-history'), buildCtx(supabase)) + const { status, body } = await parseJsonResponse<{ + data: { days: number; has_more: boolean; mails: Array> } + }>(res) + expect(status).toBe(200) + expect(body.data.days).toBe(30) + expect(body.data.has_more).toBe(false) + expect(body.data.mails).toEqual([ + { + event_id: 'evt-1', + email_id: '6ec3164f-1f59-40e7-acd2-ebedf9c4d56e', + occurred_at: '2026-09-02T14:21:56Z', + ...EVENT_ROW.payload, + // Resolved at read time for the company's own members; the event + // itself carries no address. + inbox_local_part: 'acme-ab-x7f2', + inbox_status: 'active', + }, + ]) + const inboxLookup = calls.find((c) => c.table === 'company_inboxes' && c.method === 'eq') + expect(inboxLookup?.args).toEqual(['company_id', 'company-1']) + + const eqs = calls.filter((c) => c.table === 'processing_history' && c.method === 'eq').map((c) => c.args) + expect(eqs).toEqual([ + ['company_id', 'company-1'], + ['event_type', 'InboundMailReceived'], + ]) + const gte = calls.find((c) => c.table === 'processing_history' && c.method === 'gte') + expect(gte?.args[0]).toBe('occurred_at') + const order = calls.find((c) => c.table === 'processing_history' && c.method === 'order') + expect(order?.args).toEqual(['occurred_at', { ascending: false }]) + }) + + it('marks a mail whose inbox row is gone with no address', async () => { + const { supabase, enqueue } = createQueuedMockSupabase() + enqueue({ data: [EVENT_ROW] }) + enqueue({ data: [] }) // inbox rotated away and removed + const res = await route.handler(createMockRequest('/inbound-history'), buildCtx(supabase)) + const { body } = await parseJsonResponse<{ data: { mails: Array<{ inbox_local_part: string | null }> } }>(res) + expect(body.data.mails[0].inbox_local_part).toBeNull() + }) + + it('honours an explicit window', async () => { + const { supabase, enqueue } = createQueuedMockSupabase() + enqueue({ data: [] }) + const res = await route.handler(createMockRequest('/inbound-history?days=7'), buildCtx(supabase)) + const { body } = await parseJsonResponse<{ data: { days: number; mails: unknown[] } }>(res) + expect(body.data).toEqual({ days: 7, has_more: false, mails: [] }) + }) + + it('caps the list at 200 and says when older mail in the window was cut off', async () => { + const { supabase, enqueue, calls } = createQueuedMockSupabase() + enqueue({ data: Array.from({ length: 201 }, (_, i) => ({ ...EVENT_ROW, event_id: `evt-${i}` })) }) + enqueue({ data: [{ id: 'inbox-1', local_part: 'acme-ab-x7f2', status: 'active' }] }) + const res = await route.handler(createMockRequest('/inbound-history'), buildCtx(supabase)) + const { body } = await parseJsonResponse<{ data: { has_more: boolean; mails: unknown[] } }>(res) + expect(body.data.has_more).toBe(true) + expect(body.data.mails).toHaveLength(200) + const limit = calls.find((c) => c.table === 'processing_history' && c.method === 'limit') + expect(limit?.args).toEqual([201]) + }) + + it('returns 500 when the read fails', async () => { + const { supabase, enqueue } = createQueuedMockSupabase() + enqueue({ data: null, error: { message: 'boom' } }) + const res = await route.handler(createMockRequest('/inbound-history'), buildCtx(supabase)) + expect(res.status).toBe(500) + }) +}) diff --git a/extensions/general/invoice-inbox/__tests__/inbound-webhook.test.ts b/extensions/general/invoice-inbox/__tests__/inbound-webhook.test.ts index 6b02b0a5..e27472b0 100644 --- a/extensions/general/invoice-inbox/__tests__/inbound-webhook.test.ts +++ b/extensions/general/invoice-inbox/__tests__/inbound-webhook.test.ts @@ -48,9 +48,17 @@ vi.mock('@/lib/rate-limits/inbox', () => ({ checkInboxUploadRateLimit: vi.fn().mockResolvedValue({ ok: true }), })) +// The webhook appends one InboundMailReceived event per mail and inbox +// (#2181); keep the history write off the network and observable. +vi.mock('@/lib/processing-history/append', () => ({ + appendProcessingHistory: vi.fn().mockResolvedValue('event-id'), +})) + import { verifyInboundWebhook, fetchReceivingEmail, fetchInboundAttachment } from '@/extensions/general/invoice-inbox/lib/resend-inbound' import { uploadAndExtract } from '@/extensions/general/invoice-inbox/lib/upload-and-extract' import { createClient } from '@supabase/supabase-js' +import { appendProcessingHistory } from '@/lib/processing-history/append' +import { checkInboxUploadRateLimit } from '@/lib/rate-limits/inbox' function findRoute(method: string, path: string) { return invoiceInboxExtension.apiRoutes!.find((r) => r.method === method && r.path === path)! @@ -721,3 +729,396 @@ describe('POST /inbound', () => { expect(rejection?.args[0]).toMatchObject({ status: 'error', kind_hint: 'receipt' }) }) }) + +/** Every InboundMailReceived append, in order. */ +function receivedEvents() { + return vi + .mocked(appendProcessingHistory) + .mock.calls.map(([input]) => input) + .filter((input) => input.eventType === 'InboundMailReceived') +} + +function fullEmailFor(to: string[], attachments: unknown[], overrides: Record = {}) { + return { + object: 'email', + id: 'em_123', + to, + from: 'billing@supplier.com', + created_at: '2026-04-20T10:00:00Z', + subject: 'Faktura', + bcc: null, + cc: null, + reply_to: null, + html: null, + text: 'Se bifogad faktura', + headers: {}, + message_id: '', + raw: null, + attachments, + ...overrides, + } +} + +const PDF_ATTACHMENT = { + id: 'att_1', + filename: 'faktura.pdf', + size: 100, + content_type: 'application/pdf', + content_id: 'cid', + content_disposition: 'attachment', +} + +const PDF_DOWNLOAD = { + id: 'att_1', + filename: 'faktura.pdf', + contentType: 'application/pdf', + buffer: new Uint8Array([0x25, 0x50, 0x44, 0x46]).buffer as ArrayBuffer, +} + +describe('POST /inbound with several recipients (#2181)', () => { + const originalEnv = { ...process.env } + + beforeEach(() => { + vi.clearAllMocks() + process.env.RESEND_INBOUND_DOMAIN = 'arcim.io' + process.env.NEXT_PUBLIC_SUPABASE_URL = 'http://localhost' + process.env.SUPABASE_SERVICE_ROLE_KEY = 'test-service-key' + vi.mocked(appendProcessingHistory).mockResolvedValue('event-id') + }) + + afterEach(() => { + process.env = { ...originalEnv } + }) + + it('files a mail sent to +lev and +ver of the same inbox once, with no kind hint', async () => { + const to = ['acme-ab-x7f2+lev@arcim.io', 'acme-ab-x7f2+ver@arcim.io'] + vi.mocked(verifyInboundWebhook).mockReturnValue(mockReceivedEvent({ to }) as never) + const { supabase, enqueue, calls } = createQueuedMockSupabase() + enqueue({ data: { id: 'inbox-1', company_id: 'company-1', status: 'active' } }) // one inbox lookup + enqueue({ data: { created_by: 'user-owner-1' } }) + enqueue({ data: null }) // per-attachment dup check finds nothing + vi.mocked(createClient).mockReturnValue(supabase as never) + vi.mocked(uploadAndExtract).mockResolvedValue({ inbox_item_id: 'item-1' } as never) + vi.mocked(fetchReceivingEmail).mockResolvedValue(fullEmailFor(to, [PDF_ATTACHMENT]) as never) + vi.mocked(fetchInboundAttachment).mockResolvedValue(PDF_DOWNLOAD) + + const res = await webhookRoute.handler(createMockRequest('/inbound', { method: 'POST', body: {} })) + const body = await res.json() + expect(res.status).toBe(200) + expect(body.data.processed).toBe(1) + expect(body.data.results).toEqual([{ attachment_id: 'att_1', inbox_item_id: 'item-1' }]) + + // One inbox lookup, one upload, no hint: the sender said two things. + expect(calls.filter((c) => c.table === 'company_inboxes' && c.method === 'eq')).toHaveLength(1) + expect(uploadAndExtract).toHaveBeenCalledTimes(1) + const [, , , , , emailMeta] = vi.mocked(uploadAndExtract).mock.calls[0] + expect(emailMeta?.kindHint).toBeNull() + + const events = receivedEvents() + expect(events).toHaveLength(1) + expect(events[0].companyId).toBe('company-1') + expect(events[0].correlationId).toBe('em_123') + expect(events[0].payload).toMatchObject({ + inbox_id: 'inbox-1', + custom_domain: false, + tags: ['lev', 'ver'], + unknown_tag_count: 0, + kind_hint: null, + tag_conflict: true, + outcome: 'attachments', + attachment_count: 1, + attachments: [{ id: 'att_1', outcome: 'filed', inbox_item_id: 'item-1' }], + }) + expect(JSON.stringify(events[0].payload)).not.toContain('billing@supplier.com') + expect(JSON.stringify(events[0].payload)).not.toContain('Faktura') + // No address at all: an enskild firma's local part is the owner's name. + expect(JSON.stringify(events[0].payload)).not.toContain('@') + expect(JSON.stringify(events[0].payload)).not.toContain('acme-ab-x7f2') + }) + + it('counts a sender-typed tag without storing it, so a numeric tag cannot trip the PII validator', async () => { + const to = ['acme-ab-x7f2+8501011234@arcim.io', 'acme-ab-x7f2+lev@arcim.io'] + vi.mocked(verifyInboundWebhook).mockReturnValue(mockReceivedEvent({ to, attachments: [] }) as never) + const { supabase, enqueue } = createQueuedMockSupabase() + enqueue({ data: { id: 'inbox-1', company_id: 'company-1', status: 'active' } }) + enqueue({ data: { created_by: 'user-owner-1' } }) + enqueue({ data: null }) // body-document dedupe check + vi.mocked(createClient).mockReturnValue(supabase as never) + vi.mocked(uploadAndExtract).mockResolvedValue({ inbox_item_id: 'item-body' } as never) + vi.mocked(fetchReceivingEmail).mockResolvedValue(fullEmailFor(to, [], { html: '

    Kvitto

    ' }) as never) + + const res = await webhookRoute.handler(createMockRequest('/inbound', { method: 'POST', body: {} })) + expect(res.status).toBe(200) + const [event] = receivedEvents() + expect(event.payload).toMatchObject({ tags: ['lev'], unknown_tag_count: 1, kind_hint: 'supplier_invoice', tag_conflict: false }) + expect(JSON.stringify(event.payload)).not.toContain('8501011234') + }) + + it('files a mail addressed to two inboxes once per inbox, deduped per company', async () => { + const to = ['acme-ab-x7f2+lev@arcim.io', 'beta-ab-q9z1@arcim.io'] + vi.mocked(verifyInboundWebhook).mockReturnValue(mockReceivedEvent({ to }) as never) + const { supabase, enqueue, calls } = createQueuedMockSupabase() + enqueue({ data: { id: 'inbox-1', company_id: 'company-1', status: 'active' } }) + enqueue({ data: { id: 'inbox-2', company_id: 'company-2', status: 'active' } }) + enqueue({ data: { created_by: 'user-owner-1' } }) + enqueue({ data: { created_by: 'user-owner-2' } }) + enqueue({ data: null }) // company-1 dup check + enqueue({ data: null }) // company-2 dup check + vi.mocked(createClient).mockReturnValue(supabase as never) + vi.mocked(uploadAndExtract) + .mockResolvedValueOnce({ inbox_item_id: 'item-c1' } as never) + .mockResolvedValueOnce({ inbox_item_id: 'item-c2' } as never) + vi.mocked(fetchReceivingEmail).mockResolvedValue(fullEmailFor(to, [PDF_ATTACHMENT]) as never) + vi.mocked(fetchInboundAttachment).mockResolvedValue(PDF_DOWNLOAD) + + const res = await webhookRoute.handler(createMockRequest('/inbound', { method: 'POST', body: {} })) + const body = await res.json() + expect(res.status).toBe(200) + expect(body.data.processed).toBe(2) + expect(body.data.targets).toEqual([ + { company_id: 'company-1', processed: 1, results: [{ attachment_id: 'att_1', inbox_item_id: 'item-c1' }] }, + { company_id: 'company-2', processed: 1, results: [{ attachment_id: 'att_1', inbox_item_id: 'item-c2' }] }, + ]) + + expect(uploadAndExtract).toHaveBeenCalledTimes(2) + expect(vi.mocked(uploadAndExtract).mock.calls[0][2]).toBe('company-1') + expect(vi.mocked(uploadAndExtract).mock.calls[0][5]?.kindHint).toBe('supplier_invoice') + expect(vi.mocked(uploadAndExtract).mock.calls[1][2]).toBe('company-2') + expect(vi.mocked(uploadAndExtract).mock.calls[1][5]?.kindHint).toBeNull() + + // The idempotency lookup is company-scoped: the second inbox's copy is + // not "already processed" because the first inbox filed it. + const dupChecks = calls.filter( + (c) => c.table === 'invoice_inbox_items' && c.method === 'eq' && c.args[0] === 'company_id', + ) + expect(dupChecks.map((c) => c.args[1])).toEqual(['company-1', 'company-2']) + + const events = receivedEvents() + expect(events.map((e) => e.companyId)).toEqual(['company-1', 'company-2']) + expect(events[0].payload).toMatchObject({ inbox_id: 'inbox-1', tags: ['lev'], kind_hint: 'supplier_invoice', tag_conflict: false }) + expect(events[1].payload).toMatchObject({ inbox_id: 'inbox-2', tags: [], kind_hint: null, custom_domain: false }) + }) + + it('skips a retired address in the list and still files for the active one', async () => { + const to = ['old-inbox-abcd+lev@arcim.io', 'acme-ab-x7f2+ver@arcim.io'] + vi.mocked(verifyInboundWebhook).mockReturnValue(mockReceivedEvent({ to, attachments: [] }) as never) + const { supabase, enqueue } = createQueuedMockSupabase() + enqueue({ data: { id: 'inbox-old', company_id: 'company-old', status: 'deprecated' } }) + enqueue({ data: { id: 'inbox-1', company_id: 'company-1', status: 'active' } }) + enqueue({ data: { created_by: 'user-owner-1' } }) + enqueue({ data: null }) // body-document dedupe check + vi.mocked(createClient).mockReturnValue(supabase as never) + vi.mocked(uploadAndExtract).mockResolvedValue({ inbox_item_id: 'item-body' } as never) + vi.mocked(fetchReceivingEmail).mockResolvedValue(fullEmailFor(to, [], { html: '

    Kvitto

    ' }) as never) + + const res = await webhookRoute.handler(createMockRequest('/inbound', { method: 'POST', body: {} })) + const body = await res.json() + expect(res.status).toBe(200) + expect(body.data.reason).toBe('email_body') + expect(vi.mocked(uploadAndExtract).mock.calls[0][2]).toBe('company-1') + // The retired address's tag does not leak onto the active inbox's hint. + expect(vi.mocked(uploadAndExtract).mock.calls[0][5]?.kindHint).toBe('receipt') + expect(receivedEvents()[0].payload).toMatchObject({ outcome: 'email_body', inbox_item_id: 'item-body' }) + }) + + it('lets a Resend redelivery replace a transient error row and file the attachment (self-heal kept)', async () => { + vi.mocked(verifyInboundWebhook).mockReturnValue(mockReceivedEvent() as never) + const { supabase, enqueue, calls } = createQueuedMockSupabase() + enqueue({ data: { id: 'inbox-1', company_id: 'company-1', status: 'active' } }) + enqueue({ data: { created_by: 'user-owner-1' } }) + // The first delivery's download failed and the catch wrote this row. + enqueue({ data: { id: 'err-1', status: 'error', raw_email_payload: { messageId: 'm', transient: true } } }) + enqueue({ data: null }) // delete of the transient row + vi.mocked(createClient).mockReturnValue(supabase as never) + vi.mocked(uploadAndExtract).mockResolvedValue({ inbox_item_id: 'item-healed' } as never) + vi.mocked(fetchReceivingEmail).mockResolvedValue(fullEmailFor(['acme-ab-x7f2@arcim.io'], [PDF_ATTACHMENT]) as never) + vi.mocked(fetchInboundAttachment).mockResolvedValue(PDF_DOWNLOAD) + + const res = await webhookRoute.handler(createMockRequest('/inbound', { method: 'POST', body: {} })) + const body = await res.json() + expect(res.status).toBe(200) + expect(body.data.results).toEqual([{ attachment_id: 'att_1', inbox_item_id: 'item-healed' }]) + const del = calls.find((c) => c.table === 'invoice_inbox_items' && c.method === 'delete') + expect(del).toBeDefined() + const delEq = calls.filter((c) => c.table === 'invoice_inbox_items' && c.method === 'eq').find((c) => c.args[0] === 'id') + expect(delEq?.args).toEqual(['id', 'err-1']) + // The replacement leaves a trace: the record names the row it replaced. + expect(receivedEvents()[0].payload).toMatchObject({ + attachments: [{ id: 'att_1', outcome: 'filed', inbox_item_id: 'item-healed', replaced_item_id: 'err-1' }], + }) + }) + + it('processes five inboxes per mail and records the rest as not processed', async () => { + // Two unknown local parts first: they must not use up the cap. + const to = [ + 'unknown-a-0000@arcim.io', + 'unknown-b-0000@arcim.io', + ...Array.from({ length: 7 }, (_, i) => `inbox-${i}-abcd@arcim.io`), + ] + vi.mocked(verifyInboundWebhook).mockReturnValue(mockReceivedEvent({ to, attachments: [] }) as never) + const { supabase, enqueue, calls } = createQueuedMockSupabase() + enqueue({ data: null }) // unknown-a + enqueue({ data: null }) // unknown-b + for (let i = 0; i < 7; i++) { + enqueue({ data: { id: `inbox-${i}`, company_id: `company-${i}`, status: 'active' } }) + } + for (let i = 0; i < 5; i++) enqueue({ data: { created_by: `owner-${i}` } }) + for (let i = 0; i < 5; i++) enqueue({ data: null }) // body-document dedupe checks + vi.mocked(createClient).mockReturnValue(supabase as never) + vi.mocked(uploadAndExtract).mockResolvedValue({ inbox_item_id: 'item-body' } as never) + vi.mocked(fetchReceivingEmail).mockResolvedValue(fullEmailFor(to, [], { html: '

    Kvitto

    ' }) as never) + + const res = await webhookRoute.handler(createMockRequest('/inbound', { method: 'POST', body: {} })) + const body = await res.json() + expect(res.status).toBe(200) + // Every addressed inbox is looked up; the first five resolved are processed. + expect(calls.filter((c) => c.table === 'company_inboxes' && c.method === 'eq')).toHaveLength(9) + expect(body.data.targets.map((t: { company_id: string }) => t.company_id)).toEqual( + ['company-0', 'company-1', 'company-2', 'company-3', 'company-4'], + ) + expect(body.data.deferred).toEqual([ + { company_id: 'company-5', reason: 'fan_out_capped' }, + { company_id: 'company-6', reason: 'fan_out_capped' }, + ]) + expect(uploadAndExtract).toHaveBeenCalledTimes(5) + // The two past the cap still get a record: arrived, not processed. + const events = receivedEvents() + expect(events).toHaveLength(7) + expect(events.slice(5).map((e) => [e.companyId, e.payload.outcome])).toEqual([ + ['company-5', 'fan_out_capped'], + ['company-6', 'fan_out_capped'], + ]) + }) + + it('keeps a rejected attachment (bad type) as a duplicate on redelivery', async () => { + vi.mocked(verifyInboundWebhook).mockReturnValue(mockReceivedEvent() as never) + const { supabase, enqueue, calls } = createQueuedMockSupabase() + enqueue({ data: { id: 'inbox-1', company_id: 'company-1', status: 'active' } }) + enqueue({ data: { created_by: 'user-owner-1' } }) + enqueue({ data: { id: 'rej-1', status: 'error', raw_email_payload: { messageId: 'm', mime: 'application/zip' } } }) + vi.mocked(createClient).mockReturnValue(supabase as never) + vi.mocked(fetchReceivingEmail).mockResolvedValue(fullEmailFor(['acme-ab-x7f2@arcim.io'], [PDF_ATTACHMENT]) as never) + + const res = await webhookRoute.handler(createMockRequest('/inbound', { method: 'POST', body: {} })) + const body = await res.json() + expect(body.data.results[0]).toEqual({ attachment_id: 'att_1', inbox_item_id: 'rej-1', duplicate: true }) + expect(fetchInboundAttachment).not.toHaveBeenCalled() + expect(calls.find((c) => c.table === 'invoice_inbox_items' && c.method === 'delete')).toBeUndefined() + }) + + it('records a duplicate outcome when Resend retries the webhook', async () => { + vi.mocked(verifyInboundWebhook).mockReturnValue(mockReceivedEvent() as never) + const { supabase, enqueue } = createQueuedMockSupabase() + enqueue({ data: { id: 'inbox-1', company_id: 'company-1', status: 'active' } }) + enqueue({ data: { created_by: 'user-owner-1' } }) + enqueue({ data: { id: 'existing-item-1' } }) // dup check finds the first delivery's row + vi.mocked(createClient).mockReturnValue(supabase as never) + vi.mocked(fetchReceivingEmail).mockResolvedValue(fullEmailFor(['acme-ab-x7f2@arcim.io'], [PDF_ATTACHMENT]) as never) + + const res = await webhookRoute.handler(createMockRequest('/inbound', { method: 'POST', body: {} })) + expect(res.status).toBe(200) + expect(receivedEvents()[0].payload).toMatchObject({ + attachments: [{ id: 'att_1', outcome: 'duplicate', inbox_item_id: 'existing-item-1' }], + }) + }) + + it('records a rejected outcome for an attachment type outside the allowlist', async () => { + vi.mocked(verifyInboundWebhook).mockReturnValue(mockReceivedEvent() as never) + const { supabase, enqueue } = createQueuedMockSupabase() + enqueue({ data: { id: 'inbox-1', company_id: 'company-1', status: 'active' } }) + enqueue({ data: { created_by: 'user-owner-1' } }) + enqueue({ data: null }) // dup check + enqueue({ data: { id: 'rejected-row-1' } }) // rejection-row insert returns its id + vi.mocked(createClient).mockReturnValue(supabase as never) + vi.mocked(fetchReceivingEmail).mockResolvedValue( + fullEmailFor(['acme-ab-x7f2@arcim.io'], [{ ...PDF_ATTACHMENT, id: 'att_zip', filename: 'x.zip', content_type: 'application/zip' }]) as never, + ) + vi.mocked(fetchInboundAttachment).mockResolvedValue({ + id: 'att_zip', + filename: 'x.zip', + contentType: 'application/zip', + buffer: new ArrayBuffer(8), + }) + + const res = await webhookRoute.handler(createMockRequest('/inbound', { method: 'POST', body: {} })) + expect(res.status).toBe(200) + expect(receivedEvents()[0].payload).toMatchObject({ + attachments: [ + { id: 'att_zip', outcome: 'rejected', reason: 'unsupported_type', mime: 'application/zip', inbox_item_id: 'rejected-row-1' }, + ], + }) + }) + + it('keeps an attachment whose processing threw as an error row and records it as failed', async () => { + vi.mocked(verifyInboundWebhook).mockReturnValue(mockReceivedEvent({ to: ['acme-ab-x7f2+lev@arcim.io'] }) as never) + const { supabase, enqueue, calls } = createQueuedMockSupabase() + enqueue({ data: { id: 'inbox-1', company_id: 'company-1', status: 'active' } }) + enqueue({ data: { created_by: 'user-owner-1' } }) + enqueue({ data: null }) // dup check + enqueue({ data: null }) // post-failure lookup: the upload never made its row + enqueue({ data: { id: 'failed-row-1' } }) // error-row insert + vi.mocked(createClient).mockReturnValue(supabase as never) + vi.mocked(fetchReceivingEmail).mockResolvedValue(fullEmailFor(['acme-ab-x7f2+lev@arcim.io'], [PDF_ATTACHMENT]) as never) + vi.mocked(fetchInboundAttachment).mockRejectedValue(new Error('Download URL returned 503 for attachment att_1')) + + const res = await webhookRoute.handler(createMockRequest('/inbound', { method: 'POST', body: {} })) + const body = await res.json() + expect(res.status).toBe(200) + expect(body.data.results[0].error).toBe('Download URL returned 503 for attachment att_1') + + // Before #2181 this path wrote nothing: the mail was accepted, answered + // 200 and gone. Now the attachment is an error row the user can see. + const insert = calls.find((c) => c.table === 'invoice_inbox_items' && c.method === 'insert') + expect(insert?.args[0]).toMatchObject({ + status: 'error', + resend_email_id: 'em_123', + resend_attachment_id: 'att_1', + kind_hint: 'supplier_invoice', + raw_email_payload: { transient: true }, + }) + expect((insert?.args[0] as { error_message: string }).error_message).toMatch(/^Bilagan kunde inte tas emot: Download URL returned 503/) + + expect(receivedEvents()[0].payload).toMatchObject({ + attachments: [{ id: 'att_1', outcome: 'failed', inbox_item_id: 'failed-row-1' }], + }) + }) + + it('does not write a second error row when the upload made its own row before throwing', async () => { + vi.mocked(verifyInboundWebhook).mockReturnValue(mockReceivedEvent() as never) + const { supabase, enqueue, calls } = createQueuedMockSupabase() + enqueue({ data: { id: 'inbox-1', company_id: 'company-1', status: 'active' } }) + enqueue({ data: { created_by: 'user-owner-1' } }) + enqueue({ data: null }) // dup check + enqueue({ data: { id: 'partial-row-1' } }) // post-failure lookup finds the upload's row + vi.mocked(createClient).mockReturnValue(supabase as never) + vi.mocked(fetchReceivingEmail).mockResolvedValue(fullEmailFor(['acme-ab-x7f2@arcim.io'], [PDF_ATTACHMENT]) as never) + vi.mocked(fetchInboundAttachment).mockResolvedValue(PDF_DOWNLOAD) + vi.mocked(uploadAndExtract).mockRejectedValue(new Error('extraction timed out')) + + const res = await webhookRoute.handler(createMockRequest('/inbound', { method: 'POST', body: {} })) + expect(res.status).toBe(200) + expect(calls.find((c) => c.table === 'invoice_inbox_items' && c.method === 'insert')).toBeUndefined() + expect(receivedEvents()[0].payload).toMatchObject({ + attachments: [{ id: 'att_1', outcome: 'failed', inbox_item_id: 'partial-row-1' }], + }) + }) + + it('records the rate-limited drop on the mail record too', async () => { + vi.mocked(verifyInboundWebhook).mockReturnValue(mockReceivedEvent() as never) + vi.mocked(checkInboxUploadRateLimit).mockResolvedValueOnce({ ok: false, scope: 'minute', retryAfterSec: 60 } as never) + const { supabase, enqueue } = createQueuedMockSupabase() + enqueue({ data: { id: 'inbox-1', company_id: 'company-1', status: 'active' } }) + enqueue({ data: { created_by: 'user-owner-1' } }) + vi.mocked(createClient).mockReturnValue(supabase as never) + vi.mocked(fetchReceivingEmail).mockResolvedValue(fullEmailFor(['acme-ab-x7f2@arcim.io'], [PDF_ATTACHMENT]) as never) + + const res = await webhookRoute.handler(createMockRequest('/inbound', { method: 'POST', body: {} })) + const body = await res.json() + expect(body.data.reason).toBe('rate_limited') + expect(vi.mocked(appendProcessingHistory).mock.calls.map(([i]) => i.eventType)).toEqual([ + 'RateLimitedDropped', + 'InboundMailReceived', + ]) + expect(receivedEvents()[0].payload).toMatchObject({ outcome: 'rate_limited', attachments: [] }) + }) +}) diff --git a/extensions/general/invoice-inbox/__tests__/resend-inbound.test.ts b/extensions/general/invoice-inbox/__tests__/resend-inbound.test.ts index 86ee59e8..a19a303a 100644 --- a/extensions/general/invoice-inbox/__tests__/resend-inbound.test.ts +++ b/extensions/general/invoice-inbox/__tests__/resend-inbound.test.ts @@ -1,8 +1,12 @@ import { describe, it, expect } from 'vitest' import { extractLocalPartForDomain, + extractSharedRecipientsForDomain, + groupSharedRecipientsByInbox, kindHintFromTag, parseRecipients, + resolveKindHintForTags, + splitKnownInboxTags, } from '@/extensions/general/invoice-inbox/lib/resend-inbound' describe('extractLocalPartForDomain', () => { @@ -122,3 +126,82 @@ describe('parseRecipients', () => { expect(parseRecipients([])).toEqual([]) }) }) + +describe('extractSharedRecipientsForDomain (#2181)', () => { + it('returns every shared-domain recipient in order with its tag', () => { + expect( + extractSharedRecipientsForDomain( + ['acme-ab-x7f2+lev@arcim.io', 'billing@acme.se', 'Acme-AB-x7f2+VER@arcim.io', 'other-1234@arcim.io'], + 'arcim.io', + ), + ).toEqual([ + { localPart: 'acme-ab-x7f2', tag: 'lev' }, + { localPart: 'acme-ab-x7f2', tag: 'ver' }, + { localPart: 'other-1234', tag: null }, + ]) + }) + + it('collapses the same address listed twice', () => { + expect( + extractSharedRecipientsForDomain(['acme-ab-x7f2+lev@arcim.io', 'ACME-AB-X7F2+lev@arcim.io'], 'arcim.io'), + ).toEqual([{ localPart: 'acme-ab-x7f2', tag: 'lev' }]) + }) + + it('skips foreign domains and malformed addresses', () => { + expect( + extractSharedRecipientsForDomain(['x+lev@acme.se', 'not-an-email', '@arcim.io', 'foo@'], 'arcim.io'), + ).toEqual([]) + }) + + it('keeps extractLocalPartForDomain as the first entry', () => { + const to = ['second-efgh+ver@arcim.io', 'first-abcd@arcim.io'] + expect(extractLocalPartForDomain(to, 'arcim.io')).toEqual(extractSharedRecipientsForDomain(to, 'arcim.io')[0]) + }) +}) + +describe('groupSharedRecipientsByInbox (#2181)', () => { + it('groups recipients per local part with distinct tags in first-seen order', () => { + expect( + groupSharedRecipientsByInbox([ + { localPart: 'acme', tag: 'lev' }, + { localPart: 'other', tag: null }, + { localPart: 'acme', tag: 'ver' }, + { localPart: 'acme', tag: 'lev' }, + ]), + ).toEqual([ + { localPart: 'acme', tags: ['lev', 'ver'] }, + { localPart: 'other', tags: [] }, + ]) + }) +}) + +describe('resolveKindHintForTags (#2181)', () => { + it('maps a single documented tag', () => { + expect(resolveKindHintForTags(['lev'])).toEqual({ kindHint: 'supplier_invoice', conflict: false }) + expect(resolveKindHintForTags(['ver'])).toEqual({ kindHint: 'receipt', conflict: false }) + }) + + it('returns no hint and no conflict for unknown or missing tags', () => { + expect(resolveKindHintForTags([])).toEqual({ kindHint: null, conflict: false }) + expect(resolveKindHintForTags(['faktura'])).toEqual({ kindHint: null, conflict: false }) + }) + + it('ignores unknown tags next to a documented one', () => { + expect(resolveKindHintForTags(['faktura', 'lev'])).toEqual({ kindHint: 'supplier_invoice', conflict: false }) + }) + + it('resolves +lev and +ver on one mail to no hint, flagged as a conflict', () => { + expect(resolveKindHintForTags(['lev', 'ver'])).toEqual({ kindHint: null, conflict: true }) + expect(resolveKindHintForTags(['ver', 'lev'])).toEqual({ kindHint: null, conflict: true }) + }) +}) + +describe('splitKnownInboxTags (#2181)', () => { + it('keeps the documented tags and only counts the rest', () => { + expect(splitKnownInboxTags(['lev', '8501011234', 'ver', 'anna-svensson'])).toEqual({ + known: ['lev', 'ver'], + unknownCount: 2, + }) + expect(splitKnownInboxTags([])).toEqual({ known: [], unknownCount: 0 }) + }) +}) diff --git a/extensions/general/invoice-inbox/index.ts b/extensions/general/invoice-inbox/index.ts index 5dd60c60..feaacf00 100644 --- a/extensions/general/invoice-inbox/index.ts +++ b/extensions/general/invoice-inbox/index.ts @@ -33,8 +33,10 @@ import { verifyInboundWebhook, fetchReceivingEmail, fetchInboundAttachment, - extractLocalPartForDomain, - kindHintFromTag, + extractSharedRecipientsForDomain, + groupSharedRecipientsByInbox, + resolveKindHintForTags, + splitKnownInboxTags, type InboxKindHint, parseRecipients, isEmailReceivedEvent, @@ -100,6 +102,19 @@ import { simpleParser } from 'mailparser' import type { InboxChannelContext, InvoiceExtractionResult, InvoiceInboxItem, SupplierInvoice, SupplierInvoiceItem } from '@/types' const MAX_ATTACHMENTS_PER_EMAIL = 20 +// Received-mail panel window (#2181): 30 days covers "the mail I sent last +// week"; the hard cap keeps the query bounded. +const INBOUND_HISTORY_DEFAULT_DAYS = 30 +const INBOUND_HISTORY_MAX_DAYS = 365 +const INBOUND_HISTORY_LIMIT = 200 +// One mail can name several inboxes (#2181), but the recipient list is +// sender-controlled and every target costs a full download-and-extract +// pass: cap the fan-out so a mail addressed to fifty known inboxes cannot +// multiply the work fifty times. A consultant forwarding to a handful of +// clients fits. Every addressed inbox is still resolved (one cheap lookup +// each) and the ones past the cap get a history record saying the mail +// arrived and was not processed, so no company is left without a trace. +const MAX_INBOUND_TARGETS_PER_EMAIL = 5 // Partial-update schema for the /items/:id/fields PATCH route. Only the // scalar fields the UI exposes for inline editing: line items and @@ -1777,49 +1792,95 @@ export const invoiceInboxExtension: Extension = { process.env.SUPABASE_SERVICE_ROLE_KEY! ) - // Recipient → company resolution. Shared-domain addresses first - // (existing local_part flow), then per-company verified custom - // domains. Custom domains are catch-all by design: MX routing is - // per-domain, and a supplier typing fakturor@ instead of faktura@ - // must land in the inbox rather than silently vanish (Resend has - // already accepted the message; there is no bounce path). - let companyId: string | null = null + // Recipient → company resolution. Every shared-domain recipient is + // read (#2181): one mail can name the same inbox under two tags, or + // two inboxes at once, and reading only the first lost the rest with + // no trace. Per-company verified custom domains come after, and only + // when no shared address resolved. Custom domains are catch-all by + // design: MX routing is per-domain, and a supplier typing fakturor@ + // instead of faktura@ must land in the inbox rather than silently + // vanish (Resend has already accepted the message; there is no + // bounce path). + interface InboundTarget { + companyId: string + inboxId: string | null + customDomain: boolean + /** The documented tags (+lev / +ver) that reached this inbox. */ + tags: string[] + /** Tags outside the documented set: counted, never stored. */ + unknownTagCount: number + kindHint: InboxKindHint | null + tagConflict: boolean + } + const targets: InboundTarget[] = [] let sharedInboxStatus: string | null = null + let localPart: string | null = null + const domainLower = domain.toLowerCase() - const sharedRecipient = extractLocalPartForDomain(to, domain) - const localPart = sharedRecipient?.localPart ?? null - // Sender-declared kind from the +lev / +ver tag on the shared - // address. Set only when that address is the one that resolved the - // company: a tag on an unknown or retired shared address must not - // ride along onto a custom-domain match further down. Custom domains - // are catch-all and stay unhinted. - let kindHint: InboxKindHint | null = null - if (localPart) { + const sharedRecipients = extractSharedRecipientsForDomain(to, domain) + for (const group of groupSharedRecipientsByInbox(sharedRecipients)) { + localPart ??= group.localPart const { data: inbox } = await serviceSupabase .from('company_inboxes') .select('id, company_id, status') - .eq('local_part', localPart) + .eq('local_part', group.localPart) .maybeSingle() - if (inbox) { - sharedInboxStatus = inbox.status - if (inbox.status === 'active') { - companyId = inbox.company_id - kindHint = kindHintFromTag(sharedRecipient?.tag) + if (!inbox) continue + if (inbox.status !== 'active') { + sharedInboxStatus ??= inbox.status + continue + } + // Two active addresses of one company file once. + if (targets.some((t) => t.companyId === inbox.company_id)) continue + // Sender-declared kind from the +lev / +ver tag. Set only from the + // addresses that resolved this company: a tag on an unknown or + // retired shared address must not ride along onto a custom-domain + // match further down. Custom domains are catch-all and stay + // unhinted. Contradicting tags on one mail resolve to no hint. + const { kindHint, conflict } = resolveKindHintForTags(group.tags) + const { known, unknownCount } = splitKnownInboxTags(group.tags) + targets.push({ + companyId: inbox.company_id, + inboxId: inbox.id, + customDomain: false, + tags: known, + unknownTagCount: unknownCount, + kindHint, + tagConflict: conflict, + }) + } + + if (targets.length === 0) { + const customDomains = parseRecipients(to) + .map((r) => r.domain) + .filter((d) => d !== domainLower) + if (customDomains.length > 0) { + const match = await findCompanyForRecipientDomains(serviceSupabase, customDomains) + if (match) { + targets.push({ + companyId: match.companyId, + inboxId: null, + customDomain: true, + tags: [], + unknownTagCount: 0, + kindHint: null, + tagConflict: false, + }) } } } - if (!companyId) { - const customDomains = parseRecipients(to) - .map((r) => r.domain) - .filter((d) => d !== domain.toLowerCase()) - if (customDomains.length > 0) { - const match = await findCompanyForRecipientDomains(serviceSupabase, customDomains) - if (match) companyId = match.companyId - } + // Fan-out cap: the first targets in recipient order are processed, + // the rest only recorded (below), never downloaded or extracted. + const deferredTargets = targets.splice(MAX_INBOUND_TARGETS_PER_EMAIL) + if (deferredTargets.length > 0) { + console.warn('[invoice-inbox/inbound] Recipient fan-out capped', { + addressed: targets.length + deferredTargets.length, + processed: targets.length, + }) } - if (!companyId) { + if (targets.length === 0) { // Preserve the pre-custom-domain status semantics: 410 for a // deprecated/blocked shared address, 404 otherwise. if (sharedInboxStatus && sharedInboxStatus !== 'active') { @@ -1832,17 +1893,20 @@ export const invoiceInboxExtension: Extension = { ) } - const { data: company } = await serviceSupabase - .from('companies') - .select('created_by') - .eq('id', companyId) - .single() + const owners = new Map() + for (const target of targets) { + const { data: company } = await serviceSupabase + .from('companies') + .select('created_by') + .eq('id', target.companyId) + .single() - if (!company?.created_by) { - console.error('[invoice-inbox/inbound] Company has no created_by', companyId) - return NextResponse.json({ error: 'Company owner missing' }, { status: 500 }) + if (!company?.created_by) { + console.error('[invoice-inbox/inbound] Company has no created_by', target.companyId) + return NextResponse.json({ error: 'Company owner missing' }, { status: 500 }) + } + owners.set(target.companyId, company.created_by) } - const userId = company.created_by let fullEmail try { @@ -1856,150 +1920,143 @@ export const invoiceInboxExtension: Extension = { const bodyText = fullEmail.text ?? null const rawAttachments = fullEmail.attachments ?? [] - // Per-company rate limit (30/min, 500/day). Same Postgres-backed - // RPC as /upload. Acknowledge + drop on cap: returning 429 to - // Resend would just consume more budget via their retry. - const limit = await checkInboxUploadRateLimit(serviceSupabase, companyId) - if (!limit.ok) { - try { - await appendProcessingHistory({ - companyId, - correlationId: email_id, - aggregateType: 'System', - aggregateId: email_id, - eventType: 'RateLimitedDropped', - // No `from` / `subject`: processing_history is append-only - // (UPDATE is trigger-blocked) and outside the archive's erasure - // path, so the sender address and the free-text subject may not - // land here. correlationId is the Resend email_id, which reaches - // both through invoice_inbox_items. - payload: { - scope: limit.scope, - retry_after_sec: limit.retryAfterSec, - attachment_count: rawAttachments.length, - }, - actor: { type: 'system', id: 'resend-inbound' }, - occurredAt: new Date(), - }) - } catch (err) { - console.error('[invoice-inbox/inbound] RateLimitedDropped append failed:', err) - } - return NextResponse.json({ data: { processed: 0, reason: 'rate_limited' } }) - } - // Per-email attachment cap. 20 covers any legitimate batched // supplier email; an attacker stuffing 500 PDFs into one message // gets truncated and a single history event records the drop. const totalAttachments = rawAttachments.length const attachments = rawAttachments.slice(0, MAX_ATTACHMENTS_PER_EMAIL) const truncatedCount = totalAttachments - attachments.length - if (truncatedCount > 0) { - try { - await appendProcessingHistory({ - companyId, - correlationId: email_id, - aggregateType: 'System', - aggregateId: email_id, - eventType: 'AttachmentsTruncated', - // Counts only, for the same reason as RateLimitedDropped above. - payload: { - total: totalAttachments, - processed: attachments.length, - dropped: truncatedCount, - }, - actor: { type: 'system', id: 'resend-inbound' }, - occurredAt: new Date(), - }) - } catch (err) { - console.error('[invoice-inbox/inbound] AttachmentsTruncated append failed:', err) - } + + type AttachmentResult = { attachment_id: string; inbox_item_id?: string; error?: string; duplicate?: boolean } + /** + * What became of one attachment, for the mail's history event. Codes + * only: the free-text error and the filename stay out of + * processing_history (append-only, outside the erasure path). + */ + type AttachmentOutcome = { + id: string + outcome: 'filed' | 'duplicate' | 'rejected' | 'failed' + inbox_item_id?: string + reason?: string + mime?: string + /** The transient error row a redelivery replaced (BFL 5 kap 5 §: the replacement leaves a trace). */ + replaced_item_id?: string + } + type TargetOutcome = { + processed: number + reason?: string + inbox_item_id?: string + results?: AttachmentResult[] + attachments: AttachmentOutcome[] } - if (attachments.length === 0) { - // Body-only mail: for many suppliers the HTML body IS the invoice - // (SaaS receipts, e-mail invoices), and often the only underlag the - // user has. Store the body as a text/html document and run the - // normal extract pipeline instead of dead-ending in an error row. - // Mails with an empty body keep the old error row. - const bodyDoc = buildEmailBodyHtmlDocument(fullEmail.html ?? null, bodyText) - if (bodyDoc && bodyDoc.byteLength <= MAX_FILE_SIZE) { - // Resend retries the webhook on failure: a retry after success - // must not duplicate the body document. Body items carry the - // email_id with a NULL attachment id. - const { data: existingBody } = await serviceSupabase - .from('invoice_inbox_items') - .select('id') - .eq('resend_email_id', email_id) - .is('resend_attachment_id', null) - .maybeSingle() - if (existingBody) { - return NextResponse.json( - { data: { processed: 0, reason: 'email_body_duplicate', inbox_item_id: existingBody.id } } - ) - } + const processForTarget = async (target: InboundTarget): Promise => { + const { companyId, kindHint } = target + const userId = owners.get(companyId)! + const attachmentOutcomes: AttachmentOutcome[] = [] + + // Per-company rate limit (30/min, 500/day). Same Postgres-backed + // RPC as /upload. Acknowledge + drop on cap: returning 429 to + // Resend would just consume more budget via their retry. + const limit = await checkInboxUploadRateLimit(serviceSupabase, companyId) + if (!limit.ok) { try { - const result = await uploadAndExtract( - serviceSupabase, - userId, + await appendProcessingHistory({ companyId, - { - name: `mail-${sanitiseFilename(subject, 'meddelande')}.html`, - buffer: bodyDoc, - type: 'text/html', + correlationId: email_id, + aggregateType: 'System', + aggregateId: email_id, + eventType: 'RateLimitedDropped', + // No `from` / `subject`: processing_history is append-only + // (UPDATE is trigger-blocked) and outside the archive's erasure + // path, so the sender address and the free-text subject may not + // land here. correlationId is the Resend email_id, which reaches + // both through invoice_inbox_items. + payload: { + scope: limit.scope, + retry_after_sec: limit.retryAfterSec, + attachment_count: rawAttachments.length, }, - 'email', - { - from, - subject, - receivedAt: created_at, - messageId: message_id, - bodyText, - resendEmailId: email_id, - kindHint, - } - ) - return NextResponse.json( - { data: { processed: 1, reason: 'email_body', inbox_item_id: result.inbox_item_id } } - ) + actor: { type: 'system', id: 'resend-inbound' }, + occurredAt: new Date(), + }) } catch (err) { - // Fall through to the error row so the mail never vanishes. - console.error('[invoice-inbox/inbound] Email-body document failed:', err) + console.error('[invoice-inbox/inbound] RateLimitedDropped append failed:', err) + } + return { processed: 0, reason: 'rate_limited', attachments: [] } + } + + if (truncatedCount > 0) { + try { + await appendProcessingHistory({ + companyId, + correlationId: email_id, + aggregateType: 'System', + aggregateId: email_id, + eventType: 'AttachmentsTruncated', + // Counts only, for the same reason as RateLimitedDropped above. + payload: { + total: totalAttachments, + processed: attachments.length, + dropped: truncatedCount, + }, + actor: { type: 'system', id: 'resend-inbound' }, + occurredAt: new Date(), + }) + } catch (err) { + console.error('[invoice-inbox/inbound] AttachmentsTruncated append failed:', err) } } - await serviceSupabase.from('invoice_inbox_items').insert({ - company_id: companyId, - user_id: userId, - status: 'error', - source: 'email', - email_from: from, - email_subject: subject, - email_received_at: created_at, - email_body_text: bodyText, - resend_email_id: email_id, - kind_hint: kindHint, - error_message: 'Email had no attachments', - raw_email_payload: { messageId: message_id }, - }) - return NextResponse.json({ data: { processed: 0, reason: 'no_attachments' } }) - } - const results: Array<{ attachment_id: string; inbox_item_id?: string; error?: string; duplicate?: boolean }> = [] - - // Persist a "rejected" inbox row so the user has visibility into the drop. - // Without this, attachments that fail MIME validation vanish silently, - // a common Gmail "forward as attachment" foot-gun until we added .eml - // handling below. - const logRejection = async ( - attachmentId: string, - attachmentName: string | null, - mime: string, - reason: string, - ) => { - // attachment_name and mime are attacker-controlled (they come from the - // forwarded email headers); sanitise before they land in the JSONB - // raw_email_payload column so they can't surface as injection or - // oversized values when read back into the UI / audit trails. - try { + if (attachments.length === 0) { + // Body-only mail: for many suppliers the HTML body IS the invoice + // (SaaS receipts, e-mail invoices), and often the only underlag the + // user has. Store the body as a text/html document and run the + // normal extract pipeline instead of dead-ending in an error row. + // Mails with an empty body keep the old error row. + const bodyDoc = buildEmailBodyHtmlDocument(fullEmail.html ?? null, bodyText) + if (bodyDoc && bodyDoc.byteLength <= MAX_FILE_SIZE) { + // Resend retries the webhook on failure: a retry after success + // must not duplicate the body document. Body items carry the + // email_id with a NULL attachment id. Scoped to the company: + // one mail to two inboxes files once per inbox (#2181). + const { data: existingBody } = await serviceSupabase + .from('invoice_inbox_items') + .select('id') + .eq('company_id', companyId) + .eq('resend_email_id', email_id) + .is('resend_attachment_id', null) + .maybeSingle() + if (existingBody) { + return { processed: 0, reason: 'email_body_duplicate', inbox_item_id: existingBody.id, attachments: [] } + } + try { + const result = await uploadAndExtract( + serviceSupabase, + userId, + companyId, + { + name: `mail-${sanitiseFilename(subject, 'meddelande')}.html`, + buffer: bodyDoc, + type: 'text/html', + }, + 'email', + { + from, + subject, + receivedAt: created_at, + messageId: message_id, + bodyText, + resendEmailId: email_id, + kindHint, + } + ) + return { processed: 1, reason: 'email_body', inbox_item_id: result.inbox_item_id, attachments: [] } + } catch (err) { + // Fall through to the error row so the mail never vanishes. + console.error('[invoice-inbox/inbound] Email-body document failed:', err) + } + } await serviceSupabase.from('invoice_inbox_items').insert({ company_id: companyId, user_id: userId, @@ -2010,61 +2067,172 @@ export const invoiceInboxExtension: Extension = { email_received_at: created_at, email_body_text: bodyText, resend_email_id: email_id, - resend_attachment_id: attachmentId, kind_hint: kindHint, - error_message: reason.slice(0, 500), - raw_email_payload: { - messageId: message_id, - attachment_name: sanitiseFilename(attachmentName, 'unknown'), - mime: sanitiseMime(mime), - }, + error_message: 'Email had no attachments', + raw_email_payload: { messageId: message_id }, }) - } catch (insertErr) { - console.error('[invoice-inbox/inbound] Failed to log rejected attachment:', insertErr) + return { processed: 0, reason: 'no_attachments', attachments: [] } } - } - for (const att of attachments) { - try { - const { data: existing } = await serviceSupabase - .from('invoice_inbox_items') - .select('id') - .eq('resend_email_id', email_id) - .eq('resend_attachment_id', att.id) - .maybeSingle() - if (existing) { - results.push({ attachment_id: att.id, inbox_item_id: existing.id, duplicate: true }) - continue + const results: AttachmentResult[] = [] + + // Persist a "rejected" inbox row so the user has visibility into the drop. + // Without this, attachments that fail MIME validation vanish silently, + // a common Gmail "forward as attachment" foot-gun until we added .eml + // handling below. + const logRejection = async ( + attachmentId: string, + attachmentName: string | null, + mime: string, + reason: string, + // A catch-path row: the attachment itself was fine, our side + // failed. A Resend retry may replace it (see the loop below). + transient = false, + ): Promise => { + // attachment_name and mime are attacker-controlled (they come from the + // forwarded email headers); sanitise before they land in the JSONB + // raw_email_payload column so they can't surface as injection or + // oversized values when read back into the UI / audit trails. + try { + const { data: row } = await serviceSupabase + .from('invoice_inbox_items') + .insert({ + company_id: companyId, + user_id: userId, + status: 'error', + source: 'email', + email_from: from, + email_subject: subject, + email_received_at: created_at, + email_body_text: bodyText, + resend_email_id: email_id, + resend_attachment_id: attachmentId, + kind_hint: kindHint, + error_message: reason.slice(0, 500), + raw_email_payload: { + messageId: message_id, + attachment_name: sanitiseFilename(attachmentName, 'unknown'), + mime: sanitiseMime(mime), + ...(transient ? { transient: true } : {}), + }, + }) + .select('id') + .maybeSingle() + return row?.id ?? undefined + } catch (insertErr) { + console.error('[invoice-inbox/inbound] Failed to log rejected attachment:', insertErr) + return undefined } + } - const download = await fetchInboundAttachment(email_id, att.id) + for (const att of attachments) { + let replacedItemId: string | undefined + try { + // Scoped to the company so one mail to two inboxes files once + // per inbox rather than treating the second as a retry (#2181). + const { data: existing } = await serviceSupabase + .from('invoice_inbox_items') + .select('id, status, raw_email_payload') + .eq('company_id', companyId) + .eq('resend_email_id', email_id) + .eq('resend_attachment_id', att.id) + .maybeSingle() + if (existing) { + // A row the catch below wrote for a failure on our side + // (download, storage) is not a filing: a redelivery gets to + // try again, as it did before the row existed. The row is + // replaced so the unique key stays free; if the retry fails + // too, the catch writes a fresh one. Rejections (bad type, + // too large) and filed rows stay duplicates. + const transient = + existing.status === 'error' && + (existing.raw_email_payload as { transient?: unknown } | null)?.transient === true + if (!transient) { + results.push({ attachment_id: att.id, inbox_item_id: existing.id, duplicate: true }) + attachmentOutcomes.push({ id: att.id, outcome: 'duplicate', inbox_item_id: existing.id }) + continue + } + await serviceSupabase.from('invoice_inbox_items').delete().eq('id', existing.id) + replacedItemId = existing.id + } - // Gmail "Forward as attachment" wraps the original email as message/rfc822. - // Unwrap it and process the inner attachments as if they had arrived - // directly, carrying the inner email's subject/from into our metadata. - if (download.contentType === 'message/rfc822') { - const parsed = await simpleParser(Buffer.from(download.buffer)) - const innerAttachments = parsed.attachments || [] - const innerFrom = parsed.from?.text || from - const innerSubject = parsed.subject || subject - if (innerAttachments.length === 0) { - // Gmail "Forward as attachment" of a body-only HTML invoice: - // the forwarded mail's body is the underlag. Same treatment - // as a direct body-only mail; empty bodies keep the rejection. - const innerBodyDoc = buildEmailBodyHtmlDocument( - typeof parsed.html === 'string' ? parsed.html : null, - parsed.text ?? null - ) - if (innerBodyDoc && innerBodyDoc.byteLength <= MAX_FILE_SIZE) { - const innerBodyResult = await uploadAndExtract( + const download = await fetchInboundAttachment(email_id, att.id) + + // Gmail "Forward as attachment" wraps the original email as message/rfc822. + // Unwrap it and process the inner attachments as if they had arrived + // directly, carrying the inner email's subject/from into our metadata. + if (download.contentType === 'message/rfc822') { + const parsed = await simpleParser(Buffer.from(download.buffer)) + const innerAttachments = parsed.attachments || [] + const innerFrom = parsed.from?.text || from + const innerSubject = parsed.subject || subject + if (innerAttachments.length === 0) { + // Gmail "Forward as attachment" of a body-only HTML invoice: + // the forwarded mail's body is the underlag. Same treatment + // as a direct body-only mail; empty bodies keep the rejection. + const innerBodyDoc = buildEmailBodyHtmlDocument( + typeof parsed.html === 'string' ? parsed.html : null, + parsed.text ?? null + ) + if (innerBodyDoc && innerBodyDoc.byteLength <= MAX_FILE_SIZE) { + const innerBodyResult = await uploadAndExtract( + serviceSupabase, + userId, + companyId, + { + name: `mail-${sanitiseFilename(innerSubject, 'meddelande')}.html`, + buffer: innerBodyDoc, + type: 'text/html', + }, + 'email', + { + from: innerFrom, + subject: innerSubject, + receivedAt: created_at, + messageId: message_id, + bodyText, + resendEmailId: email_id, + resendAttachmentId: att.id, + kindHint, + } + ) + results.push({ attachment_id: att.id, inbox_item_id: innerBodyResult.inbox_item_id }) + attachmentOutcomes.push({ id: att.id, outcome: 'filed', inbox_item_id: innerBodyResult.inbox_item_id }) + continue + } + const rejectedId = await logRejection(att.id, download.filename, download.contentType, 'Det vidarebefordrade meddelandet innehöll inga bilagor') + results.push({ attachment_id: att.id, error: 'eml_no_inner_attachments' }) + attachmentOutcomes.push({ id: att.id, outcome: 'rejected', reason: 'eml_no_inner_attachments', inbox_item_id: rejectedId }) + continue + } + for (let i = 0; i < innerAttachments.length; i++) { + const inner = innerAttachments[i] + const innerType = sanitiseMime(inner.contentType) + const innerName = sanitiseFilename(inner.filename, `attachment-${i}`) + const innerBuffer = inner.content + if (!innerBuffer) continue + const innerId = `${att.id}#${i}` + if (!EMAIL_ALLOWED_MIME_TYPES.has(innerType)) { + const rejectedId = await logRejection(innerId, innerName, innerType, `Avvisad bilaga från vidarebefordrat mejl: filtypen ${innerType} stöds inte`) + results.push({ attachment_id: innerId, error: `Unsupported type ${innerType}` }) + attachmentOutcomes.push({ id: innerId, outcome: 'rejected', reason: 'unsupported_type', mime: innerType, inbox_item_id: rejectedId }) + continue + } + if (innerBuffer.byteLength > MAX_FILE_SIZE) { + const rejectedId = await logRejection(innerId, innerName, innerType, 'Bilagan i det vidarebefordrade mejlet är för stor') + results.push({ attachment_id: innerId, error: 'Inner attachment too large' }) + attachmentOutcomes.push({ id: innerId, outcome: 'rejected', reason: 'too_large', inbox_item_id: rejectedId }) + continue + } + const innerArrayBuffer = + innerType === 'text/html' + ? ensureHtmlDocument(innerBuffer.toString('utf8')) + : new Uint8Array(innerBuffer).buffer + const innerResult = await uploadAndExtract( serviceSupabase, userId, companyId, - { - name: `mail-${sanitiseFilename(innerSubject, 'meddelande')}.html`, - buffer: innerBodyDoc, - type: 'text/html', - }, + { name: innerName, buffer: innerArrayBuffer, type: innerType }, 'email', { from: innerFrom, @@ -2073,106 +2241,268 @@ export const invoiceInboxExtension: Extension = { messageId: message_id, bodyText, resendEmailId: email_id, - resendAttachmentId: att.id, + resendAttachmentId: innerId, kindHint, } ) - results.push({ attachment_id: att.id, inbox_item_id: innerBodyResult.inbox_item_id }) - continue + results.push({ attachment_id: innerId, inbox_item_id: innerResult.inbox_item_id }) + attachmentOutcomes.push({ id: innerId, outcome: 'filed', inbox_item_id: innerResult.inbox_item_id }) } - await logRejection(att.id, download.filename, download.contentType, 'Det vidarebefordrade meddelandet innehöll inga bilagor') - results.push({ attachment_id: att.id, error: 'eml_no_inner_attachments' }) continue } - for (let i = 0; i < innerAttachments.length; i++) { - const inner = innerAttachments[i] - const innerType = sanitiseMime(inner.contentType) - const innerName = sanitiseFilename(inner.filename, `attachment-${i}`) - const innerBuffer = inner.content - if (!innerBuffer) continue - const innerId = `${att.id}#${i}` - if (!EMAIL_ALLOWED_MIME_TYPES.has(innerType)) { - await logRejection(innerId, innerName, innerType, `Avvisad bilaga från vidarebefordrat mejl: filtypen ${innerType} stöds inte`) - results.push({ attachment_id: innerId, error: `Unsupported type ${innerType}` }) - continue + + if (!EMAIL_ALLOWED_MIME_TYPES.has(download.contentType)) { + const rejectedId = await logRejection(att.id, download.filename, download.contentType, `Avvisad: filtypen ${download.contentType} stöds inte`) + results.push({ attachment_id: att.id, error: `Unsupported type ${download.contentType}` }) + attachmentOutcomes.push({ id: att.id, outcome: 'rejected', reason: 'unsupported_type', mime: sanitiseMime(download.contentType), inbox_item_id: rejectedId }) + continue + } + if (download.buffer.byteLength > MAX_FILE_SIZE) { + const rejectedId = await logRejection(att.id, download.filename, download.contentType, 'Bilagan är för stor') + results.push({ attachment_id: att.id, error: 'Attachment too large' }) + attachmentOutcomes.push({ id: att.id, outcome: 'rejected', reason: 'too_large', inbox_item_id: rejectedId }) + continue + } + + // Attached .html invoices are often fragments; wrap them into a + // self-contained document so the archive holds a renderable file. + const attachmentBuffer = + download.contentType === 'text/html' + ? ensureHtmlDocument(Buffer.from(download.buffer).toString('utf8')) + : download.buffer + + const result = await uploadAndExtract( + serviceSupabase, + userId, + companyId, + { name: download.filename, buffer: attachmentBuffer, type: download.contentType }, + 'email', + { + from, + subject, + receivedAt: created_at, + messageId: message_id, + bodyText, + resendEmailId: email_id, + resendAttachmentId: att.id, + kindHint, } - if (innerBuffer.byteLength > MAX_FILE_SIZE) { - await logRejection(innerId, innerName, innerType, 'Bilagan i det vidarebefordrade mejlet är för stor') - results.push({ attachment_id: innerId, error: 'Inner attachment too large' }) - continue - } - const innerArrayBuffer = - innerType === 'text/html' - ? ensureHtmlDocument(innerBuffer.toString('utf8')) - : new Uint8Array(innerBuffer).buffer - const innerResult = await uploadAndExtract( - serviceSupabase, - userId, - companyId, - { name: innerName, buffer: innerArrayBuffer, type: innerType }, - 'email', - { - from: innerFrom, - subject: innerSubject, - receivedAt: created_at, - messageId: message_id, - bodyText, - resendEmailId: email_id, - resendAttachmentId: innerId, - kindHint, - } + ) + results.push({ attachment_id: att.id, inbox_item_id: result.inbox_item_id }) + attachmentOutcomes.push({ + id: att.id, + outcome: 'filed', + inbox_item_id: result.inbox_item_id, + ...(replacedItemId ? { replaced_item_id: replacedItemId } : {}), + }) + } catch (err) { + console.error('[invoice-inbox/inbound] Attachment processing failed:', err) + const message = err instanceof Error ? err.message : 'Unknown error' + results.push({ attachment_id: att.id, error: message }) + // The failure used to leave nothing behind: the webhook answered + // 200, Resend never retried, and the attachment was gone (#2181, + // the reporter's second PDF). An error row keeps it in the inbox + // where the user can see it and send it again. Skipped when the + // upload got as far as its own row before throwing. + let failedItemId: string | undefined + try { + const { data: partial } = await serviceSupabase + .from('invoice_inbox_items') + .select('id') + .eq('company_id', companyId) + .eq('resend_email_id', email_id) + .eq('resend_attachment_id', att.id) + .maybeSingle() + failedItemId = partial?.id ?? undefined + } catch { + failedItemId = undefined + } + if (!failedItemId) { + failedItemId = await logRejection( + att.id, + att.filename ?? null, + att.content_type ?? 'application/octet-stream', + `Bilagan kunde inte tas emot: ${message.slice(0, 200)}. Skicka den igen.`, + true, ) - results.push({ attachment_id: innerId, inbox_item_id: innerResult.inbox_item_id }) } - continue + attachmentOutcomes.push({ + id: att.id, + outcome: 'failed', + inbox_item_id: failedItemId, + ...(replacedItemId ? { replaced_item_id: replacedItemId } : {}), + }) } + } - if (!EMAIL_ALLOWED_MIME_TYPES.has(download.contentType)) { - await logRejection(att.id, download.filename, download.contentType, `Avvisad: filtypen ${download.contentType} stöds inte`) - results.push({ attachment_id: att.id, error: `Unsupported type ${download.contentType}` }) - continue - } - if (download.buffer.byteLength > MAX_FILE_SIZE) { - await logRejection(att.id, download.filename, download.contentType, 'Bilagan är för stor') - results.push({ attachment_id: att.id, error: 'Attachment too large' }) - continue - } + return { processed: results.length, results, attachments: attachmentOutcomes } + } - // Attached .html invoices are often fragments; wrap them into a - // self-contained document so the archive holds a renderable file. - const attachmentBuffer = - download.contentType === 'text/html' - ? ensureHtmlDocument(Buffer.from(download.buffer).toString('utf8')) - : download.buffer - - const result = await uploadAndExtract( - serviceSupabase, - userId, - companyId, - { name: download.filename, buffer: attachmentBuffer, type: download.contentType }, - 'email', - { - from, - subject, - receivedAt: created_at, - messageId: message_id, - bodyText, - resendEmailId: email_id, - resendAttachmentId: att.id, - kindHint, - } - ) - results.push({ attachment_id: att.id, inbox_item_id: result.inbox_item_id }) - } catch (err) { - console.error('[invoice-inbox/inbound] Attachment processing failed:', err) - results.push({ - attachment_id: att.id, - error: err instanceof Error ? err.message : 'Unknown error', + const outcomes: Array<{ target: InboundTarget; outcome: TargetOutcome }> = [] + for (const target of targets) { + const outcome = await processForTarget(target) + outcomes.push({ target, outcome }) + // One durable record per mail and inbox, whatever became of it + // (#2181): the inbox list only shows rows that were filed, so a + // mail whose every attachment failed had no trace a user could + // find. Ids and a closed vocabulary only: no sender, no subject, + // no address (the local part of an enskild firma's inbox is the + // owner's name) and no sender-typed tag. processing_history is + // append-only and outside the erasure path, and the PII validator + // in appendProcessingHistory would refuse a numeric tag outright, + // losing the one record this exists to keep. The panel derives + // the address from inbox_id at read time. + try { + await appendProcessingHistory({ + companyId: target.companyId, + correlationId: email_id, + aggregateType: 'System', + aggregateId: email_id, + eventType: 'InboundMailReceived', + payload: { + inbox_id: target.inboxId, + custom_domain: target.customDomain, + tags: target.tags, + unknown_tag_count: target.unknownTagCount, + kind_hint: target.kindHint, + tag_conflict: target.tagConflict, + outcome: outcome.reason ?? 'attachments', + attachment_count: totalAttachments, + inbox_item_id: outcome.inbox_item_id ?? null, + attachments: outcome.attachments, + }, + actor: { type: 'system', id: 'resend-inbound' }, + occurredAt: new Date(), }) + } catch (err) { + console.error('[invoice-inbox/inbound] InboundMailReceived append failed:', err) } } - return NextResponse.json({ data: { processed: results.length, results } }) + for (const target of deferredTargets) { + try { + await appendProcessingHistory({ + companyId: target.companyId, + correlationId: email_id, + aggregateType: 'System', + aggregateId: email_id, + eventType: 'InboundMailReceived', + payload: { + inbox_id: target.inboxId, + custom_domain: target.customDomain, + tags: target.tags, + unknown_tag_count: target.unknownTagCount, + kind_hint: target.kindHint, + tag_conflict: target.tagConflict, + outcome: 'fan_out_capped', + attachment_count: totalAttachments, + inbox_item_id: null, + attachments: [], + }, + actor: { type: 'system', id: 'resend-inbound' }, + occurredAt: new Date(), + }) + } catch (err) { + console.error('[invoice-inbox/inbound] InboundMailReceived (capped) append failed:', err) + } + } + + if (outcomes.length === 1 && deferredTargets.length === 0) { + const { processed, reason, inbox_item_id, results } = outcomes[0].outcome + return NextResponse.json({ + data: { + processed, + ...(reason ? { reason } : {}), + ...(inbox_item_id ? { inbox_item_id } : {}), + ...(results ? { results } : {}), + }, + }) + } + return NextResponse.json({ + data: { + processed: outcomes.reduce((sum, o) => sum + o.outcome.processed, 0), + targets: outcomes.map(({ target, outcome }) => ({ + company_id: target.companyId, + processed: outcome.processed, + ...(outcome.reason ? { reason: outcome.reason } : {}), + ...(outcome.inbox_item_id ? { inbox_item_id: outcome.inbox_item_id } : {}), + ...(outcome.results ? { results: outcome.results } : {}), + })), + ...(deferredTargets.length > 0 + ? { deferred: deferredTargets.map((t) => ({ company_id: t.companyId, reason: 'fan_out_capped' })) } + : {}), + }, + }) + }, + }, + + // ── Received-mail history (#2181) ───────────────────────── + // One row per mail and inbox from the InboundMailReceived events the + // webhook appends, so a user can tell "never arrived" from "arrived + // and was rejected" or "arrived and is hidden by a filter". Sender and + // subject are not in the payload (see the webhook); the filed item ids + // are, which is what the panel links to. + { + method: 'GET', + path: '/inbound-history', + handler: async (request: Request, ctx?: ExtensionContext) => { + if (!ctx) return NextResponse.json({ error: 'Unauthorized' }, { status: 401 }) + + const url = new URL(request.url) + const rawDays = url.searchParams.get('days') + const days = rawDays === null ? INBOUND_HISTORY_DEFAULT_DAYS : Number(rawDays) + if (!Number.isInteger(days) || days < 1 || days > INBOUND_HISTORY_MAX_DAYS) { + return NextResponse.json( + { error: `days must be an integer between 1 and ${INBOUND_HISTORY_MAX_DAYS}` }, + { status: 400 } + ) + } + const since = new Date(Date.now() - days * 24 * 60 * 60 * 1000).toISOString() + + const { data, error } = await ctx.supabase + .from('processing_history') + .select('event_id, correlation_id, occurred_at, payload') + .eq('company_id', ctx.companyId) + .eq('event_type', 'InboundMailReceived') + .gte('occurred_at', since) + .order('occurred_at', { ascending: false }) + // One past the cap: the extra row only says whether older mail in + // the window was cut off, so the panel can say so. + .limit(INBOUND_HISTORY_LIMIT + 1) + + if (error) return NextResponse.json({ error: error.message }, { status: 500 }) + const hasMore = (data ?? []).length > INBOUND_HISTORY_LIMIT + const page = (data ?? []).slice(0, INBOUND_HISTORY_LIMIT) + + // The event stores inbox_id only (no address, see the webhook); the + // company's own addresses are resolved here, for its own members. + const inboxes = new Map() + if (page.length > 0) { + const { data: rows } = await ctx.supabase + .from('company_inboxes') + .select('id, local_part, status') + .eq('company_id', ctx.companyId) + for (const row of rows ?? []) inboxes.set(row.id, { local_part: row.local_part, status: row.status }) + } + + return NextResponse.json({ + data: { + days, + has_more: hasMore, + mails: page.map((row) => { + const payload = row.payload as Record + const inbox = typeof payload.inbox_id === 'string' ? inboxes.get(payload.inbox_id) : undefined + return { + event_id: row.event_id, + email_id: row.correlation_id, + occurred_at: row.occurred_at, + ...payload, + inbox_local_part: inbox?.local_part ?? null, + inbox_status: inbox?.status ?? null, + } + }), + }, + }) }, }, diff --git a/extensions/general/invoice-inbox/lib/resend-inbound.ts b/extensions/general/invoice-inbox/lib/resend-inbound.ts index 7c569b27..aaeaa6de 100644 --- a/extensions/general/invoice-inbox/lib/resend-inbound.ts +++ b/extensions/general/invoice-inbox/lib/resend-inbound.ts @@ -86,29 +86,62 @@ export interface SharedInboxRecipient { tag: string | null } -// Parses the first recipient whose domain matches our configured inbound -// domain, returning its local_part split at the first `+` (RFC 5233 -// sub-addressing): `acme-x7f2+lev@inbox` matches the inbox row for -// `acme-x7f2` and carries tag `lev`. Before this split a +tagged mail 404ed, -// because the whole `acme-x7f2+lev` was looked up as the local_part. -// Returns null if no recipient is on the domain. -export function extractLocalPartForDomain( +// Parses every recipient whose domain matches our configured inbound domain, +// in original order, each split at the first `+` (RFC 5233 sub-addressing): +// `acme-x7f2+lev@inbox` matches the inbox row for `acme-x7f2` and carries +// tag `lev`. Duplicate addresses collapse to one entry. Returns all of them +// because one mail can name the same inbox twice with different tags, or two +// inboxes at once (#2181): reading only the first silently lost the rest. +export function extractSharedRecipientsForDomain( recipients: string[], domain: string, -): SharedInboxRecipient | null { +): SharedInboxRecipient[] { const normalized = domain.toLowerCase() + const seen = new Set() + const out: SharedInboxRecipient[] = [] for (const addr of recipients) { const match = addr.match(/^\s*([^@\s]+)@([^@\s]+?)\s*$/) if (!match) continue const [, rawLocal, addrDomain] = match if (addrDomain.toLowerCase() !== normalized) continue const lower = rawLocal.toLowerCase() + if (seen.has(lower)) continue + seen.add(lower) const plus = lower.indexOf('+') - if (plus === -1) return { localPart: lower, tag: null } + if (plus === -1) { + out.push({ localPart: lower, tag: null }) + continue + } const tag = lower.slice(plus + 1) - return { localPart: lower.slice(0, plus), tag: tag.length > 0 ? tag : null } + out.push({ localPart: lower.slice(0, plus), tag: tag.length > 0 ? tag : null }) } - return null + return out +} + +// The first shared-domain recipient, or null if no recipient is on the +// domain. Kept for callers that only need to know whether the mail is on +// the shared domain at all; the webhook reads every recipient. +export function extractLocalPartForDomain( + recipients: string[], + domain: string, +): SharedInboxRecipient | null { + return extractSharedRecipientsForDomain(recipients, domain)[0] ?? null +} + +/** Recipients grouped by inbox local part, tags in first-seen order. */ +export interface SharedInboxTarget { + localPart: string + tags: string[] +} + +export function groupSharedRecipientsByInbox(recipients: SharedInboxRecipient[]): SharedInboxTarget[] { + const byLocalPart = new Map() + for (const r of recipients) { + const tags = byLocalPart.get(r.localPart) ?? [] + if (r.tag !== null && !tags.includes(r.tag)) tags.push(r.tag) + byLocalPart.set(r.localPart, tags) + } + return Array.from(byLocalPart, ([localPart, tags]) => ({ localPart, tags })) } /** Sender-declared document kind, stored on invoice_inbox_items.kind_hint. */ @@ -123,6 +156,45 @@ export function kindHintFromTag(tag: string | null | undefined): InboxKindHint | return null } +/** The documented plus-address tags. Anything else is sender-typed free text. */ +export const KNOWN_INBOX_TAGS: readonly string[] = ['lev', 'ver'] + +/** + * Splits a mail's tags into the documented ones (safe to store: a closed + * vocabulary) and a count of the rest. The rest is never stored: the part + * after `+` is whatever the sender typed, and processing_history is + * append-only and outside the erasure path (#2181 skeptic finding). + */ +export function splitKnownInboxTags(tags: readonly string[]): { known: string[]; unknownCount: number } { + const known: string[] = [] + let unknownCount = 0 + for (const tag of tags) { + if (KNOWN_INBOX_TAGS.includes(tag)) known.push(tag) + else unknownCount += 1 + } + return { known, unknownCount } +} + +/** + * The one kind hint for a mail that reached the same inbox under several + * tags. Distinct documented tags contradict each other (+lev and +ver on one + * mail, #2181): then the sender has said nothing usable, the hint is null so + * extraction classifies, and `conflict` is recorded on the mail's history + * event. Unknown tags never count. + */ +export function resolveKindHintForTags(tags: readonly string[]): { + kindHint: InboxKindHint | null + conflict: boolean +} { + const hints = new Set() + for (const tag of tags) { + const hint = kindHintFromTag(tag) + if (hint) hints.add(hint) + } + if (hints.size > 1) return { kindHint: null, conflict: true } + return { kindHint: hints.values().next().value ?? null, conflict: false } +} + // Splits every parseable recipient into { localPart, domain }, lowercased and // in original order. Used to match recipients against per-company verified // custom domains when none of them is on the shared inbound domain. diff --git a/lib/processing-history/append.ts b/lib/processing-history/append.ts index 151e2d86..64b9533f 100644 --- a/lib/processing-history/append.ts +++ b/lib/processing-history/append.ts @@ -54,6 +54,7 @@ export const PROCESSING_EVENT_TYPES = [ 'DocumentExtractionOverridden', 'DocumentExtractionRetried', 'DocumentIngested', + 'InboundMailReceived', 'InboxUnderlagReconciled', 'InvoiceDuplicatePaymentDismissed', 'InvoiceJournalEntrySkipped', diff --git a/messages/en.json b/messages/en.json index 89519307..d07b8616 100644 --- a/messages/en.json +++ b/messages/en.json @@ -3329,6 +3329,28 @@ "kind_filter_underlag": "Receipts and other", "empty_no_kind_hits": "No items of that type here. Pick All types to see the rest.", "address_plus_hint": "Add +lev before the @ for supplier invoices and +ver for receipts: {lev} or {ver}", + "hidden_by_kind_filter": "{count, plural, =1 {1 item is hidden} other {# items are hidden}} by the type filter.", + "inbound_mail_title": "Received mail", + "inbound_mail_hint": "Every mail that reached the address in the last {days} days, including the ones that were rejected.", + "inbound_mail_empty": "No mail has reached the address in the last {days} days.", + "inbound_mail_loading": "Loading mail history…", + "inbound_mail_load_failed": "Could not load the mail history.", + "inbound_outcome_filed": "{count, plural, =1 {1 attachment received} other {# attachments received}}", + "inbound_outcome_duplicate": "{count, plural, =1 {1 duplicate} other {# duplicates}}", + "inbound_outcome_rejected": "{count, plural, =1 {1 rejected} other {# rejected}}", + "inbound_outcome_failed": "{count, plural, =1 {1 failed} other {# failed}}", + "inbound_outcome_rate_limited": "Dropped: too many mails in a short time", + "inbound_outcome_empty": "Empty mail without attachments", + "inbound_outcome_body": "Mail body saved as a document", + "inbound_outcome_body_duplicate": "Duplicate of an earlier mail", + "inbound_outcome_fan_out_capped": "Not processed: the mail went to more than five inboxes. Send it to this address on its own.", + "inbound_tag_conflict": "Both +lev and +ver on one mail: the type is decided by extraction.", + "inbound_open": "Open", + "inbound_open_nth": "Open {n}", + "inbound_mail_custom_domain": "Custom domain", + "inbound_mail_former_address": "Former inbox address", + "inbound_unknown_tags": "{count, plural, =1 {1 unknown tag} other {# unknown tags}}", + "inbound_mail_truncated": "More mail than this reached the address in the last {days} days: the latest {count} are shown.", "payment_label": "Paid with", "payment_card": "Card", "payment_swish": "Swish", diff --git a/messages/sv.json b/messages/sv.json index a4879647..393c6ad0 100644 --- a/messages/sv.json +++ b/messages/sv.json @@ -3329,6 +3329,28 @@ "kind_filter_underlag": "Underlag", "empty_no_kind_hits": "Inga poster av den typen här. Välj Alla typer för att se resten.", "address_plus_hint": "Skriv +lev före @ för leverantörsfakturor och +ver för underlag: {lev} eller {ver}", + "hidden_by_kind_filter": "{count, plural, =1 {1 post är dold} other {# poster är dolda}} av typfiltret.", + "inbound_mail_title": "Inkomna mejl", + "inbound_mail_hint": "Varje mejl som nått adressen de senaste {days} dagarna, även de som avvisades.", + "inbound_mail_empty": "Inga mejl har nått adressen de senaste {days} dagarna.", + "inbound_mail_loading": "Läser mejlhistoriken…", + "inbound_mail_load_failed": "Kunde inte läsa mejlhistoriken.", + "inbound_outcome_filed": "{count, plural, =1 {1 bilaga mottagen} other {# bilagor mottagna}}", + "inbound_outcome_duplicate": "{count, plural, =1 {1 dubblett} other {# dubbletter}}", + "inbound_outcome_rejected": "{count, plural, =1 {1 avvisad} other {# avvisade}}", + "inbound_outcome_failed": "{count, plural, =1 {1 misslyckades} other {# misslyckades}}", + "inbound_outcome_rate_limited": "Stoppat: för många mejl på kort tid", + "inbound_outcome_empty": "Tomt mejl utan bilagor", + "inbound_outcome_body": "Mejltexten sparad som underlag", + "inbound_outcome_body_duplicate": "Dubblett av ett tidigare mejl", + "inbound_outcome_fan_out_capped": "Inte behandlat: mejlet gick till fler än fem inkorgar. Skicka det till den här adressen separat.", + "inbound_tag_conflict": "Både +lev och +ver i samma mejl: typen avgörs av tolkningen.", + "inbound_open": "Öppna", + "inbound_open_nth": "Öppna {n}", + "inbound_mail_custom_domain": "Egen domän", + "inbound_mail_former_address": "Tidigare inkorgsadress", + "inbound_unknown_tags": "{count, plural, =1 {1 okänd tagg} other {# okända taggar}}", + "inbound_mail_truncated": "Fler mejl än så nådde adressen de senaste {days} dagarna: de {count} senaste visas.", "payment_label": "Betalsätt", "payment_card": "Kort", "payment_swish": "Swish", diff --git a/supabase/migrations/20260904001000_inbound_mail_received_and_inbox_dedupe_per_company.sql b/supabase/migrations/20260904001000_inbound_mail_received_and_inbox_dedupe_per_company.sql new file mode 100644 index 00000000..67eb8540 --- /dev/null +++ b/supabase/migrations/20260904001000_inbound_mail_received_and_inbox_dedupe_per_company.sql @@ -0,0 +1,70 @@ +-- Inbound mail traceability (#2181). +-- +-- 1. Register the InboundMailReceived behandlingshistorik event: one row per +-- received mail and inbox, written by the Resend inbound webhook +-- (extensions/general/invoice-inbox/index.ts) with the recipient +-- addresses, the +lev/+ver tags, the resolved kind hint and the outcome +-- per attachment (filed, duplicate, rejected, failed). A mail whose every +-- attachment failed used to leave no trace a user could find. +-- +-- processing_history.event_type has an FK to processing_event_types, so an +-- unregistered type fails the insert; the append is best-effort, so the +-- record would be silently lost. Catalog row only: aggregate_type 'System' +-- is already permitted by the CHECK. +-- +-- The event carries ids and a closed vocabulary (inbox_id, +lev/+ver +-- tags, outcome codes). The database strips any address or free text an +-- older or wrong emitter might send, the same way 20260901110000 does for +-- RateLimitedDropped and AttachmentsTruncated: the invariant belongs to +-- the table, not to one emitter's good behaviour. +-- +-- 2. Make the per-attachment idempotency key company-scoped. One mail can be +-- addressed to two companies' inbox addresses at once; the webhook now +-- files it once per inbox, and the old (resend_email_id, +-- resend_attachment_id) unique index refused the second company's row. +-- Resend retries still dedupe: the same mail, attachment and company hit +-- the new index. + +INSERT INTO public.processing_event_types (event_type) VALUES + ('InboundMailReceived') +ON CONFLICT (event_type) DO NOTHING; + +CREATE OR REPLACE FUNCTION public.strip_inbound_mail_pii_from_processing_history() +RETURNS trigger +LANGUAGE plpgsql +SET search_path = pg_catalog, public +AS $$ +BEGIN + IF NEW.event_type IN ('RateLimitedDropped', 'AttachmentsTruncated', 'InboundMailReceived') + AND jsonb_typeof(NEW.payload) = 'object' + THEN + NEW.payload := NEW.payload - 'from' - 'subject' - 'recipients' - 'to'; + END IF; + RETURN NEW; +END; +$$; + +REVOKE ALL ON FUNCTION public.strip_inbound_mail_pii_from_processing_history() + FROM PUBLIC, anon, authenticated; + +-- Recreated rather than assumed: the trigger ships in 20260901110000, but a +-- database that skipped that file (the staging branch did) would otherwise +-- carry the new function with nothing calling it. +DROP TRIGGER IF EXISTS processing_history_strip_inbound_mail_pii ON public.processing_history; + +CREATE TRIGGER processing_history_strip_inbound_mail_pii + BEFORE INSERT ON public.processing_history + FOR EACH ROW + EXECUTE FUNCTION public.strip_inbound_mail_pii_from_processing_history(); + +-- Plain DROP INDEX / CREATE INDEX (not CONCURRENTLY): Supabase branching +-- applies migrations inside a transaction, where CONCURRENTLY is not allowed +-- (same call as 20260706120000 and 20260710101000). invoice_inbox_items is a +-- few thousand rows on prod; both statements take milliseconds. +DROP INDEX IF EXISTS public.idx_invoice_inbox_items_resend_email_attachment; + +CREATE UNIQUE INDEX IF NOT EXISTS idx_invoice_inbox_items_company_resend_email_attachment + ON public.invoice_inbox_items(company_id, resend_email_id, resend_attachment_id) + WHERE resend_email_id IS NOT NULL; + +NOTIFY pgrst, 'reload schema'; diff --git a/tests/pg/inbox-inbound-mail-received.pg.test.ts b/tests/pg/inbox-inbound-mail-received.pg.test.ts new file mode 100644 index 00000000..d44e613a --- /dev/null +++ b/tests/pg/inbox-inbound-mail-received.pg.test.ts @@ -0,0 +1,147 @@ +import { describe, it, expect } from 'vitest' +import { randomUUID } from 'node:crypto' +import { getClient, getPool } from '@/tests/pg/setup' +import { seedCompany } from '@/tests/pg/fixtures' + +/** + * Migration 20260904001000 (#2181): the InboundMailReceived catalog row, and + * the per-attachment idempotency index on invoice_inbox_items becoming + * company-scoped so one mail addressed to two inboxes files once per inbox. + * tests/pg/processing-event-types.pg.test.ts already proves every emitted + * type is registered; this file pins the two behaviours the webhook now + * relies on. + */ +describe('InboundMailReceived event type (#2181)', () => { + it('is registered in the catalog', async () => { + const { rows } = await getPool().query( + `SELECT 1 FROM public.processing_event_types WHERE event_type = 'InboundMailReceived'`, + ) + expect(rows).toHaveLength(1) + }) + + it('still refuses an unregistered event type through the FK', async () => { + const { companyId } = await seedCompany() + const client = await getClient() + try { + await client.query('BEGIN') + await expect( + client.query( + `INSERT INTO public.processing_history + (company_id, correlation_id, aggregate_type, aggregate_id, event_type, + payload, actor, occurred_at) + VALUES ($1, $2, 'System', $2, 'InboundMailReceivedTypo', '{}'::jsonb, + '{"type":"system","id":"inbound-mail-received-test"}', now())`, + [companyId, randomUUID()], + ), + ).rejects.toMatchObject({ code: '23503' }) + } finally { + await client.query('ROLLBACK').catch(() => {}) + client.release() + } + }) +}) + +describe('InboundMailReceived DB-side PII strip (#2181)', () => { + it('drops address and free-text keys on insert but keeps the ids and codes', async () => { + const { companyId } = await seedCompany() + const client = await getClient() + try { + await client.query('BEGIN') + const aggregateId = randomUUID() + const { rows } = await client.query<{ payload: Record }>( + `INSERT INTO public.processing_history + (company_id, correlation_id, aggregate_type, aggregate_id, event_type, + payload, actor, occurred_at) + VALUES ($1, $2, 'System', $2, 'InboundMailReceived', $3::jsonb, + '{"type":"system","id":"inbound-mail-received-test"}', now()) + RETURNING payload`, + [ + companyId, + aggregateId, + JSON.stringify({ + recipients: ['anna-andersson-x7f2+lev@example.test'], + to: ['anna-andersson-x7f2+lev@example.test'], + from: 'avsandare@example.test', + subject: 'Faktura', + inbox_id: aggregateId, + tags: ['lev'], + unknown_tag_count: 0, + outcome: 'attachments', + }), + ], + ) + expect(rows[0].payload).not.toHaveProperty('recipients') + expect(rows[0].payload).not.toHaveProperty('to') + expect(rows[0].payload).not.toHaveProperty('from') + expect(rows[0].payload).not.toHaveProperty('subject') + expect(rows[0].payload).toMatchObject({ inbox_id: aggregateId, tags: ['lev'], unknown_tag_count: 0, outcome: 'attachments' }) + } finally { + await client.query('ROLLBACK').catch(() => {}) + client.release() + } + }) +}) + +describe('invoice_inbox_items idempotency per company (#2181)', () => { + async function insertItem( + client: Awaited>, + companyId: string, + userId: string, + emailId: string, + attachmentId: string | null, + ) { + return client.query( + `INSERT INTO public.invoice_inbox_items + (company_id, user_id, status, source, resend_email_id, resend_attachment_id) + VALUES ($1, $2, 'received', 'email', $3, $4) + RETURNING id`, + [companyId, userId, emailId, attachmentId], + ) + } + + it('lets two companies file the same mail attachment, and refuses a repeat for one company', async () => { + const a = await seedCompany() + const b = await seedCompany() + const emailId = randomUUID() + const attachmentId = randomUUID() + const client = await getClient() + try { + await client.query('BEGIN') + await insertItem(client, a.companyId, a.userId, emailId, attachmentId) + // Before the migration this insert hit the (email, attachment) unique + // index and the second inbox's copy was lost. + await insertItem(client, b.companyId, b.userId, emailId, attachmentId) + + // A Resend retry for the same company still dedupes at the index. + await client.query('SAVEPOINT repeat') + await expect( + insertItem(client, a.companyId, a.userId, emailId, attachmentId), + ).rejects.toMatchObject({ code: '23505' }) + await client.query('ROLLBACK TO SAVEPOINT repeat') + + const { rows } = await client.query<{ n: string }>( + `SELECT count(*)::text AS n FROM public.invoice_inbox_items + WHERE resend_email_id = $1 AND resend_attachment_id = $2`, + [emailId, attachmentId], + ) + expect(rows[0].n).toBe('2') + } finally { + await client.query('ROLLBACK').catch(() => {}) + client.release() + } + }) + + it('keeps the old single-company index gone', async () => { + const { rows } = await getPool().query<{ indexname: string }>( + `SELECT indexname FROM pg_indexes + WHERE tablename = 'invoice_inbox_items' + AND indexname IN ( + 'idx_invoice_inbox_items_resend_email_attachment', + 'idx_invoice_inbox_items_company_resend_email_attachment' + )`, + ) + expect(rows.map((r) => r.indexname)).toEqual([ + 'idx_invoice_inbox_items_company_resend_email_attachment', + ]) + }) +})