fix(inbox): trace every received mail and file multi-recipient mail once per inbox (#2181) (#2244)

* fix(inbox): trace every received mail and file multi-recipient mail once per inbox (#2181)

A mail sent to both the +lev and +ver address of one inbox was read as
its first recipient only, and an attachment whose processing threw left
no row at all: the webhook answered 200, Resend never retried, and the
document was gone with nothing for the user to find. Prod showed both
shapes for the reporter (a +lev mail Resend accepted with zero inbox
rows, and the second PDF of the +ver mail missing).

- The webhook now reads every shared-domain recipient, groups them per
  inbox, files once per inbox with a company-scoped dedupe key, and
  resolves contradicting tags (+lev and +ver on one mail) to no hint so
  extraction classifies.
- The per-attachment catch writes an error row instead of only a
  console line.
- One InboundMailReceived behandlingshistorik event per mail and inbox
  records recipients, tags, hint, conflict and the outcome per
  attachment (filed, duplicate, rejected, failed). No sender or
  subject, matching the existing PII rule.
- GET /inbound-history?days=30 serves those events, company-scoped, and
  the inbox workspace shows them under Källor as "Inkomna mejl", each
  filed row a click away.
- The list says how many rows the type filter is hiding, with a click
  back to all types.
- Migration 20260903190000 registers the event type and replaces the
  (email, attachment) unique index with (company, email, attachment).

Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01CoG2CXf8B33Q5wp8gk4kW4

* fix(inbox): keep addresses and sender-typed tags out of the mail record, and let redelivery heal a transient failure

Skeptic pass on #2244, two refutations:

- The InboundMailReceived payload carried the recipient addresses and every
  plus-tag verbatim. An enskild firma's inbox local part is the owner's
  name, the tag is whatever the sender typed, and processing_history is
  append-only and outside the erasure path; a numeric tag also tripped the
  PII validator so the record was silently dropped. The event now carries
  inbox_id, the documented tags (+lev/+ver), an unknown-tag count and the
  outcome codes. The history route resolves inbox_id to the company's own
  address at read time. The DB strip trigger from 20260901110000 covers the
  new type (and is recreated, since staging skipped that file).
- The catch-path error row made a Resend redelivery report "duplicate", so
  a transient download or storage failure that used to self-heal on retry
  became permanent. The row is marked transient and a redelivery replaces
  it; rejections (bad type, too large) stay duplicates.

Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01CoG2CXf8B33Q5wp8gk4kW4

* fix(inbox): cap inbound fan-out, flag a truncated mail history, and name a replaced transient row

Review pass on #2244: Superagent (bound the number of inboxes one mail can
fan out to: five), CodeRabbit (the history route now returns has_more past
200 rows and the panel says so instead of "every mail"), and the Swedish
accounting review (a redelivery that replaces a transient error row names
the replaced row on the InboundMailReceived record, so the replacement
leaves a trace). The migration comment states why the index swap is not
CONCURRENTLY: Supabase branching applies migrations in a transaction.

Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>

* fix(inbox): resolve every addressed inbox and record the ones past the fan-out cap

CodeRabbit and the Swedish accounting review on #2244: slicing recipient
groups before the lookup let five unknown local parts starve a real inbox
and left companies past the cap with no trace. Every addressed inbox is
now resolved (one cheap lookup each), the first five are processed, and
the rest get their own InboundMailReceived record with outcome
fan_out_capped, shown in the panel as "not processed".

Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>

* chore(inbox): move the inbound-mail migration past the parties versions merged tonight

Main moved party_decision_undo to 20260904000100 and added
20260904000200 (#2257, #2258). A version below prod's head is skipped by
Supabase branching, so 20260903190000 becomes 20260904001000 unchanged.
Staging re-tracked under the new version.

Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>

---------

Co-authored-by: Claude Fable 5.1 <noreply@anthropic.com>
This commit is contained in:
Mattsson
2026-09-03 20:47:24 +02:00
committed by GitHub
co-authored by Claude Fable 5.1
parent b07efcafd4
commit 88a5d78594
13 changed files with 1846 additions and 312 deletions
@@ -37,6 +37,7 @@ vi.mock('@/extensions/general/invoice-inbox/lib/upload-and-extract', async () =>
)
return { ...actual, uploadAndExtract: vi.fn() }
})
import { uploadAndExtract } from '@/extensions/general/invoice-inbox/lib/upload-and-extract'
vi.mock('@/lib/rate-limits/inbox', () => ({
checkInboxUploadRateLimit: vi.fn(),
@@ -181,6 +182,49 @@ describe('POST /inbound behandlingshistorik payloads', () => {
expect(JSON.stringify(payload)).not.toContain(SUBJECT)
})
it('records InboundMailReceived with ids and a closed vocabulary only: no sender, subject, address or sender-typed tag', async () => {
const to = [
'Anna-Andersson-x7f2+lev@arcim.io',
'anna-andersson-x7f2+ref-19850101-1234@arcim.io',
]
vi.mocked(verifyInboundWebhook).mockReturnValue({
...mockReceivedEvent(0),
data: { ...mockReceivedEvent(0).data, to },
} as never)
vi.mocked(fetchReceivingEmail).mockResolvedValue({ ...mockFullEmail(0), to, html: '<p>Faktura</p>' } as never)
vi.mocked(checkInboxUploadRateLimit).mockResolvedValue({ ok: true } as never)
vi.mocked(uploadAndExtract).mockResolvedValue({ inbox_item_id: 'item-body-1' } as never)
const { supabase, enqueue } = createQueuedMockSupabase()
enqueue({ data: { id: 'inbox-1', company_id: 'company-1', status: 'active' } })
enqueue({ data: { created_by: 'user-owner-1' } })
enqueue({ data: null }) // body-document dedupe check
vi.mocked(createClient).mockReturnValue(supabase as never)
const res = await webhookRoute.handler(createMockRequest('/inbound', { method: 'POST', body: {} }))
expect(res.status).toBe(200)
const payload = historyPayload('InboundMailReceived')
expect(payload).toEqual({
inbox_id: 'inbox-1',
custom_domain: false,
tags: ['lev'],
unknown_tag_count: 1,
kind_hint: 'supplier_invoice',
tag_conflict: false,
outcome: 'email_body',
attachment_count: 0,
inbox_item_id: 'item-body-1',
attachments: [],
})
const json = JSON.stringify(payload)
expect(json).not.toContain(SENDER)
expect(json).not.toContain(SUBJECT)
expect(json).not.toContain('@')
expect(json).not.toContain('anna')
expect(json).not.toContain('19850101')
})
it('keeps the mail traceable through the Resend email id', async () => {
vi.mocked(verifyInboundWebhook).mockReturnValue(mockReceivedEvent(1) as never)
vi.mocked(fetchReceivingEmail).mockResolvedValue(mockFullEmail(1) as never)
@@ -0,0 +1,138 @@
import { describe, it, expect, vi, beforeEach } from 'vitest'
import { invoiceInboxExtension } from '@/extensions/general/invoice-inbox'
import { createQueuedMockSupabase, createMockRequest, parseJsonResponse } from '@/tests/helpers'
import type { ExtensionContext } from '@/lib/extensions/types'
vi.mock('@/lib/rate-limits/inbox', () => ({
checkInboxUploadRateLimit: vi.fn().mockResolvedValue({ ok: true }),
}))
const route = invoiceInboxExtension.apiRoutes!.find(
(r) => r.method === 'GET' && r.path === '/inbound-history'
)!
function buildCtx(supabase: unknown): ExtensionContext {
return {
userId: 'user-1',
companyId: 'company-1',
extensionId: 'invoice-inbox',
supabase: supabase as ExtensionContext['supabase'],
emit: vi.fn(),
settings: { get: vi.fn(), set: vi.fn() },
storage: { from: vi.fn() } as unknown as ExtensionContext['storage'],
log: { info: vi.fn(), warn: vi.fn(), error: vi.fn(), debug: vi.fn() } as unknown as ExtensionContext['log'],
services: {},
} as unknown as ExtensionContext
}
const EVENT_ROW = {
event_id: 'evt-1',
correlation_id: '6ec3164f-1f59-40e7-acd2-ebedf9c4d56e',
occurred_at: '2026-09-02T14:21:56Z',
payload: {
inbox_id: 'inbox-1',
custom_domain: false,
tags: ['ver'],
unknown_tag_count: 0,
kind_hint: 'receipt',
tag_conflict: false,
outcome: 'attachments',
attachment_count: 2,
inbox_item_id: null,
attachments: [
{ id: 'att-1', outcome: 'filed', inbox_item_id: 'item-1' },
{ id: 'att-2', outcome: 'failed', inbox_item_id: 'item-2' },
],
},
}
describe('GET /inbound-history (#2181)', () => {
beforeEach(() => vi.clearAllMocks())
it('returns 401 without context', async () => {
const res = await route.handler(createMockRequest('/inbound-history'), undefined)
expect(res.status).toBe(401)
})
it.each(['0', '-1', '366', 'abc', '1.5'])('returns 400 for days=%s', async (days) => {
const { supabase } = createQueuedMockSupabase()
const res = await route.handler(
createMockRequest(`/inbound-history?days=${days}`),
buildCtx(supabase)
)
expect(res.status).toBe(400)
})
it('returns the company-scoped InboundMailReceived events, newest first, 30 days by default', async () => {
const { supabase, enqueue, calls } = createQueuedMockSupabase()
enqueue({ data: [EVENT_ROW] })
enqueue({ data: [{ id: 'inbox-1', local_part: 'acme-ab-x7f2', status: 'active' }] }) // company's inboxes
const res = await route.handler(createMockRequest('/inbound-history'), buildCtx(supabase))
const { status, body } = await parseJsonResponse<{
data: { days: number; has_more: boolean; mails: Array<Record<string, unknown>> }
}>(res)
expect(status).toBe(200)
expect(body.data.days).toBe(30)
expect(body.data.has_more).toBe(false)
expect(body.data.mails).toEqual([
{
event_id: 'evt-1',
email_id: '6ec3164f-1f59-40e7-acd2-ebedf9c4d56e',
occurred_at: '2026-09-02T14:21:56Z',
...EVENT_ROW.payload,
// Resolved at read time for the company's own members; the event
// itself carries no address.
inbox_local_part: 'acme-ab-x7f2',
inbox_status: 'active',
},
])
const inboxLookup = calls.find((c) => c.table === 'company_inboxes' && c.method === 'eq')
expect(inboxLookup?.args).toEqual(['company_id', 'company-1'])
const eqs = calls.filter((c) => c.table === 'processing_history' && c.method === 'eq').map((c) => c.args)
expect(eqs).toEqual([
['company_id', 'company-1'],
['event_type', 'InboundMailReceived'],
])
const gte = calls.find((c) => c.table === 'processing_history' && c.method === 'gte')
expect(gte?.args[0]).toBe('occurred_at')
const order = calls.find((c) => c.table === 'processing_history' && c.method === 'order')
expect(order?.args).toEqual(['occurred_at', { ascending: false }])
})
it('marks a mail whose inbox row is gone with no address', async () => {
const { supabase, enqueue } = createQueuedMockSupabase()
enqueue({ data: [EVENT_ROW] })
enqueue({ data: [] }) // inbox rotated away and removed
const res = await route.handler(createMockRequest('/inbound-history'), buildCtx(supabase))
const { body } = await parseJsonResponse<{ data: { mails: Array<{ inbox_local_part: string | null }> } }>(res)
expect(body.data.mails[0].inbox_local_part).toBeNull()
})
it('honours an explicit window', async () => {
const { supabase, enqueue } = createQueuedMockSupabase()
enqueue({ data: [] })
const res = await route.handler(createMockRequest('/inbound-history?days=7'), buildCtx(supabase))
const { body } = await parseJsonResponse<{ data: { days: number; mails: unknown[] } }>(res)
expect(body.data).toEqual({ days: 7, has_more: false, mails: [] })
})
it('caps the list at 200 and says when older mail in the window was cut off', async () => {
const { supabase, enqueue, calls } = createQueuedMockSupabase()
enqueue({ data: Array.from({ length: 201 }, (_, i) => ({ ...EVENT_ROW, event_id: `evt-${i}` })) })
enqueue({ data: [{ id: 'inbox-1', local_part: 'acme-ab-x7f2', status: 'active' }] })
const res = await route.handler(createMockRequest('/inbound-history'), buildCtx(supabase))
const { body } = await parseJsonResponse<{ data: { has_more: boolean; mails: unknown[] } }>(res)
expect(body.data.has_more).toBe(true)
expect(body.data.mails).toHaveLength(200)
const limit = calls.find((c) => c.table === 'processing_history' && c.method === 'limit')
expect(limit?.args).toEqual([201])
})
it('returns 500 when the read fails', async () => {
const { supabase, enqueue } = createQueuedMockSupabase()
enqueue({ data: null, error: { message: 'boom' } })
const res = await route.handler(createMockRequest('/inbound-history'), buildCtx(supabase))
expect(res.status).toBe(500)
})
})
@@ -48,9 +48,17 @@ vi.mock('@/lib/rate-limits/inbox', () => ({
checkInboxUploadRateLimit: vi.fn().mockResolvedValue({ ok: true }),
}))
// The webhook appends one InboundMailReceived event per mail and inbox
// (#2181); keep the history write off the network and observable.
vi.mock('@/lib/processing-history/append', () => ({
appendProcessingHistory: vi.fn().mockResolvedValue('event-id'),
}))
import { verifyInboundWebhook, fetchReceivingEmail, fetchInboundAttachment } from '@/extensions/general/invoice-inbox/lib/resend-inbound'
import { uploadAndExtract } from '@/extensions/general/invoice-inbox/lib/upload-and-extract'
import { createClient } from '@supabase/supabase-js'
import { appendProcessingHistory } from '@/lib/processing-history/append'
import { checkInboxUploadRateLimit } from '@/lib/rate-limits/inbox'
function findRoute(method: string, path: string) {
return invoiceInboxExtension.apiRoutes!.find((r) => r.method === method && r.path === path)!
@@ -721,3 +729,396 @@ describe('POST /inbound', () => {
expect(rejection?.args[0]).toMatchObject({ status: 'error', kind_hint: 'receipt' })
})
})
/** Every InboundMailReceived append, in order. */
function receivedEvents() {
return vi
.mocked(appendProcessingHistory)
.mock.calls.map(([input]) => input)
.filter((input) => input.eventType === 'InboundMailReceived')
}
function fullEmailFor(to: string[], attachments: unknown[], overrides: Record<string, unknown> = {}) {
return {
object: 'email',
id: 'em_123',
to,
from: 'billing@supplier.com',
created_at: '2026-04-20T10:00:00Z',
subject: 'Faktura',
bcc: null,
cc: null,
reply_to: null,
html: null,
text: 'Se bifogad faktura',
headers: {},
message_id: '<msg@x>',
raw: null,
attachments,
...overrides,
}
}
const PDF_ATTACHMENT = {
id: 'att_1',
filename: 'faktura.pdf',
size: 100,
content_type: 'application/pdf',
content_id: 'cid',
content_disposition: 'attachment',
}
const PDF_DOWNLOAD = {
id: 'att_1',
filename: 'faktura.pdf',
contentType: 'application/pdf',
buffer: new Uint8Array([0x25, 0x50, 0x44, 0x46]).buffer as ArrayBuffer,
}
describe('POST /inbound with several recipients (#2181)', () => {
const originalEnv = { ...process.env }
beforeEach(() => {
vi.clearAllMocks()
process.env.RESEND_INBOUND_DOMAIN = 'arcim.io'
process.env.NEXT_PUBLIC_SUPABASE_URL = 'http://localhost'
process.env.SUPABASE_SERVICE_ROLE_KEY = 'test-service-key'
vi.mocked(appendProcessingHistory).mockResolvedValue('event-id')
})
afterEach(() => {
process.env = { ...originalEnv }
})
it('files a mail sent to +lev and +ver of the same inbox once, with no kind hint', async () => {
const to = ['acme-ab-x7f2+lev@arcim.io', 'acme-ab-x7f2+ver@arcim.io']
vi.mocked(verifyInboundWebhook).mockReturnValue(mockReceivedEvent({ to }) as never)
const { supabase, enqueue, calls } = createQueuedMockSupabase()
enqueue({ data: { id: 'inbox-1', company_id: 'company-1', status: 'active' } }) // one inbox lookup
enqueue({ data: { created_by: 'user-owner-1' } })
enqueue({ data: null }) // per-attachment dup check finds nothing
vi.mocked(createClient).mockReturnValue(supabase as never)
vi.mocked(uploadAndExtract).mockResolvedValue({ inbox_item_id: 'item-1' } as never)
vi.mocked(fetchReceivingEmail).mockResolvedValue(fullEmailFor(to, [PDF_ATTACHMENT]) as never)
vi.mocked(fetchInboundAttachment).mockResolvedValue(PDF_DOWNLOAD)
const res = await webhookRoute.handler(createMockRequest('/inbound', { method: 'POST', body: {} }))
const body = await res.json()
expect(res.status).toBe(200)
expect(body.data.processed).toBe(1)
expect(body.data.results).toEqual([{ attachment_id: 'att_1', inbox_item_id: 'item-1' }])
// One inbox lookup, one upload, no hint: the sender said two things.
expect(calls.filter((c) => c.table === 'company_inboxes' && c.method === 'eq')).toHaveLength(1)
expect(uploadAndExtract).toHaveBeenCalledTimes(1)
const [, , , , , emailMeta] = vi.mocked(uploadAndExtract).mock.calls[0]
expect(emailMeta?.kindHint).toBeNull()
const events = receivedEvents()
expect(events).toHaveLength(1)
expect(events[0].companyId).toBe('company-1')
expect(events[0].correlationId).toBe('em_123')
expect(events[0].payload).toMatchObject({
inbox_id: 'inbox-1',
custom_domain: false,
tags: ['lev', 'ver'],
unknown_tag_count: 0,
kind_hint: null,
tag_conflict: true,
outcome: 'attachments',
attachment_count: 1,
attachments: [{ id: 'att_1', outcome: 'filed', inbox_item_id: 'item-1' }],
})
expect(JSON.stringify(events[0].payload)).not.toContain('billing@supplier.com')
expect(JSON.stringify(events[0].payload)).not.toContain('Faktura')
// No address at all: an enskild firma's local part is the owner's name.
expect(JSON.stringify(events[0].payload)).not.toContain('@')
expect(JSON.stringify(events[0].payload)).not.toContain('acme-ab-x7f2')
})
it('counts a sender-typed tag without storing it, so a numeric tag cannot trip the PII validator', async () => {
const to = ['acme-ab-x7f2+8501011234@arcim.io', 'acme-ab-x7f2+lev@arcim.io']
vi.mocked(verifyInboundWebhook).mockReturnValue(mockReceivedEvent({ to, attachments: [] }) as never)
const { supabase, enqueue } = createQueuedMockSupabase()
enqueue({ data: { id: 'inbox-1', company_id: 'company-1', status: 'active' } })
enqueue({ data: { created_by: 'user-owner-1' } })
enqueue({ data: null }) // body-document dedupe check
vi.mocked(createClient).mockReturnValue(supabase as never)
vi.mocked(uploadAndExtract).mockResolvedValue({ inbox_item_id: 'item-body' } as never)
vi.mocked(fetchReceivingEmail).mockResolvedValue(fullEmailFor(to, [], { html: '<p>Kvitto</p>' }) as never)
const res = await webhookRoute.handler(createMockRequest('/inbound', { method: 'POST', body: {} }))
expect(res.status).toBe(200)
const [event] = receivedEvents()
expect(event.payload).toMatchObject({ tags: ['lev'], unknown_tag_count: 1, kind_hint: 'supplier_invoice', tag_conflict: false })
expect(JSON.stringify(event.payload)).not.toContain('8501011234')
})
it('files a mail addressed to two inboxes once per inbox, deduped per company', async () => {
const to = ['acme-ab-x7f2+lev@arcim.io', 'beta-ab-q9z1@arcim.io']
vi.mocked(verifyInboundWebhook).mockReturnValue(mockReceivedEvent({ to }) as never)
const { supabase, enqueue, calls } = createQueuedMockSupabase()
enqueue({ data: { id: 'inbox-1', company_id: 'company-1', status: 'active' } })
enqueue({ data: { id: 'inbox-2', company_id: 'company-2', status: 'active' } })
enqueue({ data: { created_by: 'user-owner-1' } })
enqueue({ data: { created_by: 'user-owner-2' } })
enqueue({ data: null }) // company-1 dup check
enqueue({ data: null }) // company-2 dup check
vi.mocked(createClient).mockReturnValue(supabase as never)
vi.mocked(uploadAndExtract)
.mockResolvedValueOnce({ inbox_item_id: 'item-c1' } as never)
.mockResolvedValueOnce({ inbox_item_id: 'item-c2' } as never)
vi.mocked(fetchReceivingEmail).mockResolvedValue(fullEmailFor(to, [PDF_ATTACHMENT]) as never)
vi.mocked(fetchInboundAttachment).mockResolvedValue(PDF_DOWNLOAD)
const res = await webhookRoute.handler(createMockRequest('/inbound', { method: 'POST', body: {} }))
const body = await res.json()
expect(res.status).toBe(200)
expect(body.data.processed).toBe(2)
expect(body.data.targets).toEqual([
{ company_id: 'company-1', processed: 1, results: [{ attachment_id: 'att_1', inbox_item_id: 'item-c1' }] },
{ company_id: 'company-2', processed: 1, results: [{ attachment_id: 'att_1', inbox_item_id: 'item-c2' }] },
])
expect(uploadAndExtract).toHaveBeenCalledTimes(2)
expect(vi.mocked(uploadAndExtract).mock.calls[0][2]).toBe('company-1')
expect(vi.mocked(uploadAndExtract).mock.calls[0][5]?.kindHint).toBe('supplier_invoice')
expect(vi.mocked(uploadAndExtract).mock.calls[1][2]).toBe('company-2')
expect(vi.mocked(uploadAndExtract).mock.calls[1][5]?.kindHint).toBeNull()
// The idempotency lookup is company-scoped: the second inbox's copy is
// not "already processed" because the first inbox filed it.
const dupChecks = calls.filter(
(c) => c.table === 'invoice_inbox_items' && c.method === 'eq' && c.args[0] === 'company_id',
)
expect(dupChecks.map((c) => c.args[1])).toEqual(['company-1', 'company-2'])
const events = receivedEvents()
expect(events.map((e) => e.companyId)).toEqual(['company-1', 'company-2'])
expect(events[0].payload).toMatchObject({ inbox_id: 'inbox-1', tags: ['lev'], kind_hint: 'supplier_invoice', tag_conflict: false })
expect(events[1].payload).toMatchObject({ inbox_id: 'inbox-2', tags: [], kind_hint: null, custom_domain: false })
})
it('skips a retired address in the list and still files for the active one', async () => {
const to = ['old-inbox-abcd+lev@arcim.io', 'acme-ab-x7f2+ver@arcim.io']
vi.mocked(verifyInboundWebhook).mockReturnValue(mockReceivedEvent({ to, attachments: [] }) as never)
const { supabase, enqueue } = createQueuedMockSupabase()
enqueue({ data: { id: 'inbox-old', company_id: 'company-old', status: 'deprecated' } })
enqueue({ data: { id: 'inbox-1', company_id: 'company-1', status: 'active' } })
enqueue({ data: { created_by: 'user-owner-1' } })
enqueue({ data: null }) // body-document dedupe check
vi.mocked(createClient).mockReturnValue(supabase as never)
vi.mocked(uploadAndExtract).mockResolvedValue({ inbox_item_id: 'item-body' } as never)
vi.mocked(fetchReceivingEmail).mockResolvedValue(fullEmailFor(to, [], { html: '<p>Kvitto</p>' }) as never)
const res = await webhookRoute.handler(createMockRequest('/inbound', { method: 'POST', body: {} }))
const body = await res.json()
expect(res.status).toBe(200)
expect(body.data.reason).toBe('email_body')
expect(vi.mocked(uploadAndExtract).mock.calls[0][2]).toBe('company-1')
// The retired address's tag does not leak onto the active inbox's hint.
expect(vi.mocked(uploadAndExtract).mock.calls[0][5]?.kindHint).toBe('receipt')
expect(receivedEvents()[0].payload).toMatchObject({ outcome: 'email_body', inbox_item_id: 'item-body' })
})
it('lets a Resend redelivery replace a transient error row and file the attachment (self-heal kept)', async () => {
vi.mocked(verifyInboundWebhook).mockReturnValue(mockReceivedEvent() as never)
const { supabase, enqueue, calls } = createQueuedMockSupabase()
enqueue({ data: { id: 'inbox-1', company_id: 'company-1', status: 'active' } })
enqueue({ data: { created_by: 'user-owner-1' } })
// The first delivery's download failed and the catch wrote this row.
enqueue({ data: { id: 'err-1', status: 'error', raw_email_payload: { messageId: 'm', transient: true } } })
enqueue({ data: null }) // delete of the transient row
vi.mocked(createClient).mockReturnValue(supabase as never)
vi.mocked(uploadAndExtract).mockResolvedValue({ inbox_item_id: 'item-healed' } as never)
vi.mocked(fetchReceivingEmail).mockResolvedValue(fullEmailFor(['acme-ab-x7f2@arcim.io'], [PDF_ATTACHMENT]) as never)
vi.mocked(fetchInboundAttachment).mockResolvedValue(PDF_DOWNLOAD)
const res = await webhookRoute.handler(createMockRequest('/inbound', { method: 'POST', body: {} }))
const body = await res.json()
expect(res.status).toBe(200)
expect(body.data.results).toEqual([{ attachment_id: 'att_1', inbox_item_id: 'item-healed' }])
const del = calls.find((c) => c.table === 'invoice_inbox_items' && c.method === 'delete')
expect(del).toBeDefined()
const delEq = calls.filter((c) => c.table === 'invoice_inbox_items' && c.method === 'eq').find((c) => c.args[0] === 'id')
expect(delEq?.args).toEqual(['id', 'err-1'])
// The replacement leaves a trace: the record names the row it replaced.
expect(receivedEvents()[0].payload).toMatchObject({
attachments: [{ id: 'att_1', outcome: 'filed', inbox_item_id: 'item-healed', replaced_item_id: 'err-1' }],
})
})
it('processes five inboxes per mail and records the rest as not processed', async () => {
// Two unknown local parts first: they must not use up the cap.
const to = [
'unknown-a-0000@arcim.io',
'unknown-b-0000@arcim.io',
...Array.from({ length: 7 }, (_, i) => `inbox-${i}-abcd@arcim.io`),
]
vi.mocked(verifyInboundWebhook).mockReturnValue(mockReceivedEvent({ to, attachments: [] }) as never)
const { supabase, enqueue, calls } = createQueuedMockSupabase()
enqueue({ data: null }) // unknown-a
enqueue({ data: null }) // unknown-b
for (let i = 0; i < 7; i++) {
enqueue({ data: { id: `inbox-${i}`, company_id: `company-${i}`, status: 'active' } })
}
for (let i = 0; i < 5; i++) enqueue({ data: { created_by: `owner-${i}` } })
for (let i = 0; i < 5; i++) enqueue({ data: null }) // body-document dedupe checks
vi.mocked(createClient).mockReturnValue(supabase as never)
vi.mocked(uploadAndExtract).mockResolvedValue({ inbox_item_id: 'item-body' } as never)
vi.mocked(fetchReceivingEmail).mockResolvedValue(fullEmailFor(to, [], { html: '<p>Kvitto</p>' }) as never)
const res = await webhookRoute.handler(createMockRequest('/inbound', { method: 'POST', body: {} }))
const body = await res.json()
expect(res.status).toBe(200)
// Every addressed inbox is looked up; the first five resolved are processed.
expect(calls.filter((c) => c.table === 'company_inboxes' && c.method === 'eq')).toHaveLength(9)
expect(body.data.targets.map((t: { company_id: string }) => t.company_id)).toEqual(
['company-0', 'company-1', 'company-2', 'company-3', 'company-4'],
)
expect(body.data.deferred).toEqual([
{ company_id: 'company-5', reason: 'fan_out_capped' },
{ company_id: 'company-6', reason: 'fan_out_capped' },
])
expect(uploadAndExtract).toHaveBeenCalledTimes(5)
// The two past the cap still get a record: arrived, not processed.
const events = receivedEvents()
expect(events).toHaveLength(7)
expect(events.slice(5).map((e) => [e.companyId, e.payload.outcome])).toEqual([
['company-5', 'fan_out_capped'],
['company-6', 'fan_out_capped'],
])
})
it('keeps a rejected attachment (bad type) as a duplicate on redelivery', async () => {
vi.mocked(verifyInboundWebhook).mockReturnValue(mockReceivedEvent() as never)
const { supabase, enqueue, calls } = createQueuedMockSupabase()
enqueue({ data: { id: 'inbox-1', company_id: 'company-1', status: 'active' } })
enqueue({ data: { created_by: 'user-owner-1' } })
enqueue({ data: { id: 'rej-1', status: 'error', raw_email_payload: { messageId: 'm', mime: 'application/zip' } } })
vi.mocked(createClient).mockReturnValue(supabase as never)
vi.mocked(fetchReceivingEmail).mockResolvedValue(fullEmailFor(['acme-ab-x7f2@arcim.io'], [PDF_ATTACHMENT]) as never)
const res = await webhookRoute.handler(createMockRequest('/inbound', { method: 'POST', body: {} }))
const body = await res.json()
expect(body.data.results[0]).toEqual({ attachment_id: 'att_1', inbox_item_id: 'rej-1', duplicate: true })
expect(fetchInboundAttachment).not.toHaveBeenCalled()
expect(calls.find((c) => c.table === 'invoice_inbox_items' && c.method === 'delete')).toBeUndefined()
})
it('records a duplicate outcome when Resend retries the webhook', async () => {
vi.mocked(verifyInboundWebhook).mockReturnValue(mockReceivedEvent() as never)
const { supabase, enqueue } = createQueuedMockSupabase()
enqueue({ data: { id: 'inbox-1', company_id: 'company-1', status: 'active' } })
enqueue({ data: { created_by: 'user-owner-1' } })
enqueue({ data: { id: 'existing-item-1' } }) // dup check finds the first delivery's row
vi.mocked(createClient).mockReturnValue(supabase as never)
vi.mocked(fetchReceivingEmail).mockResolvedValue(fullEmailFor(['acme-ab-x7f2@arcim.io'], [PDF_ATTACHMENT]) as never)
const res = await webhookRoute.handler(createMockRequest('/inbound', { method: 'POST', body: {} }))
expect(res.status).toBe(200)
expect(receivedEvents()[0].payload).toMatchObject({
attachments: [{ id: 'att_1', outcome: 'duplicate', inbox_item_id: 'existing-item-1' }],
})
})
it('records a rejected outcome for an attachment type outside the allowlist', async () => {
vi.mocked(verifyInboundWebhook).mockReturnValue(mockReceivedEvent() as never)
const { supabase, enqueue } = createQueuedMockSupabase()
enqueue({ data: { id: 'inbox-1', company_id: 'company-1', status: 'active' } })
enqueue({ data: { created_by: 'user-owner-1' } })
enqueue({ data: null }) // dup check
enqueue({ data: { id: 'rejected-row-1' } }) // rejection-row insert returns its id
vi.mocked(createClient).mockReturnValue(supabase as never)
vi.mocked(fetchReceivingEmail).mockResolvedValue(
fullEmailFor(['acme-ab-x7f2@arcim.io'], [{ ...PDF_ATTACHMENT, id: 'att_zip', filename: 'x.zip', content_type: 'application/zip' }]) as never,
)
vi.mocked(fetchInboundAttachment).mockResolvedValue({
id: 'att_zip',
filename: 'x.zip',
contentType: 'application/zip',
buffer: new ArrayBuffer(8),
})
const res = await webhookRoute.handler(createMockRequest('/inbound', { method: 'POST', body: {} }))
expect(res.status).toBe(200)
expect(receivedEvents()[0].payload).toMatchObject({
attachments: [
{ id: 'att_zip', outcome: 'rejected', reason: 'unsupported_type', mime: 'application/zip', inbox_item_id: 'rejected-row-1' },
],
})
})
it('keeps an attachment whose processing threw as an error row and records it as failed', async () => {
vi.mocked(verifyInboundWebhook).mockReturnValue(mockReceivedEvent({ to: ['acme-ab-x7f2+lev@arcim.io'] }) as never)
const { supabase, enqueue, calls } = createQueuedMockSupabase()
enqueue({ data: { id: 'inbox-1', company_id: 'company-1', status: 'active' } })
enqueue({ data: { created_by: 'user-owner-1' } })
enqueue({ data: null }) // dup check
enqueue({ data: null }) // post-failure lookup: the upload never made its row
enqueue({ data: { id: 'failed-row-1' } }) // error-row insert
vi.mocked(createClient).mockReturnValue(supabase as never)
vi.mocked(fetchReceivingEmail).mockResolvedValue(fullEmailFor(['acme-ab-x7f2+lev@arcim.io'], [PDF_ATTACHMENT]) as never)
vi.mocked(fetchInboundAttachment).mockRejectedValue(new Error('Download URL returned 503 for attachment att_1'))
const res = await webhookRoute.handler(createMockRequest('/inbound', { method: 'POST', body: {} }))
const body = await res.json()
expect(res.status).toBe(200)
expect(body.data.results[0].error).toBe('Download URL returned 503 for attachment att_1')
// Before #2181 this path wrote nothing: the mail was accepted, answered
// 200 and gone. Now the attachment is an error row the user can see.
const insert = calls.find((c) => c.table === 'invoice_inbox_items' && c.method === 'insert')
expect(insert?.args[0]).toMatchObject({
status: 'error',
resend_email_id: 'em_123',
resend_attachment_id: 'att_1',
kind_hint: 'supplier_invoice',
raw_email_payload: { transient: true },
})
expect((insert?.args[0] as { error_message: string }).error_message).toMatch(/^Bilagan kunde inte tas emot: Download URL returned 503/)
expect(receivedEvents()[0].payload).toMatchObject({
attachments: [{ id: 'att_1', outcome: 'failed', inbox_item_id: 'failed-row-1' }],
})
})
it('does not write a second error row when the upload made its own row before throwing', async () => {
vi.mocked(verifyInboundWebhook).mockReturnValue(mockReceivedEvent() as never)
const { supabase, enqueue, calls } = createQueuedMockSupabase()
enqueue({ data: { id: 'inbox-1', company_id: 'company-1', status: 'active' } })
enqueue({ data: { created_by: 'user-owner-1' } })
enqueue({ data: null }) // dup check
enqueue({ data: { id: 'partial-row-1' } }) // post-failure lookup finds the upload's row
vi.mocked(createClient).mockReturnValue(supabase as never)
vi.mocked(fetchReceivingEmail).mockResolvedValue(fullEmailFor(['acme-ab-x7f2@arcim.io'], [PDF_ATTACHMENT]) as never)
vi.mocked(fetchInboundAttachment).mockResolvedValue(PDF_DOWNLOAD)
vi.mocked(uploadAndExtract).mockRejectedValue(new Error('extraction timed out'))
const res = await webhookRoute.handler(createMockRequest('/inbound', { method: 'POST', body: {} }))
expect(res.status).toBe(200)
expect(calls.find((c) => c.table === 'invoice_inbox_items' && c.method === 'insert')).toBeUndefined()
expect(receivedEvents()[0].payload).toMatchObject({
attachments: [{ id: 'att_1', outcome: 'failed', inbox_item_id: 'partial-row-1' }],
})
})
it('records the rate-limited drop on the mail record too', async () => {
vi.mocked(verifyInboundWebhook).mockReturnValue(mockReceivedEvent() as never)
vi.mocked(checkInboxUploadRateLimit).mockResolvedValueOnce({ ok: false, scope: 'minute', retryAfterSec: 60 } as never)
const { supabase, enqueue } = createQueuedMockSupabase()
enqueue({ data: { id: 'inbox-1', company_id: 'company-1', status: 'active' } })
enqueue({ data: { created_by: 'user-owner-1' } })
vi.mocked(createClient).mockReturnValue(supabase as never)
vi.mocked(fetchReceivingEmail).mockResolvedValue(fullEmailFor(['acme-ab-x7f2@arcim.io'], [PDF_ATTACHMENT]) as never)
const res = await webhookRoute.handler(createMockRequest('/inbound', { method: 'POST', body: {} }))
const body = await res.json()
expect(body.data.reason).toBe('rate_limited')
expect(vi.mocked(appendProcessingHistory).mock.calls.map(([i]) => i.eventType)).toEqual([
'RateLimitedDropped',
'InboundMailReceived',
])
expect(receivedEvents()[0].payload).toMatchObject({ outcome: 'rate_limited', attachments: [] })
})
})
@@ -1,8 +1,12 @@
import { describe, it, expect } from 'vitest'
import {
extractLocalPartForDomain,
extractSharedRecipientsForDomain,
groupSharedRecipientsByInbox,
kindHintFromTag,
parseRecipients,
resolveKindHintForTags,
splitKnownInboxTags,
} from '@/extensions/general/invoice-inbox/lib/resend-inbound'
describe('extractLocalPartForDomain', () => {
@@ -122,3 +126,82 @@ describe('parseRecipients', () => {
expect(parseRecipients([])).toEqual([])
})
})
describe('extractSharedRecipientsForDomain (#2181)', () => {
it('returns every shared-domain recipient in order with its tag', () => {
expect(
extractSharedRecipientsForDomain(
['acme-ab-x7f2+lev@arcim.io', 'billing@acme.se', 'Acme-AB-x7f2+VER@arcim.io', 'other-1234@arcim.io'],
'arcim.io',
),
).toEqual([
{ localPart: 'acme-ab-x7f2', tag: 'lev' },
{ localPart: 'acme-ab-x7f2', tag: 'ver' },
{ localPart: 'other-1234', tag: null },
])
})
it('collapses the same address listed twice', () => {
expect(
extractSharedRecipientsForDomain(['acme-ab-x7f2+lev@arcim.io', 'ACME-AB-X7F2+lev@arcim.io'], 'arcim.io'),
).toEqual([{ localPart: 'acme-ab-x7f2', tag: 'lev' }])
})
it('skips foreign domains and malformed addresses', () => {
expect(
extractSharedRecipientsForDomain(['x+lev@acme.se', 'not-an-email', '@arcim.io', 'foo@'], 'arcim.io'),
).toEqual([])
})
it('keeps extractLocalPartForDomain as the first entry', () => {
const to = ['second-efgh+ver@arcim.io', 'first-abcd@arcim.io']
expect(extractLocalPartForDomain(to, 'arcim.io')).toEqual(extractSharedRecipientsForDomain(to, 'arcim.io')[0])
})
})
describe('groupSharedRecipientsByInbox (#2181)', () => {
it('groups recipients per local part with distinct tags in first-seen order', () => {
expect(
groupSharedRecipientsByInbox([
{ localPart: 'acme', tag: 'lev' },
{ localPart: 'other', tag: null },
{ localPart: 'acme', tag: 'ver' },
{ localPart: 'acme', tag: 'lev' },
]),
).toEqual([
{ localPart: 'acme', tags: ['lev', 'ver'] },
{ localPart: 'other', tags: [] },
])
})
})
describe('resolveKindHintForTags (#2181)', () => {
it('maps a single documented tag', () => {
expect(resolveKindHintForTags(['lev'])).toEqual({ kindHint: 'supplier_invoice', conflict: false })
expect(resolveKindHintForTags(['ver'])).toEqual({ kindHint: 'receipt', conflict: false })
})
it('returns no hint and no conflict for unknown or missing tags', () => {
expect(resolveKindHintForTags([])).toEqual({ kindHint: null, conflict: false })
expect(resolveKindHintForTags(['faktura'])).toEqual({ kindHint: null, conflict: false })
})
it('ignores unknown tags next to a documented one', () => {
expect(resolveKindHintForTags(['faktura', 'lev'])).toEqual({ kindHint: 'supplier_invoice', conflict: false })
})
it('resolves +lev and +ver on one mail to no hint, flagged as a conflict', () => {
expect(resolveKindHintForTags(['lev', 'ver'])).toEqual({ kindHint: null, conflict: true })
expect(resolveKindHintForTags(['ver', 'lev'])).toEqual({ kindHint: null, conflict: true })
})
})
describe('splitKnownInboxTags (#2181)', () => {
it('keeps the documented tags and only counts the rest', () => {
expect(splitKnownInboxTags(['lev', '8501011234', 'ver', 'anna-svensson'])).toEqual({
known: ['lev', 'ver'],
unknownCount: 2,
})
expect(splitKnownInboxTags([])).toEqual({ known: [], unknownCount: 0 })
})
})
File diff suppressed because it is too large Load Diff
@@ -86,29 +86,62 @@ export interface SharedInboxRecipient {
tag: string | null
}
// Parses the first recipient whose domain matches our configured inbound
// domain, returning its local_part split at the first `+` (RFC 5233
// sub-addressing): `acme-x7f2+lev@inbox` matches the inbox row for
// `acme-x7f2` and carries tag `lev`. Before this split a +tagged mail 404ed,
// because the whole `acme-x7f2+lev` was looked up as the local_part.
// Returns null if no recipient is on the domain.
export function extractLocalPartForDomain(
// Parses every recipient whose domain matches our configured inbound domain,
// in original order, each split at the first `+` (RFC 5233 sub-addressing):
// `acme-x7f2+lev@inbox` matches the inbox row for `acme-x7f2` and carries
// tag `lev`. Duplicate addresses collapse to one entry. Returns all of them
// because one mail can name the same inbox twice with different tags, or two
// inboxes at once (#2181): reading only the first silently lost the rest.
export function extractSharedRecipientsForDomain(
recipients: string[],
domain: string,
): SharedInboxRecipient | null {
): SharedInboxRecipient[] {
const normalized = domain.toLowerCase()
const seen = new Set<string>()
const out: SharedInboxRecipient[] = []
for (const addr of recipients) {
const match = addr.match(/^\s*([^@\s]+)@([^@\s]+?)\s*$/)
if (!match) continue
const [, rawLocal, addrDomain] = match
if (addrDomain.toLowerCase() !== normalized) continue
const lower = rawLocal.toLowerCase()
if (seen.has(lower)) continue
seen.add(lower)
const plus = lower.indexOf('+')
if (plus === -1) return { localPart: lower, tag: null }
if (plus === -1) {
out.push({ localPart: lower, tag: null })
continue
}
const tag = lower.slice(plus + 1)
return { localPart: lower.slice(0, plus), tag: tag.length > 0 ? tag : null }
out.push({ localPart: lower.slice(0, plus), tag: tag.length > 0 ? tag : null })
}
return null
return out
}
// The first shared-domain recipient, or null if no recipient is on the
// domain. Kept for callers that only need to know whether the mail is on
// the shared domain at all; the webhook reads every recipient.
export function extractLocalPartForDomain(
recipients: string[],
domain: string,
): SharedInboxRecipient | null {
return extractSharedRecipientsForDomain(recipients, domain)[0] ?? null
}
/** Recipients grouped by inbox local part, tags in first-seen order. */
export interface SharedInboxTarget {
localPart: string
tags: string[]
}
export function groupSharedRecipientsByInbox(recipients: SharedInboxRecipient[]): SharedInboxTarget[] {
const byLocalPart = new Map<string, string[]>()
for (const r of recipients) {
const tags = byLocalPart.get(r.localPart) ?? []
if (r.tag !== null && !tags.includes(r.tag)) tags.push(r.tag)
byLocalPart.set(r.localPart, tags)
}
return Array.from(byLocalPart, ([localPart, tags]) => ({ localPart, tags }))
}
/** Sender-declared document kind, stored on invoice_inbox_items.kind_hint. */
@@ -123,6 +156,45 @@ export function kindHintFromTag(tag: string | null | undefined): InboxKindHint |
return null
}
/** The documented plus-address tags. Anything else is sender-typed free text. */
export const KNOWN_INBOX_TAGS: readonly string[] = ['lev', 'ver']
/**
* Splits a mail's tags into the documented ones (safe to store: a closed
* vocabulary) and a count of the rest. The rest is never stored: the part
* after `+` is whatever the sender typed, and processing_history is
* append-only and outside the erasure path (#2181 skeptic finding).
*/
export function splitKnownInboxTags(tags: readonly string[]): { known: string[]; unknownCount: number } {
const known: string[] = []
let unknownCount = 0
for (const tag of tags) {
if (KNOWN_INBOX_TAGS.includes(tag)) known.push(tag)
else unknownCount += 1
}
return { known, unknownCount }
}
/**
* The one kind hint for a mail that reached the same inbox under several
* tags. Distinct documented tags contradict each other (+lev and +ver on one
* mail, #2181): then the sender has said nothing usable, the hint is null so
* extraction classifies, and `conflict` is recorded on the mail's history
* event. Unknown tags never count.
*/
export function resolveKindHintForTags(tags: readonly string[]): {
kindHint: InboxKindHint | null
conflict: boolean
} {
const hints = new Set<InboxKindHint>()
for (const tag of tags) {
const hint = kindHintFromTag(tag)
if (hint) hints.add(hint)
}
if (hints.size > 1) return { kindHint: null, conflict: true }
return { kindHint: hints.values().next().value ?? null, conflict: false }
}
// Splits every parseable recipient into { localPart, domain }, lowercased and
// in original order. Used to match recipients against per-company verified
// custom domains when none of them is on the shared inbound domain.