feat: arcim inbox (Resend Inbound) + smart-match extension + commit metadata (#286)

* feat: multi-series SIE import, reusable FiscalYearSelector, library templates in picker

- SIE import preserves each voucher's source series (B/C/I/V/...), essential
  for Fortnox migrations where series carry semantic meaning (kundfakturor,
  inbetalningar, etc.). Target numbering still goes through next_voucher_number
  per series; source (series, number) is stored in the migration mapping for
  BFNAR 2013:2 audit trail.
- Execute route reads company_settings.default_voucher_series as the fallback
  for vouchers arriving without a series (SIE4I).
- Extract shared FiscalYearSelector component; adopt in /reports and
  /bookkeeping.
- Transaction TemplatePicker now surfaces user-created library templates
  (company + team scope) alongside the static registry, with a helper to
  convert simple library templates into the BookingTemplate shape.
- Exclude 8999 "Årets resultat" from income statement financial section and
  monthly breakdown so year-end closing entries don't cancel the net result.

Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>

* test: skip Bokio SIE regression when fixtures are absent

/dev_docs is gitignored (contains anonymised customer exports), so the
integration test can't find its input files in CI. Gate the suite on
fixture presence so it still runs locally.

Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>

* fix: address Greptile review feedback

- convertLibraryToBookingTemplate: default entity_applicability to 'all'
  when the source template has no entity_type, so TemplatePicker doesn't
  silently hide it for companies with a set entity type.
- FiscalYearSelector: fire onReady in the no-company early-return branch
  so consumers (e.g. ReportsPage) don't get stuck in a loading skeleton
  while the company context is still hydrating.

Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>

* feat: arcim inbox + smart-match extension + commit metadata

Three threads, all gated off in extensions.config.json (invoice-inbox and
inbox-smart-match are not in the enabled list for this PR).

invoice-inbox: Gmail OAuth -> Resend Inbound (v2.0.0)
- Remove gmail-scanner / gmail-helpers
- Add resend-inbound.ts (webhook verify, attachment fetch) and
  inbox-provisioning.ts (per-company @arcim.io address with rotation)
- Replace /gmail/* routes with /inbox/address and admin-only /inbox/rotate
- Workspace UI: card layout + MatchBlock surfacing AI transaction matches
- classify-document: tightened discount/total prompt; cap confidence at
  50% when line items do not reconcile with amount_incl_vat
- Manifest requires RESEND_API_KEY, RESEND_INBOUND_DOMAIN,
  RESEND_INBOUND_WEBHOOK_SECRET

inbox-smart-match (new extension)
- Event-driven AI matching of receipts to bank transactions
- Listens on inbox_item.classified (match now) and transaction.synced
  (retro-match receipts waiting for a transaction)
- Uses service-role client; processing_history append is scoped by
  company_id from the event payload

commit metadata + audit plumbing
- journal_entries gains commit_method and rubric_version columns
- commit_journal_entry RPC accepts both (BFNAR 2013:2 behandlingshistorik)
- processing-history PII detector strips UUID-shaped substrings before
  personnummer pattern matching (UUIDs were triggering false positives)
- New generic inbox_item.classified event

Migrations
- arcim_inbox: company_inboxes table, resend_email_id, email_body_text,
  auto-provision trigger, drops obsolete email_connections
- journal_entry_commit_metadata: new columns + updated RPC
- inbox_attachment_composite: resend_attachment_id + composite unique index
- inbox_smart_match: correlation_id, match_reasoning, expanded match_method
  CHECK, pending-match and correlation indexes

Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>

---------

Co-authored-by: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
This commit is contained in:
Jakob Wennberg
2026-04-20 21:50:07 +02:00
committed by GitHub
co-authored by Claude Opus 4.7
parent dfb953638c
commit 0076aa85f8
31 changed files with 2614 additions and 899 deletions
@@ -0,0 +1,179 @@
-- Arcim inbox: per-company @arcim.io email address for Resend Inbound
-- Replaces the Gmail OAuth model with a push-based inbound webhook flow.
--
-- Changes:
-- 1. New company_inboxes table with one active address per company
-- 2. Extend invoice_inbox_items with resend_email_id (idempotency) and email_body_text
-- 3. Drop the obsolete email_connections table (no production data; extension was disabled)
-- 4. generate_inbox_local_part() function: slug + 4-char suffix from a 30-char alphabet
-- 5. Auto-provision trigger on companies INSERT
-- 6. One-time backfill for existing companies
-- =============================================================================
-- 1. company_inboxes table
-- =============================================================================
CREATE TABLE IF NOT EXISTS public.company_inboxes (
id uuid DEFAULT gen_random_uuid() PRIMARY KEY,
company_id uuid NOT NULL REFERENCES public.companies(id) ON DELETE CASCADE,
local_part text NOT NULL,
status text NOT NULL DEFAULT 'active'
CHECK (status IN ('active', 'deprecated', 'blocked')),
slug_seed text NOT NULL,
created_at timestamptz NOT NULL DEFAULT now(),
updated_at timestamptz NOT NULL DEFAULT now(),
deprecated_at timestamptz
);
-- Globally unique addresses across all rows (deprecated rows keep their local_part reserved)
CREATE UNIQUE INDEX IF NOT EXISTS idx_company_inboxes_local_part
ON public.company_inboxes(local_part);
-- One active inbox per company at a time
CREATE UNIQUE INDEX IF NOT EXISTS idx_company_inboxes_company_active
ON public.company_inboxes(company_id) WHERE status = 'active';
-- Lookup by company
CREATE INDEX IF NOT EXISTS idx_company_inboxes_company
ON public.company_inboxes(company_id);
-- RLS
ALTER TABLE public.company_inboxes ENABLE ROW LEVEL SECURITY;
CREATE POLICY "company_inboxes_select" ON public.company_inboxes
FOR SELECT USING (company_id IN (SELECT public.user_company_ids()));
CREATE POLICY "company_inboxes_insert" ON public.company_inboxes
FOR INSERT WITH CHECK (company_id IN (SELECT public.user_company_ids()));
CREATE POLICY "company_inboxes_update" ON public.company_inboxes
FOR UPDATE USING (company_id IN (SELECT public.user_company_ids()));
-- updated_at trigger
CREATE TRIGGER company_inboxes_updated_at
BEFORE UPDATE ON public.company_inboxes
FOR EACH ROW EXECUTE FUNCTION public.update_updated_at_column();
-- =============================================================================
-- 2. Extend invoice_inbox_items
-- =============================================================================
ALTER TABLE public.invoice_inbox_items
ADD COLUMN IF NOT EXISTS resend_email_id text,
ADD COLUMN IF NOT EXISTS email_body_text text;
-- Idempotency: Resend retries the webhook on our failures; a unique index rejects duplicates
CREATE UNIQUE INDEX IF NOT EXISTS idx_invoice_inbox_items_resend_email_id
ON public.invoice_inbox_items(resend_email_id)
WHERE resend_email_id IS NOT NULL;
-- =============================================================================
-- 3. Drop obsolete email_connections (Gmail OAuth storage)
-- =============================================================================
DROP TABLE IF EXISTS public.email_connections;
-- =============================================================================
-- 4. generate_inbox_local_part(): slugify company name + 4-char suffix
-- =============================================================================
CREATE OR REPLACE FUNCTION public.generate_inbox_local_part(p_company_name text)
RETURNS text
LANGUAGE plpgsql
AS $$
DECLARE
-- Crockford-ish base32 without ambiguous chars (no i/l/o/u, digits 2-9)
v_alphabet constant text := 'abcdefghjkmnpqrstvwxyz23456789';
v_alphabet_len constant int := 30;
v_slug text;
v_suffix text;
v_candidate text;
v_attempt int := 0;
v_max_attempts constant int := 20;
BEGIN
-- Slugify: normalize Swedish vowels, strip accents, lowercase, non-alnum → hyphen
v_slug := lower(COALESCE(p_company_name, ''));
v_slug := translate(v_slug, 'åäöéèêàâüñ', 'aaoeeeaaun');
v_slug := regexp_replace(v_slug, '[^a-z0-9]+', '-', 'g');
v_slug := regexp_replace(v_slug, '^-+|-+$', '', 'g');
v_slug := substring(v_slug from 1 for 24);
v_slug := regexp_replace(v_slug, '-+$', '', 'g');
IF v_slug IS NULL OR v_slug = '' THEN
v_slug := 'company';
END IF;
LOOP
v_attempt := v_attempt + 1;
-- Build a 4-char suffix
v_suffix := '';
FOR i IN 1..4 LOOP
v_suffix := v_suffix || substr(v_alphabet, 1 + floor(random() * v_alphabet_len)::int, 1);
END LOOP;
v_candidate := v_slug || '-' || v_suffix;
IF NOT EXISTS (SELECT 1 FROM public.company_inboxes WHERE local_part = v_candidate) THEN
RETURN v_candidate;
END IF;
IF v_attempt >= v_max_attempts THEN
RAISE EXCEPTION 'Failed to generate unique inbox local_part after % attempts for slug %', v_max_attempts, v_slug;
END IF;
END LOOP;
END;
$$;
-- =============================================================================
-- 5. Auto-provision trigger on companies INSERT
-- =============================================================================
CREATE OR REPLACE FUNCTION public.auto_provision_company_inbox()
RETURNS TRIGGER
LANGUAGE plpgsql
SECURITY DEFINER
SET search_path = public
AS $$
DECLARE
v_local_part text;
v_slug_seed text;
BEGIN
v_local_part := public.generate_inbox_local_part(NEW.name);
v_slug_seed := regexp_replace(v_local_part, '-[^-]+$', '');
INSERT INTO public.company_inboxes (company_id, local_part, slug_seed, status)
VALUES (NEW.id, v_local_part, v_slug_seed, 'active');
RETURN NEW;
END;
$$;
DROP TRIGGER IF EXISTS companies_auto_provision_inbox ON public.companies;
CREATE TRIGGER companies_auto_provision_inbox
AFTER INSERT ON public.companies
FOR EACH ROW EXECUTE FUNCTION public.auto_provision_company_inbox();
-- =============================================================================
-- 6. Backfill existing companies
-- =============================================================================
DO $$
DECLARE
c RECORD;
v_local_part text;
v_slug_seed text;
BEGIN
FOR c IN
SELECT co.id, co.name
FROM public.companies co
WHERE NOT EXISTS (
SELECT 1 FROM public.company_inboxes ci WHERE ci.company_id = co.id
)
LOOP
v_local_part := public.generate_inbox_local_part(c.name);
v_slug_seed := regexp_replace(v_local_part, '-[^-]+$', '');
INSERT INTO public.company_inboxes (company_id, local_part, slug_seed, status)
VALUES (c.id, v_local_part, v_slug_seed, 'active');
END LOOP;
END $$;
NOTIFY pgrst, 'reload schema';
@@ -0,0 +1,101 @@
-- Add commit_method and rubric_version columns to journal_entries.
-- Tracks HOW each entry was committed and WHICH rubric was active.
-- Required for autonomous accounting: trust ramp, timing ceiling, audit trail.
--
-- Compliance: BFNAR 2013:2 kap 8 requires behandlingshistorik to log automated
-- processing and user actions. commit_method records this per verifikation.
-- timing_ceiling aligns with BFL 5 kap 2§ (50-day maximum via BFNAR 2013:2).
-- 1. Add columns (nullable — existing rows get NULL)
ALTER TABLE public.journal_entries
ADD COLUMN commit_method TEXT CHECK (commit_method IS NULL OR commit_method IN (
'user_accept', 'bulk_accept', 'timing_ceiling', 'migration', 'legacy'
)),
ADD COLUMN rubric_version TEXT;
-- 2. Update commit RPC to accept and set the new columns atomically
CREATE OR REPLACE FUNCTION public.commit_journal_entry(
p_company_id uuid,
p_entry_id uuid,
p_commit_method text DEFAULT NULL,
p_rubric_version text DEFAULT NULL
)
RETURNS TABLE (voucher_number integer)
LANGUAGE plpgsql
SECURITY DEFINER
AS $$
DECLARE
v_next integer;
v_fiscal_period_id uuid;
v_series text;
BEGIN
-- Fetch and lock the draft entry
SELECT je.fiscal_period_id, COALESCE(je.voucher_series, 'A')
INTO v_fiscal_period_id, v_series
FROM public.journal_entries je
WHERE je.id = p_entry_id
AND je.company_id = p_company_id
AND je.status = 'draft'
FOR UPDATE;
IF NOT FOUND THEN
RAISE EXCEPTION 'Draft journal entry not found: %', p_entry_id;
END IF;
-- Increment voucher sequence (atomic via INSERT ON CONFLICT)
INSERT INTO public.voucher_sequences (company_id, user_id, fiscal_period_id, voucher_series, last_number)
VALUES (p_company_id, auth.uid(), v_fiscal_period_id, v_series, 1)
ON CONFLICT (company_id, fiscal_period_id, voucher_series)
DO UPDATE SET
last_number = public.voucher_sequences.last_number + 1,
updated_at = now()
RETURNING last_number INTO v_next;
-- Update entry to posted with assigned voucher number and commit metadata.
-- If the balance trigger (check_balance_on_post) rejects the UPDATE,
-- the entire transaction rolls back — including the sequence increment.
UPDATE public.journal_entries
SET voucher_number = v_next,
status = 'posted',
commit_method = p_commit_method,
rubric_version = p_rubric_version
WHERE id = p_entry_id
AND company_id = p_company_id;
RETURN QUERY SELECT v_next;
END;
$$;
-- 3. Update immutability trigger: include new fields in reversal field check
CREATE OR REPLACE FUNCTION public.enforce_journal_entry_immutability()
RETURNS trigger LANGUAGE plpgsql AS $$
BEGIN
IF TG_OP = 'DELETE' THEN
RAISE EXCEPTION 'Cannot delete journal entries (id: %, status: %). Use cancelled status instead.',
OLD.id, OLD.status;
END IF;
-- Draft can transition to draft (update fields), posted, or cancelled
IF OLD.status = 'draft' AND NEW.status IN ('draft', 'posted', 'cancelled') THEN
RETURN NEW;
END IF;
-- Posted can transition to reversed (storno) or cancelled (orphaned concurrent reversal cleanup)
IF OLD.status = 'posted' AND NEW.status IN ('reversed', 'cancelled') THEN
IF NEW.status = 'reversed' THEN
IF NEW.description != OLD.description OR NEW.entry_date != OLD.entry_date
OR NEW.fiscal_period_id != OLD.fiscal_period_id
OR NEW.voucher_number != OLD.voucher_number
OR NEW.commit_method IS DISTINCT FROM OLD.commit_method
OR NEW.rubric_version IS DISTINCT FROM OLD.rubric_version THEN
RAISE EXCEPTION 'Cannot modify fields of a posted entry during reversal (id: %)', OLD.id;
END IF;
END IF;
RETURN NEW;
END IF;
RAISE EXCEPTION 'Cannot modify a % journal entry (id: %). Committed entries are immutable per Bokforingslagen.',
OLD.status, OLD.id;
END; $$;
NOTIFY pgrst, 'reload schema';
@@ -0,0 +1,14 @@
-- Allow multiple invoice_inbox_items per email (one per attachment).
-- Replaces the single-column unique on resend_email_id with a composite
-- on (resend_email_id, resend_attachment_id).
DROP INDEX IF EXISTS idx_invoice_inbox_items_resend_email_id;
ALTER TABLE public.invoice_inbox_items
ADD COLUMN IF NOT EXISTS resend_attachment_id text;
CREATE UNIQUE INDEX IF NOT EXISTS idx_invoice_inbox_items_resend_email_attachment
ON public.invoice_inbox_items(resend_email_id, resend_attachment_id)
WHERE resend_email_id IS NOT NULL;
NOTIFY pgrst, 'reload schema';
@@ -0,0 +1,36 @@
-- inbox-smart-match extension schema additions
-- Adds correlation_id (audit chain) and match_reasoning (LLM explanation)
-- and expands match_method to include 'llm' and 'pending_transaction'.
ALTER TABLE public.invoice_inbox_items
ADD COLUMN IF NOT EXISTS correlation_id uuid,
ADD COLUMN IF NOT EXISTS match_reasoning text;
-- Expand allowed match_method values
ALTER TABLE public.invoice_inbox_items
DROP CONSTRAINT IF EXISTS invoice_inbox_items_match_method_check;
ALTER TABLE public.invoice_inbox_items
ADD CONSTRAINT invoice_inbox_items_match_method_check
CHECK (match_method IN (
'payment_reference',
'amount_date',
'amount_merchant',
'receipt_match',
'llm',
'pending_transaction'
));
-- Index for retroactive matcher queries ("find all receipts awaiting a transaction for this company")
CREATE INDEX IF NOT EXISTS idx_inbox_items_pending_match
ON public.invoice_inbox_items(company_id)
WHERE document_type = 'receipt'
AND match_method = 'pending_transaction'
AND status = 'ready';
-- Index for correlation-id audit queries
CREATE INDEX IF NOT EXISTS idx_inbox_items_correlation_id
ON public.invoice_inbox_items(correlation_id)
WHERE correlation_id IS NOT NULL;
NOTIFY pgrst, 'reload schema';
@@ -0,0 +1,111 @@
-- Hardening follow-up to 20260420000000_arcim_inbox and 20260420180000_inbox_smart_match.
--
-- Fixes identified in PR #286 review:
-- 1. Non-atomic rotation in rotateCompanyInbox() — replaced with a
-- SECURITY DEFINER RPC so deprecate/generate/insert happen in one
-- Postgres transaction.
-- 2. Overly permissive RLS on company_inboxes (any member, including
-- viewers, could INSERT/UPDATE). Tightened to owner/admin only.
-- 3. Dual-match race in inbox-smart-match (two receipts could both pair
-- themselves to the same transaction). Enforced by partial unique
-- index; process-match catches 23505 and falls back to pending.
-- =============================================================================
-- 1. Tighten RLS on company_inboxes to owner/admin only (for INSERT + UPDATE)
-- =============================================================================
DROP POLICY IF EXISTS "company_inboxes_insert" ON public.company_inboxes;
CREATE POLICY "company_inboxes_insert" ON public.company_inboxes
FOR INSERT WITH CHECK (
company_id IN (
SELECT cm.company_id FROM public.company_members cm
WHERE cm.user_id = auth.uid()
AND cm.role IN ('owner', 'admin')
)
);
DROP POLICY IF EXISTS "company_inboxes_update" ON public.company_inboxes;
CREATE POLICY "company_inboxes_update" ON public.company_inboxes
FOR UPDATE USING (
company_id IN (
SELECT cm.company_id FROM public.company_members cm
WHERE cm.user_id = auth.uid()
AND cm.role IN ('owner', 'admin')
)
);
-- Note: auto_provision_company_inbox() and rotate_company_inbox() are
-- SECURITY DEFINER and bypass these policies; the policies only gate
-- direct client-side writes (defense-in-depth).
-- =============================================================================
-- 2. Atomic rotate RPC
-- =============================================================================
CREATE OR REPLACE FUNCTION public.rotate_company_inbox(p_company_id uuid)
RETURNS public.company_inboxes
LANGUAGE plpgsql
SECURITY DEFINER
SET search_path = public
AS $$
DECLARE
v_company_name text;
v_local_part text;
v_slug_seed text;
v_new_row public.company_inboxes;
BEGIN
-- Authorization: caller must be owner/admin of the company.
IF NOT EXISTS (
SELECT 1 FROM public.company_members
WHERE company_id = p_company_id
AND user_id = auth.uid()
AND role IN ('owner', 'admin')
) THEN
RAISE EXCEPTION 'Not authorized to rotate inbox for this company'
USING ERRCODE = '42501';
END IF;
SELECT name INTO v_company_name
FROM public.companies
WHERE id = p_company_id;
IF v_company_name IS NULL THEN
RAISE EXCEPTION 'Company not found' USING ERRCODE = 'P0002';
END IF;
-- All three steps share one transaction — a failure on any of them
-- rolls the whole thing back, so the company never ends up without
-- an active inbox.
UPDATE public.company_inboxes
SET status = 'deprecated',
deprecated_at = now()
WHERE company_id = p_company_id
AND status = 'active';
v_local_part := public.generate_inbox_local_part(v_company_name);
v_slug_seed := regexp_replace(v_local_part, '-[^-]+$', '');
INSERT INTO public.company_inboxes (company_id, local_part, slug_seed, status)
VALUES (p_company_id, v_local_part, v_slug_seed, 'active')
RETURNING * INTO v_new_row;
RETURN v_new_row;
END;
$$;
GRANT EXECUTE ON FUNCTION public.rotate_company_inbox(uuid) TO authenticated;
-- =============================================================================
-- 3. Prevent two inbox items from claiming the same transaction
-- =============================================================================
-- Partial unique index: once a row has matched_transaction_id set for a
-- given company, no other row in that company may claim the same one.
-- Concurrent UPDATEs from smart-match will get a 23505 and the handler
-- gracefully downgrades the loser to pending_transaction.
CREATE UNIQUE INDEX IF NOT EXISTS idx_inbox_items_matched_transaction_unique
ON public.invoice_inbox_items(company_id, matched_transaction_id)
WHERE matched_transaction_id IS NOT NULL;
NOTIFY pgrst, 'reload schema';