08440fed94
* feat(reconciliation): match migrated bank history against imported SIE verifikat A first-class Fortnox/SIE migrator path: after SIE import plus bank connect or bank CSV upload, historical bank rows are auto-matched (>= 0.9) or suggestion-matched (0.75-0.89, persisted for review) against the imported verifikat, with a guided review surface, instead of landing as anonymous "Att bokfora" rows. Phase 0: per-cash-account unattended sweep (fixes #1298 cross-account pooling); widen payment_match_log action CHECK with linked_to_existing_voucher (silently unlogged since March). Phase 1: potential_journal_entry_id/method/confidence on transactions with CHECK + invalidation triggers; persistSuggestions in runReconciliation; sweep after bank CSV import with SIE overlap (suppressing auto-categorization); sweep summaries stamped on bank_connections and bank_file_imports; POST /api/reconciliation/bank/confirm-suggestions with per-pair server-side revalidation (voucher consumption + bank-leg amount and direction). Phase 2: "Granska forslag" review tab on Transactions with chunked bulk confirm, per-row fallbacks, "Kor matchning igen" (all_accounts sweep mode, mutually exclusive with dry_run), attn line, pre-migration row marker. Phase 3: ImportResultStep dual CTA (bank connect + CSV), migrator variant of the account-picker #917 nudge, sweep outcome on the onboarding checklist bank step. Non-selection apply runs on /api/reconciliation/bank/run now floor at 0.9 and persist the review band instead of auto-committing fuzzy matches. Migrations already applied to staging under the same versions. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com> * fix(reconciliation): resolve PR review findings in one pass Swedish accounting review (both previously-deferred holes closed): - runReconciliation's >= 0.9 auto-apply now writes 'matched' to payment_match_log (behandlingshistorik, BFNAR 2013:2 kap 8); the bus event alone lands in the 30-day event_log and is not an audit record. - The three match-route storno-conflict branches detach reconciliation links via unlinkReconciliation instead of storno-reversing the linked verifikat: a reconciliation link points at an independent verifikat that may evidence other affarshandelser, and a wholesale reversal is an over-broad rattelse (BFL 5 kap 5 §). - Historical gap quantified on prod (read-only, recorded in DECISIONS): 762 unlogged manual links across 52 companies since 2026-03-23. CodeRabbit: - confirm-suggestions route: maxDuration 300 for full 500-item batches. - AccountPickerDialog: migrator-nudge buttons set lookbackTouched so the async gap-fill probe cannot override an explicit choice. - enable-banking post-backfill sweep: persistSuggestions so the review band is not dropped. - bank-file execute: sie_sweep stamp errors are logged, not swallowed. - ImportResultStep: sandbox keeps the CSV CTA (file import works there). - payment_match_log CHECK swap: NOT VALID + VALIDATE, no table scan under ACCESS EXCLUSIVE. - logMatchEvent calls awaited (serverless can freeze unawaited work). - DECISIONS.md stale version reference annotated. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com> * fix(reconciliation): defer reconciliation-link detach until the match commits Round-2 review findings: - CodeRabbit: the eager unlinkReconciliation call could orphan a transaction if the match flow failed after it. All three match routes now persist NOTHING up front: the final transaction update overwrites journal_entry_id and clears reconciliation_method in the same write, so any failure in between leaves the existing link intact. The release is logged as 'unmatched' after the commit. - Swedish review: the auto_suggested logMatchEvent in runReconciliation is now awaited like every other audit write. - DECISIONS entry split into compliance/CodeRabbit lines and updated to describe the deferred detach. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com> * fix(reconciliation): literal reconciliation_method payloads for the phantom-column scanner The conditional spreads introduced with the deferred detach pushed the scanner's unresolvable-expression count past its ceiling (380 > 378). reconciliation_method: null is correct unconditionally on a confirmed invoice/supplier match (null is already the value on every row that was not reconciliation-linked), so the payloads become plain literals the guard can verify. No behavior change. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com> --------- Co-authored-by: Claude Fable 5 <noreply@anthropic.com>
149 lines
5.8 KiB
PL/PgSQL
149 lines
5.8 KiB
PL/PgSQL
-- Migration: persisted journal-entry match suggestions on transactions
|
|
--
|
|
-- The 4-pass bank matcher proposes matches in a 0.75-0.89 confidence band
|
|
-- (auto_fuzzy, auto_date_range) that unattended sweeps must never auto-apply.
|
|
-- Until now those proposals were computed and then dropped: the migrator who
|
|
-- imported a Fortnox SIE file and connected their bank never saw them. These
|
|
-- columns persist the band as reviewable suggestions, mirroring
|
|
-- potential_invoice_id / potential_supplier_invoice_id (one candidate per row).
|
|
--
|
|
-- Suggestions are soft data: nullable columns, no journal writes, fully
|
|
-- reversible. Confirming one goes through the ordinary manual-link path
|
|
-- (server-side revalidation at click time); rejecting clears the columns.
|
|
|
|
ALTER TABLE public.transactions
|
|
ADD COLUMN potential_journal_entry_id UUID
|
|
REFERENCES public.journal_entries (id) ON DELETE SET NULL,
|
|
ADD COLUMN potential_match_method TEXT,
|
|
ADD COLUMN potential_match_confidence NUMERIC(3,2);
|
|
|
|
-- A suggestion is all-or-nothing (id + method + confidence together), and a
|
|
-- transaction that is already linked to a journal entry carries no suggestion.
|
|
-- The BEFORE-UPDATE trigger below clears the trio whenever a link lands or the
|
|
-- row is ignored, so ordinary writers never trip this. ON DELETE SET NULL on
|
|
-- the FK nulls only the id; the trigger function also re-nulls method and
|
|
-- confidence on any write, and the CHECK is written to tolerate the transient
|
|
-- id-only-null state a FK SET NULL leaves behind (method/confidence dangling
|
|
-- without an id is inert for every reader, which keys on the id).
|
|
ALTER TABLE public.transactions
|
|
ADD CONSTRAINT transactions_potential_je_check CHECK (
|
|
potential_journal_entry_id IS NULL
|
|
OR (
|
|
potential_match_method IS NOT NULL
|
|
AND potential_match_confidence IS NOT NULL
|
|
AND journal_entry_id IS NULL
|
|
)
|
|
);
|
|
|
|
-- The review surface asks "does this company have suggestions?" and lists them;
|
|
-- suggestion rows are a small fraction of transactions.
|
|
CREATE INDEX idx_transactions_potential_je
|
|
ON public.transactions (company_id)
|
|
WHERE potential_journal_entry_id IS NOT NULL;
|
|
|
|
-- ============================================================
|
|
-- Suggestion invalidation, transaction side
|
|
-- ============================================================
|
|
-- Clear the suggestion whenever the transaction stops being an open row:
|
|
-- booked or linked (journal_entry_id set, by ANY path: categorization,
|
|
-- reconciliation, MCP, RPCs) or explicitly ignored. BEFORE trigger so the
|
|
-- cleared values land in the same write and the CHECK above always sees a
|
|
-- consistent row.
|
|
|
|
CREATE OR REPLACE FUNCTION public.clear_potential_journal_entry_on_close()
|
|
RETURNS TRIGGER
|
|
LANGUAGE plpgsql
|
|
SET search_path = public
|
|
AS $$
|
|
BEGIN
|
|
IF NEW.journal_entry_id IS NOT NULL OR NEW.is_ignored IS TRUE THEN
|
|
NEW.potential_journal_entry_id := NULL;
|
|
NEW.potential_match_method := NULL;
|
|
NEW.potential_match_confidence := NULL;
|
|
END IF;
|
|
RETURN NEW;
|
|
END;
|
|
$$;
|
|
|
|
CREATE TRIGGER transactions_clear_potential_je
|
|
BEFORE INSERT OR UPDATE ON public.transactions
|
|
FOR EACH ROW
|
|
EXECUTE FUNCTION public.clear_potential_journal_entry_on_close();
|
|
|
|
-- ============================================================
|
|
-- Suggestion invalidation, journal-entry side
|
|
-- ============================================================
|
|
-- (a) The suggested entry is consumed by another transaction's link: other
|
|
-- rows still suggesting it would double-consume the verifikat on bulk
|
|
-- confirm, so their suggestions are cleared as soon as any link lands.
|
|
-- SECURITY DEFINER: the writer linking their own transaction may not have
|
|
-- UPDATE visibility over sibling rows under RLS, but the scope is derived
|
|
-- entirely from the row being written (company_id + journal_entry_id),
|
|
-- never from caller input.
|
|
|
|
CREATE OR REPLACE FUNCTION public.clear_sibling_je_suggestions_on_link()
|
|
RETURNS TRIGGER
|
|
LANGUAGE plpgsql
|
|
SECURITY DEFINER
|
|
SET search_path = public
|
|
AS $$
|
|
BEGIN
|
|
UPDATE public.transactions
|
|
SET potential_journal_entry_id = NULL,
|
|
potential_match_method = NULL,
|
|
potential_match_confidence = NULL
|
|
WHERE company_id = NEW.company_id
|
|
AND potential_journal_entry_id = NEW.journal_entry_id
|
|
AND id <> NEW.id;
|
|
RETURN NULL;
|
|
END;
|
|
$$;
|
|
|
|
CREATE TRIGGER transactions_clear_sibling_je_suggestions
|
|
AFTER UPDATE OF journal_entry_id ON public.transactions
|
|
FOR EACH ROW
|
|
WHEN (NEW.journal_entry_id IS NOT NULL
|
|
AND NEW.journal_entry_id IS DISTINCT FROM OLD.journal_entry_id)
|
|
EXECUTE FUNCTION public.clear_sibling_je_suggestions_on_link();
|
|
|
|
-- (b) The suggested entry is reversed via storno: it is no longer a live
|
|
-- verifikat to settle against, so pending suggestions pointing at it are
|
|
-- stale. Same SECURITY DEFINER rationale as above.
|
|
|
|
CREATE OR REPLACE FUNCTION public.clear_je_suggestions_on_reversal()
|
|
RETURNS TRIGGER
|
|
LANGUAGE plpgsql
|
|
SECURITY DEFINER
|
|
SET search_path = public
|
|
AS $$
|
|
BEGIN
|
|
UPDATE public.transactions
|
|
SET potential_journal_entry_id = NULL,
|
|
potential_match_method = NULL,
|
|
potential_match_confidence = NULL
|
|
WHERE company_id = NEW.company_id
|
|
AND potential_journal_entry_id = NEW.id;
|
|
RETURN NULL;
|
|
END;
|
|
$$;
|
|
|
|
CREATE TRIGGER journal_entries_clear_je_suggestions_on_reversal
|
|
AFTER UPDATE OF status ON public.journal_entries
|
|
FOR EACH ROW
|
|
WHEN (NEW.status = 'reversed' AND OLD.status IS DISTINCT FROM 'reversed')
|
|
EXECUTE FUNCTION public.clear_je_suggestions_on_reversal();
|
|
|
|
-- ============================================================
|
|
-- Sweep summaries
|
|
-- ============================================================
|
|
-- One JSONB per surface so the UI can render "Vi matchade X av Y" without
|
|
-- recomputing: {auto_linked, suggested, unmatched, date_from, date_to, ran_at}.
|
|
|
|
ALTER TABLE public.bank_connections
|
|
ADD COLUMN last_sie_sweep JSONB;
|
|
|
|
ALTER TABLE public.bank_file_imports
|
|
ADD COLUMN sie_sweep JSONB;
|
|
|
|
NOTIFY pgrst, 'reload schema';
|