feat(reconciliation): match migrated bank history against imported SIE verifikat (#1598)
* feat(reconciliation): match migrated bank history against imported SIE verifikat A first-class Fortnox/SIE migrator path: after SIE import plus bank connect or bank CSV upload, historical bank rows are auto-matched (>= 0.9) or suggestion-matched (0.75-0.89, persisted for review) against the imported verifikat, with a guided review surface, instead of landing as anonymous "Att bokfora" rows. Phase 0: per-cash-account unattended sweep (fixes #1298 cross-account pooling); widen payment_match_log action CHECK with linked_to_existing_voucher (silently unlogged since March). Phase 1: potential_journal_entry_id/method/confidence on transactions with CHECK + invalidation triggers; persistSuggestions in runReconciliation; sweep after bank CSV import with SIE overlap (suppressing auto-categorization); sweep summaries stamped on bank_connections and bank_file_imports; POST /api/reconciliation/bank/confirm-suggestions with per-pair server-side revalidation (voucher consumption + bank-leg amount and direction). Phase 2: "Granska forslag" review tab on Transactions with chunked bulk confirm, per-row fallbacks, "Kor matchning igen" (all_accounts sweep mode, mutually exclusive with dry_run), attn line, pre-migration row marker. Phase 3: ImportResultStep dual CTA (bank connect + CSV), migrator variant of the account-picker #917 nudge, sweep outcome on the onboarding checklist bank step. Non-selection apply runs on /api/reconciliation/bank/run now floor at 0.9 and persist the review band instead of auto-committing fuzzy matches. Migrations already applied to staging under the same versions. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com> * fix(reconciliation): resolve PR review findings in one pass Swedish accounting review (both previously-deferred holes closed): - runReconciliation's >= 0.9 auto-apply now writes 'matched' to payment_match_log (behandlingshistorik, BFNAR 2013:2 kap 8); the bus event alone lands in the 30-day event_log and is not an audit record. - The three match-route storno-conflict branches detach reconciliation links via unlinkReconciliation instead of storno-reversing the linked verifikat: a reconciliation link points at an independent verifikat that may evidence other affarshandelser, and a wholesale reversal is an over-broad rattelse (BFL 5 kap 5 §). - Historical gap quantified on prod (read-only, recorded in DECISIONS): 762 unlogged manual links across 52 companies since 2026-03-23. CodeRabbit: - confirm-suggestions route: maxDuration 300 for full 500-item batches. - AccountPickerDialog: migrator-nudge buttons set lookbackTouched so the async gap-fill probe cannot override an explicit choice. - enable-banking post-backfill sweep: persistSuggestions so the review band is not dropped. - bank-file execute: sie_sweep stamp errors are logged, not swallowed. - ImportResultStep: sandbox keeps the CSV CTA (file import works there). - payment_match_log CHECK swap: NOT VALID + VALIDATE, no table scan under ACCESS EXCLUSIVE. - logMatchEvent calls awaited (serverless can freeze unawaited work). - DECISIONS.md stale version reference annotated. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com> * fix(reconciliation): defer reconciliation-link detach until the match commits Round-2 review findings: - CodeRabbit: the eager unlinkReconciliation call could orphan a transaction if the match flow failed after it. All three match routes now persist NOTHING up front: the final transaction update overwrites journal_entry_id and clears reconciliation_method in the same write, so any failure in between leaves the existing link intact. The release is logged as 'unmatched' after the commit. - Swedish review: the auto_suggested logMatchEvent in runReconciliation is now awaited like every other audit write. - DECISIONS entry split into compliance/CodeRabbit lines and updated to describe the deferred detach. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com> * fix(reconciliation): literal reconciliation_method payloads for the phantom-column scanner The conditional spreads introduced with the deferred detach pushed the scanner's unresolvable-expression count past its ceiling (380 > 378). reconciliation_method: null is correct unconditionally on a confirmed invoice/supplier match (null is already the value on every row that was not reconciliation-linked), so the payloads become plain literals the guard can verify. No behavior change. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com> --------- Co-authored-by: Claude Fable 5 <noreply@anthropic.com>
This commit is contained in:
Mattsson
co-authored by
Claude Fable 5
Claude Fable 5
parent
07e89d9b52
commit
08440fed94
@@ -0,0 +1,176 @@
|
||||
/**
|
||||
* Tests for POST /api/import/bank-file/execute, focused on the SIE-overlap
|
||||
* behavior: a bank file covering a period a completed SIE import already
|
||||
* booked must (a) suppress auto-categorization to prevent double-booking and
|
||||
* (b) trigger the per-account reconciliation sweep and stamp its summary,
|
||||
* because CSV is how a migrator gets pre-PSD2 history into the system.
|
||||
*/
|
||||
import { describe, it, expect, vi, beforeEach } from 'vitest'
|
||||
import { NextResponse } from 'next/server'
|
||||
import { createQueuedMockSupabase, createMockRequest, parseJsonResponse } from '@/tests/helpers'
|
||||
|
||||
const { supabase, enqueue, reset } = createQueuedMockSupabase()
|
||||
|
||||
const requireAuthMock = vi.fn()
|
||||
vi.mock('@/lib/auth/require-auth', () => ({
|
||||
requireAuth: (...args: unknown[]) => requireAuthMock(...args),
|
||||
}))
|
||||
|
||||
vi.mock('@/lib/company/context', () => ({
|
||||
getActiveCompanyId: vi.fn().mockResolvedValue('company-1'),
|
||||
requireCompanyId: vi.fn().mockResolvedValue('company-1'),
|
||||
}))
|
||||
|
||||
const getCompanyRoleMock = vi.fn()
|
||||
vi.mock('@/lib/auth/require-write', () => ({
|
||||
getCompanyRole: (...args: unknown[]) => getCompanyRoleMock(...args),
|
||||
requireWritePermission: vi.fn().mockResolvedValue({ ok: true }),
|
||||
}))
|
||||
|
||||
vi.mock('@/lib/init', () => ({ ensureInitialized: vi.fn() }))
|
||||
|
||||
const ingestMock = vi.fn()
|
||||
vi.mock('@/lib/transactions/ingest', () => ({
|
||||
ingestTransactions: (...args: unknown[]) => ingestMock(...args),
|
||||
}))
|
||||
|
||||
const sweepMock = vi.fn()
|
||||
vi.mock('@/lib/reconciliation/unattended-sweep', async (importOriginal) => {
|
||||
const actual = await importOriginal<typeof import('@/lib/reconciliation/unattended-sweep')>()
|
||||
return {
|
||||
...actual,
|
||||
runUnattendedReconciliationSweep: (...args: unknown[]) => sweepMock(...args),
|
||||
}
|
||||
})
|
||||
|
||||
import { POST } from '../route'
|
||||
|
||||
const emptyParams = { params: Promise.resolve({}) }
|
||||
|
||||
function makeBody(overrides: Record<string, unknown> = {}) {
|
||||
return {
|
||||
transactions: [
|
||||
{ date: '2025-03-10', description: 'Hyra mars', amount: -12000, currency: 'SEK' },
|
||||
{ date: '2025-01-05', description: 'Kundbetalning', amount: 25000, currency: 'SEK' },
|
||||
],
|
||||
format: 'seb',
|
||||
filename: 'kontoutdrag.csv',
|
||||
file_hash: 'abc123',
|
||||
skip_duplicates: true,
|
||||
auto_categorize: true,
|
||||
...overrides,
|
||||
}
|
||||
}
|
||||
|
||||
function emptyIngestResult(overrides: Record<string, unknown> = {}) {
|
||||
return {
|
||||
imported: 2,
|
||||
duplicates: 0,
|
||||
reconciled: 0,
|
||||
auto_categorized: 0,
|
||||
auto_matched_invoices: 0,
|
||||
errors: 0,
|
||||
transaction_ids: ['t-1', 't-2'],
|
||||
...overrides,
|
||||
}
|
||||
}
|
||||
|
||||
function emptySweepResult(overrides: Record<string, unknown> = {}) {
|
||||
return {
|
||||
accounts: [],
|
||||
applied: 1,
|
||||
errors: 0,
|
||||
skippedBelowThreshold: 1,
|
||||
suggested: 1,
|
||||
unmatched: 0,
|
||||
...overrides,
|
||||
}
|
||||
}
|
||||
|
||||
describe('POST /api/import/bank-file/execute (SIE overlap)', () => {
|
||||
beforeEach(() => {
|
||||
vi.clearAllMocks()
|
||||
reset()
|
||||
requireAuthMock.mockResolvedValue({ user: { id: 'user-1' }, supabase })
|
||||
getCompanyRoleMock.mockResolvedValue({ ok: true, role: 'owner', companyId: 'company-1' })
|
||||
ingestMock.mockResolvedValue(emptyIngestResult())
|
||||
sweepMock.mockResolvedValue(emptySweepResult())
|
||||
})
|
||||
|
||||
it('returns 401 when unauthenticated', async () => {
|
||||
requireAuthMock.mockResolvedValue({
|
||||
user: null,
|
||||
supabase,
|
||||
error: NextResponse.json({ error: 'Unauthorized' }, { status: 401 }),
|
||||
})
|
||||
|
||||
const request = createMockRequest('/api/import/bank-file/execute', {
|
||||
method: 'POST',
|
||||
body: makeBody(),
|
||||
})
|
||||
const response = await POST(request, emptyParams)
|
||||
expect(response.status).toBe(401)
|
||||
})
|
||||
|
||||
it('suppresses auto-categorization, runs the sweep over the file window, and stamps the summary on SIE overlap', async () => {
|
||||
enqueue({ data: { id: 'import-1' } }) // bank_file_imports upsert
|
||||
enqueue({ data: { id: 'sie-1' } }) // sie_imports overlap: found
|
||||
enqueue({ data: null }) // bank_file_imports status update
|
||||
enqueue({ data: null }) // sie_sweep stamp update
|
||||
enqueue({ data: [{ id: 't-1' }, { id: 't-2' }] }) // imported tx for event
|
||||
|
||||
const request = createMockRequest('/api/import/bank-file/execute', {
|
||||
method: 'POST',
|
||||
body: makeBody(),
|
||||
})
|
||||
const response = await POST(request, emptyParams)
|
||||
const { status } = await parseJsonResponse(response)
|
||||
|
||||
expect(status).toBe(200)
|
||||
// Ingest was told to skip auto-categorization (double-booking guard).
|
||||
const ingestOptions = ingestMock.mock.calls[0][4] as Record<string, unknown>
|
||||
expect(ingestOptions.skipAutoCategorization).toBe(true)
|
||||
// Sweep ran over the file's own date window (min/max of its rows).
|
||||
expect(sweepMock).toHaveBeenCalledWith(supabase, 'company-1', 'user-1', {
|
||||
dateFrom: '2025-01-05',
|
||||
dateTo: '2025-03-10',
|
||||
})
|
||||
})
|
||||
|
||||
it('runs no sweep and keeps categorization when there is no SIE overlap', async () => {
|
||||
enqueue({ data: { id: 'import-1' } }) // upsert
|
||||
enqueue({ data: null }) // sie_imports overlap: none
|
||||
enqueue({ data: null }) // status update
|
||||
enqueue({ data: [{ id: 't-1' }] }) // imported tx for event
|
||||
|
||||
const request = createMockRequest('/api/import/bank-file/execute', {
|
||||
method: 'POST',
|
||||
body: makeBody(),
|
||||
})
|
||||
const response = await POST(request, emptyParams)
|
||||
|
||||
expect(response.status).toBe(200)
|
||||
const ingestOptions = ingestMock.mock.calls[0][4] as Record<string, unknown>
|
||||
expect(ingestOptions.skipAutoCategorization).toBeUndefined()
|
||||
expect(sweepMock).not.toHaveBeenCalled()
|
||||
})
|
||||
|
||||
it('never sweeps for a viewer (raw insert only)', async () => {
|
||||
getCompanyRoleMock.mockResolvedValue({ ok: true, role: 'viewer', companyId: 'company-1' })
|
||||
enqueue({ data: { id: 'import-1' } }) // upsert
|
||||
enqueue({ data: { id: 'sie-1' } }) // overlap found
|
||||
enqueue({ data: null }) // status update
|
||||
enqueue({ data: [{ id: 't-1' }] }) // imported tx for event
|
||||
|
||||
const request = createMockRequest('/api/import/bank-file/execute', {
|
||||
method: 'POST',
|
||||
body: makeBody(),
|
||||
})
|
||||
const response = await POST(request, emptyParams)
|
||||
|
||||
expect(response.status).toBe(200)
|
||||
const ingestOptions = ingestMock.mock.calls[0][4] as Record<string, unknown>
|
||||
expect(ingestOptions.rawInsertOnly).toBe(true)
|
||||
expect(sweepMock).not.toHaveBeenCalled()
|
||||
})
|
||||
})
|
||||
@@ -10,6 +10,10 @@ import { errorResponseFromCode } from '@/lib/errors/get-structured-error'
|
||||
import type { ParsedBankTransaction, BankFileFormatId } from '@/lib/import/bank-file/types'
|
||||
import type { Transaction } from '@/types'
|
||||
import { getErrorMessage as getUserErrorMessage } from '@/lib/errors/get-error-message'
|
||||
import {
|
||||
runUnattendedReconciliationSweep,
|
||||
toSweepSummary,
|
||||
} from '@/lib/reconciliation/unattended-sweep'
|
||||
|
||||
ensureInitialized()
|
||||
|
||||
@@ -102,9 +106,30 @@ export const POST = withRouteContext(
|
||||
import_source: format === 'camt053' ? 'camt053' : `csv_${format}`,
|
||||
}))
|
||||
|
||||
// Detect SIE overlap, mirroring the enable-banking sync paths: a bank
|
||||
// file covering a period a completed SIE import already booked must be
|
||||
// matched against the imported verifikat, not re-booked. CSV is the only
|
||||
// way a migrator gets deep history (PSD2 windows stop at ~90 days), so
|
||||
// this path is the primary one for the Fortnox/SIE migrator journey.
|
||||
const fileDateFrom = transactions.map((t) => t.date).sort()[0] || undefined
|
||||
const fileDateTo = transactions.map((t) => t.date).sort().reverse()[0] || undefined
|
||||
let sieOverlap: { id: string } | null = null
|
||||
if (fileDateFrom) {
|
||||
const { data } = await supabase
|
||||
.from('sie_imports')
|
||||
.select('id')
|
||||
.eq('company_id', companyId)
|
||||
.eq('status', 'completed')
|
||||
.gte('fiscal_year_end', fileDateFrom)
|
||||
.limit(1)
|
||||
.maybeSingle()
|
||||
sieOverlap = data ?? null
|
||||
}
|
||||
|
||||
const ingestOptions: IngestOptions = {}
|
||||
if (settlement_account) ingestOptions.settlementAccount = settlement_account
|
||||
if (role === 'viewer') ingestOptions.rawInsertOnly = true
|
||||
if (sieOverlap) ingestOptions.skipAutoCategorization = true
|
||||
const ingestResult = await ingestTransactions(supabase, companyId, user.id, rawTransactions, ingestOptions)
|
||||
|
||||
if (ingestResult.errors > 0 && ingestResult.first_error) {
|
||||
@@ -133,6 +158,44 @@ export const POST = withRouteContext(
|
||||
})
|
||||
.eq('id', importRecord.id)
|
||||
|
||||
// SIE-overlap-gated reconciliation sweep (issue: no sweep fired after a
|
||||
// bank CSV import, yet CSV is how a migrator gets pre-PSD2 history). One
|
||||
// scoped run per enabled cash account; >= 0.9 auto-links, the 0.75-0.89
|
||||
// band persists as reviewable suggestions. Viewers skip it: the sweep
|
||||
// updates transactions, which viewers cannot do.
|
||||
if (sieOverlap && ingestResult.imported > 0 && role !== 'viewer') {
|
||||
try {
|
||||
const sweepResult = await runUnattendedReconciliationSweep(supabase, companyId, user.id, {
|
||||
dateFrom: fileDateFrom,
|
||||
dateTo: fileDateTo,
|
||||
})
|
||||
const { error: stampError } = await supabase
|
||||
.from('bank_file_imports')
|
||||
.update({
|
||||
sie_sweep: toSweepSummary(sweepResult, {
|
||||
dateFrom: fileDateFrom,
|
||||
dateTo: fileDateTo,
|
||||
}),
|
||||
})
|
||||
.eq('id', importRecord.id)
|
||||
if (stampError) {
|
||||
// The links/suggestions are already written; only the UI summary
|
||||
// is missing. Say so instead of letting the sweep look unrun.
|
||||
opLog.warn('failed to stamp sie_sweep summary on bank_file_imports', stampError)
|
||||
}
|
||||
if (sweepResult.applied > 0 || sweepResult.suggested > 0) {
|
||||
opLog.info('post-import SIE reconciliation sweep', {
|
||||
applied: sweepResult.applied,
|
||||
suggested: sweepResult.suggested,
|
||||
unmatched: sweepResult.unmatched,
|
||||
})
|
||||
}
|
||||
} catch (err) {
|
||||
// Non-critical: rows stay in "Att bokföra" for manual matching.
|
||||
opLog.warn('post-import SIE reconciliation sweep failed', err as Error)
|
||||
}
|
||||
}
|
||||
|
||||
if (ingestResult.imported > 0 && ingestResult.transaction_ids.length > 0) {
|
||||
try {
|
||||
const { data: importedTransactions } = await supabase
|
||||
|
||||
Reference in New Issue
Block a user