Files
accounted/app/api/import/bank-file/execute/route.ts
T
MattssonandClaude Fable 5 08440fed94 feat(reconciliation): match migrated bank history against imported SIE verifikat (#1598)
* feat(reconciliation): match migrated bank history against imported SIE verifikat

A first-class Fortnox/SIE migrator path: after SIE import plus bank connect
or bank CSV upload, historical bank rows are auto-matched (>= 0.9) or
suggestion-matched (0.75-0.89, persisted for review) against the imported
verifikat, with a guided review surface, instead of landing as anonymous
"Att bokfora" rows.

Phase 0: per-cash-account unattended sweep (fixes #1298 cross-account
pooling); widen payment_match_log action CHECK with
linked_to_existing_voucher (silently unlogged since March).
Phase 1: potential_journal_entry_id/method/confidence on transactions with
CHECK + invalidation triggers; persistSuggestions in runReconciliation;
sweep after bank CSV import with SIE overlap (suppressing
auto-categorization); sweep summaries stamped on bank_connections and
bank_file_imports; POST /api/reconciliation/bank/confirm-suggestions with
per-pair server-side revalidation (voucher consumption + bank-leg amount
and direction).
Phase 2: "Granska forslag" review tab on Transactions with chunked bulk
confirm, per-row fallbacks, "Kor matchning igen" (all_accounts sweep mode,
mutually exclusive with dry_run), attn line, pre-migration row marker.
Phase 3: ImportResultStep dual CTA (bank connect + CSV), migrator variant
of the account-picker #917 nudge, sweep outcome on the onboarding
checklist bank step.

Non-selection apply runs on /api/reconciliation/bank/run now floor at 0.9
and persist the review band instead of auto-committing fuzzy matches.
Migrations already applied to staging under the same versions.

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>

* fix(reconciliation): resolve PR review findings in one pass

Swedish accounting review (both previously-deferred holes closed):
- runReconciliation's >= 0.9 auto-apply now writes 'matched' to
  payment_match_log (behandlingshistorik, BFNAR 2013:2 kap 8); the bus
  event alone lands in the 30-day event_log and is not an audit record.
- The three match-route storno-conflict branches detach reconciliation
  links via unlinkReconciliation instead of storno-reversing the linked
  verifikat: a reconciliation link points at an independent verifikat
  that may evidence other affarshandelser, and a wholesale reversal is
  an over-broad rattelse (BFL 5 kap 5 §).
- Historical gap quantified on prod (read-only, recorded in DECISIONS):
  762 unlogged manual links across 52 companies since 2026-03-23.

CodeRabbit:
- confirm-suggestions route: maxDuration 300 for full 500-item batches.
- AccountPickerDialog: migrator-nudge buttons set lookbackTouched so the
  async gap-fill probe cannot override an explicit choice.
- enable-banking post-backfill sweep: persistSuggestions so the review
  band is not dropped.
- bank-file execute: sie_sweep stamp errors are logged, not swallowed.
- ImportResultStep: sandbox keeps the CSV CTA (file import works there).
- payment_match_log CHECK swap: NOT VALID + VALIDATE, no table scan
  under ACCESS EXCLUSIVE.
- logMatchEvent calls awaited (serverless can freeze unawaited work).
- DECISIONS.md stale version reference annotated.

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>

* fix(reconciliation): defer reconciliation-link detach until the match commits

Round-2 review findings:
- CodeRabbit: the eager unlinkReconciliation call could orphan a
  transaction if the match flow failed after it. All three match routes
  now persist NOTHING up front: the final transaction update overwrites
  journal_entry_id and clears reconciliation_method in the same write,
  so any failure in between leaves the existing link intact. The release
  is logged as 'unmatched' after the commit.
- Swedish review: the auto_suggested logMatchEvent in runReconciliation
  is now awaited like every other audit write.
- DECISIONS entry split into compliance/CodeRabbit lines and updated to
  describe the deferred detach.

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>

* fix(reconciliation): literal reconciliation_method payloads for the phantom-column scanner

The conditional spreads introduced with the deferred detach pushed the
scanner's unresolvable-expression count past its ceiling (380 > 378).
reconciliation_method: null is correct unconditionally on a confirmed
invoice/supplier match (null is already the value on every row that was
not reconciliation-linked), so the payloads become plain literals the
guard can verify. No behavior change.

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>

---------

Co-authored-by: Claude Fable 5 <noreply@anthropic.com>
2026-08-13 23:12:27 +02:00

236 lines
9.3 KiB
TypeScript

import { NextResponse } from 'next/server'
import { eventBus } from '@/lib/events'
import { ensureInitialized } from '@/lib/init'
import { ingestTransactions, type RawTransaction } from '@/lib/transactions/ingest'
import { generateExternalId } from '@/lib/import/bank-file/parser'
import type { IngestOptions } from '@/types'
import { getCompanyRole } from '@/lib/auth/require-write'
import { withRouteContext } from '@/lib/api/with-route-context'
import { errorResponseFromCode } from '@/lib/errors/get-structured-error'
import type { ParsedBankTransaction, BankFileFormatId } from '@/lib/import/bank-file/types'
import type { Transaction } from '@/types'
import { getErrorMessage as getUserErrorMessage } from '@/lib/errors/get-error-message'
import {
runUnattendedReconciliationSweep,
toSweepSummary,
} from '@/lib/reconciliation/unattended-sweep'
ensureInitialized()
// Bank-file imports run a sequential, per-row ingest (insert + invoice/supplier
// matching + FX lookup). A full-year file (300+ rows) takes ~85s of server time,
// which sits right on the platform's default function limit and gets killed
// mid-run: the import "spins then aborts" for the user. Give it the same 5-minute
// budget the SIE import route uses (app/api/import/sie/execute/route.ts).
export const maxDuration = 300
interface ExecuteRequest {
transactions: ParsedBankTransaction[]
format: BankFileFormatId
filename: string
file_hash: string
skip_duplicates: boolean
auto_categorize: boolean
settlement_account?: string
}
/**
* POST /api/import/bank-file/execute
*
* Executes the import of confirmed bank transactions. Records the import in
* `bank_file_imports`, calls `ingestTransactions`, and emits `transaction.synced`.
*/
export const POST = withRouteContext(
'bank_file.execute',
async (request, ctx) => {
const { user, supabase, log, requestId } = ctx
// We still call getCompanyRole because viewers are allowed through with
// rawInsertOnly behavior: `requireWrite: true` would block them.
const roleCheck = await getCompanyRole(supabase, user.id)
if (!roleCheck.ok) {
// Inject the request id for traceability and pass through.
if (!roleCheck.response.headers.get('X-Request-Id')) {
roleCheck.response.headers.set('X-Request-Id', requestId)
}
return roleCheck.response
}
const { role, companyId } = roleCheck
const body: ExecuteRequest = await request.json()
const {
transactions, format, filename, file_hash,
skip_duplicates: _skip_duplicates = true,
auto_categorize: _auto_categorize = true,
settlement_account,
} = body
if (!transactions || transactions.length === 0) {
return errorResponseFromCode('BANK_FILE_NO_TRANSACTIONS', log, { requestId })
}
const opLog = log.child({ filename, fileHash: file_hash, txCount: transactions.length })
try {
const { data: importRecord, error: importError } = await supabase
.from('bank_file_imports')
.upsert({
user_id: user.id,
company_id: companyId,
filename,
file_hash,
file_format: format,
transaction_count: transactions.length,
status: 'processing',
date_from: transactions.map((t) => t.date).sort()[0] || null,
date_to: transactions.map((t) => t.date).sort().reverse()[0] || null,
}, { onConflict: 'company_id,file_hash' })
.select()
.single()
if (importError) {
opLog.error('failed to create bank_file_imports record', importError)
return errorResponseFromCode('BANK_FILE_IMPORT_RECORD_FAILED', opLog, {
requestId,
details: { reason: getUserErrorMessage(importError) },
})
}
const rawTransactions: RawTransaction[] = transactions.map((tx, index) => ({
date: tx.date,
description: tx.description,
amount: tx.amount,
currency: tx.currency || 'SEK',
external_id: generateExternalId(tx, format, index),
reference: tx.reference || null,
import_source: format === 'camt053' ? 'camt053' : `csv_${format}`,
}))
// Detect SIE overlap, mirroring the enable-banking sync paths: a bank
// file covering a period a completed SIE import already booked must be
// matched against the imported verifikat, not re-booked. CSV is the only
// way a migrator gets deep history (PSD2 windows stop at ~90 days), so
// this path is the primary one for the Fortnox/SIE migrator journey.
const fileDateFrom = transactions.map((t) => t.date).sort()[0] || undefined
const fileDateTo = transactions.map((t) => t.date).sort().reverse()[0] || undefined
let sieOverlap: { id: string } | null = null
if (fileDateFrom) {
const { data } = await supabase
.from('sie_imports')
.select('id')
.eq('company_id', companyId)
.eq('status', 'completed')
.gte('fiscal_year_end', fileDateFrom)
.limit(1)
.maybeSingle()
sieOverlap = data ?? null
}
const ingestOptions: IngestOptions = {}
if (settlement_account) ingestOptions.settlementAccount = settlement_account
if (role === 'viewer') ingestOptions.rawInsertOnly = true
if (sieOverlap) ingestOptions.skipAutoCategorization = true
const ingestResult = await ingestTransactions(supabase, companyId, user.id, rawTransactions, ingestOptions)
if (ingestResult.errors > 0 && ingestResult.first_error) {
opLog.error('bank file ingest reported insert errors', new Error(ingestResult.first_error.message), {
errorCount: ingestResult.errors,
code: ingestResult.first_error.code,
details: ingestResult.first_error.details,
hint: ingestResult.first_error.hint,
})
}
const errorMessage = ingestResult.errors > 0
? ingestResult.first_error
? `${ingestResult.errors} fel: ${ingestResult.first_error.message}${ingestResult.first_error.details ? ` (${ingestResult.first_error.details})` : ''}`
: `${ingestResult.errors} transactions failed to import`
: null
await supabase
.from('bank_file_imports')
.update({
imported_count: ingestResult.imported,
duplicate_count: ingestResult.duplicates,
matched_count: ingestResult.auto_matched_invoices,
status: ingestResult.errors > 0 && ingestResult.imported === 0 ? 'failed' : 'completed',
error_message: errorMessage,
})
.eq('id', importRecord.id)
// SIE-overlap-gated reconciliation sweep (issue: no sweep fired after a
// bank CSV import, yet CSV is how a migrator gets pre-PSD2 history). One
// scoped run per enabled cash account; >= 0.9 auto-links, the 0.75-0.89
// band persists as reviewable suggestions. Viewers skip it: the sweep
// updates transactions, which viewers cannot do.
if (sieOverlap && ingestResult.imported > 0 && role !== 'viewer') {
try {
const sweepResult = await runUnattendedReconciliationSweep(supabase, companyId, user.id, {
dateFrom: fileDateFrom,
dateTo: fileDateTo,
})
const { error: stampError } = await supabase
.from('bank_file_imports')
.update({
sie_sweep: toSweepSummary(sweepResult, {
dateFrom: fileDateFrom,
dateTo: fileDateTo,
}),
})
.eq('id', importRecord.id)
if (stampError) {
// The links/suggestions are already written; only the UI summary
// is missing. Say so instead of letting the sweep look unrun.
opLog.warn('failed to stamp sie_sweep summary on bank_file_imports', stampError)
}
if (sweepResult.applied > 0 || sweepResult.suggested > 0) {
opLog.info('post-import SIE reconciliation sweep', {
applied: sweepResult.applied,
suggested: sweepResult.suggested,
unmatched: sweepResult.unmatched,
})
}
} catch (err) {
// Non-critical: rows stay in "Att bokföra" for manual matching.
opLog.warn('post-import SIE reconciliation sweep failed', err as Error)
}
}
if (ingestResult.imported > 0 && ingestResult.transaction_ids.length > 0) {
try {
const { data: importedTransactions } = await supabase
.from('transactions')
.select('*')
.in('id', ingestResult.transaction_ids)
if (importedTransactions && importedTransactions.length > 0) {
await eventBus.emit({
type: 'transaction.synced',
payload: {
transactions: importedTransactions as Transaction[],
userId: user.id,
companyId,
},
})
}
} catch (err) {
opLog.warn('transaction.synced event emission failed', err as Error)
}
}
return NextResponse.json({
data: {
import_id: importRecord.id,
...ingestResult,
},
})
} catch (err) {
opLog.error('bank file execute failed', err as Error)
return errorResponseFromCode('BANK_FILE_EXECUTE_FAILED', opLog, {
requestId,
details: { reason: err instanceof Error ? getUserErrorMessage(err) : 'unknown' },
})
}
},
)