feat(reconciliation): match migrated bank history against imported SIE verifikat (#1598)
* feat(reconciliation): match migrated bank history against imported SIE verifikat A first-class Fortnox/SIE migrator path: after SIE import plus bank connect or bank CSV upload, historical bank rows are auto-matched (>= 0.9) or suggestion-matched (0.75-0.89, persisted for review) against the imported verifikat, with a guided review surface, instead of landing as anonymous "Att bokfora" rows. Phase 0: per-cash-account unattended sweep (fixes #1298 cross-account pooling); widen payment_match_log action CHECK with linked_to_existing_voucher (silently unlogged since March). Phase 1: potential_journal_entry_id/method/confidence on transactions with CHECK + invalidation triggers; persistSuggestions in runReconciliation; sweep after bank CSV import with SIE overlap (suppressing auto-categorization); sweep summaries stamped on bank_connections and bank_file_imports; POST /api/reconciliation/bank/confirm-suggestions with per-pair server-side revalidation (voucher consumption + bank-leg amount and direction). Phase 2: "Granska forslag" review tab on Transactions with chunked bulk confirm, per-row fallbacks, "Kor matchning igen" (all_accounts sweep mode, mutually exclusive with dry_run), attn line, pre-migration row marker. Phase 3: ImportResultStep dual CTA (bank connect + CSV), migrator variant of the account-picker #917 nudge, sweep outcome on the onboarding checklist bank step. Non-selection apply runs on /api/reconciliation/bank/run now floor at 0.9 and persist the review band instead of auto-committing fuzzy matches. Migrations already applied to staging under the same versions. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com> * fix(reconciliation): resolve PR review findings in one pass Swedish accounting review (both previously-deferred holes closed): - runReconciliation's >= 0.9 auto-apply now writes 'matched' to payment_match_log (behandlingshistorik, BFNAR 2013:2 kap 8); the bus event alone lands in the 30-day event_log and is not an audit record. - The three match-route storno-conflict branches detach reconciliation links via unlinkReconciliation instead of storno-reversing the linked verifikat: a reconciliation link points at an independent verifikat that may evidence other affarshandelser, and a wholesale reversal is an over-broad rattelse (BFL 5 kap 5 §). - Historical gap quantified on prod (read-only, recorded in DECISIONS): 762 unlogged manual links across 52 companies since 2026-03-23. CodeRabbit: - confirm-suggestions route: maxDuration 300 for full 500-item batches. - AccountPickerDialog: migrator-nudge buttons set lookbackTouched so the async gap-fill probe cannot override an explicit choice. - enable-banking post-backfill sweep: persistSuggestions so the review band is not dropped. - bank-file execute: sie_sweep stamp errors are logged, not swallowed. - ImportResultStep: sandbox keeps the CSV CTA (file import works there). - payment_match_log CHECK swap: NOT VALID + VALIDATE, no table scan under ACCESS EXCLUSIVE. - logMatchEvent calls awaited (serverless can freeze unawaited work). - DECISIONS.md stale version reference annotated. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com> * fix(reconciliation): defer reconciliation-link detach until the match commits Round-2 review findings: - CodeRabbit: the eager unlinkReconciliation call could orphan a transaction if the match flow failed after it. All three match routes now persist NOTHING up front: the final transaction update overwrites journal_entry_id and clears reconciliation_method in the same write, so any failure in between leaves the existing link intact. The release is logged as 'unmatched' after the commit. - Swedish review: the auto_suggested logMatchEvent in runReconciliation is now awaited like every other audit write. - DECISIONS entry split into compliance/CodeRabbit lines and updated to describe the deferred detach. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com> * fix(reconciliation): literal reconciliation_method payloads for the phantom-column scanner The conditional spreads introduced with the deferred detach pushed the scanner's unresolvable-expression count past its ceiling (380 > 378). reconciliation_method: null is correct unconditionally on a confirmed invoice/supplier match (null is already the value on every row that was not reconciliation-linked), so the payloads become plain literals the guard can verify. No behavior change. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com> --------- Co-authored-by: Claude Fable 5 <noreply@anthropic.com>
This commit is contained in:
Mattsson
co-authored by
Claude Fable 5
Claude Fable 5
parent
07e89d9b52
commit
08440fed94
@@ -0,0 +1,65 @@
|
||||
import { describe, expect, it } from 'vitest'
|
||||
import { seedCompany, insertTransaction } from '@/tests/pg/fixtures'
|
||||
import { getPool } from '@/tests/pg/setup'
|
||||
|
||||
/**
|
||||
* pg-real coverage for the payment_match_log action CHECK
|
||||
* (20260323120000_payment_match_log.sql +
|
||||
* 20260813210000_payment_match_log_linked_to_existing_voucher.sql).
|
||||
*
|
||||
* Locks in Gap F: the code has emitted action = 'linked_to_existing_voucher'
|
||||
* since the existing-voucher link paths shipped, but the original CHECK never
|
||||
* included the value, and because logMatchEvent is fire-and-forget the
|
||||
* constraint violation was swallowed: every existing-voucher link went
|
||||
* unlogged. This suite asserts the widened CHECK accepts the value, so a
|
||||
* future rewrite of the constraint that forgets it fails here instead of
|
||||
* silently reopening the audit-trail gap.
|
||||
*/
|
||||
|
||||
async function insertLogRow(params: {
|
||||
userId: string
|
||||
transactionId: string
|
||||
action: string
|
||||
}): Promise<void> {
|
||||
await getPool().query(
|
||||
`INSERT INTO public.payment_match_log (user_id, transaction_id, action)
|
||||
VALUES ($1, $2, $3)`,
|
||||
[params.userId, params.transactionId, params.action],
|
||||
)
|
||||
}
|
||||
|
||||
describe('payment_match_log.action CHECK', () => {
|
||||
it('accepts every action the code emits, including linked_to_existing_voucher', async () => {
|
||||
const { userId, companyId } = await seedCompany()
|
||||
const txId = await insertTransaction({ companyId, userId })
|
||||
|
||||
// The full MatchAction union from lib/invoices/match-log.ts. If a new
|
||||
// action is added there, this list (and the CHECK) must grow with it.
|
||||
const actions = [
|
||||
'matched',
|
||||
'unmatched',
|
||||
'auto_suggested',
|
||||
'suggestion_cleared',
|
||||
'storno_conflict_resolved',
|
||||
'linked_to_existing_voucher',
|
||||
]
|
||||
for (const action of actions) {
|
||||
await expect(insertLogRow({ userId, transactionId: txId, action })).resolves.not.toThrow()
|
||||
}
|
||||
|
||||
const { rows } = await getPool().query(
|
||||
`SELECT count(*)::int AS n FROM public.payment_match_log WHERE transaction_id = $1`,
|
||||
[txId],
|
||||
)
|
||||
expect(rows[0].n).toBe(actions.length)
|
||||
})
|
||||
|
||||
it('still rejects unknown actions', async () => {
|
||||
const { userId, companyId } = await seedCompany()
|
||||
const txId = await insertTransaction({ companyId, userId })
|
||||
|
||||
await expect(
|
||||
insertLogRow({ userId, transactionId: txId, action: 'not_a_real_action' }),
|
||||
).rejects.toThrow(/payment_match_log_action_check/)
|
||||
})
|
||||
})
|
||||
@@ -0,0 +1,185 @@
|
||||
import { describe, expect, it } from 'vitest'
|
||||
import { seedCompany, insertTransaction, insertDraftJournalEntry } from '@/tests/pg/fixtures'
|
||||
import { getPool } from '@/tests/pg/setup'
|
||||
|
||||
/**
|
||||
* pg-real coverage for persisted journal-entry match suggestions
|
||||
* (20260813121000_transactions_potential_journal_entry.sql).
|
||||
*
|
||||
* Locks in:
|
||||
* - the all-or-nothing CHECK on the suggestion trio,
|
||||
* - self-clear on link / ignore (BEFORE trigger),
|
||||
* - sibling-clear when the suggested verifikat is consumed by another
|
||||
* transaction's link (the double-consume guard bulk confirm relies on),
|
||||
* - clear on storno reversal of the suggested entry,
|
||||
* - the sweep-summary JSONB columns.
|
||||
*/
|
||||
|
||||
// status: 'posted' routes to insertPostedJournalEntry, which inserts a default
|
||||
// balanced 1930/3001 line pair in the same transaction: adding lines afterwards
|
||||
// would trip the line-immutability trigger.
|
||||
async function insertPostedEntry(params: {
|
||||
userId: string
|
||||
companyId: string
|
||||
fiscalPeriodId: string
|
||||
}): Promise<string> {
|
||||
return insertDraftJournalEntry({ ...params, status: 'posted' })
|
||||
}
|
||||
|
||||
async function setSuggestion(txId: string, jeId: string): Promise<void> {
|
||||
await getPool().query(
|
||||
`UPDATE public.transactions
|
||||
SET potential_journal_entry_id = $2,
|
||||
potential_match_method = 'auto_date_range',
|
||||
potential_match_confidence = 0.85
|
||||
WHERE id = $1`,
|
||||
[txId, jeId],
|
||||
)
|
||||
}
|
||||
|
||||
async function getSuggestion(txId: string): Promise<{
|
||||
potential_journal_entry_id: string | null
|
||||
potential_match_method: string | null
|
||||
potential_match_confidence: string | null
|
||||
}> {
|
||||
const { rows } = await getPool().query(
|
||||
`SELECT potential_journal_entry_id, potential_match_method, potential_match_confidence
|
||||
FROM public.transactions WHERE id = $1`,
|
||||
[txId],
|
||||
)
|
||||
return rows[0]
|
||||
}
|
||||
|
||||
describe('transactions potential_journal_entry: schema', () => {
|
||||
it('accepts a full suggestion trio on an unlinked transaction', async () => {
|
||||
const { userId, companyId, fiscalPeriodId } = await seedCompany()
|
||||
const jeId = await insertPostedEntry({ userId, companyId, fiscalPeriodId })
|
||||
const txId = await insertTransaction({ companyId, userId })
|
||||
|
||||
await setSuggestion(txId, jeId)
|
||||
|
||||
const row = await getSuggestion(txId)
|
||||
expect(row.potential_journal_entry_id).toBe(jeId)
|
||||
expect(row.potential_match_method).toBe('auto_date_range')
|
||||
expect(Number(row.potential_match_confidence)).toBe(0.85)
|
||||
})
|
||||
|
||||
it('rejects a partial trio (id without method/confidence)', async () => {
|
||||
const { userId, companyId, fiscalPeriodId } = await seedCompany()
|
||||
const jeId = await insertPostedEntry({ userId, companyId, fiscalPeriodId })
|
||||
const txId = await insertTransaction({ companyId, userId })
|
||||
|
||||
await expect(
|
||||
getPool().query(
|
||||
`UPDATE public.transactions SET potential_journal_entry_id = $2 WHERE id = $1`,
|
||||
[txId, jeId],
|
||||
),
|
||||
).rejects.toThrow(/transactions_potential_je_check/)
|
||||
})
|
||||
})
|
||||
|
||||
describe('transactions potential_journal_entry: self-clear trigger', () => {
|
||||
it('clears the suggestion when the transaction is linked to any journal entry', async () => {
|
||||
const { userId, companyId, fiscalPeriodId } = await seedCompany()
|
||||
const suggestedJe = await insertPostedEntry({ userId, companyId, fiscalPeriodId })
|
||||
const otherJe = await insertPostedEntry({ userId, companyId, fiscalPeriodId })
|
||||
const txId = await insertTransaction({ companyId, userId })
|
||||
await setSuggestion(txId, suggestedJe)
|
||||
|
||||
// Link the row elsewhere (any path setting journal_entry_id qualifies).
|
||||
await getPool().query(
|
||||
`UPDATE public.transactions SET journal_entry_id = $2 WHERE id = $1`,
|
||||
[txId, otherJe],
|
||||
)
|
||||
|
||||
const row = await getSuggestion(txId)
|
||||
expect(row.potential_journal_entry_id).toBeNull()
|
||||
expect(row.potential_match_method).toBeNull()
|
||||
expect(row.potential_match_confidence).toBeNull()
|
||||
})
|
||||
|
||||
it('clears the suggestion when the transaction is ignored', async () => {
|
||||
const { userId, companyId, fiscalPeriodId } = await seedCompany()
|
||||
const jeId = await insertPostedEntry({ userId, companyId, fiscalPeriodId })
|
||||
const txId = await insertTransaction({ companyId, userId })
|
||||
await setSuggestion(txId, jeId)
|
||||
|
||||
await getPool().query(`UPDATE public.transactions SET is_ignored = true WHERE id = $1`, [txId])
|
||||
|
||||
const row = await getSuggestion(txId)
|
||||
expect(row.potential_journal_entry_id).toBeNull()
|
||||
})
|
||||
})
|
||||
|
||||
describe('transactions potential_journal_entry: sibling-clear on consumption', () => {
|
||||
it('clears other rows suggesting a verifikat once any transaction links to it', async () => {
|
||||
const { userId, companyId, fiscalPeriodId } = await seedCompany()
|
||||
const jeId = await insertPostedEntry({ userId, companyId, fiscalPeriodId })
|
||||
const txA = await insertTransaction({ companyId, userId, externalId: 'ext-a' })
|
||||
const txB = await insertTransaction({ companyId, userId, externalId: 'ext-b' })
|
||||
await setSuggestion(txA, jeId)
|
||||
await setSuggestion(txB, jeId)
|
||||
|
||||
// txA consumes the verifikat.
|
||||
await getPool().query(
|
||||
`UPDATE public.transactions SET journal_entry_id = $2 WHERE id = $1`,
|
||||
[txA, jeId],
|
||||
)
|
||||
|
||||
// txB's suggestion is gone: bulk confirm can no longer double-consume.
|
||||
const rowB = await getSuggestion(txB)
|
||||
expect(rowB.potential_journal_entry_id).toBeNull()
|
||||
expect(rowB.potential_match_method).toBeNull()
|
||||
})
|
||||
|
||||
it('does not clear suggestions pointing at OTHER verifikat', async () => {
|
||||
const { userId, companyId, fiscalPeriodId } = await seedCompany()
|
||||
const je1 = await insertPostedEntry({ userId, companyId, fiscalPeriodId })
|
||||
const je2 = await insertPostedEntry({ userId, companyId, fiscalPeriodId })
|
||||
const txA = await insertTransaction({ companyId, userId, externalId: 'ext-a2' })
|
||||
const txB = await insertTransaction({ companyId, userId, externalId: 'ext-b2' })
|
||||
await setSuggestion(txA, je1)
|
||||
await setSuggestion(txB, je2)
|
||||
|
||||
await getPool().query(
|
||||
`UPDATE public.transactions SET journal_entry_id = $2 WHERE id = $1`,
|
||||
[txA, je1],
|
||||
)
|
||||
|
||||
const rowB = await getSuggestion(txB)
|
||||
expect(rowB.potential_journal_entry_id).toBe(je2)
|
||||
})
|
||||
})
|
||||
|
||||
describe('transactions potential_journal_entry: clear on reversal', () => {
|
||||
it('clears suggestions pointing at an entry that is reversed via storno', async () => {
|
||||
const { userId, companyId, fiscalPeriodId } = await seedCompany()
|
||||
const jeId = await insertPostedEntry({ userId, companyId, fiscalPeriodId })
|
||||
const txId = await insertTransaction({ companyId, userId })
|
||||
await setSuggestion(txId, jeId)
|
||||
|
||||
// The storno path's status transition (posted -> reversed).
|
||||
await getPool().query(
|
||||
`UPDATE public.journal_entries SET status = 'reversed' WHERE id = $1`,
|
||||
[jeId],
|
||||
)
|
||||
|
||||
const row = await getSuggestion(txId)
|
||||
expect(row.potential_journal_entry_id).toBeNull()
|
||||
})
|
||||
})
|
||||
|
||||
describe('sweep summary columns', () => {
|
||||
it('bank_file_imports.sie_sweep and bank_connections.last_sie_sweep exist as JSONB', async () => {
|
||||
const { rows } = await getPool().query(
|
||||
`SELECT table_name, column_name, data_type
|
||||
FROM information_schema.columns
|
||||
WHERE (table_name = 'bank_file_imports' AND column_name = 'sie_sweep')
|
||||
OR (table_name = 'bank_connections' AND column_name = 'last_sie_sweep')`,
|
||||
)
|
||||
expect(rows).toHaveLength(2)
|
||||
for (const row of rows) {
|
||||
expect(row.data_type).toBe('jsonb')
|
||||
}
|
||||
})
|
||||
})
|
||||
Reference in New Issue
Block a user