Files
accounted/lib/import/__tests__/sie-import.replace.pg.test.ts
T
Jakob Wennberg 831920fede fix(arcim): allow Fortnox re-sync to replace prior SIE import per fiscal year (#512)
* fix(arcim): allow Fortnox re-sync to replace prior SIE import per fiscal year

A user reported a sync failure when retrying Fortnox after adding more
verifications:

  Import failed: Failed to create pending import record:
  duplicate key value violates unique constraint
  "sie_imports_company_id_file_hash_active_idx"

Root cause: Fortnox embeds the export-time #GEN date in every SIE export,
so the file hash always differs between syncs. The wizard's hash-based
duplicate detection treated each sync as a brand new file, but the engine
still rejected the insert because the per-period import slot was held by
the prior 'completed' row.

This change reframes Fortnox re-sync as a replace operation rather than a
fresh import:

- New executeSIEImport option `onExistingPeriod: 'block' | 'replace'`.
  Manual SIE upload at /api/import/sie keeps default 'block' (current
  behavior, no regression). The Fortnox /import-sie endpoint passes
  'replace', which runs replaceSIEImport on any overlapping completed
  import before insert. Imported journal entries from the prior import are
  cancelled per BFL 5 kap 5§; user-created entries (manual, transaction,
  invoice) are untouched.

- /sie-data switches from hash-based to period-based duplicate detection
  and returns previousImport metadata per fiscal year.

- Wizard drops the alreadyImported skip filter, surfaces an amber callout
  in the confirm dialog listing fiscal years that will be replaced, and
  shows "ersatte N tidigare importerade verifikationer" per year.

- createPendingImportRecord translates 23505 partial-index violations to
  a clear Swedish recovery message instead of leaking the raw constraint
  name.

- cleanupStaleImportRecords drops the 1-hour age gate and also cleans
  status='mapped' orphans. SIE imports are single-flight per company so
  the gate just made legitimate retries fail.

- After replace, the fiscal_periods row's opening_balances_set and
  opening_balance_entry_id are cleared (only when they pointed at the
  cancelled prior IB entry), so the new IB import isn't skipped.

Schema-drift migration captures the partial unique index
sie_imports_company_id_file_hash_active_idx that already exists in
production (added out-of-band) and drops the now-superseded plain
sie_imports_company_id_file_hash_key constraint. Both statements are
idempotent — verified no-op against production.

Tests: new pg-real test covers the partial index admit-replaced
semantics, source_type='import'-only cancellation in replace_sie_import,
and the post-replace insert path.

Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>

* fix(arcim): address PR review — restore 5-min cleanup gate, fix test source_type

Greptile P2: `cleanupStaleImportRecords` was deleting `pending` rows
unconditionally, which could wipe a concurrent in-flight import in
another tab/session. Restored a 5-minute age gate (long enough for any
normal interactive import, short enough that legitimate retries after
a crash still succeed). Also dropped `mapped` from the cleanup — it is
defined in SIEImportStatus but no code path writes it, so including it
was both unnecessary and added the concurrent-session risk Greptile
flagged.

pg-real test: insertPostedEntry used `source_type='transaction'` which
is not a valid value per the journal_entries_source_type_check
constraint (migration 20260516060000). Switched to `'bank_transaction'`
— the actual source_type emitted when a user categorizes a bank
transaction in gnubok.

Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>

---------

Co-authored-by: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
2026-05-17 16:25:43 +02:00

213 lines
6.5 KiB
TypeScript

import { randomUUID } from 'node:crypto'
import { describe, expect, it } from 'vitest'
import { getPool } from '@/tests/pg/setup'
import { seedCompany } from '@/tests/pg/fixtures'
// Covers the Fortnox re-sync flow:
// 1. The partial unique index `sie_imports_company_id_file_hash_active_idx`
// (added in migration 20260517150000) blocks duplicate (company_id,
// file_hash) rows for active statuses but allows them once a prior row
// is marked 'replaced' or 'failed'.
// 2. The replace_sie_import RPC cancels journal entries with
// source_type='import' while leaving user-created entries
// (source_type='manual', 'bank_transaction', etc.) intact.
async function insertSIEImport(params: {
companyId: string
userId: string
fileHash: string
status: 'pending' | 'mapped' | 'completed' | 'failed' | 'replaced'
fiscalPeriodId?: string
openingBalanceEntryId?: string
fiscalYearStart?: string
fiscalYearEnd?: string
}): Promise<string> {
const id = randomUUID()
await getPool().query(
`INSERT INTO public.sie_imports
(id, user_id, company_id, filename, file_hash, sie_type,
fiscal_year_start, fiscal_year_end, accounts_count, transactions_count,
status, fiscal_period_id, opening_balance_entry_id, imported_at)
VALUES ($1, $2, $3, 'fortnox-export.se', $4, 4,
$5, $6, 0, 0,
$7, $8, $9, $10)`,
[
id,
params.userId,
params.companyId,
params.fileHash,
params.fiscalYearStart ?? '2026-01-01',
params.fiscalYearEnd ?? '2026-12-31',
params.status,
params.fiscalPeriodId ?? null,
params.openingBalanceEntryId ?? null,
params.status === 'completed' ? new Date().toISOString() : null,
],
)
return id
}
async function insertPostedEntry(params: {
userId: string
companyId: string
fiscalPeriodId: string
sourceType: 'import' | 'manual' | 'bank_transaction'
voucherNumber: number
entryDate?: string
}): Promise<string> {
const id = randomUUID()
await getPool().query(
`INSERT INTO public.journal_entries
(id, user_id, company_id, fiscal_period_id, voucher_number, voucher_series,
entry_date, description, source_type, status)
VALUES ($1, $2, $3, $4, $5, 'A', $6, 'Test entry', $7, 'posted')`,
[
id,
params.userId,
params.companyId,
params.fiscalPeriodId,
params.voucherNumber,
params.entryDate ?? '2026-06-01',
params.sourceType,
],
)
await getPool().query(
`INSERT INTO public.journal_entry_lines
(journal_entry_id, account_number, debit_amount, credit_amount)
VALUES ($1, '1930', 100, 0),
($1, '3001', 0, 100)`,
[id],
)
return id
}
describe('sie_imports: partial unique index + replace flow', () => {
it('blocks a second active row with the same (company_id, file_hash)', async () => {
const { companyId, userId, fiscalPeriodId } = await seedCompany()
const hash = `hash-${randomUUID()}`
await insertSIEImport({
companyId,
userId,
fileHash: hash,
status: 'completed',
fiscalPeriodId,
})
await expect(
insertSIEImport({
companyId,
userId,
fileHash: hash,
status: 'pending',
fiscalPeriodId,
}),
).rejects.toThrow(/sie_imports_company_id_file_hash_active_idx/)
})
it('allows a new pending row with the same hash once the prior row is replaced', async () => {
const { companyId, userId, fiscalPeriodId } = await seedCompany()
const hash = `hash-${randomUUID()}`
const priorId = await insertSIEImport({
companyId,
userId,
fileHash: hash,
status: 'completed',
fiscalPeriodId,
})
// Mark the prior row as replaced (simulating what replace_sie_import does)
await getPool().query(
`UPDATE public.sie_imports SET status = 'replaced', replaced_at = now() WHERE id = $1`,
[priorId],
)
// A new pending row with the same hash now succeeds
const newId = await insertSIEImport({
companyId,
userId,
fileHash: hash,
status: 'pending',
fiscalPeriodId,
})
expect(newId).toBeTruthy()
})
it('replace_sie_import cancels source_type=import entries and leaves manual/bank_transaction entries posted', async () => {
const { companyId, userId, fiscalPeriodId } = await seedCompany()
const obEntry = await insertPostedEntry({
userId,
companyId,
fiscalPeriodId,
sourceType: 'import',
voucherNumber: 1,
})
const importEntry1 = await insertPostedEntry({
userId,
companyId,
fiscalPeriodId,
sourceType: 'import',
voucherNumber: 2,
})
const importEntry2 = await insertPostedEntry({
userId,
companyId,
fiscalPeriodId,
sourceType: 'import',
voucherNumber: 3,
})
const manualEntry = await insertPostedEntry({
userId,
companyId,
fiscalPeriodId,
sourceType: 'manual',
voucherNumber: 4,
})
const txnEntry = await insertPostedEntry({
userId,
companyId,
fiscalPeriodId,
sourceType: 'bank_transaction',
voucherNumber: 5,
})
const importId = await insertSIEImport({
companyId,
userId,
fileHash: `hash-${randomUUID()}`,
status: 'completed',
fiscalPeriodId,
openingBalanceEntryId: obEntry,
})
const { rows } = await getPool().query<{ replace_sie_import: number }>(
`SELECT public.replace_sie_import($1::uuid, $2::uuid) AS replace_sie_import`,
[companyId, importId],
)
const cancelled = rows[0]!.replace_sie_import
// OB entry + 2 import entries = 3 cancelled. Manual & transaction stay posted.
expect(cancelled).toBe(3)
const statuses = await getPool().query<{ id: string; status: string }>(
`SELECT id, status FROM public.journal_entries WHERE id = ANY($1)`,
[[obEntry, importEntry1, importEntry2, manualEntry, txnEntry]],
)
const statusById = Object.fromEntries(statuses.rows.map(r => [r.id, r.status]))
expect(statusById[obEntry]).toBe('cancelled')
expect(statusById[importEntry1]).toBe('cancelled')
expect(statusById[importEntry2]).toBe('cancelled')
expect(statusById[manualEntry]).toBe('posted')
expect(statusById[txnEntry]).toBe('posted')
const importRow = await getPool().query<{ status: string; replaced_at: string | null }>(
`SELECT status, replaced_at FROM public.sie_imports WHERE id = $1`,
[importId],
)
expect(importRow.rows[0]!.status).toBe('replaced')
expect(importRow.rows[0]!.replaced_at).not.toBeNull()
})
})