feat(import): warn about already-imported rows in the bank-file wizard (#1567)

* fix(transactions): paginate the ingest dedup maps past the 1000-row cap

buildExistingTransactionMaps issued un-paginated selects for the booked and
unbooked dedup maps, so PostgREST silently truncated each at 1000 rows: a
re-import over a wide date range in an active company deduped against a
partial map and inserted everything past the cap as duplicates. Both queries
now go through fetchAllRows with a stable .order('id') for range paging.

Also exports the function and its types for the upcoming read-only duplicate
preview, which must share the exact stored-row universe execute-side ingest
dedups against.

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>

* feat(import): add read-only duplicate preview endpoint for bank files

New POST /api/import/bank-file/check-duplicates (withRouteContext + Zod,
transactions capped at 20000) computes external_ids with the exact
generateExternalId(tx, format, index) derivation execute uses and runs
previewDuplicates: Layer-1 id collisions plus the Layer-2 text bridge with
counting semantics and the currency guard, against the same stored-row maps
ingest builds (buildExistingTransactionMaps). The result is advisory; execute
stays authoritative and mirrors/settlement-account guards are documented
preview/execute differences.

A dedicated endpoint because the generic_csv path re-parses client-side and
never re-hits /parse. Also removes the dead existing_transaction_count field
from the parse response (a raw date-range count consumed by nothing).

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>

* feat(import): surface duplicate rows in the bank-file import wizard

Overlapping bank imports used to dedup silently: the wizard promised
'Importera N transaktioner', ingest skipped the twins, and the user saw fewer
rows than parsed with zero explanation. The wizard now calls check-duplicates
after a successful parse AND inside handleColumnMappingConfirm (the
generic_csv path never re-hits parse), and:

- BankFilePreviewStep: warning card in the AlertTriangle pattern ('{count}
  rader finns redan', skipped automatically) plus a 'Finns redan' badge on
  flagged rows in the 50-row table
- BankFileConfirmStep: repeats the summary card (generic path skips preview)
  and the CTA counts 'Importera {parsed - duplicates} transaktioner'
- BankFileResultStep: renders result.duplicates when > 0, closing the loop
  ingest.ts documents as unrendered

Execute semantics unchanged: all rows are sent, ingest skips; the preview is
advisory and never promises an exact final number. New strings in both
messages/sv.json and messages/en.json next to the import_psd2 anchors.

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>

---------

Co-authored-by: Jakob Wennberg <311770904+jakobwennberg-oss@users.noreply.github.com>
Co-authored-by: Claude Fable 5 <noreply@anthropic.com>
This commit is contained in:
Jakob Wennberg
2026-08-13 15:16:29 +02:00
committed by GitHub
co-authored by Claude Fable 5 Jakob Wennberg
parent 7ccaab7a08
commit e1f13f870a
15 changed files with 856 additions and 81 deletions
+54
View File
@@ -15,6 +15,7 @@ import { MAX_INVOICE_EMAIL_COPY_RECIPIENTS } from '@/lib/invoices/email-recipien
import { INVOICE_POSTING_ACCOUNT_REGEX } from '@/lib/invoices/posting-account'
import { PERSONAL_NUMBER_INPUT_RE } from '@/lib/customers/mask-personal-number'
import type { AuditAction } from '@/types'
import type { BankFileFormatId } from '@/lib/import/bank-file/types'
// ============================================================
// Shared primitives
@@ -3339,3 +3340,56 @@ export const MileageSalaryPushSchema = z
.refine((p) => p.from.slice(0, 4) === p.to.slice(0, 4), {
message: 'Milersättning bokförs per kalenderår: dela upp perioden per år',
})
// ============================================================
// Bank file import
// ============================================================
/**
* Known bank-file format ids, mirrored from `BankFileFormatId`
* (lib/import/bank-file/types.ts). `satisfies` pins every member to the union
* at compile time; a format id added to the union but not listed here only
* degrades the ADVISORY duplicate preview (400), never the import itself.
*/
const BANK_FILE_FORMAT_IDS = [
'nordea',
'nordea_business',
'seb',
'swedbank',
'handelsbanken',
'lansforsakringar',
'ica_banken',
'skandia',
'lunar',
'northmill',
'wise',
'wise_statement',
'generic_csv',
'camt053',
] as const satisfies readonly BankFileFormatId[]
/**
* POST /api/import/bank-file/check-duplicates
*
* The rows are client-supplied (the generic_csv path never round-trips through
* the parse route), so the array is hard-capped: the parse route caps files at
* 10 MB, and 20000 rows mirrors that ceiling so an oversized payload cannot
* drive the per-chunk dedup queries as a DoS vector. `raw_line` must pass
* through untouched: camt.053/Wise external_ids are derived from it, and the
* preview must compute byte-identical ids to execute.
*/
export const BankFileCheckDuplicatesSchema = z.object({
transactions: z
.array(
z.object({
date: isoDate,
description: z.string().max(1000),
amount: z.number().finite(),
currency: z.string().max(8).optional().nullable(),
raw_line: z.string().max(4000).optional().nullable(),
})
)
.min(1)
.max(20000),
format: z.enum(BANK_FILE_FORMAT_IDS),
})