feat(import): warn about already-imported rows in the bank-file wizard (#1567)
* fix(transactions): paginate the ingest dedup maps past the 1000-row cap
buildExistingTransactionMaps issued un-paginated selects for the booked and
unbooked dedup maps, so PostgREST silently truncated each at 1000 rows: a
re-import over a wide date range in an active company deduped against a
partial map and inserted everything past the cap as duplicates. Both queries
now go through fetchAllRows with a stable .order('id') for range paging.
Also exports the function and its types for the upcoming read-only duplicate
preview, which must share the exact stored-row universe execute-side ingest
dedups against.
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
* feat(import): add read-only duplicate preview endpoint for bank files
New POST /api/import/bank-file/check-duplicates (withRouteContext + Zod,
transactions capped at 20000) computes external_ids with the exact
generateExternalId(tx, format, index) derivation execute uses and runs
previewDuplicates: Layer-1 id collisions plus the Layer-2 text bridge with
counting semantics and the currency guard, against the same stored-row maps
ingest builds (buildExistingTransactionMaps). The result is advisory; execute
stays authoritative and mirrors/settlement-account guards are documented
preview/execute differences.
A dedicated endpoint because the generic_csv path re-parses client-side and
never re-hits /parse. Also removes the dead existing_transaction_count field
from the parse response (a raw date-range count consumed by nothing).
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
* feat(import): surface duplicate rows in the bank-file import wizard
Overlapping bank imports used to dedup silently: the wizard promised
'Importera N transaktioner', ingest skipped the twins, and the user saw fewer
rows than parsed with zero explanation. The wizard now calls check-duplicates
after a successful parse AND inside handleColumnMappingConfirm (the
generic_csv path never re-hits parse), and:
- BankFilePreviewStep: warning card in the AlertTriangle pattern ('{count}
rader finns redan', skipped automatically) plus a 'Finns redan' badge on
flagged rows in the 50-row table
- BankFileConfirmStep: repeats the summary card (generic path skips preview)
and the CTA counts 'Importera {parsed - duplicates} transaktioner'
- BankFileResultStep: renders result.duplicates when > 0, closing the loop
ingest.ts documents as unrendered
Execute semantics unchanged: all rows are sent, ingest skips; the preview is
advisory and never promises an exact final number. New strings in both
messages/sv.json and messages/en.json next to the import_psd2 anchors.
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
---------
Co-authored-by: Jakob Wennberg <311770904+jakobwennberg-oss@users.noreply.github.com>
Co-authored-by: Claude Fable 5 <noreply@anthropic.com>
This commit is contained in:
co-authored by
Claude Fable 5
Jakob Wennberg
parent
7ccaab7a08
commit
e1f13f870a
@@ -15,6 +15,7 @@ import { MAX_INVOICE_EMAIL_COPY_RECIPIENTS } from '@/lib/invoices/email-recipien
|
||||
import { INVOICE_POSTING_ACCOUNT_REGEX } from '@/lib/invoices/posting-account'
|
||||
import { PERSONAL_NUMBER_INPUT_RE } from '@/lib/customers/mask-personal-number'
|
||||
import type { AuditAction } from '@/types'
|
||||
import type { BankFileFormatId } from '@/lib/import/bank-file/types'
|
||||
|
||||
// ============================================================
|
||||
// Shared primitives
|
||||
@@ -3339,3 +3340,56 @@ export const MileageSalaryPushSchema = z
|
||||
.refine((p) => p.from.slice(0, 4) === p.to.slice(0, 4), {
|
||||
message: 'Milersättning bokförs per kalenderår: dela upp perioden per år',
|
||||
})
|
||||
|
||||
// ============================================================
|
||||
// Bank file import
|
||||
// ============================================================
|
||||
|
||||
/**
|
||||
* Known bank-file format ids, mirrored from `BankFileFormatId`
|
||||
* (lib/import/bank-file/types.ts). `satisfies` pins every member to the union
|
||||
* at compile time; a format id added to the union but not listed here only
|
||||
* degrades the ADVISORY duplicate preview (400), never the import itself.
|
||||
*/
|
||||
const BANK_FILE_FORMAT_IDS = [
|
||||
'nordea',
|
||||
'nordea_business',
|
||||
'seb',
|
||||
'swedbank',
|
||||
'handelsbanken',
|
||||
'lansforsakringar',
|
||||
'ica_banken',
|
||||
'skandia',
|
||||
'lunar',
|
||||
'northmill',
|
||||
'wise',
|
||||
'wise_statement',
|
||||
'generic_csv',
|
||||
'camt053',
|
||||
] as const satisfies readonly BankFileFormatId[]
|
||||
|
||||
/**
|
||||
* POST /api/import/bank-file/check-duplicates
|
||||
*
|
||||
* The rows are client-supplied (the generic_csv path never round-trips through
|
||||
* the parse route), so the array is hard-capped: the parse route caps files at
|
||||
* 10 MB, and 20000 rows mirrors that ceiling so an oversized payload cannot
|
||||
* drive the per-chunk dedup queries as a DoS vector. `raw_line` must pass
|
||||
* through untouched: camt.053/Wise external_ids are derived from it, and the
|
||||
* preview must compute byte-identical ids to execute.
|
||||
*/
|
||||
export const BankFileCheckDuplicatesSchema = z.object({
|
||||
transactions: z
|
||||
.array(
|
||||
z.object({
|
||||
date: isoDate,
|
||||
description: z.string().max(1000),
|
||||
amount: z.number().finite(),
|
||||
currency: z.string().max(8).optional().nullable(),
|
||||
raw_line: z.string().max(4000).optional().nullable(),
|
||||
})
|
||||
)
|
||||
.min(1)
|
||||
.max(20000),
|
||||
format: z.enum(BANK_FILE_FORMAT_IDS),
|
||||
})
|
||||
|
||||
Reference in New Issue
Block a user