fix(reconciliation): exclude ignored transactions from the bank total and bridge whitespace-drifted duplicate descriptions (#1705)

Bank reconciliation counted ignored transactions in bank_transaction_total
while excluding them from the unmatched count, so after the sanctioned
duplicate cleanup (ignore one twin) the differens showed the ignored sum
forever and is_reconciled was unreachable: observed live as a permanent
116 367 kr differens on a fully booked enskild firma (78 867 kr ignored
reconnect duplicates + 37 500 kr genuinely unbooked). The ignore toast
already promised 'försvinner från avstämningen'; now the engine keeps
that promise. Ignored rows are surfaced separately (count + sum) in the
status object, the UI card, and the v1 API, mirroring the IB pattern.

The duplicates themselves came from a PSD2 reconnect: the new connection
re-rendered identical transactions with drifted whitespace (CRLF vs
space, and a DROPPED space), so the prefix-containment content bridge
missed every twin. descriptionsBridge now strips all whitespace before
comparing: char-filtering preserves existing prefix relations, and the
compare stays confined to a (date, öre) bucket.

Co-authored-by: Jakob Wennberg <311770904+jakobwennberg-oss@users.noreply.github.com>
Co-authored-by: Claude Fable 5 <noreply@anthropic.com>
This commit is contained in:
Jakob Wennberg
2026-08-19 20:19:26 +02:00
committed by GitHub
co-authored by Jakob Wennberg Claude Fable 5
parent 6b9338f17e
commit 506d030bb1
10 changed files with 195 additions and 13 deletions
@@ -134,6 +134,39 @@ describe('descriptionsBridge', () => {
expect(descriptionsBridge(' Mataffär Solna ', 'mataffär solna')).toBe(true)
})
it('bridges whitespace-only rendering drift, including a DROPPED space (2026-08-18 reconnect incident)', () => {
// The same bank re-rendered the same transactions with different spacing
// after a PSD2 reconnect; every pair below is a real observed twin shape.
// CRLF vs space:
expect(
descriptionsBridge(
'2025DEC58 BRANDHEROES\r\nURSPRUNGLIGT BELOPP M M: EUR',
'2025DEC58 BRANDHEROES URSPRUNGLIGT BELOPP M M: EUR'
)
).toBe(true)
// Dropped space (collapse-only normalization would still miss this):
expect(
descriptionsBridge(
'005 BaBylissURSPRUNGLIGT BELOPP M M: SEK',
'005 BaByliss URSPRUNGLIGT BELOPP M M: SEK'
)
).toBe(true)
// Trailing padding + reference tail growth still bridges via prefix rule:
expect(
descriptionsBridge(
'BRANDHEROES APSFAK 008URSPRUNGLIGT BELOPP M M: EUR',
'BRANDHEROES APSFAK 008 URSPRUNGLIGT BELOPP M M: EUR 2162943703022614'
)
).toBe(true)
})
it('whitespace stripping does not bridge genuinely distinct references', () => {
expect(descriptionsBridge('REF-AAAA 1111', 'REF-BBBB 2222')).toBe(false)
// A whitespace-only description is still treated as blank (no wildcard).
expect(descriptionsBridge(' \r\n ', 'anything')).toBe(false)
expect(descriptionsBridge(' \r\n ', ' ')).toBe(true)
})
it('does NOT bridge genuinely distinct descriptions sharing a date+amount', () => {
// Distinct reference codes on same-day same-amount rows (e.g. verification
// micro-deposits) must NOT collapse: each is a real transaction.
+16 -3
View File
@@ -139,7 +139,9 @@ export function contentBucketKey(date: string, amount: number | string): string
* ("TIC" → "TIC BG 0000005786439 Bg-bet. via internet", "UTBETALNING" →
* "UTBETALNING Insättning"), so prefix-containment bridges the two where a
* fixed-length prefix *equality* check (the pre-June-2026 scheme) missed and
* re-imported. A blank description carries no signal, so it never bridges a
* re-imported. Whitespace is not part of the signal: both sides are compared
* with ALL whitespace stripped, because banks reformat spacing between API
* sessions (see the inline comment). A blank description carries no signal, so it never bridges a
* *described* row: otherwise an empty title would wildcard-match any
* same-(date,öre) transaction and could silently consume a real one; only two
* blanks bridge each other (date+öre identity). In practice every caller
@@ -158,8 +160,19 @@ export function descriptionsBridge(
a: string | null | undefined,
b: string | null | undefined
): boolean {
const x = (a ?? '').toLowerCase().trim()
const y = (b ?? '').toLowerCase().trim()
// Whitespace is stripped ENTIRELY (not collapsed) before comparing: the same
// bank renders the same transaction with drifting whitespace between API
// sessions ("BaBylissURSPRUNGLIGT" vs "BaByliss URSPRUNGLIGT", CRLF vs
// space), and a collapse-only normalization still misses the dropped-space
// form. Observed in the 2026-08-18 reconnect incident: five historical rows
// re-imported as twins whose descriptions differed only in whitespace, and
// every dedup layer missed them. Stripping is safe for the title-prefix
// bridge too: char-filtering is concatenation-homomorphic, so every existing
// prefix relation is preserved (it can only ADD bridges, never remove one).
// The compare stays confined to a (date, öre) bucket, so the widened match
// can only collapse same-day same-amount rows.
const x = (a ?? '').toLowerCase().replace(/\s+/g, '')
const y = (b ?? '').toLowerCase().replace(/\s+/g, '')
// A blank never wildcards a described row; only two blanks bridge each other.
if (x === '' || y === '') return x === y
return x.startsWith(y) || y.startsWith(x)