fix(reconciliation): exclude ignored transactions from the bank total and bridge whitespace-drifted duplicate descriptions (#1705)

Bank reconciliation counted ignored transactions in bank_transaction_total
while excluding them from the unmatched count, so after the sanctioned
duplicate cleanup (ignore one twin) the differens showed the ignored sum
forever and is_reconciled was unreachable: observed live as a permanent
116 367 kr differens on a fully booked enskild firma (78 867 kr ignored
reconnect duplicates + 37 500 kr genuinely unbooked). The ignore toast
already promised 'försvinner från avstämningen'; now the engine keeps
that promise. Ignored rows are surfaced separately (count + sum) in the
status object, the UI card, and the v1 API, mirroring the IB pattern.

The duplicates themselves came from a PSD2 reconnect: the new connection
re-rendered identical transactions with drifted whitespace (CRLF vs
space, and a DROPPED space), so the prefix-containment content bridge
missed every twin. descriptionsBridge now strips all whitespace before
comparing: char-filtering preserves existing prefix relations, and the
compare stays confined to a (date, öre) bucket.

Co-authored-by: Jakob Wennberg <311770904+jakobwennberg-oss@users.noreply.github.com>
Co-authored-by: Claude Fable 5 <noreply@anthropic.com>

lib/reconciliation/__tests__/bank-reconciliation.test.ts

@@ -1336,6 +1336,86 @@ describe('getReconciliationStatus', () => {
     expect(status.gl_1930_period_movement).toBe(200)
   })
 
+  it('excludes ignored transactions from the bank total and difference, surfacing them separately', async () => {
+    // The 2026-08-18 duplicate-cleanup shape: a PSD2 reconnect re-imported
+    // history, the user booked one twin and IGNORED the other. The ignored
+    // duplicate never gets a ledger counterpart, so counting it in the bank
+    // total manufactured a permanent difference no amount of booking could
+    // clear (observed live: a 78 867 kr differens over a fully booked account).
+    const { supabase, enqueue } = createQueueMockSupabase()
+
+    // 1) transactions: booked original + its ignored duplicate + one real
+    //    unbooked deposit.
+    enqueue({
+      data: [
+        { amount: 8730, journal_entry_id: 'je-1', reconciliation_method: 'manual', is_ignored: false },
+        { amount: 8730, journal_entry_id: null, reconciliation_method: null, is_ignored: true },
+        { amount: 500, journal_entry_id: null, reconciliation_method: null, is_ignored: false },
+      ],
+    })
+    // 2) GL: only the booked original is on 1930.
+    enqueue({ data: [{ id: 'je-1', status: 'posted', source_type: 'bank_import' }] })
+    enqueue({ data: [{ debit_amount: 8730, credit_amount: 0, journal_entry_id: 'je-1' }] })
+    // 3) RPC: no unlinked lines
+    enqueue({ data: [] })
+
+    const status = await getReconciliationStatus(supabase as never, 'company-1')
+
+    expect(status.bank_transaction_total).toBe(9230) // 8730 + 500, NOT the ignored twin
+    expect(status.ignored_transaction_count).toBe(1)
+    expect(status.ignored_transaction_total).toBe(8730)
+    expect(status.gl_1930_period_movement).toBe(8730)
+    expect(status.difference).toBe(500) // only the real unbooked deposit remains
+    expect(status.matched_count).toBe(1)
+    expect(status.unmatched_transaction_count).toBe(1)
+    expect(status.is_reconciled).toBe(false)
+  })
+
+  it('reports is_reconciled=true once everything non-ignored is booked, despite ignored rows', async () => {
+    const { supabase, enqueue } = createQueueMockSupabase()
+
+    enqueue({
+      data: [
+        { amount: 1000, journal_entry_id: 'je-1', reconciliation_method: 'auto_exact', is_ignored: false },
+        { amount: 1000, journal_entry_id: null, reconciliation_method: null, is_ignored: true },
+      ],
+    })
+    enqueue({ data: [{ id: 'je-1', status: 'posted', source_type: 'bank_import' }] })
+    enqueue({ data: [{ debit_amount: 1000, credit_amount: 0, journal_entry_id: 'je-1' }] })
+    enqueue({ data: [] })
+
+    const status = await getReconciliationStatus(supabase as never, 'company-1')
+
+    expect(status.bank_transaction_total).toBe(1000)
+    expect(status.ignored_transaction_total).toBe(1000)
+    expect(status.difference).toBe(0)
+    expect(status.unmatched_transaction_count).toBe(0)
+    // Before the fix this was unreachable for any company with an ignored row:
+    // the ignored amount sat in the bank total forever.
+    expect(status.is_reconciled).toBe(true)
+  })
+
+  it('does not count an ignored row that somehow retains a journal_entry_id as matched', async () => {
+    const { supabase, enqueue } = createQueueMockSupabase()
+
+    enqueue({
+      data: [
+        { amount: 700, journal_entry_id: 'je-x', reconciliation_method: 'manual', is_ignored: true },
+      ],
+    })
+    enqueue({ data: [] })
+    enqueue({ data: [] })
+    enqueue({ data: [] })
+
+    const status = await getReconciliationStatus(supabase as never, 'company-1')
+
+    // matched + unmatched partition the reconcilable (non-ignored) set.
+    expect(status.matched_count).toBe(0)
+    expect(status.unmatched_transaction_count).toBe(0)
+    expect(status.ignored_transaction_count).toBe(1)
+    expect(status.bank_transaction_total).toBe(0)
+  })
+
   it('reconciles a corrected bank receipt and keeps gl_1930_balance equal to the balance sheet', async () => {
     // A +25000 deposit was booked to the wrong counter-account, then corrected
     // via the storno flow: the original flips to 'reversed', a storno (credit

lib/reconciliation/bank-reconciliation.ts

@@ -100,7 +100,20 @@ export interface ReconciliationStatus {
    * this is a no-op and every figure below is exactly what it always was.
    */
   currency: string
+  /**
+   * Sum of the window's bank-feed transactions EXCLUDING ignored rows: the
+   * bank side of the reconciliation. Ignored rows (feed duplicates from a
+   * PSD2 reconnect, non-business noise) never get a ledger counterpart, so
+   * counting them here manufactured a permanent unfixable difference; they are
+   * surfaced separately below instead, mirroring how the opening balance is
+   * excluded-but-shown.
+   */
   bank_transaction_total: number
+  /** Sum of ignored bank transactions in the window. NOT part of
+   *  bank_transaction_total or difference; informational, like the IB. */
+  ignored_transaction_total: number
+  /** Number of ignored bank transactions in the window. */
+  ignored_transaction_count: number
   /**
    * The real ledger balance on the bank account, incl. IB: computed from the
    * SAME `['posted','reversed']` lines the trial balance and balance sheet sum.
@@ -655,10 +668,11 @@ export async function getReconciliationStatus(
   includeUnassigned: boolean = true,
 ): Promise<ReconciliationStatus> {
   // Get all transactions in range, scoped to the selected cash account. Ignored
-  // rows are pulled too so the totals card still reflects what the bank
-  // actually moved, but they're excluded from the "unmatched" count below: the
-  // user has explicitly said they don't want them surfacing as something to
-  // reconcile. Scoping by cash account (not just currency) is what stops a
+  // rows are pulled too, but only to be COUNTED AND SUMMED separately: they are
+  // excluded from the bank total, the difference and the matched/unmatched
+  // counts below, because the user has explicitly said they are not something
+  // to reconcile (duplicates, non-business noise). Scoping by cash account
+  // (not just currency) is what stops a
   // second same-currency account from inflating bankTotal here.
   // Paginated (fetchAllRows): PostgREST silently caps un-ranged selects at 1000
   // rows, which would undercount bank_transaction_total for a busy company and
@@ -799,7 +813,7 @@ export async function getReconciliationStatus(
     onOrAfterFloor((tx as { date?: string | null }).date),
   )
 
-  // Bank side: every feed transaction in the (floored) window, full stop. We
+  // Bank side: every NON-IGNORED feed transaction in the (floored) window. We
   // deliberately do NOT special-case rows linked to a reversed entry any more.
   // Because the GL side now counts the reversed original, its storno AND the
   // correction together (just like the balance sheet), a corrected bank line nets
@@ -809,7 +823,20 @@ export async function getReconciliationStatus(
   // transactions.amount is denominated in transactions.currency, and
   // scopeTransactionsToAccount pinned that to `currency`, so this total is
   // already in the account's own currency: the unit lineAmount() resolves to.
-  const bankTotal = countedTx.reduce((sum, tx) => sum + (Number(tx.amount) || 0), 0)
+  //
+  // Ignored rows are EXCLUDED from the total, exactly as they are excluded from
+  // the unmatched count: ignoring is the sanctioned handling for feed
+  // duplicates (a reconnect re-importing history) and non-business noise, and
+  // by definition an ignored row will never get a ledger counterpart. Counting
+  // it in the bank total manufactured a permanent difference the user could
+  // never book away: after a correct duplicate cleanup the card showed a
+  // six-figure differens over a fully booked account, and is_reconciled was
+  // unreachable forever. They are surfaced separately (count + sum) instead,
+  // the same pattern as the opening balance, so nothing is silently hidden.
+  const reconcilableTx = countedTx.filter((tx) => tx.is_ignored !== true)
+  const ignoredTx = countedTx.filter((tx) => tx.is_ignored === true)
+  const bankTotal = reconcilableTx.reduce((sum, tx) => sum + (Number(tx.amount) || 0), 0)
+  const ignoredTotal = ignoredTx.reduce((sum, tx) => sum + (Number(tx.amount) || 0), 0)
 
   // Ledger lines the account's currency cannot express: a foreign account whose
   // lines hold only SEK figures with no per-row rate (SIE imports, pre-FX
@@ -844,10 +871,13 @@ export async function getReconciliationStatus(
   // a bank-feed counterpart, so it stays in.
   const glPeriodMovement = glBalance - glOpeningBalance
 
-  const matchedCount = countedTx.filter((tx) => tx.journal_entry_id !== null).length
+  // Matched/unmatched partition the RECONCILABLE (non-ignored) set, so
+  // matched_count + unmatched_transaction_count always equals the number of
+  // rows behind bank_transaction_total.
+  const matchedCount = reconcilableTx.filter((tx) => tx.journal_entry_id !== null).length
 
-  const unmatchedTransactionCount = countedTx.filter(
-    (tx) => tx.journal_entry_id === null && tx.is_ignored !== true
+  const unmatchedTransactionCount = reconcilableTx.filter(
+    (tx) => tx.journal_entry_id === null
   ).length
 
   // Unmatched GL lines count (RPC excludes opening_balance, storno and correction
@@ -866,6 +896,8 @@ export async function getReconciliationStatus(
   return {
     currency,
     bank_transaction_total: Math.round(bankTotal * 100) / 100,
+    ignored_transaction_total: Math.round(ignoredTotal * 100) / 100,
+    ignored_transaction_count: ignoredTx.length,
     gl_1930_balance: Math.round(glBalance * 100) / 100,
     gl_1930_period_movement: Math.round(glPeriodMovement * 100) / 100,
     gl_1930_opening_balance: Math.round(glOpeningBalance * 100) / 100,

lib/transactions/__tests__/external-id.test.ts

@@ -134,6 +134,39 @@ describe('descriptionsBridge', () => {
     expect(descriptionsBridge('  Mataffär Solna  ', 'mataffär solna')).toBe(true)
   })
 
+  it('bridges whitespace-only rendering drift, including a DROPPED space (2026-08-18 reconnect incident)', () => {
+    // The same bank re-rendered the same transactions with different spacing
+    // after a PSD2 reconnect; every pair below is a real observed twin shape.
+    // CRLF vs space:
+    expect(
+      descriptionsBridge(
+        '2025DEC58 BRANDHEROES\r\nURSPRUNGLIGT BELOPP M M: EUR',
+        '2025DEC58 BRANDHEROES URSPRUNGLIGT BELOPP M M: EUR'
+      )
+    ).toBe(true)
+    // Dropped space (collapse-only normalization would still miss this):
+    expect(
+      descriptionsBridge(
+        '005 BaBylissURSPRUNGLIGT BELOPP M M: SEK',
+        '005 BaByliss URSPRUNGLIGT BELOPP M M: SEK'
+      )
+    ).toBe(true)
+    // Trailing padding + reference tail growth still bridges via prefix rule:
+    expect(
+      descriptionsBridge(
+        'BRANDHEROES APSFAK 008URSPRUNGLIGT BELOPP M M: EUR',
+        'BRANDHEROES APSFAK 008 URSPRUNGLIGT BELOPP M M: EUR                 2162943703022614'
+      )
+    ).toBe(true)
+  })
+
+  it('whitespace stripping does not bridge genuinely distinct references', () => {
+    expect(descriptionsBridge('REF-AAAA 1111', 'REF-BBBB 2222')).toBe(false)
+    // A whitespace-only description is still treated as blank (no wildcard).
+    expect(descriptionsBridge(' \r\n ', 'anything')).toBe(false)
+    expect(descriptionsBridge(' \r\n ', '  ')).toBe(true)
+  })
+
   it('does NOT bridge genuinely distinct descriptions sharing a date+amount', () => {
     // Distinct reference codes on same-day same-amount rows (e.g. verification
     // micro-deposits) must NOT collapse: each is a real transaction.

lib/transactions/external-id.ts

@@ -139,7 +139,9 @@ export function contentBucketKey(date: string, amount: number | string): string
  * ("TIC" → "TIC  BG 0000005786439 Bg-bet. via internet", "UTBETALNING" →
  * "UTBETALNING Insättning"), so prefix-containment bridges the two where a
  * fixed-length prefix *equality* check (the pre-June-2026 scheme) missed and
- * re-imported. A blank description carries no signal, so it never bridges a
+ * re-imported. Whitespace is not part of the signal: both sides are compared
+ * with ALL whitespace stripped, because banks reformat spacing between API
+ * sessions (see the inline comment). A blank description carries no signal, so it never bridges a
  * *described* row: otherwise an empty title would wildcard-match any
  * same-(date,öre) transaction and could silently consume a real one; only two
  * blanks bridge each other (date+öre identity). In practice every caller
@@ -158,8 +160,19 @@ export function descriptionsBridge(
   a: string | null | undefined,
   b: string | null | undefined
 ): boolean {
-  const x = (a ?? '').toLowerCase().trim()
-  const y = (b ?? '').toLowerCase().trim()
+  // Whitespace is stripped ENTIRELY (not collapsed) before comparing: the same
+  // bank renders the same transaction with drifting whitespace between API
+  // sessions ("BaBylissURSPRUNGLIGT" vs "BaByliss URSPRUNGLIGT", CRLF vs
+  // space), and a collapse-only normalization still misses the dropped-space
+  // form. Observed in the 2026-08-18 reconnect incident: five historical rows
+  // re-imported as twins whose descriptions differed only in whitespace, and
+  // every dedup layer missed them. Stripping is safe for the title-prefix
+  // bridge too: char-filtering is concatenation-homomorphic, so every existing
+  // prefix relation is preserved (it can only ADD bridges, never remove one).
+  // The compare stays confined to a (date, öre) bucket, so the widened match
+  // can only collapse same-day same-amount rows.
+  const x = (a ?? '').toLowerCase().replace(/\s+/g, '')
+  const y = (b ?? '').toLowerCase().replace(/\s+/g, '')
   // A blank never wildcards a described row; only two blanks bridge each other.
   if (x === '' || y === '') return x === y
   return x.startsWith(y) || y.startsWith(x)