* fix(transactions): make content-dedup bridge resilient to PSD2 description drift
Enable Banking re-syncs were re-importing every overlapping transaction as a
duplicate. Two changes in the June 1 deploy combined to defeat both dedup layers
at once: the external_id format changed (old rows' stored ids no longer match the
new scheme, so the exact-match layer misses) AND PSD2 descriptions were enriched
("TIC" -> "TIC BG 0000005786439 Bg-bet. via internet"), so the content-dedup
bridge — which compared a fixed 24-char description prefix for equality — also
missed. Result: a full re-import (observed: 53 of 54 "new" rows were dupes).
The external_id format has changed several times historically and 7,120 of 9,393
old rows have no reconstructable canonical id, so a backfill is not viable; the
content bridge is the mechanism meant to survive id-scheme changes. Harden it:
- Split the bridge into bucketing (date, öre) and matching (description), and
match by prefix-containment instead of fixed-prefix equality. PSD2 enrichment
is prefix-preserving, so the enriched re-import bridges its stored original,
while genuinely-distinct same-(date,amount) rows (distinct descriptions) are
kept apart. Consumed with counting semantics + longest-match, so N stored twins
dedup exactly N incoming.
- Replace contentDedupKey with contentBucketKey + descriptionsBridge; update the
live pipeline (ingest.ts) and the v1 dry-run preview to the same logic.
- Freeze the external_id format with a regression test + a header warning: any
future format change must ship a coordinated backfill.
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
* test/refactor: address PR review — sanitize fixtures, mirror preview counting
- Replace real customer names ("Carl Bennet AB", "Brorsan AB") and prod-derived
reference strings in tests with clearly fictional stand-ins (compliance A.8.33).
- v1 dry-run preview: use the same longest-match + counting/consume semantics as
the live pipeline so a batch of N copies against M booked twins previews M skips,
not N (greptile P2). Update stale pitfall docs: content dedup is now
date+amount+description (prefix-containment), and the preview is booked-only so
its skip count is a lower bound on the live skip count.
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
* fix(transactions): scope content-dedup bridge by cash account
The content-dedup bridge buckets by (date, öre) company-wide, with no account
scope — while bank reconciliation IS account-scoped (cash_account_id). For a
company with multiple bank accounts, a transaction on account A could therefore
deduplicate a genuinely-different transaction on account B that shares the same
date, amount, and a prefix-bridging description (round-number fees/transfers are
the realistic trigger), dropping a real row before it reaches reconciliation.
Layer 1 (external_id) is already account-safe because the account IBAN is
embedded in the id; only the fuzzy content bridge was account-blind.
Add an account guard: store cash_account_id alongside each bucket entry and only
bridge when BOTH the incoming batch and the stored entry have a known
cash_account_id that matches. A null on either side falls back to bridge-allowed,
so single-account companies and legacy (un-backfilled) rows are unchanged — and
CSV-vs-PSD2 dedup for the same account still works. Affects the 11 multi-account
companies; everyone else is behaviourally identical.
Note: external_id format heterogeneity (old entry_reference / old date+amount /
new öre+index schemes) is harmless downstream — nothing parses the id; it is an
opaque exact-match dedup key and a display string. Reconciliation, invoice/
supplier/payment matching, and reporting all key off real transaction columns,
never external_id.
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
* harden(transactions): blank description never wildcards a described row
PR review (OWASP V8.2.1 + Swedish compliance) flagged that descriptionsBridge
returned true whenever either side was empty, so a blank stored/incoming title
could wildcard-match any same-(date,öre) transaction and silently consume a real
one. Every live caller normalizes blanks to FALLBACK_DESCRIPTION upstream, so the
branch was unreachable in production — but make the function safe in isolation:
a blank now bridges only another blank (date+öre identity), never a described row.
No live behaviour change; removes the footgun.
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
---------
Co-authored-by: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Jakob Wennberg
331ae11867