122bcbbcc12f5017a48d71dccbcbb66f877d75dd
2
Commits
| Author | SHA1 | Message | Date | |
|---|---|---|---|---|
 Jakob WennbergandClaude Opus 4.8
|
331ae11867 |
fix(transactions): stop Enable Banking re-sync from re-importing duplicates (#630)
* fix(transactions): make content-dedup bridge resilient to PSD2 description drift
Enable Banking re-syncs were re-importing every overlapping transaction as a
duplicate. Two changes in the June 1 deploy combined to defeat both dedup layers
at once: the external_id format changed (old rows' stored ids no longer match the
new scheme, so the exact-match layer misses) AND PSD2 descriptions were enriched
("TIC" -> "TIC BG 0000005786439 Bg-bet. via internet"), so the content-dedup
bridge — which compared a fixed 24-char description prefix for equality — also
missed. Result: a full re-import (observed: 53 of 54 "new" rows were dupes).
The external_id format has changed several times historically and 7,120 of 9,393
old rows have no reconstructable canonical id, so a backfill is not viable; the
content bridge is the mechanism meant to survive id-scheme changes. Harden it:
- Split the bridge into bucketing (date, öre) and matching (description), and
match by prefix-containment instead of fixed-prefix equality. PSD2 enrichment
is prefix-preserving, so the enriched re-import bridges its stored original,
while genuinely-distinct same-(date,amount) rows (distinct descriptions) are
kept apart. Consumed with counting semantics + longest-match, so N stored twins
dedup exactly N incoming.
- Replace contentDedupKey with contentBucketKey + descriptionsBridge; update the
live pipeline (ingest.ts) and the v1 dry-run preview to the same logic.
- Freeze the external_id format with a regression test + a header warning: any
future format change must ship a coordinated backfill.
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
* test/refactor: address PR review — sanitize fixtures, mirror preview counting
- Replace real customer names ("Carl Bennet AB", "Brorsan AB") and prod-derived
reference strings in tests with clearly fictional stand-ins (compliance A.8.33).
- v1 dry-run preview: use the same longest-match + counting/consume semantics as
the live pipeline so a batch of N copies against M booked twins previews M skips,
not N (greptile P2). Update stale pitfall docs: content dedup is now
date+amount+description (prefix-containment), and the preview is booked-only so
its skip count is a lower bound on the live skip count.
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
* fix(transactions): scope content-dedup bridge by cash account
The content-dedup bridge buckets by (date, öre) company-wide, with no account
scope — while bank reconciliation IS account-scoped (cash_account_id). For a
company with multiple bank accounts, a transaction on account A could therefore
deduplicate a genuinely-different transaction on account B that shares the same
date, amount, and a prefix-bridging description (round-number fees/transfers are
the realistic trigger), dropping a real row before it reaches reconciliation.
Layer 1 (external_id) is already account-safe because the account IBAN is
embedded in the id; only the fuzzy content bridge was account-blind.
Add an account guard: store cash_account_id alongside each bucket entry and only
bridge when BOTH the incoming batch and the stored entry have a known
cash_account_id that matches. A null on either side falls back to bridge-allowed,
so single-account companies and legacy (un-backfilled) rows are unchanged — and
CSV-vs-PSD2 dedup for the same account still works. Affects the 11 multi-account
companies; everyone else is behaviourally identical.
Note: external_id format heterogeneity (old entry_reference / old date+amount /
new öre+index schemes) is harmless downstream — nothing parses the id; it is an
opaque exact-match dedup key and a display string. Reconciliation, invoice/
supplier/payment matching, and reporting all key off real transaction columns,
never external_id.
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
* harden(transactions): blank description never wildcards a described row
PR review (OWASP V8.2.1 + Swedish compliance) flagged that descriptionsBridge
returned true whenever either side was empty, so a blank stored/incoming title
could wildcard-match any same-(date,öre) transaction and silently consume a real
one. Every live caller normalizes blanks to FALLBACK_DESCRIPTION upstream, so the
branch was unreachable in production — but make the function safe in isolation:
a blank now bridges only another blank (date+öre identity), never a described row.
No live behaviour change; removes the footgun.
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
---------
Co-authored-by: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
|
||
|
MattssonandClaude Opus 4.8
|
c6c86cded4 |
Mcp/template data feedback (#617)
* fix(booking-templates): scope template list to the active company GET /api/settings/booking-templates relied solely on the btl_select RLS policy, which is membership-wide (user_company_ids) and returns templates from every company the user belongs to. A user who owns multiple companies saw all their templates merged regardless of which company was active. Narrow the list in the API layer (mirroring counterparty-templates) to system + the active company + the active company's team. RLS stays the security backstop; this fixes the cross-company merge within a single user's own view (it was never a cross-tenant data leak). Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com> * fix(import): show proper message for duplicate bank file upload The bank file import page mis-parsed the structured error envelope ({ error: { code, message, details } }), so a BANK_FILE_DUPLICATE (409) fell through to the generic "Kunde inte läsa filen" fallback. The upload step also hardcoded that same string as the error heading, so duplicates were doubly misreported as parse failures. - Parse the structured envelope by error.code; surface error.message for all codes instead of rendering the error object. - Add a dedicated BANK_FILE_DUPLICATE message using the importedAt / importedCount details the route already returns. - Add an optional errorTitle prop to BankFileUploadStep (defaults to the previous text) and pass "Filen är redan importerad" for dupes. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com> * feat(tests): add comprehensive tests for recordateEntry, inbox-linking, and external-id handling - Implemented unit tests for recordateEntry in the bookkeeping module to validate various scenarios including date changes, non-posted entries, and fiscal period restrictions. - Created tests for inbox-linking status in pending operations to ensure correct handling of invoice inbox items and supplier invoices, addressing historical bugs related to status updates. - Added tests for external-id utilities to ensure consistent handling of monetary amounts and deduplication keys across different transaction sources. - Introduced new functions in external-id.ts for stable external ID generation and normalization of imported descriptions, enhancing transaction deduplication reliability. feat(migrations): add new database migrations for transaction handling - Created migration to exclude storno and correction vouchers from unmatched GL lines, ensuring accurate reconciliation. - Added a migration to preserve original bank transaction descriptions in a new immutable column, allowing for user edits while maintaining audit trails and deduplication integrity. * feat(migrations): add function to exclude storno/correction vouchers from unmatched GL lines * feat(transactions): enhance transaction handling with improved description normalization and preloaded original entries --------- Co-authored-by: Claude Opus 4.8 (1M context) <noreply@anthropic.com> |