70845edf69
* feat(transactions): structured transaction_method instead of channel-in-the-name
Swedish bank feeds embed the payment channel in the description string
("Vercel Jul Överföring via internet", "ANTHROPIC* ... Kortköp/uttag"):
the PSD2 remittance array is joined into one string and the ISO 20022
type codes were dropped at insert. This promotes the channel to data:
- transactions.transaction_method (text + CHECK closed vocabulary: card,
transfer, bankgiro, plusgiro, swish, autogiro, e_invoice, international,
deposit, withdrawal, salary, fee, interest, adjustment) plus verbatim
bank_transaction_code / proprietary_bank_transaction_code evidence
columns (data_quality_master Appendix B "Layer-A capture").
- classifyTransactionMethod() in lib/transactions/transaction-method.ts:
explicit source method (Stripe txn.type) > trailing Swedish channel
phrase > ISO 20022 family/subfamily > proprietary-code keywords > MCC.
It also splits the clean display title off the description.
- Ingest stores the clean title as description and the full bank string
as original_description; dedup is untouched (external_id is date+öre,
the content bridge reads original_description and is prefix-based, and
a trailing strip leaves a prefix). Enable Banking passes the codes
through; the Stripe feed sets methods from its balance-txn types.
- Backfill migration classifies existing rows from the description text
(+ MCC and Stripe prefixes) and strips unedited titles; user-edited
titles are never rewritten.
- mapping-engine also matches original_description so user rules written
against the full bank text keep firing.
- UI: the inbox row shows the clean name; clicking it now folds out
"Betalsätt: Kortköp" etc. (sv/en), making every classified row
expandable.
A card purchase implies a physical receipt, a Bankgiro/e-invoice payment
implies a supplier invoice: downstream automations can now branch on the
rail instead of regexing display strings.
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
* fix(transactions): anchor counterparty-template identity on original_description
Audit follow-up to the phrase-strip change: counterparty template lookup
AND learning derived their key from merchant_name || description. With
the working title now stripped ("SPOTIFY AB Kortköp" -> "SPOTIFY AB"),
templates learned from the full bank string would only re-match via the
occurrence-gated single-token tier, and single-token counterparties with
fewer than 3 bookings would silently stop matching.
Both sides now read merchant_name || original_description || description:
the immutable bank original is identical across eras (and across user
renames), so every stored key and alias keeps matching exactly. Same
anchoring rationale as buildMerchantHistory in category-suggestions.
Existing tests that relied on the fixture's default original_description
now state it explicitly; two new regression tests pin the era stability
(lookup via alias on the full string, learning key derivation).
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
* fix(transactions): review follow-ups on method classification
- methodFromCodes: two-pass subfamily-then-family scan so a SALA/XBCT
refinement on the proprietary code beats a bare family match on the
ISO code, matching the documented precedence; pinned by a test.
- mapping-engine: regression tests for merchant/description patterns
that only match original_description, including the invalid-regex
substring fallback and the no-match default.
- Stripe: regression test for the SDK-unmodeled 'tax' balance-txn type
mapping to 'fee'.
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
* fix(transactions): scope method classification to feed rows + adjective guard
Multi-bank risk hardening before the backfill ships:
- Feed-row scope: classification and title stripping now require a real
import feed (import_source present, not manual/mcp), both at the
ingest boundary (USER_CREATED_IMPORT_SOURCES, now exported) and in
every backfill statement. User-authored titles like "Egen insättning"
on manual/MCP rows are never classified and never rewritten.
- Adjective guard (TS + SQL): a strip that would leave the title ending
in a possessive/scope adjective (egen/eget/privat/intern/extern ...)
is skipped, so "Egen insättning" stays whole even on bank-feed rows;
the method column still classifies (deposit).
- Unknown bank phrasings remain untouched by construction: an unmatched
phrase means no method and no rewrite, so the worst case for any bank
whose vocabulary we have not seen is the status quo.
Pinned by new unit + pg-real cases (user-created exclusion for
NULL/manual/mcp, adjective guard, feed defaults in the pg fixture).
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
* chore(migrations): re-timestamp transaction_method migrations after rebase
Main gained migrations dated 20260729-20260730 (already applied to prod)
while this branch carried 20260728 versions, which would have applied
out-of-order on merge. The files have never reached prod, so renaming to
current timestamps is safe and removes any dependence on the integration's
out-of-order handling. All code/doc references updated; the pg test reads
the backfill by its new filename.
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
* chore(migrations): bump transaction_method versions past prod's max
Main's newest applied migration is 20260730090000 (future-leaning
timestamp), so the previous 202607300731xx rename still sorted before
prod's tail and risked a silent skip on merge-time apply. Versions are
now 20260730100000/20260730100100, strictly after everything applied to
prod. References updated; full migration stream replays clean.
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
* fix(transactions): final review round: keyboard guard + bank_connection_id feed marker
- TransactionInboxCard: row-level Enter/Space handling now ignores events
bubbling from nested controls, so keyboard activation of Bokför / the
overflow menu is no longer cancelled by the (now much more common)
expandable row.
- Feed predicate parity with isImportedTransaction(): a live
bank_connection_id marks a feed row even when import_source is unset
(the oldest PSD2 rows predate that column), in both the ingest
classifier and every backfill statement: those legacy rows now get
classified instead of being skipped as user-created.
- pg fixture typing uses the TransactionMethod union.
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
* chore(migrations): re-timestamp transaction_method migrations past prod's 20260807 tail
Prod max applied is 20260807170000 (verified by name via list_migrations);
the 20260730-stamped pair would sort before it. References in code,
tests, and DECISIONS.md updated to the new versions.
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
* fix(migrations): enforce, not assume, original_description preservation in the title strip
The strip UPDATE now fills a NULL original_description from the
pre-strip description in the same statement. Prod has zero such rows
(0/25,566 feed-scope rows, verified read-only), and 20260605120000's
backfill plus ingest make the NULL case unreachable on any DB that
replayed history, but the migration should not depend on that history
to avoid losing the only copy of a bank string.
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
* docs: record the compliance-review triage of the backfill's booked-row title strip
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
---------
Co-authored-by: Jakob Wennberg <jakob.wennberg@gmail.com>
Co-authored-by: Claude Fable 5 <noreply@anthropic.com>
Co-authored-by: Jakob Wennberg <311770904+jakobwennberg-oss@users.noreply.github.com>
249 lines
9.5 KiB
TypeScript
249 lines
9.5 KiB
TypeScript
/**
|
|
* Classify HOW a bank/feed transaction moved (its payment rail) into the
|
|
* closed `TransactionMethod` vocabulary, and derive the clean display title.
|
|
*
|
|
* Swedish bank feeds embed the channel in the description text itself: the
|
|
* PSD2 remittance array arrives as ["Vercel Jul", "Överföring via internet"]
|
|
* and is joined into one string at conversion, so "Kortköp/uttag",
|
|
* "Bg-bet. via internet", "Europabetalning" etc. ride along as a trailing
|
|
* phrase. This module is the single place that vocabulary lives: it both
|
|
* classifies the phrase into a structured method AND strips it off the
|
|
* working title, so the list shows "Vercel Jul" and the method becomes a
|
|
* queryable field (a card purchase implies a physical receipt exists; a
|
|
* Bankgiro/e-invoice payment implies a supplier invoice).
|
|
*
|
|
* Precedence (most trustworthy first):
|
|
* 1. An explicit method from the source (the Stripe feed knows its own
|
|
* balance-transaction types).
|
|
* 2. The trailing Swedish channel phrase in the description: bank-authored
|
|
* and more specific than the ISO family (the ISO code for a Bankgiro
|
|
* payment is just "issued credit transfer").
|
|
* 3. ISO 20022 bank_transaction_code domain/family (+ subfamily override).
|
|
* 4. Keyword scan over the raw (often proprietary) code strings.
|
|
* 5. MCC presence: only card transactions carry an MCC (6011 = ATM).
|
|
*
|
|
* Title stripping is independent of which source won: whenever a trailing
|
|
* phrase is found it is removed, but the title is never emptied (a
|
|
* description that IS the phrase, e.g. a bare "Insättning", is kept).
|
|
* Stripping a trailing phrase leaves a PREFIX of the original string, which
|
|
* the content-dedup bridge (descriptionsBridge, prefix containment) is built
|
|
* to survive; the full bank string is preserved in original_description.
|
|
*
|
|
* Pure and side-effect free. The SQL backfill in migration 20260808090100
|
|
* duplicates the phrase vocabulary as a one-shot snapshot; it does NOT need
|
|
* to stay in sync (post-migration rows are classified here at ingest).
|
|
*/
|
|
|
|
import type { TransactionMethod } from '@/types'
|
|
|
|
export interface TransactionMethodInput {
|
|
/** Normalized full source description (the ingest boundary's original). */
|
|
description: string
|
|
/** ISO 20022 bank transaction code, e.g. "PMNT-CCRD-POSD" or "PMNT/ICDT". */
|
|
bankTransactionCode?: string | null
|
|
/** ASPSP-proprietary code (free-form, varies per bank). */
|
|
proprietaryBankTransactionCode?: string | null
|
|
/** Merchant category code: present only on card-rail transactions. */
|
|
mccCode?: number | null
|
|
/** Method the source already knows structurally (e.g. Stripe txn.type). */
|
|
explicitMethod?: TransactionMethod | null
|
|
}
|
|
|
|
export interface ClassifiedTransactionMethod {
|
|
method: TransactionMethod | null
|
|
/**
|
|
* Description with the trailing channel phrase stripped: what the user sees
|
|
* as the row title. Equals `description` when no phrase matched or when
|
|
* stripping would empty the title.
|
|
*/
|
|
displayTitle: string
|
|
}
|
|
|
|
function escapeRegExp(s: string): string {
|
|
return s.replace(/[.*+?^${}()|[\]\\]/g, '\\$&')
|
|
}
|
|
|
|
/**
|
|
* Trailing channel phrases, checked in order: first match wins, so longer /
|
|
* more specific variants must precede their substrings ("överföring via
|
|
* internet" before "överföring", "kortköp/uttag" before "kortköp"). The bare
|
|
* "lön" sits last: it is the riskiest token and must never shadow a more
|
|
* specific phrase.
|
|
*/
|
|
const TRAILING_PHRASES: ReadonlyArray<readonly [string, TransactionMethod]> = [
|
|
['kortköp/uttag', 'card'],
|
|
['kortköp', 'card'],
|
|
['kortbetalning', 'card'],
|
|
['webbköp', 'card'],
|
|
['bg-bet. via internet', 'bankgiro'],
|
|
['bg-bet via internet', 'bankgiro'],
|
|
['bg-betalning', 'bankgiro'],
|
|
['bg betalning', 'bankgiro'],
|
|
['bgmax', 'bankgiro'],
|
|
['bg-inb', 'bankgiro'],
|
|
['bankgiro', 'bankgiro'],
|
|
['bg-bet.', 'bankgiro'],
|
|
['bg-bet', 'bankgiro'],
|
|
['pg-betalning', 'plusgiro'],
|
|
['pg betalning', 'plusgiro'],
|
|
['plusgiro', 'plusgiro'],
|
|
['europabetalning', 'international'],
|
|
['utlandsbetalning', 'international'],
|
|
['löneinsättning', 'salary'],
|
|
['lönebetalning', 'salary'],
|
|
['löneutbetalning', 'salary'],
|
|
['e-faktura', 'e_invoice'],
|
|
['efaktura', 'e_invoice'],
|
|
['swish-betalning', 'swish'],
|
|
['swish betalning', 'swish'],
|
|
['swish', 'swish'],
|
|
['autogirobetalning', 'autogiro'],
|
|
['autogiro', 'autogiro'],
|
|
['pris betalning', 'fee'],
|
|
['prisbetalning', 'fee'],
|
|
['avgift', 'fee'],
|
|
['insättningsränta', 'interest'],
|
|
['ränta', 'interest'],
|
|
['kontantinsättning', 'deposit'],
|
|
['insättning', 'deposit'],
|
|
['bankomatuttag', 'withdrawal'],
|
|
['kontantuttag', 'withdrawal'],
|
|
['uttag', 'withdrawal'],
|
|
['överföring via internet', 'transfer'],
|
|
['överföring via mobil', 'transfer'],
|
|
['överföring via app', 'transfer'],
|
|
['överföring inom banken', 'transfer'],
|
|
['överföring inom bank', 'transfer'],
|
|
['överföring mellan konton', 'transfer'],
|
|
['direktöverföring', 'transfer'],
|
|
['direktbetalning', 'transfer'],
|
|
['internetbetalning', 'transfer'],
|
|
['mobilbetalning', 'transfer'],
|
|
['överföring', 'transfer'],
|
|
['lön', 'salary'],
|
|
]
|
|
|
|
const TRAILING_PHRASE_RULES = TRAILING_PHRASES.map(([phrase, method]) => ({
|
|
method,
|
|
// Anchored at the end, preceded by start-of-string or whitespace so a
|
|
// phrase inside a word never matches ("Löneinsättning" is salary, not
|
|
// deposit; "Bankavgift" is untouched by the bare "avgift" rule).
|
|
regex: new RegExp(`(?:^|\\s)${escapeRegExp(phrase)}\\s*$`, 'i'),
|
|
}))
|
|
|
|
/** Leading patterns: classification signal only, never stripped. */
|
|
const LEADING_RULES: ReadonlyArray<readonly [RegExp, TransactionMethod]> = [
|
|
[/^swish (?:till|från)(?:\s|$)/i, 'swish'],
|
|
]
|
|
|
|
/**
|
|
* Possessive/scope adjectives whose meaning depends on the noun after them:
|
|
* "Egen insättning" must not become "Egen" (the phrase IS the meaning there).
|
|
* When the stripped title would END in one of these, the strip is skipped;
|
|
* the method classification still applies. Mirrored by the adjective guard in
|
|
* the 20260808090100 backfill.
|
|
*/
|
|
const ADJECTIVE_GUARD = new Set([
|
|
'egen', 'eget', 'egna', 'privat', 'privata', 'intern', 'interna', 'extern', 'externa',
|
|
])
|
|
|
|
/** ISO 20022 External Bank Transaction Codes, keyed by DOMAIN/FAMILY. */
|
|
const ISO_FAMILY_METHODS: Record<string, TransactionMethod> = {
|
|
'PMNT/RCDT': 'transfer', // ReceivedCreditTransfers
|
|
'PMNT/ICDT': 'transfer', // IssuedCreditTransfers
|
|
'PMNT/CCRD': 'card', // CustomerCardTransactions
|
|
'PMNT/MCRD': 'card', // MerchantCardTransactions
|
|
'PMNT/RDDT': 'autogiro', // ReceivedDirectDebits
|
|
'PMNT/IDDT': 'autogiro', // IssuedDirectDebits
|
|
'PMNT/CWDL': 'withdrawal', // CashWithdrawal
|
|
'PMNT/CAJT': 'adjustment', // CashAdjustments
|
|
}
|
|
|
|
/** Subfamily refinements that beat the family default. */
|
|
const ISO_SUBFAMILY_METHODS: Record<string, TransactionMethod> = {
|
|
SALA: 'salary', // SalaryPayment
|
|
XBCT: 'international', // CrossBorderCreditTransfer
|
|
ESCT: 'international', // SEPACreditTransfer
|
|
}
|
|
|
|
/** Keyword → method over raw code strings (covers proprietary formats). */
|
|
const CODE_KEYWORD_METHODS: ReadonlyArray<readonly [RegExp, TransactionMethod]> = [
|
|
[/SWISH/i, 'swish'],
|
|
[/AUTOGIRO/i, 'autogiro'],
|
|
[/INTRST|INTEREST|RÄNTA|RANTA/i, 'interest'],
|
|
[/\bFEE\b|CHRG|CHARGE|AVGIFT/i, 'fee'],
|
|
[/ATM|CASH.?WDL|WITHDRAW|UTTAG/i, 'withdrawal'],
|
|
[/\bCARD\b|KORT|\bPOS\b/i, 'card'],
|
|
[/SALA|SALARY|\bLÖN\b|\bLON\b/i, 'salary'],
|
|
]
|
|
|
|
function methodFromCodes(codes: string[]): TransactionMethod | null {
|
|
// Two passes so a subfamily refinement on EITHER code beats a family match
|
|
// on the other: with one combined pass, a family hit on the ISO code would
|
|
// short-circuit before the proprietary code's subfamily is inspected.
|
|
const partsList = codes.map((raw) => raw.toUpperCase().split(/[/\-_.\s]+/).filter(Boolean))
|
|
for (const parts of partsList) {
|
|
if (parts.length >= 3 && ISO_SUBFAMILY_METHODS[parts[2]]) {
|
|
return ISO_SUBFAMILY_METHODS[parts[2]]
|
|
}
|
|
}
|
|
for (const parts of partsList) {
|
|
if (parts.length >= 2) {
|
|
const family = ISO_FAMILY_METHODS[`${parts[0]}/${parts[1]}`]
|
|
if (family) return family
|
|
}
|
|
}
|
|
for (const raw of codes) {
|
|
for (const [re, method] of CODE_KEYWORD_METHODS) {
|
|
if (re.test(raw)) return method
|
|
}
|
|
}
|
|
return null
|
|
}
|
|
|
|
export function classifyTransactionMethod(
|
|
input: TransactionMethodInput
|
|
): ClassifiedTransactionMethod {
|
|
const description = (input.description ?? '').trim()
|
|
|
|
// Trailing phrase: sole source of the clean title, and the strongest
|
|
// text-level method signal.
|
|
let phraseMethod: TransactionMethod | null = null
|
|
let displayTitle = description
|
|
for (const rule of TRAILING_PHRASE_RULES) {
|
|
if (rule.regex.test(description)) {
|
|
phraseMethod = rule.method
|
|
const stripped = description.replace(rule.regex, '').trim()
|
|
const lastWord = stripped.toLowerCase().split(/\s+/).filter(Boolean).at(-1)
|
|
if (stripped.length > 0 && (!lastWord || !ADJECTIVE_GUARD.has(lastWord))) {
|
|
displayTitle = stripped
|
|
}
|
|
break
|
|
}
|
|
}
|
|
|
|
let leadingMethod: TransactionMethod | null = null
|
|
for (const [re, method] of LEADING_RULES) {
|
|
if (re.test(description)) {
|
|
leadingMethod = method
|
|
break
|
|
}
|
|
}
|
|
|
|
const codes = [input.bankTransactionCode, input.proprietaryBankTransactionCode].filter(
|
|
(c): c is string => typeof c === 'string' && c.trim().length > 0
|
|
)
|
|
|
|
const mccMethod: TransactionMethod | null =
|
|
input.mccCode != null ? (input.mccCode === 6011 ? 'withdrawal' : 'card') : null
|
|
|
|
const method =
|
|
input.explicitMethod ??
|
|
phraseMethod ??
|
|
leadingMethod ??
|
|
methodFromCodes(codes) ??
|
|
mccMethod
|
|
|
|
return { method, displayTitle }
|
|
}
|