fix(db): reconcile prod-orphaned migrations blocking Supabase branching (#942)

* fix(db): reconcile prod-orphaned migrations blocking Supabase branching

Prod's schema_migrations carries three versions with no committed file on
main, leaving the default Supabase branch in MIGRATIONS_FAILED and stopping
preview branches from being created:

  20260707113729  add_transactions_enrichment    (adopted from #927)
  20260708120000  ledger_stats_committed_at_lag  (adopted from #935)
  20260708130000  ledger_deep_context            (adopted from #935)

Adopt the byte-identical SQL under the exact apply-time versions, plus the
matching pg-tests and fixtures for the two RPCs so pg-real stays green:
20260708120000 switches get_ledger_usage_stats' median_booking_lag_days to
committed_at, so the existing test now asserts the new behavior. Idempotent
(ADD COLUMN IF NOT EXISTS / CREATE OR REPLACE FUNCTION): no-op on prod,
clean on fresh replays, no-op on #927/#935's next rebase. The knowledge-page
UI/lib/i18n stay in #935.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>

* fix(deps): pin @anthropic-ai/bedrock-sdk to 0.29.1

0.32.0 (grouped dependabot bump #884) broke Bedrock streaming in prod: empty stream / "request ended without sending any chunks", taking down the in-app AI assistant and invoice OCR. Local dev ran the stale 0.29.1 in node_modules, so it only failed on deploys built fresh from the lockfile. Revert to the six-week-stable 0.29.1; creds/region were never the cause (proven AKIA key + eu-west-1).

Guard against an accidental re-bump three ways: exact pin (no caret), a dependabot ignore, and a pinned-dep check in scripts/checks/no-new-antipatterns.mjs (check:guards). Unpin only once 0.32.x streaming is verified against Bedrock. See DECISIONS.md.
This commit is contained in:
Mattsson
2026-07-08 23:54:49 +02:00
committed by GitHub
parent c068638d24
commit 8dde46ad96
11 changed files with 752 additions and 7 deletions
+9
View File
@@ -49,3 +49,12 @@ updates:
update-types:
- minor
- patch
ignore:
# @anthropic-ai/bedrock-sdk is PINNED to an exact version in package.json.
# 0.32.0 arrived inside a grouped minor-and-patch bump (#884) and broke
# Bedrock streaming in prod: the SDK returned an empty stream ("request
# ended without sending any chunks"), taking down the in-app AI assistant
# and invoice OCR. Do NOT let dependabot bump it until 0.32.x streaming is
# verified against Bedrock. Enforced by scripts/checks/no-new-antipatterns.mjs
# (pinned-dep). See DECISIONS.md (2026-07-08).
- dependency-name: "@anthropic-ai/bedrock-sdk"
+3
View File
@@ -46,3 +46,6 @@ One line per decision: `[YYYY-MM-DD] <decision>: <why>`. Appended by agents and
[2026-07-07] Reconciled ledger-context prereqs INTO dev_docs/bank_transaction_ai_normalization.md (§14): plan is the strategic superset; ledger-context RPC gets interim normalizeCounterpartyName() now, re-keys to entity_id at Phase 2/Layer F. Closed 4 gaps: RPC in Layer F substrate list, supplier-side digest patterns, storno/correction exclusion (§13+§14), pending_operations audit+FK for agent-suggestion attribution.
[2026-07-08] Ledger-context prereq trifecta folded into the P1 branch pre-merge (normalize_counterparty_key SQL mirror of normalizeCounterpartyName + supplier_patterns CTE + storno filter + evidence{seen,agree,share,last_booked} format) instead of follow-up PRs: shipping first then fixing would break the payload shape consumers had just learned. Storno filter deliberately asymmetric: account_usage excludes source_type='storno' only; counterparty CTE has NO source_type filter because correctEntry() relinks transactions.journal_entry_id to the correction (the join self-heals) and excluding 'correction' would drop exactly the human-corrected booking. Faithful-mirror discipline: bare "KORT " prefix is NOT stripped (TS doesn't either); hardening the prefix list must change the TS+SQL pair together (pg test pins this). Payload caps trimmed 20/20 -> 15/15 + supplier 10 to hold the 12 KB budget with evidence objects.
[2026-07-08] Ledger-context dominant-contra VAT bug, found by the switch-on check (calling gnubok_get_agent_briefing on real prod data, not synthetic tests): counterparty patterns for foreign SaaS (Google/ngrok/Supabase) showed dominant_account 2614 (reverse-charge output VAT) instead of 5420 (software expense). Cause: the dominant_account CTE excluded only 19xx, so on a reverse-charge booking (expense + 2645 + 2614 + 1930) the three non-bank accounts tie and the account_number ASC tiebreak picks the low VAT number 2614. Fix (migration 20260708110000): also exclude 26xx (always moms in BAS, never characterizes a counterparty); 23xx/24xx/25xx/27xx stay eligible so loan/tax counterparties (e.g. ALMI) still surface their real account. supplier_patterns unaffected (aggregates supplier_invoice_items.account_number = expense only). Regression pg test asserts 5420 over 2614; verified it fails on the old function.
[2026-07-08] Bedrock prod outage + Docker build failure both root-caused to dependabot #884 (a1fad319, 2026-07-06) bumping @anthropic-ai/bedrock-sdk 0.29.1->0.32.0. Runtime: 0.32.0 streaming returns an empty event stream ("request ended without sending any chunks", no HTTP status) - proven NOT a creds/region issue (prod diagnostic logged AKIA key + eu-west-1). Two prior sessions mis-diagnosed it as an AWS_* env collision and shipped/reverted #937 (BEDROCK_AWS_* rename) with no effect. "Works locally, fails on prod/CI" because local node_modules was stale at 0.29.1 while prod/Docker build fresh from the lockfile (0.32.0). Fix: pin back to ^0.29.1 + regenerate lockfile. FOLLOW-UP: add a dependabot ignore/exact-pin so it does not re-bump to 0.32.x and re-break both.
[2026-07-08] One reconciliation PR adopts 3 prod-orphaned migrations (20260707113729 enrichment + 20260708120000/130000 ledger-stats RPCs) plus their pg-tests/fixtures onto main, instead of waiting on #927+#935 to merge: prod ledger was 3 versions ahead of the repo, leaving the default Supabase branch MIGRATIONS_FAILED and blocking every preview branch from being created. SQL committed byte-identical under the exact apply-time versions -> no-op on prod (idempotent), clean on fresh replays, and a no-op on #927/#935's next rebase. Carries #935's DB layer only (migrations + pg-tests + fixtures), not its UI/lib/i18n. Root anti-pattern: all three applied to prod via MCP apply_migration without committing the file (CLAUDE.md "never leave the remote DB ahead of the repo").
[2026-07-08] Pinned @anthropic-ai/bedrock-sdk to exact 0.29.1 (dependabot #884 auto-bumped it to 0.32.0, which broke Bedrock streaming in prod: empty stream / "request ended without sending any chunks"). Guarded three ways against accidental re-bump: exact pin in package.json, dependabot ignore, and a pinned-dep check in scripts/checks/no-new-antipatterns.mjs (check:guards). Unpin only once 0.32.x streaming is verified against Bedrock.
+1 -1
View File
@@ -9,7 +9,7 @@
"version": "0.1.0",
"license": "AGPL-3.0-or-later",
"dependencies": {
"@anthropic-ai/bedrock-sdk": "^0.29.1",
"@anthropic-ai/bedrock-sdk": "0.29.1",
"@hookform/resolvers": "^5.4.0",
"@radix-ui/react-checkbox": "^1.3.6",
"@radix-ui/react-dialog": "^1.1.18",
+1 -1
View File
@@ -22,7 +22,7 @@
"test:pg": "vitest run --project pg-real"
},
"dependencies": {
"@anthropic-ai/bedrock-sdk": "^0.29.1",
"@anthropic-ai/bedrock-sdk": "0.29.1",
"@hookform/resolvers": "^5.4.0",
"@radix-ui/react-checkbox": "^1.3.6",
"@radix-ui/react-dialog": "^1.1.18",
+55 -1
View File
@@ -22,6 +22,10 @@
* lineDimensionColumns() from the dimensions JSONB map
* (lib/bookkeeping/dimension-resolver.ts): a new direct insert site can
* silently diverge the mirror columns. Tracked as a file-set.
* 4. pinned-dep : a dependency pinned to an exact version (PINNED_DEPS)
* whose package.json spec or locked version drifted from the pin. Guards
* against a repeat of the @anthropic-ai/bedrock-sdk 0.32.0 prod outage
* (empty Bedrock stream). No baseline: any drift is a hard failure.
*
* Usage:
* node scripts/checks/no-new-antipatterns.mjs # check (CI)
@@ -130,10 +134,44 @@ function countNaiveRound() {
return count
}
// Dependencies pinned to an EXACT version on purpose, because a bump broke prod
// and must not silently return via `npm update`, a dependabot bump, or a manual
// install. Any drift (in package.json OR the lockfile) fails CI. See DECISIONS.md.
const PINNED_DEPS = [
{
name: '@anthropic-ai/bedrock-sdk',
version: '0.29.1',
reason:
'0.32.0 (grouped dependabot bump #884) broke Bedrock streaming in prod: empty stream, ' +
'"request ended without sending any chunks", taking down the AI assistant + invoice OCR. ' +
'Keep 0.29.1 until 0.32.x streaming is verified against Bedrock.',
},
]
/** Pinned deps whose package.json spec or locked version drifted from the pin. */
function findPinnedDepViolations() {
const pkg = JSON.parse(fs.readFileSync(path.join(ROOT, 'package.json'), 'utf8'))
const lock = JSON.parse(fs.readFileSync(path.join(ROOT, 'package-lock.json'), 'utf8'))
const declared = { ...pkg.dependencies, ...pkg.devDependencies }
const out = []
for (const pin of PINNED_DEPS) {
const spec = declared[pin.name]
if (spec !== undefined && spec !== pin.version) {
out.push({ ...pin, where: 'package.json', actual: spec })
}
const locked = lock.packages?.[`node_modules/${pin.name}`]?.version
if (locked !== undefined && locked !== pin.version) {
out.push({ ...pin, where: 'package-lock.json', actual: locked })
}
}
return out
}
const current = {
rawRouteAuth: findRawRouteAuth(),
naiveOreRound: countNaiveRound(),
directJelInsert: findDirectJelInserts(),
pinnedDepViolations: findPinnedDepViolations(),
}
const isUpdate = process.argv.includes('--update')
@@ -190,6 +228,22 @@ if (current.directJelInsert.length) {
)
}
// 1c. pinned-dep: a version-pinned dependency must match its pin EXACTLY, in
// both package.json and the lockfile. No baseline: any drift is a hard failure.
if (current.pinnedDepViolations.length) {
failed = true
console.error(`\n✗ pinned-dep: ${current.pinnedDepViolations.length} version-pinned dependency change(s):`)
current.pinnedDepViolations.forEach((v) =>
console.error(
` ${v.name} in ${v.where}: found "${v.actual}", must be exactly "${v.version}".\n ${v.reason}`,
),
)
console.error(
' → restore the pin (npm install <name>@<version> --save-exact). Only change PINNED_DEPS in\n' +
' this script once the upstream regression is confirmed fixed.',
)
}
// 2. naive-ore-round: count may not increase.
if (current.naiveOreRound > baseline.naiveOreRound.count) {
failed = true
@@ -214,5 +268,5 @@ if (failed) {
process.exit(1)
}
console.log(
`\n✓ Antipattern guard passed (raw-route-auth: ${current.rawRouteAuth.length}, naive-ore-round: ${current.naiveOreRound}, direct-jel-insert: 0).`,
`\n✓ Antipattern guard passed (raw-route-auth: ${current.rawRouteAuth.length}, naive-ore-round: ${current.naiveOreRound}, direct-jel-insert: 0, pinned-dep: 0).`,
)
@@ -0,0 +1,17 @@
-- Third-party transaction enrichment (Gokind counterparty identification).
-- Stores a trimmed projection of the enrichment response, not the raw payload:
-- { provider, fetched_at, identified, counterparty { id, name, org_numbers,
-- logo_url }, industries [], tags [], flags [], payment { subscription,
-- vendor_name } }
-- Nullable: enrichment is optional and fail-soft; rows ingested while the
-- provider is unconfigured or unavailable simply have NULL here.
--
-- Adopted reconciliation migration: this SQL was applied directly to prod on
-- 2026-07-07 (apply-time version 20260707113729) but the file was never
-- committed. Committed after the fact so the prod migration ledger and the
-- repo agree; prod already has the column, so this only runs on fresh
-- branches and staging. See DECISIONS.md 2026-07-08.
ALTER TABLE public.transactions
ADD COLUMN IF NOT EXISTS enrichment jsonb;
NOTIFY pgrst, 'reload schema';
@@ -0,0 +1,260 @@
-- Fix: median_booking_lag_days measures real posting promptness (committed_at),
-- not the accounting date (entry_date).
--
-- The prior version computed median(entry_date - transaction.date). But the
-- bank-booking flow dates the voucher (entry_date) TO the transaction date, so
-- entry_date - date is ~0 by construction: a tautology, not a signal. Verified
-- on prod (company ed461bc1...): entry_date == transaction date for 151 of 152
-- booked transactions, so the median came out 0, while the REAL lag,
-- committed_at (when the voucher was actually posted) minus the transaction
-- date, had median 90 days. This switches the metric to committed_at, the
-- honest "how promptly does this company book" signal the field was meant to
-- be. committed_at is set by set_committed_at() on draft->posted and is
-- non-null for posted entries; the IS NOT NULL guard is belt-and-braces.
--
-- Everything else is unchanged from 20260708110000 (26xx dominant-contra
-- exclusion, storno filter, normalize_counterparty_key keying). Full
-- CREATE OR REPLACE, no signature change.
--
-- pg-test: tests/pg/ledger-usage-stats-rpc.pg.test.ts
CREATE OR REPLACE FUNCTION public.get_ledger_usage_stats(
p_company_id uuid,
p_from_date date
)
RETURNS jsonb
LANGUAGE sql
STABLE
SECURITY INVOKER
SET search_path TO 'public'
AS $$
SELECT jsonb_build_object(
'account_usage',
(
SELECT coalesce(
jsonb_agg(
jsonb_build_object(
'account_number', au.account_number,
'account_name', au.account_name,
'postings', au.postings,
'last_used', au.last_used
)
ORDER BY au.postings DESC, au.account_number
),
'[]'::jsonb
)
FROM (
SELECT
l.account_number,
max(coa.account_name) AS account_name,
count(*)::bigint AS postings,
max(je.entry_date) AS last_used
FROM public.journal_entry_lines l
JOIN public.journal_entries je ON je.id = l.journal_entry_id
LEFT JOIN public.chart_of_accounts coa
ON coa.company_id = p_company_id
AND coa.account_number = l.account_number
WHERE je.company_id = p_company_id
AND je.status = 'posted'
-- Stornos annul: counting their swapped lines re-inflates the
-- account the correction moved away from. Corrections stay.
AND je.source_type <> 'storno'
AND je.entry_date >= p_from_date
GROUP BY l.account_number
ORDER BY count(*) DESC, l.account_number
LIMIT 20
) au
),
'counterparty_patterns',
(
WITH booked AS (
SELECT
public.normalize_counterparty_key(t.merchant_name) AS counterparty_key,
t.merchant_name,
t.category,
t.journal_entry_id,
t.date
FROM public.transactions t
JOIN public.journal_entries je ON je.id = t.journal_entry_id
WHERE t.company_id = p_company_id
AND t.journal_entry_id IS NOT NULL
AND je.status = 'posted'
-- Defensive: no code path should link a transaction to a storno
-- (correctEntry relinks to the correction, reverseEntry unlinks),
-- but legacy rows may predate the unlink behavior. Corrections are
-- deliberately NOT excluded: they are the live booking.
AND je.source_type <> 'storno'
AND t.merchant_name IS NOT NULL
AND trim(t.merchant_name) <> ''
AND t.date >= p_from_date
),
keyed AS (
-- All-digit/reference-only merchant labels normalize to '': no
-- identity, no pattern.
SELECT * FROM booked WHERE counterparty_key <> ''
),
totals AS (
SELECT
counterparty_key,
mode() WITHIN GROUP (ORDER BY merchant_name) AS display_name,
count(*)::bigint AS occurrences,
max(date) AS last_booked
FROM keyed
GROUP BY counterparty_key
),
dominant_category AS (
SELECT DISTINCT ON (counterparty_key)
counterparty_key,
category,
cnt
FROM (
SELECT counterparty_key, category, count(*)::bigint AS cnt
FROM keyed
WHERE category IS NOT NULL AND category <> 'uncategorized'
GROUP BY counterparty_key, category
) c
ORDER BY counterparty_key, cnt DESC, category
),
dominant_account AS (
SELECT DISTINCT ON (counterparty_key)
counterparty_key,
account_number
FROM (
SELECT b.counterparty_key, l.account_number, count(*)::bigint AS cnt
FROM keyed b
JOIN public.journal_entry_lines l ON l.journal_entry_id = b.journal_entry_id
WHERE l.account_number NOT LIKE '19%'
-- Exclude VAT accounts (26xx): on a reverse-charge purchase the
-- expense, 2645 and 2614 lines tie, and the account_number
-- tiebreak would otherwise pick the low VAT number over the
-- expense. 26xx is always moms, never the informative contra.
AND l.account_number NOT LIKE '26%'
GROUP BY b.counterparty_key, l.account_number
) a
ORDER BY counterparty_key, cnt DESC, account_number
)
SELECT coalesce(
jsonb_agg(
jsonb_build_object(
'counterparty', t.display_name,
'counterparty_key', t.counterparty_key,
'occurrences', t.occurrences,
'last_booked', t.last_booked,
'dominant_category', dc.category,
'dominant_category_count', coalesce(dc.cnt, 0),
'dominant_account_number', da.account_number
)
ORDER BY t.occurrences DESC, t.display_name
),
'[]'::jsonb
)
FROM (
SELECT * FROM totals ORDER BY occurrences DESC, display_name LIMIT 25
) t
LEFT JOIN dominant_category dc ON dc.counterparty_key = t.counterparty_key
LEFT JOIN dominant_account da ON da.counterparty_key = t.counterparty_key
),
'supplier_patterns',
(
-- AP-side booking patterns: bank-transaction patterns only see rows
-- with a merchant_name, so an invoice-heavy company would be half
-- blind without this. Supplier identity here is exact (FK), no
-- normalization needed.
WITH sinv AS (
SELECT si.id, si.supplier_id, s.name AS supplier_name,
si.invoice_date, si.vat_treatment
FROM public.supplier_invoices si
JOIN public.suppliers s ON s.id = si.supplier_id
WHERE si.company_id = p_company_id
AND si.invoice_date >= p_from_date
-- Reversed bookings and credited invoices are undone business;
-- credit notes repeat their original's accounts with flipped sign.
AND si.status NOT IN ('reversed', 'credited')
AND si.is_credit_note = false
),
totals AS (
SELECT
supplier_id,
max(supplier_name) AS supplier_name,
count(*)::bigint AS invoices,
max(invoice_date) AS last_invoice,
mode() WITHIN GROUP (ORDER BY vat_treatment) AS dominant_vat
FROM sinv
GROUP BY supplier_id
),
dominant_account AS (
-- Invoices (not lines) touching each account, so a many-line invoice
-- does not outvote ten single-line ones.
SELECT DISTINCT ON (supplier_id)
supplier_id,
account_number,
cnt
FROM (
SELECT v.supplier_id, i.account_number, count(DISTINCT v.id)::bigint AS cnt
FROM sinv v
JOIN public.supplier_invoice_items i ON i.supplier_invoice_id = v.id
GROUP BY v.supplier_id, i.account_number
) a
ORDER BY supplier_id, cnt DESC, account_number
)
SELECT coalesce(
jsonb_agg(
jsonb_build_object(
'supplier', t.supplier_name,
'invoices', t.invoices,
'last_invoice', t.last_invoice,
'vat_treatment', t.dominant_vat,
'dominant_account_number', da.account_number,
'dominant_account_count', coalesce(da.cnt, 0)
)
ORDER BY t.invoices DESC, t.supplier_name
),
'[]'::jsonb
)
FROM (
SELECT * FROM totals ORDER BY invoices DESC, supplier_name LIMIT 15
) t
LEFT JOIN dominant_account da ON da.supplier_id = t.supplier_id
),
'vat_treatments_used',
(
SELECT coalesce(jsonb_agg(DISTINCT vt), '[]'::jsonb)
FROM (
SELECT i.vat_treatment AS vt
FROM public.invoices i
WHERE i.company_id = p_company_id
AND i.invoice_date >= p_from_date
AND i.vat_treatment IS NOT NULL
UNION
SELECT si.vat_treatment AS vt
FROM public.supplier_invoices si
WHERE si.company_id = p_company_id
AND si.invoice_date >= p_from_date
AND si.vat_treatment IS NOT NULL
) treatments
),
'median_booking_lag_days',
(
-- committed_at (when the voucher was POSTED), not entry_date (the
-- accounting date the bank flow sets to the transaction date, making
-- entry_date - date a ~0 tautology). This is real posting promptness.
SELECT percentile_cont(0.5) WITHIN GROUP (ORDER BY (je.committed_at::date - t.date))
FROM public.transactions t
JOIN public.journal_entries je ON je.id = t.journal_entry_id
WHERE t.company_id = p_company_id
AND je.status = 'posted'
-- A legacy transaction still linked to a storno (a reversal posted
-- long after the transaction) would inject a spurious large lag into
-- the very metric this migration exists to make honest. Exclude it,
-- matching account_usage and counterparty_patterns.
AND je.source_type <> 'storno'
AND je.committed_at IS NOT NULL
AND t.date >= p_from_date
)
);
$$;
REVOKE ALL ON FUNCTION public.get_ledger_usage_stats(uuid, date) FROM PUBLIC, anon;
GRANT EXECUTE ON FUNCTION public.get_ledger_usage_stats(uuid, date) TO authenticated, service_role;
NOTIFY pgrst, 'reload schema';
@@ -0,0 +1,211 @@
-- RPC: get_ledger_deep_context — the deep, entity-resolved analysis behind the
-- "Vad din agent vet" page. Where get_ledger_usage_stats is the light,
-- 12-month digest the agent reads at session start, THIS is the full-history
-- deep pass: it merges counterparties across name variants, mines the booked
-- verifikat for real spend, and detects recurrence.
--
-- For each counterparty entity (bank-feed side) and supplier entity (AP side)
-- it returns:
-- name display label (modal raw variant)
-- key the normalized identity key (counterparties) / supplier id
-- variants up to 8 distinct raw labels that merged into this entity
-- variant_count true number of distinct raw labels
-- occurrences number of bookings
-- total_amount total paid (gross, sum of abs bank amount / invoice total)
-- first_seen/last_seen
-- cadence_days median gap between distinct booking dates (null if <2)
-- dominant_account_number + dominant_account_share (consistency 0..1)
-- dominant_vat (supplier side; counterparties carry it via the light RPC)
--
-- Determinism: entities merge by normalize_counterparty_key() (the SQL mirror
-- of normalizeCounterpartyName), which collapses "KLARNA AB" / "SWISH KLARNA
-- AB" / "KORTKÖP KLARNA AB 2026-07-01" into one "klarna". Cross-name merges
-- that need the counterparty bank account or fuzzy matching ("Klarna" +
-- "Klarna Bank AB") are the persistent-entity-table layer
-- (bank_transaction_ai_normalization.md, deferred); this is the deterministic
-- read-side v1.
--
-- Storno excluded (source_type <> 'storno'); corrections kept (they are the
-- live booking, the transaction is relinked to them). 26xx VAT and 19xx bank
-- are excluded from the dominant contra pick, same asymmetry as the light RPC.
--
-- p_from_date NULL = full history; a date bounds the lookback (large tenants
-- pass a bound; the cache layer is deferred, same "measure before enforce"
-- posture as the light RPC).
--
-- SECURITY INVOKER: RLS on the base tables scopes to the caller's company.
--
-- pg-test: tests/pg/ledger-deep-context-rpc.pg.test.ts
CREATE OR REPLACE FUNCTION public.get_ledger_deep_context(
p_company_id uuid,
p_from_date date DEFAULT NULL
)
RETURNS jsonb
LANGUAGE sql
STABLE
SECURITY INVOKER
SET search_path TO 'public'
AS $$
SELECT jsonb_build_object(
'counterparty_entities',
(
WITH booked AS (
SELECT
public.normalize_counterparty_key(t.merchant_name) AS k,
t.merchant_name,
t.journal_entry_id,
t.date,
-- SEK: amount_sek carries the converted value on foreign-currency
-- rows and is null on SEK rows, so coalesce keeps everything in SEK
-- (mirrors the supplier side's coalesce(total_sek, total)).
abs(coalesce(t.amount_sek, t.amount)) AS amount
FROM public.transactions t
JOIN public.journal_entries je ON je.id = t.journal_entry_id
WHERE t.company_id = p_company_id
AND t.journal_entry_id IS NOT NULL
AND je.status = 'posted'
AND je.source_type <> 'storno'
AND t.merchant_name IS NOT NULL
AND trim(t.merchant_name) <> ''
AND (p_from_date IS NULL OR t.date >= p_from_date)
),
keyed AS (SELECT * FROM booked WHERE k <> ''),
-- Median gap between distinct booking dates -> recurrence cadence.
distinct_dates AS (SELECT DISTINCT k, date FROM keyed),
gaps AS (
SELECT k, (date - lag(date) OVER (PARTITION BY k ORDER BY date)) AS gap
FROM distinct_dates
),
recur AS (
SELECT k, round(percentile_cont(0.5) WITHIN GROUP (ORDER BY gap))::int AS cadence_days
FROM gaps WHERE gap IS NOT NULL GROUP BY k
),
-- Dominant contra account + its share, over the entity's verifikat lines.
acct_counts AS (
SELECT b.k, l.account_number, count(*)::bigint AS cnt
FROM keyed b
JOIN public.journal_entry_lines l ON l.journal_entry_id = b.journal_entry_id
WHERE l.account_number NOT LIKE '19%'
AND l.account_number NOT LIKE '26%'
GROUP BY b.k, l.account_number
),
acct_totals AS (SELECT k, sum(cnt) AS total FROM acct_counts GROUP BY k),
dominant_account AS (
SELECT DISTINCT ON (ac.k) ac.k, ac.account_number, ac.cnt, at.total
FROM acct_counts ac JOIN acct_totals at ON at.k = ac.k
ORDER BY ac.k, ac.cnt DESC, ac.account_number
),
agg AS (
SELECT
k,
mode() WITHIN GROUP (ORDER BY merchant_name) AS display_name,
count(*)::bigint AS occurrences,
count(DISTINCT merchant_name)::int AS variant_count,
(array_agg(DISTINCT merchant_name))[1:8] AS variants,
sum(amount) AS total_amount,
min(date) AS first_seen,
max(date) AS last_seen
FROM keyed GROUP BY k
)
SELECT coalesce(
jsonb_agg(
jsonb_build_object(
'name', a.display_name,
'key', a.k,
'variants', to_jsonb(a.variants),
'variant_count', a.variant_count,
'occurrences', a.occurrences,
'total_amount', round(a.total_amount)::bigint,
'first_seen', a.first_seen,
'last_seen', a.last_seen,
'cadence_days', r.cadence_days,
'dominant_account_number', da.account_number,
'dominant_account_share',
CASE WHEN da.total > 0 THEN round(da.cnt::numeric / da.total, 2) ELSE NULL END
)
ORDER BY a.occurrences DESC, a.total_amount DESC, a.display_name
),
'[]'::jsonb
)
FROM (SELECT * FROM agg ORDER BY occurrences DESC, total_amount DESC, display_name LIMIT 40) a
LEFT JOIN recur r ON r.k = a.k
LEFT JOIN dominant_account da ON da.k = a.k
),
'supplier_entities',
(
WITH sinv AS (
SELECT si.id, si.supplier_id, s.name AS supplier_name,
si.invoice_date, si.vat_treatment,
coalesce(si.total_sek, si.total, 0) AS amount
FROM public.supplier_invoices si
JOIN public.suppliers s ON s.id = si.supplier_id
WHERE si.company_id = p_company_id
AND si.status NOT IN ('reversed', 'credited')
AND si.is_credit_note = false
AND (p_from_date IS NULL OR si.invoice_date >= p_from_date)
),
distinct_dates AS (SELECT DISTINCT supplier_id, invoice_date FROM sinv),
gaps AS (
SELECT supplier_id,
(invoice_date - lag(invoice_date) OVER (PARTITION BY supplier_id ORDER BY invoice_date)) AS gap
FROM distinct_dates
),
recur AS (
SELECT supplier_id, round(percentile_cont(0.5) WITHIN GROUP (ORDER BY gap))::int AS cadence_days
FROM gaps WHERE gap IS NOT NULL GROUP BY supplier_id
),
acct_counts AS (
SELECT v.supplier_id, i.account_number, count(DISTINCT v.id)::bigint AS cnt
FROM sinv v JOIN public.supplier_invoice_items i ON i.supplier_invoice_id = v.id
GROUP BY v.supplier_id, i.account_number
),
acct_totals AS (SELECT supplier_id, sum(cnt) AS total FROM acct_counts GROUP BY supplier_id),
dominant_account AS (
SELECT DISTINCT ON (ac.supplier_id) ac.supplier_id, ac.account_number, ac.cnt, at.total
FROM acct_counts ac JOIN acct_totals at ON at.supplier_id = ac.supplier_id
ORDER BY ac.supplier_id, ac.cnt DESC, ac.account_number
),
agg AS (
SELECT
supplier_id,
max(supplier_name) AS supplier_name,
count(*)::bigint AS occurrences,
sum(amount) AS total_amount,
min(invoice_date) AS first_seen,
max(invoice_date) AS last_seen,
mode() WITHIN GROUP (ORDER BY vat_treatment) AS dominant_vat
FROM sinv GROUP BY supplier_id
)
SELECT coalesce(
jsonb_agg(
jsonb_build_object(
'name', a.supplier_name,
'key', a.supplier_id::text,
'variants', to_jsonb(ARRAY[a.supplier_name]),
'variant_count', 1,
'occurrences', a.occurrences,
'total_amount', round(a.total_amount)::bigint,
'first_seen', a.first_seen,
'last_seen', a.last_seen,
'cadence_days', r.cadence_days,
'dominant_account_number', da.account_number,
'dominant_account_share',
CASE WHEN da.total > 0 THEN round(da.cnt::numeric / da.total, 2) ELSE NULL END,
'dominant_vat', a.dominant_vat
)
ORDER BY a.occurrences DESC, a.total_amount DESC, a.supplier_name
),
'[]'::jsonb
)
FROM (SELECT * FROM agg ORDER BY occurrences DESC, total_amount DESC, supplier_name LIMIT 20) a
LEFT JOIN recur r ON r.supplier_id = a.supplier_id
LEFT JOIN dominant_account da ON da.supplier_id = a.supplier_id
)
);
$$;
REVOKE ALL ON FUNCTION public.get_ledger_deep_context(uuid, date) FROM PUBLIC, anon;
GRANT EXECUTE ON FUNCTION public.get_ledger_deep_context(uuid, date) TO authenticated, service_role;
NOTIFY pgrst, 'reload schema';
+6 -2
View File
@@ -175,13 +175,16 @@ export async function insertDraftJournalEntry(params: {
sourceType?: string
sourceId?: string | null
createdAt?: string
// Inserting directly as 'posted' skips the set_committed_at() trigger (it
// fires on draft->posted UPDATE), so committed_at stays null unless set here.
committedAt?: string | null
}): Promise<string> {
const id = randomUUID()
await getPool().query(
`INSERT INTO public.journal_entries
(id, user_id, company_id, fiscal_period_id, voucher_number, voucher_series,
entry_date, description, source_type, source_id, status, created_at)
VALUES ($1, $2, $3, $4, $5, $6, $7, $8, $9, $10, $11, COALESCE($12::timestamptz, now()))`,
entry_date, description, source_type, source_id, status, created_at, committed_at)
VALUES ($1, $2, $3, $4, $5, $6, $7, $8, $9, $10, $11, COALESCE($12::timestamptz, now()), $13::timestamptz)`,
[
id,
params.userId,
@@ -195,6 +198,7 @@ export async function insertDraftJournalEntry(params: {
params.sourceId ?? null,
params.status ?? 'draft',
params.createdAt ?? null,
params.committedAt ?? null,
],
)
return id
+174
View File
@@ -0,0 +1,174 @@
/**
* pg-real test for get_ledger_deep_context.
*
* The deep, full-history analysis behind the "Vad din agent vet" page: merges
* counterparties across name variants, mines booked verifikat for spend, and
* detects recurrence. Verifies variant-merging, occurrence/spend rollup,
* dominant-account + share, recurrence cadence, and supplier entities.
*/
import { describe, it, expect, beforeAll } from 'vitest'
import { randomUUID } from 'node:crypto'
import { getPool } from './setup'
import { seedCompany, insertDraftJournalEntry } from './fixtures'
async function insertLines(
journalEntryId: string,
lines: Array<{ account: string; debit: number; credit: number }>,
): Promise<void> {
for (const line of lines) {
await getPool().query(
`INSERT INTO public.journal_entry_lines
(journal_entry_id, account_number, debit_amount, credit_amount)
VALUES ($1, $2, $3, $4)`,
[journalEntryId, line.account, line.debit, line.credit],
)
}
}
async function bookMerchant(params: {
userId: string
companyId: string
fiscalPeriodId: string
merchantName: string
date: string
expenseAccount: string
amount: number
voucherNumber: number
}): Promise<void> {
const entryId = await insertDraftJournalEntry({
userId: params.userId,
companyId: params.companyId,
fiscalPeriodId: params.fiscalPeriodId,
entryDate: params.date,
status: 'posted',
voucherNumber: params.voucherNumber,
sourceType: 'bank_transaction',
})
await insertLines(entryId, [
{ account: params.expenseAccount, debit: params.amount, credit: 0 },
{ account: '1930', debit: 0, credit: params.amount },
])
await getPool().query(
`INSERT INTO public.transactions
(id, company_id, user_id, currency, amount, date, description, journal_entry_id, merchant_name, category)
VALUES ($1,$2,$3,'SEK',$4,$5,$6,$7,$8,'expense_software')`,
[randomUUID(), params.companyId, params.userId, -params.amount, params.date,
`Payment ${params.merchantName}`, entryId, params.merchantName],
)
}
type DeepEntity = {
name: string
key: string
variants: string[]
variant_count: number
occurrences: number
total_amount: number
first_seen: string
last_seen: string
cadence_days: number | null
dominant_account_number: string | null
dominant_account_share: number | null
dominant_vat?: string | null
}
type Deep = { counterparty_entities: DeepEntity[]; supplier_entities: DeepEntity[] }
async function callRpc(companyId: string, fromDate: string | null): Promise<Deep> {
const res = await getPool().query(
`SELECT public.get_ledger_deep_context($1, $2) AS d`,
[companyId, fromDate],
)
return res.rows[0].d as Deep
}
describe('get_ledger_deep_context', () => {
let userId: string
let companyId: string
let fiscalPeriodId: string
beforeAll(async () => {
const seeded = await seedCompany()
userId = seeded.userId
companyId = seeded.companyId
fiscalPeriodId = seeded.fiscalPeriodId
// Klarna under 3 name variants, monthly, all to 5420, 100 kr each.
await bookMerchant({ userId, companyId, fiscalPeriodId, merchantName: 'KLARNA AB', date: '2026-04-01', expenseAccount: '5420', amount: 100, voucherNumber: 1 })
await bookMerchant({ userId, companyId, fiscalPeriodId, merchantName: 'SWISH KLARNA AB', date: '2026-05-01', expenseAccount: '5420', amount: 100, voucherNumber: 2 })
await bookMerchant({ userId, companyId, fiscalPeriodId, merchantName: 'KORTKÖP KLARNA AB 2026-06-01', date: '2026-06-01', expenseAccount: '5420', amount: 100, voucherNumber: 3 })
// A one-off different merchant.
await bookMerchant({ userId, companyId, fiscalPeriodId, merchantName: 'SL', date: '2026-05-10', expenseAccount: '5810', amount: 50, voucherNumber: 4 })
// A supplier with 2 invoices.
const supplierId = randomUUID()
await getPool().query(`INSERT INTO public.suppliers (id, user_id, company_id, name) VALUES ($1,$2,$3,'Telia Sverige AB')`,
[supplierId, userId, companyId])
let arr = 1
for (const [d, total] of [['2026-04-15', 500], ['2026-05-15', 500]] as const) {
const invId = randomUUID()
await getPool().query(
`INSERT INTO public.supplier_invoices
(id,user_id,company_id,supplier_id,arrival_number,supplier_invoice_number,invoice_date,due_date,status,vat_treatment,is_credit_note,subtotal,vat_amount,total,total_sek)
VALUES ($1,$2,$3,$4,$5,$6,$7,$7,'registered','standard_25',false,400,100,500,$8)`,
[invId, userId, companyId, supplierId, arr++, `SI-${arr}`, d, total],
)
await getPool().query(
`INSERT INTO public.supplier_invoice_items (supplier_invoice_id,description,quantity,unit_price,line_total,account_number,vat_rate,vat_amount)
VALUES ($1,'Line',1,400,400,'6212',0.25,100)`,
[invId],
)
}
})
it('merges counterparty name variants into one entity with spend + cadence', async () => {
const deep = await callRpc(companyId, null)
const klarna = deep.counterparty_entities.find((e) => e.key === 'klarna')
expect(klarna).toBeDefined()
expect(klarna!.occurrences).toBe(3)
expect(klarna!.variant_count).toBe(3)
expect(klarna!.variants.length).toBeGreaterThanOrEqual(3)
expect(klarna!.dominant_account_number).toBe('5420')
expect(klarna!.dominant_account_share).toBe(1)
expect(klarna!.total_amount).toBe(300)
expect(klarna!.first_seen).toBe('2026-04-01')
expect(klarna!.last_seen).toBe('2026-06-01')
// Monthly cadence: gaps of 30 and 31 days -> median ~30.
expect(klarna!.cadence_days).toBeGreaterThanOrEqual(30)
expect(klarna!.cadence_days).toBeLessThanOrEqual(31)
})
it('keeps a one-off merchant as a single-occurrence entity', async () => {
const deep = await callRpc(companyId, null)
const sl = deep.counterparty_entities.find((e) => e.key === 'sl')
expect(sl!.occurrences).toBe(1)
expect(sl!.variant_count).toBe(1)
expect(sl!.cadence_days).toBeNull()
expect(sl!.dominant_account_number).toBe('5810')
})
it('aggregates supplier entities with spend and dominant account', async () => {
const deep = await callRpc(companyId, null)
const telia = deep.supplier_entities.find((e) => e.name === 'Telia Sverige AB')
expect(telia).toBeDefined()
expect(telia!.occurrences).toBe(2)
expect(telia!.total_amount).toBe(1000)
expect(telia!.dominant_account_number).toBe('6212')
expect(telia!.dominant_vat).toBe('standard_25')
expect(telia!.cadence_days).toBeGreaterThanOrEqual(30)
})
it('respects the from_date bound', async () => {
const deep = await callRpc(companyId, '2026-05-15')
const klarna = deep.counterparty_entities.find((e) => e.key === 'klarna')
// Only the 2026-06-01 Klarna booking is on/after the bound.
expect(klarna!.occurrences).toBe(1)
})
it('isolates by company', async () => {
const other = await seedCompany()
const deep = await callRpc(other.companyId, null)
expect(deep.counterparty_entities).toEqual([])
expect(deep.supplier_entities).toEqual([])
})
})
+15 -2
View File
@@ -60,6 +60,13 @@ async function insertBookedTransaction(params: {
)
}
/** ISO date + n days, as a UTC timestamptz string (for committed_at). */
function plusDays(isoDate: string, n: number): string {
const d = new Date(`${isoDate}T00:00:00Z`)
d.setUTCDate(d.getUTCDate() + n)
return d.toISOString()
}
// Posted entry + lines + a booked transaction pointing at it, in one call.
async function bookMerchant(params: {
userId: string
@@ -80,6 +87,9 @@ async function bookMerchant(params: {
status: 'posted',
voucherNumber: params.voucherNumber,
sourceType: params.sourceType ?? 'bank_transaction',
// Booked 3 days after the transaction: exercises the committed_at-based
// lag (entry_date == transaction date would give 0).
committedAt: plusDays(params.date, 3),
})
await insertLines(entryId, [
{ account: params.expenseAccount, debit: 500, credit: 0 },
@@ -299,6 +309,7 @@ describe('get_ledger_usage_stats', () => {
userId, companyId, fiscalPeriodId,
entryDate: d, status: 'posted',
voucherNumber: rcVoucher++, sourceType: 'bank_transaction',
committedAt: plusDays(d, 3),
})
await insertLines(rcEntry, [
{ account: '5420', debit: 500, credit: 0 },
@@ -441,8 +452,10 @@ describe('get_ledger_usage_stats', () => {
const stats = await callRpc(companyId, '2026-04-01')
expect(stats.vat_treatments_used).toContain('standard_25')
expect(stats.vat_treatments_used).not.toContain('reverse_charge_eu')
// All fixtures book same-day (entry_date = transaction date).
expect(stats.median_booking_lag_days).toBe(0)
// Every booked fixture sets committed_at = transaction date + 3 days, so
// the lag is measured from committed_at (not entry_date, which == the
// transaction date and would give a misleading 0).
expect(stats.median_booking_lag_days).toBe(3)
})
it('returns empty sections for a company with no data (isolation)', async () => {