feat(parties): suggestion pipeline from ledger keys and linked documents (#2172)
* feat(parties): phase 1 substrate, one party per counterpart Adds the identity layer above customers and suppliers, which keep their tables and every foreign key and gain a nullable party_id. - parties: company-scoped identity with status (suggested | confirmed), kind, alias keys, origin and merged_into. One live party per org number and company, enforced by a partial unique index; merged losers leave the index so a merge can be undone. This is the unique key the duplicate-invoice guard has lacked, since suppliers never had one. - party_facts: statements with a source, a rank (preferred | normal | deprecated) and two time axes, never overwritten. - party_identities: bankgiro, plusgiro, IBAN and friends per party, with seen and paid counts and a known | unverified status. - party_decisions: every human action on a party as a labelled example. - normalize_org_number(text): SQL mirror of lib/invariants/org-number.ts (strip separators, drop the century on 12 digits, Luhn check, 10 digits). - ensure_party(): find by org number inside the company, else create. Name-only rows never merge at insert time; a name merge is a recorded human decision. - Backfill: one party per existing supplier and customer, merged on org number, suppliers first so both roles land on one party. - Archive contract: the four tables are master data in the full archive. Observed parties (keys derived from voucher and bank text) are not stored; they stay computed by the ledger-context RPC. No posted entry is touched. Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com> * feat(parties): observed parties from voucher text, ledger_key and its mirror Migrants arrive with vouchers, not bank transactions, so the bank-keyed ledger context is empty for them. This adds the description-keyed twin. - public.ledger_key(text): legibility key on top of the frozen normalize_counterparty_key mirror: strips AP-register prefixes (levfakt, leverantörsfaktura från N, levbet, faktura, kvitto, utgift), the supplier number that follows them, and trailing 1-3 digit runs, never "inköp". Mirrored by lib/parties/ledger-key.ts; the pair is pinned by a shared fixture list in the pg test. - public.get_observed_parties(company, from_date, limit): posted vouchers grouped by ledger_key(description) with occurrences, variants, expense and revenue SEK from the lines, first/last seen, median cadence and the Laplace-smoothed dominant result account. Excludes storno, opening balance, year-end and VAT settlement, and vouchers that carry a bank merchant name (those stay with get_ledger_deep_context). SECURITY INVOKER, so RLS scopes it. Never stored. - lib/parties/classify.ts: the deterministic pre-classifier moved out of the evaluation script so product and evaluation share one implementation (0.965 agreement with the founder labels, party recall 0.99). - lib/parties/observed.ts: RPC wrapper that classifies each row and derives a display rhythm from the cadence. Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com> * fix(parties): tenant-safe composite foreign keys on every party link Facts, identities, decisions, customers.party_id, suppliers.party_id and parties.merged_into now reference parties(id, company_id), so a row can only point at a party in its own company. ON DELETE SET NULL names party_id so role rows keep their company_id. Adds a pg-real test that rejects every cross-company link and checks company_id survives a party delete. Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com> * feat(parties): suggestion pipeline from ledger keys and linked documents Phase 1c of the parties plan. Fills a migrant's register with suggested parties from what the ledger already knows, never as facts: - get_ledger_key_evidence(company): hard keys per ledger_key from the documents linked to posted vouchers (org number via normalize_org_number, VAT, bankgiro, plusgiro, printed name). Documents whose supplier org is the company's own are the company's sales invoices and only count in self_docs. - apply_party_suggestions(company, user, items): upserts suggestions. Attaches by explicit party_id, org number or an exact alias key; never by name. Identities become known at two sightings. Idempotent. - decide_parties(company, user, ids, kind, note): bulk confirm or dismiss with one party_decisions row each. - parties.suggested_reason: the evidence summary the queue shows per row. - lib/parties/suggest.ts: buildSuggestions (pure) and suggestPartiesForCompany. Keys that mix two org numbers keep neither the hard key nor identities; same-core live parties are reported as similar_to for a person to decide. coreKey() moves into ledger-key.ts. 55 unit tests and 14 pg-real tests pass locally. Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com> * fix(parties): decide_parties dismisses suggested parties only Dismiss is the queue's answer to a suggestion; a confirmed party is never archived through it. Superagent P2 on #2172. Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com> * test(parties): type the rpc mock with its args so the ratchet stays clean Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com> --------- Co-authored-by: Jakob Wennberg <311770904+jakobwennberg-oss@users.noreply.github.com> Co-authored-by: Claude Fable 5.1 <noreply@anthropic.com>
This commit is contained in:
Jakob Wennberg
@@ -1503,4 +1503,5 @@ One line per decision: `[YYYY-MM-DD] <decision>: <why>`. Appended by agents and
|
||||
[2026-09-02] Bank-sync cooldown is a durable lease column (bank_connections.sync_lease_until, migration 20260902150000) claimed with one conditional UPDATE, not a process-local attempt map: the security scan on PR #2165 showed the map is bypassed by a second serverless instance or a cold start, so two agent calls could each bill Enable Banking. A column add was chosen over reusing extension_data because PostgREST cannot express an atomic conditional upsert there; the nightly cron deliberately ignores the lease.
|
||||
[2026-09-02] Grok links carry auth=required like the claude.ai link (#2159), decided from a live test: on the lazy URL Grok's connector dialog listed all 150+ tools and never opened the sign-in, so it reads the 200 probe as an authless server exactly as claude.ai does. The flag lives in one helper (mcpServerUrl / sideDoorServerUrl in lib/onboarding/checklist.ts) so the settings row, the onboarding side door and the deep link cannot drift; ChatGPT stays lazy because its developer mode honours the 401 on the first protected call.
|
||||
[2026-09-02] parties children/roles reference parties(id, company_id) with composite FKs, not parties(id): a party UUID from another tenant is rejected by construction instead of relying on each writer to check; ON DELETE SET NULL (party_id) on customers/suppliers because a plain SET NULL would null company_id too (Superagent P2 on #2162)
|
||||
[2026-09-02] Party suggestions attach only by explicit party_id, org number or an exact ledger key already in alias_keys; same-core text is reported as similar_to for a person to decide and identities are withheld when a key mixes org numbers: the selection eval measured 9% false merges on trade names shared by distinct legal entities (Fortnox AB / Fortnox Finans), so text never merges
|
||||
[2026-09-02] Edited migration 20260902160000 after merge: its backfill failed on prod (ensure_party: name is required; 3 nameless rows) so it was never applied there, the Supabase main branch sat in MIGRATIONS_FAILED and every later migration was blocked behind it. An unapplied file is not a shipped schema; a follow-up migration could not run before it
|
||||
|
||||
@@ -0,0 +1,194 @@
|
||||
import { describe, expect, it, vi } from 'vitest'
|
||||
import { coreKey } from '../ledger-key'
|
||||
import type { ObservedParty } from '../observed'
|
||||
import { buildSuggestions, suggestPartiesForCompany, type ExistingParty, type LedgerKeyEvidence } from '../suggest'
|
||||
|
||||
function observed(over: Partial<ObservedParty> & { key: string }): ObservedParty {
|
||||
return {
|
||||
name: over.key.toUpperCase(),
|
||||
variants: [],
|
||||
variant_count: 1,
|
||||
occurrences: 3,
|
||||
expense_sek: 3000,
|
||||
revenue_sek: 0,
|
||||
first_seen: '2026-01-10',
|
||||
last_seen: '2026-03-10',
|
||||
cadence_days: 30,
|
||||
dominant_account_number: '4000',
|
||||
dominant_account_share: 0.6,
|
||||
dominant_account_count: 2,
|
||||
dominant_account_total: 3,
|
||||
label: 'party',
|
||||
rhythm: 'monthly',
|
||||
...over,
|
||||
}
|
||||
}
|
||||
|
||||
function evidence(over: Partial<LedgerKeyEvidence> & { key: string }): LedgerKeyEvidence {
|
||||
return { docs: 0, self_docs: 0, orgs: [], vat_numbers: [], names: [], bankgiro: [], plusgiro: [], ...over }
|
||||
}
|
||||
|
||||
const ORG = '5564300142'
|
||||
|
||||
describe('coreKey', () => {
|
||||
it('strips AP prefixes, digit runs and legal forms', () => {
|
||||
expect(coreKey('levfakt beijer byggmaterial ab 2089')).toBe('beijer byggmaterial')
|
||||
expect(coreKey('Fortnox Finans AB')).toBe('fortnox finans')
|
||||
expect(coreKey('inköp av varor')).toBe('av varor')
|
||||
})
|
||||
})
|
||||
|
||||
describe('buildSuggestions', () => {
|
||||
it('skips keys the pre-classifier does not call party', () => {
|
||||
const r = buildSuggestions({
|
||||
observed: [observed({ key: 'inköp av varor', label: 'category' }), observed({ key: 'lön mars', label: 'payroll' })],
|
||||
evidence: [],
|
||||
existing: [],
|
||||
})
|
||||
expect(r.items).toHaveLength(0)
|
||||
expect(r.skipped).toEqual([
|
||||
{ key: 'inköp av varor', label: 'category' },
|
||||
{ key: 'lön mars', label: 'payroll' },
|
||||
])
|
||||
})
|
||||
|
||||
it('creates a new suggested party from the ledger alone, with ledger facts and a reason', () => {
|
||||
const r = buildSuggestions({ observed: [observed({ key: 'beijer byggmaterial' })], evidence: [], existing: [] })
|
||||
expect(r.items).toHaveLength(1)
|
||||
const item = r.items[0]!
|
||||
expect(item.party_id).toBeUndefined()
|
||||
expect(item.org_number).toBeUndefined()
|
||||
expect(item.origin).toBe('ledger')
|
||||
expect(item.display_name).toBe('BEIJER BYGGMATERIAL')
|
||||
expect(item.alias_keys).toEqual(['beijer byggmaterial'])
|
||||
expect(item.reason.attach).toBe('new')
|
||||
expect(item.reason.occurrences).toBe(3)
|
||||
expect(item.facts.map((f) => f.field)).toEqual(['dominant_account', 'cadence_days'])
|
||||
expect(item.identities).toEqual([])
|
||||
})
|
||||
|
||||
it('uses the document hard key: org number, printed name, VAT and identities', () => {
|
||||
const r = buildSuggestions({
|
||||
observed: [observed({ key: 'beijer byggmaterial' })],
|
||||
evidence: [
|
||||
evidence({
|
||||
key: 'beijer byggmaterial',
|
||||
docs: 3,
|
||||
orgs: [{ org: ORG, n: 3 }],
|
||||
vat_numbers: [{ vat: `SE${ORG}01`, n: 3 }],
|
||||
names: [{ name: 'Beijer Byggmaterial AB', n: 3 }],
|
||||
bankgiro: [{ value: '53170900', n: 3, first_seen: '2026-01-10', last_seen: '2026-03-10' }],
|
||||
}),
|
||||
],
|
||||
existing: [],
|
||||
})
|
||||
const item = r.items[0]!
|
||||
expect(item.org_number).toBe(ORG)
|
||||
expect(item.origin).toBe('document')
|
||||
expect(item.display_name).toBe('Beijer Byggmaterial AB')
|
||||
expect(item.legal_name).toBe('Beijer Byggmaterial AB')
|
||||
expect(item.vat_number).toBe(`SE${ORG}01`)
|
||||
expect(item.identities).toEqual([
|
||||
{ scheme: 'bankgiro', value: '53170900', first_seen: '2026-01-10', last_seen: '2026-03-10', seen_count: 3 },
|
||||
])
|
||||
expect(item.facts.map((f) => f.field)).toEqual(['dominant_account', 'cadence_days', 'org_number', 'legal_name'])
|
||||
expect(item.reason.org_number).toBe(ORG)
|
||||
})
|
||||
|
||||
it('withholds the hard key and identities when a key mixes two org numbers', () => {
|
||||
const r = buildSuggestions({
|
||||
observed: [observed({ key: 'vattenfall' })],
|
||||
evidence: [
|
||||
evidence({
|
||||
key: 'vattenfall',
|
||||
docs: 4,
|
||||
orgs: [
|
||||
{ org: ORG, n: 2 },
|
||||
{ org: '5560125790', n: 2 },
|
||||
],
|
||||
bankgiro: [{ value: '51108348', n: 4, first_seen: '2026-01-01', last_seen: '2026-04-01' }],
|
||||
}),
|
||||
],
|
||||
existing: [],
|
||||
})
|
||||
const item = r.items[0]!
|
||||
expect(item.org_number).toBeUndefined()
|
||||
expect(item.identities).toEqual([])
|
||||
expect(item.reason.ambiguous_orgs).toEqual([ORG, '5560125790'])
|
||||
})
|
||||
|
||||
it('attaches to an existing party by org number, then by exact alias key, never by name', () => {
|
||||
const byOrg: ExistingParty = { id: 'p-org', display_name: 'Beijer AB', org_number: ORG, alias_keys: [], status: 'confirmed' }
|
||||
const byAlias: ExistingParty = { id: 'p-alias', display_name: 'Loopia', org_number: null, alias_keys: ['loopia'], status: 'suggested' }
|
||||
const lookalike: ExistingParty = { id: 'p-fortnox', display_name: 'Fortnox AB', org_number: '5566661012', alias_keys: [], status: 'confirmed' }
|
||||
const r = buildSuggestions({
|
||||
observed: [observed({ key: 'beijer byggmaterial' }), observed({ key: 'loopia' }), observed({ key: 'fortnox finans' })],
|
||||
evidence: [evidence({ key: 'beijer byggmaterial', docs: 1, orgs: [{ org: ORG, n: 1 }] })],
|
||||
existing: [byOrg, byAlias, lookalike],
|
||||
})
|
||||
const [beijer, loopia, fortnox] = r.items
|
||||
expect(beijer!.party_id).toBe('p-org')
|
||||
expect(beijer!.reason.attach).toBe('org_number')
|
||||
expect(loopia!.party_id).toBe('p-alias')
|
||||
expect(loopia!.reason.attach).toBe('alias_key')
|
||||
// Same trade name is a question for a person, not a merge.
|
||||
expect(fortnox!.party_id).toBeUndefined()
|
||||
expect(fortnox!.reason.attach).toBe('new')
|
||||
expect(fortnox!.reason.similar_to).toBeUndefined()
|
||||
})
|
||||
|
||||
it('reports same-core live parties as similar_to on new suggestions', () => {
|
||||
const existing: ExistingParty = { id: 'p1', display_name: 'Levfakt Beijer Byggmaterial AB 2089', org_number: null, alias_keys: [], status: 'suggested' }
|
||||
const r = buildSuggestions({ observed: [observed({ key: 'beijer byggmaterial' })], evidence: [], existing: [existing] })
|
||||
expect(r.items[0]!.party_id).toBeUndefined()
|
||||
expect(r.items[0]!.reason.similar_to).toEqual([{ party_id: 'p1', display_name: 'Levfakt Beijer Byggmaterial AB 2089' }])
|
||||
})
|
||||
})
|
||||
|
||||
describe('suggestPartiesForCompany', () => {
|
||||
function stubClient(opts: { observed: unknown[]; evidence: unknown[]; existing: unknown[]; apply: unknown }) {
|
||||
const rpc = vi.fn(async (name: string, _args?: Record<string, unknown>) => {
|
||||
if (name === 'get_observed_parties') return { data: opts.observed, error: null }
|
||||
if (name === 'get_ledger_key_evidence') return { data: opts.evidence, error: null }
|
||||
if (name === 'apply_party_suggestions') return { data: opts.apply, error: null }
|
||||
return { data: null, error: { message: `unexpected rpc ${name}` } }
|
||||
})
|
||||
const range = vi.fn(async () => ({ data: opts.existing, error: null }))
|
||||
const chain: Record<string, unknown> = {}
|
||||
for (const m of ['select', 'eq', 'is', 'order']) chain[m] = vi.fn(() => chain)
|
||||
chain.range = range
|
||||
const from = vi.fn(() => chain)
|
||||
return { client: { rpc, from } as never, rpc, from }
|
||||
}
|
||||
|
||||
it('runs observed -> evidence -> existing -> apply and sums the RPC summary', async () => {
|
||||
const { client, rpc } = stubClient({
|
||||
observed: [
|
||||
{ key: 'beijer byggmaterial', name: 'BEIJER', variants: [], variant_count: 1, occurrences: 3, expense_sek: 3000, revenue_sek: 0, first_seen: '2026-01-10', last_seen: '2026-03-10', cadence_days: 30, dominant_account_number: '4000', dominant_account_share: 0.6, dominant_account_count: 2, dominant_account_total: 3 },
|
||||
{ key: 'inköp av varor', name: 'Inköp av varor', variants: [], variant_count: 1, occurrences: 1, expense_sek: 300, revenue_sek: 0, first_seen: '2026-03-15', last_seen: '2026-03-15', cadence_days: null, dominant_account_number: '4010', dominant_account_share: 0.5, dominant_account_count: 1, dominant_account_total: 1 },
|
||||
],
|
||||
evidence: [],
|
||||
existing: [],
|
||||
apply: { created: 1, attached: 0, identities: 0, facts: 2 },
|
||||
})
|
||||
const summary = await suggestPartiesForCompany(client, 'co', 'user')
|
||||
expect(summary).toEqual({ observed: 2, suggested: 1, skipped: 1, created: 1, attached: 0, identities: 0, facts: 2 })
|
||||
const applyCall = rpc.mock.calls.find((c) => c[0] === 'apply_party_suggestions')!
|
||||
const args = applyCall[1] as unknown as { p_company_id: string; p_user_id: string; p_items: Array<{ key: string }> }
|
||||
expect(args.p_company_id).toBe('co')
|
||||
expect(args.p_user_id).toBe('user')
|
||||
expect(args.p_items.map((i) => i.key)).toEqual(['beijer byggmaterial'])
|
||||
})
|
||||
|
||||
it('does not call apply when nothing is a party', async () => {
|
||||
const { client, rpc } = stubClient({ observed: [], evidence: [], existing: [], apply: null })
|
||||
const summary = await suggestPartiesForCompany(client, 'co', 'user')
|
||||
expect(summary.suggested).toBe(0)
|
||||
expect(rpc.mock.calls.map((c) => c[0])).not.toContain('apply_party_suggestions')
|
||||
})
|
||||
|
||||
it('surfaces RPC errors', async () => {
|
||||
const rpc = vi.fn(async () => ({ data: null, error: { message: 'boom' } }))
|
||||
await expect(suggestPartiesForCompany({ rpc } as never, 'co', 'user')).rejects.toThrow(/get_observed_parties failed: boom/)
|
||||
})
|
||||
})
|
||||
@@ -31,3 +31,26 @@ export function ledgerKey(raw: string | null | undefined): string {
|
||||
.trim()
|
||||
return stripped === '' ? k : stripped
|
||||
}
|
||||
|
||||
const CORE_AP_PREFIX = /^(levfakt|levfkt|lev\.?fakt\.?|leverantörsfaktura från|leverantörsfaktura|levbet\.?|kvitto|faktura|utgift|inköp)\s+/
|
||||
const CORE_LEGAL_FORM = /\b(ab|aktiebolag|hb|kb|sverige|sweden|ltd|limited|oy|gmbh|inc|sarl|publ|filial)\b/g
|
||||
|
||||
/**
|
||||
* The "core" of a key: what is left when AP prefixes, digit runs and legal
|
||||
* form suffixes are gone. Two keys with one core are the same trade name,
|
||||
* which is NOT the same party (Fortnox AB and Fortnox Finans AB share one),
|
||||
* so the core only ever ranks or annotates candidates; it never merges.
|
||||
* Measured on the document-anchored gold set 2026-09-02: pair precision
|
||||
* 0.909, recall 0.776 (scripts/parties/README.md).
|
||||
*/
|
||||
export function coreKey(key: string): string {
|
||||
return key
|
||||
.toLowerCase()
|
||||
.replace(CORE_AP_PREFIX, '')
|
||||
.replace(/\b\d+\b/g, '')
|
||||
.replace(CORE_LEGAL_FORM, '')
|
||||
.replace(/[^a-zåäöé ]+/g, ' ')
|
||||
.split(/\s+/)
|
||||
.filter(Boolean)
|
||||
.join(' ')
|
||||
}
|
||||
|
||||
@@ -0,0 +1,290 @@
|
||||
/**
|
||||
* Parties, phase 1c: the suggestion pipeline.
|
||||
*
|
||||
* Turns what the ledger already knows about a company into suggested parties
|
||||
* so a migrant's register is full on arrival. Inputs are the observed parties
|
||||
* (posted vouchers grouped by ledger key, get_observed_parties) and the hard
|
||||
* keys read from documents linked to those vouchers (get_ledger_key_evidence:
|
||||
* org number, VAT number, bankgiro, plusgiro, printed name). Output is a list
|
||||
* of items for apply_party_suggestions, which writes parties with status
|
||||
* 'suggested' and never merges on name: a key attaches to an existing party
|
||||
* only through its org number or because that exact key is already an alias.
|
||||
*
|
||||
* Keys that look alike (same core) are reported in the reason as
|
||||
* similar_to, so the queue can offer "same as X?" for a person to decide.
|
||||
* The model selection step is not wired here yet; it runs in shadow through
|
||||
* scripts/parties/eval-selection.ts until its decisions are labelled.
|
||||
*/
|
||||
import type { SupabaseClient } from '@supabase/supabase-js'
|
||||
import { fetchAllRows } from '@/lib/supabase/fetch-all'
|
||||
import { coreKey } from './ledger-key'
|
||||
import { getObservedParties, type ObservedParty } from './observed'
|
||||
|
||||
export interface IdentityEvidence {
|
||||
value: string
|
||||
n: number
|
||||
first_seen: string | null
|
||||
last_seen: string | null
|
||||
}
|
||||
|
||||
export interface LedgerKeyEvidence {
|
||||
key: string
|
||||
docs: number
|
||||
self_docs: number
|
||||
orgs: Array<{ org: string; n: number }>
|
||||
vat_numbers: Array<{ vat: string; n: number }>
|
||||
names: Array<{ name: string; n: number }>
|
||||
bankgiro: IdentityEvidence[]
|
||||
plusgiro: IdentityEvidence[]
|
||||
}
|
||||
|
||||
export interface ExistingParty {
|
||||
id: string
|
||||
display_name: string
|
||||
org_number: string | null
|
||||
alias_keys: string[]
|
||||
status: 'suggested' | 'confirmed'
|
||||
}
|
||||
|
||||
export interface SuggestionFact {
|
||||
field: string
|
||||
value: unknown
|
||||
source: 'ledger' | 'document'
|
||||
reference?: Record<string, unknown>
|
||||
}
|
||||
|
||||
export interface SuggestionIdentity {
|
||||
scheme: 'bankgiro' | 'plusgiro'
|
||||
value: string
|
||||
first_seen: string | null
|
||||
last_seen: string | null
|
||||
seen_count: number
|
||||
}
|
||||
|
||||
export interface SuggestionReason {
|
||||
/** How the key attaches, or why it becomes a new party. */
|
||||
attach: 'party_id' | 'org_number' | 'alias_key' | 'new'
|
||||
occurrences: number
|
||||
expense_sek: number
|
||||
revenue_sek: number
|
||||
first_seen: string
|
||||
last_seen: string
|
||||
docs: number
|
||||
/** Documents whose supplier org number is the company's own: sales side. */
|
||||
self_docs: number
|
||||
org_number?: string
|
||||
/** More than one org number seen under the key: hard key withheld. */
|
||||
ambiguous_orgs?: string[]
|
||||
dominant_account?: string | null
|
||||
/** Live parties with the same core: a merge question, never a merge. */
|
||||
similar_to?: Array<{ party_id: string; display_name: string }>
|
||||
}
|
||||
|
||||
export interface SuggestionItem {
|
||||
key: string
|
||||
display_name: string
|
||||
legal_name?: string
|
||||
kind: 'company'
|
||||
origin: 'ledger' | 'document'
|
||||
org_number?: string
|
||||
vat_number?: string
|
||||
party_id?: string
|
||||
alias_keys: string[]
|
||||
reason: SuggestionReason
|
||||
facts: SuggestionFact[]
|
||||
identities: SuggestionIdentity[]
|
||||
}
|
||||
|
||||
export interface SuggestionSkip {
|
||||
key: string
|
||||
label: ObservedParty['label']
|
||||
}
|
||||
|
||||
export interface BuildResult {
|
||||
items: SuggestionItem[]
|
||||
skipped: SuggestionSkip[]
|
||||
}
|
||||
|
||||
function pickName(observed: ObservedParty, evidence: LedgerKeyEvidence | undefined): { display: string; legal?: string } {
|
||||
// A printed supplier name from a document beats the voucher text, which is
|
||||
// upper-cased, truncated and prefixed by whatever the source system did.
|
||||
const printed = evidence?.names[0]?.name
|
||||
if (printed && printed.length >= 2) return { display: printed, legal: printed }
|
||||
return { display: observed.name || observed.key }
|
||||
}
|
||||
|
||||
function identitiesFrom(evidence: LedgerKeyEvidence | undefined): SuggestionIdentity[] {
|
||||
if (!evidence) return []
|
||||
const out: SuggestionIdentity[] = []
|
||||
for (const scheme of ['bankgiro', 'plusgiro'] as const) {
|
||||
for (const e of evidence[scheme]) {
|
||||
out.push({ scheme, value: e.value, first_seen: e.first_seen, last_seen: e.last_seen, seen_count: e.n })
|
||||
}
|
||||
}
|
||||
return out
|
||||
}
|
||||
|
||||
/**
|
||||
* Pure: decide what each observed party key becomes. Only keys the
|
||||
* pre-classifier calls 'party' continue; the rest are returned as skipped so
|
||||
* callers can show why "Inköp av varor" is not a supplier.
|
||||
*/
|
||||
export function buildSuggestions(input: {
|
||||
observed: ObservedParty[]
|
||||
evidence: LedgerKeyEvidence[]
|
||||
existing: ExistingParty[]
|
||||
}): BuildResult {
|
||||
const evidenceByKey = new Map(input.evidence.map((e) => [e.key, e]))
|
||||
const byOrg = new Map<string, ExistingParty>()
|
||||
const byAlias = new Map<string, ExistingParty>()
|
||||
const byCore = new Map<string, ExistingParty[]>()
|
||||
for (const p of input.existing) {
|
||||
if (p.org_number && !byOrg.has(p.org_number)) byOrg.set(p.org_number, p)
|
||||
for (const a of p.alias_keys) if (!byAlias.has(a)) byAlias.set(a, p)
|
||||
const c = coreKey(p.display_name)
|
||||
if (c) byCore.set(c, [...(byCore.get(c) ?? []), p])
|
||||
for (const a of p.alias_keys) {
|
||||
const ac = coreKey(a)
|
||||
if (ac && ac !== c) byCore.set(ac, [...(byCore.get(ac) ?? []), p])
|
||||
}
|
||||
}
|
||||
|
||||
const items: SuggestionItem[] = []
|
||||
const skipped: SuggestionSkip[] = []
|
||||
for (const o of input.observed) {
|
||||
if (o.label !== 'party') {
|
||||
skipped.push({ key: o.key, label: o.label })
|
||||
continue
|
||||
}
|
||||
const ev = evidenceByKey.get(o.key)
|
||||
const orgs = ev?.orgs ?? []
|
||||
const org = orgs.length === 1 ? orgs[0]!.org : undefined
|
||||
const existing = (org && byOrg.get(org)) || byAlias.get(o.key) || undefined
|
||||
const name = pickName(o, ev)
|
||||
const reason: SuggestionReason = {
|
||||
attach: existing ? (org && byOrg.get(org) === existing ? 'org_number' : 'alias_key') : 'new',
|
||||
occurrences: o.occurrences,
|
||||
expense_sek: o.expense_sek,
|
||||
revenue_sek: o.revenue_sek,
|
||||
first_seen: o.first_seen,
|
||||
last_seen: o.last_seen,
|
||||
docs: ev?.docs ?? 0,
|
||||
self_docs: ev?.self_docs ?? 0,
|
||||
dominant_account: o.dominant_account_number,
|
||||
}
|
||||
if (org) reason.org_number = org
|
||||
if (orgs.length > 1) reason.ambiguous_orgs = orgs.map((x) => x.org)
|
||||
if (!existing) {
|
||||
const similar = (byCore.get(coreKey(o.key)) ?? []).filter((p) => !org || p.org_number !== org)
|
||||
if (similar.length) reason.similar_to = similar.slice(0, 6).map((p) => ({ party_id: p.id, display_name: p.display_name }))
|
||||
}
|
||||
|
||||
const facts: SuggestionFact[] = []
|
||||
if (o.dominant_account_number) {
|
||||
facts.push({
|
||||
field: 'dominant_account',
|
||||
value: { account: o.dominant_account_number, share: o.dominant_account_share, count: o.dominant_account_count },
|
||||
source: 'ledger',
|
||||
reference: { occurrences: o.occurrences, first_seen: o.first_seen, last_seen: o.last_seen },
|
||||
})
|
||||
}
|
||||
if (o.cadence_days != null) {
|
||||
facts.push({ field: 'cadence_days', value: o.cadence_days, source: 'ledger', reference: { occurrences: o.occurrences } })
|
||||
}
|
||||
if (org) {
|
||||
facts.push({ field: 'org_number', value: org, source: 'document', reference: { docs: orgs[0]!.n } })
|
||||
}
|
||||
if (name.legal) {
|
||||
facts.push({ field: 'legal_name', value: name.legal, source: 'document', reference: { docs: ev?.names[0]?.n ?? 0 } })
|
||||
}
|
||||
|
||||
// Identities only when the hard key is unambiguous: a key that mixes two
|
||||
// org numbers would otherwise attach one supplier's bankgiro to another.
|
||||
const identities = orgs.length > 1 ? [] : identitiesFrom(ev)
|
||||
const vat = ev?.vat_numbers[0]?.vat
|
||||
|
||||
items.push({
|
||||
key: o.key,
|
||||
display_name: name.display,
|
||||
...(name.legal ? { legal_name: name.legal } : {}),
|
||||
kind: 'company',
|
||||
origin: org ? 'document' : 'ledger',
|
||||
...(org ? { org_number: org } : {}),
|
||||
...(vat && orgs.length <= 1 ? { vat_number: vat } : {}),
|
||||
...(existing ? { party_id: existing.id } : {}),
|
||||
alias_keys: [o.key],
|
||||
reason,
|
||||
facts,
|
||||
identities,
|
||||
})
|
||||
}
|
||||
return { items, skipped }
|
||||
}
|
||||
|
||||
export interface SuggestSummary {
|
||||
observed: number
|
||||
suggested: number
|
||||
skipped: number
|
||||
created: number
|
||||
attached: number
|
||||
identities: number
|
||||
facts: number
|
||||
}
|
||||
|
||||
/**
|
||||
* Run the pipeline for one company and persist the result. Safe to re-run:
|
||||
* apply_party_suggestions is idempotent.
|
||||
*/
|
||||
export async function suggestPartiesForCompany(
|
||||
supabase: SupabaseClient,
|
||||
companyId: string,
|
||||
userId: string,
|
||||
options: { fromDate?: string | null; limit?: number; chunkSize?: number } = {},
|
||||
): Promise<SuggestSummary> {
|
||||
const observed = await getObservedParties(supabase, companyId, {
|
||||
fromDate: options.fromDate ?? null,
|
||||
limit: options.limit ?? 5000,
|
||||
})
|
||||
const { data: evidenceData, error: evidenceError } = await supabase.rpc('get_ledger_key_evidence', {
|
||||
p_company_id: companyId,
|
||||
})
|
||||
if (evidenceError) throw new Error(`get_ledger_key_evidence failed: ${evidenceError.message}`)
|
||||
const evidence = (Array.isArray(evidenceData) ? evidenceData : []) as LedgerKeyEvidence[]
|
||||
|
||||
const existing = await fetchAllRows<ExistingParty>(({ from, to }) =>
|
||||
supabase
|
||||
.from('parties')
|
||||
.select('id, display_name, org_number, alias_keys, status')
|
||||
.eq('company_id', companyId)
|
||||
.is('merged_into', null)
|
||||
.is('archived_at', null)
|
||||
.order('created_at', { ascending: true })
|
||||
.range(from, to),
|
||||
)
|
||||
|
||||
const { items, skipped } = buildSuggestions({ observed, evidence, existing })
|
||||
const summary: SuggestSummary = {
|
||||
observed: observed.length,
|
||||
suggested: items.length,
|
||||
skipped: skipped.length,
|
||||
created: 0,
|
||||
attached: 0,
|
||||
identities: 0,
|
||||
facts: 0,
|
||||
}
|
||||
const chunk = Math.max(1, options.chunkSize ?? 200)
|
||||
for (let i = 0; i < items.length; i += chunk) {
|
||||
const { data, error } = await supabase.rpc('apply_party_suggestions', {
|
||||
p_company_id: companyId,
|
||||
p_user_id: userId,
|
||||
p_items: items.slice(i, i + chunk),
|
||||
})
|
||||
if (error) throw new Error(`apply_party_suggestions failed: ${error.message}`)
|
||||
const r = (data ?? {}) as Partial<Record<'created' | 'attached' | 'identities' | 'facts', number>>
|
||||
summary.created += r.created ?? 0
|
||||
summary.attached += r.attached ?? 0
|
||||
summary.identities += r.identities ?? 0
|
||||
summary.facts += r.facts ?? 0
|
||||
}
|
||||
return summary
|
||||
}
|
||||
@@ -0,0 +1,331 @@
|
||||
-- Parties, phase 1c: the suggestion pipeline.
|
||||
--
|
||||
-- A migrant's register is filled from what the ledger already knows: posted
|
||||
-- vouchers keyed on ledger_key(description) (get_observed_parties), plus the
|
||||
-- documents linked to those vouchers, whose OCR-read supplier block carries
|
||||
-- the hard keys (org number, VAT number, bankgiro, plusgiro). Nothing here is
|
||||
-- shown as a fact: every party this pipeline writes has status 'suggested'
|
||||
-- and waits for a person to confirm or dismiss it. Text similarity never
|
||||
-- merges anything; only an org number or an exact ledger key attaches a key
|
||||
-- to an existing party (DECISIONS 2026-09-02, dedupe on org only).
|
||||
--
|
||||
-- Three functions, all SECURITY INVOKER so RLS on parties/party_* applies to
|
||||
-- authenticated callers; the service role bypasses RLS as it does everywhere.
|
||||
-- get_ledger_key_evidence(company) read: hard keys per ledger key
|
||||
-- apply_party_suggestions(company, user, items) write: upsert suggestions
|
||||
-- decide_parties(company, user, ids, kind, note) write: bulk confirm/dismiss
|
||||
|
||||
-- Why the queue can show a reason per row without a join.
|
||||
ALTER TABLE public.parties ADD COLUMN IF NOT EXISTS suggested_reason jsonb;
|
||||
COMMENT ON COLUMN public.parties.suggested_reason IS
|
||||
'Why the pipeline suggested this party (evidence summary). NULL once confirmed by a person.';
|
||||
|
||||
-- ── Evidence per ledger key ─────────────────────────────────────────────────
|
||||
-- Same voucher population as get_observed_parties, joined to current-version
|
||||
-- documents whose extracted supplier block is an object. Documents whose
|
||||
-- supplier org number is the company's own are the company's sales invoices
|
||||
-- uploaded as underlag (40% of org-bearing documents fleet-wide); they are
|
||||
-- counted in self_docs and contribute nothing else.
|
||||
CREATE OR REPLACE FUNCTION public.get_ledger_key_evidence(p_company_id uuid)
|
||||
RETURNS jsonb
|
||||
LANGUAGE sql
|
||||
STABLE
|
||||
SECURITY INVOKER
|
||||
SET search_path TO 'public'
|
||||
AS $$
|
||||
WITH own AS (
|
||||
SELECT public.normalize_org_number(c.org_number) AS org
|
||||
FROM public.companies c WHERE c.id = p_company_id
|
||||
),
|
||||
entries AS (
|
||||
SELECT je.id, je.entry_date, public.ledger_key(je.description) AS k
|
||||
FROM public.journal_entries je
|
||||
WHERE je.company_id = p_company_id
|
||||
AND je.status = 'posted'
|
||||
AND je.source_type NOT IN ('storno', 'opening_balance', 'year_end', 'vat_settlement')
|
||||
AND je.description IS NOT NULL
|
||||
AND btrim(je.description) <> ''
|
||||
AND NOT EXISTS (
|
||||
SELECT 1 FROM public.transactions t
|
||||
WHERE t.journal_entry_id = je.id
|
||||
AND t.merchant_name IS NOT NULL
|
||||
AND btrim(t.merchant_name) <> ''
|
||||
)
|
||||
),
|
||||
docs AS (
|
||||
SELECT e.k, e.entry_date, d.id AS document_id,
|
||||
public.normalize_org_number(d.extracted_data->'supplier'->>'orgNumber') AS org,
|
||||
nullif(upper(regexp_replace(coalesce(d.extracted_data->'supplier'->>'vatNumber', ''), '[^0-9A-Za-z]', '', 'g')), '') AS vat,
|
||||
nullif(regexp_replace(coalesce(d.extracted_data->'supplier'->>'bankgiro', ''), '[^0-9]', '', 'g'), '') AS bankgiro,
|
||||
nullif(regexp_replace(coalesce(d.extracted_data->'supplier'->>'plusgiro', ''), '[^0-9]', '', 'g'), '') AS plusgiro,
|
||||
nullif(btrim(d.extracted_data->'supplier'->>'name'), '') AS name
|
||||
FROM entries e
|
||||
JOIN public.document_attachments d
|
||||
ON d.journal_entry_id = e.id
|
||||
AND d.company_id = p_company_id
|
||||
AND d.is_current_version
|
||||
AND jsonb_typeof(d.extracted_data->'supplier') = 'object'
|
||||
WHERE e.k <> ''
|
||||
),
|
||||
classified AS (
|
||||
SELECT d.*, (d.org IS NOT NULL AND d.org = own.org) AS is_self
|
||||
FROM docs d CROSS JOIN own
|
||||
),
|
||||
useful AS (SELECT * FROM classified WHERE NOT is_self),
|
||||
orgs AS (
|
||||
SELECT k, org, count(*) AS n FROM useful WHERE org IS NOT NULL GROUP BY k, org
|
||||
),
|
||||
vats AS (
|
||||
SELECT k, vat, count(*) AS n FROM useful WHERE vat IS NOT NULL GROUP BY k, vat
|
||||
),
|
||||
names AS (
|
||||
SELECT k, name, count(*) AS n FROM useful WHERE name IS NOT NULL GROUP BY k, name
|
||||
),
|
||||
bg AS (
|
||||
SELECT k, bankgiro AS value, count(*) AS n, min(entry_date) AS first_seen, max(entry_date) AS last_seen
|
||||
FROM useful WHERE bankgiro IS NOT NULL AND length(bankgiro) BETWEEN 7 AND 8 GROUP BY k, bankgiro
|
||||
),
|
||||
pg AS (
|
||||
SELECT k, plusgiro AS value, count(*) AS n, min(entry_date) AS first_seen, max(entry_date) AS last_seen
|
||||
FROM useful WHERE plusgiro IS NOT NULL AND length(plusgiro) BETWEEN 5 AND 8 GROUP BY k, plusgiro
|
||||
),
|
||||
per_key AS (
|
||||
SELECT c.k,
|
||||
count(*) AS docs,
|
||||
count(*) FILTER (WHERE c.is_self) AS self_docs
|
||||
FROM classified c GROUP BY c.k
|
||||
)
|
||||
SELECT coalesce(jsonb_agg(jsonb_build_object(
|
||||
'key', p.k,
|
||||
'docs', p.docs,
|
||||
'self_docs', p.self_docs,
|
||||
'orgs', coalesce((SELECT jsonb_agg(jsonb_build_object('org', o.org, 'n', o.n) ORDER BY o.n DESC, o.org) FROM orgs o WHERE o.k = p.k), '[]'::jsonb),
|
||||
'vat_numbers', coalesce((SELECT jsonb_agg(jsonb_build_object('vat', v.vat, 'n', v.n) ORDER BY v.n DESC, v.vat) FROM vats v WHERE v.k = p.k), '[]'::jsonb),
|
||||
'names', coalesce((SELECT jsonb_agg(jsonb_build_object('name', x.name, 'n', x.n) ORDER BY x.n DESC, x.name) FROM names x WHERE x.k = p.k), '[]'::jsonb),
|
||||
'bankgiro', coalesce((SELECT jsonb_agg(jsonb_build_object('value', b.value, 'n', b.n, 'first_seen', b.first_seen, 'last_seen', b.last_seen) ORDER BY b.n DESC, b.value) FROM bg b WHERE b.k = p.k), '[]'::jsonb),
|
||||
'plusgiro', coalesce((SELECT jsonb_agg(jsonb_build_object('value', g.value, 'n', g.n, 'first_seen', g.first_seen, 'last_seen', g.last_seen) ORDER BY g.n DESC, g.value) FROM pg g WHERE g.k = p.k), '[]'::jsonb)
|
||||
) ORDER BY p.docs DESC, p.k), '[]'::jsonb)
|
||||
FROM per_key p;
|
||||
$$;
|
||||
|
||||
REVOKE ALL ON FUNCTION public.get_ledger_key_evidence(uuid) FROM PUBLIC, anon;
|
||||
GRANT EXECUTE ON FUNCTION public.get_ledger_key_evidence(uuid) TO authenticated, service_role;
|
||||
COMMENT ON FUNCTION public.get_ledger_key_evidence(uuid) IS
|
||||
'Hard keys (org, VAT, bankgiro, plusgiro, printed name) per ledger_key from documents linked to posted vouchers. Self-extracted documents (supplier org = own org) count only in self_docs.';
|
||||
|
||||
-- ── Apply suggestions ───────────────────────────────────────────────────────
|
||||
-- p_items: array of
|
||||
-- { key, display_name, kind?, origin?, org_number?, vat_number?, alias_keys?,
|
||||
-- party_id?, reason?, facts?: [{field, value, source, reference?}],
|
||||
-- identities?: [{scheme, value, first_seen?, last_seen?, seen_count?}] }
|
||||
-- Attach order: explicit party_id, then live party with the same org number,
|
||||
-- then live party whose alias_keys already contains the key; otherwise insert
|
||||
-- a suggested party. Never by name. Re-running is idempotent.
|
||||
CREATE OR REPLACE FUNCTION public.apply_party_suggestions(
|
||||
p_company_id uuid,
|
||||
p_user_id uuid,
|
||||
p_items jsonb
|
||||
)
|
||||
RETURNS jsonb
|
||||
LANGUAGE plpgsql
|
||||
SECURITY INVOKER
|
||||
SET search_path TO 'public'
|
||||
AS $$
|
||||
DECLARE
|
||||
v_item jsonb;
|
||||
v_fact jsonb;
|
||||
v_ident jsonb;
|
||||
v_party_id uuid;
|
||||
v_key text;
|
||||
v_org text;
|
||||
v_aliases text[];
|
||||
v_created integer := 0;
|
||||
v_attached integer := 0;
|
||||
v_identities integer := 0;
|
||||
v_facts integer := 0;
|
||||
v_seen integer;
|
||||
BEGIN
|
||||
IF auth.uid() IS NOT NULL AND auth.uid() <> p_user_id THEN
|
||||
RAISE EXCEPTION 'apply_party_suggestions: p_user_id must be the caller' USING ERRCODE = '42501';
|
||||
END IF;
|
||||
IF p_items IS NULL OR jsonb_typeof(p_items) <> 'array' THEN
|
||||
RAISE EXCEPTION 'apply_party_suggestions: p_items must be a JSON array' USING ERRCODE = '22023';
|
||||
END IF;
|
||||
|
||||
FOR v_item IN SELECT * FROM jsonb_array_elements(p_items) LOOP
|
||||
v_key := nullif(btrim(coalesce(v_item->>'key', '')), '');
|
||||
IF v_key IS NULL THEN
|
||||
RAISE EXCEPTION 'apply_party_suggestions: every item needs a key' USING ERRCODE = '22023';
|
||||
END IF;
|
||||
v_org := public.normalize_org_number(v_item->>'org_number');
|
||||
v_aliases := ARRAY(SELECT DISTINCT x FROM (
|
||||
SELECT v_key AS x UNION ALL SELECT jsonb_array_elements_text(coalesce(v_item->'alias_keys', '[]'::jsonb))
|
||||
) a WHERE x IS NOT NULL AND btrim(x) <> '');
|
||||
|
||||
v_party_id := NULL;
|
||||
IF v_item->>'party_id' IS NOT NULL THEN
|
||||
SELECT id INTO v_party_id FROM public.parties
|
||||
WHERE id = (v_item->>'party_id')::uuid AND company_id = p_company_id AND merged_into IS NULL;
|
||||
IF v_party_id IS NULL THEN
|
||||
RAISE EXCEPTION 'apply_party_suggestions: party % is not a live party of this company', v_item->>'party_id'
|
||||
USING ERRCODE = '23503';
|
||||
END IF;
|
||||
END IF;
|
||||
IF v_party_id IS NULL AND v_org IS NOT NULL THEN
|
||||
SELECT id INTO v_party_id FROM public.parties
|
||||
WHERE company_id = p_company_id AND org_number = v_org AND merged_into IS NULL;
|
||||
END IF;
|
||||
IF v_party_id IS NULL THEN
|
||||
SELECT id INTO v_party_id FROM public.parties
|
||||
WHERE company_id = p_company_id AND merged_into IS NULL AND alias_keys @> ARRAY[v_key]
|
||||
ORDER BY (status = 'confirmed') DESC, created_at
|
||||
LIMIT 1;
|
||||
END IF;
|
||||
|
||||
IF v_party_id IS NULL THEN
|
||||
INSERT INTO public.parties (company_id, user_id, display_name, legal_name, kind, status, org_number, vat_number, alias_keys, origin, suggested_reason)
|
||||
VALUES (
|
||||
p_company_id, p_user_id,
|
||||
coalesce(nullif(btrim(v_item->>'display_name'), ''), v_key),
|
||||
nullif(btrim(v_item->>'legal_name'), ''),
|
||||
coalesce(v_item->>'kind', 'company'),
|
||||
'suggested',
|
||||
v_org,
|
||||
nullif(btrim(v_item->>'vat_number'), ''),
|
||||
v_aliases,
|
||||
coalesce(v_item->>'origin', 'ledger'),
|
||||
v_item->'reason'
|
||||
)
|
||||
ON CONFLICT DO NOTHING
|
||||
RETURNING id INTO v_party_id;
|
||||
IF v_party_id IS NULL THEN
|
||||
-- Lost a race on (company_id, org_number): attach to the winner.
|
||||
SELECT id INTO v_party_id FROM public.parties
|
||||
WHERE company_id = p_company_id AND org_number = v_org AND merged_into IS NULL;
|
||||
v_attached := v_attached + 1;
|
||||
ELSE
|
||||
v_created := v_created + 1;
|
||||
END IF;
|
||||
ELSE
|
||||
v_attached := v_attached + 1;
|
||||
UPDATE public.parties
|
||||
SET alias_keys = ARRAY(SELECT DISTINCT x FROM unnest(alias_keys || v_aliases) AS x),
|
||||
vat_number = coalesce(vat_number, nullif(btrim(v_item->>'vat_number'), '')),
|
||||
legal_name = coalesce(legal_name, nullif(btrim(v_item->>'legal_name'), '')),
|
||||
org_number = coalesce(org_number, v_org)
|
||||
WHERE id = v_party_id
|
||||
AND (NOT (alias_keys @> v_aliases)
|
||||
OR (vat_number IS NULL AND nullif(btrim(v_item->>'vat_number'), '') IS NOT NULL)
|
||||
OR (legal_name IS NULL AND nullif(btrim(v_item->>'legal_name'), '') IS NOT NULL)
|
||||
OR (org_number IS NULL AND v_org IS NOT NULL));
|
||||
END IF;
|
||||
|
||||
FOR v_ident IN SELECT * FROM jsonb_array_elements(coalesce(v_item->'identities', '[]'::jsonb)) LOOP
|
||||
v_seen := greatest(coalesce((v_ident->>'seen_count')::integer, 1), 1);
|
||||
INSERT INTO public.party_identities (party_id, company_id, user_id, scheme, value, status, source, first_seen, last_seen, seen_count)
|
||||
VALUES (
|
||||
v_party_id, p_company_id, p_user_id,
|
||||
v_ident->>'scheme', v_ident->>'value',
|
||||
CASE WHEN v_seen >= 2 THEN 'known' ELSE 'unverified' END,
|
||||
coalesce(v_ident->>'source', 'document'),
|
||||
(v_ident->>'first_seen')::date, (v_ident->>'last_seen')::date, v_seen
|
||||
)
|
||||
ON CONFLICT (party_id, scheme, value) DO UPDATE
|
||||
SET seen_count = greatest(party_identities.seen_count, EXCLUDED.seen_count),
|
||||
first_seen = least(party_identities.first_seen, EXCLUDED.first_seen),
|
||||
last_seen = greatest(party_identities.last_seen, EXCLUDED.last_seen),
|
||||
status = CASE WHEN greatest(party_identities.seen_count, EXCLUDED.seen_count) >= 2 THEN 'known' ELSE party_identities.status END
|
||||
WHERE party_identities.seen_count < EXCLUDED.seen_count
|
||||
OR party_identities.last_seen IS DISTINCT FROM greatest(party_identities.last_seen, EXCLUDED.last_seen)
|
||||
OR party_identities.first_seen IS DISTINCT FROM least(party_identities.first_seen, EXCLUDED.first_seen);
|
||||
v_identities := v_identities + 1;
|
||||
END LOOP;
|
||||
|
||||
FOR v_fact IN SELECT * FROM jsonb_array_elements(coalesce(v_item->'facts', '[]'::jsonb)) LOOP
|
||||
INSERT INTO public.party_facts (party_id, company_id, user_id, field, value, source, reference)
|
||||
SELECT v_party_id, p_company_id, p_user_id, v_fact->>'field', v_fact->'value', v_fact->>'source', v_fact->'reference'
|
||||
WHERE NOT EXISTS (
|
||||
SELECT 1 FROM public.party_facts f
|
||||
WHERE f.party_id = v_party_id AND f.field = v_fact->>'field' AND f.source = v_fact->>'source'
|
||||
AND f.value = v_fact->'value' AND f.superseded_at IS NULL
|
||||
);
|
||||
IF FOUND THEN v_facts := v_facts + 1; END IF;
|
||||
END LOOP;
|
||||
END LOOP;
|
||||
|
||||
RETURN jsonb_build_object('created', v_created, 'attached', v_attached, 'identities', v_identities, 'facts', v_facts);
|
||||
END;
|
||||
$$;
|
||||
|
||||
REVOKE ALL ON FUNCTION public.apply_party_suggestions(uuid, uuid, jsonb) FROM PUBLIC, anon;
|
||||
GRANT EXECUTE ON FUNCTION public.apply_party_suggestions(uuid, uuid, jsonb) TO authenticated, service_role;
|
||||
COMMENT ON FUNCTION public.apply_party_suggestions(uuid, uuid, jsonb) IS
|
||||
'Upserts pipeline suggestions: attaches by explicit party_id, org number or exact ledger key, otherwise inserts a suggested party. Never merges on name. Idempotent.';
|
||||
|
||||
-- ── Decide: bulk confirm or dismiss ─────────────────────────────────────────
|
||||
CREATE OR REPLACE FUNCTION public.decide_parties(
|
||||
p_company_id uuid,
|
||||
p_user_id uuid,
|
||||
p_party_ids uuid[],
|
||||
p_kind text,
|
||||
p_note text DEFAULT NULL
|
||||
)
|
||||
RETURNS integer
|
||||
LANGUAGE plpgsql
|
||||
SECURITY INVOKER
|
||||
SET search_path TO 'public'
|
||||
AS $$
|
||||
DECLARE
|
||||
v_count integer := 0;
|
||||
BEGIN
|
||||
IF auth.uid() IS NOT NULL AND auth.uid() <> p_user_id THEN
|
||||
RAISE EXCEPTION 'decide_parties: p_user_id must be the caller' USING ERRCODE = '42501';
|
||||
END IF;
|
||||
IF p_kind NOT IN ('confirm', 'dismiss') THEN
|
||||
RAISE EXCEPTION 'decide_parties: kind must be confirm or dismiss, got %', p_kind USING ERRCODE = '22023';
|
||||
END IF;
|
||||
|
||||
IF p_kind = 'confirm' THEN
|
||||
WITH changed AS (
|
||||
UPDATE public.parties p
|
||||
SET status = 'confirmed', suggested_reason = NULL, archived_at = NULL
|
||||
WHERE p.company_id = p_company_id AND p.id = ANY(p_party_ids) AND p.merged_into IS NULL
|
||||
AND (p.status <> 'confirmed' OR p.archived_at IS NOT NULL)
|
||||
RETURNING p.id, p.display_name
|
||||
), logged AS (
|
||||
INSERT INTO public.party_decisions (party_id, company_id, user_id, kind, before, after, note)
|
||||
SELECT c.id, p_company_id, p_user_id, 'confirm',
|
||||
jsonb_build_object('status', 'suggested'), jsonb_build_object('status', 'confirmed'), p_note
|
||||
FROM changed c
|
||||
RETURNING 1
|
||||
)
|
||||
SELECT count(*) INTO v_count FROM logged;
|
||||
ELSE
|
||||
-- Dismiss is the queue's "not a party" answer: it only touches suggested
|
||||
-- rows. A confirmed party is archived through its own action later.
|
||||
WITH changed AS (
|
||||
UPDATE public.parties p
|
||||
SET archived_at = now()
|
||||
WHERE p.company_id = p_company_id AND p.id = ANY(p_party_ids) AND p.merged_into IS NULL
|
||||
AND p.status = 'suggested' AND p.archived_at IS NULL
|
||||
RETURNING p.id, p.status
|
||||
), logged AS (
|
||||
INSERT INTO public.party_decisions (party_id, company_id, user_id, kind, before, after, note)
|
||||
SELECT c.id, p_company_id, p_user_id, 'dismiss',
|
||||
jsonb_build_object('status', c.status, 'archived', false), jsonb_build_object('status', c.status, 'archived', true), p_note
|
||||
FROM changed c
|
||||
RETURNING 1
|
||||
)
|
||||
SELECT count(*) INTO v_count FROM logged;
|
||||
END IF;
|
||||
|
||||
RETURN v_count;
|
||||
END;
|
||||
$$;
|
||||
|
||||
REVOKE ALL ON FUNCTION public.decide_parties(uuid, uuid, uuid[], text, text) FROM PUBLIC, anon;
|
||||
GRANT EXECUTE ON FUNCTION public.decide_parties(uuid, uuid, uuid[], text, text) TO authenticated, service_role;
|
||||
COMMENT ON FUNCTION public.decide_parties(uuid, uuid, uuid[], text, text) IS
|
||||
'Bulk confirm (suggested -> confirmed) or dismiss (archive a suggested party), one party_decisions row each. Confirmed parties are never dismissed here. Merge and undo live in their own RPCs.';
|
||||
|
||||
NOTIFY pgrst, 'reload schema';
|
||||
@@ -0,0 +1,254 @@
|
||||
import { randomUUID } from 'node:crypto'
|
||||
import { describe, expect, it } from 'vitest'
|
||||
import { getPool, withUserContext } from './setup'
|
||||
import { insertPostedJournalEntry, seedCompany } from './fixtures'
|
||||
|
||||
const ORG = '5564300142'
|
||||
const OTHER_ORG = '5560125790'
|
||||
const LOOPIA_ORG = '5566661012'
|
||||
|
||||
const expense = (account: string, amount: number) => [
|
||||
{ accountNumber: account, debitAmount: amount, creditAmount: 0 },
|
||||
{ accountNumber: '2440', debitAmount: 0, creditAmount: amount },
|
||||
]
|
||||
|
||||
async function linkDocument(params: {
|
||||
companyId: string
|
||||
userId: string
|
||||
journalEntryId: string
|
||||
supplier: Record<string, string | null>
|
||||
}): Promise<string> {
|
||||
const id = randomUUID()
|
||||
await getPool().query(
|
||||
`INSERT INTO public.document_attachments (id, company_id, user_id, uploaded_by, storage_path, file_name, sha256_hash, mime_type, upload_source, journal_entry_id, extracted_data)
|
||||
VALUES ($1, $2, $3, $3, $4, 'f.pdf', md5($4), 'application/pdf', 'file_upload', $5, $6::jsonb)`,
|
||||
[id, params.companyId, params.userId, `docs/${id}.pdf`, params.journalEntryId, JSON.stringify({ supplier: params.supplier })],
|
||||
)
|
||||
return id
|
||||
}
|
||||
|
||||
interface Evidence {
|
||||
key: string
|
||||
docs: number
|
||||
self_docs: number
|
||||
orgs: Array<{ org: string; n: number }>
|
||||
vat_numbers: Array<{ vat: string; n: number }>
|
||||
names: Array<{ name: string; n: number }>
|
||||
bankgiro: Array<{ value: string; n: number; first_seen: string; last_seen: string }>
|
||||
plusgiro: Array<{ value: string; n: number }>
|
||||
}
|
||||
|
||||
async function evidence(companyId: string, userId: string): Promise<Evidence[]> {
|
||||
return withUserContext(userId, async (client) => {
|
||||
const { rows } = await client.query<{ r: Evidence[] }>(`SELECT public.get_ledger_key_evidence($1) AS r`, [companyId])
|
||||
return rows[0]!.r
|
||||
})
|
||||
}
|
||||
|
||||
// Writes run on the pool connection (service presentation: auth.uid() NULL,
|
||||
// committed). withUserContext rolls back and is used for RLS assertions only.
|
||||
async function apply(companyId: string, userId: string, items: unknown[]) {
|
||||
const { rows } = await getPool().query<{ r: Record<string, number> }>(
|
||||
`SELECT public.apply_party_suggestions($1, $2, $3::jsonb) AS r`,
|
||||
[companyId, userId, JSON.stringify(items)],
|
||||
)
|
||||
return rows[0]!.r
|
||||
}
|
||||
|
||||
async function decide(companyId: string, userId: string, ids: string[], kind: string, note: string | null = null): Promise<number> {
|
||||
const { rows } = await getPool().query<{ n: number }>(
|
||||
`SELECT public.decide_parties($1, $2, $3::uuid[], $4, $5) AS n`,
|
||||
[companyId, userId, ids, kind, note],
|
||||
)
|
||||
return rows[0]!.n
|
||||
}
|
||||
|
||||
describe('get_ledger_key_evidence (pg)', () => {
|
||||
it('aggregates hard keys per ledger key and counts own-org documents as self_docs only', async () => {
|
||||
const c = await seedCompany()
|
||||
await getPool().query(`UPDATE public.companies SET org_number = '556430-0142' WHERE id = $1`, [c.companyId])
|
||||
const base = { userId: c.userId, companyId: c.companyId, fiscalPeriodId: c.fiscalPeriodId, sourceType: 'import' }
|
||||
const e1 = await insertPostedJournalEntry({ ...base, entryDate: '2026-01-10', description: 'Levfakt Loopia AB (17)', lines: expense('6540', 100) })
|
||||
const e2 = await insertPostedJournalEntry({ ...base, entryDate: '2026-02-10', description: 'Levfakt Loopia AB (17)', lines: expense('6540', 100) })
|
||||
const e3 = await insertPostedJournalEntry({ ...base, entryDate: '2026-03-10', description: 'Levfakt Loopia AB (17)', lines: expense('6540', 100) })
|
||||
await linkDocument({ ...c, journalEntryId: e1, supplier: { name: 'Loopia AB', orgNumber: '556666-1012', vatNumber: 'SE556666101201', bankgiro: '5317-0900', plusgiro: null } })
|
||||
await linkDocument({ ...c, journalEntryId: e2, supplier: { name: 'Loopia AB', orgNumber: LOOPIA_ORG, vatNumber: 'SE556666101201', bankgiro: '53170900', plusgiro: null } })
|
||||
// Own sales invoice uploaded as underlag: the company's own org number.
|
||||
await linkDocument({ ...c, journalEntryId: e3, supplier: { name: 'Me AB', orgNumber: ORG, vatNumber: null, bankgiro: '11112222', plusgiro: null } })
|
||||
|
||||
const rows = await evidence(c.companyId, c.userId)
|
||||
const loopia = rows.find((r) => r.key === 'loopia')
|
||||
expect(loopia).toBeDefined()
|
||||
expect(loopia!.docs).toBe(3)
|
||||
expect(loopia!.self_docs).toBe(1)
|
||||
expect(loopia!.orgs).toEqual([{ org: LOOPIA_ORG, n: 2 }])
|
||||
expect(loopia!.vat_numbers).toEqual([{ vat: 'SE556666101201', n: 2 }])
|
||||
expect(loopia!.names).toEqual([{ name: 'Loopia AB', n: 2 }])
|
||||
expect(loopia!.bankgiro).toEqual([{ value: '53170900', n: 2, first_seen: '2026-01-10', last_seen: '2026-02-10' }])
|
||||
expect(loopia!.plusgiro).toEqual([])
|
||||
})
|
||||
|
||||
it('is invisible across companies', async () => {
|
||||
const mine = await seedCompany()
|
||||
const theirs = await seedCompany()
|
||||
const e = await insertPostedJournalEntry({ userId: theirs.userId, companyId: theirs.companyId, fiscalPeriodId: theirs.fiscalPeriodId, sourceType: 'import', description: 'Levfakt Loopia AB', lines: expense('6540', 100) })
|
||||
await linkDocument({ ...theirs, journalEntryId: e, supplier: { name: 'Loopia AB', orgNumber: LOOPIA_ORG, vatNumber: null, bankgiro: null, plusgiro: null } })
|
||||
expect(await evidence(theirs.companyId, mine.userId)).toEqual([])
|
||||
})
|
||||
})
|
||||
|
||||
describe('apply_party_suggestions (pg)', () => {
|
||||
it('inserts suggested parties, attaches by org and by alias key, and is idempotent', async () => {
|
||||
const c = await seedCompany()
|
||||
const item = (over: Record<string, unknown>) => ({
|
||||
key: 'loopia',
|
||||
display_name: 'Loopia AB',
|
||||
kind: 'company',
|
||||
origin: 'document',
|
||||
alias_keys: ['loopia'],
|
||||
reason: { attach: 'new', occurrences: 3 },
|
||||
facts: [{ field: 'dominant_account', value: { account: '6540' }, source: 'ledger' }],
|
||||
identities: [{ scheme: 'bankgiro', value: '53170900', first_seen: '2026-01-10', last_seen: '2026-02-10', seen_count: 1 }],
|
||||
...over,
|
||||
})
|
||||
|
||||
const first = await apply(c.companyId, c.userId, [item({ org_number: '556666-1012' })])
|
||||
expect(first).toEqual({ created: 1, attached: 0, identities: 1, facts: 1 })
|
||||
const party = await getPool().query<{ id: string; status: string; org_number: string; alias_keys: string[]; suggested_reason: { attach: string }; origin: string }>(
|
||||
`SELECT id, status, org_number, alias_keys, suggested_reason, origin FROM public.parties WHERE company_id = $1`,
|
||||
[c.companyId],
|
||||
)
|
||||
expect(party.rows).toHaveLength(1)
|
||||
expect(party.rows[0]).toMatchObject({ status: 'suggested', org_number: LOOPIA_ORG, alias_keys: ['loopia'], origin: 'document' })
|
||||
expect(party.rows[0]!.suggested_reason.attach).toBe('new')
|
||||
const partyId = party.rows[0]!.id
|
||||
|
||||
// Same org under another key: attaches, unions aliases, promotes the identity to known.
|
||||
const second = await apply(c.companyId, c.userId, [
|
||||
item({ key: 'loopia webbhotell', alias_keys: ['loopia webbhotell'], org_number: LOOPIA_ORG, identities: [{ scheme: 'bankgiro', value: '53170900', first_seen: '2026-03-10', last_seen: '2026-03-10', seen_count: 2 }] }),
|
||||
])
|
||||
expect(second).toEqual({ created: 0, attached: 1, identities: 1, facts: 0 })
|
||||
const after = await getPool().query<{ n: string; alias_keys: string[] }>(
|
||||
`SELECT (SELECT count(*) FROM public.parties WHERE company_id = $1)::text AS n, alias_keys FROM public.parties WHERE id = $2`,
|
||||
[c.companyId, partyId],
|
||||
)
|
||||
expect(after.rows[0]!.n).toBe('1')
|
||||
expect([...after.rows[0]!.alias_keys].sort()).toEqual(['loopia', 'loopia webbhotell'])
|
||||
const ident = await getPool().query<{ status: string; seen_count: number; first_seen: string; last_seen: string }>(
|
||||
`SELECT status, seen_count, first_seen::text, last_seen::text FROM public.party_identities WHERE party_id = $1`,
|
||||
[partyId],
|
||||
)
|
||||
expect(ident.rows).toEqual([{ status: 'known', seen_count: 2, first_seen: '2026-01-10', last_seen: '2026-03-10' }])
|
||||
|
||||
// Exact alias key without org: attaches too. Re-running creates nothing new.
|
||||
const third = await apply(c.companyId, c.userId, [item({ key: 'loopia webbhotell', alias_keys: ['loopia webbhotell'] })])
|
||||
expect(third).toMatchObject({ created: 0, attached: 1, facts: 0 })
|
||||
const facts = await getPool().query(`SELECT 1 FROM public.party_facts WHERE party_id = $1`, [partyId])
|
||||
expect(facts.rowCount).toBe(1)
|
||||
})
|
||||
|
||||
it('never merges on name: same core text becomes a second suggested party', async () => {
|
||||
const c = await seedCompany()
|
||||
await apply(c.companyId, c.userId, [{ key: 'fortnox', display_name: 'Fortnox AB', org_number: ORG }])
|
||||
await apply(c.companyId, c.userId, [{ key: 'fortnox finans', display_name: 'Fortnox Finans AB', org_number: OTHER_ORG }])
|
||||
await apply(c.companyId, c.userId, [{ key: 'fortnox ab', display_name: 'FORTNOX AB' }])
|
||||
const { rows } = await getPool().query<{ display_name: string }>(
|
||||
`SELECT display_name FROM public.parties WHERE company_id = $1 ORDER BY created_at`,
|
||||
[c.companyId],
|
||||
)
|
||||
expect(rows.map((r) => r.display_name)).toEqual(['Fortnox AB', 'Fortnox Finans AB', 'FORTNOX AB'])
|
||||
})
|
||||
|
||||
it('rejects an explicit party_id from another company and a spoofed user id', async () => {
|
||||
const mine = await seedCompany()
|
||||
const theirs = await seedCompany()
|
||||
const { rows } = await getPool().query<{ id: string }>(
|
||||
`INSERT INTO public.parties (company_id, user_id, display_name) VALUES ($1, $2, 'Theirs') RETURNING id`,
|
||||
[theirs.companyId, theirs.userId],
|
||||
)
|
||||
await expect(apply(mine.companyId, mine.userId, [{ key: 'x', display_name: 'X', party_id: rows[0]!.id }])).rejects.toMatchObject({ code: '23503' })
|
||||
await expect(
|
||||
withUserContext(mine.userId, (client) =>
|
||||
client.query(`SELECT public.apply_party_suggestions($1, $2, '[]'::jsonb)`, [mine.companyId, theirs.userId]),
|
||||
),
|
||||
).rejects.toMatchObject({ code: '42501' })
|
||||
// RLS: a member of another company cannot write into mine.
|
||||
await expect(
|
||||
withUserContext(theirs.userId, (client) =>
|
||||
client.query(`SELECT public.apply_party_suggestions($1, $2, $3::jsonb)`, [mine.companyId, theirs.userId, JSON.stringify([{ key: 'x', display_name: 'X' }])]),
|
||||
),
|
||||
).rejects.toMatchObject({ code: '42501' })
|
||||
})
|
||||
})
|
||||
|
||||
describe('decide_parties (pg)', () => {
|
||||
it('confirms and dismisses in bulk, logging one decision per party', async () => {
|
||||
const c = await seedCompany()
|
||||
await apply(c.companyId, c.userId, [
|
||||
{ key: 'loopia', display_name: 'Loopia AB', reason: { attach: 'new' } },
|
||||
{ key: 'beijer', display_name: 'Beijer AB', reason: { attach: 'new' } },
|
||||
{ key: 'noise', display_name: 'Noise', reason: { attach: 'new' } },
|
||||
])
|
||||
const ids = await getPool().query<{ id: string; display_name: string }>(
|
||||
`SELECT id, display_name FROM public.parties WHERE company_id = $1`,
|
||||
[c.companyId],
|
||||
)
|
||||
const byName = Object.fromEntries(ids.rows.map((r) => [r.display_name, r.id]))
|
||||
|
||||
const confirmed = await decide(c.companyId, c.userId, [byName['Loopia AB']!, byName['Beijer AB']!], 'confirm', 'bulk from queue')
|
||||
expect(confirmed).toBe(2)
|
||||
const dismissed = await decide(c.companyId, c.userId, [byName['Noise']!], 'dismiss')
|
||||
expect(dismissed).toBe(1)
|
||||
|
||||
const state = await getPool().query<{ display_name: string; status: string; archived: boolean; reason: unknown }>(
|
||||
`SELECT display_name, status, archived_at IS NOT NULL AS archived, suggested_reason AS reason FROM public.parties WHERE company_id = $1 ORDER BY display_name`,
|
||||
[c.companyId],
|
||||
)
|
||||
expect(state.rows).toEqual([
|
||||
{ display_name: 'Beijer AB', status: 'confirmed', archived: false, reason: null },
|
||||
{ display_name: 'Loopia AB', status: 'confirmed', archived: false, reason: null },
|
||||
{ display_name: 'Noise', status: 'suggested', archived: true, reason: { attach: 'new' } },
|
||||
])
|
||||
const decisions = await getPool().query<{ kind: string; n: string }>(
|
||||
`SELECT kind, count(*)::text AS n FROM public.party_decisions WHERE company_id = $1 GROUP BY kind ORDER BY kind`,
|
||||
[c.companyId],
|
||||
)
|
||||
expect(decisions.rows).toEqual([
|
||||
{ kind: 'confirm', n: '2' },
|
||||
{ kind: 'dismiss', n: '1' },
|
||||
])
|
||||
|
||||
// Second confirm of the same ids is a no-op: no duplicate decisions.
|
||||
const again = await decide(c.companyId, c.userId, [byName['Loopia AB']!], 'confirm')
|
||||
expect(again).toBe(0)
|
||||
// Dismiss never touches a confirmed party.
|
||||
const notDismissed = await decide(c.companyId, c.userId, [byName['Loopia AB']!], 'dismiss')
|
||||
expect(notDismissed).toBe(0)
|
||||
const loopia = await getPool().query<{ archived: boolean }>(`SELECT archived_at IS NOT NULL AS archived FROM public.parties WHERE id = $1`, [byName['Loopia AB']])
|
||||
expect(loopia.rows[0]!.archived).toBe(false)
|
||||
})
|
||||
|
||||
it('rejects unknown kinds and other companies', async () => {
|
||||
const mine = await seedCompany()
|
||||
const theirs = await seedCompany()
|
||||
await apply(theirs.companyId, theirs.userId, [{ key: 'loopia', display_name: 'Loopia AB' }])
|
||||
const { rows } = await getPool().query<{ id: string }>(`SELECT id FROM public.parties WHERE company_id = $1`, [theirs.companyId])
|
||||
await expect(
|
||||
withUserContext(mine.userId, (client) =>
|
||||
client.query(`SELECT public.decide_parties($1, $2, $3::uuid[], 'merge', NULL)`, [mine.companyId, mine.userId, [rows[0]!.id]]),
|
||||
),
|
||||
).rejects.toMatchObject({ code: '22023' })
|
||||
// Under RLS a member of another company sees no rows to update.
|
||||
const n = await withUserContext(mine.userId, async (client) => {
|
||||
const r = await client.query<{ n: number }>(`SELECT public.decide_parties($1, $2, $3::uuid[], 'confirm', NULL) AS n`, [
|
||||
theirs.companyId,
|
||||
mine.userId,
|
||||
[rows[0]!.id],
|
||||
])
|
||||
return r.rows[0]!.n
|
||||
})
|
||||
expect(n).toBe(0)
|
||||
const still = await getPool().query<{ status: string }>(`SELECT status FROM public.parties WHERE id = $1`, [rows[0]!.id])
|
||||
expect(still.rows[0]!.status).toBe('suggested')
|
||||
})
|
||||
})
|
||||
Reference in New Issue
Block a user