Files
accounted/lib/parties/suggest.ts
T
Jakob Wennberg fc04578818 feat(parties): suggestion pipeline from ledger keys and linked documents (#2172)
* feat(parties): phase 1 substrate, one party per counterpart

Adds the identity layer above customers and suppliers, which keep their
tables and every foreign key and gain a nullable party_id.

- parties: company-scoped identity with status (suggested | confirmed),
  kind, alias keys, origin and merged_into. One live party per org number
  and company, enforced by a partial unique index; merged losers leave the
  index so a merge can be undone. This is the unique key the
  duplicate-invoice guard has lacked, since suppliers never had one.
- party_facts: statements with a source, a rank (preferred | normal |
  deprecated) and two time axes, never overwritten.
- party_identities: bankgiro, plusgiro, IBAN and friends per party, with
  seen and paid counts and a known | unverified status.
- party_decisions: every human action on a party as a labelled example.
- normalize_org_number(text): SQL mirror of lib/invariants/org-number.ts
  (strip separators, drop the century on 12 digits, Luhn check, 10 digits).
- ensure_party(): find by org number inside the company, else create.
  Name-only rows never merge at insert time; a name merge is a recorded
  human decision.
- Backfill: one party per existing supplier and customer, merged on org
  number, suppliers first so both roles land on one party.
- Archive contract: the four tables are master data in the full archive.

Observed parties (keys derived from voucher and bank text) are not stored;
they stay computed by the ledger-context RPC. No posted entry is touched.

Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>

* feat(parties): observed parties from voucher text, ledger_key and its mirror

Migrants arrive with vouchers, not bank transactions, so the bank-keyed
ledger context is empty for them. This adds the description-keyed twin.

- public.ledger_key(text): legibility key on top of the frozen
  normalize_counterparty_key mirror: strips AP-register prefixes (levfakt,
  leverantörsfaktura från N, levbet, faktura, kvitto, utgift), the supplier
  number that follows them, and trailing 1-3 digit runs, never "inköp".
  Mirrored by lib/parties/ledger-key.ts; the pair is pinned by a shared
  fixture list in the pg test.
- public.get_observed_parties(company, from_date, limit): posted vouchers
  grouped by ledger_key(description) with occurrences, variants, expense
  and revenue SEK from the lines, first/last seen, median cadence and the
  Laplace-smoothed dominant result account. Excludes storno, opening
  balance, year-end and VAT settlement, and vouchers that carry a bank
  merchant name (those stay with get_ledger_deep_context). SECURITY
  INVOKER, so RLS scopes it. Never stored.
- lib/parties/classify.ts: the deterministic pre-classifier moved out of
  the evaluation script so product and evaluation share one implementation
  (0.965 agreement with the founder labels, party recall 0.99).
- lib/parties/observed.ts: RPC wrapper that classifies each row and
  derives a display rhythm from the cadence.

Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>

* fix(parties): tenant-safe composite foreign keys on every party link

Facts, identities, decisions, customers.party_id, suppliers.party_id and
parties.merged_into now reference parties(id, company_id), so a row can
only point at a party in its own company. ON DELETE SET NULL names
party_id so role rows keep their company_id. Adds a pg-real test that
rejects every cross-company link and checks company_id survives a party
delete.

Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>

* feat(parties): suggestion pipeline from ledger keys and linked documents

Phase 1c of the parties plan. Fills a migrant's register with suggested
parties from what the ledger already knows, never as facts:

- get_ledger_key_evidence(company): hard keys per ledger_key from the
  documents linked to posted vouchers (org number via normalize_org_number,
  VAT, bankgiro, plusgiro, printed name). Documents whose supplier org is
  the company's own are the company's sales invoices and only count in
  self_docs.
- apply_party_suggestions(company, user, items): upserts suggestions.
  Attaches by explicit party_id, org number or an exact alias key; never
  by name. Identities become known at two sightings. Idempotent.
- decide_parties(company, user, ids, kind, note): bulk confirm or dismiss
  with one party_decisions row each.
- parties.suggested_reason: the evidence summary the queue shows per row.
- lib/parties/suggest.ts: buildSuggestions (pure) and
  suggestPartiesForCompany. Keys that mix two org numbers keep neither the
  hard key nor identities; same-core live parties are reported as
  similar_to for a person to decide. coreKey() moves into ledger-key.ts.

55 unit tests and 14 pg-real tests pass locally.

Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>

* fix(parties): decide_parties dismisses suggested parties only

Dismiss is the queue's answer to a suggestion; a confirmed party is never
archived through it. Superagent P2 on #2172.

Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>

* test(parties): type the rpc mock with its args so the ratchet stays clean

Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>

---------

Co-authored-by: Jakob Wennberg <311770904+jakobwennberg-oss@users.noreply.github.com>
Co-authored-by: Claude Fable 5.1 <noreply@anthropic.com>
2026-09-02 18:18:12 +02:00

291 lines
10 KiB
TypeScript

/**
* Parties, phase 1c: the suggestion pipeline.
*
* Turns what the ledger already knows about a company into suggested parties
* so a migrant's register is full on arrival. Inputs are the observed parties
* (posted vouchers grouped by ledger key, get_observed_parties) and the hard
* keys read from documents linked to those vouchers (get_ledger_key_evidence:
* org number, VAT number, bankgiro, plusgiro, printed name). Output is a list
* of items for apply_party_suggestions, which writes parties with status
* 'suggested' and never merges on name: a key attaches to an existing party
* only through its org number or because that exact key is already an alias.
*
* Keys that look alike (same core) are reported in the reason as
* similar_to, so the queue can offer "same as X?" for a person to decide.
* The model selection step is not wired here yet; it runs in shadow through
* scripts/parties/eval-selection.ts until its decisions are labelled.
*/
import type { SupabaseClient } from '@supabase/supabase-js'
import { fetchAllRows } from '@/lib/supabase/fetch-all'
import { coreKey } from './ledger-key'
import { getObservedParties, type ObservedParty } from './observed'
export interface IdentityEvidence {
value: string
n: number
first_seen: string | null
last_seen: string | null
}
export interface LedgerKeyEvidence {
key: string
docs: number
self_docs: number
orgs: Array<{ org: string; n: number }>
vat_numbers: Array<{ vat: string; n: number }>
names: Array<{ name: string; n: number }>
bankgiro: IdentityEvidence[]
plusgiro: IdentityEvidence[]
}
export interface ExistingParty {
id: string
display_name: string
org_number: string | null
alias_keys: string[]
status: 'suggested' | 'confirmed'
}
export interface SuggestionFact {
field: string
value: unknown
source: 'ledger' | 'document'
reference?: Record<string, unknown>
}
export interface SuggestionIdentity {
scheme: 'bankgiro' | 'plusgiro'
value: string
first_seen: string | null
last_seen: string | null
seen_count: number
}
export interface SuggestionReason {
/** How the key attaches, or why it becomes a new party. */
attach: 'party_id' | 'org_number' | 'alias_key' | 'new'
occurrences: number
expense_sek: number
revenue_sek: number
first_seen: string
last_seen: string
docs: number
/** Documents whose supplier org number is the company's own: sales side. */
self_docs: number
org_number?: string
/** More than one org number seen under the key: hard key withheld. */
ambiguous_orgs?: string[]
dominant_account?: string | null
/** Live parties with the same core: a merge question, never a merge. */
similar_to?: Array<{ party_id: string; display_name: string }>
}
export interface SuggestionItem {
key: string
display_name: string
legal_name?: string
kind: 'company'
origin: 'ledger' | 'document'
org_number?: string
vat_number?: string
party_id?: string
alias_keys: string[]
reason: SuggestionReason
facts: SuggestionFact[]
identities: SuggestionIdentity[]
}
export interface SuggestionSkip {
key: string
label: ObservedParty['label']
}
export interface BuildResult {
items: SuggestionItem[]
skipped: SuggestionSkip[]
}
function pickName(observed: ObservedParty, evidence: LedgerKeyEvidence | undefined): { display: string; legal?: string } {
// A printed supplier name from a document beats the voucher text, which is
// upper-cased, truncated and prefixed by whatever the source system did.
const printed = evidence?.names[0]?.name
if (printed && printed.length >= 2) return { display: printed, legal: printed }
return { display: observed.name || observed.key }
}
function identitiesFrom(evidence: LedgerKeyEvidence | undefined): SuggestionIdentity[] {
if (!evidence) return []
const out: SuggestionIdentity[] = []
for (const scheme of ['bankgiro', 'plusgiro'] as const) {
for (const e of evidence[scheme]) {
out.push({ scheme, value: e.value, first_seen: e.first_seen, last_seen: e.last_seen, seen_count: e.n })
}
}
return out
}
/**
* Pure: decide what each observed party key becomes. Only keys the
* pre-classifier calls 'party' continue; the rest are returned as skipped so
* callers can show why "Inköp av varor" is not a supplier.
*/
export function buildSuggestions(input: {
observed: ObservedParty[]
evidence: LedgerKeyEvidence[]
existing: ExistingParty[]
}): BuildResult {
const evidenceByKey = new Map(input.evidence.map((e) => [e.key, e]))
const byOrg = new Map<string, ExistingParty>()
const byAlias = new Map<string, ExistingParty>()
const byCore = new Map<string, ExistingParty[]>()
for (const p of input.existing) {
if (p.org_number && !byOrg.has(p.org_number)) byOrg.set(p.org_number, p)
for (const a of p.alias_keys) if (!byAlias.has(a)) byAlias.set(a, p)
const c = coreKey(p.display_name)
if (c) byCore.set(c, [...(byCore.get(c) ?? []), p])
for (const a of p.alias_keys) {
const ac = coreKey(a)
if (ac && ac !== c) byCore.set(ac, [...(byCore.get(ac) ?? []), p])
}
}
const items: SuggestionItem[] = []
const skipped: SuggestionSkip[] = []
for (const o of input.observed) {
if (o.label !== 'party') {
skipped.push({ key: o.key, label: o.label })
continue
}
const ev = evidenceByKey.get(o.key)
const orgs = ev?.orgs ?? []
const org = orgs.length === 1 ? orgs[0]!.org : undefined
const existing = (org && byOrg.get(org)) || byAlias.get(o.key) || undefined
const name = pickName(o, ev)
const reason: SuggestionReason = {
attach: existing ? (org && byOrg.get(org) === existing ? 'org_number' : 'alias_key') : 'new',
occurrences: o.occurrences,
expense_sek: o.expense_sek,
revenue_sek: o.revenue_sek,
first_seen: o.first_seen,
last_seen: o.last_seen,
docs: ev?.docs ?? 0,
self_docs: ev?.self_docs ?? 0,
dominant_account: o.dominant_account_number,
}
if (org) reason.org_number = org
if (orgs.length > 1) reason.ambiguous_orgs = orgs.map((x) => x.org)
if (!existing) {
const similar = (byCore.get(coreKey(o.key)) ?? []).filter((p) => !org || p.org_number !== org)
if (similar.length) reason.similar_to = similar.slice(0, 6).map((p) => ({ party_id: p.id, display_name: p.display_name }))
}
const facts: SuggestionFact[] = []
if (o.dominant_account_number) {
facts.push({
field: 'dominant_account',
value: { account: o.dominant_account_number, share: o.dominant_account_share, count: o.dominant_account_count },
source: 'ledger',
reference: { occurrences: o.occurrences, first_seen: o.first_seen, last_seen: o.last_seen },
})
}
if (o.cadence_days != null) {
facts.push({ field: 'cadence_days', value: o.cadence_days, source: 'ledger', reference: { occurrences: o.occurrences } })
}
if (org) {
facts.push({ field: 'org_number', value: org, source: 'document', reference: { docs: orgs[0]!.n } })
}
if (name.legal) {
facts.push({ field: 'legal_name', value: name.legal, source: 'document', reference: { docs: ev?.names[0]?.n ?? 0 } })
}
// Identities only when the hard key is unambiguous: a key that mixes two
// org numbers would otherwise attach one supplier's bankgiro to another.
const identities = orgs.length > 1 ? [] : identitiesFrom(ev)
const vat = ev?.vat_numbers[0]?.vat
items.push({
key: o.key,
display_name: name.display,
...(name.legal ? { legal_name: name.legal } : {}),
kind: 'company',
origin: org ? 'document' : 'ledger',
...(org ? { org_number: org } : {}),
...(vat && orgs.length <= 1 ? { vat_number: vat } : {}),
...(existing ? { party_id: existing.id } : {}),
alias_keys: [o.key],
reason,
facts,
identities,
})
}
return { items, skipped }
}
export interface SuggestSummary {
observed: number
suggested: number
skipped: number
created: number
attached: number
identities: number
facts: number
}
/**
* Run the pipeline for one company and persist the result. Safe to re-run:
* apply_party_suggestions is idempotent.
*/
export async function suggestPartiesForCompany(
supabase: SupabaseClient,
companyId: string,
userId: string,
options: { fromDate?: string | null; limit?: number; chunkSize?: number } = {},
): Promise<SuggestSummary> {
const observed = await getObservedParties(supabase, companyId, {
fromDate: options.fromDate ?? null,
limit: options.limit ?? 5000,
})
const { data: evidenceData, error: evidenceError } = await supabase.rpc('get_ledger_key_evidence', {
p_company_id: companyId,
})
if (evidenceError) throw new Error(`get_ledger_key_evidence failed: ${evidenceError.message}`)
const evidence = (Array.isArray(evidenceData) ? evidenceData : []) as LedgerKeyEvidence[]
const existing = await fetchAllRows<ExistingParty>(({ from, to }) =>
supabase
.from('parties')
.select('id, display_name, org_number, alias_keys, status')
.eq('company_id', companyId)
.is('merged_into', null)
.is('archived_at', null)
.order('created_at', { ascending: true })
.range(from, to),
)
const { items, skipped } = buildSuggestions({ observed, evidence, existing })
const summary: SuggestSummary = {
observed: observed.length,
suggested: items.length,
skipped: skipped.length,
created: 0,
attached: 0,
identities: 0,
facts: 0,
}
const chunk = Math.max(1, options.chunkSize ?? 200)
for (let i = 0; i < items.length; i += chunk) {
const { data, error } = await supabase.rpc('apply_party_suggestions', {
p_company_id: companyId,
p_user_id: userId,
p_items: items.slice(i, i + chunk),
})
if (error) throw new Error(`apply_party_suggestions failed: ${error.message}`)
const r = (data ?? {}) as Partial<Record<'created' | 'attached' | 'identities' | 'facts', number>>
summary.created += r.created ?? 0
summary.attached += r.attached ?? 0
summary.identities += r.identities ?? 0
summary.facts += r.facts ?? 0
}
return summary
}