* fix(suppliers): one 10-digit org number key for matching and storage Why the problem occurred: the supplier register was written in three spellings (the form asks for XXXXXX-XXXX, the v1 API and the MCP tool stored whatever the caller sent, the AI extractor emits bare digits) while matchSupplierByIdentity compared raw strings with .eq(). The canonical rule existed three times (normalizeOrgNumber, the MCP fuzzy pass's orgNumberKey, the extractor's toOrg10) and nowhere on the path that decides a match, so every AI-extracted invoice from a hyphen-registered supplier missed the strongest key and fell to exact-name matching. Prod holds 1738 hyphenated rows against 493 bare ones. What was removed or simplified: orgNumberKey (digits only, 10 kept, last 10 of 12, no Luhn) moves into lib/invariants/org-number.ts and replaces the two other copies. The matcher scans the company's suppliers with an org_number and compares keys, the same shape as its vat_number branch, so rows written before the backfill (and self-hosted instances that never run it) match too. CreateSupplierSchema, UpdateSupplierSchema and the staged create_supplier schema store the key; the form renders it through formatOrgNumberDisplay. A backfill migration strips the formatting from existing rows, skipping migration-reset source companies. Why this and not the proposed one: the issue's third layer (CHECK plus a unique index) would fail to create on prod, which holds 94 duplicate (company_id, key) groups across 18 companies, one of them 124 rows under a single placeholder-looking number; that needs a merge decision first and is filed as #2404. Rejecting anything that is not 10 or 12 digits on write was also dropped: 68 prod rows carry foreign registration numbers (DK, DE, NL, FI, GB, IE, US, CZ, IT) in org_number, so Swedish-shaped input is canonicalised and anything else is stored as typed. Luhn stays lenient on suppliers because two rows with the same mistyped number are one supplier and parties is Luhn-strict at promotion already. Fixes #2391 Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_013yCehdxm8yUubGAmoDFZag * fix(suppliers): key only Swedish-shaped org numbers, search and dedup through the key Skeptic pass on the previous commit. Three refutations, all confirmed: 1. orgNumberKey took the last 10 of any 12 digits and stripped letters. A VAT number typed into the org field (SE556012579001, orgnr + 01) keyed to 6012579001, another company's identity, on every write path and in the backfill; 26 prod rows hold exactly that shape (prefixes 55/52/87). A Belgian BE0123456789 lost its country letters the same way. The key now strips only hyphens and spaces and unprefixes 12 digits only behind 16/18/19/20; everything else is null, stored and compared as typed. The migration carries the same rule. 2. The supplier list search, the v1 ?search= filter and the list column all used the raw stored value, so a user searching 556677-88 after the backfill found nothing. Both searches now compare without separators and the column renders XXXXXX-XXXX. 3. Storage was not canonical on every path: the CSV import and the provider migration orchestrator wrote as typed and keyed their re-sync dedup by the raw value, so a Fortnox re-sync sending 556677-8899 would have duplicated the now-bare row. Both write and key through orgNumberKey. Also: the matcher scans live suppliers only, so a register holding an archived hyphenated row next to its live replacement resolves to the live one instead of whichever id sorts first. Refs #2391 Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_013yCehdxm8yUubGAmoDFZag * fix(suppliers): review pass: foreign numbers survive display and dedup, stub key canonical CodeRabbit findings on PR #2405, all verified against the code: - The supplier list rendered through formatOrgNumber, which strips letters and would show BE0123456789 as 012345-6789; it now uses formatOrgNumberDisplay, which leaves anything not Swedish-shaped alone. - The CSV import dedup fell back to digits-only, so BE0123456789 and FR0123456789 collided; the fallback is now the value as typed, in both the parse preview and the execute route. - The provider migration's supplier-invoice stub map was keyed by the raw provider value while the stored row was canonical, so 556677-8899 and 5566778899 on two invoices produced two stubs; the key goes through orgMapKey like the other maps. - v1 response examples show the stored 10-digit form; the request example keeps the hyphenated input. Refs #2391 Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_013yCehdxm8yUubGAmoDFZag * docs(api-skill): regenerate suppliers reference for the canonical org_number example Refs #2391 Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_013yCehdxm8yUubGAmoDFZag --------- Co-authored-by: Claude Fable 5.1 <noreply@anthropic.com>
216 lines
8.4 KiB
TypeScript
216 lines
8.4 KiB
TypeScript
/**
|
|
* Supplier auto-matching for extracted documents (inbox items, uploads, MCP).
|
|
*
|
|
* Every extraction path used to inline the same two lookups: exact org_number,
|
|
* then case-insensitive full name. That works for Swedish suppliers and fails
|
|
* for every foreign one, because the extractor deliberately leaves orgNumber
|
|
* null unless the document carries a real Swedish organisationsnummer (see
|
|
* extensions/general/invoice-inbox/lib/extract-invoice-fields.ts). A supplier
|
|
* like "ADOBE SYSTEMS SOFTWARE IRELAND LTD" only ever prints a
|
|
* momsregistreringsnummer (IE6364992H), which the suppliers table stores in
|
|
* vat_number and which nothing looked at, so an exact name match was the sole
|
|
* remaining key and any rename or OCR variant broke the link.
|
|
*
|
|
* This module is the single implementation: org_number, then vat_number, then
|
|
* name. Matching is best-effort and never throws: a failed lookup leaves the
|
|
* item unmatched for the user to link by hand, it does not fail the upload.
|
|
*/
|
|
|
|
import type { SupabaseClient } from '@supabase/supabase-js'
|
|
import { fetchAllRows } from '@/lib/supabase/fetch-all'
|
|
import { orgNumberKey } from '@/lib/invariants/org-number'
|
|
|
|
export type SupplierIdentity = {
|
|
orgNumber?: string | null
|
|
vatNumber?: string | null
|
|
name?: string | null
|
|
}
|
|
|
|
/** Same shape with every key present: what supplierIdentityFrom() guarantees. */
|
|
export type ResolvedSupplierIdentity = Required<{
|
|
[K in keyof SupplierIdentity]: string | null
|
|
}>
|
|
|
|
/** How a match was found. Callers log this; it is not persisted. */
|
|
export type SupplierMatchKey = 'org_number' | 'vat_number' | 'name'
|
|
|
|
export type SupplierMatch = {
|
|
supplierId: string
|
|
matchedOn: SupplierMatchKey
|
|
}
|
|
|
|
/**
|
|
* Canonical key for a VAT registration number. Registration numbers are
|
|
* printed with spaces, dots and hyphens in every combination ("SE 556012-5790
|
|
* 01", "IE6364992H"), none of which carry identity, so the key is the
|
|
* uppercased alphanumerics. Returns null for values too short to be an
|
|
* identifier at all ("SE", "VAT", "-"), which keeps junk in the column from
|
|
* matching other junk.
|
|
*/
|
|
export function vatNumberKey(raw: string | null | undefined): string | null {
|
|
if (!raw) return null
|
|
const key = raw.toUpperCase().replace(/[^A-Z0-9]/g, '')
|
|
return key.length >= 6 ? key : null
|
|
}
|
|
|
|
/** The ISO country code an EU VAT number leads with, when present. */
|
|
const COUNTRY_PREFIX = /^[A-Z]{2}(?=[0-9])/
|
|
|
|
function withoutCountryPrefix(key: string): string | null {
|
|
return COUNTRY_PREFIX.test(key) ? key.slice(2) : null
|
|
}
|
|
|
|
/**
|
|
* True when two VAT numbers denote the same registration.
|
|
*
|
|
* Beyond canonical equality this accepts the prefix-vs-no-prefix pair
|
|
* ("SE556012579001" vs "556012579001"): suppliers are often typed in from a
|
|
* Swedish invoice without the country code while the extractor is instructed
|
|
* to always include it. The relaxation only fires when exactly ONE side
|
|
* carries a prefix, so IE6364992H and SE6364992H stay distinct.
|
|
*/
|
|
export function vatNumbersMatch(
|
|
a: string | null | undefined,
|
|
b: string | null | undefined,
|
|
): boolean {
|
|
const keyA = vatNumberKey(a)
|
|
const keyB = vatNumberKey(b)
|
|
if (!keyA || !keyB) return false
|
|
if (keyA === keyB) return true
|
|
const bareA = withoutCountryPrefix(keyA)
|
|
const bareB = withoutCountryPrefix(keyB)
|
|
if (bareA && !bareB) return bareA === keyB
|
|
if (bareB && !bareA) return bareB === keyA
|
|
return false
|
|
}
|
|
|
|
/**
|
|
* Escape the LIKE metacharacters before a name goes into .ilike(). A supplier
|
|
* named "100 % Solutions" or "Foo_Bar" would otherwise be a wildcard pattern
|
|
* and match the wrong row.
|
|
*/
|
|
function escapeLikePattern(value: string): string {
|
|
return value.replace(/[\\%_]/g, (char) => `\\${char}`)
|
|
}
|
|
|
|
/**
|
|
* Resolve an extracted supplier identity to a supplier in this company.
|
|
*
|
|
* Order is strongest-key-first: org_number is exact and unique, vat_number is
|
|
* exact once normalised, name is a heuristic that legal-form suffixes and OCR
|
|
* casing routinely break. Returns null when nothing matches.
|
|
*/
|
|
export async function matchSupplierByIdentity(
|
|
supabase: SupabaseClient,
|
|
companyId: string,
|
|
identity: SupplierIdentity,
|
|
): Promise<SupplierMatch | null> {
|
|
// The register was written by hand in the form's XXXXXX-XXXX shape, by the
|
|
// v1 API and MCP in whatever the caller sent, and the extractor emits bare
|
|
// digits: comparing raw strings missed every hyphenated row (#2391). New
|
|
// writes store the canonical key, but the comparison stays key-based so
|
|
// rows written before the backfill, and self-hosted instances that have
|
|
// not run it, match too. Normalising in SQL is not possible through
|
|
// PostgREST, so the scan happens here over the suppliers that have an
|
|
// org_number at all: a small set even for companies with thousands of
|
|
// suppliers, and the same shape as the vat_number scan below. Archived
|
|
// suppliers are skipped: a register that holds the same number twice (an
|
|
// archived hyphenated row next to its live bare replacement) must resolve
|
|
// to the live one, not to whichever id sorts first.
|
|
const orgKey = orgNumberKey(identity.orgNumber)
|
|
if (orgKey) {
|
|
try {
|
|
const rows = await fetchAllRows<{ id: string; org_number: string | null }>(
|
|
({ from, to }) =>
|
|
supabase
|
|
.from('suppliers')
|
|
.select('id, org_number')
|
|
.eq('company_id', companyId)
|
|
.not('org_number', 'is', null)
|
|
.is('archived_at', null)
|
|
.order('id', { ascending: true })
|
|
.range(from, to),
|
|
)
|
|
const hit = rows.find((row) => orgNumberKey(row.org_number) === orgKey)
|
|
if (hit) return { supplierId: hit.id, matchedOn: 'org_number' }
|
|
} catch (error) {
|
|
console.error('[match-supplier] org_number lookup failed:', error)
|
|
}
|
|
} else if (identity.orgNumber) {
|
|
// Not a Swedish org number (a foreign registration number passed through
|
|
// agent-supplied extracted_data): only an exact match can be trusted.
|
|
const { data } = await supabase
|
|
.from('suppliers')
|
|
.select('id')
|
|
.eq('company_id', companyId)
|
|
.eq('org_number', identity.orgNumber)
|
|
.limit(1)
|
|
.maybeSingle()
|
|
if (data) return { supplierId: data.id as string, matchedOn: 'org_number' }
|
|
}
|
|
|
|
// Same shape as the org_number scan: compare canonical keys in memory.
|
|
if (vatNumberKey(identity.vatNumber)) {
|
|
try {
|
|
const rows = await fetchAllRows<{ id: string; vat_number: string | null }>(
|
|
({ from, to }) =>
|
|
supabase
|
|
.from('suppliers')
|
|
.select('id, vat_number')
|
|
.eq('company_id', companyId)
|
|
.not('vat_number', 'is', null)
|
|
.order('id', { ascending: true })
|
|
.range(from, to),
|
|
)
|
|
const hit = rows.find((row) => vatNumbersMatch(row.vat_number, identity.vatNumber))
|
|
if (hit) return { supplierId: hit.id, matchedOn: 'vat_number' }
|
|
} catch (error) {
|
|
// Best-effort: fall through to the name lookup rather than fail the
|
|
// extraction that called us.
|
|
console.error('[match-supplier] vat_number lookup failed:', error)
|
|
}
|
|
}
|
|
|
|
if (identity.name) {
|
|
const { data } = await supabase
|
|
.from('suppliers')
|
|
.select('id')
|
|
.eq('company_id', companyId)
|
|
.ilike('name', escapeLikePattern(identity.name))
|
|
.limit(1)
|
|
.maybeSingle()
|
|
if (data) return { supplierId: data.id as string, matchedOn: 'name' }
|
|
}
|
|
|
|
return null
|
|
}
|
|
|
|
/**
|
|
* Read a supplier identity out of a loosely-typed `extracted_data.supplier`
|
|
* blob (MCP callers hold it as Record<string, unknown>, not the Zod type).
|
|
*
|
|
* `organizationNumber` is accepted alongside the schema's `orgNumber` because
|
|
* the MCP inbox resolver has always read that spelling, and agent-supplied
|
|
* extracted_data may still use it.
|
|
*/
|
|
export function supplierIdentityFrom(raw: unknown): ResolvedSupplierIdentity {
|
|
const supplier = (raw ?? {}) as Record<string, unknown>
|
|
const str = (value: unknown): string | null =>
|
|
typeof value === 'string' && value.trim() !== '' ? value : null
|
|
return {
|
|
orgNumber: str(supplier.orgNumber) ?? str(supplier.organizationNumber),
|
|
vatNumber: str(supplier.vatNumber),
|
|
name: str(supplier.name),
|
|
}
|
|
}
|
|
|
|
/** Convenience wrapper for the call sites that only persist the id. */
|
|
export async function matchSupplierId(
|
|
supabase: SupabaseClient,
|
|
companyId: string,
|
|
identity: SupplierIdentity,
|
|
): Promise<string | null> {
|
|
const match = await matchSupplierByIdentity(supabase, companyId, identity)
|
|
return match?.supplierId ?? null
|
|
}
|