fix(customers): personnummer via MCP lands in personal_number, masked everywhere; MCP payment terms follow settings (#1788)
* fix(customers): personnummer on the MCP path lands in personal_number, masked everywhere; MCP payment terms follow settings Follow-up to #1724 (Discord kalletoxic): the fix reached the web form and the v1 REST API, but not the MCP path, and the web customer list still showed a personnummer raw when it sat in org_number. Personnummer (MCP + every write path): - gnubok_create_customer gets a personal_number input. Until now it had none, so an agent creating a private person either dropped the number or put it in org_number, which nothing masks. Encrypted at staging (personal_number_encrypted + personal_number_masked; personal_number is now a forbidden staging key in staging-pii-guard), the approval preview shows ********-1234, commitCreateCustomer stores the ciphertext as-is. Idempotency hashes the masked preview (new StageOptions.idempotencyParams) because the random-IV ciphertext would make identical retries look like payload changes. - A personnummer-shaped org_number on customer_type=individual is the personnummer in the wrong field: it is moved into personal_number (encrypted) and org_number cleared, on CreateCustomerSchema (web POST, v1 POST, v1 bulk), both PATCH routes, MCP staging, and commitCreateCustomer for in-flight ops. Only a DIFFERENT personnummer next to personal_number is refused (new CUSTOMER_PERSONAL_NUMBER_CONFLICT). The business-type guard from #1724 is unchanged and now also fires at MCP staging, so the user never approves an operation that fails at commit. - Read side: the web customer list and gnubok_list_customers mask a legacy individual row's org_number personnummer instead of showing it raw; list_customers exposes personal_number_masked and never the ciphertext. - scripts/repair-customer-personal-number-in-org-number.ts moves the existing rows (dry run: 134 rows across 10 companies on prod); run by hand with --confirm after deploy. - customer-onboarding skill: EF customers follow the #1724 decision (individual + personal_number); ROT/RUT section names the real field. Payment terms (MCP): - gnubok_create_customer staged `payment_terms || 30`, so resolveDefaultPaymentTerms at commit always saw 30 and the company's invoice_default_days never reached MCP customers. Resolved at staging now, so the preview shows the value the row will get. tools/list payload ceiling 59.75K to 59.85K (descriptions trimmed first, rationale in payload-size.bench.test.ts). apiskill regenerated; no migrations. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01CbLqn9bgZ9NJ5qnZMeC1Bk * fix(scripts): literal update payloads in the personnummer repair script The no-phantom-columns scanner counts a runtime-built update payload as unresolvable and the ceiling (379) had no headroom; two literal payloads keep the guard able to resolve both branches. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01CbLqn9bgZ9NJ5qnZMeC1Bk --------- Co-authored-by: Jakob Wennberg <311770904+jakobwennberg-oss@users.noreply.github.com> Co-authored-by: Claude Fable 5 <noreply@anthropic.com>
This commit is contained in:
co-authored by
Claude Fable 5
Jakob Wennberg
parent
a5c8f55127
commit
13b69a2056
@@ -1151,6 +1151,7 @@ One line per decision: `[YYYY-MM-DD] <decision>: <why>`. Appended by agents and
|
||||
[2026-08-21] RIP-4 cascade step 3 (UI): the AI booking proposal is surfaced INSIDE the existing QuickReviewDialog rather than a new inline-row card, so it reuses that dialog's proven, deterministic, balanced commit path (POST /api/transactions/[id]/categorize) instead of a parallel one. components/transactions/AiCategorizeProposal.tsx fetches POST /api/agent/categorize on dialog open (keyed on tx.id so it remounts per transaction), pre-fills accountOverride + vatTreatment via handleAccountChange (class-2 VAT clearing preserved), and shows the confidence band (säker/trolig/välj konto) + "Varför" + the candidate alternatives (click to re-apply). Falls back SILENTLY to the deterministic defaults on error, and shows a soft note on 503 (ai_unconfigured) — the dialog always works without AI. NO silent auto-posting (founder call, avoids the storno-on-undo mess): "säker" = pre-filled, one-tap Bokför via the dialog's existing button; true hands-off auto-book waits for calibration. i18n: strings inline Swedish for now (assistant surface), lift to messages/{sv,en}.json before final merge. Confidence bands (0.8/0.5) are placeholders until calibration. Needs founder visual sign-off before merge ([[project_nav_ia_redesign]]).
|
||||
[2026-08-21] RIP-4 step 4 = calibration. lib/agent/categorize/calibration.ts is the engine: isotonic regression via pool-adjacent-violators (distribution-free, monotonic) over (confidence, was_correct) samples → a calibrator that turns raw selector confidence into a probability that actually means what it says; plus reliabilityByBucket/ECE and bandFor(). bandFor NEVER returns 'auto' without a fitted calibrator (no silent booking on an unproven score) and never auto-books above an amount cap (default 2000 kr) — so "säker" stays honest until proven. Measurement loop: migration 20260821100000 categorize_calibration_samples (append-only, company-scoped RLS, confidence CHECK [0,1]); POST /api/agent/categorize/outcome logs one sample (proposed vs booked account → was_correct) fire-and-forget from QuickReviewDialog on a successful book (sandbox skipped to keep the corpus clean); AiCategorizeProposal surfaces the proposal metadata via onProposal. scripts/fit-categorize-calibration.ts (READ-ONLY) prints the reliability diagram + ECE + fitted calibrator once data exists — run it in a few weeks, then store the calibrator/thresholds where bandFor reads them and only THEN consider enabling auto-book. Fitting needs >=200 real samples so nothing calibrates today; the loop just starts collecting. Migration applies on merge (auto-apply-on-merge active) — not applied manually.
|
||||
[2026-08-21] The categorize selector now reads the underlag, not just the bank line — the highest-leverage quality lever for the cold-start majority (prod: 365 companies with 32.7k unbooked tx, median 0 templates, so the LLM carries them). lib/agent/categorize/underlag.ts gathers the matched receipt/invoice text (receipts.matched_transaction_id + invoice_inbox_items.matched_transaction_id + the transaction's own document_attachments), rendered as bounded Swedish text (supplier, date, total, moms, line items). Core reads these tables directly via supabase (table names, not @/extensions imports — the tables live in the shared DB). POST /api/agent/categorize gathers it server-side when the caller didn't pass `underlag`, so the model sees the actual supplier + line items. Best-effort ('' on any failure); server-side only, no client change (so no conflict with the calibration PR #1784 which also touches the dialog). Prod read (project pwxtzglxptnnvjrpixpg) also confirmed: 3342 active counterparty templates / 26k occurrences → established users get strong instant candidates; NO backfill needed (templates already reflect historical bookings).
|
||||
[2026-08-21] A personnummer-shaped org_number on customer_type=individual is MOVED into personal_number (encrypted, masked on read) and org_number cleared, on every write path (CreateCustomerSchema transform for web/v1/bulk, both PATCH routes, MCP gnubok_create_customer at staging and commitCreateCustomer for in-flight ops), rather than rejected like the business-type case (#1724): the value is unambiguously the person's own personnummer, the MCP tool had no personal_number input until now so agents had nowhere else to put it (134 such rows across 10 companies on prod), and the v1 docs promised org_number was accepted for individuals, so a 400 would break live clients for no privacy gain. Only an org_number that is a DIFFERENT personnummer than a submitted personal_number is refused (CUSTOMER_PERSONAL_NUMBER_CONFLICT). MCP stages the personnummer encrypted (personal_number_encrypted + personal_number_masked, personal_number is now a forbidden staging key) and hashes the masked preview for idempotency, since the random-IV ciphertext would make identical retries look like payload changes. Read surfaces (web customer list, MCP list_customers) mask a legacy individual row's org_number personnummer instead of showing it raw; scripts/repair-customer-personal-number-in-org-number.ts moves the existing rows and is run by hand with --confirm after deploy. MCP create_customer payment_terms are resolved from company_settings at staging: the staging-side || 30 is why #1708's fix never reached MCP customers.
|
||||
[2026-08-21] Peppol receiving (PR2) keeps Qvalia's consolidated partner account: every company's 0007:orgnr is registered on OUR account (PUT /partner/{p}/account/{p}/peppol/{id}) and inbound documents are routed by the AccountingCustomerParty EndpointID through peppol_registrations, because Qvalia confirmed sending needs no per-company account and child accounts would only add a 100 kr/mån tenant fee per customer; the exact received UBL XML is archived as a WORM document (upload_source e_invoice, extractionOwner none) and the inbox row is filled from the structured UBL with confidence 1 (no model pass), following the mail-hunt precedent of core inserting invoice_inbox_items directly; personnummer-based companies are refused registration until 0088 GLN support exists (publishing them would put personal data in the Peppol Directory); the poll is a 10-minute cron (GET .../readinvoices marks documents read at Qvalia, so every fetched document is archived before anything else can fail).
|
||||
[2026-08-21] Confidence honesty fix, driven by a real backtest (scripts/backtest-categorize.ts, read-only: runs the real cascade on already-booked prod transactions and scores the model's pick vs the human's actual account). Backtest exposed the selector reporting 0.95 on pure category guesses → "säker" was a lie (high-conf picks only 52% accurate). Fix: confidence is now driven by DETERMINISTIC BACKING (the confidence of a candidate that independently points at the chosen account), not the model's verbalized confidence (which the backtest showed is ~always "high"). A backed pick takes the candidate confidence, reduced only when the model is unsure (BACKED_MODEL_FACTOR); an UNBACKED pick (category guess no candidate agreed with) is capped at 0.7 — below the säker band (0.8) — so a guess is never "säker". Re-backtest: säker accuracy 52% → 73%, and far fewer picks claim säker (only template-backed ones). Still not auto-book-grade (~73%, want ~95%); auto-book stays off until isotonic calibration on real approvals. Backtest caveats: exact-account match is strict (penalizes reasonable-but-different picks + companies' idiosyncratic charts), sample is established users (cold-start majority has no ground truth yet), backtest ran samples=1 (no self-consistency). Some confident-wrong cases are POISONED templates (a past mis-booking → wrong candidate the model correctly follows), a data-quality issue not fixable in the confidence math.
|
||||
[2026-08-21] Behandlingshistorik ships as a report over existing stores (journal_entries.committed_at + audit_log + rattelse log + import tables) rather than on processing_history: that table only carries Document/BankTransaction/System events in prod, while audit_log is complete, immutable and already the archive's revision/behandlingshistorik.json. Event labels stay Swedish in both locales (räkenskapsinformation, archived 7 years, same rule as SIE/grundbok); only the view chrome is translated. Bokföringsposter come from journal_entries (not audit COMMIT rows) so entries predating the audit log or from the July SIE-import window are never missing.
|
||||
|
||||
Reference in New Issue
Block a user