fix(audit): alias learning is not a rule change, stop logging it (BFNAR noise) (#2134)

Production falsified 20260901103000's exclusion list within 30 minutes of deploy: 15 of the first 16 UPDATE audit rows on categorization_templates changed only the learning columns plus counterparty_aliases, because the learning path (lib/bookkeeping/counterparty-templates.ts) merges new aliases in the same write that bumps occurrence_count. Projected ~800 noise rows/day against ~50/day of real rule changes, each one rendered into the legally-facing behandlingshistorik as "Konteringsmall aendrad: Alias". counterparty_aliases joins the trigger's strip list. The trade-off is explicit: a human editing ONLY aliases is no longer logged. Accepted because alias growth is overwhelmingly automatic, and a change that also touches accounts, VAT, pattern or the active flag still logs: the first real such row (2026-09-01 19:02:17Z, debit/credit/vat accounts changed by the learning loop, BFN's automatkontering case exactly) was captured correctly and stays captured under the new WHEN clause. The pre-fix noise rows stay in audit_log (append-only). The read model stops labelling the column, so alias-only diffs, historical ones included, render as no-ops rather than rule changes; a diff that also carries a real change shows only the real change. pg-test extended: alias+learning update writes no audit row, alias+account update still does. Read-model test pins the pre-fix noise row shape to null. Claude-Session: https://claude.ai/code/session_01L3P2hr19PhQuCoTSGoegcY Co-authored-by: Jakob Wennberg <311770904+jakobwennberg-oss@users.noreply.github.com> Co-authored-by: Claude Opus 5 <noreply@anthropic.com>
2026-09-01 21:58:02 +02:00
parent 53c9c0c4f9
commit 191fb2cfce
5 changed files with 81 additions and 12 deletions
@@ -1457,4 +1457,5 @@ One line per decision: `[YYYY-MM-DD] <decision>: <why>`. Appended by agents and
 [2026-09-01] F2 bank-data staleness: ship freshness reads only (last_synced_at/consent_expires/error_message on gnubok_connect_bank + new GET /api/v1/.../bank-connections, scope companies:read mirroring the MCP mapping): the daily cron already syncs server-side, so visibility is what the incident lacked; an agent-triggerable sync is a product bet (EB call cost, runaway agents) and was deferred by Emil.
 [2026-09-01] Verifikationsserie in the Ny verifikation modal is a closed dropdown instead of a one-letter free-text field: a typo there silently opens a brand-new series with its own number sequence, and the letters only mean anything if everyone uses the same ones. The letters are NOT prescribed by law (BFL 5 kap. 7 § requires only unbroken systematic numbering within each series), and the incumbents disagree: Björn Lundén uses A Huvudserie, F Kundfakturor, I Inbetalningar, L Leverantörsfakturor, N Löner, U Utbetalningar, J Bokslut. We ship FORTNOX's table verbatim (A Redovisning, B Kundfakturor, C Inbetalningar från kunder, D Leverantörsfakturor, E Utbetalningar till leverantörer, F Kassa, G Avskrivning, H Periodisering, I Bokslut, J Revisor, K Lön, L Kontantfaktura, M Momsrapport), from their own Systemdokumentation, because Fortnox is the system most companies migrate here from and an imported ledger should keep its meaning. REJECTED an earlier draft that labelled A as Kundfakturor: A is the general series manual entries land in (the one point Fortnox and BL agree on, and Fortnox allows manuell kontering ONLY in A), and migration 20260526120700 ships every source_type defaulting to 'A', so every existing company's A series already holds everything. Calling it Kundfakturor would mislabel their entire history and the modal's own default. The list is closed but any letter the company already configured, or that a draft was saved with, is appended so no existing value can fall out of the picker. Also: tabbing or clicking into an untouched amount field now proposes the outstanding difference (pre-selected, so typing replaces it) when the row already has an account and the difference belongs on that side. This deliberately reverses part of the note in updateLine that said a balancing amount must never auto-fill: that note was about filling on ACCOUNT selection, which stole the amount before the user had a chance to split it. Filling on focus keeps the split case intact because the proposal is selected text, and it fixes the common moms case where the last line is just the remainder.
 [2026-09-01] Settings PUT cross-field VAT validations scoped to touched field groups (vat-completeness, 40m-monthly, periodisk sammanstallning), not fixed at onboarding: partial saves from surfaces without VAT fields (invoice bank-details dialog) were hard-blocked by pre-existing vat_registered-without-number state (Marketio Lab case). The invariant still holds on every save that touches its group; explicit null now counts as a clear instead of falling back to the stored value during validation. Onboarding-side VAT number collection left as follow-up.
+[2026-09-01] counterparty_aliases joins the categorization_templates audit-trigger strip list (20260901200000) instead of staying logged: prod falsified the original exclusion list within 30 minutes of 20260901103000 going live (15 of the first 16 UPDATE audit rows were alias+learning noise, ~800/day projected vs ~50/day of real rule changes), because the learning path merges aliases in the same write that bumps occurrence_count. Explicit trade-off: a human editing ONLY aliases is no longer logged; accepted since alias growth is overwhelmingly automatic and any change also touching accounts/VAT/pattern/active still logs (first real one, 19:02:17Z same day, captured correctly). Pre-fix noise rows stay in audit_log (append-only) and the read model stops labelling the column so they render as no-ops.
 [2026-09-01] MCP catalog budget attacked at the duplicated staged envelope rather than by demoting more reads: measuring the payload by segment showed outputSchema is 38 % of the whole catalog (23 290 tokens) and STAGED_OPERATION_SCHEMA alone 14 736 of it, the same envelope transmitted 58 times, while descriptions (what the three previous rounds trimmed) are only 10 %. period_status now carries its shape in one sentence instead of declared JSON Schema, matching actor/approve/preview which were always bare objects; 2 552 tokens reclaimed with no tool demoted and no field removed. Every edit is in the LOOSER direction because the server emits structuredContent for every tool and the documented failure mode is a declaration too tight making a strict client reject a successful call. next kept additionalProperties: false: staging.test.ts pins it closed and a guard whose reason is not in front of you is not one to loosen for 420 tokens. Ceiling ratcheted to 60 000 rather than the usual ~300 margin, leaving ~1 070 deliberate working margin: server.ts took 70 commits in 14 days and the previous 116-token margin is what starts the ratchet-block-bump-demote cycle visible in the bench log.
@@ -708,15 +708,18 @@ describe('auditRowToEvent: behandlingsregler', () => {
  })

  it('categorization_templates: the learning columns never reach the report', () => {
-    // The DB trigger filters these already (migration 20260901103000); the read
-    // model must not resurrect them if a row slips through, or every booking
-    // would appear as a system change.
+    // The DB trigger filters these already (20260901103000 + 20260901200000);
+    // the read model must not resurrect them if a row slips through, or every
+    // booking would appear as a system change. counterparty_aliases counts as
+    // learning: prod's first 30 minutes of trigger rows were 15/16 alias
+    // noise, and those pre-fix rows are still in audit_log and must render as
+    // no-ops.
    const learning = auditRowToEvent(
      auditRow({
        table_name: 'categorization_templates',
        action: 'UPDATE',
-        old_state: { counterparty_name: 'Spotify AB', debit_account: '6540', occurrence_count: 4, confidence: 0.7 },
-        new_state: { counterparty_name: 'Spotify AB', debit_account: '6540', occurrence_count: 5, confidence: 0.9 },
+        old_state: { counterparty_name: 'Spotify AB', debit_account: '6540', occurrence_count: 4, confidence: 0.7, counterparty_aliases: ['SPOTIFY'] },
+        new_state: { counterparty_name: 'Spotify AB', debit_account: '6540', occurrence_count: 5, confidence: 0.9, counterparty_aliases: ['SPOTIFY', 'SPOTIFY STOCKHOLM 4711'] },
      }),
    )
    expect(learning).toBeNull()
@@ -413,7 +413,10 @@ const MAPPING_RULE_FIELDS: Record<string, string> = {

 const CATEGORIZATION_TEMPLATE_FIELDS: Record<string, string> = {
  counterparty_name: 'Motpart',
-  counterparty_aliases: 'Alias',
+  // counterparty_aliases deliberately absent: aliases grow in the same
+  // learning write as occurrence_count (migration 20260901200000 stopped
+  // logging them), and the pre-fix audit rows already in prod are alias-only
+  // noise that must render as no-ops, not as rule changes.
  debit_account: 'Debetkonto',
  credit_account: 'Kreditkonto',
  vat_treatment: 'Momshantering',
@@ -0,0 +1,47 @@
+-- Stop alias learning from flooding the behandlingshistorik audit trail.
+--
+-- 20260901103000 added the audit trigger on categorization_templates with a
+-- WHEN clause excluding the learning columns (occurrence_count, confidence,
+-- last_seen_date, updated_at). Production falsified the list within 30
+-- minutes of deploy: 15 of the first 16 UPDATE audit rows changed only those
+-- columns plus counterparty_aliases, because the learning path
+-- (lib/bookkeeping/counterparty-templates.ts) merges new aliases in the same
+-- write that bumps occurrence_count. Projected ~800 noise rows/day against
+-- ~50/day of real rule changes, every one of them rendered into the
+-- legally-facing report as "Konteringsmall aendrad: Alias ...".
+--
+-- counterparty_aliases joins the strip list. The trade-off is explicit: a
+-- human editing ONLY the aliases of a template is no longer logged either.
+-- Accepted because alias growth is overwhelmingly automatic, and a change
+-- that also touches accounts, VAT, pattern or active flag still logs (the
+-- first real such row, 2026-09-01 19:02:17Z, changed debit/credit/vat
+-- accounts alongside learning columns and was correctly captured).
+--
+-- The noise rows already written stay: audit_log is append-only. The read
+-- model stops labelling the column, so they render as no-ops.
+--
+-- pg-test: tests/pg/behandlingshistorik-audit-triggers.pg.test.ts
+
+DROP TRIGGER audit_categorization_templates_update ON public.categorization_templates;
+
+CREATE TRIGGER audit_categorization_templates_update
+  AFTER UPDATE ON public.categorization_templates
+  FOR EACH ROW
+  WHEN (
+    (to_jsonb(OLD) - ARRAY[
+      'occurrence_count',
+      'confidence',
+      'last_seen_date',
+      'updated_at',
+      'counterparty_aliases'
+    ]::text[])
+    IS DISTINCT FROM
+    (to_jsonb(NEW) - ARRAY[
+      'occurrence_count',
+      'confidence',
+      'last_seen_date',
+      'updated_at',
+      'counterparty_aliases'
+    ]::text[])
+  )
+  EXECUTE FUNCTION public.write_audit_log();
@@ -4,7 +4,7 @@ import { insertAuthUser, insertCompany, insertCompanyMember } from './fixtures'
 import { getPool, withUserContext } from './setup'

 /**
- * Behandlingshistorik, part 3 (migration 20260901103000): the behandlingsregler
+ * Behandlingshistorik, part 3 (migrations 20260901103000 + 20260901200000): the behandlingsregler
 * tables and the import logs write to the immutable audit_log, learning-only
 * updates on categorization_templates are filtered, the global payroll
 * constants are logged without a company, and app_releases is an append-only,
@@ -65,21 +65,36 @@ describe('behandlingshistorik audit triggers (BFNAR 2013:2 p. 9.16)', () => {
       VALUES ($1, $2, $3, 'Spotify AB', '6540', '1930')`,
      [templateId, userId, companyId],
    )
-    // Learning on every booking: occurrence_count / confidence / last_seen_date.
+    // Learning on every booking: occurrence_count / confidence / last_seen_date,
+    // and counterparty_aliases, which the learning path merges in the same
+    // write (20260901200000; prod showed 15 of the first 16 audit rows were
+    // alias+learning noise).
    await getPool().query(
      `UPDATE public.categorization_templates
-          SET occurrence_count = occurrence_count + 1, confidence = 0.9, last_seen_date = CURRENT_DATE
+          SET occurrence_count = occurrence_count + 1, confidence = 0.9, last_seen_date = CURRENT_DATE,
+              counterparty_aliases = ARRAY['SPOTIFY STOCKHOLM 4711']
        WHERE id = $1`,
      [templateId],
    )
    expect(await auditActions('categorization_templates', templateId)).toEqual(['INSERT'])

-    // A rule change (the account) is logged.
-    await getPool().query(`UPDATE public.categorization_templates SET debit_account = '6212' WHERE id = $1`, [templateId])
+    // But an account change arriving in the same write as learning columns is
+    // a behandlingsregel change and must still log (the automatkontering case).
+    await getPool().query(
+      `UPDATE public.categorization_templates
+          SET occurrence_count = occurrence_count + 1, credit_account = '1931',
+              counterparty_aliases = ARRAY['SPOTIFY STOCKHOLM 4711', 'SPOTIFY AB']
+        WHERE id = $1`,
+      [templateId],
+    )
    expect(await auditActions('categorization_templates', templateId)).toEqual(['INSERT', 'UPDATE'])

+    // A plain rule change (the account) is logged.
+    await getPool().query(`UPDATE public.categorization_templates SET debit_account = '6212' WHERE id = $1`, [templateId])
+    expect(await auditActions('categorization_templates', templateId)).toEqual(['INSERT', 'UPDATE', 'UPDATE'])
+
    await getPool().query(`DELETE FROM public.categorization_templates WHERE id = $1`, [templateId])
-    expect(await auditActions('categorization_templates', templateId)).toEqual(['INSERT', 'UPDATE', 'DELETE'])
+    expect(await auditActions('categorization_templates', templateId)).toEqual(['INSERT', 'UPDATE', 'UPDATE', 'DELETE'])
  })

  it('logs booking_template_library changes (no user_id column: actor falls back to auth.uid())', async () => {