feat(settings): per-company data-analysis opt-in gating the calibration corpus (#1346) (#2007)

* feat(settings): per-company opt-in for data analysis of bookkeeping outcomes (#1346)

Adds company_settings.data_analysis_opt_in (default false, no grandfathering)
and gates every path that reads bookkeeping outcomes across companies on it:
POST /api/agent/categorize/outcome stops writing calibration samples for
companies that have not opted in, and the backtest / calibration-fit scripts
filter to opted-in company ids. One helper (lib/company/data-analysis.ts)
is the single gate for future analysis paths. A toggle on Inställningar >
Företag states plainly what is analysed (proposed vs booked account, amount,
confidence; no free text, no personal data) in sv and en. The flag is UI-only
by design: consent is a human action, so it is absent from the v1 REST / MCP
settings pick lists.

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_015nAd8XJ2RPCmG2eKoLBdna

* fix(settings): make data-analysis consent copy true for the backtest path (#1346)

Addresses adversarial review findings on PR #2007:

- Findings 1-3 (consent narrower than the gated processing): the flag also
  gates scripts/backtest-categorize.ts, which re-runs transaction
  descriptions, merchant names and matched underlag through the model. The
  sv/en toggle help and disclosure now state that explicitly as "evaluation
  runs" and no longer claim that free text or underlag are excluded. The
  migration header and COMMENT, the lib/company/data-analysis.ts docstring,
  the backtest script header and the DECISIONS line say the same. Kept the
  gate (un-gating would put the script back to reading every company with
  no consent at all). A test pins that both locales name those inputs and
  contain no "no free text / no underlag" denial.
- Finding 4 (member sees an active switch that RLS rejects): the toggle is
  now enabled only for owner/admin, matching the company_settings update
  policy; the disclosure says only administrators can change the choice.

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_015nAd8XJ2RPCmG2eKoLBdna

* fix(scripts): address round-2 review findings (#1346)

1. [minor] Opted-in company filter was an unbounded PostgREST `in` list in
   the URL (scripts/fit-categorize-calibration.ts, scripts/backtest-categorize.ts).
   Both scripts now read the opted-in ids through a shared, paginated helper
   (listDataAnalysisOptedInCompanyIds, fetchAllRows so the pre-fetch no longer
   caps at 1000) and query per chunk of 100 ids (chunkCompanyIds). The fit
   script pages each chunk on the id PK; the backtest merges per-chunk
   results and re-cuts to the N most recent overall. Early exit on zero
   opt-ins is kept. Pinned with tests in lib/company/__tests__/data-analysis.test.ts.

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_015nAd8XJ2RPCmG2eKoLBdna

* fix(scripts): coerce a null transaction description in the backtest (#1346)

The typed row from the chunked consent query made description nullable,
which TransactionForSelect does not accept; fall back to the original
description or an empty string, as the untyped row did implicitly before.

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_015nAd8XJ2RPCmG2eKoLBdna

---------

Co-authored-by: Jakob Wennberg <311770904+jakobwennberg-oss@users.noreply.github.com>
Co-authored-by: Claude Fable 5 <noreply@anthropic.com>
This commit is contained in:
Jakob Wennberg
2026-08-28 17:38:36 +02:00
committed by GitHub
co-authored by Claude Fable 5 Jakob Wennberg
parent 33a58bec51
commit ad8566f1ae
16 changed files with 500 additions and 28 deletions
+1
View File
@@ -1323,6 +1323,7 @@ One line per decision: `[YYYY-MM-DD] <decision>: <why>`. Appended by agents and
[2026-08-28] AR-PDF minus fix uses ASCII hyphen formatting, not font embedding: registering a Unicode TTF for react-pdf would change the whole document's typography and bundle size to fix one glyph; formatPdfKronor keeps built-in Helvetica and sidesteps WinAnsi's missing U+2212.
[2026-08-28] Same-bank warning limited to observed one-session banks (SEB only): prod shows Handelsbanken tolerates 4 concurrent sessions, and the generic warning made a user abandon a legitimate renewal. Planned sync-death visibility work was dropped: already shipped via #1271 (health probe), #1727 (stale state), #1969 (cron unstarve).
[2026-08-28] Same-bank warning revised to three tiers after skeptic refutation: hard warn SEB, silent/calm only for verified multi-session banks (Handelsbanken, 4 distinct session_ids observed), legacy hedged warning for unknown banks (fail closed), shared-session siblings exempt (fan-out carries them).
[2026-08-28] data_analysis_opt_in (#1346) defaults false with no grandfathering and gates only bookkeeping-content analysis (categorize_calibration_samples via POST /api/agent/categorize/outcome, the calibration-fit script, and the backtest script, whose wider inputs (transaction descriptions, merchant names, matched underlag re-run through the model) are named in the consent copy as "evaluation runs" after review found the outcomes-only wording false for that path; the toggle is admin/owner only to match the company_settings RLS update policy), not PostHog product analytics or MCP reliability telemetry: those carry no bookkeeping content and stay under Art. 6.1f per ropa.yaml, so widening the flag is a founder call; consequence: the calibration corpus collects nothing until a company opts in, and the flag is UI-only (absent from v1 REST / MCP settings pick lists) because consent is a human action.
[2026-08-28] Bulk revenue rate-0 slot (#1912): item 1 (require a positive momsfri/export/EU classification) stays deferred because most real momsfri/export accounts are unconfigured today and refusing them needs a configure path first (the issue's own trade-off); resolveRevenueVatBox makes it a one-line switch later. The order-context check refuses only the export/EU-account direction (ruta 36 vs SE/EU country, ruta 40 vs SE, ruta 35/38/39 vs SE/non-EU), because the domestic-account + foreign-country direction is already advisory in the dialog (zero_rate_foreign) and hard-blocking it would regress every untemplated sweep; ruta 40 is 'omsatta utom landet', so an EU country does not contradict it. EXPORT_BOXES from moms-box-mapping was not reused because it merges EU and export boxes.
[2026-08-28] Verifikat page promotes "Stryk rader i verifikatet" to a visible outline button only when the period-status endpoint answers open (anything else or unknown keeps it in the ⋯ menu only), and the ⋯ item stays: inline rättelse is the normal path in an open period (#1554), the promoted button must never invite an action the RPC will refuse, and the menu remains the complete action list per the 2026-08-20 detail-page grammar. Struck-line actor labels are resolved server-side via resolveUserLabelsFromProfiles (profiles RLS is self-only) rather than a new column on the log.
[2026-08-28] Employee-save failure reported inline (role=alert in the dialog footer, carrying the requestId) in addition to the single destructive toast, and a missing PERSONNUMMER_ENCRYPTION_KEY typed as 503 PERSONNUMMER_ENCRYPTION_NOT_CONFIGURED rather than INTERNAL_ERROR (#1996): the Radix modal aria-hides the root-layout Toaster while the dialog is open, so the toast is invisible to assistive tech and E2E drivers; TOAST_LIMIT is 1, so a second toast is not an option; and the missing key is a permanent configuration gap where "try again later" is wrong and "contact support" is right (same reasoning as CUSTOMER_PERSONAL_NUMBER_UNREADABLE and INVOICE_SEND_EMAIL_NOT_CONFIGURED). The shared postAction helper was not extended (it takes no body and exposes no requestId): keeping the change local to the dialog avoids widening a helper other panels rely on.