feat(settings): per-company data-analysis opt-in gating the calibration corpus (#1346) (#2007)

* feat(settings): per-company opt-in for data analysis of bookkeeping outcomes (#1346)

Adds company_settings.data_analysis_opt_in (default false, no grandfathering)
and gates every path that reads bookkeeping outcomes across companies on it:
POST /api/agent/categorize/outcome stops writing calibration samples for
companies that have not opted in, and the backtest / calibration-fit scripts
filter to opted-in company ids. One helper (lib/company/data-analysis.ts)
is the single gate for future analysis paths. A toggle on Inställningar >
Företag states plainly what is analysed (proposed vs booked account, amount,
confidence; no free text, no personal data) in sv and en. The flag is UI-only
by design: consent is a human action, so it is absent from the v1 REST / MCP
settings pick lists.

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_015nAd8XJ2RPCmG2eKoLBdna

* fix(settings): make data-analysis consent copy true for the backtest path (#1346)

Addresses adversarial review findings on PR #2007:

- Findings 1-3 (consent narrower than the gated processing): the flag also
  gates scripts/backtest-categorize.ts, which re-runs transaction
  descriptions, merchant names and matched underlag through the model. The
  sv/en toggle help and disclosure now state that explicitly as "evaluation
  runs" and no longer claim that free text or underlag are excluded. The
  migration header and COMMENT, the lib/company/data-analysis.ts docstring,
  the backtest script header and the DECISIONS line say the same. Kept the
  gate (un-gating would put the script back to reading every company with
  no consent at all). A test pins that both locales name those inputs and
  contain no "no free text / no underlag" denial.
- Finding 4 (member sees an active switch that RLS rejects): the toggle is
  now enabled only for owner/admin, matching the company_settings update
  policy; the disclosure says only administrators can change the choice.

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_015nAd8XJ2RPCmG2eKoLBdna

* fix(scripts): address round-2 review findings (#1346)

1. [minor] Opted-in company filter was an unbounded PostgREST `in` list in
   the URL (scripts/fit-categorize-calibration.ts, scripts/backtest-categorize.ts).
   Both scripts now read the opted-in ids through a shared, paginated helper
   (listDataAnalysisOptedInCompanyIds, fetchAllRows so the pre-fetch no longer
   caps at 1000) and query per chunk of 100 ids (chunkCompanyIds). The fit
   script pages each chunk on the id PK; the backtest merges per-chunk
   results and re-cuts to the N most recent overall. Early exit on zero
   opt-ins is kept. Pinned with tests in lib/company/__tests__/data-analysis.test.ts.

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_015nAd8XJ2RPCmG2eKoLBdna

* fix(scripts): coerce a null transaction description in the backtest (#1346)

The typed row from the chunked consent query made description nullable,
which TransactionForSelect does not accept; fall back to the original
description or an empty string, as the untyped row did implicitly before.

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_015nAd8XJ2RPCmG2eKoLBdna

---------

Co-authored-by: Jakob Wennberg <311770904+jakobwennberg-oss@users.noreply.github.com>
Co-authored-by: Claude Fable 5 <noreply@anthropic.com>

lib/api/schemas.ts

@@ -2246,6 +2246,10 @@ export const UpdateSettingsSchema = z.object({
   // Körjournal (mileage log): UI-visibility toggle only, never load-bearing
   // for correctness (trips created via API/MCP work regardless).
   mileage_enabled: z.boolean().optional(),
+  // Data analysis consent (#1346): gates cross-company analysis of this
+  // company's bookkeeping outcomes. Flipped by a human in the settings UI
+  // only; deliberately absent from the v1 REST / MCP settings pick lists.
+  data_analysis_opt_in: z.boolean().optional(),
   // Salary payment file
   preferred_payment_format: z.enum(['bg_lb', 'pain001']).optional(),
   // Salary settings (migration 20260703190000). Day of month salaries are

lib/company/__tests__/data-analysis.test.ts

@@ -0,0 +1,121 @@
+import { describe, it, expect, beforeEach, vi } from 'vitest'
+import type { SupabaseClient } from '@supabase/supabase-js'
+import { createMockSupabase } from '@/tests/helpers'
+import { readFileSync } from 'node:fs'
+import { join } from 'node:path'
+import { createQueuedMockSupabase } from '@/tests/helpers'
+import {
+  isDataAnalysisOptedIn,
+  listDataAnalysisOptedInCompanyIds,
+  chunkCompanyIds,
+  OPTED_IN_COMPANY_ID_CHUNK,
+} from '../data-analysis'
+
+const { supabase, mockResult } = createMockSupabase()
+const client = supabase as unknown as SupabaseClient
+
+describe('isDataAnalysisOptedIn', () => {
+  beforeEach(() => {
+    vi.clearAllMocks()
+    mockResult({ data: null, error: null })
+  })
+
+  it('is true when the company has opted in', async () => {
+    mockResult({ data: { data_analysis_opt_in: true } })
+    expect(await isDataAnalysisOptedIn(client, 'company-1')).toBe(true)
+  })
+
+  it('is false when the company has not opted in', async () => {
+    mockResult({ data: { data_analysis_opt_in: false } })
+    expect(await isDataAnalysisOptedIn(client, 'company-1')).toBe(false)
+  })
+
+  it('is false when the settings row is missing', async () => {
+    mockResult({ data: null })
+    expect(await isDataAnalysisOptedIn(client, 'company-1')).toBe(false)
+  })
+
+  it('fails closed when the query errors', async () => {
+    mockResult({ data: { data_analysis_opt_in: true }, error: { message: 'boom' } })
+    expect(await isDataAnalysisOptedIn(client, 'company-1')).toBe(false)
+  })
+
+  it('reads company_settings for the given company', async () => {
+    mockResult({ data: { data_analysis_opt_in: true } })
+    await isDataAnalysisOptedIn(client, 'company-9')
+    expect(supabase.from).toHaveBeenCalledWith('company_settings')
+  })
+})
+
+describe('listDataAnalysisOptedInCompanyIds', () => {
+  // Round-2 review of #1346: the read-side scripts pre-fetched opted-in ids
+  // without paging (PostgREST caps at 1000) and then passed the whole list to
+  // one `.in()` (URL length). The helper pages, the scripts chunk.
+  it('pages through more than 1000 opted-in companies', async () => {
+    const { supabase, enqueue, findCall, findCalls } = createQueuedMockSupabase()
+    const page1 = Array.from({ length: 1000 }, (_, i) => ({ company_id: `c-${i}` }))
+    const page2 = [{ company_id: 'c-1000' }, { company_id: 'c-1001' }]
+    enqueue({ data: page1 })
+    enqueue({ data: page2 })
+    const ids = await listDataAnalysisOptedInCompanyIds(supabase as unknown as SupabaseClient)
+    expect(ids).toHaveLength(1002)
+    expect(ids[0]).toBe('c-0')
+    expect(ids[1001]).toBe('c-1001')
+    expect(findCall('company_settings', 'eq')).toEqual(['data_analysis_opt_in', true])
+    expect(findCalls('company_settings', 'range')).toEqual([[0, 999], [1000, 1999]])
+  })
+
+  it('returns an empty list when nobody has opted in', async () => {
+    const { supabase, enqueue } = createQueuedMockSupabase()
+    enqueue({ data: [] })
+    expect(await listDataAnalysisOptedInCompanyIds(supabase as unknown as SupabaseClient)).toEqual([])
+  })
+
+  it('throws on a query error instead of fitting on a partial corpus', async () => {
+    const { supabase, enqueue } = createQueuedMockSupabase()
+    enqueue({ data: null, error: { message: 'boom' } })
+    await expect(listDataAnalysisOptedInCompanyIds(supabase as unknown as SupabaseClient)).rejects.toThrow('boom')
+  })
+})
+
+describe('chunkCompanyIds', () => {
+  it('keeps every `.in()` list at or under the chunk size', () => {
+    const ids = Array.from({ length: 250 }, (_, i) => `id-${i}`)
+    const chunks = chunkCompanyIds(ids)
+    expect(OPTED_IN_COMPANY_ID_CHUNK).toBeLessThanOrEqual(100)
+    expect(chunks.map((c) => c.length)).toEqual([100, 100, 50])
+    expect(chunks.flat()).toEqual(ids)
+  })
+
+  it('returns no chunks for an empty list', () => {
+    expect(chunkCompanyIds([])).toEqual([])
+  })
+})
+
+describe('data analysis consent copy', () => {
+  // The flag also gates scripts/backtest-categorize.ts, which re-runs
+  // transaction descriptions, merchant names and matched underlag through
+  // the model. The consent copy must say so in both locales and must not
+  // claim that free text or underlag are excluded (review of #1346).
+  const locales = ['sv', 'en'] as const
+  const messages = {
+    sv: readFileSync(join(process.cwd(), 'messages/sv.json'), 'utf8'),
+    en: readFileSync(join(process.cwd(), 'messages/en.json'), 'utf8'),
+  }
+
+  it.each(locales)('%s names the evaluation-run inputs the backtest reads', (locale) => {
+    const { data_analysis } = JSON.parse(messages[locale]) as {
+      data_analysis: { settings_toggle_help: string; settings_disclosure: string }
+    }
+    const help = data_analysis.settings_toggle_help
+    const disclosure = data_analysis.settings_disclosure
+    const wordsFor = locale === 'sv'
+      ? { text: /transaktionstexter/, underlag: /underlag/, denial: /ingen fritext|inga underlag/i }
+      : { text: /transaction descriptions/, underlag: /supporting documents/, denial: /no free text|no supporting documents/i }
+    expect(help).toMatch(wordsFor.text)
+    expect(help).toMatch(wordsFor.underlag)
+    expect(help).not.toMatch(wordsFor.denial)
+    expect(disclosure).toMatch(wordsFor.text)
+    expect(disclosure).not.toMatch(wordsFor.denial)
+  })
+})

lib/company/data-analysis.ts

@@ -0,0 +1,72 @@
+import type { SupabaseClient } from '@supabase/supabase-js'
+import { fetchAllRows } from '@/lib/supabase/fetch-all'
+
+/**
+ * Data analysis consent gate (#1346).
+ *
+ * Returns true only when the company has explicitly opted in to having its
+ * bookkeeping data read across companies for Accounted's own analysis. The
+ * consent copy (messages/*.json data_analysis.*) states two scopes and this
+ * flag covers both: (1) booking outcomes (proposed vs booked account,
+ * confidence, amount) for the auto-booking calibration corpus and the
+ * calibration-fit script; (2) evaluation runs (scripts/backtest-categorize.ts)
+ * that re-run the company's transaction descriptions, counterparty names and
+ * matched underlag through the same AI model as regular booking. Anything
+ * wider than that needs new consent copy first, not just a new caller.
+ * `company_settings.data_analysis_opt_in` is the single source of truth;
+ * every analysis path checks it so a company that never opted in (or opted
+ * out again) contributes nothing.
+ *
+ * Fails closed: a missing row or a query error counts as "not opted in".
+ */
+export async function isDataAnalysisOptedIn(
+  supabase: SupabaseClient,
+  companyId: string,
+): Promise<boolean> {
+  const { data, error } = await supabase
+    .from('company_settings')
+    .select('data_analysis_opt_in')
+    .eq('company_id', companyId)
+    .maybeSingle()
+  if (error) return false
+  return data?.data_analysis_opt_in === true
+}
+
+/**
+ * Upper bound on how many company ids a single PostgREST `in.(...)` filter may
+ * carry. supabase-js encodes the list into the GET query string, so an
+ * unbounded list of 36-char UUIDs blows past common URL limits (~16 KB, a few
+ * hundred ids) with a 414/400. Read-side consumers of the consent flag
+ * (the calibration-fit and backtest scripts) must query per chunk.
+ */
+export const OPTED_IN_COMPANY_ID_CHUNK = 100
+
+/**
+ * Every company_id with data_analysis_opt_in = true, paginated so the list is
+ * not silently capped at PostgREST's 1000-row default. Throws on query error:
+ * the callers are founder-run scripts that should fail loudly, not fit on a
+ * partial corpus. Ordered by company_id for stable paging.
+ */
+export async function listDataAnalysisOptedInCompanyIds(
+  supabase: SupabaseClient,
+): Promise<string[]> {
+  const rows = await fetchAllRows<{ company_id: string }>(({ from, to }) =>
+    supabase
+      .from('company_settings')
+      .select('company_id')
+      .eq('data_analysis_opt_in', true)
+      .order('company_id', { ascending: true })
+      .range(from, to),
+  )
+  return rows.map((r) => r.company_id)
+}
+
+/** Split ids into `.in('company_id', chunk)`-sized batches. */
+export function chunkCompanyIds(
+  ids: readonly string[],
+  size: number = OPTED_IN_COMPANY_ID_CHUNK,
+): string[][] {
+  const out: string[][] = []
+  for (let i = 0; i < ids.length; i += size) out.push(ids.slice(i, i + size))
+  return out
+}