feat(settings): per-company data-analysis opt-in gating the calibration corpus (#1346) (#2007)

* feat(settings): per-company opt-in for data analysis of bookkeeping outcomes (#1346)

Adds company_settings.data_analysis_opt_in (default false, no grandfathering)
and gates every path that reads bookkeeping outcomes across companies on it:
POST /api/agent/categorize/outcome stops writing calibration samples for
companies that have not opted in, and the backtest / calibration-fit scripts
filter to opted-in company ids. One helper (lib/company/data-analysis.ts)
is the single gate for future analysis paths. A toggle on Inställningar >
Företag states plainly what is analysed (proposed vs booked account, amount,
confidence; no free text, no personal data) in sv and en. The flag is UI-only
by design: consent is a human action, so it is absent from the v1 REST / MCP
settings pick lists.

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_015nAd8XJ2RPCmG2eKoLBdna

* fix(settings): make data-analysis consent copy true for the backtest path (#1346)

Addresses adversarial review findings on PR #2007:

- Findings 1-3 (consent narrower than the gated processing): the flag also
  gates scripts/backtest-categorize.ts, which re-runs transaction
  descriptions, merchant names and matched underlag through the model. The
  sv/en toggle help and disclosure now state that explicitly as "evaluation
  runs" and no longer claim that free text or underlag are excluded. The
  migration header and COMMENT, the lib/company/data-analysis.ts docstring,
  the backtest script header and the DECISIONS line say the same. Kept the
  gate (un-gating would put the script back to reading every company with
  no consent at all). A test pins that both locales name those inputs and
  contain no "no free text / no underlag" denial.
- Finding 4 (member sees an active switch that RLS rejects): the toggle is
  now enabled only for owner/admin, matching the company_settings update
  policy; the disclosure says only administrators can change the choice.

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_015nAd8XJ2RPCmG2eKoLBdna

* fix(scripts): address round-2 review findings (#1346)

1. [minor] Opted-in company filter was an unbounded PostgREST `in` list in
   the URL (scripts/fit-categorize-calibration.ts, scripts/backtest-categorize.ts).
   Both scripts now read the opted-in ids through a shared, paginated helper
   (listDataAnalysisOptedInCompanyIds, fetchAllRows so the pre-fetch no longer
   caps at 1000) and query per chunk of 100 ids (chunkCompanyIds). The fit
   script pages each chunk on the id PK; the backtest merges per-chunk
   results and re-cuts to the N most recent overall. Early exit on zero
   opt-ins is kept. Pinned with tests in lib/company/__tests__/data-analysis.test.ts.

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_015nAd8XJ2RPCmG2eKoLBdna

* fix(scripts): coerce a null transaction description in the backtest (#1346)

The typed row from the chunked consent query made description nullable,
which TransactionForSelect does not accept; fall back to the original
description or an empty string, as the untyped row did implicitly before.

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_015nAd8XJ2RPCmG2eKoLBdna

---------

Co-authored-by: Jakob Wennberg <311770904+jakobwennberg-oss@users.noreply.github.com>
Co-authored-by: Claude Fable 5 <noreply@anthropic.com>

DECISIONS.md

@@ -1323,6 +1323,7 @@ One line per decision: `[YYYY-MM-DD] <decision>: <why>`. Appended by agents and
 [2026-08-28] AR-PDF minus fix uses ASCII hyphen formatting, not font embedding: registering a Unicode TTF for react-pdf would change the whole document's typography and bundle size to fix one glyph; formatPdfKronor keeps built-in Helvetica and sidesteps WinAnsi's missing U+2212.
 [2026-08-28] Same-bank warning limited to observed one-session banks (SEB only): prod shows Handelsbanken tolerates 4 concurrent sessions, and the generic warning made a user abandon a legitimate renewal. Planned sync-death visibility work was dropped: already shipped via #1271 (health probe), #1727 (stale state), #1969 (cron unstarve).
 [2026-08-28] Same-bank warning revised to three tiers after skeptic refutation: hard warn SEB, silent/calm only for verified multi-session banks (Handelsbanken, 4 distinct session_ids observed), legacy hedged warning for unknown banks (fail closed), shared-session siblings exempt (fan-out carries them).
+[2026-08-28] data_analysis_opt_in (#1346) defaults false with no grandfathering and gates only bookkeeping-content analysis (categorize_calibration_samples via POST /api/agent/categorize/outcome, the calibration-fit script, and the backtest script, whose wider inputs (transaction descriptions, merchant names, matched underlag re-run through the model) are named in the consent copy as "evaluation runs" after review found the outcomes-only wording false for that path; the toggle is admin/owner only to match the company_settings RLS update policy), not PostHog product analytics or MCP reliability telemetry: those carry no bookkeeping content and stay under Art. 6.1f per ropa.yaml, so widening the flag is a founder call; consequence: the calibration corpus collects nothing until a company opts in, and the flag is UI-only (absent from v1 REST / MCP settings pick lists) because consent is a human action.
 [2026-08-28] Bulk revenue rate-0 slot (#1912): item 1 (require a positive momsfri/export/EU classification) stays deferred because most real momsfri/export accounts are unconfigured today and refusing them needs a configure path first (the issue's own trade-off); resolveRevenueVatBox makes it a one-line switch later. The order-context check refuses only the export/EU-account direction (ruta 36 vs SE/EU country, ruta 40 vs SE, ruta 35/38/39 vs SE/non-EU), because the domestic-account + foreign-country direction is already advisory in the dialog (zero_rate_foreign) and hard-blocking it would regress every untemplated sweep; ruta 40 is 'omsatta utom landet', so an EU country does not contradict it. EXPORT_BOXES from moms-box-mapping was not reused because it merges EU and export boxes.
 [2026-08-28] Verifikat page promotes "Stryk rader i verifikatet" to a visible outline button only when the period-status endpoint answers open (anything else or unknown keeps it in the ⋯ menu only), and the ⋯ item stays: inline rättelse is the normal path in an open period (#1554), the promoted button must never invite an action the RPC will refuse, and the menu remains the complete action list per the 2026-08-20 detail-page grammar. Struck-line actor labels are resolved server-side via resolveUserLabelsFromProfiles (profiles RLS is self-only) rather than a new column on the log.
 [2026-08-28] Employee-save failure reported inline (role=alert in the dialog footer, carrying the requestId) in addition to the single destructive toast, and a missing PERSONNUMMER_ENCRYPTION_KEY typed as 503 PERSONNUMMER_ENCRYPTION_NOT_CONFIGURED rather than INTERNAL_ERROR (#1996): the Radix modal aria-hides the root-layout Toaster while the dialog is open, so the toast is invisible to assistive tech and E2E drivers; TOAST_LIMIT is 1, so a second toast is not an option; and the missing key is a permanent configuration gap where "try again later" is wrong and "contact support" is right (same reasoning as CUSTOMER_PERSONAL_NUMBER_UNREADABLE and INVOICE_SEND_EMAIL_NOT_CONFIGURED). The shared postAction helper was not extended (it takes no body and exposes no requestId): keeping the change local to the dialog avoids widening a helper other panels rely on.

app/api/agent/categorize/outcome/__tests__/route.test.ts

@@ -11,13 +11,18 @@ vi.mock('@/lib/sandbox/guard', () => ({ guardSandbox: () => guardSandbox() }))
 import { POST } from '../route'
 
 const inserts: Record<string, unknown>[] = []
-function makeSupabase(membership: unknown = { user_id: 'user-1' }) {
+// company_settings.maybeSingle() feeds the consent gate; company_members
+// feeds the membership check. Default: a member of an opted-in company.
+function makeSupabase(membership: unknown = { user_id: 'user-1' }, optedIn = true) {
   return {
     from(table: string) {
       const chain = {
         select: () => chain,
         eq: () => chain,
-        maybeSingle: async () => ({ data: membership }),
+        maybeSingle: async () =>
+          table === 'company_settings'
+            ? { data: { data_analysis_opt_in: optedIn }, error: null }
+            : { data: membership, error: null },
         insert: async (payload: Record<string, unknown>) => {
           if (table === 'categorize_calibration_samples') inserts.push(payload)
           return { error: null }
@@ -61,7 +66,16 @@ describe('POST /api/agent/categorize/outcome', () => {
     expect(inserts).toHaveLength(0)
   })
 
-  it('logs was_correct=true when the proposed account was booked', async () => {
+  it('skips companies that have not opted in to data analysis (204, no sample)', async () => {
+    requireAuthMock.mockResolvedValue({ user: { id: 'user-1' }, supabase: makeSupabase({ user_id: 'user-1' }, false), error: null })
+    const res = await POST(
+      createMockRequest('/x', { method: 'POST', body: body({ proposed_account: '5410', booked_account: '5410' }) }),
+    )
+    expect(res.status).toBe(204)
+    expect(inserts).toHaveLength(0)
+  })
+
+  it('logs was_correct=true when the proposed account was booked (opted-in company)', async () => {
     const res = await POST(
       createMockRequest('/x', {
         method: 'POST',

app/api/agent/categorize/outcome/route.ts

@@ -2,6 +2,7 @@ import { NextResponse } from 'next/server'
 import { z } from 'zod'
 import { requireAuth } from '@/lib/auth/require-auth'
 import { getActiveCompanyId } from '@/lib/company/context'
+import { isDataAnalysisOptedIn } from '@/lib/company/data-analysis'
 import { guardSandbox } from '@/lib/sandbox/guard'
 
 /**
@@ -14,7 +15,10 @@ import { guardSandbox } from '@/lib/sandbox/guard'
  *
  * Telemetry only: it never posts anything and is gated on auth + membership.
  * Sandbox bookings run on seed data, so they are silently skipped (a 204) to
- * keep the corpus clean.
+ * keep the corpus clean. The corpus is read across companies, so it only
+ * collects from companies that opted in to data analysis
+ * (company_settings.data_analysis_opt_in, #1346): everyone else gets the same
+ * silent 204 and no row.
  */
 
 const Schema = z.object({
@@ -58,6 +62,10 @@ export async function POST(request: Request): Promise<Response> {
   const blocked = await guardSandbox(supabase, companyId)
   if (blocked) return noContent()
 
+  // Consent gate: the corpus is analysed across companies, so a company that
+  // has not opted in contributes nothing (default false, no grandfathering).
+  if (!(await isDataAnalysisOptedIn(supabase, companyId))) return noContent()
+
   const proposed = parsed.data.proposed_account ?? null
 
   // Best-effort: a failed telemetry insert must never surface to the user.

app/api/settings/__tests__/route.test.ts

@@ -116,6 +116,39 @@ describe('PUT /api/settings', () => {
     expect(deadlineMocks.regenerate).not.toHaveBeenCalled()
   })
 
+  it('accepts the data_analysis_opt_in consent toggle', async () => {
+    enqueueMany([
+      { data: { entity_type: 'enskild_firma', onboarding_complete: true } }, // fetch oldSettings
+      { data: { id: 's1', data_analysis_opt_in: true } },                     // update ... returning
+      { data: null, count: 5 },                                               // deadlines count (has some -> no regen)
+    ])
+
+    const request = createMockRequest('/api/settings', {
+      method: 'PUT',
+      body: { data_analysis_opt_in: true },
+    })
+    const response = await PUT(request, { params: Promise.resolve({}) })
+    const { status, body } = await parseJsonResponse<{ data: { data_analysis_opt_in: boolean } }>(response)
+
+    expect(status).toBe(200)
+    expect(body.data.data_analysis_opt_in).toBe(true)
+    expect(deadlineMocks.regenerate).not.toHaveBeenCalled()
+  })
+
+  it('rejects a non-boolean data_analysis_opt_in value', async () => {
+    enqueueMany([
+      { data: { onboarding_complete: true } }, // oldSettings
+    ])
+
+    const request = createMockRequest('/api/settings', {
+      method: 'PUT',
+      body: { data_analysis_opt_in: 'yes' },
+    })
+    const response = await PUT(request, { params: Promise.resolve({}) })
+
+    expect(response.status).toBe(400)
+  })
+
   it('round-trips share capital fields and clears them with null', async () => {
     const updates = { aktiekapital: 25000, antal_aktier: 500 }
     enqueueMany([

components/settings/DataAnalysisToggle.tsx

@@ -0,0 +1,97 @@
+'use client'
+
+import { useState } from 'react'
+import Link from 'next/link'
+import { useLocale, useTranslations } from 'next-intl'
+import { Switch } from '@/components/ui/switch'
+import { useToast } from '@/components/ui/use-toast'
+import { SettingsRow, SettingsRowNote } from '@/components/settings/SettingsRows'
+import { useSettings } from '@/components/settings/useSettings'
+import { useCompany } from '@/contexts/CompanyContext'
+import { cn } from '@/lib/utils'
+import { getErrorMessage, type ErrorLocale } from '@/lib/errors/get-error-message'
+
+/**
+ * Company-level consent toggle for data analysis (#1346). Persists
+ * company_settings.data_analysis_opt_in through the standard settings PUT.
+ * The flag is enforced server-side (lib/company/data-analysis.ts) on every
+ * path that reads bookkeeping outcomes across companies; this component only
+ * mirrors it and states plainly what is analysed. Off by default.
+ *
+ * Owner / admin only: the company_settings RLS update policy is admin-gated,
+ * so a plain member's PUT would fail with a misleading error. Consent is an
+ * admin decision anyway, so the switch renders disabled for everyone else.
+ */
+export function DataAnalysisToggle() {
+  const t = useTranslations('data_analysis')
+  const errorLocale = useLocale() as ErrorLocale
+  const { settings, updateSettings } = useSettings()
+  const { role } = useCompany()
+  const canConsent = role === 'owner' || role === 'admin'
+  const { toast } = useToast()
+  const [isSaving, setIsSaving] = useState(false)
+
+  const enabled = settings?.data_analysis_opt_in ?? false
+
+  async function handleChange(next: boolean) {
+    setIsSaving(true)
+    try {
+      const res = await fetch('/api/settings', {
+        method: 'PUT',
+        headers: { 'Content-Type': 'application/json' },
+        body: JSON.stringify({ data_analysis_opt_in: next }),
+      })
+      const json = await res.json().catch(() => null)
+      if (!res.ok) {
+        toast({
+          title: t('settings_save_failed_title'),
+          description: getErrorMessage(json, { locale: errorLocale }),
+          variant: 'destructive',
+        })
+        return
+      }
+      updateSettings({ data_analysis_opt_in: next })
+    } catch (err) {
+      // A rejected fetch never reaches the !res.ok arm above, and the switch
+      // is controlled by the settings context, so it stays where it was:
+      // without this toast the click looks like a dead control.
+      toast({
+        title: t('settings_save_failed_title'),
+        description: getErrorMessage(err, { locale: errorLocale }),
+        variant: 'destructive',
+      })
+    } finally {
+      setIsSaving(false)
+    }
+  }
+
+  const locked = isSaving || !canConsent
+
+  return (
+    <SettingsRow label={t('settings_heading')} help={t('settings_toggle_help')}>
+      <Switch
+        id="data-analysis-opt-in"
+        checked={enabled}
+        onCheckedChange={(next) => void handleChange(next)}
+        disabled={locked}
+      />
+      <label
+        htmlFor="data-analysis-opt-in"
+        className={cn('text-sm', locked ? 'text-muted-foreground' : 'cursor-pointer')}
+      >
+        {t('settings_toggle_label')}
+      </label>
+      <SettingsRowNote className="basis-full">
+        {t('settings_disclosure')}{' '}
+        <Link
+          href="/privacy"
+          target="_blank"
+          rel="noopener noreferrer"
+          className="underline underline-offset-4 transition-colors hover:text-foreground"
+        >
+          {t('settings_privacy_link')}
+        </Link>
+      </SettingsRowNote>
+    </SettingsRow>
+  )
+}

components/settings/sections/CompanySettingsContent.tsx

@@ -7,12 +7,13 @@ import { CompanyDangerZone } from '@/components/settings/CompanyDangerZone'
 import { CompanyInfoForm } from '@/components/settings/CompanyInfoForm'
 import { CompanyMembersSection } from '@/components/settings/CompanyMembersSection'
 import { CompanyProfileSection } from '@/components/settings/CompanyProfileSection'
+import { DataAnalysisToggle } from '@/components/settings/DataAnalysisToggle'
 import { FiscalPeriodEditor } from '@/components/settings/FiscalPeriodEditor'
 import { LogoUpload } from '@/components/settings/LogoUpload'
 import { SettingsFormWrapper } from '@/components/settings/SettingsFormWrapper'
 import { SettingsLoadError } from '@/components/settings/SettingsLoadError'
 import { SettingsLoadingSkeleton } from '@/components/settings/SettingsLoadingSkeleton'
-import { SettingsSectionHeader } from '@/components/settings/SettingsRows'
+import { SettingsGroup, SettingsSectionHeader } from '@/components/settings/SettingsRows'
 import { ShareCapitalForm } from '@/components/settings/ShareCapitalForm'
 import { useSettings } from '@/components/settings/useSettings'
 import type { CompanySettings } from '@/types'
@@ -21,6 +22,7 @@ export function CompanySettingsContent() {
   const router = useRouter()
   const tNav = useTranslations('settings_nav')
   const tIntro = useTranslations('settings_intro')
+  const tData = useTranslations('data_analysis')
   const { settings, isLoading, updateSettings, refetch } = useSettings()
 
   // Deep-link target for "Medlemmar och roller" (/settings/company#members):
@@ -97,6 +99,10 @@ export function CompanySettingsContent() {
 
       <CompanyProfileSection />
 
+      <SettingsGroup label={tData('group_label')}>
+        <DataAnalysisToggle />
+      </SettingsGroup>
+
       <CompanyDangerZone />
     </div>
   )

lib/api/schemas.ts

@@ -2246,6 +2246,10 @@ export const UpdateSettingsSchema = z.object({
   // Körjournal (mileage log): UI-visibility toggle only, never load-bearing
   // for correctness (trips created via API/MCP work regardless).
   mileage_enabled: z.boolean().optional(),
+  // Data analysis consent (#1346): gates cross-company analysis of this
+  // company's bookkeeping outcomes. Flipped by a human in the settings UI
+  // only; deliberately absent from the v1 REST / MCP settings pick lists.
+  data_analysis_opt_in: z.boolean().optional(),
   // Salary payment file
   preferred_payment_format: z.enum(['bg_lb', 'pain001']).optional(),
   // Salary settings (migration 20260703190000). Day of month salaries are

lib/company/__tests__/data-analysis.test.ts

@@ -0,0 +1,121 @@
+import { describe, it, expect, beforeEach, vi } from 'vitest'
+import type { SupabaseClient } from '@supabase/supabase-js'
+import { createMockSupabase } from '@/tests/helpers'
+import { readFileSync } from 'node:fs'
+import { join } from 'node:path'
+import { createQueuedMockSupabase } from '@/tests/helpers'
+import {
+  isDataAnalysisOptedIn,
+  listDataAnalysisOptedInCompanyIds,
+  chunkCompanyIds,
+  OPTED_IN_COMPANY_ID_CHUNK,
+} from '../data-analysis'
+
+const { supabase, mockResult } = createMockSupabase()
+const client = supabase as unknown as SupabaseClient
+
+describe('isDataAnalysisOptedIn', () => {
+  beforeEach(() => {
+    vi.clearAllMocks()
+    mockResult({ data: null, error: null })
+  })
+
+  it('is true when the company has opted in', async () => {
+    mockResult({ data: { data_analysis_opt_in: true } })
+    expect(await isDataAnalysisOptedIn(client, 'company-1')).toBe(true)
+  })
+
+  it('is false when the company has not opted in', async () => {
+    mockResult({ data: { data_analysis_opt_in: false } })
+    expect(await isDataAnalysisOptedIn(client, 'company-1')).toBe(false)
+  })
+
+  it('is false when the settings row is missing', async () => {
+    mockResult({ data: null })
+    expect(await isDataAnalysisOptedIn(client, 'company-1')).toBe(false)
+  })
+
+  it('fails closed when the query errors', async () => {
+    mockResult({ data: { data_analysis_opt_in: true }, error: { message: 'boom' } })
+    expect(await isDataAnalysisOptedIn(client, 'company-1')).toBe(false)
+  })
+
+  it('reads company_settings for the given company', async () => {
+    mockResult({ data: { data_analysis_opt_in: true } })
+    await isDataAnalysisOptedIn(client, 'company-9')
+    expect(supabase.from).toHaveBeenCalledWith('company_settings')
+  })
+})
+
+describe('listDataAnalysisOptedInCompanyIds', () => {
+  // Round-2 review of #1346: the read-side scripts pre-fetched opted-in ids
+  // without paging (PostgREST caps at 1000) and then passed the whole list to
+  // one `.in()` (URL length). The helper pages, the scripts chunk.
+  it('pages through more than 1000 opted-in companies', async () => {
+    const { supabase, enqueue, findCall, findCalls } = createQueuedMockSupabase()
+    const page1 = Array.from({ length: 1000 }, (_, i) => ({ company_id: `c-${i}` }))
+    const page2 = [{ company_id: 'c-1000' }, { company_id: 'c-1001' }]
+    enqueue({ data: page1 })
+    enqueue({ data: page2 })
+    const ids = await listDataAnalysisOptedInCompanyIds(supabase as unknown as SupabaseClient)
+    expect(ids).toHaveLength(1002)
+    expect(ids[0]).toBe('c-0')
+    expect(ids[1001]).toBe('c-1001')
+    expect(findCall('company_settings', 'eq')).toEqual(['data_analysis_opt_in', true])
+    expect(findCalls('company_settings', 'range')).toEqual([[0, 999], [1000, 1999]])
+  })
+
+  it('returns an empty list when nobody has opted in', async () => {
+    const { supabase, enqueue } = createQueuedMockSupabase()
+    enqueue({ data: [] })
+    expect(await listDataAnalysisOptedInCompanyIds(supabase as unknown as SupabaseClient)).toEqual([])
+  })
+
+  it('throws on a query error instead of fitting on a partial corpus', async () => {
+    const { supabase, enqueue } = createQueuedMockSupabase()
+    enqueue({ data: null, error: { message: 'boom' } })
+    await expect(listDataAnalysisOptedInCompanyIds(supabase as unknown as SupabaseClient)).rejects.toThrow('boom')
+  })
+})
+
+describe('chunkCompanyIds', () => {
+  it('keeps every `.in()` list at or under the chunk size', () => {
+    const ids = Array.from({ length: 250 }, (_, i) => `id-${i}`)
+    const chunks = chunkCompanyIds(ids)
+    expect(OPTED_IN_COMPANY_ID_CHUNK).toBeLessThanOrEqual(100)
+    expect(chunks.map((c) => c.length)).toEqual([100, 100, 50])
+    expect(chunks.flat()).toEqual(ids)
+  })
+
+  it('returns no chunks for an empty list', () => {
+    expect(chunkCompanyIds([])).toEqual([])
+  })
+})
+
+describe('data analysis consent copy', () => {
+  // The flag also gates scripts/backtest-categorize.ts, which re-runs
+  // transaction descriptions, merchant names and matched underlag through
+  // the model. The consent copy must say so in both locales and must not
+  // claim that free text or underlag are excluded (review of #1346).
+  const locales = ['sv', 'en'] as const
+  const messages = {
+    sv: readFileSync(join(process.cwd(), 'messages/sv.json'), 'utf8'),
+    en: readFileSync(join(process.cwd(), 'messages/en.json'), 'utf8'),
+  }
+
+  it.each(locales)('%s names the evaluation-run inputs the backtest reads', (locale) => {
+    const { data_analysis } = JSON.parse(messages[locale]) as {
+      data_analysis: { settings_toggle_help: string; settings_disclosure: string }
+    }
+    const help = data_analysis.settings_toggle_help
+    const disclosure = data_analysis.settings_disclosure
+    const wordsFor = locale === 'sv'
+      ? { text: /transaktionstexter/, underlag: /underlag/, denial: /ingen fritext|inga underlag/i }
+      : { text: /transaction descriptions/, underlag: /supporting documents/, denial: /no free text|no supporting documents/i }
+    expect(help).toMatch(wordsFor.text)
+    expect(help).toMatch(wordsFor.underlag)
+    expect(help).not.toMatch(wordsFor.denial)
+    expect(disclosure).toMatch(wordsFor.text)
+    expect(disclosure).not.toMatch(wordsFor.denial)
+  })
+})

lib/company/data-analysis.ts

@@ -0,0 +1,72 @@
+import type { SupabaseClient } from '@supabase/supabase-js'
+import { fetchAllRows } from '@/lib/supabase/fetch-all'
+
+/**
+ * Data analysis consent gate (#1346).
+ *
+ * Returns true only when the company has explicitly opted in to having its
+ * bookkeeping data read across companies for Accounted's own analysis. The
+ * consent copy (messages/*.json data_analysis.*) states two scopes and this
+ * flag covers both: (1) booking outcomes (proposed vs booked account,
+ * confidence, amount) for the auto-booking calibration corpus and the
+ * calibration-fit script; (2) evaluation runs (scripts/backtest-categorize.ts)
+ * that re-run the company's transaction descriptions, counterparty names and
+ * matched underlag through the same AI model as regular booking. Anything
+ * wider than that needs new consent copy first, not just a new caller.
+ * `company_settings.data_analysis_opt_in` is the single source of truth;
+ * every analysis path checks it so a company that never opted in (or opted
+ * out again) contributes nothing.
+ *
+ * Fails closed: a missing row or a query error counts as "not opted in".
+ */
+export async function isDataAnalysisOptedIn(
+  supabase: SupabaseClient,
+  companyId: string,
+): Promise<boolean> {
+  const { data, error } = await supabase
+    .from('company_settings')
+    .select('data_analysis_opt_in')
+    .eq('company_id', companyId)
+    .maybeSingle()
+  if (error) return false
+  return data?.data_analysis_opt_in === true
+}
+
+/**
+ * Upper bound on how many company ids a single PostgREST `in.(...)` filter may
+ * carry. supabase-js encodes the list into the GET query string, so an
+ * unbounded list of 36-char UUIDs blows past common URL limits (~16 KB, a few
+ * hundred ids) with a 414/400. Read-side consumers of the consent flag
+ * (the calibration-fit and backtest scripts) must query per chunk.
+ */
+export const OPTED_IN_COMPANY_ID_CHUNK = 100
+
+/**
+ * Every company_id with data_analysis_opt_in = true, paginated so the list is
+ * not silently capped at PostgREST's 1000-row default. Throws on query error:
+ * the callers are founder-run scripts that should fail loudly, not fit on a
+ * partial corpus. Ordered by company_id for stable paging.
+ */
+export async function listDataAnalysisOptedInCompanyIds(
+  supabase: SupabaseClient,
+): Promise<string[]> {
+  const rows = await fetchAllRows<{ company_id: string }>(({ from, to }) =>
+    supabase
+      .from('company_settings')
+      .select('company_id')
+      .eq('data_analysis_opt_in', true)
+      .order('company_id', { ascending: true })
+      .range(from, to),
+  )
+  return rows.map((r) => r.company_id)
+}
+
+/** Split ids into `.in('company_id', chunk)`-sized batches. */
+export function chunkCompanyIds(
+  ids: readonly string[],
+  size: number = OPTED_IN_COMPANY_ID_CHUNK,
+): string[][] {
+  const out: string[][] = []
+  for (let i = 0; i < ids.length; i += size) out.push(ids.slice(i, i + size))
+  return out
+}

messages/en.json

@@ -8355,6 +8355,15 @@
     "settings_save_failed_title": "Could not save the setting",
     "settings_open_page": "Open the driving log"
   },
+  "data_analysis": {
+    "group_label": "Data and privacy",
+    "settings_heading": "Data analysis",
+    "settings_toggle_label": "Allow the company's bookkeeping data to be used for analysis",
+    "settings_toggle_help": "Off by default. When on, the company's bookkeeping data is used together with other companies that made the same choice, to evaluate and improve automatic booking. Two things are included. Continuously: booking outcomes, that is which account was proposed, which account was actually booked, the amount and the model's confidence. During Accounted's own evaluation runs: also transaction descriptions, counterparty names and data from matched supporting documents (receipts and invoices), which are then re-run through the same AI model and the same processors as regular booking. Such text can contain personal data, for example names in Swish payments. The data is never used for marketing and never leaves Accounted or the processors listed in the privacy policy. Switching it off stops collection immediately.",
+    "settings_disclosure": "Booking outcomes (proposed and booked account, amount, confidence) and, during evaluation runs, transaction descriptions and matched supporting documents. Only administrators can change this choice. Read more in our",
+    "settings_privacy_link": "privacy policy",
+    "settings_save_failed_title": "Could not save the setting"
+  },
   "reconciliation_underlag": {
     "heading": "Supporting documents",
     "heading_dated": "Supporting documents as of {date}",

messages/sv.json

@@ -8355,6 +8355,15 @@
     "settings_save_failed_title": "Kunde inte spara inställningen",
     "settings_open_page": "Öppna körjournalen"
   },
+  "data_analysis": {
+    "group_label": "Data och integritet",
+    "settings_heading": "Dataanalys",
+    "settings_toggle_label": "Tillåt att företagets bokföringsdata används för analys",
+    "settings_toggle_help": "Avstängt från början. När det är på används företagets bokföringsdata tillsammans med andra företag som valt samma sak, för att utvärdera och förbättra den automatiska konteringen. Två saker ingår. Löpande: konteringsutfall, det vill säga vilket konto som föreslogs, vilket konto som faktiskt bokfördes, beloppet och modellens säkerhet. Vid Accounteds egna utvärderingskörningar: även transaktionstexter, motpartsnamn och uppgifter ur matchade underlag (kvitton och fakturor), som då körs på nytt genom samma AI-modell och samma personuppgiftsbiträden som den vanliga konteringen. Sådana texter kan innehålla personuppgifter, till exempel namn i Swish-betalningar. Uppgifterna används aldrig för marknadsföring och lämnar inte Accounted eller de personuppgiftsbiträden som listas i integritetspolicyn. Stänger du av slutar insamlingen direkt.",
+    "settings_disclosure": "Konteringsutfall (föreslaget och bokfört konto, belopp, säkerhet) samt, vid utvärderingskörningar, transaktionstexter och matchade underlag. Endast administratörer kan ändra valet. Läs mer i vår",
+    "settings_privacy_link": "integritetspolicy",
+    "settings_save_failed_title": "Kunde inte spara inställningen"
+  },
   "reconciliation_underlag": {
     "heading": "Underlag",
     "heading_dated": "Underlag per {date}",

scripts/backtest-categorize.ts

@@ -13,6 +13,15 @@
  *   npx tsx scripts/backtest-categorize.ts [N]
  *   rm .env.local
  *
+ * Consent: only companies with company_settings.data_analysis_opt_in = true
+ * are read (#1346). This script goes beyond booking outcomes: it reads each
+ * transaction's description, merchant name and matched underlag (via
+ * gatherUnderlag) and sends them to the model again, so the consent copy in
+ * messages/*.json (data_analysis.settings_toggle_help) explicitly names
+ * "evaluation runs" with exactly those inputs. Do not add inputs here that
+ * the copy does not name. Nobody is opted in by default, so an empty run is
+ * the expected state until an admin flips the toggle in Inställningar > Företag.
+ *
  * Leakage caveat: a known vendor's counterparty template may already reflect
  * the very booking under test, inflating the "deterministic nailed it" segment.
  * The "model had to decide" segment below is the leakage-free measure.
@@ -29,23 +38,53 @@ async function main() {
   const { gatherCandidates } = await import('../lib/agent/categorize/candidates')
   const { gatherUnderlag } = await import('../lib/agent/categorize/underlag')
   const { selectAccount } = await import('../lib/agent/categorize/select-account')
+  const { chunkCompanyIds, listDataAnalysisOptedInCompanyIds } = await import('../lib/company/data-analysis')
 
   const url = process.env.NEXT_PUBLIC_SUPABASE_URL!
   const key = process.env.SUPABASE_SERVICE_ROLE_KEY!
   const supabase = createClient(url, key)
 
-  // Recent booked expense transactions with a counterparty.
-  const { data: txs, error } = await supabase
-    .from('transactions')
-    .select('id, company_id, merchant_name, description, original_description, amount, date, currency, document_id, journal_entry_id')
-    .not('journal_entry_id', 'is', null)
-    .lt('amount', 0)
-    .eq('is_business', true)
-    .not('merchant_name', 'is', null)
-    .order('created_at', { ascending: false })
-    .limit(N)
-  if (error) throw error
-  const rows = txs ?? []
+  // Consent gate (#1346): only companies that opted in to data analysis.
+  const optedInIds = await listDataAnalysisOptedInCompanyIds(supabase)
+  if (optedInIds.length === 0) {
+    console.log('\nNo company has opted in to data analysis (company_settings.data_analysis_opt_in). Nothing to backtest.')
+    return
+  }
+
+  // Recent booked expense transactions with a counterparty. Queried per chunk
+  // of company ids (`.in()` lives in the GET query string), then merged and
+  // re-cut to the N most recent overall.
+  type Tx = {
+    id: string
+    company_id: string
+    merchant_name: string | null
+    description: string | null
+    original_description: string | null
+    amount: number
+    date: string
+    currency: string | null
+    document_id: string | null
+    journal_entry_id: string | null
+    created_at: string
+  }
+  const candidatesByChunk: Tx[] = []
+  for (const chunk of chunkCompanyIds(optedInIds)) {
+    const { data: txs, error } = await supabase
+      .from('transactions')
+      .select('id, company_id, merchant_name, description, original_description, amount, date, currency, document_id, journal_entry_id, created_at')
+      .in('company_id', chunk)
+      .not('journal_entry_id', 'is', null)
+      .lt('amount', 0)
+      .eq('is_business', true)
+      .not('merchant_name', 'is', null)
+      .order('created_at', { ascending: false })
+      .limit(N)
+    if (error) throw error
+    candidatesByChunk.push(...((txs ?? []) as Tx[]))
+  }
+  const rows = candidatesByChunk
+    .sort((a, b) => (a.created_at < b.created_at ? 1 : a.created_at > b.created_at ? -1 : 0))
+    .slice(0, N)
   console.log(`\nBacktesting ${rows.length} booked transactions on ${process.env.BEDROCK_MODEL_ID ?? process.env.AI_MODEL ?? 'the configured model'}…\n`)
 
   // Ground-truth debit account per journal entry (expense line, not cash/VAT).
@@ -101,7 +140,7 @@ async function main() {
     const sel = await selectAccount({
       transaction: {
         merchantName: r.merchant_name,
-        description: r.description,
+        description: r.description ?? r.original_description ?? '',
         amount: r.amount,
         date: r.date,
         currency: r.currency,

scripts/fit-categorize-calibration.ts

@@ -11,6 +11,11 @@
  *
  * Note: .env.local points at production; this only SELECTs, so it is safe, but
  * it is still the prod corpus you are reading.
+ *
+ * Consent: samples are only written for, and only read from, companies with
+ * company_settings.data_analysis_opt_in = true (#1346). The write side is
+ * gated in POST /api/agent/categorize/outcome; the read side filters again
+ * here so a company that opted out after contributing drops out of the fit.
  */
 import { createClient } from '@supabase/supabase-js'
 import {
@@ -21,6 +26,7 @@ import {
   bandFor,
   type Sample,
 } from '@/lib/agent/categorize/calibration'
+import { chunkCompanyIds, listDataAnalysisOptedInCompanyIds } from '@/lib/company/data-analysis'
 
 const url = process.env.NEXT_PUBLIC_SUPABASE_URL
 const key = process.env.SUPABASE_SERVICE_ROLE_KEY
@@ -31,18 +37,30 @@ if (!url || !key) {
 const supabase = createClient(url, key)
 
 async function main() {
+  // Consent gate (#1346): only companies that opted in to data analysis.
+  const optedInIds = await listDataAnalysisOptedInCompanyIds(supabase)
+  if (optedInIds.length === 0) {
+    console.log('\nNo company has opted in to data analysis (company_settings.data_analysis_opt_in). Nothing to fit.')
+    return
+  }
+
+  // Query per chunk of company ids: `.in()` goes into the GET query string, so
+  // one request per few hundred opted-in companies would hit URL limits.
   const rows: { confidence: number; was_correct: boolean }[] = []
   const PAGE = 1000
-  for (let from = 0; ; from += PAGE) {
-    const { data, error } = await supabase
-      .from('categorize_calibration_samples')
-      .select('confidence, was_correct')
-      .order('created_at', { ascending: false })
-      .range(from, from + PAGE - 1)
-    if (error) throw error
-    if (!data || data.length === 0) break
-    rows.push(...(data as { confidence: number; was_correct: boolean }[]))
-    if (data.length < PAGE) break
+  for (const chunk of chunkCompanyIds(optedInIds)) {
+    for (let from = 0; ; from += PAGE) {
+      const { data, error } = await supabase
+        .from('categorize_calibration_samples')
+        .select('confidence, was_correct')
+        .in('company_id', chunk)
+        .order('id', { ascending: true })
+        .range(from, from + PAGE - 1)
+      if (error) throw error
+      if (!data || data.length === 0) break
+      rows.push(...(data as { confidence: number; was_correct: boolean }[]))
+      if (data.length < PAGE) break
+    }
   }
 
   const samples: Sample[] = rows.map((r) => ({ confidence: Number(r.confidence), correct: r.was_correct }))

supabase/migrations/20260828120000_company_settings_data_analysis_opt_in.sql

@@ -0,0 +1,34 @@
+-- Per-company opt-in for analysis of bookkeeping data (#1346).
+--
+-- Adds the consent flag that gates every path where a company's bookkeeping
+-- data is read across companies for Accounted's own analysis. Two scopes,
+-- both stated in the consent copy (messages/*.json data_analysis.*):
+--   1. Booking outcomes (proposed vs booked account, amount, confidence):
+--      the auto-booking calibration corpus (categorize_calibration_samples,
+--      written by POST /api/agent/categorize/outcome) and the calibration-fit
+--      script.
+--   2. Evaluation runs (scripts/backtest-categorize.ts, founder-run): the
+--      company's transaction descriptions, counterparty names and matched
+--      underlag (receipts, invoices) are re-run through the same AI model
+--      and processors as regular booking. That text can contain personal
+--      data (e.g. names in Swish payments), so the disclosure says so.
+-- Enforced server-side by lib/company/data-analysis.ts and by the scripts'
+-- own opted-in filter, never by the client.
+--
+-- Default false for everyone, no grandfathering: a company contributes
+-- nothing until a company owner or admin flips the toggle in
+-- Inställningar > Företag (the company_settings RLS update policy is admin
+-- only, so the switch is admin only too). Turning it off stops new collection immediately.
+-- The flag does not cover product-usage analytics or MCP reliability
+-- telemetry (no bookkeeping content, documented under legitimate interest in
+-- .compliance/ropa.yaml). Mirrors mileage_enabled (20260812193500).
+--
+-- pg-test: skip (plain column addition, no trigger/RPC/RLS)
+
+ALTER TABLE public.company_settings
+  ADD COLUMN data_analysis_opt_in boolean NOT NULL DEFAULT false;
+
+COMMENT ON COLUMN public.company_settings.data_analysis_opt_in IS
+  'Consent flag: when true, this company''s booking outcomes (proposed vs booked account, confidence, amount) may be read across companies for the auto-booking calibration corpus, and in evaluation runs (scripts/backtest-categorize.ts) its transaction descriptions, counterparty names and matched underlag may be re-run through the AI model. Default false; enforced server-side (lib/company/data-analysis.ts).';
+
+NOTIFY pgrst, 'reload schema';

tests/helpers.ts

@@ -625,6 +625,7 @@ export function makeCompanySettings(
     reminder_interest_rate_override: null,
     dimensions_enabled: false,
     mileage_enabled: false,
+    data_analysis_opt_in: false,
     preferred_payment_format: 'pain001',
     salary_pay_day: 25,
     salary_default_bank: null,

types/index.ts

@@ -571,6 +571,12 @@ export interface CompanySettings {
   // for correctness. The nav row also shows when mileage_trips rows exist.
   mileage_enabled: boolean
 
+  // Data analysis consent (migration 20260828120000): when true, the
+  // company's bookkeeping outcomes may be read across companies to evaluate
+  // and improve automatic booking. Default false, enforced server-side
+  // (lib/company/data-analysis.ts); the UI only mirrors it.
+  data_analysis_opt_in: boolean
+
   // Salary payments (migration 20260508120000 + 20260703190000).
   // preferred_payment_format defaults to 'pain001' — Bankgirot Lön is
   // retired by the banks during 2026.