feat(ai): self-host enablement for BYO endpoints: poppler in the runner image, backend-agnostic smoke script, docs (#1743)

Sovereign plan WS1 PR2, stacked on the extraction-first service (#1740). - Dockerfile (runner stage): `apk add --no-cache poppler-utils`, the one system package beyond the base image (~4 MB plus shared libs, pdftoppm 25.12 on node:22-alpine). pdftoppm renders the first pages of a PDF for AI backends with no native PDF input (an OpenAI-compatible Swedish endpoint); page images land in /tmp, which docker-compose.yml already mounts as tmpfs under the read-only root. Hosted (Bedrock) never calls it; the cron image is untouched. - scripts/smoke-ai-provider.ts: the self-hoster's "is AI wired up" command. Prints provider, models per tier, PDF mode (+ whether pdftoppm is present), vision/strict-JSON; then one text generation per tier model, one schema-shaped answer and, given a file, the exact document-extraction path an upload takes. Skips are reported as failures with the fix. Reads .env.local then .env. - docs/SELF-HOSTING.md: verifying section rewritten around the new script (smoke-ai.ts stays for the assistant's Anthropic-only parameter probes); rasterizer/tmpfs notes; .env.example gains AI_PDF_RASTERIZER_BIN; DECISIONS entry. Verified: live against hosted Bedrock (text per tier, structured, PDF extraction) and against a local OpenAI-compatible mock with AI_PROVIDER=openai-compatible (the mock received Bearer auth, per-tier model ids and one image_url part per rasterized page; extraction parsed the fenced JSON answer). poppler-utils probed on node:22-alpine. Co-authored-by: Jakob Wennberg <311770904+jakobwennberg-oss@users.noreply.github.com> Co-authored-by: Claude Fable 5 <noreply@anthropic.com>
2026-08-20 19:48:21 +02:00
parent c7a75d069d
commit 4ecc888416
5 changed files with 238 additions and 4 deletions
@@ -97,6 +97,7 @@ RECEIPT_HUNT_COMPANY_IDS=
 #                                 # (images/PDFs are then skipped honestly; HTML mail still extracts)
 # AI_PDF_MODE=auto                # auto (Claude: native, others: rasterize with poppler) | native | rasterize
 # AI_PDF_MAX_PAGES=4              # pages rasterized per PDF
+# AI_PDF_RASTERIZER_BIN=pdftoppm   # poppler binary name/path (the self-host image installs poppler-utils)
 # AI_STRICT_JSON=false            # openai-compatible only: response_format json_schema when the provider enforces it
 # AI_EXTRACTION_MAX_TOKENS=8192   # output cap for document extraction (legacy BEDROCK_MAX_TOKENS)
 # AI_PROVIDER=
@@ -1125,3 +1125,4 @@ One line per decision: `[YYYY-MM-DD] <decision>: <why>`. Appended by agents and
 [2026-08-20] Every "no extraction will ever happen" outcome is now stamped on document_attachments as skipped:<reason> (no_ai_entitlement, ai_unconfigured, system_generated, sandbox, client_opt_out, ai_no_vision, pdf_rasterizer_missing) and the extraction-status route maps the quiet ones to 'disabled' on the first poll. Prod check (30 d, read-only): 309 of the audit's 327 never-extracted uploads belonged to 29 companies with zero successful extractions, i.e. the paywall working silently and the UI spinning 30 s to find out; the remaining 18 are download failures, now stamped failed:storage_download. Self-generated documents (upload_source 'system': our own invoice PDFs, payout files, filings; 59 paid model calls in 30 d) are skipped as system_generated.
 [2026-08-20] Strict JSON on OpenAI-compatible endpoints uses a hand-maintained JSON-schema mirror of the extraction Zod schema, opt-in via AI_STRICT_JSON, never an automatic Zod-to-JSON-schema conversion: the Zod schema carries .catch()/.transform() that have no schema equivalent and a generated schema would drift silently. Zod stays the validator either way; JSON-in-prose + extractJsonObject remains the default everywhere because it works on every model and is what hosted runs.
 [2026-08-20] AI_API_KEY made optional for the OpenAI-compatible backend: a base URL alone now counts as configured (hasAiCredentials / resolveAiProvider), so a local model server (llama.cpp/Ollama/LM Studio/vLLM), which usually has no auth, works with just AI_BASE_URL + AI_MODEL. The openai-compatible service only sends an Authorization: Bearer when AI_API_KEY is set, so a keyless local server is never handed an empty bearer. Hosted providers that require a key still set AI_API_KEY. Bedrock/Anthropic credential logic unchanged.
+[2026-08-20] poppler-utils is the one system package added to the self-host runner image (Sovereign plan WS1 PR2): pdftoppm renders the first pages of a PDF for AI backends without native PDF input (an OpenAI-compatible Swedish endpoint), measured at ~4 MB plus shared libs on node:22-alpine (pdftoppm 25.12), written to /tmp which docker-compose.yml already mounts as tmpfs under the read-only root. Hosted never calls it (Bedrock reads PDFs natively) and the cron image is untouched. pdfjs-dist + @napi-rs/canvas were rejected earlier (two npm deps, memory spikes, dead weight on hosted). scripts/smoke-ai-provider.ts is the backend-agnostic "is AI wired up" check; verified live against hosted Bedrock and against a local OpenAI-compatible mock (the mock received Bearer auth, per-tier model ids and one image_url part per rasterized page).
@@ -65,6 +65,16 @@ WORKDIR /app
 RUN apk upgrade --no-cache && \
    rm -rf /usr/local/lib/node_modules/npm /usr/local/bin/npm /usr/local/bin/npx

+# poppler-utils (pdftoppm, ~4 MB plus shared libs) renders PDF pages to images
+# for AI backends that cannot read PDF bytes natively: a self-host pointing
+# AI_BASE_URL at an OpenAI-compatible endpoint (e.g. a Swedish inference
+# provider) rasterizes the first pages of a receipt/invoice before extraction
+# (lib/ai/rasterize-pdf.ts, AI_PDF_MODE). Hosted runs Claude on Bedrock, which
+# reads PDFs natively and never calls it. Deliberately the only system
+# package beyond the base image. Temp files go to /tmp, which
+# docker-compose.yml mounts as tmpfs (the root filesystem is read-only).
+RUN apk add --no-cache poppler-utils
+
 ENV NODE_ENV=production
 ENV NEXT_TELEMETRY_DISABLED=1

@@ -232,7 +232,7 @@ AI_MODEL=qwen3.8                        # a model id is required: there is no de
 Three things about such endpoints are declared rather than probed, because the app cannot tell from the outside:

 - `AI_VISION=false` says the configured model cannot read images. Images and PDFs are then skipped honestly (the inbox row lands with the empty skeleton and the "AI-tolkning kördes inte" hint) instead of failing with a 400 on every upload; HTML mail invoices still extract as text on any model.
- `AI_PDF_MODE` defaults to `rasterize` here: most such endpoints have no PDF input, so the first `AI_PDF_MAX_PAGES` pages (default 4) are rendered to images with poppler's `pdftoppm`, which the self-host image installs. If the binary is missing, PDFs are skipped with `pdf_rasterizer_missing` rather than failing. A provider that accepts the OpenAI `file` content part can use `AI_PDF_MODE=native`.
+- `AI_PDF_MODE` defaults to `rasterize` here: most such endpoints have no PDF input, so the first `AI_PDF_MAX_PAGES` pages (default 4) are rendered to images with poppler's `pdftoppm`, which the self-host image installs (`apk add poppler-utils`, the only system package beyond the base image; page images are written to `/tmp`, a tmpfs in `docker-compose.yml`). If the binary is missing, PDFs are skipped with `pdf_rasterizer_missing` rather than failing; `AI_PDF_RASTERIZER_BIN` points at a non-standard install. A provider that accepts the OpenAI `file` content part can use `AI_PDF_MODE=native`.
 - `AI_STRICT_JSON=true` asks for `response_format: json_schema` on providers that enforce it. The default (JSON answered in prose, then parsed and validated) works on every model and is what hosted runs.

 Optional model overrides, in any setup:
@@ -254,7 +254,16 @@ Without working credentials the rest of the app runs normally: uploads are store

 #### Verifying the setup

-`scripts/smoke-ai.ts` sends real traffic to whichever backend your environment resolves to, so a wrong key, an unavailable model or a rejected parameter surfaces here rather than in front of a user:
+`scripts/smoke-ai-provider.ts` is the "is AI wired up?" command. It works the same on every backend because it only talks to the app's AI service: it prints the resolved provider, the model per tier, the PDF mode (and whether `pdftoppm` is installed when PDFs are rasterized), then sends real traffic: one small text generation per tier model, one schema-shaped answer, and, when you pass a file, the exact document-extraction path an uploaded receipt takes. It exits non-zero if any step fails, so it works as a post-deploy check. Run it from a checkout next to the env file your deployment uses (`.env.local`, then `.env` are read):
+
+```bash
+npx tsx scripts/smoke-ai-provider.ts                 # provider, models, text + structured calls
+npx tsx scripts/smoke-ai-provider.ts ./receipt.pdf   # also runs document extraction end to end
+```
+
+A skipped extraction is reported as a failure with the reason: a text-only model (`ai_no_vision`, pick a vision model for `AI_EXTRACTION_MODEL`), a missing rasterizer (`pdf_rasterizer_missing`, install poppler-utils or set `AI_PDF_MODE=native`), or no credentials/model at all.
+
+On the Anthropic family, `scripts/smoke-ai.ts` additionally probes the in-app assistant's full parameter set (a streamed turn with a tool, adaptive thinking, effort and the prompt cache), which the assistant still sends through the Anthropic SDK directly:

 ```bash
 npx tsx scripts/smoke-ai.ts                  # credentials, models, chat loop
@@ -266,8 +275,6 @@ npx tsx scripts/smoke-ai.ts                  # credentials, models, chat loop
 npx tsx scripts/smoke-ai.ts ./receipt.pdf    # also runs document extraction
 ```

-It prints the resolved provider and model ids first, then exercises a plain request, a streamed turn carrying the assistant's full parameter set (adaptive thinking, effort, prompt caching and a tool), and finally extraction of the file you pass. It exits non-zero if any step fails, so it works as a post-deploy check.
-
 > **Note:** `OPENAI_API_KEY` from earlier versions is not read by any code path. To use OpenAI itself, point Option 3 at `https://api.openai.com/v1`; the app has no provider-specific OpenAI integration, only the OpenAI-compatible one. Background: [#1406](https://github.com/erp-mafia/accounted/issues/1406).

 ### Email (Invoice Sending and Reminders)
@@ -0,0 +1,215 @@
+#!/usr/bin/env npx tsx
+/**
+ * Smoke test for the configured AI backend through the job-shaped service in
+ * lib/ai: the self-hoster's "is AI wired up?" command. Works the same against
+ * AWS Bedrock, the direct Anthropic API, and any OpenAI-compatible endpoint
+ * (a Swedish inference provider on a sovereign self-host), because it only
+ * ever talks to getAiService().
+ *
+ * Steps:
+ *   0. Print the resolved status: provider, models per tier, PDF mode, vision,
+ *      strict JSON, and (for rasterized PDFs) whether pdftoppm is installed.
+ *      Exits 1 with the reason when nothing is configured.
+ *   1. A tiny text generation on each distinct tier model.
+ *   2. A schema-shaped answer (generateStructured).
+ *   3. Document extraction end to end, when given a file (PDF, JPEG, PNG,
+ *      WebP, GIF or HTML): the exact path an uploaded receipt takes, so a
+ *      missing rasterizer, a text-only model or a rejected request surfaces
+ *      here rather than as an empty inbox row.
+ *
+ * The chat loop (the in-app assistant) still speaks the Anthropic SDK
+ * directly; its parameter set (adaptive thinking, effort, prompt cache, tools)
+ * is probed by scripts/smoke-ai.ts on the Anthropic family only.
+ *
+ * Usage:
+ *   npx tsx scripts/smoke-ai-provider.ts
+ *   npx tsx scripts/smoke-ai-provider.ts ./some-receipt.pdf
+ *
+ * Environment: reads .env.local then .env (the Docker compose env file), so
+ * run it from the checkout next to the env file your deployment uses.
+ */
+
+import { config } from 'dotenv'
+config({ path: '.env.local' })
+config({ path: '.env' })
+
+import { execFile } from 'node:child_process'
+import { readFile } from 'node:fs/promises'
+import { basename, extname } from 'node:path'
+import { promisify } from 'node:util'
+
+type AiModule = typeof import('../lib/ai')
+type ExtractionModule = typeof import('../extensions/general/invoice-inbox/lib/extract-invoice-fields')
+
+let failures = 0
+
+function fail(step: string, err: unknown): void {
+  const message = err instanceof Error ? err.message : String(err)
+  console.error(`  x ${step}: ${message}`)
+  failures++
+}
+
+const MIME_BY_EXT: Record<string, string> = {
+  '.pdf': 'application/pdf',
+  '.jpg': 'image/jpeg',
+  '.jpeg': 'image/jpeg',
+  '.png': 'image/png',
+  '.webp': 'image/webp',
+  '.gif': 'image/gif',
+  '.html': 'text/html',
+  '.htm': 'text/html',
+}
+
+async function probeRasterizer(): Promise<string> {
+  const bin = process.env.AI_PDF_RASTERIZER_BIN ?? 'pdftoppm'
+  try {
+    const { stdout, stderr } = await promisify(execFile)(bin, ['-v'])
+    const firstLine = `${stdout}${stderr}`.split('\n').find((l) => l.trim().length > 0) ?? 'found'
+    return `found (${firstLine.trim()})`
+  } catch (err) {
+    const code = (err as { code?: string } | null)?.code
+    return code === 'ENOENT'
+      ? `MISSING (${bin} not on PATH: PDFs will be skipped with pdf_rasterizer_missing; install poppler-utils or set AI_PDF_MODE=native if the endpoint accepts PDF parts)`
+      : `error (${err instanceof Error ? err.message : String(err)})`
+  }
+}
+
+async function main(): Promise<void> {
+  const ai: AiModule = await import('../lib/ai')
+  const status = ai.getAiStatus()
+
+  console.log(`Provider:     ${status.provider}`)
+  if (status.provider === 'openai-compatible') {
+    let host = '(AI_BASE_URL unset)'
+    try {
+      host = new URL(process.env.AI_BASE_URL ?? '').host
+    } catch {
+      // printed as unset
+    }
+    console.log(`Endpoint:     ${host}`)
+  }
+  console.log(`Configured:   ${status.configured ? 'yes' : `NO (${status.reason})`}`)
+  console.log(`Models:       assistant=${status.models.assistant ?? '-'}  heavy=${status.models.heavy ?? '-'}  extraction=${status.models.extraction ?? '-'}`)
+  console.log(`PDF mode:     ${status.pdfMode}${status.pdfMode === 'rasterize' ? `  pdftoppm: ${await probeRasterizer()}` : ''}`)
+  console.log(`Capabilities: vision=${status.capabilities.imageInput} pdfNative=${status.capabilities.pdfNative} strictJson=${status.capabilities.strictJsonSchema}`)
+  console.log(`Assistant:    ${status.assistantAvailable ? 'available' : 'not available on this backend (extraction and single-call AI still run)'}`)
+  console.log()
+
+  if (!status.configured) {
+    console.error(
+      status.reason === 'no_credentials'
+        ? 'No AI credentials found. Set AWS_ACCESS_KEY_ID + AWS_SECRET_ACCESS_KEY (Bedrock), ANTHROPIC_API_KEY (direct API), or AI_BASE_URL + AI_API_KEY (OpenAI-compatible endpoint).'
+        : 'No model id configured. An OpenAI-compatible endpoint has no default: set AI_MODEL (and optionally AI_EXTRACTION_MODEL for a vision model).'
+    )
+    process.exit(1)
+  }
+
+  const service = ai.getAiService()
+
+  // 1. Text generation per distinct tier model.
+  console.log('1. Text generation, one call per distinct tier model')
+  const seen = new Set<string>()
+  for (const tier of ['assistant', 'heavy', 'extraction'] as const) {
+    const model = service.modelFor(tier)
+    if (seen.has(model)) continue
+    seen.add(model)
+    const start = Date.now()
+    try {
+      const result = await service.generateText({
+        tier,
+        prompt: 'Säg "hej" på svenska, ett ord.',
+        maxTokens: 32,
+      })
+      console.log(`  ok ${tier} (${model}): ${Date.now() - start}ms: "${result.text.slice(0, 40)}"  tokens in/out=${result.usage.inputTokens ?? '?'}/${result.usage.outputTokens ?? '?'}`)
+    } catch (err) {
+      fail(`${tier} (${model})`, err)
+    }
+  }
+
+  // 2. Schema-shaped answer.
+  console.log('2. Structured output')
+  {
+    const start = Date.now()
+    try {
+      const result = await service.generateStructured({
+        tier: 'assistant',
+        prompt: 'Return a Swedish one-word greeting and the ISO 639-1 code of its language.',
+        maxTokens: 200,
+        schema: {
+          name: 'greeting',
+          description: 'a greeting and its language code',
+          jsonSchema: {
+            type: 'object',
+            additionalProperties: false,
+            properties: { greeting: { type: 'string' }, language: { type: 'string' } },
+            required: ['greeting', 'language'],
+          },
+        },
+      })
+      const value = result.value as { greeting?: unknown; language?: unknown }
+      if (typeof value?.greeting !== 'string' || typeof value?.language !== 'string') {
+        throw new Error(`unexpected shape: ${JSON.stringify(result.value).slice(0, 120)}`)
+      }
+      console.log(`  ok structured: ${Date.now() - start}ms: ${JSON.stringify(value)}`)
+    } catch (err) {
+      fail('structured', err)
+    }
+  }
+
+  // 3. Document extraction.
+  const file = process.argv[2]
+  if (file) {
+    console.log('3. Document extraction')
+    const mimeType = MIME_BY_EXT[extname(file).toLowerCase()]
+    if (!mimeType) {
+      fail('extraction', new Error(`unsupported extension "${extname(file)}": use pdf, jpg, png, webp, gif or html`))
+    } else {
+      const start = Date.now()
+      try {
+        const extraction: ExtractionModule = await import(
+          '../extensions/general/invoice-inbox/lib/extract-invoice-fields'
+        )
+        const buffer = await readFile(file)
+        const result = await extraction.extractInvoiceFields({
+          buffer,
+          mimeType,
+          fileName: basename(file),
+        })
+        if (result.skipped) {
+          throw new Error(
+            `skipped (${result.skipped}): no model call was made. ` +
+              (result.skipped === 'ai_no_vision'
+                ? 'The configured model is declared text-only (AI_VISION=false); pick a vision model for AI_EXTRACTION_MODEL.'
+                : result.skipped === 'pdf_rasterizer_missing'
+                  ? 'Install poppler-utils (pdftoppm) or set AI_PDF_MODE=native.'
+                  : '')
+          )
+        }
+        if (!result.rawText) {
+          throw new Error('the model answered but nothing parseable came back (see the warning above)')
+        }
+        const d = result.data
+        console.log(
+          `  ok extraction (${result.model}): ${Date.now() - start}ms: supplier="${d.supplier.name ?? '-'}", ` +
+            `date=${d.invoice.invoiceDate ?? '-'}, total=${d.totals.total ?? '-'} ${d.invoice.currency}, kind=${d.documentKind ?? '-'}`
+        )
+      } catch (err) {
+        fail('extraction', err)
+      }
+    }
+  } else {
+    console.log('3. Document extraction: skipped (pass a file path to run it)')
+  }
+
+  console.log()
+  if (failures > 0) {
+    console.error(`${failures} step(s) failed.`)
+    process.exit(1)
+  }
+  console.log('All green.')
+}
+
+main().catch((err) => {
+  console.error(err instanceof Error ? err.stack ?? err.message : String(err))
+  process.exit(1)
+})