admin/accounted
3
0
Fork 0
Code Issues 1 Pull Requests Actions Packages Projects Releases Wiki Activity

feat(ai): job-shaped AI service with OpenAI-compatible backend, extraction-first; stop extracting every inbox document twice (#1740)

Browse Source 
* feat(ai): job-shaped AI service with OpenAI-compatible backend, extraction-first; stop extracting every inbox document twice

Sovereign plan WS1 PR1 (#1406 Tier 2, extraction-first, aligned with the
AI surface audit).

lib/ai grows a job-shaped service (generateText / generateStructured /
extractFromDocument; no streaming members yet, see plan rule R3):
- services/anthropic-family delegates to the existing createAiClient()
  and sends the exact request literals the inbox extractor sent before
  (request-shape tests deep-equal them), so hosted Bedrock stays
  byte-identical.
- services/openai-compatible talks to any chat-completions endpoint
  (BYO Swedish provider) via Vercel AI SDK 6.x, exact-pinned and
  guarded: images as parts, PDFs rasterized with poppler (AI_PDF_MODE)
  or sent natively, AI_VISION / AI_STRICT_JSON declared, honest skips
  (ai_no_vision, pdf_rasterizer_missing) instead of fake failures.
- config.ts: AI_PROVIDER/AI_BASE_URL/AI_API_KEY/AI_MODEL and per-tier
  AI_*_MODEL with the legacy BEDROCK_* names kept as the same overrides;
  getAiStatus() is the single source of truth for "is AI wired up".
- provider.ts: openai-compatible in the auto-detect chain (after Bedrock
  and the direct API); createAiClient() refuses it loudly.

Document extraction moves onto the service and gets the audit's fixes:
- Inbox documents were extracted TWICE (pipeline A ran inside
  uploadDocument() before the inbox row existed, so its dedupe branch
  never fired; 3 707 + 1 666 calls / 30 d). The inbox now declares
  extractionOwner on the upload, the extension yields, and the inbox
  mirrors its single outcome onto document_attachments from every
  writer (sync, deferred, attach, retry, MCP).
- Every "no extraction will ever happen" outcome is stamped
  (skipped:no_ai_entitlement / ai_unconfigured / system_generated /
  ...); the status route maps the quiet ones to 'disabled' on the first
  poll instead of a 30 s client timeout. Prod showed 309 of the 327
  never-extracted uploads were the paywall working silently.
- Self-generated documents (our own invoice PDFs, payout files) are no
  longer OCR'd.
- Agent invoke answers 503 ai_unconfigured when the deployment has no
  assistant backend, distinct from the paywall.

Guard: new direct-ai-client antipattern check (shrink-only allowlist of
the pre-abstraction SDK callers) plus exact pins for @anthropic-ai/sdk,
ai and @ai-sdk/openai-compatible.

Verified: 15 958 unit tests green, guards, lint ratchet, typecheck, and a
live smoke against hosted Bedrock through the new service (ping, streamed
tool turn, thinking+cache, PDF extraction).

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>

* feat(ai): make AI_API_KEY optional for OpenAI-compatible endpoints (keyless local model servers)

A local model server (llama.cpp's server, Ollama /v1, LM Studio, vLLM)
usually has no auth. Before, the OpenAI-compatible backend required both
AI_BASE_URL and AI_API_KEY to count as configured, so running Accounted on a
local model meant setting a meaningless placeholder key.

- resolveAiProvider / hasAiCredentials: a base URL alone is now enough.
- services/openai-compatible: only send Authorization: Bearer when AI_API_KEY
  is set, so a keyless server is never handed an empty bearer; a hosted
  provider that needs a key still sets it.
- Docs (SELF-HOSTING Option 3: local-model example, key marked optional),
  DECISIONS.

Verified: with no AI_API_KEY, just AI_BASE_URL + AI_MODEL, getAiStatus()
reports configured=true / provider=openai-compatible (live). lib/ai suite
71 green; tsc, guards, lint clean. Bedrock/Anthropic logic unchanged.

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>

---------

Co-authored-by: Jakob Wennberg <311770904+jakobwennberg-oss@users.noreply.github.com>
Co-authored-by: Claude Fable 5 <noreply@anthropic.com>
This commit is contained in:
Jakob Wennberg
2026-08-20 19:39:08 +02:00
committed by GitHub
co-authored by Claude Fable 5 Jakob Wennberg
parent fbf47649f2
commit c7a75d069d
37 changed files with 2865 additions and 335 deletions
Download Patch File Download Diff File Expand all files Collapse all files
docs/SELF-HOSTING.md
+29 -9
View File
@@ -199,7 +199,7 @@ Additionally, migration 048 schedules a `pg_cron` job inside the database that m
### AI Features
All AI features (automatic interpretation of uploaded receipts and invoices via the `document-extraction` and `invoice-inbox` extensions, and the in-app AI assistant) run Claude. There are two ways to provide credentials; pick one.
All AI features (automatic interpretation of uploaded receipts and invoices via the `document-extraction` and `invoice-inbox` extensions, and the in-app AI assistant) run on one configured backend. There are three ways to provide one; pick one. Note that the agent surface most integrations use, the MCP server, needs no AI backend at all: it is your own agent (Claude, Codex, a local model) talking to the ledger, so a deployment without any of the credentials below is still fully usable that way.
The stock self-hosted image includes both extraction extensions, so these credentials cover emailed invoices and documents uploaded in the app.
@@ -219,18 +219,38 @@ AWS_REGION=eu-north-1 # default
Set both static AWS keys explicitly. The AI assistant's client can fall back to the standard AWS credential provider chain (instance profile, IRSA) when they are absent, but document extraction requires `AWS_ACCESS_KEY_ID` and `AWS_SECRET_ACCESS_KEY` and silently returns empty results without them.
Optional model overrides, in either setup:
**Option 3: any OpenAI-compatible endpoint.** Any server that implements the chat-completions API: a Swedish inference provider for a fully sovereign deployment, or a **local model** on the same machine (llama.cpp's `server`, Ollama's `/v1`, LM Studio, vLLM). Document extraction (receipts, invoices, HTML mail invoices) and the single-call assistant (`/api/agent/ask`) run here on any provider; the older streaming chat panel is being replaced by that single-call surface.
```bash
BEDROCK_MODEL_ID=claude-sonnet-5 # document extraction model
BEDROCK_OPUS_MODEL_ID=... # assistant model, heavy intents
BEDROCK_SONNET_MODEL_ID=... # assistant model, standard intents
AI_PROVIDER=bedrock|anthropic # force the backend (see below)
AI_BASE_URL=http://localhost:11434/v1 # the endpoint's OpenAI-compatible base URL (here: a local Ollama)
AI_MODEL=qwen3.8 # a model id is required: there is no default for an arbitrary endpoint
# AI_API_KEY=... # OPTIONAL: only when the endpoint needs auth. A local server usually
# # has none, so leave it unset; a hosted provider gives you a key.
# AI_EXTRACTION_MODEL=... # optional: a vision model for document reading, if AI_MODEL is not one
```
When both credential sets are present, Bedrock wins, so that adding an Anthropic key for an experiment cannot silently move production inference out of eu-north-1. Set `AI_PROVIDER` to say which you mean. A model id written without a provider prefix is adapted to whichever backend is active; an id that already carries one (`eu.anthropic.…`) is used as-is.
Three things about such endpoints are declared rather than probed, because the app cannot tell from the outside:
Without working credentials the rest of the app runs normally: uploads are stored but not auto-interpreted, and the AI assistant cannot answer.
- `AI_VISION=false` says the configured model cannot read images. Images and PDFs are then skipped honestly (the inbox row lands with the empty skeleton and the "AI-tolkning kördes inte" hint) instead of failing with a 400 on every upload; HTML mail invoices still extract as text on any model.
- `AI_PDF_MODE` defaults to `rasterize` here: most such endpoints have no PDF input, so the first `AI_PDF_MAX_PAGES` pages (default 4) are rendered to images with poppler's `pdftoppm`, which the self-host image installs. If the binary is missing, PDFs are skipped with `pdf_rasterizer_missing` rather than failing. A provider that accepts the OpenAI `file` content part can use `AI_PDF_MODE=native`.
- `AI_STRICT_JSON=true` asks for `response_format: json_schema` on providers that enforce it. The default (JSON answered in prose, then parsed and validated) works on every model and is what hosted runs.
Optional model overrides, in any setup:
```bash
AI_MODEL=... # default model for every tier (OpenAI-compatible: required)
AI_EXTRACTION_MODEL=... # document extraction model
AI_HEAVY_MODEL=... # assistant model, heavy intents
AI_ASSISTANT_MODEL=... # assistant model, standard intents
AI_EXTRACTION_MAX_TOKENS=8192 # output cap for document extraction
AI_PROVIDER=bedrock|anthropic|openai-compatible # force the backend (see below)
```
The pre-existing names `BEDROCK_MODEL_ID`, `BEDROCK_OPUS_MODEL_ID`, `BEDROCK_SONNET_MODEL_ID` and `BEDROCK_MAX_TOKENS` keep working as the same overrides (extraction, heavy, standard, extraction cap) on every backend; the `AI_*` names take precedence when both are set. Claude deployments default every tier to `claude-sonnet-5`.
When several credential sets are present, Bedrock wins, then the direct Anthropic API, then the OpenAI-compatible endpoint, so that adding a key for an experiment cannot silently move production inference out of eu-north-1. Set `AI_PROVIDER` to say which you mean. A model id written without a provider prefix is adapted to whichever backend is active; an id that already carries one (`eu.anthropic.…`) is used as-is.
Without working credentials the rest of the app runs normally: uploads are stored but not auto-interpreted (the upload UI sees that immediately rather than waiting for a timeout), and the AI assistant answers `503 ai_unconfigured`.
#### Verifying the setup
@@ -248,7 +268,7 @@ npx tsx scripts/smoke-ai.ts ./receipt.pdf # also runs document extraction
It prints the resolved provider and model ids first, then exercises a plain request, a streamed turn carrying the assistant's full parameter set (adaptive thinking, effort, prompt caching and a tool), and finally extraction of the file you pass. It exits non-zero if any step fails, so it works as a post-deploy check.
> **Note:** `OPENAI_API_KEY` from earlier versions is not read by any code path; there is no OpenAI route in the app. Pluggable providers beyond Claude are tracked in [#1406](https://github.com/erp-mafia/accounted/issues/1406).
> **Note:** `OPENAI_API_KEY` from earlier versions is not read by any code path. To use OpenAI itself, point Option 3 at `https://api.openai.com/v1`; the app has no provider-specific OpenAI integration, only the OpenAI-compatible one. Background: [#1406](https://github.com/erp-mafia/accounted/issues/1406).
### Email (Invoice Sending and Reminders)