feat(ai): job-shaped AI service with OpenAI-compatible backend, extraction-first; stop extracting every inbox document twice (#1740)
* feat(ai): job-shaped AI service with OpenAI-compatible backend, extraction-first; stop extracting every inbox document twice Sovereign plan WS1 PR1 (#1406 Tier 2, extraction-first, aligned with the AI surface audit). lib/ai grows a job-shaped service (generateText / generateStructured / extractFromDocument; no streaming members yet, see plan rule R3): - services/anthropic-family delegates to the existing createAiClient() and sends the exact request literals the inbox extractor sent before (request-shape tests deep-equal them), so hosted Bedrock stays byte-identical. - services/openai-compatible talks to any chat-completions endpoint (BYO Swedish provider) via Vercel AI SDK 6.x, exact-pinned and guarded: images as parts, PDFs rasterized with poppler (AI_PDF_MODE) or sent natively, AI_VISION / AI_STRICT_JSON declared, honest skips (ai_no_vision, pdf_rasterizer_missing) instead of fake failures. - config.ts: AI_PROVIDER/AI_BASE_URL/AI_API_KEY/AI_MODEL and per-tier AI_*_MODEL with the legacy BEDROCK_* names kept as the same overrides; getAiStatus() is the single source of truth for "is AI wired up". - provider.ts: openai-compatible in the auto-detect chain (after Bedrock and the direct API); createAiClient() refuses it loudly. Document extraction moves onto the service and gets the audit's fixes: - Inbox documents were extracted TWICE (pipeline A ran inside uploadDocument() before the inbox row existed, so its dedupe branch never fired; 3 707 + 1 666 calls / 30 d). The inbox now declares extractionOwner on the upload, the extension yields, and the inbox mirrors its single outcome onto document_attachments from every writer (sync, deferred, attach, retry, MCP). - Every "no extraction will ever happen" outcome is stamped (skipped:no_ai_entitlement / ai_unconfigured / system_generated / ...); the status route maps the quiet ones to 'disabled' on the first poll instead of a 30 s client timeout. Prod showed 309 of the 327 never-extracted uploads were the paywall working silently. - Self-generated documents (our own invoice PDFs, payout files) are no longer OCR'd. - Agent invoke answers 503 ai_unconfigured when the deployment has no assistant backend, distinct from the paywall. Guard: new direct-ai-client antipattern check (shrink-only allowlist of the pre-abstraction SDK callers) plus exact pins for @anthropic-ai/sdk, ai and @ai-sdk/openai-compatible. Verified: 15 958 unit tests green, guards, lint ratchet, typecheck, and a live smoke against hosted Bedrock through the new service (ping, streamed tool turn, thinking+cache, PDF extraction). Co-Authored-By: Claude Fable 5 <noreply@anthropic.com> * feat(ai): make AI_API_KEY optional for OpenAI-compatible endpoints (keyless local model servers) A local model server (llama.cpp's server, Ollama /v1, LM Studio, vLLM) usually has no auth. Before, the OpenAI-compatible backend required both AI_BASE_URL and AI_API_KEY to count as configured, so running Accounted on a local model meant setting a meaningless placeholder key. - resolveAiProvider / hasAiCredentials: a base URL alone is now enough. - services/openai-compatible: only send Authorization: Bearer when AI_API_KEY is set, so a keyless server is never handed an empty bearer; a hosted provider that needs a key still sets it. - Docs (SELF-HOSTING Option 3: local-model example, key marked optional), DECISIONS. Verified: with no AI_API_KEY, just AI_BASE_URL + AI_MODEL, getAiStatus() reports configured=true / provider=openai-compatible (live). lib/ai suite 71 green; tsc, guards, lint clean. Bedrock/Anthropic logic unchanged. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com> --------- Co-authored-by: Jakob Wennberg <311770904+jakobwennberg-oss@users.noreply.github.com> Co-authored-by: Claude Fable 5 <noreply@anthropic.com>
This commit is contained in:
Jakob Wennberg
+23
-7
@@ -68,21 +68,37 @@ RECEIPT_HUNT_COMPANY_IDS=
|
||||
# NEXT_PUBLIC_GOOGLE_AUTH_ENABLED=true
|
||||
|
||||
# ── Optional: extension features (core runs without these) ─
|
||||
# AI features (document extraction + AI assistant). Two ways to provide a key;
|
||||
# set one of them. AI_PROVIDER (bedrock|anthropic) forces the choice if both
|
||||
# are present, which otherwise resolves to Bedrock.
|
||||
# AI features (document extraction + AI assistant). Three ways to provide a
|
||||
# backend; set one of them. AI_PROVIDER (bedrock|anthropic|openai-compatible)
|
||||
# forces the choice if several are present; otherwise Bedrock wins, then the
|
||||
# direct Anthropic API, then an OpenAI-compatible endpoint.
|
||||
#
|
||||
# 1. Claude via AWS Bedrock. Needs an AWS account with Bedrock model access to
|
||||
# Claude. Keeps inference in eu-north-1, which is what hosted runs.
|
||||
# AWS_ACCESS_KEY_ID=
|
||||
# AWS_SECRET_ACCESS_KEY=
|
||||
# AWS_REGION=eu-north-1
|
||||
# BEDROCK_MODEL_ID=
|
||||
#
|
||||
# 2. Claude via the direct Anthropic API. No AWS account needed, so this is
|
||||
# usually the self-hosted option. Note that it has no EU-residency
|
||||
# guarantee: use Bedrock if you need one.
|
||||
# 2. Claude via the direct Anthropic API. No AWS account needed. Note that it
|
||||
# has no EU-residency guarantee: use Bedrock if you need one.
|
||||
# ANTHROPIC_API_KEY=
|
||||
#
|
||||
# 3. Any OpenAI-compatible endpoint (chat-completions API), e.g. a Swedish
|
||||
# inference provider for a sovereign self-host. Document extraction and
|
||||
# single-call AI jobs run here; the in-app chat assistant does not yet.
|
||||
# A model id is required (no default exists for an arbitrary endpoint).
|
||||
# AI_BASE_URL=https://api.example.se/v1
|
||||
# AI_API_KEY=
|
||||
# AI_MODEL= # default model for every tier
|
||||
# AI_EXTRACTION_MODEL= # per-tier overrides (also AI_ASSISTANT_MODEL, AI_HEAVY_MODEL);
|
||||
# # legacy BEDROCK_MODEL_ID / BEDROCK_SONNET_MODEL_ID /
|
||||
# # BEDROCK_OPUS_MODEL_ID keep working as the same overrides
|
||||
# AI_VISION=true # openai-compatible only: set false for a text-only model
|
||||
# # (images/PDFs are then skipped honestly; HTML mail still extracts)
|
||||
# AI_PDF_MODE=auto # auto (Claude: native, others: rasterize with poppler) | native | rasterize
|
||||
# AI_PDF_MAX_PAGES=4 # pages rasterized per PDF
|
||||
# AI_STRICT_JSON=false # openai-compatible only: response_format json_schema when the provider enforces it
|
||||
# AI_EXTRACTION_MAX_TOKENS=8192 # output cap for document extraction (legacy BEDROCK_MAX_TOKENS)
|
||||
# AI_PROVIDER=
|
||||
# Bank connections (Enable Banking)
|
||||
# ENABLE_BANKING_APP_ID=
|
||||
|
||||
@@ -1119,3 +1119,9 @@ One line per decision: `[YYYY-MM-DD] <decision>: <why>`. Appended by agents and
|
||||
[2026-08-20] Unmatched bank rows that no voucher on the account could settle (direction-compatible and equal to the ore) get "Bokfor" linking to /transactions?highlight=<id> instead of a match picker. They are unbooked affarshandelser, not reconciliation work, and the picker held nothing for them. The rule is deliberately strict: a false negative offers booking on a pairable row (a legitimate outcome), a false positive sends the user into an empty picker.
|
||||
[2026-08-20] Vercel build heap is raised through vercel.json `buildCommand` (`NODE_OPTIONS=--max-old-space-size=6144 npm run build`), not a project env var and not `build.env`: a project-level NODE_OPTIONS also reaches function runtime (V8 sizes the heap against a limit the function does not have), and `build.env` is marked deprecated in the vercel.json schema; `buildCommand` scopes the flag to the build exactly like core-build.yml's 8192 does for CI. 6144 fits the standard 4-core/8 GB build machine next to the main next process; the type-check needs ~4.5 GB and was hanging at V8's ~4 GB default ceiling (4 production timeouts 2026-08-14..20).
|
||||
[2026-08-20] The production build type-checks tsconfig.build.json (tsconfig.json minus tests and mocks) via typescript.tsconfigPath; tsconfig.json stays the editor/ESLint view of the whole repo. Why: Next 16.3 runs the project-local tsc CLI by default (prerequisite for TypeScript 7's native checker, which has no JS API) and that checker checks the complete project it is given, whereas the old API checker silently dropped diagnostics from __tests__/*.test.* files. ~490 real type errors live in test files today (mostly route handlers called without the ctx argument); vitest never type-checks them, so nothing caught them. Excluding tests from the build keeps that debt where it was (invisible) instead of turning it into a red deploy; a separate tsc job for tests is the follow-up that makes it visible. Measured: tests are ~10% of the check's memory, so this is correctness, not the memory fix (that is the vercel.json heap bump).
|
||||
[2026-08-20] AI provider abstraction (#1406 Tier 2, Sovereign plan WS1 PR1) ships extraction-first: a job-shaped service in lib/ai (generateText / generateStructured / extractFromDocument, NO streaming members) with an Anthropic-family adapter that delegates to the existing createAiClient() factory and sends the exact request literals the call sites sent before (request-shape tests deep-equal them, so hosted Bedrock stays byte-identical), plus an OpenAI-compatible adapter for BYO Swedish endpoints. Only document extraction moved onto it in this PR; the chat loop, composer, receipt hunt and WhatsApp interpreter stay on the direct SDK behind a shrink-only allowlist in the new direct-ai-client antipattern check, because the AI surface audit (2026-08-18) recommends ripping the chat runtime and making the composer deterministic, and porting streaming + translation for code with a delete recommendation against it would be wasted work. The streaming port is gated on that founder call (plan rule R3).
|
||||
[2026-08-20] The OpenAI-compatible adapter uses the Vercel AI SDK 6.x (`ai` 6.0.259 + `@ai-sdk/openai-compatible` 2.0.69), exact-pinned and guarded, and NOT 7.x: 7.0.0 shipped 2026-06-25 (eight weeks old at pin time) while 6.x still receives releases, the plan approved "v6" on 2026-08-18, and the SDK is confined to lib/ai/services/openai-compatible.ts so a later major bump is one reviewed file. Lockfile regenerated with npm@10 (npm 11 drops next-intl's nested @swc/helpers entry, per the dependabot-lockfile memory).
|
||||
[2026-08-20] Inbox double extraction fixed by ownership, not by reordering: uploadDocument() awaits the document.uploaded handlers inline, so the document-extraction extension ran BEFORE the invoice-inbox row existed and its copied-from-invoice-inbox branch had fired 0 times ever (every inbox document, web/email/WhatsApp/MCP, was extracted twice: 3 707 + 1 666 calls in 30 d). The inbox now declares `extractionOwner: 'invoice-inbox'` on the upload (event payload), the extension yields, and the inbox mirrors its single outcome onto document_attachments (extracted_data / extracted_at / extraction_model) from every writer (sync, deferred, attach, retry, MCP) via one helper, so the status poll and the agent reads keep working unchanged. A DB trigger was considered and rejected: it would mirror user edits (PUT extracted-data) as if they were AI output and needs a migration + pg test for what is a TS-level ordering problem.
|
||||
[2026-08-20] Every "no extraction will ever happen" outcome is now stamped on document_attachments as skipped:<reason> (no_ai_entitlement, ai_unconfigured, system_generated, sandbox, client_opt_out, ai_no_vision, pdf_rasterizer_missing) and the extraction-status route maps the quiet ones to 'disabled' on the first poll. Prod check (30 d, read-only): 309 of the audit's 327 never-extracted uploads belonged to 29 companies with zero successful extractions, i.e. the paywall working silently and the UI spinning 30 s to find out; the remaining 18 are download failures, now stamped failed:storage_download. Self-generated documents (upload_source 'system': our own invoice PDFs, payout files, filings; 59 paid model calls in 30 d) are skipped as system_generated.
|
||||
[2026-08-20] Strict JSON on OpenAI-compatible endpoints uses a hand-maintained JSON-schema mirror of the extraction Zod schema, opt-in via AI_STRICT_JSON, never an automatic Zod-to-JSON-schema conversion: the Zod schema carries .catch()/.transform() that have no schema equivalent and a generated schema would drift silently. Zod stays the validator either way; JSON-in-prose + extractJsonObject remains the default everywhere because it works on every model and is what hosted runs.
|
||||
[2026-08-20] AI_API_KEY made optional for the OpenAI-compatible backend: a base URL alone now counts as configured (hasAiCredentials / resolveAiProvider), so a local model server (llama.cpp/Ollama/LM Studio/vLLM), which usually has no auth, works with just AI_BASE_URL + AI_MODEL. The openai-compatible service only sends an Authorization: Bearer when AI_API_KEY is set, so a keyless local server is never handed an empty bearer. Hosted providers that require a key still set AI_API_KEY. Bedrock/Anthropic credential logic unchanged.
|
||||
|
||||
@@ -50,6 +50,13 @@ vi.mock('@/lib/agent/intents/registry', () => ({
|
||||
getIntent: (...args: unknown[]) => getIntentMock(...args),
|
||||
}))
|
||||
|
||||
// Deployment-level AI availability (distinct from the per-company paywall).
|
||||
// Default: available, so the ownership tests below exercise the real flow.
|
||||
const aiStatusMock = vi.fn()
|
||||
vi.mock('@/lib/ai', () => ({
|
||||
getAiStatus: () => aiStatusMock(),
|
||||
}))
|
||||
|
||||
const runChatTurnMock = vi.fn()
|
||||
vi.mock('@/lib/agent/chat/run-turn', () => ({
|
||||
runChatTurn: (...args: unknown[]) => runChatTurnMock(...args),
|
||||
@@ -92,6 +99,7 @@ beforeEach(() => {
|
||||
checkRateMock.mockResolvedValue({ ok: true })
|
||||
getIntentMock.mockReturnValue({ id: 'general.help', sheetTitle: 'Assistenten' })
|
||||
runChatTurnMock.mockResolvedValue(undefined)
|
||||
aiStatusMock.mockReturnValue({ configured: true, assistantAvailable: true, provider: 'bedrock' })
|
||||
})
|
||||
|
||||
describe('POST /api/agent/invoke', () => {
|
||||
@@ -119,6 +127,20 @@ describe('POST /api/agent/invoke', () => {
|
||||
expect(status).toBe(400)
|
||||
})
|
||||
|
||||
// A self-host without an AI key, or on an OpenAI-compatible endpoint the
|
||||
// chat loop does not speak yet: say so up front with a distinct code, never
|
||||
// open a stream that dies on the first model call, never confuse it with
|
||||
// the paywall.
|
||||
it('returns 503 ai_unconfigured when the deployment has no assistant backend', async () => {
|
||||
aiStatusMock.mockReturnValue({ configured: false, assistantAvailable: false, provider: 'bedrock' })
|
||||
enqueuePreamble()
|
||||
const res = await POST(createMockRequest('/api/agent/invoke', { method: 'POST', body: body() }))
|
||||
const { status, body: json } = await parseJsonResponse<{ error: string; code: string }>(res)
|
||||
expect(status).toBe(503)
|
||||
expect(json.code).toBe('ai_unconfigured')
|
||||
expect(runChatTurnMock).not.toHaveBeenCalled()
|
||||
})
|
||||
|
||||
it('returns 403 when the user is not a member of the company', async () => {
|
||||
enqueue({ data: null }) // company_members: no row
|
||||
const res = await POST(createMockRequest('/api/agent/invoke', { method: 'POST', body: body() }))
|
||||
|
||||
@@ -7,6 +7,7 @@ import { getActiveCompanyId } from '@/lib/company/context'
|
||||
import { getIntent } from '@/lib/agent/intents/registry'
|
||||
import { checkAgentRateLimit, agentRateLimitResponseBody } from '@/lib/rate-limits/agent'
|
||||
import { runChatTurn, friendlyModelError } from '@/lib/agent/chat/run-turn'
|
||||
import { getAiStatus } from '@/lib/ai'
|
||||
import { guardSandbox } from '@/lib/sandbox/guard'
|
||||
import { requireCapability } from '@/lib/entitlements/has-capability'
|
||||
import { CAPABILITY } from '@/lib/entitlements/keys'
|
||||
@@ -117,6 +118,20 @@ export async function POST(request: Request) {
|
||||
const capBlocked = await requireCapability(supabase, companyId, CAPABILITY.ai)
|
||||
if (capBlocked) return capBlocked
|
||||
|
||||
// Distinct from the paywall: the deployment has no AI backend the chat
|
||||
// loop can run on (no credentials, or an OpenAI-compatible endpoint, which
|
||||
// the loop does not speak yet). Answer up front instead of opening a
|
||||
// stream that dies on the first model call.
|
||||
if (!getAiStatus().assistantAvailable) {
|
||||
return NextResponse.json(
|
||||
{
|
||||
error: 'Assistenten är inte konfigurerad på den här installationen.',
|
||||
code: 'ai_unconfigured',
|
||||
},
|
||||
{ status: 503 },
|
||||
)
|
||||
}
|
||||
|
||||
// Resolve the conversation BEFORE any side effect below (the onboarding
|
||||
// intake stamp): a request that is about to be rejected must not write.
|
||||
let conversationId = body.conversation_id ?? null
|
||||
|
||||
@@ -0,0 +1,100 @@
|
||||
import { describe, it, expect, vi, beforeEach } from 'vitest'
|
||||
import { createQueuedMockSupabase, createMockRequest, parseJsonResponse } from '@/tests/helpers'
|
||||
|
||||
const { supabase, enqueue, reset } = createQueuedMockSupabase()
|
||||
|
||||
let unauthorized = false
|
||||
vi.mock('@/lib/api/with-route-context', () => ({
|
||||
withRouteContext: (_op: string, handler: (req: unknown, ctx: unknown, extra: unknown) => unknown) => {
|
||||
return async (req: unknown, extra: unknown) => {
|
||||
if (unauthorized) {
|
||||
return new Response(JSON.stringify({ error: 'Unauthorized' }), { status: 401 })
|
||||
}
|
||||
return handler(req, { supabase, companyId: 'company-1', user: { id: 'user-1' } }, extra)
|
||||
}
|
||||
},
|
||||
}))
|
||||
|
||||
const aiStatusMock = vi.fn()
|
||||
vi.mock('@/lib/ai', () => ({
|
||||
getAiStatus: () => aiStatusMock(),
|
||||
}))
|
||||
|
||||
import { GET, deriveExtractionStatus } from '../route'
|
||||
|
||||
const params = { params: Promise.resolve({ id: 'doc-1' }) }
|
||||
|
||||
beforeEach(() => {
|
||||
vi.clearAllMocks()
|
||||
reset()
|
||||
unauthorized = false
|
||||
aiStatusMock.mockReturnValue({ configured: true })
|
||||
})
|
||||
|
||||
describe('deriveExtractionStatus', () => {
|
||||
const base = { extracted_at: '2026-08-20T10:00:00Z', extracted_data: null, extraction_model: null }
|
||||
|
||||
it('is running while unstamped on a configured deployment', () => {
|
||||
expect(deriveExtractionStatus({ ...base, extracted_at: null }, true)).toBe('running')
|
||||
})
|
||||
|
||||
// The self-host "no key yet" case: answer on the first poll, no 30 s hang.
|
||||
it('is disabled while unstamped on an unconfigured deployment', () => {
|
||||
expect(deriveExtractionStatus({ ...base, extracted_at: null }, false)).toBe('disabled')
|
||||
})
|
||||
|
||||
it('is succeeded when data landed', () => {
|
||||
expect(deriveExtractionStatus({ ...base, extracted_data: { a: 1 } }, true)).toBe('succeeded')
|
||||
})
|
||||
|
||||
it('maps the quiet skips (paywall, unconfigured, sandbox, opt-out, system) to disabled', () => {
|
||||
for (const m of [
|
||||
'skipped:no_ai_entitlement',
|
||||
'skipped:ai_unconfigured',
|
||||
'skipped:sandbox',
|
||||
'skipped:client_opt_out',
|
||||
'skipped:system_generated',
|
||||
]) {
|
||||
expect(deriveExtractionStatus({ ...base, extraction_model: m }, true)).toBe('disabled')
|
||||
}
|
||||
})
|
||||
|
||||
it('maps every other skip to unsupported and failures to failed', () => {
|
||||
expect(deriveExtractionStatus({ ...base, extraction_model: 'skipped:unsupported_mime' }, true)).toBe('unsupported')
|
||||
expect(deriveExtractionStatus({ ...base, extraction_model: 'skipped:ai_no_vision' }, true)).toBe('unsupported')
|
||||
expect(deriveExtractionStatus({ ...base, extraction_model: 'failed:no_raw_text' }, true)).toBe('failed')
|
||||
expect(deriveExtractionStatus({ ...base, extraction_model: null }, true)).toBe('failed')
|
||||
})
|
||||
})
|
||||
|
||||
describe('GET /api/documents/:id/extraction-status', () => {
|
||||
it('returns 401 when unauthenticated', async () => {
|
||||
unauthorized = true
|
||||
const res = await GET(createMockRequest('/api/documents/doc-1/extraction-status'), params)
|
||||
const { status } = await parseJsonResponse(res)
|
||||
expect(status).toBe(401)
|
||||
})
|
||||
|
||||
it('returns 404 for an unknown document', async () => {
|
||||
enqueue({ data: null })
|
||||
const res = await GET(createMockRequest('/api/documents/doc-1/extraction-status'), params)
|
||||
const { status } = await parseJsonResponse(res)
|
||||
expect(status).toBe(404)
|
||||
})
|
||||
|
||||
it('returns the derived status', async () => {
|
||||
enqueue({ data: { id: 'doc-1', extracted_at: '2026-08-20T10:00:00Z', extracted_data: null, extraction_model: 'skipped:no_ai_entitlement' } })
|
||||
const res = await GET(createMockRequest('/api/documents/doc-1/extraction-status'), params)
|
||||
const { status, body } = await parseJsonResponse<{ data: { status: string } }>(res)
|
||||
expect(status).toBe(200)
|
||||
expect(body.data.status).toBe('disabled')
|
||||
})
|
||||
|
||||
it('answers disabled immediately on an unconfigured deployment', async () => {
|
||||
aiStatusMock.mockReturnValue({ configured: false })
|
||||
enqueue({ data: { id: 'doc-1', extracted_at: null, extracted_data: null, extraction_model: null } })
|
||||
const res = await GET(createMockRequest('/api/documents/doc-1/extraction-status'), params)
|
||||
const { body } = await parseJsonResponse<{ data: { status: string } }>(res)
|
||||
expect(body.data.status).toBe('disabled')
|
||||
})
|
||||
})
|
||||
@@ -1,5 +1,6 @@
|
||||
import { NextResponse } from 'next/server'
|
||||
import { withRouteContext } from '@/lib/api/with-route-context'
|
||||
import { getAiStatus } from '@/lib/ai'
|
||||
import { getErrorMessage as getUserErrorMessage } from '@/lib/errors/get-error-message'
|
||||
|
||||
// GET /api/documents/:id/extraction-status
|
||||
@@ -9,15 +10,47 @@ import { getErrorMessage as getUserErrorMessage } from '@/lib/errors/get-error-m
|
||||
// touching storage (no signed URL creation per poll).
|
||||
//
|
||||
// Derived status:
|
||||
// running : extracted_at IS NULL (pipeline hasn't stamped yet)
|
||||
// running : extracted_at IS NULL and this deployment has AI configured
|
||||
// (the pipeline hasn't stamped yet)
|
||||
// succeeded : extracted_at IS NOT NULL AND extracted_data IS NOT NULL
|
||||
// unsupported : extraction_model = 'skipped:*' (HEIC, ZIP, …)
|
||||
// disabled : no extraction will ever happen for this document and that
|
||||
// is not the document's fault: AI is not configured on this
|
||||
// deployment (extracted_at NULL + unconfigured, or stamped
|
||||
// skipped:ai_unconfigured), the company is not entitled to
|
||||
// AI extraction (skipped:no_ai_entitlement), a sandbox or
|
||||
// client opt-out, or a document the system generated itself
|
||||
// (skipped:system_generated). The UI shows nothing.
|
||||
// unsupported : any other skipped:* (HEIC, ZIP, no vision model, …)
|
||||
// failed : extracted_at IS NOT NULL AND extracted_data IS NULL AND
|
||||
// extraction_model = 'failed:*'
|
||||
// disabled : the document-extraction extension isn't enabled (column
|
||||
// stays untouched indefinitely). Client times out and shows
|
||||
// a quiet fallback. We don't distinguish this from running
|
||||
// server-side: the client decides based on elapsed time.
|
||||
//
|
||||
// Before extraction was stamped on every outcome, free-tier uploads and
|
||||
// unconfigured self-hosts left the column NULL forever and the client had to
|
||||
// decide "disabled" by timing out after 30 s. It still keeps that timeout as
|
||||
// the last resort (the document-extraction extension may be switched off).
|
||||
export type DocumentExtractionStatus = 'running' | 'succeeded' | 'failed' | 'unsupported' | 'disabled'
|
||||
|
||||
const QUIET_SKIPS = new Set([
|
||||
'skipped:ai_unconfigured',
|
||||
'skipped:no_ai_entitlement',
|
||||
'skipped:sandbox',
|
||||
'skipped:client_opt_out',
|
||||
'skipped:system_generated',
|
||||
])
|
||||
|
||||
export function deriveExtractionStatus(row: {
|
||||
extracted_at: string | null
|
||||
extracted_data: unknown
|
||||
extraction_model: string | null
|
||||
}, aiConfigured: boolean): DocumentExtractionStatus {
|
||||
if (!row.extracted_at) return aiConfigured ? 'running' : 'disabled'
|
||||
if (row.extracted_data) return 'succeeded'
|
||||
const model = row.extraction_model ?? ''
|
||||
if (QUIET_SKIPS.has(model)) return 'disabled'
|
||||
if (model.startsWith('skipped:')) return 'unsupported'
|
||||
return 'failed'
|
||||
}
|
||||
|
||||
export const GET = withRouteContext<{ params: Promise<{ id: string }> }>(
|
||||
'document.extraction_status',
|
||||
async (_request, { supabase, companyId }, { params }) => {
|
||||
@@ -33,27 +66,19 @@ export const GET = withRouteContext<{ params: Promise<{ id: string }> }>(
|
||||
if (error) return NextResponse.json({ error: getUserErrorMessage(error) }, { status: 500 })
|
||||
if (!data) return NextResponse.json({ error: 'Not found' }, { status: 404 })
|
||||
|
||||
const extractedAt = data.extracted_at as string | null
|
||||
const extractedData = data.extracted_data as Record<string, unknown> | null
|
||||
const model = data.extraction_model as string | null
|
||||
|
||||
let status: 'running' | 'succeeded' | 'failed' | 'unsupported'
|
||||
if (!extractedAt) {
|
||||
status = 'running'
|
||||
} else if (extractedData) {
|
||||
status = 'succeeded'
|
||||
} else if (model?.startsWith('skipped:')) {
|
||||
status = 'unsupported'
|
||||
} else {
|
||||
status = 'failed'
|
||||
const row = {
|
||||
extracted_at: data.extracted_at as string | null,
|
||||
extracted_data: data.extracted_data as Record<string, unknown> | null,
|
||||
extraction_model: data.extraction_model as string | null,
|
||||
}
|
||||
const status = deriveExtractionStatus(row, getAiStatus().configured)
|
||||
|
||||
return NextResponse.json({
|
||||
data: {
|
||||
id: data.id,
|
||||
status,
|
||||
extracted_at: extractedAt,
|
||||
extraction_model: model,
|
||||
extracted_at: row.extracted_at,
|
||||
extraction_model: row.extraction_model,
|
||||
},
|
||||
})
|
||||
}
|
||||
|
||||
+29
-9
@@ -199,7 +199,7 @@ Additionally, migration 048 schedules a `pg_cron` job inside the database that m
|
||||
|
||||
### AI Features
|
||||
|
||||
All AI features (automatic interpretation of uploaded receipts and invoices via the `document-extraction` and `invoice-inbox` extensions, and the in-app AI assistant) run Claude. There are two ways to provide credentials; pick one.
|
||||
All AI features (automatic interpretation of uploaded receipts and invoices via the `document-extraction` and `invoice-inbox` extensions, and the in-app AI assistant) run on one configured backend. There are three ways to provide one; pick one. Note that the agent surface most integrations use, the MCP server, needs no AI backend at all: it is your own agent (Claude, Codex, a local model) talking to the ledger, so a deployment without any of the credentials below is still fully usable that way.
|
||||
|
||||
The stock self-hosted image includes both extraction extensions, so these credentials cover emailed invoices and documents uploaded in the app.
|
||||
|
||||
@@ -219,18 +219,38 @@ AWS_REGION=eu-north-1 # default
|
||||
|
||||
Set both static AWS keys explicitly. The AI assistant's client can fall back to the standard AWS credential provider chain (instance profile, IRSA) when they are absent, but document extraction requires `AWS_ACCESS_KEY_ID` and `AWS_SECRET_ACCESS_KEY` and silently returns empty results without them.
|
||||
|
||||
Optional model overrides, in either setup:
|
||||
**Option 3: any OpenAI-compatible endpoint.** Any server that implements the chat-completions API: a Swedish inference provider for a fully sovereign deployment, or a **local model** on the same machine (llama.cpp's `server`, Ollama's `/v1`, LM Studio, vLLM). Document extraction (receipts, invoices, HTML mail invoices) and the single-call assistant (`/api/agent/ask`) run here on any provider; the older streaming chat panel is being replaced by that single-call surface.
|
||||
|
||||
```bash
|
||||
BEDROCK_MODEL_ID=claude-sonnet-5 # document extraction model
|
||||
BEDROCK_OPUS_MODEL_ID=... # assistant model, heavy intents
|
||||
BEDROCK_SONNET_MODEL_ID=... # assistant model, standard intents
|
||||
AI_PROVIDER=bedrock|anthropic # force the backend (see below)
|
||||
AI_BASE_URL=http://localhost:11434/v1 # the endpoint's OpenAI-compatible base URL (here: a local Ollama)
|
||||
AI_MODEL=qwen3.8 # a model id is required: there is no default for an arbitrary endpoint
|
||||
# AI_API_KEY=... # OPTIONAL: only when the endpoint needs auth. A local server usually
|
||||
# # has none, so leave it unset; a hosted provider gives you a key.
|
||||
# AI_EXTRACTION_MODEL=... # optional: a vision model for document reading, if AI_MODEL is not one
|
||||
```
|
||||
|
||||
When both credential sets are present, Bedrock wins, so that adding an Anthropic key for an experiment cannot silently move production inference out of eu-north-1. Set `AI_PROVIDER` to say which you mean. A model id written without a provider prefix is adapted to whichever backend is active; an id that already carries one (`eu.anthropic.…`) is used as-is.
|
||||
Three things about such endpoints are declared rather than probed, because the app cannot tell from the outside:
|
||||
|
||||
Without working credentials the rest of the app runs normally: uploads are stored but not auto-interpreted, and the AI assistant cannot answer.
|
||||
- `AI_VISION=false` says the configured model cannot read images. Images and PDFs are then skipped honestly (the inbox row lands with the empty skeleton and the "AI-tolkning kördes inte" hint) instead of failing with a 400 on every upload; HTML mail invoices still extract as text on any model.
|
||||
- `AI_PDF_MODE` defaults to `rasterize` here: most such endpoints have no PDF input, so the first `AI_PDF_MAX_PAGES` pages (default 4) are rendered to images with poppler's `pdftoppm`, which the self-host image installs. If the binary is missing, PDFs are skipped with `pdf_rasterizer_missing` rather than failing. A provider that accepts the OpenAI `file` content part can use `AI_PDF_MODE=native`.
|
||||
- `AI_STRICT_JSON=true` asks for `response_format: json_schema` on providers that enforce it. The default (JSON answered in prose, then parsed and validated) works on every model and is what hosted runs.
|
||||
|
||||
Optional model overrides, in any setup:
|
||||
|
||||
```bash
|
||||
AI_MODEL=... # default model for every tier (OpenAI-compatible: required)
|
||||
AI_EXTRACTION_MODEL=... # document extraction model
|
||||
AI_HEAVY_MODEL=... # assistant model, heavy intents
|
||||
AI_ASSISTANT_MODEL=... # assistant model, standard intents
|
||||
AI_EXTRACTION_MAX_TOKENS=8192 # output cap for document extraction
|
||||
AI_PROVIDER=bedrock|anthropic|openai-compatible # force the backend (see below)
|
||||
```
|
||||
|
||||
The pre-existing names `BEDROCK_MODEL_ID`, `BEDROCK_OPUS_MODEL_ID`, `BEDROCK_SONNET_MODEL_ID` and `BEDROCK_MAX_TOKENS` keep working as the same overrides (extraction, heavy, standard, extraction cap) on every backend; the `AI_*` names take precedence when both are set. Claude deployments default every tier to `claude-sonnet-5`.
|
||||
|
||||
When several credential sets are present, Bedrock wins, then the direct Anthropic API, then the OpenAI-compatible endpoint, so that adding a key for an experiment cannot silently move production inference out of eu-north-1. Set `AI_PROVIDER` to say which you mean. A model id written without a provider prefix is adapted to whichever backend is active; an id that already carries one (`eu.anthropic.…`) is used as-is.
|
||||
|
||||
Without working credentials the rest of the app runs normally: uploads are stored but not auto-interpreted (the upload UI sees that immediately rather than waiting for a timeout), and the AI assistant answers `503 ai_unconfigured`.
|
||||
|
||||
#### Verifying the setup
|
||||
|
||||
@@ -248,7 +268,7 @@ npx tsx scripts/smoke-ai.ts ./receipt.pdf # also runs document extraction
|
||||
|
||||
It prints the resolved provider and model ids first, then exercises a plain request, a streamed turn carrying the assistant's full parameter set (adaptive thinking, effort, prompt caching and a tool), and finally extraction of the file you pass. It exits non-zero if any step fails, so it works as a post-deploy check.
|
||||
|
||||
> **Note:** `OPENAI_API_KEY` from earlier versions is not read by any code path; there is no OpenAI route in the app. Pluggable providers beyond Claude are tracked in [#1406](https://github.com/erp-mafia/accounted/issues/1406).
|
||||
> **Note:** `OPENAI_API_KEY` from earlier versions is not read by any code path. To use OpenAI itself, point Option 3 at `https://api.openai.com/v1`; the app has no provider-specific OpenAI integration, only the OpenAI-compatible one. Background: [#1406](https://github.com/erp-mafia/accounted/issues/1406).
|
||||
|
||||
### Email (Invoice Sending and Reminders)
|
||||
|
||||
|
||||
@@ -0,0 +1,158 @@
|
||||
import { describe, it, expect, vi, beforeEach } from 'vitest'
|
||||
import { createQueuedMockSupabase } from '@/tests/helpers'
|
||||
|
||||
const { supabase, enqueue, reset, findCalls } = createQueuedMockSupabase()
|
||||
|
||||
vi.mock('@/lib/supabase/server', () => ({
|
||||
createServiceClient: () => supabase,
|
||||
}))
|
||||
|
||||
const extractMock = vi.fn()
|
||||
vi.mock('@/extensions/general/invoice-inbox/lib/extract-invoice-fields', () => ({
|
||||
extractInvoiceFields: (...args: unknown[]) => extractMock(...args),
|
||||
}))
|
||||
|
||||
const hasCapabilityMock = vi.fn()
|
||||
vi.mock('@/lib/entitlements/has-capability', () => ({
|
||||
hasCapability: (...args: unknown[]) => hasCapabilityMock(...args),
|
||||
}))
|
||||
|
||||
const aiStatusMock = vi.fn()
|
||||
vi.mock('@/lib/ai', () => ({
|
||||
getAiStatus: () => aiStatusMock(),
|
||||
}))
|
||||
|
||||
import { documentExtractionExtension } from '../index'
|
||||
|
||||
const handler = documentExtractionExtension.eventHandlers![0].handler
|
||||
|
||||
function doc(overrides: Record<string, unknown> = {}) {
|
||||
return {
|
||||
id: 'doc-1',
|
||||
company_id: 'company-1',
|
||||
file_name: 'kvitto.pdf',
|
||||
mime_type: 'application/pdf',
|
||||
storage_path: 'company-1/user-1/kvitto.pdf',
|
||||
upload_source: 'file_upload',
|
||||
...overrides,
|
||||
}
|
||||
}
|
||||
|
||||
function payload(overrides: Record<string, unknown> = {}, document = doc()) {
|
||||
return { document, userId: 'user-1', companyId: 'company-1', ...overrides }
|
||||
}
|
||||
|
||||
/** extraction_model of the LAST document_attachments update, or undefined. */
|
||||
function lastStamp(): string | undefined {
|
||||
const updates = findCalls('document_attachments', 'update')
|
||||
const last = updates[updates.length - 1]?.[0] as { extraction_model?: string } | undefined
|
||||
return last?.extraction_model
|
||||
}
|
||||
|
||||
beforeEach(() => {
|
||||
vi.clearAllMocks()
|
||||
reset()
|
||||
aiStatusMock.mockReturnValue({ configured: true, assistantAvailable: true })
|
||||
hasCapabilityMock.mockResolvedValue(true)
|
||||
extractMock.mockResolvedValue({
|
||||
data: { supplier: { name: 'Elgiganten' } },
|
||||
rawText: '{"supplier":{"name":"Elgiganten"}}',
|
||||
model: 'eu.anthropic.claude-sonnet-5',
|
||||
})
|
||||
})
|
||||
|
||||
describe('document-extraction handler', () => {
|
||||
// THE dedupe: inbox-owned documents are extracted (and mirrored) by the
|
||||
// inbox itself. The handler used to race it and pay a second model call.
|
||||
it('yields entirely when the inbox owns extraction', async () => {
|
||||
await handler(payload({ extractionOwner: 'invoice-inbox' }))
|
||||
expect(supabase.from).not.toHaveBeenCalled()
|
||||
expect(extractMock).not.toHaveBeenCalled()
|
||||
})
|
||||
|
||||
it('stamps unsupported types from the payload without reading the row', async () => {
|
||||
enqueue({ data: null }) // the stamp update
|
||||
await handler(payload({}, doc({ mime_type: 'application/json' })))
|
||||
expect(findCalls('document_attachments', 'select')).toHaveLength(0)
|
||||
expect(lastStamp()).toBe('skipped:unsupported_mime')
|
||||
expect(extractMock).not.toHaveBeenCalled()
|
||||
})
|
||||
|
||||
// Our own invoice PDFs, payout files, filings: nothing to read, paid calls
|
||||
// to waste (on hosted and on a BYO-key self-host).
|
||||
it('stamps system-generated documents instead of extracting them', async () => {
|
||||
enqueue({ data: null })
|
||||
await handler(payload({}, doc({ upload_source: 'system' })))
|
||||
expect(lastStamp()).toBe('skipped:system_generated')
|
||||
expect(extractMock).not.toHaveBeenCalled()
|
||||
})
|
||||
|
||||
it('does nothing for a row that was already attempted', async () => {
|
||||
enqueue({ data: { id: 'doc-1', mime_type: 'application/pdf', storage_path: 'p', extracted_at: '2026-08-20T00:00:00Z' } })
|
||||
await handler(payload())
|
||||
expect(findCalls('document_attachments', 'update')).toHaveLength(0)
|
||||
expect(extractMock).not.toHaveBeenCalled()
|
||||
})
|
||||
|
||||
// Self-host without an AI key: stamp so the status route answers
|
||||
// 'disabled' on the first poll instead of after a 30 s timeout.
|
||||
it('stamps ai_unconfigured when the deployment has no AI', async () => {
|
||||
aiStatusMock.mockReturnValue({ configured: false, assistantAvailable: false })
|
||||
enqueue({ data: { id: 'doc-1', mime_type: 'application/pdf', storage_path: 'p', extracted_at: null } })
|
||||
enqueue({ data: null })
|
||||
await handler(payload())
|
||||
expect(lastStamp()).toBe('skipped:ai_unconfigured')
|
||||
expect(hasCapabilityMock).not.toHaveBeenCalled()
|
||||
expect(extractMock).not.toHaveBeenCalled()
|
||||
})
|
||||
|
||||
// The paywall, made visible: 309 of the 327 never-extracted uploads in a
|
||||
// 30-day prod window belonged to companies without the ai capability.
|
||||
it('stamps no_ai_entitlement for companies without the ai capability', async () => {
|
||||
hasCapabilityMock.mockResolvedValue(false)
|
||||
enqueue({ data: { id: 'doc-1', mime_type: 'application/pdf', storage_path: 'p', extracted_at: null } })
|
||||
enqueue({ data: null })
|
||||
await handler(payload())
|
||||
expect(lastStamp()).toBe('skipped:no_ai_entitlement')
|
||||
expect(extractMock).not.toHaveBeenCalled()
|
||||
})
|
||||
|
||||
it('stamps a storage download failure', async () => {
|
||||
enqueue({ data: { id: 'doc-1', mime_type: 'application/pdf', storage_path: 'p', extracted_at: null } })
|
||||
enqueue({ data: null })
|
||||
supabase.storage.from.mockReturnValueOnce({
|
||||
download: vi.fn().mockResolvedValue({ data: null, error: { message: 'boom' } }),
|
||||
})
|
||||
await handler(payload())
|
||||
expect(lastStamp()).toBe('failed:storage_download')
|
||||
expect(extractMock).not.toHaveBeenCalled()
|
||||
})
|
||||
|
||||
it('persists the result with the model that answered', async () => {
|
||||
enqueue({ data: { id: 'doc-1', mime_type: 'application/pdf', storage_path: 'p', extracted_at: null } })
|
||||
enqueue({ data: null })
|
||||
await handler(payload())
|
||||
expect(extractMock).toHaveBeenCalledWith(expect.objectContaining({ mimeType: 'application/pdf', fileName: 'kvitto.pdf' }))
|
||||
const updates = findCalls('document_attachments', 'update')
|
||||
expect(updates[updates.length - 1][0]).toMatchObject({
|
||||
extracted_data: { supplier: { name: 'Elgiganten' } },
|
||||
extraction_model: 'eu.anthropic.claude-sonnet-5',
|
||||
})
|
||||
})
|
||||
|
||||
it('stamps the skip reason the extractor reports (no vision, rasterizer missing, ...)', async () => {
|
||||
extractMock.mockResolvedValue({ data: {}, rawText: null, skipped: 'pdf_rasterizer_missing' })
|
||||
enqueue({ data: { id: 'doc-1', mime_type: 'application/pdf', storage_path: 'p', extracted_at: null } })
|
||||
enqueue({ data: null })
|
||||
await handler(payload())
|
||||
expect(lastStamp()).toBe('skipped:pdf_rasterizer_missing')
|
||||
})
|
||||
|
||||
it('stamps failed:no_raw_text when the model call produced nothing parseable', async () => {
|
||||
extractMock.mockResolvedValue({ data: {}, rawText: null })
|
||||
enqueue({ data: { id: 'doc-1', mime_type: 'application/pdf', storage_path: 'p', extracted_at: null } })
|
||||
enqueue({ data: null })
|
||||
await handler(payload())
|
||||
expect(lastStamp()).toBe('failed:no_raw_text')
|
||||
})
|
||||
})
|
||||
@@ -1,7 +1,7 @@
|
||||
import type { Extension } from '@/lib/extensions/types'
|
||||
import type { SupabaseClient } from '@supabase/supabase-js'
|
||||
import { extractInvoiceFields } from '@/extensions/general/invoice-inbox/lib/extract-invoice-fields'
|
||||
import { toProviderModelId } from '@/lib/ai/provider'
|
||||
import { getAiStatus } from '@/lib/ai'
|
||||
import { hasCapability } from '@/lib/entitlements/has-capability'
|
||||
import { CAPABILITY } from '@/lib/entitlements/keys'
|
||||
import { createLogger } from '@/lib/logger'
|
||||
@@ -10,8 +10,9 @@ import type { DocumentAttachment } from '@/types'
|
||||
|
||||
const log = createLogger('document-extraction')
|
||||
|
||||
// Mime types we know Claude can read directly via Bedrock. Anything else
|
||||
// (HEIC, ZIP, TXT, …) is skipped: extracted_at still gets stamped so the
|
||||
// Mime types the extraction can read (a vision model reads PDFs natively on
|
||||
// Claude; OpenAI-compatible backends rasterize them). Anything else (HEIC,
|
||||
// ZIP, TXT, bank JSON, …) is skipped: extracted_at still gets stamped so the
|
||||
// row is marked as "attempted, not eligible".
|
||||
const SUPPORTED_MIME_TYPES = new Set([
|
||||
'application/pdf',
|
||||
@@ -23,43 +24,65 @@ const SUPPORTED_MIME_TYPES = new Set([
|
||||
|
||||
// AI-extraction extension: paid AI tier only.
|
||||
//
|
||||
// Subscribes to the existing document.uploaded event bus topic and runs
|
||||
// Sonnet 4.6 (via Bedrock, reusing invoice-inbox's extractInvoiceFields) on
|
||||
// every uploaded receipt or invoice. Writes the result to
|
||||
// Subscribes to the existing document.uploaded event bus topic and runs the
|
||||
// configured extraction model (reusing invoice-inbox's extractInvoiceFields)
|
||||
// on every uploaded receipt or invoice. Writes the result to
|
||||
// document_attachments.extracted_data so the agent intent capture can use
|
||||
// it without re-asking the user.
|
||||
//
|
||||
// Idempotency: skips when extracted_at is already set on the row. Also
|
||||
// dedupes against invoice_inbox_items.extracted_data: when the inbox
|
||||
// extension already extracted the same file, we copy its result instead
|
||||
// of paying for a second Sonnet call.
|
||||
// Idempotency: skips when extracted_at is already set on the row.
|
||||
//
|
||||
// Ownership: documents that enter through the invoice inbox (web upload,
|
||||
// email, WhatsApp, MCP upload) are extracted by the inbox itself, which
|
||||
// mirrors its result onto document_attachments when done. The inbox marks
|
||||
// the upload with `extractionOwner: 'invoice-inbox'` on the event payload
|
||||
// and this handler yields. It used to race instead: the inbox row did not
|
||||
// exist yet when this handler ran inside uploadDocument(), so every inbox
|
||||
// document was extracted twice (two paid model calls per receipt).
|
||||
//
|
||||
// Every outcome that means "no extraction will ever happen here" is stamped
|
||||
// as `skipped:<reason>` so the polling UI stops immediately instead of
|
||||
// timing out: paywall (`no_ai_entitlement`), no AI configured on this
|
||||
// deployment (`ai_unconfigured`), self-generated documents (our own invoice
|
||||
// PDFs, payout files: `system_generated`), unsupported types.
|
||||
//
|
||||
// Free tier: disable this extension in extensions.config.json. Uploads
|
||||
// still work; the agent intent will see null extracted_data and either
|
||||
// ask the user or call gnubok_get_document_content at chat-time.
|
||||
//
|
||||
// See dev_docs/specialized-agent-plan.md (§ paid/free tier note): to be
|
||||
// authored.
|
||||
export const documentExtractionExtension: Extension = {
|
||||
id: 'document-extraction',
|
||||
name: 'AI document extraction',
|
||||
version: '1.0.0',
|
||||
version: '1.1.0',
|
||||
|
||||
eventHandlers: [
|
||||
{
|
||||
eventType: 'document.uploaded',
|
||||
handler: async (payload) => {
|
||||
const { document, companyId } = payload as {
|
||||
const { document, companyId, extractionOwner } = payload as {
|
||||
document: DocumentAttachment
|
||||
userId: string
|
||||
companyId: string
|
||||
extractionOwner?: 'invoice-inbox'
|
||||
}
|
||||
if (extractionOwner === 'invoice-inbox') return
|
||||
await extractAndPersist(document, companyId)
|
||||
},
|
||||
},
|
||||
],
|
||||
}
|
||||
|
||||
async function stamp(
|
||||
supabase: SupabaseClient,
|
||||
documentId: string,
|
||||
extractionModel: string
|
||||
): Promise<void> {
|
||||
const { error } = await supabase
|
||||
.from('document_attachments')
|
||||
.update({ extracted_at: new Date().toISOString(), extraction_model: extractionModel })
|
||||
.eq('id', documentId)
|
||||
if (error) log.warn('stamp failed', { doc: documentId, extractionModel, err: error.message })
|
||||
}
|
||||
|
||||
async function extractAndPersist(
|
||||
document: DocumentAttachment,
|
||||
companyId: string,
|
||||
@@ -69,9 +92,22 @@ async function extractAndPersist(
|
||||
// events have no user context.
|
||||
const supabase: SupabaseClient = createServiceClient()
|
||||
|
||||
// Cheap gates first, straight from the event payload: no DB round trip for
|
||||
// the thousands of bank-statement JSON documents a sync produces.
|
||||
const mimeType = (document.mime_type as string | null) ?? null
|
||||
if (!mimeType || !SUPPORTED_MIME_TYPES.has(mimeType)) {
|
||||
await stamp(supabase, document.id, 'skipped:unsupported_mime')
|
||||
return
|
||||
}
|
||||
// Documents the system produced itself (our own invoice PDFs, payment
|
||||
// files, filings) carry nothing to extract; reading them back with a paid
|
||||
// model is waste, on hosted and doubly so on a BYO-key self-host.
|
||||
if (document.upload_source === 'system') {
|
||||
await stamp(supabase, document.id, 'skipped:system_generated')
|
||||
return
|
||||
}
|
||||
|
||||
// Idempotency guard: never re-extract a row that already has extracted_at.
|
||||
// Note: the column may be null OR the row may not yet have the new
|
||||
// schema (legacy supabase types). Fail closed on missing schema.
|
||||
const { data: existing, error: existingErr } = await supabase
|
||||
.from('document_attachments')
|
||||
.select('id, mime_type, storage_path, extracted_at')
|
||||
@@ -88,104 +124,70 @@ async function extractAndPersist(
|
||||
return
|
||||
}
|
||||
|
||||
// Dedup against inbox: if invoice-inbox already extracted this exact file
|
||||
// (same document_id), copy its result to avoid a second AI call. If the
|
||||
// inbox row marked the upload as skip_extraction=true, the inbox row's
|
||||
// extracted_data is an empty skeleton: we must stamp the doc with a
|
||||
// 'skipped:*' model so the client-side useDocumentExtraction hook reports
|
||||
// 'unsupported' rather than 'succeeded' (otherwise the UI would claim AI
|
||||
// finished reading a doc it never opened).
|
||||
const { data: inboxRow } = await supabase
|
||||
.from('invoice_inbox_items')
|
||||
.select('extracted_data, extraction_skipped')
|
||||
.eq('document_id', document.id)
|
||||
.maybeSingle()
|
||||
|
||||
if (inboxRow?.extraction_skipped) {
|
||||
await supabase
|
||||
.from('document_attachments')
|
||||
.update({
|
||||
extracted_at: new Date().toISOString(),
|
||||
extraction_model: 'skipped:invoice_inbox_gate',
|
||||
})
|
||||
.eq('id', document.id)
|
||||
// No AI configured on this deployment: stamp and stop. This is the
|
||||
// self-host "key not set yet" state; the status route reports it as
|
||||
// disabled on the first poll instead of after a 30 s timeout.
|
||||
if (!getAiStatus().configured) {
|
||||
await stamp(supabase, document.id, 'skipped:ai_unconfigured')
|
||||
return
|
||||
}
|
||||
|
||||
// Paywall: the free/manual tier never triggers paid extraction. Stamp it so
|
||||
// the row reads "attempted, not entitled" rather than NULL forever (which
|
||||
// the polling UI could only interpret by timing out).
|
||||
if (!(await hasCapability(supabase, companyId, CAPABILITY.ai))) {
|
||||
log.info('extraction skipped, ai capability not entitled', { doc: document.id, companyId })
|
||||
await stamp(supabase, document.id, 'skipped:no_ai_entitlement')
|
||||
return
|
||||
}
|
||||
|
||||
// Download the file from Supabase Storage. The bucket is private: the
|
||||
// service-role client can read any path.
|
||||
const storagePath = existing.storage_path as string | null
|
||||
if (!storagePath) {
|
||||
log.warn('document has no storage_path, skipping', { doc: document.id })
|
||||
await stamp(supabase, document.id, 'failed:no_storage_path')
|
||||
return
|
||||
}
|
||||
|
||||
const { data: blob, error: dlErr } = await supabase.storage
|
||||
.from('documents')
|
||||
.download(storagePath)
|
||||
if (dlErr || !blob) {
|
||||
log.warn('storage download failed', { doc: document.id, err: dlErr?.message })
|
||||
await stamp(supabase, document.id, 'failed:storage_download')
|
||||
return
|
||||
}
|
||||
const buffer = Buffer.from(await blob.arrayBuffer())
|
||||
|
||||
let extractedData: Record<string, unknown> | null = null
|
||||
let model: string = 'copied-from-invoice-inbox'
|
||||
|
||||
if (inboxRow?.extracted_data) {
|
||||
extractedData = inboxRow.extracted_data as Record<string, unknown>
|
||||
} else {
|
||||
const mimeType = existing.mime_type as string | null
|
||||
if (!mimeType || !SUPPORTED_MIME_TYPES.has(mimeType)) {
|
||||
// Stamp the attempt so we don't keep retrying unsupported types.
|
||||
await supabase
|
||||
.from('document_attachments')
|
||||
.update({ extracted_at: new Date().toISOString(), extraction_model: 'skipped:unsupported_mime' })
|
||||
.eq('id', document.id)
|
||||
let model: string
|
||||
try {
|
||||
const { data, rawText, model: usedModel, skipped } = await extractInvoiceFields({
|
||||
buffer,
|
||||
mimeType,
|
||||
fileName: (document.file_name as string) || 'document',
|
||||
})
|
||||
// extractInvoiceFields returns an "empty" result on failure rather than
|
||||
// throwing. `skipped` means no model call was made (and why); a null
|
||||
// rawText with no skip means the call failed or the JSON parse did.
|
||||
if (skipped) {
|
||||
await stamp(supabase, document.id, `skipped:${skipped}`)
|
||||
return
|
||||
}
|
||||
|
||||
// Download the file from Supabase Storage. The bucket is private: the
|
||||
// service-role client can read any path.
|
||||
const storagePath = existing.storage_path as string | null
|
||||
if (!storagePath) {
|
||||
log.warn('document has no storage_path, skipping', { doc: document.id })
|
||||
return
|
||||
}
|
||||
|
||||
const { data: blob, error: dlErr } = await supabase.storage
|
||||
.from('documents')
|
||||
.download(storagePath)
|
||||
if (dlErr || !blob) {
|
||||
log.warn('storage download failed', { doc: document.id, err: dlErr?.message })
|
||||
return
|
||||
}
|
||||
const buffer = Buffer.from(await blob.arrayBuffer())
|
||||
|
||||
if (!(await hasCapability(supabase, companyId, CAPABILITY.ai))) {
|
||||
log.info('extraction skipped, ai capability not entitled', { doc: document.id, companyId })
|
||||
return
|
||||
}
|
||||
|
||||
try {
|
||||
const { data, rawText } = await extractInvoiceFields({
|
||||
buffer,
|
||||
mimeType,
|
||||
fileName: (document.file_name as string) || 'document',
|
||||
})
|
||||
// extractInvoiceFields returns an "empty" result on failure rather
|
||||
// than throwing: distinguish by checking rawText. When rawText is
|
||||
// null, the call was skipped (creds missing, unsupported type) or
|
||||
// the JSON parse failed.
|
||||
if (!rawText) {
|
||||
await supabase
|
||||
.from('document_attachments')
|
||||
.update({
|
||||
extracted_at: new Date().toISOString(),
|
||||
extraction_model: 'failed:no_raw_text',
|
||||
})
|
||||
.eq('id', document.id)
|
||||
return
|
||||
}
|
||||
extractedData = data as unknown as Record<string, unknown>
|
||||
model = toProviderModelId(process.env.BEDROCK_MODEL_ID || 'claude-sonnet-5')
|
||||
} catch (err) {
|
||||
log.warn('extraction threw', {
|
||||
doc: document.id,
|
||||
err: err instanceof Error ? err.message : String(err),
|
||||
})
|
||||
await supabase
|
||||
.from('document_attachments')
|
||||
.update({
|
||||
extracted_at: new Date().toISOString(),
|
||||
extraction_model: 'failed:exception',
|
||||
})
|
||||
.eq('id', document.id)
|
||||
if (!rawText) {
|
||||
await stamp(supabase, document.id, 'failed:no_raw_text')
|
||||
return
|
||||
}
|
||||
extractedData = data as unknown as Record<string, unknown>
|
||||
model = usedModel ?? 'unknown'
|
||||
} catch (err) {
|
||||
log.warn('extraction threw', {
|
||||
doc: document.id,
|
||||
err: err instanceof Error ? err.message : String(err),
|
||||
})
|
||||
await stamp(supabase, document.id, 'failed:exception')
|
||||
return
|
||||
}
|
||||
|
||||
const { error: updateErr } = await supabase
|
||||
|
||||
@@ -0,0 +1,53 @@
|
||||
import { describe, it, expect, vi, beforeEach } from 'vitest'
|
||||
import { createQueuedMockSupabase } from '@/tests/helpers'
|
||||
|
||||
const { supabase, enqueue, reset, findCalls } = createQueuedMockSupabase()
|
||||
vi.mock('@/lib/auth/api-keys', () => ({
|
||||
createServiceClientNoCookies: () => supabase,
|
||||
}))
|
||||
|
||||
import { mirrorExtractionToDocument } from '../lib/mirror-extraction'
|
||||
|
||||
function lastUpdate() {
|
||||
const updates = findCalls('document_attachments', 'update')
|
||||
return updates[updates.length - 1]?.[0] as { extracted_data: unknown; extraction_model: string; extracted_at: string }
|
||||
}
|
||||
|
||||
beforeEach(() => {
|
||||
vi.clearAllMocks()
|
||||
reset()
|
||||
enqueue({ data: null })
|
||||
})
|
||||
|
||||
describe('mirrorExtractionToDocument', () => {
|
||||
it('writes a successful extraction with the model that answered', async () => {
|
||||
await mirrorExtractionToDocument('doc-1', {
|
||||
data: { supplier: { name: 'x' } } as never,
|
||||
rawText: '{"supplier":{"name":"x"}}',
|
||||
model: 'google/gemma-4-31B-it',
|
||||
})
|
||||
const u = lastUpdate()
|
||||
expect(u.extracted_data).toEqual({ supplier: { name: 'x' } })
|
||||
expect(u.extraction_model).toBe('google/gemma-4-31B-it')
|
||||
expect(u.extracted_at).toBeTruthy()
|
||||
expect(findCalls('document_attachments', 'eq')[0]).toEqual(['id', 'doc-1'])
|
||||
})
|
||||
|
||||
it('writes skipped:<reason> with no data for inbox and AI skip reasons alike', async () => {
|
||||
await mirrorExtractionToDocument('doc-1', { data: {} as never, rawText: null, skipped: 'no_ai_entitlement' })
|
||||
expect(lastUpdate()).toMatchObject({ extracted_data: null, extraction_model: 'skipped:no_ai_entitlement' })
|
||||
await mirrorExtractionToDocument('doc-1', { data: {} as never, rawText: null, skipped: 'ai_no_vision' })
|
||||
expect(lastUpdate()).toMatchObject({ extracted_data: null, extraction_model: 'skipped:ai_no_vision' })
|
||||
})
|
||||
|
||||
it('writes failed:no_raw_text when the call yielded nothing parseable', async () => {
|
||||
await mirrorExtractionToDocument('doc-1', { data: {} as never, rawText: null })
|
||||
expect(lastUpdate()).toMatchObject({ extracted_data: null, extraction_model: 'failed:no_raw_text' })
|
||||
})
|
||||
|
||||
it('never throws: a failed update is logged and swallowed', async () => {
|
||||
reset()
|
||||
enqueue({ error: { message: 'rls says no' } })
|
||||
await expect(mirrorExtractionToDocument('doc-1', { data: {} as never, rawText: null })).resolves.toBeUndefined()
|
||||
})
|
||||
})
|
||||
@@ -24,6 +24,21 @@ vi.mock('@/lib/core/documents/document-service', () => ({
|
||||
uploadDocument: vi.fn().mockResolvedValue({ id: 'doc-1' }),
|
||||
}))
|
||||
|
||||
// The staged (deferred) upload path only exists when AI is configured on this
|
||||
// deployment: an unconfigured one skips synchronously (ai_unconfigured). These
|
||||
// tests simulate a configured deployment; the model call itself is mocked.
|
||||
vi.mock('@/lib/ai', () => ({
|
||||
getAiStatus: () => ({
|
||||
provider: 'bedrock',
|
||||
configured: true,
|
||||
reason: 'ok',
|
||||
capabilities: { pdfNative: true, imageInput: true, toolUse: true, forcedToolChoice: true, strictJsonSchema: false },
|
||||
models: { assistant: 'm', heavy: 'm', extraction: 'm' },
|
||||
pdfMode: 'native',
|
||||
assistantAvailable: true,
|
||||
}),
|
||||
}))
|
||||
|
||||
vi.mock('@/lib/rate-limits/inbox', () => ({
|
||||
checkInboxUploadRateLimit: vi.fn().mockResolvedValue({ ok: true }),
|
||||
}))
|
||||
|
||||
@@ -21,6 +21,21 @@ vi.mock('@/lib/core/documents/document-service', () => ({
|
||||
uploadDocument: vi.fn().mockResolvedValue({ id: 'doc-1' }),
|
||||
}))
|
||||
|
||||
// The staged (deferred) upload path only exists when AI is configured on this
|
||||
// deployment: an unconfigured one skips synchronously (ai_unconfigured). These
|
||||
// tests simulate a configured deployment; the model call itself is mocked.
|
||||
vi.mock('@/lib/ai', () => ({
|
||||
getAiStatus: () => ({
|
||||
provider: 'bedrock',
|
||||
configured: true,
|
||||
reason: 'ok',
|
||||
capabilities: { pdfNative: true, imageInput: true, toolUse: true, forcedToolChoice: true, strictJsonSchema: false },
|
||||
models: { assistant: 'm', heavy: 'm', extraction: 'm' },
|
||||
pdfMode: 'native',
|
||||
assistantAvailable: true,
|
||||
}),
|
||||
}))
|
||||
|
||||
vi.mock('@/lib/rate-limits/inbox', () => ({
|
||||
checkInboxUploadRateLimit: vi.fn().mockResolvedValue({ ok: true }),
|
||||
}))
|
||||
|
||||
@@ -6,6 +6,7 @@ import { uploadDocument } from '@/lib/core/documents/document-service'
|
||||
import { createServiceClient } from '@/lib/supabase/server'
|
||||
import { matchSupplierId } from '@/lib/suppliers/match-supplier'
|
||||
import { extractInvoiceFields, ExtractionSchema, emptyResult } from './lib/extract-invoice-fields'
|
||||
import { mirrorExtractionToDocument } from './lib/mirror-extraction'
|
||||
import {
|
||||
uploadAndExtract,
|
||||
sanitiseFilename,
|
||||
@@ -697,6 +698,9 @@ export const invoiceInboxExtension: Extension = {
|
||||
type: file.type,
|
||||
}, {
|
||||
upload_source: 'file_upload',
|
||||
// This route extracts below and mirrors the outcome onto the
|
||||
// document row; the document-extraction extension must not race it.
|
||||
extractionOwner: 'invoice-inbox',
|
||||
})
|
||||
|
||||
// Same page handling as /upload (issue #553): long PDFs extract
|
||||
@@ -726,16 +730,23 @@ export const invoiceInboxExtension: Extension = {
|
||||
: null
|
||||
const skipExtraction = skipReason !== null
|
||||
|
||||
const { data: extracted } = skipExtraction
|
||||
? { data: emptyResult() }
|
||||
const extraction = skipExtraction
|
||||
? { data: emptyResult(), rawText: null, model: null, skipped: null }
|
||||
: await extractInvoiceFields({
|
||||
buffer: Buffer.from(slicedBuffer ?? buffer),
|
||||
mimeType: file.type,
|
||||
fileName: file.name,
|
||||
})
|
||||
const { data: extracted } = extraction
|
||||
if (!skipExtraction && slicedBuffer != null && pageCount != null) {
|
||||
extracted.pages = { total: pageCount, analyzed: MAX_PAGES_FOR_AUTO_EXTRACT }
|
||||
}
|
||||
await mirrorExtractionToDocument(doc.id, {
|
||||
data: extracted,
|
||||
rawText: extraction.rawText,
|
||||
model: extraction.model ?? null,
|
||||
skipped: skipReason ?? extraction.skipped ?? null,
|
||||
})
|
||||
|
||||
const { error: linkError } = await ctx.supabase
|
||||
.from('invoice_inbox_items')
|
||||
@@ -1088,11 +1099,18 @@ export const invoiceInboxExtension: Extension = {
|
||||
|
||||
try {
|
||||
const buffer = Buffer.from(await blob.arrayBuffer())
|
||||
const { data: extracted } = await extractInvoiceFields({
|
||||
const extraction = await extractInvoiceFields({
|
||||
buffer,
|
||||
mimeType: doc.mime_type,
|
||||
fileName: doc.file_name,
|
||||
})
|
||||
const { data: extracted } = extraction
|
||||
await mirrorExtractionToDocument(item.document_id, {
|
||||
data: extracted,
|
||||
rawText: extraction.rawText,
|
||||
model: extraction.model ?? null,
|
||||
skipped: extraction.skipped ?? null,
|
||||
})
|
||||
|
||||
// Re-running the extraction has to re-run the match too, or the one
|
||||
// affordance the user reaches for when an item failed to auto-link
|
||||
|
||||
@@ -1,10 +1,12 @@
|
||||
// AI-driven invoice/receipt field extraction.
|
||||
//
|
||||
// Sends the uploaded document directly to Claude Sonnet 4.6 via AWS
|
||||
// Bedrock and asks for a structured InvoiceExtractionResult JSON. Sonnet
|
||||
// reads PDFs, images, and scans natively, which the previous regex
|
||||
// extractor couldn't: that's why English receipts (Anthropic, AWS,
|
||||
// Stripe, …) and image-only PDFs came back empty.
|
||||
// Sends the uploaded document to the configured AI backend through the
|
||||
// job-shaped service in lib/ai (Claude on Bedrock or the direct API on
|
||||
// hosted; any OpenAI-compatible endpoint, e.g. a Swedish inference provider,
|
||||
// on a sovereign self-host) and asks for a structured InvoiceExtractionResult
|
||||
// JSON. Vision models read PDFs, images and scans, which the previous regex
|
||||
// extractor couldn't: that's why English receipts (Anthropic, AWS, Stripe, …)
|
||||
// and image-only PDFs came back empty.
|
||||
//
|
||||
// The AI output is validated against a Zod schema; anything that doesn't
|
||||
// parse falls back to an empty result so the inbox row still lands and
|
||||
@@ -13,30 +15,25 @@
|
||||
import { createHash } from 'node:crypto'
|
||||
import { z } from 'zod'
|
||||
import type { InvoiceExtractionResult } from '@/types'
|
||||
import { createAiClient, hasAiCredentials, toProviderModelId } from '@/lib/ai/provider'
|
||||
import { getAiService, readAiConfig, extractJsonObject } from '@/lib/ai'
|
||||
import type { AiDocumentInput, AiImageMediaType, ExtractionSkipReason } from '@/lib/ai'
|
||||
import { createLogger } from '@/lib/logger'
|
||||
|
||||
// Re-exported for callers and tests that import it from here.
|
||||
export { extractJsonObject }
|
||||
|
||||
const log = createLogger('invoice-inbox-extract')
|
||||
|
||||
// Both overridable via env vars so ops can swap models / raise token caps
|
||||
// without a code deploy. The model id is written bare and adapted to whichever
|
||||
// backend is configured (Bedrock in eu-north-1 on hosted, the direct Anthropic
|
||||
// API on self-hosted: see lib/ai/provider.ts). 8192 tokens is enough headroom
|
||||
// for invoices with 20+ line items.
|
||||
const MODEL = toProviderModelId(process.env.BEDROCK_MODEL_ID || 'claude-sonnet-5')
|
||||
const MAX_TOKENS = (() => {
|
||||
const parsed = Number(process.env.BEDROCK_MAX_TOKENS)
|
||||
// Use the env value only if it's a positive number: `||` would also
|
||||
// fall back on a deliberate `0`, masking what is really an invalid
|
||||
// configuration rather than the intent to disable.
|
||||
return Number.isFinite(parsed) && parsed > 0 ? parsed : 8192
|
||||
})()
|
||||
// Output cap: AI_EXTRACTION_MAX_TOKENS (legacy BEDROCK_MAX_TOKENS) or 8192,
|
||||
// enough headroom for invoices with 20+ line items. The model id per tier is
|
||||
// resolved by lib/ai/config.ts (AI_EXTRACTION_MODEL, legacy BEDROCK_MODEL_ID,
|
||||
// AI_MODEL, then the Claude default on the Anthropic family).
|
||||
|
||||
// Bedrock supports these document/image media types directly. HEIC/HEIF
|
||||
// are not on the list, so we skip AI for those: the inbox row still
|
||||
// lands and the user can edit fields manually or replace the file.
|
||||
// text/html (mail-body invoices from the inbound pipeline) is not sent as
|
||||
// a document block: it is converted to plain text via htmlToText() first.
|
||||
// Media types the extraction accepts. HEIC/HEIF are not on the list, so we
|
||||
// skip AI for those: the inbox row still lands and the user can edit fields
|
||||
// manually or replace the file. text/html (mail-body invoices from the
|
||||
// inbound pipeline) is not sent as a document block: it is converted to plain
|
||||
// text via htmlToText() first, which also makes it work on text-only models.
|
||||
const SUPPORTED_MEDIA_TYPES = new Set([
|
||||
'application/pdf',
|
||||
'image/jpeg',
|
||||
@@ -52,10 +49,16 @@ export interface ExtractionInput {
|
||||
fileName: string
|
||||
}
|
||||
|
||||
export type ExtractionSkipped = ExtractionSkipReason | 'unsupported_media'
|
||||
|
||||
export interface ExtractionOutput {
|
||||
data: InvoiceExtractionResult
|
||||
/** The raw JSON string returned by the model, or null on failure. */
|
||||
/** The raw JSON string returned by the model, or null on failure/skip. */
|
||||
rawText: string | null
|
||||
/** Provider-form id of the model that answered, when a call was made. */
|
||||
model?: string | null
|
||||
/** Set when no model call was made at all (and why). */
|
||||
skipped?: ExtractionSkipped | null
|
||||
}
|
||||
|
||||
// Classification fields are nullable AND .catch(null): a hallucinated enum
|
||||
@@ -244,61 +247,6 @@ Rules:
|
||||
- lineItems: include every line. Empty array is fine if the document has no itemised lines.
|
||||
- vatBreakdown: include one entry per distinct VAT rate. Empty array is fine.`
|
||||
|
||||
// Sonnet 5 intermittently wraps its answer in markdown fences (```json ... ```)
|
||||
// or adds prose around it, despite the JSON-only instruction in the system
|
||||
// prompt. Scan for balanced top-level '{'..'}' candidates (string- and
|
||||
// escape-aware, so braces inside JSON string values don't end a candidate
|
||||
// early) and return the first one JSON.parse accepts; prose braces around the
|
||||
// object form unparseable candidates and are skipped. Returns the input
|
||||
// unchanged when no candidate parses, so the existing parse-failure path
|
||||
// handles prose-only refusals. Zod validation downstream still rejects
|
||||
// well-formed-but-wrong JSON.
|
||||
// Bounds for the candidate scan below. Real model output is already capped
|
||||
// by MAX_TOKENS (roughly 33 KB of text at 8192 tokens), so genuine responses
|
||||
// never come near these; they exist so pathological or adversarially
|
||||
// brace-laden text cannot make the scan quadratic (compliance review
|
||||
// A.8.28). Oversized or exhausted inputs fall through to the raw text and
|
||||
// land in the existing empty-result path.
|
||||
const MAX_SCAN_INPUT_LENGTH = 256 * 1024
|
||||
const MAX_CANDIDATE_ATTEMPTS = 50
|
||||
|
||||
export function extractJsonObject(raw: string): string {
|
||||
if (raw.length > MAX_SCAN_INPUT_LENGTH) return raw
|
||||
let attempts = 0
|
||||
let start = raw.indexOf('{')
|
||||
while (start !== -1 && attempts < MAX_CANDIDATE_ATTEMPTS) {
|
||||
attempts++
|
||||
let depth = 0
|
||||
let inString = false
|
||||
let escaped = false
|
||||
for (let i = start; i < raw.length; i++) {
|
||||
const ch = raw[i]
|
||||
if (inString) {
|
||||
if (escaped) escaped = false
|
||||
else if (ch === '\\') escaped = true
|
||||
else if (ch === '"') inString = false
|
||||
} else if (ch === '"') {
|
||||
inString = true
|
||||
} else if (ch === '{') {
|
||||
depth++
|
||||
} else if (ch === '}') {
|
||||
depth--
|
||||
if (depth === 0) {
|
||||
const candidate = raw.slice(start, i + 1)
|
||||
try {
|
||||
JSON.parse(candidate)
|
||||
return candidate
|
||||
} catch {
|
||||
break
|
||||
}
|
||||
}
|
||||
}
|
||||
}
|
||||
start = raw.indexOf('{', start + 1)
|
||||
}
|
||||
return raw
|
||||
}
|
||||
|
||||
export function emptyResult(): InvoiceExtractionResult {
|
||||
return {
|
||||
documentKind: null,
|
||||
@@ -427,44 +375,137 @@ export function htmlToText(html: string): string {
|
||||
.slice(0, MAX_EXTRACTION_TEXT_LENGTH)
|
||||
}
|
||||
|
||||
function buildContent(input: ExtractionInput) {
|
||||
const EXTRACTION_INSTRUCTION = 'Extract the fields per the schema. JSON only.'
|
||||
|
||||
/** Map an upload to the service's document input. HTML becomes plain text. */
|
||||
function toDocumentInput(input: ExtractionInput): AiDocumentInput {
|
||||
if (input.mimeType === 'text/html') {
|
||||
const text = htmlToText(input.buffer.toString('utf8'))
|
||||
return [
|
||||
{
|
||||
type: 'text' as const,
|
||||
text: `The document is an HTML email invoice, converted to plain text:\n\n${text}`,
|
||||
},
|
||||
{ type: 'text' as const, text: 'Extract the fields per the schema. JSON only.' },
|
||||
]
|
||||
return {
|
||||
kind: 'text',
|
||||
text: `The document is an HTML email invoice, converted to plain text:\n\n${text}`,
|
||||
}
|
||||
}
|
||||
const base64 = input.buffer.toString('base64')
|
||||
if (input.mimeType === 'application/pdf') {
|
||||
return [
|
||||
{
|
||||
type: 'document' as const,
|
||||
source: { type: 'base64' as const, media_type: 'application/pdf' as const, data: base64 },
|
||||
},
|
||||
{ type: 'text' as const, text: 'Extract the fields per the schema. JSON only.' },
|
||||
]
|
||||
return { kind: 'pdf', data: input.buffer, fileName: input.fileName }
|
||||
}
|
||||
return [
|
||||
{
|
||||
type: 'image' as const,
|
||||
source: {
|
||||
type: 'base64' as const,
|
||||
media_type: input.mimeType as 'image/jpeg' | 'image/png' | 'image/webp' | 'image/gif',
|
||||
data: base64,
|
||||
return { kind: 'image', data: input.buffer, mediaType: input.mimeType as AiImageMediaType }
|
||||
}
|
||||
|
||||
// Hand-maintained JSON-schema mirror of ExtractionSchema, used ONLY when the
|
||||
// operator opts into strict JSON mode on an OpenAI-compatible endpoint
|
||||
// (AI_STRICT_JSON=true). Deliberately not auto-converted from the Zod schema:
|
||||
// .catch()/.transform() have no JSON-schema equivalent and the conversion
|
||||
// would drift silently. Permissive on purpose (types + nullability only):
|
||||
// Zod stays the validator, this just keeps the model inside the shape.
|
||||
const nullable = (type: string) => ({ type: [type, 'null'] })
|
||||
const EXTRACTION_JSON_SCHEMA: Record<string, unknown> = {
|
||||
type: 'object',
|
||||
additionalProperties: false,
|
||||
properties: {
|
||||
documentKind: nullable('string'),
|
||||
merchantCategory: nullable('string'),
|
||||
legibility: nullable('string'),
|
||||
purchaseTime: nullable('string'),
|
||||
payment: {
|
||||
type: ['object', 'null'],
|
||||
additionalProperties: false,
|
||||
properties: { method: nullable('string'), cardLast4: nullable('string') },
|
||||
required: ['method', 'cardLast4'],
|
||||
},
|
||||
supplier: {
|
||||
type: 'object',
|
||||
additionalProperties: false,
|
||||
properties: {
|
||||
name: nullable('string'),
|
||||
orgNumber: nullable('string'),
|
||||
vatNumber: nullable('string'),
|
||||
address: nullable('string'),
|
||||
bankgiro: nullable('string'),
|
||||
plusgiro: nullable('string'),
|
||||
},
|
||||
required: ['name', 'orgNumber', 'vatNumber', 'address', 'bankgiro', 'plusgiro'],
|
||||
},
|
||||
invoice: {
|
||||
type: 'object',
|
||||
additionalProperties: false,
|
||||
properties: {
|
||||
invoiceNumber: nullable('string'),
|
||||
invoiceDate: nullable('string'),
|
||||
dueDate: nullable('string'),
|
||||
paymentReference: nullable('string'),
|
||||
currency: { type: 'string' },
|
||||
servicePeriodStart: nullable('string'),
|
||||
servicePeriodEnd: nullable('string'),
|
||||
},
|
||||
required: [
|
||||
'invoiceNumber',
|
||||
'invoiceDate',
|
||||
'dueDate',
|
||||
'paymentReference',
|
||||
'currency',
|
||||
'servicePeriodStart',
|
||||
'servicePeriodEnd',
|
||||
],
|
||||
},
|
||||
lineItems: {
|
||||
type: 'array',
|
||||
items: {
|
||||
type: 'object',
|
||||
additionalProperties: false,
|
||||
properties: {
|
||||
description: { type: 'string' },
|
||||
quantity: { type: 'number' },
|
||||
unitPrice: nullable('number'),
|
||||
lineTotal: { type: 'number' },
|
||||
vatRate: nullable('number'),
|
||||
accountSuggestion: { type: 'null' },
|
||||
},
|
||||
required: ['description', 'quantity', 'unitPrice', 'lineTotal', 'vatRate', 'accountSuggestion'],
|
||||
},
|
||||
},
|
||||
{ type: 'text' as const, text: 'Extract the fields per the schema. JSON only.' },
|
||||
]
|
||||
totals: {
|
||||
type: 'object',
|
||||
additionalProperties: false,
|
||||
properties: {
|
||||
subtotal: nullable('number'),
|
||||
vatAmount: nullable('number'),
|
||||
total: nullable('number'),
|
||||
roundingAmount: nullable('number'),
|
||||
},
|
||||
required: ['subtotal', 'vatAmount', 'total', 'roundingAmount'],
|
||||
},
|
||||
vatBreakdown: {
|
||||
type: 'array',
|
||||
items: {
|
||||
type: 'object',
|
||||
additionalProperties: false,
|
||||
properties: { rate: { type: 'number' }, base: { type: 'number' }, amount: { type: 'number' } },
|
||||
required: ['rate', 'base', 'amount'],
|
||||
},
|
||||
},
|
||||
},
|
||||
required: [
|
||||
'documentKind',
|
||||
'merchantCategory',
|
||||
'legibility',
|
||||
'purchaseTime',
|
||||
'payment',
|
||||
'supplier',
|
||||
'invoice',
|
||||
'lineItems',
|
||||
'totals',
|
||||
'vatBreakdown',
|
||||
],
|
||||
}
|
||||
|
||||
/**
|
||||
* Extract invoice fields by sending the document directly to Claude
|
||||
* Sonnet 4.6 via AWS Bedrock. Never throws on extraction failure:
|
||||
* always returns an InvoiceExtractionResult. Empty fields are null.
|
||||
* Extract invoice fields by sending the document to the configured AI
|
||||
* backend. Never throws on extraction failure: always returns an
|
||||
* InvoiceExtractionResult. Empty fields are null. `skipped` is set when no
|
||||
* model call was made (unsupported file type, AI not configured, no vision
|
||||
* on this backend, PDF rasterizer missing), so callers can tell "nothing to
|
||||
* read" from "read and found nothing".
|
||||
*/
|
||||
export async function extractInvoiceFields(
|
||||
rawInput: ExtractionInput
|
||||
@@ -474,63 +515,48 @@ export async function extractInvoiceFields(
|
||||
const input = await normalizeImageForExtraction(rawInput)
|
||||
|
||||
if (!SUPPORTED_MEDIA_TYPES.has(input.mimeType)) {
|
||||
return { data: emptyResult(), rawText: null }
|
||||
return { data: emptyResult(), rawText: null, skipped: 'unsupported_media' }
|
||||
}
|
||||
|
||||
if (!hasAiCredentials()) {
|
||||
log.warn('AI credentials missing: returning empty extraction', {
|
||||
file_name_hash: createHash('sha256').update(input.fileName).digest('hex').slice(0, 12),
|
||||
})
|
||||
return { data: emptyResult(), rawText: null }
|
||||
}
|
||||
|
||||
const client = createAiClient()
|
||||
const fileNameHash = createHash('sha256').update(input.fileName).digest('hex').slice(0, 12)
|
||||
const service = getAiService()
|
||||
|
||||
let rawText: string | null = null
|
||||
let model: string | null = null
|
||||
try {
|
||||
// SYSTEM_PROMPT is byte-stable per deploy and ~3.5 KB: marking it as
|
||||
// ephemeral lets Bedrock reuse the prompt-cache on rapid sequential
|
||||
// extractions (e.g. a user uploading a stack of receipts within minutes).
|
||||
// Bedrock supports `{ type: 'ephemeral' }` with the default short TTL;
|
||||
// the 1h TTL from the agent-native API plan (item 10) requires the direct
|
||||
// Anthropic API rather than Bedrock and is out of scope here.
|
||||
const resp = await client.messages.create({
|
||||
model: MODEL,
|
||||
max_tokens: MAX_TOKENS,
|
||||
system: [{ type: 'text', text: SYSTEM_PROMPT, cache_control: { type: 'ephemeral' } }],
|
||||
messages: [{ role: 'user', content: buildContent(input) }],
|
||||
const result = await service.extractFromDocument({
|
||||
document: toDocumentInput(input),
|
||||
system: SYSTEM_PROMPT,
|
||||
instruction: EXTRACTION_INSTRUCTION,
|
||||
maxTokens: readAiConfig().extractionMaxTokens,
|
||||
jsonSchema: EXTRACTION_JSON_SCHEMA,
|
||||
})
|
||||
|
||||
rawText = resp.content
|
||||
.flatMap((b) => (b.type === 'text' ? [b.text] : []))
|
||||
.join('')
|
||||
.trim()
|
||||
if (!result.ok) {
|
||||
// Not a failure: the deployment cannot read this document at all.
|
||||
// `ai_unconfigured` is the self-host "no key yet" case the 30 s
|
||||
// upload hang used to hide; the others are honest capability gaps.
|
||||
log.warn('AI extraction skipped', { file_name_hash: fileNameHash, reason: result.skipped })
|
||||
return { data: emptyResult(), rawText: null, skipped: result.skipped }
|
||||
}
|
||||
rawText = result.text
|
||||
model = result.model
|
||||
|
||||
// Observability for the prompt-cache hit ratio. The agent-native plan
|
||||
// targets cache_read_input_tokens / total_input_tokens ≥ 0.85 in steady
|
||||
// state; logging here makes that measurable without a separate dashboard.
|
||||
const usage = resp.usage as
|
||||
| {
|
||||
input_tokens?: number
|
||||
output_tokens?: number
|
||||
cache_creation_input_tokens?: number
|
||||
cache_read_input_tokens?: number
|
||||
}
|
||||
| undefined
|
||||
if (usage) {
|
||||
// Raw fileName can constitute personal data (e.g. "faktura_Sven_Andersson.pdf").
|
||||
// Log a short hash so the operator can correlate without exposing PII
|
||||
// to the log destination (GDPR Art. 5(1)(f)).
|
||||
const fileNameHash = createHash('sha256').update(input.fileName).digest('hex').slice(0, 12)
|
||||
log.info('ai_extraction_usage', {
|
||||
file_name_hash: fileNameHash,
|
||||
mime_type: input.mimeType,
|
||||
input_tokens: usage.input_tokens ?? null,
|
||||
output_tokens: usage.output_tokens ?? null,
|
||||
cache_creation_input_tokens: usage.cache_creation_input_tokens ?? null,
|
||||
cache_read_input_tokens: usage.cache_read_input_tokens ?? null,
|
||||
})
|
||||
}
|
||||
// Raw fileName can constitute personal data (e.g. "faktura_Sven_Andersson.pdf").
|
||||
// Log a short hash so the operator can correlate without exposing PII
|
||||
// to the log destination (GDPR Art. 5(1)(f)).
|
||||
log.info('ai_extraction_usage', {
|
||||
file_name_hash: fileNameHash,
|
||||
mime_type: input.mimeType,
|
||||
model,
|
||||
input_tokens: result.usage.inputTokens,
|
||||
output_tokens: result.usage.outputTokens,
|
||||
cache_creation_input_tokens: result.usage.cacheCreationInputTokens,
|
||||
cache_read_input_tokens: result.usage.cacheReadInputTokens,
|
||||
...(result.pagesRasterized ? { pages_rasterized: result.pagesRasterized } : {}),
|
||||
})
|
||||
|
||||
const parsed = JSON.parse(extractJsonObject(rawText))
|
||||
const validated = ExtractionSchema.parse(parsed)
|
||||
@@ -540,14 +566,15 @@ export async function extractInvoiceFields(
|
||||
// .transform, so no post-validation coercion is needed.
|
||||
data: { ...validated, confidence: 1 },
|
||||
rawText,
|
||||
model,
|
||||
}
|
||||
} catch (err) {
|
||||
log.warn('AI extraction failed', {
|
||||
file_name_hash: createHash('sha256').update(input.fileName).digest('hex').slice(0, 12),
|
||||
file_name_hash: fileNameHash,
|
||||
mimeType: input.mimeType,
|
||||
error: err instanceof Error ? err.message : String(err),
|
||||
hasRawText: rawText != null,
|
||||
})
|
||||
return { data: emptyResult(), rawText }
|
||||
return { data: emptyResult(), rawText, model }
|
||||
}
|
||||
}
|
||||
|
||||
@@ -0,0 +1,64 @@
|
||||
import { createServiceClientNoCookies } from '@/lib/auth/api-keys'
|
||||
import { createLogger } from '@/lib/logger'
|
||||
import type { InvoiceExtractionResult } from '@/types'
|
||||
|
||||
const log = createLogger('invoice-inbox-mirror')
|
||||
|
||||
export interface ExtractionOutcome {
|
||||
/** The parsed result (may be the empty skeleton). Ignored unless `rawText` is set and nothing was skipped. */
|
||||
data: InvoiceExtractionResult | null
|
||||
/** Raw model output; null when the call failed or was skipped. */
|
||||
rawText: string | null
|
||||
/** Provider-form model id that answered, when a call was made. */
|
||||
model?: string | null
|
||||
/** Why no model call was made: inbox skip reasons and lib/ai skip reasons alike. */
|
||||
skipped?: string | null
|
||||
}
|
||||
|
||||
/**
|
||||
* Mirror an inbox extraction outcome onto document_attachments
|
||||
* (extracted_data / extracted_at / extraction_model).
|
||||
*
|
||||
* The invoice inbox owns extraction for the documents it ingests, and the
|
||||
* document-extraction extension yields to it (see extractionOwner on the
|
||||
* document.uploaded event). Everything that read the document row before
|
||||
* (the extraction-status poll behind the upload UI, the agent intents'
|
||||
* "what do we already know" reads) keeps working because the inbox writes
|
||||
* the same columns the extension would have, with the one model call it
|
||||
* actually made.
|
||||
*
|
||||
* Service-role client, same as the extension: this runs from routes, the
|
||||
* deferred worker and the MCP server alike, and the write is a system
|
||||
* side-effect rather than a user action. Never throws: a failed mirror
|
||||
* leaves the document row unstamped, which the UI already tolerates.
|
||||
*/
|
||||
export async function mirrorExtractionToDocument(
|
||||
documentId: string,
|
||||
outcome: ExtractionOutcome
|
||||
): Promise<void> {
|
||||
try {
|
||||
const succeeded = !outcome.skipped && outcome.rawText != null && outcome.data != null
|
||||
const extractionModel = outcome.skipped
|
||||
? `skipped:${outcome.skipped}`
|
||||
: succeeded
|
||||
? outcome.model || 'invoice-inbox'
|
||||
: 'failed:no_raw_text'
|
||||
const supabase = createServiceClientNoCookies()
|
||||
const { error } = await supabase
|
||||
.from('document_attachments')
|
||||
.update({
|
||||
extracted_data: succeeded ? (outcome.data as unknown as Record<string, unknown>) : null,
|
||||
extracted_at: new Date().toISOString(),
|
||||
extraction_model: extractionModel,
|
||||
})
|
||||
.eq('id', documentId)
|
||||
if (error) {
|
||||
log.warn('mirror failed', { doc: documentId, extractionModel, err: error.message })
|
||||
}
|
||||
} catch (err) {
|
||||
log.warn('mirror threw', {
|
||||
doc: documentId,
|
||||
err: err instanceof Error ? err.message : String(err),
|
||||
})
|
||||
}
|
||||
}
|
||||
@@ -1,6 +1,8 @@
|
||||
import { after } from 'next/server'
|
||||
import { uploadDocument } from '@/lib/core/documents/document-service'
|
||||
import { extractInvoiceFields, emptyResult } from './extract-invoice-fields'
|
||||
import { mirrorExtractionToDocument } from './mirror-extraction'
|
||||
import { getAiStatus } from '@/lib/ai'
|
||||
import { hasCapability } from '@/lib/entitlements/has-capability'
|
||||
import { CAPABILITY } from '@/lib/entitlements/keys'
|
||||
import { appendProcessingHistory } from '@/lib/processing-history/append'
|
||||
@@ -10,6 +12,16 @@ import type { InvoiceExtractionResult } from '@/types'
|
||||
import { PDFDocument } from 'pdf-lib'
|
||||
import path from 'node:path'
|
||||
|
||||
// Verdicts decidable before touching the file. `ai_unconfigured` is the
|
||||
// deployment-level "no AI backend" state (self-host without a key), distinct
|
||||
// from the per-company paywall (`no_ai_entitlement`).
|
||||
type SyncSkipReason =
|
||||
| 'no_ai_entitlement'
|
||||
| 'client_opt_out'
|
||||
| 'sandbox'
|
||||
| 'ai_unconfigured'
|
||||
| null
|
||||
|
||||
/**
|
||||
* Defensive filename sanitisation for content arriving from .eml inner
|
||||
* attachments and rejected-attachment metadata. The document-service already
|
||||
@@ -231,6 +243,9 @@ export async function uploadAndExtract(
|
||||
// two inboxes, re-hunted by a sweep, or uploaded twice must not become a
|
||||
// second archived document.
|
||||
dedupeByContent: true,
|
||||
// This function extracts (sync or deferred) and mirrors the outcome onto
|
||||
// the document row; the document-extraction extension must not race it.
|
||||
extractionOwner: 'invoice-inbox',
|
||||
})
|
||||
|
||||
if (doc.deduplicated) {
|
||||
@@ -342,14 +357,20 @@ export async function uploadAndExtract(
|
||||
// decidable without touching the PDF; too_many_pages is not (it only fires
|
||||
// when the slice fallback fails) and is resolved below, on whichever path
|
||||
// (sync or deferred) actually attempts the slice.
|
||||
const syncSkipReason: 'no_ai_entitlement' | 'client_opt_out' | 'sandbox' | null =
|
||||
// `ai_unconfigured` (no AI credentials/model on this deployment, the
|
||||
// self-host "key not set yet" state) is decided up front as well: a call
|
||||
// that can never succeed must not take the deferred path and leave the row
|
||||
// in 'processing' until the sweep.
|
||||
const syncSkipReason: SyncSkipReason =
|
||||
!hasAiEntitlement
|
||||
? 'no_ai_entitlement'
|
||||
: sandbox
|
||||
? 'sandbox'
|
||||
: opts.skipExtraction
|
||||
? 'client_opt_out'
|
||||
: null
|
||||
: !getAiStatus().configured
|
||||
? 'ai_unconfigured'
|
||||
: null
|
||||
|
||||
if (opts.deferExtraction && syncSkipReason === null) {
|
||||
// Staged path: extraction WILL call Bedrock, so create the row now and
|
||||
@@ -424,7 +445,7 @@ export async function uploadAndExtract(
|
||||
// page-count. Opt-out outranks the page gate (an opted-out caller never
|
||||
// extracts regardless of length), and too_many_pages only fires when the
|
||||
// slice fallback also failed (encrypted/malformed PDF).
|
||||
const skipReason: 'no_ai_entitlement' | 'too_many_pages' | 'client_opt_out' | 'sandbox' | null =
|
||||
const skipReason: SyncSkipReason | 'too_many_pages' =
|
||||
syncSkipReason ??
|
||||
(gatedByPageCount && slicedBuffer == null ? 'too_many_pages' : null)
|
||||
const skipExtraction = skipReason !== null
|
||||
@@ -434,13 +455,14 @@ export async function uploadAndExtract(
|
||||
// fields via /items/:id/extracted-data before converting to a supplier
|
||||
// invoice. extracted_data is never null in the DB; an empty skeleton
|
||||
// keeps downstream readers (UI, MCP) happy.
|
||||
const { data: extracted, rawText } = skipExtraction
|
||||
? { data: emptyResult(), rawText: null }
|
||||
const extraction = skipExtraction
|
||||
? { data: emptyResult(), rawText: null, model: null, skipped: null }
|
||||
: await extractInvoiceFields({
|
||||
buffer: Buffer.from(slicedBuffer ?? file.buffer),
|
||||
mimeType: file.type,
|
||||
fileName: file.name,
|
||||
})
|
||||
const { data: extracted, rawText } = extraction
|
||||
if (!skipExtraction && slicedBuffer != null && pageCount != null) {
|
||||
extracted.pages = { total: pageCount, analyzed: MAX_PAGES_FOR_AUTO_EXTRACT }
|
||||
}
|
||||
@@ -484,6 +506,15 @@ export async function uploadAndExtract(
|
||||
|
||||
if (inboxError) throw new Error(`Failed to create inbox item: ${inboxError.message}`)
|
||||
|
||||
// The extension yielded to us (extractionOwner); put the outcome where it
|
||||
// would have written it, so the document row tells the same story.
|
||||
await mirrorExtractionToDocument(doc.id, {
|
||||
data: extracted,
|
||||
rawText,
|
||||
model: extraction.model ?? null,
|
||||
skipped: skipReason ?? extraction.skipped ?? null,
|
||||
})
|
||||
|
||||
try {
|
||||
await appendProcessingHistory({
|
||||
companyId,
|
||||
@@ -552,8 +583,9 @@ function scheduleDeferredExtraction(job: DeferredExtractionJob): void {
|
||||
// an unsliceable long PDF becomes the too_many_pages skip.
|
||||
let extracted: InvoiceExtractionResult = emptyResult()
|
||||
let rawText: string | null = null
|
||||
let model: string | null = null
|
||||
let extractionSkipped = false
|
||||
let skipReason: 'too_many_pages' | null = null
|
||||
let skipReason: string | null = null
|
||||
try {
|
||||
const slicedBuffer = job.gatedByPageCount
|
||||
? await slicePdfForExtraction(job.file.buffer, MAX_PAGES_FOR_AUTO_EXTRACT)
|
||||
@@ -569,6 +601,14 @@ function scheduleDeferredExtraction(job: DeferredExtractionJob): void {
|
||||
})
|
||||
extracted = result.data
|
||||
rawText = result.rawText
|
||||
model = result.model ?? null
|
||||
if (result.skipped) {
|
||||
// No model call could be made (AI unconfigured, no vision on this
|
||||
// backend, rasterizer missing). Same UI affordance as the other
|
||||
// skips: the row lands with the empty skeleton and the hint.
|
||||
extractionSkipped = true
|
||||
skipReason = result.skipped
|
||||
}
|
||||
if (slicedBuffer != null && job.pageCount != null) {
|
||||
extracted.pages = { total: job.pageCount, analyzed: MAX_PAGES_FOR_AUTO_EXTRACT }
|
||||
}
|
||||
@@ -608,6 +648,13 @@ function scheduleDeferredExtraction(job: DeferredExtractionJob): void {
|
||||
}
|
||||
if (!Array.isArray(claimed) || claimed.length === 0) return
|
||||
|
||||
await mirrorExtractionToDocument(job.documentId, {
|
||||
data: extracted,
|
||||
rawText,
|
||||
model,
|
||||
skipped: skipReason,
|
||||
})
|
||||
|
||||
try {
|
||||
await appendProcessingHistory({
|
||||
companyId: job.companyId,
|
||||
@@ -646,6 +693,7 @@ function scheduleDeferredExtraction(job: DeferredExtractionJob): void {
|
||||
})
|
||||
.eq('id', job.itemId)
|
||||
.eq('status', 'processing')
|
||||
await mirrorExtractionToDocument(job.documentId, { data: null, rawText: null })
|
||||
} catch {
|
||||
// The sweep cron is the recovery of last resort.
|
||||
}
|
||||
|
||||
@@ -129,6 +129,10 @@ describe('MCP model-free document upload tools', () => {
|
||||
uploadId,
|
||||
'invoice.pdf',
|
||||
'application/pdf',
|
||||
undefined,
|
||||
// The inbox item created right after owns extraction; the
|
||||
// document-extraction extension must yield on the uploaded event.
|
||||
{ extractionOwner: 'invoice-inbox' },
|
||||
)
|
||||
expect(inboxInsert.insert).toHaveBeenCalledWith(
|
||||
expect.objectContaining({ id: uploadId, document_id: uploadId }),
|
||||
|
||||
@@ -210,6 +210,7 @@ import {
|
||||
MAX_DOCUMENT_SIZE,
|
||||
} from '@/lib/core/documents/document-service'
|
||||
import { extractInvoiceFields, ExtractionSchema as InvoiceExtractionSchema, AgentExtractionSchema } from '@/extensions/general/invoice-inbox/lib/extract-invoice-fields'
|
||||
import { mirrorExtractionToDocument } from '@/extensions/general/invoice-inbox/lib/mirror-extraction'
|
||||
// Skatteverket filing tools (PR5). Cross-extension lib import, same sanctioned
|
||||
// pattern as invoice-inbox above: the CI guard only checks lib/, app/api/,
|
||||
// components/. The two submit tools stage ops whose commit dispatches back into
|
||||
@@ -497,7 +498,17 @@ async function createDocumentInboxItem(
|
||||
if (existing) return existing
|
||||
}
|
||||
|
||||
const { data: extracted } = await extractInvoiceFields({ buffer, mimeType, fileName })
|
||||
const extraction = await extractInvoiceFields({ buffer, mimeType, fileName })
|
||||
const { data: extracted } = extraction
|
||||
// uploadDocument()/completePendingDocumentUpload() were told this inbox
|
||||
// item owns extraction, so the document-extraction extension yielded;
|
||||
// mirror the outcome onto the document row in its place.
|
||||
await mirrorExtractionToDocument(documentId, {
|
||||
data: extracted,
|
||||
rawText: extraction.rawText,
|
||||
model: extraction.model ?? null,
|
||||
skipped: extraction.skipped ?? null,
|
||||
})
|
||||
|
||||
const matchedSupplierId = await matchSupplierId(supabase, companyId, extracted.supplier)
|
||||
|
||||
@@ -9777,6 +9788,8 @@ export const tools: McpTool[] = [
|
||||
uploadId,
|
||||
fileName,
|
||||
mimeType,
|
||||
undefined,
|
||||
{ extractionOwner: 'invoice-inbox' },
|
||||
)
|
||||
return createDocumentInboxItem(
|
||||
supabase,
|
||||
@@ -9837,7 +9850,7 @@ export const tools: McpTool[] = [
|
||||
name: fileName,
|
||||
buffer: buffer.buffer.slice(buffer.byteOffset, buffer.byteOffset + buffer.byteLength),
|
||||
type: mimeType,
|
||||
}, { upload_source: 'api' })
|
||||
}, { upload_source: 'api', extractionOwner: 'invoice-inbox' })
|
||||
return createDocumentInboxItem(
|
||||
supabase,
|
||||
companyId,
|
||||
|
||||
@@ -0,0 +1,188 @@
|
||||
import { describe, it, expect, vi, beforeEach, afterEach } from 'vitest'
|
||||
|
||||
// Drive the service against a fake Bedrock client: the point of these tests
|
||||
// is the REQUEST SHAPE. Hosted stays byte-identical only if the service sends
|
||||
// exactly the literals the call sites sent before the abstraction existed.
|
||||
const mockCreate = vi.fn()
|
||||
vi.mock('@anthropic-ai/bedrock-sdk', () => {
|
||||
class FakeBedrock {
|
||||
messages = { create: mockCreate }
|
||||
}
|
||||
return { default: FakeBedrock }
|
||||
})
|
||||
|
||||
import { readAiConfig } from '../config'
|
||||
import { createAnthropicFamilyService, buildAnthropicDocumentContent } from '../services/anthropic-family'
|
||||
|
||||
const ENV = ['AWS_ACCESS_KEY_ID', 'AWS_SECRET_ACCESS_KEY', 'ANTHROPIC_API_KEY', 'AI_PROVIDER', 'AI_BASE_URL', 'AI_API_KEY', 'BEDROCK_MODEL_ID', 'AI_EXTRACTION_MODEL', 'AI_MODEL'] as const
|
||||
let saved: Record<string, string | undefined> = {}
|
||||
beforeEach(() => {
|
||||
vi.clearAllMocks()
|
||||
saved = {}
|
||||
for (const k of ENV) {
|
||||
saved[k] = process.env[k]
|
||||
delete process.env[k]
|
||||
}
|
||||
process.env.AWS_ACCESS_KEY_ID = 'AKIAEXAMPLE'
|
||||
process.env.AWS_SECRET_ACCESS_KEY = 'secret'
|
||||
mockCreate.mockResolvedValue({
|
||||
content: [{ type: 'text', text: '{"ok":true}' }],
|
||||
usage: { input_tokens: 10, output_tokens: 5, cache_read_input_tokens: 7, cache_creation_input_tokens: 0 },
|
||||
})
|
||||
})
|
||||
afterEach(() => {
|
||||
for (const k of ENV) {
|
||||
if (saved[k] === undefined) delete process.env[k]
|
||||
else process.env[k] = saved[k]
|
||||
}
|
||||
})
|
||||
|
||||
const SYSTEM = 'You extract fields.'
|
||||
const INSTRUCTION = 'Extract the fields per the schema. JSON only.'
|
||||
|
||||
describe('extractFromDocument request shape (hosted regression net)', () => {
|
||||
it('sends a PDF exactly as the inbox extractor did: cached system block, document part, instruction', async () => {
|
||||
const svc = createAnthropicFamilyService(readAiConfig())
|
||||
const pdf = Buffer.from('%PDF-1.4')
|
||||
const result = await svc.extractFromDocument({
|
||||
document: { kind: 'pdf', data: pdf, fileName: 'faktura.pdf' },
|
||||
system: SYSTEM,
|
||||
instruction: INSTRUCTION,
|
||||
maxTokens: 8192,
|
||||
})
|
||||
expect(result.ok).toBe(true)
|
||||
expect(mockCreate).toHaveBeenCalledTimes(1)
|
||||
expect(mockCreate.mock.calls[0][0]).toEqual({
|
||||
model: 'eu.anthropic.claude-sonnet-5',
|
||||
max_tokens: 8192,
|
||||
system: [{ type: 'text', text: SYSTEM, cache_control: { type: 'ephemeral' } }],
|
||||
messages: [
|
||||
{
|
||||
role: 'user',
|
||||
content: [
|
||||
{
|
||||
type: 'document',
|
||||
source: { type: 'base64', media_type: 'application/pdf', data: pdf.toString('base64') },
|
||||
},
|
||||
{ type: 'text', text: INSTRUCTION },
|
||||
],
|
||||
},
|
||||
],
|
||||
})
|
||||
})
|
||||
|
||||
it('sends an image as a base64 image block with its media type', async () => {
|
||||
const svc = createAnthropicFamilyService(readAiConfig())
|
||||
const jpeg = Buffer.from('JPEG')
|
||||
await svc.extractFromDocument({
|
||||
document: { kind: 'image', data: jpeg, mediaType: 'image/jpeg' },
|
||||
system: SYSTEM,
|
||||
instruction: INSTRUCTION,
|
||||
maxTokens: 8192,
|
||||
})
|
||||
expect(mockCreate.mock.calls[0][0].messages[0].content).toEqual([
|
||||
{ type: 'image', source: { type: 'base64', media_type: 'image/jpeg', data: jpeg.toString('base64') } },
|
||||
{ type: 'text', text: INSTRUCTION },
|
||||
])
|
||||
})
|
||||
|
||||
it('sends converted HTML as two text blocks', async () => {
|
||||
const svc = createAnthropicFamilyService(readAiConfig())
|
||||
await svc.extractFromDocument({
|
||||
document: { kind: 'text', text: 'The document is an HTML email invoice, converted to plain text:\n\nTotal 100' },
|
||||
system: SYSTEM,
|
||||
instruction: INSTRUCTION,
|
||||
maxTokens: 8192,
|
||||
})
|
||||
expect(mockCreate.mock.calls[0][0].messages[0].content).toEqual([
|
||||
{ type: 'text', text: 'The document is an HTML email invoice, converted to plain text:\n\nTotal 100' },
|
||||
{ type: 'text', text: INSTRUCTION },
|
||||
])
|
||||
})
|
||||
|
||||
it('returns the text, model and usage', async () => {
|
||||
const svc = createAnthropicFamilyService(readAiConfig())
|
||||
const result = await svc.extractFromDocument({
|
||||
document: { kind: 'text', text: 'x' },
|
||||
system: SYSTEM,
|
||||
instruction: INSTRUCTION,
|
||||
maxTokens: 100,
|
||||
})
|
||||
expect(result).toEqual({
|
||||
ok: true,
|
||||
text: '{"ok":true}',
|
||||
model: 'eu.anthropic.claude-sonnet-5',
|
||||
usage: { inputTokens: 10, outputTokens: 5, cacheCreationInputTokens: 0, cacheReadInputTokens: 7 },
|
||||
})
|
||||
})
|
||||
|
||||
it('honours the extraction tier override (legacy BEDROCK_MODEL_ID)', async () => {
|
||||
process.env.BEDROCK_MODEL_ID = 'claude-sonnet-4-6'
|
||||
const svc = createAnthropicFamilyService(readAiConfig())
|
||||
await svc.extractFromDocument({ document: { kind: 'text', text: 'x' }, system: SYSTEM, instruction: INSTRUCTION, maxTokens: 1 })
|
||||
expect(mockCreate.mock.calls[0][0].model).toBe('eu.anthropic.claude-sonnet-4-6')
|
||||
})
|
||||
|
||||
it('skips without a call when the deployment has no credentials', async () => {
|
||||
delete process.env.AWS_ACCESS_KEY_ID
|
||||
delete process.env.AWS_SECRET_ACCESS_KEY
|
||||
const svc = createAnthropicFamilyService(readAiConfig())
|
||||
const result = await svc.extractFromDocument({ document: { kind: 'text', text: 'x' }, system: SYSTEM, instruction: INSTRUCTION, maxTokens: 1 })
|
||||
expect(result).toEqual({ ok: false, skipped: 'ai_unconfigured' })
|
||||
expect(mockCreate).not.toHaveBeenCalled()
|
||||
})
|
||||
})
|
||||
|
||||
describe('generateText / generateStructured', () => {
|
||||
it('generateText sends a plain user message with an optional system string', async () => {
|
||||
const svc = createAnthropicFamilyService(readAiConfig())
|
||||
const result = await svc.generateText({ tier: 'assistant', system: 'S', prompt: 'Hej', maxTokens: 50 })
|
||||
expect(mockCreate.mock.calls[0][0]).toEqual({
|
||||
model: 'eu.anthropic.claude-sonnet-5',
|
||||
max_tokens: 50,
|
||||
system: 'S',
|
||||
messages: [{ role: 'user', content: 'Hej' }],
|
||||
})
|
||||
expect(result.text).toBe('{"ok":true}')
|
||||
})
|
||||
|
||||
it('generateStructured forces one named tool and returns its input', async () => {
|
||||
mockCreate.mockResolvedValueOnce({
|
||||
content: [{ type: 'tool_use', id: 't1', name: 'verdict', input: { paired: true } }],
|
||||
usage: { input_tokens: 1, output_tokens: 1 },
|
||||
})
|
||||
const svc = createAnthropicFamilyService(readAiConfig())
|
||||
const schema = { type: 'object', properties: { paired: { type: 'boolean' } }, required: ['paired'] }
|
||||
const result = await svc.generateStructured({
|
||||
tier: 'heavy',
|
||||
prompt: 'Decide',
|
||||
maxTokens: 200,
|
||||
schema: { name: 'verdict', description: 'd', jsonSchema: schema },
|
||||
})
|
||||
expect(mockCreate.mock.calls[0][0]).toEqual({
|
||||
model: 'eu.anthropic.claude-sonnet-5',
|
||||
max_tokens: 200,
|
||||
tools: [{ name: 'verdict', description: 'd', input_schema: schema }],
|
||||
tool_choice: { type: 'tool', name: 'verdict' },
|
||||
messages: [{ role: 'user', content: 'Decide' }],
|
||||
})
|
||||
expect(result.value).toEqual({ paired: true })
|
||||
})
|
||||
|
||||
it('generateStructured throws when the model ignored the forced tool', async () => {
|
||||
mockCreate.mockResolvedValueOnce({ content: [{ type: 'text', text: 'nope' }], usage: {} })
|
||||
const svc = createAnthropicFamilyService(readAiConfig())
|
||||
await expect(
|
||||
svc.generateStructured({ tier: 'assistant', prompt: 'x', maxTokens: 10, schema: { name: 'v', jsonSchema: {} } })
|
||||
).rejects.toThrow(/forced tool/)
|
||||
})
|
||||
})
|
||||
|
||||
describe('buildAnthropicDocumentContent', () => {
|
||||
it('is a pure function of the document and instruction', () => {
|
||||
expect(buildAnthropicDocumentContent({ kind: 'text', text: 'a' }, 'b')).toEqual([
|
||||
{ type: 'text', text: 'a' },
|
||||
{ type: 'text', text: 'b' },
|
||||
])
|
||||
})
|
||||
})
|
||||
@@ -0,0 +1,226 @@
|
||||
import { describe, it, expect, beforeEach, afterEach } from 'vitest'
|
||||
import { getAiStatus, readAiConfig, resolveTierModel } from '../config'
|
||||
import { createAiClient, AiProviderUnsupportedError, hasAiCredentials, resolveAiProvider, toProviderModelId } from '../provider'
|
||||
|
||||
const KEYS = [
|
||||
'AI_PROVIDER',
|
||||
'AI_BASE_URL',
|
||||
'AI_API_KEY',
|
||||
'AI_MODEL',
|
||||
'AI_ASSISTANT_MODEL',
|
||||
'AI_HEAVY_MODEL',
|
||||
'AI_EXTRACTION_MODEL',
|
||||
'AI_EXTRACTION_MAX_TOKENS',
|
||||
'AI_VISION',
|
||||
'AI_STRICT_JSON',
|
||||
'AI_PDF_MODE',
|
||||
'AI_PDF_MAX_PAGES',
|
||||
'ANTHROPIC_API_KEY',
|
||||
'AWS_ACCESS_KEY_ID',
|
||||
'AWS_SECRET_ACCESS_KEY',
|
||||
'AWS_REGION',
|
||||
'BEDROCK_MODEL_ID',
|
||||
'BEDROCK_SONNET_MODEL_ID',
|
||||
'BEDROCK_OPUS_MODEL_ID',
|
||||
'BEDROCK_MAX_TOKENS',
|
||||
] as const
|
||||
|
||||
let saved: Partial<Record<(typeof KEYS)[number], string | undefined>> = {}
|
||||
beforeEach(() => {
|
||||
saved = {}
|
||||
for (const k of KEYS) {
|
||||
saved[k] = process.env[k]
|
||||
delete process.env[k]
|
||||
}
|
||||
})
|
||||
afterEach(() => {
|
||||
for (const k of KEYS) {
|
||||
if (saved[k] === undefined) delete process.env[k]
|
||||
else process.env[k] = saved[k]
|
||||
}
|
||||
})
|
||||
|
||||
function hosted() {
|
||||
process.env.AWS_ACCESS_KEY_ID = 'AKIAEXAMPLE'
|
||||
process.env.AWS_SECRET_ACCESS_KEY = 'secret'
|
||||
}
|
||||
function byo(model = 'google/gemma-4-31B-it') {
|
||||
process.env.AI_BASE_URL = 'https://api.berget.ai/v1'
|
||||
process.env.AI_API_KEY = 'sk-berget-example'
|
||||
if (model) process.env.AI_MODEL = model
|
||||
}
|
||||
|
||||
describe('provider resolution with an OpenAI-compatible endpoint', () => {
|
||||
it('auto-detects AI_BASE_URL + AI_API_KEY when no Anthropic-family credentials exist', () => {
|
||||
byo()
|
||||
expect(resolveAiProvider()).toBe('openai-compatible')
|
||||
expect(hasAiCredentials()).toBe(true)
|
||||
})
|
||||
|
||||
// Hosted stays hosted: a stray BYO pair must never move inference.
|
||||
it('keeps Bedrock ahead of a BYO endpoint when static AWS keys are present', () => {
|
||||
hosted()
|
||||
byo()
|
||||
expect(resolveAiProvider()).toBe('bedrock')
|
||||
})
|
||||
|
||||
it('keeps the direct Anthropic API ahead of a BYO endpoint', () => {
|
||||
process.env.ANTHROPIC_API_KEY = 'sk-ant-api03-example'
|
||||
byo()
|
||||
expect(resolveAiProvider()).toBe('anthropic')
|
||||
})
|
||||
|
||||
it('honours AI_PROVIDER=openai-compatible as the explicit escape hatch', () => {
|
||||
hosted()
|
||||
byo()
|
||||
process.env.AI_PROVIDER = 'openai-compatible'
|
||||
expect(resolveAiProvider()).toBe('openai-compatible')
|
||||
})
|
||||
|
||||
// AI_API_KEY is optional: a local OpenAI-compatible server usually has no
|
||||
// auth, so a base URL alone counts as configured. A key is added only when
|
||||
// the endpoint requires one.
|
||||
it('counts a base URL alone as credentials (keyless local server)', () => {
|
||||
process.env.AI_PROVIDER = 'openai-compatible'
|
||||
process.env.AI_BASE_URL = 'http://localhost:11434/v1'
|
||||
expect(hasAiCredentials()).toBe(true)
|
||||
})
|
||||
|
||||
it('is not configured with no base URL at all', () => {
|
||||
process.env.AI_PROVIDER = 'openai-compatible'
|
||||
expect(hasAiCredentials()).toBe(false)
|
||||
})
|
||||
|
||||
it('passes bare model ids through untouched', () => {
|
||||
expect(toProviderModelId('google/gemma-4-31B-it', 'openai-compatible')).toBe('google/gemma-4-31B-it')
|
||||
})
|
||||
|
||||
// The chat loop and the other direct SDK callers cannot run here: fail
|
||||
// loudly at construction rather than with an opaque error at call time.
|
||||
it('refuses to build an Anthropic client for it', () => {
|
||||
byo()
|
||||
expect(() => createAiClient()).toThrow(AiProviderUnsupportedError)
|
||||
})
|
||||
})
|
||||
|
||||
describe('resolveTierModel precedence', () => {
|
||||
it('defaults every tier to Claude on the Anthropic family', () => {
|
||||
hosted()
|
||||
expect(resolveTierModel('assistant')).toBe('claude-sonnet-5')
|
||||
expect(resolveTierModel('heavy')).toBe('claude-sonnet-5')
|
||||
expect(resolveTierModel('extraction')).toBe('claude-sonnet-5')
|
||||
})
|
||||
|
||||
// Hosted operators set BEDROCK_* today; those must keep winning over the
|
||||
// new generic AI_MODEL so a migration to the new names is a no-op until
|
||||
// someone deliberately moves.
|
||||
it('lets the legacy Bedrock tier variables beat AI_MODEL, and AI_<TIER>_MODEL beat both', () => {
|
||||
hosted()
|
||||
process.env.AI_MODEL = 'generic'
|
||||
process.env.BEDROCK_MODEL_ID = 'legacy-extraction'
|
||||
process.env.BEDROCK_SONNET_MODEL_ID = 'legacy-assistant'
|
||||
process.env.BEDROCK_OPUS_MODEL_ID = 'legacy-heavy'
|
||||
expect(resolveTierModel('extraction')).toBe('legacy-extraction')
|
||||
expect(resolveTierModel('assistant')).toBe('legacy-assistant')
|
||||
expect(resolveTierModel('heavy')).toBe('legacy-heavy')
|
||||
|
||||
process.env.AI_EXTRACTION_MODEL = 'specific-extraction'
|
||||
expect(resolveTierModel('extraction')).toBe('specific-extraction')
|
||||
})
|
||||
|
||||
it('has no default on an OpenAI-compatible endpoint', () => {
|
||||
byo('')
|
||||
expect(resolveTierModel('extraction')).toBeNull()
|
||||
process.env.AI_MODEL = 'x'
|
||||
expect(resolveTierModel('extraction')).toBe('x')
|
||||
process.env.AI_EXTRACTION_MODEL = 'y'
|
||||
expect(resolveTierModel('extraction')).toBe('y')
|
||||
expect(resolveTierModel('assistant')).toBe('x')
|
||||
})
|
||||
})
|
||||
|
||||
describe('readAiConfig', () => {
|
||||
it('reads the extraction output cap from the new name, then the legacy name, then 8192', () => {
|
||||
hosted()
|
||||
expect(readAiConfig().extractionMaxTokens).toBe(8192)
|
||||
process.env.BEDROCK_MAX_TOKENS = '4000'
|
||||
expect(readAiConfig().extractionMaxTokens).toBe(4000)
|
||||
process.env.AI_EXTRACTION_MAX_TOKENS = '12000'
|
||||
expect(readAiConfig().extractionMaxTokens).toBe(12000)
|
||||
})
|
||||
|
||||
// A deliberate `0` is an invalid configuration, not "disable": fall back.
|
||||
it('ignores non-positive or garbage token caps', () => {
|
||||
hosted()
|
||||
process.env.AI_EXTRACTION_MAX_TOKENS = '0'
|
||||
expect(readAiConfig().extractionMaxTokens).toBe(8192)
|
||||
process.env.AI_EXTRACTION_MAX_TOKENS = 'lots'
|
||||
expect(readAiConfig().extractionMaxTokens).toBe(8192)
|
||||
})
|
||||
|
||||
it('rasterizes PDFs by default on OpenAI-compatible endpoints and reads them natively on Claude', () => {
|
||||
byo()
|
||||
expect(readAiConfig().pdfMode).toBe('rasterize')
|
||||
process.env.AI_PDF_MODE = 'native'
|
||||
expect(readAiConfig().pdfMode).toBe('native')
|
||||
})
|
||||
|
||||
it('reads PDFs natively on the Anthropic family regardless', () => {
|
||||
hosted()
|
||||
expect(readAiConfig().pdfMode).toBe('native')
|
||||
})
|
||||
|
||||
it('parses the boolean flags with sane defaults', () => {
|
||||
byo()
|
||||
expect(readAiConfig().vision).toBe(true)
|
||||
expect(readAiConfig().strictJson).toBe(false)
|
||||
process.env.AI_VISION = 'false'
|
||||
process.env.AI_STRICT_JSON = '1'
|
||||
expect(readAiConfig().vision).toBe(false)
|
||||
expect(readAiConfig().strictJson).toBe(true)
|
||||
})
|
||||
})
|
||||
|
||||
describe('getAiStatus', () => {
|
||||
it('is unconfigured with a reason when nothing is set', () => {
|
||||
const s = getAiStatus()
|
||||
expect(s.configured).toBe(false)
|
||||
expect(s.reason).toBe('no_credentials')
|
||||
expect(s.assistantAvailable).toBe(false)
|
||||
})
|
||||
|
||||
it('is configured and assistant-capable on hosted', () => {
|
||||
hosted()
|
||||
const s = getAiStatus()
|
||||
expect(s.provider).toBe('bedrock')
|
||||
expect(s.configured).toBe(true)
|
||||
expect(s.assistantAvailable).toBe(true)
|
||||
expect(s.capabilities.pdfNative).toBe(true)
|
||||
expect(s.models.extraction).toBe('eu.anthropic.claude-sonnet-5')
|
||||
})
|
||||
|
||||
it('needs a model id on an OpenAI-compatible endpoint before it counts as configured', () => {
|
||||
byo('')
|
||||
const s = getAiStatus()
|
||||
expect(s.configured).toBe(false)
|
||||
expect(s.reason).toBe('no_model')
|
||||
})
|
||||
|
||||
// Extraction and single-call jobs run; the chat loop does not (yet).
|
||||
it('is configured but not assistant-capable on an OpenAI-compatible endpoint', () => {
|
||||
byo()
|
||||
const s = getAiStatus()
|
||||
expect(s.provider).toBe('openai-compatible')
|
||||
expect(s.configured).toBe(true)
|
||||
expect(s.assistantAvailable).toBe(false)
|
||||
expect(s.capabilities.pdfNative).toBe(false)
|
||||
expect(s.capabilities.imageInput).toBe(true)
|
||||
expect(s.models.extraction).toBe('google/gemma-4-31B-it')
|
||||
})
|
||||
|
||||
it('reports AI_VISION=false as no image input', () => {
|
||||
byo()
|
||||
process.env.AI_VISION = 'false'
|
||||
expect(getAiStatus().capabilities.imageInput).toBe(false)
|
||||
})
|
||||
})
|
||||
@@ -0,0 +1,242 @@
|
||||
import { describe, it, expect, vi, beforeEach, afterEach } from 'vitest'
|
||||
import { MockLanguageModelV3 } from 'ai/test'
|
||||
|
||||
// Replace the provider factory with one that hands out a mock model, then
|
||||
// run the REAL `generateText` from the AI SDK through it: the assertions see
|
||||
// the prompt the SDK would put on the wire (image parts, file parts, system),
|
||||
// which is what an OpenAI-compatible endpoint receives.
|
||||
const doGenerate = vi.fn()
|
||||
const createdWith = vi.fn()
|
||||
vi.mock('@ai-sdk/openai-compatible', () => ({
|
||||
createOpenAICompatible: (settings: unknown) => {
|
||||
createdWith(settings)
|
||||
const factory = (modelId: string) =>
|
||||
new MockLanguageModelV3({
|
||||
modelId,
|
||||
doGenerate: async (options: unknown) => doGenerate(options),
|
||||
})
|
||||
return factory
|
||||
},
|
||||
}))
|
||||
|
||||
const rasterizeMock = vi.fn()
|
||||
vi.mock('../rasterize-pdf', () => ({
|
||||
rasterizePdf: (...args: unknown[]) => rasterizeMock(...args),
|
||||
}))
|
||||
|
||||
import { readAiConfig } from '../config'
|
||||
import { createOpenAICompatibleService } from '../services/openai-compatible'
|
||||
|
||||
const ENV = ['AWS_ACCESS_KEY_ID', 'AWS_SECRET_ACCESS_KEY', 'ANTHROPIC_API_KEY', 'AI_PROVIDER', 'AI_BASE_URL', 'AI_API_KEY', 'AI_MODEL', 'AI_EXTRACTION_MODEL', 'AI_VISION', 'AI_STRICT_JSON', 'AI_PDF_MODE', 'AI_PDF_MAX_PAGES'] as const
|
||||
let saved: Record<string, string | undefined> = {}
|
||||
|
||||
function textResponse(text: string) {
|
||||
return {
|
||||
content: [{ type: 'text', text }],
|
||||
finishReason: { unified: 'stop', raw: undefined },
|
||||
usage: {
|
||||
inputTokens: { total: 42, noCache: 40, cacheRead: 2, cacheWrite: undefined },
|
||||
outputTokens: { total: 7, text: 7, reasoning: undefined },
|
||||
},
|
||||
warnings: [],
|
||||
}
|
||||
}
|
||||
|
||||
beforeEach(() => {
|
||||
vi.clearAllMocks()
|
||||
saved = {}
|
||||
for (const k of ENV) {
|
||||
saved[k] = process.env[k]
|
||||
delete process.env[k]
|
||||
}
|
||||
process.env.AI_BASE_URL = 'https://api.berget.ai/v1'
|
||||
process.env.AI_API_KEY = 'sk-berget-example'
|
||||
process.env.AI_MODEL = 'google/gemma-4-31B-it'
|
||||
doGenerate.mockResolvedValue(textResponse('{"supplier":"x"}'))
|
||||
})
|
||||
afterEach(() => {
|
||||
for (const k of ENV) {
|
||||
if (saved[k] === undefined) delete process.env[k]
|
||||
else process.env[k] = saved[k]
|
||||
}
|
||||
})
|
||||
|
||||
const SYSTEM = 'You extract fields.'
|
||||
const INSTRUCTION = 'Extract the fields per the schema. JSON only.'
|
||||
|
||||
function promptOf(call = 0) {
|
||||
return doGenerate.mock.calls[call][0].prompt as Array<{ role: string; content: unknown }>
|
||||
}
|
||||
|
||||
describe('createOpenAICompatibleService', () => {
|
||||
it('builds the provider from AI_BASE_URL / AI_API_KEY', () => {
|
||||
createOpenAICompatibleService(readAiConfig())
|
||||
expect(createdWith).toHaveBeenCalledWith(
|
||||
expect.objectContaining({ baseURL: 'https://api.berget.ai/v1', apiKey: 'sk-berget-example', supportsStructuredOutputs: false })
|
||||
)
|
||||
})
|
||||
|
||||
it('reports capabilities from the config: rasterized PDFs, vision on, no strict JSON by default', () => {
|
||||
const svc = createOpenAICompatibleService(readAiConfig())
|
||||
expect(svc.capabilities).toEqual({
|
||||
pdfNative: false,
|
||||
imageInput: true,
|
||||
toolUse: true,
|
||||
forcedToolChoice: false,
|
||||
strictJsonSchema: false,
|
||||
})
|
||||
expect(svc.modelFor('extraction')).toBe('google/gemma-4-31B-it')
|
||||
})
|
||||
|
||||
it('generateText sends system + prompt and maps usage', async () => {
|
||||
const svc = createOpenAICompatibleService(readAiConfig())
|
||||
const result = await svc.generateText({ tier: 'assistant', system: 'S', prompt: 'Hej', maxTokens: 50 })
|
||||
const prompt = promptOf()
|
||||
expect(prompt[0]).toEqual({ role: 'system', content: 'S' })
|
||||
expect(prompt[1].role).toBe('user')
|
||||
expect(doGenerate.mock.calls[0][0].maxOutputTokens).toBe(50)
|
||||
expect(result).toEqual({
|
||||
text: '{"supplier":"x"}',
|
||||
model: 'google/gemma-4-31B-it',
|
||||
usage: { inputTokens: 42, outputTokens: 7, cacheCreationInputTokens: null, cacheReadInputTokens: 2 },
|
||||
})
|
||||
})
|
||||
|
||||
it('extractFromDocument sends an image as an image part followed by the instruction', async () => {
|
||||
const svc = createOpenAICompatibleService(readAiConfig())
|
||||
const jpeg = Buffer.from('JPEG')
|
||||
const result = await svc.extractFromDocument({
|
||||
document: { kind: 'image', data: jpeg, mediaType: 'image/jpeg' },
|
||||
system: SYSTEM,
|
||||
instruction: INSTRUCTION,
|
||||
maxTokens: 8192,
|
||||
})
|
||||
expect(result.ok).toBe(true)
|
||||
const prompt = promptOf()
|
||||
expect(prompt[0]).toEqual({ role: 'system', content: SYSTEM })
|
||||
const user = prompt[1].content as Array<{ type: string; mediaType?: string; text?: string }>
|
||||
expect(user[0].type).toBe('file')
|
||||
expect(user[0].mediaType).toBe('image/jpeg')
|
||||
expect(user[1]).toEqual({ type: 'text', text: INSTRUCTION })
|
||||
})
|
||||
|
||||
it('extractFromDocument sends plain text as two text parts (works on text-only models)', async () => {
|
||||
process.env.AI_VISION = 'false'
|
||||
const svc = createOpenAICompatibleService(readAiConfig())
|
||||
const result = await svc.extractFromDocument({
|
||||
document: { kind: 'text', text: 'Total 100' },
|
||||
system: SYSTEM,
|
||||
instruction: INSTRUCTION,
|
||||
maxTokens: 100,
|
||||
})
|
||||
expect(result.ok).toBe(true)
|
||||
const user = promptOf()[1].content as Array<{ type: string; text?: string }>
|
||||
expect(user).toEqual([
|
||||
{ type: 'text', text: 'Total 100' },
|
||||
{ type: 'text', text: INSTRUCTION },
|
||||
])
|
||||
})
|
||||
|
||||
it('rasterizes PDFs by default and sends one image part per page', async () => {
|
||||
rasterizeMock.mockResolvedValue({
|
||||
ok: true,
|
||||
pages: [Buffer.from('p1'), Buffer.from('p2')],
|
||||
mediaType: 'image/png',
|
||||
pageCount: 2,
|
||||
})
|
||||
const svc = createOpenAICompatibleService(readAiConfig())
|
||||
const result = await svc.extractFromDocument({
|
||||
document: { kind: 'pdf', data: Buffer.from('%PDF'), fileName: 'f.pdf' },
|
||||
system: SYSTEM,
|
||||
instruction: INSTRUCTION,
|
||||
maxTokens: 100,
|
||||
})
|
||||
expect(rasterizeMock).toHaveBeenCalledWith(expect.any(Buffer), { maxPages: 4 })
|
||||
expect(result).toMatchObject({ ok: true, pagesRasterized: 2 })
|
||||
const user = promptOf()[1].content as Array<{ type: string; mediaType?: string }>
|
||||
expect(user.map((p) => p.type)).toEqual(['file', 'file', 'text'])
|
||||
expect(user[0].mediaType).toBe('image/png')
|
||||
})
|
||||
|
||||
it('sends the PDF as a native file part when AI_PDF_MODE=native', async () => {
|
||||
process.env.AI_PDF_MODE = 'native'
|
||||
const svc = createOpenAICompatibleService(readAiConfig())
|
||||
await svc.extractFromDocument({
|
||||
document: { kind: 'pdf', data: Buffer.from('%PDF'), fileName: 'f.pdf' },
|
||||
system: SYSTEM,
|
||||
instruction: INSTRUCTION,
|
||||
maxTokens: 100,
|
||||
})
|
||||
expect(rasterizeMock).not.toHaveBeenCalled()
|
||||
const user = promptOf()[1].content as Array<{ type: string; mediaType?: string; filename?: string }>
|
||||
expect(user[0]).toMatchObject({ type: 'file', mediaType: 'application/pdf', filename: 'f.pdf' })
|
||||
})
|
||||
|
||||
// Honest skips, never fake failures: the caller stamps the reason.
|
||||
it('skips images and PDFs when AI_VISION=false', async () => {
|
||||
process.env.AI_VISION = 'false'
|
||||
const svc = createOpenAICompatibleService(readAiConfig())
|
||||
const image = await svc.extractFromDocument({
|
||||
document: { kind: 'image', data: Buffer.from('x'), mediaType: 'image/png' },
|
||||
system: SYSTEM,
|
||||
instruction: INSTRUCTION,
|
||||
maxTokens: 1,
|
||||
})
|
||||
const pdf = await svc.extractFromDocument({
|
||||
document: { kind: 'pdf', data: Buffer.from('%PDF') },
|
||||
system: SYSTEM,
|
||||
instruction: INSTRUCTION,
|
||||
maxTokens: 1,
|
||||
})
|
||||
expect(image).toEqual({ ok: false, skipped: 'ai_no_vision' })
|
||||
expect(pdf).toEqual({ ok: false, skipped: 'ai_no_vision' })
|
||||
expect(doGenerate).not.toHaveBeenCalled()
|
||||
})
|
||||
|
||||
it('skips with pdf_rasterizer_missing when poppler is not installed', async () => {
|
||||
rasterizeMock.mockResolvedValue({ ok: false, reason: 'rasterizer_missing' })
|
||||
const svc = createOpenAICompatibleService(readAiConfig())
|
||||
const result = await svc.extractFromDocument({
|
||||
document: { kind: 'pdf', data: Buffer.from('%PDF') },
|
||||
system: SYSTEM,
|
||||
instruction: INSTRUCTION,
|
||||
maxTokens: 1,
|
||||
})
|
||||
expect(result).toEqual({ ok: false, skipped: 'pdf_rasterizer_missing' })
|
||||
expect(doGenerate).not.toHaveBeenCalled()
|
||||
})
|
||||
|
||||
it('skips with ai_unconfigured when the model is missing', async () => {
|
||||
delete process.env.AI_MODEL
|
||||
const svc = createOpenAICompatibleService(readAiConfig())
|
||||
const result = await svc.extractFromDocument({
|
||||
document: { kind: 'text', text: 'x' },
|
||||
system: SYSTEM,
|
||||
instruction: INSTRUCTION,
|
||||
maxTokens: 1,
|
||||
})
|
||||
expect(result).toEqual({ ok: false, skipped: 'ai_unconfigured' })
|
||||
})
|
||||
|
||||
it('generateStructured without strict JSON embeds the schema and parses the first JSON object', async () => {
|
||||
doGenerate.mockResolvedValueOnce(textResponse('Here you go:\n```json\n{"paired": true}\n```'))
|
||||
const svc = createOpenAICompatibleService(readAiConfig())
|
||||
const result = await svc.generateStructured({
|
||||
tier: 'heavy',
|
||||
prompt: 'Decide',
|
||||
maxTokens: 100,
|
||||
schema: { name: 'verdict', description: 'pairing verdict', jsonSchema: { type: 'object' } },
|
||||
})
|
||||
expect(result.value).toEqual({ paired: true })
|
||||
const system = promptOf()[0].content as string
|
||||
expect(system).toContain('JSON Schema')
|
||||
expect(system).toContain('pairing verdict')
|
||||
})
|
||||
|
||||
it('turns on structured outputs at the provider when AI_STRICT_JSON=true', () => {
|
||||
process.env.AI_STRICT_JSON = 'true'
|
||||
const svc = createOpenAICompatibleService(readAiConfig())
|
||||
expect(createdWith).toHaveBeenLastCalledWith(expect.objectContaining({ supportsStructuredOutputs: true }))
|
||||
expect(svc.capabilities.strictJsonSchema).toBe(true)
|
||||
})
|
||||
})
|
||||
@@ -11,6 +11,8 @@ import {
|
||||
|
||||
const AI_ENV_KEYS = [
|
||||
'AI_PROVIDER',
|
||||
'AI_BASE_URL',
|
||||
'AI_API_KEY',
|
||||
'ANTHROPIC_API_KEY',
|
||||
'AWS_ACCESS_KEY_ID',
|
||||
'AWS_SECRET_ACCESS_KEY',
|
||||
@@ -100,6 +102,11 @@ describe('hasAiCredentials', () => {
|
||||
expect(hasAiCredentials()).toBe(false)
|
||||
})
|
||||
|
||||
it('is true for an OpenAI-compatible base URL with no key (keyless local server)', () => {
|
||||
process.env.AI_BASE_URL = 'http://localhost:11434/v1'
|
||||
expect(hasAiCredentials()).toBe(true)
|
||||
})
|
||||
|
||||
it('is true for an Anthropic key', () => {
|
||||
process.env.ANTHROPIC_API_KEY = 'sk-ant-api03-example'
|
||||
expect(hasAiCredentials()).toBe(true)
|
||||
|
||||
@@ -0,0 +1,76 @@
|
||||
import { describe, it, expect, vi, beforeEach } from 'vitest'
|
||||
import { writeFile } from 'node:fs/promises'
|
||||
|
||||
// Fake pdftoppm: the module shells out with (-r dpi -png -f 1 -l N input prefix)
|
||||
// and reads back <prefix>-<n>.png. The fake writes those files itself, so the
|
||||
// read-back, ordering and cleanup paths run for real on a temp dir.
|
||||
const execFileMock = vi.fn()
|
||||
vi.mock('node:child_process', () => ({
|
||||
execFile: (...args: unknown[]) => execFileMock(...args),
|
||||
}))
|
||||
|
||||
import { rasterizePdf } from '../rasterize-pdf'
|
||||
|
||||
type Cb = (err: Error | null, out?: { stdout: string; stderr: string }) => void
|
||||
|
||||
beforeEach(() => {
|
||||
vi.clearAllMocks()
|
||||
})
|
||||
|
||||
describe('rasterizePdf', () => {
|
||||
it('renders the first pages in order and cleans up', async () => {
|
||||
execFileMock.mockImplementation((bin: string, args: string[], _opts: unknown, cb: Cb) => {
|
||||
expect(bin).toBe('pdftoppm')
|
||||
expect(args.slice(0, 6)).toEqual(['-r', '110', '-png', '-f', '1', '-l'])
|
||||
const prefix = args[args.length - 1]
|
||||
void (async () => {
|
||||
// Two pages, written out of order to prove numeric sorting.
|
||||
await writeFile(`${prefix}-2.png`, Buffer.from('PAGE2'))
|
||||
await writeFile(`${prefix}-1.png`, Buffer.from('PAGE1'))
|
||||
cb(null, { stdout: '', stderr: '' })
|
||||
})()
|
||||
})
|
||||
const result = await rasterizePdf(Buffer.from('%PDF'), { maxPages: 4 })
|
||||
expect(result.ok).toBe(true)
|
||||
if (!result.ok) return
|
||||
expect(result.pageCount).toBe(2)
|
||||
expect(result.pages.map((p) => p.toString())).toEqual(['PAGE1', 'PAGE2'])
|
||||
expect(result.mediaType).toBe('image/png')
|
||||
const args = execFileMock.mock.calls[0][1] as string[]
|
||||
expect(args[6]).toBe('4') // -l maxPages
|
||||
})
|
||||
|
||||
it('reports a missing binary as rasterizer_missing', async () => {
|
||||
execFileMock.mockImplementation((_bin: string, _args: string[], _opts: unknown, cb: Cb) => {
|
||||
const err = Object.assign(new Error('spawn pdftoppm ENOENT'), { code: 'ENOENT' })
|
||||
cb(err)
|
||||
})
|
||||
const result = await rasterizePdf(Buffer.from('%PDF'), { maxPages: 4 })
|
||||
expect(result).toEqual({ ok: false, reason: 'rasterizer_missing' })
|
||||
})
|
||||
|
||||
it('reports any other failure as failed with the message', async () => {
|
||||
execFileMock.mockImplementation((_bin: string, _args: string[], _opts: unknown, cb: Cb) => {
|
||||
cb(new Error('Syntax Error: Couldn\'t read xref table'))
|
||||
})
|
||||
const result = await rasterizePdf(Buffer.from('not a pdf'), { maxPages: 2 })
|
||||
expect(result).toMatchObject({ ok: false, reason: 'failed' })
|
||||
})
|
||||
|
||||
it('treats a run that produced no pages as failed', async () => {
|
||||
execFileMock.mockImplementation((_bin: string, _args: string[], _opts: unknown, cb: Cb) => {
|
||||
cb(null, { stdout: '', stderr: '' })
|
||||
})
|
||||
const result = await rasterizePdf(Buffer.from('%PDF'), { maxPages: 1 })
|
||||
expect(result).toMatchObject({ ok: false, reason: 'failed' })
|
||||
})
|
||||
|
||||
it('honours a binary override', async () => {
|
||||
execFileMock.mockImplementation((bin: string, _args: string[], _opts: unknown, cb: Cb) => {
|
||||
expect(bin).toBe('/opt/poppler/bin/pdftoppm')
|
||||
cb(new Error('x'))
|
||||
})
|
||||
await rasterizePdf(Buffer.from('%PDF'), { maxPages: 1, binary: '/opt/poppler/bin/pdftoppm' })
|
||||
expect(execFileMock).toHaveBeenCalled()
|
||||
})
|
||||
})
|
||||
@@ -0,0 +1,180 @@
|
||||
import {
|
||||
hasAiCredentials,
|
||||
resolveAiProvider,
|
||||
toProviderModelId,
|
||||
type AiProvider,
|
||||
} from './provider'
|
||||
import type { AiCapabilities, AiPdfMode, AiStatus, AiTier } from './types'
|
||||
|
||||
/**
|
||||
* Environment parsing for the AI layer. Read on every call (cheap string
|
||||
* reads) so tests and the smoke script see env changes; the service cache in
|
||||
* lib/ai/index.ts keys on the resolved config, not on this module's state.
|
||||
*
|
||||
* Variables (all optional; legacy BEDROCK_* names keep working):
|
||||
*
|
||||
* AI_PROVIDER bedrock | anthropic | openai-compatible (else auto-detect)
|
||||
* AI_BASE_URL OpenAI-compatible endpoint (Swedish provider or a local model)
|
||||
* AI_API_KEY optional: only when that endpoint requires auth
|
||||
* AI_MODEL default model id for every tier
|
||||
* AI_ASSISTANT_MODEL per-tier overrides (fallbacks: BEDROCK_SONNET_MODEL_ID,
|
||||
* AI_HEAVY_MODEL BEDROCK_OPUS_MODEL_ID, BEDROCK_MODEL_ID respectively)
|
||||
* AI_EXTRACTION_MODEL
|
||||
* AI_EXTRACTION_MAX_TOKENS output cap for document extraction (fallback BEDROCK_MAX_TOKENS, then 8192)
|
||||
* AI_VISION OpenAI-compatible only: the configured models accept images (default true)
|
||||
* AI_STRICT_JSON OpenAI-compatible only: use response_format json_schema (default false)
|
||||
* AI_PDF_MODE auto | native | rasterize (auto = native on Claude, rasterize elsewhere)
|
||||
* AI_PDF_MAX_PAGES pages rasterized per PDF (default 4)
|
||||
*/
|
||||
|
||||
export interface ResolvedAiConfig {
|
||||
provider: AiProvider
|
||||
/** Credentials present (and, for OpenAI-compatible, a model id). */
|
||||
configured: boolean
|
||||
reason: AiStatus['reason']
|
||||
baseUrl: string | null
|
||||
apiKey: string | null
|
||||
/** Bare model ids per tier (null only when OpenAI-compatible has none configured). */
|
||||
models: Record<AiTier, string | null>
|
||||
extractionMaxTokens: number
|
||||
vision: boolean
|
||||
strictJson: boolean
|
||||
pdfMode: AiPdfMode
|
||||
pdfMaxPages: number
|
||||
}
|
||||
|
||||
const DEFAULT_CLAUDE_MODEL = 'claude-sonnet-5'
|
||||
const DEFAULT_EXTRACTION_MAX_TOKENS = 8192
|
||||
const DEFAULT_PDF_MAX_PAGES = 4
|
||||
|
||||
function env(name: string): string | null {
|
||||
const v = process.env[name]
|
||||
if (v === undefined) return null
|
||||
const trimmed = v.trim()
|
||||
return trimmed.length > 0 ? trimmed : null
|
||||
}
|
||||
|
||||
function envBool(name: string, fallback: boolean): boolean {
|
||||
const v = env(name)?.toLowerCase()
|
||||
if (v === null || v === undefined) return fallback
|
||||
if (v === 'true' || v === '1' || v === 'yes' || v === 'on') return true
|
||||
if (v === 'false' || v === '0' || v === 'no' || v === 'off') return false
|
||||
return fallback
|
||||
}
|
||||
|
||||
// Use the env value only if it's a positive number: `||` would also fall back
|
||||
// on a deliberate `0`, masking what is really an invalid configuration rather
|
||||
// than the intent to disable.
|
||||
function envPositiveInt(name: string): number | null {
|
||||
const raw = env(name)
|
||||
if (raw === null) return null
|
||||
const parsed = Number(raw)
|
||||
return Number.isFinite(parsed) && parsed > 0 ? Math.floor(parsed) : null
|
||||
}
|
||||
|
||||
const LEGACY_TIER_VARS: Record<AiTier, string> = {
|
||||
assistant: 'BEDROCK_SONNET_MODEL_ID',
|
||||
heavy: 'BEDROCK_OPUS_MODEL_ID',
|
||||
extraction: 'BEDROCK_MODEL_ID',
|
||||
}
|
||||
|
||||
const TIER_VARS: Record<AiTier, string> = {
|
||||
assistant: 'AI_ASSISTANT_MODEL',
|
||||
heavy: 'AI_HEAVY_MODEL',
|
||||
extraction: 'AI_EXTRACTION_MODEL',
|
||||
}
|
||||
|
||||
/**
|
||||
* Model id for a tier, most specific wins: AI_<TIER>_MODEL, then the legacy
|
||||
* Bedrock-era tier variable, then AI_MODEL, then the Claude default on the
|
||||
* Anthropic family. An OpenAI-compatible endpoint has no sane default model,
|
||||
* so the result is null there and the status reports `no_model`.
|
||||
*/
|
||||
export function resolveTierModel(tier: AiTier, provider: AiProvider = resolveAiProvider()): string | null {
|
||||
const specific = env(TIER_VARS[tier]) ?? env(LEGACY_TIER_VARS[tier]) ?? env('AI_MODEL')
|
||||
if (specific) return specific
|
||||
return provider === 'openai-compatible' ? null : DEFAULT_CLAUDE_MODEL
|
||||
}
|
||||
|
||||
export function readAiConfig(): ResolvedAiConfig {
|
||||
const provider = resolveAiProvider()
|
||||
const models: Record<AiTier, string | null> = {
|
||||
assistant: resolveTierModel('assistant', provider),
|
||||
heavy: resolveTierModel('heavy', provider),
|
||||
extraction: resolveTierModel('extraction', provider),
|
||||
}
|
||||
const credentials = hasAiCredentials()
|
||||
const isOpenAiCompatible = provider === 'openai-compatible'
|
||||
const hasModels = !isOpenAiCompatible || Object.values(models).every((m) => m !== null)
|
||||
|
||||
const pdfModeRaw = env('AI_PDF_MODE')?.toLowerCase()
|
||||
const pdfMode: AiPdfMode =
|
||||
pdfModeRaw === 'native' || pdfModeRaw === 'rasterize'
|
||||
? pdfModeRaw
|
||||
: isOpenAiCompatible
|
||||
? 'rasterize'
|
||||
: 'native'
|
||||
|
||||
return {
|
||||
provider,
|
||||
configured: credentials && hasModels,
|
||||
reason: !credentials ? 'no_credentials' : !hasModels ? 'no_model' : 'ok',
|
||||
baseUrl: isOpenAiCompatible ? env('AI_BASE_URL') : null,
|
||||
apiKey: isOpenAiCompatible ? env('AI_API_KEY') : null,
|
||||
models,
|
||||
extractionMaxTokens:
|
||||
envPositiveInt('AI_EXTRACTION_MAX_TOKENS') ??
|
||||
envPositiveInt('BEDROCK_MAX_TOKENS') ??
|
||||
DEFAULT_EXTRACTION_MAX_TOKENS,
|
||||
vision: envBool('AI_VISION', true),
|
||||
strictJson: envBool('AI_STRICT_JSON', false),
|
||||
pdfMode,
|
||||
pdfMaxPages: envPositiveInt('AI_PDF_MAX_PAGES') ?? DEFAULT_PDF_MAX_PAGES,
|
||||
}
|
||||
}
|
||||
|
||||
export function capabilitiesFor(cfg: ResolvedAiConfig): AiCapabilities {
|
||||
if (cfg.provider === 'openai-compatible') {
|
||||
return {
|
||||
pdfNative: cfg.pdfMode === 'native',
|
||||
imageInput: cfg.vision,
|
||||
toolUse: true,
|
||||
forcedToolChoice: false,
|
||||
strictJsonSchema: cfg.strictJson,
|
||||
}
|
||||
}
|
||||
return {
|
||||
pdfNative: true,
|
||||
imageInput: true,
|
||||
toolUse: true,
|
||||
forcedToolChoice: true,
|
||||
strictJsonSchema: false,
|
||||
}
|
||||
}
|
||||
|
||||
/**
|
||||
* Single source of truth for "is AI wired up here, and for what". Cheap: env
|
||||
* reads only, no network. Drives the extraction fail-fast path (a document is
|
||||
* stamped `skipped:ai_unconfigured` instead of waiting 30 s for nothing), the
|
||||
* status route, agent routes (503 instead of a stream that dies), and the
|
||||
* smoke script.
|
||||
*/
|
||||
export function getAiStatus(): AiStatus {
|
||||
const cfg = readAiConfig()
|
||||
const capabilities = capabilitiesFor(cfg)
|
||||
const models: Record<AiTier, string | null> = {
|
||||
assistant: cfg.models.assistant ? toProviderModelId(cfg.models.assistant, cfg.provider) : null,
|
||||
heavy: cfg.models.heavy ? toProviderModelId(cfg.models.heavy, cfg.provider) : null,
|
||||
extraction: cfg.models.extraction ? toProviderModelId(cfg.models.extraction, cfg.provider) : null,
|
||||
}
|
||||
return {
|
||||
provider: cfg.provider,
|
||||
configured: cfg.configured,
|
||||
reason: cfg.reason,
|
||||
capabilities,
|
||||
models,
|
||||
pdfMode: cfg.pdfMode,
|
||||
// The chat loop still speaks the Anthropic messages surface directly.
|
||||
assistantAvailable: cfg.configured && cfg.provider !== 'openai-compatible',
|
||||
}
|
||||
}
|
||||
@@ -0,0 +1,66 @@
|
||||
import { getAiStatus, readAiConfig, type ResolvedAiConfig } from './config'
|
||||
import { createAnthropicFamilyService } from './services/anthropic-family'
|
||||
import { createOpenAICompatibleService } from './services/openai-compatible'
|
||||
import type { AiService } from './types'
|
||||
|
||||
export type {
|
||||
AiCapabilities,
|
||||
AiDocumentInput,
|
||||
AiImageMediaType,
|
||||
AiPdfMode,
|
||||
AiProviderKind,
|
||||
AiService,
|
||||
AiStatus,
|
||||
AiTier,
|
||||
AiUsage,
|
||||
ExtractFromDocumentRequest,
|
||||
ExtractFromDocumentResult,
|
||||
ExtractionSkipReason,
|
||||
GenerateStructuredRequest,
|
||||
GenerateStructuredResult,
|
||||
GenerateTextRequest,
|
||||
GenerateTextResult,
|
||||
} from './types'
|
||||
export { getAiStatus, readAiConfig, resolveTierModel } from './config'
|
||||
export { extractJsonObject } from './json'
|
||||
|
||||
// One service per resolved configuration. Keyed on the non-secret parts of
|
||||
// the config plus credential presence, so a changed environment (tests, the
|
||||
// smoke script) gets a fresh service while a long-lived process reuses one.
|
||||
let cached: { key: string; service: AiService } | null = null
|
||||
|
||||
function cacheKey(cfg: ResolvedAiConfig): string {
|
||||
return JSON.stringify({
|
||||
provider: cfg.provider,
|
||||
configured: cfg.configured,
|
||||
baseUrl: cfg.baseUrl,
|
||||
hasKey: !!cfg.apiKey,
|
||||
models: cfg.models,
|
||||
vision: cfg.vision,
|
||||
strictJson: cfg.strictJson,
|
||||
pdfMode: cfg.pdfMode,
|
||||
pdfMaxPages: cfg.pdfMaxPages,
|
||||
})
|
||||
}
|
||||
|
||||
/**
|
||||
* The AI service for this deployment. Never throws on construction: an
|
||||
* unconfigured deployment still gets a service whose extractFromDocument
|
||||
* answers `skipped: ai_unconfigured`, so upload paths degrade quietly.
|
||||
*/
|
||||
export function getAiService(): AiService {
|
||||
const cfg = readAiConfig()
|
||||
const key = cacheKey(cfg)
|
||||
if (cached && cached.key === key) return cached.service
|
||||
const service =
|
||||
cfg.provider === 'openai-compatible'
|
||||
? createOpenAICompatibleService(cfg)
|
||||
: createAnthropicFamilyService(cfg)
|
||||
cached = { key, service }
|
||||
return service
|
||||
}
|
||||
|
||||
/** Tests only: drop the cached service so the next call re-reads the environment. */
|
||||
export function resetAiServiceForTests(): void {
|
||||
cached = null
|
||||
}
|
||||
@@ -0,0 +1,55 @@
|
||||
// Bounds for the candidate scan below. Real model output is already capped by
|
||||
// the caller's max_tokens (roughly 33 KB of text at 8192 tokens), so genuine
|
||||
// responses never come near these; they exist so pathological or adversarially
|
||||
// brace-laden text cannot make the scan quadratic (compliance review A.8.28).
|
||||
// Oversized or exhausted inputs fall through to the raw text and land in the
|
||||
// caller's existing parse-failure path.
|
||||
const MAX_SCAN_INPUT_LENGTH = 256 * 1024
|
||||
const MAX_CANDIDATE_ATTEMPTS = 50
|
||||
|
||||
/**
|
||||
* Models intermittently wrap a JSON answer in markdown fences (```json ... ```)
|
||||
* or add prose around it despite a JSON-only instruction. Scan for balanced
|
||||
* top-level '{'..'}' candidates (string- and escape-aware, so braces inside
|
||||
* JSON string values don't end a candidate early) and return the first one
|
||||
* JSON.parse accepts; prose braces around the object form unparseable
|
||||
* candidates and are skipped. Returns the input unchanged when no candidate
|
||||
* parses, so the caller's parse-failure path handles prose-only refusals.
|
||||
* Schema validation downstream still rejects well-formed-but-wrong JSON.
|
||||
*/
|
||||
export function extractJsonObject(raw: string): string {
|
||||
if (raw.length > MAX_SCAN_INPUT_LENGTH) return raw
|
||||
let attempts = 0
|
||||
let start = raw.indexOf('{')
|
||||
while (start !== -1 && attempts < MAX_CANDIDATE_ATTEMPTS) {
|
||||
attempts++
|
||||
let depth = 0
|
||||
let inString = false
|
||||
let escaped = false
|
||||
for (let i = start; i < raw.length; i++) {
|
||||
const ch = raw[i]
|
||||
if (inString) {
|
||||
if (escaped) escaped = false
|
||||
else if (ch === '\\') escaped = true
|
||||
else if (ch === '"') inString = false
|
||||
} else if (ch === '"') {
|
||||
inString = true
|
||||
} else if (ch === '{') {
|
||||
depth++
|
||||
} else if (ch === '}') {
|
||||
depth--
|
||||
if (depth === 0) {
|
||||
const candidate = raw.slice(start, i + 1)
|
||||
try {
|
||||
JSON.parse(candidate)
|
||||
return candidate
|
||||
} catch {
|
||||
break
|
||||
}
|
||||
}
|
||||
}
|
||||
}
|
||||
start = raw.indexOf('{', start + 1)
|
||||
}
|
||||
return raw
|
||||
}
|
||||
+58
-13
@@ -2,17 +2,43 @@ import Anthropic from '@anthropic-ai/sdk'
|
||||
import AnthropicBedrock from '@anthropic-ai/bedrock-sdk'
|
||||
|
||||
/**
|
||||
* Which backend Claude traffic goes to.
|
||||
* Which backend AI traffic goes to.
|
||||
*
|
||||
* Hosted runs on AWS Bedrock: keeping inference inside eu-north-1 is a
|
||||
* deliberate BFL/GDPR posture for Swedish accounting data, not an
|
||||
* implementation detail. Self-hosted deployments generally have no AWS
|
||||
* account at all, so they get the direct Anthropic API with a plain
|
||||
* ANTHROPIC_API_KEY.
|
||||
* ANTHROPIC_API_KEY, or any OpenAI-compatible endpoint (the Swedish inference
|
||||
* providers a sovereign self-host points at, or a local model) via
|
||||
* AI_BASE_URL (+ AI_API_KEY when the endpoint requires auth).
|
||||
*
|
||||
* The two Anthropic-family backends share the `messages.create/stream`
|
||||
* surface this factory hands out. The OpenAI-compatible backend does not: it
|
||||
* is reachable only through the job-shaped service in lib/ai (getAiService),
|
||||
* and createAiClient() refuses it loudly rather than returning a client that
|
||||
* would fail at call time.
|
||||
*
|
||||
* See https://github.com/erp-mafia/accounted/issues/1406.
|
||||
*/
|
||||
export type AiProvider = 'bedrock' | 'anthropic'
|
||||
export type AiProvider = 'bedrock' | 'anthropic' | 'openai-compatible'
|
||||
|
||||
/**
|
||||
* Thrown by createAiClient() when the resolved backend has no Anthropic
|
||||
* messages surface. Surfaces that still call the SDK directly (the chat loop,
|
||||
* composer, receipt hunt, WhatsApp interpreter) are unavailable on such a
|
||||
* backend until they move onto getAiService(); routes check
|
||||
* getAiStatus().assistantAvailable first so users get a 503 instead of this.
|
||||
*/
|
||||
export class AiProviderUnsupportedError extends Error {
|
||||
readonly provider: AiProvider
|
||||
constructor(provider: AiProvider) {
|
||||
super(
|
||||
`AI provider "${provider}" has no Anthropic messages surface; use getAiService() from @/lib/ai instead of createAiClient()`
|
||||
)
|
||||
this.name = 'AiProviderUnsupportedError'
|
||||
this.provider = provider
|
||||
}
|
||||
}
|
||||
|
||||
export type AiClient = Anthropic | AnthropicBedrock
|
||||
|
||||
@@ -29,6 +55,8 @@ export type AiClient = Anthropic | AnthropicBedrock
|
||||
* silently move production inference out of eu-north-1.
|
||||
* 3. Otherwise an Anthropic key means the direct API. This is the
|
||||
* self-hosted path.
|
||||
* 3b. Otherwise an OpenAI-compatible base URL + key means that endpoint.
|
||||
* This is the sovereign self-hosted path (BYO Swedish provider).
|
||||
* 4. Otherwise Bedrock without static keys, so the AWS credential provider
|
||||
* chain (instance profile, IRSA, EKS pod identity) still resolves on
|
||||
* hosted infrastructure that injects credentials rather than setting env
|
||||
@@ -38,10 +66,13 @@ export type AiClient = Anthropic | AnthropicBedrock
|
||||
*/
|
||||
export function resolveAiProvider(): AiProvider {
|
||||
const explicit = (process.env.AI_PROVIDER ?? '').trim().toLowerCase()
|
||||
if (explicit === 'bedrock' || explicit === 'anthropic') return explicit
|
||||
if (explicit === 'bedrock' || explicit === 'anthropic' || explicit === 'openai-compatible') {
|
||||
return explicit
|
||||
}
|
||||
|
||||
if (process.env.AWS_ACCESS_KEY_ID && process.env.AWS_SECRET_ACCESS_KEY) return 'bedrock'
|
||||
if (process.env.ANTHROPIC_API_KEY) return 'anthropic'
|
||||
if (process.env.AI_BASE_URL) return 'openai-compatible'
|
||||
return 'bedrock'
|
||||
}
|
||||
|
||||
@@ -55,18 +86,28 @@ export function resolveAiProvider(): AiProvider {
|
||||
* option existed.
|
||||
*/
|
||||
export function hasAiCredentials(): boolean {
|
||||
return resolveAiProvider() === 'anthropic'
|
||||
? !!process.env.ANTHROPIC_API_KEY
|
||||
: !!(process.env.AWS_ACCESS_KEY_ID && process.env.AWS_SECRET_ACCESS_KEY)
|
||||
const provider = resolveAiProvider()
|
||||
if (provider === 'anthropic') return !!process.env.ANTHROPIC_API_KEY
|
||||
// AI_API_KEY is optional: a local OpenAI-compatible server (llama.cpp,
|
||||
// Ollama, LM Studio, vLLM) usually has no auth, so a base URL alone counts
|
||||
// as configured. When a hosted Swedish provider needs a key, the operator
|
||||
// sets AI_API_KEY and the service sends it as a Bearer token.
|
||||
if (provider === 'openai-compatible') return !!process.env.AI_BASE_URL
|
||||
return !!(process.env.AWS_ACCESS_KEY_ID && process.env.AWS_SECRET_ACCESS_KEY)
|
||||
}
|
||||
|
||||
/**
|
||||
* Build a client for the resolved provider. Both expose the same
|
||||
* `messages.create` / `messages.stream` surface, which is all this codebase
|
||||
* uses of either SDK.
|
||||
* Build a client for the resolved Anthropic-family provider. Both expose the
|
||||
* same `messages.create` / `messages.stream` surface, which is all the direct
|
||||
* SDK callers in this codebase use. Throws AiProviderUnsupportedError for the
|
||||
* OpenAI-compatible backend: see the class doc. New code should not call this
|
||||
* at all; go through getAiService() from @/lib/ai (antipattern guard
|
||||
* direct-ai-client enforces that outside lib/ai/services).
|
||||
*/
|
||||
export function createAiClient(): AiClient {
|
||||
if (resolveAiProvider() === 'anthropic') {
|
||||
const provider = resolveAiProvider()
|
||||
if (provider === 'openai-compatible') throw new AiProviderUnsupportedError(provider)
|
||||
if (provider === 'anthropic') {
|
||||
const apiKey = process.env.ANTHROPIC_API_KEY
|
||||
// Omit the key when unset so the SDK resolves it itself and fails at call
|
||||
// time: throwing here would take down every route that merely imports a
|
||||
@@ -97,7 +138,7 @@ export function createAiClient(): AiClient {
|
||||
* operator-supplied override in either form keeps working.
|
||||
*/
|
||||
export function toProviderModelId(bareModelId: string, provider = resolveAiProvider()): string {
|
||||
if (provider === 'anthropic') return bareModelId
|
||||
if (provider === 'anthropic' || provider === 'openai-compatible') return bareModelId
|
||||
if (bareModelId.startsWith('eu.') || bareModelId.startsWith('anthropic.')) return bareModelId
|
||||
return `eu.anthropic.${bareModelId}`
|
||||
}
|
||||
@@ -110,8 +151,12 @@ export function toProviderModelId(bareModelId: string, provider = resolveAiProvi
|
||||
* any part of a secret.
|
||||
*/
|
||||
export function aiCredentialPrefix(): string | null {
|
||||
if (resolveAiProvider() === 'anthropic') {
|
||||
const provider = resolveAiProvider()
|
||||
if (provider === 'anthropic') {
|
||||
return process.env.ANTHROPIC_API_KEY?.slice(0, 12) ?? null
|
||||
}
|
||||
// OpenAI-compatible keys have no standard public prefix, so there is no
|
||||
// non-secret slice to log: identify the endpoint instead.
|
||||
if (provider === 'openai-compatible') return null
|
||||
return process.env.AWS_ACCESS_KEY_ID?.slice(0, 4) ?? null
|
||||
}
|
||||
|
||||
@@ -0,0 +1,88 @@
|
||||
import { execFile } from 'node:child_process'
|
||||
import { mkdtemp, readdir, readFile, rm, writeFile } from 'node:fs/promises'
|
||||
import { tmpdir } from 'node:os'
|
||||
import { join } from 'node:path'
|
||||
import { promisify } from 'node:util'
|
||||
|
||||
const execFileAsync = promisify(execFile)
|
||||
|
||||
/**
|
||||
* PDF -> page images for backends that cannot read PDF bytes natively (most
|
||||
* OpenAI-compatible endpoints: the OpenAI `file` content part is not part of
|
||||
* the de-facto chat-completions standard the Swedish providers implement).
|
||||
*
|
||||
* Uses poppler's `pdftoppm`, a system binary the self-host Docker image
|
||||
* installs (hosted runs Claude on Bedrock, which reads PDFs natively and
|
||||
* never needs this). A missing binary is a configuration state, not a crash:
|
||||
* callers stamp `skipped:pdf_rasterizer_missing` and move on.
|
||||
*
|
||||
* Rejected alternatives: pdfjs-dist + @napi-rs/canvas (two new npm
|
||||
* dependencies, memory spikes on large scans, dead weight on hosted).
|
||||
*/
|
||||
|
||||
export type RasterizePdfResult =
|
||||
| { ok: true; pages: Buffer[]; mediaType: 'image/png'; pageCount: number }
|
||||
| { ok: false; reason: 'rasterizer_missing' | 'failed'; error?: string }
|
||||
|
||||
export interface RasterizePdfOptions {
|
||||
/** Pages rendered from the start of the document; invoice data sits on the first page(s). */
|
||||
maxPages: number
|
||||
/** Render resolution. 110 dpi keeps an A4 page under 1000x1300 px: readable, cheap in tokens. */
|
||||
dpi?: number
|
||||
/** Binary name or path; overridable for tests and unusual installs. */
|
||||
binary?: string
|
||||
timeoutMs?: number
|
||||
}
|
||||
|
||||
const DEFAULT_DPI = 110
|
||||
const DEFAULT_TIMEOUT_MS = 60_000
|
||||
|
||||
function pageNumberOf(fileName: string): number {
|
||||
// pdftoppm names pages <prefix>-1.png, <prefix>-01.png, <prefix>-001.png
|
||||
// depending on the page count's digit width. Sort by the numeric suffix.
|
||||
const match = /-(\d+)\.png$/.exec(fileName)
|
||||
return match ? Number(match[1]) : Number.MAX_SAFE_INTEGER
|
||||
}
|
||||
|
||||
export async function rasterizePdf(
|
||||
pdf: Buffer,
|
||||
opts: RasterizePdfOptions
|
||||
): Promise<RasterizePdfResult> {
|
||||
const binary = opts.binary ?? process.env.AI_PDF_RASTERIZER_BIN ?? 'pdftoppm'
|
||||
const dpi = opts.dpi ?? DEFAULT_DPI
|
||||
const maxPages = Math.max(1, Math.floor(opts.maxPages))
|
||||
|
||||
let dir: string | null = null
|
||||
try {
|
||||
dir = await mkdtemp(join(tmpdir(), 'accounted-pdf-'))
|
||||
const input = join(dir, 'input.pdf')
|
||||
const prefix = join(dir, 'page')
|
||||
await writeFile(input, pdf)
|
||||
|
||||
await execFileAsync(
|
||||
binary,
|
||||
['-r', String(dpi), '-png', '-f', '1', '-l', String(maxPages), input, prefix],
|
||||
{ timeout: opts.timeoutMs ?? DEFAULT_TIMEOUT_MS, maxBuffer: 1024 * 1024 }
|
||||
)
|
||||
|
||||
const names = (await readdir(dir))
|
||||
.filter((n) => n.startsWith('page-') && n.endsWith('.png'))
|
||||
.sort((a, b) => pageNumberOf(a) - pageNumberOf(b))
|
||||
if (names.length === 0) {
|
||||
return { ok: false, reason: 'failed', error: 'pdftoppm produced no pages' }
|
||||
}
|
||||
const pages: Buffer[] = []
|
||||
for (const name of names) pages.push(await readFile(join(dir, name)))
|
||||
return { ok: true, pages, mediaType: 'image/png', pageCount: pages.length }
|
||||
} catch (err) {
|
||||
const code = (err as { code?: string } | null)?.code
|
||||
if (code === 'ENOENT') return { ok: false, reason: 'rasterizer_missing' }
|
||||
return {
|
||||
ok: false,
|
||||
reason: 'failed',
|
||||
error: err instanceof Error ? err.message : String(err),
|
||||
}
|
||||
} finally {
|
||||
if (dir) await rm(dir, { recursive: true, force: true }).catch(() => undefined)
|
||||
}
|
||||
}
|
||||
@@ -0,0 +1,151 @@
|
||||
import type Anthropic from '@anthropic-ai/sdk'
|
||||
import { createAiClient, toProviderModelId, type AiClient } from '../provider'
|
||||
import type { ResolvedAiConfig } from '../config'
|
||||
import { capabilitiesFor } from '../config'
|
||||
import type {
|
||||
AiDocumentInput,
|
||||
AiService,
|
||||
AiTier,
|
||||
AiUsage,
|
||||
ExtractFromDocumentRequest,
|
||||
ExtractFromDocumentResult,
|
||||
GenerateStructuredRequest,
|
||||
GenerateStructuredResult,
|
||||
GenerateTextRequest,
|
||||
GenerateTextResult,
|
||||
} from '../types'
|
||||
|
||||
/**
|
||||
* The Anthropic family (AWS Bedrock and the direct API) behind the job-shaped
|
||||
* interface. Delegates to the existing client factory and builds the EXACT
|
||||
* request literals the call sites built before the abstraction existed, so
|
||||
* hosted stays byte-identical on the wire: the request-shape tests in
|
||||
* lib/ai/__tests__ deep-equal those literals.
|
||||
*/
|
||||
|
||||
type MessageCreateParams = Anthropic.Messages.MessageCreateParamsNonStreaming
|
||||
|
||||
function usageOf(resp: { usage?: unknown }): AiUsage {
|
||||
const usage = (resp.usage ?? {}) as {
|
||||
input_tokens?: number
|
||||
output_tokens?: number
|
||||
cache_creation_input_tokens?: number | null
|
||||
cache_read_input_tokens?: number | null
|
||||
}
|
||||
return {
|
||||
inputTokens: usage.input_tokens ?? null,
|
||||
outputTokens: usage.output_tokens ?? null,
|
||||
cacheCreationInputTokens: usage.cache_creation_input_tokens ?? null,
|
||||
cacheReadInputTokens: usage.cache_read_input_tokens ?? null,
|
||||
}
|
||||
}
|
||||
|
||||
function textOf(resp: { content: Array<{ type: string; text?: string }> }): string {
|
||||
return resp.content
|
||||
.flatMap((b) => (b.type === 'text' && typeof b.text === 'string' ? [b.text] : []))
|
||||
.join('')
|
||||
.trim()
|
||||
}
|
||||
|
||||
/** User content blocks for one document + trailing instruction. Same literals the inbox extractor sent. */
|
||||
export function buildAnthropicDocumentContent(
|
||||
document: AiDocumentInput,
|
||||
instruction: string
|
||||
): Anthropic.Messages.ContentBlockParam[] {
|
||||
const tail = { type: 'text' as const, text: instruction }
|
||||
if (document.kind === 'text') {
|
||||
return [{ type: 'text' as const, text: document.text }, tail]
|
||||
}
|
||||
const base64 = document.data.toString('base64')
|
||||
if (document.kind === 'pdf') {
|
||||
return [
|
||||
{
|
||||
type: 'document' as const,
|
||||
source: { type: 'base64' as const, media_type: 'application/pdf' as const, data: base64 },
|
||||
},
|
||||
tail,
|
||||
]
|
||||
}
|
||||
return [
|
||||
{
|
||||
type: 'image' as const,
|
||||
source: { type: 'base64' as const, media_type: document.mediaType, data: base64 },
|
||||
},
|
||||
tail,
|
||||
]
|
||||
}
|
||||
|
||||
export function createAnthropicFamilyService(cfg: ResolvedAiConfig): AiService {
|
||||
let client: AiClient | null = null
|
||||
const getClient = (): AiClient => {
|
||||
if (!client) client = createAiClient()
|
||||
return client
|
||||
}
|
||||
const modelFor = (tier: AiTier): string =>
|
||||
// The Anthropic family always has a model (Claude default), so the null
|
||||
// branch is unreachable; keep the fallback so the type stays honest.
|
||||
toProviderModelId(cfg.models[tier] ?? 'claude-sonnet-5', cfg.provider)
|
||||
|
||||
return {
|
||||
provider: cfg.provider,
|
||||
capabilities: capabilitiesFor(cfg),
|
||||
modelFor,
|
||||
|
||||
async generateText(req: GenerateTextRequest): Promise<GenerateTextResult> {
|
||||
const model = modelFor(req.tier)
|
||||
const params: MessageCreateParams = {
|
||||
model,
|
||||
max_tokens: req.maxTokens,
|
||||
...(req.system ? { system: req.system } : {}),
|
||||
messages: [{ role: 'user', content: req.prompt }],
|
||||
}
|
||||
const resp = await getClient().messages.create(params)
|
||||
return { text: textOf(resp), model, usage: usageOf(resp) }
|
||||
},
|
||||
|
||||
async generateStructured(req: GenerateStructuredRequest): Promise<GenerateStructuredResult> {
|
||||
const model = modelFor(req.tier)
|
||||
// Forced tool choice is the reliable way to get schema-shaped output
|
||||
// from Claude (mirrors receipt-hunt's adjudicate/mail-intelligence).
|
||||
const params: MessageCreateParams = {
|
||||
model,
|
||||
max_tokens: req.maxTokens,
|
||||
...(req.system ? { system: req.system } : {}),
|
||||
tools: [
|
||||
{
|
||||
name: req.schema.name,
|
||||
...(req.schema.description ? { description: req.schema.description } : {}),
|
||||
input_schema: req.schema.jsonSchema as Anthropic.Messages.Tool['input_schema'],
|
||||
},
|
||||
],
|
||||
tool_choice: { type: 'tool', name: req.schema.name },
|
||||
messages: [{ role: 'user', content: req.prompt }],
|
||||
}
|
||||
const resp = await getClient().messages.create(params)
|
||||
const toolUse = resp.content.find(
|
||||
(b): b is Anthropic.Messages.ToolUseBlock => b.type === 'tool_use' && b.name === req.schema.name
|
||||
)
|
||||
if (!toolUse) {
|
||||
throw new Error(`Model answered without the forced tool "${req.schema.name}"`)
|
||||
}
|
||||
return { value: toolUse.input, model, usage: usageOf(resp) }
|
||||
},
|
||||
|
||||
async extractFromDocument(req: ExtractFromDocumentRequest): Promise<ExtractFromDocumentResult> {
|
||||
if (!cfg.configured) return { ok: false, skipped: 'ai_unconfigured' }
|
||||
const model = modelFor('extraction')
|
||||
// The system prompt is byte-stable per deploy and a few KB: marking it
|
||||
// ephemeral lets the backend reuse the prompt cache on rapid sequential
|
||||
// extractions (a user uploading a stack of receipts within minutes).
|
||||
// Bedrock honours the short default TTL; the direct API the same.
|
||||
const params: MessageCreateParams = {
|
||||
model,
|
||||
max_tokens: req.maxTokens,
|
||||
system: [{ type: 'text', text: req.system, cache_control: { type: 'ephemeral' } }],
|
||||
messages: [{ role: 'user', content: buildAnthropicDocumentContent(req.document, req.instruction) }],
|
||||
}
|
||||
const resp = await getClient().messages.create(params)
|
||||
return { ok: true, text: textOf(resp), model, usage: usageOf(resp) }
|
||||
},
|
||||
}
|
||||
}
|
||||
@@ -0,0 +1,198 @@
|
||||
import { createOpenAICompatible } from '@ai-sdk/openai-compatible'
|
||||
import { generateText, jsonSchema, Output, type ModelMessage, type UserContent } from 'ai'
|
||||
import { capabilitiesFor, type ResolvedAiConfig } from '../config'
|
||||
import { extractJsonObject } from '../json'
|
||||
import { rasterizePdf } from '../rasterize-pdf'
|
||||
import type {
|
||||
AiDocumentInput,
|
||||
AiService,
|
||||
AiTier,
|
||||
AiUsage,
|
||||
ExtractFromDocumentRequest,
|
||||
ExtractFromDocumentResult,
|
||||
ExtractionSkipReason,
|
||||
GenerateStructuredRequest,
|
||||
GenerateStructuredResult,
|
||||
GenerateTextRequest,
|
||||
GenerateTextResult,
|
||||
} from '../types'
|
||||
|
||||
/**
|
||||
* Any endpoint speaking the OpenAI chat-completions API, through the Vercel
|
||||
* AI SDK's openai-compatible provider. This is the sovereign self-host path:
|
||||
* the operator points AI_BASE_URL + AI_API_KEY at a Swedish inference
|
||||
* provider (Berget AI, evroc, ...) and names the models per tier.
|
||||
*
|
||||
* Scope discipline: the AI SDK is used ONLY here. The hosted Bedrock /
|
||||
* direct-API path stays on the Anthropic SDK (services/anthropic-family.ts)
|
||||
* and no call site imports `ai` directly (antipattern guard direct-ai-client).
|
||||
*
|
||||
* Provider quirks this has to absorb, by design choice:
|
||||
* - PDFs: most such endpoints have no PDF part; the default is to rasterize
|
||||
* the first pages with poppler (AI_PDF_MODE=rasterize). Operators whose
|
||||
* provider accepts the OpenAI `file` part can set AI_PDF_MODE=native.
|
||||
* - Vision: AI_VISION=false declares a text-only model; images and PDFs are
|
||||
* then skipped honestly (`ai_no_vision`) instead of failing with a 400.
|
||||
* HTML mail invoices arrive as text and extract on every model.
|
||||
* - JSON: the default is JSON-in-prose plus the caller's extraction + Zod,
|
||||
* which works everywhere; AI_STRICT_JSON=true opts into response_format
|
||||
* json_schema for providers that enforce it.
|
||||
*/
|
||||
|
||||
export function createOpenAICompatibleService(cfg: ResolvedAiConfig): AiService {
|
||||
const provider = createOpenAICompatible({
|
||||
name: 'accounted-byo',
|
||||
baseURL: cfg.baseUrl ?? '',
|
||||
// Only send a key when one is configured: a keyless local server would
|
||||
// reject or ignore an empty Bearer, and omitting it means no auth header.
|
||||
...(cfg.apiKey ? { apiKey: cfg.apiKey } : {}),
|
||||
supportsStructuredOutputs: cfg.strictJson,
|
||||
})
|
||||
const capabilities = capabilitiesFor(cfg)
|
||||
const modelFor = (tier: AiTier): string => {
|
||||
const id = cfg.models[tier]
|
||||
if (!id) throw new Error(`No AI model configured for tier "${tier}" (set AI_MODEL or AI_${tier.toUpperCase()}_MODEL)`)
|
||||
return id
|
||||
}
|
||||
|
||||
function usageOf(result: { usage: { inputTokens?: number; outputTokens?: number; inputTokenDetails?: { cacheReadTokens?: number; cacheWriteTokens?: number } } }): AiUsage {
|
||||
const u = result.usage
|
||||
return {
|
||||
inputTokens: u.inputTokens ?? null,
|
||||
outputTokens: u.outputTokens ?? null,
|
||||
cacheCreationInputTokens: u.inputTokenDetails?.cacheWriteTokens ?? null,
|
||||
cacheReadInputTokens: u.inputTokenDetails?.cacheReadTokens ?? null,
|
||||
}
|
||||
}
|
||||
|
||||
async function buildUserContent(
|
||||
document: AiDocumentInput,
|
||||
instruction: string
|
||||
): Promise<
|
||||
| { ok: true; content: UserContent; pagesRasterized?: number }
|
||||
| { ok: false; skipped: ExtractionSkipReason }
|
||||
> {
|
||||
const tail = { type: 'text' as const, text: instruction }
|
||||
if (document.kind === 'text') {
|
||||
return { ok: true, content: [{ type: 'text', text: document.text }, tail] }
|
||||
}
|
||||
if (!capabilities.imageInput) return { ok: false, skipped: 'ai_no_vision' }
|
||||
if (document.kind === 'image') {
|
||||
return {
|
||||
ok: true,
|
||||
content: [{ type: 'image', image: document.data, mediaType: document.mediaType }, tail],
|
||||
}
|
||||
}
|
||||
// PDF
|
||||
if (capabilities.pdfNative) {
|
||||
return {
|
||||
ok: true,
|
||||
content: [
|
||||
{
|
||||
type: 'file',
|
||||
data: document.data,
|
||||
mediaType: 'application/pdf',
|
||||
...(document.fileName ? { filename: document.fileName } : {}),
|
||||
},
|
||||
tail,
|
||||
],
|
||||
}
|
||||
}
|
||||
const raster = await rasterizePdf(document.data, { maxPages: cfg.pdfMaxPages })
|
||||
if (!raster.ok) {
|
||||
return {
|
||||
ok: false,
|
||||
skipped: raster.reason === 'rasterizer_missing' ? 'pdf_rasterizer_missing' : 'pdf_rasterize_failed',
|
||||
}
|
||||
}
|
||||
return {
|
||||
ok: true,
|
||||
pagesRasterized: raster.pageCount,
|
||||
content: [
|
||||
...raster.pages.map((page) => ({ type: 'image' as const, image: page, mediaType: raster.mediaType })),
|
||||
tail,
|
||||
],
|
||||
}
|
||||
}
|
||||
|
||||
return {
|
||||
provider: cfg.provider,
|
||||
capabilities,
|
||||
modelFor,
|
||||
|
||||
async generateText(req: GenerateTextRequest): Promise<GenerateTextResult> {
|
||||
const model = modelFor(req.tier)
|
||||
const result = await generateText({
|
||||
model: provider(model),
|
||||
...(req.system ? { system: req.system } : {}),
|
||||
prompt: req.prompt,
|
||||
maxOutputTokens: req.maxTokens,
|
||||
})
|
||||
return { text: result.text.trim(), model, usage: usageOf(result) }
|
||||
},
|
||||
|
||||
async generateStructured(req: GenerateStructuredRequest): Promise<GenerateStructuredResult> {
|
||||
const model = modelFor(req.tier)
|
||||
if (cfg.strictJson) {
|
||||
const result = await generateText({
|
||||
model: provider(model),
|
||||
...(req.system ? { system: req.system } : {}),
|
||||
prompt: req.prompt,
|
||||
maxOutputTokens: req.maxTokens,
|
||||
output: Output.object({ schema: jsonSchema<Record<string, unknown>>(req.schema.jsonSchema) }),
|
||||
})
|
||||
return { value: result.output, model, usage: usageOf(result) }
|
||||
}
|
||||
// Prose JSON: ask for the shape in the prompt, then pull the first
|
||||
// parseable object out of whatever the model wrapped it in.
|
||||
const schemaHint =
|
||||
`Answer with ONLY a single JSON object${req.schema.description ? ` (${req.schema.description})` : ''}` +
|
||||
` matching this JSON Schema, no prose, no markdown fences:\n${JSON.stringify(req.schema.jsonSchema)}`
|
||||
const result = await generateText({
|
||||
model: provider(model),
|
||||
system: req.system ? `${req.system}\n\n${schemaHint}` : schemaHint,
|
||||
prompt: req.prompt,
|
||||
maxOutputTokens: req.maxTokens,
|
||||
})
|
||||
const value: unknown = JSON.parse(extractJsonObject(result.text))
|
||||
return { value, model, usage: usageOf(result) }
|
||||
},
|
||||
|
||||
async extractFromDocument(req: ExtractFromDocumentRequest): Promise<ExtractFromDocumentResult> {
|
||||
if (!cfg.configured) return { ok: false, skipped: 'ai_unconfigured' }
|
||||
const model = modelFor('extraction')
|
||||
const built = await buildUserContent(req.document, req.instruction)
|
||||
if (!built.ok) return { ok: false, skipped: built.skipped }
|
||||
const messages: ModelMessage[] = [{ role: 'user', content: built.content }]
|
||||
if (cfg.strictJson && req.jsonSchema) {
|
||||
const result = await generateText({
|
||||
model: provider(model),
|
||||
system: req.system,
|
||||
messages,
|
||||
maxOutputTokens: req.maxTokens,
|
||||
output: Output.object({ schema: jsonSchema<Record<string, unknown>>(req.jsonSchema) }),
|
||||
})
|
||||
return {
|
||||
ok: true,
|
||||
text: JSON.stringify(result.output),
|
||||
model,
|
||||
usage: usageOf(result),
|
||||
...(built.pagesRasterized ? { pagesRasterized: built.pagesRasterized } : {}),
|
||||
}
|
||||
}
|
||||
const result = await generateText({
|
||||
model: provider(model),
|
||||
system: req.system,
|
||||
messages,
|
||||
maxOutputTokens: req.maxTokens,
|
||||
})
|
||||
return {
|
||||
ok: true,
|
||||
text: result.text.trim(),
|
||||
model,
|
||||
usage: usageOf(result),
|
||||
...(built.pagesRasterized ? { pagesRasterized: built.pagesRasterized } : {}),
|
||||
}
|
||||
},
|
||||
}
|
||||
}
|
||||
+135
@@ -0,0 +1,135 @@
|
||||
// Job-shaped AI service interface.
|
||||
//
|
||||
// Call sites describe WHAT they need (a text answer, a schema-shaped object,
|
||||
// the fields read out of a document) rather than HOW a particular backend is
|
||||
// spoken to. The Anthropic-family service (Bedrock and the direct API) keeps
|
||||
// hosted byte-identical by delegating to the existing client factory in
|
||||
// lib/ai/provider.ts; the OpenAI-compatible service talks to any endpoint
|
||||
// that implements the chat-completions API (the Swedish inference providers a
|
||||
// sovereign self-host points at) through the Vercel AI SDK.
|
||||
//
|
||||
// Streaming members (the chat loop) are deliberately absent until the chat
|
||||
// runtime decision is taken: see the Sovereign plan, alignment rule R3.
|
||||
|
||||
export type AiProviderKind = 'bedrock' | 'anthropic' | 'openai-compatible'
|
||||
|
||||
/**
|
||||
* Model tiers. `assistant` is the conversational/default tier, `heavy` the
|
||||
* deep-reasoning tier (supplier-invoice review, VAT review, bokslut), and
|
||||
* `extraction` the document-reading tier (a vision model on OpenAI-compatible
|
||||
* endpoints; Claude reads PDFs natively).
|
||||
*/
|
||||
export type AiTier = 'assistant' | 'heavy' | 'extraction'
|
||||
|
||||
export interface AiCapabilities {
|
||||
/** PDF bytes can be sent as a native document part, no rasterization. */
|
||||
pdfNative: boolean
|
||||
/** Images (and therefore scanned receipts) can be read at all. */
|
||||
imageInput: boolean
|
||||
toolUse: boolean
|
||||
/** The backend can be forced to answer with one named tool. */
|
||||
forcedToolChoice: boolean
|
||||
/** The backend enforces a JSON schema on the output (response_format). */
|
||||
strictJsonSchema: boolean
|
||||
}
|
||||
|
||||
export type AiImageMediaType = 'image/jpeg' | 'image/png' | 'image/webp' | 'image/gif'
|
||||
|
||||
export type AiDocumentInput =
|
||||
| { kind: 'pdf'; data: Buffer; fileName?: string }
|
||||
| { kind: 'image'; data: Buffer; mediaType: AiImageMediaType }
|
||||
/** Plain text already extracted by the caller (HTML mail invoices). Works on every model, vision or not. */
|
||||
| { kind: 'text'; text: string }
|
||||
|
||||
export interface AiUsage {
|
||||
inputTokens: number | null
|
||||
outputTokens: number | null
|
||||
cacheCreationInputTokens: number | null
|
||||
cacheReadInputTokens: number | null
|
||||
}
|
||||
|
||||
export interface GenerateTextRequest {
|
||||
tier: AiTier
|
||||
system?: string
|
||||
prompt: string
|
||||
maxTokens: number
|
||||
}
|
||||
|
||||
export interface GenerateTextResult {
|
||||
text: string
|
||||
model: string
|
||||
usage: AiUsage
|
||||
}
|
||||
|
||||
export interface GenerateStructuredRequest {
|
||||
tier: AiTier
|
||||
system?: string
|
||||
prompt: string
|
||||
maxTokens: number
|
||||
schema: {
|
||||
name: string
|
||||
description?: string
|
||||
/** JSON Schema (draft-07 subset) for the expected object. Hand-maintained by the caller. */
|
||||
jsonSchema: Record<string, unknown>
|
||||
}
|
||||
}
|
||||
|
||||
export interface GenerateStructuredResult {
|
||||
/** The model's object, NOT validated: callers run their own Zod parse. */
|
||||
value: unknown
|
||||
model: string
|
||||
usage: AiUsage
|
||||
}
|
||||
|
||||
export interface ExtractFromDocumentRequest {
|
||||
document: AiDocumentInput
|
||||
/** Byte-stable system prompt. The Anthropic-family service marks it as a prompt-cache breakpoint. */
|
||||
system: string
|
||||
/** Trailing user instruction placed after the document part(s). */
|
||||
instruction: string
|
||||
maxTokens: number
|
||||
/**
|
||||
* Optional JSON schema for the answer. Used only when strict JSON mode is
|
||||
* on AND the backend supports it; otherwise the model answers in prose and
|
||||
* the caller's JSON extraction + Zod parse do the work (works everywhere).
|
||||
*/
|
||||
jsonSchema?: Record<string, unknown>
|
||||
}
|
||||
|
||||
export type ExtractionSkipReason =
|
||||
| 'ai_unconfigured'
|
||||
| 'ai_no_vision'
|
||||
| 'pdf_rasterizer_missing'
|
||||
| 'pdf_rasterize_failed'
|
||||
|
||||
export type ExtractFromDocumentResult =
|
||||
| { ok: true; text: string; model: string; usage: AiUsage; pagesRasterized?: number }
|
||||
| { ok: false; skipped: ExtractionSkipReason }
|
||||
|
||||
export interface AiService {
|
||||
readonly provider: AiProviderKind
|
||||
readonly capabilities: AiCapabilities
|
||||
/** Provider-form model id for a tier (Bedrock inference-profile prefix applied, etc.). */
|
||||
modelFor(tier: AiTier): string
|
||||
generateText(req: GenerateTextRequest): Promise<GenerateTextResult>
|
||||
generateStructured(req: GenerateStructuredRequest): Promise<GenerateStructuredResult>
|
||||
extractFromDocument(req: ExtractFromDocumentRequest): Promise<ExtractFromDocumentResult>
|
||||
}
|
||||
|
||||
export type AiPdfMode = 'native' | 'rasterize'
|
||||
|
||||
export interface AiStatus {
|
||||
provider: AiProviderKind
|
||||
/** Credentials AND (for OpenAI-compatible) a model id are present. */
|
||||
configured: boolean
|
||||
reason: 'ok' | 'no_credentials' | 'no_model'
|
||||
capabilities: AiCapabilities
|
||||
models: Record<AiTier, string | null>
|
||||
pdfMode: AiPdfMode
|
||||
/**
|
||||
* Whether the in-app assistant (chat loop) can run. The loop still speaks
|
||||
* the Anthropic messages surface directly, so it needs the Anthropic family
|
||||
* until its streaming port lands; extraction and single-call jobs do not.
|
||||
*/
|
||||
assistantAvailable: boolean
|
||||
}
|
||||
@@ -503,7 +503,8 @@ export async function completePendingDocumentUpload(
|
||||
uploadId: string,
|
||||
fileName: string,
|
||||
mimeType: string,
|
||||
now: number = Date.now()
|
||||
now: number = Date.now(),
|
||||
options: { extractionOwner?: 'invoice-inbox' } = {}
|
||||
): Promise<CompletedPendingDocumentUpload> {
|
||||
const serviceClient = createServiceClientNoCookies()
|
||||
const storage = serviceClient.storage.from(DOCUMENTS_BUCKET)
|
||||
@@ -594,7 +595,12 @@ export async function completePendingDocumentUpload(
|
||||
const document = data as DocumentAttachment
|
||||
await eventBus.emit({
|
||||
type: 'document.uploaded',
|
||||
payload: { document, userId, companyId },
|
||||
payload: {
|
||||
document,
|
||||
userId,
|
||||
companyId,
|
||||
...(options.extractionOwner ? { extractionOwner: options.extractionOwner } : {}),
|
||||
},
|
||||
})
|
||||
|
||||
return { document, buffer }
|
||||
@@ -646,6 +652,13 @@ export async function uploadDocument(
|
||||
* WhatsApp intake precedent: the loser stores a copy, nothing corrupts.
|
||||
*/
|
||||
dedupeByContent?: boolean
|
||||
/**
|
||||
* Who runs AI extraction on this document. The invoice inbox extracts
|
||||
* the documents it ingests itself (and mirrors the result onto the
|
||||
* document row), so it declares ownership here and the
|
||||
* document-extraction extension yields. Default: the extension extracts.
|
||||
*/
|
||||
extractionOwner?: 'invoice-inbox'
|
||||
} = {}
|
||||
): Promise<DocumentAttachment & { deduplicated?: boolean }> {
|
||||
await ensureDocumentsBucket()
|
||||
@@ -775,7 +788,12 @@ export async function uploadDocument(
|
||||
|
||||
await eventBus.emit({
|
||||
type: 'document.uploaded',
|
||||
payload: { document: result, userId, companyId },
|
||||
payload: {
|
||||
document: result,
|
||||
userId,
|
||||
companyId,
|
||||
...(metadata.extractionOwner ? { extractionOwner: metadata.extractionOwner } : {}),
|
||||
},
|
||||
})
|
||||
|
||||
return result
|
||||
|
||||
+4
-1
@@ -26,7 +26,10 @@ export type CoreEvent =
|
||||
| { type: 'journal_entry.reversed'; payload: { originalEntry: JournalEntry; reversalEntry: JournalEntry; userId: string; companyId: string } }
|
||||
| { type: 'journal_entry.deleted'; payload: { entryId: string; voucherSeries: string; voucherNumber: number; userId: string; companyId: string } }
|
||||
// Documents
|
||||
| { type: 'document.uploaded'; payload: { document: DocumentAttachment; userId: string; companyId: string } }
|
||||
// extractionOwner: set by the invoice inbox on documents it extracts itself,
|
||||
// so the document-extraction extension yields instead of racing it (the
|
||||
// inbox row does not exist yet when this event fires inside uploadDocument).
|
||||
| { type: 'document.uploaded'; payload: { document: DocumentAttachment; userId: string; companyId: string; extractionOwner?: 'invoice-inbox' } }
|
||||
| { type: 'document.accessed'; payload: { document: { id: string; file_name: string }; userId: string; companyId: string } }
|
||||
| { type: 'document.deleted'; payload: { document: { id: string; file_name: string }; userId: string; companyId: string } }
|
||||
// Invoicing
|
||||
|
||||
@@ -6,11 +6,15 @@ import { useEffect, useState } from 'react'
|
||||
// pipeline completes, fails, or times out. Returns the derived status the
|
||||
// upload UI binds to.
|
||||
//
|
||||
// "Disabled" semantics: if the document-extraction extension isn't enabled
|
||||
// (the column stays NULL forever), we don't know server-side. Instead we
|
||||
// stop polling after EXTRACTION_TIMEOUT_MS and bubble status='disabled' so
|
||||
// the UI can quietly fall back ("Uppladdat" without an AI hint): no scary
|
||||
// error for a feature the customer didn't pay for.
|
||||
// "Disabled" semantics: the server answers 'disabled' as soon as it knows
|
||||
// no extraction will happen (AI not configured on this deployment, company
|
||||
// not entitled, self-generated document: see the route), so the UI can
|
||||
// quietly fall back ("Uppladdat" without an AI hint) on the first poll: no
|
||||
// scary error for a feature the customer didn't pay for, and no 30 s hang
|
||||
// on a self-host without an AI key. The client-side timeout stays as the
|
||||
// last resort for the one case the server cannot see: the
|
||||
// document-extraction extension switched off entirely (the column stays
|
||||
// NULL forever).
|
||||
//
|
||||
// Reasonable timeout: typical extraction takes 2-8s on Sonnet via Bedrock.
|
||||
// 30s is generous and keeps the UX responsive on flaky links.
|
||||
|
||||
Generated
+125
@@ -9,6 +9,7 @@
|
||||
"version": "0.1.0",
|
||||
"license": "AGPL-3.0-or-later",
|
||||
"dependencies": {
|
||||
"@ai-sdk/openai-compatible": "2.0.69",
|
||||
"@anthropic-ai/bedrock-sdk": "0.29.1",
|
||||
"@anthropic-ai/sdk": "0.95.0",
|
||||
"@hookform/resolvers": "^5.4.0",
|
||||
@@ -32,6 +33,7 @@
|
||||
"@upstash/redis": "^1.38.0",
|
||||
"@use-gesture/react": "^10.3.1",
|
||||
"@vercel/speed-insights": "^2.0.0",
|
||||
"ai": "6.0.259",
|
||||
"class-variance-authority": "^0.7.1",
|
||||
"clsx": "^2.1.1",
|
||||
"date-fns": "^4.4.0",
|
||||
@@ -85,6 +87,69 @@
|
||||
"vitest": "^4.1.9"
|
||||
}
|
||||
},
|
||||
"node_modules/@ai-sdk/gateway": {
|
||||
"version": "3.0.177",
|
||||
"resolved": "https://registry.npmjs.org/@ai-sdk/gateway/-/gateway-3.0.177.tgz",
|
||||
"integrity": "sha512-U579dA3K2UcezpzJHjyw6UhctFtzFRKOSIrAgxr77teluPGP7vj8aLJpZvEEPf+JbUqbvsHRKKYZzip/+NQ89A==",
|
||||
"license": "Apache-2.0",
|
||||
"dependencies": {
|
||||
"@ai-sdk/provider": "3.0.15",
|
||||
"@ai-sdk/provider-utils": "4.0.46",
|
||||
"@vercel/oidc": "3.2.0"
|
||||
},
|
||||
"engines": {
|
||||
"node": ">=18"
|
||||
},
|
||||
"peerDependencies": {
|
||||
"zod": "^3.25.76 || ^4.1.8"
|
||||
}
|
||||
},
|
||||
"node_modules/@ai-sdk/openai-compatible": {
|
||||
"version": "2.0.69",
|
||||
"resolved": "https://registry.npmjs.org/@ai-sdk/openai-compatible/-/openai-compatible-2.0.69.tgz",
|
||||
"integrity": "sha512-C99M0T0SpRkcCClmJxbQkpSqGmxLfh3NhTsNF3aNaUQZZ7oXN5sPWi9LGs49X5Q/r9FWxBYeZARXs15xxtGIig==",
|
||||
"license": "Apache-2.0",
|
||||
"dependencies": {
|
||||
"@ai-sdk/provider": "3.0.15",
|
||||
"@ai-sdk/provider-utils": "4.0.46"
|
||||
},
|
||||
"engines": {
|
||||
"node": ">=18"
|
||||
},
|
||||
"peerDependencies": {
|
||||
"zod": "^3.25.76 || ^4.1.8"
|
||||
}
|
||||
},
|
||||
"node_modules/@ai-sdk/provider": {
|
||||
"version": "3.0.15",
|
||||
"resolved": "https://registry.npmjs.org/@ai-sdk/provider/-/provider-3.0.15.tgz",
|
||||
"integrity": "sha512-XeZW1CcDF2GMbH4wejW6xBRI2QCOgnkVYUnxoeDadB1mf85riL2bMUeDoh+6gJ/r4mjNfzUPW8OjLjvwTP0u1Q==",
|
||||
"license": "Apache-2.0",
|
||||
"dependencies": {
|
||||
"json-schema": "^0.4.0"
|
||||
},
|
||||
"engines": {
|
||||
"node": ">=18"
|
||||
}
|
||||
},
|
||||
"node_modules/@ai-sdk/provider-utils": {
|
||||
"version": "4.0.46",
|
||||
"resolved": "https://registry.npmjs.org/@ai-sdk/provider-utils/-/provider-utils-4.0.46.tgz",
|
||||
"integrity": "sha512-tEtld97plCFiYevsJuOkGkeuhQndeMWFBVrJS4AjnbD5AqrNSXRCe0p+BZ3Cju/sxDeeZ9ym3q9YUV8fASA7aQ==",
|
||||
"license": "Apache-2.0",
|
||||
"dependencies": {
|
||||
"@ai-sdk/provider": "3.0.15",
|
||||
"@standard-schema/spec": "^1.1.0",
|
||||
"eventsource-parser": "^3.0.8",
|
||||
"undici": "^6.28.0"
|
||||
},
|
||||
"engines": {
|
||||
"node": ">=18.17"
|
||||
},
|
||||
"peerDependencies": {
|
||||
"zod": "^3.25.76 || ^4.1.8"
|
||||
}
|
||||
},
|
||||
"node_modules/@alloc/quick-lru": {
|
||||
"version": "5.2.0",
|
||||
"resolved": "https://registry.npmjs.org/@alloc/quick-lru/-/quick-lru-5.2.0.tgz",
|
||||
@@ -3455,6 +3520,15 @@
|
||||
"node": ">=12.4.0"
|
||||
}
|
||||
},
|
||||
"node_modules/@opentelemetry/api": {
|
||||
"version": "1.9.1",
|
||||
"resolved": "https://registry.npmjs.org/@opentelemetry/api/-/api-1.9.1.tgz",
|
||||
"integrity": "sha512-gLyJlPHPZYdAk1JENA9LeHejZe1Ti77/pTeFm/nMXmQH/HFZlcS/O2XJB+L8fkbrNSqhdtlvjBVjxwUYanNH5Q==",
|
||||
"license": "Apache-2.0",
|
||||
"engines": {
|
||||
"node": ">=8.0.0"
|
||||
}
|
||||
},
|
||||
"node_modules/@oxc-project/types": {
|
||||
"version": "0.138.0",
|
||||
"resolved": "https://registry.npmjs.org/@oxc-project/types/-/types-0.138.0.tgz",
|
||||
@@ -8382,6 +8456,15 @@
|
||||
"react": ">= 16.8.0"
|
||||
}
|
||||
},
|
||||
"node_modules/@vercel/oidc": {
|
||||
"version": "3.2.0",
|
||||
"resolved": "https://registry.npmjs.org/@vercel/oidc/-/oidc-3.2.0.tgz",
|
||||
"integrity": "sha512-UycprH3T6n3jH0k44NHMa7pnFHGu/N05MjojYr+Mc6I7obkoLIJujSWwin1pCvdy/eOxrI/l3uDLQsmcrOb4ug==",
|
||||
"license": "Apache-2.0",
|
||||
"engines": {
|
||||
"node": ">= 20"
|
||||
}
|
||||
},
|
||||
"node_modules/@vercel/speed-insights": {
|
||||
"version": "2.0.0",
|
||||
"resolved": "https://registry.npmjs.org/@vercel/speed-insights/-/speed-insights-2.0.0.tgz",
|
||||
@@ -8582,6 +8665,24 @@
|
||||
"node": ">= 14"
|
||||
}
|
||||
},
|
||||
"node_modules/ai": {
|
||||
"version": "6.0.259",
|
||||
"resolved": "https://registry.npmjs.org/ai/-/ai-6.0.259.tgz",
|
||||
"integrity": "sha512-/KvzoDkzqRjb/wlNFguETnMW4QBqNlAqB9rAi0dl36giF2pXJzTlWZCIONlppCplPbAckhcN3Sd9PvsKfi+7ag==",
|
||||
"license": "Apache-2.0",
|
||||
"dependencies": {
|
||||
"@ai-sdk/gateway": "3.0.177",
|
||||
"@ai-sdk/provider": "3.0.15",
|
||||
"@ai-sdk/provider-utils": "4.0.46",
|
||||
"@opentelemetry/api": "^1.9.0"
|
||||
},
|
||||
"engines": {
|
||||
"node": ">=18"
|
||||
},
|
||||
"peerDependencies": {
|
||||
"zod": "^3.25.76 || ^4.1.8"
|
||||
}
|
||||
},
|
||||
"node_modules/ajv": {
|
||||
"version": "6.12.6",
|
||||
"resolved": "https://registry.npmjs.org/ajv/-/ajv-6.12.6.tgz",
|
||||
@@ -10560,6 +10661,15 @@
|
||||
"node": ">=0.8.x"
|
||||
}
|
||||
},
|
||||
"node_modules/eventsource-parser": {
|
||||
"version": "3.1.1",
|
||||
"resolved": "https://registry.npmjs.org/eventsource-parser/-/eventsource-parser-3.1.1.tgz",
|
||||
"integrity": "sha512-EKN1vKAMcZ8MlYMpaNuxN6R9yakzH6uajHcHVTqWJzvu5pWw9DyhbP35HH8MVBQ+dZjAfDxk+A8NiR9KWaXiyQ==",
|
||||
"license": "MIT",
|
||||
"engines": {
|
||||
"node": ">=18.0.0"
|
||||
}
|
||||
},
|
||||
"node_modules/expect-type": {
|
||||
"version": "1.3.0",
|
||||
"resolved": "https://registry.npmjs.org/expect-type/-/expect-type-1.3.0.tgz",
|
||||
@@ -12056,6 +12166,12 @@
|
||||
"dev": true,
|
||||
"license": "MIT"
|
||||
},
|
||||
"node_modules/json-schema": {
|
||||
"version": "0.4.0",
|
||||
"resolved": "https://registry.npmjs.org/json-schema/-/json-schema-0.4.0.tgz",
|
||||
"integrity": "sha512-es94M3nTIfsEPisRafak+HDLfHXnKBhV3vU5eqPcS3flIWqcxJWgXHXiey3YrpaNsanY5ei1VoYEbOzijuq9BA==",
|
||||
"license": "(AFL-2.1 OR BSD-3-Clause)"
|
||||
},
|
||||
"node_modules/json-schema-to-ts": {
|
||||
"version": "3.1.1",
|
||||
"resolved": "https://registry.npmjs.org/json-schema-to-ts/-/json-schema-to-ts-3.1.1.tgz",
|
||||
@@ -16208,6 +16324,15 @@
|
||||
"integrity": "sha512-Ql87qFHB3s/De2ClA9e0gsnS6zXG27SkTiSJwjCc9MebbfapQfuPzumMIUMi38ezPZVNFcHI9sUIepeQfw8J8Q==",
|
||||
"license": "MIT"
|
||||
},
|
||||
"node_modules/undici": {
|
||||
"version": "6.28.0",
|
||||
"resolved": "https://registry.npmjs.org/undici/-/undici-6.28.0.tgz",
|
||||
"integrity": "sha512-LIY910g9TI13YS95lrMFrs8Rm/u/irgHeTWoKCoteeJ04CUJ92eEfj0rVn+7VKMPBpUPiUoBKfhNyLI23EE/KA==",
|
||||
"license": "MIT",
|
||||
"engines": {
|
||||
"node": ">=18.17"
|
||||
}
|
||||
},
|
||||
"node_modules/undici-types": {
|
||||
"version": "6.21.0",
|
||||
"resolved": "https://registry.npmjs.org/undici-types/-/undici-types-6.21.0.tgz",
|
||||
|
||||
@@ -27,6 +27,7 @@
|
||||
"test:pg": "vitest run --project pg-real"
|
||||
},
|
||||
"dependencies": {
|
||||
"@ai-sdk/openai-compatible": "2.0.69",
|
||||
"@anthropic-ai/bedrock-sdk": "0.29.1",
|
||||
"@anthropic-ai/sdk": "0.95.0",
|
||||
"@hookform/resolvers": "^5.4.0",
|
||||
@@ -50,6 +51,7 @@
|
||||
"@upstash/redis": "^1.38.0",
|
||||
"@use-gesture/react": "^10.3.1",
|
||||
"@vercel/speed-insights": "^2.0.0",
|
||||
"ai": "6.0.259",
|
||||
"class-variance-authority": "^0.7.1",
|
||||
"clsx": "^2.1.1",
|
||||
"date-fns": "^4.4.0",
|
||||
|
||||
@@ -102,6 +102,13 @@
|
||||
*
|
||||
* Usage:
|
||||
* node scripts/checks/no-new-antipatterns.mjs # check (CI)
|
||||
* 11. direct-ai-client: a file outside lib/ai that imports createAiClient,
|
||||
* calls `.messages.create/stream(` on an Anthropic client, or imports
|
||||
* the Vercel AI SDK. Every model call goes through getAiService() so the
|
||||
* backend (Bedrock on hosted, a Swedish OpenAI-compatible endpoint on a
|
||||
* sovereign self-host) stays an environment decision. Allowlist of the
|
||||
* pre-abstraction call sites in this file, may only shrink.
|
||||
*
|
||||
* node scripts/checks/no-new-antipatterns.mjs --update # re-baseline after a migration ratchets the count down
|
||||
*
|
||||
* Exit code 1 if either check regressed past its baseline.
|
||||
@@ -516,6 +523,51 @@ function findFoldedPublicFlags() {
|
||||
// - MockDataImportDialog: CSV preview built on the Table primitive, which
|
||||
// self-wraps in overflow-auto (components/ui/table.tsx).
|
||||
// - PaymentFileDialog: payment-line table wrapped in an overflow-x-auto div.
|
||||
// 11. direct-ai-client. Every model call goes through the job-shaped service
|
||||
// in lib/ai (getAiService): that is what lets hosted stay on Bedrock while a
|
||||
// sovereign self-host points at an OpenAI-compatible Swedish endpoint, and
|
||||
// what stops new AI surfaces from hard-wiring one SDK. Outside lib/ai/, a
|
||||
// file may not import createAiClient, call `.messages.create/stream(` on an
|
||||
// Anthropic client, or import the Vercel AI SDK (`ai`, `@ai-sdk/*`). The
|
||||
// allowlist is the pre-abstraction call sites that still speak the Anthropic
|
||||
// SDK directly (chat loop, composer, receipt hunt, WhatsApp interpreter, the
|
||||
// legacy smoke script); it may only shrink as they migrate or are deleted.
|
||||
const DIRECT_AI_CLIENT_ALLOWED = new Set([
|
||||
'lib/agent/chat/run-turn.ts',
|
||||
'lib/agent/composer/atom-selection.ts',
|
||||
'lib/agent/composer/client.ts',
|
||||
'lib/agent/composer/narrative.ts',
|
||||
'lib/agent/composer/prewarm.ts',
|
||||
'lib/receipt-hunt/adjudicate.ts',
|
||||
'lib/receipt-hunt/mail-intelligence.ts',
|
||||
'extensions/general/whatsapp-inbox/lib/interpret-answer.ts',
|
||||
'scripts/smoke-ai.ts',
|
||||
// Out-of-tree CI reviewer with its own pinned SDK install (see the
|
||||
// compliance workflow); deliberately not part of the app's AI layer.
|
||||
'scripts/swedish-compliance-review.mjs',
|
||||
])
|
||||
const DIRECT_AI_CLIENT_RES = [
|
||||
{ rule: 'createAiClient-import', re: /import[^;]*\bcreateAiClient\b[^;]*from\s+['"]@\/lib\/ai\/provider['"]/ },
|
||||
{ rule: 'anthropic-messages-call', re: /\.messages\.(create|stream)\(/ },
|
||||
{ rule: 'ai-sdk-import', re: /from\s+['"](ai|ai\/[\w-]+|@ai-sdk\/[\w-]+)['"]/ },
|
||||
]
|
||||
|
||||
function findDirectAiClients() {
|
||||
const out = []
|
||||
for (const dir of ['lib', 'app', 'extensions', 'components', 'scripts']) {
|
||||
for (const file of walk(path.join(ROOT, dir), ['.ts', '.tsx', '.mjs'])) {
|
||||
const r = rel(file)
|
||||
if (r.startsWith('lib/ai/')) continue
|
||||
if (r.includes('/__tests__/') || r.endsWith('.test.ts') || r.endsWith('.test.tsx')) continue
|
||||
const src = fs.readFileSync(file, 'utf8')
|
||||
for (const { rule, re } of DIRECT_AI_CLIENT_RES) {
|
||||
if (re.test(src)) out.push({ file: r, rule })
|
||||
}
|
||||
}
|
||||
}
|
||||
return out
|
||||
}
|
||||
|
||||
const DIALOG_NOWRAP_ALLOWED = new Set([
|
||||
'components/extensions/shared/MockDataImportDialog.tsx',
|
||||
'components/supplier-invoices/PaymentFileDialog.tsx',
|
||||
@@ -600,6 +652,27 @@ const PINNED_DEPS = [
|
||||
'"request ended without sending any chunks", taking down the AI assistant + invoice OCR. ' +
|
||||
'Keep 0.29.1 until 0.32.x streaming is verified against Bedrock.',
|
||||
},
|
||||
{
|
||||
name: '@anthropic-ai/sdk',
|
||||
version: '0.95.0',
|
||||
reason:
|
||||
'Declared explicitly at the version bedrock-sdk 0.29.1 pulls in transitively (#1406 Tier 1), so ' +
|
||||
'the lockfile dedupes to one copy; a drift here is a second SDK copy and an untested wire surface.',
|
||||
},
|
||||
{
|
||||
name: 'ai',
|
||||
version: '6.0.259',
|
||||
reason:
|
||||
'Vercel AI SDK backs lib/ai/services/openai-compatible.ts (Tier 2 BYO endpoints). Major versions ' +
|
||||
'rename core APIs; upgrades are deliberate PRs with the provider test suite, never a silent bump.',
|
||||
},
|
||||
{
|
||||
name: '@ai-sdk/openai-compatible',
|
||||
version: '2.0.69',
|
||||
reason:
|
||||
'Paired with ai 6.x; the provider package follows its own major cadence and must move together with ' +
|
||||
'the core pin in one reviewed change.',
|
||||
},
|
||||
]
|
||||
|
||||
const escapeRegExp = (s) => s.replace(/[.*+?^${}()|[\]\\]/g, '\\$&')
|
||||
@@ -931,6 +1004,7 @@ const current = {
|
||||
offLadderRadii: findOffLadderRadii(),
|
||||
foldedPublicFlags: findFoldedPublicFlags(),
|
||||
dialogOverflowRisk: findDialogOverflowRisks(),
|
||||
directAiClients: findDirectAiClients(),
|
||||
}
|
||||
|
||||
const dialogOverflowFiles = [...new Set(current.dialogOverflowRisk.map((f) => f.file))].sort()
|
||||
@@ -1093,6 +1167,25 @@ if (current.foldedPublicFlags.length) {
|
||||
)
|
||||
}
|
||||
|
||||
// 1e1c. direct-ai-client: allowlist in this file, may only shrink. A file
|
||||
// outside the allowlist that talks to a model SDK directly is a NEW
|
||||
// violation; allowlisted files that no longer do are reported as progress.
|
||||
const newDirectAi = current.directAiClients.filter((f) => !DIRECT_AI_CLIENT_ALLOWED.has(f.file))
|
||||
const directAiFilesNow = new Set(current.directAiClients.map((f) => f.file))
|
||||
const migratedDirectAi = [...DIRECT_AI_CLIENT_ALLOWED].filter((f) => !directAiFilesNow.has(f))
|
||||
if (newDirectAi.length) {
|
||||
failed = true
|
||||
console.error(
|
||||
`\n✗ direct-ai-client: ${newDirectAi.length} file(s) outside lib/ai talk to a model SDK directly:`,
|
||||
)
|
||||
newDirectAi.forEach((f) => console.error(` ${f.file} (${f.rule})`))
|
||||
console.error(
|
||||
' → use getAiService() from @/lib/ai (generateText / generateStructured / extractFromDocument).\n' +
|
||||
' Hosted and self-host resolve the backend from the environment there; a direct SDK call\n' +
|
||||
' hard-wires one provider and breaks the sovereign self-host path.',
|
||||
)
|
||||
}
|
||||
|
||||
// 1e2. hand-rolled-invariant: counted, may only go down.
|
||||
if (current.handRolledInvariants > (baseline.handRolledInvariants?.count ?? Infinity)) {
|
||||
failed = true
|
||||
@@ -1223,6 +1316,13 @@ if (
|
||||
console.log(` naive-ore-round: -${baseline.naiveOreRound.count - current.naiveOreRound} occurrence(s)`)
|
||||
console.log(' Run with --update to ratchet the baseline down and lock in the gains.')
|
||||
}
|
||||
if (migratedDirectAi.length) {
|
||||
console.log(
|
||||
`\n✓ direct-ai-client progress: ${migratedDirectAi.length} allowlisted file(s) no longer call a model SDK directly.` +
|
||||
' Remove them from DIRECT_AI_CLIENT_ALLOWED in this script to lock it in:',
|
||||
)
|
||||
migratedDirectAi.forEach((f) => console.log(` ${f}`))
|
||||
}
|
||||
if (gatedSinceBaseline.length) {
|
||||
console.log(
|
||||
`\n✓ ungated-extension-route progress: ${gatedSinceBaseline.length} allowlisted route(s) now gated or gone.` +
|
||||
@@ -1236,5 +1336,5 @@ if (failed) {
|
||||
process.exit(1)
|
||||
}
|
||||
console.log(
|
||||
`\n✓ Antipattern guard passed (raw-route-auth: ${current.rawRouteAuth.length}, naive-ore-round: ${current.naiveOreRound}, hand-rolled-invariant: ${current.handRolledInvariants}, ledger-scanning-report: ${current.ledgerScanningReports.length}, direct-jel-insert: 0, leaky-supabase-client: 0, pinned-dep: 0, raw-user-error: 0, sek-labelled-amount: 0, off-ladder-radius: 0, folded-public-flag: 0, cross-extension-import: 0, ungated-extension-route: ${current.extensionRoutes.ungated.length}/${UNGATED_EXTENSION_ROUTES.size} allowlisted, dialog-overflow-risk: ${dialogOverflowFiles.length} file(s)).`,
|
||||
`\n✓ Antipattern guard passed (raw-route-auth: ${current.rawRouteAuth.length}, naive-ore-round: ${current.naiveOreRound}, hand-rolled-invariant: ${current.handRolledInvariants}, ledger-scanning-report: ${current.ledgerScanningReports.length}, direct-jel-insert: 0, leaky-supabase-client: 0, pinned-dep: 0, raw-user-error: 0, sek-labelled-amount: 0, off-ladder-radius: 0, folded-public-flag: 0, cross-extension-import: 0, ungated-extension-route: ${current.extensionRoutes.ungated.length}/${UNGATED_EXTENSION_ROUTES.size} allowlisted, dialog-overflow-risk: ${dialogOverflowFiles.length} file(s), direct-ai-client: ${current.directAiClients.length}/${DIRECT_AI_CLIENT_ALLOWED.size} allowlisted).`,
|
||||
)
|
||||
|
||||
Reference in New Issue
Block a user