feat(ai): job-shaped AI service with OpenAI-compatible backend, extraction-first; stop extracting every inbox document twice (#1740)

* feat(ai): job-shaped AI service with OpenAI-compatible backend, extraction-first; stop extracting every inbox document twice

Sovereign plan WS1 PR1 (#1406 Tier 2, extraction-first, aligned with the
AI surface audit).

lib/ai grows a job-shaped service (generateText / generateStructured /
extractFromDocument; no streaming members yet, see plan rule R3):
- services/anthropic-family delegates to the existing createAiClient()
  and sends the exact request literals the inbox extractor sent before
  (request-shape tests deep-equal them), so hosted Bedrock stays
  byte-identical.
- services/openai-compatible talks to any chat-completions endpoint
  (BYO Swedish provider) via Vercel AI SDK 6.x, exact-pinned and
  guarded: images as parts, PDFs rasterized with poppler (AI_PDF_MODE)
  or sent natively, AI_VISION / AI_STRICT_JSON declared, honest skips
  (ai_no_vision, pdf_rasterizer_missing) instead of fake failures.
- config.ts: AI_PROVIDER/AI_BASE_URL/AI_API_KEY/AI_MODEL and per-tier
  AI_*_MODEL with the legacy BEDROCK_* names kept as the same overrides;
  getAiStatus() is the single source of truth for "is AI wired up".
- provider.ts: openai-compatible in the auto-detect chain (after Bedrock
  and the direct API); createAiClient() refuses it loudly.

Document extraction moves onto the service and gets the audit's fixes:
- Inbox documents were extracted TWICE (pipeline A ran inside
  uploadDocument() before the inbox row existed, so its dedupe branch
  never fired; 3 707 + 1 666 calls / 30 d). The inbox now declares
  extractionOwner on the upload, the extension yields, and the inbox
  mirrors its single outcome onto document_attachments from every
  writer (sync, deferred, attach, retry, MCP).
- Every "no extraction will ever happen" outcome is stamped
  (skipped:no_ai_entitlement / ai_unconfigured / system_generated /
  ...); the status route maps the quiet ones to 'disabled' on the first
  poll instead of a 30 s client timeout. Prod showed 309 of the 327
  never-extracted uploads were the paywall working silently.
- Self-generated documents (our own invoice PDFs, payout files) are no
  longer OCR'd.
- Agent invoke answers 503 ai_unconfigured when the deployment has no
  assistant backend, distinct from the paywall.

Guard: new direct-ai-client antipattern check (shrink-only allowlist of
the pre-abstraction SDK callers) plus exact pins for @anthropic-ai/sdk,
ai and @ai-sdk/openai-compatible.

Verified: 15 958 unit tests green, guards, lint ratchet, typecheck, and a
live smoke against hosted Bedrock through the new service (ping, streamed
tool turn, thinking+cache, PDF extraction).

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>

* feat(ai): make AI_API_KEY optional for OpenAI-compatible endpoints (keyless local model servers)

A local model server (llama.cpp's server, Ollama /v1, LM Studio, vLLM)
usually has no auth. Before, the OpenAI-compatible backend required both
AI_BASE_URL and AI_API_KEY to count as configured, so running Accounted on a
local model meant setting a meaningless placeholder key.

- resolveAiProvider / hasAiCredentials: a base URL alone is now enough.
- services/openai-compatible: only send Authorization: Bearer when AI_API_KEY
  is set, so a keyless server is never handed an empty bearer; a hosted
  provider that needs a key still sets it.
- Docs (SELF-HOSTING Option 3: local-model example, key marked optional),
  DECISIONS.

Verified: with no AI_API_KEY, just AI_BASE_URL + AI_MODEL, getAiStatus()
reports configured=true / provider=openai-compatible (live). lib/ai suite
71 green; tsc, guards, lint clean. Bedrock/Anthropic logic unchanged.

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>

---------

Co-authored-by: Jakob Wennberg <311770904+jakobwennberg-oss@users.noreply.github.com>
Co-authored-by: Claude Fable 5 <noreply@anthropic.com>
This commit is contained in:
Jakob Wennberg
2026-08-20 19:39:08 +02:00
committed by GitHub
co-authored by Claude Fable 5 Jakob Wennberg
parent fbf47649f2
commit c7a75d069d
37 changed files with 2865 additions and 335 deletions
@@ -0,0 +1,158 @@
import { describe, it, expect, vi, beforeEach } from 'vitest'
import { createQueuedMockSupabase } from '@/tests/helpers'
const { supabase, enqueue, reset, findCalls } = createQueuedMockSupabase()
vi.mock('@/lib/supabase/server', () => ({
createServiceClient: () => supabase,
}))
const extractMock = vi.fn()
vi.mock('@/extensions/general/invoice-inbox/lib/extract-invoice-fields', () => ({
extractInvoiceFields: (...args: unknown[]) => extractMock(...args),
}))
const hasCapabilityMock = vi.fn()
vi.mock('@/lib/entitlements/has-capability', () => ({
hasCapability: (...args: unknown[]) => hasCapabilityMock(...args),
}))
const aiStatusMock = vi.fn()
vi.mock('@/lib/ai', () => ({
getAiStatus: () => aiStatusMock(),
}))
import { documentExtractionExtension } from '../index'
const handler = documentExtractionExtension.eventHandlers![0].handler
function doc(overrides: Record<string, unknown> = {}) {
return {
id: 'doc-1',
company_id: 'company-1',
file_name: 'kvitto.pdf',
mime_type: 'application/pdf',
storage_path: 'company-1/user-1/kvitto.pdf',
upload_source: 'file_upload',
...overrides,
}
}
function payload(overrides: Record<string, unknown> = {}, document = doc()) {
return { document, userId: 'user-1', companyId: 'company-1', ...overrides }
}
/** extraction_model of the LAST document_attachments update, or undefined. */
function lastStamp(): string | undefined {
const updates = findCalls('document_attachments', 'update')
const last = updates[updates.length - 1]?.[0] as { extraction_model?: string } | undefined
return last?.extraction_model
}
beforeEach(() => {
vi.clearAllMocks()
reset()
aiStatusMock.mockReturnValue({ configured: true, assistantAvailable: true })
hasCapabilityMock.mockResolvedValue(true)
extractMock.mockResolvedValue({
data: { supplier: { name: 'Elgiganten' } },
rawText: '{"supplier":{"name":"Elgiganten"}}',
model: 'eu.anthropic.claude-sonnet-5',
})
})
describe('document-extraction handler', () => {
// THE dedupe: inbox-owned documents are extracted (and mirrored) by the
// inbox itself. The handler used to race it and pay a second model call.
it('yields entirely when the inbox owns extraction', async () => {
await handler(payload({ extractionOwner: 'invoice-inbox' }))
expect(supabase.from).not.toHaveBeenCalled()
expect(extractMock).not.toHaveBeenCalled()
})
it('stamps unsupported types from the payload without reading the row', async () => {
enqueue({ data: null }) // the stamp update
await handler(payload({}, doc({ mime_type: 'application/json' })))
expect(findCalls('document_attachments', 'select')).toHaveLength(0)
expect(lastStamp()).toBe('skipped:unsupported_mime')
expect(extractMock).not.toHaveBeenCalled()
})
// Our own invoice PDFs, payout files, filings: nothing to read, paid calls
// to waste (on hosted and on a BYO-key self-host).
it('stamps system-generated documents instead of extracting them', async () => {
enqueue({ data: null })
await handler(payload({}, doc({ upload_source: 'system' })))
expect(lastStamp()).toBe('skipped:system_generated')
expect(extractMock).not.toHaveBeenCalled()
})
it('does nothing for a row that was already attempted', async () => {
enqueue({ data: { id: 'doc-1', mime_type: 'application/pdf', storage_path: 'p', extracted_at: '2026-08-20T00:00:00Z' } })
await handler(payload())
expect(findCalls('document_attachments', 'update')).toHaveLength(0)
expect(extractMock).not.toHaveBeenCalled()
})
// Self-host without an AI key: stamp so the status route answers
// 'disabled' on the first poll instead of after a 30 s timeout.
it('stamps ai_unconfigured when the deployment has no AI', async () => {
aiStatusMock.mockReturnValue({ configured: false, assistantAvailable: false })
enqueue({ data: { id: 'doc-1', mime_type: 'application/pdf', storage_path: 'p', extracted_at: null } })
enqueue({ data: null })
await handler(payload())
expect(lastStamp()).toBe('skipped:ai_unconfigured')
expect(hasCapabilityMock).not.toHaveBeenCalled()
expect(extractMock).not.toHaveBeenCalled()
})
// The paywall, made visible: 309 of the 327 never-extracted uploads in a
// 30-day prod window belonged to companies without the ai capability.
it('stamps no_ai_entitlement for companies without the ai capability', async () => {
hasCapabilityMock.mockResolvedValue(false)
enqueue({ data: { id: 'doc-1', mime_type: 'application/pdf', storage_path: 'p', extracted_at: null } })
enqueue({ data: null })
await handler(payload())
expect(lastStamp()).toBe('skipped:no_ai_entitlement')
expect(extractMock).not.toHaveBeenCalled()
})
it('stamps a storage download failure', async () => {
enqueue({ data: { id: 'doc-1', mime_type: 'application/pdf', storage_path: 'p', extracted_at: null } })
enqueue({ data: null })
supabase.storage.from.mockReturnValueOnce({
download: vi.fn().mockResolvedValue({ data: null, error: { message: 'boom' } }),
})
await handler(payload())
expect(lastStamp()).toBe('failed:storage_download')
expect(extractMock).not.toHaveBeenCalled()
})
it('persists the result with the model that answered', async () => {
enqueue({ data: { id: 'doc-1', mime_type: 'application/pdf', storage_path: 'p', extracted_at: null } })
enqueue({ data: null })
await handler(payload())
expect(extractMock).toHaveBeenCalledWith(expect.objectContaining({ mimeType: 'application/pdf', fileName: 'kvitto.pdf' }))
const updates = findCalls('document_attachments', 'update')
expect(updates[updates.length - 1][0]).toMatchObject({
extracted_data: { supplier: { name: 'Elgiganten' } },
extraction_model: 'eu.anthropic.claude-sonnet-5',
})
})
it('stamps the skip reason the extractor reports (no vision, rasterizer missing, ...)', async () => {
extractMock.mockResolvedValue({ data: {}, rawText: null, skipped: 'pdf_rasterizer_missing' })
enqueue({ data: { id: 'doc-1', mime_type: 'application/pdf', storage_path: 'p', extracted_at: null } })
enqueue({ data: null })
await handler(payload())
expect(lastStamp()).toBe('skipped:pdf_rasterizer_missing')
})
it('stamps failed:no_raw_text when the model call produced nothing parseable', async () => {
extractMock.mockResolvedValue({ data: {}, rawText: null })
enqueue({ data: { id: 'doc-1', mime_type: 'application/pdf', storage_path: 'p', extracted_at: null } })
enqueue({ data: null })
await handler(payload())
expect(lastStamp()).toBe('failed:no_raw_text')
})
})
+109 -107
View File
@@ -1,7 +1,7 @@
import type { Extension } from '@/lib/extensions/types'
import type { SupabaseClient } from '@supabase/supabase-js'
import { extractInvoiceFields } from '@/extensions/general/invoice-inbox/lib/extract-invoice-fields'
import { toProviderModelId } from '@/lib/ai/provider'
import { getAiStatus } from '@/lib/ai'
import { hasCapability } from '@/lib/entitlements/has-capability'
import { CAPABILITY } from '@/lib/entitlements/keys'
import { createLogger } from '@/lib/logger'
@@ -10,8 +10,9 @@ import type { DocumentAttachment } from '@/types'
const log = createLogger('document-extraction')
// Mime types we know Claude can read directly via Bedrock. Anything else
// (HEIC, ZIP, TXT, …) is skipped: extracted_at still gets stamped so the
// Mime types the extraction can read (a vision model reads PDFs natively on
// Claude; OpenAI-compatible backends rasterize them). Anything else (HEIC,
// ZIP, TXT, bank JSON, …) is skipped: extracted_at still gets stamped so the
// row is marked as "attempted, not eligible".
const SUPPORTED_MIME_TYPES = new Set([
'application/pdf',
@@ -23,43 +24,65 @@ const SUPPORTED_MIME_TYPES = new Set([
// AI-extraction extension: paid AI tier only.
//
// Subscribes to the existing document.uploaded event bus topic and runs
// Sonnet 4.6 (via Bedrock, reusing invoice-inbox's extractInvoiceFields) on
// every uploaded receipt or invoice. Writes the result to
// Subscribes to the existing document.uploaded event bus topic and runs the
// configured extraction model (reusing invoice-inbox's extractInvoiceFields)
// on every uploaded receipt or invoice. Writes the result to
// document_attachments.extracted_data so the agent intent capture can use
// it without re-asking the user.
//
// Idempotency: skips when extracted_at is already set on the row. Also
// dedupes against invoice_inbox_items.extracted_data: when the inbox
// extension already extracted the same file, we copy its result instead
// of paying for a second Sonnet call.
// Idempotency: skips when extracted_at is already set on the row.
//
// Ownership: documents that enter through the invoice inbox (web upload,
// email, WhatsApp, MCP upload) are extracted by the inbox itself, which
// mirrors its result onto document_attachments when done. The inbox marks
// the upload with `extractionOwner: 'invoice-inbox'` on the event payload
// and this handler yields. It used to race instead: the inbox row did not
// exist yet when this handler ran inside uploadDocument(), so every inbox
// document was extracted twice (two paid model calls per receipt).
//
// Every outcome that means "no extraction will ever happen here" is stamped
// as `skipped:<reason>` so the polling UI stops immediately instead of
// timing out: paywall (`no_ai_entitlement`), no AI configured on this
// deployment (`ai_unconfigured`), self-generated documents (our own invoice
// PDFs, payout files: `system_generated`), unsupported types.
//
// Free tier: disable this extension in extensions.config.json. Uploads
// still work; the agent intent will see null extracted_data and either
// ask the user or call gnubok_get_document_content at chat-time.
//
// See dev_docs/specialized-agent-plan.md (§ paid/free tier note): to be
// authored.
export const documentExtractionExtension: Extension = {
id: 'document-extraction',
name: 'AI document extraction',
version: '1.0.0',
version: '1.1.0',
eventHandlers: [
{
eventType: 'document.uploaded',
handler: async (payload) => {
const { document, companyId } = payload as {
const { document, companyId, extractionOwner } = payload as {
document: DocumentAttachment
userId: string
companyId: string
extractionOwner?: 'invoice-inbox'
}
if (extractionOwner === 'invoice-inbox') return
await extractAndPersist(document, companyId)
},
},
],
}
async function stamp(
supabase: SupabaseClient,
documentId: string,
extractionModel: string
): Promise<void> {
const { error } = await supabase
.from('document_attachments')
.update({ extracted_at: new Date().toISOString(), extraction_model: extractionModel })
.eq('id', documentId)
if (error) log.warn('stamp failed', { doc: documentId, extractionModel, err: error.message })
}
async function extractAndPersist(
document: DocumentAttachment,
companyId: string,
@@ -69,9 +92,22 @@ async function extractAndPersist(
// events have no user context.
const supabase: SupabaseClient = createServiceClient()
// Cheap gates first, straight from the event payload: no DB round trip for
// the thousands of bank-statement JSON documents a sync produces.
const mimeType = (document.mime_type as string | null) ?? null
if (!mimeType || !SUPPORTED_MIME_TYPES.has(mimeType)) {
await stamp(supabase, document.id, 'skipped:unsupported_mime')
return
}
// Documents the system produced itself (our own invoice PDFs, payment
// files, filings) carry nothing to extract; reading them back with a paid
// model is waste, on hosted and doubly so on a BYO-key self-host.
if (document.upload_source === 'system') {
await stamp(supabase, document.id, 'skipped:system_generated')
return
}
// Idempotency guard: never re-extract a row that already has extracted_at.
// Note: the column may be null OR the row may not yet have the new
// schema (legacy supabase types). Fail closed on missing schema.
const { data: existing, error: existingErr } = await supabase
.from('document_attachments')
.select('id, mime_type, storage_path, extracted_at')
@@ -88,104 +124,70 @@ async function extractAndPersist(
return
}
// Dedup against inbox: if invoice-inbox already extracted this exact file
// (same document_id), copy its result to avoid a second AI call. If the
// inbox row marked the upload as skip_extraction=true, the inbox row's
// extracted_data is an empty skeleton: we must stamp the doc with a
// 'skipped:*' model so the client-side useDocumentExtraction hook reports
// 'unsupported' rather than 'succeeded' (otherwise the UI would claim AI
// finished reading a doc it never opened).
const { data: inboxRow } = await supabase
.from('invoice_inbox_items')
.select('extracted_data, extraction_skipped')
.eq('document_id', document.id)
.maybeSingle()
if (inboxRow?.extraction_skipped) {
await supabase
.from('document_attachments')
.update({
extracted_at: new Date().toISOString(),
extraction_model: 'skipped:invoice_inbox_gate',
})
.eq('id', document.id)
// No AI configured on this deployment: stamp and stop. This is the
// self-host "key not set yet" state; the status route reports it as
// disabled on the first poll instead of after a 30 s timeout.
if (!getAiStatus().configured) {
await stamp(supabase, document.id, 'skipped:ai_unconfigured')
return
}
// Paywall: the free/manual tier never triggers paid extraction. Stamp it so
// the row reads "attempted, not entitled" rather than NULL forever (which
// the polling UI could only interpret by timing out).
if (!(await hasCapability(supabase, companyId, CAPABILITY.ai))) {
log.info('extraction skipped, ai capability not entitled', { doc: document.id, companyId })
await stamp(supabase, document.id, 'skipped:no_ai_entitlement')
return
}
// Download the file from Supabase Storage. The bucket is private: the
// service-role client can read any path.
const storagePath = existing.storage_path as string | null
if (!storagePath) {
log.warn('document has no storage_path, skipping', { doc: document.id })
await stamp(supabase, document.id, 'failed:no_storage_path')
return
}
const { data: blob, error: dlErr } = await supabase.storage
.from('documents')
.download(storagePath)
if (dlErr || !blob) {
log.warn('storage download failed', { doc: document.id, err: dlErr?.message })
await stamp(supabase, document.id, 'failed:storage_download')
return
}
const buffer = Buffer.from(await blob.arrayBuffer())
let extractedData: Record<string, unknown> | null = null
let model: string = 'copied-from-invoice-inbox'
if (inboxRow?.extracted_data) {
extractedData = inboxRow.extracted_data as Record<string, unknown>
} else {
const mimeType = existing.mime_type as string | null
if (!mimeType || !SUPPORTED_MIME_TYPES.has(mimeType)) {
// Stamp the attempt so we don't keep retrying unsupported types.
await supabase
.from('document_attachments')
.update({ extracted_at: new Date().toISOString(), extraction_model: 'skipped:unsupported_mime' })
.eq('id', document.id)
let model: string
try {
const { data, rawText, model: usedModel, skipped } = await extractInvoiceFields({
buffer,
mimeType,
fileName: (document.file_name as string) || 'document',
})
// extractInvoiceFields returns an "empty" result on failure rather than
// throwing. `skipped` means no model call was made (and why); a null
// rawText with no skip means the call failed or the JSON parse did.
if (skipped) {
await stamp(supabase, document.id, `skipped:${skipped}`)
return
}
// Download the file from Supabase Storage. The bucket is private: the
// service-role client can read any path.
const storagePath = existing.storage_path as string | null
if (!storagePath) {
log.warn('document has no storage_path, skipping', { doc: document.id })
return
}
const { data: blob, error: dlErr } = await supabase.storage
.from('documents')
.download(storagePath)
if (dlErr || !blob) {
log.warn('storage download failed', { doc: document.id, err: dlErr?.message })
return
}
const buffer = Buffer.from(await blob.arrayBuffer())
if (!(await hasCapability(supabase, companyId, CAPABILITY.ai))) {
log.info('extraction skipped, ai capability not entitled', { doc: document.id, companyId })
return
}
try {
const { data, rawText } = await extractInvoiceFields({
buffer,
mimeType,
fileName: (document.file_name as string) || 'document',
})
// extractInvoiceFields returns an "empty" result on failure rather
// than throwing: distinguish by checking rawText. When rawText is
// null, the call was skipped (creds missing, unsupported type) or
// the JSON parse failed.
if (!rawText) {
await supabase
.from('document_attachments')
.update({
extracted_at: new Date().toISOString(),
extraction_model: 'failed:no_raw_text',
})
.eq('id', document.id)
return
}
extractedData = data as unknown as Record<string, unknown>
model = toProviderModelId(process.env.BEDROCK_MODEL_ID || 'claude-sonnet-5')
} catch (err) {
log.warn('extraction threw', {
doc: document.id,
err: err instanceof Error ? err.message : String(err),
})
await supabase
.from('document_attachments')
.update({
extracted_at: new Date().toISOString(),
extraction_model: 'failed:exception',
})
.eq('id', document.id)
if (!rawText) {
await stamp(supabase, document.id, 'failed:no_raw_text')
return
}
extractedData = data as unknown as Record<string, unknown>
model = usedModel ?? 'unknown'
} catch (err) {
log.warn('extraction threw', {
doc: document.id,
err: err instanceof Error ? err.message : String(err),
})
await stamp(supabase, document.id, 'failed:exception')
return
}
const { error: updateErr } = await supabase
@@ -0,0 +1,53 @@
import { describe, it, expect, vi, beforeEach } from 'vitest'
import { createQueuedMockSupabase } from '@/tests/helpers'
const { supabase, enqueue, reset, findCalls } = createQueuedMockSupabase()
vi.mock('@/lib/auth/api-keys', () => ({
createServiceClientNoCookies: () => supabase,
}))
import { mirrorExtractionToDocument } from '../lib/mirror-extraction'
function lastUpdate() {
const updates = findCalls('document_attachments', 'update')
return updates[updates.length - 1]?.[0] as { extracted_data: unknown; extraction_model: string; extracted_at: string }
}
beforeEach(() => {
vi.clearAllMocks()
reset()
enqueue({ data: null })
})
describe('mirrorExtractionToDocument', () => {
it('writes a successful extraction with the model that answered', async () => {
await mirrorExtractionToDocument('doc-1', {
data: { supplier: { name: 'x' } } as never,
rawText: '{"supplier":{"name":"x"}}',
model: 'google/gemma-4-31B-it',
})
const u = lastUpdate()
expect(u.extracted_data).toEqual({ supplier: { name: 'x' } })
expect(u.extraction_model).toBe('google/gemma-4-31B-it')
expect(u.extracted_at).toBeTruthy()
expect(findCalls('document_attachments', 'eq')[0]).toEqual(['id', 'doc-1'])
})
it('writes skipped:<reason> with no data for inbox and AI skip reasons alike', async () => {
await mirrorExtractionToDocument('doc-1', { data: {} as never, rawText: null, skipped: 'no_ai_entitlement' })
expect(lastUpdate()).toMatchObject({ extracted_data: null, extraction_model: 'skipped:no_ai_entitlement' })
await mirrorExtractionToDocument('doc-1', { data: {} as never, rawText: null, skipped: 'ai_no_vision' })
expect(lastUpdate()).toMatchObject({ extracted_data: null, extraction_model: 'skipped:ai_no_vision' })
})
it('writes failed:no_raw_text when the call yielded nothing parseable', async () => {
await mirrorExtractionToDocument('doc-1', { data: {} as never, rawText: null })
expect(lastUpdate()).toMatchObject({ extracted_data: null, extraction_model: 'failed:no_raw_text' })
})
it('never throws: a failed update is logged and swallowed', async () => {
reset()
enqueue({ error: { message: 'rls says no' } })
await expect(mirrorExtractionToDocument('doc-1', { data: {} as never, rawText: null })).resolves.toBeUndefined()
})
})
@@ -24,6 +24,21 @@ vi.mock('@/lib/core/documents/document-service', () => ({
uploadDocument: vi.fn().mockResolvedValue({ id: 'doc-1' }),
}))
// The staged (deferred) upload path only exists when AI is configured on this
// deployment: an unconfigured one skips synchronously (ai_unconfigured). These
// tests simulate a configured deployment; the model call itself is mocked.
vi.mock('@/lib/ai', () => ({
getAiStatus: () => ({
provider: 'bedrock',
configured: true,
reason: 'ok',
capabilities: { pdfNative: true, imageInput: true, toolUse: true, forcedToolChoice: true, strictJsonSchema: false },
models: { assistant: 'm', heavy: 'm', extraction: 'm' },
pdfMode: 'native',
assistantAvailable: true,
}),
}))
vi.mock('@/lib/rate-limits/inbox', () => ({
checkInboxUploadRateLimit: vi.fn().mockResolvedValue({ ok: true }),
}))
@@ -21,6 +21,21 @@ vi.mock('@/lib/core/documents/document-service', () => ({
uploadDocument: vi.fn().mockResolvedValue({ id: 'doc-1' }),
}))
// The staged (deferred) upload path only exists when AI is configured on this
// deployment: an unconfigured one skips synchronously (ai_unconfigured). These
// tests simulate a configured deployment; the model call itself is mocked.
vi.mock('@/lib/ai', () => ({
getAiStatus: () => ({
provider: 'bedrock',
configured: true,
reason: 'ok',
capabilities: { pdfNative: true, imageInput: true, toolUse: true, forcedToolChoice: true, strictJsonSchema: false },
models: { assistant: 'm', heavy: 'm', extraction: 'm' },
pdfMode: 'native',
assistantAvailable: true,
}),
}))
vi.mock('@/lib/rate-limits/inbox', () => ({
checkInboxUploadRateLimit: vi.fn().mockResolvedValue({ ok: true }),
}))
+21 -3
View File
@@ -6,6 +6,7 @@ import { uploadDocument } from '@/lib/core/documents/document-service'
import { createServiceClient } from '@/lib/supabase/server'
import { matchSupplierId } from '@/lib/suppliers/match-supplier'
import { extractInvoiceFields, ExtractionSchema, emptyResult } from './lib/extract-invoice-fields'
import { mirrorExtractionToDocument } from './lib/mirror-extraction'
import {
uploadAndExtract,
sanitiseFilename,
@@ -697,6 +698,9 @@ export const invoiceInboxExtension: Extension = {
type: file.type,
}, {
upload_source: 'file_upload',
// This route extracts below and mirrors the outcome onto the
// document row; the document-extraction extension must not race it.
extractionOwner: 'invoice-inbox',
})
// Same page handling as /upload (issue #553): long PDFs extract
@@ -726,16 +730,23 @@ export const invoiceInboxExtension: Extension = {
: null
const skipExtraction = skipReason !== null
const { data: extracted } = skipExtraction
? { data: emptyResult() }
const extraction = skipExtraction
? { data: emptyResult(), rawText: null, model: null, skipped: null }
: await extractInvoiceFields({
buffer: Buffer.from(slicedBuffer ?? buffer),
mimeType: file.type,
fileName: file.name,
})
const { data: extracted } = extraction
if (!skipExtraction && slicedBuffer != null && pageCount != null) {
extracted.pages = { total: pageCount, analyzed: MAX_PAGES_FOR_AUTO_EXTRACT }
}
await mirrorExtractionToDocument(doc.id, {
data: extracted,
rawText: extraction.rawText,
model: extraction.model ?? null,
skipped: skipReason ?? extraction.skipped ?? null,
})
const { error: linkError } = await ctx.supabase
.from('invoice_inbox_items')
@@ -1088,11 +1099,18 @@ export const invoiceInboxExtension: Extension = {
try {
const buffer = Buffer.from(await blob.arrayBuffer())
const { data: extracted } = await extractInvoiceFields({
const extraction = await extractInvoiceFields({
buffer,
mimeType: doc.mime_type,
fileName: doc.file_name,
})
const { data: extracted } = extraction
await mirrorExtractionToDocument(item.document_id, {
data: extracted,
rawText: extraction.rawText,
model: extraction.model ?? null,
skipped: extraction.skipped ?? null,
})
// Re-running the extraction has to re-run the match too, or the one
// affordance the user reaches for when an item failed to auto-link
@@ -1,10 +1,12 @@
// AI-driven invoice/receipt field extraction.
//
// Sends the uploaded document directly to Claude Sonnet 4.6 via AWS
// Bedrock and asks for a structured InvoiceExtractionResult JSON. Sonnet
// reads PDFs, images, and scans natively, which the previous regex
// extractor couldn't: that's why English receipts (Anthropic, AWS,
// Stripe, …) and image-only PDFs came back empty.
// Sends the uploaded document to the configured AI backend through the
// job-shaped service in lib/ai (Claude on Bedrock or the direct API on
// hosted; any OpenAI-compatible endpoint, e.g. a Swedish inference provider,
// on a sovereign self-host) and asks for a structured InvoiceExtractionResult
// JSON. Vision models read PDFs, images and scans, which the previous regex
// extractor couldn't: that's why English receipts (Anthropic, AWS, Stripe, …)
// and image-only PDFs came back empty.
//
// The AI output is validated against a Zod schema; anything that doesn't
// parse falls back to an empty result so the inbox row still lands and
@@ -13,30 +15,25 @@
import { createHash } from 'node:crypto'
import { z } from 'zod'
import type { InvoiceExtractionResult } from '@/types'
import { createAiClient, hasAiCredentials, toProviderModelId } from '@/lib/ai/provider'
import { getAiService, readAiConfig, extractJsonObject } from '@/lib/ai'
import type { AiDocumentInput, AiImageMediaType, ExtractionSkipReason } from '@/lib/ai'
import { createLogger } from '@/lib/logger'
// Re-exported for callers and tests that import it from here.
export { extractJsonObject }
const log = createLogger('invoice-inbox-extract')
// Both overridable via env vars so ops can swap models / raise token caps
// without a code deploy. The model id is written bare and adapted to whichever
// backend is configured (Bedrock in eu-north-1 on hosted, the direct Anthropic
// API on self-hosted: see lib/ai/provider.ts). 8192 tokens is enough headroom
// for invoices with 20+ line items.
const MODEL = toProviderModelId(process.env.BEDROCK_MODEL_ID || 'claude-sonnet-5')
const MAX_TOKENS = (() => {
const parsed = Number(process.env.BEDROCK_MAX_TOKENS)
// Use the env value only if it's a positive number: `||` would also
// fall back on a deliberate `0`, masking what is really an invalid
// configuration rather than the intent to disable.
return Number.isFinite(parsed) && parsed > 0 ? parsed : 8192
})()
// Output cap: AI_EXTRACTION_MAX_TOKENS (legacy BEDROCK_MAX_TOKENS) or 8192,
// enough headroom for invoices with 20+ line items. The model id per tier is
// resolved by lib/ai/config.ts (AI_EXTRACTION_MODEL, legacy BEDROCK_MODEL_ID,
// AI_MODEL, then the Claude default on the Anthropic family).
// Bedrock supports these document/image media types directly. HEIC/HEIF
// are not on the list, so we skip AI for those: the inbox row still
// lands and the user can edit fields manually or replace the file.
// text/html (mail-body invoices from the inbound pipeline) is not sent as
// a document block: it is converted to plain text via htmlToText() first.
// Media types the extraction accepts. HEIC/HEIF are not on the list, so we
// skip AI for those: the inbox row still lands and the user can edit fields
// manually or replace the file. text/html (mail-body invoices from the
// inbound pipeline) is not sent as a document block: it is converted to plain
// text via htmlToText() first, which also makes it work on text-only models.
const SUPPORTED_MEDIA_TYPES = new Set([
'application/pdf',
'image/jpeg',
@@ -52,10 +49,16 @@ export interface ExtractionInput {
fileName: string
}
export type ExtractionSkipped = ExtractionSkipReason | 'unsupported_media'
export interface ExtractionOutput {
data: InvoiceExtractionResult
/** The raw JSON string returned by the model, or null on failure. */
/** The raw JSON string returned by the model, or null on failure/skip. */
rawText: string | null
/** Provider-form id of the model that answered, when a call was made. */
model?: string | null
/** Set when no model call was made at all (and why). */
skipped?: ExtractionSkipped | null
}
// Classification fields are nullable AND .catch(null): a hallucinated enum
@@ -244,61 +247,6 @@ Rules:
- lineItems: include every line. Empty array is fine if the document has no itemised lines.
- vatBreakdown: include one entry per distinct VAT rate. Empty array is fine.`
// Sonnet 5 intermittently wraps its answer in markdown fences (```json ... ```)
// or adds prose around it, despite the JSON-only instruction in the system
// prompt. Scan for balanced top-level '{'..'}' candidates (string- and
// escape-aware, so braces inside JSON string values don't end a candidate
// early) and return the first one JSON.parse accepts; prose braces around the
// object form unparseable candidates and are skipped. Returns the input
// unchanged when no candidate parses, so the existing parse-failure path
// handles prose-only refusals. Zod validation downstream still rejects
// well-formed-but-wrong JSON.
// Bounds for the candidate scan below. Real model output is already capped
// by MAX_TOKENS (roughly 33 KB of text at 8192 tokens), so genuine responses
// never come near these; they exist so pathological or adversarially
// brace-laden text cannot make the scan quadratic (compliance review
// A.8.28). Oversized or exhausted inputs fall through to the raw text and
// land in the existing empty-result path.
const MAX_SCAN_INPUT_LENGTH = 256 * 1024
const MAX_CANDIDATE_ATTEMPTS = 50
export function extractJsonObject(raw: string): string {
if (raw.length > MAX_SCAN_INPUT_LENGTH) return raw
let attempts = 0
let start = raw.indexOf('{')
while (start !== -1 && attempts < MAX_CANDIDATE_ATTEMPTS) {
attempts++
let depth = 0
let inString = false
let escaped = false
for (let i = start; i < raw.length; i++) {
const ch = raw[i]
if (inString) {
if (escaped) escaped = false
else if (ch === '\\') escaped = true
else if (ch === '"') inString = false
} else if (ch === '"') {
inString = true
} else if (ch === '{') {
depth++
} else if (ch === '}') {
depth--
if (depth === 0) {
const candidate = raw.slice(start, i + 1)
try {
JSON.parse(candidate)
return candidate
} catch {
break
}
}
}
}
start = raw.indexOf('{', start + 1)
}
return raw
}
export function emptyResult(): InvoiceExtractionResult {
return {
documentKind: null,
@@ -427,44 +375,137 @@ export function htmlToText(html: string): string {
.slice(0, MAX_EXTRACTION_TEXT_LENGTH)
}
function buildContent(input: ExtractionInput) {
const EXTRACTION_INSTRUCTION = 'Extract the fields per the schema. JSON only.'
/** Map an upload to the service's document input. HTML becomes plain text. */
function toDocumentInput(input: ExtractionInput): AiDocumentInput {
if (input.mimeType === 'text/html') {
const text = htmlToText(input.buffer.toString('utf8'))
return [
{
type: 'text' as const,
text: `The document is an HTML email invoice, converted to plain text:\n\n${text}`,
},
{ type: 'text' as const, text: 'Extract the fields per the schema. JSON only.' },
]
return {
kind: 'text',
text: `The document is an HTML email invoice, converted to plain text:\n\n${text}`,
}
}
const base64 = input.buffer.toString('base64')
if (input.mimeType === 'application/pdf') {
return [
{
type: 'document' as const,
source: { type: 'base64' as const, media_type: 'application/pdf' as const, data: base64 },
},
{ type: 'text' as const, text: 'Extract the fields per the schema. JSON only.' },
]
return { kind: 'pdf', data: input.buffer, fileName: input.fileName }
}
return [
{
type: 'image' as const,
source: {
type: 'base64' as const,
media_type: input.mimeType as 'image/jpeg' | 'image/png' | 'image/webp' | 'image/gif',
data: base64,
return { kind: 'image', data: input.buffer, mediaType: input.mimeType as AiImageMediaType }
}
// Hand-maintained JSON-schema mirror of ExtractionSchema, used ONLY when the
// operator opts into strict JSON mode on an OpenAI-compatible endpoint
// (AI_STRICT_JSON=true). Deliberately not auto-converted from the Zod schema:
// .catch()/.transform() have no JSON-schema equivalent and the conversion
// would drift silently. Permissive on purpose (types + nullability only):
// Zod stays the validator, this just keeps the model inside the shape.
const nullable = (type: string) => ({ type: [type, 'null'] })
const EXTRACTION_JSON_SCHEMA: Record<string, unknown> = {
type: 'object',
additionalProperties: false,
properties: {
documentKind: nullable('string'),
merchantCategory: nullable('string'),
legibility: nullable('string'),
purchaseTime: nullable('string'),
payment: {
type: ['object', 'null'],
additionalProperties: false,
properties: { method: nullable('string'), cardLast4: nullable('string') },
required: ['method', 'cardLast4'],
},
supplier: {
type: 'object',
additionalProperties: false,
properties: {
name: nullable('string'),
orgNumber: nullable('string'),
vatNumber: nullable('string'),
address: nullable('string'),
bankgiro: nullable('string'),
plusgiro: nullable('string'),
},
required: ['name', 'orgNumber', 'vatNumber', 'address', 'bankgiro', 'plusgiro'],
},
invoice: {
type: 'object',
additionalProperties: false,
properties: {
invoiceNumber: nullable('string'),
invoiceDate: nullable('string'),
dueDate: nullable('string'),
paymentReference: nullable('string'),
currency: { type: 'string' },
servicePeriodStart: nullable('string'),
servicePeriodEnd: nullable('string'),
},
required: [
'invoiceNumber',
'invoiceDate',
'dueDate',
'paymentReference',
'currency',
'servicePeriodStart',
'servicePeriodEnd',
],
},
lineItems: {
type: 'array',
items: {
type: 'object',
additionalProperties: false,
properties: {
description: { type: 'string' },
quantity: { type: 'number' },
unitPrice: nullable('number'),
lineTotal: { type: 'number' },
vatRate: nullable('number'),
accountSuggestion: { type: 'null' },
},
required: ['description', 'quantity', 'unitPrice', 'lineTotal', 'vatRate', 'accountSuggestion'],
},
},
{ type: 'text' as const, text: 'Extract the fields per the schema. JSON only.' },
]
totals: {
type: 'object',
additionalProperties: false,
properties: {
subtotal: nullable('number'),
vatAmount: nullable('number'),
total: nullable('number'),
roundingAmount: nullable('number'),
},
required: ['subtotal', 'vatAmount', 'total', 'roundingAmount'],
},
vatBreakdown: {
type: 'array',
items: {
type: 'object',
additionalProperties: false,
properties: { rate: { type: 'number' }, base: { type: 'number' }, amount: { type: 'number' } },
required: ['rate', 'base', 'amount'],
},
},
},
required: [
'documentKind',
'merchantCategory',
'legibility',
'purchaseTime',
'payment',
'supplier',
'invoice',
'lineItems',
'totals',
'vatBreakdown',
],
}
/**
* Extract invoice fields by sending the document directly to Claude
* Sonnet 4.6 via AWS Bedrock. Never throws on extraction failure:
* always returns an InvoiceExtractionResult. Empty fields are null.
* Extract invoice fields by sending the document to the configured AI
* backend. Never throws on extraction failure: always returns an
* InvoiceExtractionResult. Empty fields are null. `skipped` is set when no
* model call was made (unsupported file type, AI not configured, no vision
* on this backend, PDF rasterizer missing), so callers can tell "nothing to
* read" from "read and found nothing".
*/
export async function extractInvoiceFields(
rawInput: ExtractionInput
@@ -474,63 +515,48 @@ export async function extractInvoiceFields(
const input = await normalizeImageForExtraction(rawInput)
if (!SUPPORTED_MEDIA_TYPES.has(input.mimeType)) {
return { data: emptyResult(), rawText: null }
return { data: emptyResult(), rawText: null, skipped: 'unsupported_media' }
}
if (!hasAiCredentials()) {
log.warn('AI credentials missing: returning empty extraction', {
file_name_hash: createHash('sha256').update(input.fileName).digest('hex').slice(0, 12),
})
return { data: emptyResult(), rawText: null }
}
const client = createAiClient()
const fileNameHash = createHash('sha256').update(input.fileName).digest('hex').slice(0, 12)
const service = getAiService()
let rawText: string | null = null
let model: string | null = null
try {
// SYSTEM_PROMPT is byte-stable per deploy and ~3.5 KB: marking it as
// ephemeral lets Bedrock reuse the prompt-cache on rapid sequential
// extractions (e.g. a user uploading a stack of receipts within minutes).
// Bedrock supports `{ type: 'ephemeral' }` with the default short TTL;
// the 1h TTL from the agent-native API plan (item 10) requires the direct
// Anthropic API rather than Bedrock and is out of scope here.
const resp = await client.messages.create({
model: MODEL,
max_tokens: MAX_TOKENS,
system: [{ type: 'text', text: SYSTEM_PROMPT, cache_control: { type: 'ephemeral' } }],
messages: [{ role: 'user', content: buildContent(input) }],
const result = await service.extractFromDocument({
document: toDocumentInput(input),
system: SYSTEM_PROMPT,
instruction: EXTRACTION_INSTRUCTION,
maxTokens: readAiConfig().extractionMaxTokens,
jsonSchema: EXTRACTION_JSON_SCHEMA,
})
rawText = resp.content
.flatMap((b) => (b.type === 'text' ? [b.text] : []))
.join('')
.trim()
if (!result.ok) {
// Not a failure: the deployment cannot read this document at all.
// `ai_unconfigured` is the self-host "no key yet" case the 30 s
// upload hang used to hide; the others are honest capability gaps.
log.warn('AI extraction skipped', { file_name_hash: fileNameHash, reason: result.skipped })
return { data: emptyResult(), rawText: null, skipped: result.skipped }
}
rawText = result.text
model = result.model
// Observability for the prompt-cache hit ratio. The agent-native plan
// targets cache_read_input_tokens / total_input_tokens ≥ 0.85 in steady
// state; logging here makes that measurable without a separate dashboard.
const usage = resp.usage as
| {
input_tokens?: number
output_tokens?: number
cache_creation_input_tokens?: number
cache_read_input_tokens?: number
}
| undefined
if (usage) {
// Raw fileName can constitute personal data (e.g. "faktura_Sven_Andersson.pdf").
// Log a short hash so the operator can correlate without exposing PII
// to the log destination (GDPR Art. 5(1)(f)).
const fileNameHash = createHash('sha256').update(input.fileName).digest('hex').slice(0, 12)
log.info('ai_extraction_usage', {
file_name_hash: fileNameHash,
mime_type: input.mimeType,
input_tokens: usage.input_tokens ?? null,
output_tokens: usage.output_tokens ?? null,
cache_creation_input_tokens: usage.cache_creation_input_tokens ?? null,
cache_read_input_tokens: usage.cache_read_input_tokens ?? null,
})
}
// Raw fileName can constitute personal data (e.g. "faktura_Sven_Andersson.pdf").
// Log a short hash so the operator can correlate without exposing PII
// to the log destination (GDPR Art. 5(1)(f)).
log.info('ai_extraction_usage', {
file_name_hash: fileNameHash,
mime_type: input.mimeType,
model,
input_tokens: result.usage.inputTokens,
output_tokens: result.usage.outputTokens,
cache_creation_input_tokens: result.usage.cacheCreationInputTokens,
cache_read_input_tokens: result.usage.cacheReadInputTokens,
...(result.pagesRasterized ? { pages_rasterized: result.pagesRasterized } : {}),
})
const parsed = JSON.parse(extractJsonObject(rawText))
const validated = ExtractionSchema.parse(parsed)
@@ -540,14 +566,15 @@ export async function extractInvoiceFields(
// .transform, so no post-validation coercion is needed.
data: { ...validated, confidence: 1 },
rawText,
model,
}
} catch (err) {
log.warn('AI extraction failed', {
file_name_hash: createHash('sha256').update(input.fileName).digest('hex').slice(0, 12),
file_name_hash: fileNameHash,
mimeType: input.mimeType,
error: err instanceof Error ? err.message : String(err),
hasRawText: rawText != null,
})
return { data: emptyResult(), rawText }
return { data: emptyResult(), rawText, model }
}
}
@@ -0,0 +1,64 @@
import { createServiceClientNoCookies } from '@/lib/auth/api-keys'
import { createLogger } from '@/lib/logger'
import type { InvoiceExtractionResult } from '@/types'
const log = createLogger('invoice-inbox-mirror')
export interface ExtractionOutcome {
/** The parsed result (may be the empty skeleton). Ignored unless `rawText` is set and nothing was skipped. */
data: InvoiceExtractionResult | null
/** Raw model output; null when the call failed or was skipped. */
rawText: string | null
/** Provider-form model id that answered, when a call was made. */
model?: string | null
/** Why no model call was made: inbox skip reasons and lib/ai skip reasons alike. */
skipped?: string | null
}
/**
* Mirror an inbox extraction outcome onto document_attachments
* (extracted_data / extracted_at / extraction_model).
*
* The invoice inbox owns extraction for the documents it ingests, and the
* document-extraction extension yields to it (see extractionOwner on the
* document.uploaded event). Everything that read the document row before
* (the extraction-status poll behind the upload UI, the agent intents'
* "what do we already know" reads) keeps working because the inbox writes
* the same columns the extension would have, with the one model call it
* actually made.
*
* Service-role client, same as the extension: this runs from routes, the
* deferred worker and the MCP server alike, and the write is a system
* side-effect rather than a user action. Never throws: a failed mirror
* leaves the document row unstamped, which the UI already tolerates.
*/
export async function mirrorExtractionToDocument(
documentId: string,
outcome: ExtractionOutcome
): Promise<void> {
try {
const succeeded = !outcome.skipped && outcome.rawText != null && outcome.data != null
const extractionModel = outcome.skipped
? `skipped:${outcome.skipped}`
: succeeded
? outcome.model || 'invoice-inbox'
: 'failed:no_raw_text'
const supabase = createServiceClientNoCookies()
const { error } = await supabase
.from('document_attachments')
.update({
extracted_data: succeeded ? (outcome.data as unknown as Record<string, unknown>) : null,
extracted_at: new Date().toISOString(),
extraction_model: extractionModel,
})
.eq('id', documentId)
if (error) {
log.warn('mirror failed', { doc: documentId, extractionModel, err: error.message })
}
} catch (err) {
log.warn('mirror threw', {
doc: documentId,
err: err instanceof Error ? err.message : String(err),
})
}
}
@@ -1,6 +1,8 @@
import { after } from 'next/server'
import { uploadDocument } from '@/lib/core/documents/document-service'
import { extractInvoiceFields, emptyResult } from './extract-invoice-fields'
import { mirrorExtractionToDocument } from './mirror-extraction'
import { getAiStatus } from '@/lib/ai'
import { hasCapability } from '@/lib/entitlements/has-capability'
import { CAPABILITY } from '@/lib/entitlements/keys'
import { appendProcessingHistory } from '@/lib/processing-history/append'
@@ -10,6 +12,16 @@ import type { InvoiceExtractionResult } from '@/types'
import { PDFDocument } from 'pdf-lib'
import path from 'node:path'
// Verdicts decidable before touching the file. `ai_unconfigured` is the
// deployment-level "no AI backend" state (self-host without a key), distinct
// from the per-company paywall (`no_ai_entitlement`).
type SyncSkipReason =
| 'no_ai_entitlement'
| 'client_opt_out'
| 'sandbox'
| 'ai_unconfigured'
| null
/**
* Defensive filename sanitisation for content arriving from .eml inner
* attachments and rejected-attachment metadata. The document-service already
@@ -231,6 +243,9 @@ export async function uploadAndExtract(
// two inboxes, re-hunted by a sweep, or uploaded twice must not become a
// second archived document.
dedupeByContent: true,
// This function extracts (sync or deferred) and mirrors the outcome onto
// the document row; the document-extraction extension must not race it.
extractionOwner: 'invoice-inbox',
})
if (doc.deduplicated) {
@@ -342,14 +357,20 @@ export async function uploadAndExtract(
// decidable without touching the PDF; too_many_pages is not (it only fires
// when the slice fallback fails) and is resolved below, on whichever path
// (sync or deferred) actually attempts the slice.
const syncSkipReason: 'no_ai_entitlement' | 'client_opt_out' | 'sandbox' | null =
// `ai_unconfigured` (no AI credentials/model on this deployment, the
// self-host "key not set yet" state) is decided up front as well: a call
// that can never succeed must not take the deferred path and leave the row
// in 'processing' until the sweep.
const syncSkipReason: SyncSkipReason =
!hasAiEntitlement
? 'no_ai_entitlement'
: sandbox
? 'sandbox'
: opts.skipExtraction
? 'client_opt_out'
: null
: !getAiStatus().configured
? 'ai_unconfigured'
: null
if (opts.deferExtraction && syncSkipReason === null) {
// Staged path: extraction WILL call Bedrock, so create the row now and
@@ -424,7 +445,7 @@ export async function uploadAndExtract(
// page-count. Opt-out outranks the page gate (an opted-out caller never
// extracts regardless of length), and too_many_pages only fires when the
// slice fallback also failed (encrypted/malformed PDF).
const skipReason: 'no_ai_entitlement' | 'too_many_pages' | 'client_opt_out' | 'sandbox' | null =
const skipReason: SyncSkipReason | 'too_many_pages' =
syncSkipReason ??
(gatedByPageCount && slicedBuffer == null ? 'too_many_pages' : null)
const skipExtraction = skipReason !== null
@@ -434,13 +455,14 @@ export async function uploadAndExtract(
// fields via /items/:id/extracted-data before converting to a supplier
// invoice. extracted_data is never null in the DB; an empty skeleton
// keeps downstream readers (UI, MCP) happy.
const { data: extracted, rawText } = skipExtraction
? { data: emptyResult(), rawText: null }
const extraction = skipExtraction
? { data: emptyResult(), rawText: null, model: null, skipped: null }
: await extractInvoiceFields({
buffer: Buffer.from(slicedBuffer ?? file.buffer),
mimeType: file.type,
fileName: file.name,
})
const { data: extracted, rawText } = extraction
if (!skipExtraction && slicedBuffer != null && pageCount != null) {
extracted.pages = { total: pageCount, analyzed: MAX_PAGES_FOR_AUTO_EXTRACT }
}
@@ -484,6 +506,15 @@ export async function uploadAndExtract(
if (inboxError) throw new Error(`Failed to create inbox item: ${inboxError.message}`)
// The extension yielded to us (extractionOwner); put the outcome where it
// would have written it, so the document row tells the same story.
await mirrorExtractionToDocument(doc.id, {
data: extracted,
rawText,
model: extraction.model ?? null,
skipped: skipReason ?? extraction.skipped ?? null,
})
try {
await appendProcessingHistory({
companyId,
@@ -552,8 +583,9 @@ function scheduleDeferredExtraction(job: DeferredExtractionJob): void {
// an unsliceable long PDF becomes the too_many_pages skip.
let extracted: InvoiceExtractionResult = emptyResult()
let rawText: string | null = null
let model: string | null = null
let extractionSkipped = false
let skipReason: 'too_many_pages' | null = null
let skipReason: string | null = null
try {
const slicedBuffer = job.gatedByPageCount
? await slicePdfForExtraction(job.file.buffer, MAX_PAGES_FOR_AUTO_EXTRACT)
@@ -569,6 +601,14 @@ function scheduleDeferredExtraction(job: DeferredExtractionJob): void {
})
extracted = result.data
rawText = result.rawText
model = result.model ?? null
if (result.skipped) {
// No model call could be made (AI unconfigured, no vision on this
// backend, rasterizer missing). Same UI affordance as the other
// skips: the row lands with the empty skeleton and the hint.
extractionSkipped = true
skipReason = result.skipped
}
if (slicedBuffer != null && job.pageCount != null) {
extracted.pages = { total: job.pageCount, analyzed: MAX_PAGES_FOR_AUTO_EXTRACT }
}
@@ -608,6 +648,13 @@ function scheduleDeferredExtraction(job: DeferredExtractionJob): void {
}
if (!Array.isArray(claimed) || claimed.length === 0) return
await mirrorExtractionToDocument(job.documentId, {
data: extracted,
rawText,
model,
skipped: skipReason,
})
try {
await appendProcessingHistory({
companyId: job.companyId,
@@ -646,6 +693,7 @@ function scheduleDeferredExtraction(job: DeferredExtractionJob): void {
})
.eq('id', job.itemId)
.eq('status', 'processing')
await mirrorExtractionToDocument(job.documentId, { data: null, rawText: null })
} catch {
// The sweep cron is the recovery of last resort.
}
@@ -129,6 +129,10 @@ describe('MCP model-free document upload tools', () => {
uploadId,
'invoice.pdf',
'application/pdf',
undefined,
// The inbox item created right after owns extraction; the
// document-extraction extension must yield on the uploaded event.
{ extractionOwner: 'invoice-inbox' },
)
expect(inboxInsert.insert).toHaveBeenCalledWith(
expect.objectContaining({ id: uploadId, document_id: uploadId }),
+15 -2
View File
@@ -210,6 +210,7 @@ import {
MAX_DOCUMENT_SIZE,
} from '@/lib/core/documents/document-service'
import { extractInvoiceFields, ExtractionSchema as InvoiceExtractionSchema, AgentExtractionSchema } from '@/extensions/general/invoice-inbox/lib/extract-invoice-fields'
import { mirrorExtractionToDocument } from '@/extensions/general/invoice-inbox/lib/mirror-extraction'
// Skatteverket filing tools (PR5). Cross-extension lib import, same sanctioned
// pattern as invoice-inbox above: the CI guard only checks lib/, app/api/,
// components/. The two submit tools stage ops whose commit dispatches back into
@@ -497,7 +498,17 @@ async function createDocumentInboxItem(
if (existing) return existing
}
const { data: extracted } = await extractInvoiceFields({ buffer, mimeType, fileName })
const extraction = await extractInvoiceFields({ buffer, mimeType, fileName })
const { data: extracted } = extraction
// uploadDocument()/completePendingDocumentUpload() were told this inbox
// item owns extraction, so the document-extraction extension yielded;
// mirror the outcome onto the document row in its place.
await mirrorExtractionToDocument(documentId, {
data: extracted,
rawText: extraction.rawText,
model: extraction.model ?? null,
skipped: extraction.skipped ?? null,
})
const matchedSupplierId = await matchSupplierId(supabase, companyId, extracted.supplier)
@@ -9777,6 +9788,8 @@ export const tools: McpTool[] = [
uploadId,
fileName,
mimeType,
undefined,
{ extractionOwner: 'invoice-inbox' },
)
return createDocumentInboxItem(
supabase,
@@ -9837,7 +9850,7 @@ export const tools: McpTool[] = [
name: fileName,
buffer: buffer.buffer.slice(buffer.byteOffset, buffer.byteOffset + buffer.byteLength),
type: mimeType,
}, { upload_source: 'api' })
}, { upload_source: 'api', extractionOwner: 'invoice-inbox' })
return createDocumentInboxItem(
supabase,
companyId,