* feat(ai): resolve the Claude backend from the environment Tier 1 of #1406: a self-hosted deployment can now run every AI feature on a plain ANTHROPIC_API_KEY, with no AWS account. Hosted behaviour is unchanged. lib/ai/provider.ts resolves the backend once, from the environment: AI_PROVIDER explicit override, bedrock|anthropic AWS static key pair Bedrock ANTHROPIC_API_KEY the direct Anthropic API nothing set Bedrock, so the AWS credential provider chain (instance profile, IRSA) still resolves Bedrock deliberately wins when both credential sets are present. EU residency in eu-north-1 is a BFL/GDPR posture rather than a default, so adding an Anthropic key for an experiment must not silently move production inference out of the region. AI_PROVIDER is the way to say you meant it. Model ids are written bare in code and prefixed to eu.anthropic.* only for Bedrock, which needs the cross-region inference profile for on-demand throughput. An operator override that already carries a prefix passes through untouched, so BEDROCK_MODEL_ID and friends keep working as written. Converted call sites: the agent composer, invoice-inbox extraction, the document-extraction model label, and both receipt-hunt clients. The last two are not named in the issue, which predates receipt-hunt landing in main. @anthropic-ai/sdk is declared at 0.95.0, the version @anthropic-ai/bedrock-sdk 0.29.1 already pulled in transitively, so the lockfile dedupes to one copy with no new download. scripts/smoke-bedrock.ts becomes scripts/smoke-ai.ts and grows two steps. Unit tests can only prove which provider and model id get resolved; they cannot prove the resulting request is one the backend accepts. The script now sends real traffic over all three shapes the app uses: a plain create, a streamed turn carrying adaptive thinking, an effort level, an hour-long cache breakpoint and a tool, and document extraction end to end when given a file. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com> Signed-off-by: Bjorn Bergenheim <29535152+bjornbergenheim@users.noreply.github.com> * docs(self-hosting): document the AI smoke test The script added alongside the provider split is what closes the #1406 acceptance criterion ("document extraction and the assistant both work"), so a self-hoster needs to know it exists. Covers both invocations and states that it exits non-zero, which is what makes it usable as a post-deploy check. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com> Signed-off-by: Bjorn Bergenheim <29535152+bjornbergenheim@users.noreply.github.com> * test(ai): split the smoke test's thinking probe from its tool probe The combined probe could not falsify what it claimed to. It asked a question that needs a tool call, so the tool was used and adaptive thinking correctly declined to reason about it: the zero thinking-block count that came back was uninformative rather than a signal. 2a keeps the tool and drops thinking. 2b asks a question with several dependent steps (reverse charge, then a partial deduction, then the affected boxes) so that a model honouring the parameter must reason, and reports the thinking text length as well as the block count, since display:"summarized" can yield blocks with empty text. The cached system prompt is also padded past the 1024-token minimum cacheable prefix. Below that the API caches nothing and reports no error, so the old probe's cache counters read zero whether or not caching worked. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com> Signed-off-by: Bjorn Bergenheim <29535152+bjornbergenheim@users.noreply.github.com> * fix(document-extraction): stop requiring AWS_REGION in the manifest The extension now needs one of two credential sets, AWS static keys or ANTHROPIC_API_KEY, and the manifest schema cannot express "one of". Since requiredEnvVars only drives a build-time warning and never gates anything, listing AWS_REGION told every self-hoster running the direct API to set a variable that has no effect for them. The description was also still promising Sonnet 4.6 via Bedrock specifically, which is no longer what the extension does. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com> Signed-off-by: Bjorn Bergenheim <29535152+bjornbergenheim@users.noreply.github.com> * fix(ai): read documentKind defensively in the smoke test The field arrived with the receipt-aware extraction work, so referencing it directly stops the script compiling against any checkout from before that landed. tsconfig includes **/*.ts and next.config does not disable type checking, so on such a checkout this failed the production build rather than just the script: caught while preparing a test branch for a self-hosted instance that had not synced yet. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com> Signed-off-by: Bjorn Bergenheim <29535152+bjornbergenheim@users.noreply.github.com> * fix(deps): restore the nested @swc/helpers entry in the lockfile Declaring @anthropic-ai/sdk with `npm install --package-lock-only` also pruned node_modules/next-intl/node_modules/@swc/helpers@0.5.23, an optional peer entry the local npm 11 considers redundant and the image's npm 10.9.8 does not. The result passed every local check and failed `npm ci` inside the Docker build, which is the only place the lockfile is actually enforced. The lockfile is now the previous one plus the single root dependency line, verified with `npm ci --dry-run`. @anthropic-ai/sdk needed nothing else: it was already in the tree as a transitive dependency of @anthropic-ai/bedrock-sdk. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com> Signed-off-by: Bjorn Bergenheim <29535152+bjornbergenheim@users.noreply.github.com> * Update DECISIONS.md Co-authored-by: coderabbitai[bot] <136622811+coderabbitai[bot]@users.noreply.github.com> * Update Docker documentation for AI provider credentials Clarify the role of credentials in AI provider selection and document extraction requirements. * Update SELF-HOSTING.md with smoke-ai script details Clarify usage of smoke-ai script for credential checks and document extraction. * Improve error handling and logging in smoke-ai script * fix(ai): complete plain-key self-hosting path Signed-off-by: Emil <emilmattsson14@gmail.com> --------- Signed-off-by: Bjorn Bergenheim <29535152+bjornbergenheim@users.noreply.github.com> Signed-off-by: Emil <emilmattsson14@gmail.com> Co-authored-by: Bjorn Bergenheim <29535152+bjornbergenheim@users.noreply.github.com> Co-authored-by: Claude Opus 5 <noreply@anthropic.com> Co-authored-by: coderabbitai[bot] <136622811+coderabbitai[bot]@users.noreply.github.com>
156 lines
6.1 KiB
TypeScript
156 lines
6.1 KiB
TypeScript
/**
|
|
* The model reads mails and reports fields. These tests are about what happens
|
|
* when it reports something it should not: an id we never offered, a filename
|
|
* that is not there, a negative total, a date that is not a date. None of that
|
|
* may reach the matcher, because the matcher trusts its inputs.
|
|
*/
|
|
import { describe, it, expect, vi, beforeEach } from 'vitest'
|
|
|
|
const mockCreate = vi.fn()
|
|
vi.mock('@/lib/ai/provider', () => ({
|
|
createAiClient: () => ({ messages: { create: (...args: unknown[]) => mockCreate(...args) } }),
|
|
toProviderModelId: (id: string) => id,
|
|
}))
|
|
|
|
import { extractMailDocuments, type CandidateForReview } from '../mail-intelligence'
|
|
|
|
/** A reply in the shape forced tool use produces. */
|
|
function toolReply(input: unknown) {
|
|
return { content: [{ type: 'tool_use', name: 'x', id: 'tu', input }] }
|
|
}
|
|
|
|
function candidate(overrides: Partial<CandidateForReview> = {}): CandidateForReview {
|
|
return {
|
|
messageId: 'msg-1',
|
|
mailbox: 'invoice@arcim.io',
|
|
subject: 'Fwd: Your receipt from Anthropic, PBC',
|
|
from: 'jakob@example.com',
|
|
receivedAt: '2026-08-01T10:00:00.000Z',
|
|
bodyText: 'Vidarebefordrat meddelande Från: Anthropic, PBC Datum: mån 15 juni 2026 €180.00',
|
|
attachmentNames: ['Receipt-2066.pdf'],
|
|
...overrides,
|
|
}
|
|
}
|
|
|
|
function doc(overrides: Record<string, unknown> = {}) {
|
|
return {
|
|
message_id: 'msg-1',
|
|
attachment_name: 'Receipt-2066.pdf',
|
|
is_receipt: true,
|
|
vendor: 'Anthropic, PBC',
|
|
date: '2026-06-15',
|
|
amount: 180,
|
|
currency: 'eur',
|
|
...overrides,
|
|
}
|
|
}
|
|
|
|
beforeEach(() => vi.clearAllMocks())
|
|
|
|
describe('extractMailDocuments', () => {
|
|
it('reports the fields a matcher needs', async () => {
|
|
mockCreate.mockResolvedValue(toolReply({ documents: [doc()] }))
|
|
const [out] = await extractMailDocuments([candidate()])
|
|
|
|
expect(out).toMatchObject({
|
|
messageId: 'msg-1',
|
|
attachmentName: 'Receipt-2066.pdf',
|
|
vendor: 'Anthropic, PBC',
|
|
date: '2026-06-15',
|
|
amount: 180,
|
|
currency: 'EUR',
|
|
})
|
|
})
|
|
|
|
it('takes the purchase date from the mail, not the forwarding date', async () => {
|
|
// The mail was forwarded in August; the purchase was in June. Matching on
|
|
// the forwarding date is what made the old design miss five of six repeat
|
|
// subscriptions.
|
|
mockCreate.mockResolvedValue(toolReply({ documents: [doc()] }))
|
|
const [out] = await extractMailDocuments([candidate({ receivedAt: '2026-08-01T10:00:00.000Z' })])
|
|
expect(out.date).toBe('2026-06-15')
|
|
})
|
|
|
|
it('drops mail that is not an underlag', async () => {
|
|
mockCreate.mockResolvedValue(
|
|
toolReply({ documents: [doc({ is_receipt: false, vendor: null, amount: null })] }),
|
|
)
|
|
await expect(extractMailDocuments([candidate()])).resolves.toEqual([])
|
|
})
|
|
|
|
it('never reports a message id it was not given', async () => {
|
|
mockCreate.mockResolvedValue(toolReply({ documents: [doc({ message_id: 'invented' })] }))
|
|
await expect(extractMailDocuments([candidate()])).resolves.toEqual([])
|
|
})
|
|
|
|
it('rejects a filename that is not on the message', async () => {
|
|
// An invented filename means it was guessing about the contents.
|
|
mockCreate.mockResolvedValue(toolReply({ documents: [doc({ attachment_name: 'ghost.pdf' })] }))
|
|
await expect(extractMailDocuments([candidate()])).resolves.toEqual([])
|
|
})
|
|
|
|
it('treats a missing amount as unknown rather than as zero', async () => {
|
|
// Most receipts state their total only inside the PDF. Null must stay null:
|
|
// a zero would score as an amount that disagrees with every transaction.
|
|
mockCreate.mockResolvedValue(toolReply({ documents: [doc({ amount: null })] }))
|
|
const [out] = await extractMailDocuments([candidate()])
|
|
expect(out.amount).toBeNull()
|
|
})
|
|
|
|
it('discards a non-positive total as a misread', async () => {
|
|
mockCreate.mockResolvedValue(toolReply({ documents: [doc({ amount: -180 })] }))
|
|
const [out] = await extractMailDocuments([candidate()])
|
|
expect(out.amount).toBeNull()
|
|
})
|
|
|
|
it('discards a date that is not a date', async () => {
|
|
mockCreate.mockResolvedValue(toolReply({ documents: [doc({ date: 'juni 2026' })] }))
|
|
const [out] = await extractMailDocuments([candidate()])
|
|
expect(out.date).toBeNull()
|
|
})
|
|
|
|
it('reports one document per attachment in a batch forward', async () => {
|
|
// "Fwd: Kvitton februari" carries five receipts for five purchases.
|
|
mockCreate.mockResolvedValue(
|
|
toolReply({
|
|
documents: [
|
|
doc({ attachment_name: 'a.pdf', amount: 162.02 }),
|
|
doc({ attachment_name: 'b.pdf', amount: 425 }),
|
|
],
|
|
}),
|
|
)
|
|
const out = await extractMailDocuments([candidate({ attachmentNames: ['a.pdf', 'b.pdf'] })])
|
|
expect(out.map((d) => d.amount)).toEqual([162.02, 425])
|
|
})
|
|
|
|
it('refuses to guess which file, when the mail carries several', async () => {
|
|
// Without a filename the caller would fetch attachment number one and hope.
|
|
// On a batch forward that is a coin flip, so the document is dropped.
|
|
mockCreate.mockResolvedValue(toolReply({ documents: [doc({ attachment_name: null })] }))
|
|
await expect(
|
|
extractMailDocuments([candidate({ attachmentNames: ['a.pdf', 'b.pdf', 'c.pdf'] })]),
|
|
).resolves.toEqual([])
|
|
})
|
|
|
|
it('still accepts a body-only receipt, where there is nothing to choose', async () => {
|
|
mockCreate.mockResolvedValue(toolReply({ documents: [doc({ attachment_name: null })] }))
|
|
const out = await extractMailDocuments([candidate({ attachmentNames: [] })])
|
|
expect(out).toHaveLength(1)
|
|
})
|
|
|
|
it('accepts an array the model sent as a JSON string', async () => {
|
|
mockCreate.mockResolvedValue(toolReply({ documents: JSON.stringify([doc()]) }))
|
|
await expect(extractMailDocuments([candidate()])).resolves.toHaveLength(1)
|
|
})
|
|
|
|
it('reports nothing when the call fails', async () => {
|
|
mockCreate.mockRejectedValue(new Error('bedrock timeout'))
|
|
await expect(extractMailDocuments([candidate()])).resolves.toEqual([])
|
|
})
|
|
|
|
it('does not call the model when there is nothing to read', async () => {
|
|
await expect(extractMailDocuments([])).resolves.toEqual([])
|
|
expect(mockCreate).not.toHaveBeenCalled()
|
|
})
|
|
})
|