fix(ai): complete plain-key self-hosting path (#1584)
* feat(ai): resolve the Claude backend from the environment Tier 1 of #1406: a self-hosted deployment can now run every AI feature on a plain ANTHROPIC_API_KEY, with no AWS account. Hosted behaviour is unchanged. lib/ai/provider.ts resolves the backend once, from the environment: AI_PROVIDER explicit override, bedrock|anthropic AWS static key pair Bedrock ANTHROPIC_API_KEY the direct Anthropic API nothing set Bedrock, so the AWS credential provider chain (instance profile, IRSA) still resolves Bedrock deliberately wins when both credential sets are present. EU residency in eu-north-1 is a BFL/GDPR posture rather than a default, so adding an Anthropic key for an experiment must not silently move production inference out of the region. AI_PROVIDER is the way to say you meant it. Model ids are written bare in code and prefixed to eu.anthropic.* only for Bedrock, which needs the cross-region inference profile for on-demand throughput. An operator override that already carries a prefix passes through untouched, so BEDROCK_MODEL_ID and friends keep working as written. Converted call sites: the agent composer, invoice-inbox extraction, the document-extraction model label, and both receipt-hunt clients. The last two are not named in the issue, which predates receipt-hunt landing in main. @anthropic-ai/sdk is declared at 0.95.0, the version @anthropic-ai/bedrock-sdk 0.29.1 already pulled in transitively, so the lockfile dedupes to one copy with no new download. scripts/smoke-bedrock.ts becomes scripts/smoke-ai.ts and grows two steps. Unit tests can only prove which provider and model id get resolved; they cannot prove the resulting request is one the backend accepts. The script now sends real traffic over all three shapes the app uses: a plain create, a streamed turn carrying adaptive thinking, an effort level, an hour-long cache breakpoint and a tool, and document extraction end to end when given a file. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com> Signed-off-by: Bjorn Bergenheim <29535152+bjornbergenheim@users.noreply.github.com> * docs(self-hosting): document the AI smoke test The script added alongside the provider split is what closes the #1406 acceptance criterion ("document extraction and the assistant both work"), so a self-hoster needs to know it exists. Covers both invocations and states that it exits non-zero, which is what makes it usable as a post-deploy check. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com> Signed-off-by: Bjorn Bergenheim <29535152+bjornbergenheim@users.noreply.github.com> * test(ai): split the smoke test's thinking probe from its tool probe The combined probe could not falsify what it claimed to. It asked a question that needs a tool call, so the tool was used and adaptive thinking correctly declined to reason about it: the zero thinking-block count that came back was uninformative rather than a signal. 2a keeps the tool and drops thinking. 2b asks a question with several dependent steps (reverse charge, then a partial deduction, then the affected boxes) so that a model honouring the parameter must reason, and reports the thinking text length as well as the block count, since display:"summarized" can yield blocks with empty text. The cached system prompt is also padded past the 1024-token minimum cacheable prefix. Below that the API caches nothing and reports no error, so the old probe's cache counters read zero whether or not caching worked. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com> Signed-off-by: Bjorn Bergenheim <29535152+bjornbergenheim@users.noreply.github.com> * fix(document-extraction): stop requiring AWS_REGION in the manifest The extension now needs one of two credential sets, AWS static keys or ANTHROPIC_API_KEY, and the manifest schema cannot express "one of". Since requiredEnvVars only drives a build-time warning and never gates anything, listing AWS_REGION told every self-hoster running the direct API to set a variable that has no effect for them. The description was also still promising Sonnet 4.6 via Bedrock specifically, which is no longer what the extension does. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com> Signed-off-by: Bjorn Bergenheim <29535152+bjornbergenheim@users.noreply.github.com> * fix(ai): read documentKind defensively in the smoke test The field arrived with the receipt-aware extraction work, so referencing it directly stops the script compiling against any checkout from before that landed. tsconfig includes **/*.ts and next.config does not disable type checking, so on such a checkout this failed the production build rather than just the script: caught while preparing a test branch for a self-hosted instance that had not synced yet. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com> Signed-off-by: Bjorn Bergenheim <29535152+bjornbergenheim@users.noreply.github.com> * fix(deps): restore the nested @swc/helpers entry in the lockfile Declaring @anthropic-ai/sdk with `npm install --package-lock-only` also pruned node_modules/next-intl/node_modules/@swc/helpers@0.5.23, an optional peer entry the local npm 11 considers redundant and the image's npm 10.9.8 does not. The result passed every local check and failed `npm ci` inside the Docker build, which is the only place the lockfile is actually enforced. The lockfile is now the previous one plus the single root dependency line, verified with `npm ci --dry-run`. @anthropic-ai/sdk needed nothing else: it was already in the tree as a transitive dependency of @anthropic-ai/bedrock-sdk. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com> Signed-off-by: Bjorn Bergenheim <29535152+bjornbergenheim@users.noreply.github.com> * Update DECISIONS.md Co-authored-by: coderabbitai[bot] <136622811+coderabbitai[bot]@users.noreply.github.com> * Update Docker documentation for AI provider credentials Clarify the role of credentials in AI provider selection and document extraction requirements. * Update SELF-HOSTING.md with smoke-ai script details Clarify usage of smoke-ai script for credential checks and document extraction. * Improve error handling and logging in smoke-ai script * fix(ai): complete plain-key self-hosting path Signed-off-by: Emil <emilmattsson14@gmail.com> --------- Signed-off-by: Bjorn Bergenheim <29535152+bjornbergenheim@users.noreply.github.com> Signed-off-by: Emil <emilmattsson14@gmail.com> Co-authored-by: Bjorn Bergenheim <29535152+bjornbergenheim@users.noreply.github.com> Co-authored-by: Claude Opus 5 <noreply@anthropic.com> Co-authored-by: coderabbitai[bot] <136622811+coderabbitai[bot]@users.noreply.github.com>
This commit is contained in:
Mattsson
co-authored by
Claude Opus 5
coderabbitai[bot]
Bjorn Bergenheim
Claude Opus 5
coderabbitai[bot]
Bjorn Bergenheim
parent
0c3864cae5
commit
3829b6add3
@@ -1,75 +1,76 @@
|
||||
import AnthropicBedrock from '@anthropic-ai/bedrock-sdk'
|
||||
import {
|
||||
createAiClient,
|
||||
aiCredentialPrefix,
|
||||
hasAiCredentials,
|
||||
resolveAiProvider,
|
||||
toProviderModelId,
|
||||
type AiClient,
|
||||
} from '@/lib/ai/provider'
|
||||
import { createLogger } from '@/lib/logger'
|
||||
|
||||
const log = createLogger('agent-bedrock-client')
|
||||
const log = createLogger('agent-ai-client')
|
||||
|
||||
let cached: AnthropicBedrock | null = null
|
||||
let cached: AiClient | null = null
|
||||
|
||||
// Single AnthropicBedrock client for the agent composer + chat loop. Matches
|
||||
// the credential surface the rest of the codebase already uses (see
|
||||
// extensions/general/invoice-inbox/lib/extract-invoice-fields.ts) so:
|
||||
// Single Claude client for the agent composer + chat loop. Which backend it
|
||||
// talks to is resolved from the environment by lib/ai/provider.ts:
|
||||
//
|
||||
// 1. There's no separate ANTHROPIC_API_KEY to provision and rotate.
|
||||
// 2. All Claude traffic stays in eu-north-1: important for Swedish
|
||||
// accounting data under BFL retention.
|
||||
// 3. Failures and quotas show up in one AWS surface, not two.
|
||||
// - Hosted runs on AWS Bedrock. All Claude traffic stays in eu-north-1,
|
||||
// which matters for Swedish accounting data under BFL retention, and
|
||||
// failures and quotas show up in one AWS surface rather than two.
|
||||
// - Self-hosted deployments with no AWS account run against the direct
|
||||
// Anthropic API with a plain ANTHROPIC_API_KEY.
|
||||
//
|
||||
// Trade-off vs. the direct Anthropic API: Bedrock's prompt-cache TTL is
|
||||
// 5 minutes (default) rather than the 1h the plan §10 specifies. We still
|
||||
// pass `cache_control: { type: 'ephemeral', ttl: '1h' }` in the system
|
||||
// prompt assembly: Bedrock currently ignores the explicit TTL and uses 5m.
|
||||
// Cache effectiveness drops on multi-minute gaps but the loop still works.
|
||||
// Revisit if/when Bedrock exposes longer TTLs or if cost forces the direct
|
||||
// API.
|
||||
export function getAnthropic(): AnthropicBedrock {
|
||||
// The two SDKs expose the same `messages.create` / `messages.stream` surface,
|
||||
// which is all this loop uses, so the split is confined to the factory.
|
||||
//
|
||||
// Prompt-cache TTL differs between them. We pass
|
||||
// `cache_control: { type: 'ephemeral', ttl: '1h' }` in the system prompt
|
||||
// assembly (plan §10); Bedrock ignores the explicit TTL and uses its 5 minute
|
||||
// default, while the direct API honours the hour. Cache effectiveness on
|
||||
// multi-minute gaps is therefore better on the direct path; the loop works
|
||||
// either way.
|
||||
export function getAnthropic(): AiClient {
|
||||
if (cached) return cached
|
||||
const awsRegion = process.env.AWS_REGION || 'eu-north-1'
|
||||
const awsAccessKey = process.env.AWS_ACCESS_KEY_ID
|
||||
const awsSecretKey = process.env.AWS_SECRET_ACCESS_KEY
|
||||
|
||||
// Startup diagnostic: make a hosted misconfiguration visible in the logs
|
||||
// instead of it surfacing only as an opaque "request ended without sending
|
||||
// any chunks" at stream time. Runs once per cold start (the client is cached).
|
||||
// Never logs a secret: only the region, presence booleans, and the 4-char
|
||||
// access-key-id PREFIX (AKIA = long-term IAM user key; ASIA = STS/temporary
|
||||
// role credential, i.e. a platform-injected one rather than ours).
|
||||
if (!awsAccessKey || !awsSecretKey) {
|
||||
log.error('agent Bedrock credentials not loaded from env', undefined, {
|
||||
region: awsRegion,
|
||||
hasAccessKeyId: !!awsAccessKey,
|
||||
hasSecretAccessKey: !!awsSecretKey,
|
||||
// Startup diagnostic: make a misconfiguration visible in the logs instead of
|
||||
// it surfacing only as an opaque "request ended without sending any chunks"
|
||||
// at stream time. Runs once per cold start (the client is cached). Never
|
||||
// logs a secret: only the provider, a presence boolean, and the credential's
|
||||
// non-secret prefix (see aiCredentialPrefix).
|
||||
const provider = resolveAiProvider()
|
||||
if (!hasAiCredentials()) {
|
||||
log.error('agent AI credentials not loaded from env', undefined, {
|
||||
provider,
|
||||
hasCredentials: false,
|
||||
regionFromEnv: !!process.env.AWS_REGION,
|
||||
})
|
||||
} else {
|
||||
log.info('agent Bedrock client init', {
|
||||
region: awsRegion,
|
||||
keyPrefix: awsAccessKey.slice(0, 4),
|
||||
log.info('agent AI client init', {
|
||||
provider,
|
||||
keyPrefix: aiCredentialPrefix(),
|
||||
hasSessionToken: !!process.env.AWS_SESSION_TOKEN,
|
||||
regionFromEnv: !!process.env.AWS_REGION,
|
||||
})
|
||||
}
|
||||
|
||||
// When both static keys are present, pass them. Otherwise omit them so the
|
||||
// SDK falls back to the AWS credential provider chain (instance profile,
|
||||
// IRSA, EKS pod identity, ...). The two-overload SDK refuses a mix.
|
||||
cached =
|
||||
awsAccessKey && awsSecretKey
|
||||
? new AnthropicBedrock({ awsRegion, awsAccessKey, awsSecretKey })
|
||||
: new AnthropicBedrock({ awsRegion })
|
||||
cached = createAiClient()
|
||||
return cached
|
||||
}
|
||||
|
||||
// Bedrock model IDs. Region prefix `eu.` keeps inference inside eu-north-1
|
||||
// (a bare `anthropic.claude-sonnet-5` is rejected: on-demand throughput needs
|
||||
// the cross-region inference profile). Both are env-overridable so ops can
|
||||
// swap models without a code deploy.
|
||||
// Model ids, written bare and adapted to the resolved provider: Bedrock needs
|
||||
// the `eu.` inference-profile prefix, the direct API needs it absent. Both stay
|
||||
// env-overridable so ops can swap models without a code deploy; an override is
|
||||
// used verbatim, in whichever form the provider it was written for expects.
|
||||
//
|
||||
// Both point at Sonnet 5, verified enabled on this Bedrock account. The two
|
||||
// names are kept because the intents split on them: OPUS_MODEL marks the
|
||||
// Both point at Sonnet 5, verified enabled on the hosted Bedrock account. The
|
||||
// two names are kept because the intents split on them: OPUS_MODEL marks the
|
||||
// heavy-reasoning intents (supplier-invoice review, VAT review, bokslut) so
|
||||
// that split survives if a genuinely larger model is enabled here later.
|
||||
export const OPUS_MODEL = process.env.BEDROCK_OPUS_MODEL_ID || 'eu.anthropic.claude-sonnet-5'
|
||||
export const SONNET_MODEL = process.env.BEDROCK_SONNET_MODEL_ID || 'eu.anthropic.claude-sonnet-5'
|
||||
export const OPUS_MODEL =
|
||||
toProviderModelId(process.env.BEDROCK_OPUS_MODEL_ID || 'claude-sonnet-5')
|
||||
export const SONNET_MODEL =
|
||||
toProviderModelId(process.env.BEDROCK_SONNET_MODEL_ID || 'claude-sonnet-5')
|
||||
|
||||
// Reasoning depth for the chat intents.
|
||||
//
|
||||
|
||||
Reference in New Issue
Block a user