feat(agent): let the single-call assistant read the ledger via read-only MCP tools (#1767)
The /chat assistant (audit Option A / rip) shipped in #1759 reading only the company name + entity type, so it answered "jag har ingen bokföringsdata" to every figures question ("vad är min största utgiftspost?"). It now behaves like an MCP client: it answers over a bounded, READ-only tool loop across the same MCP read tools the old streaming assistant had, plus an always-on company snapshot as the backstop. Provider-agnostic by construction, so it still runs on a local model: - lib/ai generateText gains optional `tools` + `maxSteps`. The OpenAI-compatible service forwards them to the Vercel AI SDK (stopWhen: stepCountIs), which runs the loop; the Anthropic-family service hand-rolls a small loop against messages.create. Kept on the raw Anthropic SDK: no new deps, and the no-tools path is byte-identical, so hosted extraction/composer/etc. are unchanged. - lib/agent/ask/ledger-tools.ts: the read slice of general.help's whitelist (income statement, VAT, ledgers, query_journal, reskontror, lists…) from agentToolRegistry, dispatched with the agent_chat actor run-turn uses. Write/ staging + memory-write tools are excluded; readOnlyHint/destructiveHint are re-checked. Empty in a core-only build → snapshot-only, graceful. - lib/agent/ask/snapshot.ts: a compact company_settings + deadlines block so a model that can't/won't call tools still answers status questions. Never carries figures (those come from the live tools). - ask-service attaches tools + snapshot when a userId is present and uses a tool-aware system prompt; the route calls ensureInitialized() so the registry is populated and threads userId/conversationId through. Works on Bedrock and on any local model with function-calling (Qwen). Tests: the anthropic hand-rolled loop (tool call → result → answer, is_error handling, step-budget forced answer), openai tool forwarding, the read-only adapter filter, the snapshot format, and the ask-service wiring. 457 agent+ai tests green, lint/guards clean. Co-authored-by: Jakob Wennberg <311770904+jakobwennberg-oss@users.noreply.github.com> Co-authored-by: Claude Opus 4.8 <noreply@anthropic.com>
This commit is contained in:
Jakob Wennberg
co-authored by
Jakob Wennberg
Claude Opus 4.8
Jakob Wennberg
Claude Opus 4.8
parent
febb4cc0c2
commit
ff4425d10e
@@ -186,3 +186,117 @@ describe('buildAnthropicDocumentContent', () => {
|
||||
])
|
||||
})
|
||||
})
|
||||
|
||||
describe('generateText read-only tool loop', () => {
|
||||
it('calls a tool, feeds the JSON result back, and returns the final answer with summed usage', async () => {
|
||||
const execute = vi.fn().mockResolvedValue({ largest: '5010', amount: 12345 })
|
||||
const svc = createAnthropicFamilyService(readAiConfig())
|
||||
|
||||
// Turn 1: the model asks for a report.
|
||||
mockCreate.mockResolvedValueOnce({
|
||||
stop_reason: 'tool_use',
|
||||
content: [
|
||||
{ type: 'text', text: 'Kollar.' },
|
||||
{ type: 'tool_use', id: 'tu_1', name: 'gnubok_get_income_statement', input: { period: '2026-07' } },
|
||||
],
|
||||
usage: { input_tokens: 10, output_tokens: 5 },
|
||||
})
|
||||
// Turn 2: the model answers with the figure.
|
||||
mockCreate.mockResolvedValueOnce({
|
||||
stop_reason: 'end_turn',
|
||||
content: [{ type: 'text', text: 'Din största utgift är 12 345 kr (konto 5010).' }],
|
||||
usage: { input_tokens: 20, output_tokens: 8 },
|
||||
})
|
||||
|
||||
const result = await svc.generateText({
|
||||
tier: 'assistant',
|
||||
system: 'S',
|
||||
prompt: 'Vad är min största utgift?',
|
||||
maxTokens: 500,
|
||||
tools: [
|
||||
{ name: 'gnubok_get_income_statement', description: 'd', jsonSchema: { type: 'object' }, execute },
|
||||
],
|
||||
maxSteps: 4,
|
||||
})
|
||||
|
||||
expect(execute).toHaveBeenCalledWith({ period: '2026-07' })
|
||||
expect(result.text).toBe('Din största utgift är 12 345 kr (konto 5010).')
|
||||
expect(mockCreate).toHaveBeenCalledTimes(2)
|
||||
|
||||
// First turn carries the tools; second turn carries the tool_result.
|
||||
expect(mockCreate.mock.calls[0][0].tools[0].name).toBe('gnubok_get_income_statement')
|
||||
const secondMessages = mockCreate.mock.calls[1][0].messages
|
||||
expect(secondMessages).toHaveLength(3) // user, assistant(tool_use), user(tool_result)
|
||||
const toolResult = secondMessages[2].content[0]
|
||||
expect(toolResult.type).toBe('tool_result')
|
||||
expect(toolResult.tool_use_id).toBe('tu_1')
|
||||
expect(JSON.parse(toolResult.content)).toEqual({ largest: '5010', amount: 12345 })
|
||||
|
||||
// Usage summed across both turns.
|
||||
expect(result.usage.inputTokens).toBe(30)
|
||||
expect(result.usage.outputTokens).toBe(13)
|
||||
})
|
||||
|
||||
it('surfaces a tool failure as an is_error result rather than throwing', async () => {
|
||||
const execute = vi.fn().mockRejectedValue(new Error('report timeout'))
|
||||
const svc = createAnthropicFamilyService(readAiConfig())
|
||||
mockCreate.mockResolvedValueOnce({
|
||||
stop_reason: 'tool_use',
|
||||
content: [{ type: 'tool_use', id: 'tu', name: 'gnubok_get_vat_report', input: {} }],
|
||||
usage: { input_tokens: 1, output_tokens: 1 },
|
||||
})
|
||||
mockCreate.mockResolvedValueOnce({
|
||||
stop_reason: 'end_turn',
|
||||
content: [{ type: 'text', text: 'Kunde inte hämta momsrapporten just nu.' }],
|
||||
usage: { input_tokens: 1, output_tokens: 1 },
|
||||
})
|
||||
const result = await svc.generateText({
|
||||
tier: 'assistant',
|
||||
prompt: 'Momsen?',
|
||||
maxTokens: 100,
|
||||
tools: [{ name: 'gnubok_get_vat_report', description: 'd', jsonSchema: {}, execute }],
|
||||
})
|
||||
const toolResult = mockCreate.mock.calls[1][0].messages[2].content[0]
|
||||
expect(toolResult.is_error).toBe(true)
|
||||
expect(JSON.parse(toolResult.content).error).toContain('report timeout')
|
||||
expect(result.text).toBe('Kunde inte hämta momsrapporten just nu.')
|
||||
})
|
||||
|
||||
it('forces a final tools-off answer when the step budget is exhausted', async () => {
|
||||
const execute = vi.fn().mockResolvedValue({ ok: true })
|
||||
const svc = createAnthropicFamilyService(readAiConfig())
|
||||
// Every turn keeps asking for the tool → never terminates on its own.
|
||||
mockCreate.mockResolvedValue({
|
||||
stop_reason: 'tool_use',
|
||||
content: [{ type: 'tool_use', id: 'tu', name: 't', input: {} }],
|
||||
usage: { input_tokens: 1, output_tokens: 1 },
|
||||
})
|
||||
// The forced final turn (index 2) returns real text.
|
||||
mockCreate.mockResolvedValueOnce({
|
||||
stop_reason: 'tool_use',
|
||||
content: [{ type: 'tool_use', id: 'tu', name: 't', input: {} }],
|
||||
usage: { input_tokens: 1, output_tokens: 1 },
|
||||
})
|
||||
mockCreate.mockResolvedValueOnce({
|
||||
stop_reason: 'tool_use',
|
||||
content: [{ type: 'tool_use', id: 'tu', name: 't', input: {} }],
|
||||
usage: { input_tokens: 1, output_tokens: 1 },
|
||||
})
|
||||
mockCreate.mockResolvedValueOnce({
|
||||
stop_reason: 'end_turn',
|
||||
content: [{ type: 'text', text: 'Sammanfattning utan fler verktygsanrop.' }],
|
||||
usage: { input_tokens: 1, output_tokens: 1 },
|
||||
})
|
||||
const result = await svc.generateText({
|
||||
tier: 'assistant',
|
||||
prompt: 'x',
|
||||
maxTokens: 100,
|
||||
tools: [{ name: 't', description: 'd', jsonSchema: {}, execute }],
|
||||
maxSteps: 2,
|
||||
})
|
||||
// 2 loop turns + 1 forced final = 3 calls; the final call omits tools.
|
||||
expect(mockCreate).toHaveBeenCalledTimes(3)
|
||||
expect(mockCreate.mock.calls[2][0].tools).toBeUndefined()
|
||||
expect(result.text).toBe('Sammanfattning utan fler verktygsanrop.')
|
||||
})
|
||||
})
|
||||
|
||||
@@ -102,6 +102,31 @@ describe('createOpenAICompatibleService', () => {
|
||||
})
|
||||
})
|
||||
|
||||
it('generateText forwards read-only tools to the model when provided', async () => {
|
||||
const svc = createOpenAICompatibleService(readAiConfig())
|
||||
const execute = vi.fn().mockResolvedValue({ ok: true })
|
||||
await svc.generateText({
|
||||
tier: 'assistant',
|
||||
prompt: 'Vad är min största utgift?',
|
||||
maxTokens: 50,
|
||||
tools: [
|
||||
{ name: 'gnubok_get_income_statement', description: 'd', jsonSchema: { type: 'object' }, execute },
|
||||
],
|
||||
maxSteps: 4,
|
||||
})
|
||||
// The AI SDK converts our defs and hands them to the model on the wire.
|
||||
const passedTools = doGenerate.mock.calls[0][0].tools as Array<{ name: string }>
|
||||
expect(Array.isArray(passedTools)).toBe(true)
|
||||
expect(passedTools.some((t) => t.name === 'gnubok_get_income_statement')).toBe(true)
|
||||
})
|
||||
|
||||
it('generateText attaches no tools when none are provided (plain single call)', async () => {
|
||||
const svc = createOpenAICompatibleService(readAiConfig())
|
||||
await svc.generateText({ tier: 'assistant', prompt: 'Hej', maxTokens: 50 })
|
||||
const passedTools = doGenerate.mock.calls[0][0].tools
|
||||
expect(passedTools == null || (Array.isArray(passedTools) && passedTools.length === 0)).toBe(true)
|
||||
})
|
||||
|
||||
it('extractFromDocument sends an image as an image part followed by the instruction', async () => {
|
||||
const svc = createOpenAICompatibleService(readAiConfig())
|
||||
const jpeg = Buffer.from('JPEG')
|
||||
|
||||
Reference in New Issue
Block a user