fix(agent): let the assistant see answers the user gave in WhatsApp (#1425)

Two field findings from the first live receipts.

1. The assistant re-asked for information the user had already given.
   The user answered the representation question in WhatsApp ("Elias
   Karlsson från Canguro Media, Jakob Wennberg från Arcim"), the answer
   was stored correctly on invoice_inbox_items.channel_context, and then
   the in-app assistant said it could see no participant names and asked
   for them again. The intent's inbox query selected only document_id and
   extracted_data, and nothing under lib/agent/ read channel_context at
   all. It is now selected, threaded onto each underlag as chat_answers,
   and rendered into the prompt as "uppgivna av användaren" with an
   explicit instruction that human answers outrank anything read off the
   image and must never be re-asked. Also backfilled by document_id: a
   receipt can reach the intent through the document paths without its
   inbox row being matched to the transaction.

2. The representation question accepted half an answer in silence.
   Naming participants but no purpose stored purpose=null and replied
   "Tack!", leaving the deduction undocumented while looking complete.
   Skatteverket wants both (BFL 5 kap 6-7 §). It now asks once, for the
   missing half only, and keeps the question open so the reply routes
   back to the same receipt. Anti-loop: the follow-up fires only when no
   representation block exists yet, so a second incomplete answer is
   taken as-is rather than nagging.

Tests cover the prompt half and the query half separately: the earlier
prompt tests injected chat_answers directly and would have stayed green
with the column still missing from the select, which is precisely how the
bug shipped. Both mutation-checked.

Co-authored-by: Jakob Wennberg <311770904+jakobwennberg-oss@users.noreply.github.com>
Co-authored-by: Claude Fable 5 <noreply@anthropic.com>
This commit is contained in:
Jakob Wennberg
2026-08-05 20:08:15 +02:00
committed by GitHub
co-authored by Jakob Wennberg Claude Fable 5
parent c84f951a5c
commit 576ed4d290
6 changed files with 276 additions and 20 deletions
@@ -1,5 +1,6 @@
import { describe, it, expect } from 'vitest'
import { transactionCategorization } from '../transaction-categorization'
import type { InboxChannelContext } from '@/types'
// Locks in the prose-drift fix from /Users/jakobwennberg/.claude/plans/.
// The promptTemplate must instruct the agent to narrate using CATEGORY
@@ -23,6 +24,7 @@ function stripUuidsAndDates(text: string): string {
function renderPrompt(opts: {
hasUnderlag: boolean
profileSummary?: string | null
chatAnswers?: InboxChannelContext | null
}) {
const captured = {
transaction: {
@@ -46,6 +48,7 @@ function renderPrompt(opts: {
is_restaurant: null,
is_systembolaget: null,
raw_extraction: null,
chat_answers: opts.chatAnswers ?? null,
},
]
: [],
@@ -183,3 +186,90 @@ describe('transaction.categorization prompt template', () => {
expect(withUnderlag).toMatch(/bifogas till\s+VERIFIKATIONEN/)
})
})
// Field report 2026-08-05: a user answered the representation question in
// WhatsApp, then the in-app assistant asked for the same names again because
// invoice_inbox_items.channel_context never reached the prompt.
describe('chat answers from another channel', () => {
const chatAnswers: InboxChannelContext = {
channel: 'whatsapp',
representation: {
participants: [
{ name: 'Elias Karlsson', company: 'Canguro Media' },
{ name: 'Jakob Wennberg', company: 'Arcim Technology AB' },
],
purpose: 'Lunch med samarbetspartner',
event_date: null,
raw_answer: 'Elias Karlsson från Canguro Media, Jakob Wennberg från Arcim',
answered_at: '2026-08-05T14:29:00.000Z',
},
}
it('puts the participants the user already gave into the prompt', () => {
const out = renderPrompt({ hasUnderlag: true, chatAnswers })
expect(out).toContain('Elias Karlsson (Canguro Media)')
expect(out).toContain('Jakob Wennberg (Arcim Technology AB)')
expect(out).toContain('Lunch med samarbetspartner')
})
it('marks them as human-supplied and forbids re-asking', () => {
const out = renderPrompt({ hasUnderlag: true, chatAnswers })
expect(out).toMatch(/uppgivna av användaren/)
expect(out).toMatch(/Fråga ALDRIG om något som redan står där/)
})
it('names the missing half when only participants were captured', () => {
const out = renderPrompt({
hasUnderlag: true,
chatAnswers: {
...chatAnswers,
representation: { ...chatAnswers.representation!, purpose: null },
},
})
expect(out).toMatch(/syfte SAKNAS/)
expect(out).toContain('Elias Karlsson (Canguro Media)')
})
it('says nothing about chat answers when there are none', () => {
const out = renderPrompt({ hasUnderlag: true })
expect(out).not.toMatch(/uppgivna av användaren/)
})
})
// The prompt tests above inject chat_answers directly, so they would still
// pass if capture() never selected the column: which is exactly the bug that
// shipped. This covers the query half.
describe('transaction.categorization capture', () => {
function supabaseStub() {
const selects: string[] = []
const from = (table: string) => {
const chain: Record<string, unknown> = {}
const self = () => chain
chain.select = (cols: string) => {
selects.push(`${table}:${cols}`)
return chain
}
for (const m of ['eq', 'in', 'not']) chain[m] = self
chain.single = async () => ({ data: { id: 'tx-1', date: '2026-08-05', description: 'X', amount: -425, currency: 'SEK', document_id: null, journal_entry_id: null } })
chain.maybeSingle = async () => ({ data: null })
chain.then = (resolve: (v: unknown) => unknown) =>
resolve({
data: table === 'invoice_inbox_items'
? [{ document_id: 'doc-1', extracted_data: {}, channel_context: { channel: 'whatsapp', user_note: 'lunch med kund' } }]
: [],
})
return chain
}
return { from, selects }
}
it('selects channel_context and threads it onto the underlag', async () => {
const stub = supabaseStub()
const captured = await transactionCategorization.capture!(
{ transaction_id: 'tx-1' },
{ supabase: stub as never, companyId: 'company-1', userId: 'user-1' } as never,
)
expect(stub.selects.some((s) => s.startsWith('invoice_inbox_items:') && s.includes('channel_context'))).toBe(true)
expect(captured.underlag[0]?.chat_answers).toEqual({ channel: 'whatsapp', user_note: 'lunch med kund' })
})
})
@@ -1,5 +1,6 @@
import { defineAgentIntent } from './types'
import { SONNET_MODEL, EFFORT_STANDARD } from '@/lib/agent/composer/client'
import type { InboxChannelContext } from '@/types'
// transaction.categorization: "Fråga om denna transaktion" on a transaction
// row.
@@ -44,6 +45,16 @@ interface CapturedTransaction {
// agent can paraphrase context-specific signals (line items, dates,
// payment reference) without us pre-modeling every field.
raw_extraction: Record<string, unknown> | null
/**
* Answers the user already gave about THIS underlag in another channel
* (today: the WhatsApp intake conversation). Verified human input, not
* OCR guesswork, so it outranks anything read off the image.
*
* Without it the assistant asked for participant names the user had
* typed into WhatsApp minutes earlier, which is the worst thing a
* system that already holds the answer can do.
*/
chat_answers: InboxChannelContext | null
}[]
}
@@ -113,7 +124,7 @@ export const transactionCategorization = defineAgentIntent<
.eq('matched_transaction_id', transaction_id),
supabase
.from('invoice_inbox_items')
.select('document_id, extracted_data')
.select('document_id, extracted_data, channel_context')
.eq('company_id', companyId)
.eq('matched_transaction_id', transaction_id),
directDocumentId
@@ -157,11 +168,13 @@ export const transactionCategorization = defineAgentIntent<
is_restaurant: r.is_restaurant,
is_systembolaget: r.is_systembolaget,
raw_extraction: r.raw_extraction,
chat_answers: null,
})
}
for (const it of (inboxItems ?? []) as {
document_id: string | null
extracted_data: Record<string, unknown> | null
channel_context: InboxChannelContext | null
}[]) {
const ex = it.extracted_data ?? {}
const supplier = (ex.supplier as { name?: string | null } | undefined) ?? null
@@ -178,6 +191,7 @@ export const transactionCategorization = defineAgentIntent<
is_restaurant: null,
is_systembolaget: null,
raw_extraction: ex,
chat_answers: it.channel_context ?? null,
})
}
@@ -221,6 +235,7 @@ export const transactionCategorization = defineAgentIntent<
is_restaurant: null,
is_systembolaget: null,
raw_extraction: null,
chat_answers: null,
})
continue
}
@@ -238,9 +253,37 @@ export const transactionCategorization = defineAgentIntent<
is_restaurant: null,
is_systembolaget: null,
raw_extraction: ex,
chat_answers: null,
})
}
// Chat answers live on the inbox item, but an underlag can reach this
// point through the document paths above (the transaction's own
// document_id, or a doc anchored to the journal entry) without the
// inbox row ever being matched to the transaction. Backfill by
// document_id so the answers are found either way.
const missingContextDocIds = underlag
.filter((u) => u.chat_answers == null && u.document_id != null)
.map((u) => u.document_id as string)
if (missingContextDocIds.length > 0) {
const { data: byDoc } = await supabase
.from('invoice_inbox_items')
.select('document_id, channel_context')
.eq('company_id', companyId)
.in('document_id', missingContextDocIds)
.not('channel_context', 'is', null)
for (const row of (byDoc ?? []) as {
document_id: string | null
channel_context: InboxChannelContext | null
}[]) {
for (const u of underlag) {
if (u.document_id === row.document_id && u.chat_answers == null) {
u.chat_answers = row.channel_context ?? null
}
}
}
}
return {
transaction: tx
? {
@@ -311,9 +354,34 @@ export const transactionCategorization = defineAgentIntent<
if (u.is_restaurant) parts.push('restaurang=ja')
if (u.is_systembolaget) parts.push('systembolaget=ja')
lines.push(` • ${u.kind}: ${parts.join(', ') || '(ingen extraherad data: läs underlaget med gnubok_get_document_content)'}`)
// Answers the user already typed in another channel. Own indented
// block so it reads as human input, not one more OCR field.
const chat = u.chat_answers
if (chat) {
const rep = chat.representation
if (rep) {
const who = (rep.participants ?? [])
.map((pp) => (pp.company ? `${pp.name} (${pp.company})` : pp.name))
.join(', ')
if (who) lines.push(` - deltagare (uppgivna av användaren): ${who}`)
if (rep.purpose) lines.push(` - syfte (uppgivet av användaren): ${rep.purpose}`)
if (rep.event_date) lines.push(` - datum (uppgivet av användaren): ${rep.event_date}`)
if (!rep.purpose) {
lines.push(' - syfte SAKNAS: fråga bara efter syftet, inte om deltagarna igen.')
}
}
if (chat.user_note) lines.push(` - anteckning från användaren: ${chat.user_note}`)
if (chat.caption) lines.push(` - bildtext vid inskick: ${chat.caption}`)
}
}
lines.push('')
lines.push('VIKTIGT: extraktionen ovan är det vi REDAN VET. Återupprepa inte frågor som "vilken leverantör är det?" eller "vad var beloppet?": det står ovan. Använd uppgifterna direkt och föreslå kategori + moms-behandling.')
// Conditional: an unconditional line is prompt bloat on the common
// transaction that has no chat history.
if (captured.underlag.some((u) => u.chat_answers != null)) {
lines.push('Rader märkta "uppgivna av användaren" kommer från en tidigare konversation om samma underlag (t.ex. WhatsApp när kvittot skickades in). Det är MÄNSKLIGT bekräftade uppgifter och väger tyngre än vad du själv läser ut ur bilden. Fråga ALDRIG om något som redan står där; behöver du komplettera, fråga bara om den del som faktiskt saknas. När du stagear: ta med deltagare och syfte i notes så de följer med till verifikationen.')
}
}
lines.push('')
lines.push('Arbetssätt: hämta information via verktygsanrop FÖRST (tyst: statusraderna visar att du söker, och ditt resonemang sker i tankekanalen), föreslå sedan. Skriv din förklaring EN gång efteråt, inte i flera block runt anropen.')