fix(agent): let the assistant see answers the user gave in WhatsApp (#1425)

Two field findings from the first live receipts. 1. The assistant re-asked for information the user had already given. The user answered the representation question in WhatsApp ("Elias Karlsson från Canguro Media, Jakob Wennberg från Arcim"), the answer was stored correctly on invoice_inbox_items.channel_context, and then the in-app assistant said it could see no participant names and asked for them again. The intent's inbox query selected only document_id and extracted_data, and nothing under lib/agent/ read channel_context at all. It is now selected, threaded onto each underlag as chat_answers, and rendered into the prompt as "uppgivna av användaren" with an explicit instruction that human answers outrank anything read off the image and must never be re-asked. Also backfilled by document_id: a receipt can reach the intent through the document paths without its inbox row being matched to the transaction. 2. The representation question accepted half an answer in silence. Naming participants but no purpose stored purpose=null and replied "Tack!", leaving the deduction undocumented while looking complete. Skatteverket wants both (BFL 5 kap 6-7 §). It now asks once, for the missing half only, and keeps the question open so the reply routes back to the same receipt. Anti-loop: the follow-up fires only when no representation block exists yet, so a second incomplete answer is taken as-is rather than nagging. Tests cover the prompt half and the query half separately: the earlier prompt tests injected chat_answers directly and would have stayed green with the column still missing from the select, which is precisely how the bug shipped. Both mutation-checked. Co-authored-by: Jakob Wennberg <311770904+jakobwennberg-oss@users.noreply.github.com> Co-authored-by: Claude Fable 5 <noreply@anthropic.com>
2026-08-05 20:08:15 +02:00
parent c84f951a5c
commit 576ed4d290
6 changed files with 276 additions and 20 deletions
@@ -794,3 +794,5 @@ One line per decision: `[YYYY-MM-DD] <decision>: <why>`. Appended by agents and
 [2026-08-03] whatsapp-inbox raw_payload: the sender's plaintext E.164 number is stripped before persisting (redactRawPayload) and replies resolve the recipient by decrypting the link's phone_enc. Storing the number verbatim on every message row defeated the point of the AES-256-GCM column and contradicted the RoPA claim that it is never in the clear. Legacy rows still holding `from` keep working via a fallback read.
 [2026-08-05] Representation clarifying question has NO amount floor (removed the 150 kr gate after the Swedish compliance review on PR #1340): documenting deltagare + syfte is what makes a representation expense deductible at all (BFL 5 kap 6-7 §) and that duty is not conditioned on any sum; the 300 kr/person figure is the VAT-deduction base cap, an unrelated rule. Noise is bounded by the triggers instead (receipt-shaped + restaurant/cafe/hotel merchant, <=1 question per receipt, <=2 per burst, <=6 per sender per day, one "nej" dismisses).
 [2026-08-05] Dropped "ML 13 kap 8 §" cites for kontantmetoden VAT timing (comments/docs only): section is the old ML 1994:200 numbering; in ML 2023:200, 13 kap is input-VAT deduction. Rule stated without section cite until the current-law section is verified.
+[2026-08-05] The in-app assistant now reads invoice_inbox_items.channel_context (the answers a user gave in WhatsApp) as first-class underlag context, marked "uppgivna av användaren" and ranked above OCR output. Found in the field: the assistant asked for representation participants the user had typed into WhatsApp minutes earlier, because the intent's inbox query selected only document_id + extracted_data. Also backfilled by document_id, since a receipt can reach the intent through the document paths without the inbox row being matched to the transaction.
+[2026-08-05] The WhatsApp representation question now asks ONCE for a missing purpose instead of silently storing participants with purpose=null. Skatteverket wants participants AND purpose; accepting half and saying "Tack!" produced an undocumented deduction. Anti-loop: the follow-up fires only when no representation block exists yet, so a second incomplete answer is accepted as-is rather than nagging.
@@ -217,6 +217,60 @@ describe('answer flow (text rows through processInboundMessage)', () => {
    expect(confirm.inboxItemId).toBe('item-9')
  })

+  // Skatteverket needs participants AND purpose. An answer with only the
+  // names used to be accepted silently, leaving the deduction undocumented
+  // (found in the first live receipt, 2026-08-05).
+  it('participants without a purpose: asks once for the purpose and keeps the question open', async () => {
+    interpretMock.mockResolvedValue({
+      ok: true,
+      data: {
+        is_denial: false,
+        participants: [{ name: 'Elias Karlsson', company: 'Canguro Media' }],
+        purpose: null,
+        event_date: null,
+        note: null,
+      },
+    })
+    const { supabase, enqueue, findCalls } = createQueuedMockSupabase()
+    enqueue({ data: makeTextRow('Elias Karlsson från Canguro Media') })
+    enqueue({ data: { id: 'msg-t1' } })
+    enqueue({ data: makeLink() })
+    enqueue({ data: awaitingConversation('representation') })
+    enqueue({ data: openItemContext('representation') })
+    enqueue({ data: null }) // item update
+    enqueue({ data: null }) // conversation update
+    enqueue({ data: { company_id: 'company-1', correlation_id: null } })
+    enqueue({ data: awaitingConversation('representation', { state: 'idle', context: {} }) })
+    enqueue({ data: null }) // markStatus
+
+    await processInboundMessage(supabase as unknown as SupabaseClient, 'msg-t1')
+
+    const itemPatch = findCalls('invoice_inbox_items', 'update')[0][0] as {
+      channel_context: {
+        representation: { participants: unknown[]; purpose: string | null }
+        pending_question?: { status?: string }
+      }
+    }
+    // Names are kept, and the question stays OPEN for the missing half.
+    expect(itemPatch.channel_context.representation.participants).toHaveLength(1)
+    expect(itemPatch.channel_context.representation.purpose).toBeNull()
+    expect(itemPatch.channel_context.pending_question?.status).toBe('open')
+
+    // The follow-up asks only for the purpose, never for the names again.
+    const reply = sendTextMock.mock.calls[0][1]
+    expect(reply.template).toBe(TEMPLATE.m8RepNeedPurpose)
+    expect(reply.body).toContain('Elias Karlsson (Canguro Media)')
+    expect(reply.body.toLowerCase()).toContain('syftet')
+
+    // Conversation must NOT go idle, or the next reply draws the fallback.
+    const conversationPatch = findCalls('whatsapp_conversations', 'update')[0][0] as {
+      state: string
+      context: { pending_question?: unknown }
+    }
+    expect(conversationPatch.state).toBe('awaiting_representation')
+    expect(conversationPatch.context.pending_question).toBeDefined()
+  })
+
  it('garbage interpretation degrades to raw-note storage + M8 partial (no retry, no error)', async () => {
    interpretMock.mockResolvedValue({ ok: false })
    const { supabase, enqueue, findCalls } = createQueuedMockSupabase()
@@ -40,6 +40,7 @@ export const TEMPLATE = {
  m6BytPin: 'm6_byt_pin',
  m7Representation: 'm7_representation',
  m8RepConfirmed: 'm8_rep_confirmed',
+  m8RepNeedPurpose: 'm8_rep_need_purpose',
  m8RepPartial: 'm8_rep_partial',
  m8RepDenied: 'm8_rep_denied',
  m9Resend: 'm9_resend',
@@ -133,6 +134,11 @@ const SV = {
  m8RepConfirmed: ({ participants, purpose }: { participants: string; purpose?: string | null }) =>
    `Tack! Noterat: ${participants}${purpose ? ` · Syfte: ${purpose}` : ''}. Följer med när kvittot bokförs.`,

+  // Participants captured, purpose missing. Skatteverket wants both, so ask
+  // once for the missing half only, never for the names again.
+  m8RepNeedPurpose: ({ participants }: { participants: string }) =>
+    `Tack! Noterat: ${participants}. En sak till: vad var syftet med mötet? Skriv en kort rad, t.ex. _uppföljning av avtal_.`,
+
  m8RepPartial: () =>
    'Tack! Jag har sparat ditt svar som anteckning på kvittot. Kolla att deltagare och syfte kom med när du bokför i appen.',

@@ -261,6 +267,9 @@ const EN: typeof SV = {
  m8RepConfirmed: ({ participants, purpose }: { participants: string; purpose?: string | null }) =>
    `Thanks! Noted: ${participants}${purpose ? ` · Purpose: ${purpose}` : ''}. It follows the receipt when it is booked.`,

+  m8RepNeedPurpose: ({ participants }: { participants: string }) =>
+    `Thanks! Noted: ${participants}. One more thing: what was the purpose of the meeting? A short line is enough, e.g. _contract follow-up_.`,
+
  m8RepPartial: () =>
    'Thanks! I saved your reply as a note on the receipt. Check that attendees and purpose came through when you book it in the app.',

@@ -1044,6 +1044,9 @@ async function processAnswerMessage(
    const nowIso = new Date().toISOString()
    let confirmBody: string
    let confirmTemplate: TemplateId
+    /** Set when the answer named participants but no purpose: the question
+     *  stays open for exactly one targeted follow-up. */
+    let keepOpenForPurpose = false

    if (target.type === 'representation' && text.toLowerCase() === 'nej') {
      // Exact 'nej' short-circuits WITHOUT an LLM call.
@@ -1096,25 +1099,48 @@ async function processAnswerMessage(
          ((interpretation.data.participants?.length ?? 0) > 0 || interpretation.data.purpose)
        ) {
          const data = interpretation.data
-          await updateItemContext(supabase, target.inboxItemId, (itemContext) => ({
-            ...itemContext,
-            representation: {
-              participants: data.participants ?? [],
-              purpose: data.purpose,
-              event_date: data.event_date,
-              raw_answer: text,
-              answered_at: nowIso,
-            },
-            ...(data.note ? { user_note: data.note } : {}),
-            pending_question: itemContext.pending_question
-              ? { ...itemContext.pending_question, status: 'answered' }
-              : undefined,
-          }))
-          confirmBody = copy.m8RepConfirmed({
-            participants: renderParticipants(data.participants ?? []),
-            purpose: data.purpose,
+          await updateItemContext(supabase, target.inboxItemId, (itemContext) => {
+            // Skatteverket wants participants AND purpose; an answer with
+            // only the names used to be accepted silently, leaving the
+            // deduction undocumented. Ask once more for the missing half,
+            // and only once: if a previous answer already wrote a
+            // representation block, take whatever we have and stop.
+            const isFirstAnswer = itemContext.representation == null
+            keepOpenForPurpose =
+              isFirstAnswer && (data.participants?.length ?? 0) > 0 && !data.purpose
+            return {
+              ...itemContext,
+              representation: {
+                participants: data.participants ?? [],
+                // A follow-up must not erase a purpose captured earlier.
+                purpose: data.purpose ?? itemContext.representation?.purpose ?? null,
+                event_date: data.event_date ?? itemContext.representation?.event_date ?? null,
+                raw_answer: itemContext.representation
+                  ? `${itemContext.representation.raw_answer}\n${text}`
+                  : text,
+                answered_at: nowIso,
+              },
+              ...(data.note ? { user_note: data.note } : {}),
+              pending_question: itemContext.pending_question
+                ? {
+                    ...itemContext.pending_question,
+                    status: keepOpenForPurpose ? 'open' : 'answered',
+                  }
+                : undefined,
+            }
          })
-          confirmTemplate = TEMPLATE.m8RepConfirmed
+          if (keepOpenForPurpose) {
+            confirmBody = copy.m8RepNeedPurpose({
+              participants: renderParticipants(data.participants ?? []),
+            })
+            confirmTemplate = TEMPLATE.m8RepNeedPurpose
+          } else {
+            confirmBody = copy.m8RepConfirmed({
+              participants: renderParticipants(data.participants ?? []),
+              purpose: data.purpose ?? null,
+            })
+            confirmTemplate = TEMPLATE.m8RepConfirmed
+          }
        } else {
          await updateItemContext(supabase, target.inboxItemId, (itemContext) => ({
            ...itemContext,
@@ -1165,10 +1191,17 @@ async function processAnswerMessage(
        recent_questions: markRecentQuestion(currentContext, target.inboxItemId, 'answered'),
      }
      nextState = current.state
-      if (!target.late && currentContext.pending_question?.inbox_item_id === target.inboxItemId) {
+      if (
+        !target.late &&
+        !keepOpenForPurpose &&
+        currentContext.pending_question?.inbox_item_id === target.inboxItemId
+      ) {
        delete patchContext.pending_question
        nextState = 'idle'
      }
+      // keepOpenForPurpose: leave the pending question and the awaiting
+      // state in place so the next free-text reply routes back to this same
+      // receipt instead of drawing the M16 fallback.
      return { state: nextState, context: patchContext }
    })

@@ -1,5 +1,6 @@
 import { describe, it, expect } from 'vitest'
 import { transactionCategorization } from '../transaction-categorization'
+import type { InboxChannelContext } from '@/types'

 // Locks in the prose-drift fix from /Users/jakobwennberg/.claude/plans/.
 // The promptTemplate must instruct the agent to narrate using CATEGORY
@@ -23,6 +24,7 @@ function stripUuidsAndDates(text: string): string {
 function renderPrompt(opts: {
  hasUnderlag: boolean
  profileSummary?: string | null
+  chatAnswers?: InboxChannelContext | null
 }) {
  const captured = {
    transaction: {
@@ -46,6 +48,7 @@ function renderPrompt(opts: {
            is_restaurant: null,
            is_systembolaget: null,
            raw_extraction: null,
+            chat_answers: opts.chatAnswers ?? null,
          },
        ]
      : [],
@@ -183,3 +186,90 @@ describe('transaction.categorization prompt template', () => {
    expect(withUnderlag).toMatch(/bifogas till\s+VERIFIKATIONEN/)
  })
 })
+
+// Field report 2026-08-05: a user answered the representation question in
+// WhatsApp, then the in-app assistant asked for the same names again because
+// invoice_inbox_items.channel_context never reached the prompt.
+describe('chat answers from another channel', () => {
+  const chatAnswers: InboxChannelContext = {
+    channel: 'whatsapp',
+    representation: {
+      participants: [
+        { name: 'Elias Karlsson', company: 'Canguro Media' },
+        { name: 'Jakob Wennberg', company: 'Arcim Technology AB' },
+      ],
+      purpose: 'Lunch med samarbetspartner',
+      event_date: null,
+      raw_answer: 'Elias Karlsson från Canguro Media, Jakob Wennberg från Arcim',
+      answered_at: '2026-08-05T14:29:00.000Z',
+    },
+  }
+
+  it('puts the participants the user already gave into the prompt', () => {
+    const out = renderPrompt({ hasUnderlag: true, chatAnswers })
+    expect(out).toContain('Elias Karlsson (Canguro Media)')
+    expect(out).toContain('Jakob Wennberg (Arcim Technology AB)')
+    expect(out).toContain('Lunch med samarbetspartner')
+  })
+
+  it('marks them as human-supplied and forbids re-asking', () => {
+    const out = renderPrompt({ hasUnderlag: true, chatAnswers })
+    expect(out).toMatch(/uppgivna av användaren/)
+    expect(out).toMatch(/Fråga ALDRIG om något som redan står där/)
+  })
+
+  it('names the missing half when only participants were captured', () => {
+    const out = renderPrompt({
+      hasUnderlag: true,
+      chatAnswers: {
+        ...chatAnswers,
+        representation: { ...chatAnswers.representation!, purpose: null },
+      },
+    })
+    expect(out).toMatch(/syfte SAKNAS/)
+    expect(out).toContain('Elias Karlsson (Canguro Media)')
+  })
+
+  it('says nothing about chat answers when there are none', () => {
+    const out = renderPrompt({ hasUnderlag: true })
+    expect(out).not.toMatch(/uppgivna av användaren/)
+  })
+})
+
+// The prompt tests above inject chat_answers directly, so they would still
+// pass if capture() never selected the column: which is exactly the bug that
+// shipped. This covers the query half.
+describe('transaction.categorization capture', () => {
+  function supabaseStub() {
+    const selects: string[] = []
+    const from = (table: string) => {
+      const chain: Record<string, unknown> = {}
+      const self = () => chain
+      chain.select = (cols: string) => {
+        selects.push(`${table}:${cols}`)
+        return chain
+      }
+      for (const m of ['eq', 'in', 'not']) chain[m] = self
+      chain.single = async () => ({ data: { id: 'tx-1', date: '2026-08-05', description: 'X', amount: -425, currency: 'SEK', document_id: null, journal_entry_id: null } })
+      chain.maybeSingle = async () => ({ data: null })
+      chain.then = (resolve: (v: unknown) => unknown) =>
+        resolve({
+          data: table === 'invoice_inbox_items'
+            ? [{ document_id: 'doc-1', extracted_data: {}, channel_context: { channel: 'whatsapp', user_note: 'lunch med kund' } }]
+            : [],
+        })
+      return chain
+    }
+    return { from, selects }
+  }
+
+  it('selects channel_context and threads it onto the underlag', async () => {
+    const stub = supabaseStub()
+    const captured = await transactionCategorization.capture!(
+      { transaction_id: 'tx-1' },
+      { supabase: stub as never, companyId: 'company-1', userId: 'user-1' } as never,
+    )
+    expect(stub.selects.some((s) => s.startsWith('invoice_inbox_items:') && s.includes('channel_context'))).toBe(true)
+    expect(captured.underlag[0]?.chat_answers).toEqual({ channel: 'whatsapp', user_note: 'lunch med kund' })
+  })
+})
@@ -1,5 +1,6 @@
 import { defineAgentIntent } from './types'
 import { SONNET_MODEL, EFFORT_STANDARD } from '@/lib/agent/composer/client'
+import type { InboxChannelContext } from '@/types'

 // transaction.categorization: "Fråga om denna transaktion" on a transaction
 // row.
@@ -44,6 +45,16 @@ interface CapturedTransaction {
    // agent can paraphrase context-specific signals (line items, dates,
    // payment reference) without us pre-modeling every field.
    raw_extraction: Record<string, unknown> | null
+    /**
+     * Answers the user already gave about THIS underlag in another channel
+     * (today: the WhatsApp intake conversation). Verified human input, not
+     * OCR guesswork, so it outranks anything read off the image.
+     *
+     * Without it the assistant asked for participant names the user had
+     * typed into WhatsApp minutes earlier, which is the worst thing a
+     * system that already holds the answer can do.
+     */
+    chat_answers: InboxChannelContext | null
  }[]
 }

@@ -113,7 +124,7 @@ export const transactionCategorization = defineAgentIntent<
        .eq('matched_transaction_id', transaction_id),
      supabase
        .from('invoice_inbox_items')
-        .select('document_id, extracted_data')
+        .select('document_id, extracted_data, channel_context')
        .eq('company_id', companyId)
        .eq('matched_transaction_id', transaction_id),
      directDocumentId
@@ -157,11 +168,13 @@ export const transactionCategorization = defineAgentIntent<
        is_restaurant: r.is_restaurant,
        is_systembolaget: r.is_systembolaget,
        raw_extraction: r.raw_extraction,
+        chat_answers: null,
      })
    }
    for (const it of (inboxItems ?? []) as {
      document_id: string | null
      extracted_data: Record<string, unknown> | null
+      channel_context: InboxChannelContext | null
    }[]) {
      const ex = it.extracted_data ?? {}
      const supplier = (ex.supplier as { name?: string | null } | undefined) ?? null
@@ -178,6 +191,7 @@ export const transactionCategorization = defineAgentIntent<
        is_restaurant: null,
        is_systembolaget: null,
        raw_extraction: ex,
+        chat_answers: it.channel_context ?? null,
      })
    }

@@ -221,6 +235,7 @@ export const transactionCategorization = defineAgentIntent<
          is_restaurant: null,
          is_systembolaget: null,
          raw_extraction: null,
+          chat_answers: null,
        })
        continue
      }
@@ -238,9 +253,37 @@ export const transactionCategorization = defineAgentIntent<
        is_restaurant: null,
        is_systembolaget: null,
        raw_extraction: ex,
+        chat_answers: null,
      })
    }

+    // Chat answers live on the inbox item, but an underlag can reach this
+    // point through the document paths above (the transaction's own
+    // document_id, or a doc anchored to the journal entry) without the
+    // inbox row ever being matched to the transaction. Backfill by
+    // document_id so the answers are found either way.
+    const missingContextDocIds = underlag
+      .filter((u) => u.chat_answers == null && u.document_id != null)
+      .map((u) => u.document_id as string)
+    if (missingContextDocIds.length > 0) {
+      const { data: byDoc } = await supabase
+        .from('invoice_inbox_items')
+        .select('document_id, channel_context')
+        .eq('company_id', companyId)
+        .in('document_id', missingContextDocIds)
+        .not('channel_context', 'is', null)
+      for (const row of (byDoc ?? []) as {
+        document_id: string | null
+        channel_context: InboxChannelContext | null
+      }[]) {
+        for (const u of underlag) {
+          if (u.document_id === row.document_id && u.chat_answers == null) {
+            u.chat_answers = row.channel_context ?? null
+          }
+        }
+      }
+    }
+
    return {
      transaction: tx
        ? {
@@ -311,9 +354,34 @@ export const transactionCategorization = defineAgentIntent<
        if (u.is_restaurant) parts.push('restaurang=ja')
        if (u.is_systembolaget) parts.push('systembolaget=ja')
        lines.push(`  • ${u.kind}: ${parts.join(', ') || '(ingen extraherad data: läs underlaget med gnubok_get_document_content)'}`)
+
+        // Answers the user already typed in another channel. Own indented
+        // block so it reads as human input, not one more OCR field.
+        const chat = u.chat_answers
+        if (chat) {
+          const rep = chat.representation
+          if (rep) {
+            const who = (rep.participants ?? [])
+              .map((pp) => (pp.company ? `${pp.name} (${pp.company})` : pp.name))
+              .join(', ')
+            if (who) lines.push(`    - deltagare (uppgivna av användaren): ${who}`)
+            if (rep.purpose) lines.push(`    - syfte (uppgivet av användaren): ${rep.purpose}`)
+            if (rep.event_date) lines.push(`    - datum (uppgivet av användaren): ${rep.event_date}`)
+            if (!rep.purpose) {
+              lines.push('    - syfte SAKNAS: fråga bara efter syftet, inte om deltagarna igen.')
+            }
+          }
+          if (chat.user_note) lines.push(`    - anteckning från användaren: ${chat.user_note}`)
+          if (chat.caption) lines.push(`    - bildtext vid inskick: ${chat.caption}`)
+        }
      }
      lines.push('')
      lines.push('VIKTIGT: extraktionen ovan är det vi REDAN VET. Återupprepa inte frågor som "vilken leverantör är det?" eller "vad var beloppet?": det står ovan. Använd uppgifterna direkt och föreslå kategori + moms-behandling.')
+      // Conditional: an unconditional line is prompt bloat on the common
+      // transaction that has no chat history.
+      if (captured.underlag.some((u) => u.chat_answers != null)) {
+        lines.push('Rader märkta "uppgivna av användaren" kommer från en tidigare konversation om samma underlag (t.ex. WhatsApp när kvittot skickades in). Det är MÄNSKLIGT bekräftade uppgifter och väger tyngre än vad du själv läser ut ur bilden. Fråga ALDRIG om något som redan står där; behöver du komplettera, fråga bara om den del som faktiskt saknas. När du stagear: ta med deltagare och syfte i notes så de följer med till verifikationen.')
+      }
    }
    lines.push('')
    lines.push('Arbetssätt: hämta information via verktygsanrop FÖRST (tyst: statusraderna visar att du söker, och ditt resonemang sker i tankekanalen), föreslå sedan. Skriv din förklaring EN gång efteråt, inte i flera block runt anropen.')