Files
accounted/extensions/general/whatsapp-inbox/lib/conversation.ts
T
88760ae6f6 fix(whatsapp-inbox): harden against adversarial review findings (#1342)
* fix(whatsapp-inbox): erase the WhatsApp channel on account deletion

whatsapp_phone_links relied on the auth.users ON DELETE CASCADE, but
Accounted never deletes auth.users: account deletion is
anonymize_user_account plus a ~100-year ban that keeps the auth row as a
tombstone, so the cascade never fires and nothing revokes the link. After
erasure the link stayed active with a decryptable phone_enc,
lookupActiveLink kept resolving the number, and every further inbound
message was persisted with body_text and the verbatim raw_payload while
the bot kept replying: GDPR Art 17 plus continued collection with no
lawful basis.

The RPC is re-created verbatim from 20260724150000 with one added block
that revokes and crypto-shreds the link, resets its conversation, nulls
body_text/raw_payload on that link's messages and deletes outstanding
link codes, plus a guarded repair pass for tombstones anonymized before
this migration. Covered by a pg-real test that fails against the previous
definition.

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>

* fix(whatsapp-inbox): pepper the link-code hash and bound code minting

hashLinkCode stored a bare sha256 over CODE_ALPHABET^6 = 30^6 values
behind a fixed 'AC-' prefix. The module cited the invite-token pattern,
but invite tokens are 256-bit random; this space enumerates offline in
about a second, so hashing at rest protected nothing. The sibling
phone-crypto.ts already states the team's own threat model for a LARGER
space ("a plain sha256 would be brute-forceable ... hence the pepper"),
so link codes now hash through the same env-mandated pepper.

/link/start was also an authenticated unbounded INSERT that left every
earlier code valid. Minting now burns the caller's unused codes (the code
the panel shows is the only one that works) and is capped per TTL window,
with the route answering 429 instead of throwing.

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>

* fix(whatsapp-inbox): harden the conversation layer against the review findings

Pre-merge hardening of the unshipped chat layer. Every change below has a
test that fails without it.

Lifecycle and races:
- conversation writes go through updateConversation(), an optimistic
  compare-and-set on updated_at (the trigger makes it a revision counter).
  The ack winner, the answer worker, the pin refresh and the sweep hold
  different claims, so blind whole-jsonb writes resurrected answered
  questions, wiped pending_question and dropped queue entries.
- terminal markStatus writes are guarded on processing_status='processing'
  so a losing worker cannot overwrite the winner's 'done' and null its
  inbox_item_id.
- the message -> inbox item path is idempotent: a pre-check plus a 23505
  fallback adopt the item a concurrent worker created, instead of throwing
  after the WORM document is already committed.
- PROCESSING_STUCK_MS 90s -> 5 min. The enforced step budget of one media
  row already exceeds 90s, so the sweep was re-claiming live workers.
- sweep 2b re-arms only when the conversation itself has been quiet, not
  just the rows: pending_ack=false plus unacked rows is also the state of a
  live finalize, which produced a duplicate combined ack.
- pin expiry re-checks against fresh state instead of writing back a stale
  whole context, which reverted company choices applied mid-pass.
- askNextQueuedQuestion claims the pop before sending, so two answer
  workers cannot ask the same question twice.

Company question:
- the state is rolled back when the M6 send fails, so the next receipt
  re-asks instead of parking receipts behind a question nobody received.
- applyCompanyChoice claims the open question (company_options) rather
  than the state: a double tap confirms once, a transient membership-query
  error is no longer read as "not a member", and a LATE answer still lands.
- at the 48h TTL the parked receipts are kept, not discarded: options and
  staged rows survive so a late digit or tap still files them, and only
  rows past Meta's ~30-day media window get the terminal marker.
- an out-of-range digit or a typed company name now gets the options
  repeated instead of silence or the "I cannot answer questions" reply.

Inline dispositions:
- stop/start/byt/company answers run their side effect BEFORE the terminal
  wamid row, with a SELECT pre-check for dedupe. Writing the row 'done'
  first made them at-most-once: a crash in between lost the action forever.

Copy and answers:
- acks state the extracted currency instead of labelling every total 'kr'.
- M17 stops promising "about 10 minutes" when the daily quota tripped.
- M18 is sent once per message tracked by the outbound row, so a file
  whose first attempt died still reaches the sender, including from the
  max-attempts path.
- M11 no longer claims the number is disconnected: 'stopp' pauses, and
  muted senders now persist no chat content at all.
- 'byt' is recognized in every state but awaiting_company (m6-confirm
  teaches the word, and it was being stored as answer data instead).
- text sent while a re-send question is open is kept as a note on THAT
  receipt with the question left open, instead of binding to another
  receipt's question.
- a quoted reply wins over the pending question and is appended when the
  quoted question is already answered, so corrections stop landing on the
  wrong receipt.
- context answers keep raw_answer + answered_at like representation does.
- finalizeBurst checks the send result: on failure it rolls the question
  back and leaves the rows unacked for the sweep.

PII:
- the sender's plaintext number is stripped from raw_payload before it is
  persisted; replies decrypt the link's phone_enc instead.

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>

* docs(whatsapp-inbox): record the erasure path and the hardening decisions

RoPA gains the account-deletion row (immediate, not via the cron: the
auth.users cascade never fires because the row is tombstoned) plus the
two new security measures, and its "never in the clear" phone claim is
now true of the stored payload. DECISIONS.md records the non-obvious
calls: revoke-not-delete on erasure, commit-then-roll-back for the
company question, keeping expired company choices answerable, the
compare-and-set conversation write, effect-before-terminal-row for inline
dispositions, honest M11 copy, and the raw_payload redaction.

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>

* fix(whatsapp-inbox): stop the answer re-claim from following a confirm with M16

A worker that died after applying an answer and sending its confirmation
leaves the row 'processing'. The sweep re-runs it, resolveAnswerTarget
finds the question already answered, and the user got "I did not
understand" immediately after the confirmation they had just received.
The fallback is now first-attempt only.

The catch comment claiming the sweep retries these rows is corrected
too: 'error' is terminal for the sweep, and nothing on the answer path
throws anyway (interpretChatAnswer degrades, sends never throw,
supabase-js returns errors), so the catch is a programming-error net.

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>

* fix(whatsapp-inbox): drop the amount floor on the representation question

The Swedish compliance review on #1340 caught a real error in the trigger
rules: the representation question only fired above 150 kr, but the duty to
document deltagare and syfte is what makes the expense deductible at all
(BFL 5 kap 6-7 §) and it is not conditioned on any amount. The 300 kr per
person figure I had in mind is the VAT-deduction base cap, a different rule.
A 120 kr business lunch would have been booked with no participant trail,
which is exactly the deduction Skatteverket denies later.

Noise stays bounded by the triggers that were already there: the question
fires only for receipt-shaped documents from restaurant, cafe or hotel
merchants, at most once per receipt, twice per burst and six times per
sender per day, and a single "nej" dismisses it.

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>

---------

Co-authored-by: Jakob Wennberg <jakob.wennberg@gmail.com>
Co-authored-by: Claude Fable 5 <noreply@anthropic.com>
Co-authored-by: Jakob Wennberg <311770904+jakobwennberg-oss@users.noreply.github.com>
2026-08-05 15:47:23 +02:00

419 lines
16 KiB
TypeScript

/**
* Deterministic conversation state helpers (LLM-last: everything here is
* plain code over whatsapp_conversations).
*
* The `context` jsonb column carries, per conversation:
* - pin_expires_at 8h sliding company-pin TTL (companion of company_id)
* - pin_source how the live pin was chosen (button/list/numbered)
* - company_options ordered options behind an open company question,
* so a digit reply maps to a company id
* - pending_question the ONE in-flight question (type + item + asked_at)
* - question_queue questions admitted for later (burst budget of 2)
* - recent_questions asked-question log for late answers (<= 7 days)
* - budget daily content-question counter (Europe/Stockholm)
*
* Concurrency note: context is read-modify-write via PostgREST, and the
* writers (burst-ack winner, answer worker, company-choice handler, sweep)
* hold DIFFERENT claims (pending_ack vs processing_status), so they are not
* serialized against each other. Every context write therefore goes through
* updateConversation(), which guards on the updated_at it read and re-applies
* the mutation against fresh state when it loses. Media staging deliberately
* does NOT go through context (staged refs live as whatsapp_messages rows) so
* parallel webhook invocations never race on this column at all.
*/
import type { SupabaseClient } from '@supabase/supabase-js'
import { createLogger } from '@/lib/logger'
import type { WhatsAppConversation, WhatsAppMessage } from '@/types'
import { decryptPhone } from './phone-crypto'
const log = createLogger('whatsapp-inbox/conversation')
export const COMPANY_PIN_TTL_MS = 8 * 60 * 60 * 1000
export const QUESTION_TTL_MS = 48 * 60 * 60 * 1000
export const LATE_ANSWER_MAX_AGE_MS = 7 * 24 * 60 * 60 * 1000
export const DEBOUNCE_WINDOW_MS = 12 * 1000
export const MAX_QUESTIONS_PER_BURST = 2
export const MAX_QUESTIONS_PER_DAY = 6
/** error_message marker on whatsapp_messages rows parked while the company
* question is open. The answer handler re-opens exactly these. */
export const STAGED_AWAITING_COMPANY = 'staged_awaiting_company'
/** Marker after the 48h TTL expired: excluded from any later re-open. */
export const COMPANY_CHOICE_EXPIRED = 'company_choice_expired'
export type QuestionType = 'representation' | 'context' | 'resend'
export type ConversationQuestionType = QuestionType | 'company'
export interface PendingQuestion {
type: ConversationQuestionType
/** Null for company questions (no item exists before the answer). */
inbox_item_id: string | null
asked_at: string
}
export interface QueuedQuestion {
type: QuestionType
inbox_item_id: string
}
export interface RecentQuestion {
type: QuestionType
inbox_item_id: string
asked_at: string
status: 'open' | 'answered' | 'moved_to_app'
}
export interface ConversationContext {
pin_expires_at?: string
pin_source?: 'button' | 'list' | 'numbered'
company_options?: { id: string; name: string }[]
pending_question?: PendingQuestion
question_queue?: QueuedQuestion[]
recent_questions?: RecentQuestion[]
budget?: { day_key: string; count: number }
}
export function getContext(conversation: WhatsAppConversation): ConversationContext {
return (conversation.context ?? {}) as ConversationContext
}
/** Load (or lazily create) the conversation row for a phone link. */
export async function getOrCreateConversation(
supabase: SupabaseClient,
phoneLinkId: string,
): Promise<WhatsAppConversation | null> {
const { data: existing } = await supabase
.from('whatsapp_conversations')
.select('*')
.eq('phone_link_id', phoneLinkId)
.maybeSingle()
if (existing) return existing as WhatsAppConversation
const { data: created } = await supabase
.from('whatsapp_conversations')
.insert({ phone_link_id: phoneLinkId })
.select('*')
.maybeSingle()
return (created as WhatsAppConversation | null) ?? null
}
export async function loadConversation(
supabase: SupabaseClient,
conversationId: string,
): Promise<WhatsAppConversation | null> {
const { data } = await supabase
.from('whatsapp_conversations')
.select('*')
.eq('id', conversationId)
.maybeSingle()
return (data as WhatsAppConversation | null) ?? null
}
/** Fields a conversation write may touch. Anything omitted stays as it is. */
export interface ConversationPatch {
state?: WhatsAppConversation['state']
context?: ConversationContext
company_id?: string | null
last_outbound_at?: string
pending_ack?: boolean
debounce_until?: string
}
/**
* Optimistic-concurrency write of a conversation row.
*
* `context` is a whole-jsonb column written read-modify-write, and its writers
* hold different claims (the ack winner holds pending_ack, an answer worker
* holds its own row's processing_status, the sweep holds nothing), so a blind
* `.eq('id')` update silently clobbers whatever landed in between: resurrected
* questions, wiped pending_question, dropped queue entries.
*
* The updated_at trigger makes that column a revision counter, so guarding on
* the value the mutation was derived from turns every write into a
* compare-and-set. On a lost race the row is reloaded and `mutate` runs again
* against fresh state; returning null from it aborts (the work is already
* done, or no longer applies).
*/
export async function updateConversation(
supabase: SupabaseClient,
conversation: WhatsAppConversation,
mutate: (
current: WhatsAppConversation,
context: ConversationContext,
) => ConversationPatch | null,
maxAttempts = 4,
): Promise<WhatsAppConversation | null> {
let current: WhatsAppConversation | null = conversation
for (let attempt = 0; attempt < maxAttempts; attempt++) {
if (!current) return null
const patch = mutate(current, getContext(current))
if (!patch) return null
// Literal payload so the phantom-column scanner can verify the column
// names; undefined values drop out of the JSON body PostgREST receives.
const { data } = await supabase
.from('whatsapp_conversations')
.update({
state: patch.state,
context: patch.context as Record<string, unknown> | undefined,
company_id: patch.company_id,
last_outbound_at: patch.last_outbound_at,
pending_ack: patch.pending_ack,
debounce_until: patch.debounce_until,
})
.eq('id', current.id)
.eq('updated_at', current.updated_at)
.select('*')
// PostgREST returns [] only when the revision guard did not match.
if (!Array.isArray(data) || data.length > 0) {
const written = Array.isArray(data) ? (data[0] as WhatsAppConversation) : null
return written ?? ({ ...current, ...patch } as WhatsAppConversation)
}
current = await loadConversation(supabase, current.id)
}
log.warn('conversation write gave up after concurrent modifications', {
conversationId: conversation.id,
})
return null
}
/**
* Atomic burst-ack claim. Exactly one caller per debounce window gets a row
* back; everyone else stays silent. `debounce_until <= now` uses the caller's
* clock: harmless skew only shifts WHEN the winner fires, never how many win
* (pending_ack is the exclusivity bit).
*/
export async function claimAck(
supabase: SupabaseClient,
conversationId: string,
): Promise<boolean> {
const { data } = await supabase
.from('whatsapp_conversations')
.update({ pending_ack: false })
.eq('id', conversationId)
.eq('pending_ack', true)
.lte('debounce_until', new Date().toISOString())
.select('id')
return Array.isArray(data) && data.length > 0
}
/** Day key for the daily question budget, in the sender's civil day. */
export function stockholmDayKey(date: Date = new Date()): string {
return new Intl.DateTimeFormat('sv-SE', {
timeZone: 'Europe/Stockholm',
year: 'numeric',
month: '2-digit',
day: '2-digit',
}).format(date)
}
/** Content questions already asked today (rolls over at Stockholm midnight). */
export function questionsAskedToday(context: ConversationContext, now: Date = new Date()): number {
const budget = context.budget
if (!budget || budget.day_key !== stockholmDayKey(now)) return 0
return budget.count
}
export function bumpBudget(
context: ConversationContext,
asked: number,
now: Date = new Date(),
): ConversationContext['budget'] {
const dayKey = stockholmDayKey(now)
const current = context.budget?.day_key === dayKey ? context.budget.count : 0
// Negative `asked` refunds a question whose send failed; never below zero.
return { day_key: dayKey, count: Math.max(0, current + asked) }
}
/** True when the company pin on the conversation is present and unexpired. */
export function hasLivePin(
conversation: WhatsAppConversation,
now: Date = new Date(),
): boolean {
if (!conversation.company_id) return false
const expiresAt = getContext(conversation).pin_expires_at
if (!expiresAt) return false
return new Date(expiresAt).getTime() > now.getTime()
}
/** True when the 24h service window is open (we may send free-form replies). */
export function serviceWindowOpen(
conversation: WhatsAppConversation,
now: Date = new Date(),
): boolean {
const expiresAt = conversation.service_window_expires_at
return expiresAt != null && new Date(expiresAt).getTime() > now.getTime()
}
/** Question types a free-text reply can answer. Resend needs media. */
export type TextAnswerableQuestionType = Extract<QuestionType, 'representation' | 'context'>
export interface AnswerTarget {
type: TextAnswerableQuestionType
inboxItemId: string
/** True when matched outside an awaiting_* state (quoted or recent). */
late: boolean
/** The quoted receipt's question was already answered: this is a follow-up
* correction, appended to the note instead of overwriting the answer. */
followUp?: boolean
}
const TEXT_ANSWERABLE: ReadonlySet<string> = new Set(['representation', 'context'])
function isTextAnswerable(
question: RecentQuestion,
): question is RecentQuestion & { type: TextAnswerableQuestionType } {
return TEXT_ANSWERABLE.has(question.type)
}
/**
* Resolve which question a free-text message answers.
*
* 1. A quoted reply (context.message_id -> whatsapp_messages.wamid, either
* direction) names its receipt, so it wins over everything else, including
* an unrelated pending question and including a question that receipt has
* already answered (the reply is then a follow-up correction). Binding a
* quoted correction to some OTHER receipt because the quoted one was
* settled is how the wrong item gets the note.
* 2. Otherwise an awaiting_representation/awaiting_context state answers its
* own pending question.
* 3. Otherwise the single most recent question asked within 7 days that has
* not been answered yet (moved_to_app still accepts a late answer).
*/
export async function resolveAnswerTarget(
supabase: SupabaseClient,
conversation: WhatsAppConversation,
quotedWamid: string | null,
now: Date = new Date(),
): Promise<AnswerTarget | null> {
const context = getContext(conversation)
const pending =
(conversation.state === 'awaiting_representation' ||
conversation.state === 'awaiting_context') &&
context.pending_question &&
TEXT_ANSWERABLE.has(context.pending_question.type) &&
context.pending_question.inbox_item_id
? {
type: context.pending_question.type as TextAnswerableQuestionType,
inboxItemId: context.pending_question.inbox_item_id,
}
: null
const withinWindow = (question: RecentQuestion): boolean =>
now.getTime() - new Date(question.asked_at).getTime() <= LATE_ANSWER_MAX_AGE_MS
if (quotedWamid) {
const { data: quotedRow } = await supabase
.from('whatsapp_messages')
.select('inbox_item_id')
.eq('wamid', quotedWamid)
.not('inbox_item_id', 'is', null)
.limit(1)
.maybeSingle()
const quotedItemId = (quotedRow as { inbox_item_id: string | null } | null)?.inbox_item_id
if (quotedItemId) {
// Quoting the very question that is open: an ordinary answer, not late.
if (pending && pending.inboxItemId === quotedItemId) {
return { ...pending, late: false }
}
const quoted = [...(context.recent_questions ?? [])]
.filter((q) => isTextAnswerable(q) && withinWindow(q) && q.inbox_item_id === quotedItemId)
.sort((a, b) => new Date(b.asked_at).getTime() - new Date(a.asked_at).getTime())[0]
if (quoted && isTextAnswerable(quoted)) {
return {
type: quoted.type,
inboxItemId: quoted.inbox_item_id,
late: true,
followUp: quoted.status === 'answered',
}
}
}
}
if (pending) return { ...pending, late: false }
const recent = (context.recent_questions ?? []).filter(
(q): q is RecentQuestion & { type: TextAnswerableQuestionType } =>
isTextAnswerable(q) && q.status !== 'answered' && withinWindow(q),
)
if (recent.length > 0) {
const latest = [...recent].sort(
(a, b) => new Date(b.asked_at).getTime() - new Date(a.asked_at).getTime(),
)[0]
return { type: latest.type, inboxItemId: latest.inbox_item_id, late: true }
}
return null
}
/** Mark one recent-question entry with a new status (pure helper). */
export function markRecentQuestion(
context: ConversationContext,
inboxItemId: string,
status: RecentQuestion['status'],
): RecentQuestion[] {
return (context.recent_questions ?? []).map((q) =>
q.inbox_item_id === inboxItemId ? { ...q, status } : q,
)
}
/** Append an asked question to the recent log, pruning entries beyond 7 days
* and capping the log at 10 entries. */
export function appendRecentQuestion(
context: ConversationContext,
entry: RecentQuestion,
now: Date = new Date(),
): RecentQuestion[] {
const kept = (context.recent_questions ?? []).filter(
(q) => now.getTime() - new Date(q.asked_at).getTime() <= LATE_ANSWER_MAX_AGE_MS,
)
return [...kept, entry].slice(-10)
}
/**
* Legacy recipient read: rows persisted before the raw payload was redacted
* still carry the sender's plaintext number under `from`. New rows do not,
* so this returns null for them and the caller decrypts the phone link.
*/
export function extractRecipient(row: WhatsAppMessage): string | null {
const raw = row.raw_payload as { from?: unknown } | null
return raw && typeof raw.from === 'string' && raw.from.length > 0 ? raw.from : null
}
/**
* The recipient phone (E.164 digits) for replies.
*
* The authoritative copy is whatsapp_phone_links.phone_enc (AES-256-GCM):
* persisting the plaintext number in every message's raw_payload defeated the
* whole point of encrypting it once on the link. Falls back to the legacy
* raw_payload copy for rows written before the redaction.
*/
export function resolveRecipient(
row: WhatsAppMessage,
link: { phone_enc?: string | null } | null,
): string | null {
const legacy = extractRecipient(row)
if (legacy) return legacy
if (!link?.phone_enc) return null
try {
const phone = decryptPhone(link.phone_enc)
return phone.length > 0 ? phone : null
} catch (err) {
// Shredded (retention/erasure sets phone_enc = '') or a key rotation:
// there is no one left to reply to, and that must not throw here.
log.warn('could not decrypt phone link for reply', {
error: err instanceof Error ? err.message : String(err),
})
return null
}
}
/** Strip the sender's plaintext phone number out of the payload we persist.
* Everything else (type, timestamp, quoted context.id) is kept verbatim. */
export function redactRawPayload(raw: unknown): Record<string, unknown> | null {
if (!raw || typeof raw !== 'object') return null
const { from: _from, ...rest } = raw as Record<string, unknown>
return rest
}