feat(mcp): unify missing-document surfaces on one predicate (#876)
* feat(mcp): unify missing-document surfaces on one predicate (P1-3) The two MCP surfaces told different truths: the transactions tool keyed 'has underlag' on transactions.document_id while the verifikat tool keyed on document_attachments — and neither respected the source-type semantics, version chains, or journal_entry_no_doc_required waivers that lib/worklist's canonical count applies. Measured on prod: 22,046 waived verifikat still listed to agents, 2,370 doc-exempt source types listed, ~87 docs attached to transactions but never propagated to the verifikat, 1,100 transactions flagged missing-receipt although their verifikat HAS the underlag. One predicate now lives in SQL — posted, needs-doc source type (mirrors NEEDS_DOC_SOURCE_TYPES), no current-version doc, no waiver: - verifikat_without_documents RPC v2 adopts the canonical predicate. - New transactions_without_documents RPC: the bank-driven subset of the same predicate, joined through transactions.journal_entry_id — a strict subset of the verifikat surface by construction. Rows expose qualified transaction_id (P1-2 forward-compat); bare id deprecated. - Both tools become thin RPC wrappers; descriptions state the actual set relationship. - lib/worklist countVerifikatMissingDocument delegates to the RPC (previously three full-table pulls set-differenced client-side) — badge count and agent surfaces can no longer drift. - Backfill: propagate transaction-attached docs to their verifikat where the attachment was never linked (open periods only; never steals a doc linked to another verifikat). pg-real: fixture matrix (no-doc/with-doc/waived/stale-version/ doc-exempt-source/import), strict-subset assertion, per-source-type pin of the SQL list against the TS constant, tenant guard. Part of dev_docs/mcp_optimization_plan.md (P1-3). Co-Authored-By: Claude Fable 5 <noreply@anthropic.com> * docs(mcp): explicit grants restated + count-call comment (#876 review) - Restate REVOKE/GRANT on verifikat_without_documents so the migration is self-contained (CREATE OR REPLACE preserves the 20260703130000 grants — verified on prod: authenticated + service_role only). - Comment on the p_limit:1 count call: total_count is computed over the full filtered set, independent of page size. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com> --------- Co-authored-by: Claude Fable 5 <noreply@anthropic.com>
This commit is contained in:
Jakob Wennberg
+75
-95
@@ -2,149 +2,129 @@ import { describe, it, expect, beforeEach, vi } from 'vitest'
|
||||
import { createQueuedMockSupabase } from '@/tests/helpers'
|
||||
import { tools } from '../server'
|
||||
|
||||
/**
|
||||
* The tool is a thin wrapper over the transactions_without_documents RPC —
|
||||
* the bank-driven subset of the verifikat surface, keyed on the SAME document
|
||||
* truth (document_attachments + waivers), never transactions.document_id.
|
||||
* Predicate semantics are pinned by
|
||||
* tests/pg/document-surfaces-unification.pg.test.ts; these tests cover the
|
||||
* wrapper contract (envelope unwrap, pagination math, error paths).
|
||||
*/
|
||||
const tool = tools.find((t) => t.name === 'gnubok_list_transactions_without_documents')!
|
||||
|
||||
function envelope(transactions: unknown[], totalCount: number) {
|
||||
return { data: { ok: true, total_count: totalCount, transactions }, error: null }
|
||||
}
|
||||
|
||||
const row = (id: string, jeId: string) => ({
|
||||
id,
|
||||
transaction_id: id,
|
||||
date: '2026-04-12',
|
||||
description: 'HOTELL ANGLAIS',
|
||||
amount: -1247,
|
||||
currency: 'SEK',
|
||||
merchant_name: null,
|
||||
reference: null,
|
||||
is_business: null,
|
||||
category: null,
|
||||
journal_entry_id: jeId,
|
||||
})
|
||||
|
||||
beforeEach(() => {
|
||||
vi.clearAllMocks()
|
||||
})
|
||||
|
||||
describe('gnubok_list_transactions_without_documents', () => {
|
||||
it('is registered as a read-only paginated tool', () => {
|
||||
it('is registered as a read-only paginated tool with qualified transaction_id', () => {
|
||||
expect(tool).toBeDefined()
|
||||
expect(tool.annotations?.readOnlyHint).toBe(true)
|
||||
const schema = tool.outputSchema as Record<string, unknown>
|
||||
expect((schema.properties as Record<string, unknown>).transactions).toBeDefined()
|
||||
expect((schema.properties as Record<string, unknown>).total_count).toBeDefined()
|
||||
const schema = tool.outputSchema as { properties: Record<string, unknown> }
|
||||
expect(schema.properties.transactions).toBeDefined()
|
||||
expect(schema.properties.total_count).toBeDefined()
|
||||
const items = (schema.properties.transactions as { items: { properties: Record<string, unknown> } })
|
||||
.items
|
||||
expect(items.properties.transaction_id).toBeDefined()
|
||||
expect(items.properties.journal_entry_id).toBeDefined()
|
||||
})
|
||||
|
||||
it('returns booked transactions that have no document attached', async () => {
|
||||
const rows = [
|
||||
{
|
||||
id: 't1',
|
||||
date: '2026-04-12',
|
||||
description: 'HOTELL ANGLAIS',
|
||||
amount: -1247,
|
||||
currency: 'SEK',
|
||||
merchant_name: 'Hotell Anglais',
|
||||
reference: null,
|
||||
is_business: true,
|
||||
category: 'travel',
|
||||
journal_entry_id: 'je-1',
|
||||
},
|
||||
]
|
||||
it('unwraps the RPC envelope and passes filters through', async () => {
|
||||
const { supabase, enqueue } = createQueuedMockSupabase()
|
||||
enqueue({ data: null, error: null, count: 1 }) // count query
|
||||
enqueue({ data: rows, error: null }) // data query
|
||||
enqueue(envelope([row('t1', 'je-1')], 1))
|
||||
|
||||
const result = (await tool.execute(
|
||||
{ limit: 20 },
|
||||
{ limit: 20, since: '2026-01-01' },
|
||||
'company-1',
|
||||
'user-1',
|
||||
supabase as never
|
||||
supabase as never,
|
||||
)) as {
|
||||
transactions: typeof rows
|
||||
transactions: Array<{ id: string; transaction_id: string; journal_entry_id: string }>
|
||||
count: number
|
||||
total_count: number
|
||||
has_more: boolean
|
||||
}
|
||||
|
||||
expect(supabase.rpc).toHaveBeenCalledWith('transactions_without_documents', {
|
||||
p_company_id: 'company-1',
|
||||
p_since: '2026-01-01',
|
||||
p_limit: 20,
|
||||
p_offset: 0,
|
||||
})
|
||||
expect(result.count).toBe(1)
|
||||
expect(result.total_count).toBe(1)
|
||||
expect(result.has_more).toBe(false)
|
||||
expect(result.transactions[0].id).toBe('t1')
|
||||
expect(result.transactions[0].transaction_id).toBe('t1')
|
||||
expect(result.transactions[0].journal_entry_id).toBe('je-1')
|
||||
})
|
||||
|
||||
it('returns rows when DB has null merchant_name, reference, is_business, category (MCP structured output)', async () => {
|
||||
const rows = [
|
||||
{
|
||||
id: 't-no-doc-1',
|
||||
date: '2026-03-17',
|
||||
description: 'Nolla skuld',
|
||||
amount: -2745,
|
||||
currency: 'SEK',
|
||||
merchant_name: null,
|
||||
reference: null,
|
||||
is_business: null,
|
||||
category: null,
|
||||
journal_entry_id: 'je-nolla-1',
|
||||
},
|
||||
]
|
||||
const { supabase, enqueue } = createQueuedMockSupabase()
|
||||
enqueue({ data: null, error: null, count: 1 })
|
||||
enqueue({ data: rows, error: null })
|
||||
|
||||
const result = (await tool.execute(
|
||||
{ limit: 20 },
|
||||
'company-1',
|
||||
'user-1',
|
||||
supabase as never
|
||||
)) as {
|
||||
transactions: typeof rows
|
||||
count: number
|
||||
total_count: number
|
||||
has_more: boolean
|
||||
}
|
||||
|
||||
expect(result.count).toBe(1)
|
||||
expect(result.transactions[0].journal_entry_id).toBe('je-nolla-1')
|
||||
expect(result.transactions[0].merchant_name).toBeNull()
|
||||
expect(result.transactions[0].reference).toBeNull()
|
||||
expect(result.transactions[0].is_business).toBeNull()
|
||||
expect(result.transactions[0].category).toBeNull()
|
||||
})
|
||||
|
||||
it('returns empty result when nothing matches', async () => {
|
||||
const { supabase, enqueue } = createQueuedMockSupabase()
|
||||
enqueue({ data: null, error: null, count: 0 })
|
||||
enqueue({ data: [], error: null })
|
||||
enqueue(envelope([], 0))
|
||||
|
||||
const result = (await tool.execute(
|
||||
{},
|
||||
'company-1',
|
||||
'user-1',
|
||||
supabase as never
|
||||
)) as { count: number; total_count: number; has_more: boolean }
|
||||
const result = (await tool.execute({}, 'company-1', 'user-1', supabase as never)) as {
|
||||
count: number
|
||||
total_count: number
|
||||
has_more: boolean
|
||||
}
|
||||
|
||||
expect(result.count).toBe(0)
|
||||
expect(result.total_count).toBe(0)
|
||||
expect(result.has_more).toBe(false)
|
||||
})
|
||||
|
||||
it('signals more pages with next_offset when total exceeds the page', async () => {
|
||||
const page = Array.from({ length: 20 }, (_, i) => ({
|
||||
id: `t${i}`,
|
||||
date: '2026-04-01',
|
||||
description: 'tx',
|
||||
amount: -100,
|
||||
currency: 'SEK',
|
||||
merchant_name: null,
|
||||
reference: null,
|
||||
is_business: true,
|
||||
category: null,
|
||||
journal_entry_id: `je-${i}`,
|
||||
}))
|
||||
it('signals more pages with next_offset advancing by rows consumed', async () => {
|
||||
const page = Array.from({ length: 20 }, (_, i) => row(`t${i}`, `je-${i}`))
|
||||
const { supabase, enqueue } = createQueuedMockSupabase()
|
||||
enqueue({ data: null, error: null, count: 50 })
|
||||
enqueue({ data: page, error: null })
|
||||
enqueue(envelope(page, 45))
|
||||
|
||||
const result = (await tool.execute(
|
||||
{ limit: 20, offset: 0 },
|
||||
{ limit: 20 },
|
||||
'company-1',
|
||||
'user-1',
|
||||
supabase as never
|
||||
)) as { has_more: boolean; next_offset?: number }
|
||||
supabase as never,
|
||||
)) as {
|
||||
has_more: boolean
|
||||
next_offset?: number
|
||||
}
|
||||
|
||||
expect(result.has_more).toBe(true)
|
||||
expect(result.next_offset).toBe(20)
|
||||
})
|
||||
|
||||
it('throws on database errors', async () => {
|
||||
it('throws on an RPC error', async () => {
|
||||
const { supabase, enqueue } = createQueuedMockSupabase()
|
||||
enqueue({ data: null, error: { message: 'connection refused' }, count: null })
|
||||
enqueue({ data: null, error: { message: 'connection refused' } })
|
||||
|
||||
await expect(
|
||||
tool.execute({}, 'company-1', 'user-1', supabase as never)
|
||||
).rejects.toThrow(/connection refused/)
|
||||
await expect(tool.execute({}, 'company-1', 'user-1', supabase as never)).rejects.toThrow(
|
||||
/connection refused/,
|
||||
)
|
||||
})
|
||||
|
||||
it('throws on a not-ok envelope (tenant guard)', async () => {
|
||||
const { supabase, enqueue } = createQueuedMockSupabase()
|
||||
enqueue({ data: { ok: false, code: 'TRANSACTIONS_WITHOUT_DOCUMENTS_FORBIDDEN' }, error: null })
|
||||
|
||||
await expect(tool.execute({}, 'company-1', 'user-1', supabase as never)).rejects.toThrow(
|
||||
/TRANSACTIONS_WITHOUT_DOCUMENTS_FORBIDDEN/,
|
||||
)
|
||||
})
|
||||
})
|
||||
|
||||
@@ -2663,7 +2663,7 @@ export const tools: McpTool[] = [
|
||||
{
|
||||
name: 'gnubok_list_transactions_without_documents',
|
||||
title: 'List Transactions Missing Receipts',
|
||||
description: 'List BANK TRANSACTIONS booked without an attached underlag. For imported/manual verifikat (no bank tx row) call gnubok_list_verifikat_without_documents — this tool only covers bank-driven entries.',
|
||||
description: 'List booked bank transactions whose verifikat lacks an underlag. Strict subset of gnubok_list_verifikat_without_documents (same document truth, waivers respected) — use that tool for full coverage incl. imported/manual verifikat.',
|
||||
inputSchema: {
|
||||
type: 'object',
|
||||
additionalProperties: false,
|
||||
@@ -2677,7 +2677,8 @@ export const tools: McpTool[] = [
|
||||
type: 'object',
|
||||
additionalProperties: false,
|
||||
properties: {
|
||||
id: { type: 'string' },
|
||||
id: { type: 'string', description: 'Deprecated — read transaction_id instead' },
|
||||
transaction_id: { type: 'string' },
|
||||
date: { type: 'string' },
|
||||
description: { type: 'string' },
|
||||
amount: { type: 'number' },
|
||||
@@ -2700,42 +2701,39 @@ export const tools: McpTool[] = [
|
||||
const offset = Math.max(0, Number(args.offset) || 0)
|
||||
const since = typeof args.since === 'string' ? args.since : null
|
||||
|
||||
let countQuery = supabase
|
||||
.from('transactions')
|
||||
.select('id', { count: 'exact', head: true })
|
||||
.eq('company_id', companyId)
|
||||
.not('journal_entry_id', 'is', null)
|
||||
.is('document_id', null)
|
||||
if (since) countQuery = countQuery.gte('date', since)
|
||||
|
||||
const { count: totalCount, error: countError } = await countQuery
|
||||
if (countError) throw new Error(`Database error: ${countError.message}`)
|
||||
|
||||
let dataQuery = supabase
|
||||
.from('transactions')
|
||||
.select(
|
||||
'id, date, description, amount, currency, merchant_name, reference, is_business, category, journal_entry_id'
|
||||
)
|
||||
.eq('company_id', companyId)
|
||||
.not('journal_entry_id', 'is', null)
|
||||
.is('document_id', null)
|
||||
if (since) dataQuery = dataQuery.gte('date', since)
|
||||
|
||||
const { data, error } = await dataQuery
|
||||
.order('date', { ascending: false })
|
||||
.range(offset, offset + limit - 1)
|
||||
|
||||
// Same document truth as the verifikat surface: the RPC keys "has
|
||||
// underlag" on document_attachments (current version) + waivers, never
|
||||
// transactions.document_id — the two columns diverged historically
|
||||
// (P1-3, dev_docs/mcp_optimization_plan.md) and this surface is the
|
||||
// bank-driven SUBSET of gnubok_list_verifikat_without_documents by
|
||||
// construction.
|
||||
const { data, error } = await supabase.rpc('transactions_without_documents', {
|
||||
p_company_id: companyId,
|
||||
p_since: since,
|
||||
p_limit: limit,
|
||||
p_offset: offset,
|
||||
})
|
||||
if (error) throw new Error(`Database error: ${error.message}`)
|
||||
|
||||
const total = totalCount ?? 0
|
||||
const hasMore = total > offset + (data?.length ?? 0)
|
||||
const result = data as {
|
||||
ok: boolean
|
||||
code?: string
|
||||
total_count?: number
|
||||
transactions?: unknown[]
|
||||
} | null
|
||||
if (!result?.ok) {
|
||||
throw new Error(`transactions_without_documents failed: ${result?.code ?? 'unknown error'}`)
|
||||
}
|
||||
|
||||
const rows = result.transactions ?? []
|
||||
const total = result.total_count ?? 0
|
||||
const hasMore = offset + rows.length < total
|
||||
return {
|
||||
transactions: data,
|
||||
count: data?.length ?? 0,
|
||||
transactions: rows,
|
||||
count: rows.length,
|
||||
total_count: total,
|
||||
has_more: hasMore,
|
||||
...(hasMore ? { next_offset: offset + (data?.length ?? 0) } : {}),
|
||||
...(hasMore ? { next_offset: offset + rows.length } : {}),
|
||||
}
|
||||
},
|
||||
},
|
||||
@@ -2743,7 +2741,7 @@ export const tools: McpTool[] = [
|
||||
{
|
||||
name: 'gnubok_list_verifikat_without_documents',
|
||||
title: 'List Verifikat Missing Documents',
|
||||
description: 'List POSTED journal entries (verifikat) that have no document_attachments row. Covers SIE-imported, manual and salary vouchers that the transactions-based tool misses. Newest first, paginated.',
|
||||
description: 'List posted verifikat that genuinely lack an underlag: needs-doc source types only, current document versions, user waivers respected. Superset of gnubok_list_transactions_without_documents (covers imported/manual too). Newest first, paginated.',
|
||||
inputSchema: {
|
||||
type: 'object',
|
||||
additionalProperties: false,
|
||||
|
||||
@@ -78,57 +78,27 @@ describe('countInboxDocuments', () => {
|
||||
})
|
||||
|
||||
describe('countVerifikatMissingDocument', () => {
|
||||
it('counts posted document-requiring entries with neither document nor exemption', async () => {
|
||||
// 6 posted entries: je-1 documented+exempt, je-2 documented, je-3 exempt
|
||||
// → je-4, je-5, je-6 missing.
|
||||
enqueue({
|
||||
data: [
|
||||
{ id: 'je-1' },
|
||||
{ id: 'je-2' },
|
||||
{ id: 'je-3' },
|
||||
{ id: 'je-4' },
|
||||
{ id: 'je-5' },
|
||||
{ id: 'je-6' },
|
||||
],
|
||||
})
|
||||
enqueue({
|
||||
data: [
|
||||
{ journal_entry_id: 'je-1' },
|
||||
{ journal_entry_id: 'je-1' }, // second doc on the same entry — still one entry
|
||||
{ journal_entry_id: 'je-2' },
|
||||
],
|
||||
})
|
||||
enqueue({
|
||||
data: [{ journal_entry_id: 'je-1' }, { journal_entry_id: 'je-3' }],
|
||||
})
|
||||
// Predicate semantics (needs-doc source types, current versions, waivers)
|
||||
// now live in the verifikat_without_documents RPC and are pinned by
|
||||
// tests/pg/document-surfaces-unification.pg.test.ts against real Postgres.
|
||||
// These tests cover only the delegation contract.
|
||||
it('delegates to the verifikat_without_documents RPC and returns its total', async () => {
|
||||
enqueue({ data: { ok: true, total_count: 3, verifikat: [] }, error: null })
|
||||
await expect(countVerifikatMissingDocument(supabase, COMPANY)).resolves.toBe(3)
|
||||
expect(mockSupabase.rpc).toHaveBeenCalledWith('verifikat_without_documents', {
|
||||
p_company_id: COMPANY,
|
||||
p_limit: 1,
|
||||
p_offset: 0,
|
||||
})
|
||||
})
|
||||
|
||||
it('ignores documents attached to entries outside the document-requiring set', async () => {
|
||||
// The doc on je-99 (e.g. a VAT-settlement entry) must not shrink the count.
|
||||
enqueue({ data: [{ id: 'je-1' }] })
|
||||
enqueue({ data: [{ journal_entry_id: 'je-99' }] })
|
||||
enqueue({ data: [] })
|
||||
await expect(countVerifikatMissingDocument(supabase, COMPANY)).resolves.toBe(1)
|
||||
})
|
||||
|
||||
it('soft-fails to 0 when a paginated read errors', async () => {
|
||||
enqueue({ error: { message: 'boom' } })
|
||||
enqueue({ data: [] })
|
||||
enqueue({ data: [] })
|
||||
it('soft-fails to 0 when the RPC errors', async () => {
|
||||
enqueue({ data: null, error: { message: 'boom' } })
|
||||
await expect(countVerifikatMissingDocument(supabase, COMPANY)).resolves.toBe(0)
|
||||
})
|
||||
|
||||
it('soft-fails to 0 (never a silent partial) when pagination errors mid-stream', async () => {
|
||||
// First page of entries is full (1000 = fetchAllRows page size), so a
|
||||
// second page is requested and errors. fetchAllRows must throw — the
|
||||
// count drops to a logged 0 rather than computing from a truncated set.
|
||||
enqueue({
|
||||
data: Array.from({ length: 1000 }, (_, i) => ({ id: `je-${i}` })),
|
||||
})
|
||||
enqueue({ data: [] }) // document_attachments page 1
|
||||
enqueue({ data: [] }) // exemptions page 1
|
||||
enqueue({ error: { message: 'mid-stream failure' } }) // entries page 2
|
||||
it('soft-fails to 0 on a not-ok envelope (tenant guard)', async () => {
|
||||
enqueue({ data: { ok: false, code: 'VERIFIKAT_WITHOUT_DOCUMENTS_FORBIDDEN' }, error: null })
|
||||
await expect(countVerifikatMissingDocument(supabase, COMPANY)).resolves.toBe(0)
|
||||
})
|
||||
})
|
||||
|
||||
+20
-43
@@ -10,7 +10,6 @@
|
||||
|
||||
import type { SupabaseClient } from '@supabase/supabase-js'
|
||||
import { createLogger } from '@/lib/logger'
|
||||
import { fetchAllRows } from '@/lib/supabase/fetch-all'
|
||||
import type { SuggestedMatch } from './types'
|
||||
|
||||
const log = createLogger('worklist')
|
||||
@@ -161,55 +160,33 @@ export async function countSupplierInvoicesAwaitingApproval(
|
||||
* source types that have neither a current-version document nor a
|
||||
* journal_entry_no_doc_required exemption.
|
||||
*
|
||||
* Computed as an exact per-entry set difference (the home page previously
|
||||
* subtracted set SIZES, which both let documents on non-document-requiring
|
||||
* entries shrink the count and silently truncated at the PostgREST row cap).
|
||||
* All three reads paginate via fetchAllRows; row volume is bounded by the
|
||||
* company's posted-entry history (id-only columns).
|
||||
* Delegates to the verifikat_without_documents RPC — the SAME predicate the
|
||||
* MCP surfaces use (single truth in SQL; the RPC's needs-doc source-type
|
||||
* list mirrors NEEDS_DOC_SOURCE_TYPES, pinned by
|
||||
* tests/pg/document-surfaces-unification.pg.test.ts). Previously this
|
||||
* fetched three full id-column tables and set-differenced client-side.
|
||||
*/
|
||||
export async function countVerifikatMissingDocument(
|
||||
supabase: SupabaseClient,
|
||||
companyId: string,
|
||||
): Promise<number> {
|
||||
try {
|
||||
const [entries, docs, exemptions] = await Promise.all([
|
||||
fetchAllRows<{ id: string }>(({ from, to }) =>
|
||||
supabase
|
||||
.from('journal_entries')
|
||||
.select('id')
|
||||
.eq('company_id', companyId)
|
||||
.eq('status', 'posted')
|
||||
.in('source_type', [...NEEDS_DOC_SOURCE_TYPES])
|
||||
.order('id')
|
||||
.range(from, to),
|
||||
),
|
||||
fetchAllRows<{ journal_entry_id: string }>(({ from, to }) =>
|
||||
supabase
|
||||
.from('document_attachments')
|
||||
.select('journal_entry_id')
|
||||
.eq('company_id', companyId)
|
||||
.eq('is_current_version', true)
|
||||
.not('journal_entry_id', 'is', null)
|
||||
.order('id')
|
||||
.range(from, to),
|
||||
),
|
||||
fetchAllRows<{ journal_entry_id: string }>(({ from, to }) =>
|
||||
supabase
|
||||
.from('journal_entry_no_doc_required')
|
||||
.select('journal_entry_id')
|
||||
.eq('company_id', companyId)
|
||||
.order('journal_entry_id')
|
||||
.range(from, to),
|
||||
),
|
||||
])
|
||||
|
||||
const withDoc = new Set(docs.map((d) => d.journal_entry_id))
|
||||
const exempt = new Set(exemptions.map((e) => e.journal_entry_id))
|
||||
let missing = 0
|
||||
for (const entry of entries) {
|
||||
if (!withDoc.has(entry.id) && !exempt.has(entry.id)) missing++
|
||||
// p_limit only sizes the page — total_count is computed over the FULL
|
||||
// filtered set inside the RPC (independent CTE), so 1 is the cheapest
|
||||
// valid page size for a count-only call.
|
||||
const { data, error } = await supabase.rpc('verifikat_without_documents', {
|
||||
p_company_id: companyId,
|
||||
p_limit: 1,
|
||||
p_offset: 0,
|
||||
})
|
||||
if (error) return logAndZero('verifikat_missing_document', companyId, error)
|
||||
const result = data as { ok?: boolean; code?: string; total_count?: number } | null
|
||||
if (!result?.ok) {
|
||||
return logAndZero('verifikat_missing_document', companyId, {
|
||||
message: result?.code ?? 'rpc returned not-ok',
|
||||
})
|
||||
}
|
||||
return missing
|
||||
return result.total_count ?? 0
|
||||
} catch (err) {
|
||||
return logAndZero(
|
||||
'verifikat_missing_document',
|
||||
|
||||
@@ -0,0 +1,281 @@
|
||||
-- Unify the missing-document surfaces on ONE truth (mcp_optimization_plan
|
||||
-- P1-3).
|
||||
--
|
||||
-- Before this migration the two MCP surfaces disagreed:
|
||||
-- - gnubok_list_transactions_without_documents keyed on
|
||||
-- transactions.document_id
|
||||
-- - gnubok_list_verifikat_without_documents keyed on document_attachments
|
||||
-- rows — and ignored source-type semantics, version chains and the
|
||||
-- journal_entry_no_doc_required waiver table that lib/worklist's
|
||||
-- canonical count respects.
|
||||
-- Measured divergence on prod (2026-07-03): 22,046 waived verifikat still
|
||||
-- listed to agents, 2,370 doc-exempt source types listed, ~87 bank
|
||||
-- transactions whose attached doc was never propagated to the verifikat
|
||||
-- (all in open periods), and 1,100 transactions listed as missing receipts
|
||||
-- although their verifikat HAS the underlag.
|
||||
--
|
||||
-- After: both RPCs implement the same predicate — a posted journal entry of a
|
||||
-- needs-doc source type, with no CURRENT-version document_attachments row and
|
||||
-- no journal_entry_no_doc_required waiver. The transactions surface is the
|
||||
-- bank-driven SUBSET of the verifikat surface by construction (it joins the
|
||||
-- same predicate through transactions.journal_entry_id).
|
||||
--
|
||||
-- The needs-doc source-type list mirrors NEEDS_DOC_SOURCE_TYPES in
|
||||
-- lib/worklist/categories.ts — keep them in lockstep (pinned by
|
||||
-- tests/pg/document-surfaces-unification.pg.test.ts, which imports the TS
|
||||
-- constant and probes the RPC per source type).
|
||||
--
|
||||
-- pg-test: tests/pg/document-surfaces-unification.pg.test.ts
|
||||
|
||||
-- ────────────────────────────────────────────────────────────────────
|
||||
-- 1. Verifikat surface: canonical predicate
|
||||
-- ────────────────────────────────────────────────────────────────────
|
||||
|
||||
CREATE OR REPLACE FUNCTION public.verifikat_without_documents(
|
||||
p_company_id uuid,
|
||||
p_since date DEFAULT NULL,
|
||||
p_min_amount numeric DEFAULT 0,
|
||||
p_limit integer DEFAULT 20,
|
||||
p_offset integer DEFAULT 0
|
||||
)
|
||||
RETURNS jsonb
|
||||
LANGUAGE plpgsql
|
||||
STABLE
|
||||
SECURITY DEFINER
|
||||
SET search_path TO 'public'
|
||||
AS $$
|
||||
DECLARE
|
||||
v_jwt_role text := coalesce(nullif(current_setting('request.jwt.claims', true), '')::jsonb ->> 'role', '');
|
||||
v_limit integer := least(greatest(coalesce(p_limit, 20), 1), 100);
|
||||
v_offset integer := greatest(coalesce(p_offset, 0), 0);
|
||||
v_min numeric := greatest(coalesce(p_min_amount, 0), 0);
|
||||
v_result jsonb;
|
||||
BEGIN
|
||||
IF v_jwt_role IN ('anon', 'authenticated') THEN
|
||||
IF p_company_id IS NULL OR NOT EXISTS (
|
||||
SELECT 1 FROM public.user_company_ids() AS c(id) WHERE c.id = p_company_id
|
||||
) THEN
|
||||
RETURN jsonb_build_object('ok', false, 'code', 'VERIFIKAT_WITHOUT_DOCUMENTS_FORBIDDEN');
|
||||
END IF;
|
||||
END IF;
|
||||
|
||||
WITH candidates AS (
|
||||
SELECT
|
||||
je.id,
|
||||
je.voucher_series,
|
||||
je.voucher_number,
|
||||
je.entry_date,
|
||||
je.description,
|
||||
je.source_type,
|
||||
round(coalesce(sum(l.debit_amount), 0), 2) AS gross_amount
|
||||
FROM journal_entries je
|
||||
LEFT JOIN journal_entry_lines l ON l.journal_entry_id = je.id
|
||||
WHERE je.company_id = p_company_id
|
||||
AND je.status = 'posted'
|
||||
-- Only source types whose affärshändelse requires an underlag.
|
||||
-- Mirrors NEEDS_DOC_SOURCE_TYPES (lib/worklist/categories.ts).
|
||||
AND je.source_type IN (
|
||||
'manual',
|
||||
'bank_transaction',
|
||||
'supplier_invoice_registered',
|
||||
'supplier_invoice_paid',
|
||||
'supplier_invoice_cash_payment',
|
||||
'import'
|
||||
)
|
||||
-- Superseded document versions do not satisfy BFL underlag.
|
||||
AND NOT EXISTS (
|
||||
SELECT 1 FROM document_attachments d
|
||||
WHERE d.journal_entry_id = je.id AND d.is_current_version = true
|
||||
)
|
||||
-- Explicitly waived (e.g. internal transfers) — user decided no
|
||||
-- underlag is required; do not resurface to agents.
|
||||
AND NOT EXISTS (
|
||||
SELECT 1 FROM journal_entry_no_doc_required x
|
||||
WHERE x.journal_entry_id = je.id
|
||||
)
|
||||
AND (p_since IS NULL OR je.entry_date >= p_since)
|
||||
GROUP BY je.id
|
||||
HAVING round(coalesce(sum(l.debit_amount), 0), 2) >= v_min
|
||||
),
|
||||
total AS (
|
||||
SELECT count(*) AS n FROM candidates
|
||||
),
|
||||
page AS (
|
||||
SELECT * FROM candidates
|
||||
ORDER BY entry_date DESC, voucher_number DESC, id DESC
|
||||
LIMIT v_limit OFFSET v_offset
|
||||
)
|
||||
SELECT jsonb_build_object(
|
||||
'ok', true,
|
||||
'total_count', (SELECT n FROM total),
|
||||
'verifikat', coalesce(
|
||||
(SELECT jsonb_agg(
|
||||
jsonb_build_object(
|
||||
'journal_entry_id', p.id,
|
||||
'voucher_series', p.voucher_series,
|
||||
'voucher_number', p.voucher_number,
|
||||
'entry_date', p.entry_date,
|
||||
'description', p.description,
|
||||
'source_type', p.source_type,
|
||||
'gross_amount', p.gross_amount
|
||||
)
|
||||
ORDER BY p.entry_date DESC, p.voucher_number DESC, p.id DESC
|
||||
) FROM page p),
|
||||
'[]'::jsonb
|
||||
)
|
||||
)
|
||||
INTO v_result;
|
||||
|
||||
RETURN v_result;
|
||||
END;
|
||||
$$;
|
||||
|
||||
-- CREATE OR REPLACE preserves the grants from 20260703130000 (verified on
|
||||
-- prod: authenticated + service_role only). Restated explicitly so this
|
||||
-- migration is self-contained and safe even standalone.
|
||||
REVOKE ALL ON FUNCTION public.verifikat_without_documents(uuid, date, numeric, integer, integer) FROM PUBLIC, anon;
|
||||
GRANT EXECUTE ON FUNCTION public.verifikat_without_documents(uuid, date, numeric, integer, integer) TO authenticated, service_role;
|
||||
|
||||
-- ────────────────────────────────────────────────────────────────────
|
||||
-- 2. Transactions surface: the bank-driven subset of the same predicate
|
||||
-- ────────────────────────────────────────────────────────────────────
|
||||
|
||||
CREATE OR REPLACE FUNCTION public.transactions_without_documents(
|
||||
p_company_id uuid,
|
||||
p_since date DEFAULT NULL,
|
||||
p_limit integer DEFAULT 20,
|
||||
p_offset integer DEFAULT 0
|
||||
)
|
||||
RETURNS jsonb
|
||||
LANGUAGE plpgsql
|
||||
STABLE
|
||||
SECURITY DEFINER
|
||||
SET search_path TO 'public'
|
||||
AS $$
|
||||
DECLARE
|
||||
v_jwt_role text := coalesce(nullif(current_setting('request.jwt.claims', true), '')::jsonb ->> 'role', '');
|
||||
v_limit integer := least(greatest(coalesce(p_limit, 20), 1), 100);
|
||||
v_offset integer := greatest(coalesce(p_offset, 0), 0);
|
||||
v_result jsonb;
|
||||
BEGIN
|
||||
IF v_jwt_role IN ('anon', 'authenticated') THEN
|
||||
IF p_company_id IS NULL OR NOT EXISTS (
|
||||
SELECT 1 FROM public.user_company_ids() AS c(id) WHERE c.id = p_company_id
|
||||
) THEN
|
||||
RETURN jsonb_build_object('ok', false, 'code', 'TRANSACTIONS_WITHOUT_DOCUMENTS_FORBIDDEN');
|
||||
END IF;
|
||||
END IF;
|
||||
|
||||
WITH candidates AS (
|
||||
SELECT
|
||||
t.id,
|
||||
t.date,
|
||||
t.description,
|
||||
t.amount,
|
||||
t.currency,
|
||||
t.merchant_name,
|
||||
t.reference,
|
||||
t.is_business,
|
||||
t.category,
|
||||
t.journal_entry_id
|
||||
FROM transactions t
|
||||
JOIN journal_entries je ON je.id = t.journal_entry_id
|
||||
WHERE t.company_id = p_company_id
|
||||
AND je.status = 'posted'
|
||||
-- Same predicate as verifikat_without_documents — this surface is the
|
||||
-- bank-driven subset, keyed on the SAME document truth
|
||||
-- (document_attachments), never transactions.document_id.
|
||||
AND je.source_type IN (
|
||||
'manual',
|
||||
'bank_transaction',
|
||||
'supplier_invoice_registered',
|
||||
'supplier_invoice_paid',
|
||||
'supplier_invoice_cash_payment',
|
||||
'import'
|
||||
)
|
||||
AND NOT EXISTS (
|
||||
SELECT 1 FROM document_attachments d
|
||||
WHERE d.journal_entry_id = je.id AND d.is_current_version = true
|
||||
)
|
||||
AND NOT EXISTS (
|
||||
SELECT 1 FROM journal_entry_no_doc_required x
|
||||
WHERE x.journal_entry_id = je.id
|
||||
)
|
||||
AND (p_since IS NULL OR t.date >= p_since)
|
||||
),
|
||||
total AS (
|
||||
SELECT count(*) AS n FROM candidates
|
||||
),
|
||||
page AS (
|
||||
SELECT * FROM candidates
|
||||
ORDER BY date DESC, id DESC
|
||||
LIMIT v_limit OFFSET v_offset
|
||||
)
|
||||
SELECT jsonb_build_object(
|
||||
'ok', true,
|
||||
'total_count', (SELECT n FROM total),
|
||||
'transactions', coalesce(
|
||||
(SELECT jsonb_agg(
|
||||
jsonb_build_object(
|
||||
'id', p.id,
|
||||
'transaction_id', p.id,
|
||||
'date', p.date,
|
||||
'description', p.description,
|
||||
'amount', p.amount,
|
||||
'currency', p.currency,
|
||||
'merchant_name', p.merchant_name,
|
||||
'reference', p.reference,
|
||||
'is_business', p.is_business,
|
||||
'category', p.category,
|
||||
'journal_entry_id', p.journal_entry_id
|
||||
)
|
||||
ORDER BY p.date DESC, p.id DESC
|
||||
) FROM page p),
|
||||
'[]'::jsonb
|
||||
)
|
||||
)
|
||||
INTO v_result;
|
||||
|
||||
RETURN v_result;
|
||||
END;
|
||||
$$;
|
||||
|
||||
REVOKE ALL ON FUNCTION public.transactions_without_documents(uuid, date, integer, integer) FROM PUBLIC, anon;
|
||||
GRANT EXECUTE ON FUNCTION public.transactions_without_documents(uuid, date, integer, integer) TO authenticated, service_role;
|
||||
|
||||
-- ────────────────────────────────────────────────────────────────────
|
||||
-- 3. Backfill the historical propagation gap: docs attached to a booked
|
||||
-- transaction whose verifikat never received the document_attachments
|
||||
-- link. Only docs that are currently unlinked (never steal a doc that
|
||||
-- points at another verifikat) and only into open, unlocked periods
|
||||
-- (the enforce_period_lock trigger raises on journal_entry_id writes in
|
||||
-- locked/closed periods). ~87 rows on prod, all in open periods.
|
||||
-- ────────────────────────────────────────────────────────────────────
|
||||
|
||||
DO $$
|
||||
DECLARE
|
||||
v_updated integer;
|
||||
BEGIN
|
||||
WITH gap AS (
|
||||
SELECT t.document_id, t.journal_entry_id
|
||||
FROM transactions t
|
||||
JOIN journal_entries je ON je.id = t.journal_entry_id
|
||||
JOIN fiscal_periods fp ON fp.id = je.fiscal_period_id
|
||||
WHERE t.document_id IS NOT NULL
|
||||
AND je.status = 'posted'
|
||||
AND fp.is_closed = false
|
||||
AND fp.locked_at IS NULL
|
||||
)
|
||||
UPDATE document_attachments d
|
||||
SET journal_entry_id = gap.journal_entry_id
|
||||
FROM gap
|
||||
WHERE d.id = gap.document_id
|
||||
AND d.journal_entry_id IS NULL
|
||||
AND d.is_current_version = true;
|
||||
|
||||
GET DIAGNOSTICS v_updated = ROW_COUNT;
|
||||
RAISE NOTICE 'document_surfaces_unification: propagated % transaction-attached documents to their verifikat', v_updated;
|
||||
END;
|
||||
$$;
|
||||
|
||||
NOTIFY pgrst, 'reload schema';
|
||||
@@ -0,0 +1,211 @@
|
||||
import { randomUUID } from 'crypto'
|
||||
import { beforeAll, describe, expect, it } from 'vitest'
|
||||
import { NEEDS_DOC_SOURCE_TYPES } from '@/lib/worklist/categories'
|
||||
import { getPool } from './setup'
|
||||
import {
|
||||
seedCompany,
|
||||
insertDraftJournalEntry,
|
||||
insertBalancedLines,
|
||||
insertTransaction,
|
||||
} from './fixtures'
|
||||
|
||||
/**
|
||||
* P1-3 (mcp_optimization_plan): both missing-document surfaces implement ONE
|
||||
* predicate — posted, needs-doc source type, no CURRENT-version
|
||||
* document_attachments row, no journal_entry_no_doc_required waiver — and the
|
||||
* transactions surface is a strict subset of the verifikat surface.
|
||||
*
|
||||
* Also pins the SQL needs-doc source-type list to the TS constant
|
||||
* NEEDS_DOC_SOURCE_TYPES (lib/worklist/categories.ts): a divergence between
|
||||
* the two lists fails the per-source-type probe below.
|
||||
*/
|
||||
|
||||
type VerifikatResult = {
|
||||
ok: boolean
|
||||
total_count?: number
|
||||
verifikat?: Array<{ journal_entry_id: string; source_type: string }>
|
||||
}
|
||||
type TransactionsResult = {
|
||||
ok: boolean
|
||||
code?: string
|
||||
total_count?: number
|
||||
transactions?: Array<{ id: string; transaction_id: string; journal_entry_id: string }>
|
||||
}
|
||||
|
||||
async function verifikatSurface(companyId: string): Promise<VerifikatResult> {
|
||||
const { rows } = await getPool().query<{ r: VerifikatResult }>(
|
||||
`SELECT public.verifikat_without_documents($1, NULL, 0, 100, 0) AS r`,
|
||||
[companyId],
|
||||
)
|
||||
return rows[0].r
|
||||
}
|
||||
|
||||
async function transactionsSurface(companyId: string): Promise<TransactionsResult> {
|
||||
const { rows } = await getPool().query<{ r: TransactionsResult }>(
|
||||
`SELECT public.transactions_without_documents($1, NULL, 100, 0) AS r`,
|
||||
[companyId],
|
||||
)
|
||||
return rows[0].r
|
||||
}
|
||||
|
||||
async function attachDocument(params: {
|
||||
userId: string
|
||||
companyId: string
|
||||
journalEntryId: string
|
||||
isCurrentVersion?: boolean
|
||||
}): Promise<string> {
|
||||
const id = randomUUID()
|
||||
await getPool().query(
|
||||
`INSERT INTO public.document_attachments
|
||||
(id, user_id, company_id, journal_entry_id, file_name, mime_type,
|
||||
file_size_bytes, storage_path, sha256_hash, upload_source, is_current_version)
|
||||
VALUES ($1, $2, $3, $4, 'underlag.pdf', 'application/pdf', 1024, $5, $6, 'file_upload', $7)`,
|
||||
[
|
||||
id,
|
||||
params.userId,
|
||||
params.companyId,
|
||||
params.journalEntryId,
|
||||
`documents/${params.companyId}/${id}.pdf`,
|
||||
randomUUID().replace(/-/g, '').padEnd(64, '0'),
|
||||
params.isCurrentVersion ?? true,
|
||||
],
|
||||
)
|
||||
return id
|
||||
}
|
||||
|
||||
async function waive(params: { userId: string; companyId: string; journalEntryId: string }) {
|
||||
await getPool().query(
|
||||
`INSERT INTO public.journal_entry_no_doc_required (journal_entry_id, company_id, user_id, reason)
|
||||
VALUES ($1, $2, $3, 'internal transfer — no underlag required')`,
|
||||
[params.journalEntryId, params.companyId, params.userId],
|
||||
)
|
||||
}
|
||||
|
||||
describe('document surfaces unification', () => {
|
||||
let userId: string
|
||||
let companyId: string
|
||||
let fiscalPeriodId: string
|
||||
|
||||
// Fixture matrix ids
|
||||
let jeBankNoDoc: string // bank tx JE, no doc → BOTH surfaces
|
||||
let jeBankWithDoc: string // bank tx JE, current doc → NEITHER
|
||||
let jeBankWaived: string // bank tx JE, waived → NEITHER
|
||||
let jeBankStaleDoc: string // bank tx JE, only superseded doc version → BOTH
|
||||
let jeInvoiceCreated: string // doc-exempt source type → NEITHER
|
||||
let jeImportNoDoc: string // import JE, no tx → verifikat surface only
|
||||
|
||||
beforeAll(async () => {
|
||||
const s = await seedCompany()
|
||||
userId = s.userId
|
||||
companyId = s.companyId
|
||||
fiscalPeriodId = s.fiscalPeriodId
|
||||
|
||||
const mkJe = async (n: number, sourceType: string) => {
|
||||
const id = await insertDraftJournalEntry({
|
||||
userId,
|
||||
companyId,
|
||||
fiscalPeriodId,
|
||||
status: 'posted',
|
||||
voucherNumber: n,
|
||||
entryDate: `2026-06-0${n}`,
|
||||
description: `${sourceType} ${n}`,
|
||||
sourceType,
|
||||
})
|
||||
await insertBalancedLines(id, n * 100)
|
||||
return id
|
||||
}
|
||||
|
||||
jeBankNoDoc = await mkJe(1, 'bank_transaction')
|
||||
jeBankWithDoc = await mkJe(2, 'bank_transaction')
|
||||
jeBankWaived = await mkJe(3, 'bank_transaction')
|
||||
jeBankStaleDoc = await mkJe(4, 'bank_transaction')
|
||||
jeInvoiceCreated = await mkJe(5, 'invoice_created')
|
||||
jeImportNoDoc = await mkJe(6, 'import')
|
||||
|
||||
// Bank transactions pointing at the four bank-driven entries. The
|
||||
// with-doc tx deliberately keeps document_id NULL (the 1,100-row reverse
|
||||
// gap on prod): the surface must key on document_attachments, not
|
||||
// transactions.document_id.
|
||||
for (const [jeId, date] of [
|
||||
[jeBankNoDoc, '2026-06-01'],
|
||||
[jeBankWithDoc, '2026-06-02'],
|
||||
[jeBankWaived, '2026-06-03'],
|
||||
[jeBankStaleDoc, '2026-06-04'],
|
||||
] as const) {
|
||||
await insertTransaction({ userId, companyId, journalEntryId: jeId, date })
|
||||
}
|
||||
|
||||
await attachDocument({ userId, companyId, journalEntryId: jeBankWithDoc })
|
||||
await attachDocument({
|
||||
userId,
|
||||
companyId,
|
||||
journalEntryId: jeBankStaleDoc,
|
||||
isCurrentVersion: false,
|
||||
})
|
||||
await waive({ userId, companyId, journalEntryId: jeBankWaived })
|
||||
})
|
||||
|
||||
it('verifikat surface: needs-doc entries without current docs or waivers, nothing else', async () => {
|
||||
const res = await verifikatSurface(companyId)
|
||||
expect(res.ok).toBe(true)
|
||||
const ids = (res.verifikat ?? []).map((v) => v.journal_entry_id).sort()
|
||||
expect(ids).toEqual([jeBankNoDoc, jeBankStaleDoc, jeImportNoDoc].sort())
|
||||
expect(res.total_count).toBe(3)
|
||||
// Doc-exempt source type never appears even when undocumented.
|
||||
expect(ids).not.toContain(jeInvoiceCreated)
|
||||
})
|
||||
|
||||
it('transactions surface: the bank-driven rows of the same set, keyed on document_attachments', async () => {
|
||||
const res = await transactionsSurface(companyId)
|
||||
expect(res.ok).toBe(true)
|
||||
const jeIds = (res.transactions ?? []).map((t) => t.journal_entry_id).sort()
|
||||
// jeBankWithDoc excluded even though its tx.document_id is NULL — the
|
||||
// doc truth is document_attachments. jeImportNoDoc has no tx row.
|
||||
expect(jeIds).toEqual([jeBankNoDoc, jeBankStaleDoc].sort())
|
||||
// P1-2 forward-compat: rows expose the qualified id.
|
||||
expect(res.transactions![0].transaction_id).toBe(res.transactions![0].id)
|
||||
})
|
||||
|
||||
it('transactions surface is a strict subset of the verifikat surface', async () => {
|
||||
const [ver, tx] = await Promise.all([verifikatSurface(companyId), transactionsSurface(companyId)])
|
||||
const verIds = new Set((ver.verifikat ?? []).map((v) => v.journal_entry_id))
|
||||
for (const row of tx.transactions ?? []) {
|
||||
expect(verIds.has(row.journal_entry_id), `tx surface row ${row.journal_entry_id} missing from verifikat surface`).toBe(true)
|
||||
}
|
||||
})
|
||||
|
||||
it('pins the SQL needs-doc list to NEEDS_DOC_SOURCE_TYPES per source type', async () => {
|
||||
// Each needs-doc source type must appear when undocumented; a canary
|
||||
// non-needs-doc type must not. Uses a fresh company per probe set to
|
||||
// keep assertions exact.
|
||||
const s = await seedCompany()
|
||||
let voucher = 1
|
||||
const expected: string[] = []
|
||||
for (const sourceType of NEEDS_DOC_SOURCE_TYPES) {
|
||||
const id = await insertDraftJournalEntry({
|
||||
userId: s.userId,
|
||||
companyId: s.companyId,
|
||||
fiscalPeriodId: s.fiscalPeriodId,
|
||||
status: 'posted',
|
||||
voucherNumber: voucher,
|
||||
entryDate: '2026-06-15',
|
||||
description: sourceType,
|
||||
sourceType,
|
||||
})
|
||||
await insertBalancedLines(id, 100 * voucher)
|
||||
expected.push(id)
|
||||
voucher++
|
||||
}
|
||||
const res = await verifikatSurface(s.companyId)
|
||||
expect((res.verifikat ?? []).map((v) => v.journal_entry_id).sort()).toEqual(expected.sort())
|
||||
})
|
||||
|
||||
it('tenant guard on the transactions surface (NULL + foreign company)', async () => {
|
||||
const { rows } = await getPool().query<{ r: TransactionsResult }>(
|
||||
`SELECT public.transactions_without_documents(NULL, NULL, 20, 0) AS r`,
|
||||
)
|
||||
// Superuser pool bypasses the guard by role; assert the NULL-company path
|
||||
// simply returns an empty ok result rather than leaking cross-tenant rows.
|
||||
expect((rows[0].r.transactions ?? []).length).toBe(0)
|
||||
})
|
||||
})
|
||||
Reference in New Issue
Block a user