admin/accounted
3
0
Fork 0
Code Issues 1 Pull Requests Actions Packages Projects Releases Wiki Activity

fix(bookkeeping): send each missing-underlag lookup chunk once per URL (#2430)

Browse Source 
* fix(bookkeeping): send each missing-underlag lookup chunk once per URL

The "verifikat utan underlag" filter (/bookkeeping?missingUnderlag=true)
failed with "Verifikaten kunde inte hamtas" on a self-hosted instance as
soon as the candidate set passed one chunk of 150 ids.

Why it occurred: resolveMissingUnderlagEntries issues four lookups per
chunk. Three carry the id list once; the supplier_invoices lookup
interpolated the same chunk twice into a single .or() over
registration_journal_entry_id and payment_journal_entry_id. That URL
alone crossed the 8 KB header buffer nginx/Kong ship with, so the
gateway answered 414 before PostgREST saw the request. Hosted sits at
roughly half of Cloudflare's 16 KB ceiling on the same query. The
LOOKUP_CHUNK docblock acknowledged the doubling without sizing for it.

What was removed: the runtime-built .or() string, the chunkInList
helper and its uuid guard (the .in() array filter is injection-safe on
its own). The supplier-invoice lookup is now two .in() queries, one per
FK column, merged into the same set, so every request carries the chunk
exactly once and the proxy limit stops being a dependency rather than
moving. LOOKUP_CHUNK stays 150 and its comment is now true. The
literal filter also leaves the phantom-column scanner's unresolvable
budget.

Two secondary defects from the same report: MissingUnderlagQueryError
now carries the raw driver error as `cause` and the journal-entries
route logs it, while the response keeps the Swedish text (the log used
to say only "Nagot gick fel", hiding the 414). The "Visa saknade
underlag" badge renders the total of the last successful filtered fetch
and hides on failure, instead of borrowing the list count (0 on a
failed first load, the whole ledger after a toggle).

Alternatives: halving LOOKUP_CHUNK moves the wall instead of removing
it. Pushing the list filters into the verifikat_without_documents RPC
and deleting the TS mirror leaves one predicate instead of two, but
moves search, series, date and sort into SQL; recorded in DECISIONS.md
as the intended next step for the surface owner.

Fixes #2395

Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01Q4kpVfYooLp8CWeUguRVpQ

* fix(bookkeeping): hide the underlag badge on network failure, log the bulk route's cause

Skeptic findings on aacb17634. The badge contract is "no honest source,
no badge": the non-OK branch cleared missingCount but the network-level
catch (offline, aborted body, JSON parse rejection) did not, so a period
or series change that failed at that level kept the previous filtered
total next to the toggle. The bulk "Inget underlag kravs" route had the
same log gap as the list route: it returned the mapped text without
logging the driver error, so a gateway 414 on that path stayed
invisible.

Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01Q4kpVfYooLp8CWeUguRVpQ

---------

Co-authored-by: Claude Fable 5.1 <noreply@anthropic.com>
This commit is contained in:
Mattsson
2026-09-08 16:35:41 +02:00
committed by GitHub
co-authored by Claude Fable 5.1
parent b2b714f829
commit 2f787a2b3c
8 changed files with 270 additions and 48 deletions
Download Patch File Download Diff File Expand all files Collapse all files
lib/bookkeeping/__tests__/missing-underlag.test.ts
+137
View File
@@ -0,0 +1,137 @@
import { describe, it, expect, vi, beforeEach } from 'vitest'
import { createQueuedMockSupabase } from '@/tests/helpers'
import type { SupabaseClient } from '@supabase/supabase-js'
import { getErrorMessage } from '@/lib/errors/get-error-message'
import {
MissingUnderlagQueryError,
resolveMissingUnderlagEntries,
} from '@/lib/bookkeeping/missing-underlag'
const { supabase, enqueue, reset, findCalls } = createQueuedMockSupabase()
const client = supabase as unknown as SupabaseClient
const E1 = '11111111-1111-4111-8111-111111111111'
const E2 = '22222222-2222-4222-8222-222222222222'
const E3 = '33333333-3333-4333-8333-333333333333'
const candidate = (id: string) => ({ id })
beforeEach(() => {
vi.clearAllMocks()
reset()
})
describe('resolveMissingUnderlagEntries: supplier-invoice reference lookup (#2395)', () => {
it('sends the candidate chunk once per URL: two .in() lookups, never one .or() over both FKs', async () => {
enqueue({ data: [candidate(E1), candidate(E2), candidate(E3)], error: null }) // candidates
enqueue({ data: [], error: null }) // no direct documents
enqueue({ data: [], error: null }) // SI by registration FK
enqueue({ data: [], error: null }) // SI by payment FK
enqueue({ data: [], error: null }) // no SI payment rows
enqueue({ data: [], error: null }) // no exemptions
enqueue({ data: [], error: null }) // no customer invoices pointing at the entries
enqueue({ data: [], error: null }) // no invoice payment rows
const missing = await resolveMissingUnderlagEntries(client, 'company-1', {}, { idOnly: true })
expect(missing.map((e) => e.id)).toEqual([E1, E2, E3])
// The doubled .or() is what pushed one lookup over an 8 KB proxy header
// buffer (414 on self-hosted Kong) while the single-list lookups passed.
expect(findCalls('supplier_invoices', 'or')).toHaveLength(0)
expect(findCalls('supplier_invoices', 'in')).toEqual([
['registration_journal_entry_id', [E1, E2, E3]],
['payment_journal_entry_id', [E1, E2, E3]],
])
// Both lookups stay scoped to the company and to invoices with a document.
expect(findCalls('supplier_invoices', 'eq')).toEqual([
['company_id', 'company-1'],
['company_id', 'company-1'],
])
expect(findCalls('supplier_invoices', 'not')).toEqual([
['document_id', 'is', null],
['document_id', 'is', null],
])
})
it('treats an anchored reference from either lookup as underlag, unanchored from neither', async () => {
enqueue({ data: [candidate(E1), candidate(E2), candidate(E3)], error: null })
enqueue({ data: [], error: null }) // no direct documents
enqueue({
data: [
{
registration_journal_entry_id: E1,
payment_journal_entry_id: null,
document: { journal_entry_id: E1 }, // anchored
},
],
error: null,
})
enqueue({
data: [
{
registration_journal_entry_id: null,
payment_journal_entry_id: E2,
document: { journal_entry_id: E2 }, // anchored
},
{
registration_journal_entry_id: null,
payment_journal_entry_id: E3,
document: { journal_entry_id: null }, // unanchored: deletable, not underlag
},
],
error: null,
})
enqueue({ data: [], error: null }) // no SI payment rows
enqueue({ data: [], error: null }) // no exemptions
enqueue({ data: [], error: null }) // no customer invoices pointing at the entries
enqueue({ data: [], error: null }) // no invoice payment rows
const missing = await resolveMissingUnderlagEntries(client, 'company-1', {}, { idOnly: true })
expect(missing.map((e) => e.id)).toEqual([E3])
})
it('chunks the candidate list at 150 ids and issues both supplier-invoice lookups per chunk', async () => {
const ids = Array.from(
{ length: 151 },
(_, i) => `${String(i).padStart(8, '0')}-0000-4000-8000-000000000000`,
)
enqueue({ data: ids.map(candidate), error: null })
for (let chunk = 0; chunk < 2; chunk++) {
enqueue({ data: [], error: null }) // documents
enqueue({ data: [], error: null }) // SI by registration FK
enqueue({ data: [], error: null }) // SI by payment FK
enqueue({ data: [], error: null }) // SI payment rows
enqueue({ data: [], error: null }) // exemptions
enqueue({ data: [], error: null }) // customer invoices
enqueue({ data: [], error: null }) // invoice payment rows
}
const missing = await resolveMissingUnderlagEntries(client, 'company-1', {}, { idOnly: true })
expect(missing).toHaveLength(151)
const inCalls = findCalls('supplier_invoices', 'in')
expect(inCalls.map(([column, list]) => [column, (list as string[]).length])).toEqual([
['registration_journal_entry_id', 150],
['payment_journal_entry_id', 150],
['registration_journal_entry_id', 1],
['payment_journal_entry_id', 1],
])
expect(findCalls('supplier_invoices', 'or')).toHaveLength(0)
})
it('carries the driver error as cause alongside the user-facing Swedish text', async () => {
enqueue({ data: [candidate(E1)], error: null })
enqueue({ data: [], error: null }) // documents
const driverError = { message: 'Request-URI Too Large', code: '414', details: null, hint: null }
enqueue({ data: null, error: driverError }) // SI by registration FK fails
const thrown = await resolveMissingUnderlagEntries(client, 'company-1').catch((e) => e)
expect(thrown).toBeInstanceOf(MissingUnderlagQueryError)
// The raw driver error survives for the server log, unmapped.
expect(thrown.cause).toBe(driverError)
// What the user sees went through the shared mapper, same as before.
expect(thrown.userMessage).toBe(getErrorMessage(driverError))
expect(thrown.message).toBe(thrown.userMessage)
})
})
lib/bookkeeping/missing-underlag.ts
+40 -34
View File
@@ -1,5 +1,4 @@
import type { SupabaseClient } from '@supabase/supabase-js'
import { z } from 'zod'
import type { PostgrestError, SupabaseClient } from '@supabase/supabase-js'
import { fetchAllRows } from '@/lib/supabase/fetch-all'
import { NEEDS_DOC_SOURCE_TYPES } from '@/lib/worklist/categories'
import { escapeLikePattern } from '@/lib/invoices/duplicate-payment-guard'
@@ -45,23 +44,26 @@ export interface MissingUnderlagEntry {
/**
* Sub-query failure. `userMessage` is already mapped through getErrorMessage()
* (user-facing Swedish), never a raw driver message.
* (user-facing Swedish), never a raw driver message. `cause` is the raw
* PostgREST error for the server log: the mapped text alone hid a gateway 414
* behind "Något gick fel" for a whole evening of proxy-log reading (#2395).
*/
export class MissingUnderlagQueryError extends Error {
constructor(public readonly userMessage: string) {
constructor(
public readonly userMessage: string,
public readonly cause: PostgrestError | unknown,
) {
super(userMessage)
}
}
// Journal-entry ids are interpolated into the supplier-invoice .or() filter
// string below, so they must be UUIDs. They originate from journal_entries.id
// (DB-sourced, never request input), but this guard keeps the injection-safety
// contract identical to /api/documents/counts.
const uuidSchema = z.string().uuid()
// 150 keeps the embedded id lists well under PostgREST's URL-length limit:
// the supplier-invoice .or() below repeats the chunk twice (registration +
// payment FK), so a larger chunk would risk truncating the GET filter.
/**
* Ids per PostgREST .in() filter. Ids travel in the GET query string; 150
* UUIDs is about 5.6 KB, under the 8 KB header buffer that nginx/Kong ship
* with and that self-hosted Supabase inherits. Every lookup below carries the
* chunk exactly ONCE: a filter that repeats it (one .or() over two FK columns)
* doubles the URL and is answered 414 before PostgREST ever sees it (#2395).
*/
const LOOKUP_CHUNK = 150
/**
@@ -169,20 +171,14 @@ export async function resolveMissingUnderlagEntries(
const exempt = new Set<string>()
for (let i = 0; i < candidateIds.length; i += LOOKUP_CHUNK) {
const chunk = candidateIds.slice(i, i + LOOKUP_CHUNK)
// Only UUIDs reach the interpolated .or() string (the .in() array filters
// are already injection-safe); mirrors the guard in documents/counts.
const chunkInList = `(${chunk.filter((id) => uuidSchema.safeParse(id).success).join(',')})`
const [docRes, siRefRes, sipRefRes, exemptRes] = await Promise.all([
supabase
.from('document_attachments')
.select('journal_entry_id')
.eq('company_id', companyId)
.eq('is_current_version', true)
.in('journal_entry_id', chunk),
// BFL 5 kap 7 § hänvisning: an entry referenced by a supplier invoice
// whose source document is retained AND anchored to a journal entry
// is NOT missing underlag (only anchored docs sit behind the WORM
// deletion guards). Mirrors the verifikat_without_documents RPC.
// BFL 5 kap 7 § hänvisning: an entry referenced by a supplier invoice
// whose source document is retained AND anchored to a journal entry
// is NOT missing underlag (only anchored docs sit behind the WORM
// deletion guards). Mirrors the verifikat_without_documents RPC.
// One query per FK column, never one .or() over both: the chunk must
// appear once per URL (see LOOKUP_CHUNK), and a literal .in() keeps the
// filter resolvable for tests/schema/no-phantom-columns.test.ts.
const supplierInvoiceRefs = () =>
supabase
.from('supplier_invoices')
.select(
@@ -190,9 +186,15 @@ export async function resolveMissingUnderlagEntries(
)
.eq('company_id', companyId)
.not('document_id', 'is', null)
.or(
`registration_journal_entry_id.in.${chunkInList},payment_journal_entry_id.in.${chunkInList}`,
),
const [docRes, siRegRes, siPayRes, sipRefRes, exemptRes] = await Promise.all([
supabase
.from('document_attachments')
.select('journal_entry_id')
.eq('company_id', companyId)
.eq('is_current_version', true)
.in('journal_entry_id', chunk),
supplierInvoiceRefs().in('registration_journal_entry_id', chunk),
supplierInvoiceRefs().in('payment_journal_entry_id', chunk),
supabase
.from('supplier_invoice_payments')
.select(
@@ -206,13 +208,17 @@ export async function resolveMissingUnderlagEntries(
.eq('company_id', companyId)
.in('journal_entry_id', chunk),
])
for (const res of [docRes, siRefRes, sipRefRes, exemptRes]) {
if (res.error) throw new MissingUnderlagQueryError(getUserErrorMessage(res.error))
for (const res of [docRes, siRegRes, siPayRes, sipRefRes, exemptRes]) {
if (res.error) {
throw new MissingUnderlagQueryError(getUserErrorMessage(res.error), res.error)
}
}
for (const r of (docRes.data ?? []) as { journal_entry_id: string }[]) {
withDoc.add(r.journal_entry_id)
}
for (const r of (siRefRes.data ?? []) as unknown as {
// Both lookups return the same row shape; an invoice matched by both
// columns lands twice, harmlessly, in the set.
for (const r of [...(siRegRes.data ?? []), ...(siPayRes.data ?? [])] as unknown as {
registration_journal_entry_id: string | null
payment_journal_entry_id: string | null
document: { journal_entry_id: string | null } | null
@@ -243,7 +249,7 @@ export async function resolveMissingUnderlagEntries(
try {
invoiceRefs = await getInvoiceReferencesForJournalEntries(supabase, companyId, chunk)
} catch (err) {
throw new MissingUnderlagQueryError(getUserErrorMessage(err))
throw new MissingUnderlagQueryError(getUserErrorMessage(err), err)
}
for (const journalEntryId of invoiceRefs.keys()) withDoc.add(journalEntryId)
}