diff --git a/DECISIONS.md b/DECISIONS.md index c17fd0f7..904f1aea 100644 --- a/DECISIONS.md +++ b/DECISIONS.md @@ -993,3 +993,19 @@ One line per decision: `[YYYY-MM-DD] : `. Appended by agents and [2026-08-14] /sie-data validation stays newest-file-only (not per-file, not on the merged parse): preserves exactly which datasets are accepted today, and validateSIEFile assumes single-file invariants (balance yearIndexes relative to ONE current year) that mergeParsedSIEFiles deliberately does not preserve. Older files' problems still surface per-file at import time. [2026-08-14] SKV manual-verifikat deep link payload moved from URL params to single-use sessionStorage (supersedes same-day URL-params decision): compliance swarm flagged financial data in query strings landing in history/access logs/Referer (GDPR Art.5(1)(f), ISO A.8.12); URL now carries only the opaque row id. [2026-08-14] SKV prefill sessionStorage XSS window accepted as residual risk (ISO A.8.12 low, swarm PR #1621): script execution already implies full ledger read via authenticated APIs; a server-issued staging token adds a roundtrip, not protection. Documented in manual-verifikat-prefill.ts header. +[2026-08-15] Underlag import (attaching a folder of receipts to SIE-migrated verifikat) matches on journal_entries.source_voucher_series/number, never on our own voucher_number: the importer renumbers per target series, so a file named after our number would land on the wrong verifikat exactly when the import skipped an empty/unbalanced voucher. The (period, series, number) resolver was lifted out of extensions/general/arcim-migration into lib/documents/voucher-ref-resolver.ts and is now shared by the provider sweep and the filename flow, so both have one resolution truth. +[2026-08-15] Underlag import is a SEPARATE optional import mode (/import?mode=underlag), not a step inside the SIE wizard (founder call 2026-08-15): the receipts normally arrive later and from a different export, so a migration must never be blocked on having them ready. +[2026-08-15] A filename that parses to a number with no series (31.pdf) is resolved but NEVER auto-selected, even when the lookup returns exactly one candidate, and a date-shaped name is refused outright rather than read as a voucher number. Linking a document to a posted verifikat is irreversible räkenskapsinformation (BFL 7 kap), so the cost of a wrong parse is permanent and the cost of asking is one click. The date guard is deliberately looser than the parser (unpadded components, two-digit years, space and slash separators, and any bare four-digit year-shaped number): a false positive costs one manual assignment, a false negative costs a permanent wrong link. +[2026-08-15] Every underlag-import batch is scoped to a fiscal year the USER declares, and resolution filters candidates to that year before the index is built. Rejected the original design, which resolved company-wide and treated "exactly one candidate exists" as proof of identity: source systems restart voucher numbering annually and a filename carries no year, so with a partial migration (or with that year's A31 among the vouchers the importer routinely skips as empty/single-line/unbalanced) a 2023 receipt was silently and permanently attached to a 2025 verifikat. Cardinality is not identity. Scoping cannot make the year inferable, so it makes it asserted; the year is read back in the confirm dialog because it is the one input the files cannot corroborate. Found by the /skeptic pass on PR #1627, not by the tests. +[2026-08-15] SUPERSEDED same day, see the entry below: the attach route took the fiscal year for its re-resolution from the TARGET entry, never from the client. Picking among candidates the server itself proposed is NOT an override: only a filename the user resolved by hand sets that flag. Flagging candidate picks would have switched the consistency check off on exactly the ambiguous rows it exists to protect. +[2026-08-15] Column lists for the two voucher-ref reads are written inline at the call site instead of a shared VOUCHER_SELECT constant: tests/schema/no-phantom-columns.test.ts resolves .select() literals by AST scan, so a constant is opaque to it and hoisting dropped all eight journal_entries columns out of the phantom-column net (and pushed the unresolved ceiling from 377 to 379, failing npm test). The provider sweep and the plan read now select different lists anyway: the sweep resolves thousands of entries and needs none of the display columns. +[2026-08-15] SUPERSEDES the entry above: the attach route takes the declared fiscal year from the REQUEST (echoed back from the plan the user reviewed) and asserts it equals the target entry's own fiscal_period_id, before any other check and including overrides. Deriving the year from the target was tautological: an entry is by construction inside its own period, so the filter could never drop it and the year axis was unfalsifiable. Server-side year enforcement was zero; it existed only as React state. The regression test that "proved" otherwise passed only because the mock let one journal_entries row report two different fiscal_period_id values to two different reads, a state Postgres cannot produce. Found by the second /skeptic pass on PR #1627. +[2026-08-15] The wizard snapshots the plan's fiscal year and every downstream read (summary, confirm dialog, manual re-resolution, each attach request) uses that snapshot, never the live picker. The picker stays interactive while a preview of up to 2000 filenames is in flight, so a confirm dialog reading live state could read back a different year than the plan was built from, which is worse than no confirm dialog at all. +[2026-08-15] `ver` is excluded from being a voucher series in filename parsing. Without it `ver 31.pdf` parsed as series VER and came back auto-selectable, while the spelled-out `Verifikat 31.pdf` correctly yielded a series-less reference needing confirmation: the same filename got two trust levels decided by an abbreviation, and the loose one was the abbreviation. The date guard also covers day-first and US order (31.01.2024, 12-24-2024), where the day would otherwise become a voucher number that always exists in the year. +[2026-08-15] FyPicker gained an opt-in `requireExplicitChoice` prop and the underlag import uses it. Default behaviour (auto-select the newest period when nothing is stored) is right for a filter, where a sensible default beats an empty page, and wrong where the year is an ASSERTION the user is making: the third /skeptic pass showed the wizard silently pre-filled the newest year, so a user migrating 2023 receipts who never opened the picker would resolve them against 2026, get `matched` rows pre-selected, and have only the confirm dialog between that and permanent links. The whole "the user declares the year" guarantee was fiction until this. A previously stored explicit choice for the same surface is still restored; no other caller changes behaviour. +[2026-08-15] `requireExplicitChoice` gates FyPicker's ENTIRE auto-selection block with one outer condition, not individual branches, and also suppresses the localStorage write. The fourth /skeptic pass showed the per-branch gate had missed the restore branch, so the guarantee held only for a user's first-ever batch: from the second on, the year was silently pre-filled from an earlier unrelated batch, and in a multi-year migration last-used is the worst possible default since the user is by definition moving to a different year. The write suppression matters because FyPicker persisted BEFORE onChange, so a pick the wizard rejected mid-preview would still have been recorded. Within one sitting, the wizard's reset() carries the year across batches; nothing else does, deliberately. +[2026-08-15] Filename parsing demotes (never refuses) collision-famous refs from auto-selection: A0-A6 (paper sizes; every scanner emits A4.pdf), K2-K13/N1-N9/T1-T2 (Skatteverket blanketter), Q1-Q4 (quarters), and any three-letter series (IMG/DSC/DOC/SCN are cameras; real SIE series are 1-2 chars). Two independent /skeptic passes flagged the same inconsistency: `31.pdf` required confirmation while `A4 scan.pdf`, which carries LESS voucher evidence, was pre-ticked. Verifikat A4 exists in every migrated ledger, so refusing would orphan the genuine A4 receipt; demotion costs it one click. Residual accepted and documented: an existing 1-2 letter series plus a small number in an ad-hoc name (`B2 hyra.pdf`) is indistinguishable from a real ref by filename alone. +[2026-08-15] The attach route runs the filename resolver on EVERY request; `override` is honored only when the filename is unresolvable in the declared year (no parse, or zero candidates), never for a filename the resolver can place elsewhere (CodeRabbit major on PR #1627). The shipped UI only overrides rows whose filenames resolved to nothing, so the tightening costs it no capability; it closes the API path where a lying client set override=true and scattered cleanly-named underlag across arbitrary same-year verifikat. planAcceptsTarget was renamed planPermitsAttach to carry the override semantics in one place. +[2026-08-15] Declined (recorded per /resolve-pr triage): classifying validateDocumentFile failures by regex on the localized message in the attach route stays as-is; it mirrors the established pattern in app/api/documents/route.ts, and moving the validator to coded returns is a core-service contract change outside this PR. The Swedish-review notes on idempotency (content hash IS in deterministicDocumentId) and the period-lock error regex (DB trigger is the real guard) require no change. +[2026-08-15] Underlag may attach only to posted or reversed verifikat, enforced in the attach route AND in the resolver reads (Swedish-review round on PR #1627). The SIE import RPC posts every entry inside its own transaction, so a draft with a source ref should be unobservable; the enforcement exists because the link is irreversible and an invariant living in another file is not one this surface may lean on. Reversed stays attachable: a storno'd original remains räkenskapsinformation and its underlag belongs on it. +[2026-08-15] Confirmed intentional (Swedish-review note): with override=true and an unresolvable filename, the attach endpoint links a document to any same-company, same-declared-year, posted verifikat, migrated or not. This mirrors /api/documents/[id]/link, which imposes no filename check at all, so it introduces no new capability class; tenant, year and period-lock enforcement always apply. diff --git a/app/(dashboard)/import/page.tsx b/app/(dashboard)/import/page.tsx index 5728c481..29c66c11 100644 --- a/app/(dashboard)/import/page.tsx +++ b/app/(dashboard)/import/page.tsx @@ -114,6 +114,7 @@ const AccountMappingStep = dynamic(() => import('@/components/import/AccountMapp const ImportReviewStep = dynamic(() => import('@/components/import/ImportReviewStep'), { loading: ImportStepLoading }) const ImportResultStep = dynamic(() => import('@/components/import/ImportResultStep'), { loading: ImportStepLoading }) const SIEImportHistory = dynamic(() => import('@/components/import/SIEImportHistory'), { loading: ImportStepLoading }) +const UnderlagImportWizard = dynamic(() => import('@/components/import/UnderlagImportWizard'), { loading: ImportStepLoading }) // ============================================================ // Bank File Import Wizard Steps @@ -2096,7 +2097,7 @@ const ShopifyPanel = getSettingsPanel('shopify') // Import Page with Selection Cards // ============================================================ -type ImportMode = null | 'psd2' | 'stripe' | 'woocommerce' | 'shopify' | 'bank' | 'sie' | 'csv_data' | 'migration' +type ImportMode = null | 'psd2' | 'stripe' | 'woocommerce' | 'shopify' | 'bank' | 'sie' | 'underlag' | 'csv_data' | 'migration' export default function ImportPage() { const { isSandbox } = useCompany() @@ -2129,8 +2130,8 @@ export default function ImportPage() { // third-party credentials, so their deep links are ignored in the sandbox. // Manual file-import modes (bank file, CSV/Excel, SIE) stay reachable. const allowedModes = isSandbox - ? ['bank', 'sie', 'csv_data'] - : ['psd2', 'stripe', 'woocommerce', 'shopify', 'bank', 'sie', 'csv_data', 'migration'] + ? ['bank', 'sie', 'underlag', 'csv_data'] + : ['psd2', 'stripe', 'woocommerce', 'shopify', 'bank', 'sie', 'underlag', 'csv_data', 'migration'] if (!isSandbox && searchParams.get('migration')) { setMode('migration') } else { @@ -2294,6 +2295,13 @@ export default function ImportPage() { sub={t('sie_description')} onClick={() => setMode('sie')} /> + {/* Optional follow-up to a SIE import, never a step inside it: + the receipts usually arrive later and from another export. */} + setMode('underlag')} + /> } {mode === 'sie' && } + {mode === 'underlag' && } {mode === 'csv_data' && } {mode === 'migration' && ( diff --git a/app/api/import/documents/attach/__tests__/route.test.ts b/app/api/import/documents/attach/__tests__/route.test.ts new file mode 100644 index 00000000..1a46eaa9 --- /dev/null +++ b/app/api/import/documents/attach/__tests__/route.test.ts @@ -0,0 +1,342 @@ +/** + * Tests for POST /api/import/documents/attach: archives one underlag file and + * links it to the SIE-migrated verifikat its filename names. + * + * The link is irreversible räkenskapsinformation (BFL 7 kap), so the guards + * matter more than the happy path: the target must belong to the company, and + * the automatic path must land where the preview said it would. + */ +import { describe, it, expect, vi, beforeEach } from 'vitest' +import { NextResponse } from 'next/server' +import { parseJsonResponse } from '@/tests/helpers' + +const PERIOD_OPEN = '44444444-4444-4444-8444-444444444444' +const PERIOD_OTHER = '55555555-5555-4555-8555-555555555555' + +const PERIODS = [ + { + id: PERIOD_OPEN, + period_start: '2024-01-01', + period_end: '2024-12-31', + is_closed: false, + locked_at: null, + }, +] + +const TARGET_ID = '11111111-1111-4111-8111-111111111111' +const OTHER_ID = '22222222-2222-4222-8222-222222222222' + +const VOUCHER = { + id: TARGET_ID, + fiscal_period_id: PERIOD_OPEN, + entry_date: '2024-03-14', + description: 'Inköp kontorsmaterial', + voucher_series: 'A', + voucher_number: 47, + source_voucher_series: 'A', + source_voucher_number: 31, +} + +/** The row the tenant-ownership lookup finds, or null for "not this company". */ +let targetEntry: Record | null = null +/** What the ledger holds for the filename's ref, as the plan would see it. */ +let vouchers: Record[] = [] + +const supabase = { + from(table: string) { + let filteredByNumber = false + let single = false + + const result = () => { + if (table === 'fiscal_periods') return { data: PERIODS, error: null, count: PERIODS.length } + if (single) return { data: targetEntry, error: null } + return { data: filteredByNumber ? vouchers : [], error: null, count: vouchers.length } + } + + const chain: Record = {} + for (const method of ['select', 'eq', 'not', 'order', 'limit']) { + chain[method] = () => chain + } + chain.maybeSingle = () => { + single = true + return chain + } + chain.in = () => { + filteredByNumber = true + return chain + } + chain.range = () => Promise.resolve(result()) + chain.then = (onFulfilled: (value: unknown) => unknown) => + Promise.resolve(result()).then(onFulfilled) + return chain + }, +} + +const requireAuthMock = vi.fn() +vi.mock('@/lib/auth/require-auth', () => ({ + requireAuth: (...args: unknown[]) => requireAuthMock(...args), +})) + +// Guideline mock: no real Supabase client may ever be constructed in a route +// test, even though this suite injects its double through requireAuth. +vi.mock('@/lib/supabase/server', () => ({ + createClient: () => Promise.resolve(supabase), +})) + +vi.mock('@/lib/company/context', () => ({ + getActiveCompanyId: vi.fn().mockResolvedValue('company-1'), + requireCompanyId: vi.fn().mockResolvedValue('company-1'), +})) + +const getCompanyRoleMock = vi.fn() +vi.mock('@/lib/auth/require-write', () => ({ + getCompanyRole: (...args: unknown[]) => getCompanyRoleMock(...args), + requireWritePermission: vi.fn().mockResolvedValue({ ok: true }), +})) + +vi.mock('@/lib/init', () => ({ ensureInitialized: vi.fn() })) + +const uploadDocumentMock = vi.fn() +vi.mock('@/lib/core/documents/document-service', async (importOriginal) => { + const actual = await importOriginal() + return { + ...actual, + uploadDocument: (...args: unknown[]) => uploadDocumentMock(...args), + } +}) + +import { POST } from '../route' + +const emptyParams = { params: Promise.resolve({}) } + +function makeRequest(options: { + fileName?: string + journalEntryId?: string | null + /** The year the plan was built against, as the wizard echoes it back. */ + declaredPeriodId?: string | null + override?: boolean + withFile?: boolean + mimeType?: string +}) { + const form = new FormData() + if (options.withFile !== false) { + form.append( + 'file', + new File(['%PDF-1.4 underlag'], options.fileName ?? 'A31_8c2db060.pdf', { + type: options.mimeType ?? 'application/pdf', + }), + ) + } + if (options.journalEntryId !== null) { + form.append('journal_entry_id', options.journalEntryId ?? TARGET_ID) + } + if (options.declaredPeriodId !== null) { + form.append('fiscal_period_id', options.declaredPeriodId ?? PERIOD_OPEN) + } + if (options.override) form.append('override', 'true') + + return new Request('http://localhost:3000/api/import/documents/attach', { + method: 'POST', + body: form, + }) +} + +describe('POST /api/import/documents/attach', () => { + beforeEach(() => { + vi.clearAllMocks() + targetEntry = { id: TARGET_ID, fiscal_period_id: PERIOD_OPEN, status: 'posted', source_voucher_series: 'A', source_voucher_number: 31 } + vouchers = [VOUCHER] + requireAuthMock.mockResolvedValue({ user: { id: 'user-1' }, supabase }) + getCompanyRoleMock.mockResolvedValue({ ok: true, role: 'owner', companyId: 'company-1' }) + uploadDocumentMock.mockResolvedValue({ id: 'doc-1', file_name: 'A31_8c2db060.pdf' }) + }) + + it('returns 401 when unauthenticated', async () => { + requireAuthMock.mockResolvedValue({ + user: null, + supabase, + error: NextResponse.json({ error: 'Unauthorized' }, { status: 401 }), + }) + + const res = await POST(makeRequest({}), emptyParams) + + expect(res.status).toBe(401) + expect(uploadDocumentMock).not.toHaveBeenCalled() + }) + + it('returns 400 when no file was sent', async () => { + const res = await POST(makeRequest({ withFile: false }), emptyParams) + + expect(res.status).toBe(400) + }) + + it('returns 400 when journal_entry_id is missing or not a uuid', async () => { + expect((await POST(makeRequest({ journalEntryId: null }), emptyParams)).status).toBe(400) + expect((await POST(makeRequest({ journalEntryId: 'A31' }), emptyParams)).status).toBe(400) + expect(uploadDocumentMock).not.toHaveBeenCalled() + }) + + it('returns 404 when the target entry belongs to another company', async () => { + targetEntry = null + + const res = await POST(makeRequest({}), emptyParams) + + expect(res.status).toBe(404) + expect(uploadDocumentMock).not.toHaveBeenCalled() + }) + + it('refuses a target the filename does not resolve to', async () => { + // The entry exists and is ours, but A31 belongs to a different verifikat: + // a stale plan in the browser must not scatter underlag permanently. + vouchers = [{ ...VOUCHER, id: OTHER_ID }] + + const res = await POST(makeRequest({}), emptyParams) + const { status, body } = await parseJsonResponse<{ error: { code: string } }>(res) + + expect(status).toBe(409) + expect(body.error.code).toBe('UNDERLAG_REF_MISMATCH') + expect(uploadDocumentMock).not.toHaveBeenCalled() + }) + + it('returns 400 when the declared fiscal year is missing or not a uuid', async () => { + expect((await POST(makeRequest({ declaredPeriodId: null }), emptyParams)).status).toBe(400) + expect((await POST(makeRequest({ declaredPeriodId: '2024' }), emptyParams)).status).toBe(400) + expect(uploadDocumentMock).not.toHaveBeenCalled() + }) + + it('refuses a target outside the fiscal year the batch declared', async () => { + // The user reviewed a plan for one year; this target lives in another. + // Nothing about the filename can make that acceptable. + const res = await POST(makeRequest({ declaredPeriodId: PERIOD_OTHER }), emptyParams) + const { status, body } = await parseJsonResponse<{ error: { code: string } }>(res) + + expect(status).toBe(409) + expect(body.error.code).toBe('UNDERLAG_PERIOD_MISMATCH') + expect(uploadDocumentMock).not.toHaveBeenCalled() + }) + + it('enforces the declared year even on a manual override', async () => { + // An override is a statement about WHICH verifikat, never about which + // year, so it must not widen the batch across fiscal years. + const res = await POST( + makeRequest({ declaredPeriodId: PERIOD_OTHER, override: true, fileName: 'kvitto.pdf' }), + emptyParams, + ) + const { status, body } = await parseJsonResponse<{ error: { code: string } }>(res) + + expect(status).toBe(409) + expect(body.error.code).toBe('UNDERLAG_PERIOD_MISMATCH') + expect(uploadDocumentMock).not.toHaveBeenCalled() + }) + + it('explains a target that never came from a SIE import', async () => { + targetEntry = { id: TARGET_ID, fiscal_period_id: PERIOD_OPEN, status: 'posted', source_voucher_series: null, source_voucher_number: null } + + const res = await POST(makeRequest({}), emptyParams) + const { status, body } = await parseJsonResponse<{ error: { code: string } }>(res) + + expect(status).toBe(400) + expect(body.error.code).toBe('UNDERLAG_ENTRY_NOT_MIGRATED') + expect(uploadDocumentMock).not.toHaveBeenCalled() + }) + + it('refuses a target that is not posted, overrides included', async () => { + // The SIE import posts entries inside its own transaction, so a draft with + // a source ref should be unreachable. This route does not lean on an + // invariant enforced in another file: underlag references a verifikation + // (BFL 5 kap 6-7 §), so the target must BE one. + targetEntry = { ...targetEntry!, status: 'draft' } + + const auto = await POST(makeRequest({}), emptyParams) + const { status, body } = await parseJsonResponse<{ error: { code: string } }>(auto) + expect(status).toBe(409) + expect(body.error.code).toBe('UNDERLAG_ENTRY_NOT_POSTED') + + const overridden = await POST( + makeRequest({ fileName: 'kvitto ica.pdf', override: true }), + emptyParams, + ) + expect(overridden.status).toBe(409) + expect(uploadDocumentMock).not.toHaveBeenCalled() + }) + + it('accepts a REVERSED target: a storno original keeps its underlag', async () => { + targetEntry = { ...targetEntry!, status: 'reversed' } + + const res = await POST(makeRequest({}), emptyParams) + + expect(res.status).toBe(200) + expect(uploadDocumentMock).toHaveBeenCalledOnce() + }) + + it('refuses an override for a filename that resolves to a DIFFERENT target', async () => { + // The server honors override only for filenames it cannot place. A31.pdf + // resolves cleanly to another verifikat, so a lying client cannot use + // override to scatter it. + vouchers = [{ ...VOUCHER, id: OTHER_ID }] + + const res = await POST(makeRequest({ override: true }), emptyParams) + const { status, body } = await parseJsonResponse<{ error: { code: string } }>(res) + + expect(status).toBe(409) + expect(body.error.code).toBe('UNDERLAG_REF_MISMATCH') + expect(uploadDocumentMock).not.toHaveBeenCalled() + }) + + it('allows an explicit manual assignment to skip the filename check', async () => { + targetEntry = { id: TARGET_ID, fiscal_period_id: PERIOD_OPEN, status: 'posted', source_voucher_series: null, source_voucher_number: null } + + const res = await POST( + makeRequest({ fileName: 'kvitto ica.pdf', override: true }), + emptyParams, + ) + + expect(res.status).toBe(200) + expect(uploadDocumentMock).toHaveBeenCalledOnce() + }) + + it('archives and links the file, scoping idempotency to the verifikat', async () => { + const res = await POST(makeRequest({}), emptyParams) + const { status, body } = await parseJsonResponse<{ data: { id: string } }>(res) + + expect(status).toBe(200) + expect(body.data.id).toBe('doc-1') + + const [, userId, companyId, file, metadata] = uploadDocumentMock.mock.calls[0] + expect(userId).toBe('user-1') + expect(companyId).toBe('company-1') + expect(file).toMatchObject({ name: 'A31_8c2db060.pdf', type: 'application/pdf' }) + expect(metadata).toMatchObject({ + journal_entry_id: TARGET_ID, + // Same content on a different verifikat must archive separately, so the + // key is the entry, not the bytes. + idempotency_key: TARGET_ID, + upload_source: 'file_upload', + }) + }) + + it('maps the period-lock trigger to a usable error instead of a 500', async () => { + uploadDocumentMock.mockRejectedValue( + new Error('Cannot attach documents to entries in a locked/closed fiscal period'), + ) + + const res = await POST(makeRequest({}), emptyParams) + const { status, body } = await parseJsonResponse<{ error: { code: string } }>(res) + + expect(status).toBe(400) + expect(body.error.code).toBe('DOC_UPLOAD_PERIOD_LOCKED') + }) + + it('does not leak storage internals when the archive write fails', async () => { + uploadDocumentMock.mockRejectedValue(new Error('bucket s3://internal-prod-bucket exploded')) + + const res = await POST(makeRequest({}), emptyParams) + const { status, body } = await parseJsonResponse<{ error: { code: string; message: string } }>( + res, + ) + + expect(status).toBe(500) + expect(body.error.code).toBe('DOC_UPLOAD_STORAGE_FAILED') + expect(body.error.message).not.toContain('internal-prod-bucket') + }) +}) diff --git a/app/api/import/documents/attach/route.ts b/app/api/import/documents/attach/route.ts new file mode 100644 index 00000000..6f5f0b30 --- /dev/null +++ b/app/api/import/documents/attach/route.ts @@ -0,0 +1,212 @@ +import { NextResponse } from 'next/server' +import { z } from 'zod' +import { ensureInitialized } from '@/lib/init' +import { withRouteContext } from '@/lib/api/with-route-context' +import { uploadDocument, validateDocumentFile } from '@/lib/core/documents/document-service' +import { planPermitsAttach } from '@/lib/documents/underlag-import' +import { getErrorMessage } from '@/lib/errors/get-error-message' +import { errorResponseFromCode } from '@/lib/errors/get-structured-error' + +ensureInitialized() + +const AttachFieldsSchema = z.object({ + journal_entry_id: z.string().uuid(), + /** + * The fiscal year the user reviewed this batch against, echoed back from the + * plan. Required, and checked against the target's own year on every request + * including overrides: it is the only thing that makes the year enforceable + * server-side. Deriving it from the target instead would be tautological, an + * entry is by construction inside its own period, and that is precisely the + * hole that let a 2023 receipt land on a 2025 verifikat. + */ + fiscal_period_id: z.string().uuid(), + /** + * Set ONLY when the user resolved this file by hand because its filename + * carries no usable reference. Honored server-side ONLY for filenames the + * resolver cannot place in the declared year: a resolvable filename must + * land where it points, override or not. Company ownership, the year + * assertion and the period lock are always enforced. + * + * Choosing among candidates the server itself proposed is NOT an override: + * those targets pass the check already, and flagging them would switch the + * guard off on exactly the ambiguous rows it exists for. + */ + override: z.boolean(), +}) + +/** + * POST /api/import/documents/attach: archive one underlag file and link it to a + * migrated verifikat. + * + * multipart/form-data: + * file: the underlag + * journal_entry_id: the target the user approved in the preview + * fiscal_period_id: the year the plan was built against (echoed back) + * override: 'true' when the target was chosen by hand + * + * One file per request on purpose: a folder migration is hundreds of files, the + * browser streams them one at a time with visible progress, and a failure on + * file 200 leaves the first 199 correctly attached instead of rolling back work + * that is legally irreversible anyway. + * + * Idempotent per (verifikat, content): re-running the same import converges on + * the same document row rather than archiving duplicates, via the deterministic + * document id that `idempotency_key` reserves. + */ +export const POST = withRouteContext( + 'import.documents.attach', + async (request, ctx) => { + const { user, supabase, companyId, log, requestId } = ctx + + const formData = await request.formData() + const file = formData.get('file') as File | null + + if (!file) { + return errorResponseFromCode('DOC_UPLOAD_NO_FILE', log, { requestId }) + } + + const fields = AttachFieldsSchema.safeParse({ + journal_entry_id: formData.get('journal_entry_id'), + fiscal_period_id: formData.get('fiscal_period_id'), + override: formData.get('override') === 'true', + }) + if (!fields.success) { + return errorResponseFromCode('VALIDATION_ERROR', log, { + requestId, + details: { + issues: fields.error.issues.map((i) => ({ + field: i.path.join('.'), + reason: i.message, + })), + }, + }) + } + const { + journal_entry_id: journalEntryId, + fiscal_period_id: fiscalPeriodId, + override, + } = fields.data + + const validationError = validateDocumentFile({ size: file.size, type: file.type }) + if (validationError) { + const code = /storlek|stor|MB/i.test(validationError) + ? 'DOC_UPLOAD_TOO_LARGE' + : 'DOC_UPLOAD_UNSUPPORTED_TYPE' + return errorResponseFromCode(code, log, { + requestId, + details: { reason: validationError, sizeBytes: file.size, mimeType: file.type }, + }) + } + + const opLog = log.child({ filename: file.name, journalEntryId }) + + // Tenant check first and explicitly: RLS covers the cookie session, but the + // link is irreversible, so the route never takes the client's word for which + // company an entry belongs to. + const { data: entry, error: entryError } = await supabase + .from('journal_entries') + .select('id, fiscal_period_id, status, source_voucher_series, source_voucher_number') + .eq('id', journalEntryId) + .eq('company_id', companyId!) + .maybeSingle() + + if (entryError) { + opLog.error('underlag attach target lookup failed', entryError) + return errorResponseFromCode('DOC_LINK_FAILED', opLog, { requestId }) + } + if (!entry) { + return errorResponseFromCode('DOC_LINK_ENTRY_NOT_FOUND', opLog, { requestId }) + } + + // Underlag references a verifikation (BFL 5 kap 6-7 §), so the target must + // BE one: posted, or reversed (a storno'd original keeps its underlag). The + // SIE import posts entries inside its own transaction, so a draft here + // should be unreachable, but this route writes irreversible links and does + // not lean on an invariant enforced in another file. + if (entry.status !== 'posted' && entry.status !== 'reversed') { + opLog.warn('underlag attach refused: target entry is not posted', { + entryStatus: entry.status, + }) + return errorResponseFromCode('UNDERLAG_ENTRY_NOT_POSTED', opLog, { requestId }) + } + + // The batch declared a fiscal year and the user reviewed the plan against + // it. The target must actually be in that year. Enforced FIRST and + // unconditionally, overrides included: a hand-resolved filename is a + // statement about which verifikat, never about which year, and this is the + // only check that makes the declared year mean anything on the server. + if (entry.fiscal_period_id !== fiscalPeriodId) { + opLog.warn('underlag attach refused: target sits in a different fiscal year', { + declaredFiscalPeriodId: fiscalPeriodId, + entryFiscalPeriodId: entry.fiscal_period_id, + }) + return errorResponseFromCode('UNDERLAG_PERIOD_MISMATCH', opLog, { requestId }) + } + + // The file must additionally land where the preview said it would. A stale + // plan in the browser (the user re-imported SIE in another tab, say) would + // otherwise scatter underlag across the wrong verifikat. The resolver runs + // on EVERY request: an override only relaxes it for filenames it cannot + // place at all, never for a filename that resolves elsewhere. + if (!override && entry.source_voucher_number == null) { + // Not a SIE-migrated verifikat, so no filename can ever resolve to it. + // Say that plainly instead of reporting a mismatch the user can't fix. + return errorResponseFromCode('UNDERLAG_ENTRY_NOT_MIGRATED', opLog, { requestId }) + } + const permitted = await planPermitsAttach( + supabase, + companyId!, + file.name, + journalEntryId, + fiscalPeriodId, + override, + ) + if (!permitted) { + opLog.warn('underlag attach refused: filename does not resolve to the target', { override }) + return errorResponseFromCode('UNDERLAG_REF_MISMATCH', opLog, { requestId }) + } + + try { + const buffer = await file.arrayBuffer() + + const document = await uploadDocument( + supabase, + user.id, + companyId!, + { name: file.name, buffer, type: file.type }, + { + upload_source: 'file_upload', + journal_entry_id: journalEntryId, + // Scope the deterministic id to the target verifikat: the same + // receipt may legitimately back several verifikat, so content alone + // must not dedupe across them. + idempotency_key: journalEntryId, + }, + ) + + return NextResponse.json({ data: document }) + } catch (err) { + const message = err instanceof Error ? err.message : 'unknown' + + // enforce_period_lock_documents: a migrated year is often closed by the + // time the receipts arrive, and the trigger blocks the link outright. + if (/locked\/closed fiscal period|Bokföringen är låst/i.test(message)) { + return errorResponseFromCode('DOC_UPLOAD_PERIOD_LOCKED', opLog, { + requestId, + details: { reason: getErrorMessage(err) }, + }) + } + if (/kunde inte verifieras|matchar inte den angivna filtypen/i.test(message)) { + opLog.warn('underlag attach rejected by content validation', { reason: message }) + return errorResponseFromCode('DOC_UPLOAD_INVALID_CONTENT', opLog, { + requestId, + details: { reason: getErrorMessage(err) }, + }) + } + + opLog.error('underlag attach failed', err as Error) + return errorResponseFromCode('DOC_UPLOAD_STORAGE_FAILED', opLog, { requestId }) + } + }, + { requireWrite: true }, +) diff --git a/app/api/import/documents/preview/__tests__/route.test.ts b/app/api/import/documents/preview/__tests__/route.test.ts new file mode 100644 index 00000000..7664af95 --- /dev/null +++ b/app/api/import/documents/preview/__tests__/route.test.ts @@ -0,0 +1,180 @@ +/** + * Tests for POST /api/import/documents/preview: the read-only match plan that + * pairs underlag filenames with SIE-migrated verifikat before anything is + * uploaded or linked. + */ +import { describe, it, expect, vi, beforeEach } from 'vitest' +import { NextResponse } from 'next/server' +import { createMockRequest, parseJsonResponse } from '@/tests/helpers' +import type { UnderlagPlan } from '@/lib/documents/underlag-import' + +const PERIOD_OPEN = '33333333-3333-4333-8333-333333333333' + +const PERIODS = [ + { + id: PERIOD_OPEN, + period_start: '2024-01-01', + period_end: '2024-12-31', + is_closed: false, + locked_at: null, + }, +] + +let vouchers: Record[] = [] +/** The row the fiscal-period ownership lookup finds, or null for "not ours". */ +let ownedPeriod: Record | null = null + +/** Shape-keyed double: the plan reads vouchers and periods concurrently. */ +const supabase = { + from(table: string) { + let filteredByNumber = false + let single = false + const result = () => { + if (table === 'fiscal_periods') { + return single + ? { data: ownedPeriod, error: null } + : { data: PERIODS, error: null, count: PERIODS.length } + } + return { + data: filteredByNumber ? vouchers : [], + error: null, + count: vouchers.length, + } + } + + const chain: Record = {} + for (const method of ['select', 'eq', 'not', 'order', 'limit', 'single']) { + chain[method] = () => chain + } + chain.maybeSingle = () => { + single = true + return chain + } + chain.in = () => { + filteredByNumber = true + return chain + } + chain.range = () => Promise.resolve(result()) + chain.then = (onFulfilled: (value: unknown) => unknown) => + Promise.resolve(result()).then(onFulfilled) + return chain + }, +} + +const requireAuthMock = vi.fn() +vi.mock('@/lib/auth/require-auth', () => ({ + requireAuth: (...args: unknown[]) => requireAuthMock(...args), +})) + +// Guideline mock: no real Supabase client may ever be constructed in a route +// test, even though this suite injects its double through requireAuth. +vi.mock('@/lib/supabase/server', () => ({ + createClient: () => Promise.resolve(supabase), +})) + +vi.mock('@/lib/company/context', () => ({ + getActiveCompanyId: vi.fn().mockResolvedValue('company-1'), + requireCompanyId: vi.fn().mockResolvedValue('company-1'), +})) + +vi.mock('@/lib/init', () => ({ ensureInitialized: vi.fn() })) + +import { POST } from '../route' + +const emptyParams = { params: Promise.resolve({}) } + +function makeRequest(body: Record) { + return createMockRequest('/api/import/documents/preview', { + method: 'POST', + body: { fiscal_period_id: PERIOD_OPEN, ...body }, + }) +} + +describe('POST /api/import/documents/preview', () => { + beforeEach(() => { + vi.clearAllMocks() + vouchers = [ + { + id: 'je-1', + fiscal_period_id: PERIOD_OPEN, + entry_date: '2024-03-14', + description: 'Inköp kontorsmaterial', + voucher_series: 'A', + voucher_number: 47, + source_voucher_series: 'A', + source_voucher_number: 31, + }, + ] + ownedPeriod = { id: PERIOD_OPEN } + requireAuthMock.mockResolvedValue({ user: { id: 'user-1' }, supabase }) + }) + + it('returns 401 when unauthenticated', async () => { + requireAuthMock.mockResolvedValue({ + user: null, + supabase, + error: NextResponse.json({ error: 'Unauthorized' }, { status: 401 }), + }) + + const res = await POST(makeRequest({ file_names: ['A31.pdf'] }), emptyParams) + + expect(res.status).toBe(401) + }) + + it('returns 400 when file_names is missing or empty', async () => { + expect((await POST(makeRequest({}), emptyParams)).status).toBe(400) + expect((await POST(makeRequest({ file_names: [] }), emptyParams)).status).toBe(400) + }) + + it('returns 400 without a fiscal year: a filename alone cannot identify a verifikat', async () => { + const noYear = createMockRequest('/api/import/documents/preview', { + method: 'POST', + body: { file_names: ['A31.pdf'] }, + }) + expect((await POST(noYear, emptyParams)).status).toBe(400) + + const badYear = await POST( + makeRequest({ file_names: ['A31.pdf'], fiscal_period_id: 'not-a-uuid' }), + emptyParams, + ) + expect(badYear.status).toBe(400) + }) + + it("returns 404 for a fiscal year that is not this company's", async () => { + ownedPeriod = null + + const res = await POST(makeRequest({ file_names: ['A31.pdf'] }), emptyParams) + + expect(res.status).toBe(404) + }) + + it('returns 400 when the batch exceeds the 2000-file cap', async () => { + const oversized = Array.from({ length: 2001 }, (_, i) => `A${i + 1}.pdf`) + + const res = await POST(makeRequest({ file_names: oversized }), emptyParams) + + expect(res.status).toBe(400) + }) + + it('returns the match plan for the submitted filenames (happy path)', async () => { + const res = await POST( + makeRequest({ file_names: ['A31_8c2db060.pdf', 'kvitto.pdf'] }), + emptyParams, + ) + const { status, body } = await parseJsonResponse<{ data: UnderlagPlan }>(res) + + expect(status).toBe(200) + expect(body.data.rows).toHaveLength(2) + expect(body.data.rows[0]).toMatchObject({ status: 'matched', journal_entry_id: 'je-1' }) + expect(body.data.rows[1]).toMatchObject({ status: 'unparsed', journal_entry_id: null }) + expect(body.data.summary).toMatchObject({ total: 2, matched: 1, unparsed: 1 }) + }) + + it('never returns a target the filename does not name', async () => { + const res = await POST(makeRequest({ file_names: ['A99.pdf'] }), emptyParams) + const { body } = await parseJsonResponse<{ data: UnderlagPlan }>(res) + + expect(body.data.rows[0].status).toBe('no_match') + expect(body.data.rows[0].journal_entry_id).toBeNull() + }) +}) diff --git a/app/api/import/documents/preview/route.ts b/app/api/import/documents/preview/route.ts new file mode 100644 index 00000000..e5e290a6 --- /dev/null +++ b/app/api/import/documents/preview/route.ts @@ -0,0 +1,54 @@ +import { NextResponse } from 'next/server' +import { ensureInitialized } from '@/lib/init' +import { withRouteContext } from '@/lib/api/with-route-context' +import { validateBody } from '@/lib/api/validate' +import { UnderlagImportPreviewSchema } from '@/lib/api/schemas' +import { buildUnderlagPlan } from '@/lib/documents/underlag-import' +import { errorResponse, errorResponseFromCode } from '@/lib/errors/get-structured-error' + +ensureInitialized() + +/** + * POST /api/import/documents/preview: plan which verifikat each underlag file + * belongs to, from the voucher reference in its filename. + * + * Read-only by construction. Only the NAMES are sent, so a user can review the + * whole plan (including the misses) before a single byte is uploaded, and + * before a single irreversible link is written. + * + * Scoped to the fiscal year the caller declares: `A31` identifies a verifikat + * only inside a year, because source systems restart numbering annually. + */ +export const POST = withRouteContext('import.documents.preview', async (request, ctx) => { + const { supabase, companyId, log, requestId } = ctx + + const validation = await validateBody(request, UnderlagImportPreviewSchema) + if (!validation.success) return validation.response + const { file_names: fileNames, fiscal_period_id: fiscalPeriodId } = validation.data + + // The period must belong to this company: buildUnderlagPlan filters entries + // by it, and an unowned id would silently produce an all-misses plan rather + // than an honest error. + const { data: period, error: periodError } = await supabase + .from('fiscal_periods') + .select('id') + .eq('id', fiscalPeriodId) + .eq('company_id', companyId!) + .maybeSingle() + + if (periodError) { + log.error('underlag import preview period lookup failed', periodError) + return errorResponse(periodError, log, { requestId }) + } + if (!period) { + return errorResponseFromCode('FISCAL_PERIOD_NOT_FOUND', log, { requestId }) + } + + try { + const plan = await buildUnderlagPlan(supabase, companyId!, fileNames, fiscalPeriodId) + return NextResponse.json({ data: plan }) + } catch (err) { + log.error('underlag import preview failed', err as Error, { fileCount: fileNames.length }) + return errorResponse(err, log, { requestId }) + } +}) diff --git a/components/common/FyPicker.tsx b/components/common/FyPicker.tsx index 952f1111..4f669402 100644 --- a/components/common/FyPicker.tsx +++ b/components/common/FyPicker.tsx @@ -30,6 +30,23 @@ interface FyPickerProps { * wrong there. Manual picks still work and are still persisted. */ preferLatestEnded?: boolean + /** + * Never auto-select on load, from ANY source: not the newest-period + * fallback, and not a selection persisted by an earlier session. The picker + * stays empty until the user chooses, every session. + * + * The default behaviour (restore or pick the newest) is right for a filter, + * where a sensible default beats an empty page. It is wrong where the year + * is an ASSERTION the user is making rather than a view they are narrowing: + * the underlag import resolves voucher references inside the chosen year and + * writes irreversible links. A pre-filled newest year would let a 2023 batch + * land in 2026, and a restored LAST-USED year is aimed even worse: in a + * multi-year migration the user is by definition moving to a year other than + * last time. Within one sitting the caller carries the choice in its own + * state (the wizard's reset() keeps it), which covers multi-batch runs + * without any cross-session hazard. + */ + requireExplicitChoice?: boolean /** Fires once after the initial period load completes. */ onReady?: () => void /** Server-loaded periods for the first render, scoped to initialCompanyId. */ @@ -65,6 +82,7 @@ export function FyPicker({ includeAllOption = true, hideFuturePeriods = false, preferLatestEnded = false, + requireExplicitChoice = false, onReady, initialPeriods, initialCompanyId, @@ -108,7 +126,13 @@ export function FyPicker({ // Restore last selection (same key as FiscalYearSelector so pages keep // their scope when the picker swaps in). - if (value === null && typeof window !== 'undefined') { + // + // requireExplicitChoice gates this WHOLE block, not individual branches: + // every path in here ends in an unprompted onChange (restore, the + // ALL_YEARS-stored fallback, newest-period, preferLatestEnded), and a + // per-branch gate already missed one of them once. Nothing auto-fires; + // the picker stays empty until a human picks. + if (value === null && !requireExplicitChoice && typeof window !== 'undefined') { if (preferLatestEnded) { // Filing surfaces: ignore the shared scope memory and open on the // most recently ended period (fetched is sorted newest-first). @@ -136,11 +160,15 @@ export function FyPicker({ // onReady is a lifecycle callback: fire once per load, not on parent // re-renders that re-create it. // eslint-disable-next-line react-hooks/exhaustive-deps - }, [company?.id, hideFuturePeriods, includeAllOption, preferLatestEnded, initialCompanyId, initialPeriods, storageKeyPrefix]) + }, [company?.id, hideFuturePeriods, includeAllOption, preferLatestEnded, requireExplicitChoice, initialCompanyId, initialPeriods, storageKeyPrefix]) const handleChange = (id: string) => { const nextId = id === ALL_YEARS_VALUE ? null : id - if (company?.id && typeof window !== 'undefined') { + // A per-batch assertion is never restored, so persisting it would be a + // write nothing reads. Worse than useless: this write happens BEFORE + // onChange, so a pick the caller rejects (e.g. mid-preview) would still + // be recorded as if it had taken effect. + if (!requireExplicitChoice && company?.id && typeof window !== 'undefined') { window.localStorage.setItem(storageKeyPrefix + company.id, nextId ?? ALL_YEARS_VALUE) } onChange(nextId, nextId ? periods.find((p) => p.id === nextId) ?? null : null) diff --git a/components/import/UnderlagImportWizard.tsx b/components/import/UnderlagImportWizard.tsx new file mode 100644 index 00000000..fae57072 --- /dev/null +++ b/components/import/UnderlagImportWizard.tsx @@ -0,0 +1,676 @@ +'use client' + +import { useCallback, useMemo, useRef, useState } from 'react' +import { useTranslations } from 'next-intl' +import { Card, CardContent } from '@/components/ui/card' +import { Progress } from '@/components/ui/progress' +import { Button } from '@/components/ui/button' +import { Badge } from '@/components/ui/badge' +import { Input } from '@/components/ui/input' +import { AttnLine } from '@/components/ui/attn-line' +import { EmptyState } from '@/components/ui/empty-state' +import { TD_CLASS, TH_CLASS } from '@/components/ui/dry-table' +import { useToast } from '@/components/ui/use-toast' +import { + DestructiveConfirmDialog, + useDestructiveConfirm, +} from '@/components/ui/destructive-confirm-dialog' +import { FyPicker } from '@/components/common/FyPicker' +import { getErrorMessage } from '@/lib/errors/get-error-message' +import { cn, formatDate } from '@/lib/utils' +import { FileUp, Loader2 } from 'lucide-react' +import type { FiscalPeriod } from '@/types' +import type { + UnderlagPlan, + UnderlagPlanCandidate, + UnderlagPlanRow, + UnderlagPlanStatus, +} from '@/lib/documents/underlag-import' + +// UnderlagImportWizard +// +// Attaches a folder of receipt files to verifikat that a SIE import already +// created, by reading the source voucher reference out of each filename +// (`A31_.pdf`). Deliberately NOT a step inside the SIE wizard: the receipts +// usually arrive later, from a different export, and a migration must not be +// blocked on having them ready. +// +// The plan is built from filenames alone and shown in full before anything is +// uploaded. Attaching a document to a posted verifikat makes it +// räkenskapsinformation, which cannot be re-pointed afterwards (BFL 7 kap), so +// nothing is ever attached without the user seeing exactly where it lands. +// +// The fiscal year is chosen first and every row resolves inside it. A filename +// carries no year and source systems restart voucher numbering annually, so +// `A31` names a verifikat only within a year. The year is the one piece of the +// mapping the files cannot supply, which is why the user supplies it. + +type Step = 'select' | 'review' | 'result' + +const ACCEPTED_TYPES = 'application/pdf,image/jpeg,image/png,image/webp' + +/** Message keys per status, spelled out so next-intl keeps checking them. */ +const STATUS_KEY: Record = { + matched: 'underlag_status_matched', + needs_confirmation: 'underlag_status_needs_confirmation', + ambiguous: 'underlag_status_ambiguous', + period_locked: 'underlag_status_period_locked', + no_match: 'underlag_status_no_match', + unparsed: 'underlag_status_unparsed', +} + +type Translate = (key: string, values?: Record) => string + +interface ReviewRow extends UnderlagPlanRow { + /** Position in the batch: two folders can contribute the same filename. */ + id: string + file: File + selected: boolean + targetId: string | null + /** The user picked this target by hand, so the server skips the name check. */ + manual: boolean + /** Free-text reference the user typed for a row the filename could not resolve. */ + manualRef: string + resolving: boolean +} + +interface AttachOutcome { + file_name: string + ok: boolean + message?: string +} + +/** Statuses whose single resolved target is safe to pre-select. */ +function isPreselected(status: UnderlagPlanStatus): boolean { + return status === 'matched' +} + +function badgeVariant(status: UnderlagPlanStatus): 'secondary' | 'warning' | 'destructive' { + if (status === 'ambiguous' || status === 'needs_confirmation') return 'warning' + if (status === 'period_locked') return 'destructive' + return 'secondary' +} + +export default function UnderlagImportWizard() { + const t = useTranslations('import') + const { toast } = useToast() + const { dialogProps, confirm } = useDestructiveConfirm() + const fileInputRef = useRef(null) + + const [step, setStep] = useState('select') + const [isLoading, setIsLoading] = useState(false) + const [error, setError] = useState(null) + const [fiscalPeriodId, setFiscalPeriodId] = useState(null) + const [fiscalPeriod, setFiscalPeriod] = useState(null) + /** + * The year the CURRENT plan was resolved against, snapshotted when the plan + * was requested. Everything downstream (summary, confirm text, manual + * re-resolution, the attach requests) reads this, never the live picker: the + * picker can move while a preview of 2000 filenames is in flight, and a + * confirm dialog that reads back a different year than the plan was built + * from is worse than no confirm dialog at all. + */ + const [planPeriod, setPlanPeriod] = useState(null) + const [plan, setPlan] = useState(null) + const [rows, setRows] = useState([]) + const [attached, setAttached] = useState(0) + const [outcomes, setOutcomes] = useState([]) + + const steps: Step[] = ['select', 'review', 'result'] + const stepLabels: Record = { + select: t('underlag_step_select'), + review: t('underlag_step_review'), + result: t('underlag_step_result'), + } + const currentStepIndex = steps.indexOf(step) + const progress = ((currentStepIndex + 1) / steps.length) * 100 + + const selectedRows = useMemo( + () => rows.filter((row) => row.selected && row.targetId), + [rows], + ) + + /** + * Resolve filenames server-side. Only names travel: the bytes stay here. + * The year is an explicit argument rather than read from state, so a caller + * cannot accidentally resolve against a year the user has since changed. + */ + const fetchPlan = useCallback( + async (fileNames: string[], periodId: string): Promise => { + const res = await fetch('/api/import/documents/preview', { + method: 'POST', + headers: { 'Content-Type': 'application/json' }, + body: JSON.stringify({ file_names: fileNames, fiscal_period_id: periodId }), + }) + const data = await res.json() + if (!res.ok) { + setError(getErrorMessage(data, { statusCode: res.status })) + return null + } + return data.data as UnderlagPlan + }, + [], + ) + + const handleFilesSelected = useCallback( + async (fileList: FileList | null) => { + if (!fileList || fileList.length === 0) return + if (!fiscalPeriodId) return + const files = Array.from(fileList) + // Snapshot the year for this batch up front. The picker stays on screen + // while the request is in flight, so state read afterwards may not be + // the year the plan was built from. + const batchPeriodId = fiscalPeriodId + const batchPeriod = fiscalPeriod + + setError(null) + setIsLoading(true) + try { + const nextPlan = await fetchPlan( + files.map((f) => f.name), + batchPeriodId, + ) + if (!nextPlan) return + + setPlan(nextPlan) + setPlanPeriod(batchPeriod) + setRows( + nextPlan.rows.map((row, index) => ({ + ...row, + id: `${index}:${row.file_name}`, + file: files[index], + selected: isPreselected(row.status), + // A resolved-but-locked target stays unselectable: the DB trigger + // would refuse it, so offering the checkbox would only mislead. + targetId: row.status === 'period_locked' ? null : row.journal_entry_id, + manual: false, + manualRef: '', + resolving: false, + })), + ) + setStep('review') + } catch (err) { + setError(getErrorMessage(err)) + } finally { + setIsLoading(false) + } + }, + [fetchPlan, fiscalPeriod, fiscalPeriodId], + ) + + const updateRow = useCallback((id: string, patch: Partial) => { + setRows((prev) => prev.map((row) => (row.id === id ? { ...row, ...patch } : row))) + }, []) + + /** + * Resolve a reference the user typed for a row whose filename said nothing. + * Goes through the same resolver as the automatic path: the user supplies the + * verifikat reference, never a free-choice target. + */ + const resolveManualRef = useCallback( + async (row: ReviewRow) => { + const ref = row.manualRef.trim() + if (!ref || !plan) return + + updateRow(row.id, { resolving: true }) + try { + // Resolve inside the year THIS PLAN was built against, straight from + // the server's own echo. Reading live state here would let a row join + // the batch from a different year than every other row in it. + const refPlan = await fetchPlan([ref], plan.fiscal_period_id) + const resolved = refPlan?.rows[0] + const candidates = resolved?.candidates ?? [] + + if (candidates.length === 0) { + toast({ + title: t('underlag_manual_not_found_title'), + description: t('underlag_manual_not_found_body', { ref }), + variant: 'destructive', + }) + updateRow(row.id, { resolving: false }) + return + } + + const single = candidates.length === 1 ? candidates[0] : null + updateRow(row.id, { + candidates, + resolving: false, + // Hand-resolved: the filename itself still says nothing, so the + // server cannot re-derive this target and the row carries an override. + manual: true, + targetId: single && !single.period_locked ? single.journal_entry_id : null, + selected: Boolean(single && !single.period_locked), + // Without this the row keeps rendering "Kan inte tolkas" while + // sitting checked and queued for an irreversible write. + status: single + ? single.period_locked + ? 'period_locked' + : 'needs_confirmation' + : 'ambiguous', + }) + } catch (err) { + updateRow(row.id, { resolving: false }) + toast({ title: t('underlag_manual_not_found_title'), description: getErrorMessage(err) }) + } + }, + [fetchPlan, plan, t, toast, updateRow], + ) + + const runAttach = useCallback(async () => { + if (!plan) return + // The year is named in the confirm text on purpose: it is the one input + // the files cannot corroborate, so it is the one worth reading back. + const ok = await confirm({ + title: t('underlag_confirm_title'), + description: t('underlag_confirm_body', { + count: selectedRows.length, + year: planPeriod?.name ?? '', + }), + confirmLabel: t('underlag_confirm_action'), + variant: 'warning', + }) + if (!ok) return + + setIsLoading(true) + setAttached(0) + const results: AttachOutcome[] = [] + + try { + // Sequential on purpose: hundreds of uploads in parallel would swamp the + // browser and the storage bucket, and a visible one-by-one count is what + // makes a long migration legible. + for (const row of selectedRows) { + const formData = new FormData() + formData.append('file', row.file) + formData.append('journal_entry_id', row.targetId as string) + // The year the plan was built against, echoed from the server. The + // route refuses any target outside it, overrides included. + formData.append('fiscal_period_id', plan.fiscal_period_id) + if (row.manual) formData.append('override', 'true') + + try { + const res = await fetch('/api/import/documents/attach', { + method: 'POST', + body: formData, + }) + if (!res.ok) { + const data = await res.json().catch(() => null) + results.push({ + file_name: row.file_name, + ok: false, + message: getErrorMessage(data, { statusCode: res.status }), + }) + } else { + results.push({ file_name: row.file_name, ok: true }) + } + } catch (err) { + results.push({ file_name: row.file_name, ok: false, message: getErrorMessage(err) }) + } + + setAttached((n) => n + 1) + } + } finally { + // Whatever happens above, the wizard must not stay stuck "loading": + // that state also freezes the year picker. + setIsLoading(false) + } + + setOutcomes(results) + setStep('result') + + const failed = results.filter((r) => !r.ok).length + toast({ + title: t('underlag_done_title'), + description: t('underlag_done_body', { + linked: results.length - failed, + failed, + }), + variant: failed > 0 ? 'destructive' : 'default', + }) + }, [confirm, plan, planPeriod, selectedRows, t, toast]) + + // The fiscal year survives a reset but never a session: within one sitting + // a migration is several batches from the same year's export, so in-state + // carry-over is the convenience. Cross-session persistence is the hazard + // (last-used is the wrong default for a user moving year by year), which is + // why the picker neither restores nor writes localStorage on this surface. + const reset = () => { + setStep('select') + setPlan(null) + setPlanPeriod(null) + setRows([]) + setOutcomes([]) + setAttached(0) + setError(null) + if (fileInputRef.current) fileInputRef.current.value = '' + } + + return ( +
+ + +
+
+ + {t('underlag_step_counter', { + current: currentStepIndex + 1, + total: steps.length, + label: stepLabels[step], + })} + + {steps.map((s, i) => ( + + {stepLabels[s]} + + ))} +
+ +
+ + + + {error && {error}} + + {step === 'select' && ( + + +
+

{t('underlag_intro')}

+

{t('underlag_intro_formats')}

+
+ +
+

{t('underlag_year_label')}

+ { + // Ignored while a preview is in flight: that request already + // captured a year and the plan must not disagree with the + // control the user is looking at. + if (isLoading) return + setFiscalPeriodId(id) + setFiscalPeriod(period ?? null) + }} + includeAllOption={false} + // The year is the user's assertion, so it must be the user who + // makes it, EVERY session. Defaulting to the newest year lets + // a 2023 batch resolve against 2026; restoring last-used is + // aimed even worse, since a multi-year migration by definition + // moves to a different year each round. Within one sitting, + // reset() carries the choice across batches; nothing else does. + requireExplicitChoice + className={isLoading ? 'pointer-events-none opacity-60' : undefined} + /> +

+ {t('underlag_year_help')} +

+
+ + handleFilesSelected(e.target.files)} + /> + + + + {/* Both the picker and the button are disabled until a year is + chosen, and if the company has no fiscal years at all the + picker never becomes usable. Say why rather than leave two + dead controls on screen. */} + {!fiscalPeriodId && ( +

+ {t('underlag_year_required')} +

+ )} + + + )} + + {step === 'review' && plan && ( +
+ {plan.no_source_refs && {t('underlag_no_source_refs')}} + {!plan.no_source_refs && plan.summary.period_locked > 0 && ( + + {t('underlag_locked_warning', { count: plan.summary.period_locked })} + + )} + +

+ {t('underlag_summary', { + matched: plan.summary.matched, + total: plan.summary.total, + year: planPeriod?.name ?? '', + })} +

+ +
+ + + + + + + + + + + {rows.map((row) => ( + + + + + + + ))} + +
+ {t('underlag_col_include')} + {t('underlag_col_file')}{t('underlag_col_ref')}{t('underlag_col_target')}
+ + updateRow(row.id, { selected: e.target.checked }) + } + /> + + {row.file_name} + + {row.parsed_ref + ? `${row.parsed_ref.series ?? ''}${row.parsed_ref.number}` + : {t('underlag_ref_none')}} + + + updateRow(row.id, { + targetId: candidate.journal_entry_id, + selected: !candidate.period_locked, + // Deliberately NOT `manual`: the server proposed + // this candidate itself, so it can re-derive it. + // Flagging it would switch the filename check off + // on exactly the rows it exists to protect. + }) + } + onManualRefChange={(value) => + updateRow(row.id, { manualRef: value }) + } + onManualRefSubmit={() => resolveManualRef(row)} + t={t} + /> +
+
+ +
+ + +
+
+ )} + + {step === 'result' && ( + + +

+ {t('underlag_done_body', { + linked: outcomes.filter((o) => o.ok).length, + failed: outcomes.filter((o) => !o.ok).length, + })} +

+ + {outcomes.some((o) => !o.ok) ? ( +
+ + + + + + + + + {outcomes + .filter((o) => !o.ok) + .map((o, index) => ( + + + + + ))} + +
{t('underlag_col_file')}{t('underlag_col_error')}
{o.file_name}{o.message}
+
+ ) : ( + + )} + + + + + )} + + +
+ ) +} + +function TargetCell({ + row, + onPick, + onManualRefChange, + onManualRefSubmit, + t, +}: { + row: ReviewRow + onPick: (candidate: UnderlagPlanCandidate) => void + onManualRefChange: (value: string) => void + onManualRefSubmit: () => void + t: Translate +}) { + // A single candidate is shown even when it is not selectable (locked period): + // the user needs to see WHICH verifikat the file wanted before deciding + // whether to unlock the year. + if (row.candidates.length === 1) { + const only = row.candidates[0] + return ( +
+ {only.voucher_label} + {formatDate(only.entry_date)} + {row.status !== 'matched' && ( + + {t(STATUS_KEY[row.status])} + + )} +
+ ) + } + + if (row.candidates.length > 1) { + return ( +
+ + + {t('underlag_status_ambiguous')} + +
+ ) + } + + return ( +
+ onManualRefChange(e.target.value)} + onBlur={onManualRefSubmit} + onKeyDown={(e) => { + if (e.key === 'Enter') { + e.preventDefault() + onManualRefSubmit() + } + }} + /> + {row.resolving ? ( + + ) : ( + + {t(STATUS_KEY[row.status])} + + )} +
+ ) +} diff --git a/extensions/general/arcim-migration/__tests__/import-documents.test.ts b/extensions/general/arcim-migration/__tests__/import-documents.test.ts index 47a95133..563b6568 100644 --- a/extensions/general/arcim-migration/__tests__/import-documents.test.ts +++ b/extensions/general/arcim-migration/__tests__/import-documents.test.ts @@ -65,6 +65,7 @@ function rangeMockSupabase(byTable: Record): SupabaseClient { select: () => node, eq: () => node, not: () => node, + in: () => node, order: () => node, range: () => Promise.resolve({ data: byTable[table] ?? [], error: null }), } diff --git a/extensions/general/arcim-migration/lib/import-documents.ts b/extensions/general/arcim-migration/lib/import-documents.ts index b891d8eb..b4cef44f 100644 --- a/extensions/general/arcim-migration/lib/import-documents.ts +++ b/extensions/general/arcim-migration/lib/import-documents.ts @@ -47,6 +47,12 @@ import { detectFileMagic, ALLOWED_DOCUMENT_TYPES, } from '@/lib/core/documents/document-service' +import { + buildVoucherIndex, + fetchFiscalPeriods, + fetchSourceRefVouchers, + resolveDatedRef, +} from '@/lib/documents/voucher-ref-resolver' import { fetchAllRows } from '@/lib/supabase/fetch-all' import { createLogger } from '@/lib/logger' @@ -87,20 +93,6 @@ export interface ImportDocumentsResult { unmatchedSamples: { uploadId: string; voucher: string; date: string }[] } -interface FiscalPeriodRow { - id: string - period_start: string - period_end: string -} - -interface VoucherRow { - id: string - fiscal_period_id: string - entry_date: string - source_voucher_series: string | null - source_voucher_number: number | null -} - interface ProviderAttachment { id: string fileName: string | null @@ -121,20 +113,6 @@ const EXTENSION_BY_TYPE: Record = { 'image/webp': 'webp', } -/** Find the fiscal period whose date range contains a given date. */ -function periodIdForDate(periods: FiscalPeriodRow[], date: string): string | null { - const period = periods.find((p) => p.period_start <= date && date <= p.period_end) - return period?.id ?? null -} - -/** - * In-memory key for a verifikat: fiscal period + series + number. Scoping by - * period is essential: providers may reuse voucher numbers across fiscal years. - */ -function voucherKey(periodId: string, series: string, number: number): string { - return `${periodId}|${series}|${number}` -} - /** Remove path/control characters while retaining a readable archive name. */ function sanitizeProviderFileName(fileName: string): string { return ( @@ -283,28 +261,11 @@ export async function importProviderDocuments( // ── Bulk reads (one round of paged requests each, no per-item N+1) ── const [attachments, periods, vouchers, existingAttachments] = await Promise.all([ source().list(), + fetchFiscalPeriods(supabase, companyId), + fetchSourceRefVouchers(supabase, companyId), // A stable `.order('id')` is required: fetchAllRows pages with `.range()`, // and PostgREST paging without a deterministic order can skip or repeat - // rows once a table exceeds one page (journal_entries crosses 1000 once - // several years are migrated), which would defeat both resolution and the - // hash dedup below. - fetchAllRows(({ from, to }) => - supabase - .from('fiscal_periods') - .select('id, period_start, period_end') - .eq('company_id', companyId) - .order('id', { ascending: true }) - .range(from, to), - ), - fetchAllRows(({ from, to }) => - supabase - .from('journal_entries') - .select('id, fiscal_period_id, entry_date, source_voucher_series, source_voucher_number') - .eq('company_id', companyId) - .not('source_voucher_number', 'is', null) - .order('id', { ascending: true }) - .range(from, to), - ), + // rows once a table exceeds one page, which would defeat the hash dedup. fetchAllRows<{ sha256_hash: string; journal_entry_id: string | null }>(({ from, to }) => supabase .from('document_attachments') @@ -316,24 +277,7 @@ export async function importProviderDocuments( ]) // Index gnubok verifikat by (period, series, number) for in-memory resolution. - const journalEntryByKey = new Map() - const journalEntriesBySourceRef = new Map() - const ambiguousVoucherKeys = new Set() - for (const v of vouchers) { - if (v.source_voucher_series == null || v.source_voucher_number == null) continue - const sourceRef = `${v.source_voucher_series}|${v.source_voucher_number}` - journalEntriesBySourceRef.set(sourceRef, [ - ...(journalEntriesBySourceRef.get(sourceRef) ?? []), - v, - ]) - const key = voucherKey(v.fiscal_period_id, v.source_voucher_series, v.source_voucher_number) - if (journalEntryByKey.has(key)) { - journalEntryByKey.delete(key) - ambiguousVoucherKeys.add(key) - } else if (!ambiguousVoucherKeys.has(key)) { - journalEntryByKey.set(key, v.id) - } - } + const voucherIndex = buildVoucherIndex(vouchers) // (content, verifikat) pairs already archived → idempotent skip set. Keyed // on hash + journal entry, NOT hash alone: the same content may back @@ -363,17 +307,7 @@ export async function importProviderDocuments( continue } - let journalEntryId: string | undefined - if (ref.dateTo) { - const candidates = (journalEntriesBySourceRef.get(`${ref.series}|${ref.number}`) ?? []) - .filter((voucher) => ref.date <= voucher.entry_date && voucher.entry_date <= ref.dateTo!) - journalEntryId = candidates.length === 1 ? candidates[0].id : undefined - } else { - const periodId = periodIdForDate(periods, ref.date) - journalEntryId = periodId - ? journalEntryByKey.get(voucherKey(periodId, ref.series, ref.number)) - : undefined - } + const journalEntryId = resolveDatedRef(voucherIndex, periods, ref) if (!journalEntryId) { recordUnmatched(attachment.id, `${ref.series}${ref.number}`, ref.date) diff --git a/lib/api/schemas.ts b/lib/api/schemas.ts index 67c212de..76776418 100644 --- a/lib/api/schemas.ts +++ b/lib/api/schemas.ts @@ -3191,6 +3191,20 @@ export const LinkDocumentSchema = z.object({ transaction_id: uuid.optional(), }) +/** + * Underlag import preview: filenames only, never file contents. The plan is + * built from the voucher reference in each name, so the bytes stay in the + * browser until the user has approved where each file will land. + * + * `fiscal_period_id` is required, not optional: a filename carries no year and + * source systems restart voucher numbering annually, so a plan with no declared + * year cannot identify a verifikat at all. + */ +export const UnderlagImportPreviewSchema = z.object({ + file_names: z.array(z.string().min(1).max(400)).min(1).max(2000), + fiscal_period_id: uuid, +}) + // ============================================================ // Shift-premium rules (OB-tillägg och övertid) // ============================================================ diff --git a/lib/documents/__tests__/filename-voucher-ref.test.ts b/lib/documents/__tests__/filename-voucher-ref.test.ts new file mode 100644 index 00000000..922f127d --- /dev/null +++ b/lib/documents/__tests__/filename-voucher-ref.test.ts @@ -0,0 +1,179 @@ +import { describe, it, expect } from 'vitest' +import { parseVoucherRefFromFileName } from '@/lib/documents/filename-voucher-ref' + +describe('parseVoucherRefFromFileName', () => { + it('parses the SpeedLedger prefix form (series + number + internal id)', () => { + expect(parseVoucherRefFromFileName('A31_8c2db060-79ba-4b6e-9f3d-4b0042aa5c52.pdf')).toEqual({ + series: 'A', + number: 31, + pattern: 'series_number', + autoSelectable: true, + }) + }) + + it('parses a bare series + number filename', () => { + expect(parseVoucherRefFromFileName('V123.pdf')).toMatchObject({ series: 'V', number: 123 }) + }) + + it.each([ + ['A-31 kvitto.pdf', 'A', 31], + ['A_31.jpg', 'A', 31], + ['A 31 leverantorsfaktura.png', 'A', 31], + ['2024-A-31.pdf', 'A', 31], + ['2024_A31_underlag.pdf', 'A', 31], + ['ver_A31.pdf', 'A', 31], + ['Verifikat A31.pdf', 'A', 31], + ['BC7.pdf', 'BC', 7], + ])('parses %s', (fileName, series, number) => { + expect(parseVoucherRefFromFileName(fileName)).toMatchObject({ series, number }) + }) + + it('uppercases the series so a lowercase export still joins', () => { + expect(parseVoucherRefFromFileName('a31_x.pdf')).toMatchObject({ series: 'A' }) + }) + + it.each([ + // Paper sizes: every scanner emits an A4.pdf. + 'A4.pdf', + 'A4 scan.pdf', + 'a4.pdf', + 'A3 ritning.pdf', + // A batch scanner's zero-padded counter normalizes onto the same refs. + 'A0004.pdf', + 'A001.pdf', + // Skatteverket blanketter and quarters. + 'K10.pdf', + 'K10 blankett 2024.pdf', + 'K4.pdf', + 'N9.pdf', + 'Q1 2024.pdf', + ])('parses %s but never pre-selects it: more often a document name than a ref', (fileName) => { + const parsed = parseVoucherRefFromFileName(fileName) + expect(parsed).not.toBeNull() + expect(parsed?.autoSelectable).toBe(false) + }) + + it.each(['IMG_0031.jpg', 'DSC00123.JPG', 'DOC001.pdf', 'SCN0007.pdf', 'Del 1 av 3.pdf'])( + 'parses %s but never pre-selects a three-letter series: cameras, not ledgers', + (fileName) => { + const parsed = parseVoucherRefFromFileName(fileName) + expect(parsed).not.toBeNull() + expect(parsed?.autoSelectable).toBe(false) + }, + ) + + it.each([ + ['A7.pdf', 'A', 7], + ['A31.pdf', 'A', 31], + ['K1.pdf', 'K', 1], + ['K14.pdf', 'K', 14], + ['LB2.pdf', 'LB', 2], + ])('keeps %s auto-selectable: just outside the collision list', (fileName, series, number) => { + expect(parseVoucherRefFromFileName(fileName)).toEqual({ + series, + number, + pattern: 'series_number', + autoSelectable: true, + }) + }) + + it.each([ + 'underlag/2024/A31_kvitto.pdf', + 'underlag\\A31.pdf', + // The manual-reference box feeds arbitrary typed text through this same + // parser. Splitting on the separator would turn a typed date into a + // voucher number and hand the user an irreversible link to approve. + '2024/01/31 kvitto.pdf', + '2024/01/31', + ])('does not strip a path component out of %s', (input) => { + expect(parseVoucherRefFromFileName(input)).toBeNull() + }) + + it.each([ + ['Verifikation 31.pdf', 31], + ['verifikation31.pdf', 31], + ['Verifikat 31.pdf', 31], + // `ver` is a prefix word, not a series: without that rule this one form + // came back auto-selectable while every spelled-out variant did not. + ['ver 31.pdf', 31], + ['ver31.pdf', 31], + ['VER-31.pdf', 31], + ['ver.31.pdf', 31], + ])('reads %s as a series-less reference, not a bogus series', (fileName, number) => { + expect(parseVoucherRefFromFileName(fileName)).toEqual({ + series: null, + number, + pattern: 'number_only', + autoSelectable: false, + }) + }) + + it('returns a series-less parse for a number-only name, never auto-selectable', () => { + expect(parseVoucherRefFromFileName('31.pdf')).toEqual({ + series: null, + number: 31, + pattern: 'number_only', + autoSelectable: false, + }) + expect(parseVoucherRefFromFileName('31_kvitto.pdf')).toMatchObject({ series: null, number: 31 }) + }) + + it.each([ + '20240131.pdf', + '20240131_kvitto.pdf', + '2024-01-31 kvitto.pdf', + '2024_01_31.pdf', + // Unpadded components, two-digit years and space separators are just as + // common in receipt exports and used to slip through as voucher 2024 / 24. + '2024-1-31 kvitto.pdf', + '2024_1_31.pdf', + '2024.1.31.pdf', + '2024 01 31 kvitto.pdf', + '24-01-31 kvitto.pdf', + '2024/01/31.pdf', + // Day-first and US order: the day would otherwise become a voucher number + // that always exists in the year. + '31.01.2024.pdf', + '31-01-2024.pdf', + '31_01_2024.pdf', + '31.1.2024.pdf', + '24.12.2024 julbord.pdf', + '03.04.2025 ICA.pdf', + '12.24.2024.pdf', + '01-31-2024.pdf', + '1-31-2024 receipt.pdf', + '31/1/2024.pdf', + ])('refuses the date-named file %s rather than reading it as a number', (fileName) => { + expect(parseVoucherRefFromFileName(fileName)).toBeNull() + }) + + it.each(['2024.pdf', '2024_kvitto.pdf', '1999.pdf'])( + 'refuses the year-shaped series-less name %s', + (fileName) => { + expect(parseVoucherRefFromFileName(fileName)).toBeNull() + }, + ) + + it.each([ + 'kvitto.pdf', + 'Faktura2024.pdf', + 'A31kvitto.pdf', + 'Version2_kvitto.pdf', + '', + '.pdf', + ])('returns null for %s instead of guessing', (fileName) => { + expect(parseVoucherRefFromFileName(fileName)).toBeNull() + }) + + it('rejects a zero voucher number', () => { + expect(parseVoucherRefFromFileName('A0.pdf')).toBeNull() + expect(parseVoucherRefFromFileName('0.pdf')).toBeNull() + }) + + it('handles a filename with no extension at all', () => { + expect(parseVoucherRefFromFileName('A31_8c2db060-79ba-4b6e-9f3d-4b0042aa5c52')).toMatchObject({ + series: 'A', + number: 31, + }) + }) +}) diff --git a/lib/documents/__tests__/underlag-import.test.ts b/lib/documents/__tests__/underlag-import.test.ts new file mode 100644 index 00000000..d520996e --- /dev/null +++ b/lib/documents/__tests__/underlag-import.test.ts @@ -0,0 +1,354 @@ +import { describe, it, expect } from 'vitest' +import type { SupabaseClient } from '@supabase/supabase-js' +import { buildUnderlagPlan, planPermitsAttach } from '@/lib/documents/underlag-import' +import type { FiscalPeriodRow, VoucherRow } from '@/lib/documents/voucher-ref-resolver' + +const PERIOD_OPEN = 'period-open' +const PERIOD_LOCKED = 'period-locked' + +const PERIODS: FiscalPeriodRow[] = [ + { + id: PERIOD_OPEN, + period_start: '2024-01-01', + period_end: '2024-12-31', + is_closed: false, + locked_at: null, + }, + { + id: PERIOD_LOCKED, + period_start: '2023-01-01', + period_end: '2023-12-31', + is_closed: true, + locked_at: null, + }, +] + +function makeVoucher(overrides: Partial & Pick): VoucherRow { + return { + fiscal_period_id: PERIOD_OPEN, + entry_date: '2024-03-14', + description: 'Inköp kontorsmaterial', + voucher_series: 'A', + voucher_number: 47, + source_voucher_series: 'A', + source_voucher_number: 31, + ...overrides, + } +} + +/** + * Minimal Supabase double keyed on the query SHAPE rather than call order: + * buildUnderlagPlan fires the voucher and period reads concurrently, so an + * order-sensitive queue would make these tests flaky for no benefit. + */ +function makeSupabase(opts: { + vouchers: VoucherRow[] + periods?: FiscalPeriodRow[] + /** Total migrated entries in the company, regardless of the number filter. */ + sourceRefCount?: number +}): SupabaseClient { + const periods = opts.periods ?? PERIODS + + const from = (table: string) => { + let filteredByNumber = false + + const result = () => { + if (table === 'fiscal_periods') return { data: periods, error: null, count: periods.length } + // Only the number-filtered read returns the voucher rows; the bare + // count read answers "does this ledger have ANY source refs at all". + return { + data: filteredByNumber ? opts.vouchers : [], + error: null, + count: opts.sourceRefCount ?? opts.vouchers.length, + } + } + + const chain: Record = {} + for (const method of ['select', 'eq', 'not', 'order', 'limit', 'maybeSingle', 'single']) { + chain[method] = () => chain + } + chain.in = () => { + filteredByNumber = true + return chain + } + chain.range = () => Promise.resolve(result()) + chain.then = (onFulfilled: (value: unknown) => unknown) => + Promise.resolve(result()).then(onFulfilled) + return chain + } + + return { from } as unknown as SupabaseClient +} + +describe('buildUnderlagPlan', () => { + it('matches a filename prefix to the migrated verifikat it names', async () => { + const supabase = makeSupabase({ vouchers: [makeVoucher({ id: 'je-1' })] }) + + const plan = await buildUnderlagPlan(supabase, 'company-1', [ + 'A31_8c2db060-79ba-4b6e-9f3d-4b0042aa5c52.pdf', + ], PERIOD_OPEN) + + expect(plan.rows[0]).toMatchObject({ + file_name: 'A31_8c2db060-79ba-4b6e-9f3d-4b0042aa5c52.pdf', + status: 'matched', + parsed_ref: { series: 'A', number: 31 }, + journal_entry_id: 'je-1', + }) + expect(plan.rows[0].candidates[0]).toMatchObject({ + voucher_label: 'A47', + source_voucher_label: 'A31', + period_locked: false, + }) + expect(plan.summary).toMatchObject({ total: 1, matched: 1 }) + }) + + it('matches on the SOURCE number, not our renumbered one', async () => { + // The import renumbered source A31 to A47. A file named after our own + // number must NOT match: A47 does not exist in the source system. + const supabase = makeSupabase({ vouchers: [makeVoucher({ id: 'je-1' })] }) + + const plan = await buildUnderlagPlan(supabase, 'company-1', ['A47_kvitto.pdf'], PERIOD_OPEN) + + expect(plan.rows[0].status).toBe('no_match') + }) + + it('reports a locked period instead of proposing a link the DB will refuse', async () => { + const supabase = makeSupabase({ + vouchers: [makeVoucher({ id: 'je-1', fiscal_period_id: PERIOD_LOCKED })], + }) + + const plan = await buildUnderlagPlan(supabase, 'company-1', ['A31.pdf'], PERIOD_LOCKED) + + expect(plan.rows[0]).toMatchObject({ status: 'period_locked', journal_entry_id: 'je-1' }) + expect(plan.rows[0].candidates[0].period_locked).toBe(true) + expect(plan.summary.period_locked).toBe(1) + }) + + it('treats a period with locked_at set as locked even when not closed', async () => { + const supabase = makeSupabase({ + vouchers: [makeVoucher({ id: 'je-1' })], + periods: [{ ...PERIODS[0], locked_at: '2025-01-31T00:00:00Z' }], + }) + + const plan = await buildUnderlagPlan(supabase, 'company-1', ['A31.pdf'], PERIOD_OPEN) + + expect(plan.rows[0].status).toBe('period_locked') + }) + + it('resolves ONLY inside the declared year when a ref exists in several', async () => { + const supabase = makeSupabase({ + vouchers: [ + makeVoucher({ id: 'je-2023', fiscal_period_id: PERIOD_LOCKED, entry_date: '2023-03-14' }), + makeVoucher({ id: 'je-2024' }), + ], + }) + + const plan = await buildUnderlagPlan(supabase, 'company-1', ['A31.pdf'], PERIOD_OPEN) + + expect(plan.rows[0].status).toBe('matched') + expect(plan.rows[0].journal_entry_id).toBe('je-2024') + // The other year's A31 is not even offered as a candidate. + expect(plan.rows[0].candidates.map((c) => c.journal_entry_id)).toEqual(['je-2024']) + }) + + it('NEVER proposes a verifikat outside the declared year, even as the only one', async () => { + // The defect this scoping exists for: a partial migration, or a year whose + // A31 the importer skipped, leaves exactly one A31 in the whole ledger. + // Treating that single hit as identity attached a 2023 receipt to a 2025 + // verifikat, permanently and undetectably. + const supabase = makeSupabase({ + vouchers: [makeVoucher({ id: 'je-other-year', fiscal_period_id: PERIOD_LOCKED })], + }) + + const plan = await buildUnderlagPlan(supabase, 'company-1', ['A31.pdf'], PERIOD_OPEN) + + expect(plan.rows[0].status).toBe('no_match') + expect(plan.rows[0].journal_entry_id).toBeNull() + expect(plan.rows[0].candidates).toEqual([]) + }) + + it('still hands back a choice when one ref repeats INSIDE the declared year', async () => { + const supabase = makeSupabase({ + vouchers: [ + makeVoucher({ id: 'je-a' }), + makeVoucher({ id: 'je-b', entry_date: '2024-09-02' }), + ], + }) + + const plan = await buildUnderlagPlan(supabase, 'company-1', ['A31.pdf'], PERIOD_OPEN) + + expect(plan.rows[0].status).toBe('ambiguous') + expect(plan.rows[0].journal_entry_id).toBeNull() + expect(plan.rows[0].candidates.map((c) => c.journal_entry_id)).toEqual(['je-a', 'je-b']) + expect(plan.summary.ambiguous).toBe(1) + }) + + it('reports which year the plan was resolved against', async () => { + const supabase = makeSupabase({ vouchers: [makeVoucher({ id: 'je-1' })] }) + + const plan = await buildUnderlagPlan(supabase, 'company-1', ['A31.pdf'], PERIOD_OPEN) + + expect(plan.fiscal_period_id).toBe(PERIOD_OPEN) + }) + + it('never auto-selects a collision-list ref, even on a clean single hit', async () => { + // "A4.pdf" is a scanner's paper size far more often than verifikat A4, + // and verifikat A4 exists in every migrated ledger, so the collision is + // guaranteed. The parse survives, the pre-tick does not. + const supabase = makeSupabase({ + vouchers: [makeVoucher({ id: 'je-a4', source_voucher_number: 4, voucher_number: 4 })], + }) + + const plan = await buildUnderlagPlan(supabase, 'company-1', ['A4 scan.pdf'], PERIOD_OPEN) + + expect(plan.rows[0]).toMatchObject({ + status: 'needs_confirmation', + parsed_ref: { series: 'A', number: 4 }, + journal_entry_id: 'je-a4', + }) + }) + + it('never auto-selects a series-less filename, even on a single hit', async () => { + const supabase = makeSupabase({ vouchers: [makeVoucher({ id: 'je-1' })] }) + + const plan = await buildUnderlagPlan(supabase, 'company-1', ['31.pdf'], PERIOD_OPEN) + + expect(plan.rows[0]).toMatchObject({ + status: 'needs_confirmation', + parsed_ref: { series: null, number: 31 }, + journal_entry_id: 'je-1', + }) + expect(plan.summary.needs_confirmation).toBe(1) + }) + + it('reports an unreadable filename as unparsed without touching the ledger', async () => { + const supabase = makeSupabase({ vouchers: [makeVoucher({ id: 'je-1' })] }) + + const plan = await buildUnderlagPlan(supabase, 'company-1', ['kvitto ica.pdf'], PERIOD_OPEN) + + expect(plan.rows[0]).toMatchObject({ + status: 'unparsed', + parsed_ref: null, + journal_entry_id: null, + candidates: [], + }) + expect(plan.summary.unparsed).toBe(1) + }) + + it('flags a ledger with no source refs at all, so the miss is explained', async () => { + const supabase = makeSupabase({ vouchers: [], sourceRefCount: 0 }) + + const plan = await buildUnderlagPlan(supabase, 'company-1', ['A31.pdf'], PERIOD_OPEN) + + expect(plan.rows[0].status).toBe('no_match') + expect(plan.no_source_refs).toBe(true) + }) + + it('does not blame missing source refs when the ledger has them', async () => { + const supabase = makeSupabase({ vouchers: [], sourceRefCount: 120 }) + + const plan = await buildUnderlagPlan(supabase, 'company-1', ['A31.pdf'], PERIOD_OPEN) + + expect(plan.rows[0].status).toBe('no_match') + expect(plan.no_source_refs).toBe(false) + }) + + it('skips the diagnostic round-trip once anything matched', async () => { + const supabase = makeSupabase({ vouchers: [makeVoucher({ id: 'je-1' })], sourceRefCount: 0 }) + + const plan = await buildUnderlagPlan(supabase, 'company-1', ['A31.pdf', 'kvitto.pdf'], PERIOD_OPEN) + + expect(plan.no_source_refs).toBe(false) + }) + + it('counts a mixed batch correctly', async () => { + const supabase = makeSupabase({ vouchers: [makeVoucher({ id: 'je-1' })] }) + + const plan = await buildUnderlagPlan(supabase, 'company-1', [ + 'A31_a.pdf', + 'A31_b.pdf', + 'A99.pdf', + 'kvitto.pdf', + ], PERIOD_OPEN) + + expect(plan.summary).toEqual({ + total: 4, + matched: 2, + needs_confirmation: 0, + ambiguous: 0, + period_locked: 0, + no_match: 1, + unparsed: 1, + }) + }) +}) + +describe('planPermitsAttach', () => { + it('override permits an unresolvable filename onto any same-year target', async () => { + const supabase = makeSupabase({ vouchers: [] }) + + await expect( + planPermitsAttach(supabase, 'company-1', 'kvitto ica.pdf', 'je-1', PERIOD_OPEN, true), + ).resolves.toBe(true) + // A parsed ref with no candidate in the year is unresolvable too. + await expect( + planPermitsAttach(supabase, 'company-1', 'A99.pdf', 'je-1', PERIOD_OPEN, true), + ).resolves.toBe(true) + }) + + it('override does NOT permit a resolvable filename onto a different target', async () => { + // The hole this closes: a lying client could set override=true and scatter + // cleanly-named underlag across arbitrary same-year verifikat. + const supabase = makeSupabase({ vouchers: [makeVoucher({ id: 'je-1' })] }) + + await expect( + planPermitsAttach(supabase, 'company-1', 'A31.pdf', 'je-other', PERIOD_OPEN, true), + ).resolves.toBe(false) + // The target the filename actually points at stays permitted, of course. + await expect( + planPermitsAttach(supabase, 'company-1', 'A31.pdf', 'je-1', PERIOD_OPEN, true), + ).resolves.toBe(true) + }) + + it('accepts the entry the filename resolves to', async () => { + const supabase = makeSupabase({ vouchers: [makeVoucher({ id: 'je-1' })] }) + + await expect(planPermitsAttach(supabase, 'company-1', 'A31.pdf', 'je-1', PERIOD_OPEN, false)).resolves.toBe(true) + }) + + it('accepts any candidate of an ambiguous filename: the user picked one', async () => { + const supabase = makeSupabase({ + vouchers: [makeVoucher({ id: 'je-a' }), makeVoucher({ id: 'je-b' })], + }) + + await expect( + planPermitsAttach(supabase, 'company-1', 'A31.pdf', 'je-b', PERIOD_OPEN, false), + ).resolves.toBe(true) + }) + + it('refuses a target in another fiscal year even when the ref matches', async () => { + const supabase = makeSupabase({ + vouchers: [makeVoucher({ id: 'je-other-year', fiscal_period_id: PERIOD_LOCKED })], + }) + + await expect( + planPermitsAttach(supabase, 'company-1', 'A31.pdf', 'je-other-year', PERIOD_OPEN, false), + ).resolves.toBe(false) + }) + + it('refuses an entry the filename does not point at', async () => { + const supabase = makeSupabase({ vouchers: [makeVoucher({ id: 'je-1' })] }) + + await expect(planPermitsAttach(supabase, 'company-1', 'A31.pdf', 'je-other', PERIOD_OPEN, false)).resolves.toBe( + false, + ) + }) + + it('refuses an unreadable filename outright', async () => { + const supabase = makeSupabase({ vouchers: [makeVoucher({ id: 'je-1' })] }) + + await expect(planPermitsAttach(supabase, 'company-1', 'kvitto.pdf', 'je-1', PERIOD_OPEN, false)).resolves.toBe( + false, + ) + }) +}) diff --git a/lib/documents/__tests__/voucher-ref-resolver.test.ts b/lib/documents/__tests__/voucher-ref-resolver.test.ts new file mode 100644 index 00000000..e4564183 --- /dev/null +++ b/lib/documents/__tests__/voucher-ref-resolver.test.ts @@ -0,0 +1,193 @@ +import { describe, it, expect } from 'vitest' +import { + buildVoucherIndex, + candidatesForNumber, + candidatesForRef, + periodIdForDate, + resolveDatedRef, + sourceVoucherLabel, + voucherLabel, + type FiscalPeriodRow, + type VoucherRow, +} from '@/lib/documents/voucher-ref-resolver' + +const PERIOD_2024 = 'period-2024' +const PERIOD_2025 = 'period-2025' + +const periods: FiscalPeriodRow[] = [ + { + id: PERIOD_2024, + period_start: '2024-01-01', + period_end: '2024-12-31', + is_closed: false, + locked_at: null, + }, + { + id: PERIOD_2025, + period_start: '2025-01-01', + period_end: '2025-12-31', + is_closed: false, + locked_at: null, + }, +] + +function makeVoucher(overrides: Partial & Pick): VoucherRow { + return { + fiscal_period_id: PERIOD_2024, + entry_date: '2024-03-14', + description: 'Import: A31', + voucher_series: 'A', + voucher_number: 47, + source_voucher_series: 'A', + source_voucher_number: 31, + ...overrides, + } +} + +describe('buildVoucherIndex', () => { + it('indexes an entry by both its period key and its source ref', () => { + const index = buildVoucherIndex([makeVoucher({ id: 'je-1' })]) + + expect(index.byPeriodKey.get(`${PERIOD_2024}|A|31`)).toBe('je-1') + expect(candidatesForRef(index, { series: 'A', number: 31 })).toHaveLength(1) + expect(index.ambiguousPeriodKeys.size).toBe(0) + }) + + it('skips entries with no source ref (non-SIE and pre-2026-04 imports)', () => { + const index = buildVoucherIndex([ + makeVoucher({ id: 'je-1', source_voucher_series: null, source_voucher_number: null }), + ]) + + expect(index.byPeriodKey.size).toBe(0) + expect(index.bySourceRef.size).toBe(0) + expect(index.byNumber.size).toBe(0) + }) + + it('drops BOTH entries when one source ref repeats inside a fiscal year', () => { + const index = buildVoucherIndex([ + makeVoucher({ id: 'je-1' }), + makeVoucher({ id: 'je-2' }), + ]) + + expect(index.byPeriodKey.has(`${PERIOD_2024}|A|31`)).toBe(false) + expect(index.ambiguousPeriodKeys.has(`${PERIOD_2024}|A|31`)).toBe(true) + // Both stay discoverable so a caller can present the choice. + expect(candidatesForRef(index, { series: 'A', number: 31 })).toHaveLength(2) + }) + + it('keeps a third repeat out of byPeriodKey once the key is ambiguous', () => { + const index = buildVoucherIndex([ + makeVoucher({ id: 'je-1' }), + makeVoucher({ id: 'je-2' }), + makeVoucher({ id: 'je-3' }), + ]) + + expect(index.byPeriodKey.has(`${PERIOD_2024}|A|31`)).toBe(false) + expect(candidatesForRef(index, { series: 'A', number: 31 })).toHaveLength(3) + }) + + it('keeps the same source ref in different fiscal years apart', () => { + const index = buildVoucherIndex([ + makeVoucher({ id: 'je-2024' }), + makeVoucher({ id: 'je-2025', fiscal_period_id: PERIOD_2025, entry_date: '2025-03-14' }), + ]) + + expect(index.byPeriodKey.get(`${PERIOD_2024}|A|31`)).toBe('je-2024') + expect(index.byPeriodKey.get(`${PERIOD_2025}|A|31`)).toBe('je-2025') + expect(candidatesForRef(index, { series: 'A', number: 31 })).toHaveLength(2) + }) + + it('matches series case-insensitively on both sides', () => { + const index = buildVoucherIndex([makeVoucher({ id: 'je-1', source_voucher_series: 'a' })]) + + expect(candidatesForRef(index, { series: 'A', number: 31 })).toHaveLength(1) + expect(index.byPeriodKey.get(`${PERIOD_2024}|A|31`)).toBe('je-1') + }) +}) + +describe('candidatesForNumber', () => { + it('finds a number across every series', () => { + const index = buildVoucherIndex([ + makeVoucher({ id: 'je-a', source_voucher_series: 'A' }), + makeVoucher({ id: 'je-b', source_voucher_series: 'B' }), + ]) + + expect(candidatesForNumber(index, 31).map((v) => v.id)).toEqual(['je-a', 'je-b']) + expect(candidatesForNumber(index, 999)).toEqual([]) + }) +}) + +describe('periodIdForDate', () => { + it('finds the period containing the date, inclusive of both bounds', () => { + expect(periodIdForDate(periods, '2024-01-01')).toBe(PERIOD_2024) + expect(periodIdForDate(periods, '2024-12-31')).toBe(PERIOD_2024) + expect(periodIdForDate(periods, '2025-06-01')).toBe(PERIOD_2025) + expect(periodIdForDate(periods, '2023-06-01')).toBeNull() + }) +}) + +describe('resolveDatedRef', () => { + const index = buildVoucherIndex([ + makeVoucher({ id: 'je-2024' }), + makeVoucher({ id: 'je-2025', fiscal_period_id: PERIOD_2025, entry_date: '2025-03-14' }), + ]) + + it('resolves via the fiscal period the attachment date falls in', () => { + expect(resolveDatedRef(index, periods, { series: 'A', number: 31, date: '2024-05-02' })).toBe( + 'je-2024', + ) + expect(resolveDatedRef(index, periods, { series: 'A', number: 31, date: '2025-05-02' })).toBe( + 'je-2025', + ) + }) + + it('returns undefined when the date falls outside every known period', () => { + expect( + resolveDatedRef(index, periods, { series: 'A', number: 31, date: '2023-05-02' }), + ).toBeUndefined() + }) + + it('resolves a financial-year window when exactly one entry falls inside it', () => { + expect( + resolveDatedRef(index, periods, { + series: 'A', + number: 31, + date: '2024-01-01', + dateTo: '2024-12-31', + }), + ).toBe('je-2024') + }) + + it('refuses a financial-year window that spans two candidates', () => { + expect( + resolveDatedRef(index, periods, { + series: 'A', + number: 31, + date: '2024-01-01', + dateTo: '2025-12-31', + }), + ).toBeUndefined() + }) + + it('returns undefined for an ambiguous key rather than picking one', () => { + const ambiguous = buildVoucherIndex([makeVoucher({ id: 'je-1' }), makeVoucher({ id: 'je-2' })]) + + expect( + resolveDatedRef(ambiguous, periods, { series: 'A', number: 31, date: '2024-05-02' }), + ).toBeUndefined() + }) +}) + +describe('labels', () => { + it('separates our voucher label from the source label', () => { + const entry = makeVoucher({ id: 'je-1' }) + + expect(voucherLabel(entry)).toBe('A47') + expect(sourceVoucherLabel(entry)).toBe('A31') + }) + + it('returns null when a label is not fully populated', () => { + expect(voucherLabel({ voucher_series: 'A', voucher_number: null })).toBeNull() + expect(sourceVoucherLabel({ source_voucher_series: null, source_voucher_number: 31 })).toBeNull() + }) +}) diff --git a/lib/documents/filename-voucher-ref.ts b/lib/documents/filename-voucher-ref.ts new file mode 100644 index 00000000..70462e9c --- /dev/null +++ b/lib/documents/filename-voucher-ref.ts @@ -0,0 +1,174 @@ +/** + * Read a source-system voucher reference out of an underlag filename. + * + * Systems that export receipts alongside a SIE file name each file after the + * verifikat it belongs to: SpeedLedger writes `A31_.pdf`, Fortnox + * `V123.pdf`, others `2024-A-31 kvitto.pdf`. That prefix is a deterministic + * pointer into the ledger, which is why underlag import does not need to read + * the document at all: no AI, no amount matching, no date windows. + * + * Design rule: an unrecognised name returns null. A wrong parse attaches + * räkenskapsinformation to the wrong verifikat, and that cannot be undone + * (BFL 7 kap), so the cost of guessing is far higher than the cost of asking. + */ + +export type VoucherRefPattern = + /** `A31`, `A31_uuid`, `A-31 kvitto`, `2024_A31`, `ver A31` */ + | 'series_number' + /** `31`, `31_kvitto`: a number with no series at all. */ + | 'number_only' + +export interface ParsedFileNameRef { + /** Null when the filename carried a number but no series. */ + series: string | null + number: number + pattern: VoucherRefPattern + /** + * Whether this parse may be pre-selected in a bulk plan. Three classes are + * never auto-selectable, even on a single-candidate hit: + * - series-less parses (`31.pdf` can point at any series); + * - refs on the collision list (`A4.pdf` is far more often a scanner's + * paper size than verifikat A4, `K10.pdf` a blankett); + * - three-letter series (`IMG_0031.jpg`: real SIE series are 1-2 chars, + * three letters is a camera or scanner prefix). + * They all still parse and resolve; a human confirms with one click. + */ + autoSelectable: boolean +} + +/** + * Optional noise ahead of the reference: a year folder prefix and the words + * some exporters prepend. Kept tight on purpose, `(?:19|20)\d{2}` rather than + * any 4 digits, so a voucher number is never eaten as a year. + * + * `ifikation` must precede `ifikat` in the alternation: regex alternation is + * first-match, so the short branch would otherwise consume `Verifikat` out of + * `Verifikation 31` and leave `ion` for the series group to swallow. + */ +const YEAR_NOISE = '(?:(?:19|20)\\d{2}[-_. ]+)?' +/** + * The lookahead is load-bearing, not decoration. Without it the engine + * backtracks into the shorter alternatives and `Verifikation 31` matches `ver` + * + `ifikat`, leaving `ion` for the series group to swallow as series `ION`. + * Requiring the word to end here means the prefix is either the whole word or + * not consumed at all. + */ +const VER_NOISE = '(?:ver(?:ifikation|ifikat)?(?![A-Za-zÅÄÖåäö])[-_. ]*)?' + +/** `A31`, `A-31`, `A_31`, `A 31`, optionally followed by `_`/`-`/space + anything. */ +const SERIES_NUMBER_RE = new RegExp( + `^${YEAR_NOISE}${VER_NOISE}([A-Za-zÅÄÖåäö]{1,3})[-_. ]?(\\d{1,7})(?:[-_. ].*)?$`, + 'i', +) + +/** + * `31`, `31_kvitto`, `Verifikat 31`. The `ver` prefix is allowed here but the + * year prefix is NOT: `2024 31` is far more likely a date fragment than + * voucher 31 of 2024, and this branch has no series to corroborate it with. + */ +const NUMBER_ONLY_RE = new RegExp(`^${VER_NOISE}(\\d{1,6})(?:[-_. ].*)?$`, 'i') + +/** + * A date-named file, never a voucher number. Deliberately loose where the + * parser is strict: unpadded components (`2024-1-31`), two-digit years + * (`24-01-31`), any of `-_. /` as separator, and the compact `20240131`. + * A false positive here costs one manual assignment; a false negative attaches + * a receipt to a verifikat whose number happens to equal a year fragment. + */ +const DATE_PREFIX_RE = new RegExp( + '^(?:' + + // 20240131 + '(?:19|20)\\d{6}' + + // 2024-01-31, 2024-1-31, 24-01-31, 2024 01 31, 2024/01/31 + '|(?:19|20)?\\d{2}[-_. /]\\d{1,2}[-_. /]\\d{1,2}' + + // 31.01.2024, 31/1/2024, 12-24-2024: day-first and US order. Without this + // the day becomes a voucher number that always exists in the year. + '|\\d{1,2}[-_. /]\\d{1,2}[-_. /](?:19|20)\\d{2}' + + ')(?!\\d)', +) + +/** + * `ver` is a prefix word, never a series. Without this `ver 31.pdf` parses as + * series VER and comes back auto-selectable, while the spelled-out + * `Verifikat 31.pdf` correctly yields a series-less reference that requires + * confirmation. Same filename, two trust levels, decided by an abbreviation. + */ +const NOT_A_SERIES = new Set(['VER']) + +/** + * Refs that are, in the wild, far more often document names than voucher + * references: A0-A6 are paper sizes (every scanner emits an `A4.pdf`), + * K2-K13 / N1-N9 / T1-T2 are Skatteverket blanketter, Q1-Q4 are quarters. + * Verifikat A4 genuinely exists in every migrated ledger, which is exactly + * why these must not be pre-selected: the review table cannot tell a scanned + * "A4.pdf" from the real receipt for voucher A4, and a wrong link is + * permanent. Demoted, not refused: a genuine A4 costs one click. + * + * The inconsistency this fixes: `31.pdf` already required confirmation while + * `A4 scan.pdf`, which carries LESS voucher evidence in a single-series + * company, was pre-ticked. + */ +const COLLISION_REFS = new Set([ + 'A0', 'A1', 'A2', 'A3', 'A4', 'A5', 'A6', + 'K2', 'K3', 'K4', 'K5', 'K6', 'K7', 'K8', 'K9', 'K10', 'K11', 'K12', 'K13', + 'N1', 'N2', 'N3', 'N4', 'N5', 'N6', 'N7', 'N8', 'N9', + 'T1', 'T2', + 'Q1', 'Q2', 'Q3', 'Q4', +]) + +/** Real SIE series are 1-2 characters; three letters is IMG/DSC/DOC/SCN. */ +const MAX_AUTO_SERIES_LENGTH = 2 + +/** + * Any four-digit run that reads as a calendar year. Used to refuse a + * SERIES-LESS parse: `2024` alone is overwhelmingly a year, not verifikat 2024. + */ +const YEAR_LIKE_RE = /^(?:19|20)\d{2}$/ + +/** + * Trim only. Directory components are NOT stripped: `file.name` from an + * `` never carries a path, while the manual-reference box + * feeds arbitrary user text through this same parser, where splitting on `/` + * would quietly turn the typed date `2024/01/31` into voucher 31. + */ +function baseName(fileName: string): string { + return fileName.trim() +} + +/** Drop the extension, but only a real-looking one (`.pdf`, `.jpeg`). */ +function stripExtension(name: string): string { + return name.replace(/\.[A-Za-z0-9]{1,5}$/, '') +} + +export function parseVoucherRefFromFileName(fileName: string): ParsedFileNameRef | null { + const stem = stripExtension(baseName(fileName)) + if (!stem) return null + + // A file named after its date is the single most common false positive: the + // digits parse cleanly and point at a verifikat number that has nothing to do + // with the receipt. Refuse the whole name rather than try to be clever. + if (DATE_PREFIX_RE.test(stem)) return null + + const seriesMatch = SERIES_NUMBER_RE.exec(stem) + if (seriesMatch) { + const series = seriesMatch[1].toUpperCase() + const number = Number(seriesMatch[2]) + if (!NOT_A_SERIES.has(series) && Number.isInteger(number) && number > 0) { + // The check runs on the NORMALIZED ref: a scanner's `A0004.pdf` parses + // to number 4 and must be caught by the same A4 entry. + const autoSelectable = + series.length <= MAX_AUTO_SERIES_LENGTH && !COLLISION_REFS.has(`${series}${number}`) + return { series, number, pattern: 'series_number', autoSelectable } + } + } + + const numberMatch = NUMBER_ONLY_RE.exec(stem) + if (numberMatch && !YEAR_LIKE_RE.test(numberMatch[1])) { + const number = Number(numberMatch[1]) + if (Number.isInteger(number) && number > 0) { + return { series: null, number, pattern: 'number_only', autoSelectable: false } + } + } + + return null +} diff --git a/lib/documents/underlag-import.ts b/lib/documents/underlag-import.ts new file mode 100644 index 00000000..5ef46c71 --- /dev/null +++ b/lib/documents/underlag-import.ts @@ -0,0 +1,286 @@ +/** + * Underlag import: attach a folder of receipt files to already-migrated + * verifikat, using nothing but the voucher reference in each filename. + * + * A SIE file carries the ledger but not the underlag, so a migrating customer + * has to bring the receipts over separately. Systems that export both name each + * receipt after its verifikat (`A31_.pdf`), and the SIE import preserved + * that same identity on every entry, so the pairing is a lookup rather than an + * interpretation. No AI, no amount matching, no date windows. + * + * This module only PLANS. Nothing here writes: the plan goes back to the user, + * who approves it, and each approved row is then attached one file at a time. + * That split is deliberate: linking a document to a posted verifikat makes it + * räkenskapsinformation, which can never be re-pointed (BFL 7 kap), so a bulk + * write with no preview would be an unrecoverable mistake by design. + * + * EVERY plan is scoped to one fiscal year, which the user declares. That is not + * ceremony. Source systems restart voucher numbering every year and a filename + * carries no year, so `A31` alone does not identify a verifikat. An earlier + * version resolved company-wide and treated "only one candidate exists" as + * proof of identity: with a partial migration, or with the year's A31 among the + * vouchers the importer skipped (empty, single-line, unbalanced), that silently + * attached a 2023 receipt to a 2025 verifikat, permanently. Cardinality is not + * identity. Scoping cannot make the year inferable, so it makes it asserted: + * a file can only ever land in the year the user named. + */ + +import type { SupabaseClient } from '@supabase/supabase-js' +import { parseVoucherRefFromFileName } from '@/lib/documents/filename-voucher-ref' +import { + buildVoucherIndex, + candidatesForNumber, + candidatesForRef, + fetchFiscalPeriods, + fetchVouchersForNumbers, + hasSourceRefVouchers, + sourceVoucherLabel, + voucherLabel, + type FiscalPeriodRow, + type VoucherRow, +} from '@/lib/documents/voucher-ref-resolver' + +export type UnderlagPlanStatus = + /** Exactly one open verifikat: safe to pre-select. */ + | 'matched' + /** Resolved, but the filename gave no series: a human confirms the pick. */ + | 'needs_confirmation' + /** The ref exists in several fiscal years: the user picks which one. */ + | 'ambiguous' + /** The only candidate sits in a closed or locked period: the DB will refuse. */ + | 'period_locked' + /** Parsed a ref, but no migrated verifikat carries it. */ + | 'no_match' + /** The filename carries no readable voucher reference. */ + | 'unparsed' + +export interface UnderlagPlanCandidate { + journal_entry_id: string + /** Our own label after renumbering, e.g. `A47`. */ + voucher_label: string | null + /** The label the source system used, i.e. what the filename says, e.g. `A31`. */ + source_voucher_label: string | null + entry_date: string + description: string | null + period_locked: boolean +} + +export interface UnderlagPlanRow { + file_name: string + status: UnderlagPlanStatus + parsed_ref: { series: string | null; number: number } | null + /** The single resolved target, when there is exactly one. */ + journal_entry_id: string | null + /** Every candidate, so an ambiguous row can be resolved by hand. */ + candidates: UnderlagPlanCandidate[] +} + +export interface UnderlagPlanSummary { + total: number + matched: number + needs_confirmation: number + ambiguous: number + period_locked: number + no_match: number + unparsed: number +} + +export interface UnderlagPlan { + rows: UnderlagPlanRow[] + summary: UnderlagPlanSummary + /** The fiscal year every row in this plan was resolved against. */ + fiscal_period_id: string + /** + * True when the SELECTED fiscal year holds no migrated entry carrying a + * source voucher ref. Either that year was never imported from SIE, or the + * import predates the columns (added 2026-04-21 and never backfilled), in + * which case filename matching cannot work for it at all and the UI must say + * so instead of showing 400 misses. Often it just means the wrong year is + * selected, which is the first thing worth telling the user. + */ + no_source_refs: boolean +} + +function isPeriodLocked(period: FiscalPeriodRow | undefined): boolean { + return period ? period.is_closed || period.locked_at !== null : false +} + +function toCandidate(entry: VoucherRow, periods: Map): UnderlagPlanCandidate { + return { + journal_entry_id: entry.id, + voucher_label: voucherLabel(entry), + source_voucher_label: sourceVoucherLabel(entry), + entry_date: entry.entry_date, + description: entry.description ?? null, + period_locked: isPeriodLocked(periods.get(entry.fiscal_period_id)), + } +} + +function emptySummary(): UnderlagPlanSummary { + return { + total: 0, + matched: 0, + needs_confirmation: 0, + ambiguous: 0, + period_locked: 0, + no_match: 0, + unparsed: 0, + } +} + +/** + * Build the match plan for a set of filenames, inside one declared fiscal year. + * Reads only: no upload, no link. + * + * The same function backs the preview and the per-file attach check, so the + * server can never link a file to a verifikat the preview would not have + * proposed. + */ +export async function buildUnderlagPlan( + supabase: SupabaseClient, + companyId: string, + fileNames: string[], + fiscalPeriodId: string, +): Promise { + const parsed = fileNames.map((fileName) => ({ + fileName, + ref: parseVoucherRefFromFileName(fileName), + })) + + const numbers = parsed.map((p) => p.ref?.number).filter((n): n is number => n != null) + + const [vouchers, periodRows] = await Promise.all([ + fetchVouchersForNumbers(supabase, companyId, numbers, fiscalPeriodId), + fetchFiscalPeriods(supabase, companyId), + ]) + + // The single most important line in this module: candidates outside the + // declared year are dropped BEFORE the index is built, so no downstream + // branch can ever see, count or propose one. + const index = buildVoucherIndex( + vouchers.filter((entry) => entry.fiscal_period_id === fiscalPeriodId), + ) + const periods = new Map(periodRows.map((p) => [p.id, p])) + + const summary = emptySummary() + summary.total = parsed.length + + const rows: UnderlagPlanRow[] = parsed.map(({ fileName, ref }) => { + if (!ref) { + summary.unparsed++ + return { + file_name: fileName, + status: 'unparsed', + parsed_ref: null, + journal_entry_id: null, + candidates: [], + } + } + + const parsedRef = { series: ref.series, number: ref.number } + const entries = ref.series + ? candidatesForRef(index, { series: ref.series, number: ref.number }) + : candidatesForNumber(index, ref.number) + const candidates = entries.map((entry) => toCandidate(entry, periods)) + + if (candidates.length === 0) { + summary.no_match++ + return { + file_name: fileName, + status: 'no_match', + parsed_ref: parsedRef, + journal_entry_id: null, + candidates: [], + } + } + + if (candidates.length > 1) { + // Inside one fiscal year a source ref should be unique, so this is either + // a re-imported year or a series-less filename hitting several series. + // Hand the choice back rather than pick. + summary.ambiguous++ + return { + file_name: fileName, + status: 'ambiguous', + parsed_ref: parsedRef, + journal_entry_id: null, + candidates, + } + } + + const only = candidates[0] + if (only.period_locked) { + summary.period_locked++ + return { + file_name: fileName, + status: 'period_locked', + parsed_ref: parsedRef, + journal_entry_id: only.journal_entry_id, + candidates, + } + } + + if (!ref.autoSelectable) { + summary.needs_confirmation++ + return { + file_name: fileName, + status: 'needs_confirmation', + parsed_ref: parsedRef, + journal_entry_id: only.journal_entry_id, + candidates, + } + } + + summary.matched++ + return { + file_name: fileName, + status: 'matched', + parsed_ref: parsedRef, + journal_entry_id: only.journal_entry_id, + candidates, + } + }) + + // Only worth a round-trip when nothing landed: the answer distinguishes + // "wrong filenames" from "this year holds no source refs to match against", + // which most often means the wrong year is selected. + const no_source_refs = + summary.matched + summary.needs_confirmation + summary.ambiguous + summary.period_locked === 0 + ? !(await hasSourceRefVouchers(supabase, companyId, fiscalPeriodId)) + : false + + return { rows, summary, fiscal_period_id: fiscalPeriodId, no_source_refs } +} + +/** + * Whether attaching `fileName` to `journalEntryId` is permitted by the plan. + * + * Guards the attach route against a stale or wrong client: the file the browser + * uploads must land on a verifikat the preview would propose for that name. + * `fiscalPeriodId` is the caller-supplied declared year, which the route has + * ALREADY asserted equal to the target entry's own period before calling this; + * this function only decides the filename-to-target question inside that year. + * + * `override` marks a deliberate manual assignment. It is honored ONLY when the + * filename is unresolvable in the declared year (no parse, or no candidate): + * a filename the resolver CAN place must land where it points, override or + * not, otherwise a lying client could scatter cleanly-named underlag across + * arbitrary same-year verifikat. The shipped UI only ever overrides rows whose + * filenames resolved to nothing, so this costs it no capability. + */ +export async function planPermitsAttach( + supabase: SupabaseClient, + companyId: string, + fileName: string, + journalEntryId: string, + fiscalPeriodId: string, + override: boolean, +): Promise { + const plan = await buildUnderlagPlan(supabase, companyId, [fileName], fiscalPeriodId) + const row = plan.rows[0] + if (!row) return false + if (row.candidates.some((candidate) => candidate.journal_entry_id === journalEntryId)) { + return true + } + return override && row.candidates.length === 0 +} diff --git a/lib/documents/voucher-ref-resolver.ts b/lib/documents/voucher-ref-resolver.ts new file mode 100644 index 00000000..9b7a7d18 --- /dev/null +++ b/lib/documents/voucher-ref-resolver.ts @@ -0,0 +1,314 @@ +/** + * Resolve a SOURCE-system voucher reference to the gnubok verifikat it became. + * + * The SIE importer renumbers vouchers per target series (so source `A31` may + * land as `A47` here), but it preserves the source identity on every entry: + * `journal_entries.source_voucher_series` + `source_voucher_number`, written by + * the `import_sie_journal_entries` RPC straight from #VER. That pair is the only + * safe join key back to the old system. Matching on our own `voucher_number` + * instead silently attaches underlag to the wrong verifikat as soon as the + * import skipped an empty or unbalanced voucher, which it routinely does. + * + * Two consumers, one resolution truth: + * - the provider migration sweep (Bokio/Fortnox), which knows each attachment's + * voucher ref AND its date/financial year, and + * - the underlag file import, which only knows what the filename says. + * + * Hence two entry points: `resolveDatedRef` when a date narrows the candidates, + * and `candidatesForRef` when it does not and the caller must handle ambiguity + * itself (surface the choice rather than guess: an underlag on the wrong + * verifikat is räkenskapsinformation and cannot be re-pointed afterwards). + */ + +import type { SupabaseClient } from '@supabase/supabase-js' +import { fetchAllRows } from '@/lib/supabase/fetch-all' + +/** A voucher as written in the source system. */ +export interface SourceVoucherRef { + series: string + number: number +} + +/** A source ref plus the date window that narrows it to one fiscal year. */ +export interface DatedSourceVoucherRef extends SourceVoucherRef { + /** Attachment date, or the financial year start when only that is known. */ + date: string + /** Financial year end, when the source only pins the attachment to a year. */ + dateTo?: string +} + +export interface VoucherRow { + id: string + fiscal_period_id: string + entry_date: string + source_voucher_series: string | null + source_voucher_number: number | null + // Display-only, and fetched only by the reads that need them: the provider + // sweep resolves thousands of entries and never renders any of this. + description?: string | null + voucher_series?: string | null + voucher_number?: number | null +} + +export interface FiscalPeriodRow { + id: string + period_start: string + period_end: string + is_closed: boolean + locked_at: string | null +} + +export interface VoucherIndex { + /** + * (period, series, number) → entry id, for keys that resolve to exactly one + * verifikat. Keys seen more than once are removed and recorded as ambiguous. + */ + byPeriodKey: Map + /** "series|number" → every entry carrying it, across all fiscal years. */ + bySourceRef: Map + /** number → every entry carrying it in ANY series, for series-less filenames. */ + byNumber: Map + ambiguousPeriodKeys: Set +} + +/** Find the fiscal period whose date range contains a given date. */ +export function periodIdForDate(periods: FiscalPeriodRow[], date: string): string | null { + const period = periods.find((p) => p.period_start <= date && date <= p.period_end) + return period?.id ?? null +} + +/** + * Series comparison is case-insensitive on both sides of the join. SIE writes + * series uppercase in practice but the spec does not require it, and a filename + * is whatever the user's export tool produced: `a31.pdf` must still find `A31`. + */ +function normalizeSeries(series: string): string { + return series.trim().toUpperCase() +} + +/** + * In-memory key for a verifikat: fiscal period + series + number. Scoping by + * period is essential: source systems restart voucher numbering every year, so + * `A31` alone is not unique once several years are migrated. + */ +export function voucherKey(periodId: string, series: string, number: number): string { + return `${periodId}|${normalizeSeries(series)}|${number}` +} + +/** "series|number", the period-agnostic key. */ +export function sourceRefKey(series: string, number: number): string { + return `${normalizeSeries(series)}|${number}` +} + +export function buildVoucherIndex(vouchers: VoucherRow[]): VoucherIndex { + const byPeriodKey = new Map() + const bySourceRef = new Map() + const byNumber = new Map() + const ambiguousPeriodKeys = new Set() + + // Get-or-create + push, never copy: the provider sweep indexes every + // migrated entry in the company, and per-row array copies turn that O(n²). + const appendTo = (map: Map, key: K, row: VoucherRow) => { + const list = map.get(key) + if (list) list.push(row) + else map.set(key, [row]) + } + + for (const v of vouchers) { + if (v.source_voucher_series == null || v.source_voucher_number == null) continue + + appendTo(bySourceRef, sourceRefKey(v.source_voucher_series, v.source_voucher_number), v) + appendTo(byNumber, v.source_voucher_number, v) + + const key = voucherKey(v.fiscal_period_id, v.source_voucher_series, v.source_voucher_number) + if (byPeriodKey.has(key)) { + // Two entries share one source ref inside one fiscal year: neither can be + // chosen without guessing, so drop both rather than attach blind. + byPeriodKey.delete(key) + ambiguousPeriodKeys.add(key) + } else if (!ambiguousPeriodKeys.has(key)) { + byPeriodKey.set(key, v.id) + } + } + + return { byPeriodKey, bySourceRef, byNumber, ambiguousPeriodKeys } +} + +/** + * Resolve a ref that carries date information. Returns undefined when the ref + * matches nothing or is ambiguous: callers count it as unmatched, never guess. + */ +export function resolveDatedRef( + index: VoucherIndex, + periods: FiscalPeriodRow[], + ref: DatedSourceVoucherRef, +): string | undefined { + if (ref.dateTo) { + // The source pinned the attachment to a financial year, not a day: accept + // it only when exactly one migrated verifikat in that window carries the ref. + const candidates = (index.bySourceRef.get(sourceRefKey(ref.series, ref.number)) ?? []).filter( + (voucher) => ref.date <= voucher.entry_date && voucher.entry_date <= ref.dateTo!, + ) + return candidates.length === 1 ? candidates[0].id : undefined + } + + const periodId = periodIdForDate(periods, ref.date) + return periodId ? index.byPeriodKey.get(voucherKey(periodId, ref.series, ref.number)) : undefined +} + +/** + * Every migrated verifikat carrying a source ref, across all fiscal years. + * A filename gives no date, so a ref that hits several years is genuinely + * ambiguous and the caller must ask instead of picking one. + */ +export function candidatesForRef(index: VoucherIndex, ref: SourceVoucherRef): VoucherRow[] { + return index.bySourceRef.get(sourceRefKey(ref.series, ref.number)) ?? [] +} + +/** + * Candidates for a filename that carried a number but no series (`31.pdf`). + * Searches every series, so this is only usable when it yields exactly one hit, + * and the caller must still make a human confirm it. + */ +export function candidatesForNumber(index: VoucherIndex, number: number): VoucherRow[] { + return index.byNumber.get(number) ?? [] +} + +// Both selects are written out inline at their call site rather than hoisted +// into a shared constant. tests/schema/no-phantom-columns.test.ts resolves +// column lists by scanning the AST for string literals passed to .select(); +// a constant is opaque to it, and hiding this query surface would drop all +// eight journal_entries columns out of the phantom-column net on the one code +// path that writes irreversible räkenskapsinformation links. + +/** + * Statuses a resolved verifikat may have to receive underlag. Posted is the + * normal case; reversed stays in, because a storno'd original remains + * räkenskapsinformation and its underlag belongs on it. Draft and cancelled + * are excluded: the SIE import RPC posts every entry inside its own + * transaction, so a draft with a source ref should be unobservable, but the + * link is irreversible, and an invariant that lives in another file is not an + * invariant this module may lean on. + */ +const ATTACHABLE_STATUSES = ['posted', 'reversed'] + +/** + * All entries that carry a source voucher ref, resolution columns only. + * + * A stable `.order('id')` is required: fetchAllRows pages with `.range()`, and + * PostgREST paging without a deterministic order can skip or repeat rows once + * the table exceeds one page (journal_entries crosses 1000 after a couple of + * migrated years), which would defeat both resolution and any dedup built on it. + */ +export async function fetchSourceRefVouchers( + supabase: SupabaseClient, + companyId: string, +): Promise { + return fetchAllRows(({ from, to }) => + supabase + .from('journal_entries') + .select('id, fiscal_period_id, entry_date, source_voucher_series, source_voucher_number') + .eq('company_id', companyId) + .not('source_voucher_number', 'is', null) + .in('status', ATTACHABLE_STATUSES) + .order('id', { ascending: true }) + .range(from, to), + ) +} + +/** PostgREST puts `.in()` lists in the URL, so the filter is chunked. */ +const REF_QUERY_CHUNK = 200 + +/** + * Only the entries that could match one of `numbers`, rather than every + * migrated entry in the company. The filename flow resolves a handful of refs + * per request and would otherwise pull thousands of rows into memory each time. + * Series is filtered in memory afterwards: it is case-insensitive here and a + * series-less filename has to search across all of them anyway. + * + * Carries the display columns too, because this is the read behind a plan the + * user has to be able to read before approving it. + * + * `fiscalPeriodId` narrows the read at the DB. It is an OPTIMIZATION, not the + * enforcement: buildUnderlagPlan re-filters the rows in memory before indexing, + * and that in-memory filter is the line the year guarantee rests on. + */ +export async function fetchVouchersForNumbers( + supabase: SupabaseClient, + companyId: string, + numbers: number[], + fiscalPeriodId?: string, +): Promise { + const unique = [...new Set(numbers)] + if (unique.length === 0) return [] + + const rows: VoucherRow[] = [] + for (let i = 0; i < unique.length; i += REF_QUERY_CHUNK) { + const chunk = unique.slice(i, i + REF_QUERY_CHUNK) + const chunkRows = await fetchAllRows(({ from, to }) => { + let query = supabase + .from('journal_entries') + .select( + 'id, fiscal_period_id, entry_date, description, voucher_series, voucher_number, source_voucher_series, source_voucher_number', + ) + .eq('company_id', companyId) + .in('source_voucher_number', chunk) + .in('status', ATTACHABLE_STATUSES) + if (fiscalPeriodId) query = query.eq('fiscal_period_id', fiscalPeriodId) + return query.order('id', { ascending: true }).range(from, to) + }) + rows.push(...chunkRows) + } + return rows +} + +/** + * Whether a fiscal year holds any SIE-imported entry carrying a source ref. + * Distinguishes "the filenames are wrong" from "this year was never imported + * from SIE", which are the same empty plan on screen but different problems. + */ +export async function hasSourceRefVouchers( + supabase: SupabaseClient, + companyId: string, + fiscalPeriodId: string, +): Promise { + const { count, error } = await supabase + .from('journal_entries') + .select('id', { count: 'exact', head: true }) + .eq('company_id', companyId) + .eq('fiscal_period_id', fiscalPeriodId) + .not('source_voucher_number', 'is', null) + + if (error) throw new Error(`Failed to count migrated vouchers: ${error.message}`) + return (count ?? 0) > 0 +} + +export async function fetchFiscalPeriods( + supabase: SupabaseClient, + companyId: string, +): Promise { + return fetchAllRows(({ from, to }) => + supabase + .from('fiscal_periods') + .select('id, period_start, period_end, is_closed, locked_at') + .eq('company_id', companyId) + .order('id', { ascending: true }) + .range(from, to), + ) +} + +/** Our own label for a verifikat (`A47`), as opposed to the source label. */ +export function voucherLabel(entry: Pick): string | null { + return entry.voucher_series && entry.voucher_number != null + ? `${entry.voucher_series}${entry.voucher_number}` + : null +} + +/** The label the source system used (`A31`), which is what filenames carry. */ +export function sourceVoucherLabel( + entry: Pick, +): string | null { + return entry.source_voucher_series && entry.source_voucher_number != null + ? `${entry.source_voucher_series}${entry.source_voucher_number}` + : null +} diff --git a/lib/errors/structured-errors.ts b/lib/errors/structured-errors.ts index 667aee8f..a6ebbcd7 100644 --- a/lib/errors/structured-errors.ts +++ b/lib/errors/structured-errors.ts @@ -2015,6 +2015,30 @@ const DOCUMENT: Record = { message_sv: 'Kopplingen misslyckades.', message_en: 'Failed to link document to journal entry.', }, + UNDERLAG_REF_MISMATCH: { + httpStatus: 409, + message_sv: + 'Filnamnet pekar inte på den verifikation som valdes. Ladda om förhandsgranskningen och försök igen.', + message_en: + 'The filename does not point at the selected verifikat. Reload the preview and try again.', + }, + UNDERLAG_PERIOD_MISMATCH: { + httpStatus: 409, + message_sv: + 'Verifikationen tillhör ett annat räkenskapsår än det du valde för underlagen. Ladda om förhandsgranskningen och försök igen.', + message_en: + 'The verifikat belongs to a different fiscal year than the one selected for these files. Reload the preview and try again.', + }, + UNDERLAG_ENTRY_NOT_POSTED: { + httpStatus: 409, + message_sv: 'Verifikationen är inte bokförd, så underlag kan inte kopplas till den ännu.', + message_en: 'The journal entry is not posted, so documents cannot be attached to it yet.', + }, + UNDERLAG_ENTRY_NOT_MIGRATED: { + httpStatus: 400, + message_sv: 'Verifikationen kommer inte från en SIE-import och kan inte matchas mot filnamn.', + message_en: 'The journal entry did not come from a SIE import and cannot be matched by filename.', + }, } // Invoice-inbox manual upload and attach-document (extension REST routes). diff --git a/messages/en.json b/messages/en.json index e0f3c2c9..f43fe463 100644 --- a/messages/en.json +++ b/messages/en.json @@ -6987,6 +6987,48 @@ "csv_chip_articles": "Articles", "sie_title": "SIE file", "sie_description": "Bookkeeping from another system or from your accountant", + "underlag_title": "Receipts for imported vouchers", + "underlag_description": "Attach receipts and invoices to verifikat from a SIE import, using the voucher number in the filename", + "underlag_step_select": "Pick files", + "underlag_step_review": "Review", + "underlag_step_result": "Result", + "underlag_step_counter": "Step {current}/{total}: {label}", + "underlag_intro": "A SIE file carries the bookkeeping but not the receipts. If your old system named each receipt after its verifikat, they can be attached automatically, with no AI reading required.", + "underlag_intro_formats": "Filenames that work: A31_abc123.pdf, A31.pdf, A-31 receipt.pdf, 2024-A-31.pdf. Matching uses the voucher number from your old system, not the number here.", + "underlag_pick_files": "Pick files", + "underlag_year_help": "The old system restarts voucher numbering every year, so A31 only identifies a verifikat within one year. Pick the year the export came from. Files are never attached to any other year.", + "underlag_year_required": "Pick a fiscal year above to continue. If the year you want is missing, its bookkeeping was not imported via SIE.", + "underlag_year_label": "Which fiscal year are these receipts from?", + "underlag_summary": "{matched} of {total} files matched a verifikat in {year}.", + "underlag_no_source_refs": "The selected fiscal year holds no verifikat from a SIE import, so there is nothing to match the filenames against. Check that you picked the right year.", + "underlag_locked_warning": "{count, plural, one {1 file points at a closed or locked period. Unlock the period first, otherwise it cannot be attached.} other {# files point at a closed or locked period. Unlock the period first, otherwise they cannot be attached.}}", + "underlag_col_include": "Include", + "underlag_col_file": "File", + "underlag_col_ref": "Parsed", + "underlag_col_target": "Verifikat", + "underlag_col_error": "Error", + "underlag_ref_none": "No voucher number", + "underlag_status_matched": "Matched", + "underlag_status_needs_confirmation": "Confirm", + "underlag_status_ambiguous": "Several matches", + "underlag_status_period_locked": "Locked period", + "underlag_status_no_match": "No match", + "underlag_status_unparsed": "Cannot be parsed", + "underlag_pick_candidate": "Pick a verifikat", + "underlag_manual_placeholder": "e.g. A31", + "underlag_manual_not_found_title": "No verifikat found", + "underlag_manual_not_found_body": "No imported verifikat carries the number {ref} from the old system.", + "underlag_confirm_title": "Attach the receipts?", + "underlag_confirm_body": "{count, plural, one {1 file will be archived and attached to its verifikat in {year}.} other {# files will be archived and attached to verifikat in {year}.}} A link to a posted verifikat is accounting information and cannot be moved afterwards.", + "underlag_confirm_action": "Attach", + "underlag_run": "{count, plural, one {Attach 1 receipt} other {Attach # receipts}}", + "underlag_running": "Attaching {done} of {total}...", + "underlag_back": "Start over", + "underlag_done_title": "Receipts attached", + "underlag_done_body": "{linked} attached, {failed} failed.", + "underlag_all_ok_title": "All receipts attached", + "underlag_all_ok_body": "Every selected file is now attached to its verifikat.", + "underlag_new_import": "Import more receipts", "loading_migration": "Loading migration tool...", "export_heading": "Export", "export_sie_title": "SIE 4", diff --git a/messages/sv.json b/messages/sv.json index cfc635a5..6bd74a94 100644 --- a/messages/sv.json +++ b/messages/sv.json @@ -6987,6 +6987,48 @@ "csv_chip_articles": "Artiklar", "sie_title": "SIE-fil", "sie_description": "Bokföring från ett annat system eller från revisorn", + "underlag_title": "Underlag till importerade verifikat", + "underlag_description": "Koppla kvitton och fakturor till verifikat från en SIE-import, via verifikatnumret i filnamnet", + "underlag_step_select": "Välj filer", + "underlag_step_review": "Granska", + "underlag_step_result": "Resultat", + "underlag_step_counter": "Steg {current}/{total}: {label}", + "underlag_intro": "En SIE-fil innehåller bokföringen men inte underlagen. Om ditt gamla system namngav varje underlag efter sitt verifikat kan de kopplas automatiskt, utan AI-tolkning.", + "underlag_intro_formats": "Filnamn som fungerar: A31_abc123.pdf, A31.pdf, A-31 kvitto.pdf, 2024-A-31.pdf. Matchningen sker mot verifikatnumret i ditt gamla system, inte mot numret här.", + "underlag_pick_files": "Välj filer", + "underlag_year_help": "Verifikatnumret i filnamnet räknas om varje år i det gamla systemet, så A31 pekar bara ut ett verifikat inom ett år. Välj det år exporten kommer från. Filerna kopplas aldrig till något annat år.", + "underlag_year_required": "Välj ett räkenskapsår ovan för att fortsätta. Saknas det år du söker har bokföringen för det året inte importerats via SIE.", + "underlag_year_label": "Vilket räkenskapsår gäller underlagen?", + "underlag_summary": "{matched} av {total} filer matchade ett verifikat i {year}.", + "underlag_no_source_refs": "Det valda räkenskapsåret innehåller inga verifikat från en SIE-import, så det finns inget att matcha filnamnen mot. Kontrollera att du valt rätt år.", + "underlag_locked_warning": "{count, plural, one {1 fil pekar på en stängd eller låst period. Lås upp perioden först, annars går den inte att koppla.} other {# filer pekar på en stängd eller låst period. Lås upp perioden först, annars går de inte att koppla.}}", + "underlag_col_include": "Ta med", + "underlag_col_file": "Fil", + "underlag_col_ref": "Tolkat", + "underlag_col_target": "Verifikat", + "underlag_col_error": "Fel", + "underlag_ref_none": "Inget verifikatnummer", + "underlag_status_matched": "Matchad", + "underlag_status_needs_confirmation": "Bekräfta", + "underlag_status_ambiguous": "Flera träffar", + "underlag_status_period_locked": "Låst period", + "underlag_status_no_match": "Ingen träff", + "underlag_status_unparsed": "Kan inte tolkas", + "underlag_pick_candidate": "Välj verifikat", + "underlag_manual_placeholder": "T.ex. A31", + "underlag_manual_not_found_title": "Hittade inget verifikat", + "underlag_manual_not_found_body": "Ingen importerad verifikation har numret {ref} från det gamla systemet.", + "underlag_confirm_title": "Koppla underlagen?", + "underlag_confirm_body": "{count, plural, one {1 fil arkiveras och kopplas till sitt verifikat i {year}.} other {# filer arkiveras och kopplas till verifikat i {year}.}} En koppling till ett bokfört verifikat är räkenskapsinformation och går inte att flytta i efterhand.", + "underlag_confirm_action": "Koppla", + "underlag_run": "{count, plural, one {Koppla 1 underlag} other {Koppla # underlag}}", + "underlag_running": "Kopplar {done} av {total}...", + "underlag_back": "Börja om", + "underlag_done_title": "Underlagen är kopplade", + "underlag_done_body": "{linked} kopplade, {failed} misslyckades.", + "underlag_all_ok_title": "Alla underlag kopplade", + "underlag_all_ok_body": "Varje vald fil ligger nu som underlag på sitt verifikat.", + "underlag_new_import": "Importera fler underlag", "loading_migration": "Laddar migreringsverktyg...", "export_heading": "Exportera", "export_sie_title": "SIE 4", diff --git a/tests/pg/underlag-attach-period-lock.pg.test.ts b/tests/pg/underlag-attach-period-lock.pg.test.ts new file mode 100644 index 00000000..6e5a42d3 --- /dev/null +++ b/tests/pg/underlag-attach-period-lock.pg.test.ts @@ -0,0 +1,132 @@ +import { randomUUID } from 'node:crypto' +import { describe, expect, it } from 'vitest' +import { getPool } from '@/tests/pg/setup' +import { insertBalancedLines, insertDraftJournalEntry, seedCompany } from '@/tests/pg/fixtures' + +// The underlag import (lib/documents/underlag-import.ts) refuses to propose a +// file whose target verifikat sits in a closed or locked period, and the attach +// route maps the failure to DOC_UPLOAD_PERIOD_LOCKED. Both rest on +// enforce_period_lock_documents (migration 20240101000017): a BEFORE +// INSERT/UPDATE trigger on document_attachments that blocks the LINK, not the +// archive. These tests pin that contract, because the plan surface would +// otherwise be free to drift into promising links the database refuses. +const PERIOD_LOCK_ERROR = /locked\/closed fiscal period/i + +async function insertPostedEntry(params: { + userId: string + companyId: string + fiscalPeriodId: string + voucherNumber: number +}): Promise { + const entryId = await insertDraftJournalEntry({ + userId: params.userId, + companyId: params.companyId, + fiscalPeriodId: params.fiscalPeriodId, + voucherNumber: params.voucherNumber, + }) + await insertBalancedLines(entryId) + await getPool().query(`UPDATE public.journal_entries SET status = 'posted' WHERE id = $1`, [ + entryId, + ]) + return entryId +} + +async function insertDocument(params: { + userId: string + companyId: string + journalEntryId: string | null +}): Promise { + const id = randomUUID() + await getPool().query( + `INSERT INTO public.document_attachments + (id, user_id, company_id, storage_path, file_name, file_size_bytes, + mime_type, sha256_hash, journal_entry_id) + VALUES ($1, $2, $3, $4, $5, $6, $7, $8, $9)`, + [ + id, + params.userId, + params.companyId, + `documents/${params.userId}/${id}.pdf`, + 'A31_underlag.pdf', + 1024, + 'application/pdf', + randomUUID().replace(/-/g, '').padEnd(64, '0'), + params.journalEntryId, + ], + ) + return id +} + +describe('underlag-attach-period-lock.pg: attaching underlag across a period lock', () => { + it('allows attaching to a verifikat in an open period', async () => { + const { userId, companyId, fiscalPeriodId } = await seedCompany() + const entryId = await insertPostedEntry({ + userId, companyId, fiscalPeriodId, voucherNumber: 1, + }) + + await expect( + insertDocument({ userId, companyId, journalEntryId: entryId }), + ).resolves.toBeDefined() + }) + + it('rejects attaching to a verifikat in a CLOSED period', async () => { + const { userId, companyId, fiscalPeriodId } = await seedCompany() + const entryId = await insertPostedEntry({ + userId, companyId, fiscalPeriodId, voucherNumber: 1, + }) + await getPool().query( + `UPDATE public.fiscal_periods SET is_closed = true, closed_at = now() WHERE id = $1`, + [fiscalPeriodId], + ) + + await expect( + insertDocument({ userId, companyId, journalEntryId: entryId }), + ).rejects.toThrow(PERIOD_LOCK_ERROR) + }) + + it('rejects attaching to a verifikat in a LOCKED period', async () => { + const { userId, companyId, fiscalPeriodId } = await seedCompany() + const entryId = await insertPostedEntry({ + userId, companyId, fiscalPeriodId, voucherNumber: 1, + }) + await getPool().query(`UPDATE public.fiscal_periods SET locked_at = now() WHERE id = $1`, [ + fiscalPeriodId, + ]) + + await expect( + insertDocument({ userId, companyId, journalEntryId: entryId }), + ).rejects.toThrow(PERIOD_LOCK_ERROR) + }) + + it('still archives an UNLINKED document in a closed period: the lock guards the link', async () => { + const { userId, companyId, fiscalPeriodId } = await seedCompany() + await getPool().query( + `UPDATE public.fiscal_periods SET is_closed = true, closed_at = now() WHERE id = $1`, + [fiscalPeriodId], + ) + + await expect( + insertDocument({ userId, companyId, journalEntryId: null }), + ).resolves.toBeDefined() + }) + + it('rejects linking an already-archived document into a locked period', async () => { + // The UPDATE path matters as much as the INSERT: /api/documents/[id]/link + // moves an existing inbox document onto a verifikat. + const { userId, companyId, fiscalPeriodId } = await seedCompany() + const entryId = await insertPostedEntry({ + userId, companyId, fiscalPeriodId, voucherNumber: 1, + }) + const docId = await insertDocument({ userId, companyId, journalEntryId: null }) + await getPool().query(`UPDATE public.fiscal_periods SET locked_at = now() WHERE id = $1`, [ + fiscalPeriodId, + ]) + + await expect( + getPool().query( + `UPDATE public.document_attachments SET journal_entry_id = $1 WHERE id = $2`, + [entryId, docId], + ), + ).rejects.toThrow(PERIOD_LOCK_ERROR) + }) +})