Files
accounted/lib/documents/filename-voucher-ref.ts
T
MattssonandClaude Opus 5 2deea05d42 feat(import): attach underlag to SIE-migrated verifikat by filename (#1627)
* refactor(documents): lift the SIE voucher-ref resolver into core

The provider migration sweep resolved a source voucher reference to the
verifikat it became with an in-memory (period, series, number) index built
inside extensions/general/arcim-migration. The underlag filename import needs
the identical resolution, and core must never import from @/extensions, so the
index, its ambiguity handling and the two paged reads move to
lib/documents/voucher-ref-resolver.ts.

Behaviour-preserving for the extension: same index construction, same "drop
both when one key repeats inside a fiscal year" rule, same dateTo-window
resolution. The arcim tests pass unchanged.

Two deliberate additions on top of the lift:
  - series comparison is now case-insensitive on both sides. SIE writes series
    uppercase in practice but the spec does not require it, and a filename is
    whatever the exporting tool produced.
  - byNumber and fetchVouchersForNumbers serve the filename flow, which
    resolves a handful of refs per request and must not pull every migrated
    entry into memory to do it.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>

* feat(import): attach underlag to SIE-migrated verifikat by filename

A SIE file carries the ledger but not the underlag, so a migrating customer
brings the receipts over separately and today has to open every verifikat and
attach them by hand. Systems that export both name each receipt after its
verifikat (A31_<internal-id>.pdf), and the SIE import already preserves that
identity on every entry (source_voucher_series / source_voucher_number), so
the pairing is a lookup, not an interpretation: no AI, no amount matching, no
date windows.

Separate optional import mode (/import?mode=underlag), NOT a step inside the
SIE wizard: the receipts normally arrive later and from a different export, so
a migration must never be blocked on having them ready.

  lib/documents/filename-voucher-ref.ts  reads the ref out of a filename
  lib/documents/underlag-import.ts       builds the plan (reads only)
  POST /api/import/documents/preview     filenames in, match plan out
  POST /api/import/documents/attach      one file, archived and linked
  components/import/UnderlagImportWizard review, adjust, run

Guards, because a document linked to a posted verifikat is
räkenskapsinformation and can never be re-pointed (BFL 7 kap):

  - Matching keys on the SOURCE voucher number, never our own. The importer
    renumbers per target series, so a file named after our number would land
    on the wrong verifikat exactly when the import skipped a voucher.
  - Nothing is uploaded until the whole plan has been shown: the preview
    sends filenames only, the bytes stay in the browser.
  - A ref that hits several migrated years is surfaced as a choice, never
    resolved by guessing. So is a filename with a number but no series, which
    is resolved but never pre-selected.
  - A date-named file (20240131.pdf) is refused outright rather than read as
    voucher 20240131.
  - A target in a closed or locked period is shown but not selectable:
    enforce_period_lock_documents would refuse the write anyway.
  - The attach route re-resolves the filename server-side and 409s when it
    does not name the target the client sent, so a stale plan cannot scatter
    underlag permanently. An explicit manual assignment opts out of that check
    and is flagged as such; company ownership of the entry is always verified.
  - Idempotent per (verifikat, content): a re-run converges on the same
    document row instead of archiving duplicates.

tests/pg/underlag-attach-period-lock.pg.test.ts pins the period-lock contract
the plan surface promises, including that the lock guards the LINK and still
lets an unlinked document be archived.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>

* fix(import): scope underlag matching to a declared fiscal year

Adversarial review of #1627 refuted the resolver: it looked a ref up
company-wide and treated "exactly one candidate exists" as proof of identity.
Source systems restart voucher numbering every year and a filename carries no
year, so with a partial migration, or with that year's A31 among the vouchers
the importer routinely skips (empty, single-line, unbalanced), a 2023 receipt
was silently attached to a 2025 verifikat. Permanent under BFL 7 kap, and
invisible afterwards. Cardinality is not identity.

Every batch now declares its fiscal year and candidates outside it are dropped
before the index is built, so no downstream branch can see, count or propose
one. The attach route takes the year for its re-resolution from the TARGET
entry, never from the client, so the check cannot be widened by naming a
different year. Scoping cannot make the year inferable; it makes it asserted,
and the confirm dialog reads it back because it is the one input the files
cannot corroborate.

Four further defects from the same review:

  - npm test went red: hoisting the column list into a VOUCHER_SELECT constant
    hid it from the no-phantom-columns AST scan (ceiling 377 -> 379) and
    dropped all eight journal_entries columns out of the guard on the one path
    that writes irreversible links. Both selects are inline again, and split:
    the provider sweep no longer fetches three display columns it never reads.
  - The date guard only caught zero-padded hyphenated dates, so
    `2024-1-31 kvitto.pdf`, `2024 01 31 ...`, `2024.1.31` and `24-01-31` all
    parsed as voucher 2024 or 24. Widened to unpadded components, two-digit
    years and space/slash separators; a bare year-shaped number is refused.
  - `Verifikation 31.pdf` parsed as series ION: the alternation matched
    `ifikat` and left `ion` for the series group. Reordering alone was not
    enough (the engine backtracks into it), so the prefix now requires the
    word to end.
  - The manual-reference box was an unguarded write path: typing a date got
    path-split down to a voucher number, marked the row selected, and posted
    with override, which skips both server checks, while the row still showed
    "Kan inte tolkas". Directory splitting is gone from the parser, the row
    status is updated on resolve, and picking a server-proposed candidate no
    longer counts as an override, which had disabled the filename check on
    exactly the ambiguous rows it exists to protect.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>

* fix(import): enforce the declared fiscal year on the server

The second adversarial pass refuted the previous fix. The attach route took
the year for its re-resolution from the TARGET entry, which is tautological:
an entry is by construction inside its own fiscal_period_id, so the filter
could never drop it and the year axis was unfalsifiable. Server-side year
enforcement was zero; the declared year existed only as React state and was
never sent. The regression test that "proved" otherwise passed only because
the mock let one journal_entries row report two different fiscal_period_id
values to two different reads, a state Postgres cannot produce. A test that
could not fail.

The attach request now carries the year the user actually reviewed, echoed
back from the plan, and the route asserts it equals the target's own period
BEFORE any other check and including overrides: an override is a statement
about which verifikat, never about which year. Its test asserts that directly
instead of a mock artifact.

Also from the same pass, a UI race that made the confirm dialog lie: FyPicker
stayed interactive while a preview of up to 2000 filenames was in flight, so
the summary and the confirm text could read back a year the plan was not built
from, and a manually resolved row could join the batch from another year
entirely. The wizard snapshots the plan's year, every downstream read uses the
snapshot, manual re-resolution goes through the server's own echoed
plan.fiscal_period_id, and the picker is frozen while a preview runs.

Parser, from the corpus pass (~360 realistic filenames plus 200k random uuids,
no ReDoS found: 2000 hostile inputs in 26ms):

  - Day-first and US dates parsed as voucher numbers: `31.01.2024` became
    voucher 31, a number that always exists in the year. The guard now covers
    both orders.
  - `ver 31.pdf` parsed as series VER and came back auto-selectable, while
    every spelled-out `Verifikat 31.pdf` correctly yielded a series-less
    reference needing confirmation. Same filename, two trust levels, decided
    by an abbreviation. `ver` is no longer a series.

Known residual, stated rather than papered over: a scanner's `A4.pdf` or a
`K10.pdf` blankett in the receipts folder still matches verifikat A4 or K10
when that year has them. No parser can separate those from a genuine
reference; they appear in the review table with the target's date and
description.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>

* fix(import): make the user actually declare the fiscal year

The third adversarial pass found that the central guarantee of the previous
two commits was fiction. FyPicker auto-selects the newest fiscal period when
nothing is stored, and the wizard passes a page-specific storage key, so that
branch fired on every first use. A user migrating 2023 receipts who never
opened the picker resolved them against the newest year; A31 exists in
essentially every year, so those rows came back `matched`, pre-selected, with
only the confirm dialog between them and permanent links. Every commit message
and code comment claiming "the year the user named" described behaviour the UI
did not have.

FyPicker gains an opt-in `requireExplicitChoice` prop, default off so no other
caller changes, and the wizard uses it. The picker starts empty and the batch
cannot proceed until someone picks. A previously stored explicit choice for
this surface is still restored, which is what makes a multi-batch migration
bearable.

Also: a company with zero fiscal periods hit a disabled picker and a disabled
button with no explanation. There is now a line saying why.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>

* fix(import): close the restore-branch hole and demote collision-prone refs

Round four of adversarial review, two findings, both fixed.

1. `requireExplicitChoice` gated only the newest-period fallback, not the
   localStorage restore branch above it, so the "user declares the year"
   guarantee held only for a user's first-ever batch. From the second on, the
   year was silently pre-filled from an earlier unrelated batch, and in a
   multi-year migration last-used is the worst possible default: the user is
   by definition moving to a different year each round. The prop now gates
   FyPicker's ENTIRE auto-selection block with one outer condition (restore,
   the ALL_YEARS-stored fallback, newest-period, preferLatestEnded), because a
   per-branch gate already missed one branch once. It also suppresses the
   localStorage write, which fired BEFORE onChange and so recorded picks the
   wizard had rejected mid-preview. The wizard drops its storage prefix
   entirely: within one sitting reset() carries the year in state, and
   nothing survives the session.

2. The filename parser pre-ticked `A4 scan.pdf` and `K10.pdf` while requiring
   a click for `31.pdf`, which carries MORE voucher evidence in a
   single-series company. Two independent review passes flagged the same
   inconsistency. Collision-famous refs (A0-A6 paper sizes, K2-K13/N1-N9/
   T1-T2 blanketter, Q1-Q4 quarters) and three-letter series (IMG/DSC/DOC/
   SCN are cameras; real SIE series are 1-2 chars) still parse and resolve
   but are never auto-selected. Demoted, not refused: verifikat A4 genuinely
   exists in every migrated ledger, and its real receipt costs one click.
   Residual documented: an existing short series plus a small number in an
   ad-hoc name (`B2 hyra.pdf`) is indistinguishable from a real ref by
   filename alone.

Also: the attach route's multipart doc now names the required
fiscal_period_id field, and the stale reset() comment describes the actual
persistence model.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>

* fix(import): honor override only for unresolvable filenames + review round

Resolution pass for the PR #1627 review reports (CodeRabbit, Swedish
accounting review, compliance swarm).

The one substantive finding (CodeRabbit, major): `override: true` skipped the
filename consistency check entirely, so a crafted client could attach a
cleanly-named file to any same-year verifikat. The resolver now runs on every
request; an override is honored only when the filename is unresolvable in the
declared year (no parse, or no candidate) or already resolves to the requested
target. The shipped UI only overrides unresolvable rows, so nothing
user-facing changes. planAcceptsTarget is renamed planPermitsAttach and
carries the semantics in one place, with tests for both directions.

The Swedish review finding (BFNAR 2013:2 systemdokumentation): the
planPermitsAttach JSDoc still described the superseded derive-the-year-from-
the-target design. It now states the actual control: the route asserts the
caller-declared year equals the target's own period before this function runs.

CodeRabbit minors and nitpicks:
  - underlag_confirm_body / underlag_run / underlag_locked_warning use ICU
    plural forms in both locales; "1 filer arkiveras" was wrong Swedish.
  - The attach and preview route tests mock @/lib/supabase/server per the
    repo test guideline.
  - fetchVouchersForNumbers narrows to the declared fiscal year at the DB;
    the in-memory filter in buildUnderlagPlan remains the enforced truth.
  - buildVoucherIndex appends into existing arrays instead of copying per
    row: the provider sweep indexes every migrated entry in the company and
    per-row copies made that O(n^2).
  - The pg test reuses its insertDocument helper instead of a duplicated
    INSERT; runAttach clears isLoading in a finally.

Declined, with reasons in DECISIONS.md: message-regex classification of
validateDocumentFile failures (established sibling pattern; validator
contract change is out of scope).

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>

* fix(import): attach only to posted or reversed verifikat

Second review cycle on PR #1627: the Swedish accounting review's re-run found
that nothing in the attach route verified the target entry's status. The SIE
import RPC posts every entry inside its own transaction, so a draft carrying a
source ref should be unobservable, but the link this route writes is
irreversible räkenskapsinformation, and an invariant enforced in another file
is not one this surface may lean on. Underlag references a verifikation
(BFL 5 kap 6-7 §), so the target must BE one.

Enforced twice: the route rejects non-posted targets with
UNDERLAG_ENTRY_NOT_POSTED (overrides included), and the resolver reads filter
to posted/reversed so a draft can never even become a candidate. Reversed
stays attachable: a storno'd original remains räkenskapsinformation and its
underlag belongs on it.

Also recorded as confirmed-intentional (review note, no code change): with
override and an unresolvable filename the endpoint links to any same-company,
same-declared-year, posted verifikat, migrated or not, which mirrors the
existing /api/documents/[id]/link capability. The period-lock error-string
regex note restates a disposition already recorded in DECISIONS.md.

The arcim test's Supabase double learns .in(), which the shared resolver read
now uses for the status filter.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>

---------

Co-authored-by: Claude Opus 5 (1M context) <noreply@anthropic.com>
2026-08-15 19:50:32 +02:00

175 lines
7.3 KiB
TypeScript

/**
* Read a source-system voucher reference out of an underlag filename.
*
* Systems that export receipts alongside a SIE file name each file after the
* verifikat it belongs to: SpeedLedger writes `A31_<internal-uuid>.pdf`, Fortnox
* `V123.pdf`, others `2024-A-31 kvitto.pdf`. That prefix is a deterministic
* pointer into the ledger, which is why underlag import does not need to read
* the document at all: no AI, no amount matching, no date windows.
*
* Design rule: an unrecognised name returns null. A wrong parse attaches
* räkenskapsinformation to the wrong verifikat, and that cannot be undone
* (BFL 7 kap), so the cost of guessing is far higher than the cost of asking.
*/
export type VoucherRefPattern =
/** `A31`, `A31_uuid`, `A-31 kvitto`, `2024_A31`, `ver A31` */
| 'series_number'
/** `31`, `31_kvitto`: a number with no series at all. */
| 'number_only'
export interface ParsedFileNameRef {
/** Null when the filename carried a number but no series. */
series: string | null
number: number
pattern: VoucherRefPattern
/**
* Whether this parse may be pre-selected in a bulk plan. Three classes are
* never auto-selectable, even on a single-candidate hit:
* - series-less parses (`31.pdf` can point at any series);
* - refs on the collision list (`A4.pdf` is far more often a scanner's
* paper size than verifikat A4, `K10.pdf` a blankett);
* - three-letter series (`IMG_0031.jpg`: real SIE series are 1-2 chars,
* three letters is a camera or scanner prefix).
* They all still parse and resolve; a human confirms with one click.
*/
autoSelectable: boolean
}
/**
* Optional noise ahead of the reference: a year folder prefix and the words
* some exporters prepend. Kept tight on purpose, `(?:19|20)\d{2}` rather than
* any 4 digits, so a voucher number is never eaten as a year.
*
* `ifikation` must precede `ifikat` in the alternation: regex alternation is
* first-match, so the short branch would otherwise consume `Verifikat` out of
* `Verifikation 31` and leave `ion` for the series group to swallow.
*/
const YEAR_NOISE = '(?:(?:19|20)\\d{2}[-_. ]+)?'
/**
* The lookahead is load-bearing, not decoration. Without it the engine
* backtracks into the shorter alternatives and `Verifikation 31` matches `ver`
* + `ifikat`, leaving `ion` for the series group to swallow as series `ION`.
* Requiring the word to end here means the prefix is either the whole word or
* not consumed at all.
*/
const VER_NOISE = '(?:ver(?:ifikation|ifikat)?(?![A-Za-zÅÄÖåäö])[-_. ]*)?'
/** `A31`, `A-31`, `A_31`, `A 31`, optionally followed by `_`/`-`/space + anything. */
const SERIES_NUMBER_RE = new RegExp(
`^${YEAR_NOISE}${VER_NOISE}([A-Za-zÅÄÖåäö]{1,3})[-_. ]?(\\d{1,7})(?:[-_. ].*)?$`,
'i',
)
/**
* `31`, `31_kvitto`, `Verifikat 31`. The `ver` prefix is allowed here but the
* year prefix is NOT: `2024 31` is far more likely a date fragment than
* voucher 31 of 2024, and this branch has no series to corroborate it with.
*/
const NUMBER_ONLY_RE = new RegExp(`^${VER_NOISE}(\\d{1,6})(?:[-_. ].*)?$`, 'i')
/**
* A date-named file, never a voucher number. Deliberately loose where the
* parser is strict: unpadded components (`2024-1-31`), two-digit years
* (`24-01-31`), any of `-_. /` as separator, and the compact `20240131`.
* A false positive here costs one manual assignment; a false negative attaches
* a receipt to a verifikat whose number happens to equal a year fragment.
*/
const DATE_PREFIX_RE = new RegExp(
'^(?:' +
// 20240131
'(?:19|20)\\d{6}' +
// 2024-01-31, 2024-1-31, 24-01-31, 2024 01 31, 2024/01/31
'|(?:19|20)?\\d{2}[-_. /]\\d{1,2}[-_. /]\\d{1,2}' +
// 31.01.2024, 31/1/2024, 12-24-2024: day-first and US order. Without this
// the day becomes a voucher number that always exists in the year.
'|\\d{1,2}[-_. /]\\d{1,2}[-_. /](?:19|20)\\d{2}' +
')(?!\\d)',
)
/**
* `ver` is a prefix word, never a series. Without this `ver 31.pdf` parses as
* series VER and comes back auto-selectable, while the spelled-out
* `Verifikat 31.pdf` correctly yields a series-less reference that requires
* confirmation. Same filename, two trust levels, decided by an abbreviation.
*/
const NOT_A_SERIES = new Set(['VER'])
/**
* Refs that are, in the wild, far more often document names than voucher
* references: A0-A6 are paper sizes (every scanner emits an `A4.pdf`),
* K2-K13 / N1-N9 / T1-T2 are Skatteverket blanketter, Q1-Q4 are quarters.
* Verifikat A4 genuinely exists in every migrated ledger, which is exactly
* why these must not be pre-selected: the review table cannot tell a scanned
* "A4.pdf" from the real receipt for voucher A4, and a wrong link is
* permanent. Demoted, not refused: a genuine A4 costs one click.
*
* The inconsistency this fixes: `31.pdf` already required confirmation while
* `A4 scan.pdf`, which carries LESS voucher evidence in a single-series
* company, was pre-ticked.
*/
const COLLISION_REFS = new Set([
'A0', 'A1', 'A2', 'A3', 'A4', 'A5', 'A6',
'K2', 'K3', 'K4', 'K5', 'K6', 'K7', 'K8', 'K9', 'K10', 'K11', 'K12', 'K13',
'N1', 'N2', 'N3', 'N4', 'N5', 'N6', 'N7', 'N8', 'N9',
'T1', 'T2',
'Q1', 'Q2', 'Q3', 'Q4',
])
/** Real SIE series are 1-2 characters; three letters is IMG/DSC/DOC/SCN. */
const MAX_AUTO_SERIES_LENGTH = 2
/**
* Any four-digit run that reads as a calendar year. Used to refuse a
* SERIES-LESS parse: `2024` alone is overwhelmingly a year, not verifikat 2024.
*/
const YEAR_LIKE_RE = /^(?:19|20)\d{2}$/
/**
* Trim only. Directory components are NOT stripped: `file.name` from an
* `<input type=file>` never carries a path, while the manual-reference box
* feeds arbitrary user text through this same parser, where splitting on `/`
* would quietly turn the typed date `2024/01/31` into voucher 31.
*/
function baseName(fileName: string): string {
return fileName.trim()
}
/** Drop the extension, but only a real-looking one (`.pdf`, `.jpeg`). */
function stripExtension(name: string): string {
return name.replace(/\.[A-Za-z0-9]{1,5}$/, '')
}
export function parseVoucherRefFromFileName(fileName: string): ParsedFileNameRef | null {
const stem = stripExtension(baseName(fileName))
if (!stem) return null
// A file named after its date is the single most common false positive: the
// digits parse cleanly and point at a verifikat number that has nothing to do
// with the receipt. Refuse the whole name rather than try to be clever.
if (DATE_PREFIX_RE.test(stem)) return null
const seriesMatch = SERIES_NUMBER_RE.exec(stem)
if (seriesMatch) {
const series = seriesMatch[1].toUpperCase()
const number = Number(seriesMatch[2])
if (!NOT_A_SERIES.has(series) && Number.isInteger(number) && number > 0) {
// The check runs on the NORMALIZED ref: a scanner's `A0004.pdf` parses
// to number 4 and must be caught by the same A4 entry.
const autoSelectable =
series.length <= MAX_AUTO_SERIES_LENGTH && !COLLISION_REFS.has(`${series}${number}`)
return { series, number, pattern: 'series_number', autoSelectable }
}
}
const numberMatch = NUMBER_ONLY_RE.exec(stem)
if (numberMatch && !YEAR_LIKE_RE.test(numberMatch[1])) {
const number = Number(numberMatch[1])
if (Number.isInteger(number) && number > 0) {
return { series: null, number, pattern: 'number_only', autoSelectable: false }
}
}
return null
}