c0ecf2fa3bebd46bdfd0169efd73b89653d1dfed
7
Commits
| Author | SHA1 | Message | Date | |
|---|---|---|---|---|
|
524d9978f1 |
fix(migration): resumable underlag import without inline extraction + same-origin MCP storage URLs (#1783)
* fix(migration): resumable underlag import without inline extraction, same-origin MCP storage URLs The Fortnox underlag import ran every file's AI extraction inline inside one request and hit the hosted 300 s function limit after ~17 of 113 files (twice on 2026-08-21); the UI showed the generic "underlagen kunde inte importeras" although the files it did reach were linked. The import now works in time-budgeted slices with a stable cursor (the UI loops until the server reports the end and shows "x av y") and opts out of extraction (extractionOwner 'none', stamped skipped:opted_out): every file is linked to its posted verifikat on arrival, so the booking is already known. MCP signed Storage URLs (upload_url, signed_url, download_url) are served through a same-origin proxy, /api/storage/[...path], because Claude Desktop's sandbox only reaches the MCP host and blocked the PUT to <project>.supabase.co. The signed token stays the only credential; the proxy forwards only signed documents-bucket paths to our own Storage host and is a no-op rewrite when NEXT_PUBLIC_APP_URL is unset. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_013YoZ8iboyTj221axW6Gdtm * fix(mcp): keep the storage-proxy note out of the size-capped tool descriptions The per-tool 280-char cap and the tools/list payload ceiling both tripped on the two sentences added to gnubok_create_document_upload and gnubok_get_document_content; the why now lives in a code comment. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_013YoZ8iboyTj221axW6Gdtm * fix(review): id cursor, stall = error, capped upload body, encoded dot segments Review follow-ups on #1783: - the import cursor is the last handled provider attachment id, not an index, so a file Fortnox adds or removes mid-sweep shifts nothing - a partial answer whose cursor does not advance (or the round guard) is reported as ARCIM_DOCUMENT_IMPORT_STALLED instead of "complete"; the slices already landed stay reported and the retry button resumes - the storage proxy reads the PUT body as a capped stream instead of buffering an unbounded payload before measuring it - object paths are rejected when any segment decodes to "." or ".." (or holds a separator), and the URL fetch() would actually request is re-checked against the allowlist after normalisation - download_url description no longer claims a direct Storage URL Co-Authored-By: Claude Fable 5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_013YoZ8iboyTj221axW6Gdtm --------- Co-authored-by: Jakob Wennberg <311770904+jakobwennberg-oss@users.noreply.github.com> Co-authored-by: Claude Fable 5 <noreply@anthropic.com> |
||
|
|
e733ab7c43 |
fix(arsredovisning): unblock the signing flow, accept foreign parent org nr, explain Fortnox underlag failures (#1738)
Batch from a real migration walkthrough (Fortnox -> Accounted, 2026-08-20): - Årsredovisning: the "Låst version" select was empty with no explanation because the only version was a draft and "Lås version för underskrift" is disabled while the four Lagstadgade upplysningar checkboxes and the content confirmation count as blockers. The select is now disabled with a hint that names the blocker count and links to Fullständighetskontroll, the four AR-NOTE-*-UNCONFIRMED issues carry remediation text, the lock button explains why it is grey, and "Markera som signerad" says what it still needs (locked version, bevisreferens, date). - Moderföretagets org.nr accepts a foreign registration identifier (CHE-123.456.789, HRB 12345, 923 609 016); personnummer shapes stay out. - Fortnox underlag discovery: log status, body and Fortnox's message on failure, show the message in the UI, treat a 400 with behörighet/scope text as scopes-required, and fall back to an unfiltered voucherfileconnections list when the financialyear filter answers 400. - Kontomapping: the Momskod column had min-w only; table-fixed collapsed it and its selects overflowed into Konfidens. Real w-72 now. - SIE import warnings pluralise correctly for one skipped voucher; the Verifikationsserie option says the source series is preserved. Co-authored-by: Jakob Wennberg <311770904+jakobwennberg-oss@users.noreply.github.com> Co-authored-by: Claude Fable 5 <noreply@anthropic.com> |
||
|
MattssonandClaude Opus 5
|
2deea05d42 |
feat(import): attach underlag to SIE-migrated verifikat by filename (#1627)
* refactor(documents): lift the SIE voucher-ref resolver into core
The provider migration sweep resolved a source voucher reference to the
verifikat it became with an in-memory (period, series, number) index built
inside extensions/general/arcim-migration. The underlag filename import needs
the identical resolution, and core must never import from @/extensions, so the
index, its ambiguity handling and the two paged reads move to
lib/documents/voucher-ref-resolver.ts.
Behaviour-preserving for the extension: same index construction, same "drop
both when one key repeats inside a fiscal year" rule, same dateTo-window
resolution. The arcim tests pass unchanged.
Two deliberate additions on top of the lift:
- series comparison is now case-insensitive on both sides. SIE writes series
uppercase in practice but the spec does not require it, and a filename is
whatever the exporting tool produced.
- byNumber and fetchVouchersForNumbers serve the filename flow, which
resolves a handful of refs per request and must not pull every migrated
entry into memory to do it.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
* feat(import): attach underlag to SIE-migrated verifikat by filename
A SIE file carries the ledger but not the underlag, so a migrating customer
brings the receipts over separately and today has to open every verifikat and
attach them by hand. Systems that export both name each receipt after its
verifikat (A31_<internal-id>.pdf), and the SIE import already preserves that
identity on every entry (source_voucher_series / source_voucher_number), so
the pairing is a lookup, not an interpretation: no AI, no amount matching, no
date windows.
Separate optional import mode (/import?mode=underlag), NOT a step inside the
SIE wizard: the receipts normally arrive later and from a different export, so
a migration must never be blocked on having them ready.
lib/documents/filename-voucher-ref.ts reads the ref out of a filename
lib/documents/underlag-import.ts builds the plan (reads only)
POST /api/import/documents/preview filenames in, match plan out
POST /api/import/documents/attach one file, archived and linked
components/import/UnderlagImportWizard review, adjust, run
Guards, because a document linked to a posted verifikat is
räkenskapsinformation and can never be re-pointed (BFL 7 kap):
- Matching keys on the SOURCE voucher number, never our own. The importer
renumbers per target series, so a file named after our number would land
on the wrong verifikat exactly when the import skipped a voucher.
- Nothing is uploaded until the whole plan has been shown: the preview
sends filenames only, the bytes stay in the browser.
- A ref that hits several migrated years is surfaced as a choice, never
resolved by guessing. So is a filename with a number but no series, which
is resolved but never pre-selected.
- A date-named file (20240131.pdf) is refused outright rather than read as
voucher 20240131.
- A target in a closed or locked period is shown but not selectable:
enforce_period_lock_documents would refuse the write anyway.
- The attach route re-resolves the filename server-side and 409s when it
does not name the target the client sent, so a stale plan cannot scatter
underlag permanently. An explicit manual assignment opts out of that check
and is flagged as such; company ownership of the entry is always verified.
- Idempotent per (verifikat, content): a re-run converges on the same
document row instead of archiving duplicates.
tests/pg/underlag-attach-period-lock.pg.test.ts pins the period-lock contract
the plan surface promises, including that the lock guards the LINK and still
lets an unlinked document be archived.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
* fix(import): scope underlag matching to a declared fiscal year
Adversarial review of #1627 refuted the resolver: it looked a ref up
company-wide and treated "exactly one candidate exists" as proof of identity.
Source systems restart voucher numbering every year and a filename carries no
year, so with a partial migration, or with that year's A31 among the vouchers
the importer routinely skips (empty, single-line, unbalanced), a 2023 receipt
was silently attached to a 2025 verifikat. Permanent under BFL 7 kap, and
invisible afterwards. Cardinality is not identity.
Every batch now declares its fiscal year and candidates outside it are dropped
before the index is built, so no downstream branch can see, count or propose
one. The attach route takes the year for its re-resolution from the TARGET
entry, never from the client, so the check cannot be widened by naming a
different year. Scoping cannot make the year inferable; it makes it asserted,
and the confirm dialog reads it back because it is the one input the files
cannot corroborate.
Four further defects from the same review:
- npm test went red: hoisting the column list into a VOUCHER_SELECT constant
hid it from the no-phantom-columns AST scan (ceiling 377 -> 379) and
dropped all eight journal_entries columns out of the guard on the one path
that writes irreversible links. Both selects are inline again, and split:
the provider sweep no longer fetches three display columns it never reads.
- The date guard only caught zero-padded hyphenated dates, so
`2024-1-31 kvitto.pdf`, `2024 01 31 ...`, `2024.1.31` and `24-01-31` all
parsed as voucher 2024 or 24. Widened to unpadded components, two-digit
years and space/slash separators; a bare year-shaped number is refused.
- `Verifikation 31.pdf` parsed as series ION: the alternation matched
`ifikat` and left `ion` for the series group. Reordering alone was not
enough (the engine backtracks into it), so the prefix now requires the
word to end.
- The manual-reference box was an unguarded write path: typing a date got
path-split down to a voucher number, marked the row selected, and posted
with override, which skips both server checks, while the row still showed
"Kan inte tolkas". Directory splitting is gone from the parser, the row
status is updated on resolve, and picking a server-proposed candidate no
longer counts as an override, which had disabled the filename check on
exactly the ambiguous rows it exists to protect.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
* fix(import): enforce the declared fiscal year on the server
The second adversarial pass refuted the previous fix. The attach route took
the year for its re-resolution from the TARGET entry, which is tautological:
an entry is by construction inside its own fiscal_period_id, so the filter
could never drop it and the year axis was unfalsifiable. Server-side year
enforcement was zero; the declared year existed only as React state and was
never sent. The regression test that "proved" otherwise passed only because
the mock let one journal_entries row report two different fiscal_period_id
values to two different reads, a state Postgres cannot produce. A test that
could not fail.
The attach request now carries the year the user actually reviewed, echoed
back from the plan, and the route asserts it equals the target's own period
BEFORE any other check and including overrides: an override is a statement
about which verifikat, never about which year. Its test asserts that directly
instead of a mock artifact.
Also from the same pass, a UI race that made the confirm dialog lie: FyPicker
stayed interactive while a preview of up to 2000 filenames was in flight, so
the summary and the confirm text could read back a year the plan was not built
from, and a manually resolved row could join the batch from another year
entirely. The wizard snapshots the plan's year, every downstream read uses the
snapshot, manual re-resolution goes through the server's own echoed
plan.fiscal_period_id, and the picker is frozen while a preview runs.
Parser, from the corpus pass (~360 realistic filenames plus 200k random uuids,
no ReDoS found: 2000 hostile inputs in 26ms):
- Day-first and US dates parsed as voucher numbers: `31.01.2024` became
voucher 31, a number that always exists in the year. The guard now covers
both orders.
- `ver 31.pdf` parsed as series VER and came back auto-selectable, while
every spelled-out `Verifikat 31.pdf` correctly yielded a series-less
reference needing confirmation. Same filename, two trust levels, decided
by an abbreviation. `ver` is no longer a series.
Known residual, stated rather than papered over: a scanner's `A4.pdf` or a
`K10.pdf` blankett in the receipts folder still matches verifikat A4 or K10
when that year has them. No parser can separate those from a genuine
reference; they appear in the review table with the target's date and
description.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
* fix(import): make the user actually declare the fiscal year
The third adversarial pass found that the central guarantee of the previous
two commits was fiction. FyPicker auto-selects the newest fiscal period when
nothing is stored, and the wizard passes a page-specific storage key, so that
branch fired on every first use. A user migrating 2023 receipts who never
opened the picker resolved them against the newest year; A31 exists in
essentially every year, so those rows came back `matched`, pre-selected, with
only the confirm dialog between them and permanent links. Every commit message
and code comment claiming "the year the user named" described behaviour the UI
did not have.
FyPicker gains an opt-in `requireExplicitChoice` prop, default off so no other
caller changes, and the wizard uses it. The picker starts empty and the batch
cannot proceed until someone picks. A previously stored explicit choice for
this surface is still restored, which is what makes a multi-batch migration
bearable.
Also: a company with zero fiscal periods hit a disabled picker and a disabled
button with no explanation. There is now a line saying why.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
* fix(import): close the restore-branch hole and demote collision-prone refs
Round four of adversarial review, two findings, both fixed.
1. `requireExplicitChoice` gated only the newest-period fallback, not the
localStorage restore branch above it, so the "user declares the year"
guarantee held only for a user's first-ever batch. From the second on, the
year was silently pre-filled from an earlier unrelated batch, and in a
multi-year migration last-used is the worst possible default: the user is
by definition moving to a different year each round. The prop now gates
FyPicker's ENTIRE auto-selection block with one outer condition (restore,
the ALL_YEARS-stored fallback, newest-period, preferLatestEnded), because a
per-branch gate already missed one branch once. It also suppresses the
localStorage write, which fired BEFORE onChange and so recorded picks the
wizard had rejected mid-preview. The wizard drops its storage prefix
entirely: within one sitting reset() carries the year in state, and
nothing survives the session.
2. The filename parser pre-ticked `A4 scan.pdf` and `K10.pdf` while requiring
a click for `31.pdf`, which carries MORE voucher evidence in a
single-series company. Two independent review passes flagged the same
inconsistency. Collision-famous refs (A0-A6 paper sizes, K2-K13/N1-N9/
T1-T2 blanketter, Q1-Q4 quarters) and three-letter series (IMG/DSC/DOC/
SCN are cameras; real SIE series are 1-2 chars) still parse and resolve
but are never auto-selected. Demoted, not refused: verifikat A4 genuinely
exists in every migrated ledger, and its real receipt costs one click.
Residual documented: an existing short series plus a small number in an
ad-hoc name (`B2 hyra.pdf`) is indistinguishable from a real ref by
filename alone.
Also: the attach route's multipart doc now names the required
fiscal_period_id field, and the stale reset() comment describes the actual
persistence model.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
* fix(import): honor override only for unresolvable filenames + review round
Resolution pass for the PR #1627 review reports (CodeRabbit, Swedish
accounting review, compliance swarm).
The one substantive finding (CodeRabbit, major): `override: true` skipped the
filename consistency check entirely, so a crafted client could attach a
cleanly-named file to any same-year verifikat. The resolver now runs on every
request; an override is honored only when the filename is unresolvable in the
declared year (no parse, or no candidate) or already resolves to the requested
target. The shipped UI only overrides unresolvable rows, so nothing
user-facing changes. planAcceptsTarget is renamed planPermitsAttach and
carries the semantics in one place, with tests for both directions.
The Swedish review finding (BFNAR 2013:2 systemdokumentation): the
planPermitsAttach JSDoc still described the superseded derive-the-year-from-
the-target design. It now states the actual control: the route asserts the
caller-declared year equals the target's own period before this function runs.
CodeRabbit minors and nitpicks:
- underlag_confirm_body / underlag_run / underlag_locked_warning use ICU
plural forms in both locales; "1 filer arkiveras" was wrong Swedish.
- The attach and preview route tests mock @/lib/supabase/server per the
repo test guideline.
- fetchVouchersForNumbers narrows to the declared fiscal year at the DB;
the in-memory filter in buildUnderlagPlan remains the enforced truth.
- buildVoucherIndex appends into existing arrays instead of copying per
row: the provider sweep indexes every migrated entry in the company and
per-row copies made that O(n^2).
- The pg test reuses its insertDocument helper instead of a duplicated
INSERT; runAttach clears isLoading in a finally.
Declined, with reasons in DECISIONS.md: message-regex classification of
validateDocumentFile failures (established sibling pattern; validator
contract change is out of scope).
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
* fix(import): attach only to posted or reversed verifikat
Second review cycle on PR #1627: the Swedish accounting review's re-run found
that nothing in the attach route verified the target entry's status. The SIE
import RPC posts every entry inside its own transaction, so a draft carrying a
source ref should be unobservable, but the link this route writes is
irreversible räkenskapsinformation, and an invariant enforced in another file
is not one this surface may lean on. Underlag references a verifikation
(BFL 5 kap 6-7 §), so the target must BE one.
Enforced twice: the route rejects non-posted targets with
UNDERLAG_ENTRY_NOT_POSTED (overrides included), and the resolver reads filter
to posted/reversed so a draft can never even become a candidate. Reversed
stays attachable: a storno'd original remains räkenskapsinformation and its
underlag belongs on it.
Also recorded as confirmed-intentional (review note, no code change): with
override and an unresolvable filename the endpoint links to any same-company,
same-declared-year, posted verifikat, migrated or not, which mirrors the
existing /api/documents/[id]/link capability. The period-lock error-string
regex note restates a disposition already recorded in DECISIONS.md.
The arcim test's Supabase double learns .in(), which the shared resolver read
now uses for the status filter.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
---------
Co-authored-by: Claude Opus 5 (1M context) <noreply@anthropic.com>
|
||
|
Mattsson
|
a97b0023d4 |
feat: import Fortnox voucher attachments (#1541)
* feat: import Fortnox voucher attachments * fix: show Fortnox document import follow-up * fix: harden optional Fortnox document import * test: pin optional Fortnox import flow * fix: use browser timer handle type * fix: avoid serializing OAuth resume state |
||
|
Jonas Hagberg
|
378611a2dc |
fix(arcim-migration): dedup underlag per verifikat and sniff file type from bytes (#1065)
First production sweep of /import-documents (921 Bokio receipts) surfaced two defects that together dropped 7 of 666 resolvable receipts: - The idempotency key was company-wide (company_id, sha256), but the same file content legitimately backs several verifikat (one arrende contract attached to each year's arrende voucher, one insurance letter on two vouchers). The second and later verifikat silently lost their underlag. The key is now (company_id, sha256, journal_entry_id). - Bokio's uploads list occasionally declares the wrong contentType (a JPEG stored as image/png); magic-byte validation then correctly rejects the mismatch, failing a perfectly good receipt. The importer now sniffs the real format from the bytes (detectFileMagic, now exported from the document service) and only falls back to the declared type when no signature is recognised. The synthesised filename extension follows the effective type. Signed-off-by: Jonas Hagberg <jonas@lindan.se> |
||
|
Jakob WennbergandClaude Sonnet 5
|
ec27228a8e |
style: remove em/en dashes repo-wide, add CLAUDE.md rule against them (#890)
Em dashes (—) and en dashes (–) had spread across comments, docs, tests, and a few UI strings, reading as AI-generated boilerplate rather than house style. Replaced each with punctuation matching its context: colon for explanatory clauses, comma for asides, plain hyphen for numeric/legal ranges (e.g. "21-23§"), "to"/"till" for date ranges, parentheses for paired-dash asides. messages/en.json and messages/sv.json were fixed by hand together to keep sv/en in sync. Left untouched where the dash is the functional subject rather than decorative punctuation: date-range-parser.ts's separator regex, charset-repair.ts's CP1252 byte-mapping table (and its test), the SIE encoding mojibake docs, generic-csv.ts's minus-sign normalizer, the agent system-prompt files that already instruct against em dashes, and a golden iXBRL test fixture compared byte-for-byte. Also fixes two bugs surfaced along the way: an off-by-one in ApiKeysPanel's scope-label split (a leftover from an earlier partial pass), and a charset-repair test that had lost the literal en-dash it exists to verify. Regenerated the agent atom seed migration (skills:generate) since 27 SKILL.md files changed. Added a CLAUDE.md rule against em/en dashes, with an explicit carve-out for the functional-dash cases above. Co-authored-by: Claude Sonnet 5 <noreply@anthropic.com> |
||
|
Jonas Hagberg
|
47f03f4238 |
feat(arcim): import Bokio underlag and link to verifikat (#786) (#813)
* feat(arcim): import Bokio underlag and link to verifikat Adds an optional, re-runnable step that pages the Bokio /uploads, resolves each receipt's target verifikat via the SIE-preserved voucher number, and archives it through the document service linked to the journal entry. Closes the gap where neither the SIE GL import nor the entity import carries the receipts/underlag attached to each verifikat. - lib/providers/bokio: getBytes() binary download + an attachments resource module (uploads list, GUID->voucher index, per-upload download); pageSize capped at 100, file type taken from the upload's contentType since the download is octet-stream - importProviderDocuments: bulk in-memory resolution keyed on (fiscal period, series, number) — scoped per fiscal year because Bokio restarts numbering at V1 each year; idempotent on (company_id, sha256) so re-runs don't duplicate the undeletable BFL-linked rows - POST /import-documents route, kept off the migration critical path because the Bokio document API is rate-limited (200 req/60s) - journal_entry_id link only for v1; reuses the document-service link path (same module as #804) rather than forking it Closes #786 Signed-off-by: Jonas Hagberg <jonas@lindan.se> * fix(arcim): stable pagination order + account for unresolvable receipts Addresses two findings from a Codex review pass on the import step: - Add .order('id') to the paged journal_entries / document_attachments / fiscal_periods reads. fetchAllRows pages with .range(), and PostgREST paging without a deterministic order can skip/repeat rows once a table exceeds one page (journal_entries crosses 1000 across several migrated years), which would defeat both voucher resolution and the sha256 dedup. - Keep every upload carrying a journalEntryId in scope instead of pre-filtering on a resolvable voucher ref, so a receipt whose Bokio entry number didn't parse (or resolves to no verifikat) is counted as unmatched rather than silently dropped from the best-effort report. Tests: add unresolvable-ref and zero-uploads cases; mock now supports .order(). Signed-off-by: Jonas Hagberg <jonas@lindan.se> --------- Signed-off-by: Jonas Hagberg <jonas@lindan.se> |