fix(migration): complete the rows of migrated sales invoices the hydration budget did not reach (#2291)

* fix(migration): complete the rows of migrated sales invoices the hydration budget did not reach The migration maps sales invoices from the provider's list payload and hydrates the detail form (rows, net, VAT) inside a fixed 90 s budget, open invoices first. Fortnox, Briox and Björn Lundén ship no rows in a list response, so every invoice the budget did not reach was imported as a header with a total and no invoice_items, and nothing ever came back for it: the wizard never showed the hydration report, so the user found out on the invoice page. Measured on prod today: Profilio 384 of 384 (migrated before hydration existed), Loftux 311 of 672, Damac 182 of 542, Clearstoq 1 125 of 1 125. - lib/providers: hydrateSalesInvoices() hydrates a caller-chosen subset of an already-listed register, so a follow-up can spend its budget on the invoices still incomplete on our side instead of re-walking the register open-first and never reaching the rest. - arcim-migration: completeMigratedInvoiceLines() starts from OUR row-less non-draft invoices, joins them to the provider register on number + date (unique on both sides), hydrates only that subset and writes each invoice's rows once the detail total matches the stored total to the öre. The header VAT split is rewritten only when the stored one holds no evidence (null rate, or a non-zero rate label beside 0 kr VAT and subtotal = total). Never the total, status, payments or a journal entry. - Hourly cron (/api/extensions/arcim-migration/complete-invoice-lines/cron, vercel.json + Docker crontabs) drives the pass over consents accepted in the last 60 days, newest first, with a per-company share of the run. - The wizard's result screen now shows "x av y fakturor hämtade med rader" and that the rest are fetched in the background within the hour. Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01DG5aYcshzKJ1EA7PPhGtVf * fix(migration): write the header VAT fill as a literal, raise the schema-guard ceiling for the row inserts The phantom-column scanner resolves only object-literal payloads. The header update is now a literal (so its six columns are checked); the two invoice_items inserts are runtime row arrays from mapSalesInvoiceLine, the same shape the orchestrator already inserts, so the ceiling moves 399 to 401 with the reason recorded beside the earlier ones. Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01DG5aYcshzKJ1EA7PPhGtVf * fix(migration): gate the completion cron on token freshness, not consent age, and visit every usable consent Two review findings held. Prod holds 57 accepted consents from the last 60 days, so a fixed page of the newest 25 would leave older companies with row-less invoices waiting behind companies that are already done: the cap is gone (a company with nothing left costs one query and no provider call). And the consent's created_at said nothing about whether its credentials still work: Fortnox refresh tokens live 45 days and rotate on every refresh, so eligibility is now read off the token row (access token expired within the last 45 days, or no expiry at all), which also stops a dead consent from being retried every hour. Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01DG5aYcshzKJ1EA7PPhGtVf --------- Co-authored-by: Jakob Wennberg <311770904+jakobwennberg-oss@users.noreply.github.com> Co-authored-by: Claude Fable 5.1 <noreply@anthropic.com>
2026-09-05 10:20:10 +02:00
parent 0bd3c27fba
commit 8e1f9d5201
14 changed files with 1291 additions and 2 deletions
@@ -1586,4 +1586,5 @@ One line per decision: `[YYYY-MM-DD] <decision>: <why>`. Appended by agents and
 [2026-09-04] company_settings.pays_salaries (and a never-attested employer_registered) now follow the ledger: trigger salary_runs_booked_marks_employer (20260904191000) sets them when a salary run is booked, with a backfill for the 12 companies already booking payroll with the flag at its default. Support case 2026-09-04: the assistant told a payroll-running aktiebolag in every answer that it "betalar inte löner" because lib/agent/ask/snapshot.ts asserted the column default (NOT NULL DEFAULT false, only written by the Skatt settings form) as a fact. The snapshot and the composer's KÄNDA FAKTA now apply the employee-facts doctrine (positive evidence or an attested negative, never the default) and the snapshot names Inställningar > Skatt / > Bokföring so the model can point at the page. Chose a trigger over calling a helper from the booking code because there are three booking writers (lib/salary/book-run.ts, the v1 route's own copy, seeders) and 20260717151000 already treated in-app payroll as employer evidence once as a one-off backfill; the trigger makes that rule continuous. An explicit employer_registered = false stays the user's answer.
 [2026-09-04] Supplier credit notes rest at 'credited' from creation (one row builder in lib/supplier-invoices/credit-note.ts for the dashboard route, the MCP executor and the v1 API) and a CHECK (supplier_invoices_credit_note_not_payable, 20260904190000) keeps every writer out of the payable states; the worklist attest count also excludes is_credit_note. Support case 2026-09-04: a credit note was inserted at 'registered' (the attest entry state) while the detail page rightly had no attest button for it, so Att göra showed "1 leverantörsfaktura att attestera" that nobody could clear; 14 such rows on prod plus one MCP-approved credit note, all backfilled by the migration (one immutable reset-source row skipped, hence NOT VALID). Chose the DB CHECK over fixing only the three insert literals because the invariant is "a credit note is never a payable", not "this literal says registered"; chose 'credited' over 'approved'+approved_at (would surface Markera betald and the Att betala tab) and over 'paid' (the customer side forbids paid credit notes by CHECK); the provider importers already used 'credited'. Also: the GET route now hydrates credited_original with a second scoped query because PostgREST cannot pick a direction for a self-referencing embed hint and returned the one-to-many side (an empty array), rendered as "Krediterar: Ankomst #" with no number.
 [2026-09-04] Underlag attach on a folder-picked Fortnox export (Loftux, 50 of 50 files refused with UNDERLAG_REF_MISMATCH): the multipart filename is reduced to its basename at the route boundary (lib/documents/upload-file-name.ts), rather than teaching the voucher-ref parser to strip directories or adding a client-supplied file_name field. Chrome writes webkitRelativePath as the multipart filename for folder selections, so the attach check saw "2026/06/Leverantörsfakturor/A166_x.pdf" while the preview had resolved File.name "A166_x.pdf"; the two endpoints received the same file under two names and the guard compared them. Stripping inside the parser would turn a typed manual ref "2024/01/31" into voucher 31 (the manual box shares the parser), and a second client-supplied name is no more trustworthy than the first, so the boundary is the only level that fixes the class.
+[2026-09-04] Migrated sales invoices without rows (Profilio 384/384, Loftux 311/672, Damac 182/542, Clearstoq 1 125/1 125): completed by an hourly re-runnable pass (extensions/general/arcim-migration/lib/complete-invoice-lines.ts, cron /api/extensions/arcim-migration/complete-invoice-lines/cron) that starts from OUR row-less invoices, joins them to the provider register on number + date, hydrates only that subset and writes rows once the detail total matches the stored total to the öre; the header VAT split is rewritten only when the stored one holds no evidence (null rate, or 0 kr VAT beside subtotal = total). Why not a bigger in-run budget: the largest register (1 911 invoices at Fortnox's platform-wide 4 req/s) does not fit one 300 s function whatever the split, and a budget-bounded one-shot pass leaves whatever it misses missing forever, silently (the wizard never showed the hydration report; it does now). Why not re-running fetchSalesInvoicesHydrated: it sorts the whole register open-first every time, so a second run re-spends its budget on the same invoices and never reaches the rest. Why not reset + re-import or an arithmetic backfill: reset deletes rows that payments and vouchers already point at, and total/1,25 asserts a rate the source never stated (DECISIONS 2026-08-22). The pass reuses mapSalesInvoice, so a row it writes is indistinguishable from a fully hydrated import; it never touches totals, status, payments or any journal entry (momsdeklaration and every report read the ledger).
 [2026-09-04] Connector-hop failures (timeout, error envelope, wire-contract mismatch) are transient in every sync path: the row keeps its status and the user message says no renewal is needed, same as AspspUnavailableError (#2202), and the cron now treats AspspUnavailableError the same way instead of parking it in 'error'. Why: on 2026-09-04 the Connect service answered a shape the client rejects and the cron flipped four canary companies to 'error' with SYNC_FAILED_MESSAGE, so users re-authorized consents that were fine. The Zod issues are logged (field paths) because a bare 'unexpected shape' left the failure undiagnosable. Rejected: a new 'degraded' connection status (one more state every filter and the probe would have to learn; the health probe already catches a dead session on the same run) and removing the canary companies from the env (hides the contract bug instead of exposing its field paths).