fix(migration): Visma pagination + chunk-insert resilience (the '300 misslyckades' case) (#1455)

* fix(providers): paginate Visma eAccounting with $page/$pagesize

eAccounting silently ignores OData $top/$skip, so every request returned
page 1 and getPaginated appended the first page TotalNumberOfPages times:
customers were imported in triplicate and invoice chunks hit unique
violations. Also stop on an empty page so a stale Meta can never loop or
duplicate.

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>

* fix(migration): survive bad rows in entity imports instead of failing whole chunks

One PostgREST insert per 500-row chunk is all-or-nothing, so a single
duplicate reported every row as failed ('300 misslyckades') with no cause
shown. Now: dedupe repeats within the fetched data (paging faults, source
duplicates), fall back to per-row inserts when a chunk is rejected, store
empty invoice numbers as NULL instead of colliding '', surface the first
DB error in the result UI, and mark all-failed steps with an error icon.
Sales invoices also carry remaining_amount so open invoices no longer
land as settled.

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>

* fix(migration): never per-row retry after a successful bulk insert with short read-back

A succeeded statement whose .select() returns fewer rows than sent means
the rows ARE in the table; retrying them one by one would duplicate every
unreturned row. Pair what came back and report the tail instead.

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>

* fix(migration): count stub-insert casualties as failed and sample enrichment errors

Review follow-ups: invoices dropped because their customer/supplier stub
insert errored are DB failures, not matching misses; classifying them as
noMatch rendered a green result row with the database error hidden.
Enrichment failures now also feed errorSample.

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>

---------

Co-authored-by: Jakob Wennberg <311770904+jakobwennberg-oss@users.noreply.github.com>
Co-authored-by: Claude Fable 5 <noreply@anthropic.com>
This commit is contained in:
Jakob Wennberg
2026-08-08 10:45:02 +02:00
committed by GitHub
co-authored by Claude Fable 5 Jakob Wennberg
parent cf373e9dd1
commit a49d75db77
9 changed files with 577 additions and 132 deletions
+11 -2
View File
@@ -88,8 +88,12 @@ export class VismaClient {
const page = options?.page ?? 1;
const params = new URLSearchParams();
params.set('$top', String(pageSize));
params.set('$skip', String((page - 1) * pageSize));
// eAccounting paginates with $page/$pagesize (default 50, max 1000).
// OData-style $top/$skip are silently IGNORED by the API: every request
// returns page 1, so a multi-page fetch yields the first page N times
// and never reaches the rest of the collection.
params.set('$page', String(page));
params.set('$pagesize', String(pageSize));
if (options?.modifiedSince && options?.modifiedField) {
params.set(
@@ -135,6 +139,11 @@ export class VismaClient {
modifiedField: options?.modifiedField,
});
// An empty page means the collection is exhausted regardless of what
// Meta claims; trusting a stale/buggy TotalNumberOfPages here would
// re-append duplicate rows or loop for nothing.
if (result.items.length === 0) break;
allItems.push(...result.items);
totalPages = result.totalPages;
page++;