fix(import): chunk SIE account creation and close the import row on every exit (#2451)
* fix(import): chunk SIE account creation and close the import row on every exit
A full-BAS Bokio SIE file creates 1 200+ chart_of_accounts rows in one
INSERT. PostgREST runs it under the authenticated role's 8 s
statement_timeout, and with four row-level triggers plus the RLS WITH
CHECK that single statement measured 6.5 s to 8.2 s on prod: it was
cancelled for one company and passed for the next (2026-09-09).
- syncMappedAccounts inserts in chunks of 100 rows (INSERT_CHUNK_SIZE),
so every statement stays an order of magnitude inside the limit. A
failed chunk leaves the earlier ones committed; the next attempt reads
the chart again and inserts only what is still missing.
- executeSIEImport closes its pending sie_imports row in a finally
block. Every early `return result` after createPendingImportRecord
(account sync failure, missing fiscal year, overlapping import,
vouchers outside the year) used to leave the row 'pending'. That row
holds the (company_id, file_hash) slot in the partial unique index,
so a retry inside the five-minute cleanup gate failed on the index
instead of on the real error.
- The slot-held message no longer names "gnubok", an "Ersätt import"
button the import history has never had, or Fortnox; it says the
same file is being imported or was interrupted moments ago and to
retry in a few minutes. The thrown-error prefix is Swedish
("Importen misslyckades:") and the two undo hints name
accounted_undo_sie_import.
Tests: chunk sizes and first-failing-chunk behaviour in
account-sync.test.ts; failed-row finalize on two early exits and the
new slot message in sie-import.account-names.test.ts.
Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01WBTRraVXkCnx93Pa9wxJ6G
* fix(import): write account chunks as ignore-duplicates upserts with an exact count
Review follow-up on #2451. A plain INSERT per chunk still had the old
race: a concurrent import (or the replace flow) creating one account
between our read and write raised a duplicate-key error that the code
swallowed as success, while PostgREST had rolled back the whole chunk,
so every other account in it was silently missing.
Each chunk is now an upsert with onConflict (company_id, account_number)
and ignoreDuplicates, selecting the landed rows: the race becomes a
skipped row, `created` counts exactly what was written (also across a
mid-loop failure, which the compliance review flagged), and the
"duplicate" string special-case is gone.
Tests: conflict-skipped row not counted; a race yields no error; a
failing later chunk reports the rows the earlier chunks committed.
Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01WBTRraVXkCnx93Pa9wxJ6G
---------
Co-authored-by: Jakob Wennberg <311770904+jakobwennberg-oss@users.noreply.github.com>
Co-authored-by: Claude Fable 5.1 <noreply@anthropic.com>
This commit is contained in:
Jakob Wennberg
@@ -1684,6 +1684,7 @@ One line per decision: `[YYYY-MM-DD] <decision>: <why>`. Appended by agents and
|
||||
[2026-09-08] Issue #2224 follow-up from the correctness skeptic: the quote decision (open/declined) is now locked in the database while a live kundorder exists (migration 20260908165100 extends invoices_quote_decision_guard), reversing the earlier call to leave it open; a declined offert behind a confirmed, invoiced order was a contradictory agreement trail and the dashboard hid the re-accept button, so the quote was stuck. The three source and decision guards run as SECURITY DEFINER: a SELECT FOR UPDATE under RLS admits only the caller's active company, so a multi-company member writing for another company through raw PostgREST got no row, no lock and no guard. Both landed as a second migration rather than an edit of 20260908165000, which was already applied to staging under that version.
|
||||
[2026-09-08] Draft invoice PDF marks a draft with one diagonal, faint word (UTKAST / DRAFT) across every page instead of a banner in the top margin (#2437): a banner reads as UI chrome on a document, a watermark reads as a stamp and leaves the preview pixel-identical to the final print. The long legal sentence (saknar löpnummer, ML 17 kap 24 §) is dropped on purpose: the word alone says the document is not a valid invoice, and the download dialog (#2399) already explains why before the file exists. Rotation and opacity sit on a padded wrapper View so the word turns about its own centre. Skeptic refutation accepted: the first cut (#6b7280 at 0.14, about 92% brightness) would drop out of a monochrome print or greyscale scan, and a numbered draft otherwise prints title, number and OCR like a real faktura; now #4b5563 at 0.3 (about 79% brightness), with a test pinning the composited grey between 70% and 85%. A 1-bit scan can still threshold the word away; a second explicit line on numbered drafts was left out because the request was the word alone, and that residual is Emil's call. Second refutation accepted: the overlay is emitted as the LAST child of the Page, because react-pdf paints in document order and `fixed` does not hoist, so an overlay emitted first was painted under the opaque payment and customer boxes and the word vanished on the page that carries totals and OCR; a test now inflates the PDF content streams and asserts the glyph run comes after the last rectangle fill on every page. BETALD and MAKULERAD banners are left as they are.
|
||||
[2026-09-08] Negative journal-line amounts: fixed the sign at three levels (producers flip the SIDE via lib/bookkeeping/line-side.ts, the engine refuses negative amounts before any write, and a NOT VALID CHECK on journal_entry_lines) instead of only patching the supplier-invoice generator or hiding negative items in the form. Why: the invariant lived nowhere (no Zod rule, no engine check, no constraint), so MCP, templates and any future producer could repeat it; negative items themselves are valid input (rabatt, öresavrundning), so rejecting them at input would break real invoices. reverseEntry now swaps on the net so legacy negative lines storno cleanly before the data repair runs.
|
||||
[2026-09-09] SIE import account creation (syncMappedAccounts) inserts in chunks of 100 rows instead of one statement per chart, and executeSIEImport closes its pending sie_imports row in a finally block on every exit: a full-BAS Bokio import (1 242 accounts) hit the authenticated role's 8 s statement_timeout on prod (8.2 s cancelled for one company, 6.5 s passed for the next), and the early `return result` after the failed account sync left the row 'pending', so the user's retry 40 s later was refused by the file-hash unique index with a message that named "gnubok" and an "Ersätt import" button that does not exist. Chunking was chosen over moving the insert into an RPC with SET statement_timeout (the import_sie_journal_entries precedent): the insert is idempotent per row, a partial success is safe to retry, and no migration or pg-real ratchet is needed; the per-row trigger cost (audit log, writer-role guard, RLS WITH CHECK) is untouched and stays a separate concern. The slot-held message is static (no lookup of the holder) so it stays deterministic in the queued-mock tests.
|
||||
[2026-09-09] Onboarding search-as-you-type (#2448) reads SCB's företagsregister per debounced keystroke and runs TIC once, on the pick, instead of TIC per keystroke: the 3000/month Lens budget is why #2421 fired on Enter only, and SCB is free; the pick still needs TIC because SCB knows no F-skatt, VAT or fiscal year. No rate limit on either route, founder's call (2026-09-09): the debounce, the 3-char minimum and the abort of superseded requests are the only throttles. Sole traders are included in the onboarding search (SCB legal form 10) but stay excluded from the parties picker; the row names the form, never the personnummer.
|
||||
[2026-09-09] Bank picker: an UNCHECKED account holds its bokföringskonto only as a soft claim (yields to a checked account that wants it), instead of either a hard claim (status quo: 400 "Flera bankkonton kan inte bokföras på samma konto", the support dead end where 1930 could never move from the wrong bank account to the right one because the picker hides the ledger dropdown for unchecked rows and disconnect + reconnect re-claims the same rows by IBAN) or a full release on every save (fewer states, but it demotes every unchecked row to a manual ghost holding its ledger and loses the "re-check lands back on the same account" prefill). Contested rows are demoted to manual in ONE update before the mirror, and upsertFromPsd2 then promotes the manual holder in place, which keeps row ids, transactions.cash_account_id links and the is_primary flag on the 1930 row; the same release pass also makes two checked accounts swapping ledgers work (previously both upserts tripped the unique constraint and were swallowed per-account). Another connection's UNCHECKED row yields the same way (cash_accounts.enabled = false), matching how session sharing already counts only enabled rows as claims; a synced-elsewhere row stays a 400. No new client/UI: the picker needed no change once the server stopped counting unchecked rows.
|
||||
|
||||
@@ -1692,3 +1693,4 @@ One line per decision: `[YYYY-MM-DD] <decision>: <why>`. Appended by agents and
|
||||
[2026-09-08] Zettle orders cron pages candidates (range) and caps entitled syncs at 50 instead of limit(50) before hasCapability: entitlement skips must not consume the batch or advance last_order_synced_at (purchase recovery cursor). Declined a separate cron_checked_at column for now; revisit if scanned non-entitled volume becomes a time-budget problem.
|
||||
[2026-09-09] PR #2416 (Zettle, community-authored) adopted on the contributor's branch instead of re-implemented: kept the Shopify-shaped feed-only design, added migration 20260909100400 for the sites that enumerate platforms/connection tables (platform CHECKs on webshop_orders and webshop_store_settings, writer-role gate trigger, migration-reset snapshot and lock via the 20260826150000 wrapper pattern rather than re-issuing the 400-line reset body), renamed the four PR migrations past prod's 20260908143051, froze the validated connect origin on zettle_connections.return_origin so white-label users return to their brand domain (Zettle has one registered callback URL), and derive per-rate VAT net from Zettle's own product rows (tax / rate drifted from what was charged) with a one-öre-per-row line-sum tolerance instead of 0.005 kr (silently dropped multi-row 12%/6% underlag). Per-purchase rows kept for v1; daily kassarapport aggregation and Finance API fees/payouts are the follow-up.
|
||||
[2026-09-09] Zettle v1 imports split-tender, gift-card (sale or tender) and tip-carrying purchases as is_paid = false rows titled 'bokför manuellt' and skips their refunds, instead of booking them through the one-account / revenue-per-rate model: the skeptic showed a card+cash split would put the whole gross on 1686, a card+invoice split would count as paid, a 0 % gift-card row would land on 3004 / ruta 42 (it is a 2421 liability), and tips would book as momsfri sale. Proper support needs per-payment amounts and a voucher liability on webshop_orders (follow-up). Sync runs claim the connection (sync_lock_until) before refreshing the rotating token: a concurrent cron + manual sync otherwise reuses a refresh token, Zettle answers 400, and the connection flips to revoked. The cron snapshots its candidate list before syncing because each sync moves the row to the tail of the last_order_synced_at ordering, so live offset paging re-fetched synced rows and skipped unseen ones.
|
||||
[2026-09-09] SIE account creation writes each chunk as an ignore-duplicates upsert (ON CONFLICT (company_id, account_number) DO NOTHING, returning the landed rows) instead of a plain INSERT whose duplicate-key error was swallowed: under PostgREST one request is one transaction, so a single concurrent duplicate rolled back the whole statement while the caller counted it as created and moved on with accounts missing. The conflict clause makes the race a skipped row, `created` counts exactly what landed, and the "duplicate" string special-case is gone. Raised by CodeRabbit and the compliance review on PR #2451 (count accuracy under BFNAR 2013:2 p. 9.16); the audit_log trigger stays the per-row record.
|
||||
|
||||
@@ -1,5 +1,5 @@
|
||||
import { describe, it, expect, vi } from 'vitest'
|
||||
import { syncMappedAccounts } from '../account-sync'
|
||||
import { syncMappedAccounts, INSERT_CHUNK_SIZE } from '../account-sync'
|
||||
import { getBASReference } from '@/lib/bookkeeping/bas-reference'
|
||||
import type { AccountMapping } from '../types'
|
||||
import type { SupabaseClient } from '@supabase/supabase-js'
|
||||
@@ -30,11 +30,21 @@ function mapping(
|
||||
function buildCapturingSupabase(opts?: {
|
||||
existingAccounts?: Array<{ account_number: string; account_name: string }>
|
||||
insertError?: { message: string } | null
|
||||
/** Zero-based index of the insert statement that should fail (default: all). */
|
||||
insertErrorOnBatch?: number
|
||||
/**
|
||||
* Account numbers that already exist at write time although the read pass
|
||||
* did not see them (a concurrent import): the upsert's ON CONFLICT DO
|
||||
* NOTHING skips them, and PostgREST does not return skipped rows.
|
||||
*/
|
||||
conflictAccounts?: string[]
|
||||
updateError?: { message: string } | null
|
||||
selectError?: { message: string } | null
|
||||
}) {
|
||||
const existing = opts?.existingAccounts ?? []
|
||||
const inserts: Array<Record<string, unknown>> = []
|
||||
/** Row count of every insert statement, in call order. */
|
||||
const insertBatches: number[] = []
|
||||
const updates: Array<{
|
||||
payload: Record<string, unknown>
|
||||
filters: Record<string, string>
|
||||
@@ -66,9 +76,35 @@ function buildCapturingSupabase(opts?: {
|
||||
}),
|
||||
}),
|
||||
}),
|
||||
insert: (rows: Array<Record<string, unknown>>) => {
|
||||
inserts.push(...rows)
|
||||
return Promise.resolve({ error: opts?.insertError ?? null })
|
||||
upsert: (
|
||||
rows: Array<Record<string, unknown>>,
|
||||
upsertOpts: { onConflict?: string; ignoreDuplicates?: boolean }
|
||||
) => {
|
||||
if (
|
||||
upsertOpts?.onConflict !== 'company_id,account_number' ||
|
||||
upsertOpts?.ignoreDuplicates !== true
|
||||
) {
|
||||
throw new Error(`Unexpected upsert options: ${JSON.stringify(upsertOpts)}`)
|
||||
}
|
||||
const batchIndex = insertBatches.length
|
||||
insertBatches.push(rows.length)
|
||||
const failsThisBatch =
|
||||
opts?.insertError != null &&
|
||||
(opts.insertErrorOnBatch === undefined || opts.insertErrorOnBatch === batchIndex)
|
||||
const conflicts = new Set(opts?.conflictAccounts ?? [])
|
||||
const landed = rows.filter((r) => !conflicts.has(String(r.account_number)))
|
||||
return {
|
||||
select: () => {
|
||||
if (failsThisBatch) {
|
||||
return Promise.resolve({ data: null, error: opts?.insertError ?? null })
|
||||
}
|
||||
inserts.push(...landed)
|
||||
return Promise.resolve({
|
||||
data: landed.map((r) => ({ account_number: r.account_number })),
|
||||
error: null,
|
||||
})
|
||||
},
|
||||
}
|
||||
},
|
||||
update: (payload: Record<string, unknown>) => {
|
||||
const filters: Record<string, string> = {}
|
||||
@@ -88,7 +124,7 @@ function buildCapturingSupabase(opts?: {
|
||||
}),
|
||||
}
|
||||
|
||||
return { supabase: supabase as unknown as SupabaseClient, inserts, updates }
|
||||
return { supabase: supabase as unknown as SupabaseClient, inserts, updates, insertBatches }
|
||||
}
|
||||
|
||||
function run(
|
||||
@@ -280,9 +316,13 @@ describe('syncMappedAccounts: create pass', () => {
|
||||
expect(inserts[0].account_name).toBe('Konto 1932')
|
||||
})
|
||||
|
||||
it('swallows duplicate-key insert errors (concurrent import race)', async () => {
|
||||
const { supabase } = buildCapturingSupabase({
|
||||
insertError: { message: 'duplicate key value violates unique constraint' },
|
||||
it('absorbs a concurrent import race through ON CONFLICT DO NOTHING', async () => {
|
||||
// A plain INSERT raised a duplicate-key error here, which the old code
|
||||
// swallowed while the whole statement had rolled back. The write is now an
|
||||
// ignore-duplicates upsert (the mock refuses any other shape), so the
|
||||
// race surfaces as a skipped row, never as an error.
|
||||
const { supabase, insertBatches } = buildCapturingSupabase({
|
||||
conflictAccounts: ['1930'],
|
||||
})
|
||||
|
||||
const result = await run(supabase, [
|
||||
@@ -290,6 +330,8 @@ describe('syncMappedAccounts: create pass', () => {
|
||||
])
|
||||
|
||||
expect(result.error).toBeNull()
|
||||
expect(result.created).toBe(0)
|
||||
expect(insertBatches).toEqual([1])
|
||||
})
|
||||
|
||||
it('returns a fatal error for non-duplicate insert failures', async () => {
|
||||
@@ -511,3 +553,73 @@ describe('syncMappedAccounts: rename pass', () => {
|
||||
expect(result.renamed).toBe(1)
|
||||
})
|
||||
})
|
||||
|
||||
// A full-BAS SIE import creates 1 200+ accounts. PostgREST runs each request
|
||||
// under the authenticated role's 8 s statement_timeout, and one 1 200-row
|
||||
// INSERT measured 6.5 s to 8.2 s on prod (2026-09-09: cancelled for one
|
||||
// company, passed for the next). The create pass must therefore never send
|
||||
// the whole chart as one statement.
|
||||
describe('syncMappedAccounts: chunked create pass', () => {
|
||||
function fullChart(count: number): AccountMapping[] {
|
||||
// 1000..1000+count: every number is a valid class-1 account, BAS or not.
|
||||
return Array.from({ length: count }, (_, i) => {
|
||||
const num = String(1000 + i)
|
||||
return mapping({ sourceAccount: num, targetAccount: num, sourceName: `Konto ${num}` })
|
||||
})
|
||||
}
|
||||
|
||||
it('inserts missing accounts in statements of at most INSERT_CHUNK_SIZE rows', async () => {
|
||||
const { supabase, inserts, insertBatches } = buildCapturingSupabase()
|
||||
const total = INSERT_CHUNK_SIZE * 2 + 42
|
||||
|
||||
const result = await run(supabase, fullChart(total))
|
||||
|
||||
expect(result.error).toBeNull()
|
||||
expect(result.created).toBe(total)
|
||||
expect(inserts).toHaveLength(total)
|
||||
expect(insertBatches).toEqual([INSERT_CHUNK_SIZE, INSERT_CHUNK_SIZE, 42])
|
||||
expect(new Set(inserts.map((r) => r.account_number)).size).toBe(total)
|
||||
})
|
||||
|
||||
it('keeps a small chart in a single statement', async () => {
|
||||
const { supabase, insertBatches } = buildCapturingSupabase()
|
||||
|
||||
await run(supabase, fullChart(3))
|
||||
|
||||
expect(insertBatches).toEqual([3])
|
||||
})
|
||||
|
||||
it('counts only the rows the conflict clause let through', async () => {
|
||||
// The read pass saw neither account, but 1930 was created by a concurrent
|
||||
// import before our write: ON CONFLICT DO NOTHING skips it and the count
|
||||
// must not claim it.
|
||||
const { supabase, inserts, insertBatches } = buildCapturingSupabase({
|
||||
conflictAccounts: ['1930'],
|
||||
})
|
||||
|
||||
const result = await run(supabase, [
|
||||
mapping({ sourceAccount: '1930', targetAccount: '1930', sourceName: 'Företagskonto' }),
|
||||
mapping({ sourceAccount: '6110', targetAccount: '6110', sourceName: 'Kontorsmateriel' }),
|
||||
])
|
||||
|
||||
expect(result.error).toBeNull()
|
||||
expect(insertBatches).toEqual([2])
|
||||
expect(result.created).toBe(1)
|
||||
expect(inserts.map((r) => r.account_number)).toEqual(['6110'])
|
||||
})
|
||||
|
||||
it('stops at the first failing statement and reports its error', async () => {
|
||||
const { supabase, insertBatches } = buildCapturingSupabase({
|
||||
insertError: { message: 'canceling statement due to statement timeout' },
|
||||
insertErrorOnBatch: 1,
|
||||
})
|
||||
|
||||
const result = await run(supabase, fullChart(INSERT_CHUNK_SIZE * 3))
|
||||
|
||||
expect(result.error).toBe('canceling statement due to statement timeout')
|
||||
// First chunk committed, second failed, third never sent: the count
|
||||
// reports what actually landed in chart_of_accounts.
|
||||
expect(result.created).toBe(INSERT_CHUNK_SIZE)
|
||||
expect(insertBatches).toEqual([INSERT_CHUNK_SIZE, INSERT_CHUNK_SIZE])
|
||||
})
|
||||
})
|
||||
|
||||
@@ -248,3 +248,104 @@ describe('executeSIEImport: account name sync wiring', () => {
|
||||
expect(result.accountsCreated).toBeUndefined()
|
||||
})
|
||||
})
|
||||
|
||||
// Regression for 2026-09-09: the account insert timed out, executeSIEImport
|
||||
// returned through the early `return result` after syncMappedAccounts, and
|
||||
// the sie_imports row it had just created stayed 'pending'. That row holds
|
||||
// the (company_id, file_hash) slot in the partial unique index, so the user's
|
||||
// retry 40 s later failed on the index with a message that named the retired
|
||||
// brand and a button that does not exist.
|
||||
describe('executeSIEImport: pending import record on early exit', () => {
|
||||
function importWith(mock: ReturnType<typeof createQueuedMockSupabase>) {
|
||||
return executeSIEImport(
|
||||
mock.supabase as unknown as SupabaseClient,
|
||||
'company-1',
|
||||
'user-1',
|
||||
makeParsedFile(),
|
||||
makeMappings(),
|
||||
{
|
||||
filename: 'bokio.se',
|
||||
fileContent: '#dummy',
|
||||
createFiscalPeriod: false,
|
||||
importOpeningBalances: false,
|
||||
importTransactions: true,
|
||||
}
|
||||
)
|
||||
}
|
||||
|
||||
it('closes the pending record as failed when the create pass aborts', async () => {
|
||||
mockSync.mockResolvedValue({
|
||||
created: 0,
|
||||
renamed: 0,
|
||||
renamedAccounts: [],
|
||||
renameFailed: 0,
|
||||
error: 'canceling statement due to statement timeout',
|
||||
})
|
||||
const mock = createQueuedMockSupabase()
|
||||
mock.enqueueMany([
|
||||
{ data: null }, // checkDuplicateImport: no prior import
|
||||
{ data: null }, // cleanupStaleImportRecords
|
||||
{ data: { id: 'imp-1' } }, // createPendingImportRecord insert
|
||||
{ data: null }, // finalizeImportRecord update
|
||||
])
|
||||
|
||||
const result = await importWith(mock)
|
||||
|
||||
expect(result.success).toBe(false)
|
||||
expect(result.importId).toBe('imp-1')
|
||||
const updates = mock.findCalls('sie_imports', 'update')
|
||||
expect(updates).toHaveLength(1)
|
||||
expect(updates[0][0]).toMatchObject({
|
||||
status: 'failed',
|
||||
imported_at: null,
|
||||
transactions_count: 0,
|
||||
error_message: expect.stringContaining('statement timeout'),
|
||||
})
|
||||
})
|
||||
|
||||
it('closes the pending record as failed when the file has no fiscal year', async () => {
|
||||
const mock = createQueuedMockSupabase()
|
||||
mock.enqueueMany([
|
||||
{ data: null },
|
||||
{ data: null },
|
||||
{ data: { id: 'imp-2' } },
|
||||
{ data: null },
|
||||
])
|
||||
|
||||
const result = await importWith(mock)
|
||||
|
||||
expect(result.errors).toContain('No fiscal year defined in the SIE file')
|
||||
const updates = mock.findCalls('sie_imports', 'update')
|
||||
expect(updates).toHaveLength(1)
|
||||
expect(updates[0][0]).toMatchObject({ status: 'failed' })
|
||||
})
|
||||
|
||||
it('explains a held file-hash slot in Swedish, without the retired brand or a missing button', async () => {
|
||||
const mock = createQueuedMockSupabase()
|
||||
mock.enqueueMany([
|
||||
{ data: null }, // checkDuplicateImport
|
||||
{ data: null }, // cleanupStaleImportRecords
|
||||
{
|
||||
data: null,
|
||||
error: {
|
||||
code: '23505',
|
||||
message:
|
||||
'duplicate key value violates unique constraint "sie_imports_company_id_file_hash_active_idx"',
|
||||
},
|
||||
},
|
||||
])
|
||||
|
||||
const result = await importWith(mock)
|
||||
|
||||
expect(result.success).toBe(false)
|
||||
expect(result.importId).toBeNull()
|
||||
expect(result.errors).toHaveLength(1)
|
||||
expect(result.errors[0]).toMatch(
|
||||
/^Importen misslyckades: Samma SIE-fil håller redan på att importeras/
|
||||
)
|
||||
expect(result.errors[0]).toMatch(/Vänta några minuter och försök igen/)
|
||||
expect(result.errors[0]).not.toMatch(/gnubok|Ersätt import|Fortnox/)
|
||||
// No row was created, so there is nothing to close.
|
||||
expect(mock.findCalls('sie_imports', 'update')).toHaveLength(0)
|
||||
})
|
||||
})
|
||||
|
||||
@@ -38,6 +38,20 @@ function emptyResult(): AccountSyncResult {
|
||||
return { created: 0, renamed: 0, renamedAccounts: [], renameFailed: 0, error: null }
|
||||
}
|
||||
|
||||
/**
|
||||
* Rows per INSERT statement in the create pass.
|
||||
*
|
||||
* PostgREST runs every request under the authenticated role's 8 s
|
||||
* statement_timeout, and each chart_of_accounts row fires four row-level
|
||||
* triggers (audit log, writer-role guard, updated_at, known VAT rate) plus the
|
||||
* RLS WITH CHECK. A full-BAS SIE import creates 1 200+ accounts in one go; on
|
||||
* prod (2026-09-09) that single statement took 6.5 s for one company and was
|
||||
* cancelled at 8.2 s for the next, so whether an import went through was a
|
||||
* coin flip. 100 rows keeps every statement an order of magnitude inside the
|
||||
* limit. Exported for the chunking test.
|
||||
*/
|
||||
export const INSERT_CHUNK_SIZE = 100
|
||||
|
||||
/**
|
||||
* Build a chart_of_accounts insert row with the richest metadata available:
|
||||
* BAS reference when the number is in BAS_REFERENCE (incl. description and
|
||||
@@ -226,15 +240,30 @@ export async function syncMappedAccounts(
|
||||
return buildInsertRow(num, name, basRef, companyId, userId, vatDefaults.get(num))
|
||||
})
|
||||
|
||||
const { error: insertError } = await supabase.from('chart_of_accounts').insert(inserts)
|
||||
// A duplicate means a concurrent import (or the replace flow) created the
|
||||
// account between our read and write: the account exists, which is all
|
||||
// this pass guarantees.
|
||||
if (insertError && !insertError.message.includes('duplicate')) {
|
||||
result.error = insertError.message
|
||||
return result
|
||||
// One statement per chunk (see INSERT_CHUNK_SIZE). A chunk that fails
|
||||
// leaves the earlier ones committed, which is safe: the next attempt reads
|
||||
// the chart again and only inserts what is still missing.
|
||||
//
|
||||
// ON CONFLICT (company_id, account_number) DO NOTHING: a concurrent import
|
||||
// (or the replace flow) can create an account between our read and this
|
||||
// write. With a plain INSERT that duplicate rolled back the whole
|
||||
// statement while the caller treated "duplicate" as success, silently
|
||||
// dropping every other account in it. Rows the conflict skipped are not
|
||||
// returned, so `created` counts exactly what landed, chunk by chunk (the
|
||||
// audit_log trigger on chart_of_accounts stays the per-row source of
|
||||
// truth for behandlingshistoriken).
|
||||
for (let i = 0; i < inserts.length; i += INSERT_CHUNK_SIZE) {
|
||||
const chunk = inserts.slice(i, i + INSERT_CHUNK_SIZE)
|
||||
const { data: insertedRows, error: insertError } = await supabase
|
||||
.from('chart_of_accounts')
|
||||
.upsert(chunk, { onConflict: 'company_id,account_number', ignoreDuplicates: true })
|
||||
.select('account_number')
|
||||
if (insertError) {
|
||||
result.error = insertError.message
|
||||
return result
|
||||
}
|
||||
result.created += insertedRows?.length ?? 0
|
||||
}
|
||||
result.created = missing.length
|
||||
}
|
||||
|
||||
const existingVatUpdates = [...vatDefaults]
|
||||
|
||||
@@ -1956,8 +1956,14 @@ async function createPendingImportRecord(
|
||||
pgMessage.includes('sie_imports_company_id_file_hash_active_idx')
|
||||
|
||||
if (hitsActiveIdx) {
|
||||
// A 'completed' row is caught by checkDuplicateImport before we get
|
||||
// here, so the slot holder is a 'pending' row younger than the
|
||||
// five-minute cleanup gate: the same file is being imported in another
|
||||
// tab, or an attempt died seconds ago without closing its row. Say so
|
||||
// and name the way out. The previous text pointed at an "Ersätt import"
|
||||
// button the import history has never had.
|
||||
throw new Error(
|
||||
'En tidigare SIE-import för samma fil finns redan i gnubok. Öppna importhistoriken och välj "Ersätt import" på den befintliga raden, eller använd Fortnox-synkningen för att hämta uppdaterad data automatiskt.'
|
||||
'Samma SIE-fil håller redan på att importeras, eller så avbröts en import av den för mindre än fem minuter sedan. Vänta några minuter och försök igen. Står filen som importerad i importhistoriken, ångra den importen där först.'
|
||||
)
|
||||
}
|
||||
|
||||
@@ -2234,6 +2240,11 @@ export async function executeSIEImport(
|
||||
// result.journalEntryIds because that also holds opening_balance entries.
|
||||
const importTypedEntryIds: string[] = []
|
||||
|
||||
// True once the sie_imports row created by createPendingImportRecord has
|
||||
// been finalized (completed or failed). The finally block below closes it
|
||||
// on every other exit.
|
||||
let importRecordClosed = false
|
||||
|
||||
const onExistingPeriod = options.onExistingPeriod ?? 'block'
|
||||
const updateAccountNames = options.updateAccountNames ?? true
|
||||
|
||||
@@ -2402,7 +2413,7 @@ export async function executeSIEImport(
|
||||
// (company_id, file_hash) slot until it is undone; agents reported
|
||||
// being stuck here without knowing undo-then-retry is the path.
|
||||
result.errors.push(
|
||||
`Den här filen har redan importerats ${duplicate.imported_at ? new Date(duplicate.imported_at).toLocaleDateString('sv-SE') : 'vid okänt datum'} (import ${duplicate.id}, ${duplicate.transactions_count} verifikat). Ångra den importen först (Ångra import i webbappen, eller gnubok_undo_sie_import via MCP) och importera sedan igen.`
|
||||
`Den här filen har redan importerats ${duplicate.imported_at ? new Date(duplicate.imported_at).toLocaleDateString('sv-SE') : 'vid okänt datum'} (import ${duplicate.id}, ${duplicate.transactions_count} verifikat). Ångra den importen först (Ångra import i webbappen, eller accounted_undo_sie_import via MCP) och importera sedan igen.`
|
||||
)
|
||||
return result
|
||||
}
|
||||
@@ -2488,7 +2499,7 @@ export async function executeSIEImport(
|
||||
)
|
||||
if (periodDuplicate) {
|
||||
result.errors.push(
|
||||
`En SIE-import för ett överlappande räkenskapsår (${periodDuplicate.fiscal_year_start} till ${periodDuplicate.fiscal_year_end}) finns redan (import ${periodDuplicate.id}, ${periodDuplicate.transactions_count} verifikat). Ångra den importen först (Ångra import i webbappen, eller gnubok_undo_sie_import via MCP) och importera sedan igen.`
|
||||
`En SIE-import för ett överlappande räkenskapsår (${periodDuplicate.fiscal_year_start} till ${periodDuplicate.fiscal_year_end}) finns redan (import ${periodDuplicate.id}, ${periodDuplicate.transactions_count} verifikat). Ångra den importen först (Ångra import i webbappen, eller accounted_undo_sie_import via MCP) och importera sedan igen.`
|
||||
)
|
||||
return result
|
||||
}
|
||||
@@ -3161,6 +3172,7 @@ export async function executeSIEImport(
|
||||
options.fileContent,
|
||||
documentation
|
||||
)
|
||||
importRecordClosed = true
|
||||
|
||||
// Populate counterparty templates from voucher patterns (non-blocking)
|
||||
if (result.success && parsed.vouchers.length > 0) {
|
||||
@@ -3218,11 +3230,20 @@ export async function executeSIEImport(
|
||||
|
||||
} catch (error) {
|
||||
result.errors.push(
|
||||
`Import failed: ${error instanceof Error ? error.message : 'Unknown error'}`
|
||||
`Importen misslyckades: ${error instanceof Error ? error.message : 'Unknown error'}`
|
||||
)
|
||||
|
||||
// Mark the pending import as failed if we created one
|
||||
if (result.importId) {
|
||||
} finally {
|
||||
// Close the pending sie_imports row on every exit that did not reach the
|
||||
// normal finalize: a thrown error, and every early `return result` after
|
||||
// createPendingImportRecord (account sync failure, missing fiscal year,
|
||||
// overlapping import, vouchers outside the year, ...). Those early
|
||||
// returns used to leave the row in 'pending'. A pending row holds the
|
||||
// (company_id, file_hash) slot in the partial unique index, so a retry
|
||||
// inside the five-minute cleanup gate failed on the index instead of on
|
||||
// the real error: on 2026-09-09 a user whose account insert timed out
|
||||
// retried 40 s later and was told the file "already existed".
|
||||
if (result.importId && !importRecordClosed) {
|
||||
importRecordClosed = true
|
||||
try {
|
||||
await finalizeImportRecord(
|
||||
supabase,
|
||||
|
||||
Reference in New Issue
Block a user