fix(import): let provider re-sync re-import an earlier fiscal year after data deletion (#1763)

* fix(import): let provider re-sync re-import an earlier fiscal year after data deletion

After partially deleting imported data, a provider re-sync could not bring
back the previous fiscal year: the sie_imports 'completed' watermark
survives data deletion, the replace path aborted the whole year when the
prior import row could not be resolved, and prior-import detection picked
an arbitrary row when several overlapped the same year.

- findOverlappingPeriodImports returns ALL overlapping completed rows,
  newest first; checkDuplicatePeriodImport now picks deterministically.
- executeSIEImport replace mode resolves every overlapping row. A row that
  is gone or no longer 'completed' (replaceSIEImport codes not_found /
  not_completed) is a stale watermark: skip it with a warning and import
  the year fresh instead of stranding the user. Locked/closed periods and
  RPC failures still abort the year.
- The arcim-migration wizard names the fiscal year in every per-file
  import failure and shows the newest prior import in the options step.

Fixes #1667

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Signed-off-by: Emil <emilmattsson14@gmail.com>

* fix(import): fail closed when the replace pre-check query errors

replaceSIEImport's pre-check discarded the .single() error, so a
transient query failure (statement timeout, network error, 5xx via
PostgREST) was indistinguishable from a genuinely absent row and got
classified not_found. The replace loop in executeSIEImport then treated
it as a stale watermark and imported the fiscal year fresh while the
prior completed import's verifikationer were still in the ledger, with
duplicate checks skipped in replace mode: silent duplicate
verifikationer for a whole year (BFL 4:1 risk).

Only PGRST116 (zero rows from .single()) now classifies as not_found;
any other pre-check error returns rpc_error, which aborts the year in
the replace loop. Tests cover both classifications plus the
executeSIEImport-level abort.

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Signed-off-by: Emil <emilmattsson14@gmail.com>

* fix(import): fail closed on overlap lookup, verify zero surviving entries before a stale-watermark skip

Review findings on #1763 (CodeRabbit + Swedish compliance review):
- findOverlappingPeriodImports now uses fetchAllRows: query errors throw
  instead of returning [] (which let replace mode import fresh over rows it
  never resolved), pagination passes the PostgREST row cap, id tiebreak
  keeps the order total.
- A stale-watermark skip (not_found/not_completed) is only trusted after a
  positive check that zero posted import entries survive in the fiscal
  year: replace_sie_import deletes by fiscal period, so entries can outlive
  their sie_imports row. Survivors or a failed check abort the year.
- Contract comment tying the stale-race regex to the RPC's RAISE wording.
- Suite-level beforeEach clears mocks and the event bus (repo convention).

Signed-off-by: Emil <emilmattsson14@gmail.com>

---------

Signed-off-by: Emil <emilmattsson14@gmail.com>
Co-authored-by: Claude Fable 5 <noreply@anthropic.com>
This commit is contained in:
Mattsson
2026-08-20 21:11:15 +02:00
committed by GitHub
parent 3f6f1ab06e
commit febb4cc0c2
5 changed files with 689 additions and 47 deletions
+1
View File
@@ -1127,3 +1127,4 @@ One line per decision: `[YYYY-MM-DD] <decision>: <why>`. Appended by agents and
[2026-08-20] AI_API_KEY made optional for the OpenAI-compatible backend: a base URL alone now counts as configured (hasAiCredentials / resolveAiProvider), so a local model server (llama.cpp/Ollama/LM Studio/vLLM), which usually has no auth, works with just AI_BASE_URL + AI_MODEL. The openai-compatible service only sends an Authorization: Bearer when AI_API_KEY is set, so a keyless local server is never handed an empty bearer. Hosted providers that require a key still set AI_API_KEY. Bedrock/Anthropic credential logic unchanged.
[2026-08-20] poppler-utils is the one system package added to the self-host runner image (Sovereign plan WS1 PR2): pdftoppm renders the first pages of a PDF for AI backends without native PDF input (an OpenAI-compatible Swedish endpoint), measured at ~4 MB plus shared libs on node:22-alpine (pdftoppm 25.12), written to /tmp which docker-compose.yml already mounts as tmpfs under the read-only root. Hosted never calls it (Bedrock reads PDFs natively) and the cron image is untouched. pdfjs-dist + @napi-rs/canvas were rejected earlier (two npm deps, memory spikes, dead weight on hosted). scripts/smoke-ai-provider.ts is the backend-agnostic "is AI wired up" check; verified live against hosted Bedrock and against a local OpenAI-compatible mock (the mock received Bearer auth, per-tier model ids and one image_url part per rasterized page).
[2026-08-20] RIP-3 chat cutover is scoped to general.help only: the free-form Q&A /chat panel now runs on a page-scoped single-call console (AskConsole → POST /api/agent/ask, persist:true), so it works on any configured backend incl. a local OpenAI-compatible model. The tool-loop intents (transaction.categorization, invoice.draft, supplier_invoice.review) and the docked AgentSheet still use AgentChat + run-turn.ts because they stage operations and need the tool loop, so run-turn.ts is NOT deleted here (the plan gates its deletion on "once nothing calls them"; RIP-4 migrates the rest). Persistence is an opt-in branch on the existing /api/agent/ask route rather than a new endpoint, so page-scoped one-off asks (a report page) stay stateless; the console writes both turns to agent_conversations/agent_messages as canonical Anthropic text blocks so the /chat sidebar and resume keep working across old streaming threads and new single-call ones.
[2026-08-19] Provider re-sync replace mode resolves EVERY overlapping completed sie_imports row and treats one it cannot resolve (not_found/not_completed) as a stale watermark to skip, importing the year fresh, but still aborts on a locked or closed period: an unresolvable row has nothing left to delete, while importing over entries that could not be deleted would duplicate verifikationer.
@@ -2711,6 +2711,13 @@ export default function ArcimMigrationWorkspace({
setMigrationProgress(progress)
setMigrationStep(`Importerar bokföringsdata (SIE): fil ${i + 1} av ${filesToImport.length}...`)
// Name the year in every per-file failure: a multi-year re-sync
// that dies on ONE year must say which, or the user cannot act on
// it (issue #1667: the current year re-imported, the prior year
// refused, and the error never said so).
const fiscalYear = filesToImport[i].status?.fiscalYear
const yearLabel = fiscalYear ? `Räkenskapsår ${fiscalYear}: ` : ''
const res = await fetch('/api/extensions/ext/arcim-migration/import-sie', {
method: 'POST',
headers: { 'Content-Type': 'application/json' },
@@ -2728,7 +2735,10 @@ export default function ArcimMigrationWorkspace({
if (!res.ok) {
const data = await res.json().catch(() => ({}))
throw apiError(data, `SIE import HTTP ${res.status}`)
const err = apiError(data, `SIE import HTTP ${res.status}`)
throw err instanceof UserFacingError
? new UserFacingError(`${yearLabel}${err.message}`)
: err
}
const result = await res.json() as ImportResult
@@ -2740,8 +2750,8 @@ export default function ArcimMigrationWorkspace({
// först" message masks the real error.
if (!result.success) {
throw new UserFacingError(result.errors.length > 0
? result.errors.join('\n')
: 'SIE-importen misslyckades utan felmeddelande.')
? `${yearLabel}${result.errors.join('\n')}`
: `${yearLabel}SIE-importen misslyckades utan felmeddelande.`)
}
}
}
+12 -11
View File
@@ -31,7 +31,7 @@ import { parseSIEFile, validateSIEFile } from '@/lib/import/sie-parser'
import { mergeParsedSIEFiles } from '@/lib/import/sie-merge'
import { scanSieForCp1252Artifacts, formatSieArtifactWarning } from '@/lib/import/sie-artifact-scan'
import { suggestMappings, getMappingStats, isSystemAccount } from '@/lib/import/account-mapper'
import { loadMappings, generateImportPreview, executeSIEImport } from '@/lib/import/sie-import'
import { loadMappings, generateImportPreview, executeSIEImport, findOverlappingPeriodImports } from '@/lib/import/sie-import'
import { BAS_REFERENCE } from '@/lib/bookkeeping/bas-reference'
import type { ProviderName } from '@/lib/providers/types'
import { errorResponseFromCode } from '@/lib/errors/get-structured-error'
@@ -921,16 +921,17 @@ export const arcimMigrationExtension: Extension = {
} | null = null
if (fyStart && fyEnd) {
const { data } = await supabase
.from('sie_imports')
.select('imported_at, fiscal_year_start, fiscal_year_end')
.eq('company_id', companyId)
.eq('status', 'completed')
.lte('fiscal_year_start', fyEnd)
.gte('fiscal_year_end', fyStart)
.limit(1)
.maybeSingle()
priorImport = data
// Newest first: several completed rows can overlap the same
// year (manual upload + provider sync, or residue from
// partially deleted data). The unordered .limit(1) this
// replaces picked an arbitrary row, so the wizard could show
// a stale "ersätter tidigare import från <date>" (issue #1667).
// Import-time replace resolves ALL rows regardless of which
// one is displayed here.
const overlapping = await findOverlappingPeriodImports(
supabase, companyId, fyStart, fyEnd
)
priorImport = overlapping[0] ?? null
}
fileStatuses.push({
@@ -0,0 +1,486 @@
/**
* Regression suite for issue #1667 (Boltonshield AB support case, 2026-08-17).
*
* The bug: after partially deleting imported data, a provider re-sync could
* not re-import an earlier fiscal year. Three defects compounded:
*
* 1. The sie_imports 'completed' watermark survives data deletion, and the
* replace path aborted the WHOLE year when replaceSIEImport could not
* resolve the prior row (its data already gone).
* 2. Prior-import detection used `.limit(1).maybeSingle()` with no
* ordering: with multiple overlapping completed rows for the same year
* it picked an arbitrary one and left the rest standing.
* 3. The wizard error did not say which fiscal year failed (covered in the
* component, not testable here).
*
* The fix: findOverlappingPeriodImports returns ALL overlapping completed
* rows newest-first (failing closed on query errors); replace mode resolves
* every row and treats an unresolvable row (not_found / not_completed) as a
* stale watermark: warn and continue as a fresh import — but only after
* positively confirming no posted import entries survive in the year, since
* replace_sie_import deletes by fiscal period and entries can outlive their
* sie_imports row. Genuine refusals (locked period, RPC errors) still abort.
*/
import { describe, it, expect, vi, beforeEach } from 'vitest'
import {
executeSIEImport,
replaceSIEImport,
checkDuplicatePeriodImport,
findOverlappingPeriodImports,
} from '../sie-import'
import { createQueuedMockSupabase } from '@/tests/helpers'
import { eventBus } from '@/lib/events'
import type { ParsedSIEFile, AccountMapping } from '../types'
import type { SupabaseClient } from '@supabase/supabase-js'
beforeEach(() => {
vi.clearAllMocks()
eventBus.clear()
})
function makeParsedFile(overrides?: Partial<ParsedSIEFile>): ParsedSIEFile {
return {
header: {
sieType: 4,
flagga: 0,
program: 'TestProg',
programVersion: '1.0',
generatedDate: '2026-08-17',
format: 'PC8',
companyName: 'Boltonshield Mock AB',
orgNumber: '5567201701',
address: null,
fiscalYears: [{ yearIndex: 0, start: '2025-01-01', end: '2025-12-31' }],
currency: 'SEK',
kontoPlanType: null,
},
accounts: [
{ number: '1930', name: 'Företagskonto' },
{ number: '6110', name: 'Kontorsmaterial' },
],
openingBalances: [],
closingBalances: [],
resultBalances: [],
dimensions: [],
dimensionValues: [],
vouchers: [
{
series: 'A',
number: 1,
date: new Date(2025, 0, 15),
description: 'Inköp',
lines: [
{ account: '6110', amount: 1000 },
{ account: '1930', amount: -1000 },
],
},
],
issues: [],
stats: {
totalAccounts: 2,
totalVouchers: 1,
totalTransactionLines: 2,
fiscalYearStart: '2025-01-01',
fiscalYearEnd: '2025-12-31',
},
...overrides,
}
}
function makeMapping(source: string, target: string): AccountMapping {
return {
sourceAccount: source,
sourceName: `Account ${source}`,
targetAccount: target,
targetName: `Target ${target}`,
confidence: 1,
matchType: 'exact',
isOverride: false,
}
}
const mappings = [makeMapping('1930', '1930'), makeMapping('6110', '6110')]
const importOptions = {
filename: 'migration-sie-2025.se',
fileContent: '#dummy',
createFiscalPeriod: false,
importOpeningBalances: false,
importTransactions: true,
onExistingPeriod: 'replace' as const,
}
function makePriorImportRow(overrides: Record<string, unknown> = {}) {
return {
id: 'imp-prior',
user_id: 'user-1',
filename: 'old.se',
file_hash: 'hash-old',
fiscal_year_start: '2025-01-01',
fiscal_year_end: '2025-12-31',
transactions_count: 10,
status: 'completed',
fiscal_period_id: 'fp-2025',
opening_balance_entry_id: null,
imported_at: '2026-06-01T00:00:00Z',
created_at: '2026-06-01T00:00:00Z',
...overrides,
}
}
describe('findOverlappingPeriodImports', () => {
it('returns every overlapping completed row, and [] when there are none', async () => {
const { supabase, enqueue } = createQueuedMockSupabase()
const rows = [makePriorImportRow({ id: 'a' }), makePriorImportRow({ id: 'b' })]
enqueue({ data: rows })
const result = await findOverlappingPeriodImports(
supabase as unknown as SupabaseClient, 'company-1', '2025-01-01', '2025-12-31'
)
expect(result.map((r) => r.id)).toEqual(['a', 'b'])
const empty = await findOverlappingPeriodImports(
supabase as unknown as SupabaseClient, 'company-1', '2025-01-01', '2025-12-31'
)
expect(empty).toEqual([])
})
it('orders newest first so the pick is deterministic', async () => {
const { supabase, enqueue, findCalls } = createQueuedMockSupabase()
enqueue({ data: [] })
await findOverlappingPeriodImports(
supabase as unknown as SupabaseClient, 'company-1', '2025-01-01', '2025-12-31'
)
const orderCalls = findCalls('sie_imports', 'order')
expect(orderCalls).toEqual([
['imported_at', { ascending: false, nullsFirst: false }],
['created_at', { ascending: false }],
['id', { ascending: false }],
])
})
it('throws on a query error instead of returning [] (fail closed)', async () => {
const { supabase, enqueue } = createQueuedMockSupabase()
enqueue({ data: null, error: { message: 'connection reset' } })
await expect(
findOverlappingPeriodImports(
supabase as unknown as SupabaseClient, 'company-1', '2025-01-01', '2025-12-31'
)
).rejects.toThrow(/connection reset/)
})
})
describe('checkDuplicatePeriodImport', () => {
it('returns the newest row when several overlap the period', async () => {
const { supabase, enqueue } = createQueuedMockSupabase()
enqueue({
data: [
makePriorImportRow({ id: 'newest', imported_at: '2026-08-01T00:00:00Z' }),
makePriorImportRow({ id: 'older', imported_at: '2026-05-01T00:00:00Z' }),
],
})
const result = await checkDuplicatePeriodImport(
supabase as unknown as SupabaseClient, 'company-1', '2025-01-01', '2025-12-31'
)
expect(result?.id).toBe('newest')
})
it('returns null when nothing overlaps', async () => {
const { supabase, enqueue } = createQueuedMockSupabase()
enqueue({ data: null })
const result = await checkDuplicatePeriodImport(
supabase as unknown as SupabaseClient, 'company-1', '2025-01-01', '2025-12-31'
)
expect(result).toBeNull()
})
})
describe('replaceSIEImport: failure classification', () => {
it('classifies a vanished sie_imports row as not_found', async () => {
const { supabase, enqueue } = createQueuedMockSupabase()
enqueue({ data: null }) // sie_imports fetch: row gone
const result = await replaceSIEImport(
supabase as unknown as SupabaseClient, 'company-1', 'imp-x', 'user-1'
)
expect(result.success).toBe(false)
expect(result.code).toBe('not_found')
})
it('classifies PGRST116 (zero rows) on the pre-check as not_found', async () => {
const { supabase, enqueue } = createQueuedMockSupabase()
enqueue({
data: null,
error: { code: 'PGRST116', message: 'JSON object requested, multiple (or no) rows returned' },
}) // sie_imports fetch: .single() zero-rows error
const result = await replaceSIEImport(
supabase as unknown as SupabaseClient, 'company-1', 'imp-x', 'user-1'
)
expect(result.success).toBe(false)
expect(result.code).toBe('not_found')
})
it('classifies a non-PGRST116 pre-check query failure as rpc_error, never not_found (fail closed)', async () => {
const { supabase, enqueue } = createQueuedMockSupabase()
enqueue({
data: null,
error: { code: '57014', message: 'canceling statement due to statement timeout' },
}) // sie_imports fetch: transient query failure, row state UNKNOWN
const result = await replaceSIEImport(
supabase as unknown as SupabaseClient, 'company-1', 'imp-x', 'user-1'
)
expect(result.success).toBe(false)
expect(result.code).toBe('rpc_error')
expect(result.error).toMatch(/statement timeout/i)
// Failed before the delete RPC: nothing was touched.
const rpcCalls = (supabase.rpc.mock.calls as [string][])
.filter(([fn]) => fn === 'replace_sie_import')
expect(rpcCalls).toHaveLength(0)
})
it('classifies a row that already left completed as not_completed', async () => {
const { supabase, enqueue } = createQueuedMockSupabase()
enqueue({ data: { status: 'replaced', fiscal_period_id: null } })
const result = await replaceSIEImport(
supabase as unknown as SupabaseClient, 'company-1', 'imp-x', 'user-1'
)
expect(result.success).toBe(false)
expect(result.code).toBe('not_completed')
})
it('classifies a locked fiscal period as period_locked', async () => {
const { supabase, enqueueMany } = createQueuedMockSupabase()
enqueueMany([
{ data: { status: 'completed', fiscal_period_id: 'fp-1' } },
{ data: { is_closed: false, locked_at: '2026-01-31T00:00:00Z' } },
])
const result = await replaceSIEImport(
supabase as unknown as SupabaseClient, 'company-1', 'imp-x', 'user-1'
)
expect(result.success).toBe(false)
expect(result.code).toBe('period_locked')
})
it('classifies the RPC status-race raise as not_completed and other RPC errors as rpc_error', async () => {
const { supabase, enqueueMany } = createQueuedMockSupabase()
enqueueMany([
{ data: { status: 'completed', fiscal_period_id: null } },
{ data: null, error: { message: 'Import imp-x not found or not in completed status' } },
])
const raced = await replaceSIEImport(
supabase as unknown as SupabaseClient, 'company-1', 'imp-x', 'user-1'
)
expect(raced.success).toBe(false)
expect(raced.code).toBe('not_completed')
enqueueMany([
{ data: { status: 'completed', fiscal_period_id: null } },
{ data: null, error: { message: 'canceling statement due to statement timeout' } },
])
const failed = await replaceSIEImport(
supabase as unknown as SupabaseClient, 'company-1', 'imp-x', 'user-1'
)
expect(failed.success).toBe(false)
expect(failed.code).toBe('rpc_error')
})
})
describe('executeSIEImport replace mode: re-sync after deletion (issue #1667)', () => {
it('replaces EVERY overlapping completed import, not just the newest', async () => {
const { supabase, enqueueMany } = createQueuedMockSupabase()
const rowNew = makePriorImportRow({
id: 'imp-new',
opening_balance_entry_id: 'ob-1',
imported_at: '2026-08-01T00:00:00Z',
})
const rowOld = makePriorImportRow({
id: 'imp-old',
imported_at: '2026-05-01T00:00:00Z',
})
enqueueMany([
{ data: [rowNew, rowOld] }, // findOverlappingPeriodImports
{ data: { status: 'completed', fiscal_period_id: 'fp-2025' } }, // replace imp-new: fetch
{ data: { is_closed: false, locked_at: null } }, // replace imp-new: period check
{ data: 5 }, // rpc replace_sie_import → 5 deleted
{ data: null }, // OB safety-net update (imp-new had an OB entry)
{ data: { status: 'completed', fiscal_period_id: 'fp-2025' } }, // replace imp-old: fetch
{ data: { is_closed: false, locked_at: null } }, // replace imp-old: period check
{ data: 3 }, // rpc → 3 deleted
{ data: null }, // cleanupStaleImportRecords delete
{ data: { id: 'imp-created' } }, // pending sie_imports insert
{ data: [] }, // syncMappedAccounts chart fetch
{ data: null }, // chart insert (missing accounts)
{ data: null }, // find existing fiscal period → clean stop
])
const result = await executeSIEImport(
supabase as unknown as SupabaseClient,
'company-1',
'user-1',
makeParsedFile(),
mappings,
importOptions,
)
// Both prior rows were pushed through the replace RPC.
const rpcCalls = (supabase.rpc.mock.calls as [string, Record<string, unknown>][])
.filter(([fn]) => fn === 'replace_sie_import')
expect(rpcCalls.map(([, params]) => params.p_import_id)).toEqual(['imp-new', 'imp-old'])
// The reported shape stays a single object: newest id, total deleted.
expect(result.replacedPriorImport).toEqual({ importId: 'imp-new', deletedEntries: 8 })
// The run then proceeds past the replace block (stops at the benign
// fiscal-period stop this queue sets up, NOT a replace failure).
expect(result.errors.join(' ')).toMatch(/No matching fiscal period found/i)
expect(result.errors.join(' ')).not.toMatch(/ersätta/i)
})
it('treats a stale watermark (prior row unresolvable) as a fresh import instead of aborting the year', async () => {
const { supabase, enqueueMany } = createQueuedMockSupabase()
enqueueMany([
{ data: [makePriorImportRow({ id: 'imp-stale' })] }, // findOverlappingPeriodImports
{ data: null }, // replaceSIEImport fetch: row gone (deleted between the two queries)
{ data: null, count: 0 }, // survivor guard: no posted import entries remain
{ data: null }, // cleanupStaleImportRecords delete
{ data: { id: 'imp-created' } }, // pending sie_imports insert
{ data: [] }, // chart fetch
{ data: null }, // chart insert
{ data: null }, // find existing fiscal period → clean stop
])
const result = await executeSIEImport(
supabase as unknown as SupabaseClient,
'company-1',
'user-1',
makeParsedFile(),
mappings,
importOptions,
)
// Not stranded: the import continued past the replace block.
expect(result.errors.join(' ')).not.toMatch(/ersätta/i)
expect(result.errors.join(' ')).toMatch(/No matching fiscal period found/i)
expect(result.warnings.join(' ')).toMatch(/fortsätter som ny import/i)
expect(result.replacedPriorImport).toBeNull()
const rpcCalls = (supabase.rpc.mock.calls as [string][])
.filter(([fn]) => fn === 'replace_sie_import')
expect(rpcCalls).toHaveLength(0)
})
it('aborts when posted import entries survive a stale watermark, instead of duplicating them', async () => {
const { supabase, enqueueMany } = createQueuedMockSupabase()
enqueueMany([
{ data: [makePriorImportRow({ id: 'imp-stale' })] }, // findOverlappingPeriodImports
{ data: null }, // replaceSIEImport fetch: metadata row gone
{ data: null, count: 42 }, // survivor guard: 42 posted import entries REMAIN in the year
])
const result = await executeSIEImport(
supabase as unknown as SupabaseClient,
'company-1',
'user-1',
makeParsedFile(),
mappings,
importOptions,
)
// The metadata row is gone but its verifikationer are not: importing
// fresh would duplicate them (BFL 4:1), so the year aborts.
expect(result.success).toBe(false)
expect(result.errors.join(' ')).toMatch(/42 kvarvarande verifikationer/)
expect(result.importId).toBeNull()
})
it('aborts when the survivor check itself fails, instead of assuming zero survivors', async () => {
const { supabase, enqueueMany } = createQueuedMockSupabase()
enqueueMany([
{ data: [makePriorImportRow({ id: 'imp-stale' })] }, // findOverlappingPeriodImports
{ data: null }, // replaceSIEImport fetch: metadata row gone
{
data: null,
count: null,
error: { message: 'canceling statement due to statement timeout' },
}, // survivor guard query fails: survivor state UNKNOWN
])
const result = await executeSIEImport(
supabase as unknown as SupabaseClient,
'company-1',
'user-1',
makeParsedFile(),
mappings,
importOptions,
)
expect(result.success).toBe(false)
expect(result.errors.join(' ')).toMatch(/Kunde inte verifiera att tidigare importdata är borttagen/)
expect(result.importId).toBeNull()
})
it('aborts the year when the replace pre-check query fails, instead of importing fresh', async () => {
const { supabase, enqueueMany } = createQueuedMockSupabase()
enqueueMany([
{ data: [makePriorImportRow({ id: 'imp-prior' })] }, // findOverlappingPeriodImports
{
data: null,
error: { code: '57014', message: 'canceling statement due to statement timeout' },
}, // replaceSIEImport fetch: transient failure, prior entries may still exist
])
const result = await executeSIEImport(
supabase as unknown as SupabaseClient,
'company-1',
'user-1',
makeParsedFile(),
mappings,
importOptions,
)
// NOT treated as a stale watermark: the year aborts fail-closed.
expect(result.success).toBe(false)
expect(result.errors.join(' ')).toMatch(/statement timeout/i)
expect(result.warnings.join(' ')).not.toMatch(/fortsätter som ny import/i)
// Aborted before any sie_imports row was created: nothing imported fresh.
expect(result.importId).toBeNull()
const rpcCalls = (supabase.rpc.mock.calls as [string][])
.filter(([fn]) => fn === 'replace_sie_import')
expect(rpcCalls).toHaveLength(0)
})
it('still aborts the year when the prior import sits in a locked period', async () => {
const { supabase, enqueueMany } = createQueuedMockSupabase()
enqueueMany([
{ data: [makePriorImportRow({ id: 'imp-locked' })] }, // findOverlappingPeriodImports
{ data: { status: 'completed', fiscal_period_id: 'fp-2025' } }, // replace fetch
{ data: { is_closed: false, locked_at: '2026-01-31T00:00:00Z' } }, // period check: LOCKED
])
const result = await executeSIEImport(
supabase as unknown as SupabaseClient,
'company-1',
'user-1',
makeParsedFile(),
mappings,
importOptions,
)
expect(result.success).toBe(false)
expect(result.errors.join(' ')).toMatch(/låst eller stängt/i)
// Aborted before any sie_imports row was created.
expect(result.importId).toBeNull()
})
})
+177 -33
View File
@@ -32,6 +32,7 @@ import {
// Re-export from the parser (moved there to avoid an import cycle:
// getEffectiveOpeningBalances needs it) so existing importers keep working.
export { isBalanceSheetAccount } from './sie-parser'
import { fetchAllRows } from '@/lib/supabase/fetch-all'
import { getBASReference } from '@/lib/bookkeeping/bas-reference'
import { classifyAccount } from '@/lib/bookkeeping/account-classifier'
import { computeSRUCode } from '@/lib/bookkeeping/bas-data/sru-mapping'
@@ -133,11 +134,56 @@ export async function checkDuplicateImport(
return data as SIEImport | null
}
/**
* Every completed SIE import whose fiscal year overlaps the given range,
* newest first (imported_at desc, created_at as tiebreak).
*
* More than one row can overlap the same räkenskapsår: a manual upload plus
* a provider sync, or residue from partially deleted data. The old
* `.limit(1).maybeSingle()` shape picked an arbitrary row in that case, so
* the replace flow resolved one watermark and left the others standing,
* permanently blocking a re-sync of that year (issue #1667). Callers that
* resolve prior imports must handle every returned row.
*/
export async function findOverlappingPeriodImports(
supabase: SupabaseClient,
companyId: string,
fiscalYearStart: string,
fiscalYearEnd: string
): Promise<SIEImport[]> {
// Range overlap check: start <= other_end AND end >= other_start.
// Two imports whose räkenskapsår overlap would produce duplicate
// verifikationer, violating BFL 4:1 (löpande bokföring).
//
// fetchAllRows throws on any query error: a swallowed error here would
// return [] and let replace mode import fresh on top of rows it never
// resolved. It also pages past PostgREST's row cap, and the id tiebreak
// keeps the pagination order total.
const rows = await fetchAllRows<SIEImport>(
({ from, to }) =>
supabase
.from('sie_imports')
.select('*')
.eq('company_id', companyId)
.eq('status', 'completed')
.lte('fiscal_year_start', fiscalYearEnd)
.gte('fiscal_year_end', fiscalYearStart)
.order('imported_at', { ascending: false, nullsFirst: false })
.order('created_at', { ascending: false })
.order('id', { ascending: false })
.range(from, to),
{ dedupeBy: (r) => r.id }
)
return rows
}
/**
* Check if a completed SIE import already exists for the same fiscal year period.
* Prevents importing two different SIE files that cover the same accounting period,
* which would create duplicate verifikationer violating BFL 4:1 (löpande bokföring).
* Only blocks on status='completed': failed/pending imports don't prevent retries.
* When several rows overlap, the newest is returned (deterministic, see
* findOverlappingPeriodImports).
*/
export async function checkDuplicatePeriodImport(
supabase: SupabaseClient,
@@ -145,20 +191,10 @@ export async function checkDuplicatePeriodImport(
fiscalYearStart: string,
fiscalYearEnd: string
): Promise<SIEImport | null> {
// Range overlap check: start <= other_end AND end >= other_start.
// Two imports whose räkenskapsår overlap would produce duplicate
// verifikationer, violating BFL 4:1 (löpande bokföring).
const { data } = await supabase
.from('sie_imports')
.select('*')
.eq('company_id', companyId)
.eq('status', 'completed')
.lte('fiscal_year_start', fiscalYearEnd)
.gte('fiscal_year_end', fiscalYearStart)
.limit(1)
.maybeSingle()
return data as SIEImport | null
const overlapping = await findOverlappingPeriodImports(
supabase, companyId, fiscalYearStart, fiscalYearEnd
)
return overlapping[0] ?? null
}
/**
@@ -209,27 +245,55 @@ async function rpcClientForBulkDelete(fallback: SupabaseClient): Promise<Supabas
* actor the gate can never match and always raises. Making the parameter
* required means a caller that has no authenticated user fails to compile
* rather than discovering the closed gate at runtime.
*
* On failure `code` classifies why, so callers can tell a stale watermark
* (the row vanished or already left 'completed': nothing left to replace,
* safe to proceed as a fresh import) from a real refusal (locked period,
* authorization, RPC failure) that must abort.
*/
export type ReplaceSIEImportResult = {
success: boolean
deletedEntries: number
error?: string
code?: 'not_found' | 'not_completed' | 'period_locked' | 'rpc_error'
}
export async function replaceSIEImport(
supabase: SupabaseClient,
companyId: string,
importId: string,
userId: string
): Promise<{ success: boolean; deletedEntries: number; error?: string }> {
// 1. Fetch and validate the import record
const { data: importRecord } = await supabase
): Promise<ReplaceSIEImportResult> {
// 1. Fetch and validate the import record. Only PGRST116 (zero rows from
// .single()) means the row is genuinely absent; any other error (statement
// timeout, network failure, 5xx surfaced as a PostgREST error) tells us
// nothing about whether the prior import's verifikationer are still in the
// ledger. Classifying such a failure as not_found would let the replace
// loop in executeSIEImport treat it as a stale watermark and import the
// year fresh on top of the old entries (silent duplicates, BFL 4:1), so it
// must fail closed as rpc_error and abort instead.
const { data: importRecord, error: fetchError } = await supabase
.from('sie_imports')
.select('status, fiscal_period_id')
.eq('id', importId)
.eq('company_id', companyId)
.single()
if (fetchError && fetchError.code !== 'PGRST116') {
return {
success: false,
deletedEntries: 0,
error: `Kunde inte läsa tidigare import: ${fetchError.message}`,
code: 'rpc_error',
}
}
if (!importRecord) {
return { success: false, deletedEntries: 0, error: 'Import hittades inte' }
return { success: false, deletedEntries: 0, error: 'Import hittades inte', code: 'not_found' }
}
if (importRecord.status !== 'completed') {
return { success: false, deletedEntries: 0, error: `Kan bara ersätta slutförda importer (status: ${importRecord.status})` }
return { success: false, deletedEntries: 0, error: `Kan bara ersätta slutförda importer (status: ${importRecord.status})`, code: 'not_completed' }
}
// 2. Check that the fiscal period is not closed or locked
@@ -242,7 +306,7 @@ export async function replaceSIEImport(
.single()
if (period?.is_closed || period?.locked_at) {
return { success: false, deletedEntries: 0, error: 'Kan inte ersätta import i ett låst eller stängt räkenskapsår. Öppna perioden först.' }
return { success: false, deletedEntries: 0, error: 'Kan inte ersätta import i ett låst eller stängt räkenskapsår. Öppna perioden först.', code: 'period_locked' }
}
}
@@ -259,7 +323,21 @@ export async function replaceSIEImport(
})
if (rpcError) {
return { success: false, deletedEntries: 0, error: `Kunde inte ersätta import: ${rpcError.message}` }
// The RPC re-checks status inside its transaction; a row that lost the
// race between our pre-check and the RPC surfaces here as "not found or
// not in completed status": classify it as the stale watermark it is.
// CONTRACT: this regex must match the RAISE EXCEPTION wording in
// replace_sie_import (supabase/migrations/*replace_sie_import*.sql).
// If that wording changes without this regex, a genuine stale race is
// reclassified as rpc_error — which fails closed (the import aborts),
// never open.
const staleRace = /not found or not in completed status/i.test(rpcError.message)
return {
success: false,
deletedEntries: 0,
error: `Kunde inte ersätta import: ${rpcError.message}`,
code: staleRace ? 'not_completed' : 'rpc_error',
}
}
return { success: true, deletedEntries: deletedCount as number }
@@ -1890,10 +1968,13 @@ export async function loadMappings(supabase: SupabaseClient, companyId: string):
* the new SIE's fiscal year is handled:
* - 'block' (default): refuse with a Swedish error. Used by the manual
* upload route in app/api/import/sie. Preserves prior behavior.
* - 'replace': automatically call replaceSIEImport on the prior row
* (marks it 'replaced', cancels its imported journal entries) and
* proceed. Used by the Fortnox re-sync flow so the user can pull
* updated data from Fortnox without manual cleanup.
* - 'replace': automatically call replaceSIEImport on EVERY overlapping
* completed row (marks them 'replaced', cancels their imported journal
* entries) and proceed. A row that can no longer be resolved (deleted
* or already left 'completed') is a stale watermark: it is skipped with
* a warning and the year imports fresh (issue #1667). Used by the
* provider re-sync flow so the user can pull updated data without
* manual cleanup.
*
* Replace only cancels journal entries with source_type='import'; entries
* the user created natively in Accounted (categorized transactions, invoices,
@@ -1987,34 +2068,62 @@ export async function executeSIEImport(
return result
}
// Replace mode: if a prior completed import overlaps the new SIE's fiscal
// year, mark it 'replaced' (and cancel its imported entries) before we
// Replace mode: if prior completed imports overlap the new SIE's fiscal
// year, mark them 'replaced' (and cancel their imported entries) before we
// try to insert. Done before checkDuplicateImport / checkDuplicatePeriodImport
// since both of those would otherwise reject the replace flow.
//
// ALL overlapping rows are resolved, not just the newest: more than one
// completed row can cover the same year (manual upload + provider sync,
// or residue from partially deleted data), and leaving one standing
// permanently blocks or corrupts the next re-sync of that year
// (issue #1667).
if (onExistingPeriod === 'replace') {
const fyStart = parsed.stats.fiscalYearStart
const fyEnd = parsed.stats.fiscalYearEnd
if (fyStart && fyEnd) {
const priorPeriodImport = await checkDuplicatePeriodImport(
const priorPeriodImports = await findOverlappingPeriodImports(
supabase, companyId, fyStart, fyEnd
)
if (priorPeriodImport) {
let replacedNewestId: string | null = null
let replacedDeletedEntries = 0
let staleSkips = 0
for (const priorPeriodImport of priorPeriodImports) {
// Pass the authorising user: this path often runs on an API-key /
// MCP client where auth.uid() is NULL, and the replace_sie_import
// owner/admin gate would otherwise fail closed.
const replaceResult = await replaceSIEImport(
supabase, companyId, priorPeriodImport.id, userId
)
if (!replaceResult.success) {
// Stale watermark: the sie_imports row is gone or no longer
// 'completed' (its data was already deleted or another actor
// resolved it). There is nothing left to replace for that row,
// so treat this year as a fresh import instead of stranding the
// user between states (issue #1667: re-sync could not re-import
// an earlier fiscal year after deletion).
if (replaceResult.code === 'not_found' || replaceResult.code === 'not_completed') {
staleSkips += 1
result.warnings.push(
`Tidigare import ${priorPeriodImport.id} kunde inte ersättas (${replaceResult.error ?? 'okänd orsak'}): dess data är redan borttagen, importen fortsätter som ny import.`
)
continue
}
// Real refusals (låst/stängd period, behörighet, RPC-fel) still
// abort the year: importing on top of entries we could not
// delete would duplicate verifikationer.
result.errors.push(
replaceResult.error ?? 'Kunde inte ersätta tidigare SIE-import'
)
return result
}
result.replacedPriorImport = {
importId: priorPeriodImport.id,
deletedEntries: replaceResult.deletedEntries,
}
// Rows arrive newest first: report the newest replaced import's id
// (the shape consumers already render) with the total entries
// deleted across every replaced row.
replacedNewestId ??= priorPeriodImport.id
replacedDeletedEntries += replaceResult.deletedEntries
// The replace_sie_import RPC clears fiscal_periods
// opening_balance_entry_id and opening_balances_set inside its
@@ -2034,6 +2143,41 @@ export async function executeSIEImport(
.eq('opening_balance_entry_id', priorPeriodImport.opening_balance_entry_id)
}
}
// A stale watermark (not_found / not_completed) only proves the
// sie_imports METADATA row is gone: replace_sie_import deletes
// entries by (company, fiscal_period, source_type='import'), so
// posted entries can outlive their import row. Before trusting the
// skip, positively confirm the year holds no surviving posted
// import entries — importing on top of survivors would duplicate
// verifikationer (BFL 4:1). Fail closed on a failed check.
if (staleSkips > 0) {
const { count: survivorCount, error: survivorError } = await supabase
.from('journal_entries')
.select('id', { count: 'exact', head: true })
.eq('company_id', companyId)
.eq('source_type', 'import')
.eq('status', 'posted')
.gte('entry_date', fyStart)
.lte('entry_date', fyEnd)
if (survivorError || typeof survivorCount !== 'number') {
result.errors.push(
`Kunde inte verifiera att tidigare importdata är borttagen: ${survivorError?.message ?? 'okänt fel'}. Importen avbryts.`
)
return result
}
if (survivorCount > 0) {
result.errors.push(
`Räkenskapsåret har ${survivorCount} kvarvarande verifikationer från en tidigare import vars importpost saknas. Importen avbryts för att undvika dubbletter (BFL 4:1). Ta bort de kvarvarande verifikationerna först.`
)
return result
}
}
if (replacedNewestId) {
result.replacedPriorImport = {
importId: replacedNewestId,
deletedEntries: replacedDeletedEntries,
}
}
}
}