fix(arcim): allow Fortnox re-sync to replace prior SIE import per fiscal year (#512)
* fix(arcim): allow Fortnox re-sync to replace prior SIE import per fiscal year A user reported a sync failure when retrying Fortnox after adding more verifications: Import failed: Failed to create pending import record: duplicate key value violates unique constraint "sie_imports_company_id_file_hash_active_idx" Root cause: Fortnox embeds the export-time #GEN date in every SIE export, so the file hash always differs between syncs. The wizard's hash-based duplicate detection treated each sync as a brand new file, but the engine still rejected the insert because the per-period import slot was held by the prior 'completed' row. This change reframes Fortnox re-sync as a replace operation rather than a fresh import: - New executeSIEImport option `onExistingPeriod: 'block' | 'replace'`. Manual SIE upload at /api/import/sie keeps default 'block' (current behavior, no regression). The Fortnox /import-sie endpoint passes 'replace', which runs replaceSIEImport on any overlapping completed import before insert. Imported journal entries from the prior import are cancelled per BFL 5 kap 5§; user-created entries (manual, transaction, invoice) are untouched. - /sie-data switches from hash-based to period-based duplicate detection and returns previousImport metadata per fiscal year. - Wizard drops the alreadyImported skip filter, surfaces an amber callout in the confirm dialog listing fiscal years that will be replaced, and shows "ersatte N tidigare importerade verifikationer" per year. - createPendingImportRecord translates 23505 partial-index violations to a clear Swedish recovery message instead of leaking the raw constraint name. - cleanupStaleImportRecords drops the 1-hour age gate and also cleans status='mapped' orphans. SIE imports are single-flight per company so the gate just made legitimate retries fail. - After replace, the fiscal_periods row's opening_balances_set and opening_balance_entry_id are cleared (only when they pointed at the cancelled prior IB entry), so the new IB import isn't skipped. Schema-drift migration captures the partial unique index sie_imports_company_id_file_hash_active_idx that already exists in production (added out-of-band) and drops the now-superseded plain sie_imports_company_id_file_hash_key constraint. Both statements are idempotent — verified no-op against production. Tests: new pg-real test covers the partial index admit-replaced semantics, source_type='import'-only cancellation in replace_sie_import, and the post-replace insert path. Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com> * fix(arcim): address PR review — restore 5-min cleanup gate, fix test source_type Greptile P2: `cleanupStaleImportRecords` was deleting `pending` rows unconditionally, which could wipe a concurrent in-flight import in another tab/session. Restored a 5-minute age gate (long enough for any normal interactive import, short enough that legitimate retries after a crash still succeed). Also dropped `mapped` from the cleanup — it is defined in SIEImportStatus but no code path writes it, so including it was both unnecessary and added the concurrent-session risk Greptile flagged. pg-real test: insertPostedEntry used `source_type='transaction'` which is not a valid value per the journal_entries_source_type_check constraint (migration 20260516060000). Switched to `'bank_transaction'` — the actual source_type emitted when a user categorizes a bank transaction in gnubok. Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com> --------- Co-authored-by: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
This commit is contained in:
Jakob Wennberg
@@ -28,6 +28,7 @@ import {
|
||||
ExternalLink,
|
||||
Info,
|
||||
RotateCcw,
|
||||
RefreshCw,
|
||||
AlertTriangle,
|
||||
ChevronDown,
|
||||
ChevronRight,
|
||||
@@ -135,8 +136,17 @@ interface PreviewData {
|
||||
|
||||
interface SIEFileStatus {
|
||||
fiscalYear: number
|
||||
// Legacy field for older builds — read previousImport instead.
|
||||
alreadyImported: boolean
|
||||
importedAt: string | null
|
||||
// New (period-based) detection. When present, this fiscal year already has a
|
||||
// completed import in gnubok and a re-sync will replace it (cancelling the
|
||||
// imported journal entries; user-created entries are untouched).
|
||||
previousImport: {
|
||||
importedAt: string | null
|
||||
fiscalYearStart: string | null
|
||||
fiscalYearEnd: string | null
|
||||
} | null
|
||||
}
|
||||
|
||||
interface SIEData {
|
||||
@@ -147,6 +157,7 @@ interface SIEData {
|
||||
fileStatuses: SIEFileStatus[]
|
||||
allImported: boolean
|
||||
newFileCount: number
|
||||
replacedFileCount?: number
|
||||
basAccounts: BASAccount[]
|
||||
}
|
||||
|
||||
@@ -776,8 +787,11 @@ function OptionsStep({
|
||||
}
|
||||
|
||||
const fileStatuses = sieData?.fileStatuses ?? []
|
||||
const allSieImported = sieData?.allImported ?? false
|
||||
const newFileCount = sieData?.newFileCount ?? 0
|
||||
const replacedFileCount = fileStatuses.filter(fs => fs.previousImport).length
|
||||
const yearsToReplace = fileStatuses
|
||||
.filter(fs => fs.previousImport)
|
||||
.map(fs => fs.fiscalYear)
|
||||
|
||||
const selectedItems: string[] = []
|
||||
if (options.importCompanyInfo) selectedItems.push('Företagsinformation')
|
||||
@@ -810,26 +824,31 @@ function OptionsStep({
|
||||
<OptionRow
|
||||
icon={<Database className="h-4 w-4" />}
|
||||
label="Bokföringsdata (SIE)"
|
||||
description={allSieImported
|
||||
? 'All bokföringsdata är redan importerad — inga ändringar'
|
||||
: newFileCount > 0
|
||||
? `${newFileCount} ny(a) räkenskapsår att importera`
|
||||
: 'Kontoplan, ingående balanser och verifikationer'
|
||||
description={
|
||||
replacedFileCount > 0 && newFileCount > 0
|
||||
? `${newFileCount} nya och ${replacedFileCount} uppdaterade räkenskapsår`
|
||||
: replacedFileCount > 0
|
||||
? `${replacedFileCount} räkenskapsår med uppdaterad data — tidigare import ersätts`
|
||||
: newFileCount > 0
|
||||
? `${newFileCount} ny(a) räkenskapsår att importera`
|
||||
: 'Kontoplan, ingående balanser och verifikationer'
|
||||
}
|
||||
checked={options.importSIEData && !allSieImported}
|
||||
onChange={() => !allSieImported && toggleOption('importSIEData')}
|
||||
disabled={allSieImported}
|
||||
checked={options.importSIEData}
|
||||
onChange={() => toggleOption('importSIEData')}
|
||||
/>
|
||||
{/* Per-file import status */}
|
||||
{fileStatuses.length > 0 && (
|
||||
<div className="ml-4 space-y-1.5">
|
||||
{fileStatuses.map((fs) => (
|
||||
<div key={fs.fiscalYear} className="flex items-center gap-2 text-xs">
|
||||
{fs.alreadyImported ? (
|
||||
{fs.previousImport ? (
|
||||
<>
|
||||
<CheckCircle className="h-3.5 w-3.5 text-emerald-500" />
|
||||
<RefreshCw className="h-3.5 w-3.5 text-amber-500" />
|
||||
<span className="text-muted-foreground">
|
||||
Räkenskapsår {fs.fiscalYear} — importerad {fs.importedAt ? new Date(fs.importedAt).toLocaleDateString('sv-SE') : ''}
|
||||
Räkenskapsår {fs.fiscalYear} — ersätter tidigare import
|
||||
{fs.previousImport.importedAt
|
||||
? ` från ${new Date(fs.previousImport.importedAt).toLocaleDateString('sv-SE')}`
|
||||
: ''}
|
||||
</span>
|
||||
</>
|
||||
) : (
|
||||
@@ -842,7 +861,7 @@ function OptionsStep({
|
||||
))}
|
||||
</div>
|
||||
)}
|
||||
{options.importSIEData && !allSieImported && (
|
||||
{options.importSIEData && (
|
||||
<div className="flex items-center gap-3 rounded-lg border border-border p-3 ml-4">
|
||||
<div className="text-muted-foreground">
|
||||
<FileText className="h-4 w-4" />
|
||||
@@ -918,16 +937,38 @@ function OptionsStep({
|
||||
warningText={`Bokföringsdata, kunder, leverantörer och fakturor importeras till ${branding.appName.toLowerCase()}. Se till att ingen annan import pågår.`}
|
||||
confirmLabel="Starta migrering"
|
||||
>
|
||||
<div className="space-y-2">
|
||||
<p className="text-sm font-medium">Följande importeras:</p>
|
||||
<ul className="space-y-1">
|
||||
{selectedItems.map((item) => (
|
||||
<li key={item} className="flex items-center gap-2 text-sm text-muted-foreground">
|
||||
<CheckCircle className="h-3.5 w-3.5 text-primary" />
|
||||
{item}
|
||||
</li>
|
||||
))}
|
||||
</ul>
|
||||
<div className="space-y-3">
|
||||
<div className="space-y-2">
|
||||
<p className="text-sm font-medium">Följande importeras:</p>
|
||||
<ul className="space-y-1">
|
||||
{selectedItems.map((item) => (
|
||||
<li key={item} className="flex items-center gap-2 text-sm text-muted-foreground">
|
||||
<CheckCircle className="h-3.5 w-3.5 text-primary" />
|
||||
{item}
|
||||
</li>
|
||||
))}
|
||||
</ul>
|
||||
</div>
|
||||
|
||||
{options.importSIEData && yearsToReplace.length > 0 && (
|
||||
<div className="rounded-md border border-amber-500/30 bg-amber-50/50 p-3 dark:bg-amber-950/20">
|
||||
<div className="flex items-start gap-2">
|
||||
<RefreshCw className="mt-0.5 h-4 w-4 shrink-0 text-amber-600" />
|
||||
<div className="space-y-1">
|
||||
<p className="text-sm font-medium">
|
||||
{yearsToReplace.length === 1
|
||||
? `Räkenskapsår ${yearsToReplace[0]} ersätts`
|
||||
: `Räkenskapsår ${yearsToReplace.join(', ')} ersätts`}
|
||||
</p>
|
||||
<p className="text-xs text-muted-foreground">
|
||||
Tidigare importerade verifikationer markeras som annullerade och ersätts av
|
||||
uppdaterad data från källsystemet. Verifikationer som du själv skapat i {branding.appName.toLowerCase()}
|
||||
(kategoriserade banktransaktioner, fakturor m.m.) påverkas inte.
|
||||
</p>
|
||||
</div>
|
||||
</div>
|
||||
</div>
|
||||
)}
|
||||
</div>
|
||||
</ConfirmationDialog>
|
||||
</div>
|
||||
@@ -1061,6 +1102,11 @@ function FiscalYearResult({ result, index }: { result: ImportResult; index: numb
|
||||
{' · '}{d.skippedVouchers.total} hoppade över
|
||||
</span>
|
||||
)}
|
||||
{result.replacedPriorImport && result.replacedPriorImport.cancelledEntries > 0 && (
|
||||
<span>
|
||||
{' · '}ersatte {result.replacedPriorImport.cancelledEntries.toLocaleString('sv-SE')} tidigare importerade verifikationer
|
||||
</span>
|
||||
)}
|
||||
</p>
|
||||
</div>
|
||||
{expanded
|
||||
@@ -1823,43 +1869,43 @@ export default function ArcimMigrationWorkspace(_props: WorkspaceComponentProps)
|
||||
setMigrationProgress(10)
|
||||
setSieImportResults([])
|
||||
|
||||
// Only import files that haven't been imported yet
|
||||
const filesToImport = sieData.rawContent
|
||||
.map((content, i) => ({ content, status: sieData.fileStatuses?.[i] }))
|
||||
.filter(f => !f.status?.alreadyImported)
|
||||
// Send every file to the engine. The Fortnox endpoint runs in
|
||||
// replace-mode, so a year that already has a completed import
|
||||
// gets its prior import marked 'replaced' (imported entries
|
||||
// cancelled, user-created entries untouched) before the new
|
||||
// SIE is loaded. The per-file result reports replacedPriorImport.
|
||||
const filesToImport = sieData.rawContent.map((content, i) => ({
|
||||
content,
|
||||
status: sieData.fileStatuses?.[i],
|
||||
}))
|
||||
|
||||
if (filesToImport.length === 0) {
|
||||
// All files already imported — skip SIE phase
|
||||
setSieImportResults([])
|
||||
} else {
|
||||
for (let i = 0; i < filesToImport.length; i++) {
|
||||
const progress = 10 + Math.round((i / filesToImport.length) * 40)
|
||||
setMigrationProgress(progress)
|
||||
setMigrationStep(`Importerar bokföringsdata (SIE) — fil ${i + 1} av ${filesToImport.length}...`)
|
||||
for (let i = 0; i < filesToImport.length; i++) {
|
||||
const progress = 10 + Math.round((i / filesToImport.length) * 40)
|
||||
setMigrationProgress(progress)
|
||||
setMigrationStep(`Importerar bokföringsdata (SIE) — fil ${i + 1} av ${filesToImport.length}...`)
|
||||
|
||||
const res = await fetch('/api/extensions/ext/arcim-migration/import-sie', {
|
||||
method: 'POST',
|
||||
headers: { 'Content-Type': 'application/json' },
|
||||
body: JSON.stringify({
|
||||
rawContent: filesToImport[i].content,
|
||||
mappings: sieData.mappings,
|
||||
options: {
|
||||
createFiscalPeriod: true,
|
||||
importOpeningBalances: true,
|
||||
importTransactions: true,
|
||||
voucherSeries: migrationOptions.voucherSeries,
|
||||
},
|
||||
}),
|
||||
})
|
||||
const res = await fetch('/api/extensions/ext/arcim-migration/import-sie', {
|
||||
method: 'POST',
|
||||
headers: { 'Content-Type': 'application/json' },
|
||||
body: JSON.stringify({
|
||||
rawContent: filesToImport[i].content,
|
||||
mappings: sieData.mappings,
|
||||
options: {
|
||||
createFiscalPeriod: true,
|
||||
importOpeningBalances: true,
|
||||
importTransactions: true,
|
||||
voucherSeries: migrationOptions.voucherSeries,
|
||||
},
|
||||
}),
|
||||
})
|
||||
|
||||
if (!res.ok) {
|
||||
const data = await res.json().catch(() => ({}))
|
||||
throw new Error(data.error || `SIE import HTTP ${res.status}`)
|
||||
}
|
||||
|
||||
const result = await res.json() as ImportResult
|
||||
setSieImportResults(prev => [...prev, result])
|
||||
if (!res.ok) {
|
||||
const data = await res.json().catch(() => ({}))
|
||||
throw new Error(data.error || `SIE import HTTP ${res.status}`)
|
||||
}
|
||||
|
||||
const result = await res.json() as ImportResult
|
||||
setSieImportResults(prev => [...prev, result])
|
||||
}
|
||||
}
|
||||
|
||||
|
||||
@@ -17,7 +17,7 @@ import { mapCompanyInfo } from './lib/entity-mapper'
|
||||
import { executeMigration } from './lib/migration-orchestrator'
|
||||
import type { ArcimProvider } from './types'
|
||||
import { ARCIM_PROVIDERS } from './types'
|
||||
import { parseSIEFile, validateSIEFile, calculateFileHash } from '@/lib/import/sie-parser'
|
||||
import { parseSIEFile, validateSIEFile } from '@/lib/import/sie-parser'
|
||||
import { suggestMappings, getMappingStats, isSystemAccount } from '@/lib/import/account-mapper'
|
||||
import { loadMappings, generateImportPreview, executeSIEImport, saveMappings } from '@/lib/import/sie-import'
|
||||
import { BAS_REFERENCE, getBASReference } from '@/lib/bookkeeping/bas-reference'
|
||||
@@ -724,28 +724,57 @@ export const arcimMigrationExtension: Extension = {
|
||||
|
||||
const preview = generateImportPreview(parsed, mappings)
|
||||
|
||||
// Check each file's hash against existing imports
|
||||
const fileStatuses: { fiscalYear: number; rawContent: string; alreadyImported: boolean; importedAt: string | null }[] = []
|
||||
// Detect prior imports by *fiscal period overlap*, not file hash.
|
||||
// Fortnox embeds the export-time #GEN date in every SIE export so
|
||||
// the hash always changes between syncs; only the period stays
|
||||
// stable. A re-sync replaces the prior import for the same period.
|
||||
const fileStatuses: {
|
||||
fiscalYear: number
|
||||
rawContent: string
|
||||
previousImport: {
|
||||
importedAt: string | null
|
||||
fiscalYearStart: string | null
|
||||
fiscalYearEnd: string | null
|
||||
} | null
|
||||
}[] = []
|
||||
for (const file of sieFiles) {
|
||||
const fileHash = await calculateFileHash(file.rawContent)
|
||||
const { data: existingImport } = await supabase
|
||||
.from('sie_imports')
|
||||
.select('imported_at')
|
||||
.eq('company_id', companyId)
|
||||
.eq('file_hash', fileHash)
|
||||
.eq('status', 'completed')
|
||||
.maybeSingle()
|
||||
const fileParsed = parseSIEFile(file.rawContent)
|
||||
const fyStart = fileParsed.stats.fiscalYearStart
|
||||
const fyEnd = fileParsed.stats.fiscalYearEnd
|
||||
|
||||
let priorImport: {
|
||||
imported_at: string | null
|
||||
fiscal_year_start: string | null
|
||||
fiscal_year_end: string | null
|
||||
} | null = null
|
||||
|
||||
if (fyStart && fyEnd) {
|
||||
const { data } = await supabase
|
||||
.from('sie_imports')
|
||||
.select('imported_at, fiscal_year_start, fiscal_year_end')
|
||||
.eq('company_id', companyId)
|
||||
.eq('status', 'completed')
|
||||
.lte('fiscal_year_start', fyEnd)
|
||||
.gte('fiscal_year_end', fyStart)
|
||||
.limit(1)
|
||||
.maybeSingle()
|
||||
priorImport = data
|
||||
}
|
||||
|
||||
fileStatuses.push({
|
||||
fiscalYear: file.fiscalYear,
|
||||
rawContent: file.rawContent,
|
||||
alreadyImported: !!existingImport,
|
||||
importedAt: existingImport?.imported_at ?? null,
|
||||
previousImport: priorImport
|
||||
? {
|
||||
importedAt: priorImport.imported_at,
|
||||
fiscalYearStart: priorImport.fiscal_year_start,
|
||||
fiscalYearEnd: priorImport.fiscal_year_end,
|
||||
}
|
||||
: null,
|
||||
})
|
||||
}
|
||||
|
||||
const allImported = fileStatuses.every(f => f.alreadyImported)
|
||||
const newFiles = fileStatuses.filter(f => !f.alreadyImported)
|
||||
const replacedFileCount = fileStatuses.filter(f => f.previousImport).length
|
||||
|
||||
return NextResponse.json({
|
||||
parsed,
|
||||
@@ -756,11 +785,15 @@ export const arcimMigrationExtension: Extension = {
|
||||
rawContent: fileStatuses.map(f => f.rawContent),
|
||||
fileStatuses: fileStatuses.map(f => ({
|
||||
fiscalYear: f.fiscalYear,
|
||||
alreadyImported: f.alreadyImported,
|
||||
importedAt: f.importedAt,
|
||||
previousImport: f.previousImport,
|
||||
// Back-compat for older wizard builds: an `alreadyImported`
|
||||
// boolean. The new wizard reads `previousImport` directly.
|
||||
alreadyImported: !!f.previousImport,
|
||||
importedAt: f.previousImport?.importedAt ?? null,
|
||||
})),
|
||||
allImported,
|
||||
newFileCount: newFiles.length,
|
||||
allImported: false,
|
||||
newFileCount: fileStatuses.length - replacedFileCount,
|
||||
replacedFileCount,
|
||||
basAccounts: BAS_REFERENCE,
|
||||
})
|
||||
} catch (error) {
|
||||
@@ -917,6 +950,12 @@ export const arcimMigrationExtension: Extension = {
|
||||
importOpeningBalances: options.importOpeningBalances,
|
||||
importTransactions: options.importTransactions,
|
||||
voucherSeries: options.voucherSeries,
|
||||
// Fortnox re-sync semantics: a prior completed import for the
|
||||
// same fiscal year is automatically replaced (its imported
|
||||
// entries are cancelled) so the user can pull updated data
|
||||
// without manual cleanup. Manual SIE upload keeps default
|
||||
// 'block' behavior.
|
||||
onExistingPeriod: 'replace',
|
||||
})
|
||||
|
||||
log.info('SIE import completed:', {
|
||||
|
||||
@@ -0,0 +1,212 @@
|
||||
import { randomUUID } from 'node:crypto'
|
||||
import { describe, expect, it } from 'vitest'
|
||||
import { getPool } from '@/tests/pg/setup'
|
||||
import { seedCompany } from '@/tests/pg/fixtures'
|
||||
|
||||
// Covers the Fortnox re-sync flow:
|
||||
// 1. The partial unique index `sie_imports_company_id_file_hash_active_idx`
|
||||
// (added in migration 20260517150000) blocks duplicate (company_id,
|
||||
// file_hash) rows for active statuses but allows them once a prior row
|
||||
// is marked 'replaced' or 'failed'.
|
||||
// 2. The replace_sie_import RPC cancels journal entries with
|
||||
// source_type='import' while leaving user-created entries
|
||||
// (source_type='manual', 'bank_transaction', etc.) intact.
|
||||
|
||||
async function insertSIEImport(params: {
|
||||
companyId: string
|
||||
userId: string
|
||||
fileHash: string
|
||||
status: 'pending' | 'mapped' | 'completed' | 'failed' | 'replaced'
|
||||
fiscalPeriodId?: string
|
||||
openingBalanceEntryId?: string
|
||||
fiscalYearStart?: string
|
||||
fiscalYearEnd?: string
|
||||
}): Promise<string> {
|
||||
const id = randomUUID()
|
||||
await getPool().query(
|
||||
`INSERT INTO public.sie_imports
|
||||
(id, user_id, company_id, filename, file_hash, sie_type,
|
||||
fiscal_year_start, fiscal_year_end, accounts_count, transactions_count,
|
||||
status, fiscal_period_id, opening_balance_entry_id, imported_at)
|
||||
VALUES ($1, $2, $3, 'fortnox-export.se', $4, 4,
|
||||
$5, $6, 0, 0,
|
||||
$7, $8, $9, $10)`,
|
||||
[
|
||||
id,
|
||||
params.userId,
|
||||
params.companyId,
|
||||
params.fileHash,
|
||||
params.fiscalYearStart ?? '2026-01-01',
|
||||
params.fiscalYearEnd ?? '2026-12-31',
|
||||
params.status,
|
||||
params.fiscalPeriodId ?? null,
|
||||
params.openingBalanceEntryId ?? null,
|
||||
params.status === 'completed' ? new Date().toISOString() : null,
|
||||
],
|
||||
)
|
||||
return id
|
||||
}
|
||||
|
||||
async function insertPostedEntry(params: {
|
||||
userId: string
|
||||
companyId: string
|
||||
fiscalPeriodId: string
|
||||
sourceType: 'import' | 'manual' | 'bank_transaction'
|
||||
voucherNumber: number
|
||||
entryDate?: string
|
||||
}): Promise<string> {
|
||||
const id = randomUUID()
|
||||
await getPool().query(
|
||||
`INSERT INTO public.journal_entries
|
||||
(id, user_id, company_id, fiscal_period_id, voucher_number, voucher_series,
|
||||
entry_date, description, source_type, status)
|
||||
VALUES ($1, $2, $3, $4, $5, 'A', $6, 'Test entry', $7, 'posted')`,
|
||||
[
|
||||
id,
|
||||
params.userId,
|
||||
params.companyId,
|
||||
params.fiscalPeriodId,
|
||||
params.voucherNumber,
|
||||
params.entryDate ?? '2026-06-01',
|
||||
params.sourceType,
|
||||
],
|
||||
)
|
||||
await getPool().query(
|
||||
`INSERT INTO public.journal_entry_lines
|
||||
(journal_entry_id, account_number, debit_amount, credit_amount)
|
||||
VALUES ($1, '1930', 100, 0),
|
||||
($1, '3001', 0, 100)`,
|
||||
[id],
|
||||
)
|
||||
return id
|
||||
}
|
||||
|
||||
describe('sie_imports: partial unique index + replace flow', () => {
|
||||
it('blocks a second active row with the same (company_id, file_hash)', async () => {
|
||||
const { companyId, userId, fiscalPeriodId } = await seedCompany()
|
||||
const hash = `hash-${randomUUID()}`
|
||||
|
||||
await insertSIEImport({
|
||||
companyId,
|
||||
userId,
|
||||
fileHash: hash,
|
||||
status: 'completed',
|
||||
fiscalPeriodId,
|
||||
})
|
||||
|
||||
await expect(
|
||||
insertSIEImport({
|
||||
companyId,
|
||||
userId,
|
||||
fileHash: hash,
|
||||
status: 'pending',
|
||||
fiscalPeriodId,
|
||||
}),
|
||||
).rejects.toThrow(/sie_imports_company_id_file_hash_active_idx/)
|
||||
})
|
||||
|
||||
it('allows a new pending row with the same hash once the prior row is replaced', async () => {
|
||||
const { companyId, userId, fiscalPeriodId } = await seedCompany()
|
||||
const hash = `hash-${randomUUID()}`
|
||||
|
||||
const priorId = await insertSIEImport({
|
||||
companyId,
|
||||
userId,
|
||||
fileHash: hash,
|
||||
status: 'completed',
|
||||
fiscalPeriodId,
|
||||
})
|
||||
|
||||
// Mark the prior row as replaced (simulating what replace_sie_import does)
|
||||
await getPool().query(
|
||||
`UPDATE public.sie_imports SET status = 'replaced', replaced_at = now() WHERE id = $1`,
|
||||
[priorId],
|
||||
)
|
||||
|
||||
// A new pending row with the same hash now succeeds
|
||||
const newId = await insertSIEImport({
|
||||
companyId,
|
||||
userId,
|
||||
fileHash: hash,
|
||||
status: 'pending',
|
||||
fiscalPeriodId,
|
||||
})
|
||||
expect(newId).toBeTruthy()
|
||||
})
|
||||
|
||||
it('replace_sie_import cancels source_type=import entries and leaves manual/bank_transaction entries posted', async () => {
|
||||
const { companyId, userId, fiscalPeriodId } = await seedCompany()
|
||||
|
||||
const obEntry = await insertPostedEntry({
|
||||
userId,
|
||||
companyId,
|
||||
fiscalPeriodId,
|
||||
sourceType: 'import',
|
||||
voucherNumber: 1,
|
||||
})
|
||||
const importEntry1 = await insertPostedEntry({
|
||||
userId,
|
||||
companyId,
|
||||
fiscalPeriodId,
|
||||
sourceType: 'import',
|
||||
voucherNumber: 2,
|
||||
})
|
||||
const importEntry2 = await insertPostedEntry({
|
||||
userId,
|
||||
companyId,
|
||||
fiscalPeriodId,
|
||||
sourceType: 'import',
|
||||
voucherNumber: 3,
|
||||
})
|
||||
const manualEntry = await insertPostedEntry({
|
||||
userId,
|
||||
companyId,
|
||||
fiscalPeriodId,
|
||||
sourceType: 'manual',
|
||||
voucherNumber: 4,
|
||||
})
|
||||
const txnEntry = await insertPostedEntry({
|
||||
userId,
|
||||
companyId,
|
||||
fiscalPeriodId,
|
||||
sourceType: 'bank_transaction',
|
||||
voucherNumber: 5,
|
||||
})
|
||||
|
||||
const importId = await insertSIEImport({
|
||||
companyId,
|
||||
userId,
|
||||
fileHash: `hash-${randomUUID()}`,
|
||||
status: 'completed',
|
||||
fiscalPeriodId,
|
||||
openingBalanceEntryId: obEntry,
|
||||
})
|
||||
|
||||
const { rows } = await getPool().query<{ replace_sie_import: number }>(
|
||||
`SELECT public.replace_sie_import($1::uuid, $2::uuid) AS replace_sie_import`,
|
||||
[companyId, importId],
|
||||
)
|
||||
const cancelled = rows[0]!.replace_sie_import
|
||||
|
||||
// OB entry + 2 import entries = 3 cancelled. Manual & transaction stay posted.
|
||||
expect(cancelled).toBe(3)
|
||||
|
||||
const statuses = await getPool().query<{ id: string; status: string }>(
|
||||
`SELECT id, status FROM public.journal_entries WHERE id = ANY($1)`,
|
||||
[[obEntry, importEntry1, importEntry2, manualEntry, txnEntry]],
|
||||
)
|
||||
const statusById = Object.fromEntries(statuses.rows.map(r => [r.id, r.status]))
|
||||
expect(statusById[obEntry]).toBe('cancelled')
|
||||
expect(statusById[importEntry1]).toBe('cancelled')
|
||||
expect(statusById[importEntry2]).toBe('cancelled')
|
||||
expect(statusById[manualEntry]).toBe('posted')
|
||||
expect(statusById[txnEntry]).toBe('posted')
|
||||
|
||||
const importRow = await getPool().query<{ status: string; replaced_at: string | null }>(
|
||||
`SELECT status, replaced_at FROM public.sie_imports WHERE id = $1`,
|
||||
[importId],
|
||||
)
|
||||
expect(importRow.rows[0]!.status).toBe('replaced')
|
||||
expect(importRow.rows[0]!.replaced_at).not.toBeNull()
|
||||
})
|
||||
})
|
||||
+128
-21
@@ -191,23 +191,41 @@ export async function replaceSIEImport(
|
||||
}
|
||||
|
||||
/**
|
||||
* Clean up stale pending/failed import records for a given file hash.
|
||||
* Prevents UNIQUE constraint conflicts when re-importing after a failure.
|
||||
* Clean up orphan in-flight import records for a given file hash.
|
||||
*
|
||||
* Targets rows in status='pending' — left behind when a prior import
|
||||
* crashed (or short-circuited at checkDuplicatePeriodImport) before
|
||||
* reaching finalizeImportRecord. They hold the slot in the partial
|
||||
* unique index `sie_imports_company_id_file_hash_active_idx`, so a
|
||||
* retry would fail with a constraint violation.
|
||||
*
|
||||
* Five-minute age gate protects an in-flight import in another tab/
|
||||
* session: createPendingImportRecord → ... → finalizeImportRecord can
|
||||
* take tens of seconds for large SIE files. Without the gate, a
|
||||
* concurrent retry of the same file would delete the live pending row
|
||||
* mid-flight and the original session's finalize would silently no-op.
|
||||
* Five minutes is long enough for any normal interactive import yet
|
||||
* short enough that legitimate retries after a crash succeed.
|
||||
*
|
||||
* The 'mapped' status is defined in the type but never written by any
|
||||
* code path, so we don't include it. 'failed' and 'replaced' rows are
|
||||
* allowed by the partial index (excluded from its predicate), so they
|
||||
* stay in place for the audit trail.
|
||||
*/
|
||||
async function cleanupStaleImportRecords(
|
||||
supabase: SupabaseClient,
|
||||
companyId: string,
|
||||
fileHash: string
|
||||
): Promise<void> {
|
||||
const oneHourAgo = new Date(Date.now() - 60 * 60 * 1000).toISOString()
|
||||
const fiveMinutesAgo = new Date(Date.now() - 5 * 60 * 1000).toISOString()
|
||||
|
||||
await supabase
|
||||
.from('sie_imports')
|
||||
.delete()
|
||||
.eq('company_id', companyId)
|
||||
.eq('file_hash', fileHash)
|
||||
.in('status', ['pending', 'failed'])
|
||||
.lt('created_at', oneHourAgo)
|
||||
.eq('status', 'pending')
|
||||
.lt('created_at', fiveMinutesAgo)
|
||||
}
|
||||
|
||||
/**
|
||||
@@ -1377,7 +1395,22 @@ async function createPendingImportRecord(
|
||||
.single()
|
||||
|
||||
if (error || !data) {
|
||||
throw new Error(`Failed to create pending import record: ${error?.message}`)
|
||||
// PG error 23505 (unique_violation) on the partial index means another
|
||||
// active row exists for the same (company_id, file_hash). Surface the
|
||||
// recovery path in Swedish instead of leaking the raw Postgres message.
|
||||
const pgCode = (error as { code?: string } | null | undefined)?.code
|
||||
const pgMessage = error?.message ?? ''
|
||||
const hitsActiveIdx =
|
||||
pgCode === '23505' &&
|
||||
pgMessage.includes('sie_imports_company_id_file_hash_active_idx')
|
||||
|
||||
if (hitsActiveIdx) {
|
||||
throw new Error(
|
||||
'En tidigare SIE-import för samma fil finns redan i gnubok. Öppna importhistoriken och välj "Ersätt import" på den befintliga raden, eller använd Fortnox-synkningen för att hämta uppdaterad data automatiskt.'
|
||||
)
|
||||
}
|
||||
|
||||
throw new Error(`Failed to create pending import record: ${pgMessage}`)
|
||||
}
|
||||
|
||||
return data.id
|
||||
@@ -1490,6 +1523,19 @@ export async function loadMappings(supabase: SupabaseClient, companyId: string):
|
||||
|
||||
/**
|
||||
* Execute the full SIE import
|
||||
*
|
||||
* `onExistingPeriod` controls how a prior completed import that overlaps
|
||||
* the new SIE's fiscal year is handled:
|
||||
* - 'block' (default): refuse with a Swedish error. Used by the manual
|
||||
* upload route in app/api/import/sie. Preserves prior behavior.
|
||||
* - 'replace': automatically call replaceSIEImport on the prior row
|
||||
* (marks it 'replaced', cancels its imported journal entries) and
|
||||
* proceed. Used by the Fortnox re-sync flow so the user can pull
|
||||
* updated data from Fortnox without manual cleanup.
|
||||
*
|
||||
* Replace only cancels journal entries with source_type='import' — entries
|
||||
* the user created natively in gnubok (categorized transactions, invoices,
|
||||
* etc.) are left alone. See the replace_sie_import RPC.
|
||||
*/
|
||||
export async function executeSIEImport(
|
||||
supabase: SupabaseClient,
|
||||
@@ -1504,6 +1550,7 @@ export async function executeSIEImport(
|
||||
importOpeningBalances: boolean
|
||||
importTransactions: boolean
|
||||
voucherSeries?: string
|
||||
onExistingPeriod?: 'block' | 'replace'
|
||||
}
|
||||
): Promise<ImportResult> {
|
||||
const result: ImportResult = {
|
||||
@@ -1515,8 +1562,11 @@ export async function executeSIEImport(
|
||||
journalEntryIds: [],
|
||||
errors: [],
|
||||
warnings: [],
|
||||
replacedPriorImport: null,
|
||||
}
|
||||
|
||||
const onExistingPeriod = options.onExistingPeriod ?? 'block'
|
||||
|
||||
try {
|
||||
// Validate all accounts are mapped
|
||||
const unmapped = mappings.filter((m) => !m.targetAccount)
|
||||
@@ -1527,13 +1577,66 @@ export async function executeSIEImport(
|
||||
return result
|
||||
}
|
||||
|
||||
// Check for duplicate import (only completed imports count as duplicates)
|
||||
const duplicate = await checkDuplicateImport(supabase, companyId, options.fileContent)
|
||||
if (duplicate) {
|
||||
result.errors.push(
|
||||
`This file has already been imported on ${duplicate.imported_at ? new Date(duplicate.imported_at).toLocaleDateString('sv-SE') : 'okänt datum'}`
|
||||
)
|
||||
return result
|
||||
// Replace mode: if a prior completed import overlaps the new SIE's fiscal
|
||||
// year, mark it 'replaced' (and cancel its imported entries) before we
|
||||
// try to insert. Done before checkDuplicateImport / checkDuplicatePeriodImport
|
||||
// since both of those would otherwise reject the replace flow.
|
||||
if (onExistingPeriod === 'replace') {
|
||||
const fyStart = parsed.stats.fiscalYearStart
|
||||
const fyEnd = parsed.stats.fiscalYearEnd
|
||||
if (fyStart && fyEnd) {
|
||||
const priorPeriodImport = await checkDuplicatePeriodImport(
|
||||
supabase, companyId, fyStart, fyEnd
|
||||
)
|
||||
if (priorPeriodImport) {
|
||||
const replaceResult = await replaceSIEImport(
|
||||
supabase, companyId, priorPeriodImport.id
|
||||
)
|
||||
if (!replaceResult.success) {
|
||||
result.errors.push(
|
||||
replaceResult.error ?? 'Kunde inte ersätta tidigare SIE-import'
|
||||
)
|
||||
return result
|
||||
}
|
||||
result.replacedPriorImport = {
|
||||
importId: priorPeriodImport.id,
|
||||
cancelledEntries: replaceResult.cancelledEntries,
|
||||
}
|
||||
|
||||
// The replace_sie_import RPC cancelled the prior import's opening
|
||||
// balance entry, but the fiscal_periods row still flags
|
||||
// opening_balances_set=true and points opening_balance_entry_id at
|
||||
// the now-cancelled row. Without clearing those, the IB import
|
||||
// below would skip ("Ingående balanser finns redan...") and the
|
||||
// new IB would be lost. Only reset when the cleared entry was the
|
||||
// prior import's IB — if the period's IB came from somewhere else
|
||||
// (manual entry, year-end carryover), we must not touch it.
|
||||
if (priorPeriodImport.fiscal_period_id && priorPeriodImport.opening_balance_entry_id) {
|
||||
await supabase
|
||||
.from('fiscal_periods')
|
||||
.update({
|
||||
opening_balances_set: false,
|
||||
opening_balance_entry_id: null,
|
||||
})
|
||||
.eq('id', priorPeriodImport.fiscal_period_id)
|
||||
.eq('company_id', companyId)
|
||||
.eq('opening_balance_entry_id', priorPeriodImport.opening_balance_entry_id)
|
||||
}
|
||||
}
|
||||
}
|
||||
}
|
||||
|
||||
// Block mode (default): the hash and period checks reject duplicates with
|
||||
// graceful Swedish errors. Skipped in replace mode because we've already
|
||||
// resolved any prior import above.
|
||||
if (onExistingPeriod === 'block') {
|
||||
const duplicate = await checkDuplicateImport(supabase, companyId, options.fileContent)
|
||||
if (duplicate) {
|
||||
result.errors.push(
|
||||
`This file has already been imported on ${duplicate.imported_at ? new Date(duplicate.imported_at).toLocaleDateString('sv-SE') : 'okänt datum'}`
|
||||
)
|
||||
return result
|
||||
}
|
||||
}
|
||||
|
||||
// Create pending import record early — ensures tracking even if later steps fail
|
||||
@@ -1628,15 +1731,19 @@ export async function executeSIEImport(
|
||||
return result
|
||||
}
|
||||
|
||||
// Safety net: reject if a completed import already exists for this period
|
||||
const periodDuplicate = await checkDuplicatePeriodImport(
|
||||
supabase, companyId, fiscalYearStart, fiscalYearEnd
|
||||
)
|
||||
if (periodDuplicate) {
|
||||
result.errors.push(
|
||||
`En SIE-import för ett överlappande räkenskapsår (${periodDuplicate.fiscal_year_start} – ${periodDuplicate.fiscal_year_end}) finns redan`
|
||||
// Safety net: reject if a completed import already exists for this period.
|
||||
// Skipped in replace mode — any overlapping prior import was already
|
||||
// marked 'replaced' at the top of executeSIEImport.
|
||||
if (onExistingPeriod === 'block') {
|
||||
const periodDuplicate = await checkDuplicatePeriodImport(
|
||||
supabase, companyId, fiscalYearStart, fiscalYearEnd
|
||||
)
|
||||
return result
|
||||
if (periodDuplicate) {
|
||||
result.errors.push(
|
||||
`En SIE-import för ett överlappande räkenskapsår (${periodDuplicate.fiscal_year_start} – ${periodDuplicate.fiscal_year_end}) finns redan`
|
||||
)
|
||||
return result
|
||||
}
|
||||
}
|
||||
|
||||
if (options.createFiscalPeriod) {
|
||||
|
||||
@@ -296,6 +296,11 @@ export interface ImportResult {
|
||||
|
||||
// Structured details for UI (populated alongside warnings for backwards compat)
|
||||
details?: ImportResultDetails
|
||||
|
||||
// If this import replaced a prior completed import for the same fiscal year
|
||||
// (Fortnox re-sync flow), the prior import's id and the count of journal
|
||||
// entries that were cancelled as a result.
|
||||
replacedPriorImport?: { importId: string; cancelledEntries: number } | null
|
||||
}
|
||||
|
||||
/**
|
||||
|
||||
@@ -0,0 +1,22 @@
|
||||
-- Capture the partial unique index on sie_imports that already exists in
|
||||
-- production but was never expressed as a repo migration (schema drift).
|
||||
--
|
||||
-- The plain UNIQUE (company_id, file_hash) constraint added in
|
||||
-- 20260330130000_multi_tenant_company_refactor.sql blocks a fiscal-year
|
||||
-- re-sync from Fortnox: the prior 'completed' row keeps the slot even
|
||||
-- though we want a new import to take its place (with the old one marked
|
||||
-- 'replaced'). The partial index relaxes uniqueness for 'replaced' and
|
||||
-- 'failed' rows so a replace-and-reimport works.
|
||||
--
|
||||
-- Both statements are idempotent. Against production this migration is a
|
||||
-- no-op; on dev/staging databases that still carry the plain constraint
|
||||
-- it brings them in line.
|
||||
|
||||
CREATE UNIQUE INDEX IF NOT EXISTS sie_imports_company_id_file_hash_active_idx
|
||||
ON public.sie_imports (company_id, file_hash)
|
||||
WHERE status <> ALL (ARRAY['replaced'::text, 'failed'::text]);
|
||||
|
||||
ALTER TABLE public.sie_imports
|
||||
DROP CONSTRAINT IF EXISTS sie_imports_company_id_file_hash_key;
|
||||
|
||||
NOTIFY pgrst, 'reload schema';
|
||||
Reference in New Issue
Block a user