Files
accounted/extensions/general/arcim-migration/lib/migration-orchestrator.ts
T
Jakob WennbergandClaude Opus 4.6 98cd253bce feat: enable banking hardening, arcim inference, SIE fixes, onboarding (#32)
* feat: import system improvements, INK2 fix, and Swedish text corrections

- SIE parser: Windows-1252 and CP437 encoding detection and decoding
- Bank file parser: add Nordea Business (Företag) CSV format
- Bank file parser: improve format detection for SEB, Länsförsäkringar, generic CSV
- INK2 engine: calculate årets resultat (7222) from income statement for open fiscal years
- Dashboard: parallel Supabase queries, simplified dashboard page
- Fix Swedish characters (å, ä, ö) in BAS data descriptions, validation messages, AI consent disclosures
- Import wizard UI improvements across all steps
- Migration: add 'bas_range' match type to sie_account_mappings constraint
- Extensive new tests for SIE parser encoding and bank file parser

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>

* feat: arcim migration wizard UX fixes, Sentry setup, and extension scaffolding

Arcim migration wizard improvements:
- Progress bar now excludes non-interactive steps (migrating/result)
- Fix OAuth text to match target="_blank" behavior (new tab, not redirect)
- Display month names instead of "Månad X" in preview
- Fix Swedish typo "förifylla" in no-company-info message
- Replace native checkboxes with shadcn Switch in options step
- Add ConfirmationDialog before starting migration
- Show progress percentage during migration
- Add "Nästa steg" guidance and navigation links in result step
- Add "Försök igen" button in error state (returns to options)
- Add Bokio company ID help text (GUID from URL)
- Add Fortnox integration add-on hint on connection failure

Also includes: SIE import system improvements, INK2 fixes, Swedish text
corrections, Sentry error tracking setup, and arcim-migration extension
scaffolding.

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>

* fix: address PR review feedback

- Fix OAuth error recovery blank page (restore provider from URL params)
- Pass real userId to MigrationWizard instead of empty string
- Remove ~50 debug console.log statements from sie-import.ts
- Fix comment referencing account 3740 → 3741

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>

* feat: comprehensive UI design audit and normalization

Dashboard audit:
- Fix muted-foreground contrast (4.31:1 → 5.08:1) for WCAG AA
- Add prefers-reduced-motion media query for all animations
- Replace border-l-2 accent anti-pattern with subtle full-border colors
- Add aria-expanded to toggle buttons, role="status" to live counters
- Fix touch targets on deadline buttons (28px → 36px)
- Vary section spacing for rhythm (mb-12/mb-10/mb-8)
- Remove unused imports and dead code

Transactions audit + hardening:
- Add pagination (200 per page) with "Ladda fler" button
- Replace height animation with transform-only exit animation
- Show batch progress in floating action bar during processing
- Fix batch bar mobile overlap (bottom-20 on mobile)
- Replace clickable badges with proper button elements
- Add safe area padding to fullscreen swipe view
- Add response.ok check to suggestion fetch
- Add truncation to invoice number buttons

Invoicing audit:
- Remove border-l-4 accent pattern from invoice cards
- Replace string concatenation with cn() utility

Systemic sweep (34 files):
- All page headings: font-bold → font-display font-medium (Fraunces)
- All stat numbers: font-bold → font-display font-medium tabular-nums
- All hard-coded blue/amber/emerald colors → design tokens
- Remove all dark mode overrides (tokens handle automatically)
- Tint pure white card background to 99%

Design context added to CLAUDE.md with brand personality,
aesthetic direction, and 5 design principles.

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>

* fix: bookkeeping flow audit — design system, accessibility, UX

- Replace raw <select> with shadcn Select component (JournalEntryForm)
- Add confirmation dialog for account deletion (ChartOfAccountsManager)
- Remove console.error from production code (JournalEntryList, JournalEntryForm)
- Fix contradictory h-7/min-h-[44px] button sizing → h-10 (ChartOfAccountsManager)
- Increase BAS catalog "Lägg till" touch target h-7 → h-9
- Improve loading state with spinner (JournalEntryList)
- Improve empty state with icon, description, and guidance (JournalEntryList)
- Add response.ok check on journal entry fetch
- Add aria-expanded to entry expand buttons
- Add tabular-nums to desktop debit/credit columns

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>

* fix: onboarding and empty state improvements

Onboarding:
- Replace font-serif with font-display (Fraunces) for brand consistency
- Remove console.error calls from production code

Empty states:
- Fix broken /transactions/new link in EmptyTransactions (route doesn't exist)
- Add actionHref fallback to EmptyCustomers when no onAction prop provided
- Improve EmptyTransactions description copy

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>

* fix: clarify Swedish UX copy — terminology, errors, descriptions

Terminology consistency:
- "Försenad" → "Förfallen" for overdue invoices (customers/[id])
- "bokföringsorder" → actionable description in bookkeeping page
- "verifikation har bifogats" → "underlag har bifogats" in doc warning
- "Fortsätt ändå" → "Bokför utan underlag" (specific action)

Error messages — replace generic "Fel" + "Något gick fel" with specific:
- "Något gick fel vid bokföring" → "Transaktionen kunde inte bokföras"
- "Något gick fel vid matchning" → "Transaktionen kunde inte matchas"
- "Kunde inte hämta X" → "Kunde inte ladda X" + recovery hint
- Add "Försök igen" guidance to all error toasts

Page descriptions — replace redundant with actionable:
- Invoices: "Skapa och hantera" → "Skicka, följ betalningar, skapa kreditnotor"
- Bookkeeping: list of features → actionable description

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>

* fix: design critique — dashboard affordance, reports description

Dashboard:
- Add ChevronRight indicator to clickable summary cards
  (Att få betalt, Koppla bank) to distinguish from static cards
- Add cursor-pointer to linked cards

Reports:
- Replace feature list description with actionable guidance
  "Huvudbok, grundbok..." → "Generera skattedeklarationer..."

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>

* fix: replace generic "Fel" error toasts with specific messages

Deadlines: 5 generic "Fel" → specific per-action titles
  (create, toggle, edit, delete, load)
Expenses detail: 5 generic "Fel" → specific per-action titles
  (load, approve, pay, credit, delete)
Expenses new: 3 generic "Fel" → instructional validation messages
  (supplier name, supplier selection, invoice number)
Customers: 1 generic "Fel" → specific load error with recovery hint

All error toasts now follow pattern:
  title = what failed, description = how to recover

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>

* fix: replace all remaining generic "Fel" error toasts (37 instances)

Systematic sweep across 12 dashboard pages replacing generic
title: 'Fel' with context-specific error titles:

- Load errors: "Kunde inte ladda [resurs]"
- Action errors: "[Åtgärd] misslyckades"
- Validation: "[Fält] saknas"

Every error toast now tells the user what failed without needing
to read the description. Recovery hints added where missing.

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>

* fix: import flow — normalize stat typography, remove console.warn

- Replace font-bold with font-display font-medium on 13 stat numbers
  across SIEPreviewStep, BankFilePreviewStep, BankFileConfirmStep,
  ImportResultStep (missed by systemic sweep since these are in
  components/import/, not app/(dashboard)/)
- Add tabular-nums to stat numbers displaying counts/currency
- Remove console.warn in ArcimMigrationWorkspace

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>

* fix: final cleanup — console statements, remaining font-bold stats

Remove production console statements:
- Step1EntityType: remove debug console.warn (dead code after onNext)
- TransactionBookingDialog: remove console.error on doc link failure
- JournalEntryAttachments: remove 3 console.error calls

Normalize remaining font-bold stat displays:
- SwipeCategorizationView: 3 instances (completion, amount displays)
- NEDeclarationView: yearly result heading + value

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>

* fix: address Greptile review feedback

loadMoreTransactions: add inbox item enrichment matching fetchTransactions
- Paginated transactions now fetch invoice_inbox_items in parallel
- Fixes missing document indicator, template suggestions, and inbox
  match card for transactions loaded via "Ladda fler"

fetchAllPages: add maxPages guard (default 500) to prevent infinite loop
- If Arcim gateway returns hasMore:true indefinitely, the loop now
  exits after 500 pages instead of running forever

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>

* docs: minimize CLAUDE.md — remove derivable content, fix stale data

Remove ~230 lines (51% reduction) of content that duplicates what's
already in the source code (directory tree, function tables, type
definitions, migration lists). Update migration count (63→65), add
missing test helpers, fix cron job list. Keep all high-value sections:
accounting guard rails, BAS accounts, VAT rutor, design context.

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>

* feat: enable banking hardening, arcim entity inference, SIE import fixes, and onboarding improvements

- Enable Banking: OAuth CSRF state tokens, JWT caching, retry with timeouts, raw PSD2 response archival (BFL 7 kap), expired/error connection UI, consent expiry notifications, pagination safety limits
- Arcim migration: Smarter entity type inference from org numbers, VAT prefixes, company name suffixes (GmbH, Ltd, etc.), and country codes
- SIE import: Parser and import fixes with new migration
- BAS accounts: Added vehicle accounts (1241, 1242, 1249, 1259)
- Dashboard: New SIE import and stale uncategorized transaction queries
- Onboarding: Enhanced NewUserChecklist
- Period service: Improvements with updated tests
- Transaction ingest: Updated logic and tests

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>

* fix: address Greptile review — credit note type, EU country codes, notification thresholds, migration timestamps

- Fix dead ternary: credit notes now correctly stored as 'credit_note' instead of 'invoice'
- Add 'GR' (Greece ISO 3166-1) to EU_COUNTRIES alongside 'EL' (VAT prefix)
- Fix consent notification condition: fire at exactly 7 days or ≤3 days, not every day in 7-day window
- Deduplicate migration timestamps: rename SIE migration to 20260316120100

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>

---------

Co-authored-by: Claude Opus 4.6 <noreply@anthropic.com>
2026-03-16 14:21:32 +01:00

453 lines
17 KiB
TypeScript

/**
* Migration orchestrator — coordinates the data migration from
* an external accounting system via Arcim Sync into gnubok.
*
* Bookkeeping data (accounts, balances, vouchers) is now imported
* via SIE files through the core SIE import engine. This orchestrator
* handles only entity-level imports:
* 1. Company info → pre-fill company_settings
* 2. Customers → needed before sales invoices
* 3. Suppliers → needed before supplier invoices
* 4. Sales invoices (open only)
* 5. Supplier invoices (open only)
*/
import type { SupabaseClient } from '@supabase/supabase-js'
import type { MigrationProgress, MigrationResults } from '../types'
import {
fetchCompanyInfo,
fetchCustomers,
fetchSuppliers,
fetchSalesInvoices,
fetchSupplierInvoices,
} from './arcim-client'
import {
mapCustomer,
mapSupplier,
mapSalesInvoice,
mapSupplierInvoice,
mapCompanyInfo,
inferTypeFromParty,
} from './entity-mapper'
export interface MigrationOptions {
consentId: string
userId: string
supabase: SupabaseClient
importCompanyInfo?: boolean
importCustomers?: boolean
importSuppliers?: boolean
importSalesInvoices?: boolean
importSupplierInvoices?: boolean
onProgress?: (progress: MigrationProgress) => void
}
function emitProgress(options: MigrationOptions, progress: MigrationProgress) {
options.onProgress?.(progress)
}
// ── Main orchestrator ─────────────────────────────────────────────
export async function executeMigration(options: MigrationOptions): Promise<MigrationResults> {
const { consentId, userId, supabase } = options
const results: MigrationResults = {}
try {
// ── Step 1: Company information ───────────────────────────────
if (options.importCompanyInfo !== false) {
emitProgress(options, { status: 'fetching', currentStep: 'Hämtar företagsinformation...', progress: 5 })
try {
const companyInfo = await fetchCompanyInfo(consentId)
if (companyInfo) {
const mapped = mapCompanyInfo(companyInfo)
const { data: existing } = await supabase
.from('company_settings')
.select('company_name, org_number, vat_number')
.eq('user_id', userId)
.single()
const updates: Record<string, unknown> = {}
if (!existing?.company_name && mapped.company_name) updates.company_name = mapped.company_name
if (!existing?.org_number && mapped.org_number) updates.org_number = mapped.org_number
if (!existing?.vat_number && mapped.vat_number) {
updates.vat_number = mapped.vat_number
updates.vat_registered = true
}
if (mapped.fiscal_year_start_month !== 1) {
updates.fiscal_year_start_month = mapped.fiscal_year_start_month
}
if (mapped.address_line1) updates.address_line1 = mapped.address_line1
if (mapped.postal_code) updates.postal_code = mapped.postal_code
if (mapped.city) updates.city = mapped.city
if (mapped.phone) updates.phone = mapped.phone
if (mapped.email) updates.email = mapped.email
if (Object.keys(updates).length > 0) {
await supabase.from('company_settings').update(updates).eq('user_id', userId)
}
results.companyInfo = { imported: true }
}
} catch (err) {
console.error('Failed to import company info:', err)
results.companyInfo = { imported: false }
}
}
// ── Step 2: Customers ─────────────────────────────────────────
const customerIdMap = new Map<string, string>()
if (options.importCustomers !== false) {
emitProgress(options, { status: 'importing', currentStep: 'Importerar kunder...', progress: 20 })
try {
const customers = await fetchCustomers(consentId)
let imported = 0
let skipped = 0
for (const customer of customers) {
if (!customer.active) {
console.log(`[migration] Customer skipped (inactive): ${customer.party.name}`)
skipped++
continue
}
const orgNumber = customer.party.legalEntity?.companyId ||
customer.party.identifications?.find(i => i.schemeId === 'SE:ORGNR')?.id
if (orgNumber) {
const { data: existing } = await supabase
.from('customers')
.select('id')
.eq('user_id', userId)
.eq('org_number', orgNumber)
.limit(1)
if (existing && existing.length > 0) {
console.log(`[migration] Customer skipped (duplicate org_number ${orgNumber}): ${customer.party.name}`)
customerIdMap.set(customer.id, existing[0].id)
skipped++
continue
}
}
const mapped = mapCustomer(customer, userId)
const { data: inserted, error } = await supabase
.from('customers')
.insert(mapped)
.select('id')
.single()
if (error || !inserted) {
console.error(`[migration] Customer insert failed: ${customer.party.name}`, error?.message)
skipped++
} else {
customerIdMap.set(customer.id, inserted.id)
imported++
}
}
results.customers = { total: customers.length, imported, skipped }
} catch (err) {
console.error('Failed to import customers:', err)
}
}
// ── Step 3: Suppliers ─────────────────────────────────────────
const supplierIdMap = new Map<string, string>()
if (options.importSuppliers !== false) {
emitProgress(options, { status: 'importing', currentStep: 'Importerar leverantörer...', progress: 40 })
try {
const suppliers = await fetchSuppliers(consentId)
let imported = 0
let skipped = 0
for (const supplier of suppliers) {
if (!supplier.active) {
console.log(`[migration] Supplier skipped (inactive): ${supplier.party.name}`)
skipped++
continue
}
const orgNumber = supplier.party.legalEntity?.companyId ||
supplier.party.identifications?.find(i => i.schemeId === 'SE:ORGNR')?.id
if (orgNumber) {
const { data: existing } = await supabase
.from('suppliers')
.select('id')
.eq('user_id', userId)
.eq('org_number', orgNumber)
.limit(1)
if (existing && existing.length > 0) {
console.log(`[migration] Supplier skipped (duplicate org_number ${orgNumber}): ${supplier.party.name}`)
supplierIdMap.set(supplier.id, existing[0].id)
skipped++
continue
}
}
const mapped = mapSupplier(supplier, userId)
const { data: inserted, error } = await supabase
.from('suppliers')
.insert(mapped)
.select('id')
.single()
if (error || !inserted) {
console.error(`[migration] Supplier insert failed: ${supplier.party.name}`, error?.message)
skipped++
} else {
supplierIdMap.set(supplier.id, inserted.id)
imported++
}
}
results.suppliers = { total: suppliers.length, imported, skipped }
} catch (err) {
console.error('Failed to import suppliers:', err)
}
}
// ── Step 4: Sales invoices (open/unpaid only) ─────────────────
if (options.importSalesInvoices !== false) {
emitProgress(options, { status: 'importing', currentStep: 'Importerar kundfakturor...', progress: 60 })
try {
const invoices = await fetchSalesInvoices(consentId)
const openInvoices = invoices.filter(i =>
i.status === 'sent' || i.status === 'overdue' || i.status === 'booked'
)
console.log(`[migration] Sales invoices: ${invoices.length} total, ${openInvoices.length} open (filtered by status: sent/overdue/booked)`)
let imported = 0
let skipped = 0
for (const inv of openInvoices) {
const customerOrgNumber = inv.customer.legalEntity?.companyId ||
inv.customer.identifications?.find(i => i.schemeId === 'SE:ORGNR')?.id
let customerId: string | null = null
if (customerOrgNumber) {
const { data: match } = await supabase
.from('customers')
.select('id')
.eq('user_id', userId)
.eq('org_number', customerOrgNumber)
.limit(1)
if (match?.[0]) customerId = match[0].id
}
if (!customerId) {
const { data: match } = await supabase
.from('customers')
.select('id')
.eq('user_id', userId)
.eq('name', inv.customer.name)
.limit(1)
if (match?.[0]) customerId = match[0].id
}
if (!customerId) {
const customerType = inferTypeFromParty(inv.customer)
const minimalCustomer = {
user_id: userId,
name: inv.customer.name,
customer_type: customerType,
default_payment_terms: 30,
country: inv.customer.postalAddress?.countryCode || (customerType === 'swedish_business' ? 'SE' : null),
vat_number_validated: false,
}
const { data: created, error: custErr } = await supabase
.from('customers')
.insert(minimalCustomer)
.select('id')
.single()
if (created) {
customerId = created.id
} else {
console.error(`[migration] Sales invoice ${inv.invoiceNumber} skipped — could not create customer "${inv.customer.name}":`, custErr?.message)
}
}
if (!customerId) {
console.log(`[migration] Sales invoice ${inv.invoiceNumber} skipped — no customer match for "${inv.customer.name}" (org: ${customerOrgNumber || 'n/a'})`)
skipped++
continue
}
const { data: existingInv } = await supabase
.from('invoices')
.select('id')
.eq('user_id', userId)
.eq('invoice_number', inv.invoiceNumber)
.limit(1)
if (existingInv && existingInv.length > 0) {
console.log(`[migration] Sales invoice ${inv.invoiceNumber} skipped — already exists`)
skipped++
continue
}
const { invoice: mappedInvoice, items: mappedItems } = mapSalesInvoice(inv, userId, customerId)
const { data: insertedInv, error: invError } = await supabase
.from('invoices')
.insert(mappedInvoice)
.select('id')
.single()
if (invError || !insertedInv) {
console.error(`[migration] Sales invoice ${inv.invoiceNumber} insert failed:`, invError?.message)
skipped++
continue
}
if (mappedItems.length > 0) {
const itemsWithInvoiceId = mappedItems.map(item => ({
...item,
invoice_id: insertedInv.id,
}))
await supabase.from('invoice_items').insert(itemsWithInvoiceId)
}
imported++
}
results.salesInvoices = { total: openInvoices.length, imported, skipped }
} catch (err) {
console.error('Failed to import sales invoices:', err)
}
}
// ── Step 5: Supplier invoices (open/unpaid only) ──────────────
if (options.importSupplierInvoices !== false) {
emitProgress(options, { status: 'importing', currentStep: 'Importerar leverantörsfakturor...', progress: 80 })
try {
const invoices = await fetchSupplierInvoices(consentId)
const openInvoices = invoices.filter(i =>
i.status === 'sent' || i.status === 'overdue' || i.status === 'booked' || i.status === 'draft'
)
console.log(`[migration] Supplier invoices: ${invoices.length} total, ${openInvoices.length} open (filtered by status: sent/overdue/booked/draft)`)
let imported = 0
let skipped = 0
for (const inv of openInvoices) {
const supplierOrgNumber = inv.supplier.legalEntity?.companyId ||
inv.supplier.identifications?.find(i => i.schemeId === 'SE:ORGNR')?.id
let supplierId: string | null = null
if (supplierOrgNumber) {
const { data: match } = await supabase
.from('suppliers')
.select('id')
.eq('user_id', userId)
.eq('org_number', supplierOrgNumber)
.limit(1)
if (match?.[0]) supplierId = match[0].id
}
if (!supplierId) {
const { data: match } = await supabase
.from('suppliers')
.select('id')
.eq('user_id', userId)
.eq('name', inv.supplier.name)
.limit(1)
if (match?.[0]) supplierId = match[0].id
}
if (!supplierId) {
const supplierType = inferTypeFromParty(inv.supplier)
const minimalSupplier = {
user_id: userId,
name: inv.supplier.name,
supplier_type: supplierType,
default_payment_terms: 30,
default_currency: 'SEK',
country: inv.supplier.postalAddress?.countryCode || (supplierType === 'swedish_business' ? 'SE' : null),
}
const { data: created, error: supErr } = await supabase
.from('suppliers')
.insert(minimalSupplier)
.select('id')
.single()
if (created) {
supplierId = created.id
} else {
console.error(`[migration] Supplier invoice ${inv.invoiceNumber} skipped — could not create supplier "${inv.supplier.name}":`, supErr?.message)
}
}
if (!supplierId) {
console.log(`[migration] Supplier invoice ${inv.invoiceNumber} skipped — no supplier match for "${inv.supplier.name}" (org: ${supplierOrgNumber || 'n/a'})`)
skipped++
continue
}
const { data: existingInv } = await supabase
.from('supplier_invoices')
.select('id')
.eq('user_id', userId)
.eq('supplier_invoice_number', inv.invoiceNumber)
.eq('supplier_id', supplierId)
.limit(1)
if (existingInv && existingInv.length > 0) {
console.log(`[migration] Supplier invoice ${inv.invoiceNumber} skipped — already exists for supplier "${inv.supplier.name}"`)
skipped++
continue
}
const { invoice: mappedInvoice, items: mappedItems } = mapSupplierInvoice(inv, userId, supplierId)
// Get next arrival number (ankomstnummer) — required NOT NULL column
const { data: arrivalNum, error: arrivalError } = await supabase
.rpc('get_next_arrival_number', { p_user_id: userId })
if (arrivalError || arrivalNum == null) {
console.error(`[migration] Supplier invoice ${inv.invoiceNumber} skipped — could not get arrival number:`, arrivalError?.message)
skipped++
continue
}
mappedInvoice.arrival_number = arrivalNum
const { data: insertedInv, error: invError } = await supabase
.from('supplier_invoices')
.insert(mappedInvoice)
.select('id')
.single()
if (invError || !insertedInv) {
console.error(`[migration] Supplier invoice ${inv.invoiceNumber} insert failed for "${inv.supplier.name}":`, invError?.message, JSON.stringify(mappedInvoice, null, 2))
skipped++
continue
}
if (mappedItems.length > 0) {
const itemsWithInvoiceId = mappedItems.map(item => ({
...item,
supplier_invoice_id: insertedInv.id,
}))
await supabase.from('supplier_invoice_items').insert(itemsWithInvoiceId)
}
imported++
}
results.supplierInvoices = { total: openInvoices.length, imported, skipped }
} catch (err) {
console.error('Failed to import supplier invoices:', err)
}
}
emitProgress(options, { status: 'completed', progress: 100, results })
return results
} catch (error) {
const message = error instanceof Error ? error.message : 'Migration failed'
emitProgress(options, { status: 'failed', progress: 0, error: message })
throw error
}
}