* refactor: remove unnecessary secondary action from EmptyInvoices component

* feat: add direct provider layer and provider_consents migration

Replace Arcim Sync gateway dependency with direct provider clients
for Fortnox, Visma, Briox, Bokio, and Björn Lundén. Adds OAuth
config, rate limiting, retry logic, data fetching, and consent
storage via new provider_consents/tokens/otc tables.

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>

* refactor: migrate arcim extension to direct provider APIs

Replace Arcim Sync gateway calls with direct provider API access.
Use FortnoxClient.getText() for SIE endpoints that return plain text
instead of JSON. OAuth callback now returns HTML with postMessage
to communicate with the opener window instead of redirecting.

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>

* fix: use OAuth popup window instead of new tab

Open provider login in a centered popup that auto-closes on
completion via postMessage, keeping the user on a single tab.
Falls back to redirect flow if popup is blocked.

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>

* feat: add connection status, consent reuse, and SIE duplicate detection

- Add listConsents() to query active consents by company
- Add GET /status route returning consents, SIE import history, and
  entity counts
- /connect reuses existing accepted consent instead of creating
  duplicates, and cleans up abandoned (status 0) consents
- /status only returns accepted (status 1) consents
- /sie-data checks each file's SHA-256 hash against sie_imports to
  report per-file import status (alreadyImported, importedAt)
- /sie-data blocks on SIE validation failure (mirrors manual upload)
- /import-sie validates unmapped accounts and auto-activates missing
  BAS accounts in chart_of_accounts (mirrors manual upload)

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>

* feat: show active connections, SIE file status, and smart re-sync UI

- ProviderStep shows active connections with last import date, entity
  counts, "Synka igen" button, and disconnect option
- Already-connected providers greyed out in selection grid
- OptionsStep shows per-fiscal-year import status (imported vs new)
- SIE toggle disabled with explanation when all files already imported
- handleStartMigration skips already-imported SIE files
- Auto-skip mapping step and disable SIE on re-sync when up to date
- Result step hides empty "0 importerade" rows and shows
  "Allt är uppdaterat" when nothing new was fetched
- OptionRow supports disabled state

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>

* fix: adjust COMING_SOON_PROVIDERS based on NODE_ENV for development and production

* Removed duplicate

* Removed duplicate

* refactor: redesign reports page navigation from grid boxes to bordered card layout

Replace the 4-column grid of uneven TabsList boxes with a CSS grid card
using auto-sized columns separated by 1px border dividers. All sections
now share equal height via items-stretch, with clear visual separation
between groups.

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>

* chore: trigger Vercel deployment

* Update supabase/migrations/20260402010000_provider_consents.sql

Co-authored-by: greptile-apps[bot] <165735046+greptile-apps[bot]@users.noreply.github.com>

* Update lib/providers/rate-limiter.ts

Co-authored-by: greptile-apps[bot] <165735046+greptile-apps[bot]@users.noreply.github.com>

* chore: re-trigger checks after migration sync

---------

Co-authored-by: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
Co-authored-by: greptile-apps[bot] <165735046+greptile-apps[bot]@users.noreply.github.com>
This commit is contained in:
Mattsson
2026-04-07 10:30:20 +02:00
committed by GitHub
co-authored by Claude Opus 4.6 greptile-apps[bot]
parent e2d9c8bb85
commit e42da5c32b
41 changed files with 5219 additions and 488 deletions
+392 -90
View File
@@ -3,45 +3,47 @@ import { NextResponse } from 'next/server'
import {
createConsent,
getConsent,
listConsents,
generateOtc,
getAuthUrl,
exchangeAuthToken,
submitProviderToken,
deleteConsent,
fetchCompanyInfo,
fetchSIEExport,
} from './lib/arcim-client'
resolveConsent,
fetchCompanyInfoDirect,
} from './lib/provider-client'
import { mapCompanyInfo } from './lib/entity-mapper'
import { executeMigration } from './lib/migration-orchestrator'
import type { ArcimProvider } from './types'
import { ARCIM_PROVIDERS } from './types'
import { parseSIEFile, validateSIEFile } from '@/lib/import/sie-parser'
import { parseSIEFile, validateSIEFile, calculateFileHash } from '@/lib/import/sie-parser'
import { suggestMappings, getMappingStats, isSystemAccount } from '@/lib/import/account-mapper'
import { loadMappings, generateImportPreview, executeSIEImport, saveMappings } from '@/lib/import/sie-import'
import { BAS_REFERENCE } from '@/lib/bookkeeping/bas-reference'
import { BAS_REFERENCE, getBASReference } from '@/lib/bookkeeping/bas-reference'
import { fetchAllRows } from '@/lib/supabase/fetch-all'
import { FortnoxClient } from '@/lib/providers/fortnox/client'
import type { ProviderName } from '@/lib/providers/types'
/** Fiscal years we support importing — older data is not needed */
const ALLOWED_FISCAL_YEARS = new Set([2024, 2025, 2026])
const fortnoxClient = new FortnoxClient()
/**
* Arcim Migration extension
* Provider Migration extension
*
* Migrates bookkeeping data from external Swedish accounting systems
* (Fortnox, Visma, Bokio, Björn Lundén, Briox) into gnubok via
* the Arcim Sync unified API gateway.
* (Fortnox, Visma, Bokio, Björn Lundén, Briox) into gnubok by talking
* directly to each provider's API.
*
* Bookkeeping data (accounts, balances, vouchers) is imported via SIE
* files fetched from the gateway. Entity data (customers, suppliers,
* invoices) is imported via the REST API.
*
* Required environment variables:
* - ARCIM_SYNC_GATEWAY_URL
* - ARCIM_SYNC_API_KEY
* files fetched from providers. Entity data (customers, suppliers,
* invoices) is imported via the provider REST APIs.
*/
export const arcimMigrationExtension: Extension = {
id: 'arcim-migration',
name: 'Systemmigration (Arcim Sync)',
version: '1.0.0',
name: 'Systemmigration',
version: '2.0.0',
apiRoutes: [
// ── List available providers ───────────────────────────────────
@@ -53,6 +55,68 @@ export const arcimMigrationExtension: Extension = {
},
},
// ── Check existing connections and import history ──────────────
{
method: 'GET',
path: '/status',
handler: async (request: Request, ctx?: ExtensionContext) => {
const supabase = ctx?.supabase ?? await (await import('@/lib/supabase/server')).createClient()
const { data: { user } } = await supabase.auth.getUser()
if (!user) {
return NextResponse.json({ error: 'Unauthorized' }, { status: 401 })
}
const companyId = ctx?.companyId ?? user.id
try {
// Get accepted consents only (status 1) — not abandoned/created ones
const allConsents = await listConsents(companyId)
const consents = allConsents.filter(c => c.status === 1)
// Get SIE import history
const { data: sieImports } = await supabase
.from('sie_imports')
.select('id, filename, status, accounts_count, transactions_count, company_name, fiscal_year_start, fiscal_year_end, imported_at, created_at')
.eq('company_id', companyId)
.order('created_at', { ascending: false })
.limit(10)
// Get entity counts (to show what's already been imported)
const [
{ count: customerCount },
{ count: supplierCount },
{ count: invoiceCount },
] = await Promise.all([
supabase.from('customers').select('*', { count: 'exact', head: true }).eq('company_id', companyId),
supabase.from('suppliers').select('*', { count: 'exact', head: true }).eq('company_id', companyId),
supabase.from('invoices').select('*', { count: 'exact', head: true }).eq('company_id', companyId),
])
return NextResponse.json({
consents: consents.map(c => ({
id: c.id,
provider: c.provider,
status: c.status,
companyName: c.companyName,
createdAt: c.createdAt,
})),
sieImports: sieImports ?? [],
entityCounts: {
customers: customerCount ?? 0,
suppliers: supplierCount ?? 0,
invoices: invoiceCount ?? 0,
},
})
} catch (error) {
return NextResponse.json(
{ error: error instanceof Error ? error.message : 'Failed to fetch status' },
{ status: 500 }
)
}
},
},
// ── Start consent flow (create consent + OTC) ─────────────────
{
method: 'POST',
@@ -66,6 +130,8 @@ export const arcimMigrationExtension: Extension = {
return NextResponse.json({ error: 'Unauthorized' }, { status: 401 })
}
const companyId = ctx?.companyId ?? user.id
const { provider, companyName, orgNumber } = await request.json() as {
provider: ArcimProvider
companyName?: string
@@ -82,15 +148,39 @@ export const arcimMigrationExtension: Extension = {
}
try {
// Create consent in Arcim Sync
// Reuse existing accepted consent if one exists for this provider
const existingConsents = await listConsents(companyId)
const existing = existingConsents.find(c => c.provider === provider && c.status === 1)
if (existing) {
// Already connected — skip OAuth, go straight to preview
if (ctx?.settings) {
await ctx.settings.set('consent_id', existing.id)
await ctx.settings.set('provider', provider)
}
return NextResponse.json({
consentId: existing.id,
authType: providerInfo.authType,
alreadyConnected: true,
})
}
// Clean up abandoned consents (status 0 = Created but never completed OAuth)
const abandoned = existingConsents.filter(c => c.provider === provider && c.status === 0)
for (const a of abandoned) {
await deleteConsent(a.id)
}
// Create new consent
const consent = await createConsent(
companyId,
provider,
`gnubok-migration-${user.id}`,
orgNumber,
companyName
)
// Store consent ID in extension settings for this user
if (ctx?.settings) {
await ctx.settings.set('consent_id', consent.id)
await ctx.settings.set('provider', provider)
@@ -100,15 +190,14 @@ export const arcimMigrationExtension: Extension = {
// Generate OTC for OAuth flow
const otc = await generateOtc(consent.id)
// Build the OAuth callback URL using the current app URL (localhost in dev, production in prod)
// Build the OAuth callback URL
const appUrl = process.env.NEXT_PUBLIC_APP_URL || ''
const callbackUrl = `${appUrl}/api/extensions/ext/arcim-migration/callback`
// Encode consentId + provider in state so the callback doesn't depend on session storage
// Encode consentId + provider in state
const statePayload = JSON.stringify({ otc: otc.code, consentId: consent.id, provider })
const stateEncoded = Buffer.from(statePayload).toString('base64url')
// Pass callbackUrl as redirectUri so Fortnox redirects here (works for localhost and production)
const { url } = await getAuthUrl(provider, stateEncoded, callbackUrl)
return NextResponse.json({
@@ -162,7 +251,6 @@ export const arcimMigrationExtension: Extension = {
}
// BL uses server-side client credentials — only needs companyId
// Bokio and Briox need an API token
if (provider !== 'bjornlunden' && !apiToken) {
return NextResponse.json(
{ error: 'apiToken is required for this provider' },
@@ -170,7 +258,6 @@ export const arcimMigrationExtension: Extension = {
)
}
// Bokio and BL require companyId
if ((provider === 'bokio' || provider === 'bjornlunden') && !companyId) {
return NextResponse.json(
{ error: 'companyId is required for this provider' },
@@ -192,10 +279,6 @@ export const arcimMigrationExtension: Extension = {
},
// ── OAuth callback ────────────────────────────────────────────
// This handler is called by the OAuth provider redirect. It does NOT
// require user auth — the request comes from the provider, not the user's
// browser session. Authentication is validated via the OTC code + consent.
// The 'skipAuth' flag is checked by the extension dispatch route.
{
method: 'GET',
path: '/callback',
@@ -211,24 +294,19 @@ export const arcimMigrationExtension: Extension = {
}
try {
// Decode state — supports both new format (base64url JSON with consentId/provider)
// and legacy format (plain OTC code string)
let consentId: string | null = null
let provider: ArcimProvider | null = null
let otcCode: string = stateRaw
try {
const decoded = JSON.parse(Buffer.from(stateRaw, 'base64url').toString())
if (decoded.consentId && decoded.provider && decoded.otc) {
if (decoded.consentId && decoded.provider) {
consentId = decoded.consentId
provider = decoded.provider as ArcimProvider
otcCode = decoded.otc
}
} catch {
// Legacy: state is just the OTC code — fall back to ctx.settings
// Legacy fallback
}
// Fall back to session-based settings if state didn't contain the data
if (!consentId || !provider) {
consentId = ctx?.settings
? await ctx.settings.get<string>('consent_id')
@@ -242,18 +320,43 @@ export const arcimMigrationExtension: Extension = {
return NextResponse.json({ error: 'No active migration session' }, { status: 400 })
}
// The redirectUri used for the token exchange must match the one used in the auth URL
const appUrl = process.env.NEXT_PUBLIC_APP_URL || ''
const redirectUri = `${appUrl}/api/extensions/ext/arcim-migration/callback`
await exchangeAuthToken(consentId, provider, otcCode, code, redirectUri)
// Exchange OAuth code directly with the provider
await exchangeAuthToken(consentId, provider, code, redirectUri)
// Redirect to import page with success
return NextResponse.redirect(`${appUrl}/import?migration=connected&consentId=${consentId}`)
// Return an HTML page that notifies the opener tab and closes itself
const html = `<!DOCTYPE html><html><body><script>
if (window.opener) {
window.opener.postMessage({ type: 'arcim-oauth-success', consentId: '${consentId}' }, '${appUrl}');
window.close();
} else {
window.location.href = '${appUrl}/import?migration=connected&consentId=${consentId}';
}
</script><p>Anslutningen lyckades. Du kan stänga denna flik.</p></body></html>`
return new Response(html, {
status: 200,
headers: { 'Content-Type': 'text/html' },
})
} catch (error) {
log.error('OAuth callback error:', error)
const appUrl = process.env.NEXT_PUBLIC_APP_URL || ''
return NextResponse.redirect(`${appUrl}/import?migration=error`)
const html = `<!DOCTYPE html><html><body><script>
if (window.opener) {
window.opener.postMessage({ type: 'arcim-oauth-error' }, '${appUrl}');
window.close();
} else {
window.location.href = '${appUrl}/import?migration=error';
}
</script><p>Något gick fel. Du kan stänga denna flik.</p></body></html>`
return new Response(html, {
status: 200,
headers: { 'Content-Type': 'text/html' },
})
}
},
},
@@ -271,6 +374,8 @@ export const arcimMigrationExtension: Extension = {
return NextResponse.json({ error: 'Unauthorized' }, { status: 401 })
}
const companyId = ctx?.companyId ?? user.id
const url = new URL(request.url)
const consentId = url.searchParams.get('consentId')
@@ -279,7 +384,6 @@ export const arcimMigrationExtension: Extension = {
}
try {
// Verify consent is accepted
const consent = await getConsent(consentId)
if (consent.status !== 1) {
return NextResponse.json(
@@ -288,45 +392,64 @@ export const arcimMigrationExtension: Extension = {
)
}
// Fetch company info for preview (non-blocking — continue if it fails)
// Resolve consent to get access token
const resolved = await resolveConsent(companyId, consentId)
const provider = resolved.consent.provider as ProviderName
// Fetch company info directly from provider
let mapped = null
try {
const companyInfo = await fetchCompanyInfo(consentId)
const companyInfo = await fetchCompanyInfoDirect(provider, resolved.accessToken, resolved.providerCompanyId)
mapped = companyInfo ? mapCompanyInfo(companyInfo) : null
} catch (err) {
log.info('Company info fetch failed:', err instanceof Error ? err.message : String(err))
}
// Try to fetch SIE stats (non-blocking — continue if it fails)
// Try to fetch SIE data (Fortnox has native SIE export)
let sieAvailable = false
let sieStats: { accountCount: number; transactionCount: number; fiscalYears: number[] } | null = null
try {
log.info(`Fetching SIE export for consent ${consentId}...`)
const sieResult = await fetchSIEExport(consentId, 4)
log.info(`SIE export response: ${sieResult.files.length} files returned`)
if (provider === 'fortnox') {
try {
log.info(`Fetching SIE export from Fortnox for consent ${consentId}...`)
// Fortnox SIE export endpoint: /3/sie/{type}?financialyear={id}
// First get financial years
const fyResponse = await fortnoxClient.get<Record<string, unknown>>(
resolved.accessToken,
'/financialyears'
)
const years = (fyResponse['FinancialYears'] as Record<string, unknown>[] | undefined) ?? []
const allowedYears = years
.map(fy => ({
id: fy['Id'] as number,
fromDate: fy['FromDate'] as string,
toDate: fy['ToDate'] as string,
}))
.filter(fy => {
const year = new Date(fy.fromDate).getFullYear()
return ALLOWED_FISCAL_YEARS.has(year)
})
// Only keep fiscal years we need (2024–2026)
const filteredFiles = sieResult.files.filter(f => ALLOWED_FISCAL_YEARS.has(f.fiscalYear))
if (filteredFiles.length < sieResult.files.length) {
const skippedYears = sieResult.files
.filter(f => !ALLOWED_FISCAL_YEARS.has(f.fiscalYear))
.map(f => f.fiscalYear)
log.info(`Filtered out fiscal years: ${skippedYears.join(', ')} (only importing ${[...ALLOWED_FISCAL_YEARS].join(', ')})`)
if (allowedYears.length > 0) {
// Fetch SIE type 4 for the most recent allowed year to get stats
const latestYear = allowedYears[allowedYears.length - 1]
const sieContent = await fortnoxClient.getText(
resolved.accessToken,
`/sie/4?financialyear=${latestYear.id}`
)
if (sieContent) {
const parsed = parseSIEFile(sieContent)
sieAvailable = true
sieStats = {
accountCount: parsed.accounts.length,
transactionCount: parsed.vouchers.length,
fiscalYears: allowedYears.map(fy => new Date(fy.fromDate).getFullYear()),
}
}
}
} catch (err) {
log.info('SIE export failed:', err instanceof Error ? err.message : String(err))
}
if (filteredFiles.length > 0) {
sieAvailable = true
const totalAccounts = Math.max(...filteredFiles.map(f => f.accountCount))
const totalTransactions = filteredFiles.reduce((sum, f) => sum + f.transactionCount, 0)
const fiscalYears = filteredFiles.map(f => f.fiscalYear).sort()
sieStats = { accountCount: totalAccounts, transactionCount: totalTransactions, fiscalYears }
log.info(`SIE stats: ${totalAccounts} accounts, ${totalTransactions} transactions, years: ${fiscalYears.join(', ')}`)
} else {
log.info('No SIE files within allowed fiscal years')
}
} catch (err) {
log.info('SIE export failed:', err instanceof Error ? err.message : String(err))
}
return NextResponse.json({
@@ -363,6 +486,8 @@ export const arcimMigrationExtension: Extension = {
return NextResponse.json({ error: 'Unauthorized' }, { status: 401 })
}
const companyId = ctx?.companyId ?? user.id
const url = new URL(request.url)
const consentId = url.searchParams.get('consentId')
@@ -371,22 +496,77 @@ export const arcimMigrationExtension: Extension = {
}
try {
// Fetch SIE from gateway and filter to allowed fiscal years (2024–2026)
const sieResult = await fetchSIEExport(consentId, 4)
const filteredFiles = sieResult.files.filter(f => ALLOWED_FISCAL_YEARS.has(f.fiscalYear))
if (filteredFiles.length === 0) {
// Resolve consent
const resolved = await resolveConsent(companyId, consentId)
const provider = resolved.consent.provider as ProviderName
if (provider !== 'fortnox') {
return NextResponse.json(
{ error: `SIE export is currently only supported for Fortnox. Provider: ${provider}` },
{ status: 400 }
)
}
// Fetch financial years from Fortnox
const fyResponse = await fortnoxClient.get<Record<string, unknown>>(
resolved.accessToken,
'/financialyears'
)
const years = (fyResponse['FinancialYears'] as Record<string, unknown>[] | undefined) ?? []
const allowedYears = years
.map(fy => ({
id: fy['Id'] as number,
fromDate: fy['FromDate'] as string,
toDate: fy['ToDate'] as string,
}))
.filter(fy => {
const year = new Date(fy.fromDate).getFullYear()
return ALLOWED_FISCAL_YEARS.has(year)
})
if (allowedYears.length === 0) {
return NextResponse.json({ error: 'No SIE data available for fiscal years 2024–2026' }, { status: 404 })
}
// Fetch SIE type 4 for each allowed year
const sieFiles: { fiscalYear: number; rawContent: string }[] = []
for (const fy of allowedYears) {
try {
const sieContent = await fortnoxClient.getText(
resolved.accessToken,
`/sie/4?financialyear=${fy.id}`
)
if (sieContent) {
sieFiles.push({
fiscalYear: new Date(fy.fromDate).getFullYear(),
rawContent: sieContent,
})
}
} catch (err) {
log.info(`Failed to fetch SIE for year ${fy.id}:`, err instanceof Error ? err.message : String(err))
}
}
if (sieFiles.length === 0) {
return NextResponse.json({ error: 'No SIE data available for fiscal years 2024–2026' }, { status: 404 })
}
// Parse most recent file for preview/validation
const sieFile = filteredFiles[filteredFiles.length - 1]
const sieFile = sieFiles[sieFiles.length - 1]
const parsed = parseSIEFile(sieFile.rawContent)
const validation = validateSIEFile(parsed)
if (!validation.valid) {
return NextResponse.json({
error: 'validation',
message: 'SIE file validation failed',
validation,
}, { status: 400 })
}
// Collect ALL unique accounts across ALL fiscal year files
// so mappings cover every account that will be imported
const allAccountsMap = new Map<string, { number: string; name: string }>()
for (const file of filteredFiles) {
for (const file of sieFiles) {
const fileParsed = parseSIEFile(file.rawContent)
for (const acc of fileParsed.accounts) {
if (!allAccountsMap.has(acc.number)) {
@@ -394,14 +574,12 @@ export const arcimMigrationExtension: Extension = {
}
}
}
// Filter out source-system internal accounts (e.g. Fortnox 0099)
// that have no BAS equivalent — same as core SIE import
const allAccounts = [...allAccountsMap.values()]
.filter(a => !isSystemAccount(a.number))
.map(a => ({ number: a.number, name: a.name }))
// Load existing user mappings
const existingMappings = await loadMappings(supabase, ctx?.companyId ?? user.id)
const existingMappings = await loadMappings(supabase, companyId)
const existingRecords = [...existingMappings.values()].map(m => ({
id: '',
user_id: user.id,
@@ -414,7 +592,7 @@ export const arcimMigrationExtension: Extension = {
updated_at: '',
}))
// Suggest mappings using accounts from ALL fiscal years
// Suggest mappings
const basAccounts = BAS_REFERENCE.map(b => ({
account_number: b.account_number,
account_name: b.account_name,
@@ -422,13 +600,32 @@ export const arcimMigrationExtension: Extension = {
const mappings = suggestMappings(allAccounts, basAccounts, existingRecords)
const mappingStats = getMappingStats(mappings)
log.info(`Account mapping: ${allAccounts.length} unique accounts across ${filteredFiles.length} files, ${mappingStats.unmapped} unmapped`)
log.info(`Account mapping: ${allAccounts.length} unique accounts across ${sieFiles.length} files, ${mappingStats.unmapped} unmapped`)
// Generate preview
const preview = generateImportPreview(parsed, mappings)
// Collect all raw SIE content (filtered fiscal years only)
const allRawContent = filteredFiles.map(f => f.rawContent)
// Check each file's hash against existing imports
const fileStatuses: { fiscalYear: number; rawContent: string; alreadyImported: boolean; importedAt: string | null }[] = []
for (const file of sieFiles) {
const fileHash = await calculateFileHash(file.rawContent)
const { data: existingImport } = await supabase
.from('sie_imports')
.select('imported_at')
.eq('company_id', companyId)
.eq('file_hash', fileHash)
.eq('status', 'completed')
.maybeSingle()
fileStatuses.push({
fiscalYear: file.fiscalYear,
rawContent: file.rawContent,
alreadyImported: !!existingImport,
importedAt: existingImport?.imported_at ?? null,
})
}
const allImported = fileStatuses.every(f => f.alreadyImported)
const newFiles = fileStatuses.filter(f => !f.alreadyImported)
return NextResponse.json({
parsed,
@@ -436,7 +633,14 @@ export const arcimMigrationExtension: Extension = {
mappingStats,
preview,
validation,
rawContent: allRawContent,
rawContent: fileStatuses.map(f => f.rawContent),
fileStatuses: fileStatuses.map(f => ({
fiscalYear: f.fiscalYear,
alreadyImported: f.alreadyImported,
importedAt: f.importedAt,
})),
allImported,
newFileCount: newFiles.length,
basAccounts: BAS_REFERENCE,
})
} catch (error) {
@@ -462,6 +666,8 @@ export const arcimMigrationExtension: Extension = {
return NextResponse.json({ error: 'Unauthorized' }, { status: 401 })
}
const companyId = ctx?.companyId ?? user.id
const { rawContent, mappings, options } = await request.json() as {
rawContent: string
mappings: import('@/lib/import/types').AccountMapping[]
@@ -478,14 +684,110 @@ export const arcimMigrationExtension: Extension = {
}
try {
// Parse the SIE content
const parsed = parseSIEFile(rawContent)
// Save the user's mappings for future use
// Validate all accounts are mapped (same as manual upload)
const unmapped = mappings.filter((m: import('@/lib/import/types').AccountMapping) => !m.targetAccount)
if (unmapped.length > 0) {
return NextResponse.json({
error: 'validation',
message: `${unmapped.length} account(s) are not mapped`,
unmappedAccounts: unmapped.map((m: import('@/lib/import/types').AccountMapping) => ({
account: m.sourceAccount,
name: m.sourceName,
})),
}, { status: 400 })
}
// Auto-activate mapped BAS accounts not yet in user's chart (same as manual upload)
const mappedAccountNumbers = [
...new Set(mappings.filter((m: import('@/lib/import/types').AccountMapping) => m.targetAccount).map((m: import('@/lib/import/types').AccountMapping) => m.targetAccount)),
]
const allCompanyAccounts = await fetchAllRows(({ from, to }) =>
supabase
.from('chart_of_accounts')
.select('account_number')
.eq('company_id', companyId)
.range(from, to)
)
const existingNumbers = new Set(allCompanyAccounts.map((a: { account_number: string }) => a.account_number))
const mappingNameLookup = new Map<string, string>()
for (const m of mappings) {
if (m.targetAccount) {
mappingNameLookup.set(m.targetAccount, m.targetName || m.sourceName)
}
}
const accountsToActivate = mappedAccountNumbers
.filter((num) => !existingNumbers.has(num))
.map((num) => {
const ref = getBASReference(num)
if (ref) {
return {
user_id: user.id,
company_id: companyId,
account_number: ref.account_number,
account_name: ref.account_name,
account_class: ref.account_class,
account_group: ref.account_group,
account_type: ref.account_type,
normal_balance: ref.normal_balance,
plan_type: 'full_bas' as const,
is_active: true,
is_system_account: false,
description: ref.description,
sru_code: ref.sru_code,
sort_order: parseInt(ref.account_number),
}
}
const accountClass = parseInt(num.charAt(0), 10)
const accountGroup = num.substring(0, 2)
const accountName = mappingNameLookup.get(num) || `Konto ${num}`
const accountType =
accountClass === 1 ? 'asset'
: accountClass === 2 ? 'liability'
: accountClass === 3 ? 'revenue'
: 'expense'
const normalBalance =
accountClass <= 1 || accountClass >= 4 ? 'debit' : 'credit'
return {
user_id: user.id,
company_id: companyId,
account_number: num,
account_name: accountName,
account_class: accountClass,
account_group: accountGroup,
account_type: accountType,
normal_balance: normalBalance,
plan_type: 'full_bas' as const,
is_active: true,
is_system_account: false,
description: accountName,
sru_code: null,
sort_order: parseInt(num),
}
})
if (accountsToActivate.length > 0) {
const { error: activateError } = await supabase
.from('chart_of_accounts')
.insert(accountsToActivate)
if (activateError) {
return NextResponse.json({
error: `Failed to activate accounts: ${activateError.message}`,
}, { status: 500 })
}
log.info(`Auto-activated ${accountsToActivate.length} accounts`)
}
await saveMappings(supabase, user.id, mappings)
// Execute the import via core engine
const result = await executeSIEImport(supabase, ctx?.companyId ?? user.id, user.id, parsed, mappings, {
const result = await executeSIEImport(supabase, companyId, user.id, parsed, mappings, {
filename: `migration-sie-${Date.now()}.se`,
fileContent: rawContent,
createFiscalPeriod: options.createFiscalPeriod,
@@ -525,6 +827,8 @@ export const arcimMigrationExtension: Extension = {
return NextResponse.json({ error: 'Unauthorized' }, { status: 401 })
}
const companyId = ctx?.companyId ?? user.id
const {
consentId,
importCompanyInfo = true,
@@ -546,7 +850,6 @@ export const arcimMigrationExtension: Extension = {
}
try {
// Verify consent
const consent = await getConsent(consentId)
if (consent.status !== 1) {
return NextResponse.json(
@@ -559,8 +862,8 @@ export const arcimMigrationExtension: Extension = {
const results = await executeMigration({
consentId,
companyId,
userId: user.id,
companyId: ctx?.companyId ?? user.id,
supabase,
importCompanyInfo,
importCustomers,
@@ -604,7 +907,6 @@ export const arcimMigrationExtension: Extension = {
try {
await deleteConsent(consentId)
// Clear stored consent from settings
if (ctx?.settings) {
await ctx.settings.set('consent_id', null)
await ctx.settings.set('provider', null)
@@ -16,7 +16,7 @@ import type {
CompanyInformationDto,
PostalAddress,
PartyDto,
} from '../types'
} from '@/lib/providers/dto'
// ── Helpers ─────────────────────────────────────────────────────────
@@ -1,8 +1,8 @@
/**
* Migration orchestrator — coordinates the data migration from
* an external accounting system via Arcim Sync into gnubok.
* an external accounting system directly via provider APIs into gnubok.
*
* Bookkeeping data (accounts, balances, vouchers) is now imported
* Bookkeeping data (accounts, balances, vouchers) is imported
* via SIE files through the core SIE import engine. This orchestrator
* handles only entity-level imports:
* 1. Company info → pre-fill company_settings
@@ -14,13 +14,15 @@
import type { SupabaseClient } from '@supabase/supabase-js'
import type { MigrationProgress, MigrationResults } from '../types'
import type { ProviderName } from '@/lib/providers/types'
import { resolveConsent } from '@/lib/providers/resolve-consent'
import {
fetchCompanyInfo,
fetchCustomers,
fetchSuppliers,
fetchSalesInvoices,
fetchSupplierInvoices,
} from './arcim-client'
fetchCompanyInfoDirect,
fetchCustomersDirect,
fetchSuppliersDirect,
fetchSalesInvoicesDirect,
fetchSupplierInvoicesDirect,
} from '@/lib/providers/provider-data-fetcher'
import {
mapCustomer,
mapSupplier,
@@ -32,8 +34,8 @@ import {
export interface MigrationOptions {
consentId: string
userId: string
companyId: string
userId: string
supabase: SupabaseClient
importCompanyInfo?: boolean
importCustomers?: boolean
@@ -50,15 +52,21 @@ function emitProgress(options: MigrationOptions, progress: MigrationProgress) {
// ── Main orchestrator ─────────────────────────────────────────────
export async function executeMigration(options: MigrationOptions): Promise<MigrationResults> {
const { consentId, userId, companyId, supabase } = options
const { consentId, companyId, userId, supabase } = options
const results: MigrationResults = {}
// Resolve consent to get access token and provider
const resolved = await resolveConsent(companyId, consentId)
const provider = resolved.consent.provider as ProviderName
const accessToken = resolved.accessToken
const providerCompanyId = resolved.providerCompanyId
try {
// ── Step 1: Company information ───────────────────────────────
if (options.importCompanyInfo !== false) {
emitProgress(options, { status: 'fetching', currentStep: 'Hämtar företagsinformation...', progress: 5 })
try {
const companyInfo = await fetchCompanyInfo(consentId)
const companyInfo = await fetchCompanyInfoDirect(provider, accessToken, providerCompanyId)
if (companyInfo) {
const mapped = mapCompanyInfo(companyInfo)
const { data: existing } = await supabase
@@ -100,7 +108,7 @@ export async function executeMigration(options: MigrationOptions): Promise<Migra
if (options.importCustomers !== false) {
emitProgress(options, { status: 'importing', currentStep: 'Importerar kunder...', progress: 20 })
try {
const customers = await fetchCustomers(consentId)
const customers = await fetchCustomersDirect(provider, accessToken, providerCompanyId)
let imported = 0
let skipped = 0
@@ -157,7 +165,7 @@ export async function executeMigration(options: MigrationOptions): Promise<Migra
if (options.importSuppliers !== false) {
emitProgress(options, { status: 'importing', currentStep: 'Importerar leverantörer...', progress: 40 })
try {
const suppliers = await fetchSuppliers(consentId)
const suppliers = await fetchSuppliersDirect(provider, accessToken, providerCompanyId)
let imported = 0
let skipped = 0
@@ -212,7 +220,7 @@ export async function executeMigration(options: MigrationOptions): Promise<Migra
if (options.importSalesInvoices !== false) {
emitProgress(options, { status: 'importing', currentStep: 'Importerar kundfakturor...', progress: 60 })
try {
const invoices = await fetchSalesInvoices(consentId)
const invoices = await fetchSalesInvoicesDirect(provider, accessToken, providerCompanyId)
const openInvoices = invoices.filter(i =>
i.status === 'sent' || i.status === 'overdue' || i.status === 'booked'
)
@@ -324,7 +332,7 @@ export async function executeMigration(options: MigrationOptions): Promise<Migra
if (options.importSupplierInvoices !== false) {
emitProgress(options, { status: 'importing', currentStep: 'Importerar leverantörsfakturor...', progress: 80 })
try {
const invoices = await fetchSupplierInvoices(consentId)
const invoices = await fetchSupplierInvoicesDirect(provider, accessToken, providerCompanyId)
const openInvoices = invoices.filter(i =>
i.status === 'sent' || i.status === 'overdue' || i.status === 'booked' || i.status === 'draft'
)
@@ -0,0 +1,262 @@
/**
* Direct provider client — replaces arcim-client.ts.
*
* Instead of making HTTP calls to the Arcim Sync gateway, this module
* performs consent/OTC operations directly against Supabase and delegates
* data fetching to the provider clients in lib/providers/.
*/
import { createServiceClient } from '@/lib/supabase/server'
import type { ProviderName } from '@/lib/providers/types'
import { getOAuthConfig } from '@/lib/providers/oauth-config'
import { buildFortnoxAuthUrl } from '@/lib/providers/fortnox/oauth'
import { exchangeFortnoxCode } from '@/lib/providers/fortnox/oauth'
import { buildVismaAuthUrl, exchangeVismaCode } from '@/lib/providers/visma/oauth'
import { refreshBjornLundenToken } from '@/lib/providers/bjornlunden/oauth'
import type { ConsentRecord, OtcResponse } from '../types'
// Re-export data fetching functions from the provider layer
export { resolveConsent } from '@/lib/providers/resolve-consent'
export {
fetchCompanyInfoDirect,
fetchCustomersDirect,
fetchSuppliersDirect,
fetchSalesInvoicesDirect,
fetchSupplierInvoicesDirect,
} from '@/lib/providers/provider-data-fetcher'
// ── Consent lifecycle (direct Supabase) ─────────────────────────────
export async function createConsent(
companyId: string,
provider: ProviderName,
name: string,
orgNumber?: string,
companyName?: string,
): Promise<ConsentRecord> {
const supabase = createServiceClient()
const { data, error } = await supabase
.from('provider_consents')
.insert({
company_id: companyId,
name,
provider,
org_number: orgNumber,
company_name: companyName,
status: 0, // Created
})
.select('*')
.single()
if (error || !data) {
throw new Error(`Failed to create consent: ${error?.message}`)
}
return {
id: data.id,
name: data.name,
provider: data.provider as ProviderName,
status: data.status,
orgNumber: data.org_number,
companyName: data.company_name,
}
}
export async function listConsents(companyId: string): Promise<ConsentRecord[]> {
const supabase = createServiceClient()
const { data, error } = await supabase
.from('provider_consents')
.select('*')
.eq('company_id', companyId)
.in('status', [0, 1]) // Created or Accepted
.order('created_at', { ascending: false })
if (error) {
throw new Error(`Failed to list consents: ${error.message}`)
}
return (data ?? []).map(d => ({
id: d.id,
name: d.name,
provider: d.provider as ProviderName,
status: d.status,
orgNumber: d.org_number,
companyName: d.company_name,
createdAt: d.created_at,
updatedAt: d.updated_at,
}))
}
export async function getConsent(consentId: string): Promise<ConsentRecord> {
const supabase = createServiceClient()
const { data, error } = await supabase
.from('provider_consents')
.select('*')
.eq('id', consentId)
.single()
if (error || !data) {
throw new Error(`Consent not found: ${error?.message}`)
}
return {
id: data.id,
name: data.name,
provider: data.provider as ProviderName,
status: data.status,
orgNumber: data.org_number,
companyName: data.company_name,
}
}
export async function deleteConsent(consentId: string): Promise<void> {
const supabase = createServiceClient()
// Delete tokens first (cascade should handle this, but be explicit)
await supabase.from('provider_consent_tokens').delete().eq('consent_id', consentId)
await supabase.from('provider_otc').delete().eq('consent_id', consentId)
const { error } = await supabase
.from('provider_consents')
.delete()
.eq('id', consentId)
if (error) {
throw new Error(`Failed to delete consent: ${error.message}`)
}
}
export async function generateOtc(
consentId: string,
expiresInMinutes: number = 60,
): Promise<OtcResponse> {
const supabase = createServiceClient()
const code = crypto.randomUUID().replace(/-/g, '').slice(0, 16)
const expiresAt = new Date(Date.now() + expiresInMinutes * 60 * 1000).toISOString()
const { error } = await supabase
.from('provider_otc')
.insert({
code,
consent_id: consentId,
expires_at: expiresAt,
})
if (error) {
throw new Error(`Failed to generate OTC: ${error.message}`)
}
return { code, consentId, expiresAt }
}
// ── OAuth helpers (direct provider calls) ───────────────────────────
export async function getAuthUrl(
provider: ProviderName,
state?: string,
redirectUri?: string,
): Promise<{ url: string }> {
const config = getOAuthConfig(provider)
// Override redirect URI if provided (extension callback URL)
const effectiveConfig = redirectUri
? { ...config, redirectUri }
: config
if (provider === 'fortnox') {
const url = buildFortnoxAuthUrl(effectiveConfig, { state })
return { url }
}
if (provider === 'visma') {
const url = buildVismaAuthUrl(effectiveConfig, { state })
return { url }
}
throw new Error(`OAuth is not supported for provider: ${provider}`)
}
export async function exchangeAuthToken(
consentId: string,
provider: ProviderName,
code: string,
redirectUri?: string,
): Promise<{ success: boolean; consentId: string }> {
const config = getOAuthConfig(provider)
const effectiveConfig = redirectUri ? { ...config, redirectUri } : config
const supabase = createServiceClient()
let tokenResponse: { access_token: string; refresh_token: string; expires_in: number }
if (provider === 'fortnox') {
tokenResponse = await exchangeFortnoxCode(effectiveConfig, code)
} else if (provider === 'visma') {
tokenResponse = await exchangeVismaCode(effectiveConfig, code)
} else {
throw new Error(`OAuth exchange not supported for provider: ${provider}`)
}
const expiresAt = new Date(Date.now() + tokenResponse.expires_in * 1000).toISOString()
// Store tokens
await supabase
.from('provider_consent_tokens')
.upsert({
consent_id: consentId,
provider,
access_token: tokenResponse.access_token,
refresh_token: tokenResponse.refresh_token,
token_expires_at: expiresAt,
})
// Mark consent as accepted
await supabase
.from('provider_consents')
.update({ status: 1 })
.eq('id', consentId)
return { success: true, consentId }
}
export async function submitProviderToken(
consentId: string,
provider: ProviderName,
apiToken: string,
companyId?: string,
): Promise<{ success: boolean; consentId: string }> {
const supabase = createServiceClient()
let accessToken = apiToken
let tokenExpiresAt: string | null = null
// BL uses client credentials — get a real token
if (provider === 'bjornlunden') {
const tokenResponse = await refreshBjornLundenToken()
accessToken = tokenResponse.access_token
tokenExpiresAt = new Date(Date.now() + tokenResponse.expires_in * 1000).toISOString()
}
// Store tokens
await supabase
.from('provider_consent_tokens')
.upsert({
consent_id: consentId,
provider,
access_token: accessToken,
refresh_token: null,
token_expires_at: tokenExpiresAt,
provider_company_id: companyId,
})
// Mark consent as accepted
await supabase
.from('provider_consents')
.update({ status: 1 })
.eq('id', consentId)
return { success: true, consentId }
}
@@ -4,16 +4,28 @@
"exportName": "arcimMigrationExtension",
"entryPoint": "@/extensions/general/arcim-migration",
"workspace": "@/components/extensions/general/ArcimMigrationWorkspace",
"requiredEnvVars": ["ARCIM_SYNC_GATEWAY_URL", "ARCIM_SYNC_API_KEY"],
"optionalEnvVars": [],
"npmDependencies": [],
"requiredEnvVars": [],
"optionalEnvVars": [
"FORTNOX_CLIENT_ID",
"FORTNOX_CLIENT_SECRET",
"FORTNOX_REDIRECT_URI",
"VISMA_CLIENT_ID",
"VISMA_CLIENT_SECRET",
"VISMA_REDIRECT_URI",
"BRIOX_CLIENT_ID",
"BJORN_LUNDEN_CLIENT_ID",
"BJORN_LUNDEN_CLIENT_SECRET",
"UPSTASH_REDIS_REST_URL",
"UPSTASH_REDIS_REST_TOKEN"
],
"npmDependencies": ["@upstash/redis", "@upstash/ratelimit"],
"definition": {
"name": "Systemmigration (Arcim Sync)",
"name": "Systemmigration",
"category": "import",
"icon": "ArrowRightLeft",
"dataPattern": "manual",
"hasOwnData": false,
"description": "Migrera bokföring från Fortnox, Visma, Bokio, Björn Lundén eller Briox",
"longDescription": "Flytta all bokföringsdata från ditt gamla system till gnubok. Importerar kontoplan, verifikationer, kunder, leverantörer och öppna fakturor automatiskt via säker API-integration."
"longDescription": "Flytta all bokföringsdata från ditt gamla system till gnubok. Importerar kontoplan, verifikationer, kunder, leverantörer och öppna fakturor automatiskt via säker API-integration direkt med leverantören."
}
}
+26 -190
View File
@@ -1,198 +1,34 @@
/**
* Types for the Arcim Sync migration extension.
* Types for the provider migration extension.
*
* These mirror the canonical DTOs from the Arcim Sync gateway
* (packages/core/src/types/dto/) so we don't take a runtime dependency.
* DTO types are now imported from the canonical source at lib/providers/dto.ts
* instead of being duplicated here.
*/
// ── Arcim Sync canonical DTOs (subset we consume) ──────────────────
// Re-export canonical DTOs used by entity-mapper and migration-orchestrator
export type {
AmountType,
PostalAddress,
Contact,
PartyIdentification,
PartyLegalEntity,
PartyDto,
PaginatedResponse,
TaxSubtotalDto,
TaxTotalDto,
LegalMonetaryTotalDto,
PaymentStatusDto,
CompanyInformationDto,
CustomerDto,
SupplierDto,
InvoiceStatusCode,
SalesInvoiceLineDto,
SalesInvoiceDto,
SupplierInvoiceLineDto,
SupplierInvoiceDto,
} from '@/lib/providers/dto'
export interface AmountType {
value: number
currencyCode: string
}
export interface PostalAddress {
streetName?: string
additionalStreetName?: string
buildingNumber?: string
cityName?: string
postalZone?: string
countrySubentity?: string
countryCode?: string
}
export interface Contact {
name?: string
telephone?: string
email?: string
website?: string
}
export interface PartyIdentification {
id: string
schemeId?: string
}
export interface PartyLegalEntity {
registrationName: string
companyId?: string
companyIdSchemeId?: string
}
export interface PartyDto {
name: string
identifications: PartyIdentification[]
postalAddress?: PostalAddress
legalEntity?: PartyLegalEntity
contact?: Contact
}
export interface PaginatedResponse<T> {
data: T[]
page: number
pageSize: number
totalCount: number
hasMore: boolean
}
export interface TaxSubtotalDto {
taxableAmount: AmountType
taxAmount: AmountType
taxCategory?: string
percent?: number
}
export interface TaxTotalDto {
taxAmount: AmountType
taxSubtotals?: TaxSubtotalDto[]
}
export interface LegalMonetaryTotalDto {
lineExtensionAmount: AmountType
taxExclusiveAmount?: AmountType
taxInclusiveAmount?: AmountType
payableAmount: AmountType
}
export interface PaymentStatusDto {
paid: boolean
balance: AmountType
lastPaymentDate?: string
}
// ── Company Information ─────────────────────────────────────────────
export interface CompanyInformationDto {
companyName: string
organizationNumber?: string
legalEntity?: PartyLegalEntity
address?: PostalAddress
contact?: Contact
vatNumber?: string
fiscalYearStart?: string // MM-DD
baseCurrency?: string
}
// ── Customer ────────────────────────────────────────────────────────
export type ArcimCustomerType = 'company' | 'private'
export interface CustomerDto {
id: string
customerNumber: string
type?: ArcimCustomerType
party: PartyDto
active: boolean
vatNumber?: string
defaultPaymentTermsDays?: number
note?: string
}
// ── Supplier ────────────────────────────────────────────────────────
export interface SupplierDto {
id: string
supplierNumber: string
party: PartyDto
active: boolean
vatNumber?: string
bankAccount?: string
bankGiro?: string
plusGiro?: string
defaultPaymentTermsDays?: number
note?: string
}
// ── Sales Invoice ───────────────────────────────────────────────────
export type InvoiceStatusCode = 'draft' | 'sent' | 'booked' | 'paid' | 'overdue' | 'cancelled' | 'credited'
export interface SalesInvoiceLineDto {
id: string
description?: string
quantity?: number
unitCode?: string
unitPrice?: AmountType
lineExtensionAmount: AmountType
taxPercent?: number
taxAmount?: AmountType
accountNumber?: string
itemName?: string
}
export interface SalesInvoiceDto {
id: string
invoiceNumber: string
issueDate: string
dueDate?: string
deliveryDate?: string
invoiceTypeCode?: string
currencyCode: string
status: InvoiceStatusCode
supplier: PartyDto
customer: PartyDto
lines: SalesInvoiceLineDto[]
taxTotal?: TaxTotalDto
legalMonetaryTotal: LegalMonetaryTotalDto
paymentStatus: PaymentStatusDto
paymentTerms?: string
note?: string
}
// ── Supplier Invoice ────────────────────────────────────────────────
export interface SupplierInvoiceLineDto {
id: string
description?: string
quantity?: number
unitCode?: string
unitPrice?: AmountType
lineExtensionAmount: AmountType
taxPercent?: number
taxAmount?: AmountType
accountNumber?: string
itemName?: string
}
export interface SupplierInvoiceDto {
id: string
invoiceNumber: string
issueDate: string
dueDate?: string
deliveryDate?: string
invoiceTypeCode?: string
currencyCode: string
status: InvoiceStatusCode
supplier: PartyDto
buyer: PartyDto
lines: SupplierInvoiceLineDto[]
taxTotal?: TaxTotalDto
legalMonetaryTotal: LegalMonetaryTotalDto
paymentStatus: PaymentStatusDto
ocrNumber?: string
note?: string
}
export type { CustomerType as ArcimCustomerType } from '@/lib/providers/dto'
// ── Supported providers ─────────────────────────────────────────────