sax3l/accounted
3
0
Fork 0
Code Issues 1 Pull Requests 2 Actions Packages Projects Releases Wiki Activity

feat(import): let the Fortnox import fetch older fiscal years: the three-year limit becomes a default selection (#2280)

Browse Source 
* fix(import): say which fiscal years the Fortnox connection fetches, list the ones left out

Root cause: the guided provider migration fetches SIE only for fiscal years
that start within a rolling three-calendar-year window
(getAllowedFiscalYears in extensions/general/arcim-migration/lib/sie-fetcher.ts:
current year and the two before it). A first, broken year 2022/2023 starts
in 2022 and falls outside the window in 2026, and the wizard said nothing:
not before the import, not after. Users concluded the books were complete,
or that they had done something wrong (issue #2211, second report via
support 2026-08-27).

Fix:
- The fetcher already lists every fiscal year at the source before applying
  the window, so the left-out years are derived from that same list at no
  extra provider call: `omittedYears` (years starting before the window,
  oldest first, with the provider's own from/to dates so a broken year is
  named as "2022-09-01 till 2023-12-31"). Fortnox and Briox year refs now
  carry those bounds; WINT's listYears reports the unfiltered year list.
- GET /preview returns `fiscalYearWindow` and `omittedYears`; GET /sie-data
  returns `omittedYears` next to `failedYears`.
- Wizard, preview step (before the import runs): one muted sentence that the
  direct connection fetches the three latest fiscal years (years starting in
  {fromYear} or later); when years are left out, they are named with a link
  to the SIE import (one SIE file per year under Import, oldest first).
- Wizard, result step: a "Räkenskapsår som inte följde med" section naming
  the omitted years with the same SIE pointer, shown when SIE data was
  imported in the run.
- MCP: the connect_migration tool description, its instructions and the
  onboarding skill claimed the wizard "fetches every fiscal year"; they now
  say three latest, older years via SIE.
- Strings in both messages/sv.json and messages/en.json.

Out of scope: fetching more years through the connection (#2238).

Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_015qgLgdt4mLmha1ZLFMwq1u

* feat(import): make the Fortnox import's three-year limit a default selection, not a cap

Root cause, from first principles: the guided provider migration fetched
SIE only for fiscal years starting within a rolling three-calendar-year
window (getAllowedFiscalYears in
extensions/general/arcim-migration/lib/sie-fetcher.ts, introduced in #718
with no stated reason). The window was a silent cap: the wizard never said
it existed, and a first broken year 2022/2023 simply never arrived
(#2211). The user's actual problem is that the year is missing from the
books, so explaining the cap (the previous commit) treats the symptom.

What the window gated, by evidence: only the SIE fetch. Documents already
list every Fortnox financial year and match against the vouchers that
exist locally (import-documents.ts), invoices, customers, suppliers and
assets are not year-gated, and the SIE import itself is one request per
year (hosted function limit 300 s, import_sie_journal_entries
statement_timeout 290 s), so its cost is linear in wall time and bounded
per year regardless of how many years are imported. The only place the
number of years multiplies inside one invocation is /preview and
/sie-data: one SIE export per year (Fortnox client: 15 s per-call timeout,
3 attempts, backoff up to 30 s, 4 req/s) fetched and parsed inside a
single 300 s function, and /sie-data returns every raw file in one
response. The repo holds no measurement of Fortnox's per-year SIE export
latency, and the maintainer's memory is that a full history can take
unreasonably long, so a fixed lift to every year cannot be shown safe for
a long history.

Fix: the window becomes the DEFAULT selection, and the user chooses.
- sie-fetcher: fetchProviderSieFiles takes `years` (explicit start years);
  without it the default window applies. The result carries `sourceYears`
  (every year at the source, oldest first, with the provider's own bounds
  and an inDefaultSelection flag) and `omittedYears` (source years outside
  the selection). Both derived from the year list already fetched: no
  extra provider call. Fortnox and Briox year refs carry their bounds;
  WINT's listYears reports the unfiltered list and its voucher chain
  follows the selection.
- GET /preview returns `sourceYears`; GET /sie-data honours `?years=`
  (validated, deduplicated, oldest first; 400 VALIDATION_ERROR when
  malformed) and returns `omittedYears`. PROVIDER_SIE_NO_YEARS names the
  selection.
- Wizard, preview step: a "Räkenskapsår att hämta" picker with one
  checkbox row per source year, the three latest ticked by default, older
  years marked "tar längre tid"; Fortsätt is disabled with an attn line
  until at least one year is ticked. The selection is sent to /sie-data,
  so each extra year is the user's own wait, and it fails loudly there,
  before any ledger write, if it is too much.
- Wizard, result step: the per-year lines already report exactly what was
  imported; a "Räkenskapsår som inte hämtades" section names the source
  years outside the selection, with the re-run path (documents come
  along) and the SIE path.
- MCP connect_migration description, instructions and the onboarding skill
  say "three latest by default, older years selectable" instead of "every
  fiscal year".
- Strings in both messages/sv.json and messages/en.json.

Closes #2238 as well: the wish to fetch more years is the same control.

Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_015qgLgdt4mLmha1ZLFMwq1u

* fix(import): bound the fiscal-year selection per import run

Superagent P2 on #2280: the `years` selection was unbounded and every
selected year is one provider export fetched and parsed inside the single
300 s /sie-data invocation, so nothing bounded the work before provider
calls.

The bound: MAX_SELECTED_FISCAL_YEARS = 6, exported from sie-fetcher.ts with
the derivation. One export call is 15 s per attempt (Fortnox client
FETCH_TIMEOUT_MS), 3 attempts with 1 s and 2 s backoff (retry defaults),
so a year that times out on every attempt costs 48 s; six such years are
288 s, leaving 12 s of the 300 s hosted function for the year listing,
parsing and the response; seven would be 336 s.

Enforced server-side:
- /sie-data refuses a selection of more than the cap with 400
  VALIDATION_ERROR naming the cap, before the consent is resolved, so an
  oversized request does no provider work.
- fetchProviderSieFiles throws FiscalYearSelectionError for a selected year
  the source does not have, right after the year listing and before any
  export; /sie-data maps it to 400 VALIDATION_ERROR naming the year.
- /preview returns maxSelectedYears so the picker enforces the same number
  without a client-side copy: Fortsätt is disabled and an attn line says how
  many can be fetched at once and that older years go in a second run
  (sv + en).

Tests: cap accepted at 6 and refused at 7 with no provider call, unknown
year refused (route and fetcher), the cap's arithmetic, maxSelectedYears
on /preview.

Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_015qgLgdt4mLmha1ZLFMwq1u

---------

Co-authored-by: Jakob Wennberg <311770904+jakobwennberg-oss@users.noreply.github.com>
Co-authored-by: Claude Fable 5.1 <noreply@anthropic.com>
This commit is contained in:
Jakob Wennberg
2026-09-04 17:14:49 +02:00
committed by GitHub
parent 7a9036caa6
commit 1976129478
10 changed files with 808 additions and 41 deletions
Download Patch File Download Diff File Expand all files Collapse all files
components/extensions/general/ArcimMigrationWorkspace.tsx
+135 -4
View File
@@ -9,6 +9,7 @@ import { Progress } from '@/components/ui/progress'
import { Button } from '@/components/ui/button'
import { Input } from '@/components/ui/input'
import { Switch } from '@/components/ui/switch'
import { Checkbox } from '@/components/ui/checkbox'
import { useToast } from '@/components/ui/use-toast'
import { cn } from '@/lib/utils'
import { ConfirmationDialog } from '@/components/ui/confirmation-dialog'
@@ -304,6 +305,13 @@ interface PreviewData {
transactionCount: number
fiscalYears: number[]
} | null
// Every fiscal year the source has, oldest first, with the default
// selection marked: rendered as the year picker so the user chooses
// before the import runs and no year is left out silently (#2211, #2238).
sourceYears?: SourceFiscalYear[]
// The most years one run may select: /sie-data refuses more. Read from
// the server so the picker never drifts from the route.
maxSelectedYears?: number
assetStats: {
total: number
importable: number
@@ -338,9 +346,33 @@ interface SIEData {
// Fiscal years whose provider export failed. Importing the remaining years
// anyway leaves an IB/UB gap: the options step warns before proceeding.
failedYears?: { year: number; error: string }[]
// Source fiscal years outside the selection: not fetched, named in the
// result so nobody believes the books are complete (#2211).
omittedYears?: SourceFiscalYear[]
basAccounts: BASAccount[]
}
/**
* A fiscal year as the source reports it. Mirrors SourceFiscalYear in
* extensions/general/arcim-migration/lib/sie-fetcher.ts (deliberate
* duplication: core must not import from @/extensions/).
*/
interface SourceFiscalYear {
year: number
fromDate: string | null
toDate: string | null
inDefaultSelection: boolean
}
/** "2022-09-01 till 2023-12-31" when the provider gave bounds, else the start year. */
function useFiscalYearSpanLabel(): (fy: SourceFiscalYear) => string {
const t = useTranslations('extensions')
return (fy) =>
fy.fromDate && fy.toDate
? t('ext_arcim_fiscal_year_span', { from: fy.fromDate, to: fy.toDate })
: String(fy.year)
}
// ── Shared step chrome ───────────────────────────────────────────
// Living Paper: step content sits directly on the page. The serif headline
// is the step's one display element; sections are kickers over hairline
@@ -875,6 +907,8 @@ function PreviewStep({
error,
authExpired,
licenseMissing,
selectedYears,
onSelectedYearsChange,
onReconnect,
onContinue,
onBack,
@@ -884,13 +918,30 @@ function PreviewStep({
error: string | null
authExpired: boolean
licenseMissing: boolean
/** Fiscal years (start years) ticked in the picker; default = the three latest. */
selectedYears: number[]
onSelectedYearsChange: (years: number[]) => void
onReconnect: () => void
onContinue: () => void
onBack: () => void
}) {
const t = useTranslations('extensions')
const fiscalYearSpanLabel = useFiscalYearSpanLabel()
const providerName = preview
? ARCIM_PROVIDERS.find(p => p.id === preview.consent.provider)?.name ?? preview.consent.provider
: ''
const sourceYears = preview?.sieAvailable ? preview.sourceYears ?? [] : []
const showYearPicker = sourceYears.length > 0 && !isLoading
const noYearSelected = showYearPicker && selectedYears.length === 0
const maxSelectable = preview?.maxSelectedYears ?? null
const tooManySelected = showYearPicker && maxSelectable != null && selectedYears.length > maxSelectable
const toggleYear = (year: number) => {
onSelectedYearsChange(
selectedYears.includes(year)
? selectedYears.filter((y) => y !== year)
: [...selectedYears, year].sort((a, b) => a - b),
)
}
return (
<div className="stagger-enter space-y-8">
@@ -941,6 +992,45 @@ function PreviewStep({
)}
</div>
{/* ── The year picker (issues #2211, #2238) ──
Every fiscal year the source has, as hairline rows with a checkbox.
The three latest are ticked by default (the limit that used to be a
silent cap); older years are the user's own choice and their own
wait: each one is another SIE export fetched in the next step. */}
{showYearPicker && (
<section className="space-y-3">
<SectionKicker>{t('ext_arcim_year_select_kicker')}</SectionKicker>
<p className="text-[13px] text-muted-foreground">{t('ext_arcim_year_select_lede')}</p>
<div className="stagger-enter divide-y divide-border" data-no-stagger>
{sourceYears.map((fy) => {
const id = `arcim-year-${fy.year}-${fy.fromDate ?? ''}`
return (
<label
key={id}
htmlFor={id}
className="flex min-h-10 cursor-pointer items-center gap-3 py-2 text-sm transition-colors duration-150 hover:bg-secondary/35"
>
<Checkbox
id={id}
checked={selectedYears.includes(fy.year)}
onCheckedChange={() => toggleYear(fy.year)}
aria-label={fiscalYearSpanLabel(fy)}
/>
<span className="tabular-nums">{fiscalYearSpanLabel(fy)}</span>
{!fy.inDefaultSelection && (
<span className="text-xs text-muted-foreground">{t('ext_arcim_year_select_older')}</span>
)}
</label>
)
})}
</div>
{noYearSelected && <AttnLine>{t('ext_arcim_year_select_none')}</AttnLine>}
{tooManySelected && maxSelectable != null && (
<AttnLine>{t('ext_arcim_year_select_too_many', { max: maxSelectable })}</AttnLine>
)}
</section>
)}
{error && (
<div className="space-y-3">
<div className="space-y-1">
@@ -979,7 +1069,11 @@ function PreviewStep({
<ArrowLeft className="mr-2 h-4 w-4" />
Tillbaka
</Button>
<Button className="min-h-11" onClick={onContinue} disabled={isLoading || (!!preview && !preview.sieAvailable && !preview.hasSieData)}>
<Button
className="min-h-11"
onClick={onContinue}
disabled={isLoading || noYearSelected || tooManySelected || (!!preview && !preview.sieAvailable && !preview.hasSieData)}
>
Fortsätt
<ArrowRight className="ml-2 h-4 w-4" />
</Button>
@@ -1706,6 +1800,7 @@ const NEXT_STEPS: { title: string; sub: string }[] = [
function ResultStep({
results,
sieResults,
omittedYears,
error,
documentImportState,
theaterModel,
@@ -1718,6 +1813,8 @@ function ResultStep({
}: {
results: MigrationResults | null
sieResults: ImportResult[]
/** Source fiscal years outside the selection: not fetched in this run. */
omittedYears: SourceFiscalYear[]
error: string | null
documentImportState: ArcimDocumentImportState
theaterModel: TheaterModel | null
@@ -1729,6 +1826,7 @@ function ResultStep({
onReconnectDocuments: () => void
}) {
const t = useTranslations('extensions')
const fiscalYearSpanLabel = useFiscalYearSpanLabel()
if (error) {
return (
<div className="stagger-enter space-y-8">
@@ -1954,6 +2052,27 @@ function ResultStep({
</section>
)}
{/* ── Source fiscal years outside the selection (#2211) ──
Named here so nobody believes the books are complete: a new run
with those years ticked fetches them (documents come along), or
the SIE path does. */}
{sieResults.length > 0 && omittedYears.length > 0 && (
<section className="space-y-3">
<SectionKicker>{t('ext_arcim_omitted_years_kicker')}</SectionKicker>
<div className="stagger-enter divide-y divide-border" data-no-stagger>
{omittedYears.map((fy) => (
<p key={`${fy.year}-${fy.fromDate ?? ''}`} className="py-3 text-sm tabular-nums">
{fiscalYearSpanLabel(fy)}
</p>
))}
</div>
<SieFallbackLine
message={t('ext_arcim_omitted_years_result', { count: omittedYears.length })}
label={t('ext_arcim_omitted_years_sie_link')}
/>
</section>
)}
{/* ── API import results: quiet two-column line list ── */}
{entityLines.length > 0 && (
<section className="space-y-3">
@@ -2122,6 +2241,9 @@ export default function ArcimMigrationWorkspace({
// Preview state
const [preview, setPreview] = useState<PreviewData | null>(null)
// Fiscal years (start years) ticked in the preview step's picker. Set from
// the preview's default selection on load; sent to /sie-data as `years`.
const [selectedYears, setSelectedYears] = useState<number[]>([])
// Set when a preview/sync fails because the provider connection expired
// (dead refresh token → PROVIDER_AUTH_EXPIRED). Drives the "Återanslut"
// affordance so the user can re-authorize in place instead of disconnecting.
@@ -2223,8 +2345,11 @@ export default function ArcimMigrationWorkspace({
throw new Error(apiErrorMessage(data, `HTTP ${res.status}`))
}
const data = await res.json()
const data = await res.json() as PreviewData
setPreview(data)
setSelectedYears(
(data.sourceYears ?? []).filter((fy) => fy.inDefaultSelection).map((fy) => fy.year),
)
const previewProvider = data?.consent?.provider
if (ARCIM_PROVIDERS.some((provider) => provider.id === previewProvider)) {
setSelectedProvider(previewProvider as ArcimProvider)
@@ -2701,7 +2826,10 @@ export default function ArcimMigrationWorkspace({
setErrorDetails(null)
try {
const res = await fetch(`/api/extensions/ext/arcim-migration/sie-data?consentId=${consentId}`)
// The picker's selection travels as `years`; without a picker (no
// source years known) the route falls back to its default selection.
const yearsQuery = selectedYears.length > 0 ? `&years=${selectedYears.join(',')}` : ''
const res = await fetch(`/api/extensions/ext/arcim-migration/sie-data?consentId=${consentId}${yearsQuery}`)
if (!res.ok) {
const data = await res.json().catch(() => ({})) as {
error?: unknown
@@ -2741,7 +2869,7 @@ export default function ArcimMigrationWorkspace({
} finally {
setIsLoading(false)
}
}, [consentId, refreshCompanyAccounts])
}, [consentId, refreshCompanyAccounts, selectedYears])
const handlePreviewContinue = useCallback(() => {
if (preview?.sieAvailable) {
@@ -3050,6 +3178,8 @@ export default function ArcimMigrationWorkspace({
error={error}
authExpired={authExpired}
licenseMissing={licenseMissing}
selectedYears={selectedYears}
onSelectedYearsChange={setSelectedYears}
onReconnect={() => {
if (selectedProvider && consentId) handleReconnect(selectedProvider, consentId)
}}
@@ -3101,6 +3231,7 @@ export default function ArcimMigrationWorkspace({
<ResultStep
results={migrationResults}
sieResults={sieImportResults}
omittedYears={sieData?.omittedYears ?? []}
error={error}
documentImportState={documentImportState}
theaterModel={theaterModel}
extensions/general/arcim-migration/__tests__/fiscal-year-scope.test.ts
+333
View File
@@ -0,0 +1,333 @@
import { describe, it, expect, beforeEach, vi, type Mock } from 'vitest'
import { createMockSupabase, createMockRequest, parseJsonResponse } from '@/tests/helpers'
import type { ExtensionContext } from '@/lib/extensions/types'
/**
* Issues #2211 / #2238: the direct provider connection used to fetch three
* fiscal years as a silent cap (getAllowedFiscalYears in lib/sie-fetcher).
* A first, broken year 2022/2023 simply never arrived and the user asked
* whether they had done something wrong. The cap is a default selection
* now: the preview step renders every source year as a picker, /sie-data
* fetches the ticked years, and the result step names the rest.
*
* These tests lock the plumbing the wizard reads: GET /preview carries
* `sourceYears` (with the default flag), GET /sie-data honours `years` and
* carries `omittedYears`.
*/
vi.mock('../lib/migration-orchestrator', () => ({
executeMigration: vi.fn(),
}))
vi.mock('../lib/provider-client', () => ({
createConsent: vi.fn(),
getConsent: vi.fn(),
listConsents: vi.fn(),
generateOtc: vi.fn(),
consumeOAuthState: vi.fn(),
getAuthUrl: vi.fn(),
exchangeAuthToken: vi.fn(),
submitProviderToken: vi.fn(),
acceptConsent: vi.fn(),
deleteConsent: vi.fn(),
resolveConsent: vi.fn(),
fetchCompanyInfoDirect: vi.fn(),
ProviderTokenInvalidError: class ProviderTokenInvalidError extends Error {},
ProviderCompanyMismatchError: class ProviderCompanyMismatchError extends Error {},
ConsentNotFoundError: class ConsentNotFoundError extends Error {},
}))
vi.mock('../lib/sie-fetcher', () => ({
providerSupportsSie: vi.fn().mockReturnValue(true),
fetchProviderSieFiles: vi.fn(),
getAllowedFiscalYears: vi.fn(),
FiscalYearSelectionError: class FiscalYearSelectionError extends Error {
constructor(public readonly unknownYears: number[]) {
super(`unknown ${unknownYears.join(', ')}`)
}
},
MAX_SELECTED_FISCAL_YEARS: 6,
}))
// The Fortnox asset preview would otherwise go to the network.
vi.mock('../lib/import-assets', () => ({
fetchFortnoxAssetPreview: vi.fn().mockResolvedValue(null),
}))
vi.mock('../lib/mapping-targets', () => ({
buildMappingTargets: vi.fn().mockResolvedValue([]),
}))
vi.mock('@/lib/supabase/server', () => ({
createClient: vi.fn(),
createServiceClient: vi.fn(),
}))
import { arcimMigrationExtension } from '../index'
import { getConsent, resolveConsent, fetchCompanyInfoDirect } from '../lib/provider-client'
import {
fetchProviderSieFiles,
getAllowedFiscalYears,
FiscalYearSelectionError,
MAX_SELECTED_FISCAL_YEARS,
} from '../lib/sie-fetcher'
type RouteHandler = (request: Request, ctx?: ExtensionContext) => Promise<Response>
function handler(path: string): RouteHandler {
return (arcimMigrationExtension.apiRoutes ?? []).find((r) => r.method === 'GET' && r.path === path)!
.handler as RouteHandler
}
const CY = new Date().getFullYear()
// One fiscal year inside the window, enough SIE for the parser and the
// validator (#SIETYP + #RAR) so /sie-data reaches its response.
const SIE_IN_WINDOW = [
'#FLAGGA 0',
'#SIETYP 4',
'#FNAMN "Bolaget AB"',
`#RAR 0 ${CY}0101 ${CY}1231`,
'#KONTO 1930 "Företagskonto"',
'',
].join('\n')
const BROKEN_FIRST_YEAR = {
year: CY - 4,
fromDate: `${CY - 4}-09-01`,
toDate: `${CY - 3}-12-31`,
inDefaultSelection: false,
}
const CURRENT_YEAR = { year: CY, fromDate: `${CY}-01-01`, toDate: `${CY}-12-31`, inDefaultSelection: true }
const SOURCE_YEARS = [BROKEN_FIRST_YEAR, CURRENT_YEAR]
const OMITTED = [BROKEN_FIRST_YEAR]
function buildCtx(): ExtensionContext {
const { supabase, mockResult } = createMockSupabase()
// Every query in these routes reads lists or counts: none, everywhere.
mockResult({ data: [], count: 0 })
;(supabase as unknown as { auth: unknown }).auth = {
getUser: vi.fn().mockResolvedValue({ data: { user: { id: 'user-1' } } }),
}
return { supabase, companyId: 'company-1' } as unknown as ExtensionContext
}
function request(path: string, consentId = 'consent-1') {
return createMockRequest(`http://localhost/api/extensions/ext/arcim-migration${path}`, {
searchParams: { consentId },
})
}
beforeEach(() => {
vi.clearAllMocks()
;(getConsent as Mock).mockResolvedValue({ id: 'consent-1', status: 1, provider: 'fortnox' })
;(resolveConsent as Mock).mockResolvedValue({
consent: { provider: 'fortnox' },
accessToken: 'tok',
providerCompanyId: undefined,
})
;(fetchCompanyInfoDirect as Mock).mockResolvedValue(null)
;(getAllowedFiscalYears as Mock).mockReturnValue(new Set([CY - 2, CY - 1, CY]))
;(fetchProviderSieFiles as Mock).mockResolvedValue({
files: [{ fiscalYear: CY, rawContent: SIE_IN_WINDOW }],
availableYears: [CY],
sourceYears: SOURCE_YEARS,
failedYears: [],
omittedYears: OMITTED,
})
})
describe('GET /preview: every source year, with the default selection marked (#2211)', () => {
it('answers 401 without a user', async () => {
const ctx = buildCtx()
;(ctx.supabase as unknown as { auth: { getUser: Mock } }).auth.getUser.mockResolvedValue({
data: { user: null },
})
const res = await handler('/preview')(request('/preview'), ctx)
expect(res.status).toBe(401)
})
it('carries every source year with its bounds and default flag, on the default selection', async () => {
const res = await handler('/preview')(request('/preview'), buildCtx())
const { status, body } = await parseJsonResponse<{
sieAvailable: boolean
sieStats: { fiscalYears: number[] }
sourceYears: typeof SOURCE_YEARS
}>(res)
expect(status).toBe(200)
expect(body.sieAvailable).toBe(true)
expect(body.sieStats.fiscalYears).toEqual([CY])
// The bounds as the provider reports them: a broken year is named as
// "2022-09-01 till 2023-12-31", not as a wrong calendar year, and it is
// unticked by default rather than absent.
expect(body.sourceYears).toEqual(SOURCE_YEARS)
// The preview never sends a selection: its stats are the default's.
expect((fetchProviderSieFiles as Mock).mock.calls[0][3]).toBeUndefined()
})
it('hands the picker the same cap /sie-data enforces', async () => {
const res = await handler('/preview')(request('/preview'), buildCtx())
const { body } = await parseJsonResponse<{ maxSelectedYears: number }>(res)
expect(body.maxSelectedYears).toBe(MAX_SELECTED_FISCAL_YEARS)
})
it('still lists the source years when nothing inside the default selection came back', async () => {
;(fetchProviderSieFiles as Mock).mockResolvedValue({
files: [],
availableYears: [],
sourceYears: [BROKEN_FIRST_YEAR],
failedYears: [],
omittedYears: [BROKEN_FIRST_YEAR],
})
const res = await handler('/preview')(request('/preview'), buildCtx())
const { status, body } = await parseJsonResponse<{ sieAvailable: boolean; sourceYears: unknown[] }>(res)
expect(status).toBe(200)
expect(body.sieAvailable).toBe(false)
expect(body.sourceYears).toEqual([BROKEN_FIRST_YEAR])
})
})
describe('GET /sie-data: the ticked years are fetched, the rest are named (#2211, #2238)', () => {
it('answers 400 without a consentId', async () => {
const res = await handler('/sie-data')(
createMockRequest('http://localhost/api/extensions/ext/arcim-migration/sie-data'),
buildCtx(),
)
expect(res.status).toBe(400)
})
it('answers 400 VALIDATION_ERROR on a malformed years selection, before touching the provider', async () => {
const res = await handler('/sie-data')(
createMockRequest('http://localhost/api/extensions/ext/arcim-migration/sie-data', {
searchParams: { consentId: 'consent-1', years: `${CY},abc` },
}),
buildCtx(),
)
const { status, body } = await parseJsonResponse<{ error: { code: string } }>(res)
expect(status).toBe(400)
expect(body.error.code).toBe('VALIDATION_ERROR')
expect(fetchProviderSieFiles).not.toHaveBeenCalled()
})
it('falls back to the default selection without a years param and returns omittedYears', async () => {
const res = await handler('/sie-data')(request('/sie-data'), buildCtx())
const { status, body } = await parseJsonResponse<{
fileStatuses: { fiscalYear: number }[]
failedYears: unknown[]
omittedYears: typeof OMITTED
}>(res)
expect(status).toBe(200)
expect((fetchProviderSieFiles as Mock).mock.calls[0][3]).toBeUndefined()
expect(body.fileStatuses.map((f) => f.fiscalYear)).toEqual([CY])
expect(body.failedYears).toEqual([])
expect(body.omittedYears).toEqual(OMITTED)
})
it('passes the ticked years to the fetcher, deduplicated and oldest first', async () => {
;(fetchProviderSieFiles as Mock).mockResolvedValue({
files: [
{ fiscalYear: CY - 4, rawContent: SIE_IN_WINDOW.replace(`#RAR 0 ${CY}0101 ${CY}1231`, `#RAR 0 ${CY - 4}0901 ${CY - 3}1231`) },
{ fiscalYear: CY, rawContent: SIE_IN_WINDOW },
],
availableYears: [CY - 4, CY],
sourceYears: SOURCE_YEARS,
failedYears: [],
omittedYears: [],
})
const res = await handler('/sie-data')(
createMockRequest('http://localhost/api/extensions/ext/arcim-migration/sie-data', {
searchParams: { consentId: 'consent-1', years: `${CY},${CY - 4},${CY}` },
}),
buildCtx(),
)
const { status, body } = await parseJsonResponse<{
fileStatuses: { fiscalYear: number }[]
omittedYears: unknown[]
}>(res)
expect(status).toBe(200)
expect((fetchProviderSieFiles as Mock).mock.calls[0][3]).toEqual({ years: [CY - 4, CY] })
expect(body.fileStatuses.map((f) => f.fiscalYear)).toEqual([CY - 4, CY])
expect(body.omittedYears).toEqual([])
})
it('accepts a selection at the cap and rejects one over it before any provider call', async () => {
const atCap = Array.from({ length: MAX_SELECTED_FISCAL_YEARS }, (_, i) => CY - i)
const overCap = [...atCap, CY - MAX_SELECTED_FISCAL_YEARS]
const ok = await handler('/sie-data')(
createMockRequest('http://localhost/api/extensions/ext/arcim-migration/sie-data', {
searchParams: { consentId: 'consent-1', years: atCap.join(',') },
}),
buildCtx(),
)
expect(ok.status).toBe(200)
expect((fetchProviderSieFiles as Mock).mock.calls[0][3]).toEqual({
years: [...atCap].sort((a, b) => a - b),
})
vi.clearAllMocks()
const rejected = await handler('/sie-data')(
createMockRequest('http://localhost/api/extensions/ext/arcim-migration/sie-data', {
searchParams: { consentId: 'consent-1', years: overCap.join(',') },
}),
buildCtx(),
)
const { status, body } = await parseJsonResponse<{ error: { code: string; message: string } }>(rejected)
expect(status).toBe(400)
expect(body.error.code).toBe('VALIDATION_ERROR')
expect(body.error.message).toContain(String(MAX_SELECTED_FISCAL_YEARS))
// Refused before the consent is even resolved: no provider work at all.
expect(fetchProviderSieFiles).not.toHaveBeenCalled()
expect(resolveConsent).not.toHaveBeenCalled()
})
it('rejects a ticked year the source does not have, as the fetcher reports it before any export', async () => {
;(fetchProviderSieFiles as Mock).mockRejectedValue(new FiscalYearSelectionError([CY - 7]))
const res = await handler('/sie-data')(
createMockRequest('http://localhost/api/extensions/ext/arcim-migration/sie-data', {
searchParams: { consentId: 'consent-1', years: `${CY},${CY - 7}` },
}),
buildCtx(),
)
const { status, body } = await parseJsonResponse<{ error: { code: string; message: string } }>(res)
expect(status).toBe(400)
expect(body.error.code).toBe('VALIDATION_ERROR')
expect(body.error.message).toContain(String(CY - 7))
})
it('names the selection in PROVIDER_SIE_NO_YEARS when none of the ticked years exist at the source', async () => {
;(fetchProviderSieFiles as Mock).mockResolvedValue({
files: [],
availableYears: [],
sourceYears: SOURCE_YEARS,
failedYears: [],
omittedYears: SOURCE_YEARS,
})
const res = await handler('/sie-data')(
createMockRequest('http://localhost/api/extensions/ext/arcim-migration/sie-data', {
searchParams: { consentId: 'consent-1', years: `${CY - 7}` },
}),
buildCtx(),
)
const { status, body } = await parseJsonResponse<{ error: { code: string; message: string } }>(res)
expect(status).toBe(404)
expect(body.error.code).toBe('PROVIDER_SIE_NO_YEARS')
expect(body.error.message).toContain(String(CY - 7))
})
})
extensions/general/arcim-migration/__tests__/preview-provider-errors.test.ts
+2
View File
@@ -45,6 +45,8 @@ vi.mock('../lib/sie-fetcher', () => ({
providerSupportsSie: vi.fn().mockReturnValue(false),
fetchProviderSieFiles: vi.fn(),
getAllowedFiscalYears: vi.fn().mockReturnValue([]),
FiscalYearSelectionError: class FiscalYearSelectionError extends Error {},
MAX_SELECTED_FISCAL_YEARS: 6,
}))
vi.mock('@/lib/supabase/server', () => ({
extensions/general/arcim-migration/index.ts
+81 -13
View File
@@ -17,7 +17,14 @@ import {
ProviderCompanyMismatchError,
ConsentNotFoundError,
} from './lib/provider-client'
import { providerSupportsSie, fetchProviderSieFiles, getAllowedFiscalYears } from './lib/sie-fetcher'
import {
providerSupportsSie,
fetchProviderSieFiles,
getAllowedFiscalYears,
FiscalYearSelectionError,
MAX_SELECTED_FISCAL_YEARS,
type SourceFiscalYear,
} from './lib/sie-fetcher'
import { mapCompanyInfo } from './lib/entity-mapper'
import { executeMigration } from './lib/migration-orchestrator'
import {
@@ -837,6 +844,12 @@ export const arcimMigrationExtension: Extension = {
// Try to fetch SIE data (Fortnox and Briox serve SIE over the API)
let sieAvailable = false
let sieStats: { accountCount: number; transactionCount: number; fiscalYears: number[] } | null = null
// Every fiscal year the source has, with the default selection
// marked: the preview step renders them as the year picker, so the
// user chooses BEFORE the import runs and no year is left out
// silently (issues #2211, #2238). The stats below stay on the
// default selection: that is what the preview fetched.
let sourceYears: SourceFiscalYear[] = []
if (providerSupportsSie(provider)) {
try {
@@ -848,11 +861,12 @@ export const arcimMigrationExtension: Extension = {
// "0 verifikationer" and then imported 4153). Costs one SIE
// export per extra year, the same work /sie-data repeats right
// after: honest numbers are worth it.
const { files, availableYears } = await fetchProviderSieFiles(
const { files, availableYears, sourceYears: source } = await fetchProviderSieFiles(
provider,
resolved.accessToken,
resolved.providerCompanyId,
)
sourceYears = source
if (files.length > 0) {
const merged = mergeParsedSIEFiles(files.map((f) => parseSIEFile(f.rawContent)))
sieAvailable = true
@@ -896,6 +910,10 @@ export const arcimMigrationExtension: Extension = {
companyInfo: mapped,
sieAvailable,
sieStats,
sourceYears,
// The picker enforces the same cap as /sie-data, read from here
// rather than duplicated in the client.
maxSelectedYears: MAX_SELECTED_FISCAL_YEARS,
assetStats,
hasSieData: (sieImportCount ?? 0) > 0,
})
@@ -936,6 +954,34 @@ export const arcimMigrationExtension: Extension = {
return NextResponse.json({ error: 'consentId is required' }, { status: 400 })
}
// `years`: the fiscal years the user ticked in the preview step
// (start years, comma-separated). Absent = the default selection.
// Every selected year is one SIE export fetched and parsed in THIS
// invocation, so the selection is the user's own wait; it fails
// loudly here, before any ledger write, if it is too much.
const yearsParam = url.searchParams.get('years')
let requestedYears: number[] | undefined
if (yearsParam !== null) {
const parsed = yearsParam.split(',').filter(Boolean).map(Number)
const valid = parsed.length > 0 && parsed.every((y) => Number.isInteger(y) && y >= 1900 && y <= 2200)
if (!valid) {
return errorResponseFromCode('VALIDATION_ERROR', moduleLog, {
details: { field: 'years', reason: 'comma-separated fiscal year start years' },
})
}
requestedYears = [...new Set(parsed)].sort((a, b) => a - b)
// The resource bound: each selected year is one provider export
// fetched and parsed in this invocation (derivation on the
// constant). Rejected before any provider call.
if (requestedYears.length > MAX_SELECTED_FISCAL_YEARS) {
return errorResponseFromCode('VALIDATION_ERROR', moduleLog, {
messageSv: `Högst ${MAX_SELECTED_FISCAL_YEARS} räkenskapsår kan hämtas per körning. Välj färre år; äldre år kan hämtas i en ny körning.`,
messageEn: `At most ${MAX_SELECTED_FISCAL_YEARS} fiscal years can be fetched per run. Select fewer years; older years can be fetched in a second run.`,
details: { field: 'years', reason: 'too_many', max: MAX_SELECTED_FISCAL_YEARS, requested: requestedYears.length },
})
}
}
try {
// Resolve consent
const resolved = await resolveConsent(companyId, consentId)
@@ -947,19 +993,38 @@ export const arcimMigrationExtension: Extension = {
})
}
// Fetch SIE type 4 for each allowed fiscal year
const { files: sieFiles, failedYears } = await fetchProviderSieFiles(
provider,
resolved.accessToken,
resolved.providerCompanyId,
)
// Fetch SIE type 4 for each selected fiscal year. A selected year
// the source does not have is refused by the fetcher right after
// the year listing, before any export is fetched.
let fetched: Awaited<ReturnType<typeof fetchProviderSieFiles>>
try {
fetched = await fetchProviderSieFiles(
provider,
resolved.accessToken,
resolved.providerCompanyId,
requestedYears ? { years: requestedYears } : undefined,
)
} catch (err) {
if (err instanceof FiscalYearSelectionError) {
const years = err.unknownYears.join(', ')
return errorResponseFromCode('VALIDATION_ERROR', moduleLog, {
messageSv: `Räkenskapsår ${years} finns inte hos leverantören.`,
messageEn: `Fiscal years ${years} do not exist at the provider.`,
details: { field: 'years', reason: 'unknown_year', unknownYears: err.unknownYears },
})
}
throw err
}
const { files: sieFiles, failedYears, omittedYears } = fetched
if (sieFiles.length === 0) {
// The allowed window is rolling (current year and the two before
// it): interpolate the actual range instead of the static
// registry message so the text never goes stale.
const allowedYears = [...getAllowedFiscalYears()].sort((a, b) => a - b)
const range = `${allowedYears[0]}-${allowedYears[allowedYears.length - 1]}`
// Name the selection: the explicit years when the picker sent
// them, else the rolling default window (interpolated so the
// text never goes stale).
const selection = requestedYears ?? [...getAllowedFiscalYears()].sort((a, b) => a - b)
const range = requestedYears
? requestedYears.join(', ')
: `${selection[0]}-${selection[selection.length - 1]}`
return errorResponseFromCode('PROVIDER_SIE_NO_YEARS', moduleLog, {
messageSv: `Inga räkenskapsår ${range} hittades hos leverantören.`,
messageEn: `No fiscal years available for ${range}.`,
@@ -1112,6 +1177,9 @@ export const arcimMigrationExtension: Extension = {
// Allowed years whose provider export failed: the wizard warns
// the user before proceeding so an IB/UB gap cannot slip through.
failedYears,
// Source years outside the selection: the result step names them
// so nobody believes the books are complete (#2211).
omittedYears,
basAccounts: mappingTargets,
})
} catch (error) {
extensions/general/arcim-migration/lib/__tests__/sie-fetcher.test.ts
+120 -1
View File
@@ -3,6 +3,8 @@ import {
providerSupportsSie,
fetchProviderSieFiles,
getAllowedFiscalYears,
FiscalYearSelectionError,
MAX_SELECTED_FISCAL_YEARS,
} from '../sie-fetcher'
// The allowed window is rolling (current year and the two before it): derive
@@ -54,7 +56,7 @@ function octetResponse(bytes: Uint8Array): Response {
})
}
describe('getAllowedFiscalYears', () => {
describe('getAllowedFiscalYears (the default selection)', () => {
it('is a rolling three-year window ending at the current year', () => {
const years = getAllowedFiscalYears(new Date('2031-06-15'))
expect([...years].sort((a, b) => a - b)).toEqual([2029, 2030, 2031])
@@ -68,6 +70,14 @@ describe('getAllowedFiscalYears', () => {
})
})
describe('MAX_SELECTED_FISCAL_YEARS', () => {
it('is six: 6 years x 48 s worst case (3 x 15 s timeout + 1 s + 2 s backoff) = 288 s inside the 300 s function', () => {
expect(MAX_SELECTED_FISCAL_YEARS).toBe(6)
expect(MAX_SELECTED_FISCAL_YEARS * (3 * 15 + 1 + 2)).toBeLessThan(300)
expect((MAX_SELECTED_FISCAL_YEARS + 1) * (3 * 15 + 1 + 2)).toBeGreaterThan(300)
})
})
describe('providerSupportsSie', () => {
it('is true for providers with SIE-over-API, false otherwise', () => {
expect(providerSupportsSie('fortnox')).toBe(true)
@@ -125,6 +135,17 @@ describe('fetchProviderSieFiles', () => {
expect(result.availableYears).toEqual([CY - 2, CY - 1])
expect(result.files.map((f) => f.fiscalYear)).toEqual([CY - 2, CY - 1])
expect(result.failedYears).toEqual([])
// Every source year is NAMED with the provider's own bounds and its
// default-selection flag (the picker), and the one outside the
// selection is reported as omitted (#2211): nothing is left out silently.
expect(result.sourceYears).toEqual([
{ year: CY - 3, fromDate: `${CY - 3}-01-01`, toDate: `${CY - 3}-12-31`, inDefaultSelection: false },
{ year: CY - 2, fromDate: `${CY - 2}-01-01`, toDate: `${CY - 2}-12-31`, inDefaultSelection: true },
{ year: CY - 1, fromDate: `${CY - 1}-01-01`, toDate: `${CY - 1}-12-31`, inDefaultSelection: true },
])
expect(result.omittedYears).toEqual([
{ year: CY - 3, fromDate: `${CY - 3}-01-01`, toDate: `${CY - 3}-12-31`, inDefaultSelection: false },
])
// CP437 bytes decoded into proper Swedish characters
expect(result.files[0].rawContent).toContain(`Företagskonto ${CY - 2}`)
expect(result.files[1].rawContent).toContain(`Företagskonto ${CY - 1}`)
@@ -174,6 +195,9 @@ describe('fetchProviderSieFiles', () => {
respond: () =>
jsonResponse({
FinancialYears: [
// A broken first year (issue #2211): starts before the window,
// so it is left out. Listed first as Fortnox does.
{ Id: 4, FromDate: `${CY - 4}-09-01`, ToDate: `${CY - 3}-12-31` },
{ Id: 5, FromDate: `${CY - 2}-01-01`, ToDate: `${CY - 2}-12-31` },
{ Id: 6, FromDate: `${CY - 1}-01-01`, ToDate: `${CY - 1}-12-31` },
],
@@ -198,10 +222,84 @@ describe('fetchProviderSieFiles', () => {
const sieUrls = fetchSpy.mock.calls
.map((c: unknown[]) => String(c[0]))
.filter((u: string) => u.includes('/sie/4'))
expect(sieUrls).toHaveLength(2)
expect(sieUrls[0]).toContain('financialyear=5')
expect(sieUrls[1]).toContain('financialyear=6')
})
it('names the broken first year outside the default selection, with the bounds Fortnox reports (#2211)', async () => {
routeFetch(fetchSpy, [
yearRoutes,
{
match: '/sie/4?financialyear=',
respond: () => new Response('#FLAGGA 0\n#KONTO 1930 "Företagskonto"\n', { status: 200 }),
},
])
const result = await fetchProviderSieFiles('fortnox', 'token', undefined)
// Derived from the /financialyears list already fetched: no extra call,
// and no SIE export for the year outside the selection.
const broken = { year: CY - 4, fromDate: `${CY - 4}-09-01`, toDate: `${CY - 3}-12-31`, inDefaultSelection: false }
expect(result.sourceYears[0]).toEqual(broken)
expect(result.sourceYears.map((fy) => fy.inDefaultSelection)).toEqual([false, true, true])
expect(result.omittedYears).toEqual([broken])
const fyListCalls = fetchSpy.mock.calls.filter((c: unknown[]) => String(c[0]).includes('/financialyears'))
expect(fyListCalls).toHaveLength(1)
expect(fetchSpy.mock.calls.map((c: unknown[]) => String(c[0]))).not.toContainEqual(
expect.stringContaining('financialyear=4'),
)
})
it('refuses a selected year the source does not have, before any SIE export', async () => {
routeFetch(fetchSpy, [
yearRoutes,
{
match: '/sie/4?financialyear=',
respond: () => new Response('#FLAGGA 0\n#KONTO 1930 "Företagskonto"\n', { status: 200 }),
},
])
const attempt = fetchProviderSieFiles('fortnox', 'token', undefined, { years: [CY - 1, CY - 9] })
await expect(attempt).rejects.toBeInstanceOf(FiscalYearSelectionError)
await expect(attempt).rejects.toMatchObject({ unknownYears: [CY - 9] })
// Only the year listing was called: the unknown year cost no export,
// and neither did the known one.
const urls = fetchSpy.mock.calls.map((c: unknown[]) => String(c[0]))
expect(urls.filter((u: string) => u.includes('/financialyears'))).toHaveLength(1)
expect(urls.filter((u: string) => u.includes('/sie/4'))).toHaveLength(0)
})
it('fetches exactly the years the picker selected, oldest first (#2238)', async () => {
routeFetch(fetchSpy, [
yearRoutes,
{
match: '/sie/4?financialyear=',
respond: () => new Response('#FLAGGA 0\n#KONTO 1930 "Företagskonto"\n', { status: 200 }),
},
])
// The broken first year ticked, the middle default year unticked.
const result = await fetchProviderSieFiles('fortnox', 'token', undefined, {
years: [CY - 1, CY - 4],
})
expect(result.availableYears).toEqual([CY - 4, CY - 1])
expect(result.files.map((f) => f.fiscalYear)).toEqual([CY - 4, CY - 1])
const sieUrls = fetchSpy.mock.calls
.map((c: unknown[]) => String(c[0]))
.filter((u: string) => u.includes('/sie/4'))
expect(sieUrls).toHaveLength(2)
expect(sieUrls[0]).toContain('financialyear=4')
expect(sieUrls[1]).toContain('financialyear=6')
// The unticked default year is the omitted one now; the flag still
// says what the default would have been.
expect(result.omittedYears).toEqual([
{ year: CY - 2, fromDate: `${CY - 2}-01-01`, toDate: `${CY - 2}-12-31`, inDefaultSelection: true },
])
})
it('decodes CP437 bytes from the Fortnox SIE endpoint (no blind UTF-8 text())', async () => {
// Some Fortnox endpoint variants serve the SIE body in CP437 (the SIE
// spec encoding). A blind response.text() would turn å/ä/ö into U+FFFD
@@ -237,6 +335,13 @@ describe('fetchProviderSieFiles', () => {
match: '/financialyear',
respond: () =>
jsonResponse([
{
entityId: 0,
id: `${CY - 5}01`,
fromDate: `${CY - 5}-01-01`,
toDate: `${CY - 5}-12-31`,
open: false,
},
{
entityId: 1,
id: `${CY - 1}01`,
@@ -253,6 +358,12 @@ describe('fetchProviderSieFiles', () => {
expect(result.files).toHaveLength(1)
expect(result.files[0].fiscalYear).toBe(CY - 1)
expect(result.files[0].rawContent).toContain(`Företagskonto ${CY - 1}`)
expect(result.omittedYears).toEqual([
{ year: CY - 5, fromDate: `${CY - 5}-01-01`, toDate: `${CY - 5}-12-31`, inDefaultSelection: false },
])
expect(fetchSpy.mock.calls.map((c: unknown[]) => String(c[0]))).not.toContainEqual(
expect.stringContaining(`/sie/export/${CY - 5}`),
)
const exportCall = fetchSpy.mock.calls.find((c: unknown[]) => String(c[0]).includes('/sie/export/'))
expect(exportCall).toBeDefined()
@@ -325,6 +436,8 @@ describe('fetchProviderSieFiles: wint (SIE rendered from voucher data)', () => {
Name: 'Bolaget AB',
Org: '556699-0011',
FinancialYears: [
// Before the window: named in omittedYears, never fetched.
{ Id: 0, Start: `${CY - 3}-01-01T00:00:00`, End: `${CY - 3}-12-31T00:00:00` },
{ Id: 1, Start: `${CY - 1}-01-01T00:00:00`, End: `${CY - 1}-12-31T00:00:00` },
{ Id: 2, Start: `${CY}-01-01T00:00:00`, End: `${CY}-12-31T00:00:00` },
],
@@ -385,6 +498,12 @@ describe('fetchProviderSieFiles: wint (SIE rendered from voucher data)', () => {
expect(result.failedYears).toEqual([])
expect(result.availableYears).toEqual([CY - 1, CY])
expect(result.files.map((f) => f.fiscalYear)).toEqual([CY - 1, CY])
expect(result.omittedYears).toEqual([
{ year: CY - 3, fromDate: `${CY - 3}-01-01`, toDate: `${CY - 3}-12-31`, inDefaultSelection: false },
])
expect(fetchSpy.mock.calls.map((c: unknown[]) => String(c[0]))).not.toContainEqual(
expect.stringContaining(`BookingDateStart=${CY - 3}`),
)
const prev = result.files[0].rawContent
const curr = result.files[1].rawContent
extensions/general/arcim-migration/lib/sie-fetcher.ts
+113 -18
View File
@@ -29,7 +29,13 @@ import { createLogger } from '@/lib/logger'
const log = createLogger('extensions/arcim-migration/sie-fetcher')
/**
* Fiscal years we support importing: the current year and the two before it.
* The DEFAULT selection of fiscal years: the current year and the two before
* it, keyed on each fiscal year's start year. A default, not a cap (issue
* #2211 / #2238): the wizard lets the user add older years, and
* fetchProviderSieFiles takes an explicit `years` selection. The default is
* the cost bound: every selected year is one SIE export fetched and parsed
* inside the single /sie-data invocation (hosted function limit 300 s), so
* an older history is the user's own wait, chosen in the preview step.
* Derived at call time (not a module constant) so the window rolls forward
* automatically at new year without a code change.
*/
@@ -38,6 +44,32 @@ export function getAllowedFiscalYears(now: Date = new Date()): Set<number> {
return new Set([currentYear - 2, currentYear - 1, currentYear])
}
/**
* The most fiscal years one import run may select. The bound is the single
* /sie-data invocation that fetches and parses one SIE export per selected
* year: hosted function limit 300 s; one export call is 15 s per attempt
* (FETCH_TIMEOUT_MS in lib/providers/fortnox/client.ts), 3 attempts with
* 1 s and 2 s backoff between them (lib/providers/retry.ts defaults, capped
* at 30 s), so a year that times out on every attempt costs 15 + 1 + 15 +
* 2 + 15 = 48 s. Six such years are 288 s, which leaves the remaining 12 s
* for the /financialyears listing, parsing and the response. Older years
* beyond the cap go in a second run. Enforced server-side in /sie-data and
* mirrored by the preview step's picker (which reads it from /preview).
*/
export const MAX_SELECTED_FISCAL_YEARS = 6
/**
* Thrown by fetchProviderSieFiles when an explicit selection names a year
* the source does not have: raised right after the year listing, before any
* SIE export is fetched, so an unknown year never costs a provider export.
*/
export class FiscalYearSelectionError extends Error {
constructor(public readonly unknownYears: number[]) {
super(`Fiscal years not found at the provider: ${unknownYears.join(', ')}`)
this.name = 'FiscalYearSelectionError'
}
}
export interface ProviderSieFile {
fiscalYear: number
rawContent: string
@@ -46,17 +78,45 @@ export interface ProviderSieFile {
export interface ProviderSieFetchResult {
files: ProviderSieFile[]
/**
* Every fiscal year available at the provider within the allowed window:
* also populated when latestOnly fetched just one file, so /preview can show
* the full year list without a second round-trip.
* Every fiscal year available at the provider within the selection (the
* default window, or the explicit `years`): also populated when latestOnly
* fetched just one file, so /preview can show the full year list without a
* second round-trip.
*/
availableYears: number[]
/**
* Every fiscal year the source has, oldest first, with the provider's own
* bounds and whether it is in the default selection. The preview step
* renders these as the year picker, so no year can be left out silently.
*/
sourceYears: SourceFiscalYear[]
/**
* Allowed years whose export failed (or came back empty). Callers MUST
* surface these to the user: silently importing e.g. 2024+2026 without 2025
* breaks IB/UB continuity between the years without anyone noticing.
*/
failedYears: { year: number; error: string }[]
/**
* Source fiscal years that were NOT part of this fetch (outside the
* selection), oldest first. Until issue #2211 these were never mentioned:
* the result step names them so nobody believes the books are complete.
*/
omittedYears: SourceFiscalYear[]
}
/**
* A fiscal year as the source reports it. Bounds are the provider's own
* (ISO yyyy-mm-dd) so a broken year can be named as "2022-09-01 till
* 2023-12-31" rather than as a calendar year that is wrong for it; null when
* the provider reported none. `year` (the start year) is the key the whole
* import uses for a fiscal year, and what `fetchProviderSieFiles` selects on.
*/
export interface SourceFiscalYear {
year: number
fromDate: string | null
toDate: string | null
/** True when the year falls in the default selection (getAllowedFiscalYears). */
inDefaultSelection: boolean
}
// Singleton clients (they hold rate limiters)
@@ -85,31 +145,53 @@ interface FiscalYearRef {
}
/**
* Fetch SIE type-4 exports from the provider, one file per allowed fiscal
* year (oldest first). Years whose export fails do not block the rest of the
* migration, but they are reported in `failedYears` so the caller can warn
* the user before importing a gap (IB/UB continuity).
* Fetch SIE type-4 exports from the provider, one file per selected fiscal
* year (oldest first). The selection is `opts.years` (start years, as the
* preview step's picker sends them) or, when absent, the default window.
* Years whose export fails do not block the rest of the migration, but they
* are reported in `failedYears` so the caller can warn the user before
* importing a gap (IB/UB continuity). Years at the source outside the
* selection are reported in `omittedYears`; the full list is `sourceYears`.
*/
export async function fetchProviderSieFiles(
provider: ProviderName,
accessToken: string,
providerCompanyId: string | undefined,
opts?: { latestOnly?: boolean },
opts?: { latestOnly?: boolean; years?: number[] },
): Promise<ProviderSieFetchResult> {
const fetcher = getSieFetcher(provider, providerCompanyId)
const fetcher = getSieFetcher(provider, providerCompanyId, opts?.years)
if (!fetcher) {
throw new Error(`Provider ${provider} does not support SIE over API`)
}
const allowedFiscalYears = getAllowedFiscalYears()
const allYears = await fetcher.listYears(accessToken)
const allowedYears = allYears
.filter((fy) => allowedFiscalYears.has(fy.year))
.sort((a, b) => a.year - b.year)
const defaultFiscalYears = getAllowedFiscalYears()
const selectedFiscalYears = opts?.years ? new Set(opts.years) : defaultFiscalYears
const byYear = (a: FiscalYearRef, b: FiscalYearRef) =>
a.year - b.year || (a.fromDate ?? '').localeCompare(b.fromDate ?? '')
const allYears = (await fetcher.listYears(accessToken)).sort(byYear)
if (opts?.years) {
// Reject an unknown year here, before the first export: the listing is
// the only provider call made so far.
const known = new Set(allYears.map((fy) => fy.year))
const unknownYears = opts.years.filter((y) => !known.has(y))
if (unknownYears.length > 0) throw new FiscalYearSelectionError(unknownYears)
}
const allowedYears = allYears.filter((fy) => selectedFiscalYears.has(fy.year))
const availableYears = allowedYears.map((fy) => fy.year)
const toFetch = opts?.latestOnly ? allowedYears.slice(-1) : allowedYears
// Both derived from the year list already fetched: naming every source
// year, and the ones left out, costs no extra provider call.
const describe = (fy: FiscalYearRef): SourceFiscalYear => ({
year: fy.year,
fromDate: fy.fromDate ?? null,
toDate: fy.toDate ?? null,
inDefaultSelection: defaultFiscalYears.has(fy.year),
})
const sourceYears = allYears.map(describe)
const omittedYears = allYears.filter((fy) => !selectedFiscalYears.has(fy.year)).map(describe)
const files: ProviderSieFile[] = []
const failedYears: { year: number; error: string }[] = []
for (const fy of toFetch) {
@@ -129,7 +211,7 @@ export async function fetchProviderSieFiles(
}
}
return { files, availableYears, failedYears }
return { files, availableYears, sourceYears, failedYears, omittedYears }
}
interface SieFetcher {
@@ -140,6 +222,7 @@ interface SieFetcher {
function getSieFetcher(
provider: ProviderName,
providerCompanyId: string | undefined,
selectedYears?: number[],
): SieFetcher | null {
if (provider === 'fortnox') {
return {
@@ -152,6 +235,8 @@ function getSieFetcher(
return years.map((fy) => ({
id: fy['Id'] as number,
year: new Date(fy['FromDate'] as string).getFullYear(),
fromDate: typeof fy['FromDate'] === 'string' ? fy['FromDate'] : undefined,
toDate: typeof fy['ToDate'] === 'string' ? fy['ToDate'] : undefined,
}))
},
async fetchSie(accessToken, fy) {
@@ -172,6 +257,8 @@ function getSieFetcher(
return years.map((fy) => ({
id: fy.id,
year: new Date(fy.fromdate).getFullYear(),
fromDate: fy.fromdate,
toDate: fy.todate,
}))
},
async fetchSie(accessToken, fy) {
@@ -194,6 +281,9 @@ function getSieFetcher(
companyName: string
orgNumber?: string
accounts: ReturnType<typeof mapWintAccountForSie>[]
/** Every year the source has, unfiltered: what listYears reports. */
allYears: (WintSieYear & { id: number })[]
/** The allowed subset: the years that render as SIE. */
years: (WintSieYear & { id: number })[]
vouchersByYear: Map<number, WintSieVoucher[]>
ibByYear: Map<number, Map<string, number>>
@@ -205,7 +295,9 @@ function getSieFetcher(
contextPromise ??= (async () => {
const company = await wintClient.get<Record<string, unknown>>(accessToken, '/api/Auth')
const rawYears = (company['FinancialYears'] as Record<string, unknown>[] | undefined) ?? []
const allowed = getAllowedFiscalYears()
// The same selection fetchProviderSieFiles applies: the explicit
// years when given, else the default window.
const allowed = selectedYears ? new Set(selectedYears) : getAllowedFiscalYears()
const allYears = rawYears
.map((fy) => ({
id: Number(fy['Id']),
@@ -273,6 +365,7 @@ function getSieFetcher(
companyName: (company['Name'] as string) ?? 'Okänt företag',
orgNumber: (company['Org'] as string | undefined) || undefined,
accounts,
allYears,
years,
vouchersByYear,
ibByYear,
@@ -284,8 +377,10 @@ function getSieFetcher(
return {
async listYears(accessToken) {
// The unfiltered list: fetchProviderSieFiles applies the window
// itself and needs the years outside it to name what is left out.
const context = await loadContext(accessToken)
return context.years.map((fy) => ({
return context.allYears.map((fy) => ({
id: fy.id,
year: fy.year,
fromDate: fy.start,
extensions/general/mcp-server/server.ts
+2 -2
View File
@@ -4114,7 +4114,7 @@ export const tools: McpTool[] = [
keywords: ['migrering', 'byta system', 'flytta bokföring'],
title: 'Connect Previous System',
description:
'Connect card into the migration wizard for a NAMED previous system. API systems (fortnox/bjornlunden/briox/wint) fetch all fiscal years plus invoices, customers and documents; visma/bokio complement AFTER a SIE import. Same one-click feel as the bank/Skatteverket cards.',
'Connect card into the migration wizard for a NAMED previous system. API systems (fortnox/bjornlunden/briox/wint) fetch the three latest fiscal years by default (older years selectable) plus invoices, customers and documents; visma/bokio complement AFTER a SIE import.',
inputSchema: {
type: 'object',
additionalProperties: false,
@@ -4165,7 +4165,7 @@ export const tools: McpTool[] = [
api_connected: info.api,
connect_url: connectUrl,
instructions: info.api
? `On claude.ai/Claude Desktop a connect card with an open-in-browser button renders with this result; elsewhere give the user the connect_url. The wizard connects to ${info.name} (login there), fetches every fiscal year and imports bookkeeping PLUS invoices, customers, suppliers and documents. The user comes back here when the wizard reports done.`
? `On claude.ai/Claude Desktop a connect card with an open-in-browser button renders with this result; elsewhere give the user the connect_url. The wizard connects to ${info.name} (login there), fetches the three latest fiscal years by default (older years can be ticked in the wizard; they take longer) and imports bookkeeping PLUS invoices, customers, suppliers and documents. The user comes back here when the wizard reports done.`
: `${info.name} has no API export: run the SIE-file import FIRST (gnubok_create_sie_upload drop card). This wizard link then complements with invoices and customers. On claude.ai/Desktop a connect card renders; elsewhere give the user the connect_url.`,
}
},
extensions/general/mcp-server/skills/onboarding.ts
+4 -3
View File
@@ -119,9 +119,10 @@ reaches far enough back anyway.
offer TWO paths and recommend by need. The FULL migration: call
\`gnubok_connect_migration\` with the provider; it renders a connect
card (same feel as bank/Skatteverket) whose button opens the wizard
that logs into the old system and fetches every fiscal year PLUS
invoices, customers, suppliers and documents: recommend it when they
have open fakturor or want underlag along. The QUICK path is a SIE
that logs into the old system and fetches the three latest fiscal
years by default (older years can be ticked in the wizard; they take
longer) PLUS invoices, customers, suppliers and documents: recommend
it when they have open fakturor or want underlag along. The QUICK path is a SIE
export dropped here (Fortnox: Register → Exportera → SIE 4): ledger
only, fastest. Either way the result lands in the same books.
- **Visma eEkonomi / Bokio**: no API export exists; ask for the SIE
messages/en.json
+9
View File
@@ -5627,6 +5627,15 @@
"ext_arcim_documents_provider_message": "Source system response: {message}",
"ext_arcim_option_series_help": "Vouchers keep their series from the source system (A, D, E ...). The series here is only used for vouchers without one.",
"ext_arcim_option_sie_required_hint": "The ledger (chart of accounts, opening balances and verifications) has not been imported yet. Tick Bokföringsdata (SIE) to fetch it in the same run: customers, suppliers and invoices cannot be imported without it.",
"ext_arcim_year_select_kicker": "Fiscal years to fetch",
"ext_arcim_year_select_lede": "The three most recent fiscal years are selected by default. Tick older years to fetch them too: it takes longer, and attached documents come along.",
"ext_arcim_year_select_older": "takes longer",
"ext_arcim_year_select_none": "Select at least one fiscal year.",
"ext_arcim_year_select_too_many": "At most {max} fiscal years can be fetched at once. Older years can be fetched in a second run.",
"ext_arcim_omitted_years_kicker": "Fiscal years not fetched",
"ext_arcim_omitted_years_result": "{count, plural, one {Not selected in this run. Run a new migration with it ticked to fetch that year too (attached documents come along), or upload it as a SIE file under Import.} other {Not selected in this run. Run a new migration with them ticked to fetch those years too (attached documents come along), or upload them as one SIE file per year under Import, oldest first.}}",
"ext_arcim_omitted_years_sie_link": "Go to the SIE import",
"ext_arcim_fiscal_year_span": "{from} to {to}",
"ext_arcim_documents_result_description": "The document import is complete.",
"ext_arcim_documents_imported": "Imported",
"ext_arcim_documents_skipped": "Already present",
messages/sv.json
+9
View File
@@ -5627,6 +5627,15 @@
"ext_arcim_documents_provider_message": "Svar från källsystemet: {message}",
"ext_arcim_option_series_help": "Verifikat behåller sin serie från källsystemet (A, D, E ...). Serien här används bara för verifikat som saknar serie.",
"ext_arcim_option_sie_required_hint": "Bokföringen (kontoplan, ingående balanser och verifikationer) har inte importerats än. Kryssa i Bokföringsdata (SIE) för att hämta den i samma körning: kunder, leverantörer och fakturor kan inte importeras utan den.",
"ext_arcim_year_select_kicker": "Räkenskapsår att hämta",
"ext_arcim_year_select_lede": "De tre senaste räkenskapsåren är förvalda. Bocka i äldre år för att hämta dem också: det tar längre tid, och kopplade underlag följer med.",
"ext_arcim_year_select_older": "tar längre tid",
"ext_arcim_year_select_none": "Välj minst ett räkenskapsår.",
"ext_arcim_year_select_too_many": "Högst {max} räkenskapsår kan hämtas åt gången. Äldre år kan hämtas i en ny körning.",
"ext_arcim_omitted_years_kicker": "Räkenskapsår som inte hämtades",
"ext_arcim_omitted_years_result": "{count, plural, one {Inte valt i den här körningen. Kör en ny migrering och bocka i det för att hämta även det året (kopplade underlag följer med), eller ladda upp det som en SIE-fil under Import.} other {Inte valda i den här körningen. Kör en ny migrering och bocka i dem för att hämta även de åren (kopplade underlag följer med), eller ladda upp dem som en SIE-fil per år under Import, äldsta året först.}}",
"ext_arcim_omitted_years_sie_link": "Gå till SIE-importen",
"ext_arcim_fiscal_year_span": "{from} till {to}",
"ext_arcim_documents_result_description": "Underlagsimporten är klar.",
"ext_arcim_documents_imported": "Importerade",
"ext_arcim_documents_skipped": "Fanns redan",