feat(import): let the Fortnox import fetch older fiscal years: the three-year limit becomes a default selection (#2280)
* fix(import): say which fiscal years the Fortnox connection fetches, list the ones left out Root cause: the guided provider migration fetches SIE only for fiscal years that start within a rolling three-calendar-year window (getAllowedFiscalYears in extensions/general/arcim-migration/lib/sie-fetcher.ts: current year and the two before it). A first, broken year 2022/2023 starts in 2022 and falls outside the window in 2026, and the wizard said nothing: not before the import, not after. Users concluded the books were complete, or that they had done something wrong (issue #2211, second report via support 2026-08-27). Fix: - The fetcher already lists every fiscal year at the source before applying the window, so the left-out years are derived from that same list at no extra provider call: `omittedYears` (years starting before the window, oldest first, with the provider's own from/to dates so a broken year is named as "2022-09-01 till 2023-12-31"). Fortnox and Briox year refs now carry those bounds; WINT's listYears reports the unfiltered year list. - GET /preview returns `fiscalYearWindow` and `omittedYears`; GET /sie-data returns `omittedYears` next to `failedYears`. - Wizard, preview step (before the import runs): one muted sentence that the direct connection fetches the three latest fiscal years (years starting in {fromYear} or later); when years are left out, they are named with a link to the SIE import (one SIE file per year under Import, oldest first). - Wizard, result step: a "Räkenskapsår som inte följde med" section naming the omitted years with the same SIE pointer, shown when SIE data was imported in the run. - MCP: the connect_migration tool description, its instructions and the onboarding skill claimed the wizard "fetches every fiscal year"; they now say three latest, older years via SIE. - Strings in both messages/sv.json and messages/en.json. Out of scope: fetching more years through the connection (#2238). Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_015qgLgdt4mLmha1ZLFMwq1u * feat(import): make the Fortnox import's three-year limit a default selection, not a cap Root cause, from first principles: the guided provider migration fetched SIE only for fiscal years starting within a rolling three-calendar-year window (getAllowedFiscalYears in extensions/general/arcim-migration/lib/sie-fetcher.ts, introduced in #718 with no stated reason). The window was a silent cap: the wizard never said it existed, and a first broken year 2022/2023 simply never arrived (#2211). The user's actual problem is that the year is missing from the books, so explaining the cap (the previous commit) treats the symptom. What the window gated, by evidence: only the SIE fetch. Documents already list every Fortnox financial year and match against the vouchers that exist locally (import-documents.ts), invoices, customers, suppliers and assets are not year-gated, and the SIE import itself is one request per year (hosted function limit 300 s, import_sie_journal_entries statement_timeout 290 s), so its cost is linear in wall time and bounded per year regardless of how many years are imported. The only place the number of years multiplies inside one invocation is /preview and /sie-data: one SIE export per year (Fortnox client: 15 s per-call timeout, 3 attempts, backoff up to 30 s, 4 req/s) fetched and parsed inside a single 300 s function, and /sie-data returns every raw file in one response. The repo holds no measurement of Fortnox's per-year SIE export latency, and the maintainer's memory is that a full history can take unreasonably long, so a fixed lift to every year cannot be shown safe for a long history. Fix: the window becomes the DEFAULT selection, and the user chooses. - sie-fetcher: fetchProviderSieFiles takes `years` (explicit start years); without it the default window applies. The result carries `sourceYears` (every year at the source, oldest first, with the provider's own bounds and an inDefaultSelection flag) and `omittedYears` (source years outside the selection). Both derived from the year list already fetched: no extra provider call. Fortnox and Briox year refs carry their bounds; WINT's listYears reports the unfiltered list and its voucher chain follows the selection. - GET /preview returns `sourceYears`; GET /sie-data honours `?years=` (validated, deduplicated, oldest first; 400 VALIDATION_ERROR when malformed) and returns `omittedYears`. PROVIDER_SIE_NO_YEARS names the selection. - Wizard, preview step: a "Räkenskapsår att hämta" picker with one checkbox row per source year, the three latest ticked by default, older years marked "tar längre tid"; Fortsätt is disabled with an attn line until at least one year is ticked. The selection is sent to /sie-data, so each extra year is the user's own wait, and it fails loudly there, before any ledger write, if it is too much. - Wizard, result step: the per-year lines already report exactly what was imported; a "Räkenskapsår som inte hämtades" section names the source years outside the selection, with the re-run path (documents come along) and the SIE path. - MCP connect_migration description, instructions and the onboarding skill say "three latest by default, older years selectable" instead of "every fiscal year". - Strings in both messages/sv.json and messages/en.json. Closes #2238 as well: the wish to fetch more years is the same control. Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_015qgLgdt4mLmha1ZLFMwq1u * fix(import): bound the fiscal-year selection per import run Superagent P2 on #2280: the `years` selection was unbounded and every selected year is one provider export fetched and parsed inside the single 300 s /sie-data invocation, so nothing bounded the work before provider calls. The bound: MAX_SELECTED_FISCAL_YEARS = 6, exported from sie-fetcher.ts with the derivation. One export call is 15 s per attempt (Fortnox client FETCH_TIMEOUT_MS), 3 attempts with 1 s and 2 s backoff (retry defaults), so a year that times out on every attempt costs 48 s; six such years are 288 s, leaving 12 s of the 300 s hosted function for the year listing, parsing and the response; seven would be 336 s. Enforced server-side: - /sie-data refuses a selection of more than the cap with 400 VALIDATION_ERROR naming the cap, before the consent is resolved, so an oversized request does no provider work. - fetchProviderSieFiles throws FiscalYearSelectionError for a selected year the source does not have, right after the year listing and before any export; /sie-data maps it to 400 VALIDATION_ERROR naming the year. - /preview returns maxSelectedYears so the picker enforces the same number without a client-side copy: Fortsätt is disabled and an attn line says how many can be fetched at once and that older years go in a second run (sv + en). Tests: cap accepted at 6 and refused at 7 with no provider call, unknown year refused (route and fetcher), the cap's arithmetic, maxSelectedYears on /preview. Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_015qgLgdt4mLmha1ZLFMwq1u --------- Co-authored-by: Jakob Wennberg <311770904+jakobwennberg-oss@users.noreply.github.com> Co-authored-by: Claude Fable 5.1 <noreply@anthropic.com>
This commit is contained in:
Jakob Wennberg
co-authored by
Claude Fable 5.1
Jakob Wennberg
Claude Fable 5.1
Jakob Wennberg
parent
7a9036caa6
commit
1976129478
@@ -9,6 +9,7 @@ import { Progress } from '@/components/ui/progress'
|
||||
import { Button } from '@/components/ui/button'
|
||||
import { Input } from '@/components/ui/input'
|
||||
import { Switch } from '@/components/ui/switch'
|
||||
import { Checkbox } from '@/components/ui/checkbox'
|
||||
import { useToast } from '@/components/ui/use-toast'
|
||||
import { cn } from '@/lib/utils'
|
||||
import { ConfirmationDialog } from '@/components/ui/confirmation-dialog'
|
||||
@@ -304,6 +305,13 @@ interface PreviewData {
|
||||
transactionCount: number
|
||||
fiscalYears: number[]
|
||||
} | null
|
||||
// Every fiscal year the source has, oldest first, with the default
|
||||
// selection marked: rendered as the year picker so the user chooses
|
||||
// before the import runs and no year is left out silently (#2211, #2238).
|
||||
sourceYears?: SourceFiscalYear[]
|
||||
// The most years one run may select: /sie-data refuses more. Read from
|
||||
// the server so the picker never drifts from the route.
|
||||
maxSelectedYears?: number
|
||||
assetStats: {
|
||||
total: number
|
||||
importable: number
|
||||
@@ -338,9 +346,33 @@ interface SIEData {
|
||||
// Fiscal years whose provider export failed. Importing the remaining years
|
||||
// anyway leaves an IB/UB gap: the options step warns before proceeding.
|
||||
failedYears?: { year: number; error: string }[]
|
||||
// Source fiscal years outside the selection: not fetched, named in the
|
||||
// result so nobody believes the books are complete (#2211).
|
||||
omittedYears?: SourceFiscalYear[]
|
||||
basAccounts: BASAccount[]
|
||||
}
|
||||
|
||||
/**
|
||||
* A fiscal year as the source reports it. Mirrors SourceFiscalYear in
|
||||
* extensions/general/arcim-migration/lib/sie-fetcher.ts (deliberate
|
||||
* duplication: core must not import from @/extensions/).
|
||||
*/
|
||||
interface SourceFiscalYear {
|
||||
year: number
|
||||
fromDate: string | null
|
||||
toDate: string | null
|
||||
inDefaultSelection: boolean
|
||||
}
|
||||
|
||||
/** "2022-09-01 till 2023-12-31" when the provider gave bounds, else the start year. */
|
||||
function useFiscalYearSpanLabel(): (fy: SourceFiscalYear) => string {
|
||||
const t = useTranslations('extensions')
|
||||
return (fy) =>
|
||||
fy.fromDate && fy.toDate
|
||||
? t('ext_arcim_fiscal_year_span', { from: fy.fromDate, to: fy.toDate })
|
||||
: String(fy.year)
|
||||
}
|
||||
|
||||
// ── Shared step chrome ───────────────────────────────────────────
|
||||
// Living Paper: step content sits directly on the page. The serif headline
|
||||
// is the step's one display element; sections are kickers over hairline
|
||||
@@ -875,6 +907,8 @@ function PreviewStep({
|
||||
error,
|
||||
authExpired,
|
||||
licenseMissing,
|
||||
selectedYears,
|
||||
onSelectedYearsChange,
|
||||
onReconnect,
|
||||
onContinue,
|
||||
onBack,
|
||||
@@ -884,13 +918,30 @@ function PreviewStep({
|
||||
error: string | null
|
||||
authExpired: boolean
|
||||
licenseMissing: boolean
|
||||
/** Fiscal years (start years) ticked in the picker; default = the three latest. */
|
||||
selectedYears: number[]
|
||||
onSelectedYearsChange: (years: number[]) => void
|
||||
onReconnect: () => void
|
||||
onContinue: () => void
|
||||
onBack: () => void
|
||||
}) {
|
||||
const t = useTranslations('extensions')
|
||||
const fiscalYearSpanLabel = useFiscalYearSpanLabel()
|
||||
const providerName = preview
|
||||
? ARCIM_PROVIDERS.find(p => p.id === preview.consent.provider)?.name ?? preview.consent.provider
|
||||
: ''
|
||||
const sourceYears = preview?.sieAvailable ? preview.sourceYears ?? [] : []
|
||||
const showYearPicker = sourceYears.length > 0 && !isLoading
|
||||
const noYearSelected = showYearPicker && selectedYears.length === 0
|
||||
const maxSelectable = preview?.maxSelectedYears ?? null
|
||||
const tooManySelected = showYearPicker && maxSelectable != null && selectedYears.length > maxSelectable
|
||||
const toggleYear = (year: number) => {
|
||||
onSelectedYearsChange(
|
||||
selectedYears.includes(year)
|
||||
? selectedYears.filter((y) => y !== year)
|
||||
: [...selectedYears, year].sort((a, b) => a - b),
|
||||
)
|
||||
}
|
||||
|
||||
return (
|
||||
<div className="stagger-enter space-y-8">
|
||||
@@ -941,6 +992,45 @@ function PreviewStep({
|
||||
)}
|
||||
</div>
|
||||
|
||||
{/* ── The year picker (issues #2211, #2238) ──
|
||||
Every fiscal year the source has, as hairline rows with a checkbox.
|
||||
The three latest are ticked by default (the limit that used to be a
|
||||
silent cap); older years are the user's own choice and their own
|
||||
wait: each one is another SIE export fetched in the next step. */}
|
||||
{showYearPicker && (
|
||||
<section className="space-y-3">
|
||||
<SectionKicker>{t('ext_arcim_year_select_kicker')}</SectionKicker>
|
||||
<p className="text-[13px] text-muted-foreground">{t('ext_arcim_year_select_lede')}</p>
|
||||
<div className="stagger-enter divide-y divide-border" data-no-stagger>
|
||||
{sourceYears.map((fy) => {
|
||||
const id = `arcim-year-${fy.year}-${fy.fromDate ?? ''}`
|
||||
return (
|
||||
<label
|
||||
key={id}
|
||||
htmlFor={id}
|
||||
className="flex min-h-10 cursor-pointer items-center gap-3 py-2 text-sm transition-colors duration-150 hover:bg-secondary/35"
|
||||
>
|
||||
<Checkbox
|
||||
id={id}
|
||||
checked={selectedYears.includes(fy.year)}
|
||||
onCheckedChange={() => toggleYear(fy.year)}
|
||||
aria-label={fiscalYearSpanLabel(fy)}
|
||||
/>
|
||||
<span className="tabular-nums">{fiscalYearSpanLabel(fy)}</span>
|
||||
{!fy.inDefaultSelection && (
|
||||
<span className="text-xs text-muted-foreground">{t('ext_arcim_year_select_older')}</span>
|
||||
)}
|
||||
</label>
|
||||
)
|
||||
})}
|
||||
</div>
|
||||
{noYearSelected && <AttnLine>{t('ext_arcim_year_select_none')}</AttnLine>}
|
||||
{tooManySelected && maxSelectable != null && (
|
||||
<AttnLine>{t('ext_arcim_year_select_too_many', { max: maxSelectable })}</AttnLine>
|
||||
)}
|
||||
</section>
|
||||
)}
|
||||
|
||||
{error && (
|
||||
<div className="space-y-3">
|
||||
<div className="space-y-1">
|
||||
@@ -979,7 +1069,11 @@ function PreviewStep({
|
||||
<ArrowLeft className="mr-2 h-4 w-4" />
|
||||
Tillbaka
|
||||
</Button>
|
||||
<Button className="min-h-11" onClick={onContinue} disabled={isLoading || (!!preview && !preview.sieAvailable && !preview.hasSieData)}>
|
||||
<Button
|
||||
className="min-h-11"
|
||||
onClick={onContinue}
|
||||
disabled={isLoading || noYearSelected || tooManySelected || (!!preview && !preview.sieAvailable && !preview.hasSieData)}
|
||||
>
|
||||
Fortsätt
|
||||
<ArrowRight className="ml-2 h-4 w-4" />
|
||||
</Button>
|
||||
@@ -1706,6 +1800,7 @@ const NEXT_STEPS: { title: string; sub: string }[] = [
|
||||
function ResultStep({
|
||||
results,
|
||||
sieResults,
|
||||
omittedYears,
|
||||
error,
|
||||
documentImportState,
|
||||
theaterModel,
|
||||
@@ -1718,6 +1813,8 @@ function ResultStep({
|
||||
}: {
|
||||
results: MigrationResults | null
|
||||
sieResults: ImportResult[]
|
||||
/** Source fiscal years outside the selection: not fetched in this run. */
|
||||
omittedYears: SourceFiscalYear[]
|
||||
error: string | null
|
||||
documentImportState: ArcimDocumentImportState
|
||||
theaterModel: TheaterModel | null
|
||||
@@ -1729,6 +1826,7 @@ function ResultStep({
|
||||
onReconnectDocuments: () => void
|
||||
}) {
|
||||
const t = useTranslations('extensions')
|
||||
const fiscalYearSpanLabel = useFiscalYearSpanLabel()
|
||||
if (error) {
|
||||
return (
|
||||
<div className="stagger-enter space-y-8">
|
||||
@@ -1954,6 +2052,27 @@ function ResultStep({
|
||||
</section>
|
||||
)}
|
||||
|
||||
{/* ── Source fiscal years outside the selection (#2211) ──
|
||||
Named here so nobody believes the books are complete: a new run
|
||||
with those years ticked fetches them (documents come along), or
|
||||
the SIE path does. */}
|
||||
{sieResults.length > 0 && omittedYears.length > 0 && (
|
||||
<section className="space-y-3">
|
||||
<SectionKicker>{t('ext_arcim_omitted_years_kicker')}</SectionKicker>
|
||||
<div className="stagger-enter divide-y divide-border" data-no-stagger>
|
||||
{omittedYears.map((fy) => (
|
||||
<p key={`${fy.year}-${fy.fromDate ?? ''}`} className="py-3 text-sm tabular-nums">
|
||||
{fiscalYearSpanLabel(fy)}
|
||||
</p>
|
||||
))}
|
||||
</div>
|
||||
<SieFallbackLine
|
||||
message={t('ext_arcim_omitted_years_result', { count: omittedYears.length })}
|
||||
label={t('ext_arcim_omitted_years_sie_link')}
|
||||
/>
|
||||
</section>
|
||||
)}
|
||||
|
||||
{/* ── API import results: quiet two-column line list ── */}
|
||||
{entityLines.length > 0 && (
|
||||
<section className="space-y-3">
|
||||
@@ -2122,6 +2241,9 @@ export default function ArcimMigrationWorkspace({
|
||||
|
||||
// Preview state
|
||||
const [preview, setPreview] = useState<PreviewData | null>(null)
|
||||
// Fiscal years (start years) ticked in the preview step's picker. Set from
|
||||
// the preview's default selection on load; sent to /sie-data as `years`.
|
||||
const [selectedYears, setSelectedYears] = useState<number[]>([])
|
||||
// Set when a preview/sync fails because the provider connection expired
|
||||
// (dead refresh token → PROVIDER_AUTH_EXPIRED). Drives the "Återanslut"
|
||||
// affordance so the user can re-authorize in place instead of disconnecting.
|
||||
@@ -2223,8 +2345,11 @@ export default function ArcimMigrationWorkspace({
|
||||
throw new Error(apiErrorMessage(data, `HTTP ${res.status}`))
|
||||
}
|
||||
|
||||
const data = await res.json()
|
||||
const data = await res.json() as PreviewData
|
||||
setPreview(data)
|
||||
setSelectedYears(
|
||||
(data.sourceYears ?? []).filter((fy) => fy.inDefaultSelection).map((fy) => fy.year),
|
||||
)
|
||||
const previewProvider = data?.consent?.provider
|
||||
if (ARCIM_PROVIDERS.some((provider) => provider.id === previewProvider)) {
|
||||
setSelectedProvider(previewProvider as ArcimProvider)
|
||||
@@ -2701,7 +2826,10 @@ export default function ArcimMigrationWorkspace({
|
||||
setErrorDetails(null)
|
||||
|
||||
try {
|
||||
const res = await fetch(`/api/extensions/ext/arcim-migration/sie-data?consentId=${consentId}`)
|
||||
// The picker's selection travels as `years`; without a picker (no
|
||||
// source years known) the route falls back to its default selection.
|
||||
const yearsQuery = selectedYears.length > 0 ? `&years=${selectedYears.join(',')}` : ''
|
||||
const res = await fetch(`/api/extensions/ext/arcim-migration/sie-data?consentId=${consentId}${yearsQuery}`)
|
||||
if (!res.ok) {
|
||||
const data = await res.json().catch(() => ({})) as {
|
||||
error?: unknown
|
||||
@@ -2741,7 +2869,7 @@ export default function ArcimMigrationWorkspace({
|
||||
} finally {
|
||||
setIsLoading(false)
|
||||
}
|
||||
}, [consentId, refreshCompanyAccounts])
|
||||
}, [consentId, refreshCompanyAccounts, selectedYears])
|
||||
|
||||
const handlePreviewContinue = useCallback(() => {
|
||||
if (preview?.sieAvailable) {
|
||||
@@ -3050,6 +3178,8 @@ export default function ArcimMigrationWorkspace({
|
||||
error={error}
|
||||
authExpired={authExpired}
|
||||
licenseMissing={licenseMissing}
|
||||
selectedYears={selectedYears}
|
||||
onSelectedYearsChange={setSelectedYears}
|
||||
onReconnect={() => {
|
||||
if (selectedProvider && consentId) handleReconnect(selectedProvider, consentId)
|
||||
}}
|
||||
@@ -3101,6 +3231,7 @@ export default function ArcimMigrationWorkspace({
|
||||
<ResultStep
|
||||
results={migrationResults}
|
||||
sieResults={sieImportResults}
|
||||
omittedYears={sieData?.omittedYears ?? []}
|
||||
error={error}
|
||||
documentImportState={documentImportState}
|
||||
theaterModel={theaterModel}
|
||||
|
||||
Reference in New Issue
Block a user