From 5eac2a492cda9af9aefeef6dea40a1bd9685a39b Mon Sep 17 00:00:00 2001 From: Jakob Wennberg Date: Thu, 3 Sep 2026 18:43:01 +0200 Subject: [PATCH] fix(enable-banking): stop reading every ASPSP_ERROR as a too-wide window (#2202) (#2247) MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit * fix(enable-banking): stop reading every ASPSP_ERROR as a too-wide window (#2202) ASPSP_ERROR is Enable Banking's generic wrapper for any upstream bank failure, so "history window beyond the PSD2 limit" and "the bank is refusing right now" arrived as the same string, and every rejection walked the whole 90/60/30 narrowing ladder: one user click cost up to five upstream calls against a bank that was already saying no, the failure surfaced with "förnya anslutningen" advice that fixes nothing, and a sync that did narrow was reported as complete. What the account has accepted before is the signal that tells the two apart. sync.ts now records the widest window (days before date_to) each account's bank has answered, on accounts_data as accepted_history_days (no migration; persisted by the same write-back as dedup_scope). On a rejected window: no wider than that = the bank is unavailable, stop after one call; wider = one retry straight at the accepted width, then stop. Without a record (first sync, legacy rows) the ladder runs as before, but its exhaustion is now AspspUnavailableError too. The web sync route maps that to 503 BANK_UNAVAILABLE with copy that says the connection does not need renewing and leaves the row alone; the agent path keeps the contract code BANK_SYNC_FAILED but no longer persists renewal advice. getAllTransactionsWithRaw returns the requested and the effective date_from plus a narrowed flag; the sync result and the /sync response carry them (history_from), and the settings toast says from which date the history is complete when the bank cut the window. Not done: a per-account backoff for the user-triggered route (the agent path already has the 15-minute lease from #2165), and using the envelope's `detail` field (one sample, identical to a width rejection). Closes #2202 Co-Authored-By: Claude Fable 5.1 Claude-Session: https://claude.ai/code/session_01VnConrmMCxJRQ5kfiPPWyy * docs(decisions): carry the batch's decision lines (#2237, #2203, #2214) here --------- Co-authored-by: Jakob Wennberg <311770904+jakobwennberg-oss@users.noreply.github.com> Co-authored-by: Claude Fable 5.1 --- DECISIONS.md | 5 + .../components/BankingSettingsPanel.tsx | 8 +- extensions/general/enable-banking/index.ts | 41 ++++ .../lib/__tests__/api-client.test.ts | 133 ++++++++++++- .../enable-banking/lib/__tests__/sync.test.ts | 101 +++++++++- .../general/enable-banking/lib/api-client.ts | 177 ++++++++++++++++-- .../enable-banking/lib/history-window.ts | 39 ++++ extensions/general/enable-banking/lib/sync.ts | 40 ++++ .../enable-banking/lib/trigger-sync.ts | 20 ++ extensions/general/enable-banking/types.ts | 8 + 10 files changed, 547 insertions(+), 25 deletions(-) create mode 100644 extensions/general/enable-banking/lib/history-window.ts diff --git a/DECISIONS.md b/DECISIONS.md index 1751e2a6..b0ddc948 100644 --- a/DECISIONS.md +++ b/DECISIONS.md @@ -1527,6 +1527,11 @@ One line per decision: `[YYYY-MM-DD] : `. Appended by agents and [2026-09-03] Old-address social identities are unlinked by a BEFORE UPDATE trigger on auth.users (migration 20260903110000), not by the /auth/callback done path: the callback never runs for a completing click from a browser without a session, and admin-side changes bypass it entirely; the trigger covers every path and keeps the email identity, password and BankID intact. [2026-09-03] AGI redovisningsperiod = the payout month (agiReportingPeriod on payment_date), not salary_runs.period_*: Skatteverket files per the month the pay went out (kontantprincipen), so lön i efterskott (August work paid 25 September) is declared in September. The in-period payment-date guard (dashboard PATCH, lib/salary/update-run.ts, v1 PATCH, RunHeader min/max) is lifted rather than widened: its only stated reason was that the AGI keyed on period_*, and any residual month window would bite the next efterskott variant. Existing agi_declarations rows keep their stored period (no backfill): a declaration already filed under the earned month is a real-world correction with Skatteverket, not a re-key. New AGI_PERIOD_CONFLICT (409) refuses to overwrite a live run's declaration for the same payout month, since one month's AGI must cover every payment that month and the generator cannot merge runs. Issue #2191. [2026-09-03] The cursor:// deeplink is its own allowlist provider (cursor_deeplink) rendered "Din egen dator" and never "Verifierad", after the skeptic, CodeRabbit and Superagent all made the same point: a custom scheme can be claimed by any local app (RFC 8252 section 8.4), so it carries loopback trust, not vendor trust, and the consent page must not say otherwise; https://www.cursor.com/... keeps the verified label. Same pass fixed the consent-page CSP for custom schemes: new URL('cursor://...').origin is the string "null", so form-action became `'self' null` and Chromium would have blocked the post-consent 303 (correctness skeptic refutation); the header now uses the scheme-source (`cursor:`) when the origin is opaque. Not done: rejecting a missing code_challenge at /authorize. A code minted without one is unexchangeable (verifyPkce against an empty challenge is always false, now pinned by a test), so it is fail-closed; making it fail earlier is a separate change touching every client. +[2026-09-03] Enable Banking ASPSP_ERROR ladder (#2202): the widest history window a bank has answered is stored per account as accounts_data[].accepted_history_days (no migration; same write-back as dedup_scope and the balance), not as an absolute date, because a bank's limit is a width from today and an absolute known-good date would age into a genuine width rejection. On a rejected window: no wider than accepted = bank unavailable after ONE call; wider = one retry at the accepted width, then unavailable (was up to five calls). AspspUnavailableError maps to 503 BANK_UNAVAILABLE on the web sync route with copy that says the connection does not need renewing; the agent path keeps the contract code BANK_SYNC_FAILED (adding a code touches core contract + structured-errors + v1 docs, left for a follow-up) but no longer persists renewal advice. The envelope's `detail` field is NOT used as a signal: one sample, "Unknown error", identical to a width rejection. A narrowed sync now returns history_from so the UI can say from which date it is complete. [2026-09-03] KPI monthly breakdown counts reversed originals (#2201): the monthly section of get_kpi_report_aggregates (new migration 20260903160000) and lib/reports/monthly-breakdown.ts now use tb_ex_year_end's entry set verbatim (posted + reversed, minus the undone year-end chain) instead of posted-only. A same-year storno then cancels inside the months as it does in the year total, so sum(months) = Nettoresultat; the reversal shows as negative revenue in its own month, which is the honest month view. The pg-real pin "in tb, not in monthly" was flipped, not worked around. Unblocks the per-month sum on Nyckeltal (#2196). [2026-09-03] customers.country and suppliers.country are ISO 3166-1 alpha-2 at every writer (form select, internal + v1 REST, MCP, imports, provider migration), normalised through one helper (lib/vat/country-codes.ts) that also accepts the Swedish/English names the form used to write; unknown text is a 400 on write and left as-is by the backfill (migration 20260903170000 keeps the original in country_raw for a one-UPDATE rollback, and derives the country from the VAT prefix for eu_business rows whose country was null or only the old writer default SE: on prod that is one validated row plus sixteen without a country, and without it they would flip from reverse charge to 25% on their next invoice). No CHECK constraint on the column: unmapped legacy rows would violate it, and the periodisk report already warns on those. The country-vs-type rule (swedish_business = SE, eu_business = not SE and either in the EU VAT area with a matching prefix or holding an EU-trade VAT registration such as a Swiss company with a DE number or Northern Ireland XI, non_eu_business = outside the EU) is enforced on customers only, and on update only when type, country or VAT number is part of the change so a contradictory legacy row can still change its email; individuals are free (a foreign private person is still a Swedish-VAT customer) and suppliers get normalisation without the rule, since #2025/#2028 are about sales VAT. An omitted country on create is SE for Swedish types, derived from the VAT prefix for eu_business, and a 400 for non_eu_business: guessing a non-EU country is not possible, and Sweden-by-default was the bug. vat-rules.ts takes the country as a third optional argument and refuses reverse charge only for SE (a VIES-validated number outweighs a non-EU address), and not for an unknown/unmapped country: charging Swedish VAT to a genuine German customer whose row says Deutschland (Bayern) would be the worse error. [2026-09-03] AGI kvittens cron: dropped the apigw_config bucket (#963) and its warn-once suppression for ACCESS_DENIED (#2226). The bucket existed because the APIGW client was known to lack the AGI hantera subscription in Utvecklarportalen; with that subscription expected in place, a gateway refusal is a regression and belongs in the ordinary error path (error level, generic 'error' status) rather than a status that hides it as a known gap. Same pass: the connector-mode gateway-refusal message names the connector operator by host instead of "kontakta supporten", because hosted is itself a Connect installation for the canary companies and "support" no longer says whose. Merging before the portal subscription is active means the 15-minute cron logs at error level per pending declaration until it is. +[2026-09-03] Fiscal period "first" = no existing period starts earlier (#2237): POST /api/bookkeeping/fiscal-periods decided first-period from `allPeriods.length === 0`, which refused a mid-month first räkenskapsår backfilled after a Fortnox import (2022-07-22) while the DB trigger enforce_first_of_month_for_subsequent_periods would have accepted the row. The route now mirrors the trigger instead of adding a second definition; the 1st-of-month rule (BFL 3 kap. 1 §) keeps binding subsequent years and its message now says why and what is allowed. +[2026-09-03] Tenant brand logos render with next/image `unoptimized` (#2203): the optimizer's remotePatterns allowlist is fixed at build time and the generic Docker image bakes a sentinel for NEXT_PUBLIC_SUPABASE_URL, so /_next/image rejected the runtime Supabase host with 400 and the sidebar mark broke on self-hosted while the favicon (same URL) worked. Chosen over a same-origin proxy route (more code, a second fetch hop for a 26px image) and over a runtime-configurable allowlist (Next.js has none). The remotePatterns block stays for builds that know the URL. +[2026-09-03] Danger zone (#2214): both company actions (Radera företag, Starta om migrering) and the account deletion already open a dialog that requires the company name / e-mail typed in, so nothing destructive can fire from a tap or swipe. The report came from a user who did not dare press the link to find that out, so the fix is copy on the row ("Inget händer direkt: du bekräftar i nästa steg ...") rather than a second dialog or a different control. +[2026-09-03] Decision lines for PRs #2242 (#2237), #2246 (#2203) and #2245 (#2214) are carried in this PR's commit rather than their own: DECISIONS.md is append-only, so every squash-merge flips every other open PR to CONFLICTING and costs a full CI round each; consolidating the lines into the last PR of the batch turns four rounds into one. diff --git a/extensions/general/enable-banking/components/BankingSettingsPanel.tsx b/extensions/general/enable-banking/components/BankingSettingsPanel.tsx index 6d3a211d..37d75832 100644 --- a/extensions/general/enable-banking/components/BankingSettingsPanel.tsx +++ b/extensions/general/enable-banking/components/BankingSettingsPanel.tsx @@ -532,9 +532,15 @@ export default function BankingSettingsPanel() { duplicates: data.duplicates, }) + // A bank that refused the requested window answered a narrower one: + // say from which date the sync is complete instead of presenting a + // truncated history as a full one (#2202). toast({ title: 'Synkronisering klar', - description: `${data.imported} nya transaktioner importerade`, + description: + data.history_narrowed && data.history_from + ? `${data.imported} nya transaktioner importerade. Banken lämnade bara ut historik från ${data.history_from}.` + : `${data.imported} nya transaktioner importerade`, }) setShowCsvFallback(false) diff --git a/extensions/general/enable-banking/index.ts b/extensions/general/enable-banking/index.ts index d440996a..5ca77ae2 100644 --- a/extensions/general/enable-banking/index.ts +++ b/extensions/general/enable-banking/index.ts @@ -7,8 +7,10 @@ import { deleteSession, isSandboxMode, SessionExpiredError, + AspspUnavailableError, REAUTH_REQUIRED_MESSAGE, SYNC_FAILED_MESSAGE, + BANK_UNAVAILABLE_MESSAGE, type ASPSP, } from './lib/api-client' import { syncAccountTransactions } from './lib/sync' @@ -910,12 +912,51 @@ export const enableBankingExtension: Extension = { } } + // A bank that refused the requested window answered a narrower one; + // say so instead of reporting a truncated sync as complete (#2202). + // history_from is the LATEST effective date across the accounts: + // the date from which every account is complete. + const narrowedFrom = results + .filter((r) => r.historyNarrowed && r.effectiveFromDate) + .map((r) => r.effectiveFromDate as string) + const historyFrom = narrowedFrom.length > 0 + ? narrowedFrom.reduce((a, b) => (a > b ? a : b)) + : null + return NextResponse.json({ imported: totalImported, duplicates: totalDuplicates, last_synced_at: syncedAt, + requested_from: fromDate, + history_narrowed: historyFrom !== null, + history_from: historyFrom, }) } catch (error) { + // The bank refused a window it has answered before, or every + // narrower one: not a dead session and not a broken connection, so + // the row is left alone (no 'error', no renewal advice) and the + // client is told to try again later (#2202). + if (error instanceof AspspUnavailableError) { + log.warn('[enable-banking] Sync: bank unavailable, narrowing cannot help', { + reason: error.reason, + dateFrom: error.dateFrom, + status: error.status, + body: error.body, + user_id: user.id, + connection_id, + bankName: connection.bank_name, + }) + return NextResponse.json( + { + error: BANK_UNAVAILABLE_MESSAGE, + code: 'BANK_UNAVAILABLE', + retryable: true, + connection_id: connection.id, + }, + { status: 503 } + ) + } + log.error('[enable-banking] Sync handler error', { message: error instanceof Error ? error.message : String(error), stack: error instanceof Error ? error.stack : undefined, diff --git a/extensions/general/enable-banking/lib/__tests__/api-client.test.ts b/extensions/general/enable-banking/lib/__tests__/api-client.test.ts index c38c0f7b..9244a903 100644 --- a/extensions/general/enable-banking/lib/__tests__/api-client.test.ts +++ b/extensions/general/enable-banking/lib/__tests__/api-client.test.ts @@ -18,6 +18,7 @@ import { getAccountTransactions, getAllTransactions, getAllTransactionsWithRaw, + AspspUnavailableError, convertTransaction, deleteSession, probeSessionHealth, @@ -443,9 +444,11 @@ describe('api-client', () => { // Fresh Response per call: a body can only be read once. fetchSpy.mockImplementation(() => Promise.resolve(new Response(ASPSP_ERROR_BODY, { status: 400 }))) - await expect( - getAllTransactionsWithRaw('acc-1', '2026-02-07', '2026-06-07') - ).rejects.toThrow('Failed to get transactions (400)') + const failure = await getAllTransactionsWithRaw('acc-1', '2026-02-07', '2026-06-07').catch((e) => e) + expect(failure).toBeInstanceOf(AspspUnavailableError) + expect(failure.message).toContain('Failed to get transactions (400)') + // Every narrower window refused too: the bank is refusing, not the width. + expect(failure.reason).toBe('ladder-exhausted') // full window + 90 + 60 + 30 = 4 attempts, then give up expect(fetchSpy).toHaveBeenCalledTimes(4) @@ -453,6 +456,130 @@ describe('api-client', () => { warnSpy.mockRestore() errorSpy.mockRestore() }) + + it('reports the requested and the effective date_from, and whether the window was narrowed', async () => { + const warnSpy = vi.spyOn(console, 'warn').mockImplementation(() => {}) + fetchSpy + .mockResolvedValueOnce(new Response(ASPSP_ERROR_BODY, { status: 400 })) // full window + .mockResolvedValueOnce( + new Response(JSON.stringify({ transactions: [] }), { + status: 200, + headers: { 'Content-Type': 'application/json' }, + }) + ) // 90 days → success + + const result = await getAllTransactionsWithRaw('acc-1', '2026-02-07', '2026-06-07') + expect(result).toMatchObject({ + requestedDateFrom: '2026-02-07', + effectiveDateFrom: '2026-03-09', + narrowed: true, + }) + warnSpy.mockRestore() + }) + + it('reports narrowed: false when the first call succeeds', async () => { + fetchSpy.mockResolvedValueOnce( + new Response(JSON.stringify({ transactions: [] }), { + status: 200, + headers: { 'Content-Type': 'application/json' }, + }) + ) + const result = await getAllTransactionsWithRaw('acc-1', '2026-02-07', '2026-06-07') + expect(result).toMatchObject({ + requestedDateFrom: '2026-02-07', + effectiveDateFrom: '2026-02-07', + narrowed: false, + }) + }) + + // Issue #2202: Länsförsäkringar answered a 4-month window at 23:07 (after + // narrowing to 06-26) and refused every rung of the same request at + // 23:14. ASPSP_ERROR is the same string for "too wide" and "the bank is + // refusing right now"; what the account has accepted before is the + // signal that tells them apart. + describe('accepted history width', () => { + it('a rejected window no wider than the accepted width stops after ONE call, as the bank being unavailable', async () => { + const warnSpy = vi.spyOn(console, 'warn').mockImplementation(() => {}) + fetchSpy.mockImplementation(() => Promise.resolve(new Response(ASPSP_ERROR_BODY, { status: 400 }))) + + // 2026-02-07 .. 2026-06-07 is 120 days; the bank has answered 120 before. + const failure = await getAllTransactionsWithRaw('acc-1', '2026-02-07', '2026-06-07', undefined, { + acceptedHistoryDays: 120, + }).catch((e) => e) + + expect(failure).toBeInstanceOf(AspspUnavailableError) + expect(failure.reason).toBe('window-already-accepted') + expect(failure.dateFrom).toBe('2026-02-07') + expect(fetchSpy).toHaveBeenCalledTimes(1) + warnSpy.mockRestore() + }) + + it('a rejected wider window jumps straight to the accepted width, then stops', async () => { + const warnSpy = vi.spyOn(console, 'warn').mockImplementation(() => {}) + fetchSpy.mockImplementation(() => Promise.resolve(new Response(ASPSP_ERROR_BODY, { status: 400 }))) + + // Accepted 54 days before; asking for 120. No 90/60/30 ladder walk. + const failure = await getAllTransactionsWithRaw('acc-1', '2026-02-07', '2026-06-07', undefined, { + acceptedHistoryDays: 54, + }).catch((e) => e) + + expect(failure).toBeInstanceOf(AspspUnavailableError) + expect(failure.reason).toBe('window-already-accepted') + expect(fetchSpy).toHaveBeenCalledTimes(2) + const urls = fetchSpy.mock.calls.map((c: unknown[]) => c[0] as string) + expect(urls[0]).toContain('date_from=2026-02-07') + expect(urls[1]).toContain('date_from=2026-04-14') // 54 days before 2026-06-07 + warnSpy.mockRestore() + }) + + it('a rejected wider window that succeeds at the accepted width is reported as narrowed to it', async () => { + const warnSpy = vi.spyOn(console, 'warn').mockImplementation(() => {}) + fetchSpy + .mockResolvedValueOnce(new Response(ASPSP_ERROR_BODY, { status: 400 })) + .mockResolvedValueOnce( + new Response(JSON.stringify({ transactions: [] }), { + status: 200, + headers: { 'Content-Type': 'application/json' }, + }) + ) + + const result = await getAllTransactionsWithRaw('acc-1', '2026-02-07', '2026-06-07', undefined, { + acceptedHistoryDays: 54, + }) + expect(result).toMatchObject({ effectiveDateFrom: '2026-04-14', narrowed: true }) + expect(fetchSpy).toHaveBeenCalledTimes(2) + warnSpy.mockRestore() + }) + + it('still drops an unsupported strategy before judging the window', async () => { + const warnSpy = vi.spyOn(console, 'warn').mockImplementation(() => {}) + fetchSpy.mockImplementation(() => Promise.resolve(new Response(ASPSP_ERROR_BODY, { status: 400 }))) + + const failure = await getAllTransactionsWithRaw('acc-1', '2026-02-07', '2026-06-07', 'longest', { + acceptedHistoryDays: 120, + }).catch((e) => e) + + expect(failure).toBeInstanceOf(AspspUnavailableError) + // strategy=longest, then the same window without strategy, then stop. + expect(fetchSpy).toHaveBeenCalledTimes(2) + warnSpy.mockRestore() + }) + + it('getAllTransactions applies the same policy', async () => { + const warnSpy = vi.spyOn(console, 'warn').mockImplementation(() => {}) + const errorSpy = vi.spyOn(console, 'error').mockImplementation(() => {}) + fetchSpy.mockImplementation(() => Promise.resolve(new Response(ASPSP_ERROR_BODY, { status: 400 }))) + + const failure = await getAllTransactions('acc-1', '2026-02-07', '2026-06-07', undefined, { + acceptedHistoryDays: 120, + }).catch((e) => e) + + expect(failure).toBeInstanceOf(AspspUnavailableError) + expect(fetchSpy).toHaveBeenCalledTimes(1) + warnSpy.mockRestore() + errorSpy.mockRestore() + }) + }) }) // ------------------------------------------------------------------------- diff --git a/extensions/general/enable-banking/lib/__tests__/sync.test.ts b/extensions/general/enable-banking/lib/__tests__/sync.test.ts index f95e0011..503257b9 100644 --- a/extensions/general/enable-banking/lib/__tests__/sync.test.ts +++ b/extensions/general/enable-banking/lib/__tests__/sync.test.ts @@ -152,7 +152,8 @@ describe('syncAccountTransactions', () => { 'acc-uid-1', '2024-01-01', '2024-12-31', - 'longest' + 'longest', + { acceptedHistoryDays: undefined } ) }) @@ -178,7 +179,8 @@ describe('syncAccountTransactions', () => { 'acc-uid-1', '2024-01-01', '2024-12-31', - undefined + undefined, + { acceptedHistoryDays: undefined } ) }) @@ -557,6 +559,101 @@ describe('syncAccountTransactions', () => { expect(result.returnedMaxBookingDate).toBeUndefined() }) + // Issue #2202: ASPSP_ERROR cannot say whether the window was too wide or the + // bank is refusing right now, so the sync records the widest window each + // account's bank has answered and reports a narrowed sync as narrowed. + it('hands the accepted history width to the fetch and records the widest window the bank answered', async () => { + mockGetAllTransactionsWithRaw.mockResolvedValue({ + transactions: [], + rawPages: ['{}'], + requestedDateFrom: '2026-04-27', + effectiveDateFrom: '2026-06-26', + narrowed: true, + }) + mockUploadDocument.mockResolvedValue({ id: 'doc-1' }) + const warnSpy = vi.spyOn(console, 'warn').mockImplementation(() => {}) + + const account = makeAccount({ accepted_history_days: 30 }) + const result = await syncAccountTransactions( + {} as never, + COMPANY_ID, + USER_ID, + CONNECTION_ID, + account, + '2026-04-27', + '2026-08-19', + mockIngest + ) + + expect(mockGetAllTransactionsWithRaw).toHaveBeenCalledWith( + 'acc-uid-1', + '2026-04-27', + '2026-08-19', + undefined, + { acceptedHistoryDays: 30 } + ) + // 2026-06-26 .. 2026-08-19 = 54 days: wider than the 30 on record. + expect(account.accepted_history_days).toBe(54) + expect(result).toMatchObject({ + requestedFromDate: '2026-04-27', + effectiveFromDate: '2026-06-26', + historyNarrowed: true, + }) + expect(warnSpy).toHaveBeenCalledWith( + '[enable-banking] Bank refused the requested history window; synced a narrower one', + expect.objectContaining({ requestedFromDate: '2026-04-27', effectiveFromDate: '2026-06-26' }) + ) + warnSpy.mockRestore() + }) + + it('never shrinks accepted_history_days: an incremental sync keeps the wider record', async () => { + mockGetAllTransactionsWithRaw.mockResolvedValue({ + transactions: [], + rawPages: ['{}'], + requestedDateFrom: '2026-08-12', + effectiveDateFrom: '2026-08-12', + narrowed: false, + }) + mockUploadDocument.mockResolvedValue({ id: 'doc-1' }) + + const account = makeAccount({ accepted_history_days: 90 }) + const result = await syncAccountTransactions( + {} as never, + COMPANY_ID, + USER_ID, + CONNECTION_ID, + account, + '2026-08-12', + '2026-08-19', + mockIngest + ) + + expect(account.accepted_history_days).toBe(90) + expect(result.historyNarrowed).toBe(false) + expect(result.effectiveFromDate).toBe('2026-08-12') + }) + + it('leaves accepted_history_days alone when the fetch reports no effective window (legacy shape)', async () => { + mockGetAllTransactionsWithRaw.mockResolvedValue({ transactions: [], rawPages: ['{}'] }) + mockUploadDocument.mockResolvedValue({ id: 'doc-1' }) + + const account = makeAccount() + const result = await syncAccountTransactions( + {} as never, + COMPANY_ID, + USER_ID, + CONNECTION_ID, + account, + '2026-08-12', + '2026-08-19', + mockIngest + ) + + expect(account.accepted_history_days).toBeUndefined() + expect(result.historyNarrowed).toBe(false) + expect(result.effectiveFromDate).toBeUndefined() + }) + it('passes account.ledger_account as IngestOptions.settlementAccount when set', async () => { mockGetAllTransactionsWithRaw.mockResolvedValue({ transactions: [{ transaction_amount: { amount: '100', currency: 'EUR' }, booking_date: '2026-04-01' }], diff --git a/extensions/general/enable-banking/lib/api-client.ts b/extensions/general/enable-banking/lib/api-client.ts index 03c49bf3..9e059caf 100644 --- a/extensions/general/enable-banking/lib/api-client.ts +++ b/extensions/general/enable-banking/lib/api-client.ts @@ -16,6 +16,7 @@ import { getAuthorizationHeader } from './jwt' import { deriveTransactionLabel } from './transaction-label' import { FALLBACK_DESCRIPTION } from '@/lib/transactions/external-id' import { bankConnectorMode, CONNECTOR_COMPANY_HEADER } from '@/lib/connect/instance/upstreams' +import { dateFromDaysBefore, historyWindowDays } from './history-window' // Prefer _PRODUCTION variant; sandbox uses api.tilisy.com, production uses api.enablebanking.com const ENABLE_BANKING_API_URL = @@ -232,6 +233,14 @@ export const REAUTH_REQUIRED_MESSAGE = 'Bankanslutningen har löpt ut. Förnya anslutningen för att fortsätta synka.' export const SYNC_FAILED_MESSAGE = 'Banksynkningen misslyckades. Försök igen, eller förnya anslutningen om felet kvarstår.' +/** + * The bank is refusing right now and narrowing the window cannot help. Says + * explicitly that the connection does NOT need renewing: a transient + * ASPSP_ERROR used to surface as SYNC_FAILED_MESSAGE, whose "förnya + * anslutningen" advice costs a BankID round trip and fixes nothing (#2202). + */ +export const BANK_UNAVAILABLE_MESSAGE = + 'Banken svarade inte just nu (tillfälligt fel hos banken). Försök igen om en stund. Anslutningen behöver inte förnyas.' /** * Thrown when a transactions fetch fails because the PSD2 session is dead @@ -250,6 +259,32 @@ export class SessionExpiredError extends Error { } } +/** + * Thrown when the ASPSP rejected the transactions request and narrowing the + * history window is not the answer (issue #2202): + * + * - 'window-already-accepted': the rejected window is no wider than one this + * account has fetched successfully before (StoredAccount.accepted_history_days), + * so width is not the problem; the bank is refusing for its own reasons. + * - 'ladder-exhausted': every narrower window was refused too. + * + * Either way the sync should say "try again later" and must not flip the + * connection to expired/error or prompt a consent renewal. The message keeps + * the `Failed to get transactions (status)` prefix so existing log matching + * still works; carries the raw body for the server log only. + */ +export class AspspUnavailableError extends Error { + constructor( + readonly status: number, + readonly body: string, + readonly reason: 'window-already-accepted' | 'ladder-exhausted', + readonly dateFrom: string | undefined + ) { + super(`Failed to get transactions (${status}), bank unavailable [${reason}]: ${body}`) + this.name = 'AspspUnavailableError' + } +} + // API Helper async function authenticatedFetch( @@ -822,7 +857,8 @@ export async function getAllTransactions( accountUid: string, dateFrom?: string, dateTo?: string, - strategy?: TransactionsFetchStrategy + strategy?: TransactionsFetchStrategy, + options?: HistoryWindowOptions ): Promise { const allTransactions: Transaction[] = [] let continuationKey: string | undefined @@ -854,6 +890,7 @@ export async function getAllTransactions( activeStrategy, activeDateFrom, dateTo, + acceptedHistoryDays: options?.acceptedHistoryDays, }) if (recovery.type === 'drop-strategy') { console.warn('[enable-banking] strategy rejected by API, retrying without strategy', { @@ -875,6 +912,9 @@ export async function getAllTransactions( activeDateFrom = recovery.dateFrom continue } + if (recovery.type === 'aspsp-unavailable') { + throw bankUnavailable(accountUid, err.status, err.body, recovery.reason, activeDateFrom, dateTo) + } } throw err } @@ -901,11 +941,31 @@ export async function getAllTransactions( */ const ASPSP_HISTORY_FALLBACK_DAYS = [90, 60, 30] as const +/** + * Per-account knowledge the caller can hand the pagination loops (#2202). + */ +export interface HistoryWindowOptions { + /** + * Widest window (whole days before date_to) this account's bank has + * accepted before: StoredAccount.accepted_history_days. When set, a + * rejected window that is no wider than this is reported as the bank being + * unavailable (AspspUnavailableError) instead of narrowed, and a wider + * rejected window jumps straight to this width instead of walking the + * 90/60/30 ladder. Unset (first sync, legacy rows): the ladder runs as + * before. + */ + acceptedHistoryDays?: number +} + /** * Enable Banking wraps upstream-bank failures in a generic envelope, e.g. * {"code":400,"message":"Error interacting with ASPSP","error":"ASPSP_ERROR"}. - * A too-wide history window is the most common trigger: see the date-narrowing - * fallback in getAllTransactionsWithRaw. + * The envelope is the SAME for a history window beyond the bank's PSD2 limit + * and for a bank that is refusing right now (maintenance, throttling, an + * upstream error): the one sample of a transient failure on record carried + * "detail":"Unknown error", exactly like a window rejection does. So the + * string alone cannot tell the two apart; planFirstPageRecovery uses what the + * account has accepted before to decide (issue #2202). */ function isAspspError(body: string): boolean { return body.includes('ASPSP_ERROR') || body.includes('interacting with ASPSP') @@ -944,14 +1004,22 @@ function nextNarrowerDateFrom( * continuation_key is scoped to the window/strategy that produced it, so the * query is never rewritten mid-pagination. * - * - 'drop-strategy' : an unsupported strategy enum: retry the same window. - * - 'narrow' : the ASPSP rejected the history window: retry with a - * narrower date_from (the bank caps history below the ask). - * - 'give-up' : nothing left to try; the caller should rethrow. + * - 'drop-strategy' : an unsupported strategy enum: retry the same window. + * - 'narrow' : the ASPSP rejected the history window: retry with a + * narrower date_from (the bank caps history below the + * ask). With a known accepted width the retry jumps + * straight to it; without one it steps down the + * 90/60/30 ladder. + * - 'aspsp-unavailable' : the ASPSP rejected a window it has accepted before, + * or every narrower window too: width is not the + * problem, the bank is refusing right now. The caller + * throws AspspUnavailableError (#2202). + * - 'give-up' : nothing left to try; the caller should rethrow. */ type FirstPageRecovery = | { type: 'drop-strategy' } | { type: 'narrow'; dateFrom: string } + | { type: 'aspsp-unavailable'; reason: AspspUnavailableError['reason'] } | { type: 'give-up' } function planFirstPageRecovery(args: { @@ -962,18 +1030,61 @@ function planFirstPageRecovery(args: { activeStrategy: TransactionsFetchStrategy | undefined activeDateFrom: string | undefined dateTo: string | undefined + acceptedHistoryDays: number | undefined }): FirstPageRecovery { - const { status, body, page, hasContinuationKey, activeStrategy, activeDateFrom, dateTo } = args + const { + status, + body, + page, + hasContinuationKey, + activeStrategy, + activeDateFrom, + dateTo, + acceptedHistoryDays, + } = args if (status !== 400 || page !== 0 || hasContinuationKey) return { type: 'give-up' } // Drop an unsupported strategy first: preserves the full requested window. if (activeStrategy) return { type: 'drop-strategy' } - // Then handle the ASPSP rejecting the window itself (e.g. Danske past ~90 - // days): step date_from toward date_to so a partial sync survives. - if (isAspspError(body)) { - const dateFrom = nextNarrowerDateFrom(activeDateFrom, dateTo) + if (!isAspspError(body)) return { type: 'give-up' } + + // A window this account has fetched before is the strongest signal on hand + // that width is not the problem: stop after this one call. Wider than that: + // retry once at the known-good width, which is the cheapest test with the + // best odds, instead of spending three rungs on a bank that is saying no. + const requestedDays = historyWindowDays(activeDateFrom, dateTo) + if (acceptedHistoryDays !== undefined && requestedDays !== undefined) { + if (requestedDays <= acceptedHistoryDays) { + return { type: 'aspsp-unavailable', reason: 'window-already-accepted' } + } + const dateFrom = dateFromDaysBefore(dateTo, acceptedHistoryDays) if (dateFrom) return { type: 'narrow', dateFrom } } - return { type: 'give-up' } + + // No accepted width on record (first sync, legacy rows): step date_from + // toward date_to (e.g. Danske past ~90 days) so a partial sync survives. + const dateFrom = nextNarrowerDateFrom(activeDateFrom, dateTo) + if (dateFrom) return { type: 'narrow', dateFrom } + return { type: 'aspsp-unavailable', reason: 'ladder-exhausted' } +} + +/** Log and build the error both pagination loops throw on 'aspsp-unavailable'. */ +function bankUnavailable( + accountUid: string, + status: number, + body: string, + reason: AspspUnavailableError['reason'], + activeDateFrom: string | undefined, + dateTo: string | undefined +): AspspUnavailableError { + console.warn('[enable-banking] ASPSP refused a request that narrowing cannot fix; treating the bank as unavailable', { + accountUid, + reason, + dateFrom: activeDateFrom, + dateTo, + status, + body, + }) + return new AspspUnavailableError(status, body, reason, activeDateFrom) } /** @@ -986,16 +1097,34 @@ function planFirstPageRecovery(args: { * * If the ASPSP then still rejects the first page with an ASPSP_ERROR (typically * a history window beyond the bank's PSD2 limit, e.g. Danske past ~90 days), - * progressively narrow date_from toward date_to (90→60→30 days) so a partial - * sync of the recent window survives instead of failing outright. Logs a - * warning on each narrowing. + * progressively narrow date_from toward date_to (90→60→30 days, or straight to + * options.acceptedHistoryDays when the account has one) so a partial sync of + * the recent window survives instead of failing outright. Logs a warning on + * each narrowing. A rejection that narrowing cannot fix throws + * AspspUnavailableError (see planFirstPageRecovery). + * + * The result names the date_from the bank finally ANSWERED (effectiveDateFrom) + * next to the one that was asked for, so a truncated window is visible to the + * caller instead of looking like a complete sync (#2202). */ +export interface TransactionsWithRaw { + transactions: Transaction[] + rawPages: string[] + /** The date_from the caller asked for. */ + requestedDateFrom: string | undefined + /** The date_from the bank answered: equal to requestedDateFrom unless narrowed. */ + effectiveDateFrom: string | undefined + /** True when the bank refused the requested window and a narrower one was used. */ + narrowed: boolean +} + export async function getAllTransactionsWithRaw( accountUid: string, dateFrom?: string, dateTo?: string, - strategy?: TransactionsFetchStrategy -): Promise<{ transactions: Transaction[]; rawPages: string[] }> { + strategy?: TransactionsFetchStrategy, + options?: HistoryWindowOptions +): Promise { const allTransactions: Transaction[] = [] const rawPages: string[] = [] let continuationKey: string | undefined @@ -1027,6 +1156,7 @@ export async function getAllTransactionsWithRaw( activeStrategy, activeDateFrom, dateTo, + acceptedHistoryDays: options?.acceptedHistoryDays, }) if (recovery.type === 'drop-strategy') { console.warn('[enable-banking] strategy rejected by API, retrying without strategy', { @@ -1048,6 +1178,9 @@ export async function getAllTransactionsWithRaw( activeDateFrom = recovery.dateFrom continue } + if (recovery.type === 'aspsp-unavailable') { + throw bankUnavailable(accountUid, response.status, body, recovery.reason, activeDateFrom, dateTo) + } // An expired PSD2 session is an expected end of life for a consent, not // a failure: SessionExpiredError below flips the connection to 'expired' // and asks the user to re-authorize. Log it at warn so only the genuine @@ -1087,7 +1220,13 @@ export async function getAllTransactionsWithRaw( if (!continuationKey) break } - return { transactions: allTransactions, rawPages } + return { + transactions: allTransactions, + rawPages, + requestedDateFrom: dateFrom, + effectiveDateFrom: activeDateFrom, + narrowed: activeDateFrom !== dateFrom, + } } /** diff --git a/extensions/general/enable-banking/lib/history-window.ts b/extensions/general/enable-banking/lib/history-window.ts new file mode 100644 index 00000000..fa83e3f0 --- /dev/null +++ b/extensions/general/enable-banking/lib/history-window.ts @@ -0,0 +1,39 @@ +/** + * Day arithmetic for the transactions history window (date_from .. date_to, + * both YYYY-MM-DD, evaluated in UTC). + * + * Shared by api-client.ts (the narrowing ladder) and sync.ts (recording the + * widest window a bank has accepted for an account, issue #2202). It lives in + * its own module because sync.test.ts mocks api-client wholesale: a helper + * exported from there would be undefined under test. + */ +const DAY_MS = 24 * 60 * 60 * 1000 + +function parseUtcDay(value: string | undefined): number | undefined { + if (!value) return undefined + const t = new Date(`${value}T00:00:00Z`).getTime() + return Number.isFinite(t) ? t : undefined +} + +/** + * Whole days from dateFrom to dateTo. Undefined when either end is missing or + * unparseable, or when the window is negative: callers treat undefined as + * "no width known", never as 0. + */ +export function historyWindowDays( + dateFrom: string | undefined, + dateTo: string | undefined +): number | undefined { + const from = parseUtcDay(dateFrom) + const to = parseUtcDay(dateTo) + if (from === undefined || to === undefined) return undefined + const days = Math.round((to - from) / DAY_MS) + return days >= 0 ? days : undefined +} + +/** The YYYY-MM-DD that lies `days` days before dateTo; undefined when dateTo is unparseable. */ +export function dateFromDaysBefore(dateTo: string | undefined, days: number): string | undefined { + const to = parseUtcDay(dateTo) + if (to === undefined) return undefined + return new Date(to - days * DAY_MS).toISOString().split('T')[0] +} diff --git a/extensions/general/enable-banking/lib/sync.ts b/extensions/general/enable-banking/lib/sync.ts index 230f00fb..0cd895e0 100644 --- a/extensions/general/enable-banking/lib/sync.ts +++ b/extensions/general/enable-banking/lib/sync.ts @@ -1,5 +1,6 @@ import type { SupabaseClient } from '@supabase/supabase-js' import { getAllTransactionsWithRaw, convertTransaction, getAccountBalance, SessionExpiredError } from './api-client' +import { historyWindowDays } from './history-window' import { bankSyncResponseSchema, connectorErrorSchema } from '@accounted/connect-contract' import { bankConnectorMode, CONNECTOR_COMPANY_HEADER } from '@/lib/connect/instance/upstreams' import { uploadDocument } from '@/lib/core/documents/document-service' @@ -46,6 +47,17 @@ export interface SyncResult { returnedMinBookingDate?: string /** Latest booking date the ASPSP returned. Undefined when no transactions came back. */ returnedMaxBookingDate?: string + /** The date_from asked of the bank. */ + requestedFromDate: string + /** + * The date_from the bank actually answered. Equal to requestedFromDate + * unless the bank refused the window and a narrower one was used; the sync + * is then complete only from this date on (#2202). Undefined through the + * hosted connector, which does not report it. + */ + effectiveFromDate?: string + /** True when the bank refused the requested window and the history was cut. */ + historyNarrowed: boolean } /** The subset of a converted bank transaction that the ingest mapping below reads. */ @@ -185,6 +197,8 @@ export async function syncAccountTransactions( let rawPages: string[] let bookedEntries: Array<{ tx: BookedTransactionFields; bookingDate: string }> let totalFetched: number + let effectiveFromDate: string | undefined + let historyNarrowed = false if (connector) { const remote = await fetchBookedViaConnector(connector, { supabase, @@ -217,9 +231,31 @@ export async function syncAccountTransactions( fromDate, toDate, syncOptions?.strategy, + { acceptedHistoryDays: account.accepted_history_days }, ) rawPages = fetched.rawPages totalFetched = fetched.transactions.length + effectiveFromDate = fetched.effectiveDateFrom + historyNarrowed = fetched.narrowed === true + // Remember the widest window this bank has ANSWERED for the account, so a + // later rejection of a window no wider than it is read as the bank being + // unavailable rather than as a too-wide window (#2202). Never shrinks: + // an incremental 7-day sync must not forget that 90 days once worked. + // Stamped on the account object; the caller's accounts_data write-back + // persists it, exactly like dedup_scope and the balance fields. + const acceptedDays = historyWindowDays(fetched.effectiveDateFrom, toDate) + if (acceptedDays !== undefined && acceptedDays > (account.accepted_history_days ?? -1)) { + account.accepted_history_days = acceptedDays + } + if (historyNarrowed) { + console.warn('[enable-banking] Bank refused the requested history window; synced a narrower one', { + connectionId, + accountUid: account.uid, + requestedFromDate: fromDate, + effectiveFromDate, + toDate, + }) + } const bankTransactions = fetched.transactions.map(tx => convertTransaction(tx, account.currency)) // Only ingest BOOKED transactions: those the ASPSP returned with a real // booking_date. Pending entries are intentionally skipped: a pending row is @@ -257,6 +293,7 @@ export async function syncAccountTransactions( transactionCount: totalFetched, rawPageCount: rawPages.length, requestedFromDate: fromDate, + effectiveFromDate, requestedToDate: toDate, returnedMinBookingDate: minBookingDate, returnedMaxBookingDate: maxBookingDate, @@ -407,5 +444,8 @@ export async function syncAccountTransactions( errors: ingestResult.errors, returnedMinBookingDate: minBookingDate, returnedMaxBookingDate: maxBookingDate, + requestedFromDate: fromDate, + effectiveFromDate, + historyNarrowed, } } diff --git a/extensions/general/enable-banking/lib/trigger-sync.ts b/extensions/general/enable-banking/lib/trigger-sync.ts index bea365e7..6129e41d 100644 --- a/extensions/general/enable-banking/lib/trigger-sync.ts +++ b/extensions/general/enable-banking/lib/trigger-sync.ts @@ -26,6 +26,7 @@ import type { SupabaseClient } from '@supabase/supabase-js' import { syncAccountTransactions, type SyncOptions } from './sync' import { SessionExpiredError, + AspspUnavailableError, REAUTH_REQUIRED_MESSAGE, SYNC_FAILED_MESSAGE, } from './api-client' @@ -275,6 +276,25 @@ export async function triggerConnectionSync( } } + if (error instanceof AspspUnavailableError) { + // The bank is refusing right now (a window it has answered before, or + // every narrower one): retryable, not a dead session, not a broken + // row. Warn, leave error_message alone (no renewal advice), and answer + // with the retryable code (#2202). + log.warn('agent-triggered bank sync: bank unavailable', { + connectionId, + reason: error.reason, + dateFrom: error.dateFrom, + status: error.status, + }) + return { + ok: false, + code: 'BANK_SYNC_FAILED', + connection_id: connectionId, + status: connection.status as string, + } + } + log.error('agent-triggered bank sync failed', { connectionId, message: error instanceof Error ? error.message : String(error), diff --git a/extensions/general/enable-banking/types.ts b/extensions/general/enable-banking/types.ts index 8cdaf71b..c74f2521 100644 --- a/extensions/general/enable-banking/types.ts +++ b/extensions/general/enable-banking/types.ts @@ -40,6 +40,14 @@ export interface StoredAccount { // silent; cleared by the selection save when the user re-enables the // account. deselected_elsewhere?: boolean + // Widest transactions history window (whole days before date_to) this + // account's bank has ever ACCEPTED, stamped by lib/sync.ts after each + // successful fetch. ASPSP_ERROR is Enable Banking's generic wrapper for any + // upstream failure, so a rejected request cannot say whether the window was + // too wide or the bank is refusing right now. A window no wider than this + // has worked before, so a rejection of it is treated as the bank being + // unavailable instead of walking the whole narrowing ladder (issue #2202). + accepted_history_days?: number } // Re-exported from the client for lib/sync.ts; every other api-client type