diff --git a/DECISIONS.md b/DECISIONS.md index 1751e2a6..b0ddc948 100644 --- a/DECISIONS.md +++ b/DECISIONS.md @@ -1527,6 +1527,11 @@ One line per decision: `[YYYY-MM-DD] : `. Appended by agents and [2026-09-03] Old-address social identities are unlinked by a BEFORE UPDATE trigger on auth.users (migration 20260903110000), not by the /auth/callback done path: the callback never runs for a completing click from a browser without a session, and admin-side changes bypass it entirely; the trigger covers every path and keeps the email identity, password and BankID intact. [2026-09-03] AGI redovisningsperiod = the payout month (agiReportingPeriod on payment_date), not salary_runs.period_*: Skatteverket files per the month the pay went out (kontantprincipen), so lön i efterskott (August work paid 25 September) is declared in September. The in-period payment-date guard (dashboard PATCH, lib/salary/update-run.ts, v1 PATCH, RunHeader min/max) is lifted rather than widened: its only stated reason was that the AGI keyed on period_*, and any residual month window would bite the next efterskott variant. Existing agi_declarations rows keep their stored period (no backfill): a declaration already filed under the earned month is a real-world correction with Skatteverket, not a re-key. New AGI_PERIOD_CONFLICT (409) refuses to overwrite a live run's declaration for the same payout month, since one month's AGI must cover every payment that month and the generator cannot merge runs. Issue #2191. [2026-09-03] The cursor:// deeplink is its own allowlist provider (cursor_deeplink) rendered "Din egen dator" and never "Verifierad", after the skeptic, CodeRabbit and Superagent all made the same point: a custom scheme can be claimed by any local app (RFC 8252 section 8.4), so it carries loopback trust, not vendor trust, and the consent page must not say otherwise; https://www.cursor.com/... keeps the verified label. Same pass fixed the consent-page CSP for custom schemes: new URL('cursor://...').origin is the string "null", so form-action became `'self' null` and Chromium would have blocked the post-consent 303 (correctness skeptic refutation); the header now uses the scheme-source (`cursor:`) when the origin is opaque. Not done: rejecting a missing code_challenge at /authorize. A code minted without one is unexchangeable (verifyPkce against an empty challenge is always false, now pinned by a test), so it is fail-closed; making it fail earlier is a separate change touching every client. +[2026-09-03] Enable Banking ASPSP_ERROR ladder (#2202): the widest history window a bank has answered is stored per account as accounts_data[].accepted_history_days (no migration; same write-back as dedup_scope and the balance), not as an absolute date, because a bank's limit is a width from today and an absolute known-good date would age into a genuine width rejection. On a rejected window: no wider than accepted = bank unavailable after ONE call; wider = one retry at the accepted width, then unavailable (was up to five calls). AspspUnavailableError maps to 503 BANK_UNAVAILABLE on the web sync route with copy that says the connection does not need renewing; the agent path keeps the contract code BANK_SYNC_FAILED (adding a code touches core contract + structured-errors + v1 docs, left for a follow-up) but no longer persists renewal advice. The envelope's `detail` field is NOT used as a signal: one sample, "Unknown error", identical to a width rejection. A narrowed sync now returns history_from so the UI can say from which date it is complete. [2026-09-03] KPI monthly breakdown counts reversed originals (#2201): the monthly section of get_kpi_report_aggregates (new migration 20260903160000) and lib/reports/monthly-breakdown.ts now use tb_ex_year_end's entry set verbatim (posted + reversed, minus the undone year-end chain) instead of posted-only. A same-year storno then cancels inside the months as it does in the year total, so sum(months) = Nettoresultat; the reversal shows as negative revenue in its own month, which is the honest month view. The pg-real pin "in tb, not in monthly" was flipped, not worked around. Unblocks the per-month sum on Nyckeltal (#2196). [2026-09-03] customers.country and suppliers.country are ISO 3166-1 alpha-2 at every writer (form select, internal + v1 REST, MCP, imports, provider migration), normalised through one helper (lib/vat/country-codes.ts) that also accepts the Swedish/English names the form used to write; unknown text is a 400 on write and left as-is by the backfill (migration 20260903170000 keeps the original in country_raw for a one-UPDATE rollback, and derives the country from the VAT prefix for eu_business rows whose country was null or only the old writer default SE: on prod that is one validated row plus sixteen without a country, and without it they would flip from reverse charge to 25% on their next invoice). No CHECK constraint on the column: unmapped legacy rows would violate it, and the periodisk report already warns on those. The country-vs-type rule (swedish_business = SE, eu_business = not SE and either in the EU VAT area with a matching prefix or holding an EU-trade VAT registration such as a Swiss company with a DE number or Northern Ireland XI, non_eu_business = outside the EU) is enforced on customers only, and on update only when type, country or VAT number is part of the change so a contradictory legacy row can still change its email; individuals are free (a foreign private person is still a Swedish-VAT customer) and suppliers get normalisation without the rule, since #2025/#2028 are about sales VAT. An omitted country on create is SE for Swedish types, derived from the VAT prefix for eu_business, and a 400 for non_eu_business: guessing a non-EU country is not possible, and Sweden-by-default was the bug. vat-rules.ts takes the country as a third optional argument and refuses reverse charge only for SE (a VIES-validated number outweighs a non-EU address), and not for an unknown/unmapped country: charging Swedish VAT to a genuine German customer whose row says Deutschland (Bayern) would be the worse error. [2026-09-03] AGI kvittens cron: dropped the apigw_config bucket (#963) and its warn-once suppression for ACCESS_DENIED (#2226). The bucket existed because the APIGW client was known to lack the AGI hantera subscription in Utvecklarportalen; with that subscription expected in place, a gateway refusal is a regression and belongs in the ordinary error path (error level, generic 'error' status) rather than a status that hides it as a known gap. Same pass: the connector-mode gateway-refusal message names the connector operator by host instead of "kontakta supporten", because hosted is itself a Connect installation for the canary companies and "support" no longer says whose. Merging before the portal subscription is active means the 15-minute cron logs at error level per pending declaration until it is. +[2026-09-03] Fiscal period "first" = no existing period starts earlier (#2237): POST /api/bookkeeping/fiscal-periods decided first-period from `allPeriods.length === 0`, which refused a mid-month first räkenskapsår backfilled after a Fortnox import (2022-07-22) while the DB trigger enforce_first_of_month_for_subsequent_periods would have accepted the row. The route now mirrors the trigger instead of adding a second definition; the 1st-of-month rule (BFL 3 kap. 1 §) keeps binding subsequent years and its message now says why and what is allowed. +[2026-09-03] Tenant brand logos render with next/image `unoptimized` (#2203): the optimizer's remotePatterns allowlist is fixed at build time and the generic Docker image bakes a sentinel for NEXT_PUBLIC_SUPABASE_URL, so /_next/image rejected the runtime Supabase host with 400 and the sidebar mark broke on self-hosted while the favicon (same URL) worked. Chosen over a same-origin proxy route (more code, a second fetch hop for a 26px image) and over a runtime-configurable allowlist (Next.js has none). The remotePatterns block stays for builds that know the URL. +[2026-09-03] Danger zone (#2214): both company actions (Radera företag, Starta om migrering) and the account deletion already open a dialog that requires the company name / e-mail typed in, so nothing destructive can fire from a tap or swipe. The report came from a user who did not dare press the link to find that out, so the fix is copy on the row ("Inget händer direkt: du bekräftar i nästa steg ...") rather than a second dialog or a different control. +[2026-09-03] Decision lines for PRs #2242 (#2237), #2246 (#2203) and #2245 (#2214) are carried in this PR's commit rather than their own: DECISIONS.md is append-only, so every squash-merge flips every other open PR to CONFLICTING and costs a full CI round each; consolidating the lines into the last PR of the batch turns four rounds into one. diff --git a/extensions/general/enable-banking/components/BankingSettingsPanel.tsx b/extensions/general/enable-banking/components/BankingSettingsPanel.tsx index 6d3a211d..37d75832 100644 --- a/extensions/general/enable-banking/components/BankingSettingsPanel.tsx +++ b/extensions/general/enable-banking/components/BankingSettingsPanel.tsx @@ -532,9 +532,15 @@ export default function BankingSettingsPanel() { duplicates: data.duplicates, }) + // A bank that refused the requested window answered a narrower one: + // say from which date the sync is complete instead of presenting a + // truncated history as a full one (#2202). toast({ title: 'Synkronisering klar', - description: `${data.imported} nya transaktioner importerade`, + description: + data.history_narrowed && data.history_from + ? `${data.imported} nya transaktioner importerade. Banken lämnade bara ut historik från ${data.history_from}.` + : `${data.imported} nya transaktioner importerade`, }) setShowCsvFallback(false) diff --git a/extensions/general/enable-banking/index.ts b/extensions/general/enable-banking/index.ts index d440996a..5ca77ae2 100644 --- a/extensions/general/enable-banking/index.ts +++ b/extensions/general/enable-banking/index.ts @@ -7,8 +7,10 @@ import { deleteSession, isSandboxMode, SessionExpiredError, + AspspUnavailableError, REAUTH_REQUIRED_MESSAGE, SYNC_FAILED_MESSAGE, + BANK_UNAVAILABLE_MESSAGE, type ASPSP, } from './lib/api-client' import { syncAccountTransactions } from './lib/sync' @@ -910,12 +912,51 @@ export const enableBankingExtension: Extension = { } } + // A bank that refused the requested window answered a narrower one; + // say so instead of reporting a truncated sync as complete (#2202). + // history_from is the LATEST effective date across the accounts: + // the date from which every account is complete. + const narrowedFrom = results + .filter((r) => r.historyNarrowed && r.effectiveFromDate) + .map((r) => r.effectiveFromDate as string) + const historyFrom = narrowedFrom.length > 0 + ? narrowedFrom.reduce((a, b) => (a > b ? a : b)) + : null + return NextResponse.json({ imported: totalImported, duplicates: totalDuplicates, last_synced_at: syncedAt, + requested_from: fromDate, + history_narrowed: historyFrom !== null, + history_from: historyFrom, }) } catch (error) { + // The bank refused a window it has answered before, or every + // narrower one: not a dead session and not a broken connection, so + // the row is left alone (no 'error', no renewal advice) and the + // client is told to try again later (#2202). + if (error instanceof AspspUnavailableError) { + log.warn('[enable-banking] Sync: bank unavailable, narrowing cannot help', { + reason: error.reason, + dateFrom: error.dateFrom, + status: error.status, + body: error.body, + user_id: user.id, + connection_id, + bankName: connection.bank_name, + }) + return NextResponse.json( + { + error: BANK_UNAVAILABLE_MESSAGE, + code: 'BANK_UNAVAILABLE', + retryable: true, + connection_id: connection.id, + }, + { status: 503 } + ) + } + log.error('[enable-banking] Sync handler error', { message: error instanceof Error ? error.message : String(error), stack: error instanceof Error ? error.stack : undefined, diff --git a/extensions/general/enable-banking/lib/__tests__/api-client.test.ts b/extensions/general/enable-banking/lib/__tests__/api-client.test.ts index c38c0f7b..9244a903 100644 --- a/extensions/general/enable-banking/lib/__tests__/api-client.test.ts +++ b/extensions/general/enable-banking/lib/__tests__/api-client.test.ts @@ -18,6 +18,7 @@ import { getAccountTransactions, getAllTransactions, getAllTransactionsWithRaw, + AspspUnavailableError, convertTransaction, deleteSession, probeSessionHealth, @@ -443,9 +444,11 @@ describe('api-client', () => { // Fresh Response per call: a body can only be read once. fetchSpy.mockImplementation(() => Promise.resolve(new Response(ASPSP_ERROR_BODY, { status: 400 }))) - await expect( - getAllTransactionsWithRaw('acc-1', '2026-02-07', '2026-06-07') - ).rejects.toThrow('Failed to get transactions (400)') + const failure = await getAllTransactionsWithRaw('acc-1', '2026-02-07', '2026-06-07').catch((e) => e) + expect(failure).toBeInstanceOf(AspspUnavailableError) + expect(failure.message).toContain('Failed to get transactions (400)') + // Every narrower window refused too: the bank is refusing, not the width. + expect(failure.reason).toBe('ladder-exhausted') // full window + 90 + 60 + 30 = 4 attempts, then give up expect(fetchSpy).toHaveBeenCalledTimes(4) @@ -453,6 +456,130 @@ describe('api-client', () => { warnSpy.mockRestore() errorSpy.mockRestore() }) + + it('reports the requested and the effective date_from, and whether the window was narrowed', async () => { + const warnSpy = vi.spyOn(console, 'warn').mockImplementation(() => {}) + fetchSpy + .mockResolvedValueOnce(new Response(ASPSP_ERROR_BODY, { status: 400 })) // full window + .mockResolvedValueOnce( + new Response(JSON.stringify({ transactions: [] }), { + status: 200, + headers: { 'Content-Type': 'application/json' }, + }) + ) // 90 days → success + + const result = await getAllTransactionsWithRaw('acc-1', '2026-02-07', '2026-06-07') + expect(result).toMatchObject({ + requestedDateFrom: '2026-02-07', + effectiveDateFrom: '2026-03-09', + narrowed: true, + }) + warnSpy.mockRestore() + }) + + it('reports narrowed: false when the first call succeeds', async () => { + fetchSpy.mockResolvedValueOnce( + new Response(JSON.stringify({ transactions: [] }), { + status: 200, + headers: { 'Content-Type': 'application/json' }, + }) + ) + const result = await getAllTransactionsWithRaw('acc-1', '2026-02-07', '2026-06-07') + expect(result).toMatchObject({ + requestedDateFrom: '2026-02-07', + effectiveDateFrom: '2026-02-07', + narrowed: false, + }) + }) + + // Issue #2202: Länsförsäkringar answered a 4-month window at 23:07 (after + // narrowing to 06-26) and refused every rung of the same request at + // 23:14. ASPSP_ERROR is the same string for "too wide" and "the bank is + // refusing right now"; what the account has accepted before is the + // signal that tells them apart. + describe('accepted history width', () => { + it('a rejected window no wider than the accepted width stops after ONE call, as the bank being unavailable', async () => { + const warnSpy = vi.spyOn(console, 'warn').mockImplementation(() => {}) + fetchSpy.mockImplementation(() => Promise.resolve(new Response(ASPSP_ERROR_BODY, { status: 400 }))) + + // 2026-02-07 .. 2026-06-07 is 120 days; the bank has answered 120 before. + const failure = await getAllTransactionsWithRaw('acc-1', '2026-02-07', '2026-06-07', undefined, { + acceptedHistoryDays: 120, + }).catch((e) => e) + + expect(failure).toBeInstanceOf(AspspUnavailableError) + expect(failure.reason).toBe('window-already-accepted') + expect(failure.dateFrom).toBe('2026-02-07') + expect(fetchSpy).toHaveBeenCalledTimes(1) + warnSpy.mockRestore() + }) + + it('a rejected wider window jumps straight to the accepted width, then stops', async () => { + const warnSpy = vi.spyOn(console, 'warn').mockImplementation(() => {}) + fetchSpy.mockImplementation(() => Promise.resolve(new Response(ASPSP_ERROR_BODY, { status: 400 }))) + + // Accepted 54 days before; asking for 120. No 90/60/30 ladder walk. + const failure = await getAllTransactionsWithRaw('acc-1', '2026-02-07', '2026-06-07', undefined, { + acceptedHistoryDays: 54, + }).catch((e) => e) + + expect(failure).toBeInstanceOf(AspspUnavailableError) + expect(failure.reason).toBe('window-already-accepted') + expect(fetchSpy).toHaveBeenCalledTimes(2) + const urls = fetchSpy.mock.calls.map((c: unknown[]) => c[0] as string) + expect(urls[0]).toContain('date_from=2026-02-07') + expect(urls[1]).toContain('date_from=2026-04-14') // 54 days before 2026-06-07 + warnSpy.mockRestore() + }) + + it('a rejected wider window that succeeds at the accepted width is reported as narrowed to it', async () => { + const warnSpy = vi.spyOn(console, 'warn').mockImplementation(() => {}) + fetchSpy + .mockResolvedValueOnce(new Response(ASPSP_ERROR_BODY, { status: 400 })) + .mockResolvedValueOnce( + new Response(JSON.stringify({ transactions: [] }), { + status: 200, + headers: { 'Content-Type': 'application/json' }, + }) + ) + + const result = await getAllTransactionsWithRaw('acc-1', '2026-02-07', '2026-06-07', undefined, { + acceptedHistoryDays: 54, + }) + expect(result).toMatchObject({ effectiveDateFrom: '2026-04-14', narrowed: true }) + expect(fetchSpy).toHaveBeenCalledTimes(2) + warnSpy.mockRestore() + }) + + it('still drops an unsupported strategy before judging the window', async () => { + const warnSpy = vi.spyOn(console, 'warn').mockImplementation(() => {}) + fetchSpy.mockImplementation(() => Promise.resolve(new Response(ASPSP_ERROR_BODY, { status: 400 }))) + + const failure = await getAllTransactionsWithRaw('acc-1', '2026-02-07', '2026-06-07', 'longest', { + acceptedHistoryDays: 120, + }).catch((e) => e) + + expect(failure).toBeInstanceOf(AspspUnavailableError) + // strategy=longest, then the same window without strategy, then stop. + expect(fetchSpy).toHaveBeenCalledTimes(2) + warnSpy.mockRestore() + }) + + it('getAllTransactions applies the same policy', async () => { + const warnSpy = vi.spyOn(console, 'warn').mockImplementation(() => {}) + const errorSpy = vi.spyOn(console, 'error').mockImplementation(() => {}) + fetchSpy.mockImplementation(() => Promise.resolve(new Response(ASPSP_ERROR_BODY, { status: 400 }))) + + const failure = await getAllTransactions('acc-1', '2026-02-07', '2026-06-07', undefined, { + acceptedHistoryDays: 120, + }).catch((e) => e) + + expect(failure).toBeInstanceOf(AspspUnavailableError) + expect(fetchSpy).toHaveBeenCalledTimes(1) + warnSpy.mockRestore() + errorSpy.mockRestore() + }) + }) }) // ------------------------------------------------------------------------- diff --git a/extensions/general/enable-banking/lib/__tests__/sync.test.ts b/extensions/general/enable-banking/lib/__tests__/sync.test.ts index f95e0011..503257b9 100644 --- a/extensions/general/enable-banking/lib/__tests__/sync.test.ts +++ b/extensions/general/enable-banking/lib/__tests__/sync.test.ts @@ -152,7 +152,8 @@ describe('syncAccountTransactions', () => { 'acc-uid-1', '2024-01-01', '2024-12-31', - 'longest' + 'longest', + { acceptedHistoryDays: undefined } ) }) @@ -178,7 +179,8 @@ describe('syncAccountTransactions', () => { 'acc-uid-1', '2024-01-01', '2024-12-31', - undefined + undefined, + { acceptedHistoryDays: undefined } ) }) @@ -557,6 +559,101 @@ describe('syncAccountTransactions', () => { expect(result.returnedMaxBookingDate).toBeUndefined() }) + // Issue #2202: ASPSP_ERROR cannot say whether the window was too wide or the + // bank is refusing right now, so the sync records the widest window each + // account's bank has answered and reports a narrowed sync as narrowed. + it('hands the accepted history width to the fetch and records the widest window the bank answered', async () => { + mockGetAllTransactionsWithRaw.mockResolvedValue({ + transactions: [], + rawPages: ['{}'], + requestedDateFrom: '2026-04-27', + effectiveDateFrom: '2026-06-26', + narrowed: true, + }) + mockUploadDocument.mockResolvedValue({ id: 'doc-1' }) + const warnSpy = vi.spyOn(console, 'warn').mockImplementation(() => {}) + + const account = makeAccount({ accepted_history_days: 30 }) + const result = await syncAccountTransactions( + {} as never, + COMPANY_ID, + USER_ID, + CONNECTION_ID, + account, + '2026-04-27', + '2026-08-19', + mockIngest + ) + + expect(mockGetAllTransactionsWithRaw).toHaveBeenCalledWith( + 'acc-uid-1', + '2026-04-27', + '2026-08-19', + undefined, + { acceptedHistoryDays: 30 } + ) + // 2026-06-26 .. 2026-08-19 = 54 days: wider than the 30 on record. + expect(account.accepted_history_days).toBe(54) + expect(result).toMatchObject({ + requestedFromDate: '2026-04-27', + effectiveFromDate: '2026-06-26', + historyNarrowed: true, + }) + expect(warnSpy).toHaveBeenCalledWith( + '[enable-banking] Bank refused the requested history window; synced a narrower one', + expect.objectContaining({ requestedFromDate: '2026-04-27', effectiveFromDate: '2026-06-26' }) + ) + warnSpy.mockRestore() + }) + + it('never shrinks accepted_history_days: an incremental sync keeps the wider record', async () => { + mockGetAllTransactionsWithRaw.mockResolvedValue({ + transactions: [], + rawPages: ['{}'], + requestedDateFrom: '2026-08-12', + effectiveDateFrom: '2026-08-12', + narrowed: false, + }) + mockUploadDocument.mockResolvedValue({ id: 'doc-1' }) + + const account = makeAccount({ accepted_history_days: 90 }) + const result = await syncAccountTransactions( + {} as never, + COMPANY_ID, + USER_ID, + CONNECTION_ID, + account, + '2026-08-12', + '2026-08-19', + mockIngest + ) + + expect(account.accepted_history_days).toBe(90) + expect(result.historyNarrowed).toBe(false) + expect(result.effectiveFromDate).toBe('2026-08-12') + }) + + it('leaves accepted_history_days alone when the fetch reports no effective window (legacy shape)', async () => { + mockGetAllTransactionsWithRaw.mockResolvedValue({ transactions: [], rawPages: ['{}'] }) + mockUploadDocument.mockResolvedValue({ id: 'doc-1' }) + + const account = makeAccount() + const result = await syncAccountTransactions( + {} as never, + COMPANY_ID, + USER_ID, + CONNECTION_ID, + account, + '2026-08-12', + '2026-08-19', + mockIngest + ) + + expect(account.accepted_history_days).toBeUndefined() + expect(result.historyNarrowed).toBe(false) + expect(result.effectiveFromDate).toBeUndefined() + }) + it('passes account.ledger_account as IngestOptions.settlementAccount when set', async () => { mockGetAllTransactionsWithRaw.mockResolvedValue({ transactions: [{ transaction_amount: { amount: '100', currency: 'EUR' }, booking_date: '2026-04-01' }], diff --git a/extensions/general/enable-banking/lib/api-client.ts b/extensions/general/enable-banking/lib/api-client.ts index 03c49bf3..9e059caf 100644 --- a/extensions/general/enable-banking/lib/api-client.ts +++ b/extensions/general/enable-banking/lib/api-client.ts @@ -16,6 +16,7 @@ import { getAuthorizationHeader } from './jwt' import { deriveTransactionLabel } from './transaction-label' import { FALLBACK_DESCRIPTION } from '@/lib/transactions/external-id' import { bankConnectorMode, CONNECTOR_COMPANY_HEADER } from '@/lib/connect/instance/upstreams' +import { dateFromDaysBefore, historyWindowDays } from './history-window' // Prefer _PRODUCTION variant; sandbox uses api.tilisy.com, production uses api.enablebanking.com const ENABLE_BANKING_API_URL = @@ -232,6 +233,14 @@ export const REAUTH_REQUIRED_MESSAGE = 'Bankanslutningen har löpt ut. Förnya anslutningen för att fortsätta synka.' export const SYNC_FAILED_MESSAGE = 'Banksynkningen misslyckades. Försök igen, eller förnya anslutningen om felet kvarstår.' +/** + * The bank is refusing right now and narrowing the window cannot help. Says + * explicitly that the connection does NOT need renewing: a transient + * ASPSP_ERROR used to surface as SYNC_FAILED_MESSAGE, whose "förnya + * anslutningen" advice costs a BankID round trip and fixes nothing (#2202). + */ +export const BANK_UNAVAILABLE_MESSAGE = + 'Banken svarade inte just nu (tillfälligt fel hos banken). Försök igen om en stund. Anslutningen behöver inte förnyas.' /** * Thrown when a transactions fetch fails because the PSD2 session is dead @@ -250,6 +259,32 @@ export class SessionExpiredError extends Error { } } +/** + * Thrown when the ASPSP rejected the transactions request and narrowing the + * history window is not the answer (issue #2202): + * + * - 'window-already-accepted': the rejected window is no wider than one this + * account has fetched successfully before (StoredAccount.accepted_history_days), + * so width is not the problem; the bank is refusing for its own reasons. + * - 'ladder-exhausted': every narrower window was refused too. + * + * Either way the sync should say "try again later" and must not flip the + * connection to expired/error or prompt a consent renewal. The message keeps + * the `Failed to get transactions (status)` prefix so existing log matching + * still works; carries the raw body for the server log only. + */ +export class AspspUnavailableError extends Error { + constructor( + readonly status: number, + readonly body: string, + readonly reason: 'window-already-accepted' | 'ladder-exhausted', + readonly dateFrom: string | undefined + ) { + super(`Failed to get transactions (${status}), bank unavailable [${reason}]: ${body}`) + this.name = 'AspspUnavailableError' + } +} + // API Helper async function authenticatedFetch( @@ -822,7 +857,8 @@ export async function getAllTransactions( accountUid: string, dateFrom?: string, dateTo?: string, - strategy?: TransactionsFetchStrategy + strategy?: TransactionsFetchStrategy, + options?: HistoryWindowOptions ): Promise { const allTransactions: Transaction[] = [] let continuationKey: string | undefined @@ -854,6 +890,7 @@ export async function getAllTransactions( activeStrategy, activeDateFrom, dateTo, + acceptedHistoryDays: options?.acceptedHistoryDays, }) if (recovery.type === 'drop-strategy') { console.warn('[enable-banking] strategy rejected by API, retrying without strategy', { @@ -875,6 +912,9 @@ export async function getAllTransactions( activeDateFrom = recovery.dateFrom continue } + if (recovery.type === 'aspsp-unavailable') { + throw bankUnavailable(accountUid, err.status, err.body, recovery.reason, activeDateFrom, dateTo) + } } throw err } @@ -901,11 +941,31 @@ export async function getAllTransactions( */ const ASPSP_HISTORY_FALLBACK_DAYS = [90, 60, 30] as const +/** + * Per-account knowledge the caller can hand the pagination loops (#2202). + */ +export interface HistoryWindowOptions { + /** + * Widest window (whole days before date_to) this account's bank has + * accepted before: StoredAccount.accepted_history_days. When set, a + * rejected window that is no wider than this is reported as the bank being + * unavailable (AspspUnavailableError) instead of narrowed, and a wider + * rejected window jumps straight to this width instead of walking the + * 90/60/30 ladder. Unset (first sync, legacy rows): the ladder runs as + * before. + */ + acceptedHistoryDays?: number +} + /** * Enable Banking wraps upstream-bank failures in a generic envelope, e.g. * {"code":400,"message":"Error interacting with ASPSP","error":"ASPSP_ERROR"}. - * A too-wide history window is the most common trigger: see the date-narrowing - * fallback in getAllTransactionsWithRaw. + * The envelope is the SAME for a history window beyond the bank's PSD2 limit + * and for a bank that is refusing right now (maintenance, throttling, an + * upstream error): the one sample of a transient failure on record carried + * "detail":"Unknown error", exactly like a window rejection does. So the + * string alone cannot tell the two apart; planFirstPageRecovery uses what the + * account has accepted before to decide (issue #2202). */ function isAspspError(body: string): boolean { return body.includes('ASPSP_ERROR') || body.includes('interacting with ASPSP') @@ -944,14 +1004,22 @@ function nextNarrowerDateFrom( * continuation_key is scoped to the window/strategy that produced it, so the * query is never rewritten mid-pagination. * - * - 'drop-strategy' : an unsupported strategy enum: retry the same window. - * - 'narrow' : the ASPSP rejected the history window: retry with a - * narrower date_from (the bank caps history below the ask). - * - 'give-up' : nothing left to try; the caller should rethrow. + * - 'drop-strategy' : an unsupported strategy enum: retry the same window. + * - 'narrow' : the ASPSP rejected the history window: retry with a + * narrower date_from (the bank caps history below the + * ask). With a known accepted width the retry jumps + * straight to it; without one it steps down the + * 90/60/30 ladder. + * - 'aspsp-unavailable' : the ASPSP rejected a window it has accepted before, + * or every narrower window too: width is not the + * problem, the bank is refusing right now. The caller + * throws AspspUnavailableError (#2202). + * - 'give-up' : nothing left to try; the caller should rethrow. */ type FirstPageRecovery = | { type: 'drop-strategy' } | { type: 'narrow'; dateFrom: string } + | { type: 'aspsp-unavailable'; reason: AspspUnavailableError['reason'] } | { type: 'give-up' } function planFirstPageRecovery(args: { @@ -962,18 +1030,61 @@ function planFirstPageRecovery(args: { activeStrategy: TransactionsFetchStrategy | undefined activeDateFrom: string | undefined dateTo: string | undefined + acceptedHistoryDays: number | undefined }): FirstPageRecovery { - const { status, body, page, hasContinuationKey, activeStrategy, activeDateFrom, dateTo } = args + const { + status, + body, + page, + hasContinuationKey, + activeStrategy, + activeDateFrom, + dateTo, + acceptedHistoryDays, + } = args if (status !== 400 || page !== 0 || hasContinuationKey) return { type: 'give-up' } // Drop an unsupported strategy first: preserves the full requested window. if (activeStrategy) return { type: 'drop-strategy' } - // Then handle the ASPSP rejecting the window itself (e.g. Danske past ~90 - // days): step date_from toward date_to so a partial sync survives. - if (isAspspError(body)) { - const dateFrom = nextNarrowerDateFrom(activeDateFrom, dateTo) + if (!isAspspError(body)) return { type: 'give-up' } + + // A window this account has fetched before is the strongest signal on hand + // that width is not the problem: stop after this one call. Wider than that: + // retry once at the known-good width, which is the cheapest test with the + // best odds, instead of spending three rungs on a bank that is saying no. + const requestedDays = historyWindowDays(activeDateFrom, dateTo) + if (acceptedHistoryDays !== undefined && requestedDays !== undefined) { + if (requestedDays <= acceptedHistoryDays) { + return { type: 'aspsp-unavailable', reason: 'window-already-accepted' } + } + const dateFrom = dateFromDaysBefore(dateTo, acceptedHistoryDays) if (dateFrom) return { type: 'narrow', dateFrom } } - return { type: 'give-up' } + + // No accepted width on record (first sync, legacy rows): step date_from + // toward date_to (e.g. Danske past ~90 days) so a partial sync survives. + const dateFrom = nextNarrowerDateFrom(activeDateFrom, dateTo) + if (dateFrom) return { type: 'narrow', dateFrom } + return { type: 'aspsp-unavailable', reason: 'ladder-exhausted' } +} + +/** Log and build the error both pagination loops throw on 'aspsp-unavailable'. */ +function bankUnavailable( + accountUid: string, + status: number, + body: string, + reason: AspspUnavailableError['reason'], + activeDateFrom: string | undefined, + dateTo: string | undefined +): AspspUnavailableError { + console.warn('[enable-banking] ASPSP refused a request that narrowing cannot fix; treating the bank as unavailable', { + accountUid, + reason, + dateFrom: activeDateFrom, + dateTo, + status, + body, + }) + return new AspspUnavailableError(status, body, reason, activeDateFrom) } /** @@ -986,16 +1097,34 @@ function planFirstPageRecovery(args: { * * If the ASPSP then still rejects the first page with an ASPSP_ERROR (typically * a history window beyond the bank's PSD2 limit, e.g. Danske past ~90 days), - * progressively narrow date_from toward date_to (90→60→30 days) so a partial - * sync of the recent window survives instead of failing outright. Logs a - * warning on each narrowing. + * progressively narrow date_from toward date_to (90→60→30 days, or straight to + * options.acceptedHistoryDays when the account has one) so a partial sync of + * the recent window survives instead of failing outright. Logs a warning on + * each narrowing. A rejection that narrowing cannot fix throws + * AspspUnavailableError (see planFirstPageRecovery). + * + * The result names the date_from the bank finally ANSWERED (effectiveDateFrom) + * next to the one that was asked for, so a truncated window is visible to the + * caller instead of looking like a complete sync (#2202). */ +export interface TransactionsWithRaw { + transactions: Transaction[] + rawPages: string[] + /** The date_from the caller asked for. */ + requestedDateFrom: string | undefined + /** The date_from the bank answered: equal to requestedDateFrom unless narrowed. */ + effectiveDateFrom: string | undefined + /** True when the bank refused the requested window and a narrower one was used. */ + narrowed: boolean +} + export async function getAllTransactionsWithRaw( accountUid: string, dateFrom?: string, dateTo?: string, - strategy?: TransactionsFetchStrategy -): Promise<{ transactions: Transaction[]; rawPages: string[] }> { + strategy?: TransactionsFetchStrategy, + options?: HistoryWindowOptions +): Promise { const allTransactions: Transaction[] = [] const rawPages: string[] = [] let continuationKey: string | undefined @@ -1027,6 +1156,7 @@ export async function getAllTransactionsWithRaw( activeStrategy, activeDateFrom, dateTo, + acceptedHistoryDays: options?.acceptedHistoryDays, }) if (recovery.type === 'drop-strategy') { console.warn('[enable-banking] strategy rejected by API, retrying without strategy', { @@ -1048,6 +1178,9 @@ export async function getAllTransactionsWithRaw( activeDateFrom = recovery.dateFrom continue } + if (recovery.type === 'aspsp-unavailable') { + throw bankUnavailable(accountUid, response.status, body, recovery.reason, activeDateFrom, dateTo) + } // An expired PSD2 session is an expected end of life for a consent, not // a failure: SessionExpiredError below flips the connection to 'expired' // and asks the user to re-authorize. Log it at warn so only the genuine @@ -1087,7 +1220,13 @@ export async function getAllTransactionsWithRaw( if (!continuationKey) break } - return { transactions: allTransactions, rawPages } + return { + transactions: allTransactions, + rawPages, + requestedDateFrom: dateFrom, + effectiveDateFrom: activeDateFrom, + narrowed: activeDateFrom !== dateFrom, + } } /** diff --git a/extensions/general/enable-banking/lib/history-window.ts b/extensions/general/enable-banking/lib/history-window.ts new file mode 100644 index 00000000..fa83e3f0 --- /dev/null +++ b/extensions/general/enable-banking/lib/history-window.ts @@ -0,0 +1,39 @@ +/** + * Day arithmetic for the transactions history window (date_from .. date_to, + * both YYYY-MM-DD, evaluated in UTC). + * + * Shared by api-client.ts (the narrowing ladder) and sync.ts (recording the + * widest window a bank has accepted for an account, issue #2202). It lives in + * its own module because sync.test.ts mocks api-client wholesale: a helper + * exported from there would be undefined under test. + */ +const DAY_MS = 24 * 60 * 60 * 1000 + +function parseUtcDay(value: string | undefined): number | undefined { + if (!value) return undefined + const t = new Date(`${value}T00:00:00Z`).getTime() + return Number.isFinite(t) ? t : undefined +} + +/** + * Whole days from dateFrom to dateTo. Undefined when either end is missing or + * unparseable, or when the window is negative: callers treat undefined as + * "no width known", never as 0. + */ +export function historyWindowDays( + dateFrom: string | undefined, + dateTo: string | undefined +): number | undefined { + const from = parseUtcDay(dateFrom) + const to = parseUtcDay(dateTo) + if (from === undefined || to === undefined) return undefined + const days = Math.round((to - from) / DAY_MS) + return days >= 0 ? days : undefined +} + +/** The YYYY-MM-DD that lies `days` days before dateTo; undefined when dateTo is unparseable. */ +export function dateFromDaysBefore(dateTo: string | undefined, days: number): string | undefined { + const to = parseUtcDay(dateTo) + if (to === undefined) return undefined + return new Date(to - days * DAY_MS).toISOString().split('T')[0] +} diff --git a/extensions/general/enable-banking/lib/sync.ts b/extensions/general/enable-banking/lib/sync.ts index 230f00fb..0cd895e0 100644 --- a/extensions/general/enable-banking/lib/sync.ts +++ b/extensions/general/enable-banking/lib/sync.ts @@ -1,5 +1,6 @@ import type { SupabaseClient } from '@supabase/supabase-js' import { getAllTransactionsWithRaw, convertTransaction, getAccountBalance, SessionExpiredError } from './api-client' +import { historyWindowDays } from './history-window' import { bankSyncResponseSchema, connectorErrorSchema } from '@accounted/connect-contract' import { bankConnectorMode, CONNECTOR_COMPANY_HEADER } from '@/lib/connect/instance/upstreams' import { uploadDocument } from '@/lib/core/documents/document-service' @@ -46,6 +47,17 @@ export interface SyncResult { returnedMinBookingDate?: string /** Latest booking date the ASPSP returned. Undefined when no transactions came back. */ returnedMaxBookingDate?: string + /** The date_from asked of the bank. */ + requestedFromDate: string + /** + * The date_from the bank actually answered. Equal to requestedFromDate + * unless the bank refused the window and a narrower one was used; the sync + * is then complete only from this date on (#2202). Undefined through the + * hosted connector, which does not report it. + */ + effectiveFromDate?: string + /** True when the bank refused the requested window and the history was cut. */ + historyNarrowed: boolean } /** The subset of a converted bank transaction that the ingest mapping below reads. */ @@ -185,6 +197,8 @@ export async function syncAccountTransactions( let rawPages: string[] let bookedEntries: Array<{ tx: BookedTransactionFields; bookingDate: string }> let totalFetched: number + let effectiveFromDate: string | undefined + let historyNarrowed = false if (connector) { const remote = await fetchBookedViaConnector(connector, { supabase, @@ -217,9 +231,31 @@ export async function syncAccountTransactions( fromDate, toDate, syncOptions?.strategy, + { acceptedHistoryDays: account.accepted_history_days }, ) rawPages = fetched.rawPages totalFetched = fetched.transactions.length + effectiveFromDate = fetched.effectiveDateFrom + historyNarrowed = fetched.narrowed === true + // Remember the widest window this bank has ANSWERED for the account, so a + // later rejection of a window no wider than it is read as the bank being + // unavailable rather than as a too-wide window (#2202). Never shrinks: + // an incremental 7-day sync must not forget that 90 days once worked. + // Stamped on the account object; the caller's accounts_data write-back + // persists it, exactly like dedup_scope and the balance fields. + const acceptedDays = historyWindowDays(fetched.effectiveDateFrom, toDate) + if (acceptedDays !== undefined && acceptedDays > (account.accepted_history_days ?? -1)) { + account.accepted_history_days = acceptedDays + } + if (historyNarrowed) { + console.warn('[enable-banking] Bank refused the requested history window; synced a narrower one', { + connectionId, + accountUid: account.uid, + requestedFromDate: fromDate, + effectiveFromDate, + toDate, + }) + } const bankTransactions = fetched.transactions.map(tx => convertTransaction(tx, account.currency)) // Only ingest BOOKED transactions: those the ASPSP returned with a real // booking_date. Pending entries are intentionally skipped: a pending row is @@ -257,6 +293,7 @@ export async function syncAccountTransactions( transactionCount: totalFetched, rawPageCount: rawPages.length, requestedFromDate: fromDate, + effectiveFromDate, requestedToDate: toDate, returnedMinBookingDate: minBookingDate, returnedMaxBookingDate: maxBookingDate, @@ -407,5 +444,8 @@ export async function syncAccountTransactions( errors: ingestResult.errors, returnedMinBookingDate: minBookingDate, returnedMaxBookingDate: maxBookingDate, + requestedFromDate: fromDate, + effectiveFromDate, + historyNarrowed, } } diff --git a/extensions/general/enable-banking/lib/trigger-sync.ts b/extensions/general/enable-banking/lib/trigger-sync.ts index bea365e7..6129e41d 100644 --- a/extensions/general/enable-banking/lib/trigger-sync.ts +++ b/extensions/general/enable-banking/lib/trigger-sync.ts @@ -26,6 +26,7 @@ import type { SupabaseClient } from '@supabase/supabase-js' import { syncAccountTransactions, type SyncOptions } from './sync' import { SessionExpiredError, + AspspUnavailableError, REAUTH_REQUIRED_MESSAGE, SYNC_FAILED_MESSAGE, } from './api-client' @@ -275,6 +276,25 @@ export async function triggerConnectionSync( } } + if (error instanceof AspspUnavailableError) { + // The bank is refusing right now (a window it has answered before, or + // every narrower one): retryable, not a dead session, not a broken + // row. Warn, leave error_message alone (no renewal advice), and answer + // with the retryable code (#2202). + log.warn('agent-triggered bank sync: bank unavailable', { + connectionId, + reason: error.reason, + dateFrom: error.dateFrom, + status: error.status, + }) + return { + ok: false, + code: 'BANK_SYNC_FAILED', + connection_id: connectionId, + status: connection.status as string, + } + } + log.error('agent-triggered bank sync failed', { connectionId, message: error instanceof Error ? error.message : String(error), diff --git a/extensions/general/enable-banking/types.ts b/extensions/general/enable-banking/types.ts index 8cdaf71b..c74f2521 100644 --- a/extensions/general/enable-banking/types.ts +++ b/extensions/general/enable-banking/types.ts @@ -40,6 +40,14 @@ export interface StoredAccount { // silent; cleared by the selection save when the user re-enables the // account. deselected_elsewhere?: boolean + // Widest transactions history window (whole days before date_to) this + // account's bank has ever ACCEPTED, stamped by lib/sync.ts after each + // successful fetch. ASPSP_ERROR is Enable Banking's generic wrapper for any + // upstream failure, so a rejected request cannot say whether the window was + // too wide or the bank is refusing right now. A window no wider than this + // has worked before, so a rejection of it is treated as the bank being + // unavailable instead of walking the whole narrowing ladder (issue #2202). + accepted_history_days?: number } // Re-exported from the client for lib/sync.ts; every other api-client type