diff --git a/.github/dependabot.yml b/.github/dependabot.yml new file mode 100644 index 00000000..31833e0a --- /dev/null +++ b/.github/dependabot.yml @@ -0,0 +1,19 @@ +# Security-updates-only posture: open-pull-requests-limit: 0 disables scheduled +# version-bump PRs while keeping Dependabot security PRs, and `ignore` applies +# to both kinds. +version: 2 +updates: + - package-ecosystem: "npm" + directory: "/" + schedule: + interval: "weekly" + open-pull-requests-limit: 0 + cooldown: + default-days: 7 + ignore: + # @anthropic-ai/bedrock-sdk 0.30.0+ broke invoice OCR and the assistant in + # prod (empty Bedrock streams, 2026-07 outage; see DECISIONS.md). The + # dependency is pinned to 0.29.1 in package.json; a bump must be a + # deliberate, verified change, never an automated PR. + - dependency-name: "@anthropic-ai/bedrock-sdk" + versions: [">=0.30.0"] diff --git a/DECISIONS.md b/DECISIONS.md index 92830716..11a43c8f 100644 --- a/DECISIONS.md +++ b/DECISIONS.md @@ -921,5 +921,13 @@ One line per decision: `[YYYY-MM-DD] : `. Appended by agents and [2026-08-11] Self-hosted: the 30 s poll in lib/hooks/use-document-extraction.ts left alone even though #1406 names it. It is a symptom of missing credentials rather than a bug in itself: with AI configured, extraction finishes in 2-8 s and the wait disappears. The real fix is for the extraction-status endpoint to report "not configured" instead of leaving the column NULL forever, which is a separate change. [2026-08-13] Tier 1 self-hosting packages the `document-extraction` extension in the stock image: a plain `ANTHROPIC_API_KEY` must cover documents uploaded in the app as well as emailed invoices and the assistant to satisfy issue #1406; this does not add provider abstraction or other Tier 2 work. [2026-08-13] document-extraction's manifest no longer lists AWS_REGION under requiredEnvVars: the extension accepts either AWS static keys or ANTHROPIC_API_KEY, which the manifest schema cannot express as alternatives, and requiredEnvVars only drives a build-time warning rather than gating execution. + +[2026-08-13] dependabot.yml added as security-updates-only (open-pull-requests-limit: 0) with an ignore on @anthropic-ai/bedrock-sdk >=0.30.0: enabling scheduled version bumps repo-wide just to pin one package would create review noise, and the exact 0.29.1 pin alone does not stop an automated bump PR from reintroducing the 0.32.0 empty-stream outage class; the ignore covers both paths. + +[2026-08-13] Renewal-flood guard reuses runReconciliation in the PATCH /accounts backfill instead of adding an import-time matcher: the manual /sync route and the cron already run the identical SIE-overlap gate plus unattended-threshold sweep, and a third matching implementation would drift. Unlike those two sweeps, the new call resolves a per-ledger-account scope via resolveCashAccountScope (the documented #1290/#1298 call-site fix) because the backfill knows each enabled account; a scope resolution failure skips that account's sweep rather than widening to the pooled unscoped form. + +[2026-08-13] Gap-fill renewal default anchors on the connection's newest imported transaction (feed coverage), not ledger coverage: the flood is a feed-level re-request problem, and ledger coverage already has its own suggestion box in the picker. The gap-fill start subtracts 7 days because external_id dedup makes overlap free and banks book some rows late. The async default only flips the radio to gap-fill if the user has not touched the group before the probe lands. + +[2026-08-13] Skeptic refutation fix on the renewal guard: the picker's gap-fill probe keys its state by connectionId and lists `accounts` in its deps, so the pre-existing reset effect (re-runs on every accounts identity change, e.g. the panel's visibility refetch after a BankID app switch) can never wipe the renewal default without a matching re-probe. Backfill sweep hardening from the same pass: accounts whose cash_accounts row did not resolve (found: false) are skipped instead of degrading to the pooled currency-only form, and the sweep window opens at the oldest returned booking date when the bank over-returns history. resolveGapFillStart clamps to the backend's 365-day floor so the shown date always matches the actual backfill start. [2026-08-13] WhatsApp decline observability (#1552) reuses whatsapp_messages with content-free rows for unknown-sender declines instead of a new table or aggregate RPC: no migration (no orphan risk), the wamid unique index gives redelivery dedupe for free (a redelivered bad-code or greeted message no longer earns a second reply), and the existing 30-day unknown-sender retention pass already deletes the rows. Write amplification from an over-quota flood is bounded by a 20-rows-per-hash-per-day trace cap, not by dropping the trail entirely. The settings panel gets a closed event enum derived server-side (lib/last-event.ts), never raw error_message text, so internal errors cannot leak to the client. [2026-08-13] Issue #546 ships a provider-agnostic Peppol BIS Billing 3 XML export with strict Swedish preflight, not a fake send path: certified access-point delivery, SMP lookup, receipts, inbound handling, credentials, and commercial terms depend on Emil selecting and contracting a multitenant provider, and the existing email delivery state cannot truthfully model those guarantees. diff --git a/extensions/general/enable-banking/__tests__/accounts-route.test.ts b/extensions/general/enable-banking/__tests__/accounts-route.test.ts index 225e54d4..eae1a3d0 100644 --- a/extensions/general/enable-banking/__tests__/accounts-route.test.ts +++ b/extensions/general/enable-banking/__tests__/accounts-route.test.ts @@ -27,8 +27,23 @@ vi.mock('@/lib/cash-accounts/service', () => ({ iban ? iban.replace(/\s+/g, '').toUpperCase() || null : null, })) +// Mock the reconciliation modules so the renewal-guard sweep is deterministic +// and observable. The mocks must export every symbol index.ts imports. +const { mockRunReconciliation, mockResolveCashAccountScope } = vi.hoisted(() => ({ + mockRunReconciliation: vi.fn(), + mockResolveCashAccountScope: vi.fn(), +})) +vi.mock('@/lib/reconciliation/bank-reconciliation', () => ({ + runReconciliation: (...args: unknown[]) => mockRunReconciliation(...args), + DEFAULT_UNATTENDED_CONFIDENCE_THRESHOLD: 0.9, +})) +vi.mock('@/lib/reconciliation/cash-account-scope', () => ({ + resolveCashAccountScope: (...args: unknown[]) => mockResolveCashAccountScope(...args), +})) + import { enableBankingExtension } from '../index' import { syncAccountTransactions } from '../lib/sync' +import { eventBus } from '@/lib/events/bus' import type { ExtensionContext } from '@/lib/extensions/types' import type { StoredAccount } from '../types' @@ -67,6 +82,10 @@ interface SupabaseStub { bank_connection_id: string | null ledger_account: string }> + /** Completed SIE import overlapping the backfill window (renewal-flood guard). */ + sieImportRow?: { id: string } | null + /** company_members row for the caller; role 'viewer' disables the sweep. */ + membershipRow?: { role: string } | null /** Last update payload (may be overwritten by a follow-up metadata update). */ capturedUpdate?: Record /** All update payloads in order: first is the status flip, second the initial-sync metadata. */ @@ -103,6 +122,24 @@ function buildSupabase(stub: SupabaseStub) { eq: vi.fn(() => Promise.resolve({ data: stub.cashAccountRows ?? [], error: null })), } } + // Renewal-flood guard: SIE-overlap probe before the inline backfill. + if (table === 'sie_imports') { + return { + select: vi.fn().mockReturnThis(), + eq: vi.fn().mockReturnThis(), + gte: vi.fn().mockReturnThis(), + limit: vi.fn().mockReturnThis(), + maybeSingle: vi.fn().mockResolvedValue({ data: stub.sieImportRow ?? null, error: null }), + } + } + // Role probe for the reconciliation sweep (viewers cannot write links). + if (table === 'company_members') { + return { + select: vi.fn().mockReturnThis(), + eq: vi.fn().mockReturnThis(), + maybeSingle: vi.fn().mockResolvedValue({ data: stub.membershipRow ?? null, error: null }), + } + } return { select: vi.fn().mockReturnThis(), eq: vi.fn().mockReturnThis(), @@ -167,7 +204,25 @@ const ALLOCATOR_DEFAULTS: Record = { describe('PATCH /accounts (enable-banking)', () => { beforeEach(() => { vi.clearAllMocks() + eventBus.clear() mockUpsertFromPsd2.mockResolvedValue(undefined) + mockRunReconciliation.mockResolvedValue({ + matches: [], + applied: 0, + errors: 0, + skippedBelowThreshold: 0, + }) + // Scope stand-in: echo the requested account (or the '1930' default) so + // tests can assert the sweep runs once per ledger account, correctly scoped. + mockResolveCashAccountScope.mockImplementation( + async (_supabase: unknown, _companyId: unknown, accountNumber?: string) => ({ + accountNumber: accountNumber ?? '1930', + currency: 'SEK', + cashAccountId: `ca-${accountNumber ?? '1930'}`, + includeUnassigned: accountNumber === undefined, + found: true, + }), + ) // Default: no revoked connections; individual tests override to exercise // the self-heal path. mockGetRevokedConnectionIds.mockResolvedValue(new Set()) @@ -520,6 +575,252 @@ describe('PATCH /accounts (enable-banking)', () => { expect(meta?.last_synced_at).toBeDefined() }) + it('suppresses auto-categorization and reconciles per ledger account when the window overlaps an SIE import (renewal-flood guard)', async () => { + mockedSync.mockResolvedValue({ + imported: 22, + duplicates: 0, + errors: 0, + returnedMinBookingDate: '2026-01-05', + returnedMaxBookingDate: '2026-05-06', + }) + mockRunReconciliation + .mockResolvedValueOnce({ matches: [{}, {}, {}, {}], applied: 3, errors: 0, skippedBelowThreshold: 1 }) + .mockResolvedValueOnce({ matches: [{}, {}], applied: 2, errors: 0, skippedBelowThreshold: 0 }) + + const stub: SupabaseStub = { + authUser: { id: 'user-1' }, + connectionRow: { + id: 'conn-1', + status: 'pending_selection', + accounts_data: [ + // ledger_account preset on the stored account: no account_mappings + // in the request, so the chart/collision validation stays out of scope. + { uid: 'acc-1', currency: 'SEK', enabled: true, ledger_account: '1930' }, + { uid: 'acc-2', currency: 'EUR', enabled: true }, + ] as StoredAccount[], + }, + sieImportRow: { id: 'sie-1' }, + } + const supabase = buildSupabase(stub) + const ctx = makeContext(supabase) + + const res = await accountsRoute.handler( + makeRequest({ + connection_id: 'conn-1', + enabled_uids: ['acc-1', 'acc-2'], + initial_lookback_days: 90, + }), + ctx + ) + + expect(res.status).toBe(200) + const body = await res.json() + + // Auto-categorization is suppressed: booking these rows through mapping + // rules would double-book the already-imported period. + expect(mockedSync).toHaveBeenCalledWith( + expect.anything(), + 'company-1', + 'user-1', + 'conn-1', + expect.objectContaining({ uid: 'acc-1' }), + expect.any(String), + expect.any(String), + undefined, + { strategy: 'longest', skipAutoCategorization: true } + ) + + // One scoped sweep per distinct ledger account: pooled unscoped runs can + // cross-link accounts (#1290/#1298). The EUR account had no stored + // ledger_account, but the PATCH allocator assigns one ('1932') before the + // backfill, so both sweeps run with a concrete account. + expect(mockResolveCashAccountScope).toHaveBeenCalledTimes(2) + expect(mockResolveCashAccountScope).toHaveBeenCalledWith(expect.anything(), 'company-1', '1930') + expect(mockResolveCashAccountScope).toHaveBeenCalledWith(expect.anything(), 'company-1', '1932') + expect(mockRunReconciliation).toHaveBeenCalledTimes(2) + expect(mockRunReconciliation).toHaveBeenCalledWith( + expect.anything(), + 'company-1', + 'user-1', + expect.objectContaining({ + accountNumber: '1930', + currency: 'SEK', + cashAccountId: 'ca-1930', + includeUnassigned: false, + confidenceThreshold: 0.9, + // The bank over-returned history: the sweep window opens at the + // oldest returned booking date, not the requested 90-day fromDate, + // so over-returned rows are swept too. + dateFrom: '2026-01-05', + dateTo: expect.stringMatching(/^\d{4}-\d{2}-\d{2}$/), + }) + ) + + // Applied links surface to the UI so the user sees the period was + // recognized, not re-imported as work. + expect(body.initial_sync.auto_matched).toBe(5) + }) + + it('runs no reconciliation sweep and keeps categorization on without SIE overlap', async () => { + mockedSync.mockResolvedValue({ + imported: 10, + duplicates: 0, + errors: 0, + returnedMinBookingDate: '2026-05-01', + returnedMaxBookingDate: '2026-05-13', + }) + + const stub: SupabaseStub = { + authUser: { id: 'user-1' }, + connectionRow: { + id: 'conn-1', + status: 'pending_selection', + accounts_data: [{ uid: 'acc-1', currency: 'SEK', enabled: true }], + }, + } + const supabase = buildSupabase(stub) + const ctx = makeContext(supabase) + + const res = await accountsRoute.handler( + makeRequest({ connection_id: 'conn-1', enabled_uids: ['acc-1'], initial_lookback_days: 90 }), + ctx + ) + + expect(res.status).toBe(200) + const body = await res.json() + expect(mockedSync).toHaveBeenCalledWith( + expect.anything(), + 'company-1', + 'user-1', + 'conn-1', + expect.anything(), + expect.any(String), + expect.any(String), + undefined, + { strategy: 'longest' } + ) + expect(mockRunReconciliation).not.toHaveBeenCalled() + expect(body.initial_sync.auto_matched).toBe(0) + }) + + it('gives viewers rawInsertOnly and no sweep even with SIE overlap (viewers cannot write links)', async () => { + mockedSync.mockResolvedValue({ + imported: 5, + duplicates: 0, + errors: 0, + returnedMinBookingDate: '2026-05-01', + returnedMaxBookingDate: '2026-05-13', + }) + + const stub: SupabaseStub = { + authUser: { id: 'user-1' }, + connectionRow: { + id: 'conn-1', + status: 'pending_selection', + accounts_data: [{ uid: 'acc-1', currency: 'SEK', enabled: true }], + }, + sieImportRow: { id: 'sie-1' }, + membershipRow: { role: 'viewer' }, + } + const supabase = buildSupabase(stub) + const ctx = makeContext(supabase) + + const res = await accountsRoute.handler( + makeRequest({ connection_id: 'conn-1', enabled_uids: ['acc-1'], initial_lookback_days: 90 }), + ctx + ) + + expect(res.status).toBe(200) + expect(mockedSync).toHaveBeenCalledWith( + expect.anything(), + 'company-1', + 'user-1', + 'conn-1', + expect.anything(), + expect.any(String), + expect.any(String), + undefined, + { strategy: 'longest', skipAutoCategorization: true, rawInsertOnly: true } + ) + expect(mockRunReconciliation).not.toHaveBeenCalled() + }) + + it('skips the sweep for an account whose cash_accounts row did not resolve instead of widening to the pooled form', async () => { + mockedSync.mockResolvedValue({ + imported: 6, + duplicates: 0, + errors: 0, + returnedMinBookingDate: '2026-05-01', + returnedMaxBookingDate: '2026-05-13', + }) + // found: false = no cash_accounts row (e.g. the mirror upsert failed + // earlier in the request). Running anyway would sweep currency-only + // across every same-currency account (#1290 write shape). + mockResolveCashAccountScope.mockResolvedValue({ + accountNumber: '1930', + currency: 'SEK', + cashAccountId: undefined, + includeUnassigned: true, + found: false, + }) + + const stub: SupabaseStub = { + authUser: { id: 'user-1' }, + connectionRow: { + id: 'conn-1', + status: 'pending_selection', + accounts_data: [{ uid: 'acc-1', currency: 'SEK', enabled: true, ledger_account: '1930' }] as StoredAccount[], + }, + sieImportRow: { id: 'sie-1' }, + } + const supabase = buildSupabase(stub) + const ctx = makeContext(supabase) + + const res = await accountsRoute.handler( + makeRequest({ connection_id: 'conn-1', enabled_uids: ['acc-1'], initial_lookback_days: 90 }), + ctx + ) + + expect(res.status).toBe(200) + const body = await res.json() + expect(mockResolveCashAccountScope).toHaveBeenCalled() + expect(mockRunReconciliation).not.toHaveBeenCalled() + expect(body.initial_sync.auto_matched).toBe(0) + }) + + it('keeps the backfill successful when the reconciliation sweep throws (non-critical)', async () => { + mockedSync.mockResolvedValue({ + imported: 8, + duplicates: 0, + errors: 0, + returnedMinBookingDate: '2026-05-01', + returnedMaxBookingDate: '2026-05-13', + }) + mockRunReconciliation.mockRejectedValue(new Error('recon exploded')) + + const stub: SupabaseStub = { + authUser: { id: 'user-1' }, + connectionRow: { + id: 'conn-1', + status: 'pending_selection', + accounts_data: [{ uid: 'acc-1', currency: 'SEK', enabled: true }], + }, + sieImportRow: { id: 'sie-1' }, + } + const supabase = buildSupabase(stub) + const ctx = makeContext(supabase) + + const res = await accountsRoute.handler( + makeRequest({ connection_id: 'conn-1', enabled_uids: ['acc-1'], initial_lookback_days: 90 }), + ctx + ) + + expect(res.status).toBe(200) + const body = await res.json() + expect(body.initial_sync).toMatchObject({ imported: 8, auto_matched: 0 }) + expect(body.initial_sync_error).toBeUndefined() + }) + it('does NOT run inline sync when connection is already active (selection edit)', async () => { mockedSync.mockResolvedValue({ imported: 99, diff --git a/extensions/general/enable-banking/components/AccountPickerDialog.tsx b/extensions/general/enable-banking/components/AccountPickerDialog.tsx index 87c646f8..76ee6b16 100644 --- a/extensions/general/enable-banking/components/AccountPickerDialog.tsx +++ b/extensions/general/enable-banking/components/AccountPickerDialog.tsx @@ -1,6 +1,6 @@ 'use client' -import { useEffect, useMemo, useState } from 'react' +import { useEffect, useMemo, useRef, useState } from 'react' import Link from 'next/link' import { Dialog, @@ -31,6 +31,7 @@ import { import { resolveBookedCoverage, resolveFiscalYearStart, + resolveGapFillStart, } from '../lib/date-suggestions' import type { CompanySettings } from '@/types' import type { StoredAccount } from '../types' @@ -57,7 +58,7 @@ interface ChartAccount { account_name: string } -type LookbackMode = 'fast' | 'fiscal-year' | 'custom' +type LookbackMode = 'gap-fill' | 'fast' | 'fiscal-year' | 'custom' type CustomSubMode = 'date' | 'previous-fiscal-year' // Suggested BAS account per currency. The mapping engine falls back to 1930 @@ -104,6 +105,15 @@ export function AccountPickerDialog({ const [lookbackMode, setLookbackMode] = useState('fiscal-year') const [customSubMode, setCustomSubMode] = useState('date') const [customDate, setCustomDate] = useState('') + // Newest transaction date this CONNECTION has already imported (date null on + // a first connect). A date means this is a renewal, where the default must be + // "fill the gap", not a fresh long lookback over bookkept periods. Keyed by + // connectionId so a stale value can never render into another connection's + // dialog between open and probe: the gapFill memo ignores mismatched keys. + const [latestImported, setLatestImported] = useState<{ connectionId: string; date: string | null } | null>(null) + // Ref, not state: only the async default below reads it, and putting it in + // the effect's deps would re-fire the query on the first radio click. + const lookbackTouched = useRef(false) const [progressOpen, setProgressOpen] = useState(false) const [progressState, setProgressState] = useState({ kind: 'syncing' }) @@ -126,6 +136,7 @@ export function AccountPickerDialog({ setLookbackMode('fiscal-year') setCustomSubMode('date') setCustomDate('') + lookbackTouched.current = false // Pre-populate ledger picks from existing StoredAccount values, falling // back to currency-based suggestions for accounts the user hasn't mapped @@ -213,6 +224,45 @@ export function AccountPickerDialog({ return () => { cancelled = true } }, [open, isInitialSelection, company?.id, supabase]) + // Fetch the newest transaction this connection has already imported. Any row + // means this pending_selection is a RENEWAL: the flow re-runs the initial + // backfill, and a fresh consent often makes the bank release history the + // first connect never delivered. Defaulting to the fiscal-year lookback then + // re-imports whole bookkept periods as "ohanterade" (the 2026-08 renewal + // flood), so a renewal defaults to gap-fill instead, unless the user has + // already picked a mode by the time the query lands. + useEffect(() => { + // `accounts` is deliberately a dep even though only its length is read: the + // reset effect above re-runs on every accounts identity change (the panel's + // visibility refetch produces a fresh array mid-open, e.g. returning from a + // BankID app switch) and resets the lookback default. Re-probing on the + // same trigger re-establishes the gap-fill default; without it the reset + // would silently strand a renewal back on the fiscal-year default. + if (!open || !isInitialSelection || !company?.id || accounts.length === 0) return + let cancelled = false + ;(async () => { + const { data, error } = await supabase + .from('transactions') + .select('date') + .eq('company_id', company.id) + .eq('bank_connection_id', connectionId) + .order('date', { ascending: false }) + .limit(1) + .maybeSingle() + if (cancelled) return + if (error) { + // A failed probe must not read as "first connect": deriving the + // fiscal-year default from an unknown state is the flood case itself. + console.warn('[enable-banking] latest-import probe failed', error.message) + return + } + const date = (data as { date?: string } | null)?.date || null + setLatestImported({ connectionId, date }) + if (date && !lookbackTouched.current) setLookbackMode('gap-fill') + })() + return () => { cancelled = true } + }, [open, isInitialSelection, company?.id, connectionId, supabase, accounts]) + // Load 19xx accounts from the chart for the per-account ledger combobox. // Class 19 = bank/cash on the BAS chart. useEffect(() => { @@ -301,13 +351,13 @@ export function AccountPickerDialog({ return } - // Block save when the user picked "Anpassat datum" but left the date blank. - // Without this guard, lookback.body is null and the PATCH would silently - // fall back to the backend's 120-day default, not what the user asked for. + // Block save when the chosen mode resolved to no request body: a blank + // custom date, or gap-fill whose suggestion vanished. Without this guard, + // lookback.body is null and the PATCH would silently fall back to the + // backend's 120-day default, not what the user asked for. if ( isInitialSelection && - lookbackMode === 'custom' && - customSubMode === 'date' && + ((lookbackMode === 'custom' && customSubMode === 'date') || lookbackMode === 'gap-fill') && !lookback.body ) { toast({ @@ -418,6 +468,13 @@ export function AccountPickerDialog({ [lastBookedDate], ) + const gapFill = useMemo( + () => (latestImported?.connectionId === connectionId + ? resolveGapFillStart(latestImported.date) + : null), + [latestImported, connectionId], + ) + const fiscalYearStart = useMemo( () => resolveFiscalYearStart(currentPeriodStart, companySettings), [currentPeriodStart, companySettings], @@ -430,6 +487,19 @@ export function AccountPickerDialog({ // Resolve mode → concrete request payload and a "resolved from-date" for display. const lookback = useMemo(() => { + if (lookbackMode === 'gap-fill') { + // The gap-fill radio only renders when gapFill resolved, but guard the + // body anyway: a null body falls into the same save-block as a blank + // custom date instead of silently syncing the server's 120-day default. + if (gapFill) { + return { + body: { initial_lookback_from_date: gapFill.suggestedStartDate }, + fromDate: gapFill.suggestedStartDate, + days: daysBetween(gapFill.suggestedStartDate), + } + } + return { body: null as Record | null, fromDate: null as string | null, days: 0 } + } if (lookbackMode === 'fast') { return { body: { initial_lookback_days: 90 }, fromDate: null as string | null, days: 90 } } @@ -442,10 +512,21 @@ export function AccountPickerDialog({ return { body: { initial_lookback_from_date: date }, fromDate: date, days: daysBetween(date) } } return { body: null as Record | null, fromDate: null as string | null, days: 0 } - }, [lookbackMode, customSubMode, customDate, fiscalYearStart, previousFiscalYearStart]) + }, [lookbackMode, customSubMode, customDate, fiscalYearStart, previousFiscalYearStart, gapFill]) const showLongRangeHelper = lookback.days > 90 + // On a renewal, a lookback reaching past what the connection already + // delivered re-imports periods that may already be bookkept: with a fresh + // consent the bank often releases history the first connect never returned. + const reimportsFetchedPeriod = Boolean( + gapFill && + lookbackMode !== 'gap-fill' && + (lookback.fromDate + ? lookback.fromDate < gapFill.latestImportedDate + : lookback.days > daysBetween(gapFill.latestImportedDate)), + ) + return ( <> { + lookbackTouched.current = true setLookbackMode('custom') setCustomSubMode('date') setCustomDate(bookedCoverage.suggestedStartDate) @@ -538,13 +620,33 @@ export function AccountPickerDialog({ )}
+ {gapFill && ( + + )} +
+ {reimportsFetchedPeriod && gapFill && ( +

+ Du har redan hämtat transaktioner till och med {gapFill.latestImportedDate}. Ett + tidigare startdatum kan hämta mer historik från banken, och dagar som redan är + bokförda kan då dyka upp som ohanterade. +

+ )} + {showLongRangeHelper && (

Din bank returnerar oftast max 90 dagar. Behöver du äldre transaktioner kan du{' '} diff --git a/extensions/general/enable-banking/components/BankSyncProgressDialog.tsx b/extensions/general/enable-banking/components/BankSyncProgressDialog.tsx index 4d391752..e2ec9c3e 100644 --- a/extensions/general/enable-banking/components/BankSyncProgressDialog.tsx +++ b/extensions/general/enable-banking/components/BankSyncProgressDialog.tsx @@ -18,6 +18,12 @@ import type { StoredAccount } from '../types' export interface SyncProgressSummary { imported: number duplicates: number + /** + * Rows the post-backfill reconciliation sweep linked to existing verifikat + * (renewal over an already-bookkept period). Optional: responses from before + * the sweep existed omit it. + */ + auto_matched?: number requested_from: string returned_min_date: string | null returned_max_date: string | null @@ -252,6 +258,11 @@ function DoneBody({ Datum: {summary.returned_min_date} → {summary.returned_max_date}

)} + {(summary.auto_matched ?? 0) > 0 && ( +

+ {summary.auto_matched} kopplades automatiskt till redan bokförda verifikat. +

+ )} {workCounts && workCounts.book > 0 && (

{workCounts.book} att bokföra diff --git a/extensions/general/enable-banking/index.ts b/extensions/general/enable-banking/index.ts index 0a906f80..846f55cd 100644 --- a/extensions/general/enable-banking/index.ts +++ b/extensions/general/enable-banking/index.ts @@ -17,6 +17,7 @@ import { runReconciliation, DEFAULT_UNATTENDED_CONFIDENCE_THRESHOLD, } from '@/lib/reconciliation/bank-reconciliation' +import { resolveCashAccountScope } from '@/lib/reconciliation/cash-account-scope' import { checkRateLimit } from '@/lib/auth/rate-limit-http' import { requireCapability } from '@/lib/entitlements/has-capability' import { CAPABILITY } from '@/lib/entitlements/keys' @@ -1336,6 +1337,7 @@ export const enableBankingExtension: Extension = { let initialSyncSummary: { imported: number duplicates: number + auto_matched: number requested_from: string returned_min_date: string | null returned_max_date: string | null @@ -1349,12 +1351,37 @@ export const enableBankingExtension: Extension = { .toISOString() .split('T')[0] + // Same guard the manual /sync route and the cron apply. This path also + // runs on RENEWAL (reconnect resets status to pending_selection), and a + // fresh consent often makes the bank release history the first connect + // never delivered, straight over an already-bookkept period. Without + // the guard those rows would be auto-categorized into brand-new + // verifikat (double-booking) instead of being linked to the ones that + // already describe them. + const { data: sieOverlap } = await supabase + .from('sie_imports') + .select('id') + .eq('company_id', companyId) + .eq('status', 'completed') + .gte('fiscal_year_end', fromDate) + .limit(1) + .maybeSingle() + + const { data: membership } = await supabase + .from('company_members') + .select('role') + .eq('company_id', companyId) + .eq('user_id', user.id) + .maybeSingle() + const isViewer = membership?.role === 'viewer' + log.info('[enable-banking] Starting inline initial backfill', { connectionId: connection.id, accountCount: accountsToSync.length, lookbackDays: initialLookbackDays, fromDate, toDate, + sieOverlap: Boolean(sieOverlap), }) let timeoutHandle: ReturnType | undefined @@ -1370,7 +1397,11 @@ export const enableBankingExtension: Extension = { fromDate, toDate, ingestFn, - { strategy: 'longest' } + { + strategy: 'longest', + ...(sieOverlap ? { skipAutoCategorization: true } : {}), + ...(isViewer ? { rawInsertOnly: true } : {}), + } )) ) // If the timeout wins the race, the underlying Promise.all keeps @@ -1397,6 +1428,78 @@ export const enableBankingExtension: Extension = { const returnedMin = minDates.length > 0 ? minDates.reduce((a, b) => (a < b ? a : b)) : null const returnedMax = maxDates.length > 0 ? maxDates.reduce((a, b) => (a > b ? a : b)) : null + // Post-backfill reconciliation sweep, mirroring the manual /sync + // route: link just-imported rows to the verifikat that already + // describe them so a re-released period does not resurface as + // hundreds of "ohanterade" transactions. Unlike the /sync and cron + // sweeps this runs once per enabled ledger account with a resolved + // cash-account scope: the pooled unscoped form can cross-link + // accounts (#1290/#1298). Both a thrown scope resolution AND an + // unresolved cash-account row (found: false) skip that account's + // sweep instead of widening to the pooled form. + // + // The window opens at the OLDEST booking date the bank actually + // returned when that is older than the requested fromDate: some + // ASPSPs over-return history, and rows outside the requested window + // would otherwise be ingested but never swept. + const sweepDateFrom = returnedMin && returnedMin < fromDate ? returnedMin : fromDate + // The sweep writes bank-feed metadata only (transactions.journal_entry_id, + // reconciliation_method, is_business): journal tables are never touched, + // so BFL immutability and period locks (which guard journal entries) are + // not in play. Links to opening-balance verifikat are blocked by the + // check_transaction_link_not_opening_balance trigger, and every link is + // reversible via unlinkReconciliation without any ledger write. + let totalAutoMatched = 0 + if (sieOverlap && totalImported > 0 && !isViewer) { + // Filter, not `?? undefined`: an undefined accountNumber makes + // resolveCashAccountScope fall back to the primary account with + // includeUnassigned=true, which is the pooled form this block must + // never widen to. The allocator gives every enabled account a + // concrete ledger_account, so nothing is skipped in practice. + const ledgerAccounts = Array.from( + new Set( + accountsToSync + .map(a => a.ledger_account) + .filter((l): l is string => typeof l === 'string' && l.length > 0) + ) + ) + for (const ledgerAccount of ledgerAccounts) { + try { + const scope = await resolveCashAccountScope(supabase, companyId, ledgerAccount) + if (!scope.found) { + log.warn('[enable-banking] No cash_accounts row for ledger account; skipping its reconciliation sweep', { + connectionId: connection.id, + ledgerAccount, + }) + continue + } + const reconResult = await runReconciliation(supabase, companyId, user.id, { + dateFrom: sweepDateFrom, + dateTo: toDate, + accountNumber: scope.accountNumber, + currency: scope.currency, + cashAccountId: scope.cashAccountId, + includeUnassigned: scope.includeUnassigned, + // Unattended run: nobody reviews a dry-run first, so never + // commit low-confidence (fuzzy / date-range) matches. + confidenceThreshold: DEFAULT_UNATTENDED_CONFIDENCE_THRESHOLD, + }) + totalAutoMatched += reconResult.applied + if (reconResult.applied > 0 || reconResult.skippedBelowThreshold > 0) { + log.info('[enable-banking] Post-backfill reconciliation linked imported rows to existing verifikat', { + connectionId: connection.id, + accountNumber: scope.accountNumber, + applied: reconResult.applied, + skippedBelowThreshold: reconResult.skippedBelowThreshold, + total: reconResult.matches.length, + }) + } + } catch { + // Non-critical: rows stay unmatched for manual review. + } + } + } + const completedAt = new Date().toISOString() // Don't re-write accounts_data here: the first update already wrote it. // Including it again races with any concurrent writer (e.g. cron firing in @@ -1432,6 +1535,7 @@ export const enableBankingExtension: Extension = { initialSyncSummary = { imported: totalImported, duplicates: totalDuplicates, + auto_matched: totalAutoMatched, requested_from: fromDate, returned_min_date: returnedMin, returned_max_date: returnedMax, diff --git a/extensions/general/enable-banking/lib/__tests__/date-suggestions.test.ts b/extensions/general/enable-banking/lib/__tests__/date-suggestions.test.ts index 8d95cc19..6ba252ba 100644 --- a/extensions/general/enable-banking/lib/__tests__/date-suggestions.test.ts +++ b/extensions/general/enable-banking/lib/__tests__/date-suggestions.test.ts @@ -1,5 +1,9 @@ import { describe, it, expect } from 'vitest' -import { resolveBookedCoverage, resolveFiscalYearStart } from '../date-suggestions' +import { + resolveBookedCoverage, + resolveFiscalYearStart, + resolveGapFillStart, +} from '../date-suggestions' describe('resolveBookedCoverage', () => { it('suggests the day after the last posted verifikat date', () => { @@ -43,6 +47,50 @@ describe('resolveBookedCoverage', () => { }) }) +describe('resolveGapFillStart', () => { + it('suggests one week of overlap before the newest imported transaction', () => { + const today = new Date('2026-08-13T12:00:00Z') + expect(resolveGapFillStart('2026-08-06', today)).toEqual({ + latestImportedDate: '2026-08-06', + suggestedStartDate: '2026-07-30', + }) + }) + + it('rolls over month and year boundaries', () => { + // Todays within the 365-day floor of each case so only the 7-day overlap acts. + expect(resolveGapFillStart('2026-01-03', new Date('2026-01-15T12:00:00Z'))?.suggestedStartDate).toBe('2025-12-27') + expect(resolveGapFillStart('2026-03-04', new Date('2026-03-10T12:00:00Z'))?.suggestedStartDate).toBe('2026-02-25') + }) + + it('clamps to today when bank data claims a future date (backend rejects non-past dates)', () => { + const today = new Date('2026-08-13T12:00:00Z') + expect(resolveGapFillStart('2026-09-20', today)).toEqual({ + latestImportedDate: '2026-09-20', + suggestedStartDate: '2026-08-13', + }) + }) + + it('clamps a stale renewal to the backend 365-day lookback floor so the shown date matches the actual backfill', () => { + const today = new Date('2026-08-13T12:00:00Z') + // Newest import 2025-01-10; minus 7d = 2025-01-03, older than the 365-day + // floor (2025-08-13), which the backend would silently clamp to anyway. + expect(resolveGapFillStart('2025-01-10', today)).toEqual({ + latestImportedDate: '2025-01-10', + suggestedStartDate: '2025-08-13', + }) + }) + + it('returns null when the connection has never imported anything (first connect has no gap)', () => { + expect(resolveGapFillStart(null)).toBeNull() + expect(resolveGapFillStart(undefined)).toBeNull() + expect(resolveGapFillStart('')).toBeNull() + }) + + it('returns null for an unparsable date', () => { + expect(resolveGapFillStart('not-a-date')).toBeNull() + }) +}) + describe('resolveFiscalYearStart', () => { const calendarYearSettings = { fiscal_year_start_month: 1, diff --git a/extensions/general/enable-banking/lib/date-suggestions.ts b/extensions/general/enable-banking/lib/date-suggestions.ts index d7edf8ba..5479197d 100644 --- a/extensions/general/enable-banking/lib/date-suggestions.ts +++ b/extensions/general/enable-banking/lib/date-suggestions.ts @@ -43,6 +43,63 @@ export function resolveBookedCoverage( } } +export interface GapFillSuggestion { + /** Date of the newest transaction this connection has already imported. */ + latestImportedDate: string + /** + * Suggested sync start: latestImportedDate minus GAP_FILL_OVERLAP_DAYS, + * clamped to today (UTC) so the backend accepts it. + */ + suggestedStartDate: string +} + +/** + * Overlap requested before the newest already-imported row. The external_id + * dedup makes re-imported rows no-ops, so the overlap costs nothing, and it + * catches transactions the bank booked late around the boundary. + */ +export const GAP_FILL_OVERLAP_DAYS = 7 + +/** + * Turn the newest transaction a connection has already imported into a + * "continue where the last fetch stopped" suggestion for RENEWALS. + * + * A reconnect walks the same pending_selection → active flow as a first + * connect, and a fresh consent often makes the bank release history the first + * connect never delivered. Re-requesting a long lookback then floods the inbox + * with rows over already-bookkept periods (the 2026-08 renewal flood), so a + * renewal should default to fetching only the gap since the last import. + * Returns null when the connection has never imported anything: a first + * connect has no gap to fill. + */ +export function resolveGapFillStart( + latestImportedDate: string | null | undefined, + today: Date = new Date(), +): GapFillSuggestion | null { + if (!latestImportedDate) return null + const d = new Date(latestImportedDate + 'T00:00:00Z') + if (!Number.isFinite(d.getTime())) return null + d.setUTCDate(d.getUTCDate() - GAP_FILL_OVERLAP_DAYS) + let start = d.toISOString().split('T')[0] + // The backend clamps every lookback to 365 days. A renewal staler than that + // must show the date the backfill will actually start from, not promise a + // gap it cannot fill (the >90-day helper already points at SIE/file import + // for older history). + const floor = new Date(today.getTime()) + floor.setUTCDate(floor.getUTCDate() - 365) + const floorUtc = floor.toISOString().split('T')[0] + if (start < floorUtc) start = floorUtc + // The backend PATCH handler rejects initial_lookback_from_date unless it is + // strictly in the past; today (UTC) is the newest value it accepts. A + // latestImportedDate in the future can only come from bad bank data: clamp + // rather than propagate it. + const todayUtc = today.toISOString().split('T')[0] + return { + latestImportedDate, + suggestedStartDate: start <= todayUtc ? start : todayUtc, + } +} + /** * Resolve the start of the current fiscal year, preferring the actual * fiscal_periods row that contains today over the recurring