* fix(enable-banking): reliable initial backfill, no more silent ~30-day windows (#443) PSD2 first-sync was a compound bug: the cron runs once daily so users got no data for up to 24h after activation; the "Sync now" button defaulted to 30 days and set last_synced_at, permanently locking the cron into 7-day incremental mode and discarding the 90-day backfill window. ASPSPs also truncate history below requested ranges, but the discrepancy was only logged. This change: - Runs the initial backfill inline when the user finishes account selection (PATCH /accounts), so data is available the moment they finish onboarding. - Tracks initial_sync_completed_at separately from last_synced_at; the cron now gates first-sync 90-day window on that, so manual syncs no longer clobber the backfill path. - Surfaces the actual returned date range to the UI ("Initial historik: X → Y (begärde Z)") with a warning when the bank truncated history. - Defaults manual /sync to 90 days (was 30) — matches user intent. - AccountPickerDialog uses SpeedLedger's SIE-anchor pattern when an SIE import covers prior periods (auto-defaults lookback to "day after last SIE entry"), with Bokio-style PSD2 disclosure on the standard path. Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com> * fix(enable-banking): address review feedback on PR #486 Two fixes from review: 1. Memoise the browser Supabase client in AccountPickerDialog. createClient() in the component body returned a new reference every render, and `supabase` was in the SIE-fetch effect's dep array — every checkbox tick or parent re-render re-fired the SIE-imports query. 2. Drop `accounts_data` from the second supabase update inside the activation backfill. The first update already wrote it; including it here races with any concurrent writer (e.g. cron firing in the sub-60s window) and would silently overwrite. Only initial_sync_* metadata + last_synced_at need to be persisted in the second update. Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com> * fix(enable-banking): check metadata-update error after inline backfill The second supabase.update() inside the activation backfill block didn't check its error return. Supabase client methods don't throw on DB errors; they return { data, error }. If the metadata write failed (network blip, RLS quirk, etc.), the handler still populated initialSyncSummary and returned success — UI saw "imported N transactions" while the DB had initial_sync_completed_at = NULL, causing the cron to schedule another full 90-day backfill the next morning. Capture { error } from the metadata update. On failure, surface as initial_sync_error with a metadata_update_failed: prefix and skip the initialSyncSummary population. The cron's gate (initial_sync_completed_at IS NULL) still self-heals on the next run; this just keeps the UI honest about which path got us there. New test stub: SupabaseStub.updateErrorByCall lets a test succeed the first update and fail the second. Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com> --------- Co-authored-by: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
This commit is contained in:
Jakob Wennberg
co-authored by
Claude Opus 4.7
Claude Opus 4.7
parent
5eee06a56d
commit
04f902fe8f
@@ -256,7 +256,9 @@ export const enableBankingExtension: Extension = {
|
||||
})
|
||||
if (!rl.ok) return rl.response!
|
||||
|
||||
const { connection_id, days_back: rawDaysBack = 30 } = await request.json()
|
||||
// Default 90 days: most callers (Sync Now button, post-activation gap fill)
|
||||
// want a deep refresh, not a 30-day blip. Cron uses 7-day incrementals separately.
|
||||
const { connection_id, days_back: rawDaysBack = 90 } = await request.json()
|
||||
const days_back = Math.min(Math.max(1, rawDaysBack), 365)
|
||||
|
||||
const { data: connection, error: connectionError } = await supabase
|
||||
@@ -453,6 +455,7 @@ export const enableBankingExtension: Extension = {
|
||||
const body = await request.json().catch(() => null)
|
||||
const connection_id = body?.connection_id
|
||||
const enabled_uids = body?.enabled_uids
|
||||
const rawLookback = body?.initial_lookback_days
|
||||
|
||||
if (typeof connection_id !== 'string' || !connection_id) {
|
||||
return NextResponse.json({ error: 'connection_id krävs' }, { status: 400 })
|
||||
@@ -473,6 +476,13 @@ export const enableBankingExtension: Extension = {
|
||||
)
|
||||
}
|
||||
|
||||
// initial_lookback_days only applies on the pending_selection→active transition.
|
||||
// Default 90 (PSD2 standard); clamp to [30, 365]. Ignored for selection edits.
|
||||
const initialLookbackDays = (() => {
|
||||
const n = typeof rawLookback === 'number' && Number.isFinite(rawLookback) ? rawLookback : 90
|
||||
return Math.min(365, Math.max(30, Math.round(n)))
|
||||
})()
|
||||
|
||||
const { data: connection, error: connectionError } = await supabase
|
||||
.from('bank_connections')
|
||||
.select('id, status, accounts_data, bank_name')
|
||||
@@ -568,10 +578,130 @@ export const enableBankingExtension: Extension = {
|
||||
})
|
||||
}
|
||||
|
||||
// Initial backfill on activation. Run inline so the user has data the
|
||||
// moment they finish account selection — no 24h cron wait. Failures
|
||||
// here don't fail the PATCH; the cron will retry on its next run
|
||||
// (gated on initial_sync_completed_at IS NULL).
|
||||
let initialSyncSummary: {
|
||||
imported: number
|
||||
duplicates: number
|
||||
requested_from: string
|
||||
returned_min_date: string | null
|
||||
returned_max_date: string | null
|
||||
} | null = null
|
||||
let initialSyncError: string | null = null
|
||||
|
||||
if (connection.status === 'pending_selection') {
|
||||
const accountsToSync = updatedAccounts.filter(a => a.enabled !== false)
|
||||
const toDate = new Date().toISOString().split('T')[0]
|
||||
const fromDate = new Date(Date.now() - initialLookbackDays * 24 * 60 * 60 * 1000)
|
||||
.toISOString()
|
||||
.split('T')[0]
|
||||
|
||||
log.info('[enable-banking] Starting inline initial backfill', {
|
||||
connectionId: connection.id,
|
||||
accountCount: accountsToSync.length,
|
||||
lookbackDays: initialLookbackDays,
|
||||
fromDate,
|
||||
toDate,
|
||||
})
|
||||
|
||||
let timeoutHandle: ReturnType<typeof setTimeout> | undefined
|
||||
try {
|
||||
const ingestFn = ctx?.services.ingestTransactions
|
||||
const syncPromise = Promise.all(
|
||||
accountsToSync.map(account => syncAccountTransactions(
|
||||
supabase,
|
||||
companyId,
|
||||
user.id,
|
||||
connection.id,
|
||||
account,
|
||||
fromDate,
|
||||
toDate,
|
||||
ingestFn,
|
||||
{ strategy: 'longest' }
|
||||
))
|
||||
)
|
||||
|
||||
const TIMEOUT_MS = 60_000
|
||||
const timeoutPromise = new Promise<never>((_, reject) => {
|
||||
timeoutHandle = setTimeout(() => reject(new Error('initial_sync_timeout')), TIMEOUT_MS)
|
||||
})
|
||||
const results = await Promise.race([syncPromise, timeoutPromise])
|
||||
|
||||
const totalImported = results.reduce((sum, r) => sum + r.imported, 0)
|
||||
const totalDuplicates = results.reduce((sum, r) => sum + r.duplicates, 0)
|
||||
|
||||
// Min/max booking date across all synced accounts
|
||||
const minDates = results.map(r => r.returnedMinBookingDate).filter((d): d is string => !!d)
|
||||
const maxDates = results.map(r => r.returnedMaxBookingDate).filter((d): d is string => !!d)
|
||||
const returnedMin = minDates.length > 0 ? minDates.reduce((a, b) => (a < b ? a : b)) : null
|
||||
const returnedMax = maxDates.length > 0 ? maxDates.reduce((a, b) => (a > b ? a : b)) : null
|
||||
|
||||
const completedAt = new Date().toISOString()
|
||||
// Don't re-write accounts_data here — the first update already wrote it.
|
||||
// Including it again races with any concurrent writer (e.g. cron firing in
|
||||
// the sub-60s window) and would silently overwrite their changes.
|
||||
const { error: metaUpdateError } = await supabase
|
||||
.from('bank_connections')
|
||||
.update({
|
||||
last_synced_at: completedAt,
|
||||
initial_sync_completed_at: completedAt,
|
||||
initial_sync_requested_from: fromDate,
|
||||
initial_sync_returned_min_date: returnedMin,
|
||||
initial_sync_returned_max_date: returnedMax,
|
||||
initial_sync_lookback_days: initialLookbackDays,
|
||||
})
|
||||
.eq('id', connection.id)
|
||||
|
||||
if (metaUpdateError) {
|
||||
// The sync itself succeeded (transactions are ingested) but we
|
||||
// couldn't persist that. Falsely reporting success would tell the
|
||||
// client "imported N transactions" while the DB still has
|
||||
// initial_sync_completed_at = NULL, causing the cron to re-run a
|
||||
// 90-day backfill next morning. Surface this as initial_sync_error
|
||||
// so the UI shows a "background sync needs retry" warning, and the
|
||||
// cron's gate (initial_sync_completed_at IS NULL) will self-heal.
|
||||
initialSyncError = `metadata_update_failed: ${metaUpdateError.message}`
|
||||
log.error('[enable-banking] Failed to persist initial_sync metadata after backfill', {
|
||||
connectionId: connection.id,
|
||||
error: metaUpdateError.message,
|
||||
userId: user.id,
|
||||
companyId,
|
||||
})
|
||||
} else {
|
||||
initialSyncSummary = {
|
||||
imported: totalImported,
|
||||
duplicates: totalDuplicates,
|
||||
requested_from: fromDate,
|
||||
returned_min_date: returnedMin,
|
||||
returned_max_date: returnedMax,
|
||||
}
|
||||
|
||||
log.info('[enable-banking] Inline initial backfill complete', {
|
||||
connectionId: connection.id,
|
||||
...initialSyncSummary,
|
||||
})
|
||||
}
|
||||
} catch (syncError) {
|
||||
initialSyncError = syncError instanceof Error ? syncError.message : String(syncError)
|
||||
log.error('[enable-banking] Inline initial backfill failed — cron will retry', {
|
||||
connectionId: connection.id,
|
||||
error: initialSyncError,
|
||||
userId: user.id,
|
||||
companyId,
|
||||
})
|
||||
} finally {
|
||||
if (timeoutHandle) clearTimeout(timeoutHandle)
|
||||
}
|
||||
}
|
||||
|
||||
return NextResponse.json({
|
||||
success: true,
|
||||
enabled_count: enabled_uids.length,
|
||||
total_count: existing.length,
|
||||
...(initialSyncSummary ? { initial_sync: initialSyncSummary } : {}),
|
||||
...(initialSyncError ? { initial_sync_error: initialSyncError } : {}),
|
||||
})
|
||||
},
|
||||
},
|
||||
|
||||
Reference in New Issue
Block a user