fix(sandbox): fit the nightly cleanup inside PostgREST's 8s session cap (#1451)

* fix(sandbox): fit the nightly cleanup inside PostgREST's 8s session cap Follow-up to #1449. Profiling the repaired teardown on prod puts one sandbox user at ~3s (the auth.users delete fans out over ~250 FK triggers; FK indexes were tried inside an aborted transaction and do not help), while every PostgREST session inherits authenticator's statement_timeout = 8s. The nightly RPC call therefore times out and ROLLS BACK wholesale: a second silent-failure mode for the same cron. - cleanup_expired_sandbox_users gets a function-local statement_timeout of 290s (same sanctioned pattern as undo_sie_import, 20260702154500) via migration 20260807150000. - The cron route bounds each night to 60 users (~180s), exports maxDuration = 300, and the backlog drains over a few nights. - Tests: route asserts the bounded rpc call and the maxDuration budget; the pg suite pins proconfig containing statement_timeout=290s. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com> * fix(db): index journal_entry_lines.account_id (chart-account cascade seq-scans 730k rows) Caught live during the backlog purge: DELETE FROM auth.users cascades chart_of_accounts deletion, whose ON DELETE SET NULL fires an unindexed UPDATE over journal_entry_lines per account (~37 per sandbox company). This is the bulk of the ~3s per-user teardown cost. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com> --------- Co-authored-by: Claude Fable 5 <noreply@anthropic.com>
2026-08-07 14:37:28 +02:00
parent 7411a0171b
commit bd7a423b86
5 changed files with 142 additions and 1 deletions
@@ -27,13 +27,19 @@ vi.mock('@supabase/supabase-js', () => ({
  createClient: vi.fn(() => ({ rpc: h.rpc })),
 }))

-import { GET } from '../route'
+import { GET, maxDuration } from '../route'

 function cronRequest(): Request {
  return new Request('http://localhost:3000/api/sandbox/cleanup/cron')
 }

 describe('GET /api/sandbox/cleanup/cron', () => {
+  it('reserves enough function time for a full batch', () => {
+    // 60 users at ~3s each must fit inside the route budget and the RPC's
+    // 290s statement_timeout (migration 20260807150000).
+    expect(maxDuration).toBe(300)
+  })
+
  beforeEach(() => {
    vi.clearAllMocks()
    process.env.NEXT_PUBLIC_SUPABASE_URL = 'https://example.supabase.co'
@@ -51,6 +57,7 @@ describe('GET /api/sandbox/cleanup/cron', () => {

    expect(h.rpc).toHaveBeenCalledWith('cleanup_expired_sandbox_users', {
      p_max_age_hours: 24,
+      p_limit: 60,
    })
    expect(res.status).toBe(200)
    expect(body).toEqual({ success: true, cleaned: 3, failed: 0, orphans_removed: 2 })
@@ -6,7 +6,18 @@ import { errorResponse, errorResponseFromCode } from '@/lib/errors/get-structure
 /**
 * GET /api/sandbox/cleanup/cron: daily 04:00 UTC.
 * Removes expired sandbox users (>24h old).
+ *
+ * One teardown costs ~3s on prod (the auth.users delete fans out over ~250
+ * FK triggers), so the run is bounded: BATCH_LIMIT users per night, sized to
+ * finish inside both the RPC's 290s statement_timeout (migration
+ * 20260807150000) and this route's maxDuration. The nightly intake is a
+ * fraction of this; a backlog drains over a few nights instead of timing
+ * out and rolling back wholesale.
 */
+export const maxDuration = 300
+
+const BATCH_LIMIT = 60
+
 export const GET = withCronContext('cron.sandbox_cleanup', async (_request, ctx) => {
  const supabaseUrl = process.env.NEXT_PUBLIC_SUPABASE_URL
  const supabaseServiceKey = process.env.SUPABASE_SERVICE_ROLE_KEY
@@ -22,6 +33,7 @@ export const GET = withCronContext('cron.sandbox_cleanup', async (_request, ctx)

  const { data, error } = await supabase.rpc('cleanup_expired_sandbox_users', {
    p_max_age_hours: 24,
+    p_limit: BATCH_LIMIT,
  })

  if (error) {
@@ -0,0 +1,96 @@
+-- The nightly sandbox cleanup cannot finish inside PostgREST's session cap.
+--
+-- All PostgREST sessions log in as authenticator, which carries
+-- statement_timeout = 8s at the role level, and profiling the repaired
+-- teardown (migration 20260807130000) on prod puts one sandbox user at
+-- ~3s: the auth.users delete fans out over ~250 FK triggers. FK indexes do
+-- not help (verified by replaying with them inside an aborted transaction);
+-- the cost is the trigger fan-out itself. One nightly RPC call therefore
+-- cleans at most two users before the whole batch times out and ROLLS BACK,
+-- which is a second silent-failure mode for the cron this migration chain
+-- exists to fix.
+--
+-- Fix mirrors undo_sie_import (20260702154500, pinned by
+-- sie-import.replace.pg.test.ts): a function-local statement_timeout raises
+-- the cap for this statement only. CREATE OR REPLACE resets proconfig, so
+-- the timeout must be restated if this function is ever replaced again.
+-- The cron route passes p_limit sized so a run finishes inside both this
+-- 290s cap and the route's maxDuration.
+
+DROP FUNCTION IF EXISTS public.cleanup_expired_sandbox_users(int, int);
+
+CREATE FUNCTION public.cleanup_expired_sandbox_users(
+  p_max_age_hours int DEFAULT 24,
+  p_limit int DEFAULT NULL
+)
+RETURNS jsonb
+LANGUAGE plpgsql
+SECURITY DEFINER
+SET search_path = public
+SET statement_timeout TO '290s'
+AS $$
+DECLARE
+  v_user_id uuid;
+  v_cleaned integer := 0;
+  v_failed integer := 0;
+  v_orphans integer := 0;
+BEGIN
+  FOR v_user_id IN
+    SELECT cs.user_id
+    FROM public.company_settings cs
+    WHERE cs.is_sandbox = true
+      AND cs.created_at < now() - interval '1 hour' * p_max_age_hours
+    ORDER BY cs.created_at
+    LIMIT p_limit
+  LOOP
+    BEGIN
+      PERFORM public.cleanup_sandbox_user(v_user_id);
+      v_cleaned := v_cleaned + 1;
+    EXCEPTION WHEN OTHERS THEN
+      v_failed := v_failed + 1;
+      RAISE WARNING 'Failed to clean up sandbox user %: %', v_user_id, SQLERRM;
+    END;
+  END LOOP;
+
+  IF EXISTS (
+    SELECT 1 FROM information_schema.columns
+    WHERE table_schema = 'auth' AND table_name = 'users'
+      AND column_name = 'is_anonymous'
+  ) THEN
+    FOR v_user_id IN
+      SELECT u.id
+      FROM auth.users u
+      WHERE u.is_anonymous = true
+        AND u.created_at < now() - interval '1 hour' * p_max_age_hours
+        AND NOT EXISTS (
+          SELECT 1 FROM public.company_settings cs WHERE cs.user_id = u.id
+        )
+        AND NOT EXISTS (
+          SELECT 1 FROM public.companies c WHERE c.created_by = u.id
+        )
+        AND NOT EXISTS (
+          SELECT 1 FROM public.company_members cm WHERE cm.user_id = u.id
+        )
+      ORDER BY u.created_at
+      LIMIT p_limit
+    LOOP
+      BEGIN
+        DELETE FROM auth.users WHERE id = v_user_id;
+        v_orphans := v_orphans + 1;
+      EXCEPTION WHEN OTHERS THEN
+        v_failed := v_failed + 1;
+        RAISE WARNING 'Failed to clean up orphaned anonymous user %: %', v_user_id, SQLERRM;
+      END;
+    END LOOP;
+  END IF;
+
+  RETURN jsonb_build_object(
+    'cleaned', v_cleaned,
+    'failed', v_failed,
+    'orphans_removed', v_orphans
+  );
+END;
+$$;
+
+REVOKE ALL ON FUNCTION public.cleanup_expired_sandbox_users(int, int) FROM PUBLIC, anon, authenticated;
+GRANT EXECUTE ON FUNCTION public.cleanup_expired_sandbox_users(int, int) TO service_role;
@@ -0,0 +1,11 @@
+-- journal_entry_lines.account_id (FK to chart_of_accounts, ON DELETE SET
+-- NULL) has no index, while account_number, journal_entry_id, cost_center,
+-- project and the dimension bags all do. Deleting a chart account therefore
+-- seq-scans all ~730k journal_entry_lines rows per account; sandbox teardown
+-- deletes ~37 chart accounts per company, which is where most of its ~3s
+-- per-user cost goes (caught live: the backlog purge timed out inside
+-- "UPDATE ONLY journal_entry_lines SET account_id = NULL WHERE account_id =
+-- $1" cascading from DELETE FROM auth.users).
+
+CREATE INDEX IF NOT EXISTS idx_journal_entry_lines_account_id
+  ON public.journal_entry_lines (account_id);
@@ -352,6 +352,21 @@ describe('sandbox cleanup RPCs (pg)', () => {
    }
  })

+  it('cleanup_expired_sandbox_users carries its own statement_timeout', async () => {
+    // PostgREST sessions inherit authenticator's 8s statement_timeout while
+    // one sandbox teardown costs ~3s, so without a function-local override
+    // the nightly batch times out and rolls back wholesale (migration
+    // 20260807150000, same pattern as undo_sie_import).
+    const { rows } = await getPool().query<{ proconfig: string[] | null }>(
+      `SELECT p.proconfig
+       FROM pg_proc p
+       JOIN pg_namespace n ON n.oid = p.pronamespace
+       WHERE n.nspname = 'public' AND p.proname = 'cleanup_expired_sandbox_users'`,
+    )
+    expect(rows).toHaveLength(1)
+    expect(rows[0]!.proconfig ?? []).toContain('statement_timeout=290s')
+  })
+
  it('is executable by service_role only', async () => {
    const { rows } = await getPool().query<{
      svc_user: boolean