fea5dfd1f912ec1111cec662cd8362f9aa55dfd9
3
Commits
| Author | SHA1 | Message | Date | |
|---|---|---|---|---|
 MattssonandClaude Fable 5
|
526f0315d0 |
fix(sandbox): loop small cleanup batches (8s cap is real), clear last FK blockers (#1452)
* fix(sandbox): loop small cleanup batches (8s cap is real), clear last FK blockers Draining the prod backlog exposed two final issues: - The function-level statement_timeout shipped in 20260807150000 does NOT lift authenticator's 8s cap: the timer arms when the top-level statement starts (verified empirically on prod: SET LOCAL 2s canceled the RPC despite its 290s proconfig; matches the 2026-08-04 SIE-import finding). The route now loops batches of 10 (~220ms/user with the account_id index, so ~2.2s per batch), each rpc() call being its own statement with its own 8s window. The loop stops when a batch makes no progress or the 240s time budget nears; capacity is 250 users/night. - processing_history.company_id and invoice_deliveries.company_id are plain NO ACTION FKs, so sandboxes whose visitor produced AI telemetry or sent a demo invoice could never be deleted (7 of ~510 backlog users). A data-driven sweep of every NO ACTION FK into companies confirms these two plus the already-handled audit_log are the only such tables with sandbox rows. cleanup_sandbox_user (migration 20260807160000) deletes them explicitly; the pg fixture now seeds a processing_history row. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com> * fix(test): use valid processing_history aggregate_type/event_type in sandbox fixture aggregate_type is CHECK-constrained and event_type is an FK to the seeded processing_event_types lookup; the guessed values failed all five fixture-dependent pg tests in CI. Validated against staging: Document/DocumentIngested inserts and tears down cleanly. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com> * fix(sandbox): bypass invoice-delivery delete guard in teardown, cover both blocker tables in pg fixture CodeRabbit's fixture ask exposed a real gap: enforce_invoice_delivery_immutability silently swallows DELETEs (RETURN NULL plus a SECURITY_EVENT audit row) for terminal rows, so the explicit invoice_deliveries delete was a no-op and the companies FK still blocked teardown for sandboxes that sent a demo invoice. The trigger's DELETE branch now honors the gnubok.sandbox_cleanup flag with the same per-row sandbox re-verification as every other guard; base definition 20260803224000, all other branches untouched. The pg fixture seeds an invoice plus a marked_sent manual delivery, and a new test pins the zero-settings refusal path the Swedish review asked about. Validated on staging end-to-end. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com> --------- Co-authored-by: Claude Fable 5 <noreply@anthropic.com> |
||
|
MattssonandClaude Fable 5
|
bd7a423b86 |
fix(sandbox): fit the nightly cleanup inside PostgREST's 8s session cap (#1451)
* fix(sandbox): fit the nightly cleanup inside PostgREST's 8s session cap Follow-up to #1449. Profiling the repaired teardown on prod puts one sandbox user at ~3s (the auth.users delete fans out over ~250 FK triggers; FK indexes were tried inside an aborted transaction and do not help), while every PostgREST session inherits authenticator's statement_timeout = 8s. The nightly RPC call therefore times out and ROLLS BACK wholesale: a second silent-failure mode for the same cron. - cleanup_expired_sandbox_users gets a function-local statement_timeout of 290s (same sanctioned pattern as undo_sie_import, 20260702154500) via migration 20260807150000. - The cron route bounds each night to 60 users (~180s), exports maxDuration = 300, and the backlog drains over a few nights. - Tests: route asserts the bounded rpc call and the maxDuration budget; the pg suite pins proconfig containing statement_timeout=290s. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com> * fix(db): index journal_entry_lines.account_id (chart-account cascade seq-scans 730k rows) Caught live during the backlog purge: DELETE FROM auth.users cascades chart_of_accounts deletion, whose ON DELETE SET NULL fires an unindexed UPDATE over journal_entry_lines per account (~37 per sandbox company). This is the bulk of the ~3s per-user teardown cost. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com> --------- Co-authored-by: Claude Fable 5 <noreply@anthropic.com> |
||
|
MattssonandClaude Fable 5
|
f7f3a31f8e |
fix(sandbox): repair the silently-failing nightly sandbox cleanup and lock down its RPCs (#1449)
* fix(sandbox): repair the silently-failing nightly sandbox cleanup and lock down its RPCs
The daily cleanup cron has deleted nothing for months: cleanup_sandbox_user
died on the journal-line immutability trigger for every user (the seed posts
vouchers since spring), and cleanup_expired_sandbox_users swallowed each
failure as a WARNING while reporting success. 658 expired sandbox users plus
21 orphaned anonymous users had accumulated in prod auth.users.
- cleanup_sandbox_user sets the sanctioned gnubok.allow_delete flag plus a
new transaction-local gnubok.sandbox_cleanup flag, only after verifying
is_sandbox; write_audit_log, audit_log_immutable (DELETE only, per-row
sandbox re-check), enforce_dimension_registry_guards (DELETE only) and
enforce_pending_operations_no_delete (DELETE only) respect it
- clears salary_runs voucher-link FKs and purges the sandbox company's
audit rows before the auth.users cascade
- cleanup_expired_sandbox_users returns {cleaned, failed, orphans_removed},
additionally sweeps expired anonymous users that never got a
company_settings row, and takes an optional p_limit for bounded batches;
the cron route logs failures at error level and accepts both return shapes
- both RPCs lose their default PUBLIC EXECUTE grant (anon and authenticated
could call them via PostgREST) and are now service_role-only
- validated by replaying the full delete chain against prod inside aborted
transactions (21 users sampled across all seed eras, zero failures) and a
committed staging run; pg-real suite + cron route unit tests added
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
* fix(sandbox): per-row sandbox re-verification in teardown guards, is_anonymous column guard
Resolution pass for PR #1449 review findings and the pg-real CI failure:
- Swedish accounting review: enforce_dimension_registry_guards and
enforce_pending_operations_no_delete now re-verify per row that
OLD.company_id belongs to a sandbox company (same pattern as
audit_log_immutable) instead of trusting the gnubok.sandbox_cleanup flag
alone. Because that re-check needs company_settings to still exist,
cleanup_sandbox_user deletes pending_operations and dimensions explicitly
before the auth.users cascade.
- pg-real CI: auth.users.is_anonymous does not exist in the CI
supabase/postgres image (or on older self-hosted stacks); the orphan sweep
in cleanup_expired_sandbox_users is now guarded on the column's existence,
and the pg test skips the orphan assertions on such stacks.
Re-validated on staging end-to-end: {cleaned: 5, failed: 0,
orphans_removed: 1}, fresh users and non-sandbox rows untouched.
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
* fix(sandbox): make company_settings.is_sandbox write-once, prove orphan sweep fails loudly
Round-2 review findings (Swedish accounting review on PR #1449):
- Every teardown bypass trusts company_settings.is_sandbox, and RLS lets an
owner update their own settings row via PostgREST, so a real company that
flipped the flag would become eligible for full deletion by the nightly
cron. New trigger makes the flag write-once (no application path updates
it; a future sandbox-to-real conversion would ship its own migration).
- New pg test pins the reviewer's remaining concern: an anonymous user who
somehow has bookkeeping but no company_settings row is NOT silently
deleted by the orphan sweep; the unbypasseed immutability triggers make
the deletion fail loudly into the summary's failed count.
Validated on staging: flip blocked in both directions, unrelated
company_settings updates unaffected.
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
* fix(sandbox): guard is_sandbox provenance at INSERT, make orphan sweep exclusions explicit
Round-3 review hardening, approved by Emil:
- is_sandbox = true can now only be created by an anonymous-user JWT (the
sandbox seed's actor), service_role, or a direct database session. A
regular authenticated user could previously insert their settings row
pre-flagged and have the nightly cron destroy their real books, which
BFL 7 kap. forbids even self-inflicted. Claims are read from the
request.jwt.* GUCs directly so the check behaves identically on hosted,
self-hosted, and the CI auth shim.
- The orphan sweep now explicitly excludes anonymous users attached to any
companies or company_members row, instead of relying on downstream
immutability triggers throwing (emergent safety) to protect half-seeded
users.
- pg tests updated accordingly: blocked/allowed provenance paths, and the
half-seeded user is proven unreachable rather than merely failing loudly.
Validated on staging: authed insert blocked, anonymous-claim insert
allowed, half-seeded user untouched, sweep summary failed=0.
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
* fix(sandbox): all-rows sandbox check, cleared bypass flags, tighter insert guard
CodeRabbit review pass on PR #1449 (its first non-rate-limited run):
- cleanup_sandbox_user now requires EVERY company_settings row of the user
to be sandbox-flagged, not an arbitrary single row: a hypothetical
mixed-company user would otherwise have their real company's rows reached
by the user-scoped deletes.
- Both bypass flags are cleared before the RPC returns, so later work in
the same transaction (the expired loop's next iterations, the orphan
sweep) never runs with them still armed.
- The is_sandbox insert guard now treats ANY PostgREST claims context
(claims json without a role claim included) as guarded, instead of
falling open when the role claim is absent.
- The flag-leak pg test now runs inside an explicit transaction (the old
version could not observe transaction-local GUCs at all), and a new test
covers the mixed sandbox/real user refusal.
Declined: replacing the em dashes inside the two replicated Swedish
exception messages; they are byte-identical copies of the strings already
deployed by migration 20260702084500 and changing them would alter live
user-facing errors out of scope.
Validated on staging: mixed user refused, flags cleared post-teardown,
role-less claims blocked.
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
---------
Co-authored-by: Claude Fable 5 <noreply@anthropic.com>
|