The commit dispatcher claims an op with an atomic pending -> committing
CAS; if the process dies after side-effects post but before the terminal
committed write (or that write fails, the PR #841 log line), the row sat
in status='committing' forever: the expire cron only sweeps 'pending'.
Add lib/pending-operations/recover-stuck-committing.ts, invoked from the
existing daily expire cron (no new vercel.json entry):
- Only rows whose updated_at (the claim timestamp: the CAS bumps it via
the update_updated_at_column trigger) is older than 15 minutes, well
past the 300s Vercel function ceiling, so in-flight executors are
never raced.
- Positive evidence that side-effects posted finalizes the row to
committed with result_data.recovered=true. Evidence exists only where
params identify a target with an unambiguous posted state:
categorize_transaction (is_transaction_booked RPC, skipped for
allow_duplicate), link_transaction_journal_entry (exact tx+entry
link), match_transaction_invoice (invoice_payments pair row).
- No evidence: terminal rejected with an explanatory result_data,
never back to pending (re-execution could duplicate side-effects
that posted without a trace). Reason 'stuck_committing' is distinct
from 'expired' so the UI badge never claims these rows.
- Every terminal write is CAS-guarded on status='committing'; probe
errors skip the row for the next run.
- One structured 'pending_op_recovery' warn per row (count by outcome);
runbook comment added next to the #841 finalize-failure log line.
Tests: unit coverage for the decision logic and cron wiring (401, sweep
invoked, failure isolation), plus a pg-real test proving row selection,
the trustworthy updated_at anchor, committing -> terminal transitions
through the real immutability/input-frozen triggers, and the
is_transaction_booked evidence substrate.
Fixes#843
Co-authored-by: Claude Fable 5 <noreply@anthropic.com>
Jakob Wennberg
25a7261eda