feat(webhooks): dispatch on emit instead of waiting for the next cron tick (#1256)
* feat(webhooks): dispatch on emit instead of waiting for the next cron tick The webhook dispatcher ran only on a per-minute cron, so the floor on delivery latency was up to 60 seconds plus the request. An external consumer that wanted to react as a transaction landed had only one alternative: polling /api/events, which the 100 rpm per-key limit makes expensive and which still cannot beat the tick interval. Schedules one dispatch cycle as soon as deliveries are enqueued. The cron is unchanged and remains the retry and sweep path; this only moves the first attempt forward. Wired into the event-bus fanout plus the two routes that enqueue a delivery directly: the :test verb, whose entire purpose is telling someone whether their receiver works, and the manual delivery retry. Three properties are load-bearing and covered by tests. The kick is never awaited, because eventBus.emit is awaited at ~99 call sites including journal_entry.committed and each delivery can burn a 10 s receiver timeout. It coalesces per function instance, so a bulk booking that emits once per row does not schedule one claim round trip per row. It claims 5 rows rather than the cron's 50, because it runs on the tail of a user-facing request. Double delivery is not a risk: claim_due_webhook_deliveries already claims FOR UPDATE SKIP LOCKED and flips rows to in_flight in the same statement, so a kick racing the cron sees disjoint rows. Does not close #1201, which asks for a realtime stream for API consumers. This is the cheap half. Refs #1201 Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com> * docs(webhooks): stop claiming the kick makes double delivery impossible Adversarial review of the previous commit caught an overstatement in its own comments. SKIP LOCKED keeps a kick and the cron from claiming the same row at the same moment, but claim_due_webhook_deliveries autocommits before any POST is issued, so from then on ownership is only status='in_flight' and a later cycle's recoverStuckInFlight sweep can re-arm a row still queued behind an earlier cycle's serial loop. Delivery is at-least-once, which is what the public docs already tell receivers ("the same delivery id may arrive more than once ... idempotency is on you"). The comments contradicted that. No behaviour change. The kick does not create this window: the cron claims 50 rows serially against the same 20 s stuck threshold, which is wider than what a batch of 5 can open. Refs #1201 Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com> --------- Co-authored-by: Claude Opus 5 (1M context) <noreply@anthropic.com>
This commit is contained in:
co-authored by
Claude Opus 5
parent
e171bffa97
commit
49ff234954
@@ -55,7 +55,7 @@ The first stable release of the public REST API. Six phases of development cover
|
||||
### 2026-05-15 — Webhooks (Phase 6 PR-1)
|
||||
|
||||
- **Subscriptions**: \`POST /webhooks\` (HMAC secret returned exactly once), GET list + detail, PATCH, DELETE. Per-event-type elevated scope check (\`salary_run.*\` and \`agi.generated\` require \`payroll:read\`).
|
||||
- **Delivery substrate**: per-minute Vercel cron at \`/api/webhooks/dispatch/cron\`. Due rows are claimed atomically via the \`claim_due_webhook_deliveries\` SQL function (\`FOR UPDATE SKIP LOCKED\`), with \`*.pg.test.ts\` coverage for the claim path and the webhook DB triggers. Exponential backoff \`1m / 5m / 30m / 2h / 12h / 24h / 48h\` (7 retries, ~87h total). HTTP 410 from receiver auto-disables the webhook.
|
||||
- **Delivery substrate**: dispatched immediately after the event is enqueued, with a per-minute Vercel cron at \`/api/webhooks/dispatch/cron\` as the retry and sweep path. Due rows are claimed atomically via the \`claim_due_webhook_deliveries\` SQL function (\`FOR UPDATE SKIP LOCKED\`), with \`*.pg.test.ts\` coverage for the claim path and the webhook DB triggers. Exponential backoff \`1m / 5m / 30m / 2h / 12h / 24h / 48h\` (7 retries, ~87h total). HTTP 410 from receiver auto-disables the webhook.
|
||||
- **Signature**: \`X-Gnubok-Signature: t=<unix>,v1=<hex-HMAC-SHA256>\`. Stripe-format. Sample receivers in [Node + Python](/docs/api/webhooks#verifying-signatures).
|
||||
- **SSRF protection**: webhook_url must be HTTPS; resolved IPs in private/loopback/link-local/CGNAT/cloud-metadata ranges are rejected at create AND dispatch time. Dispatch pins the validated IP through a DNS-rebinding-safe \`node:https.request\` agent (\`lib/webhooks/pinned-fetch.ts\`); redirects are refused on every outbound POST (any 3xx is treated as a blocked redirect).
|
||||
- **Audit + retention**: webhook delivery rows are *behandlingshistorik* per BFNAR 2013:2 kap 8 §: immutable once terminal so the audit trail of what an integration was notified of stays intact. Delivery rows are NOT räkenskapsinformation themselves; the 7-year statutory retention under BFL 7 kap 1 § applies only to the underlying verifikation / faktura / AGI XML in its own table, NOT to the delivery envelope. Accounted keeps accounting-event delivery rows for 7 years as a voluntary operational policy (the duration aligns with BFL 7 kap on the underlying records but is not itself a statutory obligation on delivery rows). Webhook DELETE preserves the delivery audit trail (\`ON DELETE SET NULL\` on \`webhook_id\`). Webhook lifecycle events (create / update / delete, plus dispatcher auto-disable) each write a V16 \`audit_log\` entry.
|
||||
|
||||
@@ -8,7 +8,7 @@ If you've used [Stripe webhooks](https://docs.stripe.com/webhooks), the model is
|
||||
|
||||
1. **Register a receiver** with [\`POST /api/v1/companies/{companyId}/webhooks\`](/docs/api/reference/webhooks#post-webhooks-create). The response includes an HMAC signing secret returned **exactly once**: store it on the receiver side immediately. If you lose it, rotate it with [\`POST /api/v1/companies/{companyId}/webhooks/{webhookId}/rotate-secret\`](/docs/api/reference/webhooks#post-webhooks-rotate_secret): a fresh secret is issued in place and the old one is invalidated immediately, with no change to the webhook's id or delivery history.
|
||||
2. **Accounted emits events** internally (e.g. an invoice is marked paid via the dashboard or another API call). The webhook handler enqueues a delivery row.
|
||||
3. **The dispatcher cron runs every minute**, signs the payload with HMAC-SHA256, and POSTs to your URL with a 10-second timeout.
|
||||
3. **The dispatcher runs immediately after the event**, signs the payload with HMAC-SHA256, and POSTs to your URL with a 10-second timeout. A cron sweep every minute picks up anything the immediate pass did not get to, so first-attempt latency is normally a second or two but is never guaranteed to be: treat delivery as prompt, not synchronous.
|
||||
4. **Your receiver verifies the signature**, processes the event idempotently, and returns 2xx.
|
||||
5. **Failed deliveries retry** at \`1m / 5m / 30m / 2h / 12h / 24h / 48h\` (7 retries, ~87 hours total, about 3.6 days). After all attempts the delivery is marked \`dead\`. HTTP 410 from your receiver short-circuits to \`dead\` immediately and **auto-disables** the webhook.
|
||||
|
||||
@@ -219,7 +219,7 @@ Use [\`GET /api/v1/companies/{companyId}/webhooks/{webhookId}/deliveries\`](/doc
|
||||
|
||||
To replay a \`dead\` or \`delivered\` delivery, call [\`POST /api/v1/webhook-deliveries/{deliveryId}/retry\`](/docs/api/reference/webhooks#post-webhook_deliveries-retry). The retry creates a fresh delivery row pointing at the same payload: the original audit row stays in place. Receivers must be idempotent on the \`X-Gnubok-Delivery\` header.
|
||||
|
||||
To send a synthetic test event without driving real state, call [\`POST /api/v1/companies/{companyId}/webhooks/{webhookId}/test\`](/docs/api/reference/webhooks#post-webhooks-test). The dispatcher delivers a \`webhook.test\` event with a static payload on the next per-minute tick.
|
||||
To send a synthetic test event without driving real state, call [\`POST /api/v1/companies/{companyId}/webhooks/{webhookId}/test\`](/docs/api/reference/webhooks#post-webhooks-test). The dispatcher delivers a \`webhook.test\` event with a static payload immediately, so the outcome is normally visible within a second or two.
|
||||
|
||||
## Auto-disable behaviour
|
||||
|
||||
|
||||
Reference in New Issue
Block a user