* feat(webhooks): dispatch on emit instead of waiting for the next cron tick The webhook dispatcher ran only on a per-minute cron, so the floor on delivery latency was up to 60 seconds plus the request. An external consumer that wanted to react as a transaction landed had only one alternative: polling /api/events, which the 100 rpm per-key limit makes expensive and which still cannot beat the tick interval. Schedules one dispatch cycle as soon as deliveries are enqueued. The cron is unchanged and remains the retry and sweep path; this only moves the first attempt forward. Wired into the event-bus fanout plus the two routes that enqueue a delivery directly: the :test verb, whose entire purpose is telling someone whether their receiver works, and the manual delivery retry. Three properties are load-bearing and covered by tests. The kick is never awaited, because eventBus.emit is awaited at ~99 call sites including journal_entry.committed and each delivery can burn a 10 s receiver timeout. It coalesces per function instance, so a bulk booking that emits once per row does not schedule one claim round trip per row. It claims 5 rows rather than the cron's 50, because it runs on the tail of a user-facing request. Double delivery is not a risk: claim_due_webhook_deliveries already claims FOR UPDATE SKIP LOCKED and flips rows to in_flight in the same statement, so a kick racing the cron sees disjoint rows. Does not close #1201, which asks for a realtime stream for API consumers. This is the cheap half. Refs #1201 Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com> * docs(webhooks): stop claiming the kick makes double delivery impossible Adversarial review of the previous commit caught an overstatement in its own comments. SKIP LOCKED keeps a kick and the cron from claiming the same row at the same moment, but claim_due_webhook_deliveries autocommits before any POST is issued, so from then on ownership is only status='in_flight' and a later cycle's recoverStuckInFlight sweep can re-arm a row still queued behind an earlier cycle's serial loop. Delivery is at-least-once, which is what the public docs already tell receivers ("the same delivery id may arrive more than once ... idempotency is on you"). The comments contradicted that. No behaviour change. The kick does not create this window: the cron claims 50 rows serially against the same 20 s stuck threshold, which is wider than what a batch of 5 can open. Refs #1201 Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com> --------- Co-authored-by: Claude Opus 5 (1M context) <noreply@anthropic.com>
228 lines
9.7 KiB
TypeScript
228 lines
9.7 KiB
TypeScript
/**
|
|
* /api/v1/webhook-deliveries/{id}/retry: POST :retry verb.
|
|
*
|
|
* Re-enqueues a `dead` delivery by INSERTing a fresh row pointing at the
|
|
* same payload, NOT by mutating the dead row in place (the immutability
|
|
* trigger blocks that). The new row enters `pending` and the dispatcher
|
|
* picks it up at next-minute boundary.
|
|
*
|
|
* Live (`pending` / `in_flight` / `failed`) deliveries cannot be retried
|
|
* via this endpoint: the dispatcher already retries failed ones, and the
|
|
* other states aren't terminal. Only `dead` (and `delivered`, for callers
|
|
* that explicitly want to redeliver a message) qualify.
|
|
*
|
|
* The route lives outside the /companies/{companyId}/ tree because callers
|
|
* referencing a delivery already have its id; nesting under company would
|
|
* force the receiver-debugging UI to round-trip company resolution from
|
|
* the delivery id. Tenancy is still enforced: the wrapper resolves the
|
|
* delivery's company_id via the row and verifies caller membership.
|
|
*/
|
|
|
|
import { z } from 'zod'
|
|
import { ok } from '@/lib/api/v1/response'
|
|
import { registerEndpoint, dataEnvelope } from '@/lib/api/v1/registry'
|
|
import { withApiV1 } from '@/lib/api/v1/with-api-v1'
|
|
import { v1ErrorResponse, v1ErrorResponseFromCode } from '@/lib/api/v1/errors'
|
|
import { kickWebhookDispatch } from '@/lib/webhooks/dispatch-kick'
|
|
import { minimisePayload } from '@/lib/webhooks/handler'
|
|
import { validateWebhookUrl } from '@/lib/webhooks/url-guard'
|
|
import { hasScope } from '@/lib/auth/api-keys'
|
|
|
|
registerEndpoint({
|
|
operation: 'webhook_deliveries.retry',
|
|
method: 'POST',
|
|
path: '/api/v1/webhook-deliveries/:id/retry',
|
|
summary: 'Retry a webhook delivery.',
|
|
description:
|
|
'Re-enqueues a dead (or delivered) delivery as a fresh pending row. The new delivery references the same webhook + payload; the dispatcher picks it up at the next per-minute cron tick. The original row is preserved in the audit log.',
|
|
useWhen:
|
|
'After a receiver outage you want to replay deliveries that died, or after fixing a receiver-side bug you want to redeliver a successful one.',
|
|
doNotUseFor:
|
|
'Retrying live deliveries (pending / in_flight / failed): the dispatcher is already managing them.',
|
|
pitfalls: [
|
|
'Retrying a delivered delivery causes the receiver to see the event twice. Receivers MUST be idempotent (check the X-Gnubok-Delivery header).',
|
|
],
|
|
example: {
|
|
response: {
|
|
data: {
|
|
webhook_delivery_id: 'wh_dlv_NEW',
|
|
status: 'pending',
|
|
},
|
|
meta: { request_id: 'req_…', api_version: '2026-05-12' },
|
|
},
|
|
},
|
|
// Special-case scope: this endpoint lives outside the companies/ tree
|
|
// but still belongs to the webhooks domain. Add to lib/auth/scopes.ts in
|
|
// the same commit as the v1 wiring.
|
|
scope: 'webhooks:manage',
|
|
risk: 'medium',
|
|
idempotent: false,
|
|
reversible: false,
|
|
dryRunSupported: false,
|
|
response: {
|
|
success: dataEnvelope(z.object({
|
|
webhook_delivery_id: z.string().uuid(),
|
|
status: z.literal('pending'),
|
|
})),
|
|
},
|
|
})
|
|
|
|
export const POST = withApiV1<{ params: Promise<{ id: string }> }>(
|
|
'webhook_deliveries.retry',
|
|
async (_request, ctx, params) => {
|
|
const { id } = await params.params
|
|
|
|
// Fetch the original delivery and its company to enforce tenancy.
|
|
const { data: original, error: lookupErr } = await ctx.supabase
|
|
.from('webhook_deliveries')
|
|
.select('id, webhook_id, company_id, event_type, payload, previous_attributes, api_version, status')
|
|
.eq('id', id)
|
|
.maybeSingle()
|
|
|
|
if (lookupErr) return v1ErrorResponse(lookupErr, ctx.log, { requestId: ctx.requestId })
|
|
if (!original) return v1ErrorResponseFromCode('NOT_FOUND', ctx.log, { requestId: ctx.requestId })
|
|
|
|
type O = {
|
|
id: string
|
|
webhook_id: string
|
|
company_id: string
|
|
event_type: string
|
|
payload: Record<string, unknown>
|
|
previous_attributes: Record<string, unknown> | null
|
|
api_version: string
|
|
status: 'pending' | 'in_flight' | 'delivered' | 'failed' | 'dead'
|
|
}
|
|
const o = original as O
|
|
|
|
// Tenancy check: the wrapper does not have a companyId from the URL
|
|
// here (deliberate; see file header). Verify the caller is a member of
|
|
// the delivery's company.
|
|
const { data: membership, error: membershipErr } = await ctx.supabase
|
|
.from('company_members')
|
|
.select('company_id')
|
|
.eq('user_id', ctx.userId)
|
|
.eq('company_id', o.company_id)
|
|
.maybeSingle()
|
|
|
|
if (membershipErr) return v1ErrorResponse(membershipErr, ctx.log, { requestId: ctx.requestId })
|
|
if (!membership) {
|
|
// 404 (not 403) so we don't leak existence of the delivery to a
|
|
// non-member; matches the wrapper's standard pattern.
|
|
return v1ErrorResponseFromCode('NOT_FOUND', ctx.log, { requestId: ctx.requestId })
|
|
}
|
|
|
|
if (o.status !== 'dead' && o.status !== 'delivered') {
|
|
return v1ErrorResponseFromCode('VALIDATION_ERROR', ctx.log, {
|
|
requestId: ctx.requestId,
|
|
details: {
|
|
field: 'status',
|
|
message: `Only dead or delivered deliveries can be retried (current: ${o.status}).`,
|
|
},
|
|
})
|
|
}
|
|
|
|
// Mirror the create-route elevated-scope gate. A key with only
|
|
// webhooks:manage must NOT be able to re-emit a salary_run.* / agi.*
|
|
// payload: those carry personnummer, lönesummor, skatteavdrag, and
|
|
// the original create call required webhooks:manage AND payroll:read.
|
|
// Retry checks the SAME pair against the CALLING key's scopes (which
|
|
// may differ from the key that created the webhook in the first place).
|
|
const PAYROLL_SENSITIVE = /^(salary_run\.|agi\.)/
|
|
if (PAYROLL_SENSITIVE.test(o.event_type) && !hasScope(ctx.scopes, 'payroll:read')) {
|
|
return v1ErrorResponseFromCode('INSUFFICIENT_SCOPE', ctx.log, {
|
|
requestId: ctx.requestId,
|
|
details: {
|
|
required_scope: 'payroll:read',
|
|
reason: `Retrying ${o.event_type} requires payroll:read in addition to webhooks:manage.`,
|
|
},
|
|
})
|
|
}
|
|
|
|
// Re-verify that the parent webhook still exists, still belongs to the
|
|
// delivery's company, and is still active immediately before INSERT.
|
|
// Closes the TOCTOU window between the membership check above and the
|
|
// INSERT: without this a webhook deleted in between would have its
|
|
// retry land in webhook_deliveries with a now-dangling webhook_id, and
|
|
// a webhook re-registered to a different company in between would let
|
|
// the caller redeliver an event to a webhook they never created.
|
|
const { data: webhook, error: webhookErr } = await ctx.supabase
|
|
.from('webhooks')
|
|
.select('id, webhook_url, active, disabled_at')
|
|
.eq('id', o.webhook_id)
|
|
.eq('company_id', o.company_id)
|
|
.maybeSingle()
|
|
|
|
if (webhookErr) return v1ErrorResponse(webhookErr, ctx.log, { requestId: ctx.requestId })
|
|
if (!webhook) {
|
|
// The original webhook no longer exists or is no longer in this
|
|
// company. There's nothing to redeliver to. 404, not VALIDATION_ERROR:
|
|
// the resource the caller targeted is genuinely gone.
|
|
return v1ErrorResponseFromCode('NOT_FOUND', ctx.log, { requestId: ctx.requestId })
|
|
}
|
|
const w = webhook as { id: string; webhook_url: string; active: boolean; disabled_at: string | null }
|
|
if (!w.active || w.disabled_at) {
|
|
return v1ErrorResponseFromCode('VALIDATION_ERROR', ctx.log, {
|
|
requestId: ctx.requestId,
|
|
details: { field: 'webhook.active', message: 'Webhook is disabled: re-enable before retrying.' },
|
|
})
|
|
}
|
|
|
|
// Re-run the SSRF guard against the webhook's CURRENT url. The URL
|
|
// may have changed via PATCH between the original delivery and this
|
|
// retry call. The dispatch-time guard would catch a malicious URL
|
|
// eventually, but allowing the INSERT first means a poisoned row
|
|
// sits in the queue until the next cron tick. Validating here closes
|
|
// the window: the retry refuses up-front and the audit trail gets
|
|
// a clean VALIDATION_ERROR rather than a deferred dispatch-time
|
|
// 'dead' row with reason='url_unsafe'.
|
|
const urlCheck = await validateWebhookUrl(w.webhook_url)
|
|
if (!urlCheck.ok) {
|
|
return v1ErrorResponseFromCode('VALIDATION_ERROR', ctx.log, {
|
|
requestId: ctx.requestId,
|
|
details: { field: 'webhook.webhook_url', reason: urlCheck.reason, message: urlCheck.detail },
|
|
})
|
|
}
|
|
|
|
// Re-run the data-minimisation projection on the original payload
|
|
// before re-enqueueing. If the original delivery predates a
|
|
// minimisePayload tightening (e.g. a future projection drops more
|
|
// fields), the retry must not silently re-deliver the unminimised
|
|
// shape. Idempotent on already-minimised payloads.
|
|
const minimised = minimisePayload(o.payload)
|
|
|
|
const { data: replay, error: insertErr } = await ctx.supabase
|
|
.from('webhook_deliveries')
|
|
.insert({
|
|
webhook_id: o.webhook_id,
|
|
company_id: o.company_id,
|
|
event_type: o.event_type,
|
|
payload: minimised,
|
|
previous_attributes: o.previous_attributes,
|
|
api_version: o.api_version,
|
|
// Link the retry to the API request that triggered it. The
|
|
// original delivery's request_id is preserved on its own audit
|
|
// row; the retry gets a fresh correlation pointing at the
|
|
// :retry call.
|
|
request_id: ctx.requestId,
|
|
})
|
|
.select('id')
|
|
.single()
|
|
|
|
if (insertErr || !replay) {
|
|
return v1ErrorResponse(insertErr ?? new Error('insert returned no row'), ctx.log, {
|
|
requestId: ctx.requestId,
|
|
})
|
|
}
|
|
|
|
// Deliver now rather than on the next cron tick (#1201). A manual retry
|
|
// is someone watching a failed delivery and asking for it again; the
|
|
// cron stays the path for the automatic backoff retries.
|
|
kickWebhookDispatch()
|
|
|
|
return ok(
|
|
{ webhook_delivery_id: (replay as { id: string }).id, status: 'pending' as const },
|
|
{ requestId: ctx.requestId },
|
|
)
|
|
},
|
|
)
|