feat(connect): hosted connector-key registry + validate RPC + entitlements endpoint; instance sync writes connector grants hourly (#1748)

* feat(entitlements): partition the self-host bypass so connector capabilities fall through to grants; capability_grants.source accepts 'connector'

Sovereign plan WS3 PR3: ships dark, nothing changes for hosted.

- lib/entitlements/keys.ts: CONNECTOR_CAPABILITIES = bank_sync,
  skatteverket, org_lookup, migration (services Accounted operates that a
  self-hosted instance cannot provide itself) + isConnectorCapability().
  Separate from PAID_CAPABILITIES and outside the trial-seed trigger on
  purpose: a hosted company can never hold a connector grant.
- lib/entitlements/has-capability.ts: isPaywallBypassed() -> isBypassedFor(key).
  Hosted: byte-identical (dev / DISABLE_PAYWALL bypass, FORCE_PAYWALL wins,
  else the grant lookup). Self-host: local capabilities always on
  (FORCE_PAYWALL included, as the existing test demands); connector
  capabilities behave like hosted, i.e. dev bypass, FORCE_PAYWALL, else the
  grant lookup where the connector sync will write source='connector' rows.
  getCompanyEntitlements on a self-host: local paid keys + active connector
  keys, state 'paid' with an active connector grant else 'none' (never the
  hosted trial copy).
- Migration 20260820122000: capability_grants.source CHECK gains
  'connector', found through pg_constraint (the CHECK was declared inline
  and auto-named; Postgres stores IN as = ANY, matched accordingly).
  pg-real test: connector accepted, unknown source rejected, upsert on the
  (scope, key, source) identity, trial seed writes no connector rows.
- Tests: self-hosted connector matrix (local all-on without DB, connector
  gated by grant/expiry, dev bypass all-on, FORCE_PAYWALL gates connector
  keys only, bulk resolution, entitlements shape); two pre-existing tests
  that asserted the old "self-host holds connector keys" contract updated
  to the new one.

Verified: full unit suite green, pg-real suite for lib/entitlements green
against a local supabase/postgres with every migration applied, lint
ratchet, guards. Deferred to the instance-wiring PR: adding the connector
extensions to the self-host Docker preset (dead-end upsells until a key can
be issued).

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>

* refactor(entitlements): fold the self-host branch into the existing grants query

One .or(scopeFilter), not two: the duplicated helper pushed the
no-phantom-columns unresolvable-expression count to 380/379. Behaviour is
unchanged; the self-host matrix tests still pass.

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>

* feat(connect): hosted connector-key registry + validate RPC + entitlements endpoint; instance sync writes connector grants hourly

Sovereign plan WS3 PR4 ("key infra enabling manual sales"), stacked on the
entitlement partition (#1747). Nothing is purchasable yet; this is the
plumbing both ends need before the first manually issued key.

Hosted side:
- Migration 20260820123000: connector_keys (SHA-256 key_hash, prefix,
  org_number, pinned instance_url, scopes, status, Stripe ids,
  current_period_end, per-minute rate limit, active_company_count,
  last_seen/synced) and connector_usage_events (per-request metering,
  separate from metered_events whose company_id references hosted
  companies). RLS on, NO policies: service role only. RPC
  validate_and_increment_connector_key copies the api_keys pattern (FOR
  UPDATE, minute window, suspended reported not counted, revoked = no row)
  and is REVOKEd from PUBLIC/anon/authenticated, GRANTed to service_role.
  pg-real test covers validate/count, unknown+revoked, suspended, rate
  limit, execute privileges per role, RLS invisibility, usage cascade.
- lib/connect/contract.ts (shared wire types), lib/connect/hosted/keys.ts
  (generate/hash/validate -> 401/403/429 mapping),
  with-connector-auth.ts (Bearer or X-Connector-Key, one usage row per
  request, 500 envelope on handler throw), /api/connect/entitlements GET +
  POST (records active_company_count, pins instance_url on first report,
  never moves a pinned one), scripts/issue-connector-key.ts (dry run unless
  --confirm, prints the key once + the .env lines).

Instance side:
- lib/connect/instance/config.ts (GNUBOK_CONNECTOR_KEY, GNUBOK_CONNECT_URL
  default https://app.gnubok.se), sync.ts: reports the active company count
  and writes source='connector' grants for every company x covered scope,
  expires_at = min(now+72h, period_end+3d); 401/403 or a non-active status
  deletes them (freeze-and-retain); network/5xx/429 leave them alone.
  /api/connector/sync/cron (hourly) runs it; not_configured without a key.
- Crontab generator gains EXTRA_JOBS (variant-only jobs not in vercel.json,
  with reasons) + drift tests; docker/crontab.self-hosted regenerated with
  the hourly sync. Docs (SELF-HOSTING connector section, env templates),
  DECISIONS.

Tests: 52 new unit tests (keys, auth wrapper, route, config, sync outcomes
and grant arithmetic, cron route, crontab EXTRA_JOBS) + 7 pg-real tests
run locally against supabase/postgres with every migration applied.
no-phantom-columns ceiling +1 with a reason (the bulk grant upsert).

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>

* chore(connect): update pg test to re-versioned migration 20260831190000

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01UzNkSsR18pLFitJdYn8QEb

* fix(connect): RPC errors answer 503 not 401; X-Connector-Key wins over Authorization

A hosted DB error mapped to 401 made the instance sync treat a pooler
blip as key revocation and delete its entire connector grant cache,
zeroing the 72h offline grace. 503 lands in the sync's keep-grants
branch (already test-pinned). Bearer-first extraction hashed the
upstream token on dual-header proxied calls, 401ing the exact shape
X-Connector-Key exists for.

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01UzNkSsR18pLFitJdYn8QEb

* fix(connect): sync deletes grants only on a body-proven connector rejection, never bare 401/403

A WAF challenge page, edge deployment protection, or an egress proxy
answers 401/403 without the hosted app ever running; trusting status
alone wiped the instance's 72h offline grant cache within the hour.
Deletion now requires the hosted route's own rejection code in the
JSON body; codeless 401/403 keeps grants (server_error branch).

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01UzNkSsR18pLFitJdYn8QEb

* fix(connect): PR #1748 review batch: https-only connect URL, atomic pin, prefix-gated Bearer, deferred metering, entitlements validation, integer months

- GNUBOK_CONNECT_URL must be https (http only for loopback); invalid or
  plaintext URLs disable the connector instead of sending the key.
- instance_url pin update filters on IS NULL; a lost race re-reads and
  reports the winner's pin.
- extractConnectorKey: a Bearer is the connector credential only with
  the gnubok_ck_ prefix; upstream Bearer falls through to X-Connector-Key.
- Usage metering runs via after() off the response path (inline outside
  a request scope).
- Sync validates entitlements shape: unknown status or malformed
  current_period_end keeps grants (server_error), never deletes.
- issue-connector-key rejects fractional --months.

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01UzNkSsR18pLFitJdYn8QEb

---------

Co-authored-by: Jakob Wennberg <311770904+jakobwennberg-oss@users.noreply.github.com>
Co-authored-by: Claude Fable 5 <noreply@anthropic.com>
Co-authored-by: Emil <emilmattsson14@gmail.com>
This commit is contained in:
Jakob Wennberg
2026-08-31 20:51:17 +02:00
committed by GitHub
co-authored by Claude Fable 5 Jakob Wennberg Emil
parent cfce2de925
commit 0ff1b05553
24 changed files with 1949 additions and 12 deletions
@@ -0,0 +1,127 @@
import { describe, it, expect, vi, beforeEach } from 'vitest'
import { createMockRequest, parseJsonResponse } from '@/tests/helpers'
const KEY = {
id: '11111111-1111-4111-8111-111111111111',
orgNumber: '5561234567',
instanceUrl: null as string | null,
scopes: ['bank_sync', 'skatteverket', 'org_lookup', 'migration'],
status: 'active' as const,
currentPeriodEnd: '2027-01-01T00:00:00.000Z',
}
type UpdateResult = { error: unknown; data: Array<{ instance_url: string | null }> | null }
const updateResults: UpdateResult[] = []
const updateSelect = vi.fn(() => Promise.resolve(updateResults.shift() ?? { error: null, data: [{ instance_url: null }] }))
const updateIs = vi.fn(() => ({ select: updateSelect }))
const updateEq = vi.fn(() => ({ is: updateIs, select: updateSelect }))
const update = vi.fn((_payload: Record<string, unknown>) => ({ eq: updateEq }))
const from = vi.fn(() => ({ update }))
const logWarn = vi.fn()
// The wrapper is exercised in its own test; here it is replaced by a
// pass-through that injects the validated key so the handler logic is what
// gets tested.
vi.mock('@/lib/connect/hosted/with-connector-auth', () => ({
withConnectorAuth: (_op: string, handler: (req: Request, ctx: unknown) => Promise<Response>) => (req: Request) =>
handler(req, {
requestId: 'conn_test',
log: { info: vi.fn(), warn: logWarn, error: vi.fn() },
supabase: { from },
key: { ...KEY, instanceUrl: currentInstanceUrl },
}),
}))
let currentInstanceUrl: string | null = null
import { GET, POST } from '../route'
beforeEach(() => {
vi.clearAllMocks()
currentInstanceUrl = null
updateResults.length = 0
})
describe('GET /api/connect/entitlements', () => {
it('returns the key entitlements', async () => {
const res = await GET(createMockRequest('/api/connect/entitlements'))
const { status, body } = await parseJsonResponse<{ data: Record<string, unknown> }>(res)
expect(status).toBe(200)
expect(body.data).toMatchObject({
status: 'active',
scopes: KEY.scopes,
current_period_end: KEY.currentPeriodEnd,
org_number: '5561234567',
instance_url: null,
})
expect(typeof body.data.server_time).toBe('string')
})
})
describe('POST /api/connect/entitlements', () => {
it('400 on an invalid report', async () => {
const res = await POST(createMockRequest('/api/connect/entitlements', { method: 'POST', body: { active_company_count: -1 } }))
expect(res.status).toBe(400)
expect(update).not.toHaveBeenCalled()
})
it('records the active company count and pins instance_url on first report', async () => {
const res = await POST(
createMockRequest('/api/connect/entitlements', {
method: 'POST',
body: { active_company_count: 12, instance_url: 'https://bokforing.example.se', app_version: '1.2.3' },
}),
)
const { status, body } = await parseJsonResponse<{ data: { instance_url: string | null } }>(res)
expect(status).toBe(200)
expect(from).toHaveBeenCalledWith('connector_keys')
const payload = update.mock.calls[0][0]
expect(payload).toMatchObject({ active_company_count: 12, instance_url: 'https://bokforing.example.se' })
expect(typeof payload.last_synced_at).toBe('string')
expect(updateEq).toHaveBeenCalledWith('id', KEY.id)
expect(body.data.instance_url).toBe('https://bokforing.example.se')
})
// A leaked key must not be able to re-home the subscription.
it('never moves a pinned instance_url; logs the mismatch', async () => {
currentInstanceUrl = 'https://bokforing.example.se'
const res = await POST(
createMockRequest('/api/connect/entitlements', {
method: 'POST',
body: { active_company_count: 3, instance_url: 'https://evil.example.com' },
}),
)
expect(res.status).toBe(200)
const payload = update.mock.calls[0][0]
expect(payload.instance_url).toBeUndefined()
expect(logWarn).toHaveBeenCalled()
const { body } = await parseJsonResponse<{ data: { instance_url: string | null } }>(res)
expect(body.data.instance_url).toBe('https://bokforing.example.se')
})
it('500 when the update fails', async () => {
updateResults.push({ error: { message: 'boom' }, data: null })
const res = await POST(createMockRequest('/api/connect/entitlements', { method: 'POST', body: { active_company_count: 1 } }))
expect(res.status).toBe(500)
})
// Two concurrent FIRST reports: the pinning update is conditional on
// instance_url IS NULL, so the loser affects no row and must surface the
// winner's pin instead of its own URL.
it('losing the pin race keeps the first pin and reports it back', async () => {
updateResults.push({ error: null, data: [] }) // conditional pin: no row matched
updateResults.push({ error: null, data: [{ instance_url: 'https://first.example.se' }] }) // counter-only fallback
const res = await POST(
createMockRequest('/api/connect/entitlements', {
method: 'POST',
body: { active_company_count: 2, instance_url: 'https://second.example.se' },
}),
)
const { status, body } = await parseJsonResponse<{ data: { instance_url: string | null } }>(res)
expect(status).toBe(200)
expect(body.data.instance_url).toBe('https://first.example.se')
expect(logWarn).toHaveBeenCalled()
expect(update).toHaveBeenCalledTimes(2)
expect(update.mock.calls[1][0]).not.toHaveProperty('instance_url')
})
})
+99
View File
@@ -0,0 +1,99 @@
import { NextResponse } from 'next/server'
import { z } from 'zod'
import { validateBody } from '@/lib/api/validate'
import { withConnectorAuth, type ConnectorContext } from '@/lib/connect/hosted/with-connector-auth'
import type { ConnectorEntitlements } from '@/lib/connect/contract'
/**
* /api/connect/entitlements: what a self-hosted instance's connector key
* entitles it to. The instance's hourly sync (lib/connect/instance/sync.ts)
* POSTs its active company count (quantity billing input) and gets back
* status, scopes and the paid period; it then writes source='connector'
* capability grants that expire at min(now + 72h, period_end + 3d), so the
* grant rows are the offline cache and the hosted service only has to be
* reachable once every three days.
*
* GET -> entitlements (no side effects beyond last_seen_at)
* POST -> { active_company_count, instance_url?, app_version? } -> entitlements;
* records the count and pins instance_url on first report.
*/
const SyncReportSchema = z.object({
active_company_count: z.number().int().min(0).max(1_000_000),
instance_url: z.string().url().max(512).optional(),
app_version: z.string().max(64).optional(),
})
function entitlementsOf(ctx: ConnectorContext, instanceUrl: string | null): ConnectorEntitlements {
return {
status: ctx.key.status,
scopes: ctx.key.scopes,
current_period_end: ctx.key.currentPeriodEnd,
org_number: ctx.key.orgNumber,
instance_url: instanceUrl,
server_time: new Date().toISOString(),
}
}
export const GET = withConnectorAuth('connect.entitlements', async (_request, ctx) => {
return NextResponse.json({ data: entitlementsOf(ctx, ctx.key.instanceUrl) })
})
export const POST = withConnectorAuth('connect.entitlements', async (request, ctx) => {
const parsed = await validateBody(request, SyncReportSchema, { log: ctx.log, operation: 'connect.entitlements' })
if (!parsed.success) return parsed.response
const report = parsed.data
// instance_url is pinned: the first report claims it, later reports that
// disagree are logged but never move it (a leaked key cannot re-home the
// subscription to another instance).
let instanceUrl = ctx.key.instanceUrl
const pinNow = !!report.instance_url && !instanceUrl
if (report.instance_url && instanceUrl && instanceUrl !== report.instance_url) {
ctx.log.warn('sync reported a different instance_url than the pinned one', {
pinned: instanceUrl,
reported: report.instance_url,
})
}
if (pinNow) instanceUrl = report.instance_url ?? null
const lastSyncedAt = new Date().toISOString()
// Two literal payloads rather than one built object: the no-phantom-columns
// scanner resolves literals only. The pinning update additionally filters
// on instance_url IS NULL so two concurrent first reports cannot both pin:
// the loser's conditional update matches no row and re-reads the winner's
// URL below (first-report-wins, later reports never move the pin).
const { error, data: pinned } = pinNow
? await ctx.supabase
.from('connector_keys')
.update({
active_company_count: report.active_company_count,
last_synced_at: lastSyncedAt,
instance_url: instanceUrl,
})
.eq('id', ctx.key.id)
.is('instance_url', null)
.select('instance_url')
: await ctx.supabase
.from('connector_keys')
.update({ active_company_count: report.active_company_count, last_synced_at: lastSyncedAt })
.eq('id', ctx.key.id)
.select('instance_url')
if (error) {
ctx.log.error('failed to record connector sync', error)
return NextResponse.json({ error: 'Failed to record sync', code: 'INTERNAL_ERROR' }, { status: 500 })
}
if (pinNow && (pinned ?? []).length === 0) {
// Lost the pin race: record the counters and surface the winner's pin.
const { data: existing } = await ctx.supabase
.from('connector_keys')
.update({ active_company_count: report.active_company_count, last_synced_at: lastSyncedAt })
.eq('id', ctx.key.id)
.select('instance_url')
instanceUrl = (existing?.[0] as { instance_url: string | null } | undefined)?.instance_url ?? null
ctx.log.warn('instance_url pin race lost; keeping the first pin', {
reported: report.instance_url,
pinned: instanceUrl,
})
}
return NextResponse.json({ data: entitlementsOf(ctx, instanceUrl) })
})
@@ -0,0 +1,54 @@
import { describe, it, expect, vi, beforeEach, afterEach } from 'vitest'
const h = vi.hoisted(() => ({
logInfo: vi.fn(),
logError: vi.fn(),
sync: vi.fn(),
}))
vi.mock('@/lib/api/with-cron-context', () => ({
withCronContext:
(_name: string, handler: (req: Request, ctx: unknown) => Promise<Response>) =>
(req: Request) =>
handler(req, { log: { info: h.logInfo, error: h.logError, warn: vi.fn() }, requestId: 'cron_test' }),
}))
vi.mock('@/lib/supabase/server', () => ({
createServiceClient: () => ({ from: vi.fn() }),
}))
vi.mock('@/lib/connect/instance/sync', () => ({
syncConnectorEntitlements: (...args: unknown[]) => h.sync(...args),
}))
import { GET } from '../route'
beforeEach(() => vi.clearAllMocks())
afterEach(() => vi.unstubAllEnvs())
describe('GET /api/connector/sync/cron', () => {
it('answers not_configured without a key and never calls the sync', async () => {
vi.stubEnv('GNUBOK_CONNECTOR_KEY', '')
const res = await GET(new Request('http://localhost:3000/api/connector/sync/cron'))
expect(res.status).toBe(200)
expect(await res.json()).toEqual({ data: { outcome: 'not_configured' } })
expect(h.sync).not.toHaveBeenCalled()
})
it('runs the sync with the instance origin and returns its result', async () => {
vi.stubEnv('GNUBOK_CONNECTOR_KEY', 'gnubok_ck_x')
vi.stubEnv('NEXT_PUBLIC_APP_URL', 'https://bokforing.example.se')
h.sync.mockResolvedValue({ outcome: 'synced', companies: 2, grantsUpserted: 8, grantsDeleted: 0 })
const res = await GET(new Request('http://localhost:3000/api/connector/sync/cron'))
expect(res.status).toBe(200)
expect(await res.json()).toEqual({ data: { outcome: 'synced', companies: 2, grantsUpserted: 8, grantsDeleted: 0 } })
expect(h.sync).toHaveBeenCalledWith(expect.anything(), expect.objectContaining({ instanceUrl: 'https://bokforing.example.se' }))
expect(h.logInfo).toHaveBeenCalled()
})
it('logs and returns an error envelope when the sync throws', async () => {
vi.stubEnv('GNUBOK_CONNECTOR_KEY', 'gnubok_ck_x')
h.sync.mockRejectedValue(new Error('Failed to upsert connector grants: boom'))
const res = await GET(new Request('http://localhost:3000/api/connector/sync/cron'))
expect(res.status).toBeGreaterThanOrEqual(500)
expect(h.logError).toHaveBeenCalled()
})
})
+37
View File
@@ -0,0 +1,37 @@
import { NextResponse } from 'next/server'
import { withCronContext } from '@/lib/api/with-cron-context'
import { createServiceClient } from '@/lib/supabase/server'
import { getConnectorConfig } from '@/lib/connect/instance/config'
import { syncConnectorEntitlements } from '@/lib/connect/instance/sync'
import { errorResponse } from '@/lib/errors/get-structured-error'
/**
* GET /api/connector/sync/cron: hourly on SELF-HOSTED deployments only
* (docker/crontab.self-hosted via EXTRA_JOBS in scripts/generate-crontabs.ts;
* deliberately not in vercel.json, hosted has no connector key).
*
* Validates GNUBOK_CONNECTOR_KEY against the hosted connector service,
* reports the active company count, and refreshes the source='connector'
* capability grants for every company on this instance. Grant expiry
* (min(now + 72h, period_end + 3d)) is the offline grace: a hosted outage
* shorter than that changes nothing here. An instance without a key answers
* 200 with outcome not_configured, so the schedule costs one cheap request.
*/
export const maxDuration = 60
export const GET = withCronContext('cron.connector_sync', async (_request, ctx) => {
if (!getConnectorConfig()) {
return NextResponse.json({ data: { outcome: 'not_configured' } })
}
try {
const result = await syncConnectorEntitlements(createServiceClient(), {
instanceUrl: process.env.NEXT_PUBLIC_APP_URL?.trim() || null,
appVersion: process.env.npm_package_version ?? null,
})
ctx.log.info('connector sync run', { ...result })
return NextResponse.json({ data: result })
} catch (err) {
ctx.log.error('connector sync failed', err)
return errorResponse(err, ctx.log, { requestId: ctx.requestId })
}
})