feat(connect): Enable Banking proxy for self-hosted instances, with a secret-free ownership ledger and a global rate budget (#1751)
* feat(entitlements): partition the self-host bypass so connector capabilities fall through to grants; capability_grants.source accepts 'connector' Sovereign plan WS3 PR3: ships dark, nothing changes for hosted. - lib/entitlements/keys.ts: CONNECTOR_CAPABILITIES = bank_sync, skatteverket, org_lookup, migration (services Accounted operates that a self-hosted instance cannot provide itself) + isConnectorCapability(). Separate from PAID_CAPABILITIES and outside the trial-seed trigger on purpose: a hosted company can never hold a connector grant. - lib/entitlements/has-capability.ts: isPaywallBypassed() -> isBypassedFor(key). Hosted: byte-identical (dev / DISABLE_PAYWALL bypass, FORCE_PAYWALL wins, else the grant lookup). Self-host: local capabilities always on (FORCE_PAYWALL included, as the existing test demands); connector capabilities behave like hosted, i.e. dev bypass, FORCE_PAYWALL, else the grant lookup where the connector sync will write source='connector' rows. getCompanyEntitlements on a self-host: local paid keys + active connector keys, state 'paid' with an active connector grant else 'none' (never the hosted trial copy). - Migration 20260820122000: capability_grants.source CHECK gains 'connector', found through pg_constraint (the CHECK was declared inline and auto-named; Postgres stores IN as = ANY, matched accordingly). pg-real test: connector accepted, unknown source rejected, upsert on the (scope, key, source) identity, trial seed writes no connector rows. - Tests: self-hosted connector matrix (local all-on without DB, connector gated by grant/expiry, dev bypass all-on, FORCE_PAYWALL gates connector keys only, bulk resolution, entitlements shape); two pre-existing tests that asserted the old "self-host holds connector keys" contract updated to the new one. Verified: full unit suite green, pg-real suite for lib/entitlements green against a local supabase/postgres with every migration applied, lint ratchet, guards. Deferred to the instance-wiring PR: adding the connector extensions to the self-host Docker preset (dead-end upsells until a key can be issued). Co-Authored-By: Claude Fable 5 <noreply@anthropic.com> * refactor(entitlements): fold the self-host branch into the existing grants query One .or(scopeFilter), not two: the duplicated helper pushed the no-phantom-columns unresolvable-expression count to 380/379. Behaviour is unchanged; the self-host matrix tests still pass. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com> * feat(connect): hosted connector-key registry + validate RPC + entitlements endpoint; instance sync writes connector grants hourly Sovereign plan WS3 PR4 ("key infra enabling manual sales"), stacked on the entitlement partition (#1747). Nothing is purchasable yet; this is the plumbing both ends need before the first manually issued key. Hosted side: - Migration 20260820123000: connector_keys (SHA-256 key_hash, prefix, org_number, pinned instance_url, scopes, status, Stripe ids, current_period_end, per-minute rate limit, active_company_count, last_seen/synced) and connector_usage_events (per-request metering, separate from metered_events whose company_id references hosted companies). RLS on, NO policies: service role only. RPC validate_and_increment_connector_key copies the api_keys pattern (FOR UPDATE, minute window, suspended reported not counted, revoked = no row) and is REVOKEd from PUBLIC/anon/authenticated, GRANTed to service_role. pg-real test covers validate/count, unknown+revoked, suspended, rate limit, execute privileges per role, RLS invisibility, usage cascade. - lib/connect/contract.ts (shared wire types), lib/connect/hosted/keys.ts (generate/hash/validate -> 401/403/429 mapping), with-connector-auth.ts (Bearer or X-Connector-Key, one usage row per request, 500 envelope on handler throw), /api/connect/entitlements GET + POST (records active_company_count, pins instance_url on first report, never moves a pinned one), scripts/issue-connector-key.ts (dry run unless --confirm, prints the key once + the .env lines). Instance side: - lib/connect/instance/config.ts (GNUBOK_CONNECTOR_KEY, GNUBOK_CONNECT_URL default https://app.gnubok.se), sync.ts: reports the active company count and writes source='connector' grants for every company x covered scope, expires_at = min(now+72h, period_end+3d); 401/403 or a non-active status deletes them (freeze-and-retain); network/5xx/429 leave them alone. /api/connector/sync/cron (hourly) runs it; not_configured without a key. - Crontab generator gains EXTRA_JOBS (variant-only jobs not in vercel.json, with reasons) + drift tests; docker/crontab.self-hosted regenerated with the hourly sync. Docs (SELF-HOSTING connector section, env templates), DECISIONS. Tests: 52 new unit tests (keys, auth wrapper, route, config, sync outcomes and grant arithmetic, cron route, crontab EXTRA_JOBS) + 7 pg-real tests run locally against supabase/postgres with every migration applied. no-phantom-columns ceiling +1 with a reason (the bulk grant upsert). Co-Authored-By: Claude Fable 5 <noreply@anthropic.com> * feat(connect): Enable Banking proxy for self-hosted instances, with a secret-free ownership ledger and a global rate budget Sovereign plan WS3 PR5a, stacked on the connector-key infra (#1748). A self-hosted instance with a `bank_sync`-scoped connector key can now connect a bank through Arcim's PSD2 credentials; the bank session id and all transaction data stay in the instance's own database (founder decision: tokens on the instance, proxy stateless). - Migration 20260820124000: `connector_connections` (secret-free ledger: sha256 of the EB session id + account uids, service-role only), `connector_upstream_counters` + RPC `connector_reserve_upstream` (global budget under EB Annex 1 §5's 300/min, shared with hosted), and `connector_keys.limits` jsonb; validate RPC v2 returns limits. All RPCs REVOKEd from PUBLIC/anon/authenticated, GRANTed service_role. pg-real covers all of it. - EB JWT minting moved to lib/connect/upstreams/enable-banking-jwt.ts (core must not import @/extensions/); the extension re-exports it, tests unchanged. - lib/connect/hosted/{state,ledger,upstream-budget}.ts: HMAC-signed connector state (15-min TTL) so the consent redirect can use OUR registered EB callback and bounce back to the instance, no per-instance redirect URI at EB; the callback route gains that connector branch. - app/api/connect/bank/[...path]: path allowlist (aspsps, auth, sessions, accounts/{uid}/{balances,transactions}), never open passthrough. POST /auth enforces the per-company connection quota + rewrites redirect/state; reads/deletes verify ledger ownership; every upstream call takes the global budget (429 + Retry-After when exhausted). - issue-connector-key.ts: scopes default bank_sync,skatteverket (TIC out of v1), --bank/skv-connections-per-company + --sync-min-interval. - Docs (SELF-HOSTING: bank connector live), DECISIONS. Verified: 52 connect unit tests + 13 pg-real (run locally against supabase/postgres with all migrations) + EB extension suite (225, jwt relocation intact); full unit suite 15 979 green; tsc, guards, lint clean. Not in this PR: SKV broker (PR5b) and instance wiring (PR6). Co-Authored-By: Claude Fable 5 <noreply@anthropic.com> * chore(connect): update ledger pg test to re-versioned migration 20260831200000 Co-Authored-By: Claude Fable 5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01UzNkSsR18pLFitJdYn8QEb * fix(connect): redact opaque path segments before usage metering; correct stale RPC-source comment GET/DELETE /sessions/{id} and /accounts/{uid}/... carry the raw EB session id / account uid in the pathname; metering persisted it in cleartext next to the ledger that stores only sha256(handle). Opaque segments (UUID, long hex, long base64url) now become ':id' before the connector_usage_events insert. Migration comment now cites the real prior RPC source (20260831190000). Co-Authored-By: Claude Fable 5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01UzNkSsR18pLFitJdYn8QEb * fix(connect): percent-encoded path segments count as opaque in metering redaction Co-Authored-By: Claude Fable 5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01UzNkSsR18pLFitJdYn8QEb * fix(connect): PR #1751 review batch: https-only EB URL, body-covering timeout, quota reservation, delete-after-success, doc fix Co-Authored-By: Claude Fable 5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01UzNkSsR18pLFitJdYn8QEb * fix(connect): bind the /sessions code exchange to its verified pending state; ceiling +1 Verified state signature, key/service match, and an existing pending row now precede the EB exchange; a concurrently consumed state closes the just-minted upstream session and 409s. no-phantom-columns ceiling 391 for countHeldConnections' computed .or() timestamp filter. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01UzNkSsR18pLFitJdYn8QEb --------- Co-authored-by: Jakob Wennberg <311770904+jakobwennberg-oss@users.noreply.github.com> Co-authored-by: Claude Fable 5 <noreply@anthropic.com> Co-authored-by: Emil <emilmattsson14@gmail.com>
This commit is contained in:
Jakob Wennberg
co-authored by
Claude Fable 5
Jakob Wennberg
Emil
Claude Fable 5
Jakob Wennberg
Emil
parent
0ff1b05553
commit
36123cef23
@@ -15,6 +15,7 @@ const ROW = {
|
||||
status: 'active',
|
||||
current_period_end: '2027-01-01T00:00:00.000Z',
|
||||
rate_limited: false,
|
||||
limits: { bank_connections_per_company: 2, skv_connections_per_company: 1, sync_min_interval_s: 3600 },
|
||||
}
|
||||
|
||||
describe('connector key primitives', () => {
|
||||
@@ -56,10 +57,17 @@ describe('validateConnectorKey', () => {
|
||||
scopes: ['bank_sync', 'skatteverket'],
|
||||
status: 'active',
|
||||
currentPeriodEnd: '2027-01-01T00:00:00.000Z',
|
||||
limits: { bank_connections_per_company: 2, skv_connections_per_company: 1, sync_min_interval_s: 3600 },
|
||||
},
|
||||
})
|
||||
})
|
||||
|
||||
it('fills default limits when the RPC returns null limits', async () => {
|
||||
const { key } = generateConnectorKey()
|
||||
const result = await validateConnectorKey(key, supabaseWithRpc({ data: [{ ...ROW, limits: null }] }).supabase)
|
||||
expect(result.ok && result.key.limits).toEqual({ bank_connections_per_company: 1, skv_connections_per_company: 1, sync_min_interval_s: 0 })
|
||||
})
|
||||
|
||||
it('maps no row (unknown/revoked) to 401, but an RPC error to 503', async () => {
|
||||
const { key } = generateConnectorKey()
|
||||
expect(await validateConnectorKey(key, supabaseWithRpc({ data: [] }).supabase)).toMatchObject({ ok: false, status: 401 })
|
||||
|
||||
@@ -0,0 +1,53 @@
|
||||
import { describe, it, expect, afterEach, vi } from 'vitest'
|
||||
import { isConnectorState, signConnectorState, verifyConnectorState } from '../state'
|
||||
|
||||
afterEach(() => vi.unstubAllEnvs())
|
||||
|
||||
const BASE = { kid: 'k1', svc: 'bank' as const, ret: 'https://bokforing.example.se/cb', st: 'inst-state', cref: 'company-1' }
|
||||
|
||||
describe('connector state', () => {
|
||||
it('round-trips a signed payload', () => {
|
||||
vi.stubEnv('CONNECTOR_STATE_SECRET', 'secret')
|
||||
const now = 1_000_000
|
||||
const token = signConnectorState(BASE, now)
|
||||
expect(isConnectorState(token)).toBe(true)
|
||||
const v = verifyConnectorState(token, now)
|
||||
expect(v).toEqual({ ok: true, payload: { ...BASE, iat: now } })
|
||||
})
|
||||
|
||||
it('rejects a tampered payload', () => {
|
||||
vi.stubEnv('CONNECTOR_STATE_SECRET', 'secret')
|
||||
const token = signConnectorState(BASE, 1000)
|
||||
const [ver, body, sig] = token.split('.')
|
||||
const tamperedBody = Buffer.from(JSON.stringify({ ...BASE, cref: 'other', iat: 1000 })).toString('base64url')
|
||||
expect(verifyConnectorState(`${ver}.${tamperedBody}.${sig}`, 1000)).toEqual({ ok: false, reason: 'bad_signature' })
|
||||
})
|
||||
|
||||
it('rejects a signature made with a different secret', () => {
|
||||
vi.stubEnv('CONNECTOR_STATE_SECRET', 'secret-a')
|
||||
const token = signConnectorState(BASE, 1000)
|
||||
vi.stubEnv('CONNECTOR_STATE_SECRET', 'secret-b')
|
||||
expect(verifyConnectorState(token, 1000)).toEqual({ ok: false, reason: 'bad_signature' })
|
||||
})
|
||||
|
||||
it('expires after the TTL and rejects a future iat', () => {
|
||||
vi.stubEnv('CONNECTOR_STATE_SECRET', 'secret')
|
||||
const token = signConnectorState(BASE, 1000)
|
||||
expect(verifyConnectorState(token, 1000 + 16 * 60 * 1000)).toEqual({ ok: false, reason: 'expired' })
|
||||
const future = signConnectorState(BASE, 10_000_000)
|
||||
expect(verifyConnectorState(future, 1000)).toEqual({ ok: false, reason: 'expired' })
|
||||
})
|
||||
|
||||
it('flags malformed tokens', () => {
|
||||
vi.stubEnv('CONNECTOR_STATE_SECRET', 'secret')
|
||||
expect(verifyConnectorState('nope')).toEqual({ ok: false, reason: 'malformed' })
|
||||
expect(isConnectorState('random-uuid-state')).toBe(false)
|
||||
})
|
||||
|
||||
it('derives a secret from the service-role key when none is set (still verifiable)', () => {
|
||||
vi.stubEnv('CONNECTOR_STATE_SECRET', '')
|
||||
vi.stubEnv('SUPABASE_SERVICE_ROLE_KEY', 'svc-key')
|
||||
const token = signConnectorState(BASE, 2000)
|
||||
expect(verifyConnectorState(token, 2000).ok).toBe(true)
|
||||
})
|
||||
})
|
||||
@@ -0,0 +1,37 @@
|
||||
import { describe, it, expect, vi, afterEach } from 'vitest'
|
||||
import type { SupabaseClient } from '@supabase/supabase-js'
|
||||
import { budgetFor, reserveUpstream } from '../upstream-budget'
|
||||
|
||||
afterEach(() => vi.unstubAllEnvs())
|
||||
|
||||
function supa(result: { data?: unknown; error?: unknown }) {
|
||||
const rpc = vi.fn().mockResolvedValue({ data: result.data ?? null, error: result.error ?? null })
|
||||
return { supabase: { rpc } as unknown as SupabaseClient, rpc }
|
||||
}
|
||||
|
||||
describe('budgetFor', () => {
|
||||
it('defaults sit under the EB per-minute quota and are env-overridable', () => {
|
||||
expect(budgetFor('bank').minuteMax).toBe(90)
|
||||
vi.stubEnv('CONNECT_BANK_RPM_BUDGET', '50')
|
||||
expect(budgetFor('bank').minuteMax).toBe(50)
|
||||
})
|
||||
})
|
||||
|
||||
describe('reserveUpstream', () => {
|
||||
it('passes the service and the resolved budget to the RPC and returns ok', async () => {
|
||||
const { supabase, rpc } = supa({ data: { ok: true } })
|
||||
expect(await reserveUpstream(supabase, 'bank')).toEqual({ ok: true })
|
||||
expect(rpc).toHaveBeenCalledWith('connector_reserve_upstream', { p_service: 'bank', p_minute_max: 90, p_hour_max: 3000 })
|
||||
})
|
||||
|
||||
it('maps a budget rejection to a Retry-After result', async () => {
|
||||
const { supabase } = supa({ data: { ok: false, scope: 'hour', retry_after_sec: 3600 } })
|
||||
expect(await reserveUpstream(supabase, 'bank')).toEqual({ ok: false, scope: 'hour', retryAfterSec: 3600 })
|
||||
})
|
||||
|
||||
// Fail-open: a broken counter table must not block every connector call.
|
||||
it('fails open on a DB error', async () => {
|
||||
const { supabase } = supa({ error: { message: 'boom' } })
|
||||
expect(await reserveUpstream(supabase, 'skatteverket')).toEqual({ ok: true })
|
||||
})
|
||||
})
|
||||
@@ -12,7 +12,7 @@ vi.mock('@/lib/auth/api-keys', () => ({
|
||||
createServiceClientNoCookies: () => ({ from }),
|
||||
}))
|
||||
|
||||
import { extractConnectorKey, withConnectorAuth } from '../with-connector-auth'
|
||||
import { extractConnectorKey, withConnectorAuth, redactEndpoint } from '../with-connector-auth'
|
||||
|
||||
const VALID = {
|
||||
ok: true,
|
||||
@@ -53,6 +53,23 @@ describe('extractConnectorKey', () => {
|
||||
})
|
||||
})
|
||||
|
||||
describe('redactEndpoint', () => {
|
||||
it('replaces opaque handle segments with :id so raw EB session ids never rest in metering', () => {
|
||||
expect(redactEndpoint('/api/connect/bank/sessions/8f14e45f-ceea-467f-a8d5-91be6ce7cbc4')).toBe(
|
||||
'/api/connect/bank/sessions/:id',
|
||||
)
|
||||
expect(redactEndpoint('/api/connect/bank/accounts/9a1b2c3d-0000-4111-8222-333344445555/transactions')).toBe(
|
||||
'/api/connect/bank/accounts/:id/transactions',
|
||||
)
|
||||
expect(redactEndpoint('/api/connect/bank/sessions/deadbeefdeadbeefdeadbeef')).toBe('/api/connect/bank/sessions/:id')
|
||||
expect(redactEndpoint('/api/connect/bank/sessions/8f14e45f%2Dceea-467f-a8d5-91be6ce7cbc4')).toBe(
|
||||
'/api/connect/bank/sessions/:id',
|
||||
)
|
||||
expect(redactEndpoint('/api/connect/entitlements')).toBe('/api/connect/entitlements')
|
||||
expect(redactEndpoint('/api/connect/bank/aspsps')).toBe('/api/connect/bank/aspsps')
|
||||
})
|
||||
})
|
||||
|
||||
describe('withConnectorAuth', () => {
|
||||
const handler = vi.fn(async (_req: Request, _ctx: { key: { id: string } }) => NextResponse.json({ data: 'ok' }))
|
||||
const wrapped = withConnectorAuth('connect.entitlements', handler)
|
||||
|
||||
Reference in New Issue
Block a user