feat(connect): Enable Banking proxy for self-hosted instances, with a secret-free ownership ledger and a global rate budget (#1751)

* feat(entitlements): partition the self-host bypass so connector capabilities fall through to grants; capability_grants.source accepts 'connector'

Sovereign plan WS3 PR3: ships dark, nothing changes for hosted.

- lib/entitlements/keys.ts: CONNECTOR_CAPABILITIES = bank_sync,
  skatteverket, org_lookup, migration (services Accounted operates that a
  self-hosted instance cannot provide itself) + isConnectorCapability().
  Separate from PAID_CAPABILITIES and outside the trial-seed trigger on
  purpose: a hosted company can never hold a connector grant.
- lib/entitlements/has-capability.ts: isPaywallBypassed() -> isBypassedFor(key).
  Hosted: byte-identical (dev / DISABLE_PAYWALL bypass, FORCE_PAYWALL wins,
  else the grant lookup). Self-host: local capabilities always on
  (FORCE_PAYWALL included, as the existing test demands); connector
  capabilities behave like hosted, i.e. dev bypass, FORCE_PAYWALL, else the
  grant lookup where the connector sync will write source='connector' rows.
  getCompanyEntitlements on a self-host: local paid keys + active connector
  keys, state 'paid' with an active connector grant else 'none' (never the
  hosted trial copy).
- Migration 20260820122000: capability_grants.source CHECK gains
  'connector', found through pg_constraint (the CHECK was declared inline
  and auto-named; Postgres stores IN as = ANY, matched accordingly).
  pg-real test: connector accepted, unknown source rejected, upsert on the
  (scope, key, source) identity, trial seed writes no connector rows.
- Tests: self-hosted connector matrix (local all-on without DB, connector
  gated by grant/expiry, dev bypass all-on, FORCE_PAYWALL gates connector
  keys only, bulk resolution, entitlements shape); two pre-existing tests
  that asserted the old "self-host holds connector keys" contract updated
  to the new one.

Verified: full unit suite green, pg-real suite for lib/entitlements green
against a local supabase/postgres with every migration applied, lint
ratchet, guards. Deferred to the instance-wiring PR: adding the connector
extensions to the self-host Docker preset (dead-end upsells until a key can
be issued).

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>

* refactor(entitlements): fold the self-host branch into the existing grants query

One .or(scopeFilter), not two: the duplicated helper pushed the
no-phantom-columns unresolvable-expression count to 380/379. Behaviour is
unchanged; the self-host matrix tests still pass.

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>

* feat(connect): hosted connector-key registry + validate RPC + entitlements endpoint; instance sync writes connector grants hourly

Sovereign plan WS3 PR4 ("key infra enabling manual sales"), stacked on the
entitlement partition (#1747). Nothing is purchasable yet; this is the
plumbing both ends need before the first manually issued key.

Hosted side:
- Migration 20260820123000: connector_keys (SHA-256 key_hash, prefix,
  org_number, pinned instance_url, scopes, status, Stripe ids,
  current_period_end, per-minute rate limit, active_company_count,
  last_seen/synced) and connector_usage_events (per-request metering,
  separate from metered_events whose company_id references hosted
  companies). RLS on, NO policies: service role only. RPC
  validate_and_increment_connector_key copies the api_keys pattern (FOR
  UPDATE, minute window, suspended reported not counted, revoked = no row)
  and is REVOKEd from PUBLIC/anon/authenticated, GRANTed to service_role.
  pg-real test covers validate/count, unknown+revoked, suspended, rate
  limit, execute privileges per role, RLS invisibility, usage cascade.
- lib/connect/contract.ts (shared wire types), lib/connect/hosted/keys.ts
  (generate/hash/validate -> 401/403/429 mapping),
  with-connector-auth.ts (Bearer or X-Connector-Key, one usage row per
  request, 500 envelope on handler throw), /api/connect/entitlements GET +
  POST (records active_company_count, pins instance_url on first report,
  never moves a pinned one), scripts/issue-connector-key.ts (dry run unless
  --confirm, prints the key once + the .env lines).

Instance side:
- lib/connect/instance/config.ts (GNUBOK_CONNECTOR_KEY, GNUBOK_CONNECT_URL
  default https://app.gnubok.se), sync.ts: reports the active company count
  and writes source='connector' grants for every company x covered scope,
  expires_at = min(now+72h, period_end+3d); 401/403 or a non-active status
  deletes them (freeze-and-retain); network/5xx/429 leave them alone.
  /api/connector/sync/cron (hourly) runs it; not_configured without a key.
- Crontab generator gains EXTRA_JOBS (variant-only jobs not in vercel.json,
  with reasons) + drift tests; docker/crontab.self-hosted regenerated with
  the hourly sync. Docs (SELF-HOSTING connector section, env templates),
  DECISIONS.

Tests: 52 new unit tests (keys, auth wrapper, route, config, sync outcomes
and grant arithmetic, cron route, crontab EXTRA_JOBS) + 7 pg-real tests
run locally against supabase/postgres with every migration applied.
no-phantom-columns ceiling +1 with a reason (the bulk grant upsert).

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>

* feat(connect): Enable Banking proxy for self-hosted instances, with a secret-free ownership ledger and a global rate budget

Sovereign plan WS3 PR5a, stacked on the connector-key infra (#1748). A
self-hosted instance with a `bank_sync`-scoped connector key can now connect
a bank through Arcim's PSD2 credentials; the bank session id and all
transaction data stay in the instance's own database (founder decision:
tokens on the instance, proxy stateless).

- Migration 20260820124000: `connector_connections` (secret-free ledger:
  sha256 of the EB session id + account uids, service-role only),
  `connector_upstream_counters` + RPC `connector_reserve_upstream` (global
  budget under EB Annex 1 §5's 300/min, shared with hosted), and
  `connector_keys.limits` jsonb; validate RPC v2 returns limits. All RPCs
  REVOKEd from PUBLIC/anon/authenticated, GRANTed service_role. pg-real
  covers all of it.
- EB JWT minting moved to lib/connect/upstreams/enable-banking-jwt.ts (core
  must not import @/extensions/); the extension re-exports it, tests
  unchanged.
- lib/connect/hosted/{state,ledger,upstream-budget}.ts: HMAC-signed
  connector state (15-min TTL) so the consent redirect can use OUR
  registered EB callback and bounce back to the instance, no per-instance
  redirect URI at EB; the callback route gains that connector branch.
- app/api/connect/bank/[...path]: path allowlist (aspsps, auth, sessions,
  accounts/{uid}/{balances,transactions}), never open passthrough. POST
  /auth enforces the per-company connection quota + rewrites redirect/state;
  reads/deletes verify ledger ownership; every upstream call takes the
  global budget (429 + Retry-After when exhausted).
- issue-connector-key.ts: scopes default bank_sync,skatteverket (TIC out of
  v1), --bank/skv-connections-per-company + --sync-min-interval.
- Docs (SELF-HOSTING: bank connector live), DECISIONS.

Verified: 52 connect unit tests + 13 pg-real (run locally against
supabase/postgres with all migrations) + EB extension suite (225, jwt
relocation intact); full unit suite 15 979 green; tsc, guards, lint clean.
Not in this PR: SKV broker (PR5b) and instance wiring (PR6).

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>

* chore(connect): update ledger pg test to re-versioned migration 20260831200000

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01UzNkSsR18pLFitJdYn8QEb

* fix(connect): redact opaque path segments before usage metering; correct stale RPC-source comment

GET/DELETE /sessions/{id} and /accounts/{uid}/... carry the raw EB
session id / account uid in the pathname; metering persisted it in
cleartext next to the ledger that stores only sha256(handle). Opaque
segments (UUID, long hex, long base64url) now become ':id' before the
connector_usage_events insert. Migration comment now cites the real
prior RPC source (20260831190000).

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01UzNkSsR18pLFitJdYn8QEb

* fix(connect): percent-encoded path segments count as opaque in metering redaction

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01UzNkSsR18pLFitJdYn8QEb

* fix(connect): PR #1751 review batch: https-only EB URL, body-covering timeout, quota reservation, delete-after-success, doc fix

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01UzNkSsR18pLFitJdYn8QEb

* fix(connect): bind the /sessions code exchange to its verified pending state; ceiling +1

Verified state signature, key/service match, and an existing pending
row now precede the EB exchange; a concurrently consumed state closes
the just-minted upstream session and 409s. no-phantom-columns ceiling
391 for countHeldConnections' computed .or() timestamp filter.

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01UzNkSsR18pLFitJdYn8QEb

---------

Co-authored-by: Jakob Wennberg <311770904+jakobwennberg-oss@users.noreply.github.com>
Co-authored-by: Claude Fable 5 <noreply@anthropic.com>
Co-authored-by: Emil <emilmattsson14@gmail.com>
This commit is contained in:
Jakob Wennberg
2026-08-31 21:46:50 +02:00
committed by GitHub
co-authored by Claude Fable 5 Jakob Wennberg Emil
parent 0ff1b05553
commit 36123cef23
20 changed files with 1570 additions and 120 deletions
+18 -1
View File
@@ -352,7 +352,24 @@ The cron sidecar calls `/api/connector/sync/cron` hourly (it is listed in `docke
curl -sf -H "Authorization: Bearer $CRON_SECRET" http://localhost:3000/api/connector/sync/cron
```
The proxied connector services themselves (bank, Skatteverket, lookup, migration through `app.gnubok.se/api/connect/*`) ship in a following release; until then the key is validated and the grants are written, nothing more.
The bank proxy (`app.gnubok.se/api/connect/bank/*`) is live server-side with this release; the Skatteverket broker and the instance-side client wiring that makes the proxies carry traffic ship in following releases. Until that wiring lands, the key is validated and the grants are written, nothing more.
### Connector subscription (self-hosted instances)
Everything a self-hosted instance runs itself is free (AGPL). Four capabilities depend on services only Accounted operates and are therefore gated on a self-host: bank sync (our PSD2/AISP credentials), Skatteverket API submission and skattekonto sync (our API client registration), company lookup (TIC) and migration from Fortnox/Visma/Bokio/Björn Lundén (the migration gateway). A **connector key** unlocks them for every company on the instance; it is priced per active company at parity with hosted and is issued manually by Accounted for now (self-serve later).
```bash
GNUBOK_CONNECTOR_KEY=gnubok_ck_... # issued by Accounted, shown once
# GNUBOK_CONNECT_URL=https://app.gnubok.se # default: the hosted connector service
```
The cron sidecar calls `/api/connector/sync/cron` hourly (it is listed in `docker/crontab.self-hosted` only): the instance reports its active company count, the hosted service answers with the key's status and scopes, and the instance writes `capability_grants` rows with `source = 'connector'` that expire after **72 hours** (or three days past the paid period, whichever is sooner). Those rows are the offline cache: a hosted outage shorter than that changes nothing, a revoked or lapsed key freezes the connector capabilities within days, and nothing in the instance phones home for permission to run the bookkeeping. An instance without a key answers `not_configured` and stays unaffected. To run the sync once by hand after pasting the key:
```bash
curl -sf -H "Authorization: Bearer $CRON_SECRET" http://localhost:3000/api/connector/sync/cron
```
The **bank connector** proxy is live (`app.gnubok.se/api/connect/bank/*`): with `bank_sync` in your key's scopes, the instance connects a bank through Arcim's PSD2 credentials while the bank session id and all transaction data stay in the instance's own database. Skatteverket, company lookup and migration through the connector ship in following releases; until each lands, a key is validated and its grants are written, and the unshipped services stay unconfigured.
### Push Notifications