feat(connect): Enable Banking proxy for self-hosted instances, with a secret-free ownership ledger and a global rate budget (#1751)

* feat(entitlements): partition the self-host bypass so connector capabilities fall through to grants; capability_grants.source accepts 'connector'

Sovereign plan WS3 PR3: ships dark, nothing changes for hosted.

- lib/entitlements/keys.ts: CONNECTOR_CAPABILITIES = bank_sync,
  skatteverket, org_lookup, migration (services Accounted operates that a
  self-hosted instance cannot provide itself) + isConnectorCapability().
  Separate from PAID_CAPABILITIES and outside the trial-seed trigger on
  purpose: a hosted company can never hold a connector grant.
- lib/entitlements/has-capability.ts: isPaywallBypassed() -> isBypassedFor(key).
  Hosted: byte-identical (dev / DISABLE_PAYWALL bypass, FORCE_PAYWALL wins,
  else the grant lookup). Self-host: local capabilities always on
  (FORCE_PAYWALL included, as the existing test demands); connector
  capabilities behave like hosted, i.e. dev bypass, FORCE_PAYWALL, else the
  grant lookup where the connector sync will write source='connector' rows.
  getCompanyEntitlements on a self-host: local paid keys + active connector
  keys, state 'paid' with an active connector grant else 'none' (never the
  hosted trial copy).
- Migration 20260820122000: capability_grants.source CHECK gains
  'connector', found through pg_constraint (the CHECK was declared inline
  and auto-named; Postgres stores IN as = ANY, matched accordingly).
  pg-real test: connector accepted, unknown source rejected, upsert on the
  (scope, key, source) identity, trial seed writes no connector rows.
- Tests: self-hosted connector matrix (local all-on without DB, connector
  gated by grant/expiry, dev bypass all-on, FORCE_PAYWALL gates connector
  keys only, bulk resolution, entitlements shape); two pre-existing tests
  that asserted the old "self-host holds connector keys" contract updated
  to the new one.

Verified: full unit suite green, pg-real suite for lib/entitlements green
against a local supabase/postgres with every migration applied, lint
ratchet, guards. Deferred to the instance-wiring PR: adding the connector
extensions to the self-host Docker preset (dead-end upsells until a key can
be issued).

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>

* refactor(entitlements): fold the self-host branch into the existing grants query

One .or(scopeFilter), not two: the duplicated helper pushed the
no-phantom-columns unresolvable-expression count to 380/379. Behaviour is
unchanged; the self-host matrix tests still pass.

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>

* feat(connect): hosted connector-key registry + validate RPC + entitlements endpoint; instance sync writes connector grants hourly

Sovereign plan WS3 PR4 ("key infra enabling manual sales"), stacked on the
entitlement partition (#1747). Nothing is purchasable yet; this is the
plumbing both ends need before the first manually issued key.

Hosted side:
- Migration 20260820123000: connector_keys (SHA-256 key_hash, prefix,
  org_number, pinned instance_url, scopes, status, Stripe ids,
  current_period_end, per-minute rate limit, active_company_count,
  last_seen/synced) and connector_usage_events (per-request metering,
  separate from metered_events whose company_id references hosted
  companies). RLS on, NO policies: service role only. RPC
  validate_and_increment_connector_key copies the api_keys pattern (FOR
  UPDATE, minute window, suspended reported not counted, revoked = no row)
  and is REVOKEd from PUBLIC/anon/authenticated, GRANTed to service_role.
  pg-real test covers validate/count, unknown+revoked, suspended, rate
  limit, execute privileges per role, RLS invisibility, usage cascade.
- lib/connect/contract.ts (shared wire types), lib/connect/hosted/keys.ts
  (generate/hash/validate -> 401/403/429 mapping),
  with-connector-auth.ts (Bearer or X-Connector-Key, one usage row per
  request, 500 envelope on handler throw), /api/connect/entitlements GET +
  POST (records active_company_count, pins instance_url on first report,
  never moves a pinned one), scripts/issue-connector-key.ts (dry run unless
  --confirm, prints the key once + the .env lines).

Instance side:
- lib/connect/instance/config.ts (GNUBOK_CONNECTOR_KEY, GNUBOK_CONNECT_URL
  default https://app.gnubok.se), sync.ts: reports the active company count
  and writes source='connector' grants for every company x covered scope,
  expires_at = min(now+72h, period_end+3d); 401/403 or a non-active status
  deletes them (freeze-and-retain); network/5xx/429 leave them alone.
  /api/connector/sync/cron (hourly) runs it; not_configured without a key.
- Crontab generator gains EXTRA_JOBS (variant-only jobs not in vercel.json,
  with reasons) + drift tests; docker/crontab.self-hosted regenerated with
  the hourly sync. Docs (SELF-HOSTING connector section, env templates),
  DECISIONS.

Tests: 52 new unit tests (keys, auth wrapper, route, config, sync outcomes
and grant arithmetic, cron route, crontab EXTRA_JOBS) + 7 pg-real tests
run locally against supabase/postgres with every migration applied.
no-phantom-columns ceiling +1 with a reason (the bulk grant upsert).

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>

* feat(connect): Enable Banking proxy for self-hosted instances, with a secret-free ownership ledger and a global rate budget

Sovereign plan WS3 PR5a, stacked on the connector-key infra (#1748). A
self-hosted instance with a `bank_sync`-scoped connector key can now connect
a bank through Arcim's PSD2 credentials; the bank session id and all
transaction data stay in the instance's own database (founder decision:
tokens on the instance, proxy stateless).

- Migration 20260820124000: `connector_connections` (secret-free ledger:
  sha256 of the EB session id + account uids, service-role only),
  `connector_upstream_counters` + RPC `connector_reserve_upstream` (global
  budget under EB Annex 1 §5's 300/min, shared with hosted), and
  `connector_keys.limits` jsonb; validate RPC v2 returns limits. All RPCs
  REVOKEd from PUBLIC/anon/authenticated, GRANTed service_role. pg-real
  covers all of it.
- EB JWT minting moved to lib/connect/upstreams/enable-banking-jwt.ts (core
  must not import @/extensions/); the extension re-exports it, tests
  unchanged.
- lib/connect/hosted/{state,ledger,upstream-budget}.ts: HMAC-signed
  connector state (15-min TTL) so the consent redirect can use OUR
  registered EB callback and bounce back to the instance, no per-instance
  redirect URI at EB; the callback route gains that connector branch.
- app/api/connect/bank/[...path]: path allowlist (aspsps, auth, sessions,
  accounts/{uid}/{balances,transactions}), never open passthrough. POST
  /auth enforces the per-company connection quota + rewrites redirect/state;
  reads/deletes verify ledger ownership; every upstream call takes the
  global budget (429 + Retry-After when exhausted).
- issue-connector-key.ts: scopes default bank_sync,skatteverket (TIC out of
  v1), --bank/skv-connections-per-company + --sync-min-interval.
- Docs (SELF-HOSTING: bank connector live), DECISIONS.

Verified: 52 connect unit tests + 13 pg-real (run locally against
supabase/postgres with all migrations) + EB extension suite (225, jwt
relocation intact); full unit suite 15 979 green; tsc, guards, lint clean.
Not in this PR: SKV broker (PR5b) and instance wiring (PR6).

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>

* chore(connect): update ledger pg test to re-versioned migration 20260831200000

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01UzNkSsR18pLFitJdYn8QEb

* fix(connect): redact opaque path segments before usage metering; correct stale RPC-source comment

GET/DELETE /sessions/{id} and /accounts/{uid}/... carry the raw EB
session id / account uid in the pathname; metering persisted it in
cleartext next to the ledger that stores only sha256(handle). Opaque
segments (UUID, long hex, long base64url) now become ':id' before the
connector_usage_events insert. Migration comment now cites the real
prior RPC source (20260831190000).

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01UzNkSsR18pLFitJdYn8QEb

* fix(connect): percent-encoded path segments count as opaque in metering redaction

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01UzNkSsR18pLFitJdYn8QEb

* fix(connect): PR #1751 review batch: https-only EB URL, body-covering timeout, quota reservation, delete-after-success, doc fix

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01UzNkSsR18pLFitJdYn8QEb

* fix(connect): bind the /sessions code exchange to its verified pending state; ceiling +1

Verified state signature, key/service match, and an existing pending
row now precede the EB exchange; a concurrently consumed state closes
the just-minted upstream session and 409s. no-phantom-columns ceiling
391 for countHeldConnections' computed .or() timestamp filter.

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01UzNkSsR18pLFitJdYn8QEb

---------

Co-authored-by: Jakob Wennberg <311770904+jakobwennberg-oss@users.noreply.github.com>
Co-authored-by: Claude Fable 5 <noreply@anthropic.com>
Co-authored-by: Emil <emilmattsson14@gmail.com>

DECISIONS.md

@@ -1142,6 +1142,7 @@ One line per decision: `[YYYY-MM-DD] <decision>: <why>`. Appended by agents and
 [2026-08-20] Unmatched bank rows that no voucher on the account could settle (direction-compatible and equal to the ore) get "Bokfor" linking to /transactions?highlight=<id> instead of a match picker. They are unbooked affarshandelser, not reconciliation work, and the picker held nothing for them. The rule is deliberately strict: a false negative offers booking on a pairable row (a legitimate outcome), a false positive sends the user into an empty picker.
 [2026-08-20] Entitlement partition for the sovereign self-host (plan WS3 PR3, ships dark): isPaywallBypassed() became isBypassedFor(key). Hosted behaviour is byte-identical (dev/DISABLE_PAYWALL bypass, FORCE_PAYWALL wins, otherwise the grant lookup). On a self-host every LOCAL capability stays always-on, FORCE_PAYWALL included (an AGPL operator's own instance is never gated on what it runs itself, which is also why the existing "FORCE_PAYWALL never overrides self-hosted" test still holds), and only the four CONNECTOR_CAPABILITIES (bank_sync, skatteverket, org_lookup, migration: services Accounted operates) fall through to the grant lookup, where the connector sync will write source='connector' rows. getCompanyEntitlements on a self-host reports state 'paid' when a connector grant is active and 'none' otherwise, never 'trial_expired' (that copy talks about a hosted trial). CONNECTOR_CAPABILITIES is deliberately separate from PAID_CAPABILITIES and outside the trial-seed trigger, so a hosted company can never hold a connector grant. The capability_grants.source CHECK gains 'connector' by looking the inline auto-named constraint up through pg_constraint. Adding the connector extensions to the self-host Docker preset was deferred to the instance-wiring PR: until a connector key can actually be issued, shipping those extensions in the image would only show dead-end upsells.
 [2026-08-20] Connector-key infrastructure (plan WS3 PR4) ships the hosted registry + the instance sync, nothing a customer can buy yet: connector_keys / connector_usage_events are service-role-only tables (RLS on, no policies) with an atomic validate_and_increment_connector_key RPC that copies the api_keys pattern (SHA-256 at rest, FOR UPDATE row lock, per-minute window) and is REVOKEd from PUBLIC, anon and authenticated before anyone can call it (the SECURITY DEFINER exposure lesson applied up front); /api/connect/entitlements behind withConnectorAuth (Bearer or X-Connector-Key, 401/403/429, one usage row per request); keys issued by scripts/issue-connector-key.ts (dry run unless --confirm, prints the key once). The instance side writes source='connector' grants expiring at min(now+72h, period_end+3d) on every hourly sync, deletes them on 401/403 or a non-active status, and leaves them alone on network/5xx errors: the grant rows ARE the offline cache, no new cache code. The hourly job lives only in docker/crontab.self-hosted through a new EXTRA_JOBS table in the crontab generator (with its own drift tests), because vercel.json is the hosted schedule and hosted has no connector key. connector_usage_events is a separate table because metered_events.company_id references hosted companies and a connector key belongs to an instance, not a company here. Deferred: the proxy routes (bank/skv/org/migration: founder legal check with Enable Banking/SKV/TIC is the launch blocker), a connect.gnubok.se host rewrite (the instance calls app.gnubok.se/api/connect directly; a dedicated host is a later DNS decision), the self-host Docker preset change and the settings row.
+[2026-08-20] Connector bank proxy (plan WS3 PR5a): app/api/connect/bank/[...path] brokers Enable Banking for self-hosted instances with tokens staying on the instance (founder decision) and the proxy stateless apart from a secret-free connection ledger. Design that keeps EB Annex 1 §3/§7 satisfied: the instance never holds the EB JWT (minting moved to lib/connect/upstreams/enable-banking-jwt.ts so core does not import @/extensions/; the extension re-exports it); the consent redirect goes to OUR already-registered EB callback, which detects an HMAC-signed connector state (lib/connect/hosted/state.ts, 15-min TTL, CONNECTOR_STATE_SECRET or a one-way derivation of the service-role key) and 302s the browser back to the instance, so no per-instance redirect URI is registered at EB. Ownership: connector_connections ledger stores sha256(session_id) and the account uids, never the session; GET/DELETE /sessions and /accounts/{uid} calls verify the handle/account belongs to the presenting key. Quotas: per-company bank connection limit (connector_keys.limits, sold in the package, checked at POST /auth), per-key RPM (validate RPC), and a GLOBAL budget (connector_reserve_upstream RPC, connector_upstream_counters, ~30% of EB's 300/min so hosted is never starved; fail-open on a counter error). validate_and_increment_connector_key gained a limits column (v2). issue-connector-key.ts scopes default to bank_sync,skatteverket (TIC/org_lookup out of v1 per founder) and take --bank-connections-per-company etc. Path allowlist only, never an open passthrough. Deferred: the SKV broker (PR5b, same pattern, callback branch on the SKV extension) and the instance-side wiring (PR6: EB client connector-mode branch, self-host preset, settings row).
 [2026-08-20] Entitlement partition for the sovereign self-host (plan WS3 PR3, ships dark): isPaywallBypassed() became isBypassedFor(key). Hosted behaviour is byte-identical (dev/DISABLE_PAYWALL bypass, FORCE_PAYWALL wins, otherwise the grant lookup). On a self-host every LOCAL capability stays always-on, FORCE_PAYWALL included (an AGPL operator's own instance is never gated on what it runs itself, which is also why the existing "FORCE_PAYWALL never overrides self-hosted" test still holds), and only the four CONNECTOR_CAPABILITIES (bank_sync, skatteverket, org_lookup, migration: services Accounted operates) fall through to the grant lookup, where the connector sync will write source='connector' rows. getCompanyEntitlements on a self-host reports state 'paid' when a connector grant is active and 'none' otherwise, never 'trial_expired' (that copy talks about a hosted trial). CONNECTOR_CAPABILITIES is deliberately separate from PAID_CAPABILITIES; the trial-seed trigger does seed 30-day source='trial' rows for bank_sync/skatteverket (they are PAID keys) but never writes source='connector' and never seeds the connector-only keys (org_lookup, migration), and on a self-host only source='connector' rows unlock a connector capability, so a hosted company can never hold a connector grant. The capability_grants.source CHECK gains 'connector' by looking the inline auto-named constraint up through pg_constraint. Adding the connector extensions to the self-host Docker preset was deferred to the instance-wiring PR: until a connector key can actually be issued, shipping those extensions in the image would only show dead-end upsells.
 [2026-08-20] Sovereign package docs (plan WS2 PR1): docs/SOVEREIGN.md is written as regulatory-risk elimination with a per-provider fact sheet checked on the vendors' own pages (Elastx CaaS/DBaaS/3 Stockholm AZs/ISO 27001:2022; GleSYS VPS + S3, no managed k8s, EU-owned not Swedish-owned; Safespring S3 with Object Lock COMPLIANCE/GOVERNANCE; Berget api.berget.ai/v1 with gemma-4-31B-it vision and an SLA that excludes serverless; evroc Think Models EU-only), never as "US cloud is illegal", and it leads with the MCP server as the agent surface that needs no AI provider at all (alignment rule R5). The connector subscription is described as planned and not yet available rather than documented as if it shipped. Vercel Speed Insights is now gated behind !isSelfHosted() in app/layout.tsx (the last ungated hosted-only telemetry; read via lib/env/public-flags per the folded-flag rule). Backup/restore ship as scripts/self-host/{backup,restore}.sh (pg_dump custom format + storage volume tar + optional db-config volume for the pgsodium root key, SHA-256 manifest, AWS CLI v2 against any S3-compatible endpoint, optional COMPLIANCE-mode Object Lock) with a bash -n + refusal-path test, because self-hosted Supabase has no managed backups and BFL 7 kap needs a credible 7-year archive. Stale self-host docs fixed: the 4-of-23 cron table replaced by a pointer to the generated crontab and the pgvector line corrected (nothing stores embeddings).
 [2026-08-20] Vercel build heap is raised through vercel.json `buildCommand` (`NODE_OPTIONS=--max-old-space-size=6144 npm run build`), not a project env var and not `build.env`: a project-level NODE_OPTIONS also reaches function runtime (V8 sizes the heap against a limit the function does not have), and `build.env` is marked deprecated in the vercel.json schema; `buildCommand` scopes the flag to the build exactly like core-build.yml's 8192 does for CI. 6144 fits the standard 4-core/8 GB build machine next to the main next process; the type-check needs ~4.5 GB and was hanging at V8's ~4 GB default ceiling (4 production timeouts 2026-08-14..20).
@@ -1406,3 +1407,6 @@ One line per decision: `[YYYY-MM-DD] <decision>: <why>`. Appended by agents and
 [2026-08-31] Own-credentials seam forward-ported into the entitlement partition (skeptic refutation on PR #1747): a self-host serving bank_sync/skatteverket from its OWN env credentials (the same vars the extensions activate on) counts those as local capabilities, so upgrading an own-credentials self-host never dark-launches the connector gate against a working integration (the 2026-08-17 folded-flag incident shape). lib/entitlements/own-credentials.ts mirrors the connector-mode seam arriving in the instance-wiring PR (connector mode = key AND no own creds) and must stay in sync with it. org_lookup/migration have no own-credentials form. capability_blocked copy now has a self-host variant naming GNUBOK_CONNECTOR_KEY instead of the hosted subscription upsell, which misled operators toward a product they cannot buy for a self-host.
 [2026-08-31] Connector key validation maps a database/RPC error to 503 CONNECTOR_VALIDATION_UNAVAILABLE, never 401 (skeptic refutation on PR #1748): the instance sync deletes its entire connector grant cache on 401/403 (revocation semantics), so the api-keys fail-closed-to-401 precedent would let a transient hosted DB blip destroy a paying instance's 72h offline grace; 503 lands in the sync's keep-grants branch. In the same pass X-Connector-Key now wins over Authorization in extractConnectorKey: the header exists solely for proxied calls where Authorization carries an upstream token (the SKV data proxy sends both), and Bearer-first hashed the upstream token and 401'd exactly that shape.
 [2026-08-31] Connector sync deletes its grant cache only on a 401/403 whose JSON body carries a connector rejection code (CONNECTOR_KEY_MISSING/INVALID/SUSPENDED), never on status alone (second skeptic refutation on PR #1748, same failure class as the RPC-error mapping one layer up): a Vercel WAF challenge page, edge deployment protection, or a self-host egress proxy all answer 401/403 without the hosted app running, and status-trusting deletion let any of them wipe a paying instance's 72h offline grace within the hour. A codeless 401/403 now lands in the keep-grants server_error branch and the cache expires naturally if the condition persists.
+[2026-08-31] Connector usage metering redacts opaque path segments to ':id' before insert (skeptic refutation on PR #1751): the proxied bank paths carry the raw EB session id and account uid as segments, so persisting the raw pathname in connector_usage_events put the cleartext handle next to the ledger that exists precisely to store only sha256(handle). redactEndpoint() replaces UUID/long-hex/long-base64url segments; literal route words survive so metering keys stay useful.
+[2026-08-31] Bank-proxy review batch (PR #1751): EB base URL must be https (JWT in Authorization; lazy check so a bad env 500s the request, never the build); forwardToEb reads the body inside the abort-timeout window (headers-then-stall no longer wedges the request); the per-company quota uses the pending row as a reservation (pre-count, insert, re-count, roll back own row on loss) with fresh-pending rows holding quota for the 15-min consent window, closing the concurrent-auth TOCTOU without a new RPC; DELETE revokes the ledger row only on upstream success or 404 (a transient EB error no longer strands a live remote session unreachable).
+[2026-08-31] POST /sessions binds the code exchange to its state's own pending row (Superagent P1 on PR #1751): verified signature + key/service match + existing pending row are preconditions for calling EB, and a concurrent consumption of the same state after exchange closes the upstream session and answers 409 instead of handing out a session the ledger never recorded. Full code-to-state binding at the callback (recording a code hash on the pending row) would need a column and is deferred; the pre-exchange binding plus one-shot pending->active activation removes the cross-key and stateless-exchange paths, and same-key crossover between an instance's own concurrent flows only relabels its own sessions.