c0ecf2fa3bebd46bdfd0169efd73b89653d1dfed
8 Commits
| Author | SHA1 | Message | Date | |
|---|---|---|---|---|
|
|
f266c386f3 |
chore: repo-wide bloat sweep, remove dead code and fold duplicate helpers (#2150)
* chore: repo-wide bloat sweep, remove dead code and fold duplicate helpers Remove 33 dead files, ~270 unreferenced exports/types, 13 dead i18n namespaces and 4 unused dependencies; fold byte-identical helper copies into one canonical home each (lib/utils chunk/sleep/utcDateStamp, lib/dates/iso, lib/invariants/uuid, lib/xml/escape, lib/reports/sru/format, lib/pdf/number-text, lib/browser/panel-request, lib/api/v1/body + v1ValidationError rolled out to ~55 v1 routes, booking-template schemas). No behaviour change: v1 bodies and status codes, MCP tool schemas, DB writes and money math are untouched. Naive ore rounding was deliberately not swapped for roundOre; see DECISIONS.md 2026-09-02 for the full list of things left alone on purpose. tsc, lint, 19588 unit tests and check:guards green; antipattern baseline ratcheted (naive-ore-round 622 -> 620, hand-rolled-invariant 115 -> 113). Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com> * test(transactions): import RawTransaction from @/types after the ingest re-export removal CI's type ratchet (check:types, full tsconfig) caught the one test file that still imported the type through lib/transactions/ingest. Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com> --------- Co-authored-by: Jakob Wennberg <311770904+jakobwennberg-oss@users.noreply.github.com> Co-authored-by: Claude Fable 5.1 <noreply@anthropic.com> |
||
|
|
08b1119c7d |
feat(connect): wire the SKV extension through the connector broker + data proxy (PR6b-2) (#2103)
* feat(connect): wire the SKV extension through the connector broker + data proxy (PR6b-2) In connector mode (GNUBOK_CONNECTOR_KEY set, no own SKV credentials) the Skatteverket extension now routes through the hosted connector stack (#1757) instead of calling Skatteverket directly: - skvRequestWithAuth routes to the data proxy: base URL maps to a service segment (moms/skattekonto/agd-inlamning/agd-period), the user's SKV Bearer moves to X-Connector-Upstream-Authorization, the connector key authenticates the proxy, and the gateway Client_Id/Client_Secret are omitted (the proxy adds Arcim's). Connector-layer 4xx bodies (code CONNECTOR_*) are classified before the SKV-shaped 401/403 sniffing so a broker refusal surfaces operator guidance (check GNUBOK_CONNECTOR_KEY), never APIGW/BankID guidance for knobs the instance does not have. - OAuth: /authorize starts the consent via the broker's authorize-url (persisting its redirect_uri + connector_state), the hosted SKV callback bounces the code back to the instance, and exchangeCodeForTokens / refreshAccessToken exchange through the broker's /oauth/token, unwrapping its { data } envelope. Tokens still rest encrypted on the instance; client_id/client_secret never exist there. - Broker refresh 404 CONNECTOR_NOT_OWNED maps to SESSION_EXPIRED (terminal; reconnect fixes); broker 502 stays a raw error so a transient SKV outage never re-arms the reconnect banner (#1155). - getSkatteverketEnvironment() reports 'prod' in connector mode: the upstream env is hosted's, and the instance's unset defaults would show a false Testmiljo badge on real filings. - System (CCG/ombud) auth is deliberately not brokered: hosted-only, stays direct. Hosted and own-credentials self-hosts are byte-identical: every branch gates on skatteverketConnectorMode(), which is null whenever own SKV credentials exist or no connector key is set. Direct-path tests pin that. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01KRfamAKDqvRNwbjr5XD2VS * fix(connect): classify SKV dead-refresh-token dialects broker-side; forward diagnostic headers; connector-aware gateway guidance Skeptic refutation on PR #2103 (found independently by the correctness and compliance skeptics): the broker's /oauth/token catch-all collapsed SKV's terminal dead-refresh-token dialects (404 id_not_found, 400 invalid_grant, "Refresh Token status is expired": the dominant refresh outcome, per-flow tokens live 65 minutes) into the generic 502 CONNECTOR_SKV_TOKEN_FAILED, so a connector instance could never classify ordinary session expiry: raw English 500s instead of the reconnect flow, staged filing operations consumed as non-recoverable, crons retrying raw forever. - Broker /oauth/token: re-codes those dialects as 401 CONNECTOR_SKV_REFRESH_DEAD, refresh grant only (invalid_grant on the code exchange means an expired one-shot code and keeps the generic 502). The classifier (isSkvDeadRefreshTokenError) uses the same regex set the extension's direct path classifies with. - Instance dead-token classifier maps CONNECTOR_SKV_REFRESH_DEAD to SESSION_EXPIRED alongside 404 CONNECTOR_NOT_OWNED; the generic 502 stays a raw error so a transient SKV outage never re-arms the reconnect banner. - Data proxy: forwards WWW-Authenticate and x-skv-*/x-amzn-*/x-api-* response headers (the instance's MISSING_SCOPE classification reads them; body-less gateway rejections carry no other signal). - Instance gateway-refusal guidance is connector-aware: a self-host has no SKATTEVERKET_APIGW_CLIENT_ID and no Utvecklarportalen access, so connector mode points at /api/connector/status and support instead. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01KRfamAKDqvRNwbjr5XD2VS * fix(connect): reject redirects on the instance's broker OAuth requests CodeRabbit inline finding (CWE-200): the connector-mode authorize-url and token requests followed redirects by default, so a 307/308 would resend the connector key (and code/refresh token) to the redirect target. redirect 'error', matching the broker's own postToken rule; the token response must only ever come from the broker endpoint itself. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01KRfamAKDqvRNwbjr5XD2VS --------- Co-authored-by: Claude Fable 5 <noreply@anthropic.com> |
||
|
|
ee22c9c7b7 |
feat(connect): connector status + Synka nu row in Settings -> Abonnemang (PR6b-3) (#2104)
* feat(connect): connector status + Synka nu row in Settings -> Abonnemang (PR6b-3) Self-host only: shows per-upstream connector mode, key prefix, and the active company's granted capabilities from GET /api/connector/status, plus a manual run of the entitlement sync via the new authed POST /api/connector/sync (requireWrite, 60s cooldown) instead of waiting for the hourly cron. Hidden on hosted. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01KAUKbGUGtAmRhpDKfn84F5 * fix(connect): handle sync fetch rejection, count capabilities not rows, not_configured toast Skeptic findings on the Synka nu flow: a rejected fetch (instance restarting) was a silent no-op with an unhandled rejection; the success toast printed grant rows (companies x scopes) as capabilities; a not_configured outcome claimed the hosted service was unreachable. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01KAUKbGUGtAmRhpDKfn84F5 --------- Co-authored-by: Claude Fable 5 <noreply@anthropic.com> |
||
|
|
ca12b1855e |
fix(connect): PR #1758 CodeRabbit follow-up: connector status hardening, i18n strings, doc alignment (#2098)
* fix(connect): PR #1758 CodeRabbit follow-up: harden connector status, i18n the connector-mode strings, align docs - getConnectorConfig() rebuilds baseUrl as origin + path: userinfo, query and fragment are stripped (warn-logged without the raw value) so nothing secret-shaped pasted into GNUBOK_CONNECT_URL survives into the /api/connector/status echo or the derived proxy URLs (CWE-200) - /api/connector/status responds Cache-Control: no-store on both branches (key prefix + wiring layout out of shared browser caches, CWE-525) - CWE-319 thread verified as no-change: both connector-mode helpers derive from getConnectorConfig(), which fails closed on non-https - SkatteverketConnectPanel tooltips and BankSyncNowButton gate/upsell strings moved to messages/sv.json + messages/en.json keys - DECISIONS.md: MD037 fix on line 1146 (backtick the glob), line 1147 reworded to grants-written-wiring-pending, decision lines appended (incl. declining the UpgradeNote children-append suggestion) - docs/SOVEREIGN.md availability wording aligned with SELF-HOSTING.md: infra merged, keys issued manually on request, client wiring pending Co-Authored-By: Claude Fable 5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01UKZUp1nePr8sVDoLkMSxbS * docs(connect): skeptic follow-up: bank client wiring is merged (#2094), SKV pending, no keys issued until it lands Skeptic refutation on PR #2098: SOVEREIGN.md claimed the services 'do not carry traffic' while this branch already contains #2094 (EB client proxy routing), and 'issued manually on request' contradicted the standing no-key-before-full-PR6b rule while skatteverketConnectorMode() has no client consumer yet. SOVEREIGN.md, SELF-HOSTING.md and DECISIONS.md line 1147 now all say: bank client wiring merged and carries traffic with a key, Skatteverket client wiring ships in a following release, keys are not issued until it lands. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01UKZUp1nePr8sVDoLkMSxbS --------- Co-authored-by: Claude Fable 5 <noreply@anthropic.com> |
||
|
|
50f13cf198 |
feat(connect): self-host connector enablement: EB/SKV in the preset, connector-mode seam, status endpoint (#1758)
* feat(entitlements): partition the self-host bypass so connector capabilities fall through to grants; capability_grants.source accepts 'connector' Sovereign plan WS3 PR3: ships dark, nothing changes for hosted. - lib/entitlements/keys.ts: CONNECTOR_CAPABILITIES = bank_sync, skatteverket, org_lookup, migration (services Accounted operates that a self-hosted instance cannot provide itself) + isConnectorCapability(). Separate from PAID_CAPABILITIES and outside the trial-seed trigger on purpose: a hosted company can never hold a connector grant. - lib/entitlements/has-capability.ts: isPaywallBypassed() -> isBypassedFor(key). Hosted: byte-identical (dev / DISABLE_PAYWALL bypass, FORCE_PAYWALL wins, else the grant lookup). Self-host: local capabilities always on (FORCE_PAYWALL included, as the existing test demands); connector capabilities behave like hosted, i.e. dev bypass, FORCE_PAYWALL, else the grant lookup where the connector sync will write source='connector' rows. getCompanyEntitlements on a self-host: local paid keys + active connector keys, state 'paid' with an active connector grant else 'none' (never the hosted trial copy). - Migration 20260820122000: capability_grants.source CHECK gains 'connector', found through pg_constraint (the CHECK was declared inline and auto-named; Postgres stores IN as = ANY, matched accordingly). pg-real test: connector accepted, unknown source rejected, upsert on the (scope, key, source) identity, trial seed writes no connector rows. - Tests: self-hosted connector matrix (local all-on without DB, connector gated by grant/expiry, dev bypass all-on, FORCE_PAYWALL gates connector keys only, bulk resolution, entitlements shape); two pre-existing tests that asserted the old "self-host holds connector keys" contract updated to the new one. Verified: full unit suite green, pg-real suite for lib/entitlements green against a local supabase/postgres with every migration applied, lint ratchet, guards. Deferred to the instance-wiring PR: adding the connector extensions to the self-host Docker preset (dead-end upsells until a key can be issued). Co-Authored-By: Claude Fable 5 <noreply@anthropic.com> * refactor(entitlements): fold the self-host branch into the existing grants query One .or(scopeFilter), not two: the duplicated helper pushed the no-phantom-columns unresolvable-expression count to 380/379. Behaviour is unchanged; the self-host matrix tests still pass. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com> * feat(connect): hosted connector-key registry + validate RPC + entitlements endpoint; instance sync writes connector grants hourly Sovereign plan WS3 PR4 ("key infra enabling manual sales"), stacked on the entitlement partition (#1747). Nothing is purchasable yet; this is the plumbing both ends need before the first manually issued key. Hosted side: - Migration 20260820123000: connector_keys (SHA-256 key_hash, prefix, org_number, pinned instance_url, scopes, status, Stripe ids, current_period_end, per-minute rate limit, active_company_count, last_seen/synced) and connector_usage_events (per-request metering, separate from metered_events whose company_id references hosted companies). RLS on, NO policies: service role only. RPC validate_and_increment_connector_key copies the api_keys pattern (FOR UPDATE, minute window, suspended reported not counted, revoked = no row) and is REVOKEd from PUBLIC/anon/authenticated, GRANTed to service_role. pg-real test covers validate/count, unknown+revoked, suspended, rate limit, execute privileges per role, RLS invisibility, usage cascade. - lib/connect/contract.ts (shared wire types), lib/connect/hosted/keys.ts (generate/hash/validate -> 401/403/429 mapping), with-connector-auth.ts (Bearer or X-Connector-Key, one usage row per request, 500 envelope on handler throw), /api/connect/entitlements GET + POST (records active_company_count, pins instance_url on first report, never moves a pinned one), scripts/issue-connector-key.ts (dry run unless --confirm, prints the key once + the .env lines). Instance side: - lib/connect/instance/config.ts (GNUBOK_CONNECTOR_KEY, GNUBOK_CONNECT_URL default https://app.gnubok.se), sync.ts: reports the active company count and writes source='connector' grants for every company x covered scope, expires_at = min(now+72h, period_end+3d); 401/403 or a non-active status deletes them (freeze-and-retain); network/5xx/429 leave them alone. /api/connector/sync/cron (hourly) runs it; not_configured without a key. - Crontab generator gains EXTRA_JOBS (variant-only jobs not in vercel.json, with reasons) + drift tests; docker/crontab.self-hosted regenerated with the hourly sync. Docs (SELF-HOSTING connector section, env templates), DECISIONS. Tests: 52 new unit tests (keys, auth wrapper, route, config, sync outcomes and grant arithmetic, cron route, crontab EXTRA_JOBS) + 7 pg-real tests run locally against supabase/postgres with every migration applied. no-phantom-columns ceiling +1 with a reason (the bulk grant upsert). Co-Authored-By: Claude Fable 5 <noreply@anthropic.com> * feat(connect): Enable Banking proxy for self-hosted instances, with a secret-free ownership ledger and a global rate budget Sovereign plan WS3 PR5a, stacked on the connector-key infra (#1748). A self-hosted instance with a `bank_sync`-scoped connector key can now connect a bank through Arcim's PSD2 credentials; the bank session id and all transaction data stay in the instance's own database (founder decision: tokens on the instance, proxy stateless). - Migration 20260820124000: `connector_connections` (secret-free ledger: sha256 of the EB session id + account uids, service-role only), `connector_upstream_counters` + RPC `connector_reserve_upstream` (global budget under EB Annex 1 §5's 300/min, shared with hosted), and `connector_keys.limits` jsonb; validate RPC v2 returns limits. All RPCs REVOKEd from PUBLIC/anon/authenticated, GRANTed service_role. pg-real covers all of it. - EB JWT minting moved to lib/connect/upstreams/enable-banking-jwt.ts (core must not import @/extensions/); the extension re-exports it, tests unchanged. - lib/connect/hosted/{state,ledger,upstream-budget}.ts: HMAC-signed connector state (15-min TTL) so the consent redirect can use OUR registered EB callback and bounce back to the instance, no per-instance redirect URI at EB; the callback route gains that connector branch. - app/api/connect/bank/[...path]: path allowlist (aspsps, auth, sessions, accounts/{uid}/{balances,transactions}), never open passthrough. POST /auth enforces the per-company connection quota + rewrites redirect/state; reads/deletes verify ledger ownership; every upstream call takes the global budget (429 + Retry-After when exhausted). - issue-connector-key.ts: scopes default bank_sync,skatteverket (TIC out of v1), --bank/skv-connections-per-company + --sync-min-interval. - Docs (SELF-HOSTING: bank connector live), DECISIONS. Verified: 52 connect unit tests + 13 pg-real (run locally against supabase/postgres with all migrations) + EB extension suite (225, jwt relocation intact); full unit suite 15 979 green; tsc, guards, lint clean. Not in this PR: SKV broker (PR5b) and instance wiring (PR6). Co-Authored-By: Claude Fable 5 <noreply@anthropic.com> * feat(connect): Skatteverket broker + data proxy for self-hosted instances (tokens stay on the instance) Sovereign plan WS3 PR5b, stacked on the bank proxy (#1751). A self-hosted instance with a `skatteverket`-scoped connector key can now run the BankID consent, file VAT/AGI and sync skattekonto through Arcim's registered Skatteverket client; the SKV tokens are returned to the instance and stored (encrypted) there. - lib/connect/upstreams/skatteverket-oauth.ts: core-side SKV OAuth + data helpers (authorize URL, code/refresh exchange with Arcim's client secret, the four backing-API base URLs, the API-gateway Client_Id/Client_Secret headers). Core can't import @/extensions/, so this duplicates the extension's endpoints/scope set (one integrator = Arcim), mirroring the EB JWT relocation. - app/api/connect/skv/oauth/authorize-url: builds the authorize URL against OUR registered redirect_uri + a signed connector state, per-company SKV connection quota, pending ledger row. - app/api/connect/skv/oauth/token: exchanges/refreshes and RETURNS the tokens to the instance; the ledger keeps only sha256(access_token) + sha256(refresh_token). - app/api/connect/skv/api/[...path]: allowlist over moms / skattekonto / agd-inlamning / agd-period. The instance sends the user's SKV Bearer (as X-Connector-Upstream-Authorization) + X-Connector-Key; the proxy checks the token hash against the ledger, adds Arcim's gateway credentials (never exposed to the instance), forwards. Same per-key + global budget as bank. - The Skatteverket extension /callback gains the connector branch (isConnectorState -> 302 back to the instance; code never exchanged there). - Docs (SELF-HOSTING: SKV connector live) + DECISIONS. Tests: SKV oauth lib, authorize-url, token, data proxy, callback connector branch (all green; 74 connect + 425 connect/SKV). tsc, guards, lint clean; no-phantom-columns held at 380 (literal update branches). Not in this PR: instance-side wiring (PR6). Co-Authored-By: Claude Fable 5 <noreply@anthropic.com> * feat(connect): self-host connector enablement: EB/SKV in the preset, connector-mode seam, status endpoint Sovereign plan WS3 PR6 (enablement layer), stacked on the SKV broker (#1757). - docker/extensions.self-hosted.json += enable-banking, skatteverket: a connector-key self-host now ships the bank + Skatteverket extensions; a key with the matching scope makes them work, without one they show the existing capability_blocked upsell (unconfigured extensions no-op). - lib/connect/instance/upstreams.ts: the connector-mode seam. An upstream is in connector mode only when GNUBOK_CONNECTOR_KEY is set AND the instance has no own credentials for it (hasOwnEnableBankingCredentials / hasOwnSkatteverketCredentials). Hosted always has own credentials, so hosted is provably never in connector mode: the guard is what keeps hosted byte-identical. Base URLs GNUBOK_CONNECT_URL/api/connect/{bank,skv}, headers X-Connector-Company / X-Connector-Upstream-Authorization. - GET /api/connector/status: the operator's wiring view (self_hosted, per upstream own_credentials|connector|unconfigured, key prefix never the key, granted connector capabilities). Hosted returns self_hosted:false. - Docs (SELF-HOSTING: status endpoint + extensions ship in the image), DECISIONS. Tests: connector-mode detection matrix (off without a key, off with own creds incl. the _PRODUCTION EB variants, on via the proxy, CONNECT_URL override) + status route (self-host vs hosted, unconfigured, per-upstream mode, prefix-not-key). 83 connect/connector tests green; tsc, guards, lint. DEFERRED to PR6b (needs a live connector key + a real bank/SKV to verify end to end, touches the live consent path): wiring the EB api-client / consent callback and the SKV oauth / api-client to call the proxy in connector mode, and the "Synka nu" settings row (UI, needs visual sign-off). The seam + preset + status route make PR6b a contained follow-up. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com> * refactor(connect): upstreams seam reuses lib/entitlements/own-credentials Co-Authored-By: Claude Fable 5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01UzNkSsR18pLFitJdYn8QEb * test(connector): status route tests pass the Next params argument (post-merge withRouteContext signature) Co-Authored-By: Claude Fable 5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01UzNkSsR18pLFitJdYn8QEb * docs(self-host): collapse the re-duplicated connector section; correct the crontab generator's preset comment Co-Authored-By: Claude Fable 5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01UzNkSsR18pLFitJdYn8QEb * fix(self-host): UpgradeNote and SKV tooltip name the connector key, never the hosted subscription; SOVEREIGN.md updated to merged reality Co-Authored-By: Claude Fable 5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01UzNkSsR18pLFitJdYn8QEb * fix(self-host): BankSyncNowButton gate copy branches like UpgradeNote (connector key, not hosted billing) Co-Authored-By: Claude Fable 5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01UzNkSsR18pLFitJdYn8QEb --------- Co-authored-by: Jakob Wennberg <311770904+jakobwennberg-oss@users.noreply.github.com> Co-authored-by: Claude Fable 5 <noreply@anthropic.com> Co-authored-by: Emil <emilmattsson14@gmail.com> |
||
|
|
37e50c272d |
feat(connect): Skatteverket broker + data proxy for self-hosted instances (tokens stay on the instance) (#1757)
* feat(entitlements): partition the self-host bypass so connector capabilities fall through to grants; capability_grants.source accepts 'connector' Sovereign plan WS3 PR3: ships dark, nothing changes for hosted. - lib/entitlements/keys.ts: CONNECTOR_CAPABILITIES = bank_sync, skatteverket, org_lookup, migration (services Accounted operates that a self-hosted instance cannot provide itself) + isConnectorCapability(). Separate from PAID_CAPABILITIES and outside the trial-seed trigger on purpose: a hosted company can never hold a connector grant. - lib/entitlements/has-capability.ts: isPaywallBypassed() -> isBypassedFor(key). Hosted: byte-identical (dev / DISABLE_PAYWALL bypass, FORCE_PAYWALL wins, else the grant lookup). Self-host: local capabilities always on (FORCE_PAYWALL included, as the existing test demands); connector capabilities behave like hosted, i.e. dev bypass, FORCE_PAYWALL, else the grant lookup where the connector sync will write source='connector' rows. getCompanyEntitlements on a self-host: local paid keys + active connector keys, state 'paid' with an active connector grant else 'none' (never the hosted trial copy). - Migration 20260820122000: capability_grants.source CHECK gains 'connector', found through pg_constraint (the CHECK was declared inline and auto-named; Postgres stores IN as = ANY, matched accordingly). pg-real test: connector accepted, unknown source rejected, upsert on the (scope, key, source) identity, trial seed writes no connector rows. - Tests: self-hosted connector matrix (local all-on without DB, connector gated by grant/expiry, dev bypass all-on, FORCE_PAYWALL gates connector keys only, bulk resolution, entitlements shape); two pre-existing tests that asserted the old "self-host holds connector keys" contract updated to the new one. Verified: full unit suite green, pg-real suite for lib/entitlements green against a local supabase/postgres with every migration applied, lint ratchet, guards. Deferred to the instance-wiring PR: adding the connector extensions to the self-host Docker preset (dead-end upsells until a key can be issued). Co-Authored-By: Claude Fable 5 <noreply@anthropic.com> * refactor(entitlements): fold the self-host branch into the existing grants query One .or(scopeFilter), not two: the duplicated helper pushed the no-phantom-columns unresolvable-expression count to 380/379. Behaviour is unchanged; the self-host matrix tests still pass. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com> * feat(connect): hosted connector-key registry + validate RPC + entitlements endpoint; instance sync writes connector grants hourly Sovereign plan WS3 PR4 ("key infra enabling manual sales"), stacked on the entitlement partition (#1747). Nothing is purchasable yet; this is the plumbing both ends need before the first manually issued key. Hosted side: - Migration 20260820123000: connector_keys (SHA-256 key_hash, prefix, org_number, pinned instance_url, scopes, status, Stripe ids, current_period_end, per-minute rate limit, active_company_count, last_seen/synced) and connector_usage_events (per-request metering, separate from metered_events whose company_id references hosted companies). RLS on, NO policies: service role only. RPC validate_and_increment_connector_key copies the api_keys pattern (FOR UPDATE, minute window, suspended reported not counted, revoked = no row) and is REVOKEd from PUBLIC/anon/authenticated, GRANTed to service_role. pg-real test covers validate/count, unknown+revoked, suspended, rate limit, execute privileges per role, RLS invisibility, usage cascade. - lib/connect/contract.ts (shared wire types), lib/connect/hosted/keys.ts (generate/hash/validate -> 401/403/429 mapping), with-connector-auth.ts (Bearer or X-Connector-Key, one usage row per request, 500 envelope on handler throw), /api/connect/entitlements GET + POST (records active_company_count, pins instance_url on first report, never moves a pinned one), scripts/issue-connector-key.ts (dry run unless --confirm, prints the key once + the .env lines). Instance side: - lib/connect/instance/config.ts (GNUBOK_CONNECTOR_KEY, GNUBOK_CONNECT_URL default https://app.gnubok.se), sync.ts: reports the active company count and writes source='connector' grants for every company x covered scope, expires_at = min(now+72h, period_end+3d); 401/403 or a non-active status deletes them (freeze-and-retain); network/5xx/429 leave them alone. /api/connector/sync/cron (hourly) runs it; not_configured without a key. - Crontab generator gains EXTRA_JOBS (variant-only jobs not in vercel.json, with reasons) + drift tests; docker/crontab.self-hosted regenerated with the hourly sync. Docs (SELF-HOSTING connector section, env templates), DECISIONS. Tests: 52 new unit tests (keys, auth wrapper, route, config, sync outcomes and grant arithmetic, cron route, crontab EXTRA_JOBS) + 7 pg-real tests run locally against supabase/postgres with every migration applied. no-phantom-columns ceiling +1 with a reason (the bulk grant upsert). Co-Authored-By: Claude Fable 5 <noreply@anthropic.com> * feat(connect): Enable Banking proxy for self-hosted instances, with a secret-free ownership ledger and a global rate budget Sovereign plan WS3 PR5a, stacked on the connector-key infra (#1748). A self-hosted instance with a `bank_sync`-scoped connector key can now connect a bank through Arcim's PSD2 credentials; the bank session id and all transaction data stay in the instance's own database (founder decision: tokens on the instance, proxy stateless). - Migration 20260820124000: `connector_connections` (secret-free ledger: sha256 of the EB session id + account uids, service-role only), `connector_upstream_counters` + RPC `connector_reserve_upstream` (global budget under EB Annex 1 §5's 300/min, shared with hosted), and `connector_keys.limits` jsonb; validate RPC v2 returns limits. All RPCs REVOKEd from PUBLIC/anon/authenticated, GRANTed service_role. pg-real covers all of it. - EB JWT minting moved to lib/connect/upstreams/enable-banking-jwt.ts (core must not import @/extensions/); the extension re-exports it, tests unchanged. - lib/connect/hosted/{state,ledger,upstream-budget}.ts: HMAC-signed connector state (15-min TTL) so the consent redirect can use OUR registered EB callback and bounce back to the instance, no per-instance redirect URI at EB; the callback route gains that connector branch. - app/api/connect/bank/[...path]: path allowlist (aspsps, auth, sessions, accounts/{uid}/{balances,transactions}), never open passthrough. POST /auth enforces the per-company connection quota + rewrites redirect/state; reads/deletes verify ledger ownership; every upstream call takes the global budget (429 + Retry-After when exhausted). - issue-connector-key.ts: scopes default bank_sync,skatteverket (TIC out of v1), --bank/skv-connections-per-company + --sync-min-interval. - Docs (SELF-HOSTING: bank connector live), DECISIONS. Verified: 52 connect unit tests + 13 pg-real (run locally against supabase/postgres with all migrations) + EB extension suite (225, jwt relocation intact); full unit suite 15 979 green; tsc, guards, lint clean. Not in this PR: SKV broker (PR5b) and instance wiring (PR6). Co-Authored-By: Claude Fable 5 <noreply@anthropic.com> * feat(connect): Skatteverket broker + data proxy for self-hosted instances (tokens stay on the instance) Sovereign plan WS3 PR5b, stacked on the bank proxy (#1751). A self-hosted instance with a `skatteverket`-scoped connector key can now run the BankID consent, file VAT/AGI and sync skattekonto through Arcim's registered Skatteverket client; the SKV tokens are returned to the instance and stored (encrypted) there. - lib/connect/upstreams/skatteverket-oauth.ts: core-side SKV OAuth + data helpers (authorize URL, code/refresh exchange with Arcim's client secret, the four backing-API base URLs, the API-gateway Client_Id/Client_Secret headers). Core can't import @/extensions/, so this duplicates the extension's endpoints/scope set (one integrator = Arcim), mirroring the EB JWT relocation. - app/api/connect/skv/oauth/authorize-url: builds the authorize URL against OUR registered redirect_uri + a signed connector state, per-company SKV connection quota, pending ledger row. - app/api/connect/skv/oauth/token: exchanges/refreshes and RETURNS the tokens to the instance; the ledger keeps only sha256(access_token) + sha256(refresh_token). - app/api/connect/skv/api/[...path]: allowlist over moms / skattekonto / agd-inlamning / agd-period. The instance sends the user's SKV Bearer (as X-Connector-Upstream-Authorization) + X-Connector-Key; the proxy checks the token hash against the ledger, adds Arcim's gateway credentials (never exposed to the instance), forwards. Same per-key + global budget as bank. - The Skatteverket extension /callback gains the connector branch (isConnectorState -> 302 back to the instance; code never exchanged there). - Docs (SELF-HOSTING: SKV connector live) + DECISIONS. Tests: SKV oauth lib, authorize-url, token, data proxy, callback connector branch (all green; 74 connect + 425 connect/SKV). tsc, guards, lint clean; no-phantom-columns held at 380 (literal update branches). Not in this PR: instance-side wiring (PR6). Co-Authored-By: Claude Fable 5 <noreply@anthropic.com> * fix(connect): SKV broker review+skeptic batch: state-bound exchange, owned-only refresh, identity-number redaction, quota reservation, https-only bases, docs dedupe Co-Authored-By: Claude Fable 5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01UzNkSsR18pLFitJdYn8QEb * docs(self-host): restore the instance-wiring qualifier in the connector section Co-Authored-By: Claude Fable 5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01UzNkSsR18pLFitJdYn8QEb * fix(connect): PR #1757 review batch 2: redirect 'error' on credential fetches, mandatory ledger writes before token return, MD037 Co-Authored-By: Claude Fable 5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01UzNkSsR18pLFitJdYn8QEb * fix(connect): reject encoded path separators in SKV data-proxy segments (traversal guard) The WHATWG parser normalizes raw dot segments before the route runs; what survives is an encoded separator inside a segment (a%2Fb, ..%2Fx, a%5Cb), which would escape the allowlisted service base once the upstream fetch re-normalizes. splitPath now decodes each segment and rejects dot segments and separator-bearing values. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01UzNkSsR18pLFitJdYn8QEb --------- Co-authored-by: Jakob Wennberg <311770904+jakobwennberg-oss@users.noreply.github.com> Co-authored-by: Claude Fable 5 <noreply@anthropic.com> Co-authored-by: Emil <emilmattsson14@gmail.com> |
||
|
|
36123cef23 |
feat(connect): Enable Banking proxy for self-hosted instances, with a secret-free ownership ledger and a global rate budget (#1751)
* feat(entitlements): partition the self-host bypass so connector capabilities fall through to grants; capability_grants.source accepts 'connector' Sovereign plan WS3 PR3: ships dark, nothing changes for hosted. - lib/entitlements/keys.ts: CONNECTOR_CAPABILITIES = bank_sync, skatteverket, org_lookup, migration (services Accounted operates that a self-hosted instance cannot provide itself) + isConnectorCapability(). Separate from PAID_CAPABILITIES and outside the trial-seed trigger on purpose: a hosted company can never hold a connector grant. - lib/entitlements/has-capability.ts: isPaywallBypassed() -> isBypassedFor(key). Hosted: byte-identical (dev / DISABLE_PAYWALL bypass, FORCE_PAYWALL wins, else the grant lookup). Self-host: local capabilities always on (FORCE_PAYWALL included, as the existing test demands); connector capabilities behave like hosted, i.e. dev bypass, FORCE_PAYWALL, else the grant lookup where the connector sync will write source='connector' rows. getCompanyEntitlements on a self-host: local paid keys + active connector keys, state 'paid' with an active connector grant else 'none' (never the hosted trial copy). - Migration 20260820122000: capability_grants.source CHECK gains 'connector', found through pg_constraint (the CHECK was declared inline and auto-named; Postgres stores IN as = ANY, matched accordingly). pg-real test: connector accepted, unknown source rejected, upsert on the (scope, key, source) identity, trial seed writes no connector rows. - Tests: self-hosted connector matrix (local all-on without DB, connector gated by grant/expiry, dev bypass all-on, FORCE_PAYWALL gates connector keys only, bulk resolution, entitlements shape); two pre-existing tests that asserted the old "self-host holds connector keys" contract updated to the new one. Verified: full unit suite green, pg-real suite for lib/entitlements green against a local supabase/postgres with every migration applied, lint ratchet, guards. Deferred to the instance-wiring PR: adding the connector extensions to the self-host Docker preset (dead-end upsells until a key can be issued). Co-Authored-By: Claude Fable 5 <noreply@anthropic.com> * refactor(entitlements): fold the self-host branch into the existing grants query One .or(scopeFilter), not two: the duplicated helper pushed the no-phantom-columns unresolvable-expression count to 380/379. Behaviour is unchanged; the self-host matrix tests still pass. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com> * feat(connect): hosted connector-key registry + validate RPC + entitlements endpoint; instance sync writes connector grants hourly Sovereign plan WS3 PR4 ("key infra enabling manual sales"), stacked on the entitlement partition (#1747). Nothing is purchasable yet; this is the plumbing both ends need before the first manually issued key. Hosted side: - Migration 20260820123000: connector_keys (SHA-256 key_hash, prefix, org_number, pinned instance_url, scopes, status, Stripe ids, current_period_end, per-minute rate limit, active_company_count, last_seen/synced) and connector_usage_events (per-request metering, separate from metered_events whose company_id references hosted companies). RLS on, NO policies: service role only. RPC validate_and_increment_connector_key copies the api_keys pattern (FOR UPDATE, minute window, suspended reported not counted, revoked = no row) and is REVOKEd from PUBLIC/anon/authenticated, GRANTed to service_role. pg-real test covers validate/count, unknown+revoked, suspended, rate limit, execute privileges per role, RLS invisibility, usage cascade. - lib/connect/contract.ts (shared wire types), lib/connect/hosted/keys.ts (generate/hash/validate -> 401/403/429 mapping), with-connector-auth.ts (Bearer or X-Connector-Key, one usage row per request, 500 envelope on handler throw), /api/connect/entitlements GET + POST (records active_company_count, pins instance_url on first report, never moves a pinned one), scripts/issue-connector-key.ts (dry run unless --confirm, prints the key once + the .env lines). Instance side: - lib/connect/instance/config.ts (GNUBOK_CONNECTOR_KEY, GNUBOK_CONNECT_URL default https://app.gnubok.se), sync.ts: reports the active company count and writes source='connector' grants for every company x covered scope, expires_at = min(now+72h, period_end+3d); 401/403 or a non-active status deletes them (freeze-and-retain); network/5xx/429 leave them alone. /api/connector/sync/cron (hourly) runs it; not_configured without a key. - Crontab generator gains EXTRA_JOBS (variant-only jobs not in vercel.json, with reasons) + drift tests; docker/crontab.self-hosted regenerated with the hourly sync. Docs (SELF-HOSTING connector section, env templates), DECISIONS. Tests: 52 new unit tests (keys, auth wrapper, route, config, sync outcomes and grant arithmetic, cron route, crontab EXTRA_JOBS) + 7 pg-real tests run locally against supabase/postgres with every migration applied. no-phantom-columns ceiling +1 with a reason (the bulk grant upsert). Co-Authored-By: Claude Fable 5 <noreply@anthropic.com> * feat(connect): Enable Banking proxy for self-hosted instances, with a secret-free ownership ledger and a global rate budget Sovereign plan WS3 PR5a, stacked on the connector-key infra (#1748). A self-hosted instance with a `bank_sync`-scoped connector key can now connect a bank through Arcim's PSD2 credentials; the bank session id and all transaction data stay in the instance's own database (founder decision: tokens on the instance, proxy stateless). - Migration 20260820124000: `connector_connections` (secret-free ledger: sha256 of the EB session id + account uids, service-role only), `connector_upstream_counters` + RPC `connector_reserve_upstream` (global budget under EB Annex 1 §5's 300/min, shared with hosted), and `connector_keys.limits` jsonb; validate RPC v2 returns limits. All RPCs REVOKEd from PUBLIC/anon/authenticated, GRANTed service_role. pg-real covers all of it. - EB JWT minting moved to lib/connect/upstreams/enable-banking-jwt.ts (core must not import @/extensions/); the extension re-exports it, tests unchanged. - lib/connect/hosted/{state,ledger,upstream-budget}.ts: HMAC-signed connector state (15-min TTL) so the consent redirect can use OUR registered EB callback and bounce back to the instance, no per-instance redirect URI at EB; the callback route gains that connector branch. - app/api/connect/bank/[...path]: path allowlist (aspsps, auth, sessions, accounts/{uid}/{balances,transactions}), never open passthrough. POST /auth enforces the per-company connection quota + rewrites redirect/state; reads/deletes verify ledger ownership; every upstream call takes the global budget (429 + Retry-After when exhausted). - issue-connector-key.ts: scopes default bank_sync,skatteverket (TIC out of v1), --bank/skv-connections-per-company + --sync-min-interval. - Docs (SELF-HOSTING: bank connector live), DECISIONS. Verified: 52 connect unit tests + 13 pg-real (run locally against supabase/postgres with all migrations) + EB extension suite (225, jwt relocation intact); full unit suite 15 979 green; tsc, guards, lint clean. Not in this PR: SKV broker (PR5b) and instance wiring (PR6). Co-Authored-By: Claude Fable 5 <noreply@anthropic.com> * chore(connect): update ledger pg test to re-versioned migration 20260831200000 Co-Authored-By: Claude Fable 5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01UzNkSsR18pLFitJdYn8QEb * fix(connect): redact opaque path segments before usage metering; correct stale RPC-source comment GET/DELETE /sessions/{id} and /accounts/{uid}/... carry the raw EB session id / account uid in the pathname; metering persisted it in cleartext next to the ledger that stores only sha256(handle). Opaque segments (UUID, long hex, long base64url) now become ':id' before the connector_usage_events insert. Migration comment now cites the real prior RPC source (20260831190000). Co-Authored-By: Claude Fable 5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01UzNkSsR18pLFitJdYn8QEb * fix(connect): percent-encoded path segments count as opaque in metering redaction Co-Authored-By: Claude Fable 5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01UzNkSsR18pLFitJdYn8QEb * fix(connect): PR #1751 review batch: https-only EB URL, body-covering timeout, quota reservation, delete-after-success, doc fix Co-Authored-By: Claude Fable 5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01UzNkSsR18pLFitJdYn8QEb * fix(connect): bind the /sessions code exchange to its verified pending state; ceiling +1 Verified state signature, key/service match, and an existing pending row now precede the EB exchange; a concurrently consumed state closes the just-minted upstream session and 409s. no-phantom-columns ceiling 391 for countHeldConnections' computed .or() timestamp filter. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01UzNkSsR18pLFitJdYn8QEb --------- Co-authored-by: Jakob Wennberg <311770904+jakobwennberg-oss@users.noreply.github.com> Co-authored-by: Claude Fable 5 <noreply@anthropic.com> Co-authored-by: Emil <emilmattsson14@gmail.com> |
||
|
|
0ff1b05553 |
feat(connect): hosted connector-key registry + validate RPC + entitlements endpoint; instance sync writes connector grants hourly (#1748)
* feat(entitlements): partition the self-host bypass so connector capabilities fall through to grants; capability_grants.source accepts 'connector' Sovereign plan WS3 PR3: ships dark, nothing changes for hosted. - lib/entitlements/keys.ts: CONNECTOR_CAPABILITIES = bank_sync, skatteverket, org_lookup, migration (services Accounted operates that a self-hosted instance cannot provide itself) + isConnectorCapability(). Separate from PAID_CAPABILITIES and outside the trial-seed trigger on purpose: a hosted company can never hold a connector grant. - lib/entitlements/has-capability.ts: isPaywallBypassed() -> isBypassedFor(key). Hosted: byte-identical (dev / DISABLE_PAYWALL bypass, FORCE_PAYWALL wins, else the grant lookup). Self-host: local capabilities always on (FORCE_PAYWALL included, as the existing test demands); connector capabilities behave like hosted, i.e. dev bypass, FORCE_PAYWALL, else the grant lookup where the connector sync will write source='connector' rows. getCompanyEntitlements on a self-host: local paid keys + active connector keys, state 'paid' with an active connector grant else 'none' (never the hosted trial copy). - Migration 20260820122000: capability_grants.source CHECK gains 'connector', found through pg_constraint (the CHECK was declared inline and auto-named; Postgres stores IN as = ANY, matched accordingly). pg-real test: connector accepted, unknown source rejected, upsert on the (scope, key, source) identity, trial seed writes no connector rows. - Tests: self-hosted connector matrix (local all-on without DB, connector gated by grant/expiry, dev bypass all-on, FORCE_PAYWALL gates connector keys only, bulk resolution, entitlements shape); two pre-existing tests that asserted the old "self-host holds connector keys" contract updated to the new one. Verified: full unit suite green, pg-real suite for lib/entitlements green against a local supabase/postgres with every migration applied, lint ratchet, guards. Deferred to the instance-wiring PR: adding the connector extensions to the self-host Docker preset (dead-end upsells until a key can be issued). Co-Authored-By: Claude Fable 5 <noreply@anthropic.com> * refactor(entitlements): fold the self-host branch into the existing grants query One .or(scopeFilter), not two: the duplicated helper pushed the no-phantom-columns unresolvable-expression count to 380/379. Behaviour is unchanged; the self-host matrix tests still pass. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com> * feat(connect): hosted connector-key registry + validate RPC + entitlements endpoint; instance sync writes connector grants hourly Sovereign plan WS3 PR4 ("key infra enabling manual sales"), stacked on the entitlement partition (#1747). Nothing is purchasable yet; this is the plumbing both ends need before the first manually issued key. Hosted side: - Migration 20260820123000: connector_keys (SHA-256 key_hash, prefix, org_number, pinned instance_url, scopes, status, Stripe ids, current_period_end, per-minute rate limit, active_company_count, last_seen/synced) and connector_usage_events (per-request metering, separate from metered_events whose company_id references hosted companies). RLS on, NO policies: service role only. RPC validate_and_increment_connector_key copies the api_keys pattern (FOR UPDATE, minute window, suspended reported not counted, revoked = no row) and is REVOKEd from PUBLIC/anon/authenticated, GRANTed to service_role. pg-real test covers validate/count, unknown+revoked, suspended, rate limit, execute privileges per role, RLS invisibility, usage cascade. - lib/connect/contract.ts (shared wire types), lib/connect/hosted/keys.ts (generate/hash/validate -> 401/403/429 mapping), with-connector-auth.ts (Bearer or X-Connector-Key, one usage row per request, 500 envelope on handler throw), /api/connect/entitlements GET + POST (records active_company_count, pins instance_url on first report, never moves a pinned one), scripts/issue-connector-key.ts (dry run unless --confirm, prints the key once + the .env lines). Instance side: - lib/connect/instance/config.ts (GNUBOK_CONNECTOR_KEY, GNUBOK_CONNECT_URL default https://app.gnubok.se), sync.ts: reports the active company count and writes source='connector' grants for every company x covered scope, expires_at = min(now+72h, period_end+3d); 401/403 or a non-active status deletes them (freeze-and-retain); network/5xx/429 leave them alone. /api/connector/sync/cron (hourly) runs it; not_configured without a key. - Crontab generator gains EXTRA_JOBS (variant-only jobs not in vercel.json, with reasons) + drift tests; docker/crontab.self-hosted regenerated with the hourly sync. Docs (SELF-HOSTING connector section, env templates), DECISIONS. Tests: 52 new unit tests (keys, auth wrapper, route, config, sync outcomes and grant arithmetic, cron route, crontab EXTRA_JOBS) + 7 pg-real tests run locally against supabase/postgres with every migration applied. no-phantom-columns ceiling +1 with a reason (the bulk grant upsert). Co-Authored-By: Claude Fable 5 <noreply@anthropic.com> * chore(connect): update pg test to re-versioned migration 20260831190000 Co-Authored-By: Claude Fable 5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01UzNkSsR18pLFitJdYn8QEb * fix(connect): RPC errors answer 503 not 401; X-Connector-Key wins over Authorization A hosted DB error mapped to 401 made the instance sync treat a pooler blip as key revocation and delete its entire connector grant cache, zeroing the 72h offline grace. 503 lands in the sync's keep-grants branch (already test-pinned). Bearer-first extraction hashed the upstream token on dual-header proxied calls, 401ing the exact shape X-Connector-Key exists for. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01UzNkSsR18pLFitJdYn8QEb * fix(connect): sync deletes grants only on a body-proven connector rejection, never bare 401/403 A WAF challenge page, edge deployment protection, or an egress proxy answers 401/403 without the hosted app ever running; trusting status alone wiped the instance's 72h offline grant cache within the hour. Deletion now requires the hosted route's own rejection code in the JSON body; codeless 401/403 keeps grants (server_error branch). Co-Authored-By: Claude Fable 5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01UzNkSsR18pLFitJdYn8QEb * fix(connect): PR #1748 review batch: https-only connect URL, atomic pin, prefix-gated Bearer, deferred metering, entitlements validation, integer months - GNUBOK_CONNECT_URL must be https (http only for loopback); invalid or plaintext URLs disable the connector instead of sending the key. - instance_url pin update filters on IS NULL; a lost race re-reads and reports the winner's pin. - extractConnectorKey: a Bearer is the connector credential only with the gnubok_ck_ prefix; upstream Bearer falls through to X-Connector-Key. - Usage metering runs via after() off the response path (inline outside a request scope). - Sync validates entitlements shape: unknown status or malformed current_period_end keeps grants (server_error), never deletes. - issue-connector-key rejects fractional --months. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01UzNkSsR18pLFitJdYn8QEb --------- Co-authored-by: Jakob Wennberg <311770904+jakobwennberg-oss@users.noreply.github.com> Co-authored-by: Claude Fable 5 <noreply@anthropic.com> Co-authored-by: Emil <emilmattsson14@gmail.com> |