cbe55808868301261dd64bd17994400a0cbcd90e
5 Commits
| Author | SHA1 | Message | Date | |
|---|---|---|---|---|
|
|
f31eeaa603 |
feat(connect): Peppol through the connector (hosted proxy, instance transport, ownership ledger) (#2177)
* feat(connect): peppol connector foundation: capability, ledger/budget service, quota Adds the storage + package shape for brokering Peppol through the connector with the same one-address + rate-budget model as bank/skatteverket: a peppol capability (connector-gated, free on hosted), peppol as a ledger + upstream service, a conservative rate budget, and a migration extending the ledger service CHECK and the per-key limits (peppol_connections_per_company). Proxy route + instance-side Qvalia reroute follow. Switch-on gated on the Qvalia brokering-terms check. (cherry picked from commit 3cc0da6a3, migration renumbered 20260902190000) Signed-off-by: Jakob Wennberg <jakob.wennberg@arcim.io> Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01MMUUom4fUk6zi4xYZSSfat * feat(connect): Peppol through the connector: hosted proxy, instance transport, ownership ledger Completes the Peppol upstream for self-hosted instances on the connector (WS3): an instance with a connector key carrying the peppol scope and no Qvalia keys of its own sends and receives e-invoices through Arcim's contracted access point, the same way bank and Skatteverket already route. Hosted: app/api/connect/peppol/[...path] speaks the PeppolTransport operations (lookup, submit, status, evidence, recipient PUT/DELETE, inbound list/xml) rather than proxying Qvalia paths, because the Qvalia account is shared by every hosted company and every instance: reads must be scoped to what the caller owns, and the inbound read endpoint is destructive for the whole account. Ownership: a receiving registration is a ledger row (service peppol, participant id in account_uids, sha256 in handle_hash so one key holds a participant at a time); outbound submissions land in the new connector_peppol_submissions table and gate status/evidence; inbound documents are served from the hosted archive filtered by the participants the key holds. Per-company quota (peppol_connections_per_company), the shared PEPPOL_RECEIVING_MAX_REGISTRATIONS cap, and the global peppol rate budget apply. Provider failures cross as CONNECTOR_UPSTREAM_ERROR with the adapter's retryable flag (422 or 502). Instance: lib/invoices/transports/connector.ts implements PeppolTransport over that API and registers itself in connector mode (key present, no QVALIA_* keys); getPeppolTransportAvailability() defaults to it when no provider is selected, so an instance needs no PEPPOL_TRANSPORT_PROVIDER. Webhooks are not brokered; the existing outbound status poll covers it. Hosted is byte-identical: it has its own keys, so connector mode is never on. Docs: SELF-HOSTING.md, SOVEREIGN.md, .env.example. Switch-on for third-party instances stays gated on the Qvalia brokering-terms check; without the scope every operation answers 403. Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01MMUUom4fUk6zi4xYZSSfat Signed-off-by: Jakob Wennberg <311770904+jakobwennberg-oss@users.noreply.github.com> * fix(connect): authorize Peppol participants per key, harden the proxy after review Review follow-ups on #2177. Authorization: a key may only register (and send as) participant identifiers Arcim recorded on the key at issuance (connector_keys.peppol_participants, migration 20260902191000) or the licensee's own org number, and a document may only be submitted as a sender the key has registered; X-Connector-Company stays an opaque per-company ref. Cap: the shared access-point cap now counts fresh pending reservations and is re-checked after this request's own reservation, so concurrent registrations cannot both pass. Inbound: both halves of the participant id are filtered in the archive query (over-fetched, then exact-pair checked), so foreign rows sharing an identifier cannot consume the limit. Delete: deregistration is a required transport capability, checked before the ledger row is revoked, and registration refuses an access point that cannot deregister. Instance transport: the hosted URL must be https (loopback http only, same rule as getConnectorConfig), and the response body is read inside the timeout window with body-read failures mapped to retryable transport errors. issue-connector-key.ts gains --peppol-participants and --peppol-connections-per-company. Declined: NOT VALID on the ledger CHECK (the table is empty until keys are issued; the validated scan is instant). Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01MMUUom4fUk6zi4xYZSSfat Signed-off-by: Jakob Wennberg <311770904+jakobwennberg-oss@users.noreply.github.com> * fix(connect): bind Peppol ownership to the instance company, query exact participant pairs Second review round on #2177. Ownership is now (key, company_ref), not key alone: a sender must be registered under the same company header, status and evidence reads look the submission up under the header company, DELETE and re-registration refuse a participant the key holds for another company, so one company on a multi-company instance cannot act on another company's registration through the shared key. The instance transport resolves the owning company from its own peppol_deliveries / peppol_registrations rows before status, evidence and deregistration calls (deps.companyFor, deps.companyForParticipant, wired in transports/index.ts). Inbound listing stays key-wide (the instance routes documents to its own companies by its own registrations). The archive query now runs one exact-pair query per scheme (scheme fixed, that scheme's identifiers), so neither foreign nor cross-pair rows can consume the limit. Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01MMUUom4fUk6zi4xYZSSfat Signed-off-by: Jakob Wennberg <311770904+jakobwennberg-oss@users.noreply.github.com> --------- Signed-off-by: Jakob Wennberg <jakob.wennberg@arcim.io> Signed-off-by: Jakob Wennberg <311770904+jakobwennberg-oss@users.noreply.github.com> Co-authored-by: Jakob Wennberg <311770904+jakobwennberg-oss@users.noreply.github.com> Co-authored-by: Claude Fable 5.1 <noreply@anthropic.com> |
||
|
|
d8cf78330e |
docs(self-hosting): own-credentials section, stale connector lines, complete .env.example (#2146)
* docs(self-hosting): own-credentials section, stale connector lines, complete .env.example (#2131) SELF-HOSTING.md said the Skatteverket client wiring "ships in a following release"; PR #2103 merged it, so both bank sync and Skatteverket now carry traffic through the hosted proxy with a key. The two stale sentences are replaced and SOVEREIGN.md line 48 says the same thing. New "Own credentials (no connector key)" subsection documents the path an operator takes without a key: Enable Banking app in restricted production mode with the callback URL, the Skatteverket developer-portal application with the redirect URI, every variable the code reads, the five production base URLs (all defaults point at the test environment), the kill switch, and the rule that any own credential switches that upstream out of connector mode. .env.example gains the Skatteverket block, the optional Enable Banking variables, and RESEND_INBOUND_DOMAIN / RESEND_INBOUND_WEBHOOK_SECRET, which the invoice-inbox manifest requires but the example never listed. DOCKER.md no longer claims Enable Banking is excluded from the self-host preset (docker/extensions.self-hosted.json ships it). ENABLE_BANKING_SANDBOX is removed from the enable-banking manifest and the index.ts header: declared as optional, never read anywhere; the sandbox is selected by ENABLE_BANKING_API_URL. Logged in DECISIONS.md. Closes #2131 Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_012C6M2ZoZc6QDRxU3m9WzgE Signed-off-by: Emil <emilmattsson14@gmail.com> * docs(self-hosting): correct key format, AISP scope caveat, SKV scopes and rotation note (#2131) Skeptic findings on PR #2146, one pass: - ENABLE_BANKING_PRIVATE_KEY: the decoder base64-decodes first and wraps anything else as DER, so a raw PEM fails at JWT signing. The docs and .env.example no longer claim it is accepted. - Enable Banking restricted mode covers the operator's own accounts only; an instance hosting client companies is doing licensed AIS and needs the connector key or its own AISP registration. Said so. - Listed the OAuth scopes the app requests (both AGI scopes), noted that the kill switch gates API calls, not the BankID login, and that the token encryption key has no dual-key rotation. Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_012C6M2ZoZc6QDRxU3m9WzgE Signed-off-by: Emil <emilmattsson14@gmail.com> --------- Signed-off-by: Emil <emilmattsson14@gmail.com> Co-authored-by: Claude Fable 5.1 <noreply@anthropic.com> |
||
|
|
ca12b1855e |
fix(connect): PR #1758 CodeRabbit follow-up: connector status hardening, i18n strings, doc alignment (#2098)
* fix(connect): PR #1758 CodeRabbit follow-up: harden connector status, i18n the connector-mode strings, align docs - getConnectorConfig() rebuilds baseUrl as origin + path: userinfo, query and fragment are stripped (warn-logged without the raw value) so nothing secret-shaped pasted into GNUBOK_CONNECT_URL survives into the /api/connector/status echo or the derived proxy URLs (CWE-200) - /api/connector/status responds Cache-Control: no-store on both branches (key prefix + wiring layout out of shared browser caches, CWE-525) - CWE-319 thread verified as no-change: both connector-mode helpers derive from getConnectorConfig(), which fails closed on non-https - SkatteverketConnectPanel tooltips and BankSyncNowButton gate/upsell strings moved to messages/sv.json + messages/en.json keys - DECISIONS.md: MD037 fix on line 1146 (backtick the glob), line 1147 reworded to grants-written-wiring-pending, decision lines appended (incl. declining the UpgradeNote children-append suggestion) - docs/SOVEREIGN.md availability wording aligned with SELF-HOSTING.md: infra merged, keys issued manually on request, client wiring pending Co-Authored-By: Claude Fable 5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01UKZUp1nePr8sVDoLkMSxbS * docs(connect): skeptic follow-up: bank client wiring is merged (#2094), SKV pending, no keys issued until it lands Skeptic refutation on PR #2098: SOVEREIGN.md claimed the services 'do not carry traffic' while this branch already contains #2094 (EB client proxy routing), and 'issued manually on request' contradicted the standing no-key-before-full-PR6b rule while skatteverketConnectorMode() has no client consumer yet. SOVEREIGN.md, SELF-HOSTING.md and DECISIONS.md line 1147 now all say: bank client wiring merged and carries traffic with a key, Skatteverket client wiring ships in a following release, keys are not issued until it lands. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01UKZUp1nePr8sVDoLkMSxbS --------- Co-authored-by: Claude Fable 5 <noreply@anthropic.com> |
||
|
|
50f13cf198 |
feat(connect): self-host connector enablement: EB/SKV in the preset, connector-mode seam, status endpoint (#1758)
* feat(entitlements): partition the self-host bypass so connector capabilities fall through to grants; capability_grants.source accepts 'connector' Sovereign plan WS3 PR3: ships dark, nothing changes for hosted. - lib/entitlements/keys.ts: CONNECTOR_CAPABILITIES = bank_sync, skatteverket, org_lookup, migration (services Accounted operates that a self-hosted instance cannot provide itself) + isConnectorCapability(). Separate from PAID_CAPABILITIES and outside the trial-seed trigger on purpose: a hosted company can never hold a connector grant. - lib/entitlements/has-capability.ts: isPaywallBypassed() -> isBypassedFor(key). Hosted: byte-identical (dev / DISABLE_PAYWALL bypass, FORCE_PAYWALL wins, else the grant lookup). Self-host: local capabilities always on (FORCE_PAYWALL included, as the existing test demands); connector capabilities behave like hosted, i.e. dev bypass, FORCE_PAYWALL, else the grant lookup where the connector sync will write source='connector' rows. getCompanyEntitlements on a self-host: local paid keys + active connector keys, state 'paid' with an active connector grant else 'none' (never the hosted trial copy). - Migration 20260820122000: capability_grants.source CHECK gains 'connector', found through pg_constraint (the CHECK was declared inline and auto-named; Postgres stores IN as = ANY, matched accordingly). pg-real test: connector accepted, unknown source rejected, upsert on the (scope, key, source) identity, trial seed writes no connector rows. - Tests: self-hosted connector matrix (local all-on without DB, connector gated by grant/expiry, dev bypass all-on, FORCE_PAYWALL gates connector keys only, bulk resolution, entitlements shape); two pre-existing tests that asserted the old "self-host holds connector keys" contract updated to the new one. Verified: full unit suite green, pg-real suite for lib/entitlements green against a local supabase/postgres with every migration applied, lint ratchet, guards. Deferred to the instance-wiring PR: adding the connector extensions to the self-host Docker preset (dead-end upsells until a key can be issued). Co-Authored-By: Claude Fable 5 <noreply@anthropic.com> * refactor(entitlements): fold the self-host branch into the existing grants query One .or(scopeFilter), not two: the duplicated helper pushed the no-phantom-columns unresolvable-expression count to 380/379. Behaviour is unchanged; the self-host matrix tests still pass. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com> * feat(connect): hosted connector-key registry + validate RPC + entitlements endpoint; instance sync writes connector grants hourly Sovereign plan WS3 PR4 ("key infra enabling manual sales"), stacked on the entitlement partition (#1747). Nothing is purchasable yet; this is the plumbing both ends need before the first manually issued key. Hosted side: - Migration 20260820123000: connector_keys (SHA-256 key_hash, prefix, org_number, pinned instance_url, scopes, status, Stripe ids, current_period_end, per-minute rate limit, active_company_count, last_seen/synced) and connector_usage_events (per-request metering, separate from metered_events whose company_id references hosted companies). RLS on, NO policies: service role only. RPC validate_and_increment_connector_key copies the api_keys pattern (FOR UPDATE, minute window, suspended reported not counted, revoked = no row) and is REVOKEd from PUBLIC/anon/authenticated, GRANTed to service_role. pg-real test covers validate/count, unknown+revoked, suspended, rate limit, execute privileges per role, RLS invisibility, usage cascade. - lib/connect/contract.ts (shared wire types), lib/connect/hosted/keys.ts (generate/hash/validate -> 401/403/429 mapping), with-connector-auth.ts (Bearer or X-Connector-Key, one usage row per request, 500 envelope on handler throw), /api/connect/entitlements GET + POST (records active_company_count, pins instance_url on first report, never moves a pinned one), scripts/issue-connector-key.ts (dry run unless --confirm, prints the key once + the .env lines). Instance side: - lib/connect/instance/config.ts (GNUBOK_CONNECTOR_KEY, GNUBOK_CONNECT_URL default https://app.gnubok.se), sync.ts: reports the active company count and writes source='connector' grants for every company x covered scope, expires_at = min(now+72h, period_end+3d); 401/403 or a non-active status deletes them (freeze-and-retain); network/5xx/429 leave them alone. /api/connector/sync/cron (hourly) runs it; not_configured without a key. - Crontab generator gains EXTRA_JOBS (variant-only jobs not in vercel.json, with reasons) + drift tests; docker/crontab.self-hosted regenerated with the hourly sync. Docs (SELF-HOSTING connector section, env templates), DECISIONS. Tests: 52 new unit tests (keys, auth wrapper, route, config, sync outcomes and grant arithmetic, cron route, crontab EXTRA_JOBS) + 7 pg-real tests run locally against supabase/postgres with every migration applied. no-phantom-columns ceiling +1 with a reason (the bulk grant upsert). Co-Authored-By: Claude Fable 5 <noreply@anthropic.com> * feat(connect): Enable Banking proxy for self-hosted instances, with a secret-free ownership ledger and a global rate budget Sovereign plan WS3 PR5a, stacked on the connector-key infra (#1748). A self-hosted instance with a `bank_sync`-scoped connector key can now connect a bank through Arcim's PSD2 credentials; the bank session id and all transaction data stay in the instance's own database (founder decision: tokens on the instance, proxy stateless). - Migration 20260820124000: `connector_connections` (secret-free ledger: sha256 of the EB session id + account uids, service-role only), `connector_upstream_counters` + RPC `connector_reserve_upstream` (global budget under EB Annex 1 §5's 300/min, shared with hosted), and `connector_keys.limits` jsonb; validate RPC v2 returns limits. All RPCs REVOKEd from PUBLIC/anon/authenticated, GRANTed service_role. pg-real covers all of it. - EB JWT minting moved to lib/connect/upstreams/enable-banking-jwt.ts (core must not import @/extensions/); the extension re-exports it, tests unchanged. - lib/connect/hosted/{state,ledger,upstream-budget}.ts: HMAC-signed connector state (15-min TTL) so the consent redirect can use OUR registered EB callback and bounce back to the instance, no per-instance redirect URI at EB; the callback route gains that connector branch. - app/api/connect/bank/[...path]: path allowlist (aspsps, auth, sessions, accounts/{uid}/{balances,transactions}), never open passthrough. POST /auth enforces the per-company connection quota + rewrites redirect/state; reads/deletes verify ledger ownership; every upstream call takes the global budget (429 + Retry-After when exhausted). - issue-connector-key.ts: scopes default bank_sync,skatteverket (TIC out of v1), --bank/skv-connections-per-company + --sync-min-interval. - Docs (SELF-HOSTING: bank connector live), DECISIONS. Verified: 52 connect unit tests + 13 pg-real (run locally against supabase/postgres with all migrations) + EB extension suite (225, jwt relocation intact); full unit suite 15 979 green; tsc, guards, lint clean. Not in this PR: SKV broker (PR5b) and instance wiring (PR6). Co-Authored-By: Claude Fable 5 <noreply@anthropic.com> * feat(connect): Skatteverket broker + data proxy for self-hosted instances (tokens stay on the instance) Sovereign plan WS3 PR5b, stacked on the bank proxy (#1751). A self-hosted instance with a `skatteverket`-scoped connector key can now run the BankID consent, file VAT/AGI and sync skattekonto through Arcim's registered Skatteverket client; the SKV tokens are returned to the instance and stored (encrypted) there. - lib/connect/upstreams/skatteverket-oauth.ts: core-side SKV OAuth + data helpers (authorize URL, code/refresh exchange with Arcim's client secret, the four backing-API base URLs, the API-gateway Client_Id/Client_Secret headers). Core can't import @/extensions/, so this duplicates the extension's endpoints/scope set (one integrator = Arcim), mirroring the EB JWT relocation. - app/api/connect/skv/oauth/authorize-url: builds the authorize URL against OUR registered redirect_uri + a signed connector state, per-company SKV connection quota, pending ledger row. - app/api/connect/skv/oauth/token: exchanges/refreshes and RETURNS the tokens to the instance; the ledger keeps only sha256(access_token) + sha256(refresh_token). - app/api/connect/skv/api/[...path]: allowlist over moms / skattekonto / agd-inlamning / agd-period. The instance sends the user's SKV Bearer (as X-Connector-Upstream-Authorization) + X-Connector-Key; the proxy checks the token hash against the ledger, adds Arcim's gateway credentials (never exposed to the instance), forwards. Same per-key + global budget as bank. - The Skatteverket extension /callback gains the connector branch (isConnectorState -> 302 back to the instance; code never exchanged there). - Docs (SELF-HOSTING: SKV connector live) + DECISIONS. Tests: SKV oauth lib, authorize-url, token, data proxy, callback connector branch (all green; 74 connect + 425 connect/SKV). tsc, guards, lint clean; no-phantom-columns held at 380 (literal update branches). Not in this PR: instance-side wiring (PR6). Co-Authored-By: Claude Fable 5 <noreply@anthropic.com> * feat(connect): self-host connector enablement: EB/SKV in the preset, connector-mode seam, status endpoint Sovereign plan WS3 PR6 (enablement layer), stacked on the SKV broker (#1757). - docker/extensions.self-hosted.json += enable-banking, skatteverket: a connector-key self-host now ships the bank + Skatteverket extensions; a key with the matching scope makes them work, without one they show the existing capability_blocked upsell (unconfigured extensions no-op). - lib/connect/instance/upstreams.ts: the connector-mode seam. An upstream is in connector mode only when GNUBOK_CONNECTOR_KEY is set AND the instance has no own credentials for it (hasOwnEnableBankingCredentials / hasOwnSkatteverketCredentials). Hosted always has own credentials, so hosted is provably never in connector mode: the guard is what keeps hosted byte-identical. Base URLs GNUBOK_CONNECT_URL/api/connect/{bank,skv}, headers X-Connector-Company / X-Connector-Upstream-Authorization. - GET /api/connector/status: the operator's wiring view (self_hosted, per upstream own_credentials|connector|unconfigured, key prefix never the key, granted connector capabilities). Hosted returns self_hosted:false. - Docs (SELF-HOSTING: status endpoint + extensions ship in the image), DECISIONS. Tests: connector-mode detection matrix (off without a key, off with own creds incl. the _PRODUCTION EB variants, on via the proxy, CONNECT_URL override) + status route (self-host vs hosted, unconfigured, per-upstream mode, prefix-not-key). 83 connect/connector tests green; tsc, guards, lint. DEFERRED to PR6b (needs a live connector key + a real bank/SKV to verify end to end, touches the live consent path): wiring the EB api-client / consent callback and the SKV oauth / api-client to call the proxy in connector mode, and the "Synka nu" settings row (UI, needs visual sign-off). The seam + preset + status route make PR6b a contained follow-up. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com> * refactor(connect): upstreams seam reuses lib/entitlements/own-credentials Co-Authored-By: Claude Fable 5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01UzNkSsR18pLFitJdYn8QEb * test(connector): status route tests pass the Next params argument (post-merge withRouteContext signature) Co-Authored-By: Claude Fable 5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01UzNkSsR18pLFitJdYn8QEb * docs(self-host): collapse the re-duplicated connector section; correct the crontab generator's preset comment Co-Authored-By: Claude Fable 5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01UzNkSsR18pLFitJdYn8QEb * fix(self-host): UpgradeNote and SKV tooltip name the connector key, never the hosted subscription; SOVEREIGN.md updated to merged reality Co-Authored-By: Claude Fable 5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01UzNkSsR18pLFitJdYn8QEb * fix(self-host): BankSyncNowButton gate copy branches like UpgradeNote (connector key, not hosted billing) Co-Authored-By: Claude Fable 5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01UzNkSsR18pLFitJdYn8QEb --------- Co-authored-by: Jakob Wennberg <311770904+jakobwennberg-oss@users.noreply.github.com> Co-authored-by: Claude Fable 5 <noreply@anthropic.com> Co-authored-by: Emil <emilmattsson14@gmail.com> |
||
|
|
79edee659f |
docs(self-host): sovereign Sverige guide, backup/restore scripts, Speed Insights gate (#1744)
docs/SOVEREIGN.md (run Accounted on Swedish infrastructure: providers, self-hosted Supabase gotchas, backup/restore runbook, honest dependency list), scripts/self-host/backup.sh + restore.sh (pg_dump custom format, storage tar, SHA-256 manifest, S3-compatible upload; ACLs are preserved through the restore and re-verified against an acl-manifest including sequences; the resume hook always runs after a failed quiesce and the hooks must be configured as a pair), Vercel Speed Insights gated off for self-hosted, and stale self-host docs corrected (assistant Q&A and categorization run on BYO OpenAI-compatible models; SMTP via EMAIL_PROVIDER=smtp after #1746; connector subscription described as proposed only). |