feat(connect): hosted connector-key registry + validate RPC + entitlements endpoint; instance sync writes connector grants hourly (#1748)
* feat(entitlements): partition the self-host bypass so connector capabilities fall through to grants; capability_grants.source accepts 'connector' Sovereign plan WS3 PR3: ships dark, nothing changes for hosted. - lib/entitlements/keys.ts: CONNECTOR_CAPABILITIES = bank_sync, skatteverket, org_lookup, migration (services Accounted operates that a self-hosted instance cannot provide itself) + isConnectorCapability(). Separate from PAID_CAPABILITIES and outside the trial-seed trigger on purpose: a hosted company can never hold a connector grant. - lib/entitlements/has-capability.ts: isPaywallBypassed() -> isBypassedFor(key). Hosted: byte-identical (dev / DISABLE_PAYWALL bypass, FORCE_PAYWALL wins, else the grant lookup). Self-host: local capabilities always on (FORCE_PAYWALL included, as the existing test demands); connector capabilities behave like hosted, i.e. dev bypass, FORCE_PAYWALL, else the grant lookup where the connector sync will write source='connector' rows. getCompanyEntitlements on a self-host: local paid keys + active connector keys, state 'paid' with an active connector grant else 'none' (never the hosted trial copy). - Migration 20260820122000: capability_grants.source CHECK gains 'connector', found through pg_constraint (the CHECK was declared inline and auto-named; Postgres stores IN as = ANY, matched accordingly). pg-real test: connector accepted, unknown source rejected, upsert on the (scope, key, source) identity, trial seed writes no connector rows. - Tests: self-hosted connector matrix (local all-on without DB, connector gated by grant/expiry, dev bypass all-on, FORCE_PAYWALL gates connector keys only, bulk resolution, entitlements shape); two pre-existing tests that asserted the old "self-host holds connector keys" contract updated to the new one. Verified: full unit suite green, pg-real suite for lib/entitlements green against a local supabase/postgres with every migration applied, lint ratchet, guards. Deferred to the instance-wiring PR: adding the connector extensions to the self-host Docker preset (dead-end upsells until a key can be issued). Co-Authored-By: Claude Fable 5 <noreply@anthropic.com> * refactor(entitlements): fold the self-host branch into the existing grants query One .or(scopeFilter), not two: the duplicated helper pushed the no-phantom-columns unresolvable-expression count to 380/379. Behaviour is unchanged; the self-host matrix tests still pass. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com> * feat(connect): hosted connector-key registry + validate RPC + entitlements endpoint; instance sync writes connector grants hourly Sovereign plan WS3 PR4 ("key infra enabling manual sales"), stacked on the entitlement partition (#1747). Nothing is purchasable yet; this is the plumbing both ends need before the first manually issued key. Hosted side: - Migration 20260820123000: connector_keys (SHA-256 key_hash, prefix, org_number, pinned instance_url, scopes, status, Stripe ids, current_period_end, per-minute rate limit, active_company_count, last_seen/synced) and connector_usage_events (per-request metering, separate from metered_events whose company_id references hosted companies). RLS on, NO policies: service role only. RPC validate_and_increment_connector_key copies the api_keys pattern (FOR UPDATE, minute window, suspended reported not counted, revoked = no row) and is REVOKEd from PUBLIC/anon/authenticated, GRANTed to service_role. pg-real test covers validate/count, unknown+revoked, suspended, rate limit, execute privileges per role, RLS invisibility, usage cascade. - lib/connect/contract.ts (shared wire types), lib/connect/hosted/keys.ts (generate/hash/validate -> 401/403/429 mapping), with-connector-auth.ts (Bearer or X-Connector-Key, one usage row per request, 500 envelope on handler throw), /api/connect/entitlements GET + POST (records active_company_count, pins instance_url on first report, never moves a pinned one), scripts/issue-connector-key.ts (dry run unless --confirm, prints the key once + the .env lines). Instance side: - lib/connect/instance/config.ts (GNUBOK_CONNECTOR_KEY, GNUBOK_CONNECT_URL default https://app.gnubok.se), sync.ts: reports the active company count and writes source='connector' grants for every company x covered scope, expires_at = min(now+72h, period_end+3d); 401/403 or a non-active status deletes them (freeze-and-retain); network/5xx/429 leave them alone. /api/connector/sync/cron (hourly) runs it; not_configured without a key. - Crontab generator gains EXTRA_JOBS (variant-only jobs not in vercel.json, with reasons) + drift tests; docker/crontab.self-hosted regenerated with the hourly sync. Docs (SELF-HOSTING connector section, env templates), DECISIONS. Tests: 52 new unit tests (keys, auth wrapper, route, config, sync outcomes and grant arithmetic, cron route, crontab EXTRA_JOBS) + 7 pg-real tests run locally against supabase/postgres with every migration applied. no-phantom-columns ceiling +1 with a reason (the bulk grant upsert). Co-Authored-By: Claude Fable 5 <noreply@anthropic.com> * chore(connect): update pg test to re-versioned migration 20260831190000 Co-Authored-By: Claude Fable 5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01UzNkSsR18pLFitJdYn8QEb * fix(connect): RPC errors answer 503 not 401; X-Connector-Key wins over Authorization A hosted DB error mapped to 401 made the instance sync treat a pooler blip as key revocation and delete its entire connector grant cache, zeroing the 72h offline grace. 503 lands in the sync's keep-grants branch (already test-pinned). Bearer-first extraction hashed the upstream token on dual-header proxied calls, 401ing the exact shape X-Connector-Key exists for. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01UzNkSsR18pLFitJdYn8QEb * fix(connect): sync deletes grants only on a body-proven connector rejection, never bare 401/403 A WAF challenge page, edge deployment protection, or an egress proxy answers 401/403 without the hosted app ever running; trusting status alone wiped the instance's 72h offline grant cache within the hour. Deletion now requires the hosted route's own rejection code in the JSON body; codeless 401/403 keeps grants (server_error branch). Co-Authored-By: Claude Fable 5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01UzNkSsR18pLFitJdYn8QEb * fix(connect): PR #1748 review batch: https-only connect URL, atomic pin, prefix-gated Bearer, deferred metering, entitlements validation, integer months - GNUBOK_CONNECT_URL must be https (http only for loopback); invalid or plaintext URLs disable the connector instead of sending the key. - instance_url pin update filters on IS NULL; a lost race re-reads and reports the winner's pin. - extractConnectorKey: a Bearer is the connector credential only with the gnubok_ck_ prefix; upstream Bearer falls through to X-Connector-Key. - Usage metering runs via after() off the response path (inline outside a request scope). - Sync validates entitlements shape: unknown status or malformed current_period_end keeps grants (server_error), never deletes. - issue-connector-key rejects fractional --months. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01UzNkSsR18pLFitJdYn8QEb --------- Co-authored-by: Jakob Wennberg <311770904+jakobwennberg-oss@users.noreply.github.com> Co-authored-by: Claude Fable 5 <noreply@anthropic.com> Co-authored-by: Emil <emilmattsson14@gmail.com>
This commit is contained in:
Jakob Wennberg
co-authored by
Claude Fable 5
Jakob Wennberg
Emil
Claude Fable 5
Jakob Wennberg
Emil
parent
cfce2de925
commit
0ff1b05553
@@ -7,6 +7,7 @@ import {
|
||||
readVercelCrons,
|
||||
scheduleFor,
|
||||
EXCLUDED_PATHS,
|
||||
EXTRA_JOBS,
|
||||
SCHEDULE_OVERRIDES,
|
||||
VARIANTS,
|
||||
type CrontabVariant,
|
||||
@@ -41,18 +42,29 @@ function parseCrontab(text: string): { path: string; schedule: string }[] {
|
||||
}
|
||||
|
||||
const expectedPaths = crons.map((c) => c.path).filter((p) => !(p in EXCLUDED_PATHS))
|
||||
const expectedPathsFor = (variant: CrontabVariant) => [
|
||||
...expectedPaths,
|
||||
...EXTRA_JOBS[variant].map((job) => job.path),
|
||||
]
|
||||
|
||||
describe('docker crontabs mirror vercel.json', () => {
|
||||
it.each(VARIANTS)('crontab.%s covers exactly the vercel.json path set minus exclusions', (variant) => {
|
||||
it.each(VARIANTS)('crontab.%s covers exactly the vercel.json path set minus exclusions, plus its EXTRA_JOBS', (variant) => {
|
||||
const actual = parseCrontab(crontabText(variant)).map((job) => job.path)
|
||||
|
||||
// Sorted comparison gives a readable diff of what is missing / extra;
|
||||
// the order assertion below covers sequence separately.
|
||||
expect([...actual].sort()).toEqual([...expectedPaths].sort())
|
||||
expect([...actual].sort()).toEqual([...expectedPathsFor(variant)].sort())
|
||||
})
|
||||
|
||||
it.each(VARIANTS)('crontab.%s keeps vercel.json order', (variant) => {
|
||||
expect(parseCrontab(crontabText(variant)).map((job) => job.path)).toEqual(expectedPaths)
|
||||
it.each(VARIANTS)('crontab.%s keeps vercel.json order, EXTRA_JOBS last', (variant) => {
|
||||
expect(parseCrontab(crontabText(variant)).map((job) => job.path)).toEqual(expectedPathsFor(variant))
|
||||
})
|
||||
|
||||
it.each(VARIANTS)('crontab.%s runs its EXTRA_JOBS on their declared cadence', (variant) => {
|
||||
const actual = new Map(parseCrontab(crontabText(variant)).map((job) => [job.path, job.schedule]))
|
||||
for (const job of EXTRA_JOBS[variant]) {
|
||||
expect(actual.get(job.path), `schedule for extra job ${job.path}`).toBe(job.schedule)
|
||||
}
|
||||
})
|
||||
|
||||
it.each(VARIANTS)('crontab.%s runs every path on its vercel.json cadence', (variant) => {
|
||||
@@ -80,14 +92,41 @@ describe('docker crontabs mirror vercel.json', () => {
|
||||
}
|
||||
})
|
||||
|
||||
it('keeps the two variants identical apart from the variant header line', () => {
|
||||
it('keeps the two variants identical apart from the variant header line and the EXTRA_JOBS tail', () => {
|
||||
const hosted = crontabText('hosted').split('\n')
|
||||
const selfHosted = crontabText('self-hosted').split('\n')
|
||||
const differing = hosted.filter((line, i) => line !== selfHosted[i])
|
||||
const shared = Math.min(hosted.length, selfHosted.length)
|
||||
const differing = hosted.slice(0, shared).filter((line, i) => line !== selfHosted[i])
|
||||
|
||||
// Any real divergence must come from SCHEDULE_OVERRIDES, which is empty
|
||||
// today. If that changes, widen this expectation deliberately.
|
||||
// Any real divergence in the shared prefix must come from
|
||||
// SCHEDULE_OVERRIDES, which is empty today. If that changes, widen this
|
||||
// expectation deliberately.
|
||||
expect(differing).toEqual([expect.stringContaining('# Variant: hosted')])
|
||||
|
||||
// The self-hosted file may only be longer by its EXTRA_JOBS block: one
|
||||
// blank line, one comment line, one line per extra job.
|
||||
const extraLines = EXTRA_JOBS['self-hosted'].length
|
||||
const hostedExtraLines = EXTRA_JOBS.hosted.length
|
||||
expect(selfHosted.length - hosted.length).toBe(
|
||||
(extraLines > 0 ? extraLines + 2 : 0) - (hostedExtraLines > 0 ? hostedExtraLines + 2 : 0),
|
||||
)
|
||||
})
|
||||
})
|
||||
|
||||
describe('EXTRA_JOBS', () => {
|
||||
it('names real cron routes that vercel.json does not schedule, each with a reason', () => {
|
||||
const scheduled = new Set(crons.map((c) => c.path))
|
||||
for (const variant of VARIANTS) {
|
||||
for (const job of EXTRA_JOBS[variant]) {
|
||||
expect(job.reason.trim().length, `${job.path} needs a reason`).toBeGreaterThan(0)
|
||||
expect(job.schedule.trim().split(/\s+/).length, `${job.path} needs a 5-field schedule`).toBe(5)
|
||||
expect(scheduled.has(job.path), `${job.path} is now in vercel.json: drop the extra entry`).toBe(false)
|
||||
expect(
|
||||
existsSync(join(ROOT, 'app', ...job.path.split('/').filter(Boolean), 'route.ts')),
|
||||
`${job.path} has no route.ts`,
|
||||
).toBe(true)
|
||||
}
|
||||
}
|
||||
})
|
||||
})
|
||||
|
||||
@@ -119,6 +158,7 @@ describe('exclusion and override tables', () => {
|
||||
const rendered = buildCrontab(sample, 'self-hosted', {
|
||||
excluded: { '/api/drop/cron': 'vercel-only, cannot work self-hosted' },
|
||||
overrides: { hosted: {}, 'self-hosted': { '/api/keep/cron': '*/30 * * * *' } },
|
||||
extraJobs: { hosted: [], 'self-hosted': [] },
|
||||
})
|
||||
const jobs = parseCrontab(rendered)
|
||||
|
||||
@@ -126,6 +166,29 @@ describe('exclusion and override tables', () => {
|
||||
expect(rendered).not.toContain('/api/drop/cron')
|
||||
})
|
||||
|
||||
it('renders extra jobs after the vercel.json jobs under their own comment line', () => {
|
||||
const rendered = buildCrontab([{ path: '/api/keep/cron', schedule: '0 1 * * *' }], 'self-hosted', {
|
||||
excluded: {},
|
||||
overrides: { hosted: {}, 'self-hosted': {} },
|
||||
extraJobs: {
|
||||
hosted: [],
|
||||
'self-hosted': [{ path: '/api/only-here/cron', schedule: '17 * * * *', reason: 'test' }],
|
||||
},
|
||||
})
|
||||
expect(parseCrontab(rendered)).toEqual([
|
||||
{ path: '/api/keep/cron', schedule: '0 1 * * *' },
|
||||
{ path: '/api/only-here/cron', schedule: '17 * * * *' },
|
||||
])
|
||||
expect(rendered).toContain('# self-hosted-only jobs, not in vercel.json')
|
||||
// hosted gets no tail at all when it has no extra jobs
|
||||
const hosted = buildCrontab([{ path: '/api/keep/cron', schedule: '0 1 * * *' }], 'hosted', {
|
||||
excluded: {},
|
||||
overrides: { hosted: {}, 'self-hosted': {} },
|
||||
extraJobs: { hosted: [], 'self-hosted': [] },
|
||||
})
|
||||
expect(hosted).not.toContain('not in vercel.json')
|
||||
})
|
||||
|
||||
it('renders the curl invocation with unexpanded shell variables', () => {
|
||||
const rendered = buildCrontab([{ path: '/api/x/cron', schedule: '0 1 * * *' }], 'hosted', {
|
||||
excluded: {},
|
||||
@@ -196,7 +259,10 @@ function findCronRoutes(dir: string, urlPrefix: string): string[] {
|
||||
describe('every cron route has a schedule', () => {
|
||||
it('leaves no unscheduled cron route undocumented', () => {
|
||||
const routes = findCronRoutes(join(ROOT, 'app', 'api'), '/api')
|
||||
const scheduled = new Set(crons.map((c) => c.path))
|
||||
const scheduled = new Set([
|
||||
...crons.map((c) => c.path),
|
||||
...VARIANTS.flatMap((variant) => EXTRA_JOBS[variant].map((job) => job.path)),
|
||||
])
|
||||
|
||||
const orphans = routes.filter((r) => !scheduled.has(r) && !(r in INTENTIONALLY_UNSCHEDULED))
|
||||
expect(
|
||||
|
||||
Reference in New Issue
Block a user