Files
accounted/scripts/__tests__/generate-crontabs.test.ts
Jakob Wennberg 0ff1b05553 feat(connect): hosted connector-key registry + validate RPC + entitlements endpoint; instance sync writes connector grants hourly (#1748)
* feat(entitlements): partition the self-host bypass so connector capabilities fall through to grants; capability_grants.source accepts 'connector'

Sovereign plan WS3 PR3: ships dark, nothing changes for hosted.

- lib/entitlements/keys.ts: CONNECTOR_CAPABILITIES = bank_sync,
  skatteverket, org_lookup, migration (services Accounted operates that a
  self-hosted instance cannot provide itself) + isConnectorCapability().
  Separate from PAID_CAPABILITIES and outside the trial-seed trigger on
  purpose: a hosted company can never hold a connector grant.
- lib/entitlements/has-capability.ts: isPaywallBypassed() -> isBypassedFor(key).
  Hosted: byte-identical (dev / DISABLE_PAYWALL bypass, FORCE_PAYWALL wins,
  else the grant lookup). Self-host: local capabilities always on
  (FORCE_PAYWALL included, as the existing test demands); connector
  capabilities behave like hosted, i.e. dev bypass, FORCE_PAYWALL, else the
  grant lookup where the connector sync will write source='connector' rows.
  getCompanyEntitlements on a self-host: local paid keys + active connector
  keys, state 'paid' with an active connector grant else 'none' (never the
  hosted trial copy).
- Migration 20260820122000: capability_grants.source CHECK gains
  'connector', found through pg_constraint (the CHECK was declared inline
  and auto-named; Postgres stores IN as = ANY, matched accordingly).
  pg-real test: connector accepted, unknown source rejected, upsert on the
  (scope, key, source) identity, trial seed writes no connector rows.
- Tests: self-hosted connector matrix (local all-on without DB, connector
  gated by grant/expiry, dev bypass all-on, FORCE_PAYWALL gates connector
  keys only, bulk resolution, entitlements shape); two pre-existing tests
  that asserted the old "self-host holds connector keys" contract updated
  to the new one.

Verified: full unit suite green, pg-real suite for lib/entitlements green
against a local supabase/postgres with every migration applied, lint
ratchet, guards. Deferred to the instance-wiring PR: adding the connector
extensions to the self-host Docker preset (dead-end upsells until a key can
be issued).

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>

* refactor(entitlements): fold the self-host branch into the existing grants query

One .or(scopeFilter), not two: the duplicated helper pushed the
no-phantom-columns unresolvable-expression count to 380/379. Behaviour is
unchanged; the self-host matrix tests still pass.

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>

* feat(connect): hosted connector-key registry + validate RPC + entitlements endpoint; instance sync writes connector grants hourly

Sovereign plan WS3 PR4 ("key infra enabling manual sales"), stacked on the
entitlement partition (#1747). Nothing is purchasable yet; this is the
plumbing both ends need before the first manually issued key.

Hosted side:
- Migration 20260820123000: connector_keys (SHA-256 key_hash, prefix,
  org_number, pinned instance_url, scopes, status, Stripe ids,
  current_period_end, per-minute rate limit, active_company_count,
  last_seen/synced) and connector_usage_events (per-request metering,
  separate from metered_events whose company_id references hosted
  companies). RLS on, NO policies: service role only. RPC
  validate_and_increment_connector_key copies the api_keys pattern (FOR
  UPDATE, minute window, suspended reported not counted, revoked = no row)
  and is REVOKEd from PUBLIC/anon/authenticated, GRANTed to service_role.
  pg-real test covers validate/count, unknown+revoked, suspended, rate
  limit, execute privileges per role, RLS invisibility, usage cascade.
- lib/connect/contract.ts (shared wire types), lib/connect/hosted/keys.ts
  (generate/hash/validate -> 401/403/429 mapping),
  with-connector-auth.ts (Bearer or X-Connector-Key, one usage row per
  request, 500 envelope on handler throw), /api/connect/entitlements GET +
  POST (records active_company_count, pins instance_url on first report,
  never moves a pinned one), scripts/issue-connector-key.ts (dry run unless
  --confirm, prints the key once + the .env lines).

Instance side:
- lib/connect/instance/config.ts (GNUBOK_CONNECTOR_KEY, GNUBOK_CONNECT_URL
  default https://app.gnubok.se), sync.ts: reports the active company count
  and writes source='connector' grants for every company x covered scope,
  expires_at = min(now+72h, period_end+3d); 401/403 or a non-active status
  deletes them (freeze-and-retain); network/5xx/429 leave them alone.
  /api/connector/sync/cron (hourly) runs it; not_configured without a key.
- Crontab generator gains EXTRA_JOBS (variant-only jobs not in vercel.json,
  with reasons) + drift tests; docker/crontab.self-hosted regenerated with
  the hourly sync. Docs (SELF-HOSTING connector section, env templates),
  DECISIONS.

Tests: 52 new unit tests (keys, auth wrapper, route, config, sync outcomes
and grant arithmetic, cron route, crontab EXTRA_JOBS) + 7 pg-real tests
run locally against supabase/postgres with every migration applied.
no-phantom-columns ceiling +1 with a reason (the bulk grant upsert).

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>

* chore(connect): update pg test to re-versioned migration 20260831190000

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01UzNkSsR18pLFitJdYn8QEb

* fix(connect): RPC errors answer 503 not 401; X-Connector-Key wins over Authorization

A hosted DB error mapped to 401 made the instance sync treat a pooler
blip as key revocation and delete its entire connector grant cache,
zeroing the 72h offline grace. 503 lands in the sync's keep-grants
branch (already test-pinned). Bearer-first extraction hashed the
upstream token on dual-header proxied calls, 401ing the exact shape
X-Connector-Key exists for.

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01UzNkSsR18pLFitJdYn8QEb

* fix(connect): sync deletes grants only on a body-proven connector rejection, never bare 401/403

A WAF challenge page, edge deployment protection, or an egress proxy
answers 401/403 without the hosted app ever running; trusting status
alone wiped the instance's 72h offline grant cache within the hour.
Deletion now requires the hosted route's own rejection code in the
JSON body; codeless 401/403 keeps grants (server_error branch).

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01UzNkSsR18pLFitJdYn8QEb

* fix(connect): PR #1748 review batch: https-only connect URL, atomic pin, prefix-gated Bearer, deferred metering, entitlements validation, integer months

- GNUBOK_CONNECT_URL must be https (http only for loopback); invalid or
  plaintext URLs disable the connector instead of sending the key.
- instance_url pin update filters on IS NULL; a lost race re-reads and
  reports the winner's pin.
- extractConnectorKey: a Bearer is the connector credential only with
  the gnubok_ck_ prefix; upstream Bearer falls through to X-Connector-Key.
- Usage metering runs via after() off the response path (inline outside
  a request scope).
- Sync validates entitlements shape: unknown status or malformed
  current_period_end keeps grants (server_error), never deletes.
- issue-connector-key rejects fractional --months.

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01UzNkSsR18pLFitJdYn8QEb

---------

Co-authored-by: Jakob Wennberg <311770904+jakobwennberg-oss@users.noreply.github.com>
Co-authored-by: Claude Fable 5 <noreply@anthropic.com>
Co-authored-by: Emil <emilmattsson14@gmail.com>
2026-08-31 20:51:17 +02:00

286 lines
12 KiB
TypeScript

import { describe, it, expect } from 'vitest'
import { readFileSync, readdirSync, existsSync } from 'node:fs'
import { join, dirname } from 'node:path'
import { fileURLToPath } from 'node:url'
import {
buildCrontab,
readVercelCrons,
scheduleFor,
EXCLUDED_PATHS,
EXTRA_JOBS,
SCHEDULE_OVERRIDES,
VARIANTS,
type CrontabVariant,
type VercelCron,
} from '../generate-crontabs'
const ROOT = dirname(dirname(dirname(fileURLToPath(import.meta.url))))
const crons = readVercelCrons(readFileSync(join(ROOT, 'vercel.json'), 'utf8'))
function crontabText(variant: CrontabVariant): string {
return readFileSync(join(ROOT, 'docker', `crontab.${variant}`), 'utf8')
}
/**
* Independent parser: deliberately NOT the generator's renderer, so a bug in
* the generator cannot hide the very drift this test exists to catch.
*/
function parseCrontab(text: string): { path: string; schedule: string }[] {
return text
.split('\n')
.map((line) => line.trim())
.filter((line) => line.length > 0 && !line.startsWith('#'))
.map((line) => {
const match = line.match(/^(\S+(?:\s+\S+){4})\s+(.*)$/)
if (!match) throw new Error(`crontab line is not "<5-field schedule> <command>": ${line}`)
const [, schedule, command] = match
const pathMatch = command.match(/\$\{APP_URL\}(\/\S*)$/)
if (!pathMatch) throw new Error(`crontab command does not end in \${APP_URL}<path>: ${command}`)
return { schedule, path: pathMatch[1] }
})
}
const expectedPaths = crons.map((c) => c.path).filter((p) => !(p in EXCLUDED_PATHS))
const expectedPathsFor = (variant: CrontabVariant) => [
...expectedPaths,
...EXTRA_JOBS[variant].map((job) => job.path),
]
describe('docker crontabs mirror vercel.json', () => {
it.each(VARIANTS)('crontab.%s covers exactly the vercel.json path set minus exclusions, plus its EXTRA_JOBS', (variant) => {
const actual = parseCrontab(crontabText(variant)).map((job) => job.path)
// Sorted comparison gives a readable diff of what is missing / extra;
// the order assertion below covers sequence separately.
expect([...actual].sort()).toEqual([...expectedPathsFor(variant)].sort())
})
it.each(VARIANTS)('crontab.%s keeps vercel.json order, EXTRA_JOBS last', (variant) => {
expect(parseCrontab(crontabText(variant)).map((job) => job.path)).toEqual(expectedPathsFor(variant))
})
it.each(VARIANTS)('crontab.%s runs its EXTRA_JOBS on their declared cadence', (variant) => {
const actual = new Map(parseCrontab(crontabText(variant)).map((job) => [job.path, job.schedule]))
for (const job of EXTRA_JOBS[variant]) {
expect(actual.get(job.path), `schedule for extra job ${job.path}`).toBe(job.schedule)
}
})
it.each(VARIANTS)('crontab.%s runs every path on its vercel.json cadence', (variant) => {
const actual = new Map(parseCrontab(crontabText(variant)).map((job) => [job.path, job.schedule]))
for (const cron of crons) {
if (cron.path in EXCLUDED_PATHS) continue
expect(actual.get(cron.path), `schedule for ${cron.path} in crontab.${variant}`).toBe(
scheduleFor(cron, variant),
)
}
})
it.each(VARIANTS)('crontab.%s is byte-identical to the generator output', (variant) => {
// Fails when someone hand-edits a crontab, or edits vercel.json without
// running `npm run crontabs:generate`.
expect(crontabText(variant)).toBe(buildCrontab(crons, variant))
})
it('emits LF line endings and a trailing newline (supercronic + .gitattributes)', () => {
for (const variant of VARIANTS) {
const raw = readFileSync(join(ROOT, 'docker', `crontab.${variant}`))
expect(raw.includes(0x0d), `CR byte in crontab.${variant}`).toBe(false)
expect(raw[raw.length - 1], `trailing newline in crontab.${variant}`).toBe(0x0a)
}
})
it('keeps the two variants identical apart from the variant header line and the EXTRA_JOBS tail', () => {
const hosted = crontabText('hosted').split('\n')
const selfHosted = crontabText('self-hosted').split('\n')
const shared = Math.min(hosted.length, selfHosted.length)
const differing = hosted.slice(0, shared).filter((line, i) => line !== selfHosted[i])
// Any real divergence in the shared prefix must come from
// SCHEDULE_OVERRIDES, which is empty today. If that changes, widen this
// expectation deliberately.
expect(differing).toEqual([expect.stringContaining('# Variant: hosted')])
// The self-hosted file may only be longer by its EXTRA_JOBS block: one
// blank line, one comment line, one line per extra job.
const extraLines = EXTRA_JOBS['self-hosted'].length
const hostedExtraLines = EXTRA_JOBS.hosted.length
expect(selfHosted.length - hosted.length).toBe(
(extraLines > 0 ? extraLines + 2 : 0) - (hostedExtraLines > 0 ? hostedExtraLines + 2 : 0),
)
})
})
describe('EXTRA_JOBS', () => {
it('names real cron routes that vercel.json does not schedule, each with a reason', () => {
const scheduled = new Set(crons.map((c) => c.path))
for (const variant of VARIANTS) {
for (const job of EXTRA_JOBS[variant]) {
expect(job.reason.trim().length, `${job.path} needs a reason`).toBeGreaterThan(0)
expect(job.schedule.trim().split(/\s+/).length, `${job.path} needs a 5-field schedule`).toBe(5)
expect(scheduled.has(job.path), `${job.path} is now in vercel.json: drop the extra entry`).toBe(false)
expect(
existsSync(join(ROOT, 'app', ...job.path.split('/').filter(Boolean), 'route.ts')),
`${job.path} has no route.ts`,
).toBe(true)
}
}
})
})
describe('exclusion and override tables', () => {
it('lists no path that vercel.json no longer schedules', () => {
const known = new Set(crons.map((c) => c.path))
for (const path of Object.keys(EXCLUDED_PATHS)) {
expect(known.has(path), `EXCLUDED_PATHS entry ${path} is stale`).toBe(true)
}
for (const variant of VARIANTS) {
for (const path of Object.keys(SCHEDULE_OVERRIDES[variant])) {
expect(known.has(path), `SCHEDULE_OVERRIDES.${variant} entry ${path} is stale`).toBe(true)
}
}
})
it('states a reason for every exclusion', () => {
for (const [path, reason] of Object.entries(EXCLUDED_PATHS)) {
expect(reason.trim().length, `exclusion ${path} needs a stated reason`).toBeGreaterThan(0)
}
})
it('honours an exclusion and a per-variant override', () => {
const sample: VercelCron[] = [
{ path: '/api/keep/cron', schedule: '0 1 * * *' },
{ path: '/api/drop/cron', schedule: '0 2 * * *' },
]
const rendered = buildCrontab(sample, 'self-hosted', {
excluded: { '/api/drop/cron': 'vercel-only, cannot work self-hosted' },
overrides: { hosted: {}, 'self-hosted': { '/api/keep/cron': '*/30 * * * *' } },
extraJobs: { hosted: [], 'self-hosted': [] },
})
const jobs = parseCrontab(rendered)
expect(jobs).toEqual([{ path: '/api/keep/cron', schedule: '*/30 * * * *' }])
expect(rendered).not.toContain('/api/drop/cron')
})
it('renders extra jobs after the vercel.json jobs under their own comment line', () => {
const rendered = buildCrontab([{ path: '/api/keep/cron', schedule: '0 1 * * *' }], 'self-hosted', {
excluded: {},
overrides: { hosted: {}, 'self-hosted': {} },
extraJobs: {
hosted: [],
'self-hosted': [{ path: '/api/only-here/cron', schedule: '17 * * * *', reason: 'test' }],
},
})
expect(parseCrontab(rendered)).toEqual([
{ path: '/api/keep/cron', schedule: '0 1 * * *' },
{ path: '/api/only-here/cron', schedule: '17 * * * *' },
])
expect(rendered).toContain('# self-hosted-only jobs, not in vercel.json')
// hosted gets no tail at all when it has no extra jobs
const hosted = buildCrontab([{ path: '/api/keep/cron', schedule: '0 1 * * *' }], 'hosted', {
excluded: {},
overrides: { hosted: {}, 'self-hosted': {} },
extraJobs: { hosted: [], 'self-hosted': [] },
})
expect(hosted).not.toContain('not in vercel.json')
})
it('renders the curl invocation with unexpanded shell variables', () => {
const rendered = buildCrontab([{ path: '/api/x/cron', schedule: '0 1 * * *' }], 'hosted', {
excluded: {},
overrides: { hosted: {}, 'self-hosted': {} },
})
expect(rendered).toContain(
'curl -sf -H "Authorization: Bearer ${CRON_SECRET}" ${APP_URL}/api/x/cron',
)
})
})
describe('readVercelCrons', () => {
it('rejects a schedule that is not five fields', () => {
expect(() =>
readVercelCrons(JSON.stringify({ crons: [{ path: '/api/x/cron', schedule: '0 3 * *' }] })),
).toThrow(/5-field/)
})
it('rejects a path outside /api/', () => {
expect(() =>
readVercelCrons(JSON.stringify({ crons: [{ path: 'api/x/cron', schedule: '0 3 * * *' }] })),
).toThrow(/must start with \/api\//)
})
it('rejects a duplicate path', () => {
expect(() =>
readVercelCrons(
JSON.stringify({
crons: [
{ path: '/api/x/cron', schedule: '0 3 * * *' },
{ path: '/api/x/cron', schedule: '0 4 * * *' },
],
}),
),
).toThrow(/duplicate/)
})
})
/**
* Ratchet: a cron route that nothing schedules is dead code that looks alive.
* Every app/api/**\/cron route must either be in vercel.json (and therefore in
* both crontabs) or be named here with a reason.
*/
const INTENTIONALLY_UNSCHEDULED: Readonly<Record<string, string>> = {
'/api/invoices/reminders/cron':
'Feature switched off: the route is a tombstone that logs and returns 503 { disabled: true }.',
'/api/extensions/push-notifications/cron':
'Pre-existing gap: the docstring says daily 09:00 UTC but no schedule exists anywhere. Needs a vercel.json entry, which is outside the crontab-parity change.',
'/api/extensions/stripe/sync/cron':
'Pre-existing gap: the docstring says every 15 minutes but no schedule exists anywhere. Needs a vercel.json entry, which is outside the crontab-parity change.',
}
function findCronRoutes(dir: string, urlPrefix: string): string[] {
const found: string[] = []
for (const entry of readdirSync(dir, { withFileTypes: true })) {
if (!entry.isDirectory()) continue
// Dynamic segments cannot be scheduled: a cron needs a concrete URL.
if (entry.name.startsWith('[')) continue
const urlPath = `${urlPrefix}/${entry.name}`
if (entry.name === 'cron' && existsSync(join(dir, entry.name, 'route.ts'))) {
found.push(urlPath)
}
found.push(...findCronRoutes(join(dir, entry.name), urlPath))
}
return found
}
describe('every cron route has a schedule', () => {
it('leaves no unscheduled cron route undocumented', () => {
const routes = findCronRoutes(join(ROOT, 'app', 'api'), '/api')
const scheduled = new Set([
...crons.map((c) => c.path),
...VARIANTS.flatMap((variant) => EXTRA_JOBS[variant].map((job) => job.path)),
])
const orphans = routes.filter((r) => !scheduled.has(r) && !(r in INTENTIONALLY_UNSCHEDULED))
expect(
orphans,
'These cron routes are scheduled nowhere. Add them to vercel.json (then run ' +
'`npm run crontabs:generate`), or list them in INTENTIONALLY_UNSCHEDULED with a reason.',
).toEqual([])
})
it('lists no route in INTENTIONALLY_UNSCHEDULED that has since been scheduled or deleted', () => {
const routes = new Set(findCronRoutes(join(ROOT, 'app', 'api'), '/api'))
const scheduled = new Set(crons.map((c) => c.path))
for (const [path, reason] of Object.entries(INTENTIONALLY_UNSCHEDULED)) {
expect(reason.trim().length, `${path} needs a reason`).toBeGreaterThan(0)
expect(routes.has(path), `${path} no longer exists: drop the entry`).toBe(true)
expect(scheduled.has(path), `${path} is now in vercel.json: drop the entry`).toBe(false)
}
})
})