325c827322
All 100 files in extensions/general/mcp-server/__tests__ fake supabase. query-journal.test.ts says out loud that its query chain is "exercised by the live MCP smoke test", and no such test exists in CI. So the PostgREST grammar of 157 tools, every .select() column string, every resource embed, every or=(...) form, is gated by nothing and fails first in production. pg-real cannot cover this: it holds a pg Pool and writes SQL, and none of that grammar is resolved by Postgres. It is resolved by PostgREST at request time. Adds a tool-pg vitest project, a docker-compose stack, a reset script that replays every migration the way the pg-real CI job does, and a CI job. The first sweep covers 74 read tools and finds no malformed query, across 87 real requests. That number is honest rather than impressive: with an empty argument set many tools bail before querying. Per-tool fixtures are what deepen it, and this harness is what makes writing them worth the effort. Includes a self-test that injects a bad column and asserts the harness detects it. That is not ceremony. It caught this file passing green while exercising nothing, twice: once locally where supabase-js prefixes /rest/v1 onto a bare PostgREST that does not serve it, and once on CI where Node 20 has no native WebSocket, so every client construction threw and was swallowed by the per-tool catch as a domain refusal. The client is now built once outside that catch, the proof-of-life assertion counts real requests instead of being trivially satisfiable, and realtime gets an inert transport. Also excludes .next from all three vitest projects. These projects override vitest's default excludes, so a local `npm run build` leaves a traced copy of the repo that gets collected as a second set of test files. Co-authored-by: Jakob Wennberg <311770904+jakobwennberg-oss@users.noreply.github.com> Co-authored-by: Claude Opus 5 (1M context) <noreply@anthropic.com>
106 lines
4.5 KiB
TypeScript
106 lines
4.5 KiB
TypeScript
/**
|
|
* A REAL supabase-js client, pointed at a real PostgREST, over a real Postgres
|
|
* with every migration replayed.
|
|
*
|
|
* Why this is not the same thing as the pg-real suite: those tests hold a `pg`
|
|
* Pool and write SQL. The MCP tools do not write SQL. They call
|
|
* `supabase.from('x').select('a, b:c(d)')`, and the string inside `.select()`
|
|
* is parsed by PostgREST, not by Postgres. A misspelled column, a resource
|
|
* embed whose foreign key does not exist, an `or=(...)` whose grammar is
|
|
* slightly off, a `.contains()` against a non-jsonb column: every one of those
|
|
* is a runtime 400 from PostgREST that a mocked client answers cheerfully and a
|
|
* SQL test never reaches.
|
|
*/
|
|
import { createHmac } from 'node:crypto'
|
|
import { createClient, type SupabaseClient } from '@supabase/supabase-js'
|
|
|
|
/** Must match PGRST_JWT_SECRET in tests/tool-pg/docker-compose.yml. */
|
|
export const TOOL_PG_JWT_SECRET =
|
|
'super-secret-jwt-token-with-at-least-32-characters-long'
|
|
|
|
export const TOOL_PG_REST_URL = process.env.TOOL_PG_REST_URL ?? 'http://127.0.0.1:54330'
|
|
export const TOOL_PG_DATABASE_URL =
|
|
process.env.TOOL_PG_DATABASE_URL ?? 'postgresql://postgres:postgres@127.0.0.1:54329/postgres'
|
|
|
|
function base64url(input: Buffer | string): string {
|
|
return Buffer.from(input)
|
|
.toString('base64')
|
|
.replace(/\+/g, '-')
|
|
.replace(/\//g, '_')
|
|
.replace(/=+$/, '')
|
|
}
|
|
|
|
/**
|
|
* Hand-rolled HS256 rather than a JWT library: this repo is AGPL and audits its
|
|
* dependency surface, and a signed JWT is three base64url segments and one
|
|
* HMAC. Not worth a dependency, and definitely not worth one that ships only to
|
|
* tests.
|
|
*/
|
|
export function signServiceRoleJwt(secret = TOOL_PG_JWT_SECRET): string {
|
|
const header = base64url(JSON.stringify({ alg: 'HS256', typ: 'JWT' }))
|
|
const payload = base64url(
|
|
JSON.stringify({
|
|
role: 'service_role',
|
|
iss: 'tool-pg',
|
|
// Fixed far-future expiry: the harness is disposable and a clock-derived
|
|
// value would make an otherwise deterministic suite time-dependent.
|
|
exp: 4102444800,
|
|
}),
|
|
)
|
|
const signature = base64url(
|
|
createHmac('sha256', secret).update(`${header}.${payload}`).digest(),
|
|
)
|
|
return `${header}.${payload}.${signature}`
|
|
}
|
|
|
|
/**
|
|
* Service-role client, which is what the MCP server actually uses:
|
|
* `createServiceClientNoCookies()` on the API-key path. RLS is therefore NOT
|
|
* the thing under test here; the query grammar is. Tenant isolation on this
|
|
* surface comes from explicit `.eq('company_id', ...)` discipline, and a tool
|
|
* that forgets it is exactly the kind of bug these tests can catch.
|
|
*/
|
|
/**
|
|
* `createClient` eagerly constructs a RealtimeClient, which resolves a
|
|
* WebSocket implementation and throws "native WebSocket not found" on Node 20.
|
|
* CI runs Node 20; local machines may not, which is exactly the kind of
|
|
* difference that turns into a green local run and a red CI one.
|
|
*
|
|
* Nothing here subscribes to realtime, and RealtimeClient only RESOLVES the
|
|
* constructor rather than instantiating it, so handing it an inert class is
|
|
* enough. Bumping the job to Node 22 would work too, but it would make this the
|
|
* only job in the repo on a different runtime for a feature it never uses.
|
|
*/
|
|
class UnusedRealtimeTransport {
|
|
constructor() {
|
|
throw new Error('tool-pg: realtime is not used by these tests')
|
|
}
|
|
}
|
|
|
|
export function createToolPgClient(): SupabaseClient {
|
|
const key = signServiceRoleJwt()
|
|
return createClient(TOOL_PG_REST_URL, key, {
|
|
auth: { persistSession: false, autoRefreshToken: false },
|
|
db: { schema: 'public' },
|
|
realtime: { transport: UnusedRealtimeTransport as never },
|
|
global: {
|
|
headers: { apikey: key },
|
|
// supabase-js hard-codes a `/rest/v1` prefix onto every PostgREST
|
|
// request, because that is where Supabase's own gateway mounts it. A
|
|
// bare PostgREST serves at the root, so without this rewrite every
|
|
// query 404s.
|
|
//
|
|
// That is not a hypothetical: the first version of this harness omitted
|
|
// it, all 55 sweep queries 404d, the tools reported the empty response
|
|
// as "Database error: undefined", and the suite passed green while
|
|
// exercising nothing at all. The self-test in query-grammar.tool.test.ts
|
|
// exists to make that failure mode loud.
|
|
fetch: (input: RequestInfo | URL, init?: RequestInit) => {
|
|
const url = typeof input === 'string' ? input : input instanceof URL ? input.href : input.url
|
|
const rewritten = url.replace('/rest/v1/', '/').replace(/\/rest\/v1$/, '')
|
|
return fetch(rewritten, init)
|
|
},
|
|
},
|
|
})
|
|
}
|