test(mcp): run the tools against a real PostgREST, not a fake supabase (#1983)

All 100 files in extensions/general/mcp-server/__tests__ fake supabase.
query-journal.test.ts says out loud that its query chain is "exercised by the
live MCP smoke test", and no such test exists in CI. So the PostgREST grammar
of 157 tools, every .select() column string, every resource embed, every
or=(...) form, is gated by nothing and fails first in production.

pg-real cannot cover this: it holds a pg Pool and writes SQL, and none of that
grammar is resolved by Postgres. It is resolved by PostgREST at request time.

Adds a tool-pg vitest project, a docker-compose stack, a reset script that
replays every migration the way the pg-real CI job does, and a CI job.

The first sweep covers 74 read tools and finds no malformed query, across 87
real requests. That number is honest rather than impressive: with an empty
argument set many tools bail before querying. Per-tool fixtures are what
deepen it, and this harness is what makes writing them worth the effort.

Includes a self-test that injects a bad column and asserts the harness detects
it. That is not ceremony. It caught this file passing green while exercising
nothing, twice: once locally where supabase-js prefixes /rest/v1 onto a bare
PostgREST that does not serve it, and once on CI where Node 20 has no native
WebSocket, so every client construction threw and was swallowed by the
per-tool catch as a domain refusal. The client is now built once outside that
catch, the proof-of-life assertion counts real requests instead of being
trivially satisfiable, and realtime gets an inert transport.

Also excludes .next from all three vitest projects. These projects override
vitest's default excludes, so a local `npm run build` leaves a traced copy of
the repo that gets collected as a second set of test files.

Co-authored-by: Jakob Wennberg <311770904+jakobwennberg-oss@users.noreply.github.com>
Co-authored-by: Claude Opus 5 (1M context) <noreply@anthropic.com>
This commit is contained in:
Jakob Wennberg
2026-08-27 18:11:41 +02:00
committed by GitHub
parent 5fe3ae71a1
commit 325c827322
9 changed files with 587 additions and 7 deletions
+3 -1
View File
@@ -25,7 +25,9 @@
"check:lint": "node scripts/checks/no-new-lint-errors.mjs",
"check:types": "node scripts/checks/no-new-type-errors.mjs",
"test": "vitest run --project unit",
"test:pg": "vitest run --project pg-real"
"test:pg": "vitest run --project pg-real",
"tools:pg:reset": "bash scripts/tool-pg/reset.sh",
"test:tools": "TOOL_PG_REST_URL=${TOOL_PG_REST_URL:-http://127.0.0.1:54330} vitest run --project tool-pg"
},
"dependencies": {
"@ai-sdk/openai-compatible": "2.0.69",