Files
accounted/tests/tool-pg/docker-compose.yml
T
Jakob Wennberg 325c827322 test(mcp): run the tools against a real PostgREST, not a fake supabase (#1983)
All 100 files in extensions/general/mcp-server/__tests__ fake supabase.
query-journal.test.ts says out loud that its query chain is "exercised by the
live MCP smoke test", and no such test exists in CI. So the PostgREST grammar
of 157 tools, every .select() column string, every resource embed, every
or=(...) form, is gated by nothing and fails first in production.

pg-real cannot cover this: it holds a pg Pool and writes SQL, and none of that
grammar is resolved by Postgres. It is resolved by PostgREST at request time.

Adds a tool-pg vitest project, a docker-compose stack, a reset script that
replays every migration the way the pg-real CI job does, and a CI job.

The first sweep covers 74 read tools and finds no malformed query, across 87
real requests. That number is honest rather than impressive: with an empty
argument set many tools bail before querying. Per-tool fixtures are what
deepen it, and this harness is what makes writing them worth the effort.

Includes a self-test that injects a bad column and asserts the harness detects
it. That is not ceremony. It caught this file passing green while exercising
nothing, twice: once locally where supabase-js prefixes /rest/v1 onto a bare
PostgREST that does not serve it, and once on CI where Node 20 has no native
WebSocket, so every client construction threw and was swallowed by the
per-tool catch as a domain refusal. The client is now built once outside that
catch, the proof-of-life assertion counts real requests instead of being
trivially satisfiable, and realtime gets an inert transport.

Also excludes .next from all three vitest projects. These projects override
vitest's default excludes, so a local `npm run build` leaves a traced copy of
the repo that gets collected as a second set of test files.

Co-authored-by: Jakob Wennberg <311770904+jakobwennberg-oss@users.noreply.github.com>
Co-authored-by: Claude Opus 5 (1M context) <noreply@anthropic.com>
2026-08-27 18:11:41 +02:00

49 lines
2.0 KiB
YAML

# Postgres + PostgREST for the MCP tool integration tests.
#
# The existing pg-real suite talks to Postgres through a raw `pg` Pool, which
# is the right shape for testing triggers and RLS in SQL. It is the wrong shape
# for testing the MCP tools, because every one of them queries through
# supabase-js, and supabase-js speaks PostgREST rather than SQL. The embeds,
# the `or=(col.is.null,col.not.in.(...))` forms, the `.contains()` filters and
# the column names in `.select()` strings are all resolved by PostgREST at
# request time, so a raw-SQL test cannot see any of them.
#
# That is the gap this stack exists to close: as of 2026-08-27 all 100 files in
# extensions/general/mcp-server/__tests__ fake supabase, and query-journal.test.ts
# says out loud that its query chain is "exercised by the live MCP smoke test",
# which does not exist in CI.
services:
postgres:
# Same image and tag as the pg-real CI job, deliberately: it ships the auth
# schema, auth.uid(), the anon/authenticated/service_role roles and the
# extensions this repo's migrations need. Plain postgres:15 would not.
image: supabase/postgres:15.8.1.060
environment:
POSTGRES_PASSWORD: postgres
ports:
- '54329:5432'
healthcheck:
test: ['CMD-SHELL', 'pg_isready -U postgres']
interval: 3s
timeout: 5s
retries: 30
postgrest:
image: postgrest/postgrest:v12.2.3
depends_on:
postgres:
condition: service_healthy
environment:
PGRST_DB_URI: postgres://postgres:postgres@postgres:5432/postgres
PGRST_DB_SCHEMAS: public
PGRST_DB_ANON_ROLE: anon
# Must match TOOL_PG_JWT_SECRET in tests/tool-pg/env.ts. Length matters:
# PostgREST refuses a secret shorter than 32 bytes.
PGRST_JWT_SECRET: super-secret-jwt-token-with-at-least-32-characters-long
# The tools call RPCs and read a lot of rows; the default 1000-row cap
# would silently truncate and make a passing test meaningless.
PGRST_DB_MAX_ROWS: '100000'
PGRST_DB_POOL: '10'
ports:
- '54330:3000'