test(mcp): run the tools against a real PostgREST, not a fake supabase (#1983)

All 100 files in extensions/general/mcp-server/__tests__ fake supabase. query-journal.test.ts says out loud that its query chain is "exercised by the live MCP smoke test", and no such test exists in CI. So the PostgREST grammar of 157 tools, every .select() column string, every resource embed, every or=(...) form, is gated by nothing and fails first in production. pg-real cannot cover this: it holds a pg Pool and writes SQL, and none of that grammar is resolved by Postgres. It is resolved by PostgREST at request time. Adds a tool-pg vitest project, a docker-compose stack, a reset script that replays every migration the way the pg-real CI job does, and a CI job. The first sweep covers 74 read tools and finds no malformed query, across 87 real requests. That number is honest rather than impressive: with an empty argument set many tools bail before querying. Per-tool fixtures are what deepen it, and this harness is what makes writing them worth the effort. Includes a self-test that injects a bad column and asserts the harness detects it. That is not ceremony. It caught this file passing green while exercising nothing, twice: once locally where supabase-js prefixes /rest/v1 onto a bare PostgREST that does not serve it, and once on CI where Node 20 has no native WebSocket, so every client construction threw and was swallowed by the per-tool catch as a domain refusal. The client is now built once outside that catch, the proof-of-life assertion counts real requests instead of being trivially satisfiable, and realtime gets an inert transport. Also excludes .next from all three vitest projects. These projects override vitest's default excludes, so a local `npm run build` leaves a traced copy of the repo that gets collected as a second set of test files. Co-authored-by: Jakob Wennberg <311770904+jakobwennberg-oss@users.noreply.github.com> Co-authored-by: Claude Opus 5 (1M context) <noreply@anthropic.com>
2026-08-27 18:11:41 +02:00
parent 5fe3ae71a1
commit 325c827322
9 changed files with 587 additions and 7 deletions
@@ -200,3 +200,100 @@ jobs:
          done

      - run: npm run test:pg
+
+  tool-pg:
+    # MCP tools driven through a REAL supabase-js client against a REAL
+    # PostgREST. This is NOT a duplicate of the pg-real job: that one holds a
+    # `pg` Pool and writes SQL, which cannot see the half of a tool that
+    # PostgREST resolves (the .select() column strings, the resource embeds,
+    # the or=(...) grammar). Before this job, all 100 files in
+    # extensions/general/mcp-server/__tests__ faked supabase and nothing in CI
+    # exercised that surface.
+    runs-on: ubuntu-latest
+
+    services:
+      postgres:
+        image: supabase/postgres:15.8.1.060
+        env:
+          POSTGRES_PASSWORD: postgres
+        ports:
+          - 5432:5432
+        options: >-
+          --health-cmd "pg_isready -U postgres"
+          --health-interval 5s
+          --health-timeout 5s
+          --health-retries 20
+
+    env:
+      DATABASE_URL: postgresql://postgres:postgres@localhost:5432/postgres
+      TOOL_PG_DATABASE_URL: postgresql://postgres:postgres@localhost:5432/postgres
+      TOOL_PG_REST_URL: http://127.0.0.1:3000
+      PGPASSWORD: postgres
+
+    steps:
+      - uses: actions/checkout@3d3c42e5aac5ba805825da76410c181273ba90b1 # v7
+        with:
+          persist-credentials: false
+
+      - uses: actions/setup-node@249970729cb0ef3589644e2896645e5dc5ba9c38 # v6
+        with:
+          node-version: 20
+
+      - run: npm ci
+
+      - name: Install psql client
+        run: sudo apt-get update && sudo apt-get install -y --no-install-recommends postgresql-client
+
+      - name: Bootstrap storage schema
+        run: psql "$DATABASE_URL" -v ON_ERROR_STOP=1 -q -f tests/pg/bootstrap.sql
+
+      - name: Default privileges for the supabase roles
+        # Without these PostgREST answers every request with 42501, because the
+        # grants have to exist before the migrations create ~400 tables.
+        run: |
+          psql "$DATABASE_URL" -v ON_ERROR_STOP=1 -q -c "
+            GRANT USAGE ON SCHEMA public TO postgres, anon, authenticated, service_role;
+            ALTER DEFAULT PRIVILEGES IN SCHEMA public GRANT ALL ON TABLES TO postgres, anon, authenticated, service_role;
+            ALTER DEFAULT PRIVILEGES IN SCHEMA public GRANT ALL ON ROUTINES TO postgres, anon, authenticated, service_role;
+            ALTER DEFAULT PRIVILEGES IN SCHEMA public GRANT ALL ON SEQUENCES TO postgres, anon, authenticated, service_role;
+          "
+
+      - name: Apply migrations
+        run: |
+          set -euo pipefail
+          shopt -s nullglob
+          files=(supabase/migrations/*.sql)
+          if [ ${#files[@]} -eq 0 ]; then echo "No migration files found"; exit 1; fi
+          for f in "${files[@]}"; do
+            psql "$DATABASE_URL" -v ON_ERROR_STOP=1 -q -f "$f"
+          done
+
+      - name: Grant on everything the migrations created
+        run: |
+          psql "$DATABASE_URL" -v ON_ERROR_STOP=1 -q -c "
+            GRANT ALL ON ALL TABLES IN SCHEMA public TO postgres, anon, authenticated, service_role;
+            GRANT ALL ON ALL ROUTINES IN SCHEMA public TO postgres, anon, authenticated, service_role;
+            GRANT ALL ON ALL SEQUENCES IN SCHEMA public TO postgres, anon, authenticated, service_role;
+          "
+
+      - name: Start PostgREST
+        # Deliberately `docker run --network host` rather than a service
+        # container. Service containers on a non-containerized job are reachable
+        # from the runner on localhost, but NOT from each other by name, and
+        # PostgREST has to reach Postgres. Host networking sidesteps that.
+        run: |
+          docker run -d --name postgrest --network host \
+            -e PGRST_DB_URI="postgres://postgres:postgres@127.0.0.1:5432/postgres" \
+            -e PGRST_DB_SCHEMAS=public \
+            -e PGRST_DB_ANON_ROLE=anon \
+            -e PGRST_JWT_SECRET="super-secret-jwt-token-with-at-least-32-characters-long" \
+            -e PGRST_DB_MAX_ROWS=100000 \
+            -e PGRST_SERVER_PORT=3000 \
+            postgrest/postgrest:v12.2.3
+          for _ in $(seq 1 60); do
+            if curl -sf -o /dev/null "http://127.0.0.1:3000/" ; then break; fi
+            sleep 1
+          done
+          curl -sf -o /dev/null "http://127.0.0.1:3000/" || (docker logs postgrest && exit 1)
+
+      - run: npm run test:tools