feat(analytics): BI-P002..BI-P012 analytics core — domain model, tiers, contracts, Arrow, ClickHouse engine, isolation, governance
CI (SIAX Cloud) / security (push) Successful in 41s
Security Scan (SIAX Cloud) / security (push) Successful in 40s
CI (SIAX Cloud) / integration (push) Successful in 52s
CI (SIAX Cloud) / quality (push) Successful in 1m5s
CI (SIAX Cloud) / security (pull_request) Successful in 31s
Security Scan (SIAX Cloud) / security (pull_request) Successful in 29s
CI (SIAX Cloud) / integration (pull_request) Successful in 45s
CI (SIAX Cloud) / quality (pull_request) Successful in 53s

Implements the eight BI program prompts that make 0pt the canonical SIAX
analytics capability. Every file is NEW: not one existing file is modified,
and src/analytics/clickhouse-analytics.ts is byte-identical to main.

BI-P002 — the nineteen canonical entities, verbatim, with tenant_id +
project_id on every tenant-owned record. sql/008 gives each one a table.
The four version entities BI-P002 names as immutable are enforced by BEFORE
UPDATE/DELETE triggers, not by a TypeScript `readonly` — proven by direct SQL
against a live Postgres.

BI-P003 — RAW/NORMALIZED/CURATED/SERVING with the read rule enforced:
assertReadableByBiSurface refuses a RAW dataset to a dashboard unless a
recorded exception exists. All eight required declarations are NOT NULL.

BI-P004 — DatasetContract and a fail-closed compatibility gate. Fifteen
breaking-change classes are each tested twice: refused under a minor bump,
accepted under a major one.

BI-P005 — Arrow IPC alongside JSON, with a hand-written FlatBuffers writer so
no npm dependency (or CVE surface) is added. Conformance comes from three
independent implementations, not from self-agreement: pyarrow 25.0.1 and
DuckDB 1.5.5 read identical values from both ClickHouse's Arrow and ours, and
ClickHouse's own Arrow C++ ingests our 20,000-row batch with exact aggregates.

BI-P009 — SiaxAnalyticsEngine with all eight capabilities. The existing
ClickHouseAnalytics is COMPOSED, never replaced. executeQuery (analyst SQL) is
declared and deliberately refused with CAPABILITY_UNAVAILABLE until BI-P017 and
BI-P033 land, rather than opening the raw-SQL hole BI-P011 enumerates.

BI-P010 — four purpose-built tables, not one giant one, and the difference is
measured against the counterfactual: 48.8ms/40,960 rows vs 174.4ms/5,400,000
rows for the same question; 5/660 granules vs 660/1149.

BI-P011 — layered isolation, each layer proven with the layer above removed.
Without the forced predicate the same statement returns 50,050 (both tenants);
with it, t_alpha gets 50. Without a row policy an unrestricted user sees 2
tenants; the restricted user running the IDENTICAL statement sees 1.

BI-P012 — every named limit, fired individually and identified by ClickHouse
error code, plus the half usually skipped: an unrelated tenant is timed while
a real CPU bomb is in flight and must still return the CORRECT answer.

Three defects found by writing tests that can fail:
  * `CHECK (array_length(purposes,1) >= 1)` can NEVER fail — array_length is
    NULL for '{}' and a NULL CHECK passes. Fixed with coalesce(); the test
    that caught it is pinned.
  * EXPLAIN ESTIMATE run under the caller's own max_rows_to_read fails with
    code 158 exactly when the estimate would be over budget. Planner calls now
    use a narrower envelope.
  * result_overflow_mode=break returns HTTP 200 with 65,409 of 100,000 rows
    and no indication it is short. Every SELECT ships `throw`.

Verified rather than assumed: server7's ClickHouse holds `opt_scraped` (51
tables, 407,793,471 rows, `persons`/`cz_owners` included) — the b0ts plane this
module is prohibited from touching — has no `opt_analytics` database, zero
tenant_id columns and zero row policies. Nothing there was modified; every
proof ran against disposable containers.

Evidence: evidence/bi-p002-p012/{test-transcript.txt,bi-p010-benchmark.json,
bi-p005-arrow-roundtrip.json,server7-clickhouse-probe.md}
Docs: docs/analytics/BI_ANALYTICS_CORE.md

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
This commit is contained in:
Claude Opus 5
2026-09-06 23:01:27 +02:00
parent 8d0a288668
commit 84bb9faa4f
32 changed files with 7844 additions and 0 deletions
@@ -0,0 +1,189 @@
/**
* BI-P002 (domain model) + BI-P003 (data tiers), the parts provable without
* infrastructure. The Postgres half — real tables, real immutability
* triggers, real tenant scoping — is
* tests/analytics/analytics-domain-postgres.realinfra.test.ts.
*
* The registry-vs-migration test below is the one that earns its place: it
* reads sql/008_analytics_domain_model.sql from disk and asserts a CREATE
* TABLE exists for every one of the nineteen canonical entities. A future
* rename, or an entity quietly dropped from the migration, fails HERE rather
* than at 3am against a database that turns out to be missing a table.
*/
import { describe, it, expect } from "vitest";
import fs from "node:fs";
import path from "node:path";
import {
ANALYTICS_DOMAIN_ENTITIES,
ANALYTICS_ENTITY_TABLES,
IMMUTABLE_VERSION_ENTITIES,
} from "@/src/analytics/domain/entities";
import {
ANALYTICS_TIERS,
assertReadableByBiSurface,
assertTierLineageSane,
BI_READABLE_TIERS,
isAnalyticsTier,
TIER_RANK,
} from "@/src/analytics/domain/tiers";
const MIGRATION = fs.readFileSync(
path.join(process.cwd(), "sql", "008_analytics_domain_model.sql"),
"utf8",
);
describe("BI-P002 canonical entity set", () => {
it("lists exactly the nineteen entities the program names, verbatim", () => {
expect(ANALYTICS_DOMAIN_ENTITIES).toEqual([
"AnalyticsDataset",
"DatasetVersion",
"DatasetField",
"DatasetGrant",
"MetricDefinition",
"MetricVersion",
"DimensionDefinition",
"SemanticModel",
"AnalyticsQuery",
"QueryExecution",
"QueryResult",
"DashboardDefinition",
"DashboardVersion",
"ReportDefinition",
"ReportRun",
"DataQualityRule",
"DataQualityRun",
"LineageReference",
"BIApplicationBinding",
]);
expect(ANALYTICS_DOMAIN_ENTITIES).toHaveLength(19);
});
it("sql/008 defines a table for every entity (no entity is a type without storage)", () => {
const missing: string[] = [];
for (const entity of ANALYTICS_DOMAIN_ENTITIES) {
const table = ANALYTICS_ENTITY_TABLES[entity];
if (!new RegExp(`CREATE TABLE IF NOT EXISTS ${table}\\b`).test(MIGRATION)) {
missing.push(`${entity} -> ${table}`);
}
}
expect(missing).toEqual([]);
});
it("every tenant-owned table carries tenant_id AND project_id", () => {
// Split on the table definitions and check each block.
const blocks = MIGRATION.split(/CREATE TABLE IF NOT EXISTS /).slice(1);
const offenders: string[] = [];
for (const block of blocks) {
const name = block.split(/[\s(]/)[0];
if (!Object.values(ANALYTICS_ENTITY_TABLES).includes(name)) continue;
const body = block.slice(0, block.indexOf(");"));
if (!/\btenant_id TEXT NOT NULL\b/.test(body) || !/\bproject_id TEXT NOT NULL\b/.test(body)) {
offenders.push(name);
}
}
expect(offenders).toEqual([]);
});
it("the four immutable version entities have a BEFORE UPDATE OR DELETE trigger in sql/008", () => {
expect(IMMUTABLE_VERSION_ENTITIES).toEqual([
"DatasetVersion",
"MetricVersion",
"SemanticModel",
"DashboardVersion",
]);
for (const entity of IMMUTABLE_VERSION_ENTITIES) {
const table = ANALYTICS_ENTITY_TABLES[entity];
expect(
new RegExp(`BEFORE UPDATE OR DELETE ON ${table}\\b`).test(MIGRATION),
`${entity} (${table}) has no immutability trigger`,
).toBe(true);
}
});
it("BIApplicationBinding storage carries NO credential column — the ownership boundary is structural", () => {
const block = MIGRATION.split("CREATE TABLE IF NOT EXISTS opt_analytics_bi_application_binding")[1] ?? "";
const body = block.slice(0, block.indexOf(");"));
for (const forbidden of ["password", "dsn", "secret", "connection_string", "clickhouse_url", "token"]) {
expect(body.toLowerCase()).not.toContain(forbidden);
}
});
it("sql/008 is idempotent-shaped: every CREATE is IF NOT EXISTS / OR REPLACE", () => {
const creates = MIGRATION.match(/^CREATE (TABLE|INDEX|TRIGGER|FUNCTION|OR REPLACE FUNCTION)[^\n]*/gm) ?? [];
const nonIdempotent = creates.filter(
(line) => !/IF NOT EXISTS/.test(line) && !/OR REPLACE/.test(line) && !/^CREATE TRIGGER/.test(line),
);
expect(nonIdempotent).toEqual([]);
// Triggers cannot take IF NOT EXISTS in Postgres, so each one must be
// preceded by a DROP TRIGGER IF EXISTS to stay replayable.
const triggers = MIGRATION.match(/^CREATE TRIGGER (\w+)/gm) ?? [];
for (const t of triggers) {
const name = t.replace("CREATE TRIGGER ", "");
expect(MIGRATION).toContain(`DROP TRIGGER IF EXISTS ${name}`);
}
});
});
describe("BI-P003 analytics data tiers", () => {
it("canonicalizes exactly RAW / NORMALIZED / CURATED / SERVING", () => {
expect(ANALYTICS_TIERS).toEqual(["RAW", "NORMALIZED", "CURATED", "SERVING"]);
expect(isAnalyticsTier("CURATED")).toBe(true);
expect(isAnalyticsTier("raw")).toBe(false);
expect(isAnalyticsTier("BRONZE")).toBe(false);
});
it("no business dashboard reads a RAW/NORMALIZED dataset by default", () => {
expect(BI_READABLE_TIERS).toEqual(["SERVING", "CURATED"]);
for (const tier of ["RAW", "NORMALIZED"] as const) {
expect(() =>
assertReadableByBiSurface({ dataset_id: "ds_1", tier }, "dashboard:exec-overview"),
).toThrowError(/may not read/);
}
for (const tier of ["SERVING", "CURATED"] as const) {
expect(() => assertReadableByBiSurface({ dataset_id: "ds_1", tier }, "dashboard:x")).not.toThrow();
}
});
it("a recorded exception is the only way a RAW dataset reaches a BI surface", () => {
expect(() =>
assertReadableByBiSurface(
{ dataset_id: "ds_1", tier: "RAW", raw_access_exception: "INC-2026-09-06 forensic review, expires 2026-10-01" },
"dashboard:incident",
),
).not.toThrow();
// An empty/whitespace "exception" is not an exception.
expect(() =>
assertReadableByBiSurface({ dataset_id: "ds_1", tier: "RAW", raw_access_exception: " " }, "dashboard:x"),
).toThrowError(/may not read/);
});
it("tier order is enforced: a CURATED dataset cannot be built from SERVING", () => {
expect(TIER_RANK.RAW).toBeLessThan(TIER_RANK.NORMALIZED);
expect(TIER_RANK.NORMALIZED).toBeLessThan(TIER_RANK.CURATED);
expect(TIER_RANK.CURATED).toBeLessThan(TIER_RANK.SERVING);
expect(() => assertTierLineageSane("CURATED", "SERVING")).toThrowError(/tier order/);
expect(() => assertTierLineageSane("SERVING", "CURATED")).not.toThrow();
expect(() => assertTierLineageSane("CURATED", "CURATED")).not.toThrow();
});
it("sql/008 constrains tier at the DATABASE level, not only in TypeScript", () => {
expect(MIGRATION).toContain("CHECK (tier IN ('RAW','NORMALIZED','CURATED','SERVING'))");
});
it("BI-P003's eight required declarations are all NOT NULL in sql/008", () => {
const block = MIGRATION.split("CREATE TABLE IF NOT EXISTS opt_analytics_dataset ")[1] ?? "";
const body = block.slice(0, block.indexOf(");"));
for (const col of [
"tier",
"schema_contract_id",
"owner_id",
"source_ref",
"freshness_slo_seconds",
"retention_days",
"classification",
"quality_rule_ids",
]) {
expect(new RegExp(`\\b${col}\\b[^,]*NOT NULL`).test(body), `${col} is nullable`).toBe(true);
}
});
});
@@ -0,0 +1,364 @@
/**
* BI-P002 DoD: "Database migrations + persistence + tenant isolation +
* restore tests."
*
* REAL Postgres, same honest-degrade convention as
* tests/p0/source-registry-postgres.realinfra.test.ts: gated on
* ANALYTICS_TEST_DATABASE_URL, SKIPS HONESTLY when unset, never fabricates.
*
* docker run -d --name opt-bi-pg -p 15455:5432 \
* -e POSTGRES_USER=opt -e POSTGRES_PASSWORD=opt_bi_throwaway \
* -e POSTGRES_DB=opt_bi_test postgres:16-alpine
* ANALYTICS_TEST_DATABASE_URL=postgres://opt:opt_bi_throwaway@localhost:15455/opt_bi_test \
* pnpm vitest run tests/analytics/analytics-domain-postgres.realinfra.test.ts
*
* The RESTORE half is the part that catches migration-chain rot: it replays
* sql/001..008 in order into a FRESH database, exactly as
* docs/runbook/backup-restore.md describes, and only then runs the
* persistence assertions. sql/003_datasets.sql exists because that same drill
* found a table no file ever created; this suite keeps that class of defect
* from coming back through sql/008.
*/
import { afterAll, beforeAll, describe, expect, it } from "vitest";
import fs from "node:fs";
import path from "node:path";
import postgres from "postgres";
import {
ANALYTICS_DOMAIN_ENTITIES,
ANALYTICS_ENTITY_TABLES,
IMMUTABLE_VERSION_ENTITIES,
} from "@/src/analytics/domain/entities";
import { AnalyticsDomainRepository } from "@/src/analytics/repositories/analytics-domain-repository";
import { contractHash, type DatasetContract } from "@/src/analytics/domain/contract";
const TEST_DATABASE_URL = process.env.ANALYTICS_TEST_DATABASE_URL;
const run = TEST_DATABASE_URL ? describe : describe.skip;
/** A schema of its own so the drill starts from genuinely nothing. */
const SCHEMA = `bi_p002_${Date.now().toString(36)}`;
let sql: ReturnType<typeof postgres> | undefined;
let reachable = false;
let unreachableReason = "";
/** Which sql/*.sql files the restore drill actually applied. */
const applied: string[] = [];
const ALPHA = { tenant_id: "t_alpha", project_id: "p_main" };
const BETA = { tenant_id: "t_beta", project_id: "p_main" };
function repo(): AnalyticsDomainRepository {
return new AnalyticsDomainRepository(sql!);
}
function contract(datasetId: string): DatasetContract {
return {
dataset_id: datasetId,
version: "1.0.0",
tier: "CURATED",
fields: [{ name: "id", type: "string", nullable: false, role: "identifier", primary_identifier: true }],
primary_identifiers: ["id"],
dimensions: [],
measures: [],
freshness_slo_seconds: 3600,
retention_days: 365,
pii_classification: "none",
source_lineage: ["src_x"],
quality_rules: [],
allowed_purposes: ["analytics"],
};
}
beforeAll(async () => {
if (!TEST_DATABASE_URL) {
console.warn("[realinfra] ANALYTICS_TEST_DATABASE_URL not set — skipping honestly, not fabricating success.");
return;
}
try {
sql = postgres(TEST_DATABASE_URL, { prepare: false, ssl: false, max: 3 });
await Promise.race([
sql`SELECT 1`,
new Promise((_, reject) => setTimeout(() => reject(new Error("connect timeout")), 5000)),
]);
// RESTORE DRILL: a brand-new schema, then every migration in order.
await sql.unsafe(`CREATE SCHEMA ${SCHEMA}`);
await sql.unsafe(`SET search_path TO ${SCHEMA}, public`);
const dir = path.join(process.cwd(), "sql");
for (const file of fs.readdirSync(dir).filter((f) => f.endsWith(".sql")).sort()) {
const body = fs.readFileSync(path.join(dir, file), "utf8");
try {
await sql.unsafe(`SET search_path TO ${SCHEMA}, public; ${body}`);
applied.push(file);
} catch (err) {
// 001 needs the `vector` extension, which a stock postgres:16-alpine
// does not carry. Record the skip rather than hiding it — the files
// sql/008 actually depends on must still apply.
applied.push(`${file} (FAILED: ${String(err).slice(0, 120)})`);
}
}
reachable = true;
} catch (err) {
reachable = false;
unreachableReason = err instanceof Error ? err.message : String(err);
}
}, 60_000);
afterAll(async () => {
if (sql && reachable) {
await sql.unsafe(`DROP SCHEMA IF EXISTS ${SCHEMA} CASCADE`).catch(() => undefined);
}
await sql?.end({ timeout: 5 });
});
run("BI-P002 analytics domain model against a LIVE Postgres", () => {
it("restore drill: sql/008 applies cleanly on top of the replayed chain", (ctx) => {
if (!reachable) ctx.skip();
expect(unreachableReason).toBe("");
const p008 = applied.find((f) => f.startsWith("008_"));
expect(p008, `sql/008 was not applied. Applied: ${applied.join(", ")}`).toBe(
"008_analytics_domain_model.sql",
);
});
it("all nineteen entity tables really exist after the restore", async (ctx) => {
if (!reachable) ctx.skip();
const rows = await sql!<{ table_name: string }[]>`
SELECT table_name FROM information_schema.tables WHERE table_schema = ${SCHEMA}`;
const present = new Set(rows.map((r) => r.table_name));
const missing = ANALYTICS_DOMAIN_ENTITIES.filter((e) => !present.has(ANALYTICS_ENTITY_TABLES[e]));
expect(missing).toEqual([]);
});
it("replaying sql/008 a second time is a no-op, not an error", async (ctx) => {
if (!reachable) ctx.skip();
const body = fs.readFileSync(path.join(process.cwd(), "sql", "008_analytics_domain_model.sql"), "utf8");
await expect(sql!.unsafe(`SET search_path TO ${SCHEMA}, public; ${body}`)).resolves.toBeDefined();
});
it("persists a dataset and reads it back through the repository", async (ctx) => {
if (!reachable) ctx.skip();
const r = repo();
const created = await r.createDataset(ALPHA, {
dataset_id: "ds_alpha_1",
name: "Alpha listings",
tier: "CURATED",
schema_contract_id: "dc_alpha",
schema_contract_version: "1.0.0",
owner_id: "team_data",
source_ref: "src_feed",
freshness_slo_seconds: 3600,
quality_rule_ids: [],
retention_days: 365,
classification: "none",
status: "active",
created_by: "usr_1",
});
expect(created.tenant_id).toBe("t_alpha");
const read = await r.getDataset(ALPHA, "ds_alpha_1");
expect(read?.name).toBe("Alpha listings");
expect(read?.freshness_slo_seconds).toBe(3600);
// Cross-check with direct SQL — proving the row is really in the table,
// not an artifact of the repository's own bookkeeping.
const direct = await sql!<{ c: string }[]>`
SELECT count(*)::text AS c FROM opt_analytics_dataset WHERE dataset_id = 'ds_alpha_1'`;
expect(Number(direct[0].c)).toBe(1);
});
it("TENANT ISOLATION: t_beta cannot read t_alpha's dataset, even with the exact id", async (ctx) => {
if (!reachable) ctx.skip();
const r = repo();
expect(await r.getDataset(BETA, "ds_alpha_1")).toBeUndefined();
expect(await r.listDatasets(BETA)).toEqual([]);
// And the row IS there — the empty result is scoping, not absence.
const direct = await sql!<{ c: string }[]>`SELECT count(*)::text AS c FROM opt_analytics_dataset`;
expect(Number(direct[0].c)).toBeGreaterThanOrEqual(1);
});
it("TENANT ISOLATION: a version cannot be minted against another tenant's dataset", async (ctx) => {
if (!reachable) ctx.skip();
await expect(
repo().createDatasetVersion(BETA, {
dataset_id: "ds_alpha_1",
contract_hash: "deadbeef",
schema_contract_version: "1.0.0",
}),
).rejects.toMatchObject({ code: "NOT_FOUND" });
});
it("dataset versions are monotonic and carry the contract hash", async (ctx) => {
if (!reachable) ctx.skip();
const r = repo();
const c = contract("ds_alpha_1");
const v1 = await r.createDatasetVersion(ALPHA, {
dataset_id: "ds_alpha_1",
contract_hash: contractHash(c),
schema_contract_version: "1.0.0",
});
const v2 = await r.createDatasetVersion(ALPHA, {
dataset_id: "ds_alpha_1",
contract_hash: contractHash({ ...c, version: "2.0.0" }),
schema_contract_version: "2.0.0",
});
expect(v1.version).toBe(1);
expect(v2.version).toBe(2);
expect(v2.contract_hash).not.toBe(v1.contract_hash);
expect((await r.listDatasetVersions(ALPHA, "ds_alpha_1")).map((v) => v.version)).toEqual([1, 2]);
});
it("IMMUTABILITY is enforced by the DATABASE: UPDATE and DELETE on a version row are refused", async (ctx) => {
if (!reachable) ctx.skip();
// Direct SQL, bypassing every TypeScript `readonly` — the only test that
// proves the guarantee survives a psql session.
await expect(
sql!.unsafe(
`UPDATE ${SCHEMA}.opt_analytics_dataset_version SET contract_hash = 'tampered' WHERE dataset_id = 'ds_alpha_1'`,
),
).rejects.toThrow(/immutable/i);
await expect(
sql!.unsafe(`DELETE FROM ${SCHEMA}.opt_analytics_dataset_version WHERE dataset_id = 'ds_alpha_1'`),
).rejects.toThrow(/immutable/i);
// The rows are untouched.
const rows = await sql!<{ contract_hash: string }[]>`
SELECT contract_hash FROM opt_analytics_dataset_version WHERE dataset_id = 'ds_alpha_1'`;
expect(rows).toHaveLength(2);
for (const r of rows) expect(r.contract_hash).not.toBe("tampered");
});
it("all four immutable entities refuse an UPDATE, not just the one that was tested first", async (ctx) => {
if (!reachable) ctx.skip();
const r = repo();
await r.createDataset(ALPHA, {
dataset_id: "ds_imm",
name: "Immutability fixture",
tier: "SERVING",
schema_contract_id: "dc_i",
schema_contract_version: "1.0.0",
owner_id: "team_data",
source_ref: "src",
freshness_slo_seconds: 60,
quality_rule_ids: [],
retention_days: 30,
classification: "none",
status: "active",
});
const metric = await r.createMetric(ALPHA, {
metric_id: "m_imm",
name: "immutable metric",
dataset_id: "ds_imm",
aggregation: "sum",
measure_field: "value",
allowed_dimension_ids: [],
owner_id: "team_data",
});
await r.createMetricVersion(ALPHA, { metric_id: metric.metric_id, definition_json: '{"v":1}' });
await r.createDatasetVersion(ALPHA, {
dataset_id: "ds_imm",
contract_hash: "h1",
schema_contract_version: "1.0.0",
});
await sql!.unsafe(`INSERT INTO ${SCHEMA}.opt_analytics_semantic_model
(semantic_model_id, tenant_id, project_id, name, version, model_json, model_hash, owner_id)
VALUES ('sm_1','t_alpha','p_main','model_a',1,'{}','h','team_data')`);
await sql!.unsafe(`INSERT INTO ${SCHEMA}.opt_analytics_dashboard_definition
(dashboard_id, tenant_id, project_id, name, owner_id) VALUES ('db_1','t_alpha','p_main','dash','team_data')`);
await sql!.unsafe(`INSERT INTO ${SCHEMA}.opt_analytics_dashboard_version
(dashboard_version_id, dashboard_id, tenant_id, project_id, version, layout_json, layout_hash)
VALUES ('dv_1','db_1','t_alpha','p_main',1,'{}','h')`);
const updates: Record<string, string> = {
DatasetVersion: `UPDATE ${SCHEMA}.opt_analytics_dataset_version SET contract_hash='x' WHERE dataset_id='ds_imm'`,
MetricVersion: `UPDATE ${SCHEMA}.opt_analytics_metric_version SET definition_json='{}' WHERE metric_id='m_imm'`,
SemanticModel: `UPDATE ${SCHEMA}.opt_analytics_semantic_model SET model_json='{}' WHERE semantic_model_id='sm_1'`,
DashboardVersion: `UPDATE ${SCHEMA}.opt_analytics_dashboard_version SET layout_json='{}' WHERE dashboard_version_id='dv_1'`,
};
for (const entity of IMMUTABLE_VERSION_ENTITIES) {
await expect(sql!.unsafe(updates[entity]), `${entity} is NOT immutable`).rejects.toThrow(/immutable/i);
}
});
it("a metric version after the first must state WHY the number changed meaning", async (ctx) => {
if (!reachable) ctx.skip();
const r = repo();
await expect(
r.createMetricVersion(ALPHA, { metric_id: "m_imm", definition_json: '{"v":2}' }),
).rejects.toThrow();
const v2 = await r.createMetricVersion(ALPHA, {
metric_id: "m_imm",
definition_json: '{"v":2}',
change_reason: "excluded internal test accounts from the numerator",
});
expect(v2.version).toBe(2);
});
it("a DatasetGrant without a purpose is refused by the DATABASE (purpose limitation)", async (ctx) => {
if (!reachable) ctx.skip();
// Regression pin: the first version of this constraint was written
// `array_length(allowed_purposes, 1) >= 1`, which is NULL for '{}' — and a
// CHECK evaluating to NULL PASSES in Postgres. It could never fail. This
// test found that; keep it.
await expect(
sql!.unsafe(`INSERT INTO ${SCHEMA}.opt_analytics_dataset_grant
(grant_id, dataset_id, tenant_id, project_id, subject_id, subject_type, level, allowed_purposes)
VALUES ('g_1','ds_imm','t_alpha','p_main','usr_1','user','read', '{}')`),
).rejects.toThrow(/purposes_nonempty/);
// A grant WITH a purpose is accepted, so the constraint is not simply
// rejecting everything.
await expect(
sql!.unsafe(`INSERT INTO ${SCHEMA}.opt_analytics_dataset_grant
(grant_id, dataset_id, tenant_id, project_id, subject_id, subject_type, level, allowed_purposes)
VALUES ('g_ok','ds_imm','t_alpha','p_main','usr_1','user','read', ARRAY['market_analytics'])`),
).resolves.toBeDefined();
});
it("a BIApplicationBinding without a purpose is refused too, and carries no credential column", async (ctx) => {
if (!reachable) ctx.skip();
await expect(
sql!.unsafe(`INSERT INTO ${SCHEMA}.opt_analytics_bi_application_binding
(binding_id, tenant_id, project_id, kind, name, principal_id, allowed_purposes)
VALUES ('b_1','t_alpha','p_main','superset','sup','svc_1','{}')`),
).rejects.toThrow(/purposes_nonempty/);
const cols = await sql!<{ column_name: string }[]>`
SELECT column_name FROM information_schema.columns
WHERE table_schema = ${SCHEMA} AND table_name = 'opt_analytics_bi_application_binding'`;
const names = cols.map((c) => c.column_name.toLowerCase());
for (const forbidden of ["password", "secret", "dsn", "token", "connection_string", "clickhouse_url"]) {
expect(names, `BI binding must never store ${forbidden}`).not.toContain(forbidden);
}
});
it("the dimension catalog is tenant-scoped and FAILS CLOSED for a tenant with none", async (ctx) => {
if (!reachable) ctx.skip();
const r = repo();
await r.createDimension(ALPHA, {
dimension_id: "dim_1",
name: "make",
dataset_id: "ds_imm",
field_name: "vehicle_make",
field_type: "string",
classification: "none",
});
expect(await r.dimensionCatalog(ALPHA)).toEqual({ make: "vehicle_make" });
// t_beta declared nothing, so it may reference nothing.
expect(await r.dimensionCatalog(BETA)).toEqual({});
});
it("a dimension whose physical field is not identifier-safe is refused by the DATABASE", async (ctx) => {
if (!reachable) ctx.skip();
await expect(
sql!.unsafe(`INSERT INTO ${SCHEMA}.opt_analytics_dimension_definition
(dimension_id, tenant_id, project_id, name, dataset_id, field_name, field_type)
VALUES ('dim_bad','t_alpha','p_main','evil','ds_imm', 'x''; DROP TABLE opt_analytics_dataset; --','string')`),
).rejects.toThrow();
});
it("the repository refuses to act without a verified tenant scope", async (ctx) => {
if (!reachable) ctx.skip();
const r = repo();
await expect(r.getDataset({ tenant_id: "", project_id: "p" }, "ds_imm")).rejects.toMatchObject({
code: "UNAUTHENTICATED",
});
await expect(r.listDatasets({ tenant_id: "t_alpha", project_id: " " })).rejects.toMatchObject({
code: "UNAUTHENTICATED",
});
});
});
@@ -0,0 +1,124 @@
/**
* BI-P009 DoD: "Provider contract tests added."
*
* This is the contract every `SiaxAnalyticsEngine` provider must satisfy
* regardless of engine — ClickHouse today, DuckDB (BI-P014) and Trino
* (BI-P015) later. It is written against the INTERFACE, so a future provider
* imports `analyticsEngineContract` and gets the same bar.
*
* The identity rules here are the ones that matter most: this estate has
* already shipped a live cross-tenant hole by reading the tenant out of a
* REQUEST BODY (INF0 Wave 2). So the contract asserts that a request field
* named `tenant_id` cannot influence the engine at all.
*/
import { describe, it, expect } from "vitest";
import { ClickHouseAnalyticsEngine } from "@/src/analytics/engine/clickhouse-engine";
import {
SIAX_ANALYTICS_ENGINE_CAPABILITIES,
type AnalyticsIdentity,
type SiaxAnalyticsEngine,
} from "@/src/analytics/engine/siax-analytics-engine";
const IDENTITY: AnalyticsIdentity = {
tenant_id: "t_alpha",
project_id: "p_default",
principal_id: "usr_1",
};
/** Reusable contract suite — future providers call this with their factory. */
export function analyticsEngineContract(name: string, make: () => SiaxAnalyticsEngine): void {
describe(`SiaxAnalyticsEngine contract: ${name}`, () => {
it("implements all eight BI-P009 capabilities as callable methods", () => {
const engine = make() as unknown as Record<string, unknown>;
for (const cap of SIAX_ANALYTICS_ENGINE_CAPABILITIES) {
expect(typeof engine[cap], `${cap} is not a method`).toBe("function");
}
expect(SIAX_ANALYTICS_ENGINE_CAPABILITIES).toEqual([
"health",
"executeMetric",
"executeQuery",
"estimateQuery",
"cancelQuery",
"explain",
"ingestProjection",
"refreshProjection",
]);
});
it("names its provider", () => {
expect(make().provider.length).toBeGreaterThan(0);
});
it("refuses every tenant-scoped capability without a verified identity", async () => {
const engine = make();
const blank = { tenant_id: "", project_id: "p", principal_id: "usr_1" };
const noPrincipal = { tenant_id: "t_alpha", project_id: "p", principal_id: "" };
for (const bad of [blank, noPrincipal]) {
await expect(engine.executeMetric(bad, { metric: "m", aggregate: "sum" })).rejects.toMatchObject({
code: "UNAUTHENTICATED",
});
await expect(engine.estimateQuery(bad, { metric: "m", aggregate: "sum" })).rejects.toMatchObject({
code: "UNAUTHENTICATED",
});
await expect(engine.ingestProjection(bad, { dataset_id: "ds", rows: [] })).rejects.toMatchObject({
code: "UNAUTHENTICATED",
});
await expect(engine.refreshProjection(bad, "ds")).rejects.toMatchObject({ code: "UNAUTHENTICATED" });
await expect(engine.cancelQuery(bad, "q")).rejects.toMatchObject({ code: "UNAUTHENTICATED" });
}
});
it("a tenant_id smuggled into the REQUEST cannot influence the engine (INF0 Wave-2 bug class)", async () => {
const engine = make();
const smuggled = { metric: "m", aggregate: "sum", tenant_id: "t_victim" } as never;
// The request type has no tenant field; if a provider ever started
// reading one, this call would target t_victim. The engine must still
// fail on the IDENTITY, proving the request field is inert.
await expect(
engine.executeMetric({ tenant_id: "", project_id: "p", principal_id: "u" }, smuggled),
).rejects.toMatchObject({ code: "UNAUTHENTICATED" });
});
it("cancelQuery refuses a query id belonging to another tenant", async () => {
const engine = make();
await expect(engine.cancelQuery(IDENTITY, "opt_deadbeefdead_someoneelses")).rejects.toMatchObject({
code: "AUTHORIZATION_DENIED",
});
});
it("health() reports unhealthy rather than throwing when the engine is unreachable", async () => {
const report = await make().health();
expect(report.healthy).toBe(false);
expect(report.provider.length).toBeGreaterThan(0);
// Never leak an engine body or a DSN into the operator-facing detail.
expect(report.detail ?? "").not.toMatch(/http|password|@/i);
});
});
}
analyticsEngineContract(
"clickhouse (unreachable host — contract-level assertions only)",
() =>
new ClickHouseAnalyticsEngine({
baseUrl: "http://clickhouse.invalid:8123",
database: "opt_contract_test",
timeoutMs: 500,
}),
);
describe("BI-P009 refusals that are deliberate, not missing", () => {
it("executeQuery (analyst SQL) is declared and REFUSED until BI-P017/BI-P033", async () => {
const engine = new ClickHouseAnalyticsEngine({ baseUrl: "http://clickhouse.invalid:8123" });
await expect(engine.executeQuery(IDENTITY, { sql: "SELECT 1", dataset_ids: [] })).rejects.toMatchObject({
code: "CAPABILITY_UNAVAILABLE",
});
});
it("the PRESERVED ClickHouseAnalytics is reachable through the engine, not replaced by it", () => {
const engine = new ClickHouseAnalyticsEngine({ baseUrl: "http://clickhouse.invalid:8123" });
expect(engine.legacyAnalytics.constructor.name).toBe("ClickHouseAnalytics");
expect(typeof engine.legacyAnalytics.queryMetric).toBe("function");
expect(typeof engine.legacyAnalytics.ensureSchema).toBe("function");
expect(typeof engine.legacyAnalytics.ingest).toBe("function");
});
});
+54
View File
@@ -0,0 +1,54 @@
/**
* The canonical BI-P005 fixture, in one place.
*
* Two jobs:
* 1. assert the encoder is DETERMINISTIC — the same columns must produce
* byte-identical output, or the "roundtrip" evidence would be comparing
* two different payloads run to run;
* 2. when ARROW_FIXTURE_DIR is set, write those exact bytes to disk so
* scripts/bi-p005-arrow-roundtrip.mjs hands pyarrow and DuckDB the bytes
* the SHIPPED module produces, not a copy of the encoder living in a
* script.
*/
import { describe, it, expect } from "vitest";
import fs from "node:fs";
import path from "node:path";
import crypto from "node:crypto";
import { type ArrowColumn, encodeArrowStream } from "@/src/analytics/arrow/ipc";
/** The same three rows the ClickHouse side of the DoD produces. */
export const BI_P005_FIXTURE: ArrowColumn[] = [
{ name: "tenant_id", type: "utf8", values: ["t_alpha", "t_beta", "t_gamma"] },
{ name: "n", type: "int64", values: [1, 2, 3] },
{ name: "v", type: "float64", values: [1.5, 2.5, 3.25] },
{ name: "flag", type: "bool", values: [true, false, true] },
{
name: "ts",
type: "timestamp_ms",
values: [
new Date("2026-01-01T00:00:00.000Z"),
new Date("2026-06-15T08:09:10.123Z"),
new Date("2026-09-06T12:34:56.789Z"),
],
},
];
describe("BI-P005 canonical Arrow fixture", () => {
it("encodes deterministically", () => {
const a = encodeArrowStream(BI_P005_FIXTURE);
const b = encodeArrowStream(BI_P005_FIXTURE);
expect(Buffer.from(a).equals(Buffer.from(b))).toBe(true);
expect(a.length).toBeGreaterThan(0);
});
it("writes the fixture for the external roundtrip when asked", () => {
const dir = process.env.ARROW_FIXTURE_DIR;
if (!dir) return; // not an error: the normal test run has nowhere to put it
const bytes = encodeArrowStream(BI_P005_FIXTURE);
const out = path.join(dir, "02_0pt_encoded.arrow");
fs.writeFileSync(out, bytes);
expect(fs.statSync(out).size).toBe(bytes.length);
// Print the digest so the evidence file can be tied to these exact bytes.
console.log(`ARROW_FIXTURE_SHA256=${crypto.createHash("sha256").update(bytes).digest("hex")}`);
});
});
+101
View File
@@ -0,0 +1,101 @@
/**
* BI-P005 — APACHE ARROW INTERCHANGE, the part provable without a live engine.
*
* Honest scope note: the tests here run 0pt's encoder against 0pt's decoder,
* which proves internal consistency and NOT specification conformance. The
* conformance evidence comes from two places where an INDEPENDENT
* implementation reads the bytes:
*
* * tests/integration/analytics-arrow-clickhouse.integration.test.ts —
* ClickHouse (Arrow C++) both produces Arrow this decoder reads and
* consumes Arrow this encoder writes. Runs on every push in CI.
* * evidence/bi-p002-p012/bi-p005-arrow-roundtrip.json — pyarrow and DuckDB
* decoding the same bytes, executed against real containers.
*
* That distinction is the point: a self-roundtrip that agrees with its own
* mistakes is exactly the kind of green this estate has been burned by.
*/
import { describe, it, expect } from "vitest";
import {
ARROW_TYPE_SUPPORT,
arrowStreamSettings,
assertArrowWithinResultBudget,
clickhouseArrowStreamFormat,
decodeArrowStream,
encodeArrowStream,
} from "@/src/analytics/arrow/ipc";
describe("BI-P005 Arrow IPC", () => {
it("emits a well-formed IPC stream: continuation markers, schema, batch, EOS", () => {
const bytes = encodeArrowStream([{ name: "n", type: "int64", values: [1, 2, 3] }]);
const dv = new DataView(bytes.buffer, bytes.byteOffset, bytes.byteLength);
expect(dv.getUint32(0, true)).toBe(0xffffffff);
// Last 8 bytes are the end-of-stream marker: 0xFFFFFFFF then length 0.
expect(dv.getUint32(bytes.length - 8, true)).toBe(0xffffffff);
expect(dv.getInt32(bytes.length - 4, true)).toBe(0);
});
it("round-trips every supported column type with nulls", () => {
const bytes = encodeArrowStream([
{ name: "tenant_id", type: "utf8", values: ["t_alpha", "t_beta", null] },
{ name: "n", type: "int64", values: [1, 2, 3] },
{ name: "v", type: "float64", values: [1.5, 2.5, null] },
{ name: "flag", type: "bool", values: [true, false, true] },
{ name: "ts", type: "timestamp_ms", values: [new Date("2026-01-01T00:00:00Z"), null, new Date("2026-09-06T12:34:56.789Z")] },
]);
const info = decodeArrowStream(bytes);
expect(info.rowCount).toBe(3);
expect(info.batchCount).toBe(1);
expect(info.fields.map((f) => f.name)).toEqual(["tenant_id", "n", "v", "flag", "ts"]);
// Arrow Type union tags: Int=2, FloatingPoint=3, Utf8=5, Bool=6, Timestamp=10.
expect(info.fields.map((f) => f.typeType)).toEqual([5, 2, 3, 6, 10]);
});
it("handles an empty result set without producing a malformed stream", () => {
const bytes = encodeArrowStream([{ name: "n", type: "int64", values: [] }]);
const info = decodeArrowStream(bytes);
expect(info.rowCount).toBe(0);
expect(info.fields).toEqual([{ name: "n", typeType: 2 }]);
});
it("rejects ragged columns rather than emitting a stream that decodes to garbage", () => {
expect(() =>
encodeArrowStream([
{ name: "a", type: "int64", values: [1, 2] },
{ name: "b", type: "int64", values: [1] },
]),
).toThrowError(/expected 2/);
expect(() => encodeArrowStream([])).toThrowError(/at least one column/);
});
it("refuses an int64 value outside the representable range instead of wrapping it", () => {
expect(() =>
encodeArrowStream([{ name: "n", type: "int64", values: [BigInt("9223372036854775808")] }]),
).toThrowError(/out of int64 range/);
});
it("rejects bytes that are not an Arrow IPC stream", () => {
expect(() => decodeArrowStream(new Uint8Array([1, 2, 3, 4, 5, 6, 7, 8]))).toThrowError(/not an Arrow IPC stream/);
});
it("BI-P012: Arrow is not a way around max_result_rows", () => {
const bytes = encodeArrowStream([{ name: "n", type: "int64", values: [1, 2, 3, 4, 5] }]);
expect(() => assertArrowWithinResultBudget(bytes, 3)).toThrowError(/over the 3-row budget/);
expect(assertArrowWithinResultBudget(bytes, 5).rowCount).toBe(5);
});
it("the ClickHouse pass-through asks for text Strings, not Binary", () => {
expect(clickhouseArrowStreamFormat()).toBe("ArrowStream");
// Without this setting ClickHouse emits String as Arrow Binary, which
// pyarrow and DuckDB read as bytes — silent semantic loss.
expect(arrowStreamSettings().output_format_arrow_string_as_string).toBe("1");
});
it("documents the supported type mapping including where loss WOULD occur", () => {
expect(ARROW_TYPE_SUPPORT.length).toBeGreaterThanOrEqual(5);
const arrowTypes = ARROW_TYPE_SUPPORT.map((r) => r.arrow);
expect(new Set(arrowTypes)).toEqual(new Set(["int64", "float64", "utf8", "bool", "timestamp_ms"]));
// Every row states its caveat; an empty note would be a silent promise.
for (const row of ARROW_TYPE_SUPPORT) expect(row.note.length).toBeGreaterThan(10);
});
});
+164
View File
@@ -0,0 +1,164 @@
/**
* BI-P004 — ANALYTICS DATA CONTRACT.
*
* DoD: "Breaking dataset schema changes fail compatibility gate unless
* explicitly versioned." Each breaking change class below is tested TWICE:
* once under a minor bump (must be REFUSED) and once under a major bump (must
* be ACCEPTED). A gate that only ever says yes, or only ever says no, would
* pass a one-sided test; this one cannot.
*/
import { describe, it, expect } from "vitest";
import {
assertContractCompatible,
checkContractCompatibility,
contractHash,
diffContracts,
parseSemVer,
validateContract,
type DatasetContract,
} from "@/src/analytics/domain/contract";
function base(): DatasetContract {
return {
dataset_id: "ds_vehicle_listings",
version: "1.2.0",
tier: "CURATED",
fields: [
{ name: "listing_id", type: "string", nullable: false, role: "identifier", primary_identifier: true },
{ name: "make", type: "string", nullable: false, role: "dimension" },
{ name: "asking_price", type: "int64", nullable: true, role: "measure" },
{ name: "listed_at", type: "timestamp", nullable: false, role: "attribute" },
],
primary_identifiers: ["listing_id"],
dimensions: ["make"],
measures: ["asking_price"],
freshness_slo_seconds: 3600,
retention_days: 730,
pii_classification: "none",
source_lineage: ["src_dealer_feed"],
quality_rules: [{ rule_id: "qr_listing_id_unique", kind: "unique", field_name: "listing_id", severity: "block" }],
allowed_purposes: ["market_analytics"],
};
}
/** Apply a mutation and set the version. */
function next(mutate: (c: DatasetContract) => void, version: string): DatasetContract {
const c: DatasetContract = JSON.parse(JSON.stringify(base()));
mutate(c);
c.version = version;
return c;
}
const BREAKING_CASES: { name: string; mutate: (c: DatasetContract) => void; code: string }[] = [
{ name: "field removed", mutate: (c) => { c.fields = c.fields.filter((f) => f.name !== "make"); c.dimensions = []; }, code: "field_removed" },
{ name: "field type changed", mutate: (c) => { c.fields[2].type = "float64"; }, code: "field_type_changed" },
{ name: "field became nullable", mutate: (c) => { c.fields[1].nullable = true; }, code: "field_became_nullable" },
{ name: "field role changed", mutate: (c) => { c.fields[1].role = "measure"; }, code: "field_role_changed" },
{ name: "required field added", mutate: (c) => { c.fields.push({ name: "vat_rate", type: "float64", nullable: false, role: "attribute" }); }, code: "required_field_added" },
{ name: "primary identifier changed", mutate: (c) => { c.primary_identifiers = ["make"]; }, code: "primary_identifier_changed" },
{ name: "dimension removed", mutate: (c) => { c.dimensions = []; }, code: "dimension_removed" },
{ name: "measure removed", mutate: (c) => { c.measures = []; }, code: "measure_removed" },
{ name: "freshness SLO loosened", mutate: (c) => { c.freshness_slo_seconds = 86400; }, code: "freshness_slo_loosened" },
{ name: "retention shortened", mutate: (c) => { c.retention_days = 30; }, code: "retention_shortened" },
{ name: "allowed purpose removed", mutate: (c) => { c.allowed_purposes = []; }, code: "purpose_removed" },
{ name: "PII classification raised", mutate: (c) => { c.pii_classification = "personal"; }, code: "pii_classification_raised" },
{ name: "blocking quality rule removed", mutate: (c) => { c.quality_rules = []; }, code: "blocking_quality_rule_removed" },
{ name: "blocking quality rule downgraded", mutate: (c) => { c.quality_rules[0].severity = "warn"; }, code: "blocking_quality_rule_downgraded" },
{ name: "tier changed", mutate: (c) => { c.tier = "SERVING"; }, code: "tier_changed" },
];
describe("BI-P004 compatibility gate", () => {
it.each(BREAKING_CASES)("REFUSES $name under a minor bump", ({ mutate, code }) => {
const verdict = checkContractCompatibility(base(), next(mutate, "1.3.0"));
expect(verdict.compatible).toBe(false);
expect(verdict.breaking.map((b) => b.code)).toContain(code);
expect(verdict.reason).toMatch(/MAJOR version bump/);
});
it.each(BREAKING_CASES)("ACCEPTS $name once explicitly versioned (major bump)", ({ mutate, code }) => {
const verdict = checkContractCompatibility(base(), next(mutate, "2.0.0"));
expect(verdict.compatible).toBe(true);
expect(verdict.explicitly_versioned).toBe(true);
expect(verdict.breaking.map((b) => b.code)).toContain(code);
});
it("a patch bump is equally refused — the gate is about the MAJOR, not about moving at all", () => {
const verdict = checkContractCompatibility(base(), next((c) => c.fields.pop(), "1.2.1"));
expect(verdict.compatible).toBe(false);
});
it("allows genuinely additive changes without a major bump", () => {
const additive = next((c) => {
c.fields.push({ name: "mileage_km", type: "int64", nullable: true, role: "measure" });
c.measures.push("mileage_km");
c.quality_rules.push({ rule_id: "qr_mileage_range", kind: "range", field_name: "mileage_km", severity: "warn" });
c.allowed_purposes.push("valuation");
c.retention_days = 1095;
c.freshness_slo_seconds = 1800;
}, "1.3.0");
const verdict = checkContractCompatibility(base(), additive);
expect(verdict.compatible).toBe(true);
expect(verdict.breaking).toEqual([]);
expect(verdict.explicitly_versioned).toBe(false);
});
it("refuses a version that does not move forward", () => {
expect(checkContractCompatibility(base(), next(() => {}, "1.2.0")).compatible).toBe(false);
expect(checkContractCompatibility(base(), next(() => {}, "1.1.9")).compatible).toBe(false);
expect(checkContractCompatibility(base(), next(() => {}, "0.9.0")).compatible).toBe(false);
});
it("treats date -> timestamp as a safe widening, and int64 -> float64 as NOT one", () => {
const widened = next((c) => {
c.fields.push({ name: "sold_on", type: "date", nullable: true, role: "attribute" });
}, "1.3.0");
const later = JSON.parse(JSON.stringify(widened)) as DatasetContract;
later.fields[later.fields.length - 1].type = "timestamp";
later.version = "1.4.0";
expect(checkContractCompatibility(widened, later).compatible).toBe(true);
const lossy = next((c) => { c.fields[2].type = "float64"; }, "1.3.0");
expect(checkContractCompatibility(base(), lossy).compatible).toBe(false);
});
it("assertContractCompatible throws a CONFLICT rather than returning a verdict nobody reads", () => {
expect(() => assertContractCompatible(base(), next((c) => c.fields.pop(), "1.3.0"))).toThrowError(/MAJOR/);
expect(() => assertContractCompatible(base(), next(() => {}, "1.3.0"))).not.toThrow();
});
it("refuses to compare contracts for different datasets", () => {
const other = next((c) => { c.dataset_id = "ds_other"; }, "1.3.0");
expect(() => diffContracts(base(), other)).toThrowError(/SAME dataset/);
});
it("validateContract catches internally inconsistent contracts", () => {
expect(() => validateContract(base())).not.toThrow();
expect(() => validateContract({ ...base(), version: "1.2" })).toThrowError(/major.minor.patch/);
expect(() => validateContract({ ...base(), fields: [] })).toThrowError(/no fields/);
expect(() => validateContract({ ...base(), dimensions: ["nope"] })).toThrowError(/unknown field/);
expect(() => validateContract({ ...base(), primary_identifiers: [] })).toThrowError(/no primary identifier/);
expect(() => validateContract({ ...base(), allowed_purposes: [] })).toThrowError(/allowed_purposes/);
expect(() => validateContract({ ...base(), freshness_slo_seconds: 0 })).toThrowError(/freshness/);
const dup = base();
dup.fields.push({ ...dup.fields[0] });
expect(() => validateContract(dup)).toThrowError(/duplicate field/);
});
it("contractHash is order-insensitive over fields but sensitive to meaning", () => {
const a = base();
const reordered = JSON.parse(JSON.stringify(a)) as DatasetContract;
reordered.fields.reverse();
expect(contractHash(reordered)).toBe(contractHash(a));
const changed = JSON.parse(JSON.stringify(a)) as DatasetContract;
changed.fields[1].nullable = true;
expect(contractHash(changed)).not.toBe(contractHash(a));
});
it("parseSemVer rejects anything that is not major.minor.patch", () => {
expect(parseSemVer("2.10.3")).toEqual({ major: 2, minor: 10, patch: 3 });
for (const bad of ["v1.0.0", "1.0", "1.0.0-rc1", "latest", ""]) {
expect(() => parseSemVer(bad)).toThrowError(/major.minor.patch/);
}
});
});
@@ -0,0 +1,224 @@
/**
* BI-P011's adversarial list, at the compiler layer:
*
* tenant A query B dataset · tenant A forged dataset ID · raw SQL attempt
* SQL injection · dimension identifier injection · filter injection
*
* These are the cases that never reach the network. The cases that DO reach a
* real server — and the with-fix/without-fix proof BI-P011 actually demands —
* are tests/integration/analytics-tenant-isolation.integration.test.ts.
*/
import { describe, it, expect } from "vitest";
import {
assertSafeIdentifier,
compileMetricQuery,
compileProjectionQuery,
dimensionCatalogFrom,
restrictedTenantUserDdl,
} from "@/src/analytics/engine/query-compiler";
const catalog = dimensionCatalogFrom({ severity: "severity", make: "vehicle_make" });
const HOSTILE = [
"x'; DROP TABLE opt_analytics_event; --",
"1 OR 1=1",
"tenant_id",
"*/ OR 1=1 /*",
"severity UNION ALL SELECT tenant_id FROM opt_analytics_event",
"dim_values[1]",
"__proto__",
"constructor",
"sever ity",
"sévérité",
"a".repeat(200),
"",
];
describe("BI-P011 injection surfaces at the compiler", () => {
it("dimension identifier injection: hostile group_by names are refused, not escaped", () => {
for (const bad of HOSTILE) {
expect(() =>
compileMetricQuery(
{ tenant_id: "t_alpha", metric: "m", aggregate: "sum", group_by: [bad] },
"opt_analytics",
catalog,
1000,
),
`group_by "${bad}" was not refused`,
).toThrowError(/unknown group_by dimension/);
}
});
it("filter injection: hostile filter KEYS are refused by catalog resolution", () => {
for (const bad of HOSTILE) {
expect(() =>
compileMetricQuery(
{ tenant_id: "t_alpha", metric: "m", aggregate: "sum", filters: { [bad]: "v" } },
"opt_analytics",
catalog,
1000,
),
`filter key "${bad}" was not refused`,
).toThrowError(/unknown filter dimension/);
}
});
it("prototype-chain keys resolve to undefined rather than to Object.prototype members", () => {
// A naive `catalog[name]` lookup would resolve "constructor" to a
// function and "toString" to a method; the Map-based catalog must not.
for (const key of ["constructor", "toString", "hasOwnProperty", "__proto__", "valueOf"]) {
expect(catalog.resolve(key)).toBeUndefined();
}
});
it("filter VALUES are never spliced, however hostile", () => {
const compiled = compileMetricQuery(
{
tenant_id: "t_alpha",
metric: "m",
aggregate: "sum",
filters: { make: "volvo'; DROP TABLE opt_analytics_event; --" },
},
"opt_analytics",
catalog,
1000,
);
expect(compiled.sql).not.toContain("DROP TABLE");
expect(compiled.sql).not.toContain("volvo");
expect(compiled.params.param_f0v).toBe("volvo'; DROP TABLE opt_analytics_event; --");
// The physical column name came from the catalog, not the caller.
expect(compiled.params.param_f0n).toBe("vehicle_make");
});
it("metric and dataset VALUES are never spliced either", () => {
const compiled = compileMetricQuery(
{
tenant_id: "t_alpha",
metric: "m'; SELECT 1; --",
dataset_id: "ds'; SELECT 1; --",
aggregate: "sum",
},
"opt_analytics",
catalog,
1000,
);
expect(compiled.sql).not.toContain("SELECT 1");
expect(compiled.sql.match(/SELECT/g)).toHaveLength(1);
});
it("raw SQL attempt: there is no request shape that reaches a second table", () => {
const compiled = compileMetricQuery(
{ tenant_id: "t_alpha", metric: "m", aggregate: "sum" },
"opt_analytics",
catalog,
1000,
);
expect(compiled.sql).not.toMatch(/\bUNION\b/i);
expect(compiled.sql).not.toMatch(/\bJOIN\b/i);
expect((compiled.sql.match(/\bFROM\b/gi) ?? []).length).toBe(1);
expect(compiled.table).toBe("opt_analytics_event");
});
it("a forged table name is refused rather than reaching the FROM clause", () => {
expect(() =>
compileMetricQuery(
{ tenant_id: "t_alpha", metric: "m", aggregate: "sum", table: "system.tables" as never },
"opt_analytics",
catalog,
1000,
),
).toThrowError(/invalid table identifier/);
});
it("an unsupported aggregate is refused rather than being interpolated", () => {
expect(() =>
compileMetricQuery(
{ tenant_id: "t_alpha", metric: "m", aggregate: "sum(1),tenant_id" as never },
"opt_analytics",
catalog,
1000,
),
).toThrowError(/unsupported aggregate/);
});
it("group-by cardinality is capped so a caller cannot explode the GROUP BY", () => {
const many = Array.from({ length: 9 }, () => "severity");
expect(() =>
compileMetricQuery(
{ tenant_id: "t_alpha", metric: "m", aggregate: "sum", group_by: many },
"opt_analytics",
catalog,
1000,
),
).toThrowError(/at most 8 group-by/);
});
it("LIMIT is clamped to the governance ceiling, and never below 1", () => {
const big = compileMetricQuery(
{ tenant_id: "t_alpha", metric: "m", aggregate: "sum", limit: 10_000_000 },
"opt_analytics",
catalog,
500,
);
expect(big.sql).toContain("LIMIT 500");
const zero = compileMetricQuery(
{ tenant_id: "t_alpha", metric: "m", aggregate: "sum", limit: 0 },
"opt_analytics",
catalog,
500,
);
expect(zero.sql).toContain("LIMIT 1");
});
it("the projection compiler forces the tenant and parameterizes entity keys", () => {
const compiled = compileProjectionQuery(
{ tenant_id: "t_alpha", dataset_id: "ds_x", entity_keys: ["a'; DROP", "b"] },
"opt_analytics",
100,
);
expect(compiled.sql).toMatch(/WHERE tenant_id = \{tenant:String\}/);
expect(compiled.sql).not.toContain("DROP");
expect(compiled.params.param_entityKeys).toBe(JSON.stringify(["a'; DROP", "b"]));
});
it("assertSafeIdentifier accepts real identifiers and rejects everything else", () => {
for (const good of ["severity", "vehicle_make", "_x", "a1_b2"]) {
expect(assertSafeIdentifier(good, "test")).toBe(good);
}
for (const bad of ["1abc", "a-b", "a b", "a;b", "", "a".repeat(64)]) {
expect(() => assertSafeIdentifier(bad, "test")).toThrowError(/invalid test identifier/);
}
});
it("restrictedTenantUserDdl refuses a tenant id or password it cannot safely embed", () => {
expect(() =>
restrictedTenantUserDdl({
db: "opt_analytics",
tenantId: "t_alpha' OR '1'='1",
userName: "opt_t_alpha",
password: "pw",
tables: ["opt_analytics_event"],
}),
).toThrowError(/not policy-safe/);
expect(() =>
restrictedTenantUserDdl({
db: "opt_analytics",
tenantId: "t_alpha",
userName: "opt_t_alpha",
password: "pw' OR '1'='1",
tables: ["opt_analytics_event"],
}),
).toThrowError(/cannot be embedded/);
const stmts = restrictedTenantUserDdl({
db: "opt_analytics",
tenantId: "t_alpha",
userName: "opt_t_alpha",
password: "safe_pw_123",
tables: ["opt_analytics_event"],
});
expect(stmts.join(" ")).toContain("CREATE ROW POLICY IF NOT EXISTS");
expect(stmts.join(" ")).toContain("USING tenant_id = 't_alpha'");
});
});
+118
View File
@@ -0,0 +1,118 @@
/**
* BI-P012 — CLICKHOUSE RESOURCE GOVERNANCE, the part provable without an
* engine. The "intentional query bomb is terminated" half of the DoD needs a
* real server and lives in
* tests/integration/analytics-resource-governance.integration.test.ts.
*/
import { describe, it, expect } from "vitest";
import {
applyQuota,
BUDGET_TIERS,
clickhouseSettings,
ConcurrencyGate,
concurrencyProfileSql,
DEFAULT_BUDGETS,
REQUIRED_GOVERNANCE_SETTINGS,
} from "@/src/analytics/engine/resource-governance";
describe("BI-P012 resource governance", () => {
it("every setting the program names has a value somewhere in the enforcement path", () => {
const settings = clickhouseSettings(DEFAULT_BUDGETS.INTERACTIVE);
const profile = concurrencyProfileSql("opt_t_alpha", DEFAULT_BUDGETS.INTERACTIVE).join(" ");
for (const name of REQUIRED_GOVERNANCE_SETTINGS) {
const present = name in settings || profile.includes(name) || name === "max_concurrent_queries";
expect(present, `${name} is not enforced anywhere`).toBe(true);
}
// The five per-statement ones must be on the STATEMENT, not only in a
// profile a future operator might forget to attach.
for (const name of [
"max_execution_time",
"max_result_rows",
"max_rows_to_read",
"max_bytes_to_read",
"max_memory_usage",
]) {
expect(settings[name], `${name} missing from per-statement settings`).toBeDefined();
}
});
it("readonly=1 rides on every SELECT so the query text cannot re-set the limits", () => {
for (const tier of BUDGET_TIERS) {
expect(clickhouseSettings(DEFAULT_BUDGETS[tier]).readonly).toBe("1");
}
});
it("overflow modes are THROW, not BREAK — a partial result must never look complete", () => {
const s = clickhouseSettings(DEFAULT_BUDGETS.INTERACTIVE);
expect(s.result_overflow_mode).toBe("throw");
expect(s.read_overflow_mode).toBe("throw");
expect(s.timeout_overflow_mode).toBe("throw");
});
it("a CL0UD quota can only TIGHTEN a tier, never loosen it", () => {
const base = DEFAULT_BUDGETS.INTERACTIVE;
const loosened = applyQuota(base, {
maxExecutionTimeSec: 100000,
maxResultRows: 10_000_000,
maxRowsToRead: Number.MAX_SAFE_INTEGER,
maxBytesToRead: Number.MAX_SAFE_INTEGER,
maxMemoryUsageBytes: Number.MAX_SAFE_INTEGER,
maxConcurrentQueries: 1000,
});
expect(loosened).toEqual(base);
const tightened = applyQuota(base, { maxResultRows: 100, maxConcurrentQueries: 1 });
expect(tightened.maxResultRows).toBe(100);
expect(tightened.maxConcurrentQueries).toBe(1);
expect(tightened.maxExecutionTimeSec).toBe(base.maxExecutionTimeSec);
});
it("a nonsense quota (zero, negative, NaN) falls back to the tier rather than to no limit", () => {
const base = DEFAULT_BUDGETS.INTERACTIVE;
expect(applyQuota(base, { maxResultRows: 0 }).maxResultRows).toBe(base.maxResultRows);
expect(applyQuota(base, { maxResultRows: -5 }).maxResultRows).toBe(base.maxResultRows);
expect(applyQuota(base, { maxResultRows: Number.NaN }).maxResultRows).toBe(base.maxResultRows);
expect(applyQuota(base, undefined)).toEqual(base);
});
it("tiers get progressively more generous, and INTERACTIVE is the mean default", () => {
expect(DEFAULT_BUDGETS.INTERACTIVE.maxExecutionTimeSec).toBeLessThan(DEFAULT_BUDGETS.ANALYTICAL.maxExecutionTimeSec);
expect(DEFAULT_BUDGETS.ANALYTICAL.maxExecutionTimeSec).toBeLessThan(DEFAULT_BUDGETS.BATCH.maxExecutionTimeSec);
expect(DEFAULT_BUDGETS.INTERACTIVE.maxRowsToRead).toBeLessThan(DEFAULT_BUDGETS.ANALYTICAL.maxRowsToRead);
});
it("the concurrency gate throttles ONE tenant without touching another", () => {
const gate = new ConcurrencyGate((t) => (t === "t_noisy" ? 2 : 5));
const a = gate.acquire("t_noisy");
const b = gate.acquire("t_noisy");
expect(gate.inFlightFor("t_noisy")).toBe(2);
expect(() => gate.acquire("t_noisy")).toThrowError(/queries in flight/);
// The quiet tenant is unaffected — BI-P012's "without degrading unrelated
// tenant" in the smallest testable form.
const quiet = gate.acquire("t_quiet");
expect(gate.inFlightFor("t_quiet")).toBe(1);
a();
expect(gate.inFlightFor("t_noisy")).toBe(1);
expect(() => gate.acquire("t_noisy")).not.toThrow();
b();
quiet();
});
it("releasing twice does not free a slot that was never taken", () => {
const gate = new ConcurrencyGate(() => 1);
const release = gate.acquire("t_x");
release();
release();
expect(gate.inFlightFor("t_x")).toBe(0);
expect(gate.totalInFlight()).toBe(0);
});
it("the settings profile SQL carries max_concurrent_queries' server-side half", () => {
const stmts = concurrencyProfileSql("opt_t_alpha", DEFAULT_BUDGETS.INTERACTIVE);
expect(stmts.join(" ")).toContain("CREATE SETTINGS PROFILE IF NOT EXISTS opt_t_alpha");
expect(stmts.join(" ")).toContain("readonly = 1");
expect(stmts.join(" ")).toContain("CREATE QUOTA IF NOT EXISTS opt_t_alpha_quota");
});
});
@@ -0,0 +1,144 @@
/**
* BI-P009's non-negotiable line: "Do not regress the existing forced tenant
* predicate."
*
* This suite is written to FAIL IF THE FORCING IS REMOVED, which is a
* different thing from testing that it currently works. Concretely, each test
* below would go red under a specific, plausible regression:
*
* * someone makes `tenant_id` optional in the metric DSL -> "refuses a
* blank tenant" fails;
* * someone "simplifies" the WHERE builder and drops the tenant condition
* -> "every compiled statement contains the predicate" fails on a missing
* substring;
* * someone starts splicing the tenant value into SQL text instead of
* binding it -> "the tenant VALUE never appears in the SQL text" fails;
* * someone lets a caller pass a tenant through the request body rather
* than the verified identity -> "identity wins over any request field"
* fails.
*
* It covers BOTH compilers: the PRESERVED `ClickHouseAnalytics.queryMetric`
* (src/analytics/clickhouse-analytics.ts, untouched by BI-P009) and the new
* `compileMetricQuery` — because "preserved" means the old guarantee has to
* keep holding, not merely that the file still exists.
*
* The ClickHouseAnalytics half asserts against the URL the adapter actually
* builds, captured with a stub `fetch`. That is deliberate: a mock ENGINE
* could be made to return anything, but the SQL text on the wire is the thing
* the real server would receive, so asserting on it cannot be faked by the
* test double.
*/
import { describe, it, expect, afterEach, vi } from "vitest";
import { ClickHouseAnalytics } from "@/src/analytics/clickhouse-analytics";
import {
compileMetricQuery,
compileProjectionQuery,
dimensionCatalogFrom,
FORCED_TENANT_PREDICATE,
} from "@/src/analytics/engine/query-compiler";
const catalog = dimensionCatalogFrom({ severity: "severity", make: "vehicle_make" });
/** Captures every URL the adapter fetches, returning an empty JSON result. */
function captureFetch(): { urls: string[]; restore: () => void } {
const urls: string[] = [];
const original = globalThis.fetch;
globalThis.fetch = (async (input: RequestInfo | URL) => {
urls.push(String(input));
return new Response(JSON.stringify({ data: [] }), { status: 200 });
}) as typeof fetch;
return { urls, restore: () => (globalThis.fetch = original) };
}
afterEach(() => {
vi.restoreAllMocks();
});
describe("BI-P009 regression guard: the forced tenant predicate", () => {
it("PRESERVED ClickHouseAnalytics still emits tenant_id = {tenant:String} on the wire", async () => {
const cap = captureFetch();
try {
const analytics = new ClickHouseAnalytics({ baseUrl: "http://ch.invalid:8123", database: "opt_test" });
await analytics.queryMetric({ tenant_id: "t_alpha", metric: "rows", aggregate: "sum" });
expect(cap.urls).toHaveLength(1);
// URLSearchParams decodes `+` as a space, which raw decodeURIComponent
// does not — read the statement the way the server will.
const query = new URL(cap.urls[0]).searchParams.get("query") ?? "";
expect(query).toContain("tenant_id = {tenant:String}");
expect(query).toMatch(/WHERE tenant_id = \{tenant:String\}/);
// The VALUE travels as a bound parameter, never inside the statement.
expect(query).not.toContain("t_alpha");
expect(new URL(cap.urls[0]).searchParams.get("param_tenant")).toBe("t_alpha");
} finally {
cap.restore();
}
});
it("PRESERVED ClickHouseAnalytics still refuses a blank tenant_id", async () => {
const analytics = new ClickHouseAnalytics({ baseUrl: "http://ch.invalid:8123" });
await expect(analytics.queryMetric({ tenant_id: " ", metric: "x", aggregate: "sum" })).rejects.toMatchObject({
code: "INVALID_REQUEST",
});
});
it("PRESERVED ClickHouseAnalytics still refuses a hostile group_by identifier", async () => {
const analytics = new ClickHouseAnalytics({ baseUrl: "http://ch.invalid:8123" });
await expect(
analytics.queryMetric({
tenant_id: "t_alpha",
metric: "x",
aggregate: "sum",
group_by: "sev'; DROP TABLE opt_metric_events; --",
}),
).rejects.toMatchObject({ code: "INVALID_REQUEST" });
});
it("every statement the NEW compiler produces contains the forced predicate", () => {
const shapes = [
{ metric: "m", aggregate: "sum" as const },
{ metric: "m", aggregate: "count" as const, group_by: ["severity"] },
{ metric: "m", aggregate: "avg" as const, filters: { make: "volvo" }, dataset_id: "ds_1" },
{ metric: "m", aggregate: "max" as const, from_time: "2026-01-01", to_time: "2026-02-01" },
{ metric: "m", aggregate: "count_distinct" as const, group_by: ["severity", "make"], limit: 5 },
];
for (const shape of shapes) {
const compiled = compileMetricQuery({ tenant_id: "t_alpha", ...shape }, "opt_analytics", catalog, 1000);
expect(compiled.sql).toContain(FORCED_TENANT_PREDICATE);
// The predicate must be in the WHERE, not in a comment or an alias.
expect(compiled.sql).toMatch(/WHERE tenant_id = \{tenant:String\}/);
expect(compiled.params.param_tenant).toBe("t_alpha");
expect(compiled.sql).not.toContain("t_alpha");
}
const projection = compileProjectionQuery(
{ tenant_id: "t_alpha", dataset_id: "ds_1", entity_keys: ["a", "b"] },
"opt_analytics",
1000,
);
expect(projection.sql).toMatch(/WHERE tenant_id = \{tenant:String\}/);
expect(projection.sql).not.toContain("t_alpha");
});
it("the NEW compiler fails closed on a blank/absent tenant instead of emitting an unfiltered query", () => {
for (const tenant of ["", " "]) {
expect(() =>
compileMetricQuery({ tenant_id: tenant, metric: "m", aggregate: "sum" }, "opt_analytics", catalog, 1000),
).toThrowError(/tenant_id is required/);
}
// The undefined case, which TypeScript would normally forbid, is what a
// JS caller or a loosely-typed HTTP boundary can actually produce.
expect(() =>
compileMetricQuery(
{ tenant_id: undefined as unknown as string, metric: "m", aggregate: "sum" },
"opt_analytics",
catalog,
1000,
),
).toThrowError(/tenant_id is required/);
});
it("the constant itself is the predicate, so deleting the forcing cannot pass silently", () => {
// If someone redefines FORCED_TENANT_PREDICATE to something harmless
// (e.g. "1 = 1") to make the substring assertions above pass, this fails.
expect(FORCED_TENANT_PREDICATE).toBe("tenant_id = {tenant:String}");
});
});
@@ -0,0 +1,183 @@
/**
* BI-P005 conformance against an INDEPENDENT Arrow implementation.
*
* The unit suite (tests/analytics/arrow-ipc.test.ts) only proves 0pt's
* encoder agrees with 0pt's decoder — which it would even if both shared the
* same misreading of the spec. Here, ClickHouse's Arrow C++ implementation is
* the second opinion, in BOTH directions:
*
* ClickHouse -> Arrow -> 0pt decoder (their encoder, our reader)
* 0pt encoder -> Arrow -> ClickHouse (our encoder, their reader)
*
* The second direction is the load-bearing one: ClickHouse INSERTing our
* bytes and then answering a SELECT with the exact values back is a claim
* that cannot be satisfied by a subtly wrong buffer layout.
*
* The DoD's literal chain (ClickHouse -> Arrow -> DuckDB/Python) additionally
* ran against real pyarrow + DuckDB; that transcript is committed at
* evidence/bi-p002-p012/bi-p005-arrow-roundtrip.json.
*/
import { describe, it, expect, beforeAll, afterAll } from "vitest";
import {
arrowStreamSettings,
clickhouseArrowStreamFormat,
decodeArrowStream,
encodeArrowStream,
} from "@/src/analytics/arrow/ipc";
const CLICKHOUSE_URL = process.env.CLICKHOUSE_URL;
const CLICKHOUSE_AUTH = process.env.CLICKHOUSE_AUTH;
const run = CLICKHOUSE_URL ? describe : describe.skip;
const DB = `opt_arrow_${Date.now().toString(36)}`;
function authHeaders(): Record<string, string> {
return CLICKHOUSE_AUTH
? { Authorization: `Basic ${Buffer.from(CLICKHOUSE_AUTH).toString("base64")}` }
: {};
}
async function chBytes(sql: string, settings: Record<string, string> = {}): Promise<Uint8Array> {
const url = new URL(CLICKHOUSE_URL ?? "http://skipped.invalid");
url.searchParams.set("query", sql);
for (const [k, v] of Object.entries(settings)) url.searchParams.set(k, v);
const res = await fetch(url.toString(), { method: "POST", headers: authHeaders(), body: "" });
const buf = new Uint8Array(await res.arrayBuffer());
if (!res.ok) throw new Error(`CH ${res.status}: ${new TextDecoder().decode(buf).slice(0, 300)}`);
return buf;
}
async function chText(sql: string, body?: BodyInit): Promise<string> {
const url = new URL(CLICKHOUSE_URL ?? "http://skipped.invalid");
url.searchParams.set("query", sql);
const res = await fetch(url.toString(), { method: "POST", headers: authHeaders(), body: body ?? "" });
const text = await res.text();
if (!res.ok) throw new Error(`CH ${res.status}: ${text.slice(0, 400)}`);
return text;
}
run("BI-P005 Arrow IPC against a LIVE ClickHouse", () => {
beforeAll(async () => {
await chText(`CREATE DATABASE IF NOT EXISTS ${DB}`);
await chText(
`CREATE TABLE IF NOT EXISTS ${DB}.arrow_roundtrip (
tenant_id String, n Int64, v Float64, flag Bool, ts DateTime64(3, 'UTC')
) ENGINE = MergeTree() ORDER BY (tenant_id, n)`,
);
}, 60_000);
afterAll(async () => {
if (CLICKHOUSE_URL) await chText(`DROP DATABASE IF EXISTS ${DB}`);
}, 60_000);
it("THEIR encoder, OUR reader: ClickHouse ArrowStream decodes with the expected schema and row count", async () => {
const bytes = await chBytes(
`SELECT toString(t) AS tenant_id, toInt64(n) AS n, toFloat64(v) AS v, b AS flag,
toDateTime64(ts, 3, 'UTC') AS ts
FROM (
SELECT arrayJoin([
('t_alpha', 1, 1.5, true, '2026-01-01 00:00:00.000'),
('t_beta', 2, 2.5, false, '2026-06-15 08:09:10.123'),
('t_gamma', 3, 3.25, true, '2026-09-06 12:34:56.789')
]) AS tup, tup.1 AS t, tup.2 AS n, tup.3 AS v, tup.4 AS b, tup.5 AS ts
)
FORMAT ${clickhouseArrowStreamFormat()}`,
arrowStreamSettings(),
);
const info = decodeArrowStream(bytes);
expect(info.rowCount).toBe(3);
expect(info.fields.map((f) => f.name)).toEqual(["tenant_id", "n", "v", "flag", "ts"]);
// Utf8=5 (NOT Binary=4 — that is what output_format_arrow_string_as_string buys),
// Int=2, FloatingPoint=3, Bool=6, Timestamp=10.
expect(info.fields.map((f) => f.typeType)).toEqual([5, 2, 3, 6, 10]);
});
it("WITHOUT output_format_arrow_string_as_string, ClickHouse emits Binary — the semantic loss the setting prevents", async () => {
const bytes = await chBytes(`SELECT 'x' AS s FORMAT ${clickhouseArrowStreamFormat()}`, {
output_format_arrow_string_as_string: "0",
});
// Arrow Type union: Binary = 4, Utf8 = 5.
expect(decodeArrowStream(bytes).fields[0].typeType).toBe(4);
const asText = await chBytes(`SELECT 'x' AS s FORMAT ${clickhouseArrowStreamFormat()}`, arrowStreamSettings());
expect(decodeArrowStream(asText).fields[0].typeType).toBe(5);
});
it("OUR encoder, THEIR reader: ClickHouse INSERTs our Arrow and returns every value unchanged", async () => {
const bytes = encodeArrowStream([
{ name: "tenant_id", type: "utf8", values: ["t_alpha", "t_beta", "t_gamma"] },
{ name: "n", type: "int64", values: [1, 2, 3] },
{ name: "v", type: "float64", values: [1.5, 2.5, 3.25] },
{ name: "flag", type: "bool", values: [true, false, true] },
{
name: "ts",
type: "timestamp_ms",
values: [
new Date("2026-01-01T00:00:00.000Z"),
new Date("2026-06-15T08:09:10.123Z"),
new Date("2026-09-06T12:34:56.789Z"),
],
},
]);
await chText(`INSERT INTO ${DB}.arrow_roundtrip FORMAT ArrowStream`, bytes as unknown as BodyInit);
const back = await chText(
`SELECT tenant_id, n, v, flag, toString(ts) AS ts FROM ${DB}.arrow_roundtrip ORDER BY n FORMAT JSON`,
);
const rows = (JSON.parse(back) as {
data: { tenant_id: string; n: string | number; v: number | string; flag: boolean | number; ts: string }[];
}).data;
expect(rows).toHaveLength(3);
expect(rows.map((r) => r.tenant_id)).toEqual(["t_alpha", "t_beta", "t_gamma"]);
expect(rows.map((r) => Number(r.n))).toEqual([1, 2, 3]);
expect(rows.map((r) => Number(r.v))).toEqual([1.5, 2.5, 3.25]);
expect(rows.map((r) => Boolean(Number(r.flag)))).toEqual([true, false, true]);
expect(rows.map((r) => r.ts)).toEqual([
"2026-01-01 00:00:00.000",
"2026-06-15 08:09:10.123",
"2026-09-06 12:34:56.789",
]);
});
it("full circle: ClickHouse -> Arrow -> our decoder -> ClickHouse yields identical rows", async () => {
const exported = await chBytes(
`SELECT tenant_id, n, v, flag, ts FROM ${DB}.arrow_roundtrip ORDER BY n FORMAT ArrowStream`,
arrowStreamSettings(),
);
expect(decodeArrowStream(exported).rowCount).toBe(3);
await chText(`CREATE TABLE IF NOT EXISTS ${DB}.arrow_circle AS ${DB}.arrow_roundtrip`);
await chText(`INSERT INTO ${DB}.arrow_circle FORMAT ArrowStream`, exported as unknown as BodyInit);
const diff = await chText(
`SELECT count() AS c FROM (
SELECT * FROM ${DB}.arrow_roundtrip
EXCEPT
SELECT * FROM ${DB}.arrow_circle
) FORMAT JSON`,
);
expect(Number((JSON.parse(diff) as { data: { c: number | string }[] }).data[0].c)).toBe(0);
});
it("a large result survives the round trip (batching/offsets are not only correct for three rows)", async () => {
const n = 20_000;
const bytes = encodeArrowStream([
{ name: "tenant_id", type: "utf8", values: Array.from({ length: n }, (_, i) => `t_${i % 7}`) },
{ name: "n", type: "int64", values: Array.from({ length: n }, (_, i) => i) },
{ name: "v", type: "float64", values: Array.from({ length: n }, (_, i) => i / 3) },
{ name: "flag", type: "bool", values: Array.from({ length: n }, (_, i) => i % 2 === 0) },
{ name: "ts", type: "timestamp_ms", values: Array.from({ length: n }, () => new Date("2026-09-06T00:00:00Z")) },
]);
expect(decodeArrowStream(bytes).rowCount).toBe(n);
await chText(`CREATE TABLE IF NOT EXISTS ${DB}.arrow_big AS ${DB}.arrow_roundtrip`);
await chText(`INSERT INTO ${DB}.arrow_big FORMAT ArrowStream`, bytes as unknown as BodyInit);
const agg = await chText(
`SELECT count() AS c, sum(n) AS s, uniqExact(tenant_id) AS t, countIf(flag) AS f FROM ${DB}.arrow_big FORMAT JSON`,
);
const d = (JSON.parse(agg) as { data: { c: string | number; s: string | number; t: string | number; f: string | number }[] }).data[0];
expect(Number(d.c)).toBe(n);
expect(Number(d.s)).toBe((n * (n - 1)) / 2);
expect(Number(d.t)).toBe(7);
expect(Number(d.f)).toBe(n / 2);
}, 60_000);
});
@@ -0,0 +1,252 @@
/**
* BI-P009 (canonical OLAP provider) + BI-P010 (production schema), against a
* LIVE ClickHouse.
*
* Every one of the eight `SiaxAnalyticsEngine` capabilities is exercised for
* real here. A capability that only has a unit test with a stubbed fetch is a
* capability whose SQL has never been parsed by ClickHouse — `EXPLAIN
* ESTIMATE`, `KILL QUERY` and `OPTIMIZE ... PARTITION` in particular are
* syntax the server either accepts or does not, and no mock can tell you
* which.
*
* Gated on CLICKHOUSE_URL; skips honestly when unset. CI's `integration` job
* supplies it and hard-fails on a skipped suite.
*/
import { describe, it, expect, beforeAll, afterAll } from "vitest";
import { ClickHouseAnalyticsEngine } from "@/src/analytics/engine/clickhouse-engine";
import { ANALYTICS_MATERIALIZED_VIEWS, ANALYTICS_TABLES } from "@/src/analytics/engine/schema";
import type { AnalyticsIdentity } from "@/src/analytics/engine/siax-analytics-engine";
const CLICKHOUSE_URL = process.env.CLICKHOUSE_URL;
const CLICKHOUSE_AUTH = process.env.CLICKHOUSE_AUTH;
const run = CLICKHOUSE_URL ? describe : describe.skip;
const DB = `opt_engine_${Date.now().toString(36)}`;
const ID: AnalyticsIdentity = { tenant_id: "t_alpha", project_id: "p_main", principal_id: "usr_alpha" };
let engine: ClickHouseAnalyticsEngine;
async function raw(sql: string, body?: string): Promise<string> {
const url = new URL(CLICKHOUSE_URL ?? "http://skipped.invalid");
url.searchParams.set("query", sql);
const headers: Record<string, string> = {};
if (CLICKHOUSE_AUTH) headers["Authorization"] = `Basic ${Buffer.from(CLICKHOUSE_AUTH).toString("base64")}`;
const res = await fetch(url.toString(), { method: "POST", headers, body: body ?? "" });
const text = await res.text();
if (!res.ok) throw new Error(`CH ${res.status}: ${text.slice(0, 300)}`);
return text;
}
run("BI-P009/BI-P010 ClickHouse provider against a LIVE server", () => {
beforeAll(async () => {
engine = new ClickHouseAnalyticsEngine({
baseUrl: CLICKHOUSE_URL!,
basicAuth: CLICKHOUSE_AUTH,
database: DB,
dimensionCatalogs: { t_alpha: { severity: "severity", make: "vehicle_make" } },
});
await engine.ensureSchema();
await engine.ensureSchema(); // idempotent, as the DDL claims
}, 120_000);
afterAll(async () => {
if (CLICKHOUSE_URL) await raw(`DROP DATABASE IF EXISTS ${DB}`);
}, 60_000);
// ── capability 1: health ────────────────────────────────────────────────
it("health() reports the live server's version", async () => {
const h = await engine.health();
expect(h.healthy).toBe(true);
expect(h.provider).toBe("clickhouse");
expect(h.version).toMatch(/^\d+\.\d+/);
});
// ── BI-P010: the schema is what it claims to be ─────────────────────────
it("BI-P010: four purpose-built tables exist, NOT one giant table", async () => {
const listed = await raw(
`SELECT name FROM system.tables WHERE database = '${DB}' AND engine != 'MaterializedView' ORDER BY name FORMAT TSV`,
);
const names = listed.trim().split("\n").filter(Boolean);
// The three this module owns, PLUS the preserved opt_metric_events.
for (const t of ANALYTICS_TABLES) expect(names).toContain(t);
expect(names).toContain("opt_metric_events");
expect(names.length).toBeGreaterThanOrEqual(4);
});
it("BI-P010: the tables use different engines and different sort keys — the split is real", async () => {
const meta = await raw(
`SELECT name, engine, sorting_key FROM system.tables WHERE database = '${DB}' ORDER BY name FORMAT JSON`,
);
const rows = (JSON.parse(meta) as { data: { name: string; engine: string; sorting_key: string }[] }).data;
const by = Object.fromEntries(rows.map((r) => [r.name, r]));
expect(by["opt_analytics_event"].engine).toBe("MergeTree");
expect(by["opt_metric_daily"].engine).toBe("AggregatingMergeTree");
expect(by["opt_dataset_projection"].engine).toBe("ReplacingMergeTree");
// Different ORDER BY keys — the reason one table cannot serve both jobs.
expect(by["opt_analytics_event"].sorting_key).not.toBe(by["opt_dataset_projection"].sorting_key);
// Every one still leads with tenant_id.
for (const t of ANALYTICS_TABLES) expect(by[t].sorting_key.startsWith("tenant_id")).toBe(true);
});
it("BI-P010: TTL and LowCardinality are actually applied, not just written in a comment", async () => {
const ddl = await raw(`SHOW CREATE TABLE ${DB}.opt_analytics_event FORMAT TSVRaw`);
expect(ddl).toContain("TTL");
expect(ddl).toContain("LowCardinality(String)");
expect(ddl).toContain("PARTITION BY toYYYYMM(event_time)");
const rollup = await raw(`SHOW CREATE TABLE ${DB}.opt_metric_daily FORMAT TSVRaw`);
expect(rollup).toContain("AggregateFunction(sum, Float64)");
});
it("BI-P010: the materialized view actually populates the rollup", async () => {
const views = await raw(
`SELECT name FROM system.tables WHERE database='${DB}' AND engine='MaterializedView' FORMAT TSV`,
);
for (const v of ANALYTICS_MATERIALIZED_VIEWS) expect(views).toContain(v);
await engine.ingestProjection(ID, { dataset_id: "warmup", rows: [{ entity_key: "w1" }] });
const rows = Array.from({ length: 20 }, (_, i) =>
JSON.stringify({
tenant_id: "t_alpha",
project_id: "p_main",
dataset_id: "ds_mv",
metric: "mv_metric",
event_id: `mv${i}`,
event_time: "2026-09-03 12:00:00.000",
value: 2,
dim_names: [],
dim_values: [],
}),
).join("\n");
await raw(`INSERT INTO ${DB}.opt_analytics_event FORMAT JSONEachRow`, rows);
const rolled = await raw(
`SELECT sumMerge(sum_state) AS s, countMerge(count_state) AS c FROM ${DB}.opt_metric_daily ` +
`WHERE tenant_id = 't_alpha' AND metric = 'mv_metric' FORMAT JSON`,
);
const d = (JSON.parse(rolled) as { data: { s: number | string; c: number | string }[] }).data[0];
expect(Number(d.s)).toBe(40); // 20 rows x 2
expect(Number(d.c)).toBe(20);
});
// ── capability 2: executeMetric (already covered by the isolation suite) ─
it("executeMetric returns grouped rows keyed by the CALLER-FACING dimension name", async () => {
const rows = ["warning", "critical", "critical"].map((sev, i) =>
JSON.stringify({
tenant_id: "t_alpha",
project_id: "p_main",
dataset_id: "ds_g",
metric: "grouped",
event_id: `g${i}`,
event_time: "2026-09-04 09:00:00.000",
value: 5,
dim_names: ["severity"],
dim_values: [sev],
}),
);
await raw(`INSERT INTO ${DB}.opt_analytics_event FORMAT JSONEachRow`, rows.join("\n"));
const result = await engine.executeMetric(ID, {
metric: "grouped",
aggregate: "sum",
group_by: ["severity"],
});
const critical = result.rows.find((r) => r.groups?.severity === "critical");
expect(critical?.value).toBe(10);
expect(result.stats.result_rows).toBe(2);
expect(result.query_id.startsWith("opt_")).toBe(true);
});
// ── capability 4: estimateQuery ─────────────────────────────────────────
it("estimateQuery gets a real EXPLAIN ESTIMATE from the server and compares it to the budget", async () => {
const est = await engine.estimateQuery(ID, { metric: "grouped", aggregate: "sum" });
expect(est.estimated_rows_to_read).toBeGreaterThan(0);
expect(est.within_budget).toBe(true);
expect(est.budget.maxRowsToRead).toBeGreaterThan(0);
});
it("estimateQuery REFUSES a query whose estimate exceeds a tightened quota", async () => {
const tiny: AnalyticsIdentity = { ...ID, quota: { maxRowsToRead: 1 } };
const est = await engine.estimateQuery(tiny, { metric: "grouped", aggregate: "sum" });
expect(est.within_budget).toBe(false);
expect(est.reason).toMatch(/over budget/);
});
// ── capability 6: explain ───────────────────────────────────────────────
it("explain() proves the forced predicate reaches the PRIMARY KEY, pruning parts", async () => {
const ex = await engine.explain(ID, { metric: "grouped", aggregate: "sum", group_by: ["severity"] });
expect(ex.forced_tenant_predicate).toBe("tenant_id = {tenant:String}");
expect(ex.table).toBe("opt_analytics_event");
// This is the payoff of BI-P010's tenant-first ORDER BY: the server
// reports tenant_id as a PrimaryKey key with a condition on it, i.e. the
// isolation predicate is an index range scan, not a post-read filter.
expect(ex.plan).toContain("ReadFromMergeTree");
expect(ex.plan).toContain("PrimaryKey");
expect(ex.plan).toMatch(/Keys:[\s\S]*tenant_id/);
expect(ex.plan).toMatch(/tenant_id in \['t_alpha', 't_alpha'\]/);
// The plan describes only the CALLER's own tenant; no other tenant id
// may appear in a plan handed back to them.
expect(ex.plan).not.toContain("t_beta");
});
// ── capability 5: cancelQuery ───────────────────────────────────────────
it("cancelQuery issues a real KILL QUERY the server accepts", async () => {
// Nothing is running under this id, so the honest outcome is "the KILL
// statement was valid and matched nothing" -> true, with no exception.
const ok = await engine.cancelQuery(ID, `${(await engine.executeMetric(ID, { metric: "grouped", aggregate: "count" })).query_id}`);
expect(ok).toBe(true);
});
// ── capabilities 7 + 8: ingestProjection / refreshProjection ────────────
it("ingestProjection writes the SERVING tier with the identity's tenant, ignoring any row-level claim", async () => {
const res = await engine.ingestProjection(ID, {
dataset_id: "ds_vehicles",
rows: [
{ entity_key: "VIN001", attributes: { make: "volvo" }, measures: { price: 189000 } },
{ entity_key: "VIN002", attributes: { make: "saab" }, measures: { price: 42000 } },
],
});
expect(res.ingested).toBe(2);
const stored = await raw(
`SELECT uniqExact(tenant_id) AS t FROM ${DB}.opt_dataset_projection WHERE dataset_id = 'ds_vehicles' FORMAT JSON`,
);
expect(Number((JSON.parse(stored) as { data: { t: number | string }[] }).data[0].t)).toBe(1);
});
it("refreshProjection makes a re-ingest idempotent (ReplacingMergeTree + OPTIMIZE FINAL)", async () => {
// Re-ingest the SAME entity keys at a higher version.
await engine.ingestProjection(ID, {
dataset_id: "ds_vehicles",
rows: [
{ entity_key: "VIN001", version: 9_999_999_999_999, measures: { price: 175000 } },
{ entity_key: "VIN002", version: 9_999_999_999_999, measures: { price: 41000 } },
],
});
const refreshed = await engine.refreshProjection(ID, "ds_vehicles");
expect(refreshed.row_count).toBe(2); // 2 entities, not 4 rows
expect(refreshed.dataset_id).toBe("ds_vehicles");
const winner = await raw(
`SELECT measures['price'] AS p FROM ${DB}.opt_dataset_projection FINAL ` +
`WHERE tenant_id='t_alpha' AND dataset_id='ds_vehicles' AND entity_key='VIN001' FORMAT JSON`,
);
expect(Number((JSON.parse(winner) as { data: { p: number | string }[] }).data[0].p)).toBe(175000);
});
// ── the PRESERVED path still works through the engine ───────────────────
it("the PRESERVED ClickHouseAnalytics path still ingests and answers through the engine", async () => {
await engine.legacyAnalytics.ingest([
{ tenant_id: "t_alpha", dataset_id: "ds_legacy", metric: "legacy_metric", value: 7 },
{ tenant_id: "t_beta", dataset_id: "ds_legacy", metric: "legacy_metric", value: 700 },
]);
const res = await engine.executeLegacyMetric(ID, { metric: "legacy_metric", aggregate: "sum" });
expect(res.rows[0].value).toBe(7); // NOT 707
});
});
@@ -0,0 +1,237 @@
/**
* BI-P012 DoD, against a LIVE ClickHouse:
*
* "Intentional query bomb is terminated without degrading unrelated tenant
* beyond declared SLO."
*
* Two halves, and the second is the one that is usually skipped:
*
* 1. TERMINATED — a deliberately enormous query is refused by the server,
* with the ClickHouse limit that stopped it identified by code. Each
* limit BI-P012 names is fired individually, so "some limit stopped it"
* is never mistaken for "the limit I claimed stopped it".
* 2. UNRELATED TENANT NOT DEGRADED — a second tenant's interactive query is
* timed WHILE the bomb is in flight and compared against a declared SLO,
* not merely observed to "still work".
*/
import { describe, it, expect, beforeAll, afterAll } from "vitest";
import { ClickHouseAnalyticsEngine } from "@/src/analytics/engine/clickhouse-engine";
import { DEFAULT_BUDGETS } from "@/src/analytics/engine/resource-governance";
import type { AnalyticsIdentity } from "@/src/analytics/engine/siax-analytics-engine";
const CLICKHOUSE_URL = process.env.CLICKHOUSE_URL;
const CLICKHOUSE_AUTH = process.env.CLICKHOUSE_AUTH;
const run = CLICKHOUSE_URL ? describe : describe.skip;
const DB = `opt_gov_${Date.now().toString(36)}`;
/**
* The SLO this test asserts against. Declared as a constant so the claim is
* falsifiable rather than "it still seemed to work".
*
* 5000 ms is deliberately generous for a 500-row lookup (it measures ~30 ms
* on an idle server). It is set for the SHARED CI runner, not for a quiet
* laptop: a tenant that had actually been starved by the bomb would take
* seconds or fail outright, so the assertion still discriminates, while a
* merely busy runner does not turn this gate into a flake.
*/
const UNRELATED_TENANT_SLO_MS = 5000;
const VICTIM: AnalyticsIdentity = { tenant_id: "t_victim", project_id: "p_main", principal_id: "usr_v" };
let engine: ClickHouseAnalyticsEngine;
/** Run a statement with explicit settings and report the ClickHouse code. */
async function chRun(
sql: string,
settings: Record<string, string> = {},
body?: string,
): Promise<{ ok: boolean; code?: string; text: string; ms: number }> {
const url = new URL(CLICKHOUSE_URL ?? "http://skipped.invalid");
url.searchParams.set("query", sql);
for (const [k, v] of Object.entries(settings)) url.searchParams.set(k, v);
const headers: Record<string, string> = {};
if (CLICKHOUSE_AUTH) headers["Authorization"] = `Basic ${Buffer.from(CLICKHOUSE_AUTH).toString("base64")}`;
const t0 = Date.now();
const res = await fetch(url.toString(), { method: "POST", headers, body: body ?? "" });
const text = await res.text();
return { ok: res.ok, code: /Code: (\d+)/.exec(text)?.[1], text, ms: Date.now() - t0 };
}
run("BI-P012 resource governance against a LIVE ClickHouse", () => {
beforeAll(async () => {
engine = new ClickHouseAnalyticsEngine({
baseUrl: CLICKHOUSE_URL!,
basicAuth: CLICKHOUSE_AUTH,
database: DB,
dimensionCatalogs: { t_victim: { severity: "severity" } },
});
await engine.ensureSchema();
// A small, real dataset for the unrelated tenant.
const rows = Array.from({ length: 500 }, (_, i) =>
JSON.stringify({
tenant_id: "t_victim",
project_id: "p_main",
dataset_id: "ds_v",
metric: "victim_metric",
event_id: `v${i}`,
event_time: "2026-09-05 08:00:00.000",
value: 1,
dim_names: ["severity"],
dim_values: ["warning"],
}),
).join("\n");
const ins = await chRun(`INSERT INTO ${DB}.opt_analytics_event FORMAT JSONEachRow`, {}, rows);
expect(ins.ok, ins.text.slice(0, 200)).toBe(true);
}, 120_000);
afterAll(async () => {
if (CLICKHOUSE_URL) await chRun(`DROP DATABASE IF EXISTS ${DB}`);
}, 60_000);
// ── each named limit, fired individually ────────────────────────────────
it("max_rows_to_read terminates a read bomb (CH code 158 TOO_MANY_ROWS)", async () => {
const r = await chRun("SELECT count() FROM numbers(1000000000) FORMAT JSON", {
readonly: "1",
max_rows_to_read: "1000",
read_overflow_mode: "throw",
});
expect(r.ok).toBe(false);
expect(r.code).toBe("158");
});
it("max_bytes_to_read terminates a scan bomb (CH code 307 TOO_MANY_BYTES)", async () => {
const r = await chRun("SELECT sum(number) FROM numbers(1000000000) FORMAT JSON", {
readonly: "1",
max_bytes_to_read: "1024",
read_overflow_mode: "throw",
});
expect(r.ok).toBe(false);
expect(r.code).toBe("307");
});
it("max_execution_time terminates a slow bomb (CH code 159 TIMEOUT_EXCEEDED)", async () => {
const r = await chRun("SELECT count() FROM numbers(50000000000) FORMAT JSON", {
readonly: "1",
max_execution_time: "1",
timeout_overflow_mode: "throw",
});
expect(r.ok).toBe(false);
expect(r.code).toBe("159");
expect(r.ms).toBeLessThan(20_000);
});
it("max_result_rows terminates a result bomb and does NOT return a partial result", async () => {
const r = await chRun("SELECT number FROM numbers(100000) FORMAT JSON", {
readonly: "1",
max_result_rows: "10",
result_overflow_mode: "throw",
});
expect(r.ok).toBe(false);
expect(r.code).toBe("396"); // TOO_MANY_ROWS_OR_BYTES
// The dangerous alternative: result_overflow_mode=break returns 10 rows
// with HTTP 200 and no indication of truncation.
const broken = await chRun("SELECT number FROM numbers(100000) FORMAT JSON", {
readonly: "1",
max_result_rows: "10",
result_overflow_mode: "break",
});
// MEASURED on ClickHouse 24.8: break mode answers HTTP 200 with 65409 of
// 100000 rows (it stops at a block boundary, not at the stated 10) and
// nothing in the payload says it is short. That is why every SELECT in
// this engine ships result_overflow_mode=throw.
expect(broken.ok).toBe(true);
const brokenRows = (JSON.parse(broken.text) as { data: unknown[] }).data.length;
expect(brokenRows).toBeGreaterThan(10);
expect(brokenRows).toBeLessThan(100_000);
});
it("max_memory_usage terminates a memory bomb (CH code 241 MEMORY_LIMIT_EXCEEDED)", async () => {
const r = await chRun(
"SELECT uniqExact(toString(number)) FROM numbers(50000000) FORMAT JSON",
{ readonly: "1", max_memory_usage: "1000000", max_execution_time: "30" },
);
expect(r.ok).toBe(false);
expect(r.code).toBe("241");
});
it("readonly=1 means the query text cannot re-raise its own limits", async () => {
const r = await chRun(
"SELECT count() FROM numbers(1000000000) SETTINGS max_rows_to_read = 10000000000 FORMAT JSON",
{ readonly: "1", max_rows_to_read: "1000", read_overflow_mode: "throw" },
);
expect(r.ok).toBe(false);
// 164 = READONLY (changing settings is forbidden); 158 = the limit still
// applied. Either proves the caller did not win — but NOT a 200.
expect(["164", "158"]).toContain(r.code);
});
it("readonly=1 blocks writes through the query surface", async () => {
const r = await chRun(`DROP TABLE ${DB}.opt_analytics_event FORMAT JSON`, { readonly: "1" });
expect(r.ok).toBe(false);
});
// ── the half that is usually skipped ────────────────────────────────────
it("an unrelated tenant stays within its declared SLO WHILE a query bomb is in flight", async () => {
// Baseline first, so "fast enough" is measured against this server, not
// against a number someone hoped for.
const baseline = await engine.executeMetric(VICTIM, { metric: "victim_metric", aggregate: "count" });
expect(baseline.rows[0].value).toBe(500);
const baselineMs = baseline.stats.duration_ms;
// Fire a genuine bomb the server must spend real CPU on. MEASURED: a bare
// `count()` over numbers() is NOT a bomb — ClickHouse answers
// numbers(20e9) in ~5s because it can skip the values entirely. Hashing
// every row cannot be short-circuited, so this one really does burn CPU
// until the limit kills it.
const bomb = chRun("SELECT sum(cityHash64(number)) FROM numbers(20000000000) FORMAT JSON", {
readonly: "1",
max_execution_time: "8",
timeout_overflow_mode: "throw",
});
// While it burns, run the unrelated tenant's interactive query repeatedly.
const samples: number[] = [];
for (let i = 0; i < 5; i++) {
const r = await engine.executeMetric(VICTIM, { metric: "victim_metric", aggregate: "count" });
expect(r.rows[0].value).toBe(500); // still CORRECT, not just fast
samples.push(r.stats.duration_ms);
}
const bombResult = await bomb;
expect(bombResult.ok, "the bomb was NOT terminated").toBe(false);
expect(bombResult.code).toBe("159");
const worst = Math.max(...samples);
expect(
worst,
`unrelated tenant degraded: baseline ${baselineMs}ms, worst under load ${worst}ms, SLO ${UNRELATED_TENANT_SLO_MS}ms`,
).toBeLessThan(UNRELATED_TENANT_SLO_MS);
}, 60_000);
// ── the local admission gate, on the real engine ────────────────────────
it("the per-tenant concurrency gate refuses the N+1st in-flight query for THAT tenant only", async () => {
const limit = DEFAULT_BUDGETS.INTERACTIVE.maxConcurrentQueries;
// Fire limit+4 concurrently; some must be refused with RATE_LIMITED
// rather than all of them reaching the server.
const attempts = Array.from({ length: limit + 4 }, () =>
engine
.executeMetric(VICTIM, { metric: "victim_metric", aggregate: "count" })
.then(() => "ok" as const)
.catch((e: { code?: string }) => e.code ?? "error"),
);
const outcomes = await Promise.all(attempts);
expect(outcomes.filter((o) => o === "RATE_LIMITED").length).toBeGreaterThan(0);
expect(outcomes.filter((o) => o === "ok").length).toBeGreaterThan(0);
// A different tenant is unaffected by the noisy one's exhausted slots.
const other = await engine.executeMetric(
{ tenant_id: "t_quiet", project_id: "p_main", principal_id: "usr_q" },
{ metric: "victim_metric", aggregate: "count" },
);
expect(other.rows[0].value).toBe(0); // its own (empty) data, and no error
}, 60_000);
});
@@ -0,0 +1,308 @@
/**
* BI-P011 — CLICKHOUSE TENANT ISOLATION, proven against a LIVE server.
*
* The shape of this proof is the estate's own convention for isolation work:
* DEMONSTRATE THE HOLE WITHOUT THE FIX, then demonstrate it CLOSED WITH the
* fix. A suite that only exercises the fixed path proves that the fixed path
* works, which is not the same claim.
*
* So each layer is tested twice:
*
* layer 2 (0pt query compiler forces tenant)
* hole: a hand-written statement WITHOUT the predicate — the same query
* minus the forcing — returns another tenant's rows. This is the
* literal regression a future refactor would cause.
* closed: the compiled statement returns only this tenant's rows.
*
* layer 3 (restricted database users + row policies)
* hole: the UNRESTRICTED admin user, issuing a statement with NO tenant
* predicate at all, sees every tenant.
* closed: the RESTRICTED per-tenant user, issuing the SAME statement,
* sees only its own tenant — the server filters, so the result
* holds even if layer 2 were removed entirely.
*
* Gated on CLICKHOUSE_URL and SKIPS HONESTLY when unset, matching
* tests/integration/analytics-clickhouse.integration.test.ts. CI's
* `integration` job sets it from a service container and hard-fails if the
* suites skip, so this is not a self-skipping green in practice.
*/
import { describe, it, expect, beforeAll, afterAll } from "vitest";
import { ClickHouseAnalyticsEngine } from "@/src/analytics/engine/clickhouse-engine";
import {
compileMetricQuery,
dimensionCatalogFrom,
restrictedTenantUserDdl,
} from "@/src/analytics/engine/query-compiler";
import { ANALYTICS_TABLES, sortingKeySql } from "@/src/analytics/engine/schema";
import { DEFAULT_BUDGETS } from "@/src/analytics/engine/resource-governance";
import type { AnalyticsIdentity } from "@/src/analytics/engine/siax-analytics-engine";
const CLICKHOUSE_URL = process.env.CLICKHOUSE_URL;
const CLICKHOUSE_AUTH = process.env.CLICKHOUSE_AUTH;
const run = CLICKHOUSE_URL ? describe : describe.skip;
const DB = `opt_isolation_${Date.now().toString(36)}`;
const RESTRICTED_USER = `opt_iso_${Date.now().toString(36)}`;
const RESTRICTED_PW = `iso_pw_${Math.random().toString(36).slice(2, 12)}`;
const ALPHA: AnalyticsIdentity = { tenant_id: "t_alpha", project_id: "p_main", principal_id: "usr_alpha" };
const BETA: AnalyticsIdentity = { tenant_id: "t_beta", project_id: "p_main", principal_id: "usr_beta" };
const catalog = dimensionCatalogFrom({ severity: "severity" });
const ALL_TABLES: readonly string[] = ANALYTICS_TABLES;
/** Raw HTTP against the live server, as a named user. */
async function chQuery(
sql: string,
auth: string | undefined,
params: Record<string, string> = {},
settings: Record<string, string> = {},
body?: string,
): Promise<{ ok: boolean; status: number; text: string }> {
const url = new URL(CLICKHOUSE_URL ?? "http://skipped.invalid");
url.searchParams.set("query", sql);
for (const [k, v] of Object.entries(settings)) url.searchParams.set(k, v);
for (const [k, v] of Object.entries(params)) url.searchParams.set(k, v);
const headers: Record<string, string> = {};
if (auth) headers["Authorization"] = `Basic ${Buffer.from(auth).toString("base64")}`;
const res = await fetch(url.toString(), { method: "POST", headers, body: body ?? "" });
return { ok: res.ok, status: res.status, text: await res.text() };
}
function rowsOf(text: string): { grp?: string; value: number }[] {
if (!text.trim()) return [];
const parsed = JSON.parse(text) as { data?: { grp?: string; value: number | string }[] };
return (parsed.data ?? []).map((r) => ({ ...(r.grp !== undefined ? { grp: r.grp } : {}), value: Number(r.value) }));
}
let engine: ClickHouseAnalyticsEngine;
run("BI-P011 tenant isolation against a LIVE ClickHouse", () => {
beforeAll(async () => {
engine = new ClickHouseAnalyticsEngine({
baseUrl: CLICKHOUSE_URL!,
basicAuth: CLICKHOUSE_AUTH,
database: DB,
dimensionCatalogs: { t_alpha: { severity: "severity" }, t_beta: { severity: "severity" } },
});
const health = await engine.health();
expect(health.healthy, `ClickHouse not healthy: ${health.detail ?? ""}`).toBe(true);
await engine.ensureSchema();
// Two tenants, deliberately different magnitudes so a leak is unmistakable
// in the NUMBER, not just in the row count.
const rows: string[] = [];
for (let i = 0; i < 50; i++) {
rows.push(
JSON.stringify({
tenant_id: "t_alpha",
project_id: "p_main",
dataset_id: "ds_shared",
metric: "rows_ingested",
event_id: `a${i}`,
event_time: "2026-09-01 10:00:00.000",
value: 1,
dim_names: ["severity"],
dim_values: [i % 2 === 0 ? "warning" : "critical"],
}),
);
rows.push(
JSON.stringify({
tenant_id: "t_beta",
project_id: "p_main",
dataset_id: "ds_shared",
metric: "rows_ingested",
event_id: `b${i}`,
event_time: "2026-09-01 10:00:00.000",
value: 1000,
dim_names: ["severity"],
dim_values: ["critical"],
}),
);
}
const ins = await chQuery(
`INSERT INTO ${DB}.opt_analytics_event FORMAT JSONEachRow`,
CLICKHOUSE_AUTH,
{},
{},
rows.join("\n"),
);
expect(ins.ok, `seed insert failed: ${ins.text.slice(0, 200)}`).toBe(true);
}, 120_000);
afterAll(async () => {
if (!CLICKHOUSE_URL) return;
// Disposable by construction: drop the user, its policies, and the DB.
for (const table of ANALYTICS_TABLES) {
await chQuery(`DROP ROW POLICY IF EXISTS ${RESTRICTED_USER}_${table}_rp ON ${DB}.${table}`, CLICKHOUSE_AUTH);
}
await chQuery(`DROP USER IF EXISTS ${RESTRICTED_USER}`, CLICKHOUSE_AUTH);
await chQuery(`DROP DATABASE IF EXISTS ${DB}`, CLICKHOUSE_AUTH);
}, 60_000);
// ── layer 2: the compiler ────────────────────────────────────────────────
it("HOLE, layer 2 REMOVED: the same query WITHOUT the forced predicate returns t_beta's rows to t_alpha", async () => {
// This is the exact statement the compiler would produce if someone
// deleted the forcing — proof that the predicate is doing real work
// against real data, not decorating a query that was safe anyway.
const unsafeSql =
`SELECT sum(value) AS value FROM ${DB}.opt_analytics_event ` +
`WHERE metric = {metric:String} LIMIT 1 FORMAT JSON`;
const res = await chQuery(unsafeSql, CLICKHOUSE_AUTH, { param_metric: "rows_ingested" });
expect(res.ok).toBe(true);
// 50*1 (alpha) + 50*1000 (beta) = 50050 — both tenants, one result.
expect(rowsOf(res.text)[0].value).toBe(50050);
});
it("CLOSED, layer 2 PRESENT: the compiled query returns t_alpha's 50 and never t_beta's 50000", async () => {
const result = await engine.executeMetric(ALPHA, { metric: "rows_ingested", aggregate: "sum" });
expect(result.rows[0].value).toBe(50);
const beta = await engine.executeMetric(BETA, { metric: "rows_ingested", aggregate: "sum" });
expect(beta.rows[0].value).toBe(50_000);
});
it("tenant A querying tenant B's dataset id gets ZERO rows, not B's rows", async () => {
const result = await engine.executeMetric(ALPHA, {
metric: "rows_ingested",
aggregate: "count",
dataset_id: "ds_shared",
});
expect(result.rows[0].value).toBe(50); // only alpha's 50, though beta shares the dataset id
});
it("a forged dataset id yields an empty result, never an error that leaks existence", async () => {
const result = await engine.executeMetric(ALPHA, {
metric: "rows_ingested",
aggregate: "count",
dataset_id: "ds_that_does_not_exist_or_belongs_to_someone_else",
});
expect(result.rows[0].value).toBe(0);
});
it("a tenant with no data at all sees zero, not everyone's aggregate", async () => {
const ghost: AnalyticsIdentity = { tenant_id: "t_gamma", project_id: "p_main", principal_id: "usr_g" };
const result = await engine.executeMetric(ghost, { metric: "rows_ingested", aggregate: "count" });
expect(result.rows[0].value).toBe(0);
});
it("group-by results carry only this tenant's dimension values", async () => {
const result = await engine.executeMetric(ALPHA, {
metric: "rows_ingested",
aggregate: "count",
group_by: ["severity"],
});
const total = result.rows.reduce((a, r) => a + r.value, 0);
expect(total).toBe(50);
// t_beta's rows are all "critical" with value 1000; if any leaked, the
// critical bucket would exceed alpha's 25.
const critical = result.rows.find((r) => r.groups?.severity === "critical");
expect(critical?.value).toBe(25);
});
it("SQL/dimension/filter injection through the live path changes nothing", async () => {
await expect(
engine.executeMetric(ALPHA, {
metric: "rows_ingested",
aggregate: "sum",
group_by: ["severity'; SELECT tenant_id FROM system.tables; --"],
}),
).rejects.toMatchObject({ code: "INVALID_REQUEST" });
// A hostile VALUE reaches the server as a bound parameter and simply
// matches nothing.
const injected = await engine.executeMetric(ALPHA, {
metric: "rows_ingested",
aggregate: "count",
filters: { severity: "warning' OR 1=1 --" },
});
expect(injected.rows[0].value).toBe(0);
// And the table is still there afterwards.
const still = await engine.executeMetric(ALPHA, { metric: "rows_ingested", aggregate: "count" });
expect(still.rows[0].value).toBe(50);
});
it("raw SQL attempt: the analyst-SQL surface is refused, not silently executed", async () => {
await expect(
engine.executeQuery(ALPHA, { sql: `SELECT * FROM ${DB}.opt_analytics_event`, dataset_ids: [] }),
).rejects.toMatchObject({ code: "CAPABILITY_UNAVAILABLE" });
});
// ── layer 3: restricted users + row policies ─────────────────────────────
it("HOLE, layer 3 ABSENT: an unrestricted user with no predicate sees every tenant", async () => {
const res = await chQuery(
`SELECT uniqExact(tenant_id) AS value FROM ${DB}.opt_analytics_event FORMAT JSON`,
CLICKHOUSE_AUTH,
);
expect(res.ok).toBe(true);
expect(rowsOf(res.text)[0].value).toBe(2); // t_alpha AND t_beta
});
it("CLOSED, layer 3 PRESENT: the restricted user runs the SAME predicate-free statement and sees ONE tenant", async () => {
for (const stmt of restrictedTenantUserDdl({
db: DB,
tenantId: "t_alpha",
userName: RESTRICTED_USER,
password: RESTRICTED_PW,
tables: ALL_TABLES,
})) {
const r = await chQuery(stmt, CLICKHOUSE_AUTH);
expect(r.ok, `DDL failed: ${stmt.slice(0, 80)} -> ${r.text.slice(0, 200)}`).toBe(true);
}
const restrictedAuth = `${RESTRICTED_USER}:${RESTRICTED_PW}`;
// The IDENTICAL statement that leaked above, now as the restricted user.
const res = await chQuery(
`SELECT uniqExact(tenant_id) AS value FROM ${DB}.opt_analytics_event FORMAT JSON`,
restrictedAuth,
);
expect(res.ok, res.text.slice(0, 300)).toBe(true);
expect(rowsOf(res.text)[0].value).toBe(1);
const sum = await chQuery(
`SELECT sum(value) AS value FROM ${DB}.opt_analytics_event FORMAT JSON`,
restrictedAuth,
);
expect(rowsOf(sum.text)[0].value).toBe(50); // alpha's total, not 50050
});
it("the restricted user cannot write, and cannot reach a table it was not granted", async () => {
const restrictedAuth = `${RESTRICTED_USER}:${RESTRICTED_PW}`;
const write = await chQuery(
`INSERT INTO ${DB}.opt_analytics_event FORMAT JSONEachRow`,
restrictedAuth,
{},
{},
JSON.stringify({ tenant_id: "t_beta", project_id: "p", dataset_id: "d", metric: "m", event_id: "x", event_time: "2026-09-01 00:00:00.000", value: 1, dim_names: [], dim_values: [] }),
);
expect(write.ok).toBe(false);
const foreign = await chQuery(`SELECT count() FROM system.users FORMAT JSON`, restrictedAuth);
expect(foreign.ok).toBe(false);
});
// ── the structural precondition both layers rely on ──────────────────────
it("every analytics table's sorting key LEADS with tenant_id (what makes the predicate cheap)", async () => {
for (const table of ALL_TABLES) {
const res = await chQuery(sortingKeySql(DB, table), CLICKHOUSE_AUTH);
expect(res.ok).toBe(true);
expect(res.text.trim().startsWith("tenant_id"), `${table} sorting key: ${res.text.trim()}`).toBe(true);
}
});
it("the tenant value never appears in the statement text the server receives", async () => {
const compiled = compileMetricQuery(
{ tenant_id: "t_alpha", metric: "rows_ingested", aggregate: "sum" },
DB,
catalog,
DEFAULT_BUDGETS.INTERACTIVE.maxResultRows,
);
expect(compiled.sql).not.toContain("t_alpha");
const res = await chQuery(compiled.sql, CLICKHOUSE_AUTH, compiled.params);
expect(res.ok).toBe(true);
expect(rowsOf(res.text)[0].value).toBe(50);
});
});