Files
0pt/tests/analytics/query-compiler-injection.test.ts
T
Claude Opus 5 84bb9faa4f
CI (SIAX Cloud) / security (push) Successful in 41s
Security Scan (SIAX Cloud) / security (push) Successful in 40s
CI (SIAX Cloud) / integration (push) Successful in 52s
CI (SIAX Cloud) / quality (push) Successful in 1m5s
CI (SIAX Cloud) / security (pull_request) Successful in 31s
Security Scan (SIAX Cloud) / security (pull_request) Successful in 29s
CI (SIAX Cloud) / integration (pull_request) Successful in 45s
CI (SIAX Cloud) / quality (pull_request) Successful in 53s
feat(analytics): BI-P002..BI-P012 analytics core — domain model, tiers, contracts, Arrow, ClickHouse engine, isolation, governance
Implements the eight BI program prompts that make 0pt the canonical SIAX
analytics capability. Every file is NEW: not one existing file is modified,
and src/analytics/clickhouse-analytics.ts is byte-identical to main.

BI-P002 — the nineteen canonical entities, verbatim, with tenant_id +
project_id on every tenant-owned record. sql/008 gives each one a table.
The four version entities BI-P002 names as immutable are enforced by BEFORE
UPDATE/DELETE triggers, not by a TypeScript `readonly` — proven by direct SQL
against a live Postgres.

BI-P003 — RAW/NORMALIZED/CURATED/SERVING with the read rule enforced:
assertReadableByBiSurface refuses a RAW dataset to a dashboard unless a
recorded exception exists. All eight required declarations are NOT NULL.

BI-P004 — DatasetContract and a fail-closed compatibility gate. Fifteen
breaking-change classes are each tested twice: refused under a minor bump,
accepted under a major one.

BI-P005 — Arrow IPC alongside JSON, with a hand-written FlatBuffers writer so
no npm dependency (or CVE surface) is added. Conformance comes from three
independent implementations, not from self-agreement: pyarrow 25.0.1 and
DuckDB 1.5.5 read identical values from both ClickHouse's Arrow and ours, and
ClickHouse's own Arrow C++ ingests our 20,000-row batch with exact aggregates.

BI-P009 — SiaxAnalyticsEngine with all eight capabilities. The existing
ClickHouseAnalytics is COMPOSED, never replaced. executeQuery (analyst SQL) is
declared and deliberately refused with CAPABILITY_UNAVAILABLE until BI-P017 and
BI-P033 land, rather than opening the raw-SQL hole BI-P011 enumerates.

BI-P010 — four purpose-built tables, not one giant one, and the difference is
measured against the counterfactual: 48.8ms/40,960 rows vs 174.4ms/5,400,000
rows for the same question; 5/660 granules vs 660/1149.

BI-P011 — layered isolation, each layer proven with the layer above removed.
Without the forced predicate the same statement returns 50,050 (both tenants);
with it, t_alpha gets 50. Without a row policy an unrestricted user sees 2
tenants; the restricted user running the IDENTICAL statement sees 1.

BI-P012 — every named limit, fired individually and identified by ClickHouse
error code, plus the half usually skipped: an unrelated tenant is timed while
a real CPU bomb is in flight and must still return the CORRECT answer.

Three defects found by writing tests that can fail:
  * `CHECK (array_length(purposes,1) >= 1)` can NEVER fail — array_length is
    NULL for '{}' and a NULL CHECK passes. Fixed with coalesce(); the test
    that caught it is pinned.
  * EXPLAIN ESTIMATE run under the caller's own max_rows_to_read fails with
    code 158 exactly when the estimate would be over budget. Planner calls now
    use a narrower envelope.
  * result_overflow_mode=break returns HTTP 200 with 65,409 of 100,000 rows
    and no indication it is short. Every SELECT ships `throw`.

Verified rather than assumed: server7's ClickHouse holds `opt_scraped` (51
tables, 407,793,471 rows, `persons`/`cz_owners` included) — the b0ts plane this
module is prohibited from touching — has no `opt_analytics` database, zero
tenant_id columns and zero row policies. Nothing there was modified; every
proof ran against disposable containers.

Evidence: evidence/bi-p002-p012/{test-transcript.txt,bi-p010-benchmark.json,
bi-p005-arrow-roundtrip.json,server7-clickhouse-probe.md}
Docs: docs/analytics/BI_ANALYTICS_CORE.md

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
2026-09-06 23:01:27 +02:00

225 lines
7.3 KiB
TypeScript

/**
* BI-P011's adversarial list, at the compiler layer:
*
* tenant A query B dataset · tenant A forged dataset ID · raw SQL attempt
* SQL injection · dimension identifier injection · filter injection
*
* These are the cases that never reach the network. The cases that DO reach a
* real server — and the with-fix/without-fix proof BI-P011 actually demands —
* are tests/integration/analytics-tenant-isolation.integration.test.ts.
*/
import { describe, it, expect } from "vitest";
import {
assertSafeIdentifier,
compileMetricQuery,
compileProjectionQuery,
dimensionCatalogFrom,
restrictedTenantUserDdl,
} from "@/src/analytics/engine/query-compiler";
const catalog = dimensionCatalogFrom({ severity: "severity", make: "vehicle_make" });
const HOSTILE = [
"x'; DROP TABLE opt_analytics_event; --",
"1 OR 1=1",
"tenant_id",
"*/ OR 1=1 /*",
"severity UNION ALL SELECT tenant_id FROM opt_analytics_event",
"dim_values[1]",
"__proto__",
"constructor",
"sever ity",
"sévérité",
"a".repeat(200),
"",
];
describe("BI-P011 injection surfaces at the compiler", () => {
it("dimension identifier injection: hostile group_by names are refused, not escaped", () => {
for (const bad of HOSTILE) {
expect(() =>
compileMetricQuery(
{ tenant_id: "t_alpha", metric: "m", aggregate: "sum", group_by: [bad] },
"opt_analytics",
catalog,
1000,
),
`group_by "${bad}" was not refused`,
).toThrowError(/unknown group_by dimension/);
}
});
it("filter injection: hostile filter KEYS are refused by catalog resolution", () => {
for (const bad of HOSTILE) {
expect(() =>
compileMetricQuery(
{ tenant_id: "t_alpha", metric: "m", aggregate: "sum", filters: { [bad]: "v" } },
"opt_analytics",
catalog,
1000,
),
`filter key "${bad}" was not refused`,
).toThrowError(/unknown filter dimension/);
}
});
it("prototype-chain keys resolve to undefined rather than to Object.prototype members", () => {
// A naive `catalog[name]` lookup would resolve "constructor" to a
// function and "toString" to a method; the Map-based catalog must not.
for (const key of ["constructor", "toString", "hasOwnProperty", "__proto__", "valueOf"]) {
expect(catalog.resolve(key)).toBeUndefined();
}
});
it("filter VALUES are never spliced, however hostile", () => {
const compiled = compileMetricQuery(
{
tenant_id: "t_alpha",
metric: "m",
aggregate: "sum",
filters: { make: "volvo'; DROP TABLE opt_analytics_event; --" },
},
"opt_analytics",
catalog,
1000,
);
expect(compiled.sql).not.toContain("DROP TABLE");
expect(compiled.sql).not.toContain("volvo");
expect(compiled.params.param_f0v).toBe("volvo'; DROP TABLE opt_analytics_event; --");
// The physical column name came from the catalog, not the caller.
expect(compiled.params.param_f0n).toBe("vehicle_make");
});
it("metric and dataset VALUES are never spliced either", () => {
const compiled = compileMetricQuery(
{
tenant_id: "t_alpha",
metric: "m'; SELECT 1; --",
dataset_id: "ds'; SELECT 1; --",
aggregate: "sum",
},
"opt_analytics",
catalog,
1000,
);
expect(compiled.sql).not.toContain("SELECT 1");
expect(compiled.sql.match(/SELECT/g)).toHaveLength(1);
});
it("raw SQL attempt: there is no request shape that reaches a second table", () => {
const compiled = compileMetricQuery(
{ tenant_id: "t_alpha", metric: "m", aggregate: "sum" },
"opt_analytics",
catalog,
1000,
);
expect(compiled.sql).not.toMatch(/\bUNION\b/i);
expect(compiled.sql).not.toMatch(/\bJOIN\b/i);
expect((compiled.sql.match(/\bFROM\b/gi) ?? []).length).toBe(1);
expect(compiled.table).toBe("opt_analytics_event");
});
it("a forged table name is refused rather than reaching the FROM clause", () => {
expect(() =>
compileMetricQuery(
{ tenant_id: "t_alpha", metric: "m", aggregate: "sum", table: "system.tables" as never },
"opt_analytics",
catalog,
1000,
),
).toThrowError(/invalid table identifier/);
});
it("an unsupported aggregate is refused rather than being interpolated", () => {
expect(() =>
compileMetricQuery(
{ tenant_id: "t_alpha", metric: "m", aggregate: "sum(1),tenant_id" as never },
"opt_analytics",
catalog,
1000,
),
).toThrowError(/unsupported aggregate/);
});
it("group-by cardinality is capped so a caller cannot explode the GROUP BY", () => {
const many = Array.from({ length: 9 }, () => "severity");
expect(() =>
compileMetricQuery(
{ tenant_id: "t_alpha", metric: "m", aggregate: "sum", group_by: many },
"opt_analytics",
catalog,
1000,
),
).toThrowError(/at most 8 group-by/);
});
it("LIMIT is clamped to the governance ceiling, and never below 1", () => {
const big = compileMetricQuery(
{ tenant_id: "t_alpha", metric: "m", aggregate: "sum", limit: 10_000_000 },
"opt_analytics",
catalog,
500,
);
expect(big.sql).toContain("LIMIT 500");
const zero = compileMetricQuery(
{ tenant_id: "t_alpha", metric: "m", aggregate: "sum", limit: 0 },
"opt_analytics",
catalog,
500,
);
expect(zero.sql).toContain("LIMIT 1");
});
it("the projection compiler forces the tenant and parameterizes entity keys", () => {
const compiled = compileProjectionQuery(
{ tenant_id: "t_alpha", dataset_id: "ds_x", entity_keys: ["a'; DROP", "b"] },
"opt_analytics",
100,
);
expect(compiled.sql).toMatch(/WHERE tenant_id = \{tenant:String\}/);
expect(compiled.sql).not.toContain("DROP");
expect(compiled.params.param_entityKeys).toBe(JSON.stringify(["a'; DROP", "b"]));
});
it("assertSafeIdentifier accepts real identifiers and rejects everything else", () => {
for (const good of ["severity", "vehicle_make", "_x", "a1_b2"]) {
expect(assertSafeIdentifier(good, "test")).toBe(good);
}
for (const bad of ["1abc", "a-b", "a b", "a;b", "", "a".repeat(64)]) {
expect(() => assertSafeIdentifier(bad, "test")).toThrowError(/invalid test identifier/);
}
});
it("restrictedTenantUserDdl refuses a tenant id or password it cannot safely embed", () => {
expect(() =>
restrictedTenantUserDdl({
db: "opt_analytics",
tenantId: "t_alpha' OR '1'='1",
userName: "opt_t_alpha",
password: "pw",
tables: ["opt_analytics_event"],
}),
).toThrowError(/not policy-safe/);
expect(() =>
restrictedTenantUserDdl({
db: "opt_analytics",
tenantId: "t_alpha",
userName: "opt_t_alpha",
password: "pw' OR '1'='1",
tables: ["opt_analytics_event"],
}),
).toThrowError(/cannot be embedded/);
const stmts = restrictedTenantUserDdl({
db: "opt_analytics",
tenantId: "t_alpha",
userName: "opt_t_alpha",
password: "safe_pw_123",
tables: ["opt_analytics_event"],
});
expect(stmts.join(" ")).toContain("CREATE ROW POLICY IF NOT EXISTS");
expect(stmts.join(" ")).toContain("USING tenant_id = 't_alpha'");
});
});