CI (SIAX Cloud) / integration (push) Successful in 40s
CI (SIAX Cloud) / quality (push) Successful in 55s
CI (SIAX Cloud) / security (push) Successful in 21s
Security Scan (SIAX Cloud) / security (push) Successful in 22s
CI (SIAX Cloud) / integration (pull_request) Successful in 39s
CI (SIAX Cloud) / quality (pull_request) Successful in 54s
Security Scan (SIAX Cloud) / security (pull_request) Successful in 21s
CI (SIAX Cloud) / security (pull_request) Successful in 25s
Two corrections, both of my own errors.
1. SCOPE CREEP, NOW REVERTED. Running `prettier --write` on whole directories
reformatted 18 files this branch had no business touching — the entire
merged BI-P002..P012 core (arrow/, domain/, engine/, repositories/, five
analytics integration suites) and, absurdly,
search-opensearch.integration.test.ts, which has nothing to do with this
work. The changes were purely stylistic quote churn, but they added ~1,400
lines of noise to the diff, made the PR harder to review honestly, and
invited conflicts with the four other lanes working in this estate. All 18
are now byte-identical to origin/main again (verified with `git diff
origin/main --name-only`, which reports nothing for them). Formatting is now
applied only to the files this branch actually owns.
2. A COMMIT MESSAGE THAT DESCRIBED A FIX THAT WAS NOT IN THE CODE. Commit
3ee0230 said the "does NOT ship the Iceberg table" assertion had been moved
from a filesystem check to `git ls-files`. It had not. The patch anchor used
double quotes, prettier had already converted that file to single quotes, so
the replacement silently matched nothing and the script printed its success
line anyway. The suite then passed — but only because I had just deleted the
fixture directory, so the OLD filesystem assertion happened to be true.
That is precisely the failure mode this branch keeps writing tests against:
a green run that means something different from what it appears to mean. The
fix is now genuinely applied and verified the way it should have been the
first time — by running the suite WITH the Iceberg fixture present on disk,
which is the case that previously failed. It passes, and it now also asserts
that the parquet/csv fixtures ARE tracked, so a version shipping no fixture
at all would fail rather than pass vacuously.
957 tests pass with ClickHouse, Postgres and DuckDB live. typecheck, eslint and
prettier clean on the files this branch owns.
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
97 lines
4.0 KiB
TypeScript
97 lines
4.0 KiB
TypeScript
/**
|
|
* REAL ClickHouse integration test (Master-DoD §48/§50) for the Analytics
|
|
* capability. Runs against CLICKHOUSE_URL (+ CLICKHOUSE_AUTH "user:pass"
|
|
* when the server requires it); SKIPS HONESTLY when unset.
|
|
*
|
|
* Proves against the real engine: schema creation with TTL retention,
|
|
* JSONEachRow ingest, the closed metric-query DSL, and — the §69-relevant
|
|
* part — that the FORCED tenant predicate isolates tenants on real data.
|
|
*/
|
|
import { describe, it, expect, beforeAll } from "vitest";
|
|
import { ClickHouseAnalytics } from "@/src/analytics/clickhouse-analytics";
|
|
|
|
const CLICKHOUSE_URL = process.env.CLICKHOUSE_URL;
|
|
const run = CLICKHOUSE_URL ? describe : describe.skip;
|
|
|
|
const DB = `opt_itest_${Date.now().toString(36)}`;
|
|
|
|
run("Analytics capability against a LIVE ClickHouse", () => {
|
|
// The skip-variant still evaluates this callback body, so the constructor
|
|
// must tolerate the env-less case (the tests themselves never run then).
|
|
const analytics = new ClickHouseAnalytics({
|
|
baseUrl: CLICKHOUSE_URL ?? "http://skipped.invalid",
|
|
basicAuth: process.env.CLICKHOUSE_AUTH,
|
|
database: DB,
|
|
});
|
|
|
|
beforeAll(async () => {
|
|
expect(await analytics.health()).toBe(true);
|
|
await analytics.ensureSchema(30);
|
|
// ensureSchema is idempotent (CREATE IF NOT EXISTS).
|
|
await analytics.ensureSchema(30);
|
|
}, 60_000);
|
|
|
|
it("ingests events and answers aggregate metric queries", async () => {
|
|
const res = await analytics.ingest([
|
|
{ tenant_id: "t_alpha", dataset_id: "ds_companies", metric: "rows_ingested", value: 100 },
|
|
{ tenant_id: "t_alpha", dataset_id: "ds_companies", metric: "rows_ingested", value: 50 },
|
|
{ tenant_id: "t_alpha", dataset_id: "ds_vehicles", metric: "rows_ingested", value: 7 },
|
|
{ tenant_id: "t_beta", dataset_id: "ds_companies", metric: "rows_ingested", value: 999 },
|
|
]);
|
|
expect(res.ingested).toBe(4);
|
|
|
|
const total = await analytics.queryMetric({
|
|
tenant_id: "t_alpha",
|
|
metric: "rows_ingested",
|
|
aggregate: "sum",
|
|
});
|
|
expect(total).toEqual([{ value: 157 }]);
|
|
|
|
const perDataset = await analytics.queryMetric({
|
|
tenant_id: "t_alpha",
|
|
dataset_id: "ds_companies",
|
|
metric: "rows_ingested",
|
|
aggregate: "sum",
|
|
});
|
|
expect(perDataset).toEqual([{ value: 150 }]);
|
|
}, 30_000);
|
|
|
|
it("REAL tenant isolation: t_beta's numbers never leak into t_alpha aggregates", async () => {
|
|
const alpha = await analytics.queryMetric({ tenant_id: "t_alpha", metric: "rows_ingested", aggregate: "max" });
|
|
expect(alpha[0].value).toBe(100); // NOT 999
|
|
const beta = await analytics.queryMetric({ tenant_id: "t_beta", metric: "rows_ingested", aggregate: "sum" });
|
|
expect(beta[0].value).toBe(999);
|
|
const ghost = await analytics.queryMetric({ tenant_id: "t_gamma", metric: "rows_ingested", aggregate: "count" });
|
|
expect(ghost[0].value).toBe(0);
|
|
});
|
|
|
|
it("group-by dimension queries work through the closed DSL", async () => {
|
|
await analytics.ingest([
|
|
{ tenant_id: "t_alpha", dataset_id: "ds_x", metric: "dq_score", value: 90, dimensions: { severity: "warning" } },
|
|
{ tenant_id: "t_alpha", dataset_id: "ds_x", metric: "dq_score", value: 40, dimensions: { severity: "critical" } },
|
|
]);
|
|
const rows = await analytics.queryMetric({
|
|
tenant_id: "t_alpha",
|
|
metric: "dq_score",
|
|
aggregate: "avg",
|
|
group_by: "severity",
|
|
});
|
|
expect(rows).toHaveLength(2);
|
|
expect(rows.find((r) => r.group === "warning")?.value).toBe(90);
|
|
});
|
|
|
|
it("rejects tenant-less queries and invalid group_by identifiers (injection surface closed)", async () => {
|
|
await expect(
|
|
analytics.queryMetric({ tenant_id: " ", metric: "x", aggregate: "sum" })
|
|
).rejects.toMatchObject({ code: "INVALID_REQUEST" });
|
|
await expect(
|
|
analytics.queryMetric({
|
|
tenant_id: "t_alpha",
|
|
metric: "x",
|
|
aggregate: "sum",
|
|
group_by: "sev'; DROP TABLE opt_metric_events; --",
|
|
})
|
|
).rejects.toMatchObject({ code: "INVALID_REQUEST" });
|
|
});
|
|
});
|