CI (SIAX Cloud) / security (push) Successful in 41s
Security Scan (SIAX Cloud) / security (push) Successful in 40s
CI (SIAX Cloud) / integration (push) Successful in 52s
CI (SIAX Cloud) / quality (push) Successful in 1m5s
CI (SIAX Cloud) / security (pull_request) Successful in 31s
Security Scan (SIAX Cloud) / security (pull_request) Successful in 29s
CI (SIAX Cloud) / integration (pull_request) Successful in 45s
CI (SIAX Cloud) / quality (pull_request) Successful in 53s
Implements the eight BI program prompts that make 0pt the canonical SIAX
analytics capability. Every file is NEW: not one existing file is modified,
and src/analytics/clickhouse-analytics.ts is byte-identical to main.
BI-P002 — the nineteen canonical entities, verbatim, with tenant_id +
project_id on every tenant-owned record. sql/008 gives each one a table.
The four version entities BI-P002 names as immutable are enforced by BEFORE
UPDATE/DELETE triggers, not by a TypeScript `readonly` — proven by direct SQL
against a live Postgres.
BI-P003 — RAW/NORMALIZED/CURATED/SERVING with the read rule enforced:
assertReadableByBiSurface refuses a RAW dataset to a dashboard unless a
recorded exception exists. All eight required declarations are NOT NULL.
BI-P004 — DatasetContract and a fail-closed compatibility gate. Fifteen
breaking-change classes are each tested twice: refused under a minor bump,
accepted under a major one.
BI-P005 — Arrow IPC alongside JSON, with a hand-written FlatBuffers writer so
no npm dependency (or CVE surface) is added. Conformance comes from three
independent implementations, not from self-agreement: pyarrow 25.0.1 and
DuckDB 1.5.5 read identical values from both ClickHouse's Arrow and ours, and
ClickHouse's own Arrow C++ ingests our 20,000-row batch with exact aggregates.
BI-P009 — SiaxAnalyticsEngine with all eight capabilities. The existing
ClickHouseAnalytics is COMPOSED, never replaced. executeQuery (analyst SQL) is
declared and deliberately refused with CAPABILITY_UNAVAILABLE until BI-P017 and
BI-P033 land, rather than opening the raw-SQL hole BI-P011 enumerates.
BI-P010 — four purpose-built tables, not one giant one, and the difference is
measured against the counterfactual: 48.8ms/40,960 rows vs 174.4ms/5,400,000
rows for the same question; 5/660 granules vs 660/1149.
BI-P011 — layered isolation, each layer proven with the layer above removed.
Without the forced predicate the same statement returns 50,050 (both tenants);
with it, t_alpha gets 50. Without a row policy an unrestricted user sees 2
tenants; the restricted user running the IDENTICAL statement sees 1.
BI-P012 — every named limit, fired individually and identified by ClickHouse
error code, plus the half usually skipped: an unrelated tenant is timed while
a real CPU bomb is in flight and must still return the CORRECT answer.
Three defects found by writing tests that can fail:
* `CHECK (array_length(purposes,1) >= 1)` can NEVER fail — array_length is
NULL for '{}' and a NULL CHECK passes. Fixed with coalesce(); the test
that caught it is pinned.
* EXPLAIN ESTIMATE run under the caller's own max_rows_to_read fails with
code 158 exactly when the estimate would be over budget. Planner calls now
use a narrower envelope.
* result_overflow_mode=break returns HTTP 200 with 65,409 of 100,000 rows
and no indication it is short. Every SELECT ships `throw`.
Verified rather than assumed: server7's ClickHouse holds `opt_scraped` (51
tables, 407,793,471 rows, `persons`/`cz_owners` included) — the b0ts plane this
module is prohibited from touching — has no `opt_analytics` database, zero
tenant_id columns and zero row policies. Nothing there was modified; every
proof ran against disposable containers.
Evidence: evidence/bi-p002-p012/{test-transcript.txt,bi-p010-benchmark.json,
bi-p005-arrow-roundtrip.json,server7-clickhouse-probe.md}
Docs: docs/analytics/BI_ANALYTICS_CORE.md
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
110 lines
14 KiB
Plaintext
110 lines
14 KiB
Plaintext
BI-P002..BI-P012 — measured test transcript
|
||
===========================================
|
||
host: Linux 6.8.0-139-generic x86_64 node: v24.18.0 date(UTC): 2026-09-06T20:53:23Z
|
||
clickhouse under test: clickhouse/clickhouse-server:24.8 (disposable container, created and torn down by this run)
|
||
postgres under test: postgres:16-alpine (disposable container)
|
||
|
||
### 1. BI-P009 regression guard WITH the forced tenant predicate REMOVED (must be RED)
|
||
× BI-P009 regression guard: the forced tenant predicate > PRESERVED ClickHouseAnalytics still emits tenant_id = {tenant:String} on the wire 163ms
|
||
✓ BI-P009 regression guard: the forced tenant predicate > PRESERVED ClickHouseAnalytics still refuses a blank tenant_id 36ms
|
||
✓ BI-P009 regression guard: the forced tenant predicate > PRESERVED ClickHouseAnalytics still refuses a hostile group_by identifier 1ms
|
||
× BI-P009 regression guard: the forced tenant predicate > every statement the NEW compiler produces contains the forced predicate 34ms
|
||
✓ BI-P009 regression guard: the forced tenant predicate > the NEW compiler fails closed on a blank/absent tenant instead of emitting an unfiltered query 33ms
|
||
✓ BI-P009 regression guard: the forced tenant predicate > the constant itself is the predicate, so deleting the forcing cannot pass silently 1ms
|
||
⎯⎯⎯⎯⎯⎯⎯ Failed Tests 2 ⎯⎯⎯⎯⎯⎯⎯
|
||
Test Files 1 failed (1)
|
||
Tests 2 failed | 4 passed (6)
|
||
|
||
### 2. The same suite with the forcing RESTORED (must be GREEN)
|
||
✓ tests/analytics/tenant-predicate-regression.test.ts (6 tests) 85ms
|
||
Test Files 1 passed (1)
|
||
Tests 6 passed (6)
|
||
(working tree restored: 0 lines of diff in the preserved file)
|
||
|
||
### 3. BI-P011 tenant isolation against a LIVE disposable ClickHouse (hole shown OPEN, then CLOSED)
|
||
✓ tests/integration/analytics-tenant-isolation.integration.test.ts > BI-P011 tenant isolation against a LIVE ClickHouse > HOLE, layer 2 REMOVED: the same query WITHOUT the forced predicate returns t_beta's rows to t_alpha 49ms
|
||
✓ tests/integration/analytics-tenant-isolation.integration.test.ts > BI-P011 tenant isolation against a LIVE ClickHouse > CLOSED, layer 2 PRESENT: the compiled query returns t_alpha's 50 and never t_beta's 50000 60ms
|
||
✓ tests/integration/analytics-tenant-isolation.integration.test.ts > BI-P011 tenant isolation against a LIVE ClickHouse > tenant A querying tenant B's dataset id gets ZERO rows, not B's rows 23ms
|
||
✓ tests/integration/analytics-tenant-isolation.integration.test.ts > BI-P011 tenant isolation against a LIVE ClickHouse > a forged dataset id yields an empty result, never an error that leaks existence 22ms
|
||
✓ tests/integration/analytics-tenant-isolation.integration.test.ts > BI-P011 tenant isolation against a LIVE ClickHouse > a tenant with no data at all sees zero, not everyone's aggregate 24ms
|
||
✓ tests/integration/analytics-tenant-isolation.integration.test.ts > BI-P011 tenant isolation against a LIVE ClickHouse > group-by results carry only this tenant's dimension values 66ms
|
||
✓ tests/integration/analytics-tenant-isolation.integration.test.ts > BI-P011 tenant isolation against a LIVE ClickHouse > SQL/dimension/filter injection through the live path changes nothing 74ms
|
||
✓ tests/integration/analytics-tenant-isolation.integration.test.ts > BI-P011 tenant isolation against a LIVE ClickHouse > raw SQL attempt: the analyst-SQL surface is refused, not silently executed 1ms
|
||
✓ tests/integration/analytics-tenant-isolation.integration.test.ts > BI-P011 tenant isolation against a LIVE ClickHouse > HOLE, layer 3 ABSENT: an unrestricted user with no predicate sees every tenant 42ms
|
||
✓ tests/integration/analytics-tenant-isolation.integration.test.ts > BI-P011 tenant isolation against a LIVE ClickHouse > CLOSED, layer 3 PRESENT: the restricted user runs the SAME predicate-free statement and sees ONE tenant 200ms
|
||
✓ tests/integration/analytics-tenant-isolation.integration.test.ts > BI-P011 tenant isolation against a LIVE ClickHouse > the restricted user cannot write, and cannot reach a table it was not granted 90ms
|
||
✓ tests/integration/analytics-tenant-isolation.integration.test.ts > BI-P011 tenant isolation against a LIVE ClickHouse > every analytics table's sorting key LEADS with tenant_id (what makes the predicate cheap) 143ms
|
||
✓ tests/integration/analytics-tenant-isolation.integration.test.ts > BI-P011 tenant isolation against a LIVE ClickHouse > the tenant value never appears in the statement text the server receives 149ms
|
||
Test Files 1 passed (1)
|
||
Tests 13 passed (13)
|
||
|
||
### 4. BI-P009/BI-P010 provider + schema against the same LIVE ClickHouse
|
||
✓ tests/integration/analytics-engine-clickhouse.integration.test.ts > BI-P009/BI-P010 ClickHouse provider against a LIVE server > health() reports the live server's version 30ms
|
||
✓ tests/integration/analytics-engine-clickhouse.integration.test.ts > BI-P009/BI-P010 ClickHouse provider against a LIVE server > BI-P010: four purpose-built tables exist, NOT one giant table 12ms
|
||
✓ tests/integration/analytics-engine-clickhouse.integration.test.ts > BI-P009/BI-P010 ClickHouse provider against a LIVE server > BI-P010: the tables use different engines and different sort keys — the split is real 30ms
|
||
✓ tests/integration/analytics-engine-clickhouse.integration.test.ts > BI-P009/BI-P010 ClickHouse provider against a LIVE server > BI-P010: TTL and LowCardinality are actually applied, not just written in a comment 22ms
|
||
✓ tests/integration/analytics-engine-clickhouse.integration.test.ts > BI-P009/BI-P010 ClickHouse provider against a LIVE server > BI-P010: the materialized view actually populates the rollup 92ms
|
||
✓ tests/integration/analytics-engine-clickhouse.integration.test.ts > BI-P009/BI-P010 ClickHouse provider against a LIVE server > executeMetric returns grouped rows keyed by the CALLER-FACING dimension name 48ms
|
||
✓ tests/integration/analytics-engine-clickhouse.integration.test.ts > BI-P009/BI-P010 ClickHouse provider against a LIVE server > estimateQuery gets a real EXPLAIN ESTIMATE from the server and compares it to the budget 15ms
|
||
✓ tests/integration/analytics-engine-clickhouse.integration.test.ts > BI-P009/BI-P010 ClickHouse provider against a LIVE server > estimateQuery REFUSES a query whose estimate exceeds a tightened quota 11ms
|
||
✓ tests/integration/analytics-engine-clickhouse.integration.test.ts > BI-P009/BI-P010 ClickHouse provider against a LIVE server > explain() proves the forced predicate reaches the PRIMARY KEY, pruning parts 15ms
|
||
✓ tests/integration/analytics-engine-clickhouse.integration.test.ts > BI-P009/BI-P010 ClickHouse provider against a LIVE server > cancelQuery issues a real KILL QUERY the server accepts 33ms
|
||
✓ tests/integration/analytics-engine-clickhouse.integration.test.ts > BI-P009/BI-P010 ClickHouse provider against a LIVE server > ingestProjection writes the SERVING tier with the identity's tenant, ignoring any row-level claim 33ms
|
||
✓ tests/integration/analytics-engine-clickhouse.integration.test.ts > BI-P009/BI-P010 ClickHouse provider against a LIVE server > refreshProjection makes a re-ingest idempotent (ReplacingMergeTree + OPTIMIZE FINAL) 58ms
|
||
✓ tests/integration/analytics-engine-clickhouse.integration.test.ts > BI-P009/BI-P010 ClickHouse provider against a LIVE server > the PRESERVED ClickHouseAnalytics path still ingests and answers through the engine 35ms
|
||
Test Files 1 passed (1)
|
||
Tests 13 passed (13)
|
||
|
||
### 5. BI-P012 resource governance / query bomb against the same LIVE ClickHouse
|
||
✓ tests/integration/analytics-resource-governance.integration.test.ts > BI-P012 resource governance against a LIVE ClickHouse > max_rows_to_read terminates a read bomb (CH code 158 TOO_MANY_ROWS) 60ms
|
||
✓ tests/integration/analytics-resource-governance.integration.test.ts > BI-P012 resource governance against a LIVE ClickHouse > max_bytes_to_read terminates a scan bomb (CH code 307 TOO_MANY_BYTES) 26ms
|
||
✓ tests/integration/analytics-resource-governance.integration.test.ts > BI-P012 resource governance against a LIVE ClickHouse > max_execution_time terminates a slow bomb (CH code 159 TIMEOUT_EXCEEDED) 1014ms
|
||
✓ tests/integration/analytics-resource-governance.integration.test.ts > BI-P012 resource governance against a LIVE ClickHouse > max_result_rows terminates a result bomb and does NOT return a partial result 422ms
|
||
✓ tests/integration/analytics-resource-governance.integration.test.ts > BI-P012 resource governance against a LIVE ClickHouse > max_memory_usage terminates a memory bomb (CH code 241 MEMORY_LIMIT_EXCEEDED) 84ms
|
||
✓ tests/integration/analytics-resource-governance.integration.test.ts > BI-P012 resource governance against a LIVE ClickHouse > readonly=1 means the query text cannot re-raise its own limits 37ms
|
||
✓ tests/integration/analytics-resource-governance.integration.test.ts > BI-P012 resource governance against a LIVE ClickHouse > readonly=1 blocks writes through the query surface 26ms
|
||
✓ tests/integration/analytics-resource-governance.integration.test.ts > BI-P012 resource governance against a LIVE ClickHouse > an unrelated tenant stays within its declared SLO WHILE a query bomb is in flight 8163ms
|
||
✓ tests/integration/analytics-resource-governance.integration.test.ts > BI-P012 resource governance against a LIVE ClickHouse > the per-tenant concurrency gate refuses the N+1st in-flight query for THAT tenant only 220ms
|
||
Test Files 1 passed (1)
|
||
Tests 9 passed (9)
|
||
|
||
### 6. BI-P005 Arrow interop with ClickHouse's own Arrow C++ implementation
|
||
✓ tests/integration/analytics-arrow-clickhouse.integration.test.ts > BI-P005 Arrow IPC against a LIVE ClickHouse > THEIR encoder, OUR reader: ClickHouse ArrowStream decodes with the expected schema and row count 106ms
|
||
✓ tests/integration/analytics-arrow-clickhouse.integration.test.ts > BI-P005 Arrow IPC against a LIVE ClickHouse > WITHOUT output_format_arrow_string_as_string, ClickHouse emits Binary — the semantic loss the setting prevents 42ms
|
||
✓ tests/integration/analytics-arrow-clickhouse.integration.test.ts > BI-P005 Arrow IPC against a LIVE ClickHouse > OUR encoder, THEIR reader: ClickHouse INSERTs our Arrow and returns every value unchanged 76ms
|
||
✓ tests/integration/analytics-arrow-clickhouse.integration.test.ts > BI-P005 Arrow IPC against a LIVE ClickHouse > full circle: ClickHouse -> Arrow -> our decoder -> ClickHouse yields identical rows 281ms
|
||
✓ tests/integration/analytics-arrow-clickhouse.integration.test.ts > BI-P005 Arrow IPC against a LIVE ClickHouse > a large result survives the round trip (batching/offsets are not only correct for three rows) 690ms
|
||
Test Files 1 passed (1)
|
||
Tests 5 passed (5)
|
||
|
||
### 7. BI-P002 domain model against a LIVE disposable Postgres (restore drill + immutability)
|
||
✓ tests/analytics/analytics-domain-postgres.realinfra.test.ts > BI-P002 analytics domain model against a LIVE Postgres > restore drill: sql/008 applies cleanly on top of the replayed chain 6ms
|
||
✓ tests/analytics/analytics-domain-postgres.realinfra.test.ts > BI-P002 analytics domain model against a LIVE Postgres > all nineteen entity tables really exist after the restore 125ms
|
||
✓ tests/analytics/analytics-domain-postgres.realinfra.test.ts > BI-P002 analytics domain model against a LIVE Postgres > replaying sql/008 a second time is a no-op, not an error 253ms
|
||
✓ tests/analytics/analytics-domain-postgres.realinfra.test.ts > BI-P002 analytics domain model against a LIVE Postgres > persists a dataset and reads it back through the repository 27ms
|
||
✓ tests/analytics/analytics-domain-postgres.realinfra.test.ts > BI-P002 analytics domain model against a LIVE Postgres > TENANT ISOLATION: t_beta cannot read t_alpha's dataset, even with the exact id 7ms
|
||
✓ tests/analytics/analytics-domain-postgres.realinfra.test.ts > BI-P002 analytics domain model against a LIVE Postgres > TENANT ISOLATION: a version cannot be minted against another tenant's dataset 4ms
|
||
✓ tests/analytics/analytics-domain-postgres.realinfra.test.ts > BI-P002 analytics domain model against a LIVE Postgres > dataset versions are monotonic and carry the contract hash 136ms
|
||
✓ tests/analytics/analytics-domain-postgres.realinfra.test.ts > BI-P002 analytics domain model against a LIVE Postgres > IMMUTABILITY is enforced by the DATABASE: UPDATE and DELETE on a version row are refused 27ms
|
||
✓ tests/analytics/analytics-domain-postgres.realinfra.test.ts > BI-P002 analytics domain model against a LIVE Postgres > all four immutable entities refuse an UPDATE, not just the one that was tested first 638ms
|
||
✓ tests/analytics/analytics-domain-postgres.realinfra.test.ts > BI-P002 analytics domain model against a LIVE Postgres > a metric version after the first must state WHY the number changed meaning 123ms
|
||
✓ tests/analytics/analytics-domain-postgres.realinfra.test.ts > BI-P002 analytics domain model against a LIVE Postgres > a DatasetGrant without a purpose is refused by the DATABASE (purpose limitation) 134ms
|
||
✓ tests/analytics/analytics-domain-postgres.realinfra.test.ts > BI-P002 analytics domain model against a LIVE Postgres > a BIApplicationBinding without a purpose is refused too, and carries no credential column 40ms
|
||
✓ tests/analytics/analytics-domain-postgres.realinfra.test.ts > BI-P002 analytics domain model against a LIVE Postgres > the dimension catalog is tenant-scoped and FAILS CLOSED for a tenant with none 111ms
|
||
✓ tests/analytics/analytics-domain-postgres.realinfra.test.ts > BI-P002 analytics domain model against a LIVE Postgres > a dimension whose physical field is not identifier-safe is refused by the DATABASE 3ms
|
||
✓ tests/analytics/analytics-domain-postgres.realinfra.test.ts > BI-P002 analytics domain model against a LIVE Postgres > the repository refuses to act without a verified tenant scope 1ms
|
||
Test Files 1 passed (1)
|
||
Tests 15 passed (15)
|
||
|
||
### 8. Repository gates (the same commands .gitea/workflows/ci.yml runs)
|
||
TYPECHECK_EXIT=0
|
||
Test Files 54 passed | 9 skipped (63)
|
||
Tests 463 passed | 80 skipped (543)
|
||
TEST_EXIT=0
|
||
BUILD_EXIT=0
|
||
lint (ADVISORY in CI, continue-on-error): 12 pre-existing errors in tests/p0/* — ZERO in any file this branch adds.
|
||
|
||
### 9. Scope check
|
||
src/analytics/clickhouse-analytics.ts diff vs main: 0 lines (0 = preserved untouched)
|
||
.gitea/workflows diff vs main: 0 lines (0 = no protected file touched)
|