Files
0pt/evidence/bi-p002-p012/test-transcript.txt
T
Claude Opus 5 84bb9faa4f
CI (SIAX Cloud) / security (push) Successful in 41s
Security Scan (SIAX Cloud) / security (push) Successful in 40s
CI (SIAX Cloud) / integration (push) Successful in 52s
CI (SIAX Cloud) / quality (push) Successful in 1m5s
CI (SIAX Cloud) / security (pull_request) Successful in 31s
Security Scan (SIAX Cloud) / security (pull_request) Successful in 29s
CI (SIAX Cloud) / integration (pull_request) Successful in 45s
CI (SIAX Cloud) / quality (pull_request) Successful in 53s
feat(analytics): BI-P002..BI-P012 analytics core — domain model, tiers, contracts, Arrow, ClickHouse engine, isolation, governance
Implements the eight BI program prompts that make 0pt the canonical SIAX
analytics capability. Every file is NEW: not one existing file is modified,
and src/analytics/clickhouse-analytics.ts is byte-identical to main.

BI-P002 — the nineteen canonical entities, verbatim, with tenant_id +
project_id on every tenant-owned record. sql/008 gives each one a table.
The four version entities BI-P002 names as immutable are enforced by BEFORE
UPDATE/DELETE triggers, not by a TypeScript `readonly` — proven by direct SQL
against a live Postgres.

BI-P003 — RAW/NORMALIZED/CURATED/SERVING with the read rule enforced:
assertReadableByBiSurface refuses a RAW dataset to a dashboard unless a
recorded exception exists. All eight required declarations are NOT NULL.

BI-P004 — DatasetContract and a fail-closed compatibility gate. Fifteen
breaking-change classes are each tested twice: refused under a minor bump,
accepted under a major one.

BI-P005 — Arrow IPC alongside JSON, with a hand-written FlatBuffers writer so
no npm dependency (or CVE surface) is added. Conformance comes from three
independent implementations, not from self-agreement: pyarrow 25.0.1 and
DuckDB 1.5.5 read identical values from both ClickHouse's Arrow and ours, and
ClickHouse's own Arrow C++ ingests our 20,000-row batch with exact aggregates.

BI-P009 — SiaxAnalyticsEngine with all eight capabilities. The existing
ClickHouseAnalytics is COMPOSED, never replaced. executeQuery (analyst SQL) is
declared and deliberately refused with CAPABILITY_UNAVAILABLE until BI-P017 and
BI-P033 land, rather than opening the raw-SQL hole BI-P011 enumerates.

BI-P010 — four purpose-built tables, not one giant one, and the difference is
measured against the counterfactual: 48.8ms/40,960 rows vs 174.4ms/5,400,000
rows for the same question; 5/660 granules vs 660/1149.

BI-P011 — layered isolation, each layer proven with the layer above removed.
Without the forced predicate the same statement returns 50,050 (both tenants);
with it, t_alpha gets 50. Without a row policy an unrestricted user sees 2
tenants; the restricted user running the IDENTICAL statement sees 1.

BI-P012 — every named limit, fired individually and identified by ClickHouse
error code, plus the half usually skipped: an unrelated tenant is timed while
a real CPU bomb is in flight and must still return the CORRECT answer.

Three defects found by writing tests that can fail:
  * `CHECK (array_length(purposes,1) >= 1)` can NEVER fail — array_length is
    NULL for '{}' and a NULL CHECK passes. Fixed with coalesce(); the test
    that caught it is pinned.
  * EXPLAIN ESTIMATE run under the caller's own max_rows_to_read fails with
    code 158 exactly when the estimate would be over budget. Planner calls now
    use a narrower envelope.
  * result_overflow_mode=break returns HTTP 200 with 65,409 of 100,000 rows
    and no indication it is short. Every SELECT ships `throw`.

Verified rather than assumed: server7's ClickHouse holds `opt_scraped` (51
tables, 407,793,471 rows, `persons`/`cz_owners` included) — the b0ts plane this
module is prohibited from touching — has no `opt_analytics` database, zero
tenant_id columns and zero row policies. Nothing there was modified; every
proof ran against disposable containers.

Evidence: evidence/bi-p002-p012/{test-transcript.txt,bi-p010-benchmark.json,
bi-p005-arrow-roundtrip.json,server7-clickhouse-probe.md}
Docs: docs/analytics/BI_ANALYTICS_CORE.md

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
2026-09-06 23:01:27 +02:00

110 lines
14 KiB
Plaintext
Raw Blame History

This file contains ambiguous Unicode characters
This file contains Unicode characters that might be confused with other characters. If you think that this is intentional, you can safely ignore this warning. Use the Escape button to reveal them.
BI-P002..BI-P012 — measured test transcript
===========================================
host: Linux 6.8.0-139-generic x86_64 node: v24.18.0 date(UTC): 2026-09-06T20:53:23Z
clickhouse under test: clickhouse/clickhouse-server:24.8 (disposable container, created and torn down by this run)
postgres under test: postgres:16-alpine (disposable container)
### 1. BI-P009 regression guard WITH the forced tenant predicate REMOVED (must be RED)
× BI-P009 regression guard: the forced tenant predicate > PRESERVED ClickHouseAnalytics still emits tenant_id = {tenant:String} on the wire 163ms
✓ BI-P009 regression guard: the forced tenant predicate > PRESERVED ClickHouseAnalytics still refuses a blank tenant_id 36ms
✓ BI-P009 regression guard: the forced tenant predicate > PRESERVED ClickHouseAnalytics still refuses a hostile group_by identifier 1ms
× BI-P009 regression guard: the forced tenant predicate > every statement the NEW compiler produces contains the forced predicate 34ms
✓ BI-P009 regression guard: the forced tenant predicate > the NEW compiler fails closed on a blank/absent tenant instead of emitting an unfiltered query 33ms
✓ BI-P009 regression guard: the forced tenant predicate > the constant itself is the predicate, so deleting the forcing cannot pass silently 1ms
⎯⎯⎯⎯⎯⎯⎯ Failed Tests 2 ⎯⎯⎯⎯⎯⎯⎯
Test Files 1 failed (1)
Tests 2 failed | 4 passed (6)
### 2. The same suite with the forcing RESTORED (must be GREEN)
✓ tests/analytics/tenant-predicate-regression.test.ts (6 tests) 85ms
Test Files 1 passed (1)
Tests 6 passed (6)
(working tree restored: 0 lines of diff in the preserved file)
### 3. BI-P011 tenant isolation against a LIVE disposable ClickHouse (hole shown OPEN, then CLOSED)
✓ tests/integration/analytics-tenant-isolation.integration.test.ts > BI-P011 tenant isolation against a LIVE ClickHouse > HOLE, layer 2 REMOVED: the same query WITHOUT the forced predicate returns t_beta's rows to t_alpha 49ms
✓ tests/integration/analytics-tenant-isolation.integration.test.ts > BI-P011 tenant isolation against a LIVE ClickHouse > CLOSED, layer 2 PRESENT: the compiled query returns t_alpha's 50 and never t_beta's 50000 60ms
✓ tests/integration/analytics-tenant-isolation.integration.test.ts > BI-P011 tenant isolation against a LIVE ClickHouse > tenant A querying tenant B's dataset id gets ZERO rows, not B's rows 23ms
✓ tests/integration/analytics-tenant-isolation.integration.test.ts > BI-P011 tenant isolation against a LIVE ClickHouse > a forged dataset id yields an empty result, never an error that leaks existence 22ms
✓ tests/integration/analytics-tenant-isolation.integration.test.ts > BI-P011 tenant isolation against a LIVE ClickHouse > a tenant with no data at all sees zero, not everyone's aggregate 24ms
✓ tests/integration/analytics-tenant-isolation.integration.test.ts > BI-P011 tenant isolation against a LIVE ClickHouse > group-by results carry only this tenant's dimension values 66ms
✓ tests/integration/analytics-tenant-isolation.integration.test.ts > BI-P011 tenant isolation against a LIVE ClickHouse > SQL/dimension/filter injection through the live path changes nothing 74ms
✓ tests/integration/analytics-tenant-isolation.integration.test.ts > BI-P011 tenant isolation against a LIVE ClickHouse > raw SQL attempt: the analyst-SQL surface is refused, not silently executed 1ms
✓ tests/integration/analytics-tenant-isolation.integration.test.ts > BI-P011 tenant isolation against a LIVE ClickHouse > HOLE, layer 3 ABSENT: an unrestricted user with no predicate sees every tenant 42ms
✓ tests/integration/analytics-tenant-isolation.integration.test.ts > BI-P011 tenant isolation against a LIVE ClickHouse > CLOSED, layer 3 PRESENT: the restricted user runs the SAME predicate-free statement and sees ONE tenant 200ms
✓ tests/integration/analytics-tenant-isolation.integration.test.ts > BI-P011 tenant isolation against a LIVE ClickHouse > the restricted user cannot write, and cannot reach a table it was not granted 90ms
✓ tests/integration/analytics-tenant-isolation.integration.test.ts > BI-P011 tenant isolation against a LIVE ClickHouse > every analytics table's sorting key LEADS with tenant_id (what makes the predicate cheap) 143ms
✓ tests/integration/analytics-tenant-isolation.integration.test.ts > BI-P011 tenant isolation against a LIVE ClickHouse > the tenant value never appears in the statement text the server receives 149ms
Test Files 1 passed (1)
Tests 13 passed (13)
### 4. BI-P009/BI-P010 provider + schema against the same LIVE ClickHouse
✓ tests/integration/analytics-engine-clickhouse.integration.test.ts > BI-P009/BI-P010 ClickHouse provider against a LIVE server > health() reports the live server's version 30ms
✓ tests/integration/analytics-engine-clickhouse.integration.test.ts > BI-P009/BI-P010 ClickHouse provider against a LIVE server > BI-P010: four purpose-built tables exist, NOT one giant table 12ms
✓ tests/integration/analytics-engine-clickhouse.integration.test.ts > BI-P009/BI-P010 ClickHouse provider against a LIVE server > BI-P010: the tables use different engines and different sort keys — the split is real 30ms
✓ tests/integration/analytics-engine-clickhouse.integration.test.ts > BI-P009/BI-P010 ClickHouse provider against a LIVE server > BI-P010: TTL and LowCardinality are actually applied, not just written in a comment 22ms
✓ tests/integration/analytics-engine-clickhouse.integration.test.ts > BI-P009/BI-P010 ClickHouse provider against a LIVE server > BI-P010: the materialized view actually populates the rollup 92ms
✓ tests/integration/analytics-engine-clickhouse.integration.test.ts > BI-P009/BI-P010 ClickHouse provider against a LIVE server > executeMetric returns grouped rows keyed by the CALLER-FACING dimension name 48ms
✓ tests/integration/analytics-engine-clickhouse.integration.test.ts > BI-P009/BI-P010 ClickHouse provider against a LIVE server > estimateQuery gets a real EXPLAIN ESTIMATE from the server and compares it to the budget 15ms
✓ tests/integration/analytics-engine-clickhouse.integration.test.ts > BI-P009/BI-P010 ClickHouse provider against a LIVE server > estimateQuery REFUSES a query whose estimate exceeds a tightened quota 11ms
✓ tests/integration/analytics-engine-clickhouse.integration.test.ts > BI-P009/BI-P010 ClickHouse provider against a LIVE server > explain() proves the forced predicate reaches the PRIMARY KEY, pruning parts 15ms
✓ tests/integration/analytics-engine-clickhouse.integration.test.ts > BI-P009/BI-P010 ClickHouse provider against a LIVE server > cancelQuery issues a real KILL QUERY the server accepts 33ms
✓ tests/integration/analytics-engine-clickhouse.integration.test.ts > BI-P009/BI-P010 ClickHouse provider against a LIVE server > ingestProjection writes the SERVING tier with the identity's tenant, ignoring any row-level claim 33ms
✓ tests/integration/analytics-engine-clickhouse.integration.test.ts > BI-P009/BI-P010 ClickHouse provider against a LIVE server > refreshProjection makes a re-ingest idempotent (ReplacingMergeTree + OPTIMIZE FINAL) 58ms
✓ tests/integration/analytics-engine-clickhouse.integration.test.ts > BI-P009/BI-P010 ClickHouse provider against a LIVE server > the PRESERVED ClickHouseAnalytics path still ingests and answers through the engine 35ms
Test Files 1 passed (1)
Tests 13 passed (13)
### 5. BI-P012 resource governance / query bomb against the same LIVE ClickHouse
✓ tests/integration/analytics-resource-governance.integration.test.ts > BI-P012 resource governance against a LIVE ClickHouse > max_rows_to_read terminates a read bomb (CH code 158 TOO_MANY_ROWS) 60ms
✓ tests/integration/analytics-resource-governance.integration.test.ts > BI-P012 resource governance against a LIVE ClickHouse > max_bytes_to_read terminates a scan bomb (CH code 307 TOO_MANY_BYTES) 26ms
✓ tests/integration/analytics-resource-governance.integration.test.ts > BI-P012 resource governance against a LIVE ClickHouse > max_execution_time terminates a slow bomb (CH code 159 TIMEOUT_EXCEEDED) 1014ms
✓ tests/integration/analytics-resource-governance.integration.test.ts > BI-P012 resource governance against a LIVE ClickHouse > max_result_rows terminates a result bomb and does NOT return a partial result 422ms
✓ tests/integration/analytics-resource-governance.integration.test.ts > BI-P012 resource governance against a LIVE ClickHouse > max_memory_usage terminates a memory bomb (CH code 241 MEMORY_LIMIT_EXCEEDED) 84ms
✓ tests/integration/analytics-resource-governance.integration.test.ts > BI-P012 resource governance against a LIVE ClickHouse > readonly=1 means the query text cannot re-raise its own limits 37ms
✓ tests/integration/analytics-resource-governance.integration.test.ts > BI-P012 resource governance against a LIVE ClickHouse > readonly=1 blocks writes through the query surface 26ms
✓ tests/integration/analytics-resource-governance.integration.test.ts > BI-P012 resource governance against a LIVE ClickHouse > an unrelated tenant stays within its declared SLO WHILE a query bomb is in flight 8163ms
✓ tests/integration/analytics-resource-governance.integration.test.ts > BI-P012 resource governance against a LIVE ClickHouse > the per-tenant concurrency gate refuses the N+1st in-flight query for THAT tenant only 220ms
Test Files 1 passed (1)
Tests 9 passed (9)
### 6. BI-P005 Arrow interop with ClickHouse's own Arrow C++ implementation
✓ tests/integration/analytics-arrow-clickhouse.integration.test.ts > BI-P005 Arrow IPC against a LIVE ClickHouse > THEIR encoder, OUR reader: ClickHouse ArrowStream decodes with the expected schema and row count 106ms
✓ tests/integration/analytics-arrow-clickhouse.integration.test.ts > BI-P005 Arrow IPC against a LIVE ClickHouse > WITHOUT output_format_arrow_string_as_string, ClickHouse emits Binary — the semantic loss the setting prevents 42ms
✓ tests/integration/analytics-arrow-clickhouse.integration.test.ts > BI-P005 Arrow IPC against a LIVE ClickHouse > OUR encoder, THEIR reader: ClickHouse INSERTs our Arrow and returns every value unchanged 76ms
✓ tests/integration/analytics-arrow-clickhouse.integration.test.ts > BI-P005 Arrow IPC against a LIVE ClickHouse > full circle: ClickHouse -> Arrow -> our decoder -> ClickHouse yields identical rows 281ms
✓ tests/integration/analytics-arrow-clickhouse.integration.test.ts > BI-P005 Arrow IPC against a LIVE ClickHouse > a large result survives the round trip (batching/offsets are not only correct for three rows) 690ms
Test Files 1 passed (1)
Tests 5 passed (5)
### 7. BI-P002 domain model against a LIVE disposable Postgres (restore drill + immutability)
✓ tests/analytics/analytics-domain-postgres.realinfra.test.ts > BI-P002 analytics domain model against a LIVE Postgres > restore drill: sql/008 applies cleanly on top of the replayed chain 6ms
✓ tests/analytics/analytics-domain-postgres.realinfra.test.ts > BI-P002 analytics domain model against a LIVE Postgres > all nineteen entity tables really exist after the restore 125ms
✓ tests/analytics/analytics-domain-postgres.realinfra.test.ts > BI-P002 analytics domain model against a LIVE Postgres > replaying sql/008 a second time is a no-op, not an error 253ms
✓ tests/analytics/analytics-domain-postgres.realinfra.test.ts > BI-P002 analytics domain model against a LIVE Postgres > persists a dataset and reads it back through the repository 27ms
✓ tests/analytics/analytics-domain-postgres.realinfra.test.ts > BI-P002 analytics domain model against a LIVE Postgres > TENANT ISOLATION: t_beta cannot read t_alpha's dataset, even with the exact id 7ms
✓ tests/analytics/analytics-domain-postgres.realinfra.test.ts > BI-P002 analytics domain model against a LIVE Postgres > TENANT ISOLATION: a version cannot be minted against another tenant's dataset 4ms
✓ tests/analytics/analytics-domain-postgres.realinfra.test.ts > BI-P002 analytics domain model against a LIVE Postgres > dataset versions are monotonic and carry the contract hash 136ms
✓ tests/analytics/analytics-domain-postgres.realinfra.test.ts > BI-P002 analytics domain model against a LIVE Postgres > IMMUTABILITY is enforced by the DATABASE: UPDATE and DELETE on a version row are refused 27ms
✓ tests/analytics/analytics-domain-postgres.realinfra.test.ts > BI-P002 analytics domain model against a LIVE Postgres > all four immutable entities refuse an UPDATE, not just the one that was tested first 638ms
✓ tests/analytics/analytics-domain-postgres.realinfra.test.ts > BI-P002 analytics domain model against a LIVE Postgres > a metric version after the first must state WHY the number changed meaning 123ms
✓ tests/analytics/analytics-domain-postgres.realinfra.test.ts > BI-P002 analytics domain model against a LIVE Postgres > a DatasetGrant without a purpose is refused by the DATABASE (purpose limitation) 134ms
✓ tests/analytics/analytics-domain-postgres.realinfra.test.ts > BI-P002 analytics domain model against a LIVE Postgres > a BIApplicationBinding without a purpose is refused too, and carries no credential column 40ms
✓ tests/analytics/analytics-domain-postgres.realinfra.test.ts > BI-P002 analytics domain model against a LIVE Postgres > the dimension catalog is tenant-scoped and FAILS CLOSED for a tenant with none 111ms
✓ tests/analytics/analytics-domain-postgres.realinfra.test.ts > BI-P002 analytics domain model against a LIVE Postgres > a dimension whose physical field is not identifier-safe is refused by the DATABASE 3ms
✓ tests/analytics/analytics-domain-postgres.realinfra.test.ts > BI-P002 analytics domain model against a LIVE Postgres > the repository refuses to act without a verified tenant scope 1ms
Test Files 1 passed (1)
Tests 15 passed (15)
### 8. Repository gates (the same commands .gitea/workflows/ci.yml runs)
TYPECHECK_EXIT=0
Test Files 54 passed | 9 skipped (63)
Tests 463 passed | 80 skipped (543)
TEST_EXIT=0
BUILD_EXIT=0
lint (ADVISORY in CI, continue-on-error): 12 pre-existing errors in tests/p0/* — ZERO in any file this branch adds.
### 9. Scope check
src/analytics/clickhouse-analytics.ts diff vs main: 0 lines (0 = preserved untouched)
.gitea/workflows diff vs main: 0 lines (0 = no protected file touched)