Claude Opus 5
84bb9faa4f
CI (SIAX Cloud) / security (push) Successful in 41s
Security Scan (SIAX Cloud) / security (push) Successful in 40s
CI (SIAX Cloud) / integration (push) Successful in 52s
CI (SIAX Cloud) / quality (push) Successful in 1m5s
CI (SIAX Cloud) / security (pull_request) Successful in 31s
Security Scan (SIAX Cloud) / security (pull_request) Successful in 29s
CI (SIAX Cloud) / integration (pull_request) Successful in 45s
CI (SIAX Cloud) / quality (pull_request) Successful in 53s
Implements the eight BI program prompts that make 0pt the canonical SIAX
analytics capability. Every file is NEW: not one existing file is modified,
and src/analytics/clickhouse-analytics.ts is byte-identical to main.
BI-P002 — the nineteen canonical entities, verbatim, with tenant_id +
project_id on every tenant-owned record. sql/008 gives each one a table.
The four version entities BI-P002 names as immutable are enforced by BEFORE
UPDATE/DELETE triggers, not by a TypeScript `readonly` — proven by direct SQL
against a live Postgres.
BI-P003 — RAW/NORMALIZED/CURATED/SERVING with the read rule enforced:
assertReadableByBiSurface refuses a RAW dataset to a dashboard unless a
recorded exception exists. All eight required declarations are NOT NULL.
BI-P004 — DatasetContract and a fail-closed compatibility gate. Fifteen
breaking-change classes are each tested twice: refused under a minor bump,
accepted under a major one.
BI-P005 — Arrow IPC alongside JSON, with a hand-written FlatBuffers writer so
no npm dependency (or CVE surface) is added. Conformance comes from three
independent implementations, not from self-agreement: pyarrow 25.0.1 and
DuckDB 1.5.5 read identical values from both ClickHouse's Arrow and ours, and
ClickHouse's own Arrow C++ ingests our 20,000-row batch with exact aggregates.
BI-P009 — SiaxAnalyticsEngine with all eight capabilities. The existing
ClickHouseAnalytics is COMPOSED, never replaced. executeQuery (analyst SQL) is
declared and deliberately refused with CAPABILITY_UNAVAILABLE until BI-P017 and
BI-P033 land, rather than opening the raw-SQL hole BI-P011 enumerates.
BI-P010 — four purpose-built tables, not one giant one, and the difference is
measured against the counterfactual: 48.8ms/40,960 rows vs 174.4ms/5,400,000
rows for the same question; 5/660 granules vs 660/1149.
BI-P011 — layered isolation, each layer proven with the layer above removed.
Without the forced predicate the same statement returns 50,050 (both tenants);
with it, t_alpha gets 50. Without a row policy an unrestricted user sees 2
tenants; the restricted user running the IDENTICAL statement sees 1.
BI-P012 — every named limit, fired individually and identified by ClickHouse
error code, plus the half usually skipped: an unrelated tenant is timed while
a real CPU bomb is in flight and must still return the CORRECT answer.
Three defects found by writing tests that can fail:
* `CHECK (array_length(purposes,1) >= 1)` can NEVER fail — array_length is
NULL for '{}' and a NULL CHECK passes. Fixed with coalesce(); the test
that caught it is pinned.
* EXPLAIN ESTIMATE run under the caller's own max_rows_to_read fails with
code 158 exactly when the estimate would be over budget. Planner calls now
use a narrower envelope.
* result_overflow_mode=break returns HTTP 200 with 65,409 of 100,000 rows
and no indication it is short. Every SELECT ships `throw`.
Verified rather than assumed: server7's ClickHouse holds `opt_scraped` (51
tables, 407,793,471 rows, `persons`/`cz_owners` included) — the b0ts plane this
module is prohibited from touching — has no `opt_analytics` database, zero
tenant_id columns and zero row policies. Nothing there was modified; every
proof ran against disposable containers.
Evidence: evidence/bi-p002-p012/{test-transcript.txt,bi-p010-benchmark.json,
bi-p005-arrow-roundtrip.json,server7-clickhouse-probe.md}
Docs: docs/analytics/BI_ANALYTICS_CORE.md
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
272 lines
13 KiB
Markdown
272 lines
13 KiB
Markdown
# 0pt Analytics Core — BI-P002 … BI-P012
|
||
|
||
Status of this document: it describes what **is in this repository and has been
|
||
run**, and states separately what has **not** been deployed. Nothing here is a
|
||
plan written in the present tense.
|
||
|
||
---
|
||
|
||
## 0. The ownership boundary (normative, not negotiable)
|
||
|
||
```
|
||
SIAX / 0pt owns the analytics engine, the query compiler, the tenant forcing,
|
||
the resource governance and the semantic/contract layer.
|
||
|
||
Product repos (VehIQ and the rest) own DOMAIN MEANING — what a row means,
|
||
which metric a dealer cares about, how a screen reads.
|
||
```
|
||
|
||
Concretely, and enforced rather than asserted:
|
||
|
||
* `BIApplicationBinding` (BI-P002 entity 19) has **no credential field**, in
|
||
TypeScript or in `sql/008_analytics_domain_model.sql`. A bound BI
|
||
application authenticates to the 0pt Analytics API with a CL0UD identity and
|
||
receives compiled, tenant-forced results. It never learns a ClickHouse DSN,
|
||
user, or password. Two tests assert the absence of such a column — one over
|
||
the migration text, one over `information_schema.columns` on a live
|
||
Postgres — so adding one later is a test failure.
|
||
* `AnalyticsIdentity` (BI-P009) is a **separate parameter** from the request,
|
||
never a field on it. This estate has already shipped a live cross-tenant
|
||
hole by reading a tenant out of a request body (INF0 Wave 2); the type
|
||
shape is what stops that recurring here.
|
||
|
||
---
|
||
|
||
## 1. What each prompt produced
|
||
|
||
| Prompt | Deliverable | Where |
|
||
|---|---|---|
|
||
| BI-P002 | 19 canonical entities + migrations + tenant-scoped persistence + restore drill | `src/analytics/domain/entities.ts`, `sql/008_analytics_domain_model.sql`, `src/analytics/repositories/analytics-domain-repository.ts` |
|
||
| BI-P003 | RAW / NORMALIZED / CURATED / SERVING, with the BI-surface read rule enforced | `src/analytics/domain/tiers.ts` |
|
||
| BI-P004 | `DatasetContract` + a fail-closed compatibility gate | `src/analytics/domain/contract.ts` |
|
||
| BI-P005 | Arrow IPC encode/decode + ClickHouse pass-through | `src/analytics/arrow/ipc.ts`, `src/analytics/arrow/flatbuffer-builder.ts` |
|
||
| BI-P009 | `SiaxAnalyticsEngine` interface + ClickHouse provider, existing implementation **preserved** | `src/analytics/engine/siax-analytics-engine.ts`, `src/analytics/engine/clickhouse-engine.ts` |
|
||
| BI-P010 | Four purpose-built ClickHouse tables + rollup MV, benchmarked | `src/analytics/engine/schema.ts`, `scripts/bi-p010-benchmark.mjs` |
|
||
| BI-P011 | Layered tenant isolation: verified identity, forced compiler predicate, restricted users, row policies | `src/analytics/engine/query-compiler.ts` |
|
||
| BI-P012 | Every named ClickHouse limit + CL0UD quota + per-tenant admission control | `src/analytics/engine/resource-governance.ts` |
|
||
|
||
---
|
||
|
||
## 2. BI-P009 — what "preserve" means here
|
||
|
||
`src/analytics/clickhouse-analytics.ts` is **byte-identical to main**. The new
|
||
engine composes it:
|
||
|
||
* `health()` delegates to it;
|
||
* the pre-existing `opt_metric_events` path is reachable through
|
||
`executeLegacyMetric()`, still using its own compiler;
|
||
* everything else uses the new compiler for the BI-P010 tables.
|
||
|
||
"Do not regress the existing forced tenant predicate" is enforced by
|
||
`tests/analytics/tenant-predicate-regression.test.ts`, which asserts the exact
|
||
string `tenant_id = {tenant:String}` on the URL the adapter builds. It has been
|
||
run with the forcing deliberately deleted from both compilers and goes RED
|
||
(2 of 6 tests fail); the transcript is
|
||
`evidence/bi-p002-p012/test-transcript.txt` §1–§2.
|
||
|
||
The eight capabilities are all implemented, with one **deliberate refusal**:
|
||
`executeQuery` (analyst SQL) throws `CAPABILITY_UNAVAILABLE` until BI-P017
|
||
(SQL dialect contract) and BI-P033 (analytics authorization) exist. Shipping a
|
||
raw-SQL surface first would create exactly the "raw SQL attempt" hole BI-P011
|
||
enumerates. It is declared and refused, not silently missing.
|
||
|
||
---
|
||
|
||
## 3. BI-P010 — why four tables, measured
|
||
|
||
BI-P010 warns against "one giant unbounded table for every analytics use
|
||
case". `scripts/bi-p010-benchmark.mjs` builds both fixtures and the
|
||
counterfactual, and times them. Measured on a disposable
|
||
`clickhouse/clickhouse-server:24.8` (full transcript:
|
||
`evidence/bi-p002-p012/bi-p010-benchmark.json`):
|
||
|
||
| Question | Purpose-built | One giant table |
|
||
|---|---|---|
|
||
| 90-day metric total, one tenant | **48.8 ms p50**, 40 960 rows read | 174.4 ms p50, **5 400 000** rows read |
|
||
| Vehicle point lookup by key, one tenant | **49.3 ms p50**, 8 192 rows read | 236.6 ms p50, **7 401 152** rows read |
|
||
| Avg asking price by make over 2M vehicles | **389.6 ms p50** | 680.8 ms p50 |
|
||
| Granules read for the metric total | **5 / 660** | 660 / 1 149 |
|
||
|
||
Fixtures: 5 400 000 SaaS events (200 tenants × 12 metrics × 90 days) and
|
||
4 000 000 vehicle projections (2M for the tenant under test + 2M decoys owned
|
||
by 50 other tenants).
|
||
|
||
**Honest finding that does not flatter the design:** at this scale the
|
||
`AggregatingMergeTree` rollup (37.98 ms p50, 21 416 rows read) was *not* faster
|
||
in wall time than scanning raw events (32.55 ms p50, 40 960 rows read). It
|
||
reads about half the rows, and its value is the long TTL — raw events expire
|
||
in 30 days, the rollup keeps three years — so the rollup earns its keep on
|
||
retention and on scale beyond this fixture, not on latency at 5.4M rows. Do
|
||
not quote it as a speed win.
|
||
|
||
---
|
||
|
||
## 4. BI-P011 — the isolation proof
|
||
|
||
Four layers, and each one is tested with the layer above it *removed*:
|
||
|
||
1. **Verified CL0UD identity.** `AnalyticsIdentity` is not part of the
|
||
request; a blank tenant or principal is `UNAUTHENTICATED`, never a wildcard.
|
||
2. **The 0pt query compiler forces the tenant.** Every compiled statement
|
||
contains `tenant_id = {tenant:String}` as its first WHERE condition, with
|
||
the value bound as an HTTP parameter — the tenant string never appears in
|
||
the SQL text.
|
||
3. **Restricted database users + row policies.**
|
||
`restrictedTenantUserDdl()` renders a per-tenant ClickHouse user with
|
||
`readonly = 1` and a `CREATE ROW POLICY ... USING tenant_id = '<tenant>'`
|
||
on every analytics table.
|
||
4. **Separate database/cluster tier** for very-high-isolation customers:
|
||
supported by construction — the engine takes a `database` and a `baseUrl`
|
||
per instance — but **not yet operated**, because there is no deployed
|
||
ClickHouse for 0pt analytics at all (see §7).
|
||
|
||
The proof, run against a disposable ClickHouse with 50 rows for `t_alpha`
|
||
(value 1 each) and 50 for `t_beta` (value 1000 each):
|
||
|
||
* **hole, layer 2 removed:** the same statement minus the forced predicate
|
||
returns **50 050** — both tenants in one number;
|
||
* **closed, layer 2 present:** `t_alpha` gets **50**, `t_beta` gets **50 000**;
|
||
* **hole, layer 3 absent:** an unrestricted user issuing a predicate-free
|
||
`uniqExact(tenant_id)` sees **2** tenants;
|
||
* **closed, layer 3 present:** the restricted user issuing the *identical*
|
||
statement sees **1**, and `sum(value)` returns 50 rather than 50 050.
|
||
|
||
Adversarial cases from BI-P011's list — tenant A querying B's dataset, forged
|
||
dataset IDs, raw SQL, SQL injection, dimension identifier injection, filter
|
||
injection — are covered in
|
||
`tests/analytics/query-compiler-injection.test.ts` (compiler layer) and
|
||
`tests/integration/analytics-tenant-isolation.integration.test.ts` (live).
|
||
|
||
Dimension names are **resolved through a per-tenant allow-list**
|
||
(`DimensionDefinition`), never escaped and never spliced. A tenant that has
|
||
declared no dimensions may reference none — fail closed.
|
||
|
||
---
|
||
|
||
## 5. BI-P012 — governance, and the partial-result trap
|
||
|
||
Every setting BI-P012 names is applied. Five ride on the statement itself
|
||
(`max_execution_time`, `max_result_rows`, `max_rows_to_read`,
|
||
`max_bytes_to_read`, `max_memory_usage`) under `readonly = 1`, so the query
|
||
text cannot re-raise them; `max_concurrent_queries` is a per-user quota plus a
|
||
local admission gate.
|
||
|
||
Overflow modes are **`throw`, never `break`**. That is not a style preference.
|
||
Measured on ClickHouse 24.8: with `result_overflow_mode = break` and
|
||
`max_result_rows = 10`, a 100 000-row query returns **HTTP 200 with 65 409
|
||
rows** and nothing in the payload says it is short. A truncated answer that
|
||
looks complete is the analytics form of a false green.
|
||
|
||
Query-bomb outcomes, each fired individually and identified by ClickHouse
|
||
error code: 158 `TOO_MANY_ROWS`, 307 `TOO_MANY_BYTES`, 159 `TIMEOUT_EXCEEDED`,
|
||
396 `TOO_MANY_ROWS_OR_BYTES`, 241 `MEMORY_LIMIT_EXCEEDED`.
|
||
|
||
The second half of the DoD — "without degrading unrelated tenant beyond
|
||
declared SLO" — is timed, not assumed: an unrelated tenant's interactive query
|
||
is run five times *while* an 8-second CPU bomb is in flight, must still return
|
||
the **correct** answer, and must stay under a declared 5000 ms ceiling (measured ~30 ms on an idle
|
||
server; the ceiling is set wide for a shared CI runner, and a genuinely
|
||
starved tenant would take seconds or fail).
|
||
|
||
Two measured details worth keeping:
|
||
|
||
* `EXPLAIN ESTIMATE` must **not** run under the caller's own
|
||
`max_rows_to_read`. It fails with code 158 exactly when the estimate would
|
||
be over budget — i.e. in the only case the estimate matters. Planner calls
|
||
use `planningSettings()`, a narrower envelope, instead.
|
||
* `SELECT count() FROM numbers(2e10)` is *not* a query bomb — ClickHouse
|
||
answers it in about 5 seconds because it can skip the values. The bomb in
|
||
the test hashes every row, which cannot be short-circuited.
|
||
|
||
---
|
||
|
||
## 6. BI-P005 — Arrow, and why it is not self-certified
|
||
|
||
Two paths:
|
||
|
||
* **pass-through** — ClickHouse emits Arrow IPC itself (`FORMAT ArrowStream`);
|
||
0pt streams those bytes unchanged. `output_format_arrow_string_as_string=1`
|
||
is required, or String columns arrive as Arrow **Binary** and both pyarrow
|
||
and DuckDB read them as bytes: silent semantic loss.
|
||
* **encoded** — for results 0pt assembles itself, `encodeArrowStream()` writes
|
||
the IPC stream, including a small hand-written FlatBuffers writer (no new
|
||
npm dependency, no new supply-chain surface).
|
||
|
||
A hand-rolled encoder is only worth having if somebody else's parser agrees
|
||
with it, so conformance is established by three independent implementations:
|
||
|
||
* **pyarrow 25.0.1** and **DuckDB 1.5.5** decode both ClickHouse's bytes and
|
||
0pt's bytes to *identical* values and types
|
||
(`evidence/bi-p002-p012/bi-p005-arrow-roundtrip.json`, verdict `PASS`);
|
||
* **ClickHouse's own Arrow C++** ingests 0pt-encoded Arrow — including a
|
||
20 000-row batch whose aggregates come back exact — in
|
||
`tests/integration/analytics-arrow-clickhouse.integration.test.ts`, which
|
||
runs on every push.
|
||
|
||
Supported types and their caveats are the table in `ARROW_TYPE_SUPPORT`;
|
||
`DateTime64(6|9)` and `UInt64` above 2^63 are deliberately **outside** the
|
||
supported set rather than silently lossy.
|
||
|
||
Arrow is also not an escape hatch from BI-P012:
|
||
`assertArrowWithinResultBudget()` decodes the row count and refuses a payload
|
||
over the caller's `max_result_rows`.
|
||
|
||
---
|
||
|
||
## 7. What is NOT done — verified, not assumed
|
||
|
||
* **There is no deployed ClickHouse for 0pt analytics.** Probed 2026-09-06 on
|
||
server7, the only ClickHouse in the estate: databases are
|
||
`opt_scraped`, `default`, `system`, `INFORMATION_SCHEMA`,
|
||
`information_schema`. There is no `opt_analytics` database, so
|
||
`ClickHouseAnalytics` — the "existing working" provider — has never been
|
||
run against a production instance. Everything in this document was proven
|
||
against disposable containers.
|
||
* **server7's ClickHouse must not be used for this.** It holds
|
||
`opt_scraped` — 51 tables, 407 793 471 rows, including `persons` and
|
||
`cz_owners`. That is the b0ts scraping plane, which
|
||
`src/analytics/clickhouse-analytics.ts`'s own header already prohibits this
|
||
module from touching (Ratsit/person-PII legal case, written DPIA
|
||
prohibition on quarantined market data). It also has **zero** columns named
|
||
`tenant_id` or `project_id` anywhere on the server and **zero** row
|
||
policies, so it could not satisfy BI-P011 even if it were legally
|
||
available. Nothing on that server was modified by this work.
|
||
* **BI-P011 layer 5** (separate database/cluster tier per high-isolation
|
||
customer) is supported by construction but not operated.
|
||
* **`executeQuery`** is refused pending BI-P017/BI-P033 (§2).
|
||
* The domain repository covers `AnalyticsDataset`, `DatasetVersion`,
|
||
`MetricDefinition`, `MetricVersion` and `DimensionDefinition`. The other
|
||
fourteen entities have tables, constraints and types but no repository
|
||
methods yet; their consumers (dashboards, reports, quality runs, lineage)
|
||
arrive with BI-P025 … BI-P035.
|
||
|
||
---
|
||
|
||
## 8. Running the proofs yourself
|
||
|
||
```bash
|
||
# unit suites (no infrastructure)
|
||
pnpm vitest run tests/analytics
|
||
|
||
# live ClickHouse suites
|
||
docker run -d --name opt-bi-ch -p 18123:8123 \
|
||
-e CLICKHOUSE_USER=opt_admin -e CLICKHOUSE_PASSWORD=<throwaway> \
|
||
-e CLICKHOUSE_DEFAULT_ACCESS_MANAGEMENT=1 clickhouse/clickhouse-server:24.8
|
||
CLICKHOUSE_URL=http://localhost:18123 CLICKHOUSE_AUTH=opt_admin:<throwaway> \
|
||
pnpm vitest run tests/integration
|
||
|
||
# BI-P002 restore drill
|
||
docker run -d --name opt-bi-pg -p 15455:5432 \
|
||
-e POSTGRES_USER=opt -e POSTGRES_PASSWORD=<throwaway> -e POSTGRES_DB=opt_bi_test \
|
||
postgres:16-alpine
|
||
ANALYTICS_TEST_DATABASE_URL=postgres://opt:<throwaway>@localhost:15455/opt_bi_test \
|
||
pnpm vitest run tests/analytics/analytics-domain-postgres.realinfra.test.ts
|
||
|
||
# BI-P010 benchmark and BI-P005 external roundtrip (rewrite the evidence files)
|
||
CLICKHOUSE_URL=... CLICKHOUSE_AUTH=... node scripts/bi-p010-benchmark.mjs
|
||
CLICKHOUSE_URL=... CLICKHOUSE_AUTH=... node scripts/bi-p005-arrow-roundtrip.mjs
|
||
```
|
||
|
||
Both scripts CREATE and DROP a database. Never point either at server7 or at
|
||
any ClickHouse holding b0ts or quarantined market data.
|