ec26ddc296a710c35830d03ee2458d03c8f7ef5a
5
Commits
| Author | SHA1 | Message | Date | |
|---|---|---|---|---|
|
|
6cb00add57 |
feat(analytics): BI-P025/BI-P026 — quality providers that fail on empty, freshness that refuses to be silent
CI (SIAX Cloud) / integration (push) Successful in 43s
CI (SIAX Cloud) / security (push) Successful in 23s
CI (SIAX Cloud) / quality (push) Successful in 57s
Security Scan (SIAX Cloud) / security (push) Successful in 21s
CI (SIAX Cloud) / security (pull_request) Successful in 23s
CI (SIAX Cloud) / integration (pull_request) Successful in 40s
CI (SIAX Cloud) / quality (pull_request) Successful in 55s
Security Scan (SIAX Cloud) / security (pull_request) Successful in 25s
Both prompts are about a FALSE GREEN, so both are tested by constructing the false green and requiring a failure. BI-P025. Three provider kinds, and the interface makes the difference visible instead of implying parity: `providerKind` is "executing" or "ingesting". The native provider runs checks; the dbt and Great Expectations adapters INGEST results produced elsewhere (n0d#38 owns dbt execution against BI-P022, and GE is Python — nothing here runs it). Calling all three "providers" while only one executes is how "we have Great Expectations coverage" ends up in a status report. THE FALSE GREEN, proven against a live ClickHouse 24.8.14.39: a `not_null` check over an EMPTY table returns violations=0, evaluated=0. The raw counts are asserted first — the engine really does say zero-over-zero — and then the provider is required to call it a FAILURE, not a pass. Every rule compiles to a VIOLATION expression rather than a "rows that are fine" count, so the distinction cannot be lost at the runner layer. Same treatment for the ingesting providers: a MISSING dbt result is a failure, and dbt's `error`, `warn` and `skipped` are each asserted not to be passes. `skipped` in particular is the exact shape of l0re's 28 silently-skipped Postgres tests. The gate: zero verdicts QUARANTINES. "No rules ran" and "everything passed" produce the same green in most implementations, which is how a quality gate stops gating without anyone noticing — api0's `trivy --exit-code 0` in another costume. A later passing run does not auto-release either; release takes a principal and a reason, or a flapping rule would quarantine and un-quarantine a dataset between two reads of the same dashboard. BI-P025's DoD is the READ side: a quarantined projection is REFUSED, not annotated. An annotation on a dashboard nobody reads is not a quarantine. Live counts, all exact, from evidence/bi-p025/native-quality-live.json: not_null 5/118, range 3, distribution 4, unique 11. The unique figure is 11 rather than the 6 the first version of the test asserted — the five empty-string customers are duplicates of each other, so not_null and unique overlap on the same defect. The engine was right and the fixture arithmetic was wrong; the breakdown is written into the test rather than the number quietly adjusted. Tenant isolation re-proven on this path specifically, because a quality run uses an ADMINISTRATIVE credential: with 618 rows in the base table, t_alpha evaluates 118 and t_beta 500, and t_beta's blocking failure quarantines t_beta alone. Without the forced predicate one tenant's dirty rows would quarantine another tenant's projection. BI-P026. Five states, and the DoD's operative word is "silently". So the gate sits at the boundary every result crosses: a result with NO freshness envelope is unreleasable — not defaulted to FRESH, not defaulted to UNKNOWN — because "we forgot to attach it" and "the data is current" are indistinguishable to a reader, and any default makes forgetting the silent path. A non-current state without an explicit acknowledgement is refused, so a consumer that ignores the envelope cannot render stale numbers as current. FAILED outranks age: a pipeline that just failed may still hold recently-timestamped data from the previous run, and reporting that as FRESH is how a broken pipeline stays invisible for a day. UNKNOWN is never collapsed into FRESH. 54 unit tests, 11 live tests. typecheck + eslint clean. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com> |
||
|
|
8f6357044f |
feat(analytics): BI-P018/BI-P019/BI-P021 — execution model, verified cancellation, a budget that stops queries
CI (SIAX Cloud) / integration (push) Successful in 43s
CI (SIAX Cloud) / security (push) Successful in 21s
CI (SIAX Cloud) / quality (push) Successful in 1m5s
Security Scan (SIAX Cloud) / security (push) Successful in 23s
CI (SIAX Cloud) / security (pull_request) Successful in 24s
CI (SIAX Cloud) / quality (pull_request) Successful in 1m0s
CI (SIAX Cloud) / integration (pull_request) Successful in 42s
Security Scan (SIAX Cloud) / security (pull_request) Successful in 38s
BI-P018 — every field the prompt names, plus the one it names by its absence.
The record carries `compiled_query_hash` and never the statement, and that is
ENFORCED: `assertNoRawSql` runs before any record is emitted and refuses one
whose `error` field looks like SQL. The realistic leak is ClickHouse's own
error text, which echoes the query — the test uses exactly that string. The
recorder writes at EVERY transition, because a recorder that only writes at
the end loses precisely the executions worth investigating. Illegal state
transitions throw rather than overwrite a terminal row.
The one place raw SQL is retained stays BI-P017's analyst statement history,
where a privileged human ran unreviewed SQL and "who ran what" must survive.
Different record, different retention posture, stated in both files.
BI-P019 — "stops" is VERIFIED, not requested. Aborting the client stops US
waiting, not the server; KILL QUERY without SYNC tells you nothing; even SYNC
can return while the query unwinds. So cancelAndVerify aborts, kills, then
POLLS the process list until the query is GONE or the deadline passes — and
if the deadline passes it returns `verified_stopped: false` rather than a
cheerful `true`. The return type is an outcome object, not a boolean, so a
caller cannot mistake "we asked" for "it stopped". An unreachable engine is
read as "still running" (fail closed), so the kill is still attempted at the
moment an operator most needs it to be.
Proven live against ClickHouse 24.8.14.39, with the counterfactual first:
* an identical bomb left alone IS still running after 2s — so "gone after
we cancelled" is not just "the query was short";
* the cancelled query had read 6,559,476,156 rows, and was verified gone
57ms after the request;
* system.query_log records ExceptionWhileProcessing with code 394,
QUERY_WAS_CANCELLED — ClickHouse's verdict, not ours.
BI-P021 — the honest version. A budget checked once before execution bounds
nothing, and the module says so in its first paragraph. ClickHouse's
max_rows_to_read bounds ONE query; it cannot express "this tenant may scan X
per hour", which is what a budget is for. So enforcement polls the engine's
live read_rows/read_bytes counters and CANCELS the running query, through
BI-P019 with verification, when window spend would be exceeded.
Measured live: a 1,000,000-row window cancelled the query at 130,883,409 rows,
92ms in, verified stopped, code 394. THE OVERSHOOT IS DOCUMENTED, in the module
and in the evidence: enforcement is granular to the poll interval, so this is
a bound of the form "within roughly one poll interval of engine throughput",
not a hard cap. Quoting it as a hard cap would be wrong.
Two more things stated rather than implied:
* `enforceInFlight: false` is reported by describeEnforcement() as
"NOT ENFORCED — this budget is advisory", and the constructor REFUSES to
accept enforceInFlight without a process control;
* an unshared window store is reported as non-binding across replicas
("with N replicas a tenant can spend up to N times this budget").
* a query the budget killed is still charged for the work it burned. Not
charging is how a tenant learns the cheapest query is a cancelled one.
Evidence: evidence/bi-p019-p021/cancellation-and-budget.json
268 unit tests, 7 live tests. typecheck + eslint clean.
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
|
||
|
|
d9c8f7190b |
feat(analytics): BI-P017 — analyst SQL contract, proven against live ClickHouse with the guard removed
CI (SIAX Cloud) / quality (push) Successful in 50s
CI (SIAX Cloud) / integration (push) Successful in 35s
CI (SIAX Cloud) / security (push) Successful in 21s
Security Scan (SIAX Cloud) / security (push) Successful in 26s
CI (SIAX Cloud) / quality (pull_request) Successful in 46s
CI (SIAX Cloud) / integration (pull_request) Successful in 35s
CI (SIAX Cloud) / security (pull_request) Successful in 19s
Security Scan (SIAX Cloud) / security (pull_request) Successful in 16s
AnalyticsSQLSession is a SEPARATE class, not a method on SiaxAnalyticsEngine.
executeMetric has no raw-SQL surface at all; on the same object the closed DSL
would be one refactor away from growing one. ClickHouseAnalyticsEngine
.executeQuery stays a documented refusal.
FOUR LAYERS, and the guard is deliberately called the weakest:
1 role — the CL0UD analyst role is checked when the SESSION IS OPENED, so
a caller without it never obtains an object that can run SQL.
2 guard — statement admission. Comments and literals removed with
ClickHouse's own lexical rules, ASCII-only outside literals, one
statement, SELECT/WITH only, and every table reference POSITIVELY
resolved against the tenant's catalog. Deny-lists are a second
opinion, never the mechanism.
3 grants — a per-tenant ClickHouse user with SELECT on that tenant's catalog
alone, plus a restrictive row policy on the base tables.
4 views — the tenant catalog holds only VIEWS, each baking
`WHERE tenant_id = '<tenant>'`, and tenant_id is not projected.
61 bypass attempts, in one shared corpus, run TWICE: against the guard, and
against a live ClickHouse 24.8.14.39 with LAYER 2 REMOVED. The live assertion
is not "the engine errored" — it is "no canary came back", with BETA_CANARY
seeded into the other tenant's rows and SECRET_CANARY into an ungranted table.
The counterfactual runs first: as admin the same base table returns 100 rows
and BETA_CANARY IS readable, so the isolation numbers mean something.
Result: 0 of 61 leaked. Evidence: evidence/bi-p017/analyst-sql-live-bypass.json.
FOUR DEFECTS FOUND BY WRITING TESTS THAT CAN FAIL — three of them mine:
* `CREATE ROW POLICY IF NOT EXISTS` is a SILENT NO-OP on re-provisioning.
Dropping the user empties the policy's apply-to list; the next run skips
re-creation, so the policy survives with `apply_to_list = []`, applying to
nobody, while `system.row_policies` still shows the correct select_filter.
The restricted user read 100 of 100 rows — BOTH TENANTS — from the base
table. Fixed with OR REPLACE + `rowPolicyVerificationSql`, which turns
"the policy is applied" from an assumption into a query, and a test that
asserts applied_to_user = 1 per object.
* `CREATE SETTINGS PROFILE IF NOT EXISTS` has the identical shape: a changed
budget never lands. A stale 60-row ceiling survived re-provisioning and
starved every legitimate statement, with the DDL reporting success
throughout. Also OR REPLACE now.
* `describe.skipIf(!reachable)` where `reachable` is set in `beforeAll`
skips ALWAYS — vitest evaluates skipIf at collection time. Ten tests
reported green while skipping against a live, reachable server. Replaced
with the repo's existing env-var gate, which FAILS when CLICKHOUSE_URL is
set and the server is down rather than degrading to a skip.
* MEASURED, and it changes the design: a ClickHouse user whose profile
carries `readonly = 1` cannot have ANY setting changed from the request
(Code: 164). The BI-P012 ceilings therefore cannot travel with an analyst
statement; they are baked into the user's SETTINGS PROFILE at provisioning.
Proven both ways: over-budget scan fails with code 158 from the profile
with no setting sent, and an attempt to raise it from the request fails
with code 164.
Honest about what the guard does NOT catch: `SELECT tenant_id FROM
t_alpha_sql.sales` is structurally legitimate and IS admitted. It fails at the
engine because the view does not project the column. Recorded in its own
GUARD_ADMITS_ENGINE_REFUSES list and asserted live, rather than deleted from
the corpus.
evidence/bi-p017 is pinned by tests/analytics/analyst-sql-evidence.test.ts,
which never skips: adding an attack without re-running the live suite breaks
the build, and the pin also checks the counterfactual numbers so the evidence
cannot be a recording of an empty database.
246 tests pass.
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
|
||
|
|
84bb9faa4f |
feat(analytics): BI-P002..BI-P012 analytics core — domain model, tiers, contracts, Arrow, ClickHouse engine, isolation, governance
CI (SIAX Cloud) / security (push) Successful in 41s
Security Scan (SIAX Cloud) / security (push) Successful in 40s
CI (SIAX Cloud) / integration (push) Successful in 52s
CI (SIAX Cloud) / quality (push) Successful in 1m5s
CI (SIAX Cloud) / security (pull_request) Successful in 31s
Security Scan (SIAX Cloud) / security (pull_request) Successful in 29s
CI (SIAX Cloud) / integration (pull_request) Successful in 45s
CI (SIAX Cloud) / quality (pull_request) Successful in 53s
Implements the eight BI program prompts that make 0pt the canonical SIAX
analytics capability. Every file is NEW: not one existing file is modified,
and src/analytics/clickhouse-analytics.ts is byte-identical to main.
BI-P002 — the nineteen canonical entities, verbatim, with tenant_id +
project_id on every tenant-owned record. sql/008 gives each one a table.
The four version entities BI-P002 names as immutable are enforced by BEFORE
UPDATE/DELETE triggers, not by a TypeScript `readonly` — proven by direct SQL
against a live Postgres.
BI-P003 — RAW/NORMALIZED/CURATED/SERVING with the read rule enforced:
assertReadableByBiSurface refuses a RAW dataset to a dashboard unless a
recorded exception exists. All eight required declarations are NOT NULL.
BI-P004 — DatasetContract and a fail-closed compatibility gate. Fifteen
breaking-change classes are each tested twice: refused under a minor bump,
accepted under a major one.
BI-P005 — Arrow IPC alongside JSON, with a hand-written FlatBuffers writer so
no npm dependency (or CVE surface) is added. Conformance comes from three
independent implementations, not from self-agreement: pyarrow 25.0.1 and
DuckDB 1.5.5 read identical values from both ClickHouse's Arrow and ours, and
ClickHouse's own Arrow C++ ingests our 20,000-row batch with exact aggregates.
BI-P009 — SiaxAnalyticsEngine with all eight capabilities. The existing
ClickHouseAnalytics is COMPOSED, never replaced. executeQuery (analyst SQL) is
declared and deliberately refused with CAPABILITY_UNAVAILABLE until BI-P017 and
BI-P033 land, rather than opening the raw-SQL hole BI-P011 enumerates.
BI-P010 — four purpose-built tables, not one giant one, and the difference is
measured against the counterfactual: 48.8ms/40,960 rows vs 174.4ms/5,400,000
rows for the same question; 5/660 granules vs 660/1149.
BI-P011 — layered isolation, each layer proven with the layer above removed.
Without the forced predicate the same statement returns 50,050 (both tenants);
with it, t_alpha gets 50. Without a row policy an unrestricted user sees 2
tenants; the restricted user running the IDENTICAL statement sees 1.
BI-P012 — every named limit, fired individually and identified by ClickHouse
error code, plus the half usually skipped: an unrelated tenant is timed while
a real CPU bomb is in flight and must still return the CORRECT answer.
Three defects found by writing tests that can fail:
* `CHECK (array_length(purposes,1) >= 1)` can NEVER fail — array_length is
NULL for '{}' and a NULL CHECK passes. Fixed with coalesce(); the test
that caught it is pinned.
* EXPLAIN ESTIMATE run under the caller's own max_rows_to_read fails with
code 158 exactly when the estimate would be over budget. Planner calls now
use a narrower envelope.
* result_overflow_mode=break returns HTTP 200 with 65,409 of 100,000 rows
and no indication it is short. Every SELECT ships `throw`.
Verified rather than assumed: server7's ClickHouse holds `opt_scraped` (51
tables, 407,793,471 rows, `persons`/`cz_owners` included) — the b0ts plane this
module is prohibited from touching — has no `opt_analytics` database, zero
tenant_id columns and zero row policies. Nothing there was modified; every
proof ran against disposable containers.
Evidence: evidence/bi-p002-p012/{test-transcript.txt,bi-p010-benchmark.json,
bi-p005-arrow-roundtrip.json,server7-clickhouse-probe.md}
Docs: docs/analytics/BI_ANALYTICS_CORE.md
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
|
||
|
|
c47e3b468e |
test(integration)+ci+docs: live OpenSearch/ClickHouse suites, CI service containers, STATUS.md (§56)
CI (SIAX Cloud) / integration (push) Failing after 22s
CI (SIAX Cloud) / security (push) Successful in 22s
Security Scan (SIAX Cloud) / security (push) Successful in 21s
CI (SIAX Cloud) / quality (push) Successful in 43s
CI (SIAX Cloud) / integration (pull_request) Failing after 3s
CI (SIAX Cloud) / security (pull_request) Successful in 17s
Security Scan (SIAX Cloud) / security (pull_request) Successful in 16s
CI (SIAX Cloud) / quality (pull_request) Successful in 43s
- tests/integration/: 10 tests against REAL OpenSearch 2.11 (tenant isolation, purpose/rights filters, generation rebuild/rollback, projection-disaster rebuild §68) + REAL ClickHouse 24.8 (tenant-isolated metric DSL, TTL schema, injection-surface refusals). Honest skip without env - and CI fails hard if the suites skip instead of running. - .gitea/workflows/ci.yml: new blocking 'integration' job with self-identifying service containers (st0re ci.yml's measured alias-collision discipline: run-unique cluster.name / credentials, getent-enumerated IP selection, no-match = hard fail). - STATUS.md: full 8-level Master-DoD classification per capability incl. the honest gaps blocking PRODUCTION_READY. - dependency-mode: NEXT_PHASE=phase-production-build is not a serving runtime (next build page-data collection must compile; guard re-arms at boot) - pinned by contract test. - docs/02-architecture.md: superseded the accurate-but-now-fixed 'every client runs against its mock' paragraph; .env.example: client URLs now actually read from env + new P0 backend vars. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com> |