Commit Graph
4 Commits
Author SHA1 Message Date
Claude Opus 5 c864c8c417 fix(analytics): revert 18 files reformatted by mistake, and actually apply the Iceberg fix a previous message claimed
CI (SIAX Cloud) / quality (push) Successful in 55s
CI (SIAX Cloud) / integration (push) Successful in 40s
CI (SIAX Cloud) / security (push) Successful in 21s
Security Scan (SIAX Cloud) / security (push) Successful in 22s
CI (SIAX Cloud) / integration (pull_request) Successful in 39s
CI (SIAX Cloud) / quality (pull_request) Successful in 54s
Security Scan (SIAX Cloud) / security (pull_request) Successful in 21s
CI (SIAX Cloud) / security (pull_request) Successful in 25s
Two corrections, both of my own errors.

1. SCOPE CREEP, NOW REVERTED. Running `prettier --write` on whole directories
   reformatted 18 files this branch had no business touching — the entire
   merged BI-P002..P012 core (arrow/, domain/, engine/, repositories/, five
   analytics integration suites) and, absurdly,
   search-opensearch.integration.test.ts, which has nothing to do with this
   work. The changes were purely stylistic quote churn, but they added ~1,400
   lines of noise to the diff, made the PR harder to review honestly, and
   invited conflicts with the four other lanes working in this estate. All 18
   are now byte-identical to origin/main again (verified with `git diff
   origin/main --name-only`, which reports nothing for them). Formatting is now
   applied only to the files this branch actually owns.

2. A COMMIT MESSAGE THAT DESCRIBED A FIX THAT WAS NOT IN THE CODE. Commit
   3ee0230 said the "does NOT ship the Iceberg table" assertion had been moved
   from a filesystem check to `git ls-files`. It had not. The patch anchor used
   double quotes, prettier had already converted that file to single quotes, so
   the replacement silently matched nothing and the script printed its success
   line anyway. The suite then passed — but only because I had just deleted the
   fixture directory, so the OLD filesystem assertion happened to be true.

   That is precisely the failure mode this branch keeps writing tests against:
   a green run that means something different from what it appears to mean. The
   fix is now genuinely applied and verified the way it should have been the
   first time — by running the suite WITH the Iceberg fixture present on disk,
   which is the case that previously failed. It passes, and it now also asserts
   that the parquet/csv fixtures ARE tracked, so a version shipping no fixture
   at all would fail rather than pass vacuously.

957 tests pass with ClickHouse, Postgres and DuckDB live. typecheck, eslint and
prettier clean on the files this branch owns.

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
2026-09-07 03:16:30 +02:00
Claude Opus 5 3ee023017b fix(analytics): move the DuckDB live suite out of tests/integration, and add the BI docs
CI (SIAX Cloud) / security (push) Successful in 28s
CI (SIAX Cloud) / integration (push) Successful in 42s
Security Scan (SIAX Cloud) / security (push) Successful in 21s
CI (SIAX Cloud) / quality (push) Successful in 1m2s
CI (SIAX Cloud) / security (pull_request) Successful in 23s
CI (SIAX Cloud) / integration (pull_request) Successful in 40s
CI (SIAX Cloud) / quality (pull_request) Successful in 58s
Security Scan (SIAX Cloud) / security (pull_request) Successful in 20s
CI's integration job failed, and it failed CORRECTLY. That job provides a live
OpenSearch and a live ClickHouse and then asserts that nothing in
tests/integration/ skipped, on the stated principle that "a skipped suite must
never masquerade as a green integration gate". The DuckDB suite needs a Python
interpreter that job does not provide, so it skipped, and the guard fired — the
first suite in that directory ever to skip there.

Moving the file to tests/duckdb/ respects the guard rather than weakening it.
Loosening it to tolerate skips would have traded a real gate for one green job,
which is the trade this estate keeps regretting. The suite still runs, still
fails rather than skipping when DUCKDB_PYTHON is set and DuckDB is broken, and
is still pinned by the committed evidence.

Also fixed: "does NOT ship the Iceberg table" asserted against the FILESYSTEM
when the claim is about what is COMMITTED. The live suite rebuilds that table
locally, so the assertion failed whenever both suites ran in one pass — it was
testing the wrong thing rather than catching a real problem. It now asserts
against `git ls-files`, and also asserts the parquet/csv fixtures ARE tracked,
so a version that stopped shipping any fixture would fail too.

docs/analytics/BI_QUERY_METRIC_QUALITY.md is organised around what is PROVEN
and what is NOT, because the second list is the one that gets dropped. It
records the five findings and the six open items, including that BI-P029's
MetricFlow was never executed and that BI-P021's budget is a bound rather than
a hard cap (measured 130x overshoot at a 100ms poll).

It also reports, without changing it, a latent flaw in .gitea/workflows/ci.yml
(a protected file this branch does not touch): the integration guard's second
pattern `Tests  [0-9]+ passed` never matches, because vitest writes that
summary with ANSI colour codes between the words. The guard is therefore
STRICTER than it reads — "any skip fails" rather than "skipped and nothing
passed" — so nothing is currently unguarded, but the escape hatch does not
work and someone will eventually be puzzled by it.

957 tests pass with ClickHouse, Postgres and DuckDB all live.

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
2026-09-07 03:08:11 +02:00
Claude Opus 5 adc7a9be31 fix(analytics): replace the control-character class with a code-point scan (no-control-regex)
CI (SIAX Cloud) / quality (push) Successful in 45s
CI (SIAX Cloud) / integration (push) Successful in 34s
CI (SIAX Cloud) / security (push) Successful in 19s
Security Scan (SIAX Cloud) / security (push) Successful in 17s
CI (SIAX Cloud) / quality (pull_request) Successful in 45s
Security Scan (SIAX Cloud) / security (pull_request) Successful in 27s
CI (SIAX Cloud) / security (pull_request) Successful in 37s
CI (SIAX Cloud) / integration (pull_request) Successful in 45s
The ASCII-only check used a regex character class of control characters. It
tripped eslint's no-control-regex, which the repo's quality gate enforces —
and the rule has a point: such a class is unreadable and is exactly the kind
of expression a reviewer nods at without checking. Same behaviour, written as
an explicit scan naming the permitted set.

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
2026-09-07 02:19:10 +02:00
Claude Opus 5 d9c8f7190b feat(analytics): BI-P017 — analyst SQL contract, proven against live ClickHouse with the guard removed
CI (SIAX Cloud) / quality (push) Successful in 50s
CI (SIAX Cloud) / integration (push) Successful in 35s
CI (SIAX Cloud) / security (push) Successful in 21s
Security Scan (SIAX Cloud) / security (push) Successful in 26s
CI (SIAX Cloud) / quality (pull_request) Successful in 46s
CI (SIAX Cloud) / integration (pull_request) Successful in 35s
CI (SIAX Cloud) / security (pull_request) Successful in 19s
Security Scan (SIAX Cloud) / security (pull_request) Successful in 16s
AnalyticsSQLSession is a SEPARATE class, not a method on SiaxAnalyticsEngine.
executeMetric has no raw-SQL surface at all; on the same object the closed DSL
would be one refactor away from growing one. ClickHouseAnalyticsEngine
.executeQuery stays a documented refusal.

FOUR LAYERS, and the guard is deliberately called the weakest:
  1 role   — the CL0UD analyst role is checked when the SESSION IS OPENED, so
             a caller without it never obtains an object that can run SQL.
  2 guard  — statement admission. Comments and literals removed with
             ClickHouse's own lexical rules, ASCII-only outside literals, one
             statement, SELECT/WITH only, and every table reference POSITIVELY
             resolved against the tenant's catalog. Deny-lists are a second
             opinion, never the mechanism.
  3 grants — a per-tenant ClickHouse user with SELECT on that tenant's catalog
             alone, plus a restrictive row policy on the base tables.
  4 views  — the tenant catalog holds only VIEWS, each baking
             `WHERE tenant_id = '<tenant>'`, and tenant_id is not projected.

61 bypass attempts, in one shared corpus, run TWICE: against the guard, and
against a live ClickHouse 24.8.14.39 with LAYER 2 REMOVED. The live assertion
is not "the engine errored" — it is "no canary came back", with BETA_CANARY
seeded into the other tenant's rows and SECRET_CANARY into an ungranted table.
The counterfactual runs first: as admin the same base table returns 100 rows
and BETA_CANARY IS readable, so the isolation numbers mean something.
Result: 0 of 61 leaked. Evidence: evidence/bi-p017/analyst-sql-live-bypass.json.

FOUR DEFECTS FOUND BY WRITING TESTS THAT CAN FAIL — three of them mine:

  * `CREATE ROW POLICY IF NOT EXISTS` is a SILENT NO-OP on re-provisioning.
    Dropping the user empties the policy's apply-to list; the next run skips
    re-creation, so the policy survives with `apply_to_list = []`, applying to
    nobody, while `system.row_policies` still shows the correct select_filter.
    The restricted user read 100 of 100 rows — BOTH TENANTS — from the base
    table. Fixed with OR REPLACE + `rowPolicyVerificationSql`, which turns
    "the policy is applied" from an assumption into a query, and a test that
    asserts applied_to_user = 1 per object.

  * `CREATE SETTINGS PROFILE IF NOT EXISTS` has the identical shape: a changed
    budget never lands. A stale 60-row ceiling survived re-provisioning and
    starved every legitimate statement, with the DDL reporting success
    throughout. Also OR REPLACE now.

  * `describe.skipIf(!reachable)` where `reachable` is set in `beforeAll`
    skips ALWAYS — vitest evaluates skipIf at collection time. Ten tests
    reported green while skipping against a live, reachable server. Replaced
    with the repo's existing env-var gate, which FAILS when CLICKHOUSE_URL is
    set and the server is down rather than degrading to a skip.

  * MEASURED, and it changes the design: a ClickHouse user whose profile
    carries `readonly = 1` cannot have ANY setting changed from the request
    (Code: 164). The BI-P012 ceilings therefore cannot travel with an analyst
    statement; they are baked into the user's SETTINGS PROFILE at provisioning.
    Proven both ways: over-budget scan fails with code 158 from the profile
    with no setting sent, and an attempt to raise it from the request fails
    with code 164.

Honest about what the guard does NOT catch: `SELECT tenant_id FROM
t_alpha_sql.sales` is structurally legitimate and IS admitted. It fails at the
engine because the view does not project the column. Recorded in its own
GUARD_ADMITS_ENGINE_REFUSES list and asserted live, rather than deleted from
the corpus.

evidence/bi-p017 is pinned by tests/analytics/analyst-sql-evidence.test.ts,
which never skips: adding an attack without re-running the live suite breaks
the build, and the pin also checks the counterfactual numbers so the evidence
cannot be a recording of an empty database.

246 tests pass.

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
2026-09-07 02:17:07 +02:00