Commit Graph
4 Commits
Author SHA1 Message Date
Claude Opus 5 c864c8c417 fix(analytics): revert 18 files reformatted by mistake, and actually apply the Iceberg fix a previous message claimed
CI (SIAX Cloud) / quality (push) Successful in 55s
CI (SIAX Cloud) / integration (push) Successful in 40s
CI (SIAX Cloud) / security (push) Successful in 21s
Security Scan (SIAX Cloud) / security (push) Successful in 22s
CI (SIAX Cloud) / integration (pull_request) Successful in 39s
CI (SIAX Cloud) / quality (pull_request) Successful in 54s
Security Scan (SIAX Cloud) / security (pull_request) Successful in 21s
CI (SIAX Cloud) / security (pull_request) Successful in 25s
Two corrections, both of my own errors.

1. SCOPE CREEP, NOW REVERTED. Running `prettier --write` on whole directories
   reformatted 18 files this branch had no business touching — the entire
   merged BI-P002..P012 core (arrow/, domain/, engine/, repositories/, five
   analytics integration suites) and, absurdly,
   search-opensearch.integration.test.ts, which has nothing to do with this
   work. The changes were purely stylistic quote churn, but they added ~1,400
   lines of noise to the diff, made the PR harder to review honestly, and
   invited conflicts with the four other lanes working in this estate. All 18
   are now byte-identical to origin/main again (verified with `git diff
   origin/main --name-only`, which reports nothing for them). Formatting is now
   applied only to the files this branch actually owns.

2. A COMMIT MESSAGE THAT DESCRIBED A FIX THAT WAS NOT IN THE CODE. Commit
   3ee0230 said the "does NOT ship the Iceberg table" assertion had been moved
   from a filesystem check to `git ls-files`. It had not. The patch anchor used
   double quotes, prettier had already converted that file to single quotes, so
   the replacement silently matched nothing and the script printed its success
   line anyway. The suite then passed — but only because I had just deleted the
   fixture directory, so the OLD filesystem assertion happened to be true.

   That is precisely the failure mode this branch keeps writing tests against:
   a green run that means something different from what it appears to mean. The
   fix is now genuinely applied and verified the way it should have been the
   first time — by running the suite WITH the Iceberg fixture present on disk,
   which is the case that previously failed. It passes, and it now also asserts
   that the parquet/csv fixtures ARE tracked, so a version shipping no fixture
   at all would fail rather than pass vacuously.

957 tests pass with ClickHouse, Postgres and DuckDB live. typecheck, eslint and
prettier clean on the files this branch owns.

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
2026-09-07 03:16:30 +02:00
Claude Opus 5 ba5c796b36 docs(analytics): anchor two security claims to the measurements behind them
CI (SIAX Cloud) / security (push) Successful in 30s
CI (SIAX Cloud) / integration (push) Successful in 56s
CI (SIAX Cloud) / quality (push) Successful in 1m23s
CI (SIAX Cloud) / integration (pull_request) Successful in 49s
CI (SIAX Cloud) / security (pull_request) Successful in 24s
CI (SIAX Cloud) / quality (pull_request) Successful in 1m18s
Security Scan (SIAX Cloud) / security (push) Successful in 23s
Security Scan (SIAX Cloud) / security (pull_request) Successful in 23s
Neither changes behaviour; both replace an assertion with a measurement.

BI-P014's escaping strategy rests on DuckDB treating a backslash as a literal
character, so that doubling `'` is sufficient. Verified on 1.5.5 rather than
assumed: `SELECT length('a\\b')` returns 4, and `SELECT 'a\''` is a parser
error ("unterminated quoted string") — which is exactly what a NON-escaping
backslash produces. In a dialect where `\'` closed the literal, doubling would
not be enough, so the claim is worth pinning to a run.

BI-P017's FROM/JOIN scan is conservative where it is unsure, and that has a
cost worth naming rather than discovering: ClickHouse's `ARRAY JOIN arr` takes
a COLUMN, not a table, and this guard refuses it. A false refusal an analyst
can see and report is the right side of that trade; a false ADMISSION is not.
Supporting it means teaching the scan which JOIN flavours take a non-table
target, which is the direction that grows holes — so it stays refused until
someone needs it and can add it WITH a corpus entry proving the narrowing did
not open anything else.

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
2026-09-07 03:11:19 +02:00
Claude Opus 5 3ee023017b fix(analytics): move the DuckDB live suite out of tests/integration, and add the BI docs
CI (SIAX Cloud) / security (push) Successful in 28s
CI (SIAX Cloud) / integration (push) Successful in 42s
Security Scan (SIAX Cloud) / security (push) Successful in 21s
CI (SIAX Cloud) / quality (push) Successful in 1m2s
CI (SIAX Cloud) / security (pull_request) Successful in 23s
CI (SIAX Cloud) / integration (pull_request) Successful in 40s
CI (SIAX Cloud) / quality (pull_request) Successful in 58s
Security Scan (SIAX Cloud) / security (pull_request) Successful in 20s
CI's integration job failed, and it failed CORRECTLY. That job provides a live
OpenSearch and a live ClickHouse and then asserts that nothing in
tests/integration/ skipped, on the stated principle that "a skipped suite must
never masquerade as a green integration gate". The DuckDB suite needs a Python
interpreter that job does not provide, so it skipped, and the guard fired — the
first suite in that directory ever to skip there.

Moving the file to tests/duckdb/ respects the guard rather than weakening it.
Loosening it to tolerate skips would have traded a real gate for one green job,
which is the trade this estate keeps regretting. The suite still runs, still
fails rather than skipping when DUCKDB_PYTHON is set and DuckDB is broken, and
is still pinned by the committed evidence.

Also fixed: "does NOT ship the Iceberg table" asserted against the FILESYSTEM
when the claim is about what is COMMITTED. The live suite rebuilds that table
locally, so the assertion failed whenever both suites ran in one pass — it was
testing the wrong thing rather than catching a real problem. It now asserts
against `git ls-files`, and also asserts the parquet/csv fixtures ARE tracked,
so a version that stopped shipping any fixture would fail too.

docs/analytics/BI_QUERY_METRIC_QUALITY.md is organised around what is PROVEN
and what is NOT, because the second list is the one that gets dropped. It
records the five findings and the six open items, including that BI-P029's
MetricFlow was never executed and that BI-P021's budget is a bound rather than
a hard cap (measured 130x overshoot at a 100ms poll).

It also reports, without changing it, a latent flaw in .gitea/workflows/ci.yml
(a protected file this branch does not touch): the integration guard's second
pattern `Tests  [0-9]+ passed` never matches, because vitest writes that
summary with ANSI colour codes between the words. The guard is therefore
STRICTER than it reads — "any skip fails" rather than "skipped and nothing
passed" — so nothing is currently unguarded, but the escape hatch does not
work and someone will eventually be puzzled by it.

957 tests pass with ClickHouse, Postgres and DuckDB all live.

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
2026-09-07 03:08:11 +02:00
Claude Opus 5 a9ae7f25a4 feat(analytics): BI-P014 — DuckDB provider, and the measured finding that its sandbox does not work the way it looks
CI (SIAX Cloud) / integration (push) Failing after 42s
CI (SIAX Cloud) / security (push) Successful in 24s
CI (SIAX Cloud) / quality (push) Successful in 1m6s
Security Scan (SIAX Cloud) / security (push) Successful in 22s
CI (SIAX Cloud) / security (pull_request) Successful in 23s
Security Scan (SIAX Cloud) / security (pull_request) Successful in 23s
CI (SIAX Cloud) / integration (pull_request) Failing after 43s
CI (SIAX Cloud) / quality (pull_request) Successful in 57s
BI-P014's security requirement is the whole prompt: DuckDB is a local engine
with `read_csv('/etc/passwd')` built in. Two layers.

LAYER 1, the mount registry: a caller names ARTIFACT IDS, never paths. The
registry resolves an id, checks the artifact belongs to the caller's tenant,
and realpath()s the result to confirm it is still inside the mount root once
symlinks are followed. A symlink to /etc/passwd, a traversing relative path, an
absolute path, and a sibling directory sharing the root's prefix
(/mnt/artifacts-evil for root /mnt/artifacts) are each refused. A cross-tenant
read is AUTHORIZATION_DENIED, deliberately not NOT_FOUND — an operator reading
the audit trail needs to see an attempted cross-tenant read as one.

LAYER 2, THE FINDING:

    `SET allowed_directories=[...]` ALONE IS A NO-OP.

Measured on DuckDB 1.5.5. With allowed_directories set to a single mount
directory and nothing else, `read_csv('/etc/passwd')` RETURNED THE FILE and an
https:// URL was fetched over the network. Only when `enable_external_access
= false` is ALSO set does the allow-list bind; the same reads then fail with
"Permission Error: Cannot access file". `lock_configuration=true` afterwards
stops a query re-opening either.

So the obvious spelling of this control — set the allow-list and move on —
yields a setting that appears in duckdb_settings(), looks like a sandbox, and
permits everything. That is `trivy --exit-code 0` in a different costume. The
bootstrap is therefore emitted as ONE unit, `assertSessionSandboxed` refuses a
session missing any part of it, and the measurement itself is PINNED by a test
that never skips: if someone simplifies the bootstrap, the pinned evidence
(`read_etc_passwd_succeeded: true` for the naive spelling) keeps the finding
attached to the build rather than to a comment that would rot.

Twelve bypass attempts run against a real DuckDB with LAYER 1 REMOVED — sent
straight to the engine, registry not in the picture. Zero succeeded. The
counterfactual runs in the same session and is asserted: the mounted file IS
readable, so "nothing succeeded" is not a sandbox that blocks everything, which
would be an outage rather than a control. A locked session is then shown unable
to re-open its own sandbox, all three ways.

DoD, against real fixtures: Parquet aggregates correctly (SE 394.5, NO 105.5),
CSV round-trips a value containing a quote, and an ICEBERG SNAPSHOT reads via
iceberg_scan on an exact metadata JSON — which is not a workaround: DuckDB
refuses to glob for the latest version because globbing can read uncommitted
data, and an exact metadata file is what "snapshot" means. Arrow is reported
UNPROVEN by `readiness()` with the reason: DuckDB's arrow extension is a
network install the offline environment could not download.

THREE DEFECTS IN MY OWN TESTS, found and fixed rather than tidied away:
  * a bypass-corpus entry was `SELECT 1 FROM (SELECT 1) t` — not an attack at
    all. It succeeded and was reported as a leak. A corpus entry that cannot
    fail is as useless as a check that cannot. Replaced with real ATTACH and
    COPY attempts.
  * the harness ran the session bootstrap outside its try block, so a statement
    that correctly REFUSED to re-open a locked sandbox killed the subprocess
    and surfaced as an unreadable "Command failed" instead of as the meaningful
    refusal it was.
  * the Iceberg fallback was `expect(evidence.iceberg ?? "unbuilt").toBeDefined()`
    — a check that can never fail. It let a run without the fixture ERASE the
    recorded Iceberg result from the evidence file while both suites stayed
    green. It now rebuilds the fixture when pyiceberg is available and, when it
    is not, asserts against the previously recorded result and marks it as
    carried forward.

The Iceberg fixture is deliberately NOT committed: pyiceberg writes absolute
paths into its metadata and Avro manifests, so a committed table resolves only
on the machine that built it, and a fixture that silently fails everywhere else
is worse than none. tests/fixtures/duckdb/README.md says so and gives the
rebuild command; the recorded result is the artefact instead.

Heavy artifacts are refused with CAPABILITY_UNAVAILABLE and pointed at N0D,
per BI-P014 — silently running a 40 GB scan in the API process is how a preview
engine takes down a control plane.

35 unit tests, 12 live tests. Evidence: evidence/bi-p014/duckdb-sandbox.json.

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
2026-09-07 02:54:53 +02:00