diff --git a/docs/analytics/BI_QUERY_METRIC_QUALITY.md b/docs/analytics/BI_QUERY_METRIC_QUALITY.md index 3cb8a9c..07c8040 100644 --- a/docs/analytics/BI_QUERY_METRIC_QUALITY.md +++ b/docs/analytics/BI_QUERY_METRIC_QUALITY.md @@ -100,10 +100,15 @@ env-var gate, which **fails** when the variable is set and the server is down. - **DuckDB Arrow IPC** is unproven: the `arrow` extension is a network install the offline environment could not download. `readiness()` says so. - **BI-P021's budget is a bound, not a hard cap.** In-flight enforcement is - granular to the poll interval. Measured: a 1,000,000-row window cancelled the - query at **130,883,409 rows**, 92 ms in — a 130× overshoot. Still a bound - (unwatched, the same query read 6.5 billion rows in one second), but anyone - quoting it as a hard cap is wrong. + granular to the poll interval, so the query overshoots between polls — and on + a columnar engine the overshoot is large. In the committed run + (`evidence/bi-p019-p021/`) a 1,000,000-row window cancelled the query at + **153,318,696 rows**, 55 ms in: a **153×** overshoot. The figure is + machine-dependent and moves between runs (an earlier run on the same box + recorded 131 M / 130×), which is itself the point — the bound is "roughly one + poll interval of engine throughput", not a number. It is still a bound: the + same query, unwatched, read **4,227,318,261 rows** before cancellation in the + BI-P019 case. Anyone quoting it as a hard cap is wrong. - **The per-process window store does not bind across replicas.** `describeEnforcement()` reports `"with N replicas a tenant can spend up to N times this budget"`. A shared store (Valkey, or the CL0UD quota service) diff --git a/src/analytics/cost/query-budget.ts b/src/analytics/cost/query-budget.ts index 5d1a5e9..c4e9734 100644 --- a/src/analytics/cost/query-budget.ts +++ b/src/analytics/cost/query-budget.ts @@ -35,15 +35,19 @@ * In-flight enforcement is granular to the poll interval, so a query can * OVERSHOOT its budget between polls — and on a columnar engine the overshoot * is large. Measured against ClickHouse 24.8.14.39 with a 1,000,000-row window - * and a 100ms poll: the query was cancelled at 130,883,409 rows read, 92ms in. - * A 130x overshoot. See evidence/bi-p019-p021/cancellation-and-budget.json. + * and a 100ms poll: the query was cancelled at 153,318,696 rows read, 55ms in. + * A 153x overshoot. See evidence/bi-p019-p021/cancellation-and-budget.json, + * which is regenerated by the live suite — the figure is machine-dependent and + * moves between runs (an earlier run on the same box recorded 131M / 130x). + * Do not quote a specific multiple; quote the SHAPE. * - * That is still a bound — the same query, unwatched, read 6,559,476,156 rows - * in one second — but it is a bound of the form "within roughly one poll - * interval of engine throughput", not "at exactly the budget". Anyone sizing a - * budget must size it for that, and anyone quoting this as a hard cap is - * wrong. Shortening the interval narrows the overshoot and costs a - * `system.processes` read per poll per running query. + * That shape is still a bound — the same query, unwatched, read 4,227,318,261 + * rows before it was stopped in the BI-P019 case — but it is a bound of the + * form "within roughly one poll interval of engine throughput", not "at + * exactly the budget". Anyone sizing a budget must size it for that, and + * anyone quoting this as a hard cap is wrong. Shortening the interval narrows + * the overshoot and costs a `system.processes` read per poll per running + * query. */ import { SiaxError } from '../../platform/errors'; import {