{ "prompt": "BI-P103", "title": "First measured BI SLO baselines", "recorded_at": "2026-09-07T06:26:43.457Z", "host_note": "One idle host, local ClickHouse, warm page cache, single-threaded client. A floor for the shape of the distribution on this fixture — not a production SLA.", "clickhouse_version": "26.3.10.60", "fixture": { "rows": 400000, "table": "slo_fact", "partitions": "MergeTree ORDER BY (tenant_id, bucket, ts)" }, "workload": { "cold_dashboard_queries": 120, "cached_dashboard_queries": 120, "exploration_queries": 220, "report_queries": 0 }, "window": { "from": "2026-09-07T06:26:39.422Z", "to": "2026-09-07T06:26:43.447Z" }, "catalogue_size": 11, "fully_measured": false, "unresolved_queries": 0, "summary": { "met": 7, "breached": 0, "insufficient_data": 1, "not_measurable_here": 3, "measured_fraction": 0.6363636363636364 }, "results": [ { "objective_id": "dash.availability", "workload": "interactive_dashboard", "kind": "availability", "status": "met", "target": 0.995, "unit": "ratio", "observed": 1, "sample_size": 240, "low_confidence": false, "detail": "0 failed of 240 terminal queries" }, { "objective_id": "dash.p95_first_query", "workload": "interactive_dashboard", "kind": "latency_p95_first", "status": "met", "target": 3000, "unit": "ms", "observed": 8.893520999999964, "sample_size": 120, "low_confidence": false, "detail": "p95 over 120 samples" }, { "objective_id": "dash.p95_cached", "workload": "interactive_dashboard", "kind": "latency_p95_cached", "status": "met", "target": 400, "unit": "ms", "observed": 0.024498999999877924, "sample_size": 120, "low_confidence": false, "detail": "p95 over 120 samples" }, { "objective_id": "dash.error_rate", "workload": "interactive_dashboard", "kind": "error_rate", "status": "met", "target": 0.005, "unit": "ratio", "observed": 0, "sample_size": 240, "low_confidence": false, "detail": "0 failed of 240 terminal queries" }, { "objective_id": "dash.freshness", "workload": "interactive_dashboard", "kind": "freshness", "status": "not_measurable_here", "target": 900, "unit": "seconds", "sample_size": 0, "low_confidence": false, "detail": "measured from analytics_freshness_watermark; BI usage events cannot answer it — Data age is a property of the DATASET (BI-P026 freshness, BI-P105 watermark), not of a usage event. Usage events cannot measure it and this evaluator does not pretend to." }, { "objective_id": "explore.p95", "workload": "exploration", "kind": "latency_p95", "status": "met", "target": 10000, "unit": "ms", "observed": 17.072708000000148, "sample_size": 220, "low_confidence": false, "detail": "p95 over 220 samples" }, { "objective_id": "explore.p99", "workload": "exploration", "kind": "latency_p99", "status": "met", "target": 60000, "unit": "ms", "observed": 21.640736000000288, "sample_size": 220, "low_confidence": false, "detail": "p99 over 220 samples" }, { "objective_id": "explore.max_result", "workload": "exploration", "kind": "max_result_rows", "status": "met", "target": 1000000, "unit": "rows", "observed": 96, "sample_size": 220, "low_confidence": false, "detail": "largest result seen in the window" }, { "objective_id": "explore.timeout", "workload": "exploration", "kind": "timeout", "status": "not_measurable_here", "target": 300, "unit": "seconds", "sample_size": 0, "low_confidence": false, "detail": "measured from engine_configuration; BI usage events cannot answer it — The configured ceiling (BI-P012 settings profile), not an observation." }, { "objective_id": "report.completion_sla", "workload": "report", "kind": "completion_sla", "status": "insufficient_data", "target": 900000, "unit": "ms", "sample_size": 0, "low_confidence": false, "detail": "no report_generated events" }, { "objective_id": "report.delivery_sla", "workload": "report", "kind": "delivery_sla", "status": "not_measurable_here", "target": 300000, "unit": "ms", "sample_size": 0, "low_confidence": false, "detail": "measured from n0tify_delivery_receipt; BI usage events cannot answer it — Produced is not delivered. aud0 has no delivery event; the receipt is N0TIFY's, so this objective is declared and reported unmeasured rather than proxied by completion." } ], "measured_but_narrower_than_production": [ "dash.p95_cached is the IN-PROCESS reference cache (InMemoryResultCache). A Valkey deployment adds a network round trip, so 0.03 ms here is a floor and not the number a user will see.", "Every latency is the ENGINE round trip only: no HTTP layer, no CL0UD authorization, no serialization to the caller. The end-to-end p95 is larger by whatever those cost.", "One tenant, one client, no concurrency. Contention is not represented." ], "not_measured_and_why": [ "dash.freshness — a property of the DATASET (BI-P026 watermark), not of a usage event", "explore.timeout — the configured BI-P012 ceiling, not an observation", "report.completion_sla — nothing in 0pt emits aud0.report_generated yet, so the sample is 0", "report.delivery_sla — delivery is N0TIFY's receipt; aud0 has no delivery event" ] }