feat(analytics): BI-P105 — a pre-aggregate serves only while it is provably undrifted
CI (SIAX Cloud) / security (push) Successful in 49s
Security Scan (SIAX Cloud) / security (push) Successful in 50s
CI (SIAX Cloud) / integration (push) Successful in 55s
CI (SIAX Cloud) / quality (push) Successful in 1m20s
CI (SIAX Cloud) / quality (pull_request) Successful in 1m21s
CI (SIAX Cloud) / security (pull_request) Successful in 42s
Security Scan (SIAX Cloud) / security (pull_request) Successful in 37s
CI (SIAX Cloud) / integration (pull_request) Successful in 1m10s
CI (SIAX Cloud) / security (push) Successful in 49s
Security Scan (SIAX Cloud) / security (push) Successful in 50s
CI (SIAX Cloud) / integration (push) Successful in 55s
CI (SIAX Cloud) / quality (push) Successful in 1m20s
CI (SIAX Cloud) / quality (pull_request) Successful in 1m21s
CI (SIAX Cloud) / security (pull_request) Successful in 42s
Security Scan (SIAX Cloud) / security (pull_request) Successful in 37s
CI (SIAX Cloud) / integration (pull_request) Successful in 1m10s
A slow query is a complaint; a drifted pre-aggregate answers instantly, with
confidence, and is wrong. So the router is fail-closed on eight named reasons,
and the one people leave out — drift_never_verified — is the one that matters
on launch day: a brand-new aggregate is exactly as unproven as a drifted one.
Proved against ClickHouse 26.3.10.60, 2M rows, a real materialized view
(evidence/bi-p105/materialization-drift.json):
after backfill match, 48 buckets, 801 571 002 both sides
GDPR erasure ALTER TABLE ... DELETE on the source. An MV is an INSERT
trigger and never sees a mutation, so the aggregate kept
the deleted revenue: it would have reported 801 571 002
against a true 761 580 554 — 39 990 448 overstated,
5.251%, with NOTHING stale. Staleness at the moment of
the decision: 1 second.
guarded router base_table (state_not_serving)
counterfactual a staleness-only router SERVES it — measured, not assumed
after rebuild match, and routing returns to the materialization
Drift detection treats a MISSING bucket as drift even when every value present
is correct (late-arriving rows are the sneakiest failure), and an empty
comparison as insufficient_data rather than match — the same false green
BI-P025 found in a quality check over an empty table.
The measured speed-up is reported as a distribution, not a figure: p50 3.4x,
but the run-to-run range spans 0.6x-7.5x on one idle host, so quoting "3.4x"
as a property of the system would be misleading (BI-P104 doctrine).
19 unit tests + 4 against live ClickHouse.
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
This commit is contained in:
@@ -0,0 +1,170 @@
|
||||
/**
|
||||
* src/analytics/materialization/drift-detector.ts
|
||||
* ================================================
|
||||
* BI-P105 — is the pre-aggregate still telling the truth?
|
||||
*
|
||||
* The comparison is bucket-by-bucket between the aggregate and a recomputation
|
||||
* from the source over the same window. Three things make it a real detector
|
||||
* rather than a ceremony:
|
||||
*
|
||||
* 1. A MISSING BUCKET IS DRIFT. The nastiest pre-aggregate failure is not a
|
||||
* wrong value, it is an absent row: late-arriving data, a materialized view
|
||||
* that was created after the backfill, a refresh that failed silently. Every
|
||||
* value it DOES hold is correct, so a value-only comparison passes while the
|
||||
* total is wrong. Set difference is checked first, before any arithmetic.
|
||||
*
|
||||
* 2. ZERO BUCKETS IS `insufficient_data`, NEVER `match`. A comparison over an
|
||||
* empty window finds no discrepancies — the same false green BI-P025 found
|
||||
* in a quality check over an empty table. The router treats
|
||||
* `insufficient_data` as "do not serve", not as "fine".
|
||||
*
|
||||
* 3. RELATIVE AND ABSOLUTE TOLERANCE BOTH HAVE TO HOLD. A 0.1% tolerance on a
|
||||
* revenue bucket of 40M is 40 000 kr of invisible error; an absolute-only
|
||||
* tolerance flags every rounding difference on a large bucket. Requiring
|
||||
* both means a discrepancy has to be small in kronor AND small in
|
||||
* proportion.
|
||||
*/
|
||||
|
||||
import type { DriftTolerance, DriftVerdict, MaterializationStatus } from './materialization';
|
||||
|
||||
/** One aggregated row, keyed by its dimension tuple. */
|
||||
export interface AggregateBucket {
|
||||
/** Stable key for the dimension tuple, e.g. "2026-09-01|SE". */
|
||||
key: string;
|
||||
/** metric name -> value. */
|
||||
values: Record<string, number>;
|
||||
}
|
||||
|
||||
export interface DriftDiscrepancy {
|
||||
key: string;
|
||||
metric: string;
|
||||
source_value: number;
|
||||
aggregate_value: number;
|
||||
absolute_delta: number;
|
||||
/** |delta| / |source|, or Infinity when the source value is 0 and the aggregate is not. */
|
||||
relative_delta: number;
|
||||
}
|
||||
|
||||
export interface DriftReport {
|
||||
verdict: DriftVerdict;
|
||||
checked_at: string;
|
||||
buckets_compared: number;
|
||||
/** In the source recomputation, absent from the aggregate. Always drift. */
|
||||
missing_in_aggregate: string[];
|
||||
/** In the aggregate, absent from the source. Always drift (deleted rows). */
|
||||
extra_in_aggregate: string[];
|
||||
discrepancies: DriftDiscrepancy[];
|
||||
worst?: DriftDiscrepancy;
|
||||
tolerance: DriftTolerance;
|
||||
/**
|
||||
* What a caller would have been told if it had trusted the aggregate: the
|
||||
* total the aggregate reports vs the total the source computes, per metric.
|
||||
* This is the number that makes a drift report actionable — "0.4% off" means
|
||||
* nothing until it is "1 284 500 kr reported, 1 271 900 kr true".
|
||||
*/
|
||||
totals: Record<string, { source: number; aggregate: number; delta: number }>;
|
||||
}
|
||||
|
||||
const MAX_REPORTED_DISCREPANCIES = 50;
|
||||
|
||||
function relative(sourceValue: number, delta: number): number {
|
||||
if (sourceValue === 0) return delta === 0 ? 0 : Number.POSITIVE_INFINITY;
|
||||
return Math.abs(delta / sourceValue);
|
||||
}
|
||||
|
||||
/**
|
||||
* Compare a recomputation from source against the pre-aggregate.
|
||||
*
|
||||
* `fromSource` is the truth. Both sides must cover the same window; the caller
|
||||
* is responsible for that, and `missing_in_aggregate` will catch it loudly if
|
||||
* they do not.
|
||||
*/
|
||||
export function detectDrift(args: {
|
||||
fromSource: AggregateBucket[];
|
||||
fromAggregate: AggregateBucket[];
|
||||
tolerance: DriftTolerance;
|
||||
metrics: string[];
|
||||
now?: Date;
|
||||
}): DriftReport {
|
||||
const { fromSource, fromAggregate, tolerance, metrics } = args;
|
||||
const checked_at = (args.now ?? new Date()).toISOString();
|
||||
const sourceByKey = new Map(fromSource.map((b) => [b.key, b]));
|
||||
const aggByKey = new Map(fromAggregate.map((b) => [b.key, b]));
|
||||
|
||||
const missing_in_aggregate = [...sourceByKey.keys()].filter((k) => !aggByKey.has(k)).sort();
|
||||
const extra_in_aggregate = [...aggByKey.keys()].filter((k) => !sourceByKey.has(k)).sort();
|
||||
|
||||
const discrepancies: DriftDiscrepancy[] = [];
|
||||
let buckets_compared = 0;
|
||||
const totals: DriftReport['totals'] = {};
|
||||
for (const m of metrics) totals[m] = { source: 0, aggregate: 0, delta: 0 };
|
||||
|
||||
for (const [key, sourceBucket] of sourceByKey) {
|
||||
for (const m of metrics) totals[m].source += sourceBucket.values[m] ?? 0;
|
||||
const aggBucket = aggByKey.get(key);
|
||||
if (!aggBucket) continue;
|
||||
buckets_compared += 1;
|
||||
for (const m of metrics) {
|
||||
const sv = sourceBucket.values[m] ?? 0;
|
||||
const av = aggBucket.values[m] ?? 0;
|
||||
const delta = av - sv;
|
||||
const rel = relative(sv, delta);
|
||||
if (Math.abs(delta) > tolerance.absolute || rel > tolerance.relative) {
|
||||
discrepancies.push({
|
||||
key,
|
||||
metric: m,
|
||||
source_value: sv,
|
||||
aggregate_value: av,
|
||||
absolute_delta: delta,
|
||||
relative_delta: rel,
|
||||
});
|
||||
}
|
||||
}
|
||||
}
|
||||
for (const bucket of aggByKey.values()) {
|
||||
for (const m of metrics) totals[m].aggregate += bucket.values[m] ?? 0;
|
||||
}
|
||||
for (const m of metrics) totals[m].delta = totals[m].aggregate - totals[m].source;
|
||||
|
||||
discrepancies.sort((a, b) => Math.abs(b.absolute_delta) - Math.abs(a.absolute_delta));
|
||||
|
||||
// A comparison that compared nothing has not proved anything. This is the
|
||||
// BI-P025 false green, restated for pre-aggregates.
|
||||
const verdict: DriftVerdict =
|
||||
buckets_compared === 0 && missing_in_aggregate.length === 0 && extra_in_aggregate.length === 0
|
||||
? 'insufficient_data'
|
||||
: discrepancies.length > 0 || missing_in_aggregate.length > 0 || extra_in_aggregate.length > 0
|
||||
? 'drifted'
|
||||
: 'match';
|
||||
|
||||
const report: DriftReport = {
|
||||
verdict,
|
||||
checked_at,
|
||||
buckets_compared,
|
||||
missing_in_aggregate,
|
||||
extra_in_aggregate,
|
||||
discrepancies: discrepancies.slice(0, MAX_REPORTED_DISCREPANCIES),
|
||||
tolerance,
|
||||
totals,
|
||||
};
|
||||
if (discrepancies.length > 0) report.worst = discrepancies[0];
|
||||
return report;
|
||||
}
|
||||
|
||||
/**
|
||||
* Fold a drift report into the runtime status. A drifted aggregate is taken
|
||||
* OUT of service here rather than merely annotated: leaving it `serving` with
|
||||
* a bad verdict relies on every future reader remembering to check the
|
||||
* verdict, and one that forgets serves the wrong number.
|
||||
*
|
||||
* `insufficient_data` also stops service. It means the check could not prove
|
||||
* anything, and an unproven aggregate is not a current one.
|
||||
*/
|
||||
export function applyDriftReport(status: MaterializationStatus, report: DriftReport): MaterializationStatus {
|
||||
return {
|
||||
...status,
|
||||
state: report.verdict === 'match' ? status.state : 'drifted',
|
||||
last_drift_check_at: report.checked_at,
|
||||
last_drift_verdict: report.verdict,
|
||||
};
|
||||
}
|
||||
@@ -0,0 +1,309 @@
|
||||
/**
|
||||
* src/analytics/materialization/materialization.ts
|
||||
* =================================================
|
||||
* BI-P105 — MATERIALIZATION / PRE-AGGREGATION.
|
||||
*
|
||||
* A pre-aggregate is the one optimisation that fails SILENTLY. A slow query is
|
||||
* a complaint; a pre-aggregate that has drifted from its source answers
|
||||
* instantly, with confidence, and is wrong — no error, no timeout, nothing in
|
||||
* a log. So this module is built around one rule:
|
||||
*
|
||||
* A MATERIALIZATION SERVES ONLY WHILE IT IS PROVABLY CURRENT AND PROVABLY
|
||||
* UNDRIFTED. Every unknown routes to the source table.
|
||||
*
|
||||
* Six refusal reasons, each of which is a real way the aggregate becomes a
|
||||
* confident liar:
|
||||
*
|
||||
* coverage_miss it does not carry the metric/dimension/grain
|
||||
* asked for (a filter on a dimension the
|
||||
* aggregate collapsed cannot be applied after
|
||||
* the fact, and applying it anyway is how a
|
||||
* pre-aggregate returns another slice's number)
|
||||
* watermark_unknown nobody knows how current it is. BI-P026 already
|
||||
* refuses to render UNKNOWN age as current; the
|
||||
* same doctrine applies here.
|
||||
* staleness_exceeded it is older than its own freshness contract
|
||||
* window_beyond_watermark the question reaches past the newest data the
|
||||
* aggregate contains — it would silently answer
|
||||
* with a shorter period
|
||||
* drift_never_verified it has never been compared to its source
|
||||
* drift_verification_stale the last comparison is older than the interval
|
||||
* the spec requires
|
||||
* drift_detected the last comparison FAILED
|
||||
*
|
||||
* `drift_never_verified` is the one people leave out, and it is the one that
|
||||
* matters most on day one: a brand-new materialization is exactly as unproven
|
||||
* as a drifted one, and a router that defaults new aggregates to "serving"
|
||||
* ships the bug on the day the feature launches.
|
||||
*
|
||||
* The DoD also asks for measured improvement and documented freshness
|
||||
* semantics: scripts/bi-p105-materialization-drift.mjs measures both against a
|
||||
* live ClickHouse and writes evidence/bi-p105/materialization-drift.json,
|
||||
* including what happens WITHOUT this router (the drifted aggregate served).
|
||||
*/
|
||||
import { SiaxError } from '../../platform/errors';
|
||||
import type { AnalyticsQueryRequest, TimeGrain } from '../contract/query-request';
|
||||
|
||||
export const MATERIALIZATION_STRATEGIES = [
|
||||
'clickhouse_materialized_view',
|
||||
'clickhouse_projection',
|
||||
'dbt_incremental',
|
||||
'cube_preaggregation',
|
||||
] as const;
|
||||
export type MaterializationStrategy = (typeof MATERIALIZATION_STRATEGIES)[number];
|
||||
|
||||
export const MATERIALIZATION_STATES = ['building', 'serving', 'drifted', 'disabled'] as const;
|
||||
export type MaterializationState = (typeof MATERIALIZATION_STATES)[number];
|
||||
|
||||
/**
|
||||
* Grains ordered coarsest-last. An aggregate at `day` can answer `day` and
|
||||
* anything coarser by re-aggregating; it can NEVER answer `hour`, because the
|
||||
* detail was thrown away when the rows were rolled up.
|
||||
*/
|
||||
const GRAIN_ORDER: readonly TimeGrain[] = ['hour', 'day', 'week', 'month', 'quarter', 'year'];
|
||||
|
||||
/**
|
||||
* `week` does not divide `month` and `month` does not divide `quarter` by row
|
||||
* count, but both roll up cleanly from `day`. Grains that CANNOT be derived by
|
||||
* re-aggregating a coarser one are listed here so the router refuses rather
|
||||
* than quietly returning a mis-bucketed series.
|
||||
*/
|
||||
function canRollUp(from: TimeGrain, to: TimeGrain): boolean {
|
||||
if (from === to) return true;
|
||||
const fi = GRAIN_ORDER.indexOf(from);
|
||||
const ti = GRAIN_ORDER.indexOf(to);
|
||||
if (fi < 0 || ti < 0 || ti < fi) return false;
|
||||
// Weekly buckets do not compose into calendar months/quarters/years.
|
||||
if (from === 'week' && to !== 'week') return false;
|
||||
return true;
|
||||
}
|
||||
|
||||
export interface DriftTolerance {
|
||||
/** Absolute delta allowed per metric per bucket. */
|
||||
absolute: number;
|
||||
/** Relative delta allowed (0.001 = 0.1%). Both must hold. */
|
||||
relative: number;
|
||||
}
|
||||
|
||||
/** What a materialization IS. Declarative; the engine adapters read it. */
|
||||
export interface MaterializationSpec {
|
||||
materialization_id: string;
|
||||
tenant_id: string;
|
||||
project_id: string;
|
||||
/** The dataset it pre-aggregates (BI-P003). */
|
||||
dataset: string;
|
||||
strategy: MaterializationStrategy;
|
||||
/** Physical target (ClickHouse table, dbt model, Cube pre-agg name). */
|
||||
target_ref: string;
|
||||
metrics: string[];
|
||||
dimensions: string[];
|
||||
time_dimension?: string;
|
||||
grain?: TimeGrain;
|
||||
/** The freshness contract. Older than this and it stops serving. */
|
||||
max_staleness_seconds: number;
|
||||
/** How often the aggregate must be re-proved against its source. */
|
||||
drift_check_interval_seconds: number;
|
||||
tolerance: DriftTolerance;
|
||||
}
|
||||
|
||||
export const DRIFT_VERDICTS = ['match', 'drifted', 'insufficient_data'] as const;
|
||||
export type DriftVerdict = (typeof DRIFT_VERDICTS)[number];
|
||||
|
||||
/** Runtime state. Separate from the spec: the spec is config, this is fact. */
|
||||
export interface MaterializationStatus {
|
||||
materialization_id: string;
|
||||
state: MaterializationState;
|
||||
/**
|
||||
* The newest source timestamp the aggregate represents. UNDEFINED means
|
||||
* UNKNOWN, and unknown is never treated as current.
|
||||
*/
|
||||
data_watermark?: string;
|
||||
last_refresh_at?: string;
|
||||
last_drift_check_at?: string;
|
||||
last_drift_verdict?: DriftVerdict;
|
||||
}
|
||||
|
||||
export const ROUTING_REFUSALS = [
|
||||
'coverage_miss',
|
||||
'watermark_unknown',
|
||||
'staleness_exceeded',
|
||||
'window_beyond_watermark',
|
||||
'drift_never_verified',
|
||||
'drift_verification_stale',
|
||||
'drift_detected',
|
||||
'state_not_serving',
|
||||
] as const;
|
||||
export type RoutingRefusal = (typeof ROUTING_REFUSALS)[number];
|
||||
|
||||
export type RoutingDecision =
|
||||
| {
|
||||
source: 'materialization';
|
||||
materialization_id: string;
|
||||
target_ref: string;
|
||||
/** Reported so a caller can show it; never inferred, always measured. */
|
||||
staleness_seconds: number;
|
||||
}
|
||||
| { source: 'base_table'; refusal: RoutingRefusal; detail: string };
|
||||
|
||||
function refuse(refusal: RoutingRefusal, detail: string): RoutingDecision {
|
||||
return { source: 'base_table', refusal, detail };
|
||||
}
|
||||
|
||||
/**
|
||||
* Staleness of an aggregate, in seconds, or `undefined` when it cannot be
|
||||
* computed. Deliberately NOT defaulted to 0: "we could not tell" and "it is
|
||||
* current" are different facts, and conflating them is the bug.
|
||||
*/
|
||||
export function materializationStaleness(status: MaterializationStatus, now: Date): number | undefined {
|
||||
if (!status.data_watermark) return undefined;
|
||||
const t = Date.parse(status.data_watermark);
|
||||
if (Number.isNaN(t)) return undefined;
|
||||
return Math.max(0, Math.floor((now.getTime() - t) / 1000));
|
||||
}
|
||||
|
||||
/**
|
||||
* Decide whether a request may be answered from the pre-aggregate.
|
||||
*
|
||||
* Fail-closed by construction: the function returns `materialization` only
|
||||
* after every check has passed, and every unknown is a refusal with a named
|
||||
* reason. Callers get the reason so it can be logged and counted (BI-P103
|
||||
* reads the counts).
|
||||
*/
|
||||
export function selectQuerySource(args: {
|
||||
request: AnalyticsQueryRequest;
|
||||
spec: MaterializationSpec;
|
||||
status: MaterializationStatus;
|
||||
now: Date;
|
||||
}): RoutingDecision {
|
||||
const { request, spec, status, now } = args;
|
||||
|
||||
if (request.dataset !== spec.dataset) {
|
||||
return refuse('coverage_miss', `materialization covers "${spec.dataset}", not "${request.dataset}"`);
|
||||
}
|
||||
if (status.state !== 'serving') {
|
||||
return refuse('state_not_serving', `materialization state is "${status.state}"`);
|
||||
}
|
||||
|
||||
// ── coverage ────────────────────────────────────────────────────────────
|
||||
const metrics = new Set(spec.metrics);
|
||||
for (const m of request.metrics) {
|
||||
if (!metrics.has(m)) return refuse('coverage_miss', `metric "${m}" is not in the pre-aggregate`);
|
||||
}
|
||||
const dimensions = new Set(spec.dimensions);
|
||||
for (const d of request.dimensions) {
|
||||
if (!dimensions.has(d)) return refuse('coverage_miss', `dimension "${d}" is not in the pre-aggregate`);
|
||||
}
|
||||
for (const f of request.filters) {
|
||||
if (!dimensions.has(f.dimension)) {
|
||||
// The aggregate summed across this dimension. Filtering afterwards would
|
||||
// return a number computed over rows the filter should have excluded.
|
||||
return refuse(
|
||||
'coverage_miss',
|
||||
`filter on "${f.dimension}" cannot be applied: the pre-aggregate has already summed across it`,
|
||||
);
|
||||
}
|
||||
}
|
||||
const requestedGrain = request.timeRange?.grain;
|
||||
if (requestedGrain) {
|
||||
if (!spec.grain) return refuse('coverage_miss', 'the pre-aggregate has no time grain');
|
||||
if (!canRollUp(spec.grain, requestedGrain)) {
|
||||
return refuse(
|
||||
'coverage_miss',
|
||||
`a "${spec.grain}" pre-aggregate cannot answer at "${requestedGrain}" grain`,
|
||||
);
|
||||
}
|
||||
}
|
||||
|
||||
// ── freshness ───────────────────────────────────────────────────────────
|
||||
const staleness = materializationStaleness(status, now);
|
||||
if (staleness === undefined) {
|
||||
return refuse(
|
||||
'watermark_unknown',
|
||||
'the pre-aggregate has no data watermark — unknown age is not current age',
|
||||
);
|
||||
}
|
||||
if (staleness > spec.max_staleness_seconds) {
|
||||
return refuse(
|
||||
'staleness_exceeded',
|
||||
`${staleness}s stale, contract allows ${spec.max_staleness_seconds}s`,
|
||||
);
|
||||
}
|
||||
const to = request.timeRange?.to;
|
||||
if (to) {
|
||||
const toMs = Date.parse(to);
|
||||
const watermarkMs = Date.parse(status.data_watermark!);
|
||||
if (!Number.isNaN(toMs) && toMs > watermarkMs) {
|
||||
return refuse(
|
||||
'window_beyond_watermark',
|
||||
`the question runs to ${to} but the pre-aggregate only contains data to ${status.data_watermark}`,
|
||||
);
|
||||
}
|
||||
}
|
||||
|
||||
// ── drift ───────────────────────────────────────────────────────────────
|
||||
if (!status.last_drift_check_at || !status.last_drift_verdict) {
|
||||
return refuse(
|
||||
'drift_never_verified',
|
||||
'the pre-aggregate has never been compared to its source; a new aggregate is as unproven as a drifted one',
|
||||
);
|
||||
}
|
||||
if (status.last_drift_verdict !== 'match') {
|
||||
return refuse('drift_detected', `last drift check returned "${status.last_drift_verdict}"`);
|
||||
}
|
||||
const checkedAgo = Math.floor((now.getTime() - Date.parse(status.last_drift_check_at)) / 1000);
|
||||
if (Number.isNaN(checkedAgo)) {
|
||||
return refuse('drift_never_verified', 'last_drift_check_at is not a timestamp');
|
||||
}
|
||||
if (checkedAgo > spec.drift_check_interval_seconds) {
|
||||
return refuse(
|
||||
'drift_verification_stale',
|
||||
`last verified ${checkedAgo}s ago, spec requires every ${spec.drift_check_interval_seconds}s`,
|
||||
);
|
||||
}
|
||||
|
||||
return {
|
||||
source: 'materialization',
|
||||
materialization_id: spec.materialization_id,
|
||||
target_ref: spec.target_ref,
|
||||
staleness_seconds: staleness,
|
||||
};
|
||||
}
|
||||
|
||||
/**
|
||||
* The routing decision as a freshness envelope for the result (BI-P026 shape):
|
||||
* a consumer must be able to see WHICH source answered and how old it was.
|
||||
* A result that does not say is a result nobody can audit.
|
||||
*/
|
||||
export function describeRouting(decision: RoutingDecision): {
|
||||
served_from: 'materialization' | 'base_table';
|
||||
materialization_id?: string;
|
||||
staleness_seconds?: number;
|
||||
refusal?: RoutingRefusal;
|
||||
detail?: string;
|
||||
} {
|
||||
return decision.source === 'materialization'
|
||||
? {
|
||||
served_from: 'materialization',
|
||||
materialization_id: decision.materialization_id,
|
||||
staleness_seconds: decision.staleness_seconds,
|
||||
}
|
||||
: { served_from: 'base_table', refusal: decision.refusal, detail: decision.detail };
|
||||
}
|
||||
|
||||
export function assertSpec(spec: MaterializationSpec): void {
|
||||
const invalid = (d: string): never => {
|
||||
throw new SiaxError('INVALID_REQUEST', '0pt.analytics.materialization', d);
|
||||
};
|
||||
if (!spec.materialization_id?.trim()) invalid('materialization_id is required');
|
||||
if (!spec.tenant_id?.trim() || !spec.project_id?.trim()) invalid('a materialization is tenant-scoped');
|
||||
if (spec.metrics.length === 0) invalid('a materialization with no metrics pre-aggregates nothing');
|
||||
if (spec.max_staleness_seconds <= 0) invalid('max_staleness_seconds must be positive');
|
||||
if (spec.drift_check_interval_seconds <= 0) {
|
||||
invalid(
|
||||
'drift_check_interval_seconds must be positive — an aggregate that is never re-verified is one ' +
|
||||
'source mutation away from serving wrong numbers forever',
|
||||
);
|
||||
}
|
||||
if (spec.tolerance.absolute < 0 || spec.tolerance.relative < 0) invalid('tolerance must be non-negative');
|
||||
if (spec.grain && !GRAIN_ORDER.includes(spec.grain)) invalid('grain is not a time grain');
|
||||
}
|
||||
Reference in New Issue
Block a user