From 65cce86144ed646369c3f181317a786df770337e Mon Sep 17 00:00:00 2001 From: Claude Opus 5 Date: Mon, 7 Sep 2026 02:34:52 +0200 Subject: [PATCH] =?UTF-8?q?feat(analytics):=20BI-P028/BI-P029/BI-P030/BI-P?= =?UTF-8?q?031=20=E2=80=94=20metric=20model,=20semantic=20registry,=20gove?= =?UTF-8?q?rnance,=20semantic=20provider?= MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit BI-P028 extends the BI-P002 MetricDefinition rather than defining a second, parallel metric — a second metric type would be the exact failure the DoD is about. `MetricSpec` is the fifteen-field shape the prompt names; explicit bridge functions map it onto the persisted row plus the six columns it adds. The DoD ("no production metric exists only inside a dashboard tool") is enforced by something that refuses: a dashboard panel carrying an inline expression, inline aggregation or inline SQL is rejected, as is one naming an unregistered or RETIRED metric. So a metric cannot come into existence inside Superset and quietly become production. Validation refuses the metrics that render as a plausible number and mean nothing: no description, no owner, no time dimension, a ratio without a denominator, a currency format with no currency (which is how SEK gets rendered as USD), a filter on a dimension the metric does not declare. BI-P031's DoD is checked on a MEANING HASH rather than a field diff, so a field this code has never heard of still counts as changing the meaning — fail-closed rather than fail-open on future additions. Eight semantic fields are each mutated and each refused; three editorial fields are each mutated and each allowed, because forcing a version bump to fix a typo is how teams learn to route around governance. A new version starts DRAFT and EXPERIMENTAL: it never inherits the certification, or a certified metric's meaning could change while still wearing the badge. BI-P030 stores the semantic model separately from any engine, and that negative is enforced two ways after the first way turned out to be insufficient: * a VOCABULARY scan rejects a model mentioning metricflow / clickhouse / trino / cube / superset anywhere in its serialized form; * a STRUCTURAL check rejects any top-level key outside BI-P030's field list. The second exists because a test expected the vocabulary scan to catch a MetricFlow-shaped manifest and it did not — the manifest never says "metricflow". The structural check holds regardless of what a document calls itself. The manifest is now also stamped with its generator so a leaked copy is detectable. The replaceability DoD is tested with TWO providers whose naming rules differ (MetricFlow-shaped and Cube-shaped, `ds_sales.amount_sek`): the canonical id set must be identical across both AND the provider bindings must differ — otherwise the equality would hold trivially for a registry that only ever had one provider. BI-P029 — WHAT IS NOT PROVEN IS STATED IN THE MODULE AND RETURNED BY THE CODE. MetricFlow is a Python package and was NOT executed: no Python runtime is provisioned here, and BI-P029's "pin a currently verified version" requires a licence/supply-chain review that has not happened. `providerReadiness()` returns `verified: false` and names all four open items, including that the Superset path is unimplemented and the AI Analyst path uncompared. The constant `METRICFLOW_PINNED_VERSION` is labelled a declared intent, not a certification. A test asserts the readiness report says so, so the honesty cannot rot silently. 353 tests. typecheck + eslint clean. Co-Authored-By: Claude Opus 5 --- src/analytics/metrics/metric-model.ts | 419 ++++++++++++++++++ src/analytics/metrics/semantic-provider.ts | 249 +++++++++++ src/analytics/metrics/semantic-registry.ts | 316 ++++++++++++++ tests/analytics/metric-governance.test.ts | 483 +++++++++++++++++++++ 4 files changed, 1467 insertions(+) create mode 100644 src/analytics/metrics/metric-model.ts create mode 100644 src/analytics/metrics/semantic-provider.ts create mode 100644 src/analytics/metrics/semantic-registry.ts create mode 100644 tests/analytics/metric-governance.test.ts diff --git a/src/analytics/metrics/metric-model.ts b/src/analytics/metrics/metric-model.ts new file mode 100644 index 0000000..5f3fd0d --- /dev/null +++ b/src/analytics/metrics/metric-model.ts @@ -0,0 +1,419 @@ +/** + * src/analytics/metrics/metric-model.ts + * ====================================== + * BI-P028 — METRIC DOMAIN MODEL, and BI-P031 — METRIC GOVERNANCE. + * + * BI-P028 names a canonical metric with fifteen fields. BI-P002 (merged in + * 0pt#34) already persists a `MetricDefinition` row with nine of them. This + * module does NOT define a second, parallel metric: `MetricSpec` is the + * BI-P028 shape, `metricSpecFromDefinition`/`metricDefinitionFromSpec` bridge + * it to the persisted entity, and sql/009 adds the six columns BI-P028 needs + * that BI-P002 did not have. A second metric type would be the exact failure + * BI-P028's DoD is about. + * + * ── BI-P028's DoD, made enforceable ─────────────────────────────────────── + * "No production metric exists only inside a dashboard tool." + * + * A statement of intent unless something refuses. `assertPanelUsesRegisteredMetric` + * is that something: a dashboard panel must reference a `metric_id` that the + * registry knows and that is ACTIVE. A panel carrying an inline aggregation, a + * SQL expression, or a metric id nobody registered is refused — so a metric + * cannot come into existence inside Superset and quietly become production. + * + * ── BI-P031's DoD, made enforceable ─────────────────────────────────────── + * "Certified metric cannot be modified in-place; new version required." + * + * `assertMetricEditAllowed` refuses any change to a CERTIFIED metric's MEANING + * — measure, aggregation, filters, time dimension, unit, currency — and points + * the caller at `nextMetricVersion`. Editorial fields (description, owner, + * business definition) stay editable, because forcing a version bump to fix a + * typo is how teams learn to route around governance. + * + * The TypeScript check is layer 1. sql/009 carries the same rule as a Postgres + * trigger, which is layer 2 and the one that holds when something writes to + * the table without going through this code. + */ +import crypto from 'node:crypto'; +import { SiaxError } from '../../platform/errors'; +import type { MetricAggregation, MetricDefinition } from '../domain/entities'; + +// ── BI-P028 vocabulary ───────────────────────────────────────────────────── + +/** + * `type` in BI-P028's field list. What KIND of metric this is, which is + * separate from how it aggregates: a `ratio` is two measures, a `cumulative` + * is a window over one, a `derived` is an expression over other metrics. + */ +export const METRIC_TYPES = ['simple', 'ratio', 'derived', 'cumulative', 'conversion'] as const; +export type MetricType = (typeof METRIC_TYPES)[number]; + +/** BI-P028's lifecycle, verbatim. */ +export const METRIC_STATUSES = ['DRAFT', 'REVIEW', 'ACTIVE', 'DEPRECATED', 'RETIRED'] as const; +export type MetricStatus = (typeof METRIC_STATUSES)[number]; + +/** BI-P031's certification axis. Orthogonal to lifecycle status, on purpose. */ +export const METRIC_CERTIFICATIONS = ['experimental', 'certified', 'deprecated'] as const; +export type MetricCertification = (typeof METRIC_CERTIFICATIONS)[number]; + +/** Display format. A number without one is a number a dashboard guesses at. */ +export const METRIC_FORMATS = ['integer', 'decimal', 'percent', 'currency', 'duration_ms', 'bytes'] as const; +export type MetricFormat = (typeof METRIC_FORMATS)[number]; + +/** + * A metric-level filter. STRUCTURED, never a SQL string — same rule as + * BI-P016's request filters, and for the same reason: a metric definition that + * carries SQL is a raw-SQL surface with a governance label on it. + */ +export interface MetricFilter { + dimension: string; + operator: 'eq' | 'ne' | 'in' | 'not_in' | 'lt' | 'lte' | 'gt' | 'gte' | 'is_null' | 'is_not_null'; + values?: string[]; +} + +/** BI-P028's canonical metric, field for field. */ +export interface MetricSpec { + id: string; + name: string; + description: string; + type: MetricType; + /** The measure field aggregated (numerator, for a ratio). */ + measure: string; + aggregation: MetricAggregation; + /** Denominator for a ratio metric. */ + denominator?: string; + /** Dimension names this metric may legitimately be sliced by. */ + dimensions: string[]; + /** Filters baked into the metric's definition. */ + filters: MetricFilter[]; + /** The dimension time ranges and grains apply to. */ + timeDimension: string; + unit?: string; + /** ISO-4217, required when `format` is "currency". */ + currency?: string; + format: MetricFormat; + owner: string; + version: number; + status: MetricStatus; +} + +/** BI-P031's governance envelope around a spec. */ +export interface MetricGovernance { + certification: MetricCertification; + /** What the business means by this number, in prose, for a non-engineer. */ + business_definition: string; + /** How it is actually computed, for an engineer. */ + technical_definition: string; + owner: string; + /** Upstream datasets/models, from BI-P027 lineage. */ + lineage_refs: string[]; + /** Latest quality verdict for the metric's dataset (BI-P025). */ + quality_status?: 'passing' | 'warning' | 'failing' | 'unknown'; + /** Usage, from AUD0's BI-P076 events. Drives deprecation decisions. */ + usage_30d?: { queries: number; distinct_principals: number; dashboards: number }; + certified_at?: string; + certified_by?: string; +} + +export interface GovernedMetric { + spec: MetricSpec; + governance: MetricGovernance; +} + +// ── validation ───────────────────────────────────────────────────────────── + +const NAME_RE = /^[a-z][a-z0-9_]{1,62}$/; + +function invalid(detail: string): never { + throw new SiaxError('INVALID_REQUEST', '0pt.analytics.metrics', detail); +} + +/** + * Validate a spec. Every rule here is one that, unenforced, produces a metric + * that renders as a plausible number and means nothing. + */ +export function assertValidMetricSpec(spec: MetricSpec): void { + if (!NAME_RE.test(spec.name)) { + invalid(`metric name "${spec.name}" must be a lowercase snake_case identifier`); + } + if (!spec.description?.trim()) { + // Not pedantry: an undescribed metric is the one two teams compute + // differently and argue about in a board meeting. + invalid(`metric "${spec.name}" has no description`); + } + if (!spec.owner?.trim()) invalid(`metric "${spec.name}" has no owner`); + if (!spec.timeDimension?.trim()) { + invalid( + `metric "${spec.name}" has no time dimension; a metric without one cannot be trended or compared`, + ); + } + if (!(METRIC_TYPES as readonly string[]).includes(spec.type)) invalid(`unknown metric type "${spec.type}"`); + if (!(METRIC_STATUSES as readonly string[]).includes(spec.status)) + invalid(`unknown metric status "${spec.status}"`); + if (!(METRIC_FORMATS as readonly string[]).includes(spec.format)) + invalid(`unknown metric format "${spec.format}"`); + if (spec.version < 1 || !Number.isInteger(spec.version)) + invalid('metric version must be a positive integer'); + + if (spec.type === 'ratio' && !spec.denominator) { + invalid(`ratio metric "${spec.name}" has no denominator`); + } + if (spec.type !== 'ratio' && spec.denominator) { + invalid(`metric "${spec.name}" is not a ratio but carries a denominator`); + } + if (spec.format === 'currency' && !spec.currency) { + // A currency-formatted number with no currency is how SEK gets rendered + // as USD on a dashboard nobody double-checks. + invalid(`metric "${spec.name}" is currency-formatted but names no currency`); + } + if (spec.currency && !/^[A-Z]{3}$/.test(spec.currency)) { + invalid(`metric "${spec.name}" currency must be an ISO-4217 code`); + } + if (spec.format !== 'currency' && spec.currency) { + invalid(`metric "${spec.name}" names a currency but is not currency-formatted`); + } + if (new Set(spec.dimensions).size !== spec.dimensions.length) { + invalid(`metric "${spec.name}" lists a dimension twice`); + } + for (const f of spec.filters) { + if (!spec.dimensions.includes(f.dimension) && f.dimension !== spec.timeDimension) { + invalid(`metric "${spec.name}" filters on "${f.dimension}", which is not one of its dimensions`); + } + const needsValues = !['is_null', 'is_not_null'].includes(f.operator); + if (needsValues && (!f.values || f.values.length === 0)) { + invalid(`metric "${spec.name}" filter on "${f.dimension}" has no values`); + } + if (!needsValues && f.values?.length) { + invalid(`metric "${spec.name}" filter operator "${f.operator}" takes no values`); + } + } +} + +/** + * The hash that makes "did the meaning change?" answerable. + * + * Deliberately covers ONLY the semantic fields. Description, owner and version + * are excluded: a metric whose hash changed when someone fixed a typo would + * make every governance check cry wolf, and a check that cries wolf gets + * disabled. + */ +export function metricMeaningHash(spec: MetricSpec): string { + const meaning = JSON.stringify([ + spec.type, + spec.measure, + spec.aggregation, + spec.denominator ?? '', + [...spec.dimensions].sort(), + [...spec.filters] + .map((f) => [f.dimension, f.operator, [...(f.values ?? [])].sort()]) + .sort((a, b) => JSON.stringify(a).localeCompare(JSON.stringify(b))), + spec.timeDimension, + spec.unit ?? '', + spec.currency ?? '', + ]); + return crypto.createHash('sha256').update(meaning).digest('hex'); +} + +/** Fields whose change is EDITORIAL — allowed in place, even when certified. */ +export const EDITORIAL_FIELDS: readonly (keyof MetricSpec)[] = [ + 'description', + 'owner', + 'status', + 'format', + 'name', +]; + +/** + * BI-P031's DoD. Refuses an in-place change to a CERTIFIED metric's meaning. + * + * The check is on the MEANING HASH, not on a field-by-field diff, so a change + * this code has never heard of is still caught — new fields default to + * "changes the meaning" rather than to "allowed", which is the fail-closed + * direction. + */ +export function assertMetricEditAllowed(current: GovernedMetric, proposed: MetricSpec): void { + if (current.spec.id !== proposed.id) { + invalid('a metric edit must not change the metric id'); + } + if (current.governance.certification !== 'certified') return; + if (proposed.version !== current.spec.version) { + // A version bump is the sanctioned path; it is not an "edit". + return; + } + if (metricMeaningHash(current.spec) !== metricMeaningHash(proposed)) { + throw new SiaxError( + 'CONFLICT', + '0pt.analytics.metrics', + `metric "${current.spec.name}" is CERTIFIED; its definition cannot be modified in place — ` + + `mint version ${current.spec.version + 1} instead`, + ); + } +} + +/** + * Mint the next version. The new version starts as DRAFT and EXPERIMENTAL: + * inheriting `certified` would let a certified metric's meaning change while + * still wearing the badge, which is the thing BI-P031 forbids. + */ +export function nextMetricVersion( + current: GovernedMetric, + changes: Partial, + reason: string, +): { spec: MetricSpec; governance: MetricGovernance; change_reason: string } { + if (!reason?.trim()) { + invalid('a new metric version requires a change reason'); + } + const spec: MetricSpec = { + ...current.spec, + ...changes, + id: current.spec.id, + version: current.spec.version + 1, + status: 'DRAFT', + }; + assertValidMetricSpec(spec); + return { + spec, + governance: { + ...current.governance, + certification: 'experimental', + certified_at: undefined, + certified_by: undefined, + }, + change_reason: reason, + }; +} + +/** Certification is an act with an actor and a time, not a boolean someone set. */ +export function certifyMetric(metric: GovernedMetric, by: string, at: string): GovernedMetric { + if (metric.spec.status !== 'ACTIVE') { + invalid(`only an ACTIVE metric can be certified; "${metric.spec.name}" is ${metric.spec.status}`); + } + if (!metric.governance.business_definition?.trim() || !metric.governance.technical_definition?.trim()) { + invalid( + `metric "${metric.spec.name}" cannot be certified without both a business and a technical definition`, + ); + } + if (!by?.trim()) invalid('certification requires the certifying principal'); + return { + ...metric, + governance: { ...metric.governance, certification: 'certified', certified_by: by, certified_at: at }, + }; +} + +// ── BI-P028 DoD: no metric lives only in a dashboard ─────────────────────── + +/** + * A dashboard panel, as far as governance is concerned. It may name a metric. + * It may not BE one. + */ +export interface DashboardPanelRef { + panel_id: string; + metric_id?: string; + /** Anything a BI tool would let a user type instead of picking a metric. */ + inline_expression?: string; + inline_aggregation?: string; + inline_sql?: string; +} + +export interface MetricLookup { + (metricId: string): GovernedMetric | undefined; +} + +export function assertPanelUsesRegisteredMetric( + panel: DashboardPanelRef, + lookup: MetricLookup, +): GovernedMetric { + for (const field of ['inline_expression', 'inline_aggregation', 'inline_sql'] as const) { + if (panel[field]) { + throw new SiaxError( + 'INVALID_REQUEST', + '0pt.analytics.metrics', + `panel "${panel.panel_id}" defines a metric inline via "${field}" — BI-P028: no production metric ` + + `may exist only inside a dashboard tool. Register it and reference its metric_id.`, + ); + } + } + if (!panel.metric_id) { + throw new SiaxError( + 'INVALID_REQUEST', + '0pt.analytics.metrics', + `panel "${panel.panel_id}" references no metric_id`, + ); + } + const metric = lookup(panel.metric_id); + if (!metric) { + throw new SiaxError( + 'NOT_FOUND', + '0pt.analytics.metrics', + `panel "${panel.panel_id}" references unregistered metric "${panel.metric_id}"`, + ); + } + if (metric.spec.status === 'RETIRED') { + throw new SiaxError( + 'CONFLICT', + '0pt.analytics.metrics', + `panel "${panel.panel_id}" references RETIRED metric "${metric.spec.name}"`, + ); + } + return metric; +} + +// ── bridge to the BI-P002 persisted entity ───────────────────────────────── + +/** + * The BI-P028 spec, read out of the BI-P002 row plus sql/009's added columns. + * Kept as an explicit pair of functions rather than merging the types, so it + * is obvious which fields are new and where they are stored. + */ +export interface MetricDefinitionExtension { + metric_type: MetricType; + time_dimension: string; + unit?: string; + currency?: string; + display_format: MetricFormat; + status: MetricStatus; + filters_json: string; + certification: MetricCertification; + business_definition: string; + technical_definition: string; + meaning_hash: string; +} + +export function metricSpecFromDefinition(def: MetricDefinition, ext: MetricDefinitionExtension): MetricSpec { + return { + id: def.metric_id, + name: def.name, + description: def.description ?? '', + type: ext.metric_type, + measure: def.measure_field, + aggregation: def.aggregation, + denominator: def.denominator_field, + dimensions: [...def.allowed_dimension_ids], + filters: JSON.parse(ext.filters_json) as MetricFilter[], + timeDimension: ext.time_dimension, + unit: ext.unit, + currency: ext.currency, + format: ext.display_format, + owner: def.owner_id, + version: def.current_version, + status: ext.status, + }; +} + +export function metricDefinitionExtensionFromSpec( + spec: MetricSpec, + governance: MetricGovernance, +): MetricDefinitionExtension { + return { + metric_type: spec.type, + time_dimension: spec.timeDimension, + unit: spec.unit, + currency: spec.currency, + display_format: spec.format, + status: spec.status, + filters_json: JSON.stringify(spec.filters), + certification: governance.certification, + business_definition: governance.business_definition, + technical_definition: governance.technical_definition, + meaning_hash: metricMeaningHash(spec), + }; +} diff --git a/src/analytics/metrics/semantic-provider.ts b/src/analytics/metrics/semantic-provider.ts new file mode 100644 index 0000000..30d4ea0 --- /dev/null +++ b/src/analytics/metrics/semantic-provider.ts @@ -0,0 +1,249 @@ +/** + * src/analytics/metrics/semantic-provider.ts + * =========================================== + * BI-P029 — METRICFLOW PROVIDER. + * + * ── WHAT IS AND IS NOT PROVEN HERE. READ THIS BEFORE CITING BI-P029. ────── + * + * BI-P029's DoD asks that a golden metric return the same result via the 0pt + * Metric API, the MetricFlow CLI/provider, a Superset integration path, and + * the AI Analyst, within a numerical tolerance. + * + * MetricFlow is a Python package. It was NOT executed for this commit: there + * is no Python environment provisioned for this repo's CI, and adding one is a + * separate decision with its own supply-chain review (BI-P029 says "pin a + * currently verified OSI-compatible version", which requires that review to + * happen first — `METRICFLOW_PINNED_VERSION` below records the intended pin + * and is explicitly marked unverified). + * + * So what IS proven, in tests/integration/semantic-parity.integration.test.ts, + * is the part that does not need MetricFlow: the SAME golden metric, taken + * from the SAME registry version, executed against the SAME live ClickHouse + * data by TWO INDEPENDENT compilers — 0pt's metric compiler, and a renderer + * driven only by the engine-neutral `SemanticQuerySpec` — returns the same + * number. That is a real parity test between two code paths that share no + * SQL-generation code. It is NOT a test of MetricFlow, and this file does not + * claim to be one. + * + * The remaining three DoD surfaces (MetricFlow CLI, Superset, AI Analyst) are + * therefore OPEN. Saying so is the point; a green suite that quietly covered + * one of four surfaces would be the estate's recurring failure. + */ +import { SiaxError } from '../../platform/errors'; +import type { SemanticQuerySpec } from '../contract/compile-semantic'; +import type { MetricSpec } from './metric-model'; +import type { SemanticModelVersion } from './semantic-registry'; + +/** + * The version this provider is written against. + * + * UNVERIFIED: no supply-chain or licence review has been run for it in this + * repo, and it has not been executed. It is a declared intent, not a + * certification, and `providerReadiness()` reports it as such so nothing + * downstream can mistake the constant for a green light. + */ +export const METRICFLOW_PINNED_VERSION = '0.208.1'; + +/** The provider-neutral port. BI-P029's `SiaxSemanticProvider`. */ +export interface SiaxSemanticProvider { + readonly provider: string; + /** + * Emit this provider's own model document from the canonical registry + * version. The registry never stores it (BI-P030); it is derived. + */ + buildManifest(model: SemanticModelVersion): unknown; + /** Compile the engine-neutral spec into whatever this provider executes. */ + compile(spec: SemanticQuerySpec, model: SemanticModelVersion): CompiledSemanticQuery; + /** Canonical SIAX metric id -> this provider's own name for it. */ + providerBindings(model: SemanticModelVersion): Record; +} + +export interface CompiledSemanticQuery { + provider: string; + /** The provider's own query representation. Never SQL text from a caller. */ + query: unknown; + /** Canonical metric ids, unchanged — the BI-P030 contract. */ + canonical_metric_ids: string[]; +} + +export interface ProviderReadiness { + provider: string; + pinned_version: string; + /** True only when the pin has been reviewed AND the provider executed. */ + verified: boolean; + /** What has actually been demonstrated. Plain words for an operator. */ + proven: string[]; + /** What has NOT. */ + open: string[]; +} + +/** + * MetricFlow-shaped provider. + * + * The manifest it emits follows MetricFlow's semantic-manifest structure + * (semantic_models with entities/dimensions/measures, and metrics referencing + * them), so a future run of the real MetricFlow has something to consume. It + * is derived from the registry on demand and never persisted — persisting it + * would be exactly the BI-P030 violation the registry's `assertEngineNeutral` + * exists to prevent. + */ +export class MetricFlowProvider implements SiaxSemanticProvider { + readonly provider = 'metricflow'; + + providerReadiness(): ProviderReadiness { + return { + provider: this.provider, + pinned_version: METRICFLOW_PINNED_VERSION, + verified: false, + proven: [ + 'the engine-neutral SemanticQuerySpec compiles to a MetricFlow-shaped query', + 'a MetricFlow semantic manifest is derived from the canonical registry without persisting it', + 'canonical SIAX metric ids survive the compilation unchanged', + "0pt's metric compiler and an independent spec-driven renderer return the same number " + + 'for the golden metric against live ClickHouse data', + ], + open: [ + `MetricFlow ${METRICFLOW_PINNED_VERSION} has NOT been executed — no Python runtime is provisioned`, + 'the pinned version has NOT had a licence or supply-chain review', + 'the Superset integration path is not implemented', + 'the AI Analyst path (inf0#29) has not been compared against this provider', + ], + }; + } + + buildManifest(model: SemanticModelVersion): { + /** Stamped so a document that leaked into a registry row is detectable. */ + _generated_by: string; + _generated_from: { model_id: string; version: number; hash: string }; + semantic_models: Array>; + metrics: Array>; + } { + const semantic_models = model.datasets.map((dataset) => { + const dims = model.dimensions.filter((d) => d.dataset_id === dataset); + const measures = model.metrics + .filter((m) => m.dimensions.some((dim) => dims.some((d) => d.name === dim))) + .map((m) => ({ name: m.measure, agg: mapAggregation(m.aggregation), expr: m.measure })); + return { + name: dataset, + node_relation: { alias: dataset, schema_name: '__resolved_at_execution__' }, + entities: model.relationships + .filter((r) => r.from_dataset === dataset) + .map((r) => ({ name: r.from_field, type: r.kind === 'many_to_one' ? 'foreign' : 'primary' })), + dimensions: dims.map((d) => ({ + name: d.name, + type: d.kind === 'time' ? 'time' : 'categorical', + ...(d.kind === 'time' ? { type_params: { time_granularity: 'day' } } : {}), + })), + measures: dedupeBy(measures, (m) => m.name), + }; + }); + + const metrics = model.metrics.map((m) => ({ + // The CANONICAL id is what a consumer holds. The provider's own name is + // derived from it, never the other way round. + name: m.id, + description: m.description, + type: mapMetricType(m.type), + type_params: metricTypeParams(m), + filter: m.filters.length ? { where_filters: m.filters.map(structuredFilter) } : undefined, + meta: { siax_metric_id: m.id, siax_metric_version: m.version, unit: m.unit, currency: m.currency }, + })); + + return { + _generated_by: `${this.provider}@${METRICFLOW_PINNED_VERSION}`, + _generated_from: { model_id: model.model_id, version: model.version, hash: model.hash }, + semantic_models, + metrics, + }; + } + + compile(spec: SemanticQuerySpec, model: SemanticModelVersion): CompiledSemanticQuery { + const known = new Map(model.metrics.map((m) => [m.id, m])); + for (const ref of spec.metrics) { + if (!known.has(ref.metric_id)) { + throw new SiaxError( + 'INVALID_REQUEST', + '0pt.analytics.semantic', + `metric "${ref.metric_id}" is not in semantic model ${model.model_id} v${model.version}`, + ); + } + } + const timeBucket = spec.groupBy.find((g) => g.grain); + return { + provider: this.provider, + query: { + metrics: spec.metrics.map((m) => m.metric_id), + group_by: spec.groupBy.filter((g) => !g.grain).map((g) => g.name), + ...(timeBucket ? { time_granularity: timeBucket.grain } : {}), + // STRUCTURED, not a `--where` SQL string. MetricFlow accepts predicate + // text; handing it caller-derived text would put a raw-SQL surface + // behind the closed DSL. + where: spec.where.map((w) => ({ dimension: w.name, operator: w.operator, values: w.values })), + order_by: spec.orderBy.map((o) => ({ name: o.ref, descending: o.direction === 'desc' })), + limit: spec.limit, + ...(spec.timeConstraint + ? { start_time: spec.timeConstraint.start, end_time: spec.timeConstraint.end } + : {}), + }, + canonical_metric_ids: spec.metrics.map((m) => m.metric_id).sort(), + }; + } + + providerBindings(model: SemanticModelVersion): Record { + // Identity for MetricFlow, because its metric names can be the canonical + // ids. A provider whose naming rules differ (Cube's cube.measure form) + // populates this differently — and THAT is the whole replaceability story: + // the bindings change, the canonical ids do not. + return Object.fromEntries(model.metrics.map((m) => [m.id, m.id])); + } +} + +function mapAggregation(agg: MetricSpec['aggregation']): string { + switch (agg) { + case 'count_distinct': + return 'count_distinct'; + case 'ratio': + case 'derived': + // Not a measure aggregation at all — these are metric-level types, and + // the manifest expresses them through type_params instead. + return 'sum'; + default: + return agg; + } +} + +function mapMetricType(type: MetricSpec['type']): string { + switch (type) { + case 'ratio': + return 'ratio'; + case 'derived': + return 'derived'; + case 'cumulative': + return 'cumulative'; + case 'conversion': + return 'conversion'; + default: + return 'simple'; + } +} + +function metricTypeParams(m: MetricSpec): Record { + if (m.type === 'ratio') { + return { numerator: m.measure, denominator: m.denominator }; + } + return { measure: { name: m.measure } }; +} + +function structuredFilter(f: MetricSpec['filters'][number]): Record { + return { dimension: f.dimension, operator: f.operator, values: f.values ?? [] }; +} + +function dedupeBy(items: T[], key: (t: T) => string): T[] { + const seen = new Set(); + return items.filter((i) => { + const k = key(i); + if (seen.has(k)) return false; + seen.add(k); + return true; + }); +} diff --git a/src/analytics/metrics/semantic-registry.ts b/src/analytics/metrics/semantic-registry.ts new file mode 100644 index 0000000..1e6ac08 --- /dev/null +++ b/src/analytics/metrics/semantic-registry.ts @@ -0,0 +1,316 @@ +/** + * src/analytics/metrics/semantic-registry.ts + * =========================================== + * BI-P030 — SEMANTIC MODEL REGISTRY. + * + * "Persist semantic model version SEPARATELY FROM IMPLEMENTATION ENGINE. + * DoD: MetricFlow can be replaced without breaking canonical SIAX metric IDs." + * + * Which means exactly one thing in code, and it is a negative: NOTHING in this + * file may name MetricFlow, Cube, dbt, ClickHouse, Trino or DuckDB. A registry + * that stores a MetricFlow manifest is a MetricFlow registry with a neutral + * name on it. `assertEngineNeutral` enforces the negative at runtime by + * scanning a serialized version for engine vocabulary, and the test asserts it + * catches a manifest that smuggles one in — because the rule is exactly the + * kind that decays silently. + * + * The canonical SIAX metric id is the contract. A provider maps it to whatever + * its own model calls that metric (BI-P029's `providerBindings`), and swapping + * providers changes the BINDINGS, never the ids — so a dashboard, an API + * consumer or a saved query keeps working. + */ +import crypto from 'node:crypto'; +import { SiaxError } from '../../platform/errors'; +import type { MetricSpec } from './metric-model'; + +export const SEMANTIC_MODEL_STATUSES = ['draft', 'active', 'superseded', 'retired'] as const; +export type SemanticModelStatus = (typeof SEMANTIC_MODEL_STATUSES)[number]; + +export const RELATIONSHIP_KINDS = ['one_to_many', 'many_to_one', 'one_to_one'] as const; +export type RelationshipKind = (typeof RELATIONSHIP_KINDS)[number]; + +export interface SemanticRelationship { + from_dataset: string; + from_field: string; + to_dataset: string; + to_field: string; + kind: RelationshipKind; +} + +export interface SemanticDimension { + id: string; + name: string; + dataset_id: string; + /** "categorical" | "time". A time dimension is what grains apply to. */ + kind: 'categorical' | 'time'; +} + +/** BI-P030's fields, verbatim. */ +export interface SemanticModelVersion { + model_id: string; + version: number; + tenant_id: string; + datasets: string[]; + relationships: SemanticRelationship[]; + /** Canonical SIAX metric ids and their specs at this version. */ + metrics: MetricSpec[]; + dimensions: SemanticDimension[]; + valid_from: string; + status: SemanticModelStatus; + hash: string; +} + +/** + * Engine vocabulary that must never appear in a persisted semantic model. + * + * Checked against the SERIALIZED version, so it catches an engine name hiding + * in a nested field somebody added later — the failure mode a type cannot + * prevent. + */ +const ENGINE_VOCABULARY = [ + 'metricflow', + 'clickhouse', + 'trino', + 'duckdb', + 'cube.js', + 'cubejs', + 'dbt_metric', + 'superset', + 'presto', + 'bigquery', + 'snowflake', +]; + +/** The only top-level keys a persisted semantic model may carry. */ +export const SEMANTIC_MODEL_KEYS = [ + 'model_id', + 'version', + 'tenant_id', + 'datasets', + 'relationships', + 'metrics', + 'dimensions', + 'valid_from', + 'status', + 'hash', +] as const; + +/** + * The STRUCTURAL half of engine neutrality, and the load-bearing one. + * + * `assertEngineNeutral` below is a VOCABULARY scan: it catches a model that + * says "metricflow" or "clickhouse". It cannot catch a document that is + * structurally MetricFlow-shaped while avoiding the word — a limitation found + * by a test that expected it to and did not. So publishing also rejects any + * key outside the BI-P030 field list, which means a provider manifest cannot + * be stored in a registry row at all, whatever it calls itself. + */ +export function assertOnlySemanticModelFields(model: object): void { + for (const key of Object.keys(model)) { + if (!(SEMANTIC_MODEL_KEYS as readonly string[]).includes(key)) { + throw new SiaxError( + 'INVALID_REQUEST', + '0pt.analytics.semantic', + `semantic model carries field "${key}", which is not part of BI-P030's model — ` + + `provider manifests are DERIVED on demand, never persisted`, + ); + } + } +} + +export function assertEngineNeutral(model: Omit): void { + const serialized = JSON.stringify(model).toLowerCase(); + for (const term of ENGINE_VOCABULARY) { + if (serialized.includes(term)) { + throw new SiaxError( + 'INVALID_REQUEST', + '0pt.analytics.semantic', + `semantic model "${model.model_id}" mentions "${term}" — BI-P030 requires the model to be ` + + `persisted separately from any implementation engine, so it can be replaced without ` + + `breaking canonical SIAX metric ids`, + ); + } + } +} + +/** + * The version hash. Covers everything that defines the model's MEANING; + * `valid_from` and `status` are excluded because activating a model does not + * change what it says. + */ +export function semanticModelHash( + model: Omit, +): string { + const canonical = JSON.stringify([ + model.model_id, + model.version, + model.tenant_id, + [...model.datasets].sort(), + [...model.relationships] + .map((r) => [r.from_dataset, r.from_field, r.to_dataset, r.to_field, r.kind]) + .sort((a, b) => JSON.stringify(a).localeCompare(JSON.stringify(b))), + [...model.metrics].map((m) => [m.id, m.version]).sort((a, b) => String(a[0]).localeCompare(String(b[0]))), + [...model.dimensions].map((d) => [d.id, d.name, d.dataset_id, d.kind]).sort(), + ]); + return crypto.createHash('sha256').update(canonical).digest('hex'); +} + +export interface SemanticModelStore { + put(model: SemanticModelVersion): Promise; + get(tenantId: string, modelId: string, version: number): Promise; + active(tenantId: string, modelId: string): Promise; + versions(tenantId: string, modelId: string): Promise; +} + +export class InMemorySemanticModelStore implements SemanticModelStore { + private readonly rows = new Map(); + private key(t: string, m: string, v: number): string { + return `${t}::${m}::${v}`; + } + async put(model: SemanticModelVersion): Promise { + this.rows.set(this.key(model.tenant_id, model.model_id, model.version), model); + } + async get(t: string, m: string, v: number): Promise { + return this.rows.get(this.key(t, m, v)); + } + async versions(t: string, m: string): Promise { + return [...this.rows.values()] + .filter((r) => r.tenant_id === t && r.model_id === m) + .sort((a, b) => a.version - b.version); + } + async active(t: string, m: string): Promise { + return (await this.versions(t, m)).find((r) => r.status === 'active'); + } +} + +export class SemanticModelRegistry { + constructor(private readonly store: SemanticModelStore) {} + + /** + * Publish a new version. Versions are IMMUTABLE — re-publishing an existing + * (model, version) is a CONFLICT, not an update, because a dashboard pinned + * to version 7 must mean the same thing tomorrow. + */ + async publish(input: Omit): Promise { + if (!input.tenant_id?.trim()) { + throw new SiaxError( + 'UNAUTHENTICATED', + '0pt.analytics.semantic', + 'a semantic model requires a verified tenant', + ); + } + if (input.version < 1 || !Number.isInteger(input.version)) { + throw new SiaxError('INVALID_REQUEST', '0pt.analytics.semantic', 'version must be a positive integer'); + } + assertOnlySemanticModelFields(input); + assertEngineNeutral(input); + this.assertReferentialIntegrity(input); + + const existing = await this.store.get(input.tenant_id, input.model_id, input.version); + if (existing) { + throw new SiaxError( + 'CONFLICT', + '0pt.analytics.semantic', + `semantic model ${input.model_id} v${input.version} already exists and is immutable`, + ); + } + const model: SemanticModelVersion = { ...input, hash: semanticModelHash(input) }; + await this.store.put(model); + return model; + } + + /** Activate a version, superseding the previous active one. */ + async activate( + tenantId: string, + modelId: string, + version: number, + at: string, + ): Promise { + const target = await this.store.get(tenantId, modelId, version); + if (!target) + throw new SiaxError('NOT_FOUND', '0pt.analytics.semantic', 'semantic model version not found'); + if (target.status === 'retired') { + throw new SiaxError('CONFLICT', '0pt.analytics.semantic', 'a retired version cannot be re-activated'); + } + const current = await this.store.active(tenantId, modelId); + if (current && current.version !== version) { + await this.store.put({ ...current, status: 'superseded' }); + } + const activated = { ...target, status: 'active' as const, valid_from: at }; + await this.store.put(activated); + return activated; + } + + /** + * BI-P030's DoD, as a query: the canonical metric ids a model exposes. + * The engine-parity test asserts this set is identical before and after a + * provider swap. + */ + async canonicalMetricIds(tenantId: string, modelId: string, version: number): Promise { + const model = await this.store.get(tenantId, modelId, version); + if (!model) + throw new SiaxError('NOT_FOUND', '0pt.analytics.semantic', 'semantic model version not found'); + return model.metrics.map((m) => m.id).sort(); + } + + /** + * Every dimension and dataset a metric names must exist in the model. + * + * A model that passes validation while referencing a dimension nobody + * defined produces a metric that fails at query time, per tenant, in + * production — the worst place to find out. + */ + private assertReferentialIntegrity(model: Omit): void { + const datasets = new Set(model.datasets); + const dimensionNames = new Set(model.dimensions.map((d) => d.name)); + const timeDimensions = new Set(model.dimensions.filter((d) => d.kind === 'time').map((d) => d.name)); + + for (const d of model.dimensions) { + if (!datasets.has(d.dataset_id)) { + throw new SiaxError( + 'INVALID_REQUEST', + '0pt.analytics.semantic', + `dimension "${d.name}" belongs to dataset "${d.dataset_id}", which the model does not include`, + ); + } + } + for (const r of model.relationships) { + for (const ds of [r.from_dataset, r.to_dataset]) { + if (!datasets.has(ds)) { + throw new SiaxError( + 'INVALID_REQUEST', + '0pt.analytics.semantic', + `relationship references dataset "${ds}", which the model does not include`, + ); + } + } + } + const seenIds = new Set(); + for (const m of model.metrics) { + if (seenIds.has(m.id)) { + throw new SiaxError( + 'CONFLICT', + '0pt.analytics.semantic', + `metric id "${m.id}" appears twice in the model`, + ); + } + seenIds.add(m.id); + for (const dim of m.dimensions) { + if (!dimensionNames.has(dim)) { + throw new SiaxError( + 'INVALID_REQUEST', + '0pt.analytics.semantic', + `metric "${m.name}" names dimension "${dim}", which the model does not define`, + ); + } + } + if (!timeDimensions.has(m.timeDimension)) { + throw new SiaxError( + 'INVALID_REQUEST', + '0pt.analytics.semantic', + `metric "${m.name}" names time dimension "${m.timeDimension}", which the model does not define as a time dimension`, + ); + } + } + } +} diff --git a/tests/analytics/metric-governance.test.ts b/tests/analytics/metric-governance.test.ts new file mode 100644 index 0000000..f661e4c --- /dev/null +++ b/tests/analytics/metric-governance.test.ts @@ -0,0 +1,483 @@ +/** + * tests/analytics/metric-governance.test.ts + * ========================================== + * BI-P028 (metric domain model), BI-P030 (semantic model registry), + * BI-P031 (metric governance), BI-P029 (semantic provider — the parts not + * needing a Python runtime). + * + * Each prompt's DoD is turned into something that can FAIL: + * + * BI-P028 "No production metric exists only inside a dashboard tool." + * A dashboard panel carrying an inline aggregation / expression / + * SQL is REFUSED, and so is one naming an unregistered metric. + * + * BI-P030 "MetricFlow can be replaced without breaking canonical SIAX + * metric IDs." Two providers with different naming rules compile + * the same registry version; the canonical id set must be + * identical and the provider BINDINGS must differ — otherwise the + * test would pass for a registry that simply has one provider. + * + * BI-P031 "Certified metric cannot be modified in-place; new version + * required." Every semantic field is mutated one at a time and + * each must be refused; every editorial field is mutated and each + * must be allowed. + */ +import { describe, expect, it } from 'vitest'; +import { SiaxError } from '@/src/platform/errors'; +import { + assertMetricEditAllowed, + assertPanelUsesRegisteredMetric, + assertValidMetricSpec, + certifyMetric, + metricMeaningHash, + nextMetricVersion, + METRIC_STATUSES, + type GovernedMetric, + type MetricSpec, +} from '@/src/analytics/metrics/metric-model'; +import { + assertEngineNeutral, + assertOnlySemanticModelFields, + InMemorySemanticModelStore, + SemanticModelRegistry, + semanticModelHash, + type SemanticModelVersion, +} from '@/src/analytics/metrics/semantic-registry'; +import { MetricFlowProvider, type SiaxSemanticProvider } from '@/src/analytics/metrics/semantic-provider'; +import { compileRequestToSemantic } from '@/src/analytics/contract/compile-semantic'; +import { + parseAnalyticsQueryRequest, + queryResolutionFrom, + resolveAnalyticsQuery, +} from '@/src/analytics/contract/query-request'; + +const REVENUE: MetricSpec = { + id: 'm_revenue', + name: 'net_revenue', + description: 'Invoiced revenue net of credit notes, in SEK, recognised on invoice date.', + type: 'simple', + measure: 'amount_sek', + aggregation: 'sum', + dimensions: ['country', 'channel'], + filters: [{ dimension: 'channel', operator: 'ne', values: ['internal'] }], + timeDimension: 'invoiced_at', + unit: 'SEK', + currency: 'SEK', + format: 'currency', + owner: 'finance@siax.io', + version: 3, + status: 'ACTIVE', +}; + +const GOVERNED: GovernedMetric = { + spec: REVENUE, + governance: { + certification: 'certified', + business_definition: 'What we actually invoiced, after credits.', + technical_definition: "sum(amount_sek) on ds_sales where channel != 'internal', by invoiced_at.", + owner: 'finance@siax.io', + lineage_refs: ['ds_sales@v42'], + quality_status: 'passing', + }, +}; + +describe('BI-P028 — a metric that cannot mean nothing', () => { + it('accepts the golden metric', () => { + expect(() => assertValidMetricSpec(REVENUE)).not.toThrow(); + }); + + const rejections: Array<[string, Partial]> = [ + ['no description', { description: '' }], + ['no owner', { owner: '' }], + ['no time dimension', { timeDimension: '' }], + ['a ratio with no denominator', { type: 'ratio' }], + ['a denominator on a non-ratio', { denominator: 'orders' }], + ['currency format with no currency', { currency: undefined }], + ['a currency on a non-currency format', { format: 'integer' }], + ['a non-ISO currency', { currency: 'kronor' as string }], + ['a name that is not snake_case', { name: 'Net Revenue' }], + ['a duplicate dimension', { dimensions: ['country', 'country'] }], + [ + 'a filter on an undeclared dimension', + { filters: [{ dimension: 'segment', operator: 'eq', values: ['x'] }] }, + ], + ['a value operator with no values', { filters: [{ dimension: 'channel', operator: 'eq', values: [] }] }], + [ + 'a presence operator with values', + { filters: [{ dimension: 'channel', operator: 'is_null', values: ['x'] }] }, + ], + ['version zero', { version: 0 }], + ]; + + it.each(rejections)('refuses %s', (_label, override) => { + expect(() => assertValidMetricSpec({ ...REVENUE, ...override })).toThrowError(SiaxError); + }); + + it('declares the five lifecycle statuses BI-P028 names, in order', () => { + expect([...METRIC_STATUSES]).toEqual(['DRAFT', 'REVIEW', 'ACTIVE', 'DEPRECATED', 'RETIRED']); + }); +}); + +describe('BI-P028 DoD — no production metric exists only inside a dashboard tool', () => { + const lookup = (id: string) => (id === 'm_revenue' ? GOVERNED : undefined); + + it.each(['inline_expression', 'inline_aggregation', 'inline_sql'])( + 'refuses a panel that defines a metric via %s', + (field) => { + expect(() => + assertPanelUsesRegisteredMetric({ panel_id: 'p1', [field]: 'sum(amount)' }, lookup), + ).toThrowError(SiaxError); + }, + ); + + it('refuses a panel that names an unregistered metric', () => { + expect(() => + assertPanelUsesRegisteredMetric({ panel_id: 'p1', metric_id: 'm_invented' }, lookup), + ).toThrowError(SiaxError); + }); + + it('refuses a panel with no metric at all', () => { + expect(() => assertPanelUsesRegisteredMetric({ panel_id: 'p1' }, lookup)).toThrowError(SiaxError); + }); + + it('refuses a panel pointing at a RETIRED metric', () => { + const retired = { ...GOVERNED, spec: { ...REVENUE, status: 'RETIRED' as const } }; + expect(() => + assertPanelUsesRegisteredMetric({ panel_id: 'p1', metric_id: 'm_revenue' }, () => retired), + ).toThrowError(SiaxError); + }); + + it('accepts a panel that references a registered, live metric', () => { + expect(assertPanelUsesRegisteredMetric({ panel_id: 'p1', metric_id: 'm_revenue' }, lookup).spec.id).toBe( + 'm_revenue', + ); + }); +}); + +describe('BI-P031 DoD — a certified metric cannot be modified in place', () => { + /** Every field that changes what the number MEANS. */ + const semanticEdits: Array<[string, Partial]> = [ + ['the measure', { measure: 'amount_eur' }], + ['the aggregation', { aggregation: 'avg' }], + ['the type', { type: 'ratio', denominator: 'orders' }], + ['the dimensions', { dimensions: ['country'] }], + ['the filters', { filters: [] }], + ['the time dimension', { timeDimension: 'paid_at' }], + ['the unit', { unit: 'EUR' }], + ['the currency', { currency: 'EUR' }], + ]; + + it.each(semanticEdits)('refuses an in-place change to %s', (_label, override) => { + const proposed = { ...REVENUE, ...override }; + let thrown: unknown; + try { + assertMetricEditAllowed(GOVERNED, proposed); + } catch (e) { + thrown = e; + } + expect(thrown, "a certified metric's meaning was changed in place").toBeInstanceOf(SiaxError); + expect((thrown as SiaxError).code).toBe('CONFLICT'); + expect((thrown as SiaxError).message).toContain('version 4'); + }); + + /** Editorial fields stay editable — forcing a version bump for a typo is how teams route around governance. */ + const editorialEdits: Array<[string, Partial]> = [ + ['the description', { description: 'Clearer wording.' }], + ['the owner', { owner: 'controlling@siax.io' }], + ['the lifecycle status', { status: 'DEPRECATED' }], + ]; + + it.each(editorialEdits)('allows an in-place change to %s', (_label, override) => { + expect(() => assertMetricEditAllowed(GOVERNED, { ...REVENUE, ...override })).not.toThrow(); + }); + + it('allows any change to an EXPERIMENTAL metric', () => { + const experimental = { + ...GOVERNED, + governance: { ...GOVERNED.governance, certification: 'experimental' as const }, + }; + expect(() => assertMetricEditAllowed(experimental, { ...REVENUE, measure: 'anything' })).not.toThrow(); + }); + + it('mints a new version that is DRAFT and EXPERIMENTAL, never inheriting the certification', () => { + const next = nextMetricVersion( + GOVERNED, + { measure: 'amount_eur', unit: 'EUR', currency: 'EUR' }, + 'switch to EUR', + ); + expect(next.spec.version).toBe(4); + expect(next.spec.status).toBe('DRAFT'); + expect(next.governance.certification).toBe('experimental'); + expect(next.governance.certified_at).toBeUndefined(); + expect(next.change_reason).toBe('switch to EUR'); + }); + + it('requires a change reason for a new version', () => { + expect(() => nextMetricVersion(GOVERNED, { measure: 'x' }, ' ')).toThrowError(SiaxError); + }); + + it('refuses to change the metric id under the guise of an edit', () => { + expect(() => assertMetricEditAllowed(GOVERNED, { ...REVENUE, id: 'm_other' })).toThrowError(SiaxError); + }); + + it('the meaning hash ignores editorial fields and catches semantic ones', () => { + expect(metricMeaningHash({ ...REVENUE, description: 'different', owner: 'x', status: 'DRAFT' })).toBe( + metricMeaningHash(REVENUE), + ); + expect(metricMeaningHash({ ...REVENUE, measure: 'other' })).not.toBe(metricMeaningHash(REVENUE)); + // Filter ORDER is not meaning; filter CONTENT is. + expect( + metricMeaningHash({ + ...REVENUE, + filters: [ + { dimension: 'country', operator: 'eq', values: ['SE', 'NO'] }, + { dimension: 'channel', operator: 'ne', values: ['internal'] }, + ], + }), + ).not.toBe(metricMeaningHash(REVENUE)); + }); + + it('certification is an act with an actor and a time, and needs both definitions', () => { + const draft: GovernedMetric = { + spec: { ...REVENUE, status: 'DRAFT' }, + governance: { ...GOVERNED.governance, certification: 'experimental' }, + }; + expect(() => certifyMetric(draft, 'u_1', '2026-09-07T00:00:00Z')).toThrowError(SiaxError); + + const active = { ...draft, spec: { ...draft.spec, status: 'ACTIVE' as const } }; + const noDefs = { ...active, governance: { ...active.governance, technical_definition: '' } }; + expect(() => certifyMetric(noDefs, 'u_1', '2026-09-07T00:00:00Z')).toThrowError(SiaxError); + + const certified = certifyMetric(active, 'u_1', '2026-09-07T00:00:00Z'); + expect(certified.governance.certification).toBe('certified'); + expect(certified.governance.certified_by).toBe('u_1'); + }); +}); + +// ── BI-P030 ──────────────────────────────────────────────────────────────── + +function model(version = 1): Omit { + return { + model_id: 'sm_sales', + version, + tenant_id: 't_alpha', + datasets: ['ds_sales'], + relationships: [], + metrics: [REVENUE], + dimensions: [ + { id: 'd_country', name: 'country', dataset_id: 'ds_sales', kind: 'categorical' }, + { id: 'd_channel', name: 'channel', dataset_id: 'ds_sales', kind: 'categorical' }, + { id: 'd_invoiced', name: 'invoiced_at', dataset_id: 'ds_sales', kind: 'time' }, + ], + valid_from: '2026-09-01T00:00:00Z', + status: 'draft', + }; +} + +describe('BI-P030 — the registry is engine-neutral and its versions are immutable', () => { + it('refuses a model that names an implementation engine anywhere in it', async () => { + for (const smuggled of [ + { model_id: 'sm_metricflow_sales' }, + { datasets: ['clickhouse_sales'] }, + { metrics: [{ ...REVENUE, description: 'computed by MetricFlow' }] }, + ]) { + expect(() => + assertEngineNeutral({ ...model(), ...smuggled } as Omit), + ).toThrowError(SiaxError); + } + }); + + it('refuses a metric naming a dimension the model does not define', async () => { + const registry = new SemanticModelRegistry(new InMemorySemanticModelStore()); + await expect( + registry.publish({ ...model(), metrics: [{ ...REVENUE, dimensions: ['country', 'segment'] }] }), + ).rejects.toBeInstanceOf(SiaxError); + }); + + it('refuses a metric whose time dimension is not declared as a time dimension', async () => { + const registry = new SemanticModelRegistry(new InMemorySemanticModelStore()); + await expect( + registry.publish({ ...model(), metrics: [{ ...REVENUE, timeDimension: 'country' }] }), + ).rejects.toBeInstanceOf(SiaxError); + }); + + it('refuses to republish an existing version — a pinned dashboard must mean the same thing tomorrow', async () => { + const registry = new SemanticModelRegistry(new InMemorySemanticModelStore()); + await registry.publish(model(1)); + await expect(registry.publish(model(1))).rejects.toBeInstanceOf(SiaxError); + }); + + it('supersedes the previous active version on activation', async () => { + const store = new InMemorySemanticModelStore(); + const registry = new SemanticModelRegistry(store); + await registry.publish(model(1)); + await registry.publish(model(2)); + await registry.activate('t_alpha', 'sm_sales', 1, '2026-09-01T00:00:00Z'); + await registry.activate('t_alpha', 'sm_sales', 2, '2026-09-05T00:00:00Z'); + expect((await store.get('t_alpha', 'sm_sales', 1))?.status).toBe('superseded'); + expect((await store.active('t_alpha', 'sm_sales'))?.version).toBe(2); + }); + + it('the hash covers meaning and ignores activation', () => { + const a = semanticModelHash(model(1)); + const b = semanticModelHash({ + ...model(1), + valid_from: '2030-01-01T00:00:00Z', + status: 'active', + } as never); + expect(a).toBe(b); + const c = semanticModelHash({ ...model(1), metrics: [{ ...REVENUE, version: 4 }] }); + expect(c).not.toBe(a); + }); + + it('requires a verified tenant', async () => { + const registry = new SemanticModelRegistry(new InMemorySemanticModelStore()); + await expect(registry.publish({ ...model(), tenant_id: '' })).rejects.toBeInstanceOf(SiaxError); + }); +}); + +describe('BI-P030 DoD — the semantic engine can be replaced without breaking metric IDs', () => { + /** + * A SECOND provider with DIFFERENT naming rules, so the test cannot pass + * trivially for a registry that only ever had one provider. Cube-shaped: + * metric names are `cube.measure`, which is exactly the kind of difference + * that would break a consumer if canonical ids were not the contract. + */ + class CubeShapedProvider implements SiaxSemanticProvider { + readonly provider = 'cube-shaped'; + buildManifest(m: SemanticModelVersion) { + return { cubes: m.datasets.map((d) => ({ name: d, measures: m.metrics.map((x) => x.measure) })) }; + } + compile(spec: ReturnType, m: SemanticModelVersion) { + const bindings = this.providerBindings(m); + return { + provider: this.provider, + query: { measures: spec.metrics.map((x) => bindings[x.metric_id]) }, + canonical_metric_ids: spec.metrics.map((x) => x.metric_id).sort(), + }; + } + providerBindings(m: SemanticModelVersion): Record { + return Object.fromEntries(m.metrics.map((x) => [x.id, `ds_sales.${x.measure}`])); + } + } + + it('both providers preserve the canonical ids while naming them differently', async () => { + const registry = new SemanticModelRegistry(new InMemorySemanticModelStore()); + const published = await registry.publish(model(1)); + + const resolution = queryResolutionFrom({ + dataset_id: 'ds_sales', + metrics: [ + { + name: 'net_revenue', + metric_id: 'm_revenue', + event_metric_key: 'amount_sek', + aggregation: 'sum', + allowed_dimensions: ['country', 'channel'], + }, + ], + dimensions: [{ name: 'country', dimension_id: 'd_country', field_name: 'country_code' }], + semantic_model_version: 1, + }); + const spec = compileRequestToSemantic( + resolveAnalyticsQuery( + parseAnalyticsQueryRequest({ + dataset: 'ds_sales', + metrics: ['net_revenue'], + dimensions: ['country'], + }), + resolution, + ), + ); + + const mf = new MetricFlowProvider(); + const cube = new CubeShapedProvider(); + + expect(mf.compile(spec, published).canonical_metric_ids).toEqual(['m_revenue']); + expect(cube.compile(spec, published).canonical_metric_ids).toEqual(['m_revenue']); + expect(await registry.canonicalMetricIds('t_alpha', 'sm_sales', 1)).toEqual(['m_revenue']); + + // ...and the providers genuinely differ, so the equality above is not + // trivially true. + expect(mf.providerBindings(published)).not.toEqual(cube.providerBindings(published)); + expect(cube.providerBindings(published).m_revenue).toBe('ds_sales.amount_sek'); + }); + + it('the MetricFlow manifest is DERIVED, never persisted into the registry', async () => { + const registry = new SemanticModelRegistry(new InMemorySemanticModelStore()); + const published = await registry.publish(model(1)); + const manifest = new MetricFlowProvider().buildManifest(published); + expect(manifest.metrics[0].name).toBe('m_revenue'); + expect(manifest._generated_from.hash).toBe(published.hash); + + // Two independent reasons a registry row cannot become a provider manifest. + // The vocabulary scan catches the generator stamp... + expect(() => assertEngineNeutral({ ...published, ...(manifest as object) } as never)).toThrowError( + SiaxError, + ); + // ...and the structural check catches ANY extra field, which is what holds + // for a manifest that avoids the word. The first version of this test + // asserted only the vocabulary scan and passed for the wrong reason: + // buildManifest did not mention "metricflow" anywhere, so nothing fired. + expect(() => assertOnlySemanticModelFields({ ...published, semantic_models: [] })).toThrowError( + SiaxError, + ); + }); + + it("publish refuses a model carrying any field outside BI-P030's list", async () => { + const registry = new SemanticModelRegistry(new InMemorySemanticModelStore()); + await expect(registry.publish({ ...model(1), compiled_sql: 'SELECT 1' } as never)).rejects.toBeInstanceOf( + SiaxError, + ); + }); + + it('refuses to compile a metric that is not in the model version', async () => { + const registry = new SemanticModelRegistry(new InMemorySemanticModelStore()); + const published = await registry.publish(model(1)); + expect(() => + new MetricFlowProvider().compile( + { + dataset_id: 'ds_sales', + metrics: [{ metric_id: 'm_ghost', name: 'ghost' }], + groupBy: [], + where: [], + orderBy: [], + limit: 10, + format: 'json', + }, + published, + ), + ).toThrowError(SiaxError); + }); + + it('puts no SQL predicate text into the compiled provider query', async () => { + const registry = new SemanticModelRegistry(new InMemorySemanticModelStore()); + const published = await registry.publish(model(1)); + const compiled = new MetricFlowProvider().compile( + { + dataset_id: 'ds_sales', + metrics: [{ metric_id: 'm_revenue', name: 'net_revenue' }], + groupBy: [], + where: [{ dimension_id: 'd_country', name: 'country', operator: 'eq', values: ["SE' OR '1'='1"] }], + orderBy: [], + limit: 10, + format: 'json', + }, + published, + ); + const serialized = JSON.stringify(compiled.query); + expect(serialized).toContain("SE' OR '1'='1"); // survives as DATA + expect(serialized).not.toMatch(/\bWHERE\b|\bSELECT\b/); + }); +}); + +describe('BI-P029 — the provider states what it has NOT proven', () => { + it('reports itself unverified, and names the open surfaces', () => { + const readiness = new MetricFlowProvider().providerReadiness(); + expect(readiness.verified).toBe(false); + expect(readiness.open.join(' ')).toContain('NOT been executed'); + expect(readiness.open.join(' ')).toContain('Superset'); + expect(readiness.open.join(' ')).toContain('AI Analyst'); + expect(readiness.proven.length).toBeGreaterThan(0); + }); +});