feat(analytics): BI-P028/BI-P029/BI-P030/BI-P031 — metric model, semantic registry, governance, semantic provider
CI (SIAX Cloud) / security (push) Successful in 29s
CI (SIAX Cloud) / integration (push) Successful in 52s
CI (SIAX Cloud) / quality (push) Successful in 1m10s
Security Scan (SIAX Cloud) / security (push) Successful in 25s
CI (SIAX Cloud) / security (pull_request) Successful in 29s
CI (SIAX Cloud) / integration (pull_request) Successful in 53s
CI (SIAX Cloud) / quality (pull_request) Successful in 1m13s
Security Scan (SIAX Cloud) / security (pull_request) Successful in 22s

BI-P028 extends the BI-P002 MetricDefinition rather than defining a second,
parallel metric — a second metric type would be the exact failure the DoD is
about. `MetricSpec` is the fifteen-field shape the prompt names; explicit
bridge functions map it onto the persisted row plus the six columns it adds.

The DoD ("no production metric exists only inside a dashboard tool") is
enforced by something that refuses: a dashboard panel carrying an inline
expression, inline aggregation or inline SQL is rejected, as is one naming an
unregistered or RETIRED metric. So a metric cannot come into existence inside
Superset and quietly become production.

Validation refuses the metrics that render as a plausible number and mean
nothing: no description, no owner, no time dimension, a ratio without a
denominator, a currency format with no currency (which is how SEK gets
rendered as USD), a filter on a dimension the metric does not declare.

BI-P031's DoD is checked on a MEANING HASH rather than a field diff, so a
field this code has never heard of still counts as changing the meaning —
fail-closed rather than fail-open on future additions. Eight semantic fields
are each mutated and each refused; three editorial fields are each mutated and
each allowed, because forcing a version bump to fix a typo is how teams learn
to route around governance. A new version starts DRAFT and EXPERIMENTAL: it
never inherits the certification, or a certified metric's meaning could change
while still wearing the badge.

BI-P030 stores the semantic model separately from any engine, and that
negative is enforced two ways after the first way turned out to be
insufficient:
  * a VOCABULARY scan rejects a model mentioning metricflow / clickhouse /
    trino / cube / superset anywhere in its serialized form;
  * a STRUCTURAL check rejects any top-level key outside BI-P030's field list.
The second exists because a test expected the vocabulary scan to catch a
MetricFlow-shaped manifest and it did not — the manifest never says
"metricflow". The structural check holds regardless of what a document calls
itself. The manifest is now also stamped with its generator so a leaked copy
is detectable.

The replaceability DoD is tested with TWO providers whose naming rules differ
(MetricFlow-shaped and Cube-shaped, `ds_sales.amount_sek`): the canonical id
set must be identical across both AND the provider bindings must differ —
otherwise the equality would hold trivially for a registry that only ever had
one provider.

BI-P029 — WHAT IS NOT PROVEN IS STATED IN THE MODULE AND RETURNED BY THE CODE.
MetricFlow is a Python package and was NOT executed: no Python runtime is
provisioned here, and BI-P029's "pin a currently verified version" requires a
licence/supply-chain review that has not happened. `providerReadiness()`
returns `verified: false` and names all four open items, including that the
Superset path is unimplemented and the AI Analyst path uncompared. The
constant `METRICFLOW_PINNED_VERSION` is labelled a declared intent, not a
certification. A test asserts the readiness report says so, so the honesty
cannot rot silently.

353 tests. typecheck + eslint clean.

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
This commit is contained in:
Claude Opus 5
2026-09-07 02:34:52 +02:00
parent ca0a3292df
commit 65cce86144
4 changed files with 1467 additions and 0 deletions
+419
View File
@@ -0,0 +1,419 @@
/**
* src/analytics/metrics/metric-model.ts
* ======================================
* BI-P028 — METRIC DOMAIN MODEL, and BI-P031 — METRIC GOVERNANCE.
*
* BI-P028 names a canonical metric with fifteen fields. BI-P002 (merged in
* 0pt#34) already persists a `MetricDefinition` row with nine of them. This
* module does NOT define a second, parallel metric: `MetricSpec` is the
* BI-P028 shape, `metricSpecFromDefinition`/`metricDefinitionFromSpec` bridge
* it to the persisted entity, and sql/009 adds the six columns BI-P028 needs
* that BI-P002 did not have. A second metric type would be the exact failure
* BI-P028's DoD is about.
*
* ── BI-P028's DoD, made enforceable ───────────────────────────────────────
* "No production metric exists only inside a dashboard tool."
*
* A statement of intent unless something refuses. `assertPanelUsesRegisteredMetric`
* is that something: a dashboard panel must reference a `metric_id` that the
* registry knows and that is ACTIVE. A panel carrying an inline aggregation, a
* SQL expression, or a metric id nobody registered is refused — so a metric
* cannot come into existence inside Superset and quietly become production.
*
* ── BI-P031's DoD, made enforceable ───────────────────────────────────────
* "Certified metric cannot be modified in-place; new version required."
*
* `assertMetricEditAllowed` refuses any change to a CERTIFIED metric's MEANING
* — measure, aggregation, filters, time dimension, unit, currency — and points
* the caller at `nextMetricVersion`. Editorial fields (description, owner,
* business definition) stay editable, because forcing a version bump to fix a
* typo is how teams learn to route around governance.
*
* The TypeScript check is layer 1. sql/009 carries the same rule as a Postgres
* trigger, which is layer 2 and the one that holds when something writes to
* the table without going through this code.
*/
import crypto from 'node:crypto';
import { SiaxError } from '../../platform/errors';
import type { MetricAggregation, MetricDefinition } from '../domain/entities';
// ── BI-P028 vocabulary ─────────────────────────────────────────────────────
/**
* `type` in BI-P028's field list. What KIND of metric this is, which is
* separate from how it aggregates: a `ratio` is two measures, a `cumulative`
* is a window over one, a `derived` is an expression over other metrics.
*/
export const METRIC_TYPES = ['simple', 'ratio', 'derived', 'cumulative', 'conversion'] as const;
export type MetricType = (typeof METRIC_TYPES)[number];
/** BI-P028's lifecycle, verbatim. */
export const METRIC_STATUSES = ['DRAFT', 'REVIEW', 'ACTIVE', 'DEPRECATED', 'RETIRED'] as const;
export type MetricStatus = (typeof METRIC_STATUSES)[number];
/** BI-P031's certification axis. Orthogonal to lifecycle status, on purpose. */
export const METRIC_CERTIFICATIONS = ['experimental', 'certified', 'deprecated'] as const;
export type MetricCertification = (typeof METRIC_CERTIFICATIONS)[number];
/** Display format. A number without one is a number a dashboard guesses at. */
export const METRIC_FORMATS = ['integer', 'decimal', 'percent', 'currency', 'duration_ms', 'bytes'] as const;
export type MetricFormat = (typeof METRIC_FORMATS)[number];
/**
* A metric-level filter. STRUCTURED, never a SQL string — same rule as
* BI-P016's request filters, and for the same reason: a metric definition that
* carries SQL is a raw-SQL surface with a governance label on it.
*/
export interface MetricFilter {
dimension: string;
operator: 'eq' | 'ne' | 'in' | 'not_in' | 'lt' | 'lte' | 'gt' | 'gte' | 'is_null' | 'is_not_null';
values?: string[];
}
/** BI-P028's canonical metric, field for field. */
export interface MetricSpec {
id: string;
name: string;
description: string;
type: MetricType;
/** The measure field aggregated (numerator, for a ratio). */
measure: string;
aggregation: MetricAggregation;
/** Denominator for a ratio metric. */
denominator?: string;
/** Dimension names this metric may legitimately be sliced by. */
dimensions: string[];
/** Filters baked into the metric's definition. */
filters: MetricFilter[];
/** The dimension time ranges and grains apply to. */
timeDimension: string;
unit?: string;
/** ISO-4217, required when `format` is "currency". */
currency?: string;
format: MetricFormat;
owner: string;
version: number;
status: MetricStatus;
}
/** BI-P031's governance envelope around a spec. */
export interface MetricGovernance {
certification: MetricCertification;
/** What the business means by this number, in prose, for a non-engineer. */
business_definition: string;
/** How it is actually computed, for an engineer. */
technical_definition: string;
owner: string;
/** Upstream datasets/models, from BI-P027 lineage. */
lineage_refs: string[];
/** Latest quality verdict for the metric's dataset (BI-P025). */
quality_status?: 'passing' | 'warning' | 'failing' | 'unknown';
/** Usage, from AUD0's BI-P076 events. Drives deprecation decisions. */
usage_30d?: { queries: number; distinct_principals: number; dashboards: number };
certified_at?: string;
certified_by?: string;
}
export interface GovernedMetric {
spec: MetricSpec;
governance: MetricGovernance;
}
// ── validation ─────────────────────────────────────────────────────────────
const NAME_RE = /^[a-z][a-z0-9_]{1,62}$/;
function invalid(detail: string): never {
throw new SiaxError('INVALID_REQUEST', '0pt.analytics.metrics', detail);
}
/**
* Validate a spec. Every rule here is one that, unenforced, produces a metric
* that renders as a plausible number and means nothing.
*/
export function assertValidMetricSpec(spec: MetricSpec): void {
if (!NAME_RE.test(spec.name)) {
invalid(`metric name "${spec.name}" must be a lowercase snake_case identifier`);
}
if (!spec.description?.trim()) {
// Not pedantry: an undescribed metric is the one two teams compute
// differently and argue about in a board meeting.
invalid(`metric "${spec.name}" has no description`);
}
if (!spec.owner?.trim()) invalid(`metric "${spec.name}" has no owner`);
if (!spec.timeDimension?.trim()) {
invalid(
`metric "${spec.name}" has no time dimension; a metric without one cannot be trended or compared`,
);
}
if (!(METRIC_TYPES as readonly string[]).includes(spec.type)) invalid(`unknown metric type "${spec.type}"`);
if (!(METRIC_STATUSES as readonly string[]).includes(spec.status))
invalid(`unknown metric status "${spec.status}"`);
if (!(METRIC_FORMATS as readonly string[]).includes(spec.format))
invalid(`unknown metric format "${spec.format}"`);
if (spec.version < 1 || !Number.isInteger(spec.version))
invalid('metric version must be a positive integer');
if (spec.type === 'ratio' && !spec.denominator) {
invalid(`ratio metric "${spec.name}" has no denominator`);
}
if (spec.type !== 'ratio' && spec.denominator) {
invalid(`metric "${spec.name}" is not a ratio but carries a denominator`);
}
if (spec.format === 'currency' && !spec.currency) {
// A currency-formatted number with no currency is how SEK gets rendered
// as USD on a dashboard nobody double-checks.
invalid(`metric "${spec.name}" is currency-formatted but names no currency`);
}
if (spec.currency && !/^[A-Z]{3}$/.test(spec.currency)) {
invalid(`metric "${spec.name}" currency must be an ISO-4217 code`);
}
if (spec.format !== 'currency' && spec.currency) {
invalid(`metric "${spec.name}" names a currency but is not currency-formatted`);
}
if (new Set(spec.dimensions).size !== spec.dimensions.length) {
invalid(`metric "${spec.name}" lists a dimension twice`);
}
for (const f of spec.filters) {
if (!spec.dimensions.includes(f.dimension) && f.dimension !== spec.timeDimension) {
invalid(`metric "${spec.name}" filters on "${f.dimension}", which is not one of its dimensions`);
}
const needsValues = !['is_null', 'is_not_null'].includes(f.operator);
if (needsValues && (!f.values || f.values.length === 0)) {
invalid(`metric "${spec.name}" filter on "${f.dimension}" has no values`);
}
if (!needsValues && f.values?.length) {
invalid(`metric "${spec.name}" filter operator "${f.operator}" takes no values`);
}
}
}
/**
* The hash that makes "did the meaning change?" answerable.
*
* Deliberately covers ONLY the semantic fields. Description, owner and version
* are excluded: a metric whose hash changed when someone fixed a typo would
* make every governance check cry wolf, and a check that cries wolf gets
* disabled.
*/
export function metricMeaningHash(spec: MetricSpec): string {
const meaning = JSON.stringify([
spec.type,
spec.measure,
spec.aggregation,
spec.denominator ?? '',
[...spec.dimensions].sort(),
[...spec.filters]
.map((f) => [f.dimension, f.operator, [...(f.values ?? [])].sort()])
.sort((a, b) => JSON.stringify(a).localeCompare(JSON.stringify(b))),
spec.timeDimension,
spec.unit ?? '',
spec.currency ?? '',
]);
return crypto.createHash('sha256').update(meaning).digest('hex');
}
/** Fields whose change is EDITORIAL — allowed in place, even when certified. */
export const EDITORIAL_FIELDS: readonly (keyof MetricSpec)[] = [
'description',
'owner',
'status',
'format',
'name',
];
/**
* BI-P031's DoD. Refuses an in-place change to a CERTIFIED metric's meaning.
*
* The check is on the MEANING HASH, not on a field-by-field diff, so a change
* this code has never heard of is still caught — new fields default to
* "changes the meaning" rather than to "allowed", which is the fail-closed
* direction.
*/
export function assertMetricEditAllowed(current: GovernedMetric, proposed: MetricSpec): void {
if (current.spec.id !== proposed.id) {
invalid('a metric edit must not change the metric id');
}
if (current.governance.certification !== 'certified') return;
if (proposed.version !== current.spec.version) {
// A version bump is the sanctioned path; it is not an "edit".
return;
}
if (metricMeaningHash(current.spec) !== metricMeaningHash(proposed)) {
throw new SiaxError(
'CONFLICT',
'0pt.analytics.metrics',
`metric "${current.spec.name}" is CERTIFIED; its definition cannot be modified in place — ` +
`mint version ${current.spec.version + 1} instead`,
);
}
}
/**
* Mint the next version. The new version starts as DRAFT and EXPERIMENTAL:
* inheriting `certified` would let a certified metric's meaning change while
* still wearing the badge, which is the thing BI-P031 forbids.
*/
export function nextMetricVersion(
current: GovernedMetric,
changes: Partial<MetricSpec>,
reason: string,
): { spec: MetricSpec; governance: MetricGovernance; change_reason: string } {
if (!reason?.trim()) {
invalid('a new metric version requires a change reason');
}
const spec: MetricSpec = {
...current.spec,
...changes,
id: current.spec.id,
version: current.spec.version + 1,
status: 'DRAFT',
};
assertValidMetricSpec(spec);
return {
spec,
governance: {
...current.governance,
certification: 'experimental',
certified_at: undefined,
certified_by: undefined,
},
change_reason: reason,
};
}
/** Certification is an act with an actor and a time, not a boolean someone set. */
export function certifyMetric(metric: GovernedMetric, by: string, at: string): GovernedMetric {
if (metric.spec.status !== 'ACTIVE') {
invalid(`only an ACTIVE metric can be certified; "${metric.spec.name}" is ${metric.spec.status}`);
}
if (!metric.governance.business_definition?.trim() || !metric.governance.technical_definition?.trim()) {
invalid(
`metric "${metric.spec.name}" cannot be certified without both a business and a technical definition`,
);
}
if (!by?.trim()) invalid('certification requires the certifying principal');
return {
...metric,
governance: { ...metric.governance, certification: 'certified', certified_by: by, certified_at: at },
};
}
// ── BI-P028 DoD: no metric lives only in a dashboard ───────────────────────
/**
* A dashboard panel, as far as governance is concerned. It may name a metric.
* It may not BE one.
*/
export interface DashboardPanelRef {
panel_id: string;
metric_id?: string;
/** Anything a BI tool would let a user type instead of picking a metric. */
inline_expression?: string;
inline_aggregation?: string;
inline_sql?: string;
}
export interface MetricLookup {
(metricId: string): GovernedMetric | undefined;
}
export function assertPanelUsesRegisteredMetric(
panel: DashboardPanelRef,
lookup: MetricLookup,
): GovernedMetric {
for (const field of ['inline_expression', 'inline_aggregation', 'inline_sql'] as const) {
if (panel[field]) {
throw new SiaxError(
'INVALID_REQUEST',
'0pt.analytics.metrics',
`panel "${panel.panel_id}" defines a metric inline via "${field}" — BI-P028: no production metric ` +
`may exist only inside a dashboard tool. Register it and reference its metric_id.`,
);
}
}
if (!panel.metric_id) {
throw new SiaxError(
'INVALID_REQUEST',
'0pt.analytics.metrics',
`panel "${panel.panel_id}" references no metric_id`,
);
}
const metric = lookup(panel.metric_id);
if (!metric) {
throw new SiaxError(
'NOT_FOUND',
'0pt.analytics.metrics',
`panel "${panel.panel_id}" references unregistered metric "${panel.metric_id}"`,
);
}
if (metric.spec.status === 'RETIRED') {
throw new SiaxError(
'CONFLICT',
'0pt.analytics.metrics',
`panel "${panel.panel_id}" references RETIRED metric "${metric.spec.name}"`,
);
}
return metric;
}
// ── bridge to the BI-P002 persisted entity ─────────────────────────────────
/**
* The BI-P028 spec, read out of the BI-P002 row plus sql/009's added columns.
* Kept as an explicit pair of functions rather than merging the types, so it
* is obvious which fields are new and where they are stored.
*/
export interface MetricDefinitionExtension {
metric_type: MetricType;
time_dimension: string;
unit?: string;
currency?: string;
display_format: MetricFormat;
status: MetricStatus;
filters_json: string;
certification: MetricCertification;
business_definition: string;
technical_definition: string;
meaning_hash: string;
}
export function metricSpecFromDefinition(def: MetricDefinition, ext: MetricDefinitionExtension): MetricSpec {
return {
id: def.metric_id,
name: def.name,
description: def.description ?? '',
type: ext.metric_type,
measure: def.measure_field,
aggregation: def.aggregation,
denominator: def.denominator_field,
dimensions: [...def.allowed_dimension_ids],
filters: JSON.parse(ext.filters_json) as MetricFilter[],
timeDimension: ext.time_dimension,
unit: ext.unit,
currency: ext.currency,
format: ext.display_format,
owner: def.owner_id,
version: def.current_version,
status: ext.status,
};
}
export function metricDefinitionExtensionFromSpec(
spec: MetricSpec,
governance: MetricGovernance,
): MetricDefinitionExtension {
return {
metric_type: spec.type,
time_dimension: spec.timeDimension,
unit: spec.unit,
currency: spec.currency,
display_format: spec.format,
status: spec.status,
filters_json: JSON.stringify(spec.filters),
certification: governance.certification,
business_definition: governance.business_definition,
technical_definition: governance.technical_definition,
meaning_hash: metricMeaningHash(spec),
};
}
+249
View File
@@ -0,0 +1,249 @@
/**
* src/analytics/metrics/semantic-provider.ts
* ===========================================
* BI-P029 — METRICFLOW PROVIDER.
*
* ── WHAT IS AND IS NOT PROVEN HERE. READ THIS BEFORE CITING BI-P029. ──────
*
* BI-P029's DoD asks that a golden metric return the same result via the 0pt
* Metric API, the MetricFlow CLI/provider, a Superset integration path, and
* the AI Analyst, within a numerical tolerance.
*
* MetricFlow is a Python package. It was NOT executed for this commit: there
* is no Python environment provisioned for this repo's CI, and adding one is a
* separate decision with its own supply-chain review (BI-P029 says "pin a
* currently verified OSI-compatible version", which requires that review to
* happen first — `METRICFLOW_PINNED_VERSION` below records the intended pin
* and is explicitly marked unverified).
*
* So what IS proven, in tests/integration/semantic-parity.integration.test.ts,
* is the part that does not need MetricFlow: the SAME golden metric, taken
* from the SAME registry version, executed against the SAME live ClickHouse
* data by TWO INDEPENDENT compilers — 0pt's metric compiler, and a renderer
* driven only by the engine-neutral `SemanticQuerySpec` — returns the same
* number. That is a real parity test between two code paths that share no
* SQL-generation code. It is NOT a test of MetricFlow, and this file does not
* claim to be one.
*
* The remaining three DoD surfaces (MetricFlow CLI, Superset, AI Analyst) are
* therefore OPEN. Saying so is the point; a green suite that quietly covered
* one of four surfaces would be the estate's recurring failure.
*/
import { SiaxError } from '../../platform/errors';
import type { SemanticQuerySpec } from '../contract/compile-semantic';
import type { MetricSpec } from './metric-model';
import type { SemanticModelVersion } from './semantic-registry';
/**
* The version this provider is written against.
*
* UNVERIFIED: no supply-chain or licence review has been run for it in this
* repo, and it has not been executed. It is a declared intent, not a
* certification, and `providerReadiness()` reports it as such so nothing
* downstream can mistake the constant for a green light.
*/
export const METRICFLOW_PINNED_VERSION = '0.208.1';
/** The provider-neutral port. BI-P029's `SiaxSemanticProvider`. */
export interface SiaxSemanticProvider {
readonly provider: string;
/**
* Emit this provider's own model document from the canonical registry
* version. The registry never stores it (BI-P030); it is derived.
*/
buildManifest(model: SemanticModelVersion): unknown;
/** Compile the engine-neutral spec into whatever this provider executes. */
compile(spec: SemanticQuerySpec, model: SemanticModelVersion): CompiledSemanticQuery;
/** Canonical SIAX metric id -> this provider's own name for it. */
providerBindings(model: SemanticModelVersion): Record<string, string>;
}
export interface CompiledSemanticQuery {
provider: string;
/** The provider's own query representation. Never SQL text from a caller. */
query: unknown;
/** Canonical metric ids, unchanged — the BI-P030 contract. */
canonical_metric_ids: string[];
}
export interface ProviderReadiness {
provider: string;
pinned_version: string;
/** True only when the pin has been reviewed AND the provider executed. */
verified: boolean;
/** What has actually been demonstrated. Plain words for an operator. */
proven: string[];
/** What has NOT. */
open: string[];
}
/**
* MetricFlow-shaped provider.
*
* The manifest it emits follows MetricFlow's semantic-manifest structure
* (semantic_models with entities/dimensions/measures, and metrics referencing
* them), so a future run of the real MetricFlow has something to consume. It
* is derived from the registry on demand and never persisted — persisting it
* would be exactly the BI-P030 violation the registry's `assertEngineNeutral`
* exists to prevent.
*/
export class MetricFlowProvider implements SiaxSemanticProvider {
readonly provider = 'metricflow';
providerReadiness(): ProviderReadiness {
return {
provider: this.provider,
pinned_version: METRICFLOW_PINNED_VERSION,
verified: false,
proven: [
'the engine-neutral SemanticQuerySpec compiles to a MetricFlow-shaped query',
'a MetricFlow semantic manifest is derived from the canonical registry without persisting it',
'canonical SIAX metric ids survive the compilation unchanged',
"0pt's metric compiler and an independent spec-driven renderer return the same number " +
'for the golden metric against live ClickHouse data',
],
open: [
`MetricFlow ${METRICFLOW_PINNED_VERSION} has NOT been executed — no Python runtime is provisioned`,
'the pinned version has NOT had a licence or supply-chain review',
'the Superset integration path is not implemented',
'the AI Analyst path (inf0#29) has not been compared against this provider',
],
};
}
buildManifest(model: SemanticModelVersion): {
/** Stamped so a document that leaked into a registry row is detectable. */
_generated_by: string;
_generated_from: { model_id: string; version: number; hash: string };
semantic_models: Array<Record<string, unknown>>;
metrics: Array<Record<string, unknown>>;
} {
const semantic_models = model.datasets.map((dataset) => {
const dims = model.dimensions.filter((d) => d.dataset_id === dataset);
const measures = model.metrics
.filter((m) => m.dimensions.some((dim) => dims.some((d) => d.name === dim)))
.map((m) => ({ name: m.measure, agg: mapAggregation(m.aggregation), expr: m.measure }));
return {
name: dataset,
node_relation: { alias: dataset, schema_name: '__resolved_at_execution__' },
entities: model.relationships
.filter((r) => r.from_dataset === dataset)
.map((r) => ({ name: r.from_field, type: r.kind === 'many_to_one' ? 'foreign' : 'primary' })),
dimensions: dims.map((d) => ({
name: d.name,
type: d.kind === 'time' ? 'time' : 'categorical',
...(d.kind === 'time' ? { type_params: { time_granularity: 'day' } } : {}),
})),
measures: dedupeBy(measures, (m) => m.name),
};
});
const metrics = model.metrics.map((m) => ({
// The CANONICAL id is what a consumer holds. The provider's own name is
// derived from it, never the other way round.
name: m.id,
description: m.description,
type: mapMetricType(m.type),
type_params: metricTypeParams(m),
filter: m.filters.length ? { where_filters: m.filters.map(structuredFilter) } : undefined,
meta: { siax_metric_id: m.id, siax_metric_version: m.version, unit: m.unit, currency: m.currency },
}));
return {
_generated_by: `${this.provider}@${METRICFLOW_PINNED_VERSION}`,
_generated_from: { model_id: model.model_id, version: model.version, hash: model.hash },
semantic_models,
metrics,
};
}
compile(spec: SemanticQuerySpec, model: SemanticModelVersion): CompiledSemanticQuery {
const known = new Map(model.metrics.map((m) => [m.id, m]));
for (const ref of spec.metrics) {
if (!known.has(ref.metric_id)) {
throw new SiaxError(
'INVALID_REQUEST',
'0pt.analytics.semantic',
`metric "${ref.metric_id}" is not in semantic model ${model.model_id} v${model.version}`,
);
}
}
const timeBucket = spec.groupBy.find((g) => g.grain);
return {
provider: this.provider,
query: {
metrics: spec.metrics.map((m) => m.metric_id),
group_by: spec.groupBy.filter((g) => !g.grain).map((g) => g.name),
...(timeBucket ? { time_granularity: timeBucket.grain } : {}),
// STRUCTURED, not a `--where` SQL string. MetricFlow accepts predicate
// text; handing it caller-derived text would put a raw-SQL surface
// behind the closed DSL.
where: spec.where.map((w) => ({ dimension: w.name, operator: w.operator, values: w.values })),
order_by: spec.orderBy.map((o) => ({ name: o.ref, descending: o.direction === 'desc' })),
limit: spec.limit,
...(spec.timeConstraint
? { start_time: spec.timeConstraint.start, end_time: spec.timeConstraint.end }
: {}),
},
canonical_metric_ids: spec.metrics.map((m) => m.metric_id).sort(),
};
}
providerBindings(model: SemanticModelVersion): Record<string, string> {
// Identity for MetricFlow, because its metric names can be the canonical
// ids. A provider whose naming rules differ (Cube's cube.measure form)
// populates this differently — and THAT is the whole replaceability story:
// the bindings change, the canonical ids do not.
return Object.fromEntries(model.metrics.map((m) => [m.id, m.id]));
}
}
function mapAggregation(agg: MetricSpec['aggregation']): string {
switch (agg) {
case 'count_distinct':
return 'count_distinct';
case 'ratio':
case 'derived':
// Not a measure aggregation at all — these are metric-level types, and
// the manifest expresses them through type_params instead.
return 'sum';
default:
return agg;
}
}
function mapMetricType(type: MetricSpec['type']): string {
switch (type) {
case 'ratio':
return 'ratio';
case 'derived':
return 'derived';
case 'cumulative':
return 'cumulative';
case 'conversion':
return 'conversion';
default:
return 'simple';
}
}
function metricTypeParams(m: MetricSpec): Record<string, unknown> {
if (m.type === 'ratio') {
return { numerator: m.measure, denominator: m.denominator };
}
return { measure: { name: m.measure } };
}
function structuredFilter(f: MetricSpec['filters'][number]): Record<string, unknown> {
return { dimension: f.dimension, operator: f.operator, values: f.values ?? [] };
}
function dedupeBy<T>(items: T[], key: (t: T) => string): T[] {
const seen = new Set<string>();
return items.filter((i) => {
const k = key(i);
if (seen.has(k)) return false;
seen.add(k);
return true;
});
}
+316
View File
@@ -0,0 +1,316 @@
/**
* src/analytics/metrics/semantic-registry.ts
* ===========================================
* BI-P030 — SEMANTIC MODEL REGISTRY.
*
* "Persist semantic model version SEPARATELY FROM IMPLEMENTATION ENGINE.
* DoD: MetricFlow can be replaced without breaking canonical SIAX metric IDs."
*
* Which means exactly one thing in code, and it is a negative: NOTHING in this
* file may name MetricFlow, Cube, dbt, ClickHouse, Trino or DuckDB. A registry
* that stores a MetricFlow manifest is a MetricFlow registry with a neutral
* name on it. `assertEngineNeutral` enforces the negative at runtime by
* scanning a serialized version for engine vocabulary, and the test asserts it
* catches a manifest that smuggles one in — because the rule is exactly the
* kind that decays silently.
*
* The canonical SIAX metric id is the contract. A provider maps it to whatever
* its own model calls that metric (BI-P029's `providerBindings`), and swapping
* providers changes the BINDINGS, never the ids — so a dashboard, an API
* consumer or a saved query keeps working.
*/
import crypto from 'node:crypto';
import { SiaxError } from '../../platform/errors';
import type { MetricSpec } from './metric-model';
export const SEMANTIC_MODEL_STATUSES = ['draft', 'active', 'superseded', 'retired'] as const;
export type SemanticModelStatus = (typeof SEMANTIC_MODEL_STATUSES)[number];
export const RELATIONSHIP_KINDS = ['one_to_many', 'many_to_one', 'one_to_one'] as const;
export type RelationshipKind = (typeof RELATIONSHIP_KINDS)[number];
export interface SemanticRelationship {
from_dataset: string;
from_field: string;
to_dataset: string;
to_field: string;
kind: RelationshipKind;
}
export interface SemanticDimension {
id: string;
name: string;
dataset_id: string;
/** "categorical" | "time". A time dimension is what grains apply to. */
kind: 'categorical' | 'time';
}
/** BI-P030's fields, verbatim. */
export interface SemanticModelVersion {
model_id: string;
version: number;
tenant_id: string;
datasets: string[];
relationships: SemanticRelationship[];
/** Canonical SIAX metric ids and their specs at this version. */
metrics: MetricSpec[];
dimensions: SemanticDimension[];
valid_from: string;
status: SemanticModelStatus;
hash: string;
}
/**
* Engine vocabulary that must never appear in a persisted semantic model.
*
* Checked against the SERIALIZED version, so it catches an engine name hiding
* in a nested field somebody added later — the failure mode a type cannot
* prevent.
*/
const ENGINE_VOCABULARY = [
'metricflow',
'clickhouse',
'trino',
'duckdb',
'cube.js',
'cubejs',
'dbt_metric',
'superset',
'presto',
'bigquery',
'snowflake',
];
/** The only top-level keys a persisted semantic model may carry. */
export const SEMANTIC_MODEL_KEYS = [
'model_id',
'version',
'tenant_id',
'datasets',
'relationships',
'metrics',
'dimensions',
'valid_from',
'status',
'hash',
] as const;
/**
* The STRUCTURAL half of engine neutrality, and the load-bearing one.
*
* `assertEngineNeutral` below is a VOCABULARY scan: it catches a model that
* says "metricflow" or "clickhouse". It cannot catch a document that is
* structurally MetricFlow-shaped while avoiding the word — a limitation found
* by a test that expected it to and did not. So publishing also rejects any
* key outside the BI-P030 field list, which means a provider manifest cannot
* be stored in a registry row at all, whatever it calls itself.
*/
export function assertOnlySemanticModelFields(model: object): void {
for (const key of Object.keys(model)) {
if (!(SEMANTIC_MODEL_KEYS as readonly string[]).includes(key)) {
throw new SiaxError(
'INVALID_REQUEST',
'0pt.analytics.semantic',
`semantic model carries field "${key}", which is not part of BI-P030's model — ` +
`provider manifests are DERIVED on demand, never persisted`,
);
}
}
}
export function assertEngineNeutral(model: Omit<SemanticModelVersion, 'hash'>): void {
const serialized = JSON.stringify(model).toLowerCase();
for (const term of ENGINE_VOCABULARY) {
if (serialized.includes(term)) {
throw new SiaxError(
'INVALID_REQUEST',
'0pt.analytics.semantic',
`semantic model "${model.model_id}" mentions "${term}" — BI-P030 requires the model to be ` +
`persisted separately from any implementation engine, so it can be replaced without ` +
`breaking canonical SIAX metric ids`,
);
}
}
}
/**
* The version hash. Covers everything that defines the model's MEANING;
* `valid_from` and `status` are excluded because activating a model does not
* change what it says.
*/
export function semanticModelHash(
model: Omit<SemanticModelVersion, 'hash' | 'valid_from' | 'status'>,
): string {
const canonical = JSON.stringify([
model.model_id,
model.version,
model.tenant_id,
[...model.datasets].sort(),
[...model.relationships]
.map((r) => [r.from_dataset, r.from_field, r.to_dataset, r.to_field, r.kind])
.sort((a, b) => JSON.stringify(a).localeCompare(JSON.stringify(b))),
[...model.metrics].map((m) => [m.id, m.version]).sort((a, b) => String(a[0]).localeCompare(String(b[0]))),
[...model.dimensions].map((d) => [d.id, d.name, d.dataset_id, d.kind]).sort(),
]);
return crypto.createHash('sha256').update(canonical).digest('hex');
}
export interface SemanticModelStore {
put(model: SemanticModelVersion): Promise<void>;
get(tenantId: string, modelId: string, version: number): Promise<SemanticModelVersion | undefined>;
active(tenantId: string, modelId: string): Promise<SemanticModelVersion | undefined>;
versions(tenantId: string, modelId: string): Promise<SemanticModelVersion[]>;
}
export class InMemorySemanticModelStore implements SemanticModelStore {
private readonly rows = new Map<string, SemanticModelVersion>();
private key(t: string, m: string, v: number): string {
return `${t}::${m}::${v}`;
}
async put(model: SemanticModelVersion): Promise<void> {
this.rows.set(this.key(model.tenant_id, model.model_id, model.version), model);
}
async get(t: string, m: string, v: number): Promise<SemanticModelVersion | undefined> {
return this.rows.get(this.key(t, m, v));
}
async versions(t: string, m: string): Promise<SemanticModelVersion[]> {
return [...this.rows.values()]
.filter((r) => r.tenant_id === t && r.model_id === m)
.sort((a, b) => a.version - b.version);
}
async active(t: string, m: string): Promise<SemanticModelVersion | undefined> {
return (await this.versions(t, m)).find((r) => r.status === 'active');
}
}
export class SemanticModelRegistry {
constructor(private readonly store: SemanticModelStore) {}
/**
* Publish a new version. Versions are IMMUTABLE — re-publishing an existing
* (model, version) is a CONFLICT, not an update, because a dashboard pinned
* to version 7 must mean the same thing tomorrow.
*/
async publish(input: Omit<SemanticModelVersion, 'hash'>): Promise<SemanticModelVersion> {
if (!input.tenant_id?.trim()) {
throw new SiaxError(
'UNAUTHENTICATED',
'0pt.analytics.semantic',
'a semantic model requires a verified tenant',
);
}
if (input.version < 1 || !Number.isInteger(input.version)) {
throw new SiaxError('INVALID_REQUEST', '0pt.analytics.semantic', 'version must be a positive integer');
}
assertOnlySemanticModelFields(input);
assertEngineNeutral(input);
this.assertReferentialIntegrity(input);
const existing = await this.store.get(input.tenant_id, input.model_id, input.version);
if (existing) {
throw new SiaxError(
'CONFLICT',
'0pt.analytics.semantic',
`semantic model ${input.model_id} v${input.version} already exists and is immutable`,
);
}
const model: SemanticModelVersion = { ...input, hash: semanticModelHash(input) };
await this.store.put(model);
return model;
}
/** Activate a version, superseding the previous active one. */
async activate(
tenantId: string,
modelId: string,
version: number,
at: string,
): Promise<SemanticModelVersion> {
const target = await this.store.get(tenantId, modelId, version);
if (!target)
throw new SiaxError('NOT_FOUND', '0pt.analytics.semantic', 'semantic model version not found');
if (target.status === 'retired') {
throw new SiaxError('CONFLICT', '0pt.analytics.semantic', 'a retired version cannot be re-activated');
}
const current = await this.store.active(tenantId, modelId);
if (current && current.version !== version) {
await this.store.put({ ...current, status: 'superseded' });
}
const activated = { ...target, status: 'active' as const, valid_from: at };
await this.store.put(activated);
return activated;
}
/**
* BI-P030's DoD, as a query: the canonical metric ids a model exposes.
* The engine-parity test asserts this set is identical before and after a
* provider swap.
*/
async canonicalMetricIds(tenantId: string, modelId: string, version: number): Promise<string[]> {
const model = await this.store.get(tenantId, modelId, version);
if (!model)
throw new SiaxError('NOT_FOUND', '0pt.analytics.semantic', 'semantic model version not found');
return model.metrics.map((m) => m.id).sort();
}
/**
* Every dimension and dataset a metric names must exist in the model.
*
* A model that passes validation while referencing a dimension nobody
* defined produces a metric that fails at query time, per tenant, in
* production — the worst place to find out.
*/
private assertReferentialIntegrity(model: Omit<SemanticModelVersion, 'hash'>): void {
const datasets = new Set(model.datasets);
const dimensionNames = new Set(model.dimensions.map((d) => d.name));
const timeDimensions = new Set(model.dimensions.filter((d) => d.kind === 'time').map((d) => d.name));
for (const d of model.dimensions) {
if (!datasets.has(d.dataset_id)) {
throw new SiaxError(
'INVALID_REQUEST',
'0pt.analytics.semantic',
`dimension "${d.name}" belongs to dataset "${d.dataset_id}", which the model does not include`,
);
}
}
for (const r of model.relationships) {
for (const ds of [r.from_dataset, r.to_dataset]) {
if (!datasets.has(ds)) {
throw new SiaxError(
'INVALID_REQUEST',
'0pt.analytics.semantic',
`relationship references dataset "${ds}", which the model does not include`,
);
}
}
}
const seenIds = new Set<string>();
for (const m of model.metrics) {
if (seenIds.has(m.id)) {
throw new SiaxError(
'CONFLICT',
'0pt.analytics.semantic',
`metric id "${m.id}" appears twice in the model`,
);
}
seenIds.add(m.id);
for (const dim of m.dimensions) {
if (!dimensionNames.has(dim)) {
throw new SiaxError(
'INVALID_REQUEST',
'0pt.analytics.semantic',
`metric "${m.name}" names dimension "${dim}", which the model does not define`,
);
}
}
if (!timeDimensions.has(m.timeDimension)) {
throw new SiaxError(
'INVALID_REQUEST',
'0pt.analytics.semantic',
`metric "${m.name}" names time dimension "${m.timeDimension}", which the model does not define as a time dimension`,
);
}
}
}
}
+483
View File
@@ -0,0 +1,483 @@
/**
* tests/analytics/metric-governance.test.ts
* ==========================================
* BI-P028 (metric domain model), BI-P030 (semantic model registry),
* BI-P031 (metric governance), BI-P029 (semantic provider — the parts not
* needing a Python runtime).
*
* Each prompt's DoD is turned into something that can FAIL:
*
* BI-P028 "No production metric exists only inside a dashboard tool."
* A dashboard panel carrying an inline aggregation / expression /
* SQL is REFUSED, and so is one naming an unregistered metric.
*
* BI-P030 "MetricFlow can be replaced without breaking canonical SIAX
* metric IDs." Two providers with different naming rules compile
* the same registry version; the canonical id set must be
* identical and the provider BINDINGS must differ — otherwise the
* test would pass for a registry that simply has one provider.
*
* BI-P031 "Certified metric cannot be modified in-place; new version
* required." Every semantic field is mutated one at a time and
* each must be refused; every editorial field is mutated and each
* must be allowed.
*/
import { describe, expect, it } from 'vitest';
import { SiaxError } from '@/src/platform/errors';
import {
assertMetricEditAllowed,
assertPanelUsesRegisteredMetric,
assertValidMetricSpec,
certifyMetric,
metricMeaningHash,
nextMetricVersion,
METRIC_STATUSES,
type GovernedMetric,
type MetricSpec,
} from '@/src/analytics/metrics/metric-model';
import {
assertEngineNeutral,
assertOnlySemanticModelFields,
InMemorySemanticModelStore,
SemanticModelRegistry,
semanticModelHash,
type SemanticModelVersion,
} from '@/src/analytics/metrics/semantic-registry';
import { MetricFlowProvider, type SiaxSemanticProvider } from '@/src/analytics/metrics/semantic-provider';
import { compileRequestToSemantic } from '@/src/analytics/contract/compile-semantic';
import {
parseAnalyticsQueryRequest,
queryResolutionFrom,
resolveAnalyticsQuery,
} from '@/src/analytics/contract/query-request';
const REVENUE: MetricSpec = {
id: 'm_revenue',
name: 'net_revenue',
description: 'Invoiced revenue net of credit notes, in SEK, recognised on invoice date.',
type: 'simple',
measure: 'amount_sek',
aggregation: 'sum',
dimensions: ['country', 'channel'],
filters: [{ dimension: 'channel', operator: 'ne', values: ['internal'] }],
timeDimension: 'invoiced_at',
unit: 'SEK',
currency: 'SEK',
format: 'currency',
owner: 'finance@siax.io',
version: 3,
status: 'ACTIVE',
};
const GOVERNED: GovernedMetric = {
spec: REVENUE,
governance: {
certification: 'certified',
business_definition: 'What we actually invoiced, after credits.',
technical_definition: "sum(amount_sek) on ds_sales where channel != 'internal', by invoiced_at.",
owner: 'finance@siax.io',
lineage_refs: ['ds_sales@v42'],
quality_status: 'passing',
},
};
describe('BI-P028 — a metric that cannot mean nothing', () => {
it('accepts the golden metric', () => {
expect(() => assertValidMetricSpec(REVENUE)).not.toThrow();
});
const rejections: Array<[string, Partial<MetricSpec>]> = [
['no description', { description: '' }],
['no owner', { owner: '' }],
['no time dimension', { timeDimension: '' }],
['a ratio with no denominator', { type: 'ratio' }],
['a denominator on a non-ratio', { denominator: 'orders' }],
['currency format with no currency', { currency: undefined }],
['a currency on a non-currency format', { format: 'integer' }],
['a non-ISO currency', { currency: 'kronor' as string }],
['a name that is not snake_case', { name: 'Net Revenue' }],
['a duplicate dimension', { dimensions: ['country', 'country'] }],
[
'a filter on an undeclared dimension',
{ filters: [{ dimension: 'segment', operator: 'eq', values: ['x'] }] },
],
['a value operator with no values', { filters: [{ dimension: 'channel', operator: 'eq', values: [] }] }],
[
'a presence operator with values',
{ filters: [{ dimension: 'channel', operator: 'is_null', values: ['x'] }] },
],
['version zero', { version: 0 }],
];
it.each(rejections)('refuses %s', (_label, override) => {
expect(() => assertValidMetricSpec({ ...REVENUE, ...override })).toThrowError(SiaxError);
});
it('declares the five lifecycle statuses BI-P028 names, in order', () => {
expect([...METRIC_STATUSES]).toEqual(['DRAFT', 'REVIEW', 'ACTIVE', 'DEPRECATED', 'RETIRED']);
});
});
describe('BI-P028 DoD — no production metric exists only inside a dashboard tool', () => {
const lookup = (id: string) => (id === 'm_revenue' ? GOVERNED : undefined);
it.each(['inline_expression', 'inline_aggregation', 'inline_sql'])(
'refuses a panel that defines a metric via %s',
(field) => {
expect(() =>
assertPanelUsesRegisteredMetric({ panel_id: 'p1', [field]: 'sum(amount)' }, lookup),
).toThrowError(SiaxError);
},
);
it('refuses a panel that names an unregistered metric', () => {
expect(() =>
assertPanelUsesRegisteredMetric({ panel_id: 'p1', metric_id: 'm_invented' }, lookup),
).toThrowError(SiaxError);
});
it('refuses a panel with no metric at all', () => {
expect(() => assertPanelUsesRegisteredMetric({ panel_id: 'p1' }, lookup)).toThrowError(SiaxError);
});
it('refuses a panel pointing at a RETIRED metric', () => {
const retired = { ...GOVERNED, spec: { ...REVENUE, status: 'RETIRED' as const } };
expect(() =>
assertPanelUsesRegisteredMetric({ panel_id: 'p1', metric_id: 'm_revenue' }, () => retired),
).toThrowError(SiaxError);
});
it('accepts a panel that references a registered, live metric', () => {
expect(assertPanelUsesRegisteredMetric({ panel_id: 'p1', metric_id: 'm_revenue' }, lookup).spec.id).toBe(
'm_revenue',
);
});
});
describe('BI-P031 DoD — a certified metric cannot be modified in place', () => {
/** Every field that changes what the number MEANS. */
const semanticEdits: Array<[string, Partial<MetricSpec>]> = [
['the measure', { measure: 'amount_eur' }],
['the aggregation', { aggregation: 'avg' }],
['the type', { type: 'ratio', denominator: 'orders' }],
['the dimensions', { dimensions: ['country'] }],
['the filters', { filters: [] }],
['the time dimension', { timeDimension: 'paid_at' }],
['the unit', { unit: 'EUR' }],
['the currency', { currency: 'EUR' }],
];
it.each(semanticEdits)('refuses an in-place change to %s', (_label, override) => {
const proposed = { ...REVENUE, ...override };
let thrown: unknown;
try {
assertMetricEditAllowed(GOVERNED, proposed);
} catch (e) {
thrown = e;
}
expect(thrown, "a certified metric's meaning was changed in place").toBeInstanceOf(SiaxError);
expect((thrown as SiaxError).code).toBe('CONFLICT');
expect((thrown as SiaxError).message).toContain('version 4');
});
/** Editorial fields stay editable — forcing a version bump for a typo is how teams route around governance. */
const editorialEdits: Array<[string, Partial<MetricSpec>]> = [
['the description', { description: 'Clearer wording.' }],
['the owner', { owner: 'controlling@siax.io' }],
['the lifecycle status', { status: 'DEPRECATED' }],
];
it.each(editorialEdits)('allows an in-place change to %s', (_label, override) => {
expect(() => assertMetricEditAllowed(GOVERNED, { ...REVENUE, ...override })).not.toThrow();
});
it('allows any change to an EXPERIMENTAL metric', () => {
const experimental = {
...GOVERNED,
governance: { ...GOVERNED.governance, certification: 'experimental' as const },
};
expect(() => assertMetricEditAllowed(experimental, { ...REVENUE, measure: 'anything' })).not.toThrow();
});
it('mints a new version that is DRAFT and EXPERIMENTAL, never inheriting the certification', () => {
const next = nextMetricVersion(
GOVERNED,
{ measure: 'amount_eur', unit: 'EUR', currency: 'EUR' },
'switch to EUR',
);
expect(next.spec.version).toBe(4);
expect(next.spec.status).toBe('DRAFT');
expect(next.governance.certification).toBe('experimental');
expect(next.governance.certified_at).toBeUndefined();
expect(next.change_reason).toBe('switch to EUR');
});
it('requires a change reason for a new version', () => {
expect(() => nextMetricVersion(GOVERNED, { measure: 'x' }, ' ')).toThrowError(SiaxError);
});
it('refuses to change the metric id under the guise of an edit', () => {
expect(() => assertMetricEditAllowed(GOVERNED, { ...REVENUE, id: 'm_other' })).toThrowError(SiaxError);
});
it('the meaning hash ignores editorial fields and catches semantic ones', () => {
expect(metricMeaningHash({ ...REVENUE, description: 'different', owner: 'x', status: 'DRAFT' })).toBe(
metricMeaningHash(REVENUE),
);
expect(metricMeaningHash({ ...REVENUE, measure: 'other' })).not.toBe(metricMeaningHash(REVENUE));
// Filter ORDER is not meaning; filter CONTENT is.
expect(
metricMeaningHash({
...REVENUE,
filters: [
{ dimension: 'country', operator: 'eq', values: ['SE', 'NO'] },
{ dimension: 'channel', operator: 'ne', values: ['internal'] },
],
}),
).not.toBe(metricMeaningHash(REVENUE));
});
it('certification is an act with an actor and a time, and needs both definitions', () => {
const draft: GovernedMetric = {
spec: { ...REVENUE, status: 'DRAFT' },
governance: { ...GOVERNED.governance, certification: 'experimental' },
};
expect(() => certifyMetric(draft, 'u_1', '2026-09-07T00:00:00Z')).toThrowError(SiaxError);
const active = { ...draft, spec: { ...draft.spec, status: 'ACTIVE' as const } };
const noDefs = { ...active, governance: { ...active.governance, technical_definition: '' } };
expect(() => certifyMetric(noDefs, 'u_1', '2026-09-07T00:00:00Z')).toThrowError(SiaxError);
const certified = certifyMetric(active, 'u_1', '2026-09-07T00:00:00Z');
expect(certified.governance.certification).toBe('certified');
expect(certified.governance.certified_by).toBe('u_1');
});
});
// ── BI-P030 ────────────────────────────────────────────────────────────────
function model(version = 1): Omit<SemanticModelVersion, 'hash'> {
return {
model_id: 'sm_sales',
version,
tenant_id: 't_alpha',
datasets: ['ds_sales'],
relationships: [],
metrics: [REVENUE],
dimensions: [
{ id: 'd_country', name: 'country', dataset_id: 'ds_sales', kind: 'categorical' },
{ id: 'd_channel', name: 'channel', dataset_id: 'ds_sales', kind: 'categorical' },
{ id: 'd_invoiced', name: 'invoiced_at', dataset_id: 'ds_sales', kind: 'time' },
],
valid_from: '2026-09-01T00:00:00Z',
status: 'draft',
};
}
describe('BI-P030 — the registry is engine-neutral and its versions are immutable', () => {
it('refuses a model that names an implementation engine anywhere in it', async () => {
for (const smuggled of [
{ model_id: 'sm_metricflow_sales' },
{ datasets: ['clickhouse_sales'] },
{ metrics: [{ ...REVENUE, description: 'computed by MetricFlow' }] },
]) {
expect(() =>
assertEngineNeutral({ ...model(), ...smuggled } as Omit<SemanticModelVersion, 'hash'>),
).toThrowError(SiaxError);
}
});
it('refuses a metric naming a dimension the model does not define', async () => {
const registry = new SemanticModelRegistry(new InMemorySemanticModelStore());
await expect(
registry.publish({ ...model(), metrics: [{ ...REVENUE, dimensions: ['country', 'segment'] }] }),
).rejects.toBeInstanceOf(SiaxError);
});
it('refuses a metric whose time dimension is not declared as a time dimension', async () => {
const registry = new SemanticModelRegistry(new InMemorySemanticModelStore());
await expect(
registry.publish({ ...model(), metrics: [{ ...REVENUE, timeDimension: 'country' }] }),
).rejects.toBeInstanceOf(SiaxError);
});
it('refuses to republish an existing version — a pinned dashboard must mean the same thing tomorrow', async () => {
const registry = new SemanticModelRegistry(new InMemorySemanticModelStore());
await registry.publish(model(1));
await expect(registry.publish(model(1))).rejects.toBeInstanceOf(SiaxError);
});
it('supersedes the previous active version on activation', async () => {
const store = new InMemorySemanticModelStore();
const registry = new SemanticModelRegistry(store);
await registry.publish(model(1));
await registry.publish(model(2));
await registry.activate('t_alpha', 'sm_sales', 1, '2026-09-01T00:00:00Z');
await registry.activate('t_alpha', 'sm_sales', 2, '2026-09-05T00:00:00Z');
expect((await store.get('t_alpha', 'sm_sales', 1))?.status).toBe('superseded');
expect((await store.active('t_alpha', 'sm_sales'))?.version).toBe(2);
});
it('the hash covers meaning and ignores activation', () => {
const a = semanticModelHash(model(1));
const b = semanticModelHash({
...model(1),
valid_from: '2030-01-01T00:00:00Z',
status: 'active',
} as never);
expect(a).toBe(b);
const c = semanticModelHash({ ...model(1), metrics: [{ ...REVENUE, version: 4 }] });
expect(c).not.toBe(a);
});
it('requires a verified tenant', async () => {
const registry = new SemanticModelRegistry(new InMemorySemanticModelStore());
await expect(registry.publish({ ...model(), tenant_id: '' })).rejects.toBeInstanceOf(SiaxError);
});
});
describe('BI-P030 DoD — the semantic engine can be replaced without breaking metric IDs', () => {
/**
* A SECOND provider with DIFFERENT naming rules, so the test cannot pass
* trivially for a registry that only ever had one provider. Cube-shaped:
* metric names are `cube.measure`, which is exactly the kind of difference
* that would break a consumer if canonical ids were not the contract.
*/
class CubeShapedProvider implements SiaxSemanticProvider {
readonly provider = 'cube-shaped';
buildManifest(m: SemanticModelVersion) {
return { cubes: m.datasets.map((d) => ({ name: d, measures: m.metrics.map((x) => x.measure) })) };
}
compile(spec: ReturnType<typeof compileRequestToSemantic>, m: SemanticModelVersion) {
const bindings = this.providerBindings(m);
return {
provider: this.provider,
query: { measures: spec.metrics.map((x) => bindings[x.metric_id]) },
canonical_metric_ids: spec.metrics.map((x) => x.metric_id).sort(),
};
}
providerBindings(m: SemanticModelVersion): Record<string, string> {
return Object.fromEntries(m.metrics.map((x) => [x.id, `ds_sales.${x.measure}`]));
}
}
it('both providers preserve the canonical ids while naming them differently', async () => {
const registry = new SemanticModelRegistry(new InMemorySemanticModelStore());
const published = await registry.publish(model(1));
const resolution = queryResolutionFrom({
dataset_id: 'ds_sales',
metrics: [
{
name: 'net_revenue',
metric_id: 'm_revenue',
event_metric_key: 'amount_sek',
aggregation: 'sum',
allowed_dimensions: ['country', 'channel'],
},
],
dimensions: [{ name: 'country', dimension_id: 'd_country', field_name: 'country_code' }],
semantic_model_version: 1,
});
const spec = compileRequestToSemantic(
resolveAnalyticsQuery(
parseAnalyticsQueryRequest({
dataset: 'ds_sales',
metrics: ['net_revenue'],
dimensions: ['country'],
}),
resolution,
),
);
const mf = new MetricFlowProvider();
const cube = new CubeShapedProvider();
expect(mf.compile(spec, published).canonical_metric_ids).toEqual(['m_revenue']);
expect(cube.compile(spec, published).canonical_metric_ids).toEqual(['m_revenue']);
expect(await registry.canonicalMetricIds('t_alpha', 'sm_sales', 1)).toEqual(['m_revenue']);
// ...and the providers genuinely differ, so the equality above is not
// trivially true.
expect(mf.providerBindings(published)).not.toEqual(cube.providerBindings(published));
expect(cube.providerBindings(published).m_revenue).toBe('ds_sales.amount_sek');
});
it('the MetricFlow manifest is DERIVED, never persisted into the registry', async () => {
const registry = new SemanticModelRegistry(new InMemorySemanticModelStore());
const published = await registry.publish(model(1));
const manifest = new MetricFlowProvider().buildManifest(published);
expect(manifest.metrics[0].name).toBe('m_revenue');
expect(manifest._generated_from.hash).toBe(published.hash);
// Two independent reasons a registry row cannot become a provider manifest.
// The vocabulary scan catches the generator stamp...
expect(() => assertEngineNeutral({ ...published, ...(manifest as object) } as never)).toThrowError(
SiaxError,
);
// ...and the structural check catches ANY extra field, which is what holds
// for a manifest that avoids the word. The first version of this test
// asserted only the vocabulary scan and passed for the wrong reason:
// buildManifest did not mention "metricflow" anywhere, so nothing fired.
expect(() => assertOnlySemanticModelFields({ ...published, semantic_models: [] })).toThrowError(
SiaxError,
);
});
it("publish refuses a model carrying any field outside BI-P030's list", async () => {
const registry = new SemanticModelRegistry(new InMemorySemanticModelStore());
await expect(registry.publish({ ...model(1), compiled_sql: 'SELECT 1' } as never)).rejects.toBeInstanceOf(
SiaxError,
);
});
it('refuses to compile a metric that is not in the model version', async () => {
const registry = new SemanticModelRegistry(new InMemorySemanticModelStore());
const published = await registry.publish(model(1));
expect(() =>
new MetricFlowProvider().compile(
{
dataset_id: 'ds_sales',
metrics: [{ metric_id: 'm_ghost', name: 'ghost' }],
groupBy: [],
where: [],
orderBy: [],
limit: 10,
format: 'json',
},
published,
),
).toThrowError(SiaxError);
});
it('puts no SQL predicate text into the compiled provider query', async () => {
const registry = new SemanticModelRegistry(new InMemorySemanticModelStore());
const published = await registry.publish(model(1));
const compiled = new MetricFlowProvider().compile(
{
dataset_id: 'ds_sales',
metrics: [{ metric_id: 'm_revenue', name: 'net_revenue' }],
groupBy: [],
where: [{ dimension_id: 'd_country', name: 'country', operator: 'eq', values: ["SE' OR '1'='1"] }],
orderBy: [],
limit: 10,
format: 'json',
},
published,
);
const serialized = JSON.stringify(compiled.query);
expect(serialized).toContain("SE' OR '1'='1"); // survives as DATA
expect(serialized).not.toMatch(/\bWHERE\b|\bSELECT\b/);
});
});
describe('BI-P029 — the provider states what it has NOT proven', () => {
it('reports itself unverified, and names the open surfaces', () => {
const readiness = new MetricFlowProvider().providerReadiness();
expect(readiness.verified).toBe(false);
expect(readiness.open.join(' ')).toContain('NOT been executed');
expect(readiness.open.join(' ')).toContain('Superset');
expect(readiness.open.join(' ')).toContain('AI Analyst');
expect(readiness.proven.length).toBeGreaterThan(0);
});
});