Files
0pt/tests/p0/quality-bridge.test.ts
Simon AxelssonandClaude Sonnet 5 9740e20818
CI (SIAX Cloud) / security (push) Successful in 23s
Security Scan (SIAX Cloud) / security (push) Successful in 22s
CI (SIAX Cloud) / integration (push) Successful in 29s
CI (SIAX Cloud) / quality (push) Successful in 45s
CI (SIAX Cloud) / security (pull_request) Successful in 27s
CI (SIAX Cloud) / integration (pull_request) Successful in 31s
Security Scan (SIAX Cloud) / security (pull_request) Successful in 29s
CI (SIAX Cloud) / quality (pull_request) Successful in 48s
feat(data-os): TS/Python quality bridge + provenance fields + fault-injection tests
Continues P010 (branch renamed from feat/p010-source-registry-persistence):
closes the three remaining BEFORE_STATE.md gaps (§2 TS/Python split-brain,
§3 missing provenance fields, §4 untested failure modes), all additive to
the production pipeline — the Python QA filter's delivery decision is
unchanged by any of this.

TS/Python bridge (§2):
  - New route surface app/api/0pt/quality/* (rules/runs/issues CRUD +
    POST /api/0pt/quality/bridge-run) over the shared QualityService
    singleton (src/services/quality.ts), fixing the same "no shared
    instance" defect the Source Registry had.
  - src/quality/quality-bridge.ts auto-provisions a deterministic,
    tenant-scoped baseline rule set on first use so the Python engine never
    pre-registers rules through the CRUD surface.
  - services/api/workers/quality_bridge.py: the Python-side HTTP client.
    Feature-flagged off by default (OPT_QUALITY_BRIDGE_ENABLED), fails open
    on any error (never raises into run_job, never changes `delivered`).
    Advisory only — real per-row gating against the TS engine isn't
    implementable yet (ExpectationEngine returns dataset-level pass/fail
    counts + samples, not a per-row boolean array); documented, not faked.
  - Live-verified end-to-end: `next dev` (AUTH_ENFORCE=false) + the real
    Python pipeline hitting the real brreg.no API, round-tripping through
    the real bridge route, receiving a real QualityService score.

Provenance (§3): sources/connectors.py's Provenance dataclass gains
retrieved_at, transform, connector_version, producer, quality,
entity_resolution, artifact_ref (all previously absent from the real
dataflow entirely) via a make_provenance() factory used by every connector.
resolve.py stamps entity_resolution on merged records' provenance (never on
unmerged ones). pipeline.py stamps artifact_ref at delivery and quality
once the bridge succeeds.

Failure tests (§4), real fault injection against real code (services/api,
new pytest + respx suite, not vitest mocks):
  - dedup-determinism + duplicate-input + out-of-order: resolve.py under
    shuffled/interleaved/exhaustive-permutation input, plus an explicit,
    verified test of the one case that IS legitimately order-sensitive
    (non-transitive similarity in a greedy clusterer) instead of hiding it.
  - schema-evolution: respx-mocked upstream responses with added/removed/
    alien-shaped fields through the real brreg connector + pipeline.
  - restart: connector exceptions mid-job (including one from respx itself)
    never crash run_job.
  - backfill: the real property that holds given no persistence layer
    exists (replay is compute-idempotent) — NOT a fabricated dedup-store
    test.
  - CDC-interruption and true backfill/delivery-store idempotency:
    explicitly documented as not implementable against this codebase
    (no CDC pipeline, no delivery store exist anywhere) via skip markers
    with reasons, in tests/test_known_gaps.py, rather than silently
    dropped or faked.

Testing: 256/256 vitest (249 pre-existing + 7 new bridge route/service
tests) and 28/28 pytest (2 documented skips) green locally. tsc -p
tsconfig.check.json has zero new errors (14 pre-existing errors, all in
the untracked/unaudited src/fleet, src/graph, src/metrics, src/projections
paths this pass does not touch, per BEFORE_STATE.md).

No PR opened, per instructions.

Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>
2026-09-01 21:02:14 +02:00

128 lines
5.6 KiB
TypeScript

/**
* P010: the TS/Python split-brain bridge (BEFORE_STATE.md §2 — the
* production Python pipeline and the sophisticated TS QualityService had
* "noll bryggor mellan sig"). Covers both the pure bridge helper
* (src/quality/quality-bridge.ts) and the actual HTTP route
* (app/api/0pt/quality/bridge-run/route.ts) the Python engine calls, so a
* regression here is caught even if only the route wiring breaks.
*/
import { describe, it, expect } from "vitest";
import { NextRequest } from "next/server";
import { QualityService } from "@/src/quality/quality-service";
import { ensureBaselineRules } from "@/src/quality/quality-bridge";
import { POST as bridgeRunPOST } from "@/app/api/0pt/quality/bridge-run/route";
function bridgeRequest(body: unknown): NextRequest {
return new NextRequest("http://localhost/api/0pt/quality/bridge-run", {
method: "POST",
body: JSON.stringify(body),
headers: { "content-type": "application/json" },
});
}
describe("ensureBaselineRules — the auto-provisioning half of the bridge", () => {
it("creates the baseline rule set once, and is idempotent on repeat calls", async () => {
const q = new QualityService();
const target = { tenant_id: "t_a", project_id: "p_1", dataset_id: "ds_order_42" };
await ensureBaselineRules(q, target);
const firstPass = q.listRules({ tenant_id: "t_a", dataset_id: "ds_order_42" });
expect(firstPass).toHaveLength(2);
expect(firstPass.every((r) => r.supplied_by === "0pt-quality-bridge")).toBe(true);
// Calling again must NOT throw CONFLICT or create duplicates/new versions.
await ensureBaselineRules(q, target);
const secondPass = q.listRules({ tenant_id: "t_a", dataset_id: "ds_order_42" });
expect(secondPass).toHaveLength(2);
expect(secondPass.map((r) => r.version)).toEqual([1, 1]);
});
it("scopes baseline rule ids per tenant so two tenants never collide (CONFLICT)", async () => {
const q = new QualityService();
await ensureBaselineRules(q, { tenant_id: "t_a", project_id: "p_1", dataset_id: "ds_shared_name" });
// Same dataset_id, different tenant — must not throw CONFLICT on the global rule_id map.
await expect(
ensureBaselineRules(q, { tenant_id: "t_b", project_id: "p_1", dataset_id: "ds_shared_name" })
).resolves.toBeUndefined();
expect(q.listRules({ tenant_id: "t_a", dataset_id: "ds_shared_name" })).toHaveLength(2);
expect(q.listRules({ tenant_id: "t_b", dataset_id: "ds_shared_name" })).toHaveLength(2);
});
it("never overwrites a rule a caller already customized under the same id pattern", async () => {
const q = new QualityService();
const target = { tenant_id: "t_a", project_id: "p_1", dataset_id: "ds_x" };
await ensureBaselineRules(q, target);
const before = q.getRule("baseline_name_notnull__t_a__ds_x", "t_a");
await q.updateRule("baseline_name_notnull__t_a__ds_x", "t_a", { severity: "critical" });
await ensureBaselineRules(q, target); // must not clobber the customization
const after = q.getRule("baseline_name_notnull__t_a__ds_x", "t_a");
expect(after.severity).toBe("critical");
expect(after.version).toBe(before.version + 1);
});
});
describe("POST /api/0pt/quality/bridge-run — the actual contract the Python engine calls", () => {
it("grades delivered rows for real: a row missing `name` fails the baseline and opens an issue", async () => {
const res = await bridgeRunPOST(
bridgeRequest({
tenant_id: "t_bridge",
project_id: "p_1",
dataset_id: "ds_order_1",
rows: [
{ name: "Volvo Personvagnar AB", confidence: 0.97 },
{ name: null, confidence: 0.4 },
],
})
);
expect(res.status).toBe(201);
const body = await res.json();
expect(body.run.status).toBe("failed");
expect(body.run.score).toBeGreaterThan(0);
expect(body.run.score).toBeLessThan(100);
expect(body.issues.length).toBeGreaterThan(0);
expect(body.issues.some((i: { rule_id: string }) => i.rule_id.startsWith("baseline_name_notnull__"))).toBe(true);
});
it("a clean batch of rows succeeds with score 100 and no open issues", async () => {
const res = await bridgeRunPOST(
bridgeRequest({
tenant_id: "t_bridge_clean",
project_id: "p_1",
dataset_id: "ds_order_2",
rows: [
{ name: "Ericsson AB", confidence: 0.95 },
{ name: "Atlas Copco AB", confidence: 0.88 },
],
})
);
expect(res.status).toBe(201);
const body = await res.json();
expect(body.run.status).toBe("succeeded");
expect(body.run.score).toBe(100);
expect(body.issues).toEqual([]);
});
it("rejects a malformed body with a structured 400, never a 500 (Python must be able to tell apart its own bug from a network fault)", async () => {
const res = await bridgeRunPOST(bridgeRequest({ tenant_id: "t_x" })); // missing project_id/dataset_id/rows
expect(res.status).toBe(400);
const body = await res.json();
expect(body.error).toBe("validation_error");
});
it("out-of-range confidence values (data drift from upstream) are graded, not crashed on", async () => {
const res = await bridgeRunPOST(
bridgeRequest({
tenant_id: "t_bridge_drift",
project_id: "p_1",
dataset_id: "ds_order_3",
rows: [{ name: "Weird AB", confidence: 42 }], // confidence way outside [0,1] — upstream bug simulation
})
);
expect(res.status).toBe(201);
const body = await res.json();
expect(body.run.status).toBe("failed");
expect(body.issues.some((i: { rule_id: string }) => i.rule_id.startsWith("baseline_confidence_range__"))).toBe(
true
);
});
});