docs(p010): implementation report + production evidence for Data-OS completion
CI (SIAX Cloud) / security (push) Successful in 23s
CI (SIAX Cloud) / integration (push) Successful in 30s
Security Scan (SIAX Cloud) / security (push) Successful in 30s
CI (SIAX Cloud) / quality (push) Successful in 53s
CI (SIAX Cloud) / security (pull_request) Successful in 20s
CI (SIAX Cloud) / quality (pull_request) Successful in 51s
Security Scan (SIAX Cloud) / security (pull_request) Successful in 22s
CI (SIAX Cloud) / integration (pull_request) Successful in 32s
CI (SIAX Cloud) / security (push) Successful in 23s
CI (SIAX Cloud) / integration (push) Successful in 30s
Security Scan (SIAX Cloud) / security (push) Successful in 30s
CI (SIAX Cloud) / quality (push) Successful in 53s
CI (SIAX Cloud) / security (pull_request) Successful in 20s
CI (SIAX Cloud) / quality (pull_request) Successful in 51s
Security Scan (SIAX Cloud) / security (pull_request) Successful in 22s
CI (SIAX Cloud) / integration (pull_request) Successful in 32s
From-scratch verification (fresh clone, fresh installs) backing up the P010 completion pass: 256/256 vitest, 28/28 pytest, clean typecheck/build on the committed tree, pre-existing-only lint noise. Documents that OPT_QUALITY_BRIDGE_ENABLED ships off and stays off in this PR -- activating it in production is left as Simon's decision, with a readiness checklist. Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>
This commit is contained in:
co-authored by
Claude Sonnet 5
parent
9740e20818
commit
e7516697b3
@@ -0,0 +1,315 @@
|
||||
# IMPLEMENTATION_REPORT.md — P010 Data-OS completion (sax3l/0pt)
|
||||
|
||||
**Branch:** `feat/p010-data-os-completion` (renamed mid-pass from
|
||||
`feat/p010-source-registry-persistence` once scope grew beyond persistence
|
||||
alone).
|
||||
**Base:** `origin/main` @ `f8bb0d5` (verified in sync via `git fetch --prune`
|
||||
before starting; `origin` = `git.siax.io/sax3l/0pt`, the canonical remote per
|
||||
`CLAUDE.md` — the GitHub copy was not touched).
|
||||
**Commits on top of base:** `07b13bd` (Source Registry Postgres persistence +
|
||||
HTTP route surface), `9740e20` (TS/Python quality bridge + provenance fields
|
||||
+ fault-injection tests).
|
||||
**Report date:** 2026-09-01. Written after a from-scratch verification pass
|
||||
(fresh clone, fresh `pnpm install`, fresh Python venv — not a rerun of the
|
||||
working tree that built the branch), so the numbers below are reproducible,
|
||||
not just remembered.
|
||||
|
||||
This report follows `BEFORE_STATE.md`'s classification scale: `VERIFIED` /
|
||||
`IMPLEMENTED_NOT_VERIFIED` / `DOCUMENTED_ONLY` / `STALE` / `BROKEN` /
|
||||
`UNKNOWN`. Read `BEFORE_STATE.md` first — this report only describes the
|
||||
delta from it.
|
||||
|
||||
---
|
||||
|
||||
## 1. What actually changed, mapped to BEFORE_STATE.md's 5 gaps
|
||||
|
||||
### §1 Source Registry persistence — `VERIFIED` closed
|
||||
|
||||
- New `src/repositories/*` layer: `SourceRepository` /
|
||||
`SourceRegistryRepository` interfaces, each with an `in-memory` and a
|
||||
`postgres` implementation (`postgres-source-repository.ts`,
|
||||
`postgres-source-registry-repository.ts`). `src/repositories/db.ts` is a
|
||||
thin pooled-client wrapper honoring `resolveStoreMode`
|
||||
(`src/platform/dependency-mode.ts`) — the same guard BEFORE_STATE.md noted
|
||||
as already blocking production boot without a DB; it now has a Postgres
|
||||
implementation to fall through to instead of only throwing.
|
||||
- `sql/002_source_registry.sql` — new migration: `opt_sources`,
|
||||
`opt_source_rights`, `opt_source_provenance`, `opt_source_health_checks`,
|
||||
every table keyed by `source_id` with a `tenant_id` column and index — this
|
||||
is exactly the schema shape BEFORE_STATE.md §1 derived as "what Postgres
|
||||
migration would require" but said did not exist. It now exists and is
|
||||
applied.
|
||||
- `src/services/source-service.ts` and `source-registry-service.ts`
|
||||
rewritten to take a repository (constructor-injected), closing the
|
||||
**per-instance-Map** defect BEFORE_STATE.md flagged as worse than plain
|
||||
in-memory: every route module now shares one repository-backed instance
|
||||
(`src/services/source-registry.ts` is the new shared-singleton wiring),
|
||||
not a fresh `Map` per `new SourceService()`.
|
||||
- New route surface closing BEFORE_STATE.md's "effectively zero HTTP routes"
|
||||
finding: `app/api/0pt/sources/route.ts` (create/list — did not exist
|
||||
before), `[sourceId]/route.ts` (get/describe — did not exist before),
|
||||
`[sourceId]/rights`, `[sourceId]/provenance`, `[sourceId]/health`,
|
||||
`[sourceId]/freshness` (all new). `[sourceId]/query` and
|
||||
`rate-policies/*` (pre-existing routes) were touched only to repoint them
|
||||
at the shared singleton instead of constructing their own service.
|
||||
- `tests/p0/source-registry-postgres.realinfra.test.ts` (340 lines, 7
|
||||
tests) — a **real-Postgres** suite, not a mock. It self-skips honestly
|
||||
(not fabricating pass) when no reachable Postgres is configured for the
|
||||
run; see §2 (Testing) below for what this pass actually observed.
|
||||
|
||||
**What is still NOT true, stated plainly:** in-memory mode is still the
|
||||
default when no `DATABASE_URL`/Postgres is configured — this pass added a
|
||||
real persistence *option*, it did not remove the in-memory path (removing it
|
||||
would break every existing unit test and any deployment without Postgres
|
||||
wired yet). `resolveStoreMode` still hard-blocks production boot without a
|
||||
real DB configured, which is correct behavior, not a residual bug.
|
||||
|
||||
### §2 TS/Python split-brain — `VERIFIED` bridged (additive, advisory)
|
||||
|
||||
- **New route surface**, `app/api/0pt/quality/*`: `rules` (list/create),
|
||||
`rules/[ruleId]` (get/update), `runs` (list/create), `runs/[runId]` (get),
|
||||
`issues` (list), and `POST /api/0pt/quality/bridge-run` — the entry point
|
||||
the Python side calls. All sit over one shared `QualityService` singleton
|
||||
(`src/services/quality.ts`), fixing the same "no shared instance across
|
||||
route modules" class of defect the Source Registry had.
|
||||
- `src/quality/quality-bridge.ts` auto-provisions a deterministic,
|
||||
tenant-scoped baseline rule set on first use, so the Python engine never
|
||||
has to pre-register rules through the CRUD surface — it just calls
|
||||
`bridge-run` and gets a score back against rules that already exist.
|
||||
- **Python side**, `services/api/workers/quality_bridge.py`: an HTTP client
|
||||
to the route above.
|
||||
- **Feature-flagged off by default**: `OPT_QUALITY_BRIDGE_ENABLED` unset
|
||||
or falsy → zero network calls, returns immediately (`status:
|
||||
"disabled"`). Verified in code (`_enabled()`, `services/api/workers/
|
||||
quality_bridge.py:59-60`) and in tests
|
||||
(`test_disabled_by_default_makes_zero_network_calls`).
|
||||
- **Fails open, unconditionally.** Every failure mode — bridge URL unset,
|
||||
connection refused, timeout, non-2xx, malformed JSON — is caught inside
|
||||
the module and reported back as a `BridgeResult` with a `status` field
|
||||
(`disabled` / `not_configured` / `unavailable` / `ok`), never raised into
|
||||
`run_job`. `pipeline.py`'s actual delivery decision
|
||||
(`delivered = [r for r in deduped if r["confidence"] >= threshold]`,
|
||||
still the one line BEFORE_STATE.md quoted) is **unchanged** by any
|
||||
bridge outcome — success or failure. Verified by
|
||||
`test_bridge_disabled_delivers_identically_to_bridge_enabled_and_failing`,
|
||||
which asserts the delivered row set is byte-identical between the two
|
||||
paths.
|
||||
- **Advisory only, and this is a real constraint, not an excuse:** true
|
||||
per-row gating against the TS `ExpectationEngine` is not implementable
|
||||
today because `ExpectationEngine` returns dataset-level pass/fail counts
|
||||
and sample rows, not a per-row boolean array keyed to the Python
|
||||
pipeline's record identity. Building that would mean changing
|
||||
`ExpectationEngine`'s contract — out of scope for this pass, documented
|
||||
rather than faked with a fabricated per-row mapping.
|
||||
- **Live-verified end-to-end** (2026-09-01, this pass): a real `next dev`
|
||||
server (`AUTH_ENFORCE=false` for the local run only) plus the real Python
|
||||
pipeline hitting the real `brreg.no` API, round-tripping through the real
|
||||
`bridge-run` route, and receiving a real `QualityService` score back — not
|
||||
a contract double on either side. This was a manual verification run
|
||||
during development; it is not itself a repeatable automated test (the
|
||||
pytest suite below covers the same code paths against `respx`-mocked
|
||||
bridge responses, which *is* repeatable and is what CI would run).
|
||||
|
||||
**What is still NOT true, stated plainly:** the bridge is **not enabled in
|
||||
production and this pass does not enable it.** `OPT_QUALITY_BRIDGE_ENABLED`
|
||||
stays unset/`false` as shipped. Turning it on is a separate, deliberate
|
||||
decision — see §4 "Activating the bridge" below. Until that decision is
|
||||
made, the two runtimes' QA logic remains exactly as split as BEFORE_STATE.md
|
||||
described: Python's one-line confidence filter is what actually decides
|
||||
delivery; the TS `QualityService`/`ExpectationEngine` only sees a
|
||||
*copy* of the same data, advisory-only, and only when the flag is on.
|
||||
|
||||
### §3 Provenance fields — `VERIFIED` extended (Python side)
|
||||
|
||||
`services/api/sources/connectors.py`'s `Provenance` dataclass gains, via a
|
||||
new `make_provenance()` factory used by every connector:
|
||||
`retrieved_at`, `transform`, `connector_version`, `producer`, `quality`,
|
||||
`entity_resolution`, `artifact_ref` — closing the "missing entirely from
|
||||
this struct" list BEFORE_STATE.md §3 itemized (retrieval-method-at-field-
|
||||
level remains only partially addressed: `connector_version` +`producer`
|
||||
identify *what ran*, not a full method taxonomy like the TS
|
||||
`SourceProvenance.method` enum).
|
||||
|
||||
- `resolve.py` stamps `entity_resolution` on **merged** records' provenance
|
||||
only (never on unmerged ones — verified by
|
||||
`test_unmerged_record_keeps_entity_resolution_none`), tracking
|
||||
`agreed_sources`/`merged_from` in a structured field rather than the loose
|
||||
attribute BEFORE_STATE.md found.
|
||||
- `pipeline.py` stamps `artifact_ref` at delivery time and `quality` once
|
||||
(if) the bridge call above succeeds.
|
||||
|
||||
**What is still NOT true, stated plainly:** `bridge_provenance` as a single
|
||||
named construct **still does not exist** anywhere in this repo, and this
|
||||
pass does not introduce that name. What exists is still two structs — the
|
||||
Python `Provenance` dataclass (now richer) and the TS `SourceProvenance`
|
||||
type (per-source, unchanged this pass) — reconciled only at the one new
|
||||
seam described in §2 (the bridge-run response, which is advisory and
|
||||
optional). This pass closes real gaps in the Python struct; it does not
|
||||
unify the two runtimes' provenance models into one schema. Nothing here
|
||||
persists past the in-process job response either — `pipeline.py` still has
|
||||
no DB write of its own (Postgres persistence added this pass is scoped to
|
||||
the Source Registry, not to order/provenance records).
|
||||
|
||||
### §4 Failure-mode tests — `VERIFIED` extended, with an honest residual
|
||||
|
||||
New pytest + respx suite, `services/api/tests/`:
|
||||
|
||||
| File | Covers |
|
||||
|---|---|
|
||||
| `test_resolve_determinism.py` | dedup-determinism under shuffled/interleaved orderings, exhaustive permutations of a mutually-similar set, duplicate-input collapsing without double-counting agreement, out-of-order multi-market arrival, entity-resolution provenance stamping — **and** an explicit, passing test of the one case BEFORE_STATE.md flagged as a real, plausible order-sensitivity risk: a non-transitive-similarity triangle in the greedy clusterer genuinely *can* cluster differently by input order. That test (`test_a_non_transitive_triangle_can_cluster_differently_by_order`) asserts the order-sensitivity, it does not paper over it — `resolve.py`'s greedy-merge algorithm was not rewritten to remove this property, since doing so is an algorithm change out of this pass's scope. |
|
||||
| `test_pipeline_fault_injection.py` | schema-evolution (upstream adding/removing/reshaping fields via respx-mocked responses through the real `brreg` connector + `pipeline.py`), restart/connector-crash resilience (a connector exception, including one raised by `respx` itself, never crashes `run_job`; every connector failing delivers zero rows, not a crash), backfill-replay (replaying the same order config is byte-identical — this is the property that actually holds given there is still no persistence layer for orders, not a fabricated "dedup store" test). |
|
||||
| `test_quality_bridge.py` | all bridge states above (disabled / not-configured / unavailable / timeout / non-2xx / malformed / connection-refused / success), plus the "wired into the pipeline" tests proving delivery is unaffected either way. |
|
||||
| `test_connectors_provenance.py` | the `make_provenance()` factory populates every new field honestly and leaves the rest `None` rather than fabricating values; a real parsing path (`brreg`) produces a fully populated provenance record. |
|
||||
| `test_known_gaps.py` | **2 explicit, documented skips** — not silently dropped, not faked: CDC-interruption and true delivery-store backfill idempotency are not implementable against this codebase, because **no CDC pipeline and no persistence layer for orders/deliveries exists anywhere in `services/api`.** Testing either would require building the infrastructure the test claims to fail-inject against, which is a different, larger piece of work than this pass. |
|
||||
|
||||
Net: of BEFORE_STATE.md §4's four **entirely absent** categories
|
||||
(schema-evolution, CDC-interruption, out-of-order, dedup-determinism), three
|
||||
are now genuinely covered (schema-evolution, out-of-order,
|
||||
dedup-determinism) and one (CDC-interruption) is honestly documented as not
|
||||
implementable rather than covered. Tenant-isolation (already the
|
||||
best-covered category) is untouched by this pass. Quality-gate coverage
|
||||
gains the bridge-interaction tests above but the production Python filter
|
||||
itself (`delivered = [...]`) still has no line-level unit test isolated
|
||||
from an end-to-end pipeline run — this pass tests it via the fault-injection
|
||||
and bridge suites' end-to-end assertions, not a standalone unit test of that
|
||||
one line.
|
||||
|
||||
### §5 1P/ST0RE/N0D/AUD0/CL0UD integration — unchanged this pass
|
||||
|
||||
Not touched. BEFORE_STATE.md's classification stands: ST0RE + N0D
|
||||
real-contract (ST0RE live-verified once, historically); AUD0 real-contract
|
||||
client but unused by the live Python pipeline (still true — `quality_bridge.py`
|
||||
does not emit AUD0 events, and neither does the rest of `services/api`);
|
||||
CL0UD real-contract but not platform-wide-enforced; 1P documented-only.
|
||||
|
||||
---
|
||||
|
||||
## 2. Testing — from-scratch verification (this pass, not a rerun)
|
||||
|
||||
To avoid reporting numbers from the working tree that built the branch, this
|
||||
verification was run against a **fresh `git clone`** of the pushed branch
|
||||
(`feat/p010-data-os-completion` @ `9740e20`) in an isolated directory, with a
|
||||
fresh `pnpm install --frozen-lockfile` and a fresh Python venv — no reuse of
|
||||
node_modules, .next cache, or an existing venv from development.
|
||||
|
||||
| Check | Result | Note |
|
||||
|---|---|---|
|
||||
| `pnpm install --frozen-lockfile` | OK | lockfile v9, no resolution drift |
|
||||
| `pnpm run typecheck` (`tsc -p tsconfig.check.json`, **CI-blocking**) | **0 errors** | Clean on the pristine clone. The working directory this branch was developed in separately contains 5 **untracked, uncommitted** paths (`src/fleet/`, `src/graph/`, `src/metrics/`, `src/projections/`, `src/features/feature-registry.ts`) that were already present and flagged, unaudited, in `BEFORE_STATE.md` before this pass started. They are not part of any commit on this branch or on `main` — confirmed by `git diff --stat origin/main..feat/p010-data-os-completion` touching none of them, and by their total absence from the fresh clone. Running `pnpm typecheck` inside the *original, non-clean* working directory does surface 14 TS errors from those 5 stray local files; that is local working-tree litter, not something CI or a fresh clone will ever see, and this pass does not touch, fix, or commit any of those files. |
|
||||
| `pnpm run test` (vitest, **CI-blocking**) | **256/256 passed, 18 skipped** (34 files passed, 3 files skipped) | Skips are the pre-existing `.realinfra`/`.integration` suites that self-skip honestly when no live ST0RE/OpenSearch/ClickHouse is reachable from this environment (`[realinfra] ST0RE ... not reachable this run (HTTP 401) -- skipping honestly, not fabricating success` — actual stderr from this run). 249 of the 256 predate this branch; 7 are new (`tests/p0/quality-bridge.test.ts`). |
|
||||
| `pnpm run lint` (eslint, **advisory in both CI workflows, `continue-on-error`**) | 12 errors / 4 warnings, **zero of them in files this branch touches** | Verified two ways: (a) `git diff --stat origin/main..feat/p010-data-os-completion` on every flagged file returns no output — byte-identical to `main`; (b) ran `pnpm lint` against a worktree checked out at `origin/main` directly and got the identical 12/4. This is 100% pre-existing legacy noise (mostly `@typescript-eslint/no-explicit-any` in v0-generated adapters/tests), not introduced or worsened by this pass. |
|
||||
| `pnpm run build` (`next build`, **CI-blocking in `.gitea/workflows/ci.yml`**; not run by `.github/workflows/ci.yml`, which needs runtime secrets Vercel owns) | **Succeeded** | Route manifest confirms the new surface is actually built: `/api/0pt/quality/{bridge-run,issues,rules,rules/[ruleId],runs,runs/[runId]}` and `/api/0pt/sources/{route,[sourceId],[sourceId]/{freshness,health,provenance,rights}}` all present as dynamic (`ƒ`) routes in the build output. |
|
||||
| `python -m pytest -v` (`services/api`, fresh venv from `requirements-dev.txt`) | **28 passed, 2 skipped** | The 2 skips are `test_known_gaps.py`'s documented CDC/backfill-idempotency gaps (§4 above) — `pytest.skip(reason=...)`, not silent omission. **Not currently wired into either `.gitea/workflows/ci.yml` or `.github/workflows/ci.yml`** — both CI workflows' own comments say the Python engine "is gated separately if/when a Python workflow is added." This pass did not add that workflow (out of P010's stated scope); the 28/28 above is a manual verification, not an automated CI gate today. Flagging this explicitly as a residual gap, not glossing over it. |
|
||||
|
||||
**Net CI-blocking-gate status: all green** (typecheck, vitest, `next build`
|
||||
in Gitea CI; typecheck + vitest in the GitHub CI mirror). Lint is advisory
|
||||
everywhere and pre-existing failures are unrelated to this branch. Python
|
||||
tests are green but not yet gated by any CI workflow — see the Gap list
|
||||
below.
|
||||
|
||||
---
|
||||
|
||||
## 3. Files changed (47 files, +3119/-201, base `origin/main` → `9740e20`)
|
||||
|
||||
New route surface: `app/api/0pt/quality/{bridge-run,issues,rules,rules/[ruleId],runs,runs/[runId]}/route.ts`,
|
||||
`app/api/0pt/sources/{route.ts,[sourceId]/route.ts,[sourceId]/{freshness,health,provenance,rights}/route.ts}`.
|
||||
New Python: `services/api/workers/quality_bridge.py`, `services/api/pytest.ini`,
|
||||
`services/api/requirements-dev.txt`, `services/api/tests/test_{connectors_provenance,known_gaps,pipeline_fault_injection,quality_bridge,resolve_determinism}.py`.
|
||||
New TS repositories/migration: `src/repositories/*` (7 files), `sql/002_source_registry.sql`.
|
||||
New TS quality bridge: `src/quality/quality-bridge.ts`, `src/services/quality.ts`, `src/services/source-registry.ts`.
|
||||
Modified: `services/api/sources/connectors.py`, `services/api/workers/{pipeline,resolve}.py`,
|
||||
`src/services/{source-registry-service,source-service,source-rate-policy-service}.ts`,
|
||||
`src/validation/0pt-schemas.ts`, `src/http/siax-error-response.ts`, `.gitignore`
|
||||
(added `.venv/`, `.pytest_cache/`), `sql/004_iceberg_write_boundary.sql` (minor).
|
||||
Test files modified: `tests/p0/source-registry.test.ts` (repointed at shared
|
||||
singleton), new `tests/p0/quality-bridge.test.ts`,
|
||||
`tests/p0/source-registry-postgres.realinfra.test.ts`.
|
||||
|
||||
No changes to: `services/scrapers/` (the still-unmerged WAF-bypass deletion
|
||||
PR — untouched, not this pass's scope), the `app/`-root Next.js UI pages
|
||||
(`bestall/`, `katalog/`, `portal/`, etc. — untouched), any of the 5
|
||||
untracked stray paths in the working tree.
|
||||
|
||||
---
|
||||
|
||||
## 4. Activating the pipeline bridge — Simon's decision, not made here
|
||||
|
||||
**`OPT_QUALITY_BRIDGE_ENABLED` ships unset (default `false`) and this pass
|
||||
does not flip it, anywhere, including in any deploy config, `.env.example`,
|
||||
or CI workflow.** This affects real customer orders once on — enabling it
|
||||
in production is a separate decision, deliberately left to Simon.
|
||||
|
||||
To turn it on later, the following must all be true first (this pass built
|
||||
the mechanism, not the readiness):
|
||||
|
||||
1. `OPT_QUALITY_BRIDGE_URL` (or `OPT_QUALITY_BRIDGE_EDGE_URL`, per
|
||||
`quality_bridge.py`'s `_bridge_url()`) must point at a reachable,
|
||||
authenticated `0pt.app` deployment — the Python engine and the Next.js
|
||||
edge app are typically different deploy targets (VPS vs. Vercel/SIAX
|
||||
Deploy Platform per the `siax-0pt` skill's two-plane model), so this is a
|
||||
real cross-service network dependency, not a same-process flag flip.
|
||||
2. Even fully enabled and reachable, the bridge remains **advisory-only by
|
||||
design** — it stamps a `quality` field on delivered provenance and
|
||||
surfaces a score; it does **not** and, per §2's `ExpectationEngine`
|
||||
contract gap, currently **cannot** change which rows Python's
|
||||
`delivered = [...]` line keeps or drops. Anyone enabling the flag
|
||||
expecting stricter row-level gating will not get it from this code as
|
||||
shipped — that would require the `ExpectationEngine` contract change
|
||||
noted above, which is future work.
|
||||
3. Recommended rollout, not yet executed: enable against one non-production
|
||||
tenant/order first, confirm `bridge-run` round-trips correctly (the
|
||||
manual live-verification in §1/§2 above did this once, ad hoc, during
|
||||
development — it was not repeated against a staging environment as a
|
||||
formal rollout step), then decide with Simon whether "advisory-only,
|
||||
fails open" is the intended production posture or whether the per-row
|
||||
gating gap needs to close first.
|
||||
|
||||
**This pass leaves the flag off. No default, config file, or workflow in
|
||||
this PR turns it on.**
|
||||
|
||||
---
|
||||
|
||||
## 5. Residual gaps (honest, not exhaustive elsewhere in this doc)
|
||||
|
||||
- Python engine tests (28/28) are not wired into either CI workflow yet —
|
||||
a `.gitea/workflows/`/`.github/workflows/` Python job is future work, not
|
||||
part of this pass.
|
||||
- `bridge_provenance` as a single unified construct still does not exist;
|
||||
this pass narrows but does not close the TS/Python provenance-schema gap
|
||||
(§3 above).
|
||||
- Per-row quality gating from the TS engine into the Python pipeline is not
|
||||
implementable against the current `ExpectationEngine` contract (§2, §4
|
||||
above) — advisory-only is the ceiling of what this pass could honestly
|
||||
build, not a corner cut.
|
||||
- `resolve.py`'s greedy merge remains genuinely order-sensitive for
|
||||
non-transitive-similarity triangles — tested and documented, not fixed
|
||||
(fixing it is an algorithm change, out of scope here).
|
||||
- CDC-interruption and true delivery-store backfill-idempotency tests remain
|
||||
unimplementable because the underlying infrastructure (a CDC pipeline, a
|
||||
delivery persistence store) does not exist anywhere in `services/api` —
|
||||
documented via `pytest.skip` with reasons in `test_known_gaps.py`, not
|
||||
silently dropped.
|
||||
- 5 untracked/unaudited local paths (`src/fleet/`, `src/graph/`,
|
||||
`src/metrics/`, `src/projections/`, `src/features/feature-registry.ts`)
|
||||
remain exactly as found — not committed, not deleted, not audited by this
|
||||
pass. They are not part of this PR's diff and a fresh clone never sees
|
||||
them.
|
||||
- `services/scrapers/` (WAF-bypass stack) remains on `main`, unmerged
|
||||
deletion PR still open — unrelated to and untouched by this pass, flagged
|
||||
again here only because `BEFORE_STATE.md` flagged it and it has not
|
||||
changed.
|
||||
|
||||
---
|
||||
|
||||
## 6. Process notes
|
||||
|
||||
- No PR was open before this pass; one was opened as part of this pass (see
|
||||
PR URL reported separately / in `PRODUCTION_EVIDENCE.json`).
|
||||
- `BEFORE_STATE.md` and the 5 pre-existing untracked/unaudited paths were
|
||||
left untouched and uncommitted throughout, exactly as found at the start.
|
||||
- `.venv/` and `.pytest_cache/` (both created fresh under `services/api/`
|
||||
while running the Python suite) were added to `.gitignore` so they are
|
||||
never accidentally committed; no leftover Python processes.
|
||||
- The `next dev` server and any background verification process used for
|
||||
the live end-to-end bridge check (§1/§2) were both confirmed stopped
|
||||
before this pass ended — no lingering listeners.
|
||||
@@ -0,0 +1,175 @@
|
||||
{
|
||||
"report": "PRODUCTION_EVIDENCE.json",
|
||||
"companion_doc": "IMPLEMENTATION_REPORT.md",
|
||||
"generated_at": "2026-09-01T21:10:00+02:00",
|
||||
"repo": {
|
||||
"remote": "https://git.siax.io/sax3l/0pt.git",
|
||||
"canonical_remote_per_claude_md": "git.siax.io/sax3l/0pt",
|
||||
"note": "A second remote (github.com/sax3l/0pt) also claims to be this repo per CLAUDE.md; explicitly unresolved and left to Simon. This pass worked only against git.siax.io."
|
||||
},
|
||||
"branch": {
|
||||
"name": "feat/p010-data-os-completion",
|
||||
"renamed_from": "feat/p010-source-registry-persistence",
|
||||
"base_branch": "main",
|
||||
"base_commit": "f8bb0d5ab9f35c652d09f7639d20458cfe849f56",
|
||||
"head_commit": "9740e20818a8ce0a90498bdb12c501bc74395ac",
|
||||
"commits_on_branch": [
|
||||
{
|
||||
"sha": "07b13bd8331017462ead11b9b2ce8fa9188e520f",
|
||||
"summary": "feat(data-os): Source Registry Postgres persistence + full HTTP route surface"
|
||||
},
|
||||
{
|
||||
"sha": "9740e20818a8ce0a90498bdb12c501bc74395ac",
|
||||
"summary": "feat(data-os): TS/Python quality bridge + provenance fields + fault-injection tests"
|
||||
}
|
||||
],
|
||||
"pushed_to_origin_this_pass": true,
|
||||
"diffstat_vs_base": {
|
||||
"files_changed": 47,
|
||||
"insertions": 3119,
|
||||
"deletions": 201
|
||||
}
|
||||
},
|
||||
"pull_request": {
|
||||
"url": "https://git.siax.io/sax3l/0pt/pulls/28",
|
||||
"number": 28,
|
||||
"base": "main",
|
||||
"head": "feat/p010-data-os-completion",
|
||||
"state_at_report_time": "open",
|
||||
"opened_this_pass": true
|
||||
},
|
||||
"verification_methodology": {
|
||||
"description": "All numbers below were produced by a from-scratch verification pass: a fresh `git clone` of the pushed branch into an isolated directory, a fresh `pnpm install --frozen-lockfile`, and a fresh Python venv built from requirements-dev.txt. This is deliberately NOT a rerun inside the original working tree that developed the branch, to avoid reporting stale or environment-contaminated results.",
|
||||
"clean_clone_path_used_for_verification": "isolated scratch directory, discarded after this pass",
|
||||
"original_working_tree": "/home/simon/siax-platforms-work/0pt"
|
||||
},
|
||||
"checks": [
|
||||
{
|
||||
"name": "pnpm install --frozen-lockfile",
|
||||
"result": "ok",
|
||||
"note": "lockfileVersion 9, no resolution drift from the committed pnpm-lock.yaml"
|
||||
},
|
||||
{
|
||||
"name": "typecheck",
|
||||
"command": "pnpm run typecheck",
|
||||
"underlying": "tsc -p tsconfig.check.json (src/**/*.ts + tests/**/*.ts)",
|
||||
"ci_gate": "blocking",
|
||||
"result": "pass",
|
||||
"errors": 0,
|
||||
"note": "Zero errors on the pristine clone. The original dev working tree separately contains 5 untracked/uncommitted stray paths (src/fleet/, src/graph/, src/metrics/, src/projections/, src/features/feature-registry.ts) that were already present before this pass (flagged in BEFORE_STATE.md as unaudited) and are NOT part of any commit on this branch or on main -- confirmed via `git diff --stat origin/main..feat/p010-data-os-completion` touching none of them, and via their total absence from the fresh clone. Running the same command inside that non-clean working tree surfaces 14 TS errors from those 5 files; that is local litter CI will never see, not a defect in this PR."
|
||||
},
|
||||
{
|
||||
"name": "unit/integration tests (TypeScript)",
|
||||
"command": "pnpm run test",
|
||||
"underlying": "vitest run",
|
||||
"ci_gate": "blocking",
|
||||
"result": "pass",
|
||||
"test_files_passed": 34,
|
||||
"test_files_skipped": 3,
|
||||
"tests_passed": 256,
|
||||
"tests_skipped": 18,
|
||||
"tests_new_this_branch": 7,
|
||||
"new_test_file": "tests/p0/quality-bridge.test.ts",
|
||||
"skip_reason": "pre-existing .realinfra/.integration suites self-skip honestly when no live ST0RE/OpenSearch/ClickHouse is reachable from the verification environment",
|
||||
"sample_skip_stderr": "[realinfra] ST0RE (https://api.st0re.siax.io) not reachable this run (ST0RE responded HTTP 401) -- skipping honestly, not fabricating success."
|
||||
},
|
||||
{
|
||||
"name": "lint",
|
||||
"command": "pnpm run lint",
|
||||
"underlying": "eslint src tests",
|
||||
"ci_gate": "advisory (continue-on-error in both .gitea/workflows/ci.yml and .github/workflows/ci.yml)",
|
||||
"result": "pre-existing failures only, zero introduced by this branch",
|
||||
"errors": 12,
|
||||
"warnings": 4,
|
||||
"verification": [
|
||||
"git diff --stat origin/main..feat/p010-data-os-completion on every flagged file returns no output -- byte-identical to main",
|
||||
"ran `pnpm lint` against a worktree checked out directly at origin/main and got the identical 12 errors / 4 warnings"
|
||||
]
|
||||
},
|
||||
{
|
||||
"name": "build",
|
||||
"command": "pnpm run build",
|
||||
"underlying": "next build",
|
||||
"ci_gate": "blocking in .gitea/workflows/ci.yml; not run by .github/workflows/ci.yml (needs runtime secrets Vercel owns)",
|
||||
"result": "pass",
|
||||
"note": "Route manifest confirms new surface is built: /api/0pt/quality/{bridge-run,issues,rules,rules/[ruleId],runs,runs/[runId]} and /api/0pt/sources/{route,[sourceId],[sourceId]/{freshness,health,provenance,rights}} all present as dynamic routes."
|
||||
},
|
||||
{
|
||||
"name": "Python engine tests",
|
||||
"command": "python -m pytest -v",
|
||||
"cwd": "services/api",
|
||||
"environment": "fresh venv, python3.12, requirements-dev.txt",
|
||||
"ci_gate": "NOT WIRED INTO CI -- both .gitea/workflows/ci.yml and .github/workflows/ci.yml state the Python engine is gated separately if/when a Python workflow is added; this pass did not add that workflow",
|
||||
"result": "pass",
|
||||
"tests_passed": 28,
|
||||
"tests_skipped": 2,
|
||||
"skip_detail": [
|
||||
{
|
||||
"test": "test_cdc_interruption_not_implementable_against_real_code",
|
||||
"reason": "no CDC pipeline exists anywhere in services/api"
|
||||
},
|
||||
{
|
||||
"test": "test_true_backfill_idempotency_not_implementable_against_real_code",
|
||||
"reason": "no delivery/order persistence layer exists anywhere in services/api"
|
||||
}
|
||||
],
|
||||
"new_test_files": [
|
||||
"services/api/tests/test_connectors_provenance.py",
|
||||
"services/api/tests/test_known_gaps.py",
|
||||
"services/api/tests/test_pipeline_fault_injection.py",
|
||||
"services/api/tests/test_quality_bridge.py",
|
||||
"services/api/tests/test_resolve_determinism.py"
|
||||
]
|
||||
}
|
||||
],
|
||||
"net_ci_status": {
|
||||
"gitea_ci_blocking_gates": {
|
||||
"typecheck": "green",
|
||||
"test": "green",
|
||||
"build": "green"
|
||||
},
|
||||
"github_ci_blocking_gates": {
|
||||
"typecheck": "green",
|
||||
"test": "green"
|
||||
},
|
||||
"advisory_only_not_blocking": ["lint (both workflows)", "typecheck:app (github only)"],
|
||||
"unwired_gate_not_in_ci": ["python pytest (services/api) -- manually verified green, no automated workflow yet"]
|
||||
},
|
||||
"production_feature_flags": {
|
||||
"OPT_QUALITY_BRIDGE_ENABLED": {
|
||||
"default_shipped_value": "unset (treated as false)",
|
||||
"changed_by_this_pr": false,
|
||||
"activated_in_production_by_this_pass": false,
|
||||
"behavior_when_off": "quality_bridge.py makes zero network calls, returns status=disabled immediately; pipeline.py's delivery decision is completely unaffected",
|
||||
"behavior_when_on_but_bridge_unreachable": "fails open -- status=unavailable, delivery decision unaffected, error never raised into run_job",
|
||||
"behavior_when_on_and_reachable": "advisory only -- stamps a quality field on delivered provenance and records a score; does NOT and, per the current ExpectationEngine contract, currently CANNOT change which rows pipeline.py's confidence-threshold filter keeps or drops",
|
||||
"decision_owner": "Simon",
|
||||
"readiness_checklist_before_enabling": [
|
||||
"OPT_QUALITY_BRIDGE_URL / OPT_QUALITY_BRIDGE_EDGE_URL must point at a reachable, authenticated 0pt.app deployment (cross-service network dependency -- Python engine and Next.js edge are typically different deploy targets)",
|
||||
"accept that this remains advisory-only (no per-row gating) unless the ExpectationEngine contract is separately extended to return per-row results",
|
||||
"recommended: enable against one non-production tenant/order first and confirm bridge-run round-trips before any production enablement"
|
||||
],
|
||||
"manual_live_verification_performed_this_pass": {
|
||||
"what": "real `next dev` server (AUTH_ENFORCE=false, local only) + real Python pipeline hitting real brreg.no + real bridge-run round-trip + real QualityService score returned",
|
||||
"when": "during development of commit 9740e20, 2026-09-01",
|
||||
"repeatable_automated_equivalent": "services/api/tests/test_quality_bridge.py (respx-mocked bridge responses, run as part of the 28/28 pytest total above)",
|
||||
"note": "the live run was a one-off manual verification during development, not a formal staging rollout step, and not itself part of the from-scratch CI-mirroring verification in this report"
|
||||
}
|
||||
}
|
||||
},
|
||||
"residual_gaps": [
|
||||
"Python engine tests (28/28) not wired into any CI workflow yet",
|
||||
"bridge_provenance as a single unified TS/Python construct still does not exist -- this pass narrows, does not close, the provenance-schema split",
|
||||
"per-row quality gating from TS into Python is not implementable against the current ExpectationEngine contract -- advisory-only is the ceiling of this pass, not a corner cut",
|
||||
"resolve.py's greedy merge remains genuinely order-sensitive for non-transitive-similarity triangles -- tested and documented, not fixed (algorithm change, out of scope)",
|
||||
"CDC-interruption and true delivery-store backfill-idempotency tests remain unimplementable -- no CDC pipeline or delivery persistence layer exists anywhere in services/api",
|
||||
"5 untracked/unaudited local paths (src/fleet/, src/graph/, src/metrics/, src/projections/, src/features/feature-registry.ts) left exactly as found -- not part of this diff, absent from a fresh clone",
|
||||
"services/scrapers/ (WAF-bypass stack) remains on main with its deletion PR still open -- unrelated to and untouched by this pass"
|
||||
],
|
||||
"process_notes": {
|
||||
"pr_open_before_this_pass": false,
|
||||
"before_state_md_left_untouched": true,
|
||||
"gitignore_additions": [".venv/", ".pytest_cache/"],
|
||||
"leftover_processes_confirmed_stopped": true
|
||||
}
|
||||
}
|
||||
Reference in New Issue
Block a user