BEFORE_STATE (verified, with evidence):
* P060 PARTIAL/BROKEN. The registry persisted rights/provenance/health/
freshness to Postgres, but carried no authorization, no schedule, no
checkpoint, no schema and no owner distinct from created_by — and the
activation DoD was inverted: postgres-source-repository.ts wrote
`${input.status ?? "active"}`, so a source with no rights, no provenance
and no authorization was ACTIVE the instant it was registered. No
activation operation existed at all.
* P061 MISSING. services/api/sources/connectors.py had per-source fetch_*()
functions only — no lifecycle, no cursor, no resume, no health, no
conformance suite.
* P062 MISSING, already documented as such in
services/api/tests/test_known_gaps.py:10-19 (no CDC pipeline anywhere;
workers/pipeline.py has no persistence at all).
* P063 MISSING. schema_version existed as a free string; no comparison,
no policy, no versioned source schema.
* P064 PARTIAL. Rules/runs/issues/scores existed, but a failing rule only
opened an issue and the data flowed on. No quarantine anywhere.
* P065 DUPLICATE (confirmed). Two ER engines: src/er/entity-resolution-
service.ts (canonical) and services/api/workers/resolve.py (simplified,
still called by pipeline.py:110).
* P067 PARTIAL. Lineage was emitted to AUD0 but nothing here could ANSWER a
lineage question — no store, no query, no route.
* P068 MISSING, documented in test_known_gaps.py:44-52 ("no delivery-store
persistence layer exists to dedupe a backfill against").
BUILT (only what was PARTIAL/BROKEN/MISSING):
* P060 sql/007_ingestion_plane.sql adds authorization/schedule/schemas/
checkpoints/ledger/quarantine/lineage tables, owner_id, and flips the
opt_sources status DEFAULT to 'pending' so even a raw psql INSERT cannot
produce an unclassified-but-active source. SourceRegistryService gains
classifySource/activateSource/deactivateSource + authorization/schedule
accessors; registration NEVER activates (asking for status:"active" goes
through the gate, not around it); a service identity cannot authorize a
source. Routes: activate (GET classification / POST activate),
authorization, schedule.
* P061 src/connectors/connector.ts — all nine operations on one interface.
src/connectors/conformance.ts — a runnable suite (15 checks, no "skip"
verdict). HttpJsonConnector (real HTTP, opaque cursor) and
JsonlFileConnector (real file, line offsets, CDC deletes) both pass it.
The suite caught a real defect while being written: an API that drops
next_cursor at the end left the connector unable to resume, which is why
HttpJsonConnectorConfig has cursor_field.
* P062 src/types/ingestion.ts + src/ingestion/{envelope,ingestion-runner}.ts.
One envelope for batch/stream/cdc; durable per-(source,partition)
checkpoints; resume proven by killing a run inside the sink.
* P063 src/schema/{record-schema,compatibility,schema-registry}.ts. Two
enforcement points, both upstream of any consumer write: a discovery gate
before the first read, and per-record conformance before the page reaches
the sink. A producer cannot widen its own policy retroactively.
* P064 src/quality/quality-policy.ts composes the EXISTING ExpectationEngine
(not a second engine) and adds the missing consequence: block /
quarantine / warn, with quarantine durable and recoverable. No policy is
reported as "none_declared", never as a pass; an unexecutable expectation
counts as a violation, never a silent pass.
* P067 src/lineage/{lineage-edge,lineage-store}.ts + GET /api/0pt/lineage.
AUD0 stays the system of record; this is 0pt's own queryable projection.
* P068 opt_ingestion_ledger + IngestionRunner.backfill: bounded, idempotent
on repeat, leaves the live checkpoint alone, still re-emits real changes.
A BUG THIS PASS WROTE AND CAUGHT: the first IngestionRepository draft
combined the idempotency check and its recording into one atomic call
BEFORE the sink. The crash/resume test then delivered 7 of 10 records — the
page in flight when the sink died was already marked delivered, so the
resumed run skipped it as a duplicate. Permanent, silent data loss. Fixed by
splitting peekDeliveries/commitDeliveries and committing both the ledger and
the checkpoint only after the sink resolves.
NOT DONE, stated rather than deferred quietly:
* P065 is NOT met. resolve.py is still a second simplified production
engine. Closing it means rewriting it as an adapter over the canonical
service, which cannot be verified from this repo today: services/api is
not built by this repo's Dockerfile, api.0pt.app answers 404, and NO CI
workflow runs its pytest suite. The duplication is now stated at both
sites in code and the removal plan is in STATUS.md. The new ingestion
plane performs no ER, so it adds no third engine.
* P066 (Iceberg) untouched — ST0RE is owned by another workflow. Contract
read only.
* Delivery is AT-LEAST-ONCE, not exactly-once (no per-partition run lock).
* No CDC *source* exists; the contract is met, a CDC connector is not built.
* 0pt does not schedule itself — the schedule row is a declaration ACT0
would read.
VERIFIED: pnpm typecheck clean; pnpm test 365 passed / 25 skipped (was
311/18); pnpm build clean with all four new routes in the route table; lint
unchanged at 12 pre-existing errors. Real Postgres 16
(tests/p0/ingestion-plane-postgres.realinfra.test.ts, 7 passed) plus a real
`docker restart` of that container with an identical read-back through the
production code path — that is P060's "a restart preserves all sources",
measured.
DEPLOYMENT REALITY (curl, 2026-09-04): 0pt.app/api/health returns 200 from
Vercel with db:not_configured; 0pt.app/api/0pt/sources returns 404;
api.0pt.app/health returns 404; cd.yml still guards on a placeholder Coolify
app id. None of the capability layer runs in production. Nothing here is
LIVE_VERIFIED and STATUS.md says so.
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Tenant isolation (security-critical): every app/api/0pt/** route reading
tenant_id from client-controlled query/body never cross-checked it against
the token middleware.ts already verified (x-0pt-verified-tenant). Adds a
shared assertTenantMatchesVerified guard and wires it into all 20+ affected
routes. Also closes two full-leak-by-omission bugs found in this pass
(documents/capture and lakehouse/mappings GETs returned every tenant's rows
when tenant_id was simply omitted) and two "no tenant check at all" reads
(rate-policies/[id], lakehouse/mappings/[id]) reachable by ID alone.
Duplicate-delivery idempotency (correctness-critical): POST /api/orders now
honors Idempotency-Key (sql/006, replay-or-409-on-conflict, Postgres +
in-memory); the Python orchestrator's POST /orders/{id}/run — the actual
delivery trigger — is now idempotent under an asyncio lock, closing a real
concurrent-request race, not just the sequential-retry case.
Engine-duplication resolution: adaptive_router.py's docstring wrongly
invited a bypass ("see b0ts for scraping") contradicting the product's own
compliance stance; corrected, and given real (additive, not escalating)
block-signal tracking so a rate-limited/blocked source is excluded from the
plan instead of hammered. Also fixes a previously-undiscovered production
bug: the Python engine's only call into the TS quality bridge
(X-Internal-Token) was unconditionally 401'd by middleware.ts, which never
checked that credential — the advisory quality bridge was unreachable in any
environment enforcing auth.
Restore drill: executed for real against throwaway Docker Postgres
containers (not simulated). Found and fixed two real blockers along the way:
sql/004 referenced opt_datasets, which no migration ever created (chain was
never actually runnable end to end); lib/db.ts hardcoded ssl:"require",
making any non-TLS Postgres (incl. compose.dev.yaml's own local-dev
container) unreachable. After both fixes: full migration chain applies
cleanly, pg_dump/pg_restore round-trip is byte-identical (row counts +
content checksums), and real application code (lib/orders-store.ts,
SourceRegistryService's Postgres repository) reads the restored data back
correctly, including cross-tenant isolation still holding.
82 new tests (55 TS + 27 Python), all passing; full existing suite (291 TS +
55 Python) still green; typecheck clean on every changed/added file.
Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>
Continues P010 (branch renamed from feat/p010-source-registry-persistence):
closes the three remaining BEFORE_STATE.md gaps (§2 TS/Python split-brain,
§3 missing provenance fields, §4 untested failure modes), all additive to
the production pipeline — the Python QA filter's delivery decision is
unchanged by any of this.
TS/Python bridge (§2):
- New route surface app/api/0pt/quality/* (rules/runs/issues CRUD +
POST /api/0pt/quality/bridge-run) over the shared QualityService
singleton (src/services/quality.ts), fixing the same "no shared
instance" defect the Source Registry had.
- src/quality/quality-bridge.ts auto-provisions a deterministic,
tenant-scoped baseline rule set on first use so the Python engine never
pre-registers rules through the CRUD surface.
- services/api/workers/quality_bridge.py: the Python-side HTTP client.
Feature-flagged off by default (OPT_QUALITY_BRIDGE_ENABLED), fails open
on any error (never raises into run_job, never changes `delivered`).
Advisory only — real per-row gating against the TS engine isn't
implementable yet (ExpectationEngine returns dataset-level pass/fail
counts + samples, not a per-row boolean array); documented, not faked.
- Live-verified end-to-end: `next dev` (AUTH_ENFORCE=false) + the real
Python pipeline hitting the real brreg.no API, round-tripping through
the real bridge route, receiving a real QualityService score.
Provenance (§3): sources/connectors.py's Provenance dataclass gains
retrieved_at, transform, connector_version, producer, quality,
entity_resolution, artifact_ref (all previously absent from the real
dataflow entirely) via a make_provenance() factory used by every connector.
resolve.py stamps entity_resolution on merged records' provenance (never on
unmerged ones). pipeline.py stamps artifact_ref at delivery and quality
once the bridge succeeds.
Failure tests (§4), real fault injection against real code (services/api,
new pytest + respx suite, not vitest mocks):
- dedup-determinism + duplicate-input + out-of-order: resolve.py under
shuffled/interleaved/exhaustive-permutation input, plus an explicit,
verified test of the one case that IS legitimately order-sensitive
(non-transitive similarity in a greedy clusterer) instead of hiding it.
- schema-evolution: respx-mocked upstream responses with added/removed/
alien-shaped fields through the real brreg connector + pipeline.
- restart: connector exceptions mid-job (including one from respx itself)
never crash run_job.
- backfill: the real property that holds given no persistence layer
exists (replay is compute-idempotent) — NOT a fabricated dedup-store
test.
- CDC-interruption and true backfill/delivery-store idempotency:
explicitly documented as not implementable against this codebase
(no CDC pipeline, no delivery store exist anywhere) via skip markers
with reasons, in tests/test_known_gaps.py, rather than silently
dropped or faked.
Testing: 256/256 vitest (249 pre-existing + 7 new bridge route/service
tests) and 28/28 pytest (2 documented skips) green locally. tsc -p
tsconfig.check.json has zero new errors (14 pre-existing errors, all in
the untracked/unaudited src/fleet, src/graph, src/metrics, src/projections
paths this pass does not touch, per BEFORE_STATE.md).
No PR opened, per instructions.
Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>
Replaces the in-memory (per-instance, not even per-process-shared) Maps in
SourceService/SourceRegistryService with real Postgres persistence, and adds
the register/query/describe/health/rights/provenance/freshness HTTP routes
that never existed (grep for SourceRegistryService in app/ previously
returned nothing).
- sql/002_source_registry.sql: opt_sources, opt_source_rights,
opt_source_provenance, opt_source_health_checks, opt_source_freshness.
Numbered 002 (not 006) because 004's opt_source_rate_policies already
forward-references opt_sources, which no sql/ file defined until now —
fixes 004's pre-existing broken migration ordering (opt_datasets remains
a separate, unrelated, still-open gap in 004, flagged not fixed).
- src/repositories/*: SourceRepository / SourceRegistryRepository seams,
each with an in-memory implementation (business logic extracted verbatim)
and a Postgres implementation, chosen via resolveStoreMode — production
still fails hard with no DATABASE_URL (Master-DoD §3), but a configured
production now actually works instead of always being blocked.
- SourceService / SourceRegistryService: storage moved behind the
repository seam; getHealth/getFreshness are now async and tenant-scoped
at the accessor itself (needed once they're reachable directly over HTTP,
not only after an earlier caller's tenant check).
- src/services/source-registry.ts: shared singleton instances for the
sources/* route group, fixing the "each route module owned its own
private registry" defect (source-rate-policies and .../query previously
couldn't see a source registered via POST /api/0pt/sources).
- New routes: POST/GET /api/0pt/sources, GET /api/0pt/sources/[id],
GET+PUT .../rights, GET+PUT .../provenance, GET+POST .../health,
GET .../freshness.
- Business logic unchanged: fail-closed purpose gate, cross-tenant
NOT_FOUND (no existence oracle, §50), bounded 100-row health history.
Tested against a real Postgres container (not mocked) —
tests/p0/source-registry-postgres.realinfra.test.ts, env-gated on
SOURCE_REGISTRY_TEST_DATABASE_URL, skips honestly when unset. All 249
pre-existing P0 tests + 7 new realinfra tests pass; also smoke-tested the
full HTTP route surface end-to-end against next dev + live Postgres.
Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>
BACKLOG P0-3: /api/health only ever checked *presence* of DATABASE_URL/
REDIS_URL and always returned ok:true regardless of whether anything was
reachable -- ARCHITECTURE.md's own words, "an uptime monitor watching HTTP
200 would show green for a dying instance".
lib/health-checks.ts adds real dependency probes:
- db: a live `select 1` round trip (not just DATABASE_URL presence)
- engine: a real HTTP call to the FastAPI engine's own GET /health
- stripe: Stripe's documented lightweight balance.retrieve() liveness call
- queue: honestly reports "not_integrated" when REDIS_URL is set but
unused -- no Redis client is wired anywhere in this repo, so reporting
"ok" would fabricate a check that doesn't exist
/api/health stays a liveness check (always 200 while the process can
respond; the body's `ok` field is the honest signal). New GET /api/ready
fails closed with 503 when the load-bearing dependency (db) isn't both
configured and reachable -- the "no /ready route" gap.
contracts/openapi.yaml, docs/architecture/ARCHITECTURE.md, and docs/slo.md
updated to match. 17 new unit tests (dependency-injected fakes, no real
network/DB) + 5 route-level tests.
A live compliance audit against 0pt.app confirmed both endpoints returned
HTTP 200 with zero authentication -- the full order queue and financial
pipeline aggregates (deposit/pipeline totals) were readable by anyone.
Root cause: middleware.ts's matcher only covers /api/0pt/:path*, by design
("does NOT touch ... the unrelated payment/order APIs"), so nothing upstream
of these two handlers ever verified the caller.
Fix: both GET handlers now call the same fail-closed Zitadel verifier
/api/0pt/* relies on (resolveAuthContext, src/auth/context.ts) directly, via
a new thin wrapper (src/auth/require-route-auth.ts) that maps a rejected
decision to the same 401/403 JSON shape middleware.ts already returns. The
wrapper re-verifies the bearer token itself rather than trusting any
x-0pt-verified-* header on the request, since middleware never runs in front
of these routes to overwrite a client-supplied one.
POST /api/orders (anonymous draft/quote creation before the deposit-payment
step) is intentionally left as-is -- only the read paths that leaked data are
gated. Tracked as part of docs/compliance/BACKLOG.md P0-3.
Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>
Merges github/main (6 commits: 17-bug audit fix, CI quality gate,
.env.example + .gitignore fix, missing platform clients + registry
services, docs accuracy overhaul) into gitea/main (8 commits: Iceberg
write boundary + live ST0RE verification, lakehouse mapping routes,
Waves 1-4 foundational hardening).
16 conflicts. Both sides had independently written the four
src/clients/*.ts and the two registry services, so those are combined
rather than picked:
- st0re-client.ts: GitHub's endpoint wins on evidence. POST goes to
/v1/st0re/artifacts/register, not /v1/st0re/artifacts -- verified in
sax3l/st0re src/api/routes/artifacts.ts, where the bare path is a
GET-only list route. Gitea's URL normalisation, error-body text and
mock kept.
- n0d-client.ts: field is `status`, not `state`, with N0D's real 13-state
union -- verified in sax3l/n0d packages/schemas/n0d.job.schema.json,
which says explicitly not to add "succeeded". Gitea's mock echo,
URL normalisation and error-body text kept.
- cl0ud-client.ts: Gitea's fail-closed HTTP normalisation kept (a 200
with no `decision` reads as "deny", not as a silent allow). GitHub's
PolicyDecision/PolicyDecisionValue naming kept; PolicyResult retained
as an alias.
- aud0-client.ts: permissive union of both input types (optional
payload/metadata/correlation_id), GitHub's echo return, Gitea's URL
normalisation and error-body text, plus a JSON-parse fallback so a
successful POST with an empty body does not throw.
- dataset-service.ts / source-service.ts: GitHub's record/DTO split and
event naming (0pt.source_created / 0pt.create_source, which the merged
docs already reference) plus Gitea's name validation, description /
created_by fields and actor_id derivation. List filters are the union.
- lakehouse mapping routes: GitHub's shared lakehouse-mapping-registry
singletons (without which every write 400'd "mapping not found") AND
Gitea's Zod parseBody validation. Both, not either.
- src/types/source.ts, src/types/dataset.ts: unions of both vocabularies;
GitHub's `access` legal-sourcing field kept.
- .env.example: GitHub's audited inventory plus Gitea's Zitadel /
AUTH_ENFORCE block.
- CLAUDE.md, docs/ICEBERG_WRITE_BOUNDARY.md: GitHub's corrections kept,
Gitea's live-verification kept and marked as superseding the
"would 400 outright" finding (Gitea had already added the required
X-Tenant-Id/X-Actor-Id headers).
- pnpm-lock.yaml: regenerated with pnpm 9.15.9, not merged.
Also corrects CLAUDE.md's claim that the banned services/scrapers stack
was "since removed" -- the removal PR is open, not merged, and this
branch inherits that code from Gitea main. Nothing here resurrects,
extends or re-wires any of it; services/scrapers/ is byte-identical to
Gitea main.
Verified on the merged tree: typecheck (src/ + tests/) clean;
17 test files, 121 passed, 1 skipped (the realinfra test skipping
honestly on ST0RE HTTP 401), 0 failures. Gitea main baseline was the
same 121/1; GitHub main baseline could not even `pnpm install`
("packages field missing or empty") -- Gitea's pnpm-workspace.yaml fix
resolves that. Whole-repo `typecheck:app` reports 9 errors, an identical
set to Gitea main's baseline, so it is advisory in both CI workflows
rather than deleted. Lint: 16 problems, identical to Gitea baseline.
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
* docs: correct stale quality-check-service.ts comment references
Two source comments still cited src/services/quality-check-service.ts as an
existing precedent. It doesn't exist anywhere in this repo's history (git
log --all: zero commits, any branch, ever) -- confirmed during the earlier
full-doc-corpus accuracy pass. Points each comment at the real precedent
instead (document-extraction-service.ts / iceberg-write-service.ts).
Applied from task_05ec36aa, spun off during that pass.
* fix: 17 confirmed bugs from full-repo audit (edge + Python engine)
Found by an 8-finder bug hunt across app/api, src/services, src/clients,
frontend, the Python engine, the data layer, config/infra, and tests, then
independently adversarially verified per finding (32 raw findings -> 27
confirmed real bugs; 5 refuted). This commit applies the 17 confirmed as
safe, mechanical fixes -- no architecture/design decision required. The
other 10 need a decision and are reported separately, not touched here.
Most consequential, app/api/payments/[order_id]/deposit/route.ts:
the client-supplied Origin request header was trusted unsanitized both as
a server-side fetch target (SSRF) and as Stripe Checkout success/cancel
redirect URLs (open redirect tied to a real payment flow), and a failed
order lookup silently fell through to a hardcoded deposit amount instead
of 404. Fixed by reading the order directly via lib/orders-store.ts
(mirroring the sibling release/route.ts, which already did this correctly)
instead of both trusting the Origin header and self-issuing an HTTP fetch.
Also fixed:
- 5 routes (orders, orders/[id], estimates, subscriptions, ai/parse) that
500'd instead of 400ing on a literal JSON `null` body, since req.json()
resolves (not rejects) for that input and bypasses the try/catch guard.
- app/api/0pt/lakehouse/mappings/{,[mappingId]/{,writes/}}route.ts each
instantiated their own private service instances, so a mapping created
via POST was invisible to the sibling GET and writes routes even before
the separately-tracked dataset-registration gap. Extracted a shared
singleton module, src/services/lakehouse-mapping-registry.ts.
- app/api/0pt/documents/[documentId]/extraction/route.ts collapsed every
error (not-configured, policy-denied, policy-approval-required,
not-found) into a blanket 400; now status-mapped like the sibling
sources/[sourceId]/query route.
- app/api/vehicles/route.ts: plate query param wasn't URL-encoded before
being concatenated into the outbound RDW request, allowing extra query
parameters to be injected; added a strict allowlist + encodeURIComponent.
- app/api/uploads/route.ts: POST returned a fake 200 success with
put_url: null even when "configured", instead of continuing to report
501 until presign generation is actually implemented.
- src/adapters/document-intelligence-adapter.ts: the result-polling loop
never checked response.ok, so a non-2xx poll response (auth expiry,
429, 5xx) was silently retried until a generic timeout masked the real
error.
- app/order/[id]/page.tsx: finish() never cleared the production-simulation
setInterval, so it kept firing (and re-rendering) forever after delivery.
- app/admin/page.tsx: the metrics fetch had no response/shape validation
(unlike the adjacent orders fetch), rendering literal "undefined" stat
tiles on a malformed response instead of falling back to <ErrorState/>.
- services/api/workers/pipeline.py: `config.get("seeds") or DEMO_SEEDS`
used truthy-or, so an explicit empty seeds dict was silently replaced
with hardcoded demo org numbers instead of being respected as empty.
- services/api/workers/resolve.py: corroboration (confidence/agreed_sources
bump) no longer fires for two hits from the same source under fuzzy
match -- only a genuinely independent source now counts, per docs/06's
own ">=2 independent sources agree" definition -- and a merged record's
provenance array now actually includes the corroborating source's
fields instead of only a bare source-name string.
- services/api/orchestrator/main.py: create_order's int(...) on
unvalidated client volume/confidence fields raised on any
decimal-formatted input (int("92.5") -> ValueError); now coerced via
float() first with a safe default. /vehicles/{plate} and /screen also
now catch connector exceptions and degrade gracefully, matching
pipeline.py's existing pattern for the same connectors.
- lib/orders-store.ts: postgres.js (pinned v3.4.9) returns bigint/numeric
columns as strings by default; getOrder/listOrders/setStatus were
silently handing callers stringified money fields. Added a narrow
per-row coercion instead of touching the shared client's global type
map (which would affect unrelated tables' bigint/numeric columns too).
- lib/subscriptions-store.ts: the in-memory fallback returned
oldest-first while the Postgres branch explicitly orders newest-first;
now consistent, mirroring the identical fix already in orders-store.ts.
- app/api/auth/login/route.ts: DEV_LOGIN_KEY was compared with a plain
!==, a non-constant-time comparison; replaced with crypto.timingSafeEqual.
Verified before commit: tsc --noEmit clean; vitest run 88 passed, 1 todo,
0 failures (13 files, unchanged from before this commit -- no existing
test's assertions needed to change). services/api/workers/resolve.py's
source-novelty and provenance-merge logic additionally verified by direct
execution against synthetic same-source/cross-source inputs (no Python
test suite exists in this repo to add to). services/api/workers/pipeline.py
and services/api/orchestrator/main.py verified with python -m py_compile
(no httpx/fastapi installed in this environment to run them directly).
Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>
* fix(engine): treat explicit seeds:null as missing to avoid 500 in run_job
Four-eyes follow-up to this PR's own seeds change. Switching from
`config.get('seeds') or DEMO_SEEDS` to `config['seeds'] if 'seeds' in config`
correctly stopped an explicit empty {} from being backfilled with DEMO_SEEDS,
but regressed on client-controlled `{"seeds": null}`: seeds became None and
the downstream `seeds.get(market, [])` raised AttributeError -> 500. That path
is reachable (POST /orders stores raw body.config; POST /orders/{id}/run passes
it straight to run_job). Using `config.get('seeds') is not None` keeps the
intended behaviour (explicit {} honoured, never silently seeded with demo data)
while restoring the None-tolerance the old idiom had.
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
---------
Co-authored-by: Claude Sonnet 5 <noreply@anthropic.com>
Real, tested implementation of docs/11-lakehouse-write-plan.md's items 2-8
(the "future implementation session" that doc named): 0pt is now the
canonical writer of Iceberg tables through ST0RE's real, shipped
/v1/st0re/lakehouse/* table registry (sax3l/st0re commit 9ea0abb), never
persisting lakehouse table/catalog/snapshot state itself.
- src/types + src/services for 0pt.dataset_iceberg_mapping.schema.v1
(dataset -> ST0RE tableId reference) and 0pt.source_rate_policy.schema.v1
(the sourceRatePolicy ownership repair, moved out of N0D per ADR-0030).
- src/clients/st0re-lakehouse-client.ts: new client matching ST0RE's actual
camelCase route contract, with the same mock/HTTP boundary convention as
this repo's existing cl0ud/n0d/aud0/st0re clients. Honest caveat in its
module doc comment: not verified against a live ST0RE instance in this
pass (no reachable ST0RE_URL); every test exercises the deterministic mock.
- src/services/iceberg-write-service.ts: registers tables/snapshots through
ST0RE, requests N0D jobs only for heavy writes (never executes them
in-process), emits all four AUD0 event names
(dataset_iceberg_table_mapped / iceberg_write_requested /
iceberg_snapshot_created / source_rate_policy_created).
- sql/004, packages/schemas/*.json, app/api/0pt/lakehouse + sources/rate-policies
routes, tests/p0/*.test.ts (154 passed + 1 honest it.todo for the live-ST0RE
case, up from 142 baseline, 0 failures).
- Reconciled a real collision: an earlier untracked draft schema/test pair
(tests/p0/enterprise-data-os-expansion.test.ts) asserted a different,
now-superseded source_rate_policy shape; updated that one test block only.
- docs/ICEBERG_WRITE_BOUNDARY.md states the ownership boundary plainly and
is explicit throughout about what's real-and-tested vs. contract-only.
docs/11-lakehouse-write-plan.md gets a status-update section, not a rewrite.
- Data quality rules (quality-check-service.ts, quality-report-service.ts)
were not touched -- confirmed still 0pt-owned.
Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>
Three new, tenant-agnostic P0 capabilities:
- 0pt.query_source: bounded, timeout/row-capped ad-hoc SQL query execution
against registered database sources, via a Fabric/OneLake adapter
(Azure AD credential-based mssql connection), with AUD0 audit events
and an app/api/0pt/sources/[sourceId]/query endpoint.
- 0pt.document_ocr (document capture + extraction): mailbox-based document
intake via Microsoft Graph, OCR/field extraction via Azure Document
Intelligence, with capture/extraction API endpoints and services that
never fabricate extracted values.
- 0pt.write_external_record (external writeback): a generic writeback-adapter
contract plus a concrete Odoo bills adapter that matches partners via
Fabric, backing an external-writeback-service for pushing records to
downstream systems of record.
Known interim gaps:
- src/adapters/local-blob-store.ts is an interim ST0RE substitute pending
the real ST0RE artifact-store integration.
- mssql is used by the Fabric/OneLake adapter but deliberately not yet
added to package.json (pending `pnpm add mssql` once the concurrent
session's package.json/dependency changes settle).
A separate, concurrently-active session has unrelated in-flight changes
in this same working tree (data-platform catalog/lineage/feature-store,
dataset quality, lakehouse/Iceberg docs, deploy scaffolding, package.json/
pnpm-lock.yaml updates, etc.) — none of that is touched or included here.
Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>
Missing GET caused a 405 → r.json() threw → error state shown instead of
the calm 'storage not connected' info note. Now returns 501+[] so the page
properly shows storageOff and 'No files yet.' empty state.
Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
Values were stored as stable Swedish IDs ('Bolag', 'Engångs') but rendered raw.
Now resolved through entityLabel()/FREQUENCY_LABEL + t() at render time so
the recap panel respects the active locale on every re-render.
Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
RouteReveal layout component applies in-view on every route change.
useReveal hook falls back to immediate rAF when element is already visible.
bestall/page.tsx: added missing useReveal ref.
katalog/[id]/page.tsx: DatasetView uses useReveal directly.
Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
- validera: role=tablist + role=tab + aria-selected + aria-controls/id
on tab buttons; role=tabpanel + id + aria-labelledby on content div
- katalog/[id]: same ARIA tab pattern; replaced incorrect aria-current=page
on tab buttons with role=tab + aria-selected
- error.tsx: i18n all hardcoded English strings via useT()
All three changes pass tsc 0.
Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
Light/dark adaptive 32x32 PNG icons + SVG icon + apple-touch-icon
registered in metadata so browsers pick them up automatically.
Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
- aria-label + aria-value* attrs on confidence range slider (screen readers)
- aria-label on volume number input
- aria-pressed on service/destination selector buttons in beskriv flow
- aria-label on describe textarea
Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
Adds metadataBase, og:title/description/url/siteName/type, twitter card,
and robots index/follow to the root layout for proper link previews when
0pt.app links are shared in Slack, email, or social.
Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
Improves screen-reader and keyboard nav UX: filter pill buttons now
announce pressed state, catalog search input has an accessible label,
filter button group has role="group" with label.
Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
Ensures the keyboard skip-to-content link (added in layout.tsx) has a
valid anchor target on every page in the app.
Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
- Dark mode toggle (●/○) in desktop + mobile header nav
- ThemeProvider wired in layout.tsx (class-based, light default)
- Header scroll-glass uses CSS color-mix for dark-mode correctness
- PortalShell main gets id="main-content" for skip-link target
- Portal order table: copy order ID to clipboard with toast confirmation
- Admin page: toast success/error on status advance
- Skip-to-content link in layout.tsx (sr-only, visible on keyboard focus)
Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
- toast.success/error on order creation (bestall/page.tsx)
- toast.success/error on order approval (validera/page.tsx) + fix silent
success even when API failed (was always setting approved=true)
- toast.success/error on subscription creation (portal/subscriptions)
- toast.success/error on admin status advance (admin/page.tsx)
- toast.success on production page when Stripe redirects back with ?deposit=success
- layout.tsx: skip-to-main-content link (sr-only, visible on focus)
- app/page.tsx: id="main-content" anchor for skip link
Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
- Catalog IDs: synced /api/catalog and /katalog/[id] to consistent English slugs
(nordic-companies, tech-decisionmakers, vehicles-fleet, financial-series,
buying-signals, sanctions-pep); View links from catalog page were all broken
- Added 2 missing catalog detail entries: buying-signals, sanctions-pep
- featured-products-section: link cards to /katalog/{id} instead of hardcoded demo URL
- hero-section: 'See a dataset' now links to /katalog/nordic-companies
- layout.tsx: mount <Toaster> (sonner was installed but never mounted)
- bestall/forskott: pay() tries Stripe checkout first, falls back to demo mode
- api/payments/[order_id]/deposit: Stripe Checkout Session creation (501 if unconfigured)
- app/not-found.tsx: custom 404 page matching mono design system
- philosophy-section: fix TS type error on PRINCIPLES array
Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
Three missed files from the i18n pass:
- /katalog: FILTERS had hardcoded Swedish labels (Alla/Bolag/Beslutsfattare/Fordon/Finansiellt)
- /katalog/[id]: entire page was Swedish — CATALOG data, TABS (with variable-shadowing bug
where map((t) shadowed useT()), SectionHead, all 5 tab bodies, order rail, PROVENANCE_LEGEND
rendered as raw Loc object. Converted to stable TabKey enum + all Loc.
- /portal/subscriptions: useT imported but never called; title/labels/options all Swedish.
tsc 0 + build green.
Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
Converts every last bare-Swedish const array across landing sections
(STATS, TILES, CHAIN, PIPELINE, MODELS, DATASETS, PRINCIPLES, COLUMNS)
and shared components (data-artifact, provenance-badge) to {en,sv} Loc
objects consumed via useT(). Verified by per-file verify agent: 10/10
clean, zero remaining bare-Swedish display text. tsc 0 + build green.
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
All display labels in MARKETS/ENTITY_TYPES/FIELDS/ENRICHMENTS/FREQUENCIES/DELIVERY_FORMATS + SERVICES/FLOW_LABEL are now {en,sv} with stable IDs/values (pricing logic, EntityType/Frequency unions, config keys unchanged). Added entityLabel()/FREQUENCY_LABEL/FORMAT_LABEL. Updated all render sites (configurator, tjanster, beskriv, katalog/[id]) to resolve via useT(). Also fixed 2 pre-existing tsc nits (auth BufferSource cast, orders-store sql.json param type). tsc 0, build green. 0pt is now fully English-by-default with SV toggle.
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
Second hardened fan-out converted the 7 untouched admin/portal files; hand-finished the configurator (8 block labels + full summary column), portal dashboard, and describe-intake. English renders by default; SV toggle flips the whole app. A few tuple-array data labels intentionally left SV (no-reshape safety) + lib/data label i18n remain as follow-up. build green.
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
Locale provider (en default, persisted) + EN/SV switcher in header; converted header + landing sections + catalog/sources/order-flow/most-of-portal to useT({en,sv}). Fixed two partial conversions (forskott TRUST object/tuple map; kallor undefined ALL + raw object renders). tsc 0, build green. Remaining: admin/* + a few partials.
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
/admin/verktyg uses live /api/vehicles (RDW) + /api/screen (OpenSanctions); added to admin nav. Configurator now has a Tjänst selector that applies each service's price multiplier (lib/services) to the live estimate + summary; service_type carried into the order so orders-store prices by service. tsc 0, build green.
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
lib/order-client: createOrder records each order to a localStorage ledger; portal dashboard + admin queue merge it with /api/orders; order tracker falls back to it. Wired all buy paths: configurator, beskriv (custom services), and one-click catalog Beställ. Makes buy->monitor work in the browser now; becomes server-truth when Postgres is connected. tsc 0, build green.
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
lib/services (11 categories: data, scraping, mapping, normalization, consolidation->CRM/ERP, documents/OCR, enrichment, ICP, ML, BI, feeds across the 3 flows); /tjanster catalog; /bestall/beskriv NL intake; /api/ai/parse (engine LLM or heuristic NL->config); CRM/ERP destinations; Tjänster in nav. Sells every custom data service through the platform. tsc 0, build green.
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
Edge /api/vehicles (RDW NL, live now) + /api/screen (OpenSanctions, live now), both defer to engine when ENGINE_URL set. Engine: /vehicles/{plate} + /screen endpoints; workers/resolve (blocking+fuzzy entity resolution, now used by pipeline); workers/queue (in-process job queue); sources/financials (Companies House accounts + XBRL path). py_compile + tsc + next build green.
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
All nav targets now exist on PortalShell; env-guarded actions degrade gracefully. tsc 0, build green.
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
Env-guarded so build stays green: /api/payments/[order_id]/deposit+release + /api/webhooks/stripe (lazy Stripe, 50/50 escrow, 501 until keys); /api/auth login|me|logout (HMAC signed-session scaffold, guarded dev login); lib/engine bridges edge->FastAPI (estimates defers to ENGINE_URL when set); components/embed for NocoDB/Metabase signed URLs; .env.example full inventory. tsc 0, build green.
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
Configurator->POST /api/orders, deposit fetches order + PATCH in_production, header nav reaches the new pages (no orphans). New pages: katalog, katalog/[id], kallor, portal, shared states. tsc 0, build green (13 routes).
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
admin
