Commit Graph
3 Commits
Author SHA1 Message Date
Simon AxelssonandClaude Opus 5 01cc225ffc fix: add missing 0pt platform clients + registry services, wire test/typecheck scripts
The 0pt data-engine build has been broken on main since 2026-07-08 (commit
c897311). document-capture-service.ts, document-extraction-service.ts,
source-query-service.ts, external-writeback-service.ts (and, more recently,
dataset-iceberg-mapping-service.ts, iceberg-write-service.ts,
source-rate-policy-service.ts) all import six platform client modules and two
registry services that were never committed: src/clients/{aud0,cl0ud,n0d,
st0re}-client.ts, src/services/{dataset,source}-service.ts, plus
src/types/{dataset,source}.ts. `next build` failed with 17 "Module not found"
errors, and every route under app/api/0pt/* returned HTTP 500 in dev with the
Turbopack compile error leaked straight into the JSON response body.

This adds the missing files, following the same fetch-based client shape
already established by src/clients/st0re-lakehouse-client.ts (thin interface,
HTTP implementation, deterministic in-memory mock used when no base URL is
configured) and the same "pure core, thin in-memory registry, AUD0-evented
mutation" shape the existing SourceRatePolicyService/DatasetIcebergMappingService
already use for DatasetService/SourceService. Each new client mock is
documented as a mock/degrade path, never a fabricated success, per this
repo's CLAUDE.md honesty doctrine.

Also wires up "typecheck" and "test" npm scripts (tsc --noEmit / vitest run)
and adds vitest as a devDependency -- previously `pnpm test` failed
immediately with ERR_PNPM_NO_SCRIPT even though 13 test files (~89 cases)
already existed under tests/p0/.

Verified: `pnpm run typecheck` clean, `pnpm run test` 88 passed / 1 todo
(pre-existing live-ST0RE-only skip), `pnpm run build` exits 0 (39/39 routes
compiled, only the pre-existing intentional optional `mssql` import warning
remains). Re-verified live on port 4097: GET /api/0pt/sources/rate-policies,
GET /api/0pt/lakehouse/mappings, POST /api/0pt/documents/capture, GET
/api/0pt/documents/[id]/extraction, and both GET/POST
/api/0pt/sources/[id]/query now return real application responses (200/400/405)
instead of HTTP 500 with a leaked compile error.

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
2026-07-14 01:16:50 +02:00
adminandClaude Sonnet 5 35b6ec63bf Add 0pt Iceberg write-path ownership: dataset<->table mapping + source rate policy
Real, tested implementation of docs/11-lakehouse-write-plan.md's items 2-8
(the "future implementation session" that doc named): 0pt is now the
canonical writer of Iceberg tables through ST0RE's real, shipped
/v1/st0re/lakehouse/* table registry (sax3l/st0re commit 9ea0abb), never
persisting lakehouse table/catalog/snapshot state itself.

- src/types + src/services for 0pt.dataset_iceberg_mapping.schema.v1
  (dataset -> ST0RE tableId reference) and 0pt.source_rate_policy.schema.v1
  (the sourceRatePolicy ownership repair, moved out of N0D per ADR-0030).
- src/clients/st0re-lakehouse-client.ts: new client matching ST0RE's actual
  camelCase route contract, with the same mock/HTTP boundary convention as
  this repo's existing cl0ud/n0d/aud0/st0re clients. Honest caveat in its
  module doc comment: not verified against a live ST0RE instance in this
  pass (no reachable ST0RE_URL); every test exercises the deterministic mock.
- src/services/iceberg-write-service.ts: registers tables/snapshots through
  ST0RE, requests N0D jobs only for heavy writes (never executes them
  in-process), emits all four AUD0 event names
  (dataset_iceberg_table_mapped / iceberg_write_requested /
  iceberg_snapshot_created / source_rate_policy_created).
- sql/004, packages/schemas/*.json, app/api/0pt/lakehouse + sources/rate-policies
  routes, tests/p0/*.test.ts (154 passed + 1 honest it.todo for the live-ST0RE
  case, up from 142 baseline, 0 failures).
- Reconciled a real collision: an earlier untracked draft schema/test pair
  (tests/p0/enterprise-data-os-expansion.test.ts) asserted a different,
  now-superseded source_rate_policy shape; updated that one test block only.
- docs/ICEBERG_WRITE_BOUNDARY.md states the ownership boundary plainly and
  is explicit throughout about what's real-and-tested vs. contract-only.
  docs/11-lakehouse-write-plan.md gets a status-update section, not a rewrite.
- Data quality rules (quality-check-service.ts, quality-report-service.ts)
  were not touched -- confirmed still 0pt-owned.

Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>
2026-07-12 19:46:47 +02:00
adminandClaude Sonnet 5 c89731115c feat: add 0pt.query_source, 0pt.document_ocr, 0pt.write_external_record capabilities
Three new, tenant-agnostic P0 capabilities:

- 0pt.query_source: bounded, timeout/row-capped ad-hoc SQL query execution
  against registered database sources, via a Fabric/OneLake adapter
  (Azure AD credential-based mssql connection), with AUD0 audit events
  and an app/api/0pt/sources/[sourceId]/query endpoint.

- 0pt.document_ocr (document capture + extraction): mailbox-based document
  intake via Microsoft Graph, OCR/field extraction via Azure Document
  Intelligence, with capture/extraction API endpoints and services that
  never fabricate extracted values.

- 0pt.write_external_record (external writeback): a generic writeback-adapter
  contract plus a concrete Odoo bills adapter that matches partners via
  Fabric, backing an external-writeback-service for pushing records to
  downstream systems of record.

Known interim gaps:
- src/adapters/local-blob-store.ts is an interim ST0RE substitute pending
  the real ST0RE artifact-store integration.
- mssql is used by the Fabric/OneLake adapter but deliberately not yet
  added to package.json (pending `pnpm add mssql` once the concurrent
  session's package.json/dependency changes settle).

A separate, concurrently-active session has unrelated in-flight changes
in this same working tree (data-platform catalog/lineage/feature-store,
dataset quality, lakehouse/Iceberg docs, deploy scaffolding, package.json/
pnpm-lock.yaml updates, etc.) — none of that is touched or included here.

Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>
2026-07-08 19:32:43 +02:00