Real, tested implementation of docs/11-lakehouse-write-plan.md's items 2-8 (the "future implementation session" that doc named): 0pt is now the canonical writer of Iceberg tables through ST0RE's real, shipped /v1/st0re/lakehouse/* table registry (sax3l/st0re commit 9ea0abb), never persisting lakehouse table/catalog/snapshot state itself. - src/types + src/services for 0pt.dataset_iceberg_mapping.schema.v1 (dataset -> ST0RE tableId reference) and 0pt.source_rate_policy.schema.v1 (the sourceRatePolicy ownership repair, moved out of N0D per ADR-0030). - src/clients/st0re-lakehouse-client.ts: new client matching ST0RE's actual camelCase route contract, with the same mock/HTTP boundary convention as this repo's existing cl0ud/n0d/aud0/st0re clients. Honest caveat in its module doc comment: not verified against a live ST0RE instance in this pass (no reachable ST0RE_URL); every test exercises the deterministic mock. - src/services/iceberg-write-service.ts: registers tables/snapshots through ST0RE, requests N0D jobs only for heavy writes (never executes them in-process), emits all four AUD0 event names (dataset_iceberg_table_mapped / iceberg_write_requested / iceberg_snapshot_created / source_rate_policy_created). - sql/004, packages/schemas/*.json, app/api/0pt/lakehouse + sources/rate-policies routes, tests/p0/*.test.ts (154 passed + 1 honest it.todo for the live-ST0RE case, up from 142 baseline, 0 failures). - Reconciled a real collision: an earlier untracked draft schema/test pair (tests/p0/enterprise-data-os-expansion.test.ts) asserted a different, now-superseded source_rate_policy shape; updated that one test block only. - docs/ICEBERG_WRITE_BOUNDARY.md states the ownership boundary plainly and is explicit throughout about what's real-and-tested vs. contract-only. docs/11-lakehouse-write-plan.md gets a status-update section, not a rewrite. - Data quality rules (quality-check-service.ts, quality-report-service.ts) were not touched -- confirmed still 0pt-owned. Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>
32 lines
1.2 KiB
JSON
32 lines
1.2 KiB
JSON
{
|
|
"$id": "0pt.source_rate_policy.schema.v1",
|
|
"type": "object",
|
|
"required": [
|
|
"source_rate_policy_id",
|
|
"tenant_id",
|
|
"project_id",
|
|
"source_id",
|
|
"rate_limit_per_minute",
|
|
"crawl_budget",
|
|
"status",
|
|
"created_at"
|
|
],
|
|
"properties": {
|
|
"source_rate_policy_id": { "type": "string" },
|
|
"tenant_id": { "type": "string" },
|
|
"project_id": { "type": "string" },
|
|
"source_id": { "type": "string", "description": "References 0pt.source.schema.v1's source_id." },
|
|
"rate_limit_per_minute": { "type": "number" },
|
|
"crawl_budget": { "type": "number", "description": "Max requests/pages/records this policy permits per crawl run or per day; unit documented in compliance_notes when non-obvious." },
|
|
"compliance_notes": { "type": "string" },
|
|
"domain": { "type": "string", "description": "Crawl-target domain, e.g. allabolag.se." },
|
|
"robots_ok": { "type": "boolean" },
|
|
"access": { "type": "string", "enum": ["api", "bulk", "authorized_web", "licensed"] },
|
|
"status": { "type": "string", "enum": ["active", "paused", "deprecated"] },
|
|
"correlation_id": { "type": "string" },
|
|
"created_at": { "type": "string" },
|
|
"updated_at": { "type": "string" },
|
|
"created_by": { "type": "string" }
|
|
}
|
|
}
|