Files
0pt/packages/schemas/0pt.source_rate_policy.schema.json
T
adminandClaude Sonnet 5 35b6ec63bf Add 0pt Iceberg write-path ownership: dataset<->table mapping + source rate policy
Real, tested implementation of docs/11-lakehouse-write-plan.md's items 2-8
(the "future implementation session" that doc named): 0pt is now the
canonical writer of Iceberg tables through ST0RE's real, shipped
/v1/st0re/lakehouse/* table registry (sax3l/st0re commit 9ea0abb), never
persisting lakehouse table/catalog/snapshot state itself.

- src/types + src/services for 0pt.dataset_iceberg_mapping.schema.v1
  (dataset -> ST0RE tableId reference) and 0pt.source_rate_policy.schema.v1
  (the sourceRatePolicy ownership repair, moved out of N0D per ADR-0030).
- src/clients/st0re-lakehouse-client.ts: new client matching ST0RE's actual
  camelCase route contract, with the same mock/HTTP boundary convention as
  this repo's existing cl0ud/n0d/aud0/st0re clients. Honest caveat in its
  module doc comment: not verified against a live ST0RE instance in this
  pass (no reachable ST0RE_URL); every test exercises the deterministic mock.
- src/services/iceberg-write-service.ts: registers tables/snapshots through
  ST0RE, requests N0D jobs only for heavy writes (never executes them
  in-process), emits all four AUD0 event names
  (dataset_iceberg_table_mapped / iceberg_write_requested /
  iceberg_snapshot_created / source_rate_policy_created).
- sql/004, packages/schemas/*.json, app/api/0pt/lakehouse + sources/rate-policies
  routes, tests/p0/*.test.ts (154 passed + 1 honest it.todo for the live-ST0RE
  case, up from 142 baseline, 0 failures).
- Reconciled a real collision: an earlier untracked draft schema/test pair
  (tests/p0/enterprise-data-os-expansion.test.ts) asserted a different,
  now-superseded source_rate_policy shape; updated that one test block only.
- docs/ICEBERG_WRITE_BOUNDARY.md states the ownership boundary plainly and
  is explicit throughout about what's real-and-tested vs. contract-only.
  docs/11-lakehouse-write-plan.md gets a status-update section, not a rewrite.
- Data quality rules (quality-check-service.ts, quality-report-service.ts)
  were not touched -- confirmed still 0pt-owned.

Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>
2026-07-12 19:46:47 +02:00

32 lines
1.2 KiB
JSON

{
"$id": "0pt.source_rate_policy.schema.v1",
"type": "object",
"required": [
"source_rate_policy_id",
"tenant_id",
"project_id",
"source_id",
"rate_limit_per_minute",
"crawl_budget",
"status",
"created_at"
],
"properties": {
"source_rate_policy_id": { "type": "string" },
"tenant_id": { "type": "string" },
"project_id": { "type": "string" },
"source_id": { "type": "string", "description": "References 0pt.source.schema.v1's source_id." },
"rate_limit_per_minute": { "type": "number" },
"crawl_budget": { "type": "number", "description": "Max requests/pages/records this policy permits per crawl run or per day; unit documented in compliance_notes when non-obvious." },
"compliance_notes": { "type": "string" },
"domain": { "type": "string", "description": "Crawl-target domain, e.g. allabolag.se." },
"robots_ok": { "type": "boolean" },
"access": { "type": "string", "enum": ["api", "bulk", "authorized_web", "licensed"] },
"status": { "type": "string", "enum": ["active", "paused", "deprecated"] },
"correlation_id": { "type": "string" },
"created_at": { "type": "string" },
"updated_at": { "type": "string" },
"created_by": { "type": "string" }
}
}