CI (SIAX Cloud) / integration (push) Successful in 42s
CI (SIAX Cloud) / quality (push) Successful in 1m17s
CI (SIAX Cloud) / security (push) Successful in 46s
Security Scan (SIAX Cloud) / security (push) Successful in 31s
CI (SIAX Cloud) / security (pull_request) Successful in 24s
CI (SIAX Cloud) / integration (pull_request) Successful in 36s
CI (SIAX Cloud) / quality (pull_request) Successful in 1m4s
Security Scan (SIAX Cloud) / security (pull_request) Successful in 27s
BEFORE_STATE (verified, with evidence):
* P060 PARTIAL/BROKEN. The registry persisted rights/provenance/health/
freshness to Postgres, but carried no authorization, no schedule, no
checkpoint, no schema and no owner distinct from created_by — and the
activation DoD was inverted: postgres-source-repository.ts wrote
`${input.status ?? "active"}`, so a source with no rights, no provenance
and no authorization was ACTIVE the instant it was registered. No
activation operation existed at all.
* P061 MISSING. services/api/sources/connectors.py had per-source fetch_*()
functions only — no lifecycle, no cursor, no resume, no health, no
conformance suite.
* P062 MISSING, already documented as such in
services/api/tests/test_known_gaps.py:10-19 (no CDC pipeline anywhere;
workers/pipeline.py has no persistence at all).
* P063 MISSING. schema_version existed as a free string; no comparison,
no policy, no versioned source schema.
* P064 PARTIAL. Rules/runs/issues/scores existed, but a failing rule only
opened an issue and the data flowed on. No quarantine anywhere.
* P065 DUPLICATE (confirmed). Two ER engines: src/er/entity-resolution-
service.ts (canonical) and services/api/workers/resolve.py (simplified,
still called by pipeline.py:110).
* P067 PARTIAL. Lineage was emitted to AUD0 but nothing here could ANSWER a
lineage question — no store, no query, no route.
* P068 MISSING, documented in test_known_gaps.py:44-52 ("no delivery-store
persistence layer exists to dedupe a backfill against").
BUILT (only what was PARTIAL/BROKEN/MISSING):
* P060 sql/007_ingestion_plane.sql adds authorization/schedule/schemas/
checkpoints/ledger/quarantine/lineage tables, owner_id, and flips the
opt_sources status DEFAULT to 'pending' so even a raw psql INSERT cannot
produce an unclassified-but-active source. SourceRegistryService gains
classifySource/activateSource/deactivateSource + authorization/schedule
accessors; registration NEVER activates (asking for status:"active" goes
through the gate, not around it); a service identity cannot authorize a
source. Routes: activate (GET classification / POST activate),
authorization, schedule.
* P061 src/connectors/connector.ts — all nine operations on one interface.
src/connectors/conformance.ts — a runnable suite (15 checks, no "skip"
verdict). HttpJsonConnector (real HTTP, opaque cursor) and
JsonlFileConnector (real file, line offsets, CDC deletes) both pass it.
The suite caught a real defect while being written: an API that drops
next_cursor at the end left the connector unable to resume, which is why
HttpJsonConnectorConfig has cursor_field.
* P062 src/types/ingestion.ts + src/ingestion/{envelope,ingestion-runner}.ts.
One envelope for batch/stream/cdc; durable per-(source,partition)
checkpoints; resume proven by killing a run inside the sink.
* P063 src/schema/{record-schema,compatibility,schema-registry}.ts. Two
enforcement points, both upstream of any consumer write: a discovery gate
before the first read, and per-record conformance before the page reaches
the sink. A producer cannot widen its own policy retroactively.
* P064 src/quality/quality-policy.ts composes the EXISTING ExpectationEngine
(not a second engine) and adds the missing consequence: block /
quarantine / warn, with quarantine durable and recoverable. No policy is
reported as "none_declared", never as a pass; an unexecutable expectation
counts as a violation, never a silent pass.
* P067 src/lineage/{lineage-edge,lineage-store}.ts + GET /api/0pt/lineage.
AUD0 stays the system of record; this is 0pt's own queryable projection.
* P068 opt_ingestion_ledger + IngestionRunner.backfill: bounded, idempotent
on repeat, leaves the live checkpoint alone, still re-emits real changes.
A BUG THIS PASS WROTE AND CAUGHT: the first IngestionRepository draft
combined the idempotency check and its recording into one atomic call
BEFORE the sink. The crash/resume test then delivered 7 of 10 records — the
page in flight when the sink died was already marked delivered, so the
resumed run skipped it as a duplicate. Permanent, silent data loss. Fixed by
splitting peekDeliveries/commitDeliveries and committing both the ledger and
the checkpoint only after the sink resolves.
NOT DONE, stated rather than deferred quietly:
* P065 is NOT met. resolve.py is still a second simplified production
engine. Closing it means rewriting it as an adapter over the canonical
service, which cannot be verified from this repo today: services/api is
not built by this repo's Dockerfile, api.0pt.app answers 404, and NO CI
workflow runs its pytest suite. The duplication is now stated at both
sites in code and the removal plan is in STATUS.md. The new ingestion
plane performs no ER, so it adds no third engine.
* P066 (Iceberg) untouched — ST0RE is owned by another workflow. Contract
read only.
* Delivery is AT-LEAST-ONCE, not exactly-once (no per-partition run lock).
* No CDC *source* exists; the contract is met, a CDC connector is not built.
* 0pt does not schedule itself — the schedule row is a declaration ACT0
would read.
VERIFIED: pnpm typecheck clean; pnpm test 365 passed / 25 skipped (was
311/18); pnpm build clean with all four new routes in the route table; lint
unchanged at 12 pre-existing errors. Real Postgres 16
(tests/p0/ingestion-plane-postgres.realinfra.test.ts, 7 passed) plus a real
`docker restart` of that container with an identical read-back through the
production code path — that is P060's "a restart preserves all sources",
measured.
DEPLOYMENT REALITY (curl, 2026-09-04): 0pt.app/api/health returns 200 from
Vercel with db:not_configured; 0pt.app/api/0pt/sources returns 404;
api.0pt.app/health returns 404; cd.yml still guards on a placeholder Coolify
app id. None of the capability layer runs in production. Nothing here is
LIVE_VERIFIED and STATUS.md says so.
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
512 lines
20 KiB
TypeScript
512 lines
20 KiB
TypeScript
/**
|
|
* src/services/source-registry-service.ts
|
|
* ========================================
|
|
* The P0 Source Registry capability surface (Master-DoD §10):
|
|
*
|
|
* source.register -> delegates to SourceService.createSource (owner of
|
|
* 0pt.source.schema.v1) + optional rights/provenance
|
|
* source.query -> listSources + rights/purpose filtering
|
|
* source.describe -> full SourceDescriptor (base + rights + provenance
|
|
* + health + freshness + rate policy)
|
|
* source.health -> record + aggregate health observations
|
|
* source.rate-policy -> already owned by SourceRatePolicyService; composed
|
|
*
|
|
* Tenant isolation: every read path REQUIRES a tenant_id and filters on it —
|
|
* a source registered by tenant A is invisible to tenant B (tested in
|
|
* tests/p0/source-registry.test.ts, incl. the cross-tenant attack case §50).
|
|
* `getHealth`/`getFreshness` now take tenant_id directly and run the same
|
|
* `assertSourceTenant` gate every other accessor does — they are exposed as
|
|
* standalone HTTP routes now (app/api/0pt/sources/[sourceId]/health,
|
|
* .../freshness), so they can no longer rely on an earlier caller having
|
|
* already checked tenancy on their behalf.
|
|
*
|
|
* Purpose enforcement: `assertPurposeAllowed` is the single hook other 0pt
|
|
* capabilities call before using a source for a given purpose (SearchService
|
|
* and DatasetService both do). No rights record ⇒ fail closed (deny).
|
|
*
|
|
* Persistence: rights/provenance/health/freshness now live behind
|
|
* SourceRegistryRepository (src/repositories/source-registry-repository.ts)
|
|
* — real Postgres (`opt_source_rights`/`opt_source_provenance`/
|
|
* `opt_source_health_checks`/`opt_source_freshness`,
|
|
* sql/002_source_registry.sql) when DATABASE_URL is configured, in-memory
|
|
* Maps otherwise. `resolveStoreMode` still fails production boot hard with
|
|
* no DATABASE_URL (Master-DoD §3) — the difference from before is that a
|
|
* configured production now actually WORKS (opt_sources/etc. exist), not
|
|
* just fails safely.
|
|
*/
|
|
import { createAud0Client, type Aud0Client } from "../clients/aud0-client";
|
|
import { SiaxError } from "../platform/errors";
|
|
import {
|
|
createSourceRegistryRepository,
|
|
type SourceRegistryRepository,
|
|
} from "../repositories/source-registry-repository";
|
|
import { SourceService, type SourceServiceOptions } from "./source-service";
|
|
import { SourceRatePolicyService } from "./source-rate-policy-service";
|
|
import type { CreateSourceInput, Source } from "../types/source";
|
|
import type {
|
|
SourceAuthorization,
|
|
SourceClassification,
|
|
SourceDescriptor,
|
|
SourceFreshness,
|
|
SourceHealth,
|
|
SourceHealthCheck,
|
|
SourceHealthStatus,
|
|
SourceProvenance,
|
|
SourceRights,
|
|
SourceSchedule,
|
|
} from "../types/source-registry";
|
|
|
|
export interface RegisterSourceInput extends CreateSourceInput {
|
|
rights?: Omit<SourceRights, "source_id" | "tenant_id">;
|
|
provenance?: Omit<SourceProvenance, "source_id" | "tenant_id" | "updated_at">;
|
|
authorization?: Omit<SourceAuthorization, "source_id" | "tenant_id">;
|
|
schedule?: Omit<SourceSchedule, "source_id" | "tenant_id" | "updated_at">;
|
|
}
|
|
|
|
/**
|
|
* Actor ids that may never appear as an authorization's `granted_by`.
|
|
* P060 requires an accountable HUMAN decision behind a source going live;
|
|
* a service identity granting itself access is exactly the rubber stamp the
|
|
* gate exists to prevent.
|
|
*/
|
|
const NON_ACCOUNTABLE_ACTORS = /^(system|svc_|service:|robot|automation|anonymous)/i;
|
|
|
|
export class SourceRegistryService {
|
|
private readonly repo: SourceRegistryRepository;
|
|
private readonly aud0: Aud0Client;
|
|
readonly sources: SourceService;
|
|
readonly ratePolicies: SourceRatePolicyService;
|
|
|
|
constructor(opts?: {
|
|
aud0BaseUrl?: string;
|
|
/** Explicit DATABASE_URL override — mainly for tests. Falls back to resolveStoreMode's own env lookup when omitted. */
|
|
databaseUrl?: string;
|
|
sourceService?: SourceService;
|
|
ratePolicyService?: SourceRatePolicyService;
|
|
/** Inject a repository directly (tests) — bypasses resolveStoreMode entirely. */
|
|
repository?: SourceRegistryRepository;
|
|
}) {
|
|
this.aud0 = createAud0Client(opts?.aud0BaseUrl);
|
|
this.repo = opts?.repository ?? createSourceRegistryRepository(opts?.databaseUrl);
|
|
const sourceServiceOpts: SourceServiceOptions = {
|
|
aud0BaseUrl: opts?.aud0BaseUrl,
|
|
databaseUrl: opts?.databaseUrl,
|
|
};
|
|
this.sources = opts?.sourceService ?? new SourceService(sourceServiceOpts);
|
|
this.ratePolicies =
|
|
opts?.ratePolicyService ??
|
|
new SourceRatePolicyService({ aud0BaseUrl: opts?.aud0BaseUrl, sourceService: this.sources });
|
|
}
|
|
|
|
// -------------------------------------------------------------------------
|
|
// source.register
|
|
// -------------------------------------------------------------------------
|
|
async registerSource(input: RegisterSourceInput): Promise<SourceDescriptor> {
|
|
const { rights, provenance, authorization, schedule, ...base } = input;
|
|
// P060: registration NEVER activates. A caller asking for status "active"
|
|
// gets the source created "pending" and must go through activateSource,
|
|
// which checks the classification. Silently honouring `status: "active"`
|
|
// here would leave the gate wide open to its most obvious bypass.
|
|
const requestedActive = base.status === "active";
|
|
const source = await this.sources.createSource({
|
|
...base,
|
|
status: requestedActive ? "pending" : base.status,
|
|
});
|
|
if (rights) {
|
|
await this.setRights({ ...rights, source_id: source.source_id, tenant_id: source.tenant_id });
|
|
}
|
|
if (provenance) {
|
|
await this.setProvenance({
|
|
...provenance,
|
|
source_id: source.source_id,
|
|
tenant_id: source.tenant_id,
|
|
updated_at: new Date().toISOString(),
|
|
});
|
|
}
|
|
if (authorization) {
|
|
await this.setAuthorization({
|
|
...authorization,
|
|
source_id: source.source_id,
|
|
tenant_id: source.tenant_id,
|
|
});
|
|
}
|
|
if (schedule) {
|
|
await this.setSchedule({
|
|
...schedule,
|
|
source_id: source.source_id,
|
|
tenant_id: source.tenant_id,
|
|
updated_at: new Date().toISOString(),
|
|
});
|
|
}
|
|
// A registration that arrived fully classified AND asked to be active is
|
|
// activated here — through the same gate, not around it.
|
|
if (requestedActive) {
|
|
await this.activateSource(source.source_id, source.tenant_id, base.created_by);
|
|
}
|
|
return this.describeSource(source.source_id, source.tenant_id);
|
|
}
|
|
|
|
// -------------------------------------------------------------------------
|
|
// P060 authorization / schedule
|
|
// -------------------------------------------------------------------------
|
|
async setAuthorization(auth: SourceAuthorization): Promise<SourceAuthorization> {
|
|
await this.assertSourceTenant(auth.source_id, auth.tenant_id);
|
|
if (!auth.granted_by || NON_ACCOUNTABLE_ACTORS.test(auth.granted_by)) {
|
|
throw new SiaxError(
|
|
"INVALID_REQUEST",
|
|
"0pt.source_registry",
|
|
`authorization.granted_by must name an accountable actor, not "${auth.granted_by}"`,
|
|
);
|
|
}
|
|
await this.repo.setAuthorization(auth);
|
|
await this.emit("0pt.source_authorization_set", auth.tenant_id, auth.source_id, {
|
|
basis: auth.basis,
|
|
granted_by: auth.granted_by,
|
|
expires_at: auth.expires_at,
|
|
evidence_ref: auth.evidence_ref,
|
|
});
|
|
return auth;
|
|
}
|
|
|
|
async getAuthorization(sourceId: string, tenantId: string): Promise<SourceAuthorization | undefined> {
|
|
await this.assertSourceTenant(sourceId, tenantId);
|
|
return this.repo.getAuthorization(sourceId);
|
|
}
|
|
|
|
async setSchedule(schedule: SourceSchedule): Promise<SourceSchedule> {
|
|
await this.assertSourceTenant(schedule.source_id, schedule.tenant_id);
|
|
if (schedule.interval_seconds !== undefined && schedule.interval_seconds <= 0) {
|
|
throw new SiaxError(
|
|
"INVALID_REQUEST",
|
|
"0pt.source_registry",
|
|
"schedule.interval_seconds must be positive",
|
|
);
|
|
}
|
|
await this.repo.setSchedule(schedule);
|
|
await this.emit("0pt.source_schedule_set", schedule.tenant_id, schedule.source_id, {
|
|
mode: schedule.mode,
|
|
interval_seconds: schedule.interval_seconds,
|
|
enabled: schedule.enabled,
|
|
});
|
|
return schedule;
|
|
}
|
|
|
|
async getSchedule(sourceId: string, tenantId: string): Promise<SourceSchedule | undefined> {
|
|
await this.assertSourceTenant(sourceId, tenantId);
|
|
return this.repo.getSchedule(sourceId);
|
|
}
|
|
|
|
// -------------------------------------------------------------------------
|
|
// P060 classification + activation gate
|
|
// -------------------------------------------------------------------------
|
|
/**
|
|
* Is this source classified? A source is classified when ALL of the
|
|
* following are on record:
|
|
* - an accountable owner,
|
|
* - an unexpired authorization from an accountable actor,
|
|
* - rights: a license id AND at least one allowed purpose,
|
|
* - provenance: how the data is obtained.
|
|
*
|
|
* Everything else in the registry (schedule, schema, checkpoint) describes
|
|
* HOW a source runs, not WHETHER it may — those are deliberately not part
|
|
* of the gate, and that choice is documented here rather than left implicit.
|
|
*/
|
|
async classifySource(
|
|
sourceId: string,
|
|
tenantId: string,
|
|
now: Date = new Date(),
|
|
): Promise<SourceClassification> {
|
|
const source = await this.assertSourceTenant(sourceId, tenantId);
|
|
const missing: string[] = [];
|
|
|
|
if (!source.owner_id) missing.push("owner_id is not set");
|
|
|
|
const auth = await this.repo.getAuthorization(sourceId);
|
|
if (!auth) {
|
|
missing.push("no authorization on record");
|
|
} else {
|
|
if (!auth.granted_by || NON_ACCOUNTABLE_ACTORS.test(auth.granted_by)) {
|
|
missing.push(`authorization.granted_by "${auth.granted_by}" is not an accountable actor`);
|
|
}
|
|
if (auth.expires_at && new Date(auth.expires_at).getTime() <= now.getTime()) {
|
|
missing.push(`authorization expired at ${auth.expires_at}`);
|
|
}
|
|
}
|
|
|
|
const rights = await this.repo.getRights(sourceId);
|
|
if (!rights) {
|
|
missing.push("no rights record (license/purpose) on record");
|
|
} else {
|
|
if (!rights.license_id) missing.push("rights.license_id is empty");
|
|
if (rights.allowed_purposes.length === 0) missing.push("rights.allowed_purposes is empty");
|
|
}
|
|
|
|
const provenance = await this.repo.getProvenance(sourceId);
|
|
if (!provenance) missing.push("no provenance record (sourcing method) on record");
|
|
|
|
return { source_id: sourceId, classified: missing.length === 0, missing };
|
|
}
|
|
|
|
/**
|
|
* P060 DoD: an UNCLASSIFIED source cannot be activated. Fail-closed —
|
|
* every missing fact is named at once, and nothing is written when the
|
|
* check fails.
|
|
*/
|
|
async activateSource(sourceId: string, tenantId: string, actorId?: string): Promise<Source> {
|
|
const classification = await this.classifySource(sourceId, tenantId);
|
|
if (!classification.classified) {
|
|
await this.emit("0pt.source_activation_denied", tenantId, sourceId, {
|
|
missing: classification.missing,
|
|
});
|
|
throw new SiaxError(
|
|
"AUTHORIZATION_DENIED",
|
|
"0pt.source_registry",
|
|
`source ${sourceId} is not classified and cannot be activated: ${classification.missing.join("; ")}`,
|
|
);
|
|
}
|
|
const source = await this.sources.updateStatus(sourceId, "active", actorId);
|
|
await this.emit("0pt.source_activated", tenantId, sourceId, { activated_by: actorId ?? null });
|
|
return source;
|
|
}
|
|
|
|
/** Take a source out of service. Always allowed — deactivating is never gated. */
|
|
async deactivateSource(
|
|
sourceId: string,
|
|
tenantId: string,
|
|
status: "paused" | "archived" | "deprecated" = "paused",
|
|
actorId?: string,
|
|
): Promise<Source> {
|
|
await this.assertSourceTenant(sourceId, tenantId);
|
|
const source = await this.sources.updateStatus(sourceId, status, actorId);
|
|
await this.emit("0pt.source_deactivated", tenantId, sourceId, { status });
|
|
return source;
|
|
}
|
|
|
|
// -------------------------------------------------------------------------
|
|
// rights / license / purpose metadata
|
|
// -------------------------------------------------------------------------
|
|
async setRights(rights: SourceRights): Promise<SourceRights> {
|
|
await this.assertSourceTenant(rights.source_id, rights.tenant_id);
|
|
await this.repo.setRights(rights);
|
|
await this.emit("0pt.source_rights_set", rights.tenant_id, rights.source_id, {
|
|
license_id: rights.license_id,
|
|
allowed_purposes: rights.allowed_purposes,
|
|
redistribution: rights.redistribution,
|
|
});
|
|
return rights;
|
|
}
|
|
|
|
async getRights(sourceId: string, tenantId: string): Promise<SourceRights | undefined> {
|
|
await this.assertSourceTenant(sourceId, tenantId);
|
|
return this.repo.getRights(sourceId);
|
|
}
|
|
|
|
/**
|
|
* The purpose gate other capabilities call before using a source. FAIL
|
|
* CLOSED: no rights record, or a purpose outside the allowlist, throws
|
|
* AUTHORIZATION_DENIED.
|
|
*/
|
|
async assertPurposeAllowed(sourceId: string, tenantId: string, purpose: string): Promise<SourceRights> {
|
|
await this.assertSourceTenant(sourceId, tenantId);
|
|
const rights = await this.repo.getRights(sourceId);
|
|
if (!rights) {
|
|
throw new SiaxError(
|
|
"AUTHORIZATION_DENIED",
|
|
"0pt.source_registry",
|
|
`source ${sourceId} has no rights record — purpose "${purpose}" denied (fail closed)`,
|
|
);
|
|
}
|
|
if (!rights.allowed_purposes.includes(purpose)) {
|
|
throw new SiaxError(
|
|
"AUTHORIZATION_DENIED",
|
|
"0pt.source_registry",
|
|
`purpose "${purpose}" is not in source ${sourceId}'s allowed purposes`,
|
|
);
|
|
}
|
|
return rights;
|
|
}
|
|
|
|
// -------------------------------------------------------------------------
|
|
// provenance / freshness
|
|
// -------------------------------------------------------------------------
|
|
async setProvenance(prov: SourceProvenance): Promise<SourceProvenance> {
|
|
await this.assertSourceTenant(prov.source_id, prov.tenant_id);
|
|
await this.repo.setProvenance(prov);
|
|
await this.emit("0pt.source_provenance_set", prov.tenant_id, prov.source_id, {
|
|
method: prov.method,
|
|
authority: prov.authority,
|
|
cadence: prov.cadence,
|
|
});
|
|
return prov;
|
|
}
|
|
|
|
async getProvenance(sourceId: string, tenantId: string): Promise<SourceProvenance | undefined> {
|
|
await this.assertSourceTenant(sourceId, tenantId);
|
|
return this.repo.getProvenance(sourceId);
|
|
}
|
|
|
|
/** Record a successful data observation (drives freshness). */
|
|
async recordSuccess(sourceId: string, tenantId: string, atIso?: string): Promise<void> {
|
|
await this.assertSourceTenant(sourceId, tenantId);
|
|
await this.repo.recordSuccess(sourceId, tenantId, atIso ?? new Date().toISOString());
|
|
}
|
|
|
|
async getFreshness(sourceId: string, tenantId: string, now: Date = new Date()): Promise<SourceFreshness> {
|
|
await this.assertSourceTenant(sourceId, tenantId);
|
|
const prov = await this.repo.getProvenance(sourceId);
|
|
const last = await this.repo.getLastSuccess(sourceId);
|
|
const sla = prov?.freshness_sla_hours;
|
|
if (!last) {
|
|
return { source_id: sourceId, freshness_sla_hours: sla, fresh: sla === undefined };
|
|
}
|
|
const ageHours = (now.getTime() - new Date(last).getTime()) / 3_600_000;
|
|
return {
|
|
source_id: sourceId,
|
|
last_success_at: last,
|
|
freshness_sla_hours: sla,
|
|
age_hours: Math.round(ageHours * 100) / 100,
|
|
fresh: sla === undefined ? true : ageHours <= sla,
|
|
};
|
|
}
|
|
|
|
// -------------------------------------------------------------------------
|
|
// source.health
|
|
// -------------------------------------------------------------------------
|
|
async recordHealthCheck(check: SourceHealthCheck): Promise<SourceHealth> {
|
|
await this.assertSourceTenant(check.source_id, check.tenant_id);
|
|
await this.repo.appendHealthCheck(check);
|
|
if (check.status === "up") {
|
|
await this.recordSuccess(check.source_id, check.tenant_id, check.checked_at);
|
|
}
|
|
return this.getHealth(check.source_id, check.tenant_id);
|
|
}
|
|
|
|
async getHealth(sourceId: string, tenantId: string): Promise<SourceHealth> {
|
|
await this.assertSourceTenant(sourceId, tenantId);
|
|
// Newest-first (repository contract) — index 0 is the most recent check.
|
|
const list = await this.repo.listHealthChecks(sourceId);
|
|
const last = list[0];
|
|
let consecutiveFailures = 0;
|
|
for (const check of list) {
|
|
if (check.status === "down" || check.status === "degraded") consecutiveFailures++;
|
|
else break;
|
|
}
|
|
const status: SourceHealthStatus = last?.status ?? "unknown";
|
|
return { source_id: sourceId, status, last_check: last, consecutive_failures: consecutiveFailures };
|
|
}
|
|
|
|
// -------------------------------------------------------------------------
|
|
// source.query / source.describe
|
|
// -------------------------------------------------------------------------
|
|
async querySources(filter: {
|
|
tenant_id: string;
|
|
project_id?: string;
|
|
source_type?: Source["source_type"];
|
|
status?: Source["status"];
|
|
/** Only sources whose rights allowlist includes this purpose. */
|
|
purpose?: string;
|
|
/** Only sources with at most this redistribution openness. */
|
|
redistribution?: SourceRights["redistribution"];
|
|
}): Promise<Source[]> {
|
|
if (!filter.tenant_id) {
|
|
throw new SiaxError("INVALID_REQUEST", "0pt.source_registry", "tenant_id is required on source.query");
|
|
}
|
|
let results = await this.sources.listSources({
|
|
tenant_id: filter.tenant_id,
|
|
project_id: filter.project_id,
|
|
source_type: filter.source_type,
|
|
status: filter.status,
|
|
});
|
|
if (filter.purpose !== undefined || filter.redistribution !== undefined) {
|
|
const rightsEntries = await Promise.all(
|
|
results.map(async (s) => [s.source_id, await this.repo.getRights(s.source_id)] as const),
|
|
);
|
|
const rightsBySource = new Map(rightsEntries);
|
|
if (filter.purpose !== undefined) {
|
|
results = results.filter((s) => {
|
|
const rights = rightsBySource.get(s.source_id);
|
|
return rights !== undefined && rights.allowed_purposes.includes(filter.purpose as string);
|
|
});
|
|
}
|
|
if (filter.redistribution !== undefined) {
|
|
results = results.filter(
|
|
(s) => rightsBySource.get(s.source_id)?.redistribution === filter.redistribution,
|
|
);
|
|
}
|
|
}
|
|
return results;
|
|
}
|
|
|
|
async describeSource(sourceId: string, tenantId: string): Promise<SourceDescriptor> {
|
|
const source = await this.assertSourceTenant(sourceId, tenantId);
|
|
const [policies, rights, provenance, health, freshness, authorization, schedule, classification] =
|
|
await Promise.all([
|
|
this.ratePolicies.listSourceRatePolicies({ tenant_id: tenantId, source_id: sourceId }),
|
|
this.repo.getRights(sourceId),
|
|
this.repo.getProvenance(sourceId),
|
|
this.getHealth(sourceId, tenantId),
|
|
this.getFreshness(sourceId, tenantId),
|
|
this.repo.getAuthorization(sourceId),
|
|
this.repo.getSchedule(sourceId),
|
|
this.classifySource(sourceId, tenantId),
|
|
]);
|
|
return {
|
|
source,
|
|
rights,
|
|
provenance,
|
|
health,
|
|
freshness,
|
|
rate_policy: policies[0],
|
|
authorization,
|
|
schedule,
|
|
classification,
|
|
};
|
|
}
|
|
|
|
// -------------------------------------------------------------------------
|
|
// internals
|
|
// -------------------------------------------------------------------------
|
|
/**
|
|
* Tenant gate for every read/write: resolves the source AND verifies the
|
|
* caller's tenant owns it. Cross-tenant probes get NOT_FOUND (no existence
|
|
* oracle), same 404-not-403 posture as ST0RE/N0D.
|
|
*/
|
|
private async assertSourceTenant(sourceId: string, tenantId: string): Promise<Source> {
|
|
if (!tenantId) {
|
|
throw new SiaxError("INVALID_REQUEST", "0pt.source_registry", "tenant_id is required");
|
|
}
|
|
let source: Source;
|
|
try {
|
|
source = await this.sources.getSource(sourceId);
|
|
} catch {
|
|
throw new SiaxError("NOT_FOUND", "0pt.source_registry", `source ${sourceId} not found`);
|
|
}
|
|
if (source.tenant_id !== tenantId) {
|
|
throw new SiaxError("NOT_FOUND", "0pt.source_registry", `source ${sourceId} not found`);
|
|
}
|
|
return source;
|
|
}
|
|
|
|
private async emit(
|
|
eventType: string,
|
|
tenantId: string,
|
|
sourceId: string,
|
|
payload: Record<string, unknown>,
|
|
): Promise<void> {
|
|
await this.aud0.ingestEvent({
|
|
event_id: `evt_${eventType}_${sourceId}_${Date.now()}`,
|
|
event_type: eventType,
|
|
tenant_id: tenantId,
|
|
project_id: "p_platform",
|
|
actor_id: "svc_0pt",
|
|
app_id: "0pt",
|
|
capability_id: eventType.replace(/^0pt\./, "0pt."),
|
|
resource_type: "source",
|
|
resource_id: sourceId,
|
|
risk_level: "L2",
|
|
payload,
|
|
created_at: new Date().toISOString(),
|
|
});
|
|
}
|
|
}
|