fix(data-os): P014 — close tenant-isolation + idempotency gaps, execute real restore drill
CI (SIAX Cloud) / quality (push) Successful in 51s
Security Scan (SIAX Cloud) / security (push) Successful in 26s
CI (SIAX Cloud) / security (push) Successful in 28s
CI (SIAX Cloud) / integration (push) Successful in 1m1s
CI (SIAX Cloud) / integration (pull_request) Successful in 30s
CI (SIAX Cloud) / security (pull_request) Successful in 24s
CI (SIAX Cloud) / quality (pull_request) Successful in 48s
Security Scan (SIAX Cloud) / security (pull_request) Successful in 22s

Tenant isolation (security-critical): every app/api/0pt/** route reading
tenant_id from client-controlled query/body never cross-checked it against
the token middleware.ts already verified (x-0pt-verified-tenant). Adds a
shared assertTenantMatchesVerified guard and wires it into all 20+ affected
routes. Also closes two full-leak-by-omission bugs found in this pass
(documents/capture and lakehouse/mappings GETs returned every tenant's rows
when tenant_id was simply omitted) and two "no tenant check at all" reads
(rate-policies/[id], lakehouse/mappings/[id]) reachable by ID alone.

Duplicate-delivery idempotency (correctness-critical): POST /api/orders now
honors Idempotency-Key (sql/006, replay-or-409-on-conflict, Postgres +
in-memory); the Python orchestrator's POST /orders/{id}/run — the actual
delivery trigger — is now idempotent under an asyncio lock, closing a real
concurrent-request race, not just the sequential-retry case.

Engine-duplication resolution: adaptive_router.py's docstring wrongly
invited a bypass ("see b0ts for scraping") contradicting the product's own
compliance stance; corrected, and given real (additive, not escalating)
block-signal tracking so a rate-limited/blocked source is excluded from the
plan instead of hammered. Also fixes a previously-undiscovered production
bug: the Python engine's only call into the TS quality bridge
(X-Internal-Token) was unconditionally 401'd by middleware.ts, which never
checked that credential — the advisory quality bridge was unreachable in any
environment enforcing auth.

Restore drill: executed for real against throwaway Docker Postgres
containers (not simulated). Found and fixed two real blockers along the way:
sql/004 referenced opt_datasets, which no migration ever created (chain was
never actually runnable end to end); lib/db.ts hardcoded ssl:"require",
making any non-TLS Postgres (incl. compose.dev.yaml's own local-dev
container) unreachable. After both fixes: full migration chain applies
cleanly, pg_dump/pg_restore round-trip is byte-identical (row counts +
content checksums), and real application code (lib/orders-store.ts,
SourceRegistryService's Postgres repository) reads the restored data back
correctly, including cross-tenant isolation still holding.

82 new tests (55 TS + 27 Python), all passing; full existing suite (291 TS +
55 Python) still green; typecheck clean on every changed/added file.

Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>
This commit is contained in:
Simon Axelsson
2026-09-03 12:00:43 +02:00
co-authored by Claude Sonnet 5
parent e7516697b3
commit dc6aaa0e10
50 changed files with 1820 additions and 52 deletions
@@ -125,9 +125,21 @@ export class DatasetIcebergMappingService {
return this.toMapping(record);
}
async getMapping(mappingId: string): Promise<DatasetIcebergMapping> {
// P014: `tenantId` is optional so trusted in-process callers (existing
// unit tests, and internal callers that already resolved the tenant some
// other way) are unaffected -- but both HTTP routes that reach this method
// now always pass it, closing a real gap: `getMapping` used to be
// reachable by mapping ID alone with NO tenant check anywhere, so any
// authenticated caller could read -- or, via requestIcebergWrite below,
// WRITE a new snapshot onto -- another tenant's dataset<->Iceberg-table
// mapping just by guessing/enumerating dataset_iceberg_mapping_id. A
// mismatched tenantId reports identically to "not found" (no existence
// oracle), matching SourceRegistryService.assertSourceTenant's convention.
async getMapping(mappingId: string, tenantId?: string): Promise<DatasetIcebergMapping> {
const record = this.mappings.get(mappingId);
if (!record) throw new Error(`Dataset Iceberg mapping ${mappingId} not found`);
if (!record || (tenantId && record.tenant_id !== tenantId)) {
throw new Error(`Dataset Iceberg mapping ${mappingId} not found`);
}
return this.toMapping(record);
}
+16 -1
View File
@@ -249,7 +249,22 @@ export class DocumentExtractionService {
return record;
}
async listExtractionsForDocument(documentId: string): Promise<DocumentExtractionRecord[]> {
// P014: `tenantId` is optional so existing trusted in-process callers are
// unaffected, but the HTTP route now always supplies it -- closing a real
// gap: this used to be reachable by documentId alone with NO tenant
// check anywhere, so any authenticated caller could list another
// tenant's OCR/structuring extraction results (potentially containing
// extracted document content, e.g. invoice fields) by guessing or
// enumerating documentId. A mismatched tenantId reports identically to
// "not found" (no existence oracle), same convention as the rest of the
// surface.
async listExtractionsForDocument(documentId: string, tenantId?: string): Promise<DocumentExtractionRecord[]> {
if (tenantId) {
const document = await this.captureService.getDocument(documentId).catch(() => undefined);
if (!document || document.tenant_id !== tenantId) {
throw new Error(`Document ${documentId} not found`);
}
}
return Array.from(this.extractions.values()).filter((r) => r.document_id === documentId);
}
}
+10 -1
View File
@@ -65,6 +65,15 @@ export interface MapDatasetToIcebergTableResult {
export interface RequestIcebergWriteInput {
dataset_iceberg_mapping_id: string;
/**
* P014: optional so existing trusted in-process callers are unaffected,
* but the HTTP route (app/api/0pt/lakehouse/mappings/[mappingId]/writes)
* always supplies it now. When present, it must match the mapping's own
* tenant_id or the write is refused as not-found -- see getMapping's
* doc comment for the gap this closes (a foreign mapping_id used to let
* any caller register a snapshot against another tenant's table).
*/
tenant_id?: string;
actor_id: string;
correlation_id: string;
snapshot_type?: TableSnapshotType;
@@ -176,7 +185,7 @@ export class IcebergWriteService {
}
async requestIcebergWrite(input: RequestIcebergWriteInput): Promise<RequestIcebergWriteResult> {
const mapping = await this.mappingService.getMapping(input.dataset_iceberg_mapping_id);
const mapping = await this.mappingService.getMapping(input.dataset_iceberg_mapping_id, input.tenant_id);
if (mapping.mapping_status !== "active") {
throw new Error(`Dataset Iceberg mapping ${input.dataset_iceberg_mapping_id} is not active (status=${mapping.mapping_status})`);
}
+13 -2
View File
@@ -103,9 +103,20 @@ export class SourceRatePolicyService {
return this.toPolicy(record);
}
async getSourceRatePolicy(sourceRatePolicyId: string): Promise<SourceRatePolicy> {
// P014: `tenantId` is optional so existing internal callers that already
// resolved a policy from a tenant-scoped list (or are trusted, in-process
// callers) are unaffected — but the HTTP route now always passes it, which
// closes a real gap: this lookup used to be reachable by rate-policy ID
// alone with NO tenant check anywhere, so any authenticated caller could
// read another tenant's crawl-budget/rate-limit/compliance-notes record
// just by guessing or enumerating IDs. A mismatched tenantId is reported
// identically to "not found" (no existence oracle), matching the
// convention SourceRegistryService.assertSourceTenant already established.
async getSourceRatePolicy(sourceRatePolicyId: string, tenantId?: string): Promise<SourceRatePolicy> {
const record = this.policies.get(sourceRatePolicyId);
if (!record) throw new Error(`Source rate policy ${sourceRatePolicyId} not found`);
if (!record || (tenantId && record.tenant_id !== tenantId)) {
throw new Error(`Source rate policy ${sourceRatePolicyId} not found`);
}
return this.toPolicy(record);
}