117 Commits

This Branch

This Branch

All Branches

Author	SHA1	Message	Date
opt-vehiq-work salvage	1d63f63dfd	chore(salvage): preserve uncommitted WIP from /opt/vehiq-work/0pt-p060-p068_0pt before machine cleanup (2026-09-27) ... This snapshot was found sitting uncommitted in a stale /opt/vehiq-work lane clone dated ~2026-09-09..18, base branch: feat/p060-p068-ingestion-plane. Not reviewed, not verified against current main. Pushed only so the directory could be deleted without losing the diff.	2026-09-27 12:58:50 +02:00
Simon AxelssonandClaude Opus 5	9267576b7c	feat(data-os): P060-P068 durable ingestion plane — source classification gate, connector contract, common envelope, schema evolution, quality enforcement, queryable lineage, idempotent backfill ... BEFORE_STATE (verified, with evidence): * P060 PARTIAL/BROKEN. The registry persisted rights/provenance/health/ freshness to Postgres, but carried no authorization, no schedule, no checkpoint, no schema and no owner distinct from created_by — and the activation DoD was inverted: postgres-source-repository.ts wrote `${input.status ?? "active"}`, so a source with no rights, no provenance and no authorization was ACTIVE the instant it was registered. No activation operation existed at all. * P061 MISSING. services/api/sources/connectors.py had per-source fetch_*() functions only — no lifecycle, no cursor, no resume, no health, no conformance suite. * P062 MISSING, already documented as such in services/api/tests/test_known_gaps.py:10-19 (no CDC pipeline anywhere; workers/pipeline.py has no persistence at all). * P063 MISSING. schema_version existed as a free string; no comparison, no policy, no versioned source schema. * P064 PARTIAL. Rules/runs/issues/scores existed, but a failing rule only opened an issue and the data flowed on. No quarantine anywhere. * P065 DUPLICATE (confirmed). Two ER engines: src/er/entity-resolution- service.ts (canonical) and services/api/workers/resolve.py (simplified, still called by pipeline.py:110). * P067 PARTIAL. Lineage was emitted to AUD0 but nothing here could ANSWER a lineage question — no store, no query, no route. * P068 MISSING, documented in test_known_gaps.py:44-52 ("no delivery-store persistence layer exists to dedupe a backfill against"). BUILT (only what was PARTIAL/BROKEN/MISSING): * P060 sql/007_ingestion_plane.sql adds authorization/schedule/schemas/ checkpoints/ledger/quarantine/lineage tables, owner_id, and flips the opt_sources status DEFAULT to 'pending' so even a raw psql INSERT cannot produce an unclassified-but-active source. SourceRegistryService gains classifySource/activateSource/deactivateSource + authorization/schedule accessors; registration NEVER activates (asking for status:"active" goes through the gate, not around it); a service identity cannot authorize a source. Routes: activate (GET classification / POST activate), authorization, schedule. * P061 src/connectors/connector.ts — all nine operations on one interface. src/connectors/conformance.ts — a runnable suite (15 checks, no "skip" verdict). HttpJsonConnector (real HTTP, opaque cursor) and JsonlFileConnector (real file, line offsets, CDC deletes) both pass it. The suite caught a real defect while being written: an API that drops next_cursor at the end left the connector unable to resume, which is why HttpJsonConnectorConfig has cursor_field. * P062 src/types/ingestion.ts + src/ingestion/{envelope,ingestion-runner}.ts. One envelope for batch/stream/cdc; durable per-(source,partition) checkpoints; resume proven by killing a run inside the sink. * P063 src/schema/{record-schema,compatibility,schema-registry}.ts. Two enforcement points, both upstream of any consumer write: a discovery gate before the first read, and per-record conformance before the page reaches the sink. A producer cannot widen its own policy retroactively. * P064 src/quality/quality-policy.ts composes the EXISTING ExpectationEngine (not a second engine) and adds the missing consequence: block / quarantine / warn, with quarantine durable and recoverable. No policy is reported as "none_declared", never as a pass; an unexecutable expectation counts as a violation, never a silent pass. * P067 src/lineage/{lineage-edge,lineage-store}.ts + GET /api/0pt/lineage. AUD0 stays the system of record; this is 0pt's own queryable projection. * P068 opt_ingestion_ledger + IngestionRunner.backfill: bounded, idempotent on repeat, leaves the live checkpoint alone, still re-emits real changes. A BUG THIS PASS WROTE AND CAUGHT: the first IngestionRepository draft combined the idempotency check and its recording into one atomic call BEFORE the sink. The crash/resume test then delivered 7 of 10 records — the page in flight when the sink died was already marked delivered, so the resumed run skipped it as a duplicate. Permanent, silent data loss. Fixed by splitting peekDeliveries/commitDeliveries and committing both the ledger and the checkpoint only after the sink resolves. NOT DONE, stated rather than deferred quietly: * P065 is NOT met. resolve.py is still a second simplified production engine. Closing it means rewriting it as an adapter over the canonical service, which cannot be verified from this repo today: services/api is not built by this repo's Dockerfile, api.0pt.app answers 404, and NO CI workflow runs its pytest suite. The duplication is now stated at both sites in code and the removal plan is in STATUS.md. The new ingestion plane performs no ER, so it adds no third engine. * P066 (Iceberg) untouched — ST0RE is owned by another workflow. Contract read only. * Delivery is AT-LEAST-ONCE, not exactly-once (no per-partition run lock). * No CDC *source* exists; the contract is met, a CDC connector is not built. * 0pt does not schedule itself — the schedule row is a declaration ACT0 would read. VERIFIED: pnpm typecheck clean; pnpm test 365 passed / 25 skipped (was 311/18); pnpm build clean with all four new routes in the route table; lint unchanged at 12 pre-existing errors. Real Postgres 16 (tests/p0/ingestion-plane-postgres.realinfra.test.ts, 7 passed) plus a real `docker restart` of that container with an identical read-back through the production code path — that is P060's "a restart preserves all sources", measured. DEPLOYMENT REALITY (curl, 2026-09-04): 0pt.app/api/health returns 200 from Vercel with db:not_configured; 0pt.app/api/0pt/sources returns 404; api.0pt.app/health returns 404; cd.yml still guards on a placeholder Coolify app id. None of the capability layer runs in production. Nothing here is LIVE_VERIFIED and STATUS.md says so. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>	2026-09-04 17:03:14 +02:00
admin	c5ebceff38	Merge pull request 'P014: close tenant-isolation + idempotency gaps, execute real restore drill' (#29) from feat/p014-data-os-hardening into main	2026-09-03 10:18:09 +00:00
Simon AxelssonandClaude Sonnet 5	dc6aaa0e10	fix(data-os): P014 — close tenant-isolation + idempotency gaps, execute real restore drill ... Tenant isolation (security-critical): every app/api/0pt/** route reading tenant_id from client-controlled query/body never cross-checked it against the token middleware.ts already verified (x-0pt-verified-tenant). Adds a shared assertTenantMatchesVerified guard and wires it into all 20+ affected routes. Also closes two full-leak-by-omission bugs found in this pass (documents/capture and lakehouse/mappings GETs returned every tenant's rows when tenant_id was simply omitted) and two "no tenant check at all" reads (rate-policies/[id], lakehouse/mappings/[id]) reachable by ID alone. Duplicate-delivery idempotency (correctness-critical): POST /api/orders now honors Idempotency-Key (sql/006, replay-or-409-on-conflict, Postgres + in-memory); the Python orchestrator's POST /orders/{id}/run — the actual delivery trigger — is now idempotent under an asyncio lock, closing a real concurrent-request race, not just the sequential-retry case. Engine-duplication resolution: adaptive_router.py's docstring wrongly invited a bypass ("see b0ts for scraping") contradicting the product's own compliance stance; corrected, and given real (additive, not escalating) block-signal tracking so a rate-limited/blocked source is excluded from the plan instead of hammered. Also fixes a previously-undiscovered production bug: the Python engine's only call into the TS quality bridge (X-Internal-Token) was unconditionally 401'd by middleware.ts, which never checked that credential — the advisory quality bridge was unreachable in any environment enforcing auth. Restore drill: executed for real against throwaway Docker Postgres containers (not simulated). Found and fixed two real blockers along the way: sql/004 referenced opt_datasets, which no migration ever created (chain was never actually runnable end to end); lib/db.ts hardcoded ssl:"require", making any non-TLS Postgres (incl. compose.dev.yaml's own local-dev container) unreachable. After both fixes: full migration chain applies cleanly, pg_dump/pg_restore round-trip is byte-identical (row counts + content checksums), and real application code (lib/orders-store.ts, SourceRegistryService's Postgres repository) reads the restored data back correctly, including cross-tenant isolation still holding. 82 new tests (55 TS + 27 Python), all passing; full existing suite (291 TS + 55 Python) still green; typecheck clean on every changed/added file. Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>	2026-09-03 12:00:43 +02:00
admin	40c3e2fbee	Merge pull request 'feat(data-os): P010 completion — Source Registry persistence + TS/Python quality bridge + provenance + fault-injection tests' (#28) from feat/p010-data-os-completion into main	2026-09-01 19:10:26 +00:00
Simon AxelssonandClaude Sonnet 5	e7516697b3	docs(p010): implementation report + production evidence for Data-OS completion ... From-scratch verification (fresh clone, fresh installs) backing up the P010 completion pass: 256/256 vitest, 28/28 pytest, clean typecheck/build on the committed tree, pre-existing-only lint noise. Documents that OPT_QUALITY_BRIDGE_ENABLED ships off and stays off in this PR -- activating it in production is left as Simon's decision, with a readiness checklist. Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>	2026-09-01 21:08:20 +02:00
Simon AxelssonandClaude Sonnet 5	9740e20818	feat(data-os): TS/Python quality bridge + provenance fields + fault-injection tests ... Continues P010 (branch renamed from feat/p010-source-registry-persistence): closes the three remaining BEFORE_STATE.md gaps (§2 TS/Python split-brain, §3 missing provenance fields, §4 untested failure modes), all additive to the production pipeline — the Python QA filter's delivery decision is unchanged by any of this. TS/Python bridge (§2): - New route surface app/api/0pt/quality/* (rules/runs/issues CRUD + POST /api/0pt/quality/bridge-run) over the shared QualityService singleton (src/services/quality.ts), fixing the same "no shared instance" defect the Source Registry had. - src/quality/quality-bridge.ts auto-provisions a deterministic, tenant-scoped baseline rule set on first use so the Python engine never pre-registers rules through the CRUD surface. - services/api/workers/quality_bridge.py: the Python-side HTTP client. Feature-flagged off by default (OPT_QUALITY_BRIDGE_ENABLED), fails open on any error (never raises into run_job, never changes `delivered`). Advisory only — real per-row gating against the TS engine isn't implementable yet (ExpectationEngine returns dataset-level pass/fail counts + samples, not a per-row boolean array); documented, not faked. - Live-verified end-to-end: `next dev` (AUTH_ENFORCE=false) + the real Python pipeline hitting the real brreg.no API, round-tripping through the real bridge route, receiving a real QualityService score. Provenance (§3): sources/connectors.py's Provenance dataclass gains retrieved_at, transform, connector_version, producer, quality, entity_resolution, artifact_ref (all previously absent from the real dataflow entirely) via a make_provenance() factory used by every connector. resolve.py stamps entity_resolution on merged records' provenance (never on unmerged ones). pipeline.py stamps artifact_ref at delivery and quality once the bridge succeeds. Failure tests (§4), real fault injection against real code (services/api, new pytest + respx suite, not vitest mocks): - dedup-determinism + duplicate-input + out-of-order: resolve.py under shuffled/interleaved/exhaustive-permutation input, plus an explicit, verified test of the one case that IS legitimately order-sensitive (non-transitive similarity in a greedy clusterer) instead of hiding it. - schema-evolution: respx-mocked upstream responses with added/removed/ alien-shaped fields through the real brreg connector + pipeline. - restart: connector exceptions mid-job (including one from respx itself) never crash run_job. - backfill: the real property that holds given no persistence layer exists (replay is compute-idempotent) — NOT a fabricated dedup-store test. - CDC-interruption and true backfill/delivery-store idempotency: explicitly documented as not implementable against this codebase (no CDC pipeline, no delivery store exist anywhere) via skip markers with reasons, in tests/test_known_gaps.py, rather than silently dropped or faked. Testing: 256/256 vitest (249 pre-existing + 7 new bridge route/service tests) and 28/28 pytest (2 documented skips) green locally. tsc -p tsconfig.check.json has zero new errors (14 pre-existing errors, all in the untracked/unaudited src/fleet, src/graph, src/metrics, src/projections paths this pass does not touch, per BEFORE_STATE.md). No PR opened, per instructions. Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>	2026-09-01 21:02:14 +02:00
Simon AxelssonandClaude Sonnet 5	07b13bd833	feat(data-os): Source Registry Postgres persistence + full HTTP route surface ... Replaces the in-memory (per-instance, not even per-process-shared) Maps in SourceService/SourceRegistryService with real Postgres persistence, and adds the register/query/describe/health/rights/provenance/freshness HTTP routes that never existed (grep for SourceRegistryService in app/ previously returned nothing). - sql/002_source_registry.sql: opt_sources, opt_source_rights, opt_source_provenance, opt_source_health_checks, opt_source_freshness. Numbered 002 (not 006) because 004's opt_source_rate_policies already forward-references opt_sources, which no sql/ file defined until now — fixes 004's pre-existing broken migration ordering (opt_datasets remains a separate, unrelated, still-open gap in 004, flagged not fixed). - src/repositories/*: SourceRepository / SourceRegistryRepository seams, each with an in-memory implementation (business logic extracted verbatim) and a Postgres implementation, chosen via resolveStoreMode — production still fails hard with no DATABASE_URL (Master-DoD §3), but a configured production now actually works instead of always being blocked. - SourceService / SourceRegistryService: storage moved behind the repository seam; getHealth/getFreshness are now async and tenant-scoped at the accessor itself (needed once they're reachable directly over HTTP, not only after an earlier caller's tenant check). - src/services/source-registry.ts: shared singleton instances for the sources/* route group, fixing the "each route module owned its own private registry" defect (source-rate-policies and .../query previously couldn't see a source registered via POST /api/0pt/sources). - New routes: POST/GET /api/0pt/sources, GET /api/0pt/sources/[id], GET+PUT .../rights, GET+PUT .../provenance, GET+POST .../health, GET .../freshness. - Business logic unchanged: fail-closed purpose gate, cross-tenant NOT_FOUND (no existence oracle, §50), bounded 100-row health history. Tested against a real Postgres container (not mocked) — tests/p0/source-registry-postgres.realinfra.test.ts, env-gated on SOURCE_REGISTRY_TEST_DATABASE_URL, skips honestly when unset. All 249 pre-existing P0 tests + 7 new realinfra tests pass; also smoke-tested the full HTTP route surface end-to-end against next dev + live Postgres. Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>	2026-09-01 20:42:41 +02:00
admin	f8bb0d5ab9	Merge pull request 'feat: SIAX Platforms P0 wave 2 — real-contract platform clients + Data-OS capability layer' (#25) from feat/siax-platforms-p0-data-os into main	2026-08-31 20:26:42 +00:00
Simon AxelssonandClaude Fable 5	62f7b0049f	fix(ci): pin OpenSearch service container workdir — act_runner clobbers WORKDIR-relative ENTRYPOINT ... PR #25 CI run 7745 job 'integration' failed before any test ran: 'exec: "./opensearch-docker-entrypoint.sh": stat ...: no such file or directory'. Root cause (confirmed against the job log's merged container.Config, and reproduced locally with plain docker run -w): this runner's act_runner merges the JOB's WorkingDir (/workspace/sax3l/0pt) into every SERVICE container, clobbering the OpenSearch image's own WORKDIR (/usr/share/opensearch) that its ENTRYPOINT ./opensearch-docker-entrypoint.sh is relative to. ClickHouse's entrypoint is an absolute path so it started fine under the same defect -- a green ClickHouse service is not evidence this class of bug is absent elsewhere. Fix: --workdir /usr/share/opensearch in the opt-ci-opensearch service's options, restoring the image's own working directory. Verified locally: 'docker run -w /workspace/sax3l/0pt opensearchproject/opensearch:2.11.1' reproduces the exact failure; '-w /usr/share/opensearch' fixes it. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>	2026-08-31 22:20:36 +02:00
Simon AxelssonandClaude Fable 5	c47e3b468e	test(integration)+ci+docs: live OpenSearch/ClickHouse suites, CI service containers, STATUS.md (§56) ... - tests/integration/: 10 tests against REAL OpenSearch 2.11 (tenant isolation, purpose/rights filters, generation rebuild/rollback, projection-disaster rebuild §68) + REAL ClickHouse 24.8 (tenant-isolated metric DSL, TTL schema, injection-surface refusals). Honest skip without env - and CI fails hard if the suites skip instead of running. - .gitea/workflows/ci.yml: new blocking 'integration' job with self-identifying service containers (st0re ci.yml's measured alias-collision discipline: run-unique cluster.name / credentials, getent-enumerated IP selection, no-match = hard fail). - STATUS.md: full 8-level Master-DoD classification per capability incl. the honest gaps blocking PRODUCTION_READY. - dependency-mode: NEXT_PHASE=phase-production-build is not a serving runtime (next build page-data collection must compile; guard re-arms at boot) - pinned by contract test. - docs/02-architecture.md: superseded the accurate-but-now-fixed 'every client runs against its mock' paragraph; .env.example: client URLs now actually read from env + new P0 backend vars. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>	2026-08-31 16:56:33 +02:00
admin	9f0de72456	chore(deps): update dependency @tailwindcss/postcss to v4.3.3 (#12)	2026-08-31 14:54:56 +00:00
admin	6be9992cad	chore(deps): update dev-dependencies (non-major) (#14)	2026-08-31 14:54:54 +00:00
Simon AxelssonandClaude Fable 5	b46807da75	feat(data-os): P0 capability layer — source registry, search, quality, dataset versions, metadata, analytics, features, ER ... - Source Registry (§10): rights/license/purpose metadata, provenance, freshness SLA, health checks, describe/query; purpose gate FAILS CLOSED (no rights record => deny); cross-tenant probes => NOT_FOUND. - Search (§10/§42): OpenSearch REST adapter (no SDK) + SearchService with MANDATORY tenant + purpose/rights filtering, versioned index definitions, bulk, generation-per-rebuild (opt-{name}--g{n}) with atomic alias cutover, rollback, failed-rebuild generation discard. OpenSearch never canonical. - Quality (§10): versioned rule registry (GE-JSON interchange), local engine for the GE core subset (never silently passes unsupported types), runs/results/issues/score/trend, whole-run delegation to the GE backend via N0D (0pt.quality.ge_run) - never fabricated results. - Dataset (§10/§5/§53): immutable numbered versions holding canonical st0re:artifact refs only, idempotent createVersion, export registers the ST0RE artifact + AUD0 evidence + OpenLineage. - Metadata (§10): OpenMetadata PUT /api/v1/tables projection adapter + projectAll rebuild path; projection never canonical. - Analytics (§10): ClickHouse HTTP adapter, closed metric-query DSL with FORCED tenant predicate + readonly/max_execution_time/max_result_rows, TTL retention. Zero connection to b0ts/quarantined pipelines. - Features (§10): versioned feature-view registry, real Feast get-online-features adapter, materialize/get-historical as N0D delegation. - ER (§10): generic candidate API with deterministic blocking + transparent scoring + per-field explanations, product hard-rule hook (force both directions), review-band handoff, Splink batch via N0D contract. 93 new unit/contract tests; suite 248 passed / 1 skipped. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>	2026-08-31 16:49:32 +02:00
Simon AxelssonandClaude Fable 5	da58176735	test(contracts): per-client wire-contract suites + §3 production-guard tests ... 38 tests: recording provider double answers with the sibling services' real response shapes; each suite asserts the exact request wire shape (path, method, tenant/actor/idempotency headers, camelCase vs snake_case body) so a regression to the pre-rewrite invented shapes fails loudly. Includes the fail-closed policy cases (missing decision => deny, CL0UD down => typed error, never implicit allow) and the production factory guard for all four clients. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>	2026-08-31 16:37:57 +02:00
Simon AxelssonandClaude Fable 5	b7038d7675	feat(platform): canonical SIAX context/errors + real-contract clients, no prod mock fallback ... - src/platform/: SiaxRequestContext + propagation headers + 12 canonical error codes (documented mirror of cl0ud packages/contract), SiaxError, resource refs (st0re:artifact:...), and the §3/§10 dependency-mode guard: production runtime + unconfigured dependency/store => hard DEPENDENCY_UNAVAILABLE at construction, never silent mock/in-memory. - src/clients/: all four platform clients rewritten against the REAL wire contracts verified on sibling main branches this session: * cl0ud: POST /api/policy/evaluate ({data} envelope, allow+requiresApproval) - the old /v1/cl0ud/policies/evaluate path never existed on CL0UD * st0re: camelCase ArtifactInput on /v1/st0re/artifacts/register incl. X-Tenant-Id/X-Actor-Id + X-Idempotency-Key - the old snake_case body would have been a 400 against the live registry * n0d: required_role + resource_requirements + body idempotency_key + CloudEvents callback_url per n0d createJobSchema; adds getJob/getJobResult * aud0: /v1/aud0/openlineage is AUD0's only lineage ingest - /v1/aud0/lineage/events never existed; legacy shape converts to a spec-shaped OpenLineage RunEvent (deterministic runId dedup) - src/lineage/: OpenLineage subset types + LineageEmitter (never-throw, §16 evidence semantics) - shared SiaxHttp transport: timeouts, typed SiaxError mapping, context propagation, no raw upstream bodies in errors Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>	2026-08-31 16:35:01 +02:00
admin	e452848470	Merge pull request 'fix: COOLIFY_APP_ID from variables (0pt app live in Coolify)' (#24) from fix/coolify-app-id into main	2026-08-31 11:54:34 +00:00
Simon Axelsson	18ea0ecc41	fix: wire COOLIFY_APP_ID from repo variable (0pt app provisioned via API 2026-08-31)	2026-08-31 13:54:21 +02:00
Simon Axelsson	fedaab41ba	fix: auth middleware matcher covers /api/0pt exact path + diag logging for S1 root-cause	2026-08-31 13:05:04 +02:00
admin	00a1b3997d	fix(env): NEXT_PUBLIC_API_URL -> live origin (api.0pt.app is dead) (#23) ... Co-authored-by: sax3l <simon@siax.io> Co-committed-by: sax3l <simon@siax.io>	2026-08-27 04:36:39 +00:00
admin	77e5137c9a	Merge pull request 'docs+chore: repo hygiene -- v2.0 manifests, ADRs, changelog, doctor, guides' (#20) from docs/repo-hygiene-v2-standard into main	2026-08-11 12:32:19 +00:00
admin	5f98379b80	Merge pull request 'fix(health): report real per-dependency status instead of hardcoded ok:true' (#18) from fix/honest-health-check into main	2026-08-11 12:31:36 +00:00
admin	8e4a062f02	Merge pull request 'docs(compliance): add BLOCKERS.md (§1.4)' (#16) from docs/compliance-blockers-2026-08-11 into main	2026-08-11 12:30:59 +00:00
admin	328f8a46cb	docs+chore: repo hygiene -- v2.0 manifests, ADRs, changelog, doctor, guides ... BACKLOG P1-1: siax.repo.json/siax.config.json rewritten to be schema-valid against sax3l/siax-standard's actual v2.0 schemas (lifecycle, operationalState, gitHost, domains-as-objects, consumedBy, children, contracts -- validated with jsonschema, not just eyeballed). config.json restructured to the real deploy-descriptor shape (framework/build/start/ port/healthcheck/environments) instead of duplicating repo.json's identity fields, which the schema doesn't allow. BACKLOG P1-2: 3 ADRs for decisions already made and evidenced elsewhere in this repo (Coolify-not-Vercel, Zitadel-not-NextAuth, Iceberg-format pointer to n0d ADR-0030) -- retroactive documentation, not new decisions. The Stripe/Revolut and services/scrapers/* questions are explicitly NOT written as ADRs here; both are account-owner-gated or actively under review elsewhere (see docs/architecture/decisions/README.md). BACKLOG P1-3: CHANGELOG.md from real git log history (no tags/releases exist yet -- this doesn't invent version numbers that were never cut). BACKLOG P1-19: `pnpm siax:doctor` (scripts/siax-doctor.mjs, AGENT-04) -- zero-dependency, checks required files + manifest shape, prints control-IDs. .devcontainer/devcontainer.json (DX-03). compose.dev.yaml (DX-02 -- UNVERIFIED, no Docker available in this environment to smoke-test, flagged in the file itself). All 4 prompts + the index rewritten to AGENT-03 shape (goal/context/prerequisites/scope/contract/acceptance-criteria/ control-ID/forbidden-actions). BACKLOG P1-20: docs/architecture/ARCHITECTURE.md migrated to the v2.0 9-heading template -- new §5 "Ytor och paritet" (capability x API/CLI/MCP/console coverage table, grounded in the same registry facts docs/02-architecture.md already established), §6-9 renumbered from the old §5-8 (all internal cross-references fixed, CLAUDE.md's own §-numbers left untouched). docs/api/ added (points at contracts/openapi.yaml as the one SSOT instead of duplicating it as drift-prone prose). BACKLOG P1-21: packages/schemas now has a real package.json (was loose JSON files with no package -- pnpm-workspace.yaml's packages/* glob matched an empty package). Verified `pnpm install --frozen-lockfile` and the full typecheck/test/build suite still pass with 2 real workspace projects. apps/ restructuring explicitly NOT attempted (BACKLOG's own effort estimate is "L" / large -- moving the whole app tree is a bigger, riskier change than belongs in this pass). Import-boundary check added via dependency-cruiser (ARK-14) -- installed for real, config tested against this codebase (179 modules, 310 deps, 0 violations), wired as `pnpm run deps:check` (advisory, not yet a CI gate). BACKLOG P2-5: docs/02-architecture.md's "Hosting" section marked SUPERSEDED (the Vercel plan was superseded by the Coolify deploy.target flip, 2026-08-09 -- ADR-0001). README.md/.env.example were checked and are already accurately framed (live-on-Vercel vs. config-target-is-Coolify) -- not touched. BACKLOG P3-5: 2 stale github.com/sax3l/0pt/pull/3 links removed (the PR merged long ago, .env.example is real and current) -- docs/09-build-plan.md, docs/08-infrastructure.md. The "two diverged remotes, unresolved which is canonical" language elsewhere (CLAUDE.md, AGENTS.md, SECURITY.md, etc.) is deliberately NOT touched -- that's a live, correctly-flagged, account-owner- owned open question per the Gitea-exclusivity mandate, not a stale link. BACKLOG P6 (19 items): concrete artifacts for the ones that are actually a file (.editorconfig, dependency-cruiser, deprecation policy, K8s reference manifest, .actrc, Makefile, ONBOARDING.md); docs/guides/P6-GUIDES.md for the rest (CLI-13, DOC-18, UI-30, IAM-19, PAY-11, AI-18, OBS-21, TEST-20, NET-19, SCALE-13, OSS-10, AGENT-13) -- each grounded in this repo's actual code/gaps, not generic advice. NOT attempted (out of this task's scope, noted for the record): - P2-1 (cl0ud app-registry.ts mismatch) -- lives in a different repo (sax3l/cl0ud), not cloned/authorized for this task's single-repo PR flow. - The CI "doctor step" half of P1-15 -- deliberately left out of this PR's .gitea/workflows/ci.yml to avoid a 3-way merge conflict with ci/security-headers-and-scanning, which already restructured that file; add it as a follow-up once that PR merges. This branch was authored independently of, but shares docs/architecture/ ARCHITECTURE.md and docs/slo.md with, fix/honest-health-check and ci/security-headers-and-scanning -- expect a small, normal merge conflict if this is rebased after either of those merge first (both already rebased cleanly against each other and against main as of this commit).	2026-08-11 14:18:00 +02:00
admin	a88ad04867	fix(health): report real per-dependency status instead of hardcoded ok:true ... BACKLOG P0-3: /api/health only ever checked *presence* of DATABASE_URL/ REDIS_URL and always returned ok:true regardless of whether anything was reachable -- ARCHITECTURE.md's own words, "an uptime monitor watching HTTP 200 would show green for a dying instance". lib/health-checks.ts adds real dependency probes: - db: a live `select 1` round trip (not just DATABASE_URL presence) - engine: a real HTTP call to the FastAPI engine's own GET /health - stripe: Stripe's documented lightweight balance.retrieve() liveness call - queue: honestly reports "not_integrated" when REDIS_URL is set but unused -- no Redis client is wired anywhere in this repo, so reporting "ok" would fabricate a check that doesn't exist /api/health stays a liveness check (always 200 while the process can respond; the body's `ok` field is the honest signal). New GET /api/ready fails closed with 503 when the load-bearing dependency (db) isn't both configured and reachable -- the "no /ready route" gap. contracts/openapi.yaml, docs/architecture/ARCHITECTURE.md, and docs/slo.md updated to match. 17 new unit tests (dependency-injected fakes, no real network/DB) + 5 route-level tests.	2026-08-11 13:49:23 +02:00
admin	ebffb7df86	Merge pull request 'docs(compliance): finish attributing the evasion-stack keep decision' (#17) from docs/curl-cffi-decision-attribution into main	2026-08-11 11:36:38 +00:00
adminandClaude Sonnet 5	2393f52768	docs(compliance): finish attributing the evasion-stack keep decision ... PR #15 already fixed AGENTS.md/SECURITY.md with a dated, attributed statement of the account owner's 2026-08-11 decision (Simon Axelsson, via an interactive decision prompt, responding to this repo's own removal proposal PR #7 which was verified clean/128 tests and closed unmerged -- no rationale beyond "keep it" was given or is invented). This PR finds and fixes every other place in the repo that still carried the earlier unattributed "medvetet anvandarbeslut" / "deliberate user decision" claim a compliance audit flagged as unverifiable: - docs/compliance/GOLDEN-STANDARD-REPORT.md: control C6 row + the "Bypass-undantag" formal section (the two places this doc was explicitly named to need checking) -- header gains a dated update note, C6 row and the section body get the same attributed statement as AGENTS.md/SECURITY.md. - docs/compliance/SLUTRAPPORT.md, docs/compliance/INVENTORY.md: historical session doc keeps its original text and gains a dated correction addendum; the INVENTORY.md evidence bullet is corrected in place. - docs/slo.md, CONTRIBUTING.md, prompts/002-compliance-audit.md, renovate.json, .gitea/workflows/security.yml: direct/living references to the exemption get the same attributed, dated pointer instead of the unattributed phrase. CLAUDE.md's general §1 anti-evasion policy text is untouched, per instruction -- this is a scoped exception note, not a change to the normative rule. No code changes. Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>	2026-08-11 13:34:13 +02:00
Claude (0pt session)andClaude Sonnet 5	e9dff491ca	docs(compliance): add BLOCKERS.md (§1.4) ... Golden Standard §1.4 requires an unresolved blocker to be recorded in docs/compliance/BLOCKERS.md; this file was confirmed absent in a recent audit pass, which is itself a delivery defect per the framework's own rule. Records four blockers, each freshly re-verified against current main rather than copied from prior findings: - WAF-bypass/Tor/FlareSolverr stack (services/scrapers/*): the account owner's decision (keep) is now on record via merged PR #15 as of today; PR #7 (removal) stays closed/unmerged. Documented as decided, not left as still-pending. - No Coolify app / Infisical project provisioned (cd.yml still carries the TODO placeholder on current main). - 0pt.app / 0pt.dev DNS still delegated to Vercel nameservers (live NS lookup today), despite PowerDNS holding shadow zone copies. - middleware.ts still answers anonymously live even after a merged auth fix (PR #11) — confirmed with a fresh live probe today, which sharpens the existing P0-1 backlog item. Scope: documentation only, this one new file. Does not touch services/scrapers/*, services/infra/docker-compose.data-stack.yml, docs/compliance/BACKLOG.md, or PR #7. Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>	2026-08-11 13:04:53 +02:00
admin	5c36dc300f	Merge pull request 'docs(compliance): record dated provenance for evasion-stack keep decision' (#15) from docs/evasion-stack-decision-provenance into main	2026-08-11 09:36:16 +00:00
admin	60145eee33	Merge pull request 'fix(security): require auth on GET /api/orders and GET /api/admin/metrics' (#11) from security/auth-gate-anonymous-api-routes into main	2026-08-11 09:35:48 +00:00
adminandClaude Sonnet 5	a14911b263	docs(compliance): record dated provenance for evasion-stack keep decision ... AGENTS.md and SECURITY.md previously attributed keeping the WAF-bypass/ Tor/FlareSolverr/curl_cffi/cloudscraper stack to an unattributed, unverified "medvetet användarbeslut" (deliberate user decision) with no stated author or date -- flagged by a prior audit as unverified provenance. Replace with an accurate, dated statement: the account owner (Simon Axelsson) was asked directly via an interactive decision prompt on 2026-08-11, in direct response to a compliance audit's removal proposal (the surgically-clean removal PR #7, verified 128/128 tests, was closed unmerged), and explicitly chose to keep the code. No business or legal rationale beyond "keep it" was given or is invented here. This is a narrow, scoped exception for the existing 0PT/0pt-motor sourcing engine. CLAUDE.md §1's general anti-evasion policy text is untouched and remains in force as written unless/until the account owner says otherwise. No code changes -- documentation only. Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>	2026-08-11 11:32:13 +02:00
adminandClaude Sonnet 5	15a0dbe3b1	fix(security): require auth on GET /api/orders and GET /api/admin/metrics ... A live compliance audit against 0pt.app confirmed both endpoints returned HTTP 200 with zero authentication -- the full order queue and financial pipeline aggregates (deposit/pipeline totals) were readable by anyone. Root cause: middleware.ts's matcher only covers /api/0pt/:path*, by design ("does NOT touch ... the unrelated payment/order APIs"), so nothing upstream of these two handlers ever verified the caller. Fix: both GET handlers now call the same fail-closed Zitadel verifier /api/0pt/* relies on (resolveAuthContext, src/auth/context.ts) directly, via a new thin wrapper (src/auth/require-route-auth.ts) that maps a rejected decision to the same 401/403 JSON shape middleware.ts already returns. The wrapper re-verifies the bearer token itself rather than trusting any x-0pt-verified-* header on the request, since middleware never runs in front of these routes to overwrite a client-supplied one. POST /api/orders (anonymous draft/quote creation before the deposit-payment step) is intentionally left as-is -- only the read paths that leaked data are gated. Tracked as part of docs/compliance/BACKLOG.md P0-3. Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>	2026-08-11 04:06:13 +02:00
admin	5f41047546	docs(compliance): record final CI run IDs on main 3af2ac8	2026-08-10 04:23:46 +02:00
admin	3af2ac894e	docs(compliance): FAS 8 evidence + FAS 10 final report	2026-08-10 04:18:00 +02:00
admin	8db56f1b4c	docs: FAS 9 - README/SECURITY updates, CONTRIBUTING, AGENTS, runbooks (deploy/incident/backup-restore/oncall), SLO, prompts	2026-08-10 04:16:30 +02:00
admin	2dccb8aab8	Merge pull request 'ci(workflows): split CD/security/nightly under .gitea/workflows + gitignore hygiene' (#9) from ci/workflow-split into main	2026-08-10 02:15:13 +00:00
admin	6612ab1233	ci: keep advisory security job in ci.yml so protected-branch status contexts stay reported	2026-08-10 04:11:52 +02:00
admin	48d31d0081	ci(workflows): split CD/security/nightly under .gitea/workflows + gitignore hygiene (protected-path PR)	2026-08-10 04:10:47 +02:00
admin	9888c0078c	feat(contract): OpenAPI 3.1 SSOT (orders/estimates/enrich/resolve/provenance) + AR-4 fixes (drop @vercel/analytics + next/font/google -> self-hosted fontsource) + repo hygiene	2026-08-10 04:10:30 +02:00
admin	d2b5ec8e4f	docs(compliance): FAS 0-5 - inventory, relations, OSS, golden standard, backlog + repo classification (siax.repo.json)	2026-08-10 04:03:40 +02:00
admin	1346e868bf	Merge pull request 'fix(deploy): declare Coolify as the deploy target, not Vercel' (#8) from fix/remove-vercel-deploy-target into main	2026-08-09 12:15:19 +00:00
adminandClaude Sonnet 5	ad042d825f	fix(deploy): declare Coolify as the deploy target, not Vercel ... siax.config.json declared deploy.target: "vercel", which conflicts with the SIAX Cloud platform pattern every sibling OS repo (n0d, st0re, aud0) follows: a Dockerfile "for Coolify" (already present in this repo) plus a .gitea/workflows/deploy.yml that POSTs a restart to a Coolify app UUID. Investigated before changing anything: - 0pt.app is confirmed live on Vercel (Server: Vercel, 200 OK) but its own /api/health reports db/queue "not_configured", and POSTing to /api/orders creates an order that is not durably persisted (in-memory store only, per this repo's own docs/08-infrastructure.md and docs/architecture/ ARCHITECTURE.md, both independently verified live 2026-07-19 / 2026-08-07, and reconfirmed live again today). It's a fully-built marketing/demo surface with no real backend behind it -- not a live product serving real customer data. Safe to change deploy config without a cutover plan. - Dockerfile already exists and is already written "for Coolify" (node:24- alpine multistage, non-root, /api/health healthcheck) -- no changes needed there. - No .gitea/workflows/deploy.yml existed. Added one following the exact pattern used by n0d and st0re (POST /api/v1/deploy?uuid=...&force=false with COOLIFY_TOKEN). No Coolify app has been provisioned for 0pt yet, so the app UUID is a literal placeholder and the workflow fails loudly by design until a real app + token are in place -- this does not silently ship a broken deploy config. This changes only this repo's declared *intent*. It does NOT touch the live Vercel deployment, delete the Vercel project, or move any traffic -- that cutover (provision Coolify app -> verify -> DNS -> decommission Vercel) is a separate, larger decision. README and ARCHITECTURE.md updated to state this plainly rather than let them contradict the new config. Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>	2026-08-09 12:36:24 +02:00
admin	a7146683b9	Merge pull request 'unify: merge the GitHub line into Gitea, settled against real service contracts' (#6) from unify/github-main-2026-07-28 into main	2026-08-09 09:31:05 +00:00
SIAX Ops	44e83ad1de	resolve: README adopts main (newer, fail-closed auth docs)	2026-08-09 11:30:18 +02:00
admin	39967cb3f6	Merge pull request 'docs(data-engine): 1p extends the provenance chain to the network layer' (#3) from docs/1p-egress-measurement-point into main	2026-08-09 08:50:39 +00:00
admin	4cf833bb82	Merge remote-tracking branch 'origin/main' into docs/1p-egress-measurement-point	2026-08-09 10:49:30 +02:00
admin	17309c0bf3	Merge pull request 'chore: Configure Renovate' (#5) from renovate/configure into main	2026-08-09 08:38:53 +00:00
admin	5166d85ecd	chore: add missing standard files (LICENSE/CLAUDE/health-check)	2026-08-08 23:14:50 +02:00
Claude (SIAX platform session)	bb58617540	docs(standard): add code-verified ARCHITECTURE.md + siax.config.json (S1-S7) ... Brings 0pt to the SIAX repo standard. README, LICENSE (proprietary, correct for this repo), Dockerfile and .gitea/workflows/ci.yml already existed and are left untouched; no .github/workflows exists. The architecture document was written by reading the code (entrypoints, routes, services, migrations, Dockerfile, env, tests) and by probing the live 0pt.app instance -- not from the templates. It therefore contradicts the existing docs in several places, deliberately: * /api/0pt/* answers ANONYMOUSLY in production (live-verified 2026-08-07): GET https://0pt.app/api/0pt returns 200 with no token, and 200 with a deliberately invalid one. The deployment is current (the newer lakehouse routes are present), so middleware.ts is simply not running. README lines 49-52 and merge 6531384 claim the opposite. Root cause UNKNOWN -- flagged, not guessed. * No handler reads x-0pt-verified-tenant. middleware.ts sets it; nothing else in the repo reads it. Every /api/0pt/* handler takes tenant_id from the request body/query instead, so tenant isolation is not implemented. * Everything outside /api/0pt/* has no auth at all (matcher scope), including GET /api/admin/metrics and GET /api/orders (both live-verified 200 anon) and POST /api/payments/{id}/release, which flips an order to "delivered" regardless of payment. * CL0UD policy client is fail-open and AUD0 audit is a silent no-op when their URLs are unset, with no production guard -- unlike AUTH_ENFORCE, which has one. * connections/db-grant documents AES-GCM encryption it does not implement. * The FastAPI engine (services/api) has zero authentication despite lib/engine.ts sending X-Internal-Token. * services/scrapers/waf_bypass.py (FlareSolverr, Tor, cloudscraper, proxy pool, undetected browser, fake identities) plus flaresolverr/tor-proxy in services/infra/docker-compose.data-stack.yml directly contradict CLAUDE.md §1 and the README's compliance claim. Left in place -- deletion is a Simon decision, recorded as open question 2. * Production has NO database (live /api/health reports db:not_configured) and the whole /api/0pt/* capability layer is per-module in-memory Maps, so state does not survive a cold start and is not even shared between route modules. siax.config.json deviates from the template on two fields, on purpose: deploy.target is "vercel" (live-verified Server: Vercel header), not "coolify", and designSystem is null because @siax/ui is not a dependency here. Both deviations are documented in ARCHITECTURE.md §5 and §8. Docs-only change. No code, CI, LICENSE or workflow files were modified.	2026-08-07 21:48:35 +02:00
admin	65313842d9	Merge pull request 'fix(auth): make the AUTH_ENFORCE default fail-closed -- 0pt.app was answering anonymously' (#4) from fix/fail-closed-auth-default into main	2026-08-04 13:24:16 +00:00