From-scratch verification (fresh clone, fresh installs) backing up the P010
completion pass: 256/256 vitest, 28/28 pytest, clean typecheck/build on the
committed tree, pre-existing-only lint noise. Documents that
OPT_QUALITY_BRIDGE_ENABLED ships off and stays off in this PR -- activating
it in production is left as Simon's decision, with a readiness checklist.
Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>
Continues P010 (branch renamed from feat/p010-source-registry-persistence):
closes the three remaining BEFORE_STATE.md gaps (§2 TS/Python split-brain,
§3 missing provenance fields, §4 untested failure modes), all additive to
the production pipeline — the Python QA filter's delivery decision is
unchanged by any of this.
TS/Python bridge (§2):
- New route surface app/api/0pt/quality/* (rules/runs/issues CRUD +
POST /api/0pt/quality/bridge-run) over the shared QualityService
singleton (src/services/quality.ts), fixing the same "no shared
instance" defect the Source Registry had.
- src/quality/quality-bridge.ts auto-provisions a deterministic,
tenant-scoped baseline rule set on first use so the Python engine never
pre-registers rules through the CRUD surface.
- services/api/workers/quality_bridge.py: the Python-side HTTP client.
Feature-flagged off by default (OPT_QUALITY_BRIDGE_ENABLED), fails open
on any error (never raises into run_job, never changes `delivered`).
Advisory only — real per-row gating against the TS engine isn't
implementable yet (ExpectationEngine returns dataset-level pass/fail
counts + samples, not a per-row boolean array); documented, not faked.
- Live-verified end-to-end: `next dev` (AUTH_ENFORCE=false) + the real
Python pipeline hitting the real brreg.no API, round-tripping through
the real bridge route, receiving a real QualityService score.
Provenance (§3): sources/connectors.py's Provenance dataclass gains
retrieved_at, transform, connector_version, producer, quality,
entity_resolution, artifact_ref (all previously absent from the real
dataflow entirely) via a make_provenance() factory used by every connector.
resolve.py stamps entity_resolution on merged records' provenance (never on
unmerged ones). pipeline.py stamps artifact_ref at delivery and quality
once the bridge succeeds.
Failure tests (§4), real fault injection against real code (services/api,
new pytest + respx suite, not vitest mocks):
- dedup-determinism + duplicate-input + out-of-order: resolve.py under
shuffled/interleaved/exhaustive-permutation input, plus an explicit,
verified test of the one case that IS legitimately order-sensitive
(non-transitive similarity in a greedy clusterer) instead of hiding it.
- schema-evolution: respx-mocked upstream responses with added/removed/
alien-shaped fields through the real brreg connector + pipeline.
- restart: connector exceptions mid-job (including one from respx itself)
never crash run_job.
- backfill: the real property that holds given no persistence layer
exists (replay is compute-idempotent) — NOT a fabricated dedup-store
test.
- CDC-interruption and true backfill/delivery-store idempotency:
explicitly documented as not implementable against this codebase
(no CDC pipeline, no delivery store exist anywhere) via skip markers
with reasons, in tests/test_known_gaps.py, rather than silently
dropped or faked.
Testing: 256/256 vitest (249 pre-existing + 7 new bridge route/service
tests) and 28/28 pytest (2 documented skips) green locally. tsc -p
tsconfig.check.json has zero new errors (14 pre-existing errors, all in
the untracked/unaudited src/fleet, src/graph, src/metrics, src/projections
paths this pass does not touch, per BEFORE_STATE.md).
No PR opened, per instructions.
Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>
Replaces the in-memory (per-instance, not even per-process-shared) Maps in
SourceService/SourceRegistryService with real Postgres persistence, and adds
the register/query/describe/health/rights/provenance/freshness HTTP routes
that never existed (grep for SourceRegistryService in app/ previously
returned nothing).
- sql/002_source_registry.sql: opt_sources, opt_source_rights,
opt_source_provenance, opt_source_health_checks, opt_source_freshness.
Numbered 002 (not 006) because 004's opt_source_rate_policies already
forward-references opt_sources, which no sql/ file defined until now —
fixes 004's pre-existing broken migration ordering (opt_datasets remains
a separate, unrelated, still-open gap in 004, flagged not fixed).
- src/repositories/*: SourceRepository / SourceRegistryRepository seams,
each with an in-memory implementation (business logic extracted verbatim)
and a Postgres implementation, chosen via resolveStoreMode — production
still fails hard with no DATABASE_URL (Master-DoD §3), but a configured
production now actually works instead of always being blocked.
- SourceService / SourceRegistryService: storage moved behind the
repository seam; getHealth/getFreshness are now async and tenant-scoped
at the accessor itself (needed once they're reachable directly over HTTP,
not only after an earlier caller's tenant check).
- src/services/source-registry.ts: shared singleton instances for the
sources/* route group, fixing the "each route module owned its own
private registry" defect (source-rate-policies and .../query previously
couldn't see a source registered via POST /api/0pt/sources).
- New routes: POST/GET /api/0pt/sources, GET /api/0pt/sources/[id],
GET+PUT .../rights, GET+PUT .../provenance, GET+POST .../health,
GET .../freshness.
- Business logic unchanged: fail-closed purpose gate, cross-tenant
NOT_FOUND (no existence oracle, §50), bounded 100-row health history.
Tested against a real Postgres container (not mocked) —
tests/p0/source-registry-postgres.realinfra.test.ts, env-gated on
SOURCE_REGISTRY_TEST_DATABASE_URL, skips honestly when unset. All 249
pre-existing P0 tests + 7 new realinfra tests pass; also smoke-tested the
full HTTP route surface end-to-end against next dev + live Postgres.
Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>
PR #25 CI run 7745 job 'integration' failed before any test ran: 'exec:
"./opensearch-docker-entrypoint.sh": stat ...: no such file or directory'.
Root cause (confirmed against the job log's merged container.Config, and
reproduced locally with plain docker run -w): this runner's act_runner
merges the JOB's WorkingDir (/workspace/sax3l/0pt) into every SERVICE
container, clobbering the OpenSearch image's own WORKDIR
(/usr/share/opensearch) that its ENTRYPOINT ./opensearch-docker-entrypoint.sh
is relative to. ClickHouse's entrypoint is an absolute path so it started
fine under the same defect -- a green ClickHouse service is not evidence
this class of bug is absent elsewhere.
Fix: --workdir /usr/share/opensearch in the opt-ci-opensearch service's
options, restoring the image's own working directory. Verified locally:
'docker run -w /workspace/sax3l/0pt opensearchproject/opensearch:2.11.1'
reproduces the exact failure; '-w /usr/share/opensearch' fixes it.
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
- tests/integration/: 10 tests against REAL OpenSearch 2.11 (tenant
isolation, purpose/rights filters, generation rebuild/rollback,
projection-disaster rebuild §68) + REAL ClickHouse 24.8 (tenant-isolated
metric DSL, TTL schema, injection-surface refusals). Honest skip without
env - and CI fails hard if the suites skip instead of running.
- .gitea/workflows/ci.yml: new blocking 'integration' job with
self-identifying service containers (st0re ci.yml's measured
alias-collision discipline: run-unique cluster.name / credentials,
getent-enumerated IP selection, no-match = hard fail).
- STATUS.md: full 8-level Master-DoD classification per capability incl.
the honest gaps blocking PRODUCTION_READY.
- dependency-mode: NEXT_PHASE=phase-production-build is not a serving
runtime (next build page-data collection must compile; guard re-arms at
boot) - pinned by contract test.
- docs/02-architecture.md: superseded the accurate-but-now-fixed 'every
client runs against its mock' paragraph; .env.example: client URLs now
actually read from env + new P0 backend vars.
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
38 tests: recording provider double answers with the sibling services' real
response shapes; each suite asserts the exact request wire shape (path,
method, tenant/actor/idempotency headers, camelCase vs snake_case body) so a
regression to the pre-rewrite invented shapes fails loudly. Includes the
fail-closed policy cases (missing decision => deny, CL0UD down => typed
error, never implicit allow) and the production factory guard for all four
clients.
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
- src/platform/: SiaxRequestContext + propagation headers + 12 canonical
error codes (documented mirror of cl0ud packages/contract), SiaxError,
resource refs (st0re:artifact:...), and the §3/§10 dependency-mode guard:
production runtime + unconfigured dependency/store => hard
DEPENDENCY_UNAVAILABLE at construction, never silent mock/in-memory.
- src/clients/: all four platform clients rewritten against the REAL wire
contracts verified on sibling main branches this session:
* cl0ud: POST /api/policy/evaluate ({data} envelope, allow+requiresApproval)
- the old /v1/cl0ud/policies/evaluate path never existed on CL0UD
* st0re: camelCase ArtifactInput on /v1/st0re/artifacts/register incl.
X-Tenant-Id/X-Actor-Id + X-Idempotency-Key - the old snake_case body
would have been a 400 against the live registry
* n0d: required_role + resource_requirements + body idempotency_key +
CloudEvents callback_url per n0d createJobSchema; adds getJob/getJobResult
* aud0: /v1/aud0/openlineage is AUD0's only lineage ingest -
/v1/aud0/lineage/events never existed; legacy shape converts to a
spec-shaped OpenLineage RunEvent (deterministic runId dedup)
- src/lineage/: OpenLineage subset types + LineageEmitter (never-throw,
§16 evidence semantics)
- shared SiaxHttp transport: timeouts, typed SiaxError mapping, context
propagation, no raw upstream bodies in errors
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
BACKLOG P1-1: siax.repo.json/siax.config.json rewritten to be schema-valid
against sax3l/siax-standard's actual v2.0 schemas (lifecycle,
operationalState, gitHost, domains-as-objects, consumedBy, children,
contracts -- validated with jsonschema, not just eyeballed). config.json
restructured to the real deploy-descriptor shape (framework/build/start/
port/healthcheck/environments) instead of duplicating repo.json's identity
fields, which the schema doesn't allow.
BACKLOG P1-2: 3 ADRs for decisions already made and evidenced elsewhere in
this repo (Coolify-not-Vercel, Zitadel-not-NextAuth, Iceberg-format pointer
to n0d ADR-0030) -- retroactive documentation, not new decisions. The
Stripe/Revolut and services/scrapers/* questions are explicitly NOT written
as ADRs here; both are account-owner-gated or actively under review
elsewhere (see docs/architecture/decisions/README.md).
BACKLOG P1-3: CHANGELOG.md from real git log history (no tags/releases
exist yet -- this doesn't invent version numbers that were never cut).
BACKLOG P1-19: `pnpm siax:doctor` (scripts/siax-doctor.mjs, AGENT-04) --
zero-dependency, checks required files + manifest shape, prints
control-IDs. .devcontainer/devcontainer.json (DX-03). compose.dev.yaml
(DX-02 -- UNVERIFIED, no Docker available in this environment to smoke-test,
flagged in the file itself). All 4 prompts + the index rewritten to AGENT-03
shape (goal/context/prerequisites/scope/contract/acceptance-criteria/
control-ID/forbidden-actions).
BACKLOG P1-20: docs/architecture/ARCHITECTURE.md migrated to the v2.0
9-heading template -- new §5 "Ytor och paritet" (capability x
API/CLI/MCP/console coverage table, grounded in the same registry facts
docs/02-architecture.md already established), §6-9 renumbered from the old
§5-8 (all internal cross-references fixed, CLAUDE.md's own §-numbers left
untouched). docs/api/ added (points at contracts/openapi.yaml as the one
SSOT instead of duplicating it as drift-prone prose).
BACKLOG P1-21: packages/schemas now has a real package.json (was loose JSON
files with no package -- pnpm-workspace.yaml's packages/* glob matched an
empty package). Verified `pnpm install --frozen-lockfile` and the full
typecheck/test/build suite still pass with 2 real workspace projects.
apps/ restructuring explicitly NOT attempted (BACKLOG's own effort estimate
is "L" / large -- moving the whole app tree is a bigger, riskier change than
belongs in this pass). Import-boundary check added via dependency-cruiser
(ARK-14) -- installed for real, config tested against this codebase (179
modules, 310 deps, 0 violations), wired as `pnpm run deps:check` (advisory,
not yet a CI gate).
BACKLOG P2-5: docs/02-architecture.md's "Hosting" section marked SUPERSEDED
(the Vercel plan was superseded by the Coolify deploy.target flip,
2026-08-09 -- ADR-0001). README.md/.env.example were checked and are
already accurately framed (live-on-Vercel vs. config-target-is-Coolify) --
not touched.
BACKLOG P3-5: 2 stale github.com/sax3l/0pt/pull/3 links removed (the PR
merged long ago, .env.example is real and current) -- docs/09-build-plan.md,
docs/08-infrastructure.md. The "two diverged remotes, unresolved which is
canonical" language elsewhere (CLAUDE.md, AGENTS.md, SECURITY.md, etc.) is
deliberately NOT touched -- that's a live, correctly-flagged, account-owner-
owned open question per the Gitea-exclusivity mandate, not a stale link.
BACKLOG P6 (19 items): concrete artifacts for the ones that are actually a
file (.editorconfig, dependency-cruiser, deprecation policy, K8s reference
manifest, .actrc, Makefile, ONBOARDING.md); docs/guides/P6-GUIDES.md for the
rest (CLI-13, DOC-18, UI-30, IAM-19, PAY-11, AI-18, OBS-21, TEST-20, NET-19,
SCALE-13, OSS-10, AGENT-13) -- each grounded in this repo's actual code/gaps,
not generic advice.
NOT attempted (out of this task's scope, noted for the record):
- P2-1 (cl0ud app-registry.ts mismatch) -- lives in a different repo
(sax3l/cl0ud), not cloned/authorized for this task's single-repo PR flow.
- The CI "doctor step" half of P1-15 -- deliberately left out of this PR's
.gitea/workflows/ci.yml to avoid a 3-way merge conflict with
ci/security-headers-and-scanning, which already restructured that file;
add it as a follow-up once that PR merges.
This branch was authored independently of, but shares docs/architecture/
ARCHITECTURE.md and docs/slo.md with, fix/honest-health-check and
ci/security-headers-and-scanning -- expect a small, normal merge conflict
if this is rebased after either of those merge first (both already rebased
cleanly against each other and against main as of this commit).
BACKLOG P0-3: /api/health only ever checked *presence* of DATABASE_URL/
REDIS_URL and always returned ok:true regardless of whether anything was
reachable -- ARCHITECTURE.md's own words, "an uptime monitor watching HTTP
200 would show green for a dying instance".
lib/health-checks.ts adds real dependency probes:
- db: a live `select 1` round trip (not just DATABASE_URL presence)
- engine: a real HTTP call to the FastAPI engine's own GET /health
- stripe: Stripe's documented lightweight balance.retrieve() liveness call
- queue: honestly reports "not_integrated" when REDIS_URL is set but
unused -- no Redis client is wired anywhere in this repo, so reporting
"ok" would fabricate a check that doesn't exist
/api/health stays a liveness check (always 200 while the process can
respond; the body's `ok` field is the honest signal). New GET /api/ready
fails closed with 503 when the load-bearing dependency (db) isn't both
configured and reachable -- the "no /ready route" gap.
contracts/openapi.yaml, docs/architecture/ARCHITECTURE.md, and docs/slo.md
updated to match. 17 new unit tests (dependency-injected fakes, no real
network/DB) + 5 route-level tests.
PR #15 already fixed AGENTS.md/SECURITY.md with a dated, attributed
statement of the account owner's 2026-08-11 decision (Simon Axelsson, via
an interactive decision prompt, responding to this repo's own removal
proposal PR #7 which was verified clean/128 tests and closed unmerged --
no rationale beyond "keep it" was given or is invented). This PR finds and
fixes every other place in the repo that still carried the earlier
unattributed "medvetet anvandarbeslut" / "deliberate user decision" claim
a compliance audit flagged as unverifiable:
- docs/compliance/GOLDEN-STANDARD-REPORT.md: control C6 row + the
"Bypass-undantag" formal section (the two places this doc was explicitly
named to need checking) -- header gains a dated update note, C6 row and
the section body get the same attributed statement as AGENTS.md/SECURITY.md.
- docs/compliance/SLUTRAPPORT.md, docs/compliance/INVENTORY.md: historical
session doc keeps its original text and gains a dated correction
addendum; the INVENTORY.md evidence bullet is corrected in place.
- docs/slo.md, CONTRIBUTING.md, prompts/002-compliance-audit.md,
renovate.json, .gitea/workflows/security.yml: direct/living references
to the exemption get the same attributed, dated pointer instead of the
unattributed phrase.
CLAUDE.md's general §1 anti-evasion policy text is untouched, per instruction
-- this is a scoped exception note, not a change to the normative rule. No
code changes.
Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>
Golden Standard §1.4 requires an unresolved blocker to be recorded in
docs/compliance/BLOCKERS.md; this file was confirmed absent in a recent
audit pass, which is itself a delivery defect per the framework's own
rule. Records four blockers, each freshly re-verified against current
main rather than copied from prior findings:
- WAF-bypass/Tor/FlareSolverr stack (services/scrapers/*): the account
owner's decision (keep) is now on record via merged PR #15 as of
today; PR #7 (removal) stays closed/unmerged. Documented as decided,
not left as still-pending.
- No Coolify app / Infisical project provisioned (cd.yml still carries
the TODO placeholder on current main).
- 0pt.app / 0pt.dev DNS still delegated to Vercel nameservers (live NS
lookup today), despite PowerDNS holding shadow zone copies.
- middleware.ts still answers anonymously live even after a merged
auth fix (PR #11) — confirmed with a fresh live probe today, which
sharpens the existing P0-1 backlog item.
Scope: documentation only, this one new file. Does not touch
services/scrapers/*, services/infra/docker-compose.data-stack.yml,
docs/compliance/BACKLOG.md, or PR #7.
Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>
AGENTS.md and SECURITY.md previously attributed keeping the WAF-bypass/
Tor/FlareSolverr/curl_cffi/cloudscraper stack to an unattributed,
unverified "medvetet användarbeslut" (deliberate user decision) with no
stated author or date -- flagged by a prior audit as unverified provenance.
Replace with an accurate, dated statement: the account owner (Simon
Axelsson) was asked directly via an interactive decision prompt on
2026-08-11, in direct response to a compliance audit's removal proposal
(the surgically-clean removal PR #7, verified 128/128 tests, was closed
unmerged), and explicitly chose to keep the code. No business or legal
rationale beyond "keep it" was given or is invented here.
This is a narrow, scoped exception for the existing 0PT/0pt-motor sourcing
engine. CLAUDE.md §1's general anti-evasion policy text is untouched and
remains in force as written unless/until the account owner says otherwise.
No code changes -- documentation only.
Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>
A live compliance audit against 0pt.app confirmed both endpoints returned
HTTP 200 with zero authentication -- the full order queue and financial
pipeline aggregates (deposit/pipeline totals) were readable by anyone.
Root cause: middleware.ts's matcher only covers /api/0pt/:path*, by design
("does NOT touch ... the unrelated payment/order APIs"), so nothing upstream
of these two handlers ever verified the caller.
Fix: both GET handlers now call the same fail-closed Zitadel verifier
/api/0pt/* relies on (resolveAuthContext, src/auth/context.ts) directly, via
a new thin wrapper (src/auth/require-route-auth.ts) that maps a rejected
decision to the same 401/403 JSON shape middleware.ts already returns. The
wrapper re-verifies the bearer token itself rather than trusting any
x-0pt-verified-* header on the request, since middleware never runs in front
of these routes to overwrite a client-supplied one.
POST /api/orders (anonymous draft/quote creation before the deposit-payment
step) is intentionally left as-is -- only the read paths that leaked data are
gated. Tracked as part of docs/compliance/BACKLOG.md P0-3.
Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>
siax.config.json declared deploy.target: "vercel", which conflicts with
the SIAX Cloud platform pattern every sibling OS repo (n0d, st0re, aud0)
follows: a Dockerfile "for Coolify" (already present in this repo) plus a
.gitea/workflows/deploy.yml that POSTs a restart to a Coolify app UUID.
Investigated before changing anything:
- 0pt.app is confirmed live on Vercel (Server: Vercel, 200 OK) but its own
/api/health reports db/queue "not_configured", and POSTing to /api/orders
creates an order that is not durably persisted (in-memory store only, per
this repo's own docs/08-infrastructure.md and docs/architecture/
ARCHITECTURE.md, both independently verified live 2026-07-19 / 2026-08-07,
and reconfirmed live again today). It's a fully-built marketing/demo
surface with no real backend behind it -- not a live product serving real
customer data. Safe to change deploy config without a cutover plan.
- Dockerfile already exists and is already written "for Coolify" (node:24-
alpine multistage, non-root, /api/health healthcheck) -- no changes needed
there.
- No .gitea/workflows/deploy.yml existed. Added one following the exact
pattern used by n0d and st0re (POST /api/v1/deploy?uuid=...&force=false
with COOLIFY_TOKEN). No Coolify app has been provisioned for 0pt yet, so
the app UUID is a literal placeholder and the workflow fails loudly by
design until a real app + token are in place -- this does not silently
ship a broken deploy config.
This changes only this repo's declared *intent*. It does NOT touch the live
Vercel deployment, delete the Vercel project, or move any traffic --
that cutover (provision Coolify app -> verify -> DNS -> decommission
Vercel) is a separate, larger decision. README and ARCHITECTURE.md updated
to state this plainly rather than let them contradict the new config.
Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>
Brings 0pt to the SIAX repo standard. README, LICENSE (proprietary, correct
for this repo), Dockerfile and .gitea/workflows/ci.yml already existed and are
left untouched; no .github/workflows exists.
The architecture document was written by reading the code (entrypoints,
routes, services, migrations, Dockerfile, env, tests) and by probing the live
0pt.app instance -- not from the templates. It therefore contradicts the
existing docs in several places, deliberately:
* /api/0pt/* answers ANONYMOUSLY in production (live-verified 2026-08-07):
GET https://0pt.app/api/0pt returns 200 with no token, and 200 with a
deliberately invalid one. The deployment is current (the newer lakehouse
routes are present), so middleware.ts is simply not running. README lines
49-52 and merge 6531384 claim the opposite. Root cause UNKNOWN -- flagged,
not guessed.
* No handler reads x-0pt-verified-tenant. middleware.ts sets it; nothing else
in the repo reads it. Every /api/0pt/* handler takes tenant_id from the
request body/query instead, so tenant isolation is not implemented.
* Everything outside /api/0pt/* has no auth at all (matcher scope), including
GET /api/admin/metrics and GET /api/orders (both live-verified 200 anon) and
POST /api/payments/{id}/release, which flips an order to "delivered"
regardless of payment.
* CL0UD policy client is fail-open and AUD0 audit is a silent no-op when their
URLs are unset, with no production guard -- unlike AUTH_ENFORCE, which has one.
* connections/db-grant documents AES-GCM encryption it does not implement.
* The FastAPI engine (services/api) has zero authentication despite
lib/engine.ts sending X-Internal-Token.
* services/scrapers/waf_bypass.py (FlareSolverr, Tor, cloudscraper, proxy pool,
undetected browser, fake identities) plus flaresolverr/tor-proxy in
services/infra/docker-compose.data-stack.yml directly contradict CLAUDE.md
§1 and the README's compliance claim. Left in place -- deletion is a Simon
decision, recorded as open question 2.
* Production has NO database (live /api/health reports db:not_configured) and
the whole /api/0pt/* capability layer is per-module in-memory Maps, so state
does not survive a cold start and is not even shared between route modules.
siax.config.json deviates from the template on two fields, on purpose:
deploy.target is "vercel" (live-verified Server: Vercel header), not "coolify",
and designSystem is null because @siax/ui is not a dependency here. Both
deviations are documented in ARCHITECTURE.md §5 and §8.
Docs-only change. No code, CI, LICENSE or workflow files were modified.
An independent review of this unification branch flagged that the tree asserts the
bypass/stealth/proxy-rotation paths are 'excluded by design' while
services/scrapers/ still contains all 64 files implementing exactly that --
curl_cffi TLS impersonation, cloudscraper, Tor rotation, WAFBypass, ProxyRotator,
plus committed .pyc bytecode.
Both statements were true in their own frame: CLAUDE.md §1 is the policy, and the
code is the state. The README stated the policy in the present tense, which reads
as a description of the tree.
This adds the state next to the policy and points at PR #1, which exists to delete
the directory and has been open since 2026-07-19. docs/06-data-engine.md already
documented the arrival honestly; this brings the README in line with it.
Deliberately NOT deleting services/scrapers/ here. That is PR #1's job, and folding
a policy deletion into a history-unification merge would make both harder to
review. Recording the truth is what this merge owes; removing the code is a
separate, reviewable change.
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Merges github/main (6 commits: 17-bug audit fix, CI quality gate,
.env.example + .gitignore fix, missing platform clients + registry
services, docs accuracy overhaul) into gitea/main (8 commits: Iceberg
write boundary + live ST0RE verification, lakehouse mapping routes,
Waves 1-4 foundational hardening).
16 conflicts. Both sides had independently written the four
src/clients/*.ts and the two registry services, so those are combined
rather than picked:
- st0re-client.ts: GitHub's endpoint wins on evidence. POST goes to
/v1/st0re/artifacts/register, not /v1/st0re/artifacts -- verified in
sax3l/st0re src/api/routes/artifacts.ts, where the bare path is a
GET-only list route. Gitea's URL normalisation, error-body text and
mock kept.
- n0d-client.ts: field is `status`, not `state`, with N0D's real 13-state
union -- verified in sax3l/n0d packages/schemas/n0d.job.schema.json,
which says explicitly not to add "succeeded". Gitea's mock echo,
URL normalisation and error-body text kept.
- cl0ud-client.ts: Gitea's fail-closed HTTP normalisation kept (a 200
with no `decision` reads as "deny", not as a silent allow). GitHub's
PolicyDecision/PolicyDecisionValue naming kept; PolicyResult retained
as an alias.
- aud0-client.ts: permissive union of both input types (optional
payload/metadata/correlation_id), GitHub's echo return, Gitea's URL
normalisation and error-body text, plus a JSON-parse fallback so a
successful POST with an empty body does not throw.
- dataset-service.ts / source-service.ts: GitHub's record/DTO split and
event naming (0pt.source_created / 0pt.create_source, which the merged
docs already reference) plus Gitea's name validation, description /
created_by fields and actor_id derivation. List filters are the union.
- lakehouse mapping routes: GitHub's shared lakehouse-mapping-registry
singletons (without which every write 400'd "mapping not found") AND
Gitea's Zod parseBody validation. Both, not either.
- src/types/source.ts, src/types/dataset.ts: unions of both vocabularies;
GitHub's `access` legal-sourcing field kept.
- .env.example: GitHub's audited inventory plus Gitea's Zitadel /
AUTH_ENFORCE block.
- CLAUDE.md, docs/ICEBERG_WRITE_BOUNDARY.md: GitHub's corrections kept,
Gitea's live-verification kept and marked as superseding the
"would 400 outright" finding (Gitea had already added the required
X-Tenant-Id/X-Actor-Id headers).
- pnpm-lock.yaml: regenerated with pnpm 9.15.9, not merged.
Also corrects CLAUDE.md's claim that the banned services/scrapers stack
was "since removed" -- the removal PR is open, not merged, and this
branch inherits that code from Gitea main. Nothing here resurrects,
extends or re-wires any of it; services/scrapers/ is byte-identical to
Gitea main.
Verified on the merged tree: typecheck (src/ + tests/) clean;
17 test files, 121 passed, 1 skipped (the realinfra test skipping
honestly on ST0RE HTTP 401), 0 failures. Gitea main baseline was the
same 121/1; GitHub main baseline could not even `pnpm install`
("packages field missing or empty") -- Gitea's pnpm-workspace.yaml fix
resolves that. Whole-repo `typecheck:app` reports 9 errors, an identical
set to Gitea main's baseline, so it is advisory in both CI workflows
rather than deleted. Lint: 16 problems, identical to Gitea baseline.
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
The auth seam read `process.env.AUTH_ENFORCE === 'true'`, so an absent env
var meant NO authentication: any fresh container, local run, DR restore or
redeploy that dropped the variable served /api/0pt/* anonymously, taking the
tenant from the caller-supplied X-Tenant-Id header. 0pt.app is deployed and
was answering anonymously (verified: an invalid Bearer token plus an
arbitrary X-Tenant-Id returned 200), so this was a live hole, not latent.
* authEnforced() now returns true unless AUTH_ENFORCE is explicitly "false"
(trimmed, case-insensitive). Unset/empty/misspelled => ENFORCED.
* assertAuthConfigSafe() throws AuthMisconfigurationError when
NODE_ENV=production and enforcement resolved off, naming AUTH_ENFORCE. It
runs at middleware module load (startup) and on the env-driven request
path, so a production instance can neither boot nor serve a single
unauthenticated request with auth switched off.
* .env.example no longer teaches AUTH_ENFORCE=false; README's Deploy section
states the fail-closed posture (no DEPLOY.md in this repo).
* 7 new tests, incl. the regression case: with AUTH_ENFORCE unset and no
`enforce` option, an unauthenticated request is rejected with 401. No test
was inverted or deleted -- none asserted the open default; the three
enforce:false cases now describe the explicit local-dev escape hatch.
Tests: 121 passed / 1 skipped before, 128 passed / 1 skipped after.
Typecheck and build clean; advisory lint unchanged (16 pre-existing problems).
Scope: auth default only. The matcher still covers /api/0pt/* alone, and the
handlers still take tenant_id from the query/body rather than the verified
principal -- both described in the PR body, neither fixed here.
bridge_provenance answers which source a field came from; it does not
answer what happened on the wire. 1p (the SIAX egress measurement point)
adds the robots decision + sha256 of the robots.txt it was read from,
body_sha256 per fetch, and blocked_soft detection — HTTP 200 whose body
is a block page, which otherwise gets scored as corroborating data.
Consistent with CLAUDE.md §1: a measurement point, not a bypass.
Documentation only; not in use, no worker routes through it.
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
admin