BACKLOG P0-13: next.config.mjs now sets HSTS, X-Frame-Options,
X-Content-Type-Options, Referrer-Policy, Permissions-Policy, and a
same-origin CSP on every route. Previously the only security header came
from Vercel's platform default (HSTS) -- nothing was set by the app itself,
so it would have silently disappeared on cutover to Coolify/Traefik.
Verified at runtime (curl -I against a production build), not just that the
build compiles.
BACKLOG P0-16: Trivy in security.yml/nightly.yml now actually fails the
build on HIGH/CRITICAL findings (`--exit-code 1`) instead of the previous
`--exit-code 0` (purely advisory, never blocked a merge). Verified locally
against this repo's real dependency tree -- 19 HIGH findings currently exist
in pnpm-lock.yaml (next.js, postcss, sharp, nanoid), several in next.js
itself relating to middleware/auth bypass. See PR description: this is
expected to turn the security.yml `trivy` job red until those are bumped in
a follow-up -- that's the fix working as intended, not a regression, but it
needs a human decision on timing given the sibling auth-gate PR (#11).
`services/scrapers/` stays explicitly excluded from the scan
(--skip-dirs) -- that tree is a dated, deliberate keep-decision under
active review in a separate thread (BACKLOG P3-3) and is out of scope here.
BACKLOG P0-19: dedicated gitleaks job added to security.yml, full git
history (--log-opts=--all), pinned + checksum-verified binary. Verified
locally against this repo's actual history first (81 commits, no leaks) so
this blocking gate doesn't open red on day one.
BACKLOG P1-14: SBOM (CycloneDX) + license inventory (`pnpm licenses list`)
generated as informational, non-blocking build artifacts. actions/checkout,
actions/setup-node, actions/cache, actions/upload-artifact all pinned to
commit SHA (was floating @v4 tags) across every workflow file, Trivy/gitleaks
pinned to specific released versions instead of floating install scripts.
cosign image-signing explicitly NOT added -- there is no container-registry
publish step in this repo's CI to attach a signature to yet (Coolify builds
directly from git); flagging as a real follow-up once that pipeline exists.
BACKLOG P1-15: explicit `permissions: contents: read` + `concurrency` added
to every workflow that lacked them (only .github/workflows/ci.yml already
had concurrency). Pinned pnpm-store cache added to ci.yml's quality job.
Root CODEOWNERS added (`* @sax3l`) so Gitea has someone to assign once
required-approvals is turned on (separate repo-settings change, not a file
change). The "doctor step" part of P1-15 is added in the follow-up repo-
hygiene PR, alongside the `siax:doctor` script it invokes.
AGENTS.md and SECURITY.md previously attributed keeping the WAF-bypass/
Tor/FlareSolverr/curl_cffi/cloudscraper stack to an unattributed,
unverified "medvetet användarbeslut" (deliberate user decision) with no
stated author or date -- flagged by a prior audit as unverified provenance.
Replace with an accurate, dated statement: the account owner (Simon
Axelsson) was asked directly via an interactive decision prompt on
2026-08-11, in direct response to a compliance audit's removal proposal
(the surgically-clean removal PR #7, verified 128/128 tests, was closed
unmerged), and explicitly chose to keep the code. No business or legal
rationale beyond "keep it" was given or is invented here.
This is a narrow, scoped exception for the existing 0PT/0pt-motor sourcing
engine. CLAUDE.md §1's general anti-evasion policy text is untouched and
remains in force as written unless/until the account owner says otherwise.
No code changes -- documentation only.
Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>
A live compliance audit against 0pt.app confirmed both endpoints returned
HTTP 200 with zero authentication -- the full order queue and financial
pipeline aggregates (deposit/pipeline totals) were readable by anyone.
Root cause: middleware.ts's matcher only covers /api/0pt/:path*, by design
("does NOT touch ... the unrelated payment/order APIs"), so nothing upstream
of these two handlers ever verified the caller.
Fix: both GET handlers now call the same fail-closed Zitadel verifier
/api/0pt/* relies on (resolveAuthContext, src/auth/context.ts) directly, via
a new thin wrapper (src/auth/require-route-auth.ts) that maps a rejected
decision to the same 401/403 JSON shape middleware.ts already returns. The
wrapper re-verifies the bearer token itself rather than trusting any
x-0pt-verified-* header on the request, since middleware never runs in front
of these routes to overwrite a client-supplied one.
POST /api/orders (anonymous draft/quote creation before the deposit-payment
step) is intentionally left as-is -- only the read paths that leaked data are
gated. Tracked as part of docs/compliance/BACKLOG.md P0-3.
Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>
siax.config.json declared deploy.target: "vercel", which conflicts with
the SIAX Cloud platform pattern every sibling OS repo (n0d, st0re, aud0)
follows: a Dockerfile "for Coolify" (already present in this repo) plus a
.gitea/workflows/deploy.yml that POSTs a restart to a Coolify app UUID.
Investigated before changing anything:
- 0pt.app is confirmed live on Vercel (Server: Vercel, 200 OK) but its own
/api/health reports db/queue "not_configured", and POSTing to /api/orders
creates an order that is not durably persisted (in-memory store only, per
this repo's own docs/08-infrastructure.md and docs/architecture/
ARCHITECTURE.md, both independently verified live 2026-07-19 / 2026-08-07,
and reconfirmed live again today). It's a fully-built marketing/demo
surface with no real backend behind it -- not a live product serving real
customer data. Safe to change deploy config without a cutover plan.
- Dockerfile already exists and is already written "for Coolify" (node:24-
alpine multistage, non-root, /api/health healthcheck) -- no changes needed
there.
- No .gitea/workflows/deploy.yml existed. Added one following the exact
pattern used by n0d and st0re (POST /api/v1/deploy?uuid=...&force=false
with COOLIFY_TOKEN). No Coolify app has been provisioned for 0pt yet, so
the app UUID is a literal placeholder and the workflow fails loudly by
design until a real app + token are in place -- this does not silently
ship a broken deploy config.
This changes only this repo's declared *intent*. It does NOT touch the live
Vercel deployment, delete the Vercel project, or move any traffic --
that cutover (provision Coolify app -> verify -> DNS -> decommission
Vercel) is a separate, larger decision. README and ARCHITECTURE.md updated
to state this plainly rather than let them contradict the new config.
Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>
Brings 0pt to the SIAX repo standard. README, LICENSE (proprietary, correct
for this repo), Dockerfile and .gitea/workflows/ci.yml already existed and are
left untouched; no .github/workflows exists.
The architecture document was written by reading the code (entrypoints,
routes, services, migrations, Dockerfile, env, tests) and by probing the live
0pt.app instance -- not from the templates. It therefore contradicts the
existing docs in several places, deliberately:
* /api/0pt/* answers ANONYMOUSLY in production (live-verified 2026-08-07):
GET https://0pt.app/api/0pt returns 200 with no token, and 200 with a
deliberately invalid one. The deployment is current (the newer lakehouse
routes are present), so middleware.ts is simply not running. README lines
49-52 and merge 6531384 claim the opposite. Root cause UNKNOWN -- flagged,
not guessed.
* No handler reads x-0pt-verified-tenant. middleware.ts sets it; nothing else
in the repo reads it. Every /api/0pt/* handler takes tenant_id from the
request body/query instead, so tenant isolation is not implemented.
* Everything outside /api/0pt/* has no auth at all (matcher scope), including
GET /api/admin/metrics and GET /api/orders (both live-verified 200 anon) and
POST /api/payments/{id}/release, which flips an order to "delivered"
regardless of payment.
* CL0UD policy client is fail-open and AUD0 audit is a silent no-op when their
URLs are unset, with no production guard -- unlike AUTH_ENFORCE, which has one.
* connections/db-grant documents AES-GCM encryption it does not implement.
* The FastAPI engine (services/api) has zero authentication despite
lib/engine.ts sending X-Internal-Token.
* services/scrapers/waf_bypass.py (FlareSolverr, Tor, cloudscraper, proxy pool,
undetected browser, fake identities) plus flaresolverr/tor-proxy in
services/infra/docker-compose.data-stack.yml directly contradict CLAUDE.md
§1 and the README's compliance claim. Left in place -- deletion is a Simon
decision, recorded as open question 2.
* Production has NO database (live /api/health reports db:not_configured) and
the whole /api/0pt/* capability layer is per-module in-memory Maps, so state
does not survive a cold start and is not even shared between route modules.
siax.config.json deviates from the template on two fields, on purpose:
deploy.target is "vercel" (live-verified Server: Vercel header), not "coolify",
and designSystem is null because @siax/ui is not a dependency here. Both
deviations are documented in ARCHITECTURE.md §5 and §8.
Docs-only change. No code, CI, LICENSE or workflow files were modified.
An independent review of this unification branch flagged that the tree asserts the
bypass/stealth/proxy-rotation paths are 'excluded by design' while
services/scrapers/ still contains all 64 files implementing exactly that --
curl_cffi TLS impersonation, cloudscraper, Tor rotation, WAFBypass, ProxyRotator,
plus committed .pyc bytecode.
Both statements were true in their own frame: CLAUDE.md §1 is the policy, and the
code is the state. The README stated the policy in the present tense, which reads
as a description of the tree.
This adds the state next to the policy and points at PR #1, which exists to delete
the directory and has been open since 2026-07-19. docs/06-data-engine.md already
documented the arrival honestly; this brings the README in line with it.
Deliberately NOT deleting services/scrapers/ here. That is PR #1's job, and folding
a policy deletion into a history-unification merge would make both harder to
review. Recording the truth is what this merge owes; removing the code is a
separate, reviewable change.
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Merges github/main (6 commits: 17-bug audit fix, CI quality gate,
.env.example + .gitignore fix, missing platform clients + registry
services, docs accuracy overhaul) into gitea/main (8 commits: Iceberg
write boundary + live ST0RE verification, lakehouse mapping routes,
Waves 1-4 foundational hardening).
16 conflicts. Both sides had independently written the four
src/clients/*.ts and the two registry services, so those are combined
rather than picked:
- st0re-client.ts: GitHub's endpoint wins on evidence. POST goes to
/v1/st0re/artifacts/register, not /v1/st0re/artifacts -- verified in
sax3l/st0re src/api/routes/artifacts.ts, where the bare path is a
GET-only list route. Gitea's URL normalisation, error-body text and
mock kept.
- n0d-client.ts: field is `status`, not `state`, with N0D's real 13-state
union -- verified in sax3l/n0d packages/schemas/n0d.job.schema.json,
which says explicitly not to add "succeeded". Gitea's mock echo,
URL normalisation and error-body text kept.
- cl0ud-client.ts: Gitea's fail-closed HTTP normalisation kept (a 200
with no `decision` reads as "deny", not as a silent allow). GitHub's
PolicyDecision/PolicyDecisionValue naming kept; PolicyResult retained
as an alias.
- aud0-client.ts: permissive union of both input types (optional
payload/metadata/correlation_id), GitHub's echo return, Gitea's URL
normalisation and error-body text, plus a JSON-parse fallback so a
successful POST with an empty body does not throw.
- dataset-service.ts / source-service.ts: GitHub's record/DTO split and
event naming (0pt.source_created / 0pt.create_source, which the merged
docs already reference) plus Gitea's name validation, description /
created_by fields and actor_id derivation. List filters are the union.
- lakehouse mapping routes: GitHub's shared lakehouse-mapping-registry
singletons (without which every write 400'd "mapping not found") AND
Gitea's Zod parseBody validation. Both, not either.
- src/types/source.ts, src/types/dataset.ts: unions of both vocabularies;
GitHub's `access` legal-sourcing field kept.
- .env.example: GitHub's audited inventory plus Gitea's Zitadel /
AUTH_ENFORCE block.
- CLAUDE.md, docs/ICEBERG_WRITE_BOUNDARY.md: GitHub's corrections kept,
Gitea's live-verification kept and marked as superseding the
"would 400 outright" finding (Gitea had already added the required
X-Tenant-Id/X-Actor-Id headers).
- pnpm-lock.yaml: regenerated with pnpm 9.15.9, not merged.
Also corrects CLAUDE.md's claim that the banned services/scrapers stack
was "since removed" -- the removal PR is open, not merged, and this
branch inherits that code from Gitea main. Nothing here resurrects,
extends or re-wires any of it; services/scrapers/ is byte-identical to
Gitea main.
Verified on the merged tree: typecheck (src/ + tests/) clean;
17 test files, 121 passed, 1 skipped (the realinfra test skipping
honestly on ST0RE HTTP 401), 0 failures. Gitea main baseline was the
same 121/1; GitHub main baseline could not even `pnpm install`
("packages field missing or empty") -- Gitea's pnpm-workspace.yaml fix
resolves that. Whole-repo `typecheck:app` reports 9 errors, an identical
set to Gitea main's baseline, so it is advisory in both CI workflows
rather than deleted. Lint: 16 problems, identical to Gitea baseline.
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
The auth seam read `process.env.AUTH_ENFORCE === 'true'`, so an absent env
var meant NO authentication: any fresh container, local run, DR restore or
redeploy that dropped the variable served /api/0pt/* anonymously, taking the
tenant from the caller-supplied X-Tenant-Id header. 0pt.app is deployed and
was answering anonymously (verified: an invalid Bearer token plus an
arbitrary X-Tenant-Id returned 200), so this was a live hole, not latent.
* authEnforced() now returns true unless AUTH_ENFORCE is explicitly "false"
(trimmed, case-insensitive). Unset/empty/misspelled => ENFORCED.
* assertAuthConfigSafe() throws AuthMisconfigurationError when
NODE_ENV=production and enforcement resolved off, naming AUTH_ENFORCE. It
runs at middleware module load (startup) and on the env-driven request
path, so a production instance can neither boot nor serve a single
unauthenticated request with auth switched off.
* .env.example no longer teaches AUTH_ENFORCE=false; README's Deploy section
states the fail-closed posture (no DEPLOY.md in this repo).
* 7 new tests, incl. the regression case: with AUTH_ENFORCE unset and no
`enforce` option, an unauthenticated request is rejected with 401. No test
was inverted or deleted -- none asserted the open default; the three
enforce:false cases now describe the explicit local-dev escape hatch.
Tests: 121 passed / 1 skipped before, 128 passed / 1 skipped after.
Typecheck and build clean; advisory lint unchanged (16 pre-existing problems).
Scope: auth default only. The matcher still covers /api/0pt/* alone, and the
handlers still take tenant_id from the query/body rather than the verified
principal -- both described in the PR body, neither fixed here.
bridge_provenance answers which source a field came from; it does not
answer what happened on the wire. 1p (the SIAX egress measurement point)
adds the robots decision + sha256 of the robots.txt it was read from,
body_sha256 per fetch, and blocked_soft detection — HTTP 200 whose body
is a block page, which otherwise gets scored as corroborating data.
Consistent with CLAUDE.md §1: a measurement point, not a bypass.
Documentation only; not in use, no worker routes through it.
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
* ci: add typecheck/lint/test quality gate (was deploying unguarded)
* ci: make quality gate functional — typecheck + test, drop unwired lint
The lint step ran `pnpm run lint` = `eslint .`, but eslint is not a dependency
of this repo (absent from pnpm-lock.yaml), so the step failed on command-not-found
and, since a failed step aborts the job, prevented typecheck from ever running —
a red, useless gate on every PR. Replace it with the vitest suite (`pnpm run test`,
88 passing) so the gate matches the PR's stated 'typecheck + test' intent and is
green on a clean tree. Typecheck now uses `pnpm run typecheck` (carries the
`-p tsconfig.json` the script defines). Wiring eslint is left as a separate decision.
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
---------
Co-authored-by: Claude Opus 4.8 <noreply@anthropic.com>
* docs: correct stale quality-check-service.ts comment references
Two source comments still cited src/services/quality-check-service.ts as an
existing precedent. It doesn't exist anywhere in this repo's history (git
log --all: zero commits, any branch, ever) -- confirmed during the earlier
full-doc-corpus accuracy pass. Points each comment at the real precedent
instead (document-extraction-service.ts / iceberg-write-service.ts).
Applied from task_05ec36aa, spun off during that pass.
* fix: 17 confirmed bugs from full-repo audit (edge + Python engine)
Found by an 8-finder bug hunt across app/api, src/services, src/clients,
frontend, the Python engine, the data layer, config/infra, and tests, then
independently adversarially verified per finding (32 raw findings -> 27
confirmed real bugs; 5 refuted). This commit applies the 17 confirmed as
safe, mechanical fixes -- no architecture/design decision required. The
other 10 need a decision and are reported separately, not touched here.
Most consequential, app/api/payments/[order_id]/deposit/route.ts:
the client-supplied Origin request header was trusted unsanitized both as
a server-side fetch target (SSRF) and as Stripe Checkout success/cancel
redirect URLs (open redirect tied to a real payment flow), and a failed
order lookup silently fell through to a hardcoded deposit amount instead
of 404. Fixed by reading the order directly via lib/orders-store.ts
(mirroring the sibling release/route.ts, which already did this correctly)
instead of both trusting the Origin header and self-issuing an HTTP fetch.
Also fixed:
- 5 routes (orders, orders/[id], estimates, subscriptions, ai/parse) that
500'd instead of 400ing on a literal JSON `null` body, since req.json()
resolves (not rejects) for that input and bypasses the try/catch guard.
- app/api/0pt/lakehouse/mappings/{,[mappingId]/{,writes/}}route.ts each
instantiated their own private service instances, so a mapping created
via POST was invisible to the sibling GET and writes routes even before
the separately-tracked dataset-registration gap. Extracted a shared
singleton module, src/services/lakehouse-mapping-registry.ts.
- app/api/0pt/documents/[documentId]/extraction/route.ts collapsed every
error (not-configured, policy-denied, policy-approval-required,
not-found) into a blanket 400; now status-mapped like the sibling
sources/[sourceId]/query route.
- app/api/vehicles/route.ts: plate query param wasn't URL-encoded before
being concatenated into the outbound RDW request, allowing extra query
parameters to be injected; added a strict allowlist + encodeURIComponent.
- app/api/uploads/route.ts: POST returned a fake 200 success with
put_url: null even when "configured", instead of continuing to report
501 until presign generation is actually implemented.
- src/adapters/document-intelligence-adapter.ts: the result-polling loop
never checked response.ok, so a non-2xx poll response (auth expiry,
429, 5xx) was silently retried until a generic timeout masked the real
error.
- app/order/[id]/page.tsx: finish() never cleared the production-simulation
setInterval, so it kept firing (and re-rendering) forever after delivery.
- app/admin/page.tsx: the metrics fetch had no response/shape validation
(unlike the adjacent orders fetch), rendering literal "undefined" stat
tiles on a malformed response instead of falling back to <ErrorState/>.
- services/api/workers/pipeline.py: `config.get("seeds") or DEMO_SEEDS`
used truthy-or, so an explicit empty seeds dict was silently replaced
with hardcoded demo org numbers instead of being respected as empty.
- services/api/workers/resolve.py: corroboration (confidence/agreed_sources
bump) no longer fires for two hits from the same source under fuzzy
match -- only a genuinely independent source now counts, per docs/06's
own ">=2 independent sources agree" definition -- and a merged record's
provenance array now actually includes the corroborating source's
fields instead of only a bare source-name string.
- services/api/orchestrator/main.py: create_order's int(...) on
unvalidated client volume/confidence fields raised on any
decimal-formatted input (int("92.5") -> ValueError); now coerced via
float() first with a safe default. /vehicles/{plate} and /screen also
now catch connector exceptions and degrade gracefully, matching
pipeline.py's existing pattern for the same connectors.
- lib/orders-store.ts: postgres.js (pinned v3.4.9) returns bigint/numeric
columns as strings by default; getOrder/listOrders/setStatus were
silently handing callers stringified money fields. Added a narrow
per-row coercion instead of touching the shared client's global type
map (which would affect unrelated tables' bigint/numeric columns too).
- lib/subscriptions-store.ts: the in-memory fallback returned
oldest-first while the Postgres branch explicitly orders newest-first;
now consistent, mirroring the identical fix already in orders-store.ts.
- app/api/auth/login/route.ts: DEV_LOGIN_KEY was compared with a plain
!==, a non-constant-time comparison; replaced with crypto.timingSafeEqual.
Verified before commit: tsc --noEmit clean; vitest run 88 passed, 1 todo,
0 failures (13 files, unchanged from before this commit -- no existing
test's assertions needed to change). services/api/workers/resolve.py's
source-novelty and provenance-merge logic additionally verified by direct
execution against synthetic same-source/cross-source inputs (no Python
test suite exists in this repo to add to). services/api/workers/pipeline.py
and services/api/orchestrator/main.py verified with python -m py_compile
(no httpx/fastapi installed in this environment to run them directly).
Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>
* fix(engine): treat explicit seeds:null as missing to avoid 500 in run_job
Four-eyes follow-up to this PR's own seeds change. Switching from
`config.get('seeds') or DEMO_SEEDS` to `config['seeds'] if 'seeds' in config`
correctly stopped an explicit empty {} from being backfilled with DEMO_SEEDS,
but regressed on client-controlled `{"seeds": null}`: seeds became None and
the downstream `seeds.get(market, [])` raised AttributeError -> 500. That path
is reachable (POST /orders stores raw body.config; POST /orders/{id}/run passes
it straight to run_job). Using `config.get('seeds') is not None` keeps the
intended behaviour (explicit {} honoured, never silently seeded with demo data)
while restoring the None-tolerance the old idiom had.
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
---------
Co-authored-by: Claude Sonnet 5 <noreply@anthropic.com>
* docs: full accuracy overhaul, grounded claim-by-claim in real code
Every doc in this repo (CLAUDE.md, README.md, SECURITY.md, docs/*.md) was
audited and rewritten by a multi-pass process: independent rewrite per
cluster, adversarial re-verification of that rewrite against the actual
code, then a final cross-doc consistency pass — each stage re-checking
claims against the repo directly rather than trusting the previous stage.
Two corrections matter most:
1. docs/ICEBERG_WRITE_BOUNDARY.md, docs/11-lakehouse-write-plan.md, and
docs/P0_QUERY_SOURCE_FLOW.md asserted src/services/quality-check-service.ts,
quality-report-service.ts, src/data-platform/services/catalog-service.ts,
and sql/002_p0_dataset_quality.sql/sql/003_*.sql as real, tested code.
git log --all across every branch shows zero commits for any of these
paths, ever. 0pt's data-quality/QA enforcement does not exist in code.
Corrected everywhere, with a new docs/P0_DATASET_QUALITY_FLOW.md added
as an explicit target-spec (not-yet-built) replacement.
2. github.com/sax3l/0pt (this repo) and git.siax.io/sax3l/0pt (Gitea) are
two diverged remotes sharing part of the same history (services/api/
workers/adaptive_router.py is byte-identical between them). Gitea's main
independently accumulated a WAF-bypass/proxy-rotation scraper stack in
three 2026-07-13 commits, since removed via a Gitea PR; GitHub's main
never had it. Which remote is canonical/deployed is unresolved — flagged
consistently across the corpus, not decided here.
Also newly and consistently documented: the tenant_id/project_id-read-from-
client-body gap on the P0 lakehouse/rate-policy/source-query routes (no
auth/session system exists anywhere in this repo yet); that every CL0UD/
N0D/AUD0/ST0RE platform client is real but mock-only (zero process.env
reads for any of their *_URL vars); that two P0 mutation routes
(lakehouse/mappings, sources/rate-policies) always fail over HTTP today
because their prerequisite entity can never be created via any route; a
schema bug in sql/004 (FK references to opt_datasets/opt_sources, which
don't exist); and that 0pt.app's live deployment was independently
confirmed (curled this pass) to be running this exact codebase with
DATABASE_URL/REDIS_URL unset in production.
New file: docs/12-edge-capability-api.md — the edge-native P0 TypeScript
capability contract (app/api/0pt/*, /api/screen, /api/ai/parse), previously
undocumented and distinct from docs/04-api.md's FastAPI-engine-only scope.
Verified before commit: tsc --noEmit clean; vitest run 88 passed, 1 todo,
0 failures (13 files) — docs-only change, no source touched.
Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>
* docs: reflect PR #5's lakehouse-mapping-registry + status-code fixes
Follow-up to keep docs/12-edge-capability-api.md accurate once both PRs
land: fix/0pt-full-bug-hunt (#5) fixed the shared-instance bug in the
lakehouse mapping routes and added proper status-code mapping to the
document extraction route. This doc was written against the pre-fix
state in the same pass that added it, so it needs the same correction.
Does not claim the lakehouse write path fully works now -- the deeper
dataset-registration gap (no route ever calls DatasetService.registerDataset)
is unchanged and still blocks it end-to-end; only the shared-instance half
of the bug is fixed.
---------
Co-authored-by: Claude Sonnet 5 <noreply@anthropic.com>
.env.example was referenced by README.md and required by docs/09-build-plan.md
(Phase 0) but never existed in the repo. Root cause: .gitignore's `.env*` rule
silently ignored it too, so `git add .env.example` was always a no-op even if
someone created it locally.
- Add .gitignore exception: `!.env.example`.
- Add .env.example covering every process.env.* read found in this pass
(grep-verified across app/, lib/, src/), organized by concern, each section
marked "wired in code today" or "documented target, not yet read by code"
per docs/08-infrastructure.md. No real values; matches repo convention.
Excluded ODOO_BASE_URL: only appears inside a comment describing a past
refactor away from module-level env reads, not an actual process.env read.
pnpm test: 88 passed, 1 todo, 0 failures (13 files, tests/**/*.test.ts).
tsc --noEmit: clean.
The 0pt data-engine build has been broken on main since 2026-07-08 (commit
c897311). document-capture-service.ts, document-extraction-service.ts,
source-query-service.ts, external-writeback-service.ts (and, more recently,
dataset-iceberg-mapping-service.ts, iceberg-write-service.ts,
source-rate-policy-service.ts) all import six platform client modules and two
registry services that were never committed: src/clients/{aud0,cl0ud,n0d,
st0re}-client.ts, src/services/{dataset,source}-service.ts, plus
src/types/{dataset,source}.ts. `next build` failed with 17 "Module not found"
errors, and every route under app/api/0pt/* returned HTTP 500 in dev with the
Turbopack compile error leaked straight into the JSON response body.
This adds the missing files, following the same fetch-based client shape
already established by src/clients/st0re-lakehouse-client.ts (thin interface,
HTTP implementation, deterministic in-memory mock used when no base URL is
configured) and the same "pure core, thin in-memory registry, AUD0-evented
mutation" shape the existing SourceRatePolicyService/DatasetIcebergMappingService
already use for DatasetService/SourceService. Each new client mock is
documented as a mock/degrade path, never a fabricated success, per this
repo's CLAUDE.md honesty doctrine.
Also wires up "typecheck" and "test" npm scripts (tsc --noEmit / vitest run)
and adds vitest as a devDependency -- previously `pnpm test` failed
immediately with ERR_PNPM_NO_SCRIPT even though 13 test files (~89 cases)
already existed under tests/p0/.
Verified: `pnpm run typecheck` clean, `pnpm run test` 88 passed / 1 todo
(pre-existing live-ST0RE-only skip), `pnpm run build` exits 0 (39/39 routes
compiled, only the pre-existing intentional optional `mssql` import warning
remains). Re-verified live on port 4097: GET /api/0pt/sources/rate-policies,
GET /api/0pt/lakehouse/mappings, POST /api/0pt/documents/capture, GET
/api/0pt/documents/[id]/extraction, and both GET/POST
/api/0pt/sources/[id]/query now return real application responses (200/400/405)
instead of HTTP 500 with a leaked compile error.
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Re-runs this session's task spec against 0pt's actual repo state, which
turned out to already have a substantial, tested implementation from a
prior pass (35b6ec6): dataset<->Iceberg-table mapping, source rate policy,
and an ST0RE lakehouse client that was honestly marked "not live-verified".
This pass found api.st0re.siax.io IS reachable from this environment and
used it -- first via curl to safely discover the real contract, then
through 0pt's own updated code -- to drive a real namespace -> table ->
snapshot write against production ST0RE (task item 3's "write ONE first
test dataset" proof). That surfaced and fixed three real bugs the
mock-only pass could not have caught:
- ST0RE requires X-Tenant-Id/X-Actor-Id headers on every lakehouse route;
the previous HTTP client sent none. Fixed via a threaded
St0reRequestContext.
- IcebergTableInput.datasetId is a foreign key into ST0RE's OWN dataset
table, not a free-form string -- sending 0pt's dataset_id 400s live with
a Postgres FK violation. Fixed by no longer sending it; 0pt's
dataset<->table linkage lives entirely in DatasetIcebergMapping.
- ST0RE enforces a real unique constraint on lakehouse namespaces
(tenant_id, project_id, name) with no upsert route. Fixed with a
client-side lookup-or-create (listNamespaces + findOrRegisterNamespace).
Also closed a field-level gap against the task's literal schema spec:
0pt.dataset_iceberg_mapping.schema.v1 now carries namespace_id (a real
reference to ST0RE's LakehouseNamespace.namespaceId, not guessed) and
mapped_at, both additive alongside the existing namespace/created_at
fields (sql/005, JSON schema, types, service).
tests/p0/iceberg-write-service.realinfra.test.ts replaces the prior
it.todo with a real SEAM-pattern live test: attempts the real HTTP call,
skips honestly if ST0RE is unreachable, never fabricates success. It ran
successfully against production ST0RE this session (real namespace/table/
snapshot UUIDs logged in docs/ICEBERG_WRITE_BOUNDARY.md).
Two ST0RE-side cross-tenant read bugs were found and flagged (not fixed
here -- out of scope for this repo): GET .../namespaces ignores its own
tenant header and query filters; GET .../tables/:id doesn't check the
caller's tenant against the table's owner.
npm test: 155 passed, 0 failed, 0 todo (up from 154 passed + 1 todo).
npx tsc --noEmit: 1 pre-existing, unrelated error in
quality-report-service.ts (present before this pass, part of the wider
uncommitted P0 data-platform build already in this repo) -- not touched.
Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>
Real, tested implementation of docs/11-lakehouse-write-plan.md's items 2-8
(the "future implementation session" that doc named): 0pt is now the
canonical writer of Iceberg tables through ST0RE's real, shipped
/v1/st0re/lakehouse/* table registry (sax3l/st0re commit 9ea0abb), never
persisting lakehouse table/catalog/snapshot state itself.
- src/types + src/services for 0pt.dataset_iceberg_mapping.schema.v1
(dataset -> ST0RE tableId reference) and 0pt.source_rate_policy.schema.v1
(the sourceRatePolicy ownership repair, moved out of N0D per ADR-0030).
- src/clients/st0re-lakehouse-client.ts: new client matching ST0RE's actual
camelCase route contract, with the same mock/HTTP boundary convention as
this repo's existing cl0ud/n0d/aud0/st0re clients. Honest caveat in its
module doc comment: not verified against a live ST0RE instance in this
pass (no reachable ST0RE_URL); every test exercises the deterministic mock.
- src/services/iceberg-write-service.ts: registers tables/snapshots through
ST0RE, requests N0D jobs only for heavy writes (never executes them
in-process), emits all four AUD0 event names
(dataset_iceberg_table_mapped / iceberg_write_requested /
iceberg_snapshot_created / source_rate_policy_created).
- sql/004, packages/schemas/*.json, app/api/0pt/lakehouse + sources/rate-policies
routes, tests/p0/*.test.ts (154 passed + 1 honest it.todo for the live-ST0RE
case, up from 142 baseline, 0 failures).
- Reconciled a real collision: an earlier untracked draft schema/test pair
(tests/p0/enterprise-data-os-expansion.test.ts) asserted a different,
now-superseded source_rate_policy shape; updated that one test block only.
- docs/ICEBERG_WRITE_BOUNDARY.md states the ownership boundary plainly and
is explicit throughout about what's real-and-tested vs. contract-only.
docs/11-lakehouse-write-plan.md gets a status-update section, not a rewrite.
- Data quality rules (quality-check-service.ts, quality-report-service.ts)
were not touched -- confirmed still 0pt-owned.
Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>
Points at n0d's ADR-0030 (Apache Iceberg canonical, Delta rejected for
P0/P1) and lays out 0pt's write-path plan: dataset-to-Iceberg-table
mapping, writes through ST0RE's future table registry, keeping QA and
source-rate/crawl-compliance logic in 0pt. Schema/event names specified
precisely for a future implementation session. Written adjacent to, not
touching, the substantial live P0 data-platform build in progress in this
repo (data-contract/dataset/quality-check services).
Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>
Adds a private/proprietary license notice matching the convention used in
top0, and a vulnerability-disclosure policy with contact, scope, and
response-time expectations. No code changes.
Three new, tenant-agnostic P0 capabilities:
- 0pt.query_source: bounded, timeout/row-capped ad-hoc SQL query execution
against registered database sources, via a Fabric/OneLake adapter
(Azure AD credential-based mssql connection), with AUD0 audit events
and an app/api/0pt/sources/[sourceId]/query endpoint.
- 0pt.document_ocr (document capture + extraction): mailbox-based document
intake via Microsoft Graph, OCR/field extraction via Azure Document
Intelligence, with capture/extraction API endpoints and services that
never fabricate extracted values.
- 0pt.write_external_record (external writeback): a generic writeback-adapter
contract plus a concrete Odoo bills adapter that matches partners via
Fabric, backing an external-writeback-service for pushing records to
downstream systems of record.
Known interim gaps:
- src/adapters/local-blob-store.ts is an interim ST0RE substitute pending
the real ST0RE artifact-store integration.
- mssql is used by the Fabric/OneLake adapter but deliberately not yet
added to package.json (pending `pnpm add mssql` once the concurrent
session's package.json/dependency changes settle).
A separate, concurrently-active session has unrelated in-flight changes
in this same working tree (data-platform catalog/lineage/feature-store,
dataset quality, lakehouse/Iceberg docs, deploy scaffolding, package.json/
pnpm-lock.yaml updates, etc.) — none of that is touched or included here.
Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>
Missing GET caused a 405 → r.json() threw → error state shown instead of
the calm 'storage not connected' info note. Now returns 501+[] so the page
properly shows storageOff and 'No files yet.' empty state.
Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
Values were stored as stable Swedish IDs ('Bolag', 'Engångs') but rendered raw.
Now resolved through entityLabel()/FREQUENCY_LABEL + t() at render time so
the recap panel respects the active locale on every re-render.
Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
RouteReveal layout component applies in-view on every route change.
useReveal hook falls back to immediate rAF when element is already visible.
bestall/page.tsx: added missing useReveal ref.
katalog/[id]/page.tsx: DatasetView uses useReveal directly.
Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
admin
