* docs: full accuracy overhaul, grounded claim-by-claim in real code Every doc in this repo (CLAUDE.md, README.md, SECURITY.md, docs/*.md) was audited and rewritten by a multi-pass process: independent rewrite per cluster, adversarial re-verification of that rewrite against the actual code, then a final cross-doc consistency pass — each stage re-checking claims against the repo directly rather than trusting the previous stage. Two corrections matter most: 1. docs/ICEBERG_WRITE_BOUNDARY.md, docs/11-lakehouse-write-plan.md, and docs/P0_QUERY_SOURCE_FLOW.md asserted src/services/quality-check-service.ts, quality-report-service.ts, src/data-platform/services/catalog-service.ts, and sql/002_p0_dataset_quality.sql/sql/003_*.sql as real, tested code. git log --all across every branch shows zero commits for any of these paths, ever. 0pt's data-quality/QA enforcement does not exist in code. Corrected everywhere, with a new docs/P0_DATASET_QUALITY_FLOW.md added as an explicit target-spec (not-yet-built) replacement. 2. github.com/sax3l/0pt (this repo) and git.siax.io/sax3l/0pt (Gitea) are two diverged remotes sharing part of the same history (services/api/ workers/adaptive_router.py is byte-identical between them). Gitea's main independently accumulated a WAF-bypass/proxy-rotation scraper stack in three 2026-07-13 commits, since removed via a Gitea PR; GitHub's main never had it. Which remote is canonical/deployed is unresolved — flagged consistently across the corpus, not decided here. Also newly and consistently documented: the tenant_id/project_id-read-from- client-body gap on the P0 lakehouse/rate-policy/source-query routes (no auth/session system exists anywhere in this repo yet); that every CL0UD/ N0D/AUD0/ST0RE platform client is real but mock-only (zero process.env reads for any of their *_URL vars); that two P0 mutation routes (lakehouse/mappings, sources/rate-policies) always fail over HTTP today because their prerequisite entity can never be created via any route; a schema bug in sql/004 (FK references to opt_datasets/opt_sources, which don't exist); and that 0pt.app's live deployment was independently confirmed (curled this pass) to be running this exact codebase with DATABASE_URL/REDIS_URL unset in production. New file: docs/12-edge-capability-api.md — the edge-native P0 TypeScript capability contract (app/api/0pt/*, /api/screen, /api/ai/parse), previously undocumented and distinct from docs/04-api.md's FastAPI-engine-only scope. Verified before commit: tsc --noEmit clean; vitest run 88 passed, 1 todo, 0 failures (13 files) — docs-only change, no source touched. Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com> * docs: reflect PR #5's lakehouse-mapping-registry + status-code fixes Follow-up to keep docs/12-edge-capability-api.md accurate once both PRs land: fix/0pt-full-bug-hunt (#5) fixed the shared-instance bug in the lakehouse mapping routes and added proper status-code mapping to the document extraction route. This doc was written against the pre-fix state in the same pass that added it, so it needs the same correction. Does not claim the lakehouse write path fully works now -- the deeper dataset-registration gap (no route ever calls DatasetService.registerDataset) is unchanged and still blocks it end-to-end; only the shared-instance half of the bug is fixed. --------- Co-authored-by: Claude Sonnet 5 <noreply@anthropic.com>
9.2 KiB
CLAUDE.md — 0pt (SIAX Technology AB)
Read this first. It is the contract for how this project is built. Then read
docs/in order.
What 0pt is
0pt — your data. A Nordic/European data-services platform (0pt.app) where customers:
- Compose a data job (markets, entity type, fields, enrichment, frequency, output) — or browse ready-made datasets.
- Pay a 50 % deposit to start; the remaining 50 % is held in escrow and released only on approval.
- Watch production live (sourcing → extraction → structuring → enrichment → QA).
- Validate the result with full freedom (table / map / schema / quality views) and deliver to any format/destination.
Owned by SIAX Technology AB. Feeds internal products too (Protosell/FiveCRM, VehIQ).
Non-negotiable constraints (do not violate)
- Legal sourcing only. The data engine uses, in strict priority: official APIs/bulk → authorized scraping of clearly public pages (respect
robots.txt+ rate limits) → licensed partner data → otherwise the field is delivered as "not available / licensed only". We never invent values.- Do NOT build, keep, or extend: WAF/bot-detection bypass (incl. FlareSolverr, "stealth" browsers, undetected-* used to defeat detection), fingerprint/proxy rotation to evade bans, scraping behind login walls, or automated fake-account creation (Apollo etc.).
- Path corrected this pass: no
src/workers/browser_worker.pyexists anywhere in this checkout (verified via full-repo search) — the engine's actual workers live underservices/api/workers/(adaptive_router.py,pipeline.py,queue.py,resolve.py,run.py), none of which contain a FlareSolverr/bypass path today. The precedent for this concern is real but lives on the other remote: Gitea'sgit.siax.io/sax3l/0ptmainindependently accumulated a WAF-bypass/proxy-rotation/fake-identity scraper stack (services/scrapers/*) in three 2026-07-13 commits, since removed via a Gitea PR (see README "Known issues"). If new code (from either remote) reintroduces a bypass path anywhere, remove or disable it and replace with compliant fetching. This is a product-positioning + compliance requirement, not a preference. "Honest accuracy" is the whole moat.
- Postgres is the backbone. One Postgres is the source of truth. Catalog, portal, NocoDB views, Metabase dashboards, workers, exports all read/write the same DB. A tool may only enter the stack if it (a) reads that Postgres and (b) sits behind the same auth.
- Provenance per field. Every fact carries source, freshness, confidence (
bridge_provenance). This drives the UI signature too. - One tool per job. Do not add overlapping tools. The approved stack is in
docs/02-architecture.mdanddocs/07-dependencies.md. Anything not listed needs a documented reason. - GDPR-native. DPA, lawful basis, audit log, deletion-after-job. Customer DB access is pull by the customer via a read-only, time-boxed role they create — never us asking for admin, never raw creds stored in plaintext.
- Design is fixed. Mono/editorial system (white
#fff, text#0a0a0a, hairline#e5e5e5, radius 0, Inter + JetBrains Mono, single red#e40014). Exact tokens + motion indocs/05-design-system.md. No photography — imagery is rendered data (DataArtifact).
Build on existing code
A working engine repo already exists. Corrected this pass — the previous description of it was wrong on every specific: the FastAPI orchestrator lives at services/api/orchestrator/main.py (not src/orchestrator/main.py) and registers 12 route handlers (not "25+"); the workers directory is services/api/workers/ and contains adaptive_router.py, pipeline.py, queue.py, resolve.py, and run.py (not browser_worker/ocr_worker/db — none of those three files exist anywhere in this repo); and services/api/sources/sources.yaml currently covers 7 markets (SE, NO, DK, FI, GB, NL, plus global), not 20. services/api/ holds 11 Python source files total (excluding __pycache__). Reconcile with the code as it actually is, not this description's prior claims. Where this documentation defines a target contract, migrate the existing code toward it. Flag conflicts instead of guessing.
Two remotes claim to be this repo, and it's unresolved which is authoritative. origin here is github.com/sax3l/0pt. A separate Gitea remote, git.siax.io/sax3l/0pt, shares at least part of the same history but has independently diverged (see README "Known issues" for specifics). Do not assume origin is the only copy of this codebase, and do not silently resolve the question of which is canonical — that is Simon's decision.
Before trusting any status claim in this doc corpus, read README's "Known issues" section. It covers the auth/tenant-scoping gap, the platform-client mock-only status, and the underlying fact that 0pt's quality-check/QA enforcement code was never built despite once being described as real. Corrected this pass: docs/ICEBERG_WRITE_BOUNDARY.md, docs/11-lakehouse-write-plan.md, and docs/P0_QUERY_SOURCE_FLOW.md are now all consistently corrected — each carries its own "fabricated, not built" callout and none asserts quality-check-service.ts/quality-report-service.ts/etc. as real or tested anywhere in its current body (verified this pass by reading all three in full). docs/P0_DATASET_QUALITY_FLOW.md remains the explicit target-spec replacement for what that stage should look like once built. Treat any other doc claiming the QA stage already exists as wrong.
How to work
- Work in dependency order per
docs/09-build-plan.md. Finish a phase to its acceptance criteria before the next. - Small, reviewable commits. Conventional Commits (
feat:,fix:,docs:,chore:). - Frontend = Next.js App Router + TS + Tailwind + shadcn/ui on Vercel. Backend = FastAPI + workers, not on Vercel (long jobs).
- Tech docs + code identifiers in English; all visible UI copy in Swedish.
- Never hardcode secrets — use env (
docs/08-infrastructure.md). - When unsure about a source's current access/licence terms, verify before relying on it and record it in the sourcing map; do not assume.
Documentation index
| Doc | Purpose |
|---|---|
docs/01-product.md |
Vision, the three flows, order lifecycle, business model |
docs/02-architecture.md |
System architecture, stack decisions (in/out), hosting |
docs/03-data-model.md |
Postgres schema, provenance model, migrations |
docs/04-api.md |
FastAPI contract (canonical endpoints) |
docs/05-design-system.md |
Mono tokens, motion, visual system, signature |
docs/06-data-engine.md |
Workers, legal sourcing chain, pipeline, QA, fusion, sources.yaml |
docs/07-dependencies.md |
Repos/libraries to import (+ explicit exclusions) |
docs/08-infrastructure.md |
Hosting, docker-compose, env, security, CI/CD |
docs/09-build-plan.md |
Phased milestones with acceptance criteria |
docs/10-lakehouse-decision.md |
Pointer to an ecosystem-wide lakehouse ADR made outside this repo; does not change docs/02/docs/03 |
docs/11-lakehouse-write-plan.md |
0pt's own follow-up task list for the Iceberg write boundary — corrected: item 4 (quality rules) is explicitly labeled NOT BUILT, and the old fabricated quality-check-service.ts/quality-report-service.ts claim is quoted only inside its own "Correction" callout, never asserted as fact |
docs/12-edge-capability-api.md |
Added this pass. Edge-native TypeScript P0 capability API contract (app/api/0pt/*, /api/screen, /api/ai/parse) — distinct from docs/04-api.md's FastAPI-engine contract; documents several confirmed always-fails-over-HTTP routes and the tenant_id/in-memory gaps |
docs/ICEBERG_WRITE_BOUNDARY.md |
Iceberg write-path ownership record + "real vs. contract-only" verification status — corrected: its "Data quality: not yet built" section explicitly quotes and retracts the old fabricated claim rather than repeating it as fact |
docs/P0_QUERY_SOURCE_FLOW.md |
0pt.query_source bounded source-query flow — corrected by a parallel pass this session: no longer claims quality-check-service.ts exists, now points at the real precedent (iceberg-write-service.ts) instead |
docs/P0_DATASET_QUALITY_FLOW.md |
Added by that same parallel pass: an explicit target-spec-only, not-yet-built replacement for the fabricated quality-check/QA claims — read it as "should", never "does" |
Corrected this pass: this table previously listed docs/design/0pt-v0-build-brief.md and docs/design/0pt-design-coverage-and-visuals.md as if they existed. They do not — confirmed via a full-repo search; there is no docs/design/ directory at all. No docs/ design-brief content is fabricated here to fill the gap. Either write those two docs or stop referencing them; until one of those happens, treat any prior claim that they exist as false.
Definition of done (every screen, before real data)
Mono chrome · correct scroll-reveal motion · representative placeholder content · loading + empty + error states · mobile layout · provenance signature where facts appear · red used ≤ once · visible focus ring · prefers-reduced-motion respected.