0pt/docs/09-build-plan.md

09 · Build plan

Phased, in dependency order. Finish a phase to its acceptance criteria before the next. Each phase = small, reviewable commits. Reconcile with existing code rather than rewriting wholesale.

Status callouts added 2026-07-19. Each phase below now carries a Status: line, grep/read-verified against main this pass. These are conservative on purpose: a phase is only called Done when this pass found real, working code satisfying its Accept criteria; anything else is Partial (with the specific evidence and gap named) or Not started. No phase is upgraded to Done without a code citation. Where a referenced doc (e.g. docs/design/...) doesn't exist, that's flagged rather than silently worked around. See docs/08-infrastructure.md for the full env/security grounding these statuses draw on.

Phase 0 — Foundation & repo

• Monorepo structure (docs/08). Migrate existing src/orchestrator + src/workers into services/api.
• Postgres schema + migrations (docs/03); pgvector enabled. .env.example.
• docker-compose up (postgres, redis, api, worker, nocodb, metabase, caddy). /health green.
• CI skeleton (lint/typecheck/build).
• Accept: docker compose up runs the whole stack locally; migrations apply; /health returns ok for db + queue.
• Status: Partial.
  • Repo structure exists but diverged from the apps/web/packages/shared layout docs/08 originally sketched — it settled on app/ (Next.js, repo root), lib/, src/ (platform clients/services), and services/api/ (FastAPI). docs/08-infrastructure.md now documents the structure as-built.
  • Migration to services/api/orchestrator + services/api/workers is done (verified: those directories exist and contain main.py/adaptive_router.py/pipeline.py/etc.; no src/orchestrator or src/workers remain at the old paths).
  • Postgres schema is real: sql/001_init.sql (CREATE EXTENSION vector, CREATE EXTENSION pgcrypto, 17 tables incl. orgs, users, orders, jobs, audit_log, bridge_provenance, db_grants, payments) + sql/004_iceberg_write_boundary.sql. No sql/002_* or sql/003_* exists anywhere in this repo's history — some other docs (docs/ICEBERG_WRITE_BOUNDARY.md etc.) cite a sql/002_p0_dataset_quality.sql that was never built; see docs/08-infrastructure.md's "Schema state" section.
  • .env.example — now exists, on branch feature/0pt-env-example (github.com/sax3l/0pt/pull/3), not yet merged to main. It was blocked for the life of this project by .gitignore's blanket .env* rule silently matching it too; the same PR fixes that with !.env.example. Until that PR merges, main still has no .env.example.
  • infra/docker-compose.yml and infra/Caddyfile are real, committed files (not sketches) covering exactly the services listed. This pass found no evidence they've ever actually been run — no CI, no deploy log, no reachable host checked. "File exists and looks correct" ≠ "verified to run"; treat the Accept criterion ("docker compose up runs... /health returns ok") as unverified.
  • CI skeleton — not built. There is no .github/workflows/ directory in this repo at all. pnpm lint/typecheck/test exist as local scripts only.

Phase 1 — Design system in code

• Tailwind + shadcn; globals.css (grain + reveal keyframes) and useReveal verbatim (docs/05).
• Build DataArtifact primitive (table/map/schema/chart/grid) and ProvenanceBadge (docs/design prompts 5.1, 2.11).
• Accept: a sample page renders mono tokens, a DataArtifact animates on scroll, reduced-motion respected, focus rings visible.
• Status: Partial — real components exist; scroll-motion/reduced-motion/focus-ring behavior not independently exercised this pass.
  • Tailwind + shadcn/radix stack: confirmed real in package.json (tailwindcss, @radix-ui/*, class-variance-authority, tailwind-merge, tailwindcss-animate).
  • hooks/use-reveal.ts, components/data-artifact.tsx, components/provenance-badge.tsx all exist and are used across app/katalog, app/kallor, app/bestall, app/order/[id], and most components/sections/* files (grep-confirmed).
  • docs/design/0pt-v0-build-brief.md and docs/design/0pt-design-coverage-and-visuals.md (the "prompts 5.1, 2.11" this bullet references) do not exist anywhere in this repo — confirmed via full-repo search; there is no docs/design/ directory at all. CLAUDE.md's own doc index has already been corrected to flag this as a gap rather than list them as real. Treat any reference to specific numbered "build-brief prompts" in this build plan as pointing at a doc that was never written — not this file's scope to author that content.
  • Reduced-motion handling, focus-ring visibility, and actual scroll-triggered animation behavior were not executed/verified in this pass (no browser run performed) — code presence is confirmed, runtime behavior is not.

Phase 2 — Marketing + catalog (placeholder data)

• Landing components top-to-bottom (build-brief 2.1–2.10): Header, Hero, Philosophy, Catalog teaser, Engine, Visualisation gallery, Coverage, Editorial, Proof, Footer.
• Full Catalog page + Dataset detail + Sourcing-map page (coverage doc 5.4). Postgres full-text search.
• All with loading/empty/error states + mobile.
• Accept: marketing site + catalog browse work end-to-end on placeholder catalog rows; one continuous scroll reads well; DoD met per screen.
• Status: Partial.
  • Landing section components are real: components/sections/{hero,philosophy,gallery,editorial, pricing,technology,testimonials,footer,featured-products,collection}-section.tsx all exist.
  • app/katalog/page.tsx, app/katalog/[id]/page.tsx, app/kallor/page.tsx all exist.
  • Postgres full-text search was not found — grepped for to_tsvector/websearch_to_tsquery across the whole repo, zero matches. Catalog browsing today reads from lib/data.ts placeholder data, not a full-text query against Postgres. This Accept criterion's "catalog browse... on placeholder catalog rows" half is plausible; the Postgres full-text search half is not built.
  • Loading/empty/error states + mobile + DoD-per-screen were not independently audited screen-by-screen this pass — components/states.tsx (LoadingSkeleton, ErrorState) exists and is imported widely, which is evidence toward this but not a full verification.

Phase 3 — Auth & portal shell

• Auth.js + Postgres adapter; sign in/up/verify/forgot. orgs/users.
• Portal shell + Dashboard ("Mina beställningar") with empty state.
• Accept: a user can sign up, land in an empty dashboard, sign out; routes protected.
• Status: Not done — this is the most consequential gap in the whole repo, not a minor loose end.
  • There is no Auth.js integration. lib/auth.ts is a from-scratch, ~40-line HMAC-SHA256 session scaffold (Web Crypto, no dependency) whose own comment states: "Production auth = Auth.js + Postgres adapter + OAuth (docs/02) — wired once the database is provisioned." app/api/auth/login/route.ts is a dev-only login gated by a shared DEV_LOGIN_KEY, not sign-up/verify/forgot.
  • orgs and users tables exist in sql/001_init.sql (schema is ready), but nothing in the app reads/writes them for real session-bound identity today.
  • app/portal/page.tsx and sibling portal pages exist (shell is real), but routes are not protected by any real session check tied to that schema.
  • Concretely, this means the tenant/org isolation several 0pt.* API routes assume does not exist yet. app/api/0pt/lakehouse/mappings/route.ts, .../writes/route.ts, app/api/0pt/sources/rate-policies/route.ts, app/api/0pt/sources/[sourceId]/query/route.ts, and src/services/document-capture-service.ts all read tenant_id/project_id straight from the request body — anyone can claim any tenant. app/api/orders/route.ts's own GET handler carries the literal comment // org scoping added with auth. This is an open security gap, not a two-tier design choice — see docs/08-infrastructure.md's "Security & compliance" section for the full list. Do not build any doc or feature that assumes tenant-scoped auth already exists.

Phase 4 — Order lifecycle end-to-end (test mode)

• Configurator (2.12) wired to POST /estimates (real estimate math in FastAPI).
• Deposit checkout (2.13) → Stripe test Checkout → webhook → order IN_PRODUCTION.
• Production tracker (2.14) reading GET /orders/{id}/events (SSE) — pipeline may be simulated at this phase.
• Validation & delivery (2.15): tabs render from a sample delivery; accept → POST /payments/release (test) → DELIVERED; revise leaves escrow untouched.
• AI-assist (POST /ai/parse) prefilling the configurator.
• Accept: a full custom order can be configured, deposit paid (test), watched, validated, accepted, and "delivered" — state machine in docs/01 traversed; escrow split correct.
• Status: Partial — the payment/escrow spine is genuinely real; the SSE tracker is not.
  • app/api/estimates/route.ts exists.
  • Deposit checkout is real, working code: app/api/payments/[order_id]/deposit/route.ts calls Stripe's Checkout Sessions API directly (not a stub) and app/api/webhooks/stripe/route.ts verifies the webhook signature and advances order status via setStatus.
  • app/api/payments/[order_id]/release/route.ts is also real: it enforces the escrow rule in code — balance release is rejected with 409 not_accepted unless order.status === "accepted" — matching the "escrow split correct" accept criterion's intent.
  • GET /api/orders/{id}/events (SSE) does not exist. Only GET/PATCH /api/orders/{id} were found (app/api/orders/[id]/route.ts); no /events sub-route, and no EventSource/ text/event-stream/ReadableStream usage anywhere in app/ (grep-verified). If a production tracker is live today, it's polling GET /api/orders/{id}, not the SSE stream this bullet describes — or it hasn't been built. Either way, the SSE contract as written is not implemented.
  • app/api/ai/parse/route.ts is real: a genuine (non-fabricated) keyword/regex heuristic that prefills a config object, with a documented fallback to engineFetch("/ai/parse", ...) when ENGINE_URL is set. This one holds up as described.

Phase 5 — Real data engine (Nordic slice)

• Compliant connectors first: GLEIF + Brreg + Companies House (verify access terms; record in sources.yaml). Remove any FlareSolverr/bypass path (docs/06).
• Pipeline stages produce real dim_company + fact_financials with bridge_provenance; QA threshold drops low-confidence rows.
• Production tracker now reflects real job metrics.
• Accept: a Nordic company dataset is produced for a real order, every field has provenance, quality report generated, level-0 fields shown honestly (not fabricated).
• Status: Partial.
  • services/api/sources/sources.yaml and connectors.py are real: GLEIF, Brreg, Companies House, PRH, and CVR are all registered, each with an explicit licence: "... — VERIFY" tag rather than an assumed-clean claim — matches the "verify access terms" instruction.
  • services/api/workers/adaptive_router.py's own comment confirms: "There are NO bypass/stealth/ proxy-rotation tiers here, by design (CLAUDE.md §1)" — the FlareSolverr/bypass removal is done, at least on this repo's main (see docs/08-infrastructure.md's "Known open question" about the diverged Gitea remote, which independently added such a stack on its own main — unresolved which remote is canonical).
  • services/api/workers/pipeline.py has a real, working confidence-threshold QA drop: delivered = [r for r in deduped if r["confidence"] >= threshold], plus an avg_confidence figure in its output. This is a genuine, narrow QA mechanism inside the Python engine.
  • This is not the same thing as the "quality report" / quality-check layer once described in docs/ICEBERG_WRITE_BOUNDARY.md, docs/11-lakehouse-write-plan.md, and docs/P0_QUERY_SOURCE_FLOW.md. Those three docs used to cite src/services/quality-check-service.ts, src/services/quality-report-service.ts, src/types/quality-{check,report}.ts, and src/data-platform/services/catalog-service.ts as already-built and tested. None of those files has ever existed in this repo's git history, on any branch — and all three docs have since been corrected to say so explicitly (verified this pass), rather than repeating the original claim. So: the narrow Python-side confidence filter is real; the broader TS/lakehouse "quality report" this Accept criterion likely also intends is not built (see docs/P0_DATASET_QUALITY_FLOW.md for the honest target spec). Whether pipeline.py's avg_confidence output satisfies "quality report generated" is a judgment call for whoever runs this phase — flagging rather than deciding it here.
  • Whether an actual Nordic dataset has been produced end-to-end for a real order was not verified this pass (would require running the stack).

Phase 6 — Delivery surfaces & integrations

• Exports: CSV / Parquet / JSON via R2 presigned URLs.
• Embedded NocoDB ("Min data") and Metabase dashboards via signed URLs in mono chrome.
• Integrations: FiveCRM push, webhooks, Google Sheets.
• File uploads (Uppy → R2) for matching/enrichment.
• Accept: a delivered dataset is downloadable, viewable in embedded NocoDB + Metabase (no separate login), and pushable to FiveCRM; uploads work.
• Status: Not done — this phase is UI-shell-only today.
  • CSV/Parquet/JSON exports via R2 presigned URLs: not found. app/api/uploads/route.ts (the closest R2-touching code) is a 501 stub with a hardcoded put_url: null — no presign logic exists yet, for uploads or exports.
  • components/embed.tsx (EmbeddedView) is real — a sandboxed iframe shell — but the server-side signed-URL generator for Metabase/NocoDB (METABASE_SECRET_KEY/NOCODB_API_TOKEN) that would feed it a real src was not found anywhere in this pass.
  • "FiveCRM" appears only as a destination label string in lib/data.ts, lib/services.ts, app/portal/integrationer/page.tsx (explicitly described in that file's own comment as "presentationella toggles" — i.e. presentational, not wired), and as one of several regex-matched destination names in app/api/ai/parse/route.ts's heuristic. No actual FiveCRM API push exists.
  • uppy is not a dependency in package.json (checked directly) — the Uppy → R2 upload flow this bullet describes has no corresponding package installed, let alone implemented.

Phase 7 — Subscriptions / feeds

• subscriptions; recurring jobs (weekly/daily) delivering deltas; webhooks on change.
• Freshness monitoring; Resend emails (confirmation, "din data är klar", feedback).
• Accept: an order converts to a feed; a scheduled run produces a delta + fires a webhook + email.
• Status: Partial — data layer real, scheduling and email are not.
  • lib/subscriptions-store.ts is real CRUD against the subscriptions table (with an in-memory fallback when no DB is configured) — createSubscription/listSubscriptions genuinely insert/read.
  • No recurring-job scheduler (cron, queue consumer producing deltas, etc.) was found on the TS side of this repo. services/api/workers/queue.py exists on the Python side but this pass did not trace whether it implements actual delta-producing scheduled runs — flagged as unverified rather than assumed either way.
  • Resend is not wired at all — grepped for resend/Resend across the whole repo, zero matches outside RESEND_API_KEY being listed as an env var. No confirmation/"din data är klar"/feedback email has been implemented.

Phase 8 — Admin, hardening, compliance

• Admin: order queue + scope/quote + escrow release; source registry & freshness monitoring; worker/job monitoring; customer + catalog management.
• Customer DB-access grant flow (encrypted, read-only, time-boxed) + revoke.
• Security pass: RLS, presigned-only files, signed embeds, secret rotation, audit_log everywhere; DPA published; legal pages.
• Finalize all loading/empty/error states + mobile across the app.
• Accept: admin can run the business; DB-grant flow works and self-revokes; security checklist passes; DoD met app-wide.
• Status: Partial — UI shell real, the security substance is not.
  • Admin pages exist: app/admin/{page,jobs/page,sources/page,customers/page,verktyg/page}.tsx. This pass did not verify whether they read real backend data or placeholders.
  • sql/001_init.sql defines audit_log, db_grants, and payments tables — schema is ready.
  • The DB-access grant flow is a stub, not a working feature. app/api/connections/db-grant/route.ts returns 501 without GRANT_ENC_KEY, and with it set returns a hardcoded 201 { grant_id, status: "pending" } — no AES-GCM encryption, no read-only validation, and no self-revoke logic exist in code today, despite the route's own comment describing that target behavior. See docs/08-infrastructure.md's "Security & compliance" section.
  • RLS enforcement, presigned-only files (blocked on the Phase 6 R2 gap above), signed embeds (blocked on the Phase 6 NocoDB/Metabase gap above), and a published DPA were not found as implemented in this pass. Treat the "security checklist passes" accept criterion as not met.

Phase 9 — Launch

• Domain 0pt.app (Vercel) + subdomains (api/data/dash via Caddy). Sentry + uptime live.
• Soft launch: Protosell internally + a handful of existing dealer relationships.
• Accept: a real external customer completes an order start→delivery in production; feedback loop running.
• Status: Domain is live; soft-launch/customer criterion not verified.
  • Verified this pass (network check performed): https://0pt.app returns HTTP 200 and serves this repo's own Next.js app (title "0pt — your data", SIAX branding); GET /api/health returns {"ok":true,"service":"0pt-web","db":"not_configured","queue":"not_configured",...}, an exact shape match for app/api/health/route.ts. README's "Live: https://0pt.app" is confirmed, and the response confirms the live deployment has no DATABASE_URL/REDIS_URL set — see docs/08-infrastructure.md's "Hosting topology" section. This does not confirm which git remote/commit is deployed (see the diverged-remotes note elsewhere in this doc), or that api/data/dash subdomains or Caddy are live — only the root Vercel deployment was checked.
  • Sentry is not wired — no sentry package in package.json, no Sentry config file anywhere in the repo (grep-confirmed). @vercel/analytics is genuinely wired (app/layout.tsx renders <Analytics />), which is a real but much smaller piece of the "Sentry + uptime live" bullet.
  • No evidence either way on the soft-launch/external-customer criterion — out of this doc's ability to verify from code alone.

---

Working conventions (for Claude Code)

• Tech docs + identifiers English; UI copy Swedish.
• Conventional Commits; one concern per PR; keep diffs reviewable.
• No secrets in code; update .env.example when adding env. (.env.example itself currently lives on feature/0pt-env-example / PR #3 — merge that first, or this instruction has nothing to update yet.)
• Add/extend tests for engine logic (estimate math, fusion/confidence, dedup) and API contract.
• When a source's access/licence is unclear, verify and record in sources.yaml before relying; if disallowed for bulk, route to licensed/partner or mark level 0 — never bypass, never fabricate.
• Touch the design tokens/motion only via docs/05; keep red ≤ once per screen; no photography.
• If anything here conflicts with existing code, flag it in the PR description and propose the reconciliation rather than guessing.
• Two git remotes claim to be this repo (github.com/sax3l/0pt and git.siax.io/sax3l/0pt), and they have independently diverged (see CLAUDE.md, README.md). Don't assume the checkout you're in is the only copy, and don't silently pick one as canonical — that's Simon's decision, flag it if it becomes relevant to the phase you're working on.