# 02 · Architecture

> **Known issues affecting this doc, in brief** — full detail + evidence in [`README.md`](../README.md)'s
> "Known issues" section: (1) no auth/session system exists yet, so several newer routes trust
> client-supplied `tenant_id` (see "Known gaps" below); (2) the CL0UD/N0D/AUD0/ST0RE platform clients are
> real code but mock-only in this repo; (3) `docs/ICEBERG_WRITE_BOUNDARY.md` / `docs/11-lakehouse-write-plan.md`
> / `docs/P0_QUERY_SOURCE_FLOW.md` reference several `src/services/quality-*` and `sql/002`/`sql/003` files
> that were never actually committed to this repo (out of this doc's file scope to fix — flagged here so it
> isn't silently repeated); (4) two git remotes (`github.com/sax3l/0pt` and `git.siax.io/sax3l/0pt`) have
> diverged and it is unresolved which is authoritative — Simon's call, not assumed here.

## Principle: one database, many lenses

Postgres is the source of truth. Everything else is a **lens** on it. A tool enters the stack only if it (a) reads that Postgres and (b) sits behind the same auth. This is the difference between an app and a pile of separate logins.

This principle describes — and is largely realized by — the **original edge + engine plane**: marketing, catalog, order lifecycle, and the FastAPI engine. It does **not** describe the newer **P0 data-platform surface** (`app/api/0pt/*`), which is real, additive, and deliberately does not go through this same path. See "Two API surfaces" below.

```
                 ┌──────────────────────────────────────────────┐
   COLLECTION    │                  POSTGRES                     │   LENSES (customer + admin)
  (legal engine) │  dim_company · dim_contact · dim_vehicle      │
 ─────────────── │  fact_financials · dim_ubo                    │ ─────────────────────────────
  Official APIs  │  dim_source · bridge_provenance               │   Next.js portal (Vercel)
  /bulk  ───────▶│  orders · order_items · jobs · deliveries     │◀──  Catalog + order lifecycle
  Authorized web │  subscriptions · users · orgs · files         │   NocoDB  (embedded "Min data")
  Licensed data  │  db_grants · audit_log · pgvector embeddings  │   Metabase (embedded dashboards)
  Customer upload│                                               │   Export: CSV/Parquet/JSON/API/FiveCRM
                 └───────────────────▲──────────────────────────┘
                                     │
                        services/api — FastAPI orchestrator + workers
              (estimates, orders, jobs, AI-assist, webhooks, pipeline)
```

## Components & responsibilities

- **Next.js (Vercel)** — all customer/admin UI: marketing site, catalog, configurator, checkout, production tracker, validation, portal, admin. It also hosts **two distinct families** of API route handlers under `app/api/`: the original order/catalog/estimate surface (some of which defer to the FastAPI engine), and the newer P0 data-platform surface under `app/api/0pt/*` (does not defer to the engine at all — see below).
- **FastAPI orchestrator** (`services/api/orchestrator/main.py`) — REST contract in `docs/04-api.md`: estimates, orders, sources, vehicle lookup, sanctions screening, job run/status. **Not on Vercel** (serverless timeouts kill long jobs). *Correction: earlier drafts of this doc pointed at `src/orchestrator`; that was the pre-migration location. `docs/09-build-plan.md` Phase 0 called for moving it into `services/api`, and that move has happened — the code lives at `services/api/orchestrator` today.*
- **Workers** (`services/api/workers/`) — adaptive router (`adaptive_router.py`, verified: chooses only among compliant methods — official API → bulk → authorized web → licensed, no bypass/stealth/proxy-rotation tier exists in it, by design) + pipeline stages (`pipeline.py`, `queue.py`, `resolve.py`, `run.py`). See `docs/06-data-engine.md`.
- **Postgres** — backbone (`docs/03-data-model.md`) incl. `pgvector` for matching/embeddings, for the original edge + engine plane. Wired via `lib/db.ts` (env-guarded: `DATABASE_URL`/`POSTGRES_URL`; falls back to an in-memory, non-durable store when unset, e.g. `lib/orders-store.ts`).
- **NocoDB / Metabase** — both are defined as services in `infra/docker-compose.yml`; embedding them into the portal as "Min data" / customer dashboards is the target, not yet confirmed wired into a portal page in this pass.
- **Cloudflare R2** — file storage (uploads + deliverables) via presigned URLs. Env-guarded (`app/api/uploads/route.ts` checks `R2_ACCESS_KEY_ID`/`R2_BUCKET`/`R2_ENDPOINT` and returns `501` until set); the presign call itself is still a stub (`put_url: null`) even when the env vars are present — not wired end-to-end yet.
- **Stripe** — deposit capture + escrow balance release. This one **is** wired: `app/api/payments/[order_id]/deposit/route.ts` and `.../release/route.ts` are real Stripe Checkout calls (env-guarded on `STRIPE_SECRET_KEY`, `501` until configured). See the correction in `docs/01-product.md`'s business model section for exactly which escrow mechanism is implemented.
- **Resend** — documented as the transactional-email provider (`docs/08-infrastructure.md`) but has no call site anywhere in this repo yet. Treat "+ email" as not-yet-implemented, not merely unconfigured.
- **Auth.js (NextAuth) with a Postgres adapter** — this is the **target**, listed here as a stack decision, not a built component. What exists today is a lightweight signed-session scaffold (`lib/auth.ts`: HMAC-SHA256 via Web Crypto, no external deps) behind a guarded dev-only login gate (`DEV_LOGIN_KEY`). There is **no** real session-to-org/tenant binding anywhere in this repo. Full Auth.js + OAuth is Phase 3 of `docs/09-build-plan.md` and has not been built. This matters directly for the tenant-scoping gap described below.

## Two API surfaces

This repo currently contains **two real, distinct API surfaces**, not one. Earlier versions of this doc described the edge as a uniform thin proxy to the FastAPI engine — that was true for part of the original surface and remains true for that part, but a second, newer surface was added that is a deliberate, ADR-documented exception to it. Conflating the two is the single biggest way to misread this codebase's architecture.

### 1. The original edge → engine surface

A subset of the original `app/api/*` route handlers defer to the FastAPI engine (`services/api`) via `lib/engine.ts`'s `engineFetch()`/`hasEngine()` bridge when `ENGINE_URL` is set, and fall back to local computation when it isn't:

- `app/api/estimates/route.ts` — defers to `POST {ENGINE_URL}/estimates`, falls back to `lib/data.ts`'s `computeEstimate`.
- `app/api/vehicles/route.ts`, `app/api/screen/route.ts`, `app/api/ai/parse/route.ts` — same pattern.

Other original routes are **not** engine-backed at all today: `app/api/orders/*` reads/writes `lib/orders-store.ts` directly (Postgres when configured, else in-memory); `app/api/catalog/route.ts` returns a hardcoded static array. Both of these are entirely edge-local — there is no engine call in either path currently, not "thin proxy with a local fallback."

This surface is documented in full in `docs/04-api.md` and matches the "one Postgres, many lenses" principle above (once `DATABASE_URL` is set).

### 2. The P0 data-platform surface (`app/api/0pt/*`)

A second, newer surface implements dataset registration, Iceberg lakehouse table mapping/writes, per-source rate-policy management, bounded synchronous source querying, and mailbox-based document capture — **directly in the edge app**, backed by `src/services/*` and `src/clients/*`. It does **not** call `lib/engine.ts`/`ENGINE_URL` and does **not** go through the FastAPI engine at all. This is deliberate: it is the concrete implementation of **ADR-0030** (Apache Iceberg as SIAX's canonical lakehouse table format, authored in the `n0d` repo) and the ownership split it defines. See `docs/10-lakehouse-decision.md` (the pointer to the ecosystem decision), `docs/11-lakehouse-write-plan.md` (the plan), and `docs/ICEBERG_WRITE_BOUNDARY.md` (the implementation record) for the full boundary — this doc only summarizes it.

Real, existing routes today (verified against the filesystem, not against `app/api/0pt/route.ts`'s own advertised list — see the note below):

| Route | Backed by |
|---|---|
| `GET /api/0pt` | static index (see note below) |
| `POST/GET /api/0pt/documents/capture` | `src/services/document-capture-service.ts` |
| `.../documents/{documentId}/extraction` | `src/services/document-extraction-service.ts` |
| `POST/GET /api/0pt/lakehouse/mappings` | `src/services/dataset-iceberg-mapping-service.ts`, `iceberg-write-service.ts` |
| `.../lakehouse/mappings/{mappingId}` | same |
| `POST .../lakehouse/mappings/{mappingId}/writes` | `src/services/iceberg-write-service.ts#requestIcebergWrite` |
| `POST/GET /api/0pt/sources/rate-policies` | `src/services/source-rate-policy-service.ts` |
| `.../sources/rate-policies/{sourceRatePolicyId}` | same |
| `POST /api/0pt/sources/{sourceId}/query` | `src/services/source-query-service.ts` (bounded, timeout-guarded ad-hoc SQL against a registered `database` source — see `docs/P0_QUERY_SOURCE_FLOW.md`) |

**Code-level inconsistency worth knowing about**: `app/api/0pt/route.ts`'s own `GET` handler advertises an `endpoints` list including `/api/0pt/sources`, `/api/0pt/datasets`, `/api/0pt/data-contracts`, `/api/0pt/quality/checks`, `/api/0pt/quality/reports`, `/api/0pt/features/views` — **none of these exist as routes** (confirmed by listing `app/api/0pt/` on disk); the lakehouse-mapping and rate-policy routes that *do* exist aren't listed there either. This is the same "described but never built" pattern as the `quality-check-service.ts` fabrication flagged below — this doc's route table above reflects what is actually on disk, not what the index endpoint claims.

**In-memory scope**: every P0 service (`DatasetService`, `SourceService`, `DatasetIcebergMappingService`, `SourceRatePolicyService`, `DocumentCaptureService`, …) holds its state in a plain `Map` inside the Next.js route module that instantiates it. Most route modules construct a fresh instance per module, so a record created via one route is not visible to a different route module in the same running process (each file's own header comment documents this; `document-capture-service.ts` worked around it for its own two routes with a shared singleton export). A Postgres migration exists (`sql/004_iceberg_write_boundary.sql`) but is not wired into any route handler yet. So the "one database, many lenses" principle above does not yet hold for this surface.

## Six-surface capability model & 0pt's current coverage

SIAX Cloud's target model (defined in `cl0ud`, `docs/architecture/SIAX_CLOUD_UNIFIED_ARCHITECTURE.md`) is that one capability can expose up to **six surfaces**: REST **api**, **cli** (`siax 0pt ...`), **mcp** (an agent tool), **ui** (a page/panel), **job** (an async N0D job type), and **sdk** (a generated client method). The capability registry's own `Surface` type (`cl0ud` `packages/capability-registry/src/types.ts`) currently encodes five of those six as a declarative field (`'api' | 'cli' | 'mcp' | 'ui' | 'job'`); `sdk` is generated separately, not yet a registry field.

**0pt's actual coverage today** (verified against `cl0ud packages/capability-registry/src/registry.ts`): 14 distinct `0pt.*` capabilities are registered —

`0pt.query_source`, `0pt.document_ocr`, `0pt.write_external_record`, `0pt.lakehouse_mapping_manage`, `0pt.lakehouse_write`, `0pt.source_rate_policy_manage`, `0pt.screen_sanctions`, `0pt.estimate_compute`, `0pt.vehicle_lookup`, `0pt.catalog_list`, `0pt.source_registry_list`, `0pt.order_create`, `0pt.order_payment`, `0pt.write_iceberg_table`

— and, checked entry by entry:

- **11** declare `surfaces: ['api']` only.
- **2** (`0pt.lakehouse_write`, `0pt.write_iceberg_table`) declare `surfaces: ['api', 'job']`.
- **1** (`0pt.write_external_record`) declares `surfaces: ['job']` **only** — `apiEnabled: false`, and its own comment states it has "NO dedicated HTTP route in the 0pt app yet"; it exists as a tested TS service (`src/services/external-writeback-service.ts`) invoked only via direct import today, not a live callable path.
- **Every single one** has `cliEnabled: false` and `mcpEnabled: false`. None has a `ui` surface entry either.

In plain terms: **0pt is API-only today** (plus two job-backed writes, and one job-only capability with no live route at all). If any doc — inside or outside this repo — describes a live `siax 0pt ...` CLI command or an `0pt.*` MCP tool, it is describing a target, not current reality; correct it on sight.

## Platform clients: CL0UD / N0D / AUD0 / ST0RE

`src/clients/{cl0ud,n0d,aud0,st0re,st0re-lakehouse}-client.ts` are real, typechecked TypeScript with accompanying tests (`tests/p0/*`), each following the same shape: a thin interface, an HTTP implementation written against the target service's real, committed route contract, and a deterministic in-memory mock used whenever no `baseUrl` is supplied to the constructor.

**Verified this pass**: none of the five client files, and none of their call sites in `src/services/*`, read `CL0UD_URL`, `N0D_URL`, `AUD0_URL`, or `ST0RE_URL` from `process.env` anywhere (`grep -rn "process.env" src/clients src/services` turns up nothing for any of those four names outside comments/TODOs). Every constructor call in this repo (`createCl0udClient(opts?.cl0udBaseUrl)`, etc.) is invoked with no `baseUrl` argument, so **every one of these clients runs against its in-memory mock today, in every code path that exists in this repo.** Read this as "real code, mock-only, not live-verified" — not "integrated" or "connected" — wherever platform-client status comes up.

What each client is for, concretely:

- **CL0UD** — policy evaluation (`evaluatePolicy`). `iceberg-write-service.ts` and `external-writeback-service.ts` call it before acting; the mock always returns `"allow"`.
- **N0D** — job requests (`createJob`). `document-capture-service.ts` requests document-extraction jobs; `iceberg-write-service.ts` requests heavy-write/compaction jobs. 0pt only *requests* N0D jobs — it never executes them, per `docs/ICEBERG_WRITE_BOUNDARY.md`'s "N0D executes" boundary. **No evidence supports a claim that N0D has a dedicated-node ("`t_0pt`") auth pattern already built for this**: `n0d`'s repository was searched in full (all branches, all commits, all files) and contains zero references to `t_0pt`. Treat any such claim as unconfirmed until proven otherwise.
- **AUD0** — audit/lineage events (`ingestEvent`, `ingestLineageEvent`). Every mutating P0 service emits these; the mock is a true no-op, matching AUD0's own documented stance that it has no live ingestion endpoint yet.
- **ST0RE / ST0RE-lakehouse** — artifact-metadata registration and Iceberg table/namespace/snapshot registration, respectively. `st0re-lakehouse-client.ts`'s HTTP path is written against `sax3l/st0re`'s actual shipped route contract (commit `9ea0abb`, read directly during a prior pass) — real code, but not executed against a running ST0RE in this repo's test environment either.

## Known gaps

### tenant_id is not server-bound anywhere in the P0 surface

The platform-wide rule is that `tenant_id` must be bound to an authenticated session server-side and never trusted from client input. This repo violates that rule today, and it is **not** a deliberate two-tier design — it's a direct consequence of there being no auth/session system at all yet (see the Auth.js note above; `app/api/orders/route.ts`'s own `GET` handler literally comments `// org scoping added with auth`, i.e. not yet).

Concretely: `tenant_id`/`project_id` are accepted straight from the client-supplied request body (or query string) at every P0 mutation entry point —

- `app/api/0pt/lakehouse/mappings/route.ts` (POST)
- `app/api/0pt/sources/rate-policies/route.ts` (POST)
- `app/api/0pt/sources/[sourceId]/query/route.ts` (POST)
- `app/api/0pt/documents/capture/route.ts` → `src/services/document-capture-service.ts#captureFromMailbox`

`app/api/0pt/lakehouse/mappings/[mappingId]/writes/route.ts` doesn't read `tenant_id` from its own body — but it resolves the tenant by looking up `mapping.tenant_id` from a mapping that was itself created via one of the unauthenticated routes above, so the trust boundary problem is inherited, not avoided.

Do not expose any of these routes to untrusted callers before Phase 3 of `docs/09-build-plan.md` (Auth.js + org/session binding) lands. This is Simon's call on sequencing, not something to silently patch in a docs pass.

### 0pt's data-quality/QA stage is a fabrication in sibling docs, not built

Flagged so it is not repeated: `src/services/quality-check-service.ts`, `src/services/quality-report-service.ts`, `src/types/quality-check.ts`, `src/types/quality-report.ts`, `src/data-platform/services/catalog-service.ts`, `sql/002_p0_dataset_quality.sql`, and `sql/003_*.sql` were at one point referenced in `docs/ICEBERG_WRITE_BOUNDARY.md`, `docs/11-lakehouse-write-plan.md`, and `docs/P0_QUERY_SOURCE_FLOW.md` as if already built and tested. Verified this pass: `git log --all` for every one of those paths returns zero commits, on any branch, ever — `sql/` on disk today contains only `001_init.sql` and `004_iceberg_write_boundary.sql`. Those files have never existed in this repository. 0pt's data-quality/QA enforcement is **spec/target only** — a real, open build gap, not a doc typo. **All three of the docs above have since been corrected** (verified this pass by reading each in full) — none currently asserts those files as real; each carries its own explicit "fabricated, not built" callout, and `docs/P0_DATASET_QUALITY_FLOW.md` is the honest target-spec replacement.

### Two diverged git remotes

`origin` in local checkouts of this repo is `github.com/sax3l/0pt`. A separate Gitea remote, `git.siax.io/sax3l/0pt`, shares at least part of the same history (`services/api/workers/adaptive_router.py` is byte-for-byte identical between GitHub's current `main` and a Gitea post-fix PR branch) but independently accumulated commits GitHub's `main` never received — including, at one point, a WAF-bypass/proxy-rotation/fake-identity scraper stack that directly violated the CLAUDE.md §1 sourcing rule (since removed on Gitea via a separate PR). **Which remote is authoritative, which is actually deployed to `0pt.app`, and whether the two should be kept in sync is unresolved and is Simon's decision** — do not assume `origin`/GitHub is the sole copy of this codebase, here or in any other doc.

## Stack decisions — IN

| Concern | Choice | Why |
|---|---|---|
| Frontend | Next.js App Router + TS + Tailwind + shadcn/ui | Vercel-native, matches existing skills |
| Backend/jobs | FastAPI + workers (`services/api`) | already built; long-running jobs |
| DB | Postgres (+ pgvector) | backbone; Neon free tier or on-VPS |
| Airtable layer | **NocoDB** | sits on existing tables, no duplication |
| BI | **Metabase** | customer-facing embedding, simple |
| Files | Cloudflare R2 + presigned URLs | cheapest S3-compatible |
| Payments | Stripe Checkout + webhook → Next.js API | ~1 day, no SaaS boilerplate needed |
| Auth | Auth.js + Postgres adapter | one system — **target, not yet built** (see Components above) |
| Email | Resend | best Next.js integration, free tier — **not yet wired** (no call site in this repo) |
| Quality | Great Expectations / checks **inside workers** | scores shipped with each order — **not yet built** for the P0 surface (see "Known gaps" above) |

## Stack decisions — OUT (do not add)

| Rejected | Reason |
|---|---|
| Superset | Metabase already covers BI — one tool, not two |
| DataHub / OpenMetadata | enterprise-heavy; catalog = a Postgres table in Next.js + NocoDB. Add lineage only if an enterprise customer demands it |
| Baserow / Teable | NocoDB chosen — not three Airtable clones |
| Meilisearch (v1) | Postgres full-text suffices to a few thousand rows; add later |
| Delta Sharing (v1) | unnecessary; customer pull via scoped role + exports |
| Trigger.dev | the existing queue already schedules jobs; Resend covers mail |
| FlareSolverr / stealth browsers / undetected-* / proxy-fingerprint rotation / fake-account creation | **prohibited** — see CLAUDE.md §1. Replace with compliant fetching. (The Gitea-only scraper stack described under "Two diverged git remotes" above is exactly this — removed there, never present on GitHub `main`.) |

## Hosting (cheapest coherent)

- **Vercel** (Hobby/free) — Next.js + custom domain `0pt.app`.
- **One small VPS** (e.g. Hetzner ~€5/mo) running `docker-compose` (`infra/docker-compose.yml`): FastAPI, workers, NocoDB, Metabase, (Postgres or use Neon), optional Meilisearch later. Per-service managed platforms (Railway) get expensive once you have 4–5 always-on services; a single VPS + Vercel + R2 lands ≈ <€10/mo to start.
- **Cloudflare R2** — object storage (env-guarded, presign not yet wired — see Components above).
- **Postgres** — on the VPS, or **Neon** free tier if you want it managed.

Full config, env, security, CI/CD: `docs/08-infrastructure.md`. Env var inventory: `.env.example` exists on branch/PR [`feature/0pt-env-example`](https://github.com/sax3l/0pt/pull/3) (unmerged as of this pass) — until it merges to `main`, `docs/08-infrastructure.md` is the source of truth for env var names.