0pt/docs/08-infrastructure.md

# 08 · Infrastructure

## Repo structure (monorepo)

```
0pt/
├─ CLAUDE.md
├─ docs/                      # this documentation set
├─ apps/
│  └─ web/                    # Next.js (Vercel): marketing, catalog, portal, admin
├─ services/
│  └─ api/                    # FastAPI orchestrator + workers (existing src/* migrates here)
│     ├─ orchestrator/        # main.py + routers (docs/04-api.md)
│     ├─ workers/             # adaptive_router, fetch_worker, ocr_worker, pipeline, db
│     └─ sources/             # sources.yaml + connectors (docs/06)
├─ packages/
│  └─ shared/                 # shared TS types (mirror API contract), design tokens
├─ sql/  (or services/api/alembic/)   # migrations (docs/03)
├─ infra/
│  ├─ docker-compose.yml      # VPS always-on services
│  └─ Caddyfile               # reverse proxy + TLS
└─ .github/workflows/         # CI/CD
```

## Hosting topology

- **Vercel** — `apps/web` → `0pt.app`. Free Hobby until paying customers.
- **One VPS** (Hetzner ~€5/mo) — `docker-compose`: FastAPI, workers, NocoDB, Metabase, Postgres (or Neon), Redis (if Celery), Caddy. Caddy terminates TLS and proxies `api.0pt.app`, `data.0pt.app` (NocoDB), `dash.0pt.app` (Metabase).
- **Cloudflare R2** — files.
- **Postgres** — on VPS (`pgvector/pgvector:pg16`) or **Neon** managed.

## docker-compose (infra/docker-compose.yml — sketch)

```yaml
services:
  postgres:
    image: pgvector/pgvector:pg16
    environment: { POSTGRES_USER: opt, POSTGRES_PASSWORD: ${PG_PASSWORD}, POSTGRES_DB: opt }
    volumes: ["pgdata:/var/lib/postgresql/data"]
  redis:
    image: redis:7
  api:
    build: ../services/api
    command: uvicorn orchestrator.main:app --host 0.0.0.0 --port 8000
    environment: [DATABASE_URL, R2_*, STRIPE_*, RESEND_API_KEY, INTERNAL_TOKEN]
    depends_on: [postgres, redis]
  worker:
    build: ../services/api
    command: python -m workers.run            # or celery -A workers worker
    environment: [DATABASE_URL, R2_*]
    depends_on: [postgres, redis]
  nocodb:
    image: nocodb/nocodb
    environment: { NC_DB: "pg://postgres:5432?u=opt&p=${PG_PASSWORD}&d=opt", NC_PUBLIC_URL: "https://data.0pt.app" }
    depends_on: [postgres]
  metabase:
    image: metabase/metabase
    environment: { MB_DB_TYPE: postgres, MB_DB_HOST: postgres, MB_DB_USER: opt, MB_DB_PASS: ${PG_PASSWORD}, MB_DB_DBNAME: opt }
    depends_on: [postgres]
  caddy:
    image: caddy:2
    ports: ["80:80","443:443"]
    volumes: ["./Caddyfile:/etc/caddy/Caddyfile", "caddydata:/data"]
volumes: { pgdata: {}, caddydata: {} }
```

## Environment variables (full inventory)

```
# Postgres
DATABASE_URL=postgres://opt:***@host:5432/opt
PG_PASSWORD=***
# Internal auth (Next ↔ FastAPI)
INTERNAL_TOKEN=***                 # sent as X-Internal-Token
# Auth.js
AUTH_SECRET=***
AUTH_URL=https://0pt.app
# Stripe
STRIPE_SECRET_KEY=sk_***
STRIPE_WEBHOOK_SECRET=whsec_***
NEXT_PUBLIC_STRIPE_PUBLISHABLE_KEY=pk_***
# Cloudflare R2 (S3-compatible)
R2_ACCOUNT_ID=***  R2_ACCESS_KEY_ID=***  R2_SECRET_ACCESS_KEY=***  R2_BUCKET=opt-files  R2_ENDPOINT=https://<acct>.r2.cloudflarestorage.com
# Email
RESEND_API_KEY=re_***
# Embeds (signed URLs)
METABASE_SITE_URL=https://dash.0pt.app  METABASE_SECRET_KEY=***
NOCODB_BASE_URL=https://data.0pt.app    NOCODB_API_TOKEN=***
# Encryption for db_grants
GRANT_ENC_KEY=***                  # 32-byte key, AES-GCM
# App
NEXT_PUBLIC_API_URL=https://api.0pt.app
```

Never hardcode. Vercel env for `apps/web`; VPS `.env` (git-ignored) for compose. Provide `.env.example` with empty values.

## Security & compliance (build in, not bolt on)

- **Customer DB grants**: customer creates a read-only, table-scoped, time-boxed DB role; we store the connection string **AES-GCM encrypted** (`GRANT_ENC_KEY`), validate it's read-only, and **delete after the job**. IP-allowlist; prefer a replica. Log access in `audit_log`.
- **File delivery**: short-lived **presigned R2 URLs**; never public buckets.
- **Isolation**: API filters by `org_id`; Postgres **RLS** where practical; NocoDB/Metabase exposed only via **signed embed URLs** scoped per customer/order (no separate login surfaced).
- **PII (contacts/UBO)**: lawful basis recorded per delivery; honor deletion; minimize retention.
- **DPA**: ship a Data Processing Agreement; provenance log + audit log are part of the offering.
- **Secrets**: rotate; least privilege; no secrets in client bundles (only `NEXT_PUBLIC_*`).

## Embedding NocoDB / Metabase (so it feels like one app)
- Generate a **signed embed URL** server-side (Metabase signed embedding with `METABASE_SECRET_KEY`; NocoDB shared/locked view + token) scoped to the customer's data, and render it in an iframe inside the mono chrome. The customer never sees a NocoDB/Metabase login.

## CI/CD (.github/workflows)
- **web**: lint + typecheck + build → deploy to Vercel on push to `main` (Vercel Git integration).
- **api**: lint (ruff) + tests (pytest) → build Docker image → deploy to VPS (SSH `docker compose pull && up -d`, or a small runner).
- Run DB migrations as a deploy step.

## Monitoring (cheap)
- Vercel Analytics + Speed Insights (web).
- **Sentry** free tier (web + api).
- Uptime ping on `/health`.
- Worker/job health visible in admin (reads `jobs`).