0pt/docs/07-dependencies.md

07 · Dependencies & repos to import

Curated. Each entry: what it is, where it slots in, how to install. Do not import on reputation/star-count — several items in earlier brainstorming were unverified or hallucinated (see Exclusions). Verify each repo exists and is maintained before adopting.

This is a plan, not a status report. Every table below lists what should be imported for the target architecture in docs/06-data-engine.md. A Status column has been added to each table (verified this pass against services/api/requirements.txt and root package.json — the actual, current dependency manifests, read in full). Read the status column before assuming a package is already wired in; most of the data-engine tier (OCR, NER, quality checks, orchestration) is still not installed.

Frontend (Next.js on Vercel)

Package	Purpose	Install	Status
next, react, typescript	App Router base	npx create-next-app@latest --ts	Installed — next@16.0.10, react@19.2.0, typescript@^5 in package.json
tailwindcss	styling (mono tokens)	per Next setup	Installed — tailwindcss@^4.1.9
shadcn/ui	base components	npx shadcn@latest init	Installed via copy-in, not as a runtime dep — no shadcn package in package.json (expected: the shadcn CLI copies component source into components/ui/); the underlying @radix-ui/react-* primitives it generates against are installed individually and present
@tanstack/react-table or ag-grid-react	data tables / catalog grid	npm i @tanstack/react-table	Not installed — neither package present
@uppy/core @uppy/react @uppy/dashboard @uppy/aws-s3	file upload → R2 (presigned)	npm i @uppy/core @uppy/react @uppy/dashboard @uppy/aws-s3	Not installed
next-auth (Auth.js) + @auth/pg-adapter	single auth, Postgres-backed	npm i next-auth @auth/pg-adapter	Not installed. Matches a broader, confirmed gap: there is no auth/session system anywhere in this repo yet (app/api/orders/route.ts's own GET handler has the literal comment "org scoping added with auth" — i.e. not yet). Several app/api/0pt/* routes currently read tenant_id/project_id straight from the request body as a result — a real, open security gap, not a deliberate design; see docs/09-build-plan.md Phase 3 for the planned fix (not built anywhere yet, Simon's call on design).
stripe / @stripe/stripe-js	checkout + portal	npm i stripe @stripe/stripe-js	Partially installed — stripe@^22.2.3 (server SDK) is present; @stripe/stripe-js (client SDK) is not
resend / react-email	transactional email	npm i resend @react-email/components	Not installed
lenis (optional)	smooth scroll (keeps all effects)	npm i lenis	Not installed
recharts (only if needed)	charts inside DataArtifact/dashboards	npm i recharts	Installed — recharts@2.15.4

Catalog/search in v1 = Postgres full-text. Add Meilisearch later only past a few thousand rows.

Backend / workers (Python, FastAPI — existing)

services/api/requirements.txt currently pins exactly 9 packages. Everything else in this table is still to be added — this is the single biggest gap between this doc's plan and the running code (see docs/06-data-engine.md's "Pipeline stages" section for what that means stage-by-stage: only official-API sourcing and rapidfuzz-based dedup are real today; extraction/OCR, NER, and quality-checking have no code at all).

Package	Purpose	Install	Status
fastapi, uvicorn, pydantic	API (existing)	pip install fastapi uvicorn pydantic	Installed — fastapi==0.115.6, uvicorn[standard]==0.34.0, pydantic==2.10.4
httpx	polite HTTP fetch (identifies itself, obeys rate limits)	pip install httpx	Installed and actively used — httpx==0.28.1, every connector in sources/connectors.py and sources/financials.py goes through it
pyyaml	parse sources.yaml	pip install pyyaml	Installed and used — pyyaml==6.0.2, loaded once in orchestrator/main.py
rapidfuzz	dedup / fuzzy matching	pip install rapidfuzz	Installed and used — rapidfuzz==3.11.0, the only matching engine in workers/resolve.py
python-dotenv, tenacity	env loading, retry helper	—	Installed, python-dotenv==1.0.1 / tenacity==9.0.0 — present in requirements.txt but not yet visibly imported in the worker/orchestrator/sources files read this pass; not confirmed wired up
sqlalchemy / asyncpg	Postgres access (align db.py)	pip install sqlalchemy asyncpg	asyncpg==0.30.0 is installed but unused. No file in services/api imports it or issues a query — confirmed by grep. orchestrator/main.py reads DATABASE_URL once, only to report a boolean in /health. Orders/jobs live in in-memory dicts that reset on restart. sqlalchemy is not installed at all. There is also no db.py anywhere in this repo — an older doc reference to src/workers/db.py is stale; that path does not exist.
alembic	migrations	pip install alembic	Not installed. Migrations currently ship as plain numbered SQL files in sql/ (001_init.sql, 004_iceberg_write_boundary.sql), applied ad hoc, not via Alembic.
celery + redis or existing adaptive_router/queue	job queue/concurrency	pip install celery redis	Neither Celery nor a Python Redis client is installed. workers/queue.py is a small in-process asyncio.Queue wrapper (single-worker dev only, by its own docstring). infra/docker-compose.yml does run a redis container, but nothing in services/api currently talks to it.
scrapy or crawlee	orchestrate authorized fetches	pip install scrapy	Not installed. No HTML-scraping library of any kind (also no bs4/lxml) exists in this codebase — confirmed by grep. Tier-2 "authorized public web" sourcing (docs/06) has no code path yet; only tier-1 official-API connectors exist.
playwright	public JS-rendered pages, politely	pip install playwright && playwright install chromium	Not installed. No browser_worker.py file exists anywhere in this repo either — an older doc reference to reconciling a FlareSolverr stage inside it is moot; there's nothing to reconcile in the current Python engine (see docs/06-data-engine.md's Known Issue callout for where a bypass stack did briefly exist, on a different git remote).
unstructured	parse PDF/HTML/DOCX/images → elements	pip install "unstructured[all-docs]"	Not installed.
paddleocr / pytesseract	OCR cascade	pip install paddleocr pytesseract (+ tesseract bin)	Not installed. No OCR code or ocr_worker module exists anywhere in this repo.
spacy (+ models) / transformers	NER, entity resolution (multilingual)	pip install spacy transformers	Not installed. workers/resolve.py's entity resolution is rapidfuzz string-similarity only — no NER, no multilingual normalization.
polars	fast tabular processing	pip install polars	Not installed.
great-expectations	data quality checks in pipeline	pip install great-expectations	Not installed, and nothing equivalent exists either. The entire QA stage in workers/pipeline.py is a single confidence-threshold filter (r["confidence"] >= threshold) — no validity/completeness/freshness/uniqueness checks, no quality report emitted or stored. This is a real, currently-open gap, not a doc typo: a broader, confirmed finding this session is that 0pt's data-quality/QA layer, once described elsewhere in this doc corpus as already built and tested (src/services/quality-check-service.ts etc.), does not exist in code at all — zero commits in this repo's history, any branch, for any of those files. Every doc that used to make that claim (docs/ICEBERG_WRITE_BOUNDARY.md, docs/11-lakehouse-write-plan.md, docs/P0_QUERY_SOURCE_FLOW.md) has since been corrected. The Python engine independently confirms the same absence.
dbt-postgres (optional)	transform layer	pip install dbt-postgres	Not installed.
pgvector (Postgres ext)	embeddings/matching	CREATE EXTENSION vector;	The pgvector/pgvector:pg16 image is used in infra/docker-compose.yml, so the extension is available in the container, but no code in services/api runs a vector query — not exercised by any of the files read this pass
boto3 (or aioboto3)	R2 (S3-compatible)	pip install boto3	Not installed. No R2/S3 upload code found in services/api this pass.
stripe (python)	webhook handling	pip install stripe	Not installed in services/api/requirements.txt (the JS stripe package in root package.json is separate and unrelated to this table)

Self-hosted services (docker-compose on the VPS)

Service	Purpose	Image	Status
NocoDB	Airtable-style views over existing Postgres tables; embedded "Min data"	nocodb/nocodb	Wired in infra/docker-compose.yml (nocodb/nocodb:latest, points at the same postgres service) — not independently verified running/reachable this pass
Metabase	customer-facing dashboards; embedded	metabase/metabase	Wired in infra/docker-compose.yml (metabase/metabase:latest, same Postgres backend) — not independently verified running/reachable this pass
Postgres (or Neon managed)	backbone (+ pgvector)	pgvector/pgvector:pg16	Wired in infra/docker-compose.yml — but see the backend table above: the FastAPI service that should write to it currently does not (in-memory only)
Redis (if Celery)	queue	redis	Container is wired in infra/docker-compose.yml, but no Python code in services/api currently connects to it (no Celery, no redis client installed)
Meilisearch (later)	catalog search at scale	getmeili/meilisearch	Not in infra/docker-compose.yml yet — consistent with this doc's own "later" framing, not a gap

External managed (free/cheap tiers)

• Vercel — Next.js hosting.
• Cloudflare R2 — object storage (presigned URLs). No R2 client code found in this pass (boto3 not installed; no @uppy/aws-s3 in package.json) — treat as not yet wired regardless of account status.
• Stripe — payments (test mode first). Server SDK (stripe npm package) is installed; whether a live/test key is actually configured was not verified this pass (no secret material read).
• Resend — email. Not installed (see frontend table).
• Neon (optional) — managed Postgres. infra/docker-compose.yml runs its own Postgres container instead; no evidence either way of a Neon instance actually in use.

EXCLUSIONS — do not import

• Bypass/stealth/evasion: FlareSolverr, "CloakBrowser", undetected-chromedriver/undetected-playwright (as bypass), proxy/fingerprint-rotation engines, any "WAF bypass" repo. Prohibited by CLAUDE.md §1 — they destroy the honest-accuracy positioning and create legal exposure. This is not hypothetical: a real WAF-bypass/proxy-rotation/fake-identity scraper stack (services/scrapers/*) briefly existed on this repo's Gitea remote (git.siax.io/sax3l/0pt, three commits dated 2026-07-13) and has since been removed there via PR #1. It never touched origin (github.com/sax3l/0pt), which this doc corpus was verified against. See docs/06-data-engine.md's Known Issue callout for the full, unresolved detail — which remote is canonical is Simon's call, not decided this pass.
• Fake-account automation on Apollo/Cognism/etc.
• Unverified / likely-hallucinated from earlier brainstorming: "CloakBrowser/CloakHQ", "Rota proxy engine", "OpenClaw". Do not adopt unless you confirm a real, maintained project — and even then, not if it falls in the bypass category.
• Overlapping tools already decided against (Superset, DataHub/OpenMetadata, Baserow/Teable, Trigger.dev, Delta Sharing in v1) — see docs/02-architecture.md.

Note on real-but-overhyped tools (Firecrawl, browser-use, etc.): they exist, but ignore inflated star-counts and only adopt if they fit a compliant role and don't duplicate the stack.