# 07 · Dependencies & repos to import Curated. Each entry: what it is, where it slots in, how to install. **Do not import on reputation/star-count** — several items in earlier brainstorming were unverified or hallucinated (see Exclusions). Verify each repo exists and is maintained before adopting. ## Frontend (Next.js on Vercel) | Package | Purpose | Install | |---|---|---| | next, react, typescript | App Router base | `npx create-next-app@latest --ts` | | tailwindcss | styling (mono tokens) | per Next setup | | shadcn/ui | base components | `npx shadcn@latest init` | | @tanstack/react-table **or** ag-grid-react | data tables / catalog grid | `npm i @tanstack/react-table` | | @uppy/core @uppy/react @uppy/dashboard @uppy/aws-s3 | file upload → R2 (presigned) | `npm i @uppy/core @uppy/react @uppy/dashboard @uppy/aws-s3` | | next-auth (Auth.js) + @auth/pg-adapter | single auth, Postgres-backed | `npm i next-auth @auth/pg-adapter` | | stripe / @stripe/stripe-js | checkout + portal | `npm i stripe @stripe/stripe-js` | | resend / react-email | transactional email | `npm i resend @react-email/components` | | lenis *(optional)* | smooth scroll (keeps all effects) | `npm i lenis` | | recharts *(only if needed)* | charts inside DataArtifact/dashboards | `npm i recharts` | > Catalog/search in v1 = Postgres full-text. Add Meilisearch later only past a few thousand rows. ## Backend / workers (Python, FastAPI — existing) | Package | Purpose | Install | |---|---|---| | fastapi, uvicorn, pydantic | API (existing) | `pip install fastapi uvicorn pydantic` | | sqlalchemy / asyncpg | Postgres access (align `db.py`) | `pip install sqlalchemy asyncpg` | | alembic | migrations | `pip install alembic` | | celery + redis **or** existing adaptive_router/queue | job queue/concurrency | `pip install celery redis` | | httpx | polite HTTP fetch (identifies itself, obeys rate limits) | `pip install httpx` | | scrapy **or** crawlee | orchestrate **authorized** fetches | `pip install scrapy` | | playwright | public JS-rendered pages, politely | `pip install playwright && playwright install chromium` | | unstructured | parse PDF/HTML/DOCX/images → elements | `pip install "unstructured[all-docs]"` | | paddleocr / pytesseract | OCR cascade | `pip install paddleocr pytesseract` (+ tesseract bin) | | spacy (+ models) / transformers | NER, entity resolution (multilingual) | `pip install spacy transformers` | | polars | fast tabular processing | `pip install polars` | | rapidfuzz | dedup / fuzzy matching | `pip install rapidfuzz` | | great-expectations | data quality checks in pipeline | `pip install great-expectations` | | dbt-postgres *(optional)* | transform layer | `pip install dbt-postgres` | | pgvector (Postgres ext) | embeddings/matching | `CREATE EXTENSION vector;` | | boto3 *(or aioboto3)* | R2 (S3-compatible) | `pip install boto3` | | stripe (python) | webhook handling | `pip install stripe` | ## Self-hosted services (docker-compose on the VPS) | Service | Purpose | Image | |---|---|---| | **NocoDB** | Airtable-style views over existing Postgres tables; embedded "Min data" | `nocodb/nocodb` | | **Metabase** | customer-facing dashboards; embedded | `metabase/metabase` | | **Postgres** *(or Neon managed)* | backbone (+ pgvector) | `pgvector/pgvector:pg16` | | **Redis** *(if Celery)* | queue | `redis` | | **Meilisearch** *(later)* | catalog search at scale | `getmeili/meilisearch` | ## External managed (free/cheap tiers) - **Vercel** — Next.js hosting. - **Cloudflare R2** — object storage (presigned URLs). - **Stripe** — payments (test mode first). - **Resend** — email. - **Neon** *(optional)* — managed Postgres. ## EXCLUSIONS — do not import - **Bypass/stealth/evasion**: FlareSolverr, "CloakBrowser", undetected-chromedriver/undetected-playwright (as bypass), proxy/fingerprint-rotation engines, any "WAF bypass" repo. Prohibited by CLAUDE.md §1 — they destroy the honest-accuracy positioning and create legal exposure. - **Fake-account automation** on Apollo/Cognism/etc. - **Unverified / likely-hallucinated** from earlier brainstorming: "CloakBrowser/CloakHQ", "Rota proxy engine", "OpenClaw". Do not adopt unless you confirm a real, maintained project — and even then, not if it falls in the bypass category. - **Overlapping tools** already decided against (Superset, DataHub/OpenMetadata, Baserow/Teable, Trigger.dev, Delta Sharing in v1) — see `docs/02-architecture.md`. > Note on real-but-overhyped tools (Firecrawl, browser-use, etc.): they exist, but ignore inflated star-counts and only adopt if they fit a *compliant* role and don't duplicate the stack.