Simon AxelssonandClaude Opus 5 e848992020
CI (SIAX Cloud) / security (pull_request) Successful in 19s
CI (SIAX Cloud) / quality (pull_request) Successful in 41s
CI (SIAX Cloud) / security (push) Successful in 15s
CI (SIAX Cloud) / quality (push) Successful in 40s
chore(compliance): remove the banned WAF-bypass/evasion stack (CLAUDE.md §1)
CLAUDE.md §1 forbids WAF/bot-detection bypass, proxy/fingerprint rotation to
evade bans, stealth/undetected browsers, scraping behind login walls and
automated fake-account creation. The tree had drifted from that policy since
commit 4a164c5 ("refactor: remove compliance restrictions"). This restores the
tree to the repo's own stated policy.

Removed (services/scrapers/):
- waf_bypass.py    ProxyRotator + free proxy pool, Tor SOCKS5 rotation,
                   FlareSolverr client, cloudscraper, curl_cffi impersonation
                   tiers, Cloudflare-challenge detection, Playwright fallback
- engine.py        multi-method bypass chain + CF-block detection
- engine_v2.py     METHOD_RANKING per-source bypass table + orchestration
- run_pipeline.py  batch orchestrator driving ScraperEngine
- run_benchmark.py benchmarked bypass methods against ~32 sites
- clickhouse_pipeline.py  sink for the pipeline (scrape_log.cf_blocked)
- s3_sync.py       shipped pipeline output to MinIO
- Dockerfile, docker-compose.yml  ran `python -m scrapers.run_pipeline`
- requirements.txt pinned curl_cffi + cloudscraper
- __pycache__/engine*.pyc  compiled bytecode of both engines (importable)

Removed (services/infra/docker-compose.data-stack.yml):
- flaresolverr (ghcr.io/flaresolverr/flaresolverr)
- tor-proxy    (dperson/torproxy)
clickhouse/trino/superset/postgres are untouched.

Fixed:
- adaptive_router.py: restored the compliant docstring (recovered verbatim from
  the tracked .pyc, which predates 4a164c5). The code had no bypass tier.
- ARCHITECTURE.md: S9 and §1 described the removed code as present; updated.

Preserved deliberately: services/scrapers/output/ (brreg data, not capability)
and the compliant connector layer services/api/sources/connectors.py.

Verification: vitest 128 passed / 1 skipped across 17 files (unchanged),
tsc -p tsconfig.check.json 0 errors, next build exit 0.
src/auth/zitadel.ts, tests/auth/*, vitest.config.ts all still present.

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
2026-08-07 23:06:53 +02:00
2026-06-24 14:47:12 +00:00
2026-06-24 14:47:12 +00:00
2026-06-24 14:47:12 +00:00
2026-06-24 14:47:12 +00:00
2026-06-24 14:47:12 +00:00
2026-06-24 14:51:57 +00:00

0pt — your data

Data services on demand for the Nordics + Europe. Compose a data job, pay 50 % to start, watch production live, validate with full provenance, deliver anywhere. Owned by SIAX Technology AB. Honest accuracy is the moat: every field carries source, freshness and confidence; nothing is guessed.

Live: https://0pt.app · Contract: CLAUDE.md + docs/

Architecture (verified against code, not aspirational): docs/architecture/ARCHITECTURE.md. It documents the real API surface, the real persistence story, and the known security gaps — including several places where this README is optimistic. Read it before trusting any capability claim below. Repo identity + estate classification: siax.config.json.

Two planes (see docs/02-architecture.md)

  • Edge (Vercel, this app) — marketing, catalog, the order lifecycle, customer portal, admin portal, and the API route handlers. Defers to the data engine when ENGINE_URL is set.
  • Engine (services/api, runs on a VPS) — FastAPI orchestrator + worker pipeline doing the compliant sourcing (official API → bulk → authorized web → licensed → else level 0). Never on Vercel (long jobs). Bypass/stealth/proxy-rotation/fake-account paths are excluded by design (CLAUDE.md §1).

Routes

Area Routes
Marketing / (hero, filosofi, katalog, motorn, visualiseringar, täckning, editorial, bevis, priser, footer)
Catalog /katalog, /katalog/[id]
Sourcing map /kallor
Order lifecycle /bestall → /bestall/forskott → /order/[id] → /order/[id]/validera
Customer portal /portal, /portal/subscriptions, /portal/files, /portal/integrationer, /portal/settings
Admin portal /admin, /admin/sources, /admin/jobs, /admin/customers
API /api/{health,estimates,catalog,sources,orders,orders/[id],subscriptions,admin/metrics,uploads,auth/*,payments/*,webhooks/stripe,connections/db-grant}

Design system

Mono/editorial, light, radius 0, Inter + JetBrains Mono, single red #e40014, animated grain, scroll-reveal motion. No photography — imagery is rendered data (DataArtifact). Every fact shows a provenance square (ProvenanceBadge). Full spec: docs/05-design-system.md.

Run locally

pnpm install
pnpm dev                       # Next.js on http://localhost:3000

# data engine (separate, optional)
cd services/api && pip install -r requirements.txt && uvicorn orchestrator.main:app --reload

Deploy

  • Frontend → Vercel (project 0pt, scope Simon Axelsson's projects), domain 0pt.app.
  • Engine + DB + embeds → one VPS via infra/docker-compose.yml (Postgres+pgvector, Redis, api, worker, NocoDB, Metabase, Caddy).
  • Env inventory: .env.example.
  • Auth on /api/0pt/* is fail-closed: with AUTH_ENFORCE unset the middleware REQUIRES a verified Zitadel Bearer token (401 without one). Only the literal AUTH_ENFORCE=false disables it, and that is refused when NODE_ENV=production (the process throws at startup rather than serving anonymously).

Status

Frontend, full order lifecycle, customer + admin portals, edge API, payment/auth/embed infrastructure, the compliant data engine, schema and infra are all in place and building green. Integrations are env-guarded and activate when their keys/DB are provided:

  1. Database — Vercel → Storage → Postgres (sets POSTGRES_URL) → durable persistence (schema in sql/001_init.sql).
  2. Stripe + Resend keys → payments + email.
  3. VPS for the engine → set ENGINE_URL.
  4. AUTH_SECRET + DEV_LOGIN_KEY → login (full OAuth via Auth.js lands with the DB).

© 2026 SIAX Technology AB. Not affiliated with Protosell.

S
Description
No description provided
Readme
26 MiB
Languages
TypeScript 90.3%
Python 5.5%
JavaScript 2.1%
PLpgSQL 1.4%
Shell 0.3%
Other 0.4%