feat: rebrand MONO->0pt (hero, header, layout) + add project docs/contract
Hero is now a data-native 0pt artifact (no photos, keeps grain/reveal/float); header is the 0pt wordmark with Swedish nav; layout is sv + loads JetBrains Mono + 0pt metadata; globals wires the loaded fonts. Adds CLAUDE.md + docs/01-09 as the build contract. next build green (5 routes). Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
This commit is contained in:
Claude Opus 4.8
@@ -0,0 +1,57 @@
|
||||
# CLAUDE.md — 0pt (SIAX Technology AB)
|
||||
|
||||
> Read this first. It is the contract for how this project is built. Then read `docs/` in order.
|
||||
|
||||
## What 0pt is
|
||||
|
||||
**0pt — your data.** A Nordic/European data-services platform (`0pt.app`) where customers:
|
||||
1. **Compose** a data job (markets, entity type, fields, enrichment, frequency, output) — or browse ready-made datasets.
|
||||
2. Pay a **50 % deposit** to start; the remaining 50 % is held in **escrow** and released only on approval.
|
||||
3. **Watch production live** (sourcing → extraction → structuring → enrichment → QA).
|
||||
4. **Validate** the result with full freedom (table / map / schema / quality views) and deliver to any format/destination.
|
||||
|
||||
Owned by SIAX Technology AB. Feeds internal products too (Protosell/FiveCRM, VehIQ).
|
||||
|
||||
## Non-negotiable constraints (do not violate)
|
||||
|
||||
1. **Legal sourcing only.** The data engine uses, in strict priority: official APIs/bulk → authorized scraping of clearly public pages (respect `robots.txt` + rate limits) → licensed partner data → otherwise the field is delivered as **"not available / licensed only"**. We never invent values.
|
||||
- **Do NOT build, keep, or extend**: WAF/bot-detection bypass (incl. FlareSolverr, "stealth" browsers, undetected-* used to defeat detection), fingerprint/proxy rotation to evade bans, scraping behind login walls, or automated fake-account creation (Apollo etc.).
|
||||
- If existing code (e.g. `src/workers/browser_worker.py`) contains a FlareSolverr/bypass path, **remove or disable it** and replace with compliant fetching. This is a product-positioning + compliance requirement, not a preference. "Honest accuracy" is the whole moat.
|
||||
2. **Postgres is the backbone.** One Postgres is the source of truth. Catalog, portal, NocoDB views, Metabase dashboards, workers, exports all read/write the *same* DB. A tool may only enter the stack if it (a) reads that Postgres and (b) sits behind the same auth.
|
||||
3. **Provenance per field.** Every fact carries source, freshness, confidence (`bridge_provenance`). This drives the UI signature too.
|
||||
4. **One tool per job.** Do not add overlapping tools. The approved stack is in `docs/02-architecture.md` and `docs/07-dependencies.md`. Anything not listed needs a documented reason.
|
||||
5. **GDPR-native.** DPA, lawful basis, audit log, deletion-after-job. Customer DB access is *pull by the customer* via a read-only, time-boxed role they create — never us asking for admin, never raw creds stored in plaintext.
|
||||
6. **Design is fixed.** Mono/editorial system (white `#fff`, text `#0a0a0a`, hairline `#e5e5e5`, radius 0, Inter + JetBrains Mono, single red `#e40014`). Exact tokens + motion in `docs/05-design-system.md`. No photography — imagery is rendered data (`DataArtifact`).
|
||||
|
||||
## Build on existing code
|
||||
|
||||
A working repo already exists (~145 files): FastAPI orchestrator (`src/orchestrator/main.py`, 25+ endpoints), workers (`src/workers/`: adaptive_router, browser_worker, ocr_worker, db), and country data for 20 markets. **Reconcile with it, don't blindly replace.** Where this documentation defines a target contract, migrate the existing code toward it. Flag conflicts instead of guessing.
|
||||
|
||||
## How to work
|
||||
|
||||
- Work in **dependency order** per `docs/09-build-plan.md`. Finish a phase to its acceptance criteria before the next.
|
||||
- Small, reviewable commits. Conventional Commits (`feat:`, `fix:`, `docs:`, `chore:`).
|
||||
- Frontend = Next.js App Router + TS + Tailwind + shadcn/ui on Vercel. Backend = FastAPI + workers, **not** on Vercel (long jobs).
|
||||
- Tech docs + code identifiers in **English**; all visible **UI copy in Swedish**.
|
||||
- Never hardcode secrets — use env (`docs/08-infrastructure.md`).
|
||||
- When unsure about a source's current access/licence terms, **verify before relying on it** and record it in the sourcing map; do not assume.
|
||||
|
||||
## Documentation index
|
||||
|
||||
| Doc | Purpose |
|
||||
|---|---|
|
||||
| `docs/01-product.md` | Vision, the three flows, order lifecycle, business model |
|
||||
| `docs/02-architecture.md` | System architecture, stack decisions (in/out), hosting |
|
||||
| `docs/03-data-model.md` | Postgres schema, provenance model, migrations |
|
||||
| `docs/04-api.md` | FastAPI contract (canonical endpoints) |
|
||||
| `docs/05-design-system.md` | Mono tokens, motion, visual system, signature |
|
||||
| `docs/06-data-engine.md` | Workers, legal sourcing chain, pipeline, QA, fusion, sources.yaml |
|
||||
| `docs/07-dependencies.md` | Repos/libraries to import (+ explicit exclusions) |
|
||||
| `docs/08-infrastructure.md` | Hosting, docker-compose, env, security, CI/CD |
|
||||
| `docs/09-build-plan.md` | Phased milestones with acceptance criteria |
|
||||
| `docs/design/0pt-v0-build-brief.md` | Per-component v0 prompts (design reference) |
|
||||
| `docs/design/0pt-design-coverage-and-visuals.md` | Full page/component inventory + visual system |
|
||||
|
||||
## Definition of done (every screen, before real data)
|
||||
|
||||
Mono chrome · correct scroll-reveal motion · representative placeholder content · **loading + empty + error** states · mobile layout · provenance signature where facts appear · red used ≤ once · visible focus ring · `prefers-reduced-motion` respected.
|
||||
+3
-3
@@ -75,9 +75,9 @@
|
||||
}
|
||||
|
||||
@theme inline {
|
||||
--font-sans: 'Inter', 'Inter Fallback', system-ui, sans-serif;
|
||||
--font-mono: 'JetBrains Mono', monospace;
|
||||
--font-display: 'PP Editorial New', 'Times New Roman', serif;
|
||||
--font-sans: var(--font-inter), 'Inter', system-ui, sans-serif;
|
||||
--font-mono: var(--font-jetbrains-mono), 'JetBrains Mono', ui-monospace, monospace;
|
||||
--font-display: var(--font-inter), 'Inter', system-ui, sans-serif;
|
||||
--color-background: var(--background);
|
||||
--color-foreground: var(--foreground);
|
||||
--color-card: var(--card);
|
||||
|
||||
+16
-35
@@ -1,45 +1,26 @@
|
||||
import React from "react"
|
||||
import type { Metadata } from 'next'
|
||||
import { Inter } from 'next/font/google'
|
||||
import { Analytics } from '@vercel/analytics/next'
|
||||
import './globals.css'
|
||||
import React from "react";
|
||||
import type { Metadata } from "next";
|
||||
import { Inter, JetBrains_Mono } from "next/font/google";
|
||||
import { Analytics } from "@vercel/analytics/next";
|
||||
import "./globals.css";
|
||||
|
||||
const inter = Inter({ subsets: ["latin"], variable: '--font-inter' });
|
||||
const inter = Inter({ subsets: ["latin"], variable: "--font-inter" });
|
||||
const jetbrainsMono = JetBrains_Mono({ subsets: ["latin"], variable: "--font-jetbrains-mono" });
|
||||
|
||||
export const metadata: Metadata = {
|
||||
title: 'EVASION',
|
||||
description: 'High-performance outdoor gear engineered for the modern explorer. Lightweight, durable, adventure-ready.',
|
||||
generator: 'v0.app',
|
||||
icons: {
|
||||
icon: [
|
||||
{
|
||||
url: '/icon-light-32x32.png',
|
||||
media: '(prefers-color-scheme: light)',
|
||||
},
|
||||
{
|
||||
url: '/icon-dark-32x32.png',
|
||||
media: '(prefers-color-scheme: dark)',
|
||||
},
|
||||
{
|
||||
url: '/icon.svg',
|
||||
type: 'image/svg+xml',
|
||||
},
|
||||
],
|
||||
apple: '/apple-icon.png',
|
||||
},
|
||||
}
|
||||
title: "0pt — your data",
|
||||
description:
|
||||
"Datatjänster på beställning för Norden & Europa. Härkomst per fält, GDPR-native. Sätt ihop ditt datajobb, betala 50 % för att starta, följ produktionen live och validera resultatet med full insyn.",
|
||||
generator: "0pt.app",
|
||||
};
|
||||
|
||||
export default function RootLayout({
|
||||
children,
|
||||
}: Readonly<{
|
||||
children: React.ReactNode
|
||||
}>) {
|
||||
export default function RootLayout({ children }: Readonly<{ children: React.ReactNode }>) {
|
||||
return (
|
||||
<html lang="en">
|
||||
<body className={`${inter.variable} font-sans antialiased`}>
|
||||
<html lang="sv">
|
||||
<body className={`${inter.variable} ${jetbrainsMono.variable} font-sans antialiased`}>
|
||||
{children}
|
||||
<Analytics />
|
||||
</body>
|
||||
</html>
|
||||
)
|
||||
);
|
||||
}
|
||||
|
||||
+63
-95
@@ -1,126 +1,94 @@
|
||||
"use client";
|
||||
|
||||
import { useState, useEffect } from "react";
|
||||
import Link from "next/link";
|
||||
import { Menu, X } from "lucide-react";
|
||||
import { useEffect, useState } from "react";
|
||||
|
||||
const NAV = [
|
||||
{ label: "Katalog", href: "/#katalog" },
|
||||
{ label: "Så funkar det", href: "/#motorn" },
|
||||
{ label: "Källor", href: "/#tackning" },
|
||||
{ label: "Priser", href: "/#priser" },
|
||||
];
|
||||
|
||||
function Wordmark() {
|
||||
return (
|
||||
<Link href="/" className="flex items-center gap-2">
|
||||
<span className="text-xl font-semibold tracking-[-0.02em]">0pt</span>
|
||||
<span className="inline-block h-1 w-1" style={{ backgroundColor: "var(--accent-red)" }} />
|
||||
<span className="font-mono text-[10px] uppercase tracking-[0.18em] text-muted-foreground">your data</span>
|
||||
</Link>
|
||||
);
|
||||
}
|
||||
|
||||
export function Header() {
|
||||
const [isMenuOpen, setIsMenuOpen] = useState(false);
|
||||
const [isScrolled, setIsScrolled] = useState(false);
|
||||
const [scrolled, setScrolled] = useState(false);
|
||||
const [open, setOpen] = useState(false);
|
||||
|
||||
useEffect(() => {
|
||||
const handleScroll = () => {
|
||||
setIsScrolled(window.scrollY > 50);
|
||||
};
|
||||
|
||||
window.addEventListener("scroll", handleScroll, { passive: true });
|
||||
return () => window.removeEventListener("scroll", handleScroll);
|
||||
const onScroll = () => setScrolled(window.scrollY > 40);
|
||||
window.addEventListener("scroll", onScroll, { passive: true });
|
||||
onScroll();
|
||||
return () => window.removeEventListener("scroll", onScroll);
|
||||
}, []);
|
||||
|
||||
return (
|
||||
<header
|
||||
className={`fixed top-4 left-1/2 -translate-x-1/2 z-50 w-[90%] max-w-3xl transition-all duration-300 ${isScrolled ? "bg-background/80 backdrop-blur-md rounded-full" : "bg-transparent"}`}
|
||||
<header
|
||||
className="sticky top-0 z-50 border-b border-border transition-colors"
|
||||
style={{
|
||||
boxShadow: isScrolled ? "rgba(14, 63, 126, 0.04) 0px 0px 0px 1px, rgba(42, 51, 69, 0.04) 0px 1px 1px -0.5px, rgba(42, 51, 70, 0.04) 0px 3px 3px -1.5px, rgba(42, 51, 70, 0.04) 0px 6px 6px -3px, rgba(14, 63, 126, 0.04) 0px 12px 12px -6px, rgba(14, 63, 126, 0.04) 0px 24px 24px -12px" : "none"
|
||||
backgroundColor: scrolled ? "rgba(255,255,255,0.82)" : "var(--background)",
|
||||
backdropFilter: scrolled ? "blur(8px)" : "none",
|
||||
}}
|
||||
>
|
||||
<div className="flex items-center justify-between transition-all duration-300 px-2 pl-5 py-2">
|
||||
{/* Logo */}
|
||||
<Link href="#hero" className="text-lg font-medium tracking-tight transition-colors duration-300 text-foreground">
|
||||
MONO
|
||||
</Link>
|
||||
<div className="mx-auto flex h-16 max-w-[80rem] items-center justify-between px-6">
|
||||
<Wordmark />
|
||||
|
||||
{/* Desktop Navigation */}
|
||||
<nav className="hidden items-center gap-10 md:flex">
|
||||
<Link
|
||||
href="#technology"
|
||||
className="text-sm transition-colors text-muted-foreground hover:text-foreground"
|
||||
>
|
||||
Design
|
||||
</Link>
|
||||
<Link
|
||||
href="#gallery"
|
||||
className="text-sm transition-colors text-muted-foreground hover:text-foreground"
|
||||
>
|
||||
Gallery
|
||||
</Link>
|
||||
<Link
|
||||
href="#accessories"
|
||||
className="text-sm transition-colors text-muted-foreground hover:text-foreground"
|
||||
>
|
||||
Models
|
||||
</Link>
|
||||
<Link
|
||||
href="#about"
|
||||
className="text-sm transition-colors text-muted-foreground hover:text-foreground"
|
||||
>
|
||||
About
|
||||
</Link>
|
||||
<nav className="hidden items-center gap-7 md:flex">
|
||||
{NAV.map((n) => (
|
||||
<Link
|
||||
key={n.label}
|
||||
href={n.href}
|
||||
className="font-mono text-[12px] uppercase tracking-[0.1em] text-muted-foreground transition-colors hover:text-foreground"
|
||||
>
|
||||
{n.label}
|
||||
</Link>
|
||||
))}
|
||||
</nav>
|
||||
|
||||
{/* CTA */}
|
||||
<div className="hidden items-center gap-6 md:flex">
|
||||
<div className="hidden items-center gap-5 md:flex">
|
||||
<Link href="/bestall" className="font-mono text-[12px] uppercase tracking-[0.1em] text-muted-foreground transition-colors hover:text-foreground">
|
||||
Logga in
|
||||
</Link>
|
||||
<Link
|
||||
href="#reserve"
|
||||
className="px-4 py-2 text-sm font-medium transition-all rounded-full bg-foreground text-background hover:opacity-80"
|
||||
href="/bestall"
|
||||
className="bg-foreground px-5 py-2.5 font-mono text-[12px] uppercase tracking-[0.1em] text-background transition-colors hover:bg-[var(--accent-red)]"
|
||||
>
|
||||
Contact
|
||||
Starta beställning
|
||||
</Link>
|
||||
</div>
|
||||
|
||||
{/* Mobile Menu Button */}
|
||||
<button
|
||||
type="button"
|
||||
onClick={() => setIsMenuOpen(!isMenuOpen)}
|
||||
className="transition-colors md:hidden text-foreground"
|
||||
aria-label="Toggle menu"
|
||||
aria-label="Meny"
|
||||
onClick={() => setOpen((v) => !v)}
|
||||
className="flex h-9 w-9 items-center justify-center border border-border md:hidden"
|
||||
>
|
||||
{isMenuOpen ? <X size={24} /> : <Menu size={24} />}
|
||||
<span className="font-mono text-sm">{open ? "✕" : "≡"}</span>
|
||||
</button>
|
||||
</div>
|
||||
|
||||
{/* Mobile Menu */}
|
||||
{isMenuOpen && (
|
||||
<div className="border-t border-border bg-background px-6 py-8 md:hidden rounded-b-2xl">
|
||||
<nav className="flex flex-col gap-6">
|
||||
<Link
|
||||
href="#technology"
|
||||
className="text-lg text-foreground"
|
||||
onClick={() => setIsMenuOpen(false)}
|
||||
>
|
||||
Design
|
||||
{open ? (
|
||||
<div className="border-t border-border bg-background md:hidden">
|
||||
<div className="flex flex-col gap-1 px-6 py-4">
|
||||
{NAV.map((n) => (
|
||||
<Link key={n.label} href={n.href} onClick={() => setOpen(false)} className="py-2 font-mono text-[12px] uppercase tracking-[0.1em] text-muted-foreground">
|
||||
{n.label}
|
||||
</Link>
|
||||
))}
|
||||
<Link href="/bestall" onClick={() => setOpen(false)} className="mt-2 bg-foreground px-5 py-3 text-center font-mono text-[12px] uppercase tracking-[0.1em] text-background">
|
||||
Starta beställning
|
||||
</Link>
|
||||
<Link
|
||||
href="#gallery"
|
||||
className="text-lg text-foreground"
|
||||
onClick={() => setIsMenuOpen(false)}
|
||||
>
|
||||
Gallery
|
||||
</Link>
|
||||
<Link
|
||||
href="#accessories"
|
||||
className="text-lg text-foreground"
|
||||
onClick={() => setIsMenuOpen(false)}
|
||||
>
|
||||
Models
|
||||
</Link>
|
||||
<Link
|
||||
href="#about"
|
||||
className="text-lg text-foreground"
|
||||
onClick={() => setIsMenuOpen(false)}
|
||||
>
|
||||
About
|
||||
</Link>
|
||||
<Link
|
||||
href="#reserve"
|
||||
className="mt-4 bg-foreground px-5 py-3 text-center text-sm font-medium text-background rounded-full"
|
||||
onClick={() => setIsMenuOpen(false)}
|
||||
>
|
||||
Contact
|
||||
</Link>
|
||||
</nav>
|
||||
</div>
|
||||
</div>
|
||||
)}
|
||||
) : null}
|
||||
</header>
|
||||
);
|
||||
}
|
||||
|
||||
@@ -1,208 +1,100 @@
|
||||
"use client";
|
||||
|
||||
import Image from "next/image";
|
||||
import { useEffect, useRef, useState } from "react";
|
||||
import Link from "next/link";
|
||||
import { useReveal } from "@/hooks/use-reveal";
|
||||
|
||||
const word = "MONO";
|
||||
|
||||
const sideImages = [
|
||||
{
|
||||
src: "/images/hero-side-1.png",
|
||||
alt: "Modern architecture with corten steel",
|
||||
position: "left",
|
||||
span: 1,
|
||||
},
|
||||
{
|
||||
src: "/images/hero-side-2.png",
|
||||
alt: "Aerial view of modern home",
|
||||
position: "left",
|
||||
span: 1,
|
||||
},
|
||||
{
|
||||
src: "/images/hero-side-3.png",
|
||||
alt: "Interior view with landscape",
|
||||
position: "right",
|
||||
span: 1,
|
||||
},
|
||||
{
|
||||
src: "/images/hero-side-4.png",
|
||||
alt: "Modern architecture at night",
|
||||
position: "right",
|
||||
span: 1,
|
||||
},
|
||||
const STATS = [
|
||||
["20", "marknader täckta"],
|
||||
["Per fält", "härkomst & konfidens"],
|
||||
["50/50", "start & godkänt"],
|
||||
["GDPR", "native, med DPA"],
|
||||
];
|
||||
|
||||
// Data-native hero artifact: a miniature company table that "is" the product.
|
||||
const ROWS: [string, string, string, 0 | 1 | 2 | 3][] = [
|
||||
["556016-0680", "Volvo Group", "GLEIF", 3],
|
||||
["915933149", "Equinor ASA", "Brreg", 3],
|
||||
["25052943", "Maersk A/S", "CVR", 2],
|
||||
["0112038-9", "Nokia Oyj", "PRH", 2],
|
||||
["—", "E-post (FI)", "licens", 0],
|
||||
];
|
||||
const PROV = ["var(--prov-0)", "var(--prov-1)", "var(--prov-2)", "var(--prov-3)"];
|
||||
|
||||
export function HeroSection() {
|
||||
const sectionRef = useRef<HTMLElement>(null);
|
||||
const [scrollProgress, setScrollProgress] = useState(0);
|
||||
|
||||
useEffect(() => {
|
||||
const handleScroll = () => {
|
||||
if (!sectionRef.current) return;
|
||||
|
||||
const rect = sectionRef.current.getBoundingClientRect();
|
||||
const scrollableHeight = window.innerHeight * 2;
|
||||
const scrolled = -rect.top;
|
||||
const progress = Math.max(0, Math.min(1, scrolled / scrollableHeight));
|
||||
|
||||
setScrollProgress(progress);
|
||||
};
|
||||
|
||||
window.addEventListener("scroll", handleScroll, { passive: true });
|
||||
handleScroll();
|
||||
|
||||
return () => {
|
||||
window.removeEventListener("scroll", handleScroll);
|
||||
};
|
||||
}, []);
|
||||
|
||||
// Text fades out first (0 to 0.2)
|
||||
const textOpacity = Math.max(0, 1 - (scrollProgress / 0.2));
|
||||
|
||||
// Image transforms start after text fades (0.2 to 1)
|
||||
const imageProgress = Math.max(0, Math.min(1, (scrollProgress - 0.2) / 0.8));
|
||||
|
||||
// Smooth interpolations - More balanced distribution
|
||||
const centerWidth = 100 - (imageProgress * 80); // 100% to 20% (same as each side image)
|
||||
const centerHeight = 100; // Always 100% height
|
||||
const sideWidth = imageProgress * 40; // 0% to 40% (20% per image, 2 images = 40%)
|
||||
const sideOpacity = imageProgress;
|
||||
const sideTranslateLeft = -100 + (imageProgress * 100); // -100% to 0%
|
||||
const sideTranslateRight = 100 - (imageProgress * 100); // 100% to 0%
|
||||
const borderRadius = 0; // No border radius
|
||||
const gap = imageProgress * 8; // 0px to 8px
|
||||
|
||||
// Vertical offset for side columns to move them up on mobile
|
||||
const sideTranslateY = -(imageProgress * 15); // Move up by 15% when fully expanded
|
||||
const ref = useReveal<HTMLElement>();
|
||||
|
||||
return (
|
||||
<section ref={sectionRef} className="relative bg-background">
|
||||
{/* Sticky container for scroll animation */}
|
||||
<div className="sticky top-0 h-screen overflow-hidden">
|
||||
<div className="flex h-full w-full items-center justify-center">
|
||||
{/* Bento Grid Container */}
|
||||
<div
|
||||
className="relative flex h-full w-full items-stretch justify-center"
|
||||
style={{ gap: `${gap}px` }}
|
||||
>
|
||||
|
||||
{/* Left Column */}
|
||||
<div
|
||||
className="flex h-full flex-row will-change-transform"
|
||||
style={{
|
||||
width: `${sideWidth}%`,
|
||||
gap: `${gap}px`,
|
||||
transform: `translateX(${sideTranslateLeft}%) translateY(${sideTranslateY}%)`,
|
||||
opacity: sideOpacity,
|
||||
}}
|
||||
>
|
||||
{sideImages.filter(img => img.position === "left").map((img, idx) => (
|
||||
<div
|
||||
key={idx}
|
||||
className="relative h-full overflow-hidden will-change-transform"
|
||||
style={{
|
||||
flex: img.span,
|
||||
borderRadius: `${borderRadius}px`,
|
||||
}}
|
||||
>
|
||||
<Image
|
||||
src={img.src || "/placeholder.svg"}
|
||||
alt={img.alt}
|
||||
fill
|
||||
className="object-cover"
|
||||
/>
|
||||
</div>
|
||||
))}
|
||||
</div>
|
||||
<section ref={ref} className="grain-overlay relative overflow-hidden border-b border-border bg-background">
|
||||
{/* faint oversized wordmark */}
|
||||
<div aria-hidden className="pointer-events-none absolute -right-10 top-1/2 -translate-y-1/2 select-none">
|
||||
<span className="text-[34vw] font-semibold leading-none tracking-tighter text-border/60 md:text-[22vw]">0pt</span>
|
||||
</div>
|
||||
<span aria-hidden className="animate-float pointer-events-none absolute right-[8%] top-[22%] inline-block h-3 w-3" style={{ backgroundColor: "var(--accent-red)" }} />
|
||||
|
||||
{/* Main Hero Image - Center */}
|
||||
<div
|
||||
className="relative overflow-hidden will-change-transform"
|
||||
style={{
|
||||
width: `${centerWidth}%`,
|
||||
height: `${centerHeight}%`,
|
||||
flex: "0 0 auto",
|
||||
borderRadius: `${borderRadius}px`,
|
||||
}}
|
||||
<div className="relative mx-auto grid max-w-[80rem] grid-cols-1 gap-12 px-6 py-24 lg:grid-cols-[1.1fr_0.9fr] lg:py-32">
|
||||
<div className="flex flex-col justify-center">
|
||||
<p className="reveal delay-1 font-mono text-[11px] uppercase tracking-[0.14em] text-muted-foreground">
|
||||
00 — Datatjänster på beställning · Norden & EU
|
||||
</p>
|
||||
<h1 className="reveal delay-2 mt-5 text-[clamp(2.6rem,7vw,6rem)] font-medium leading-[0.95] tracking-[-0.025em]">
|
||||
Beställ exakt den data du behöver.
|
||||
<br />
|
||||
<span style={{ color: "var(--accent-red)" }}>Validera varje fält.</span>
|
||||
</h1>
|
||||
<p className="reveal delay-3 mt-6 max-w-[54ch] text-lg text-muted-foreground">
|
||||
Sätt ihop ditt eget datajobb. Betala 50 % för att starta. Följ produktionen i realtid. Godkänn
|
||||
resultatet med full insyn i var varje uppgift kommer ifrån — och hur säker den är.
|
||||
</p>
|
||||
<div className="reveal delay-4 mt-8 flex flex-wrap gap-3">
|
||||
<Link
|
||||
href="/bestall"
|
||||
className="bg-foreground px-6 py-3.5 font-mono text-[12px] uppercase tracking-[0.1em] text-background transition-colors hover:bg-[var(--accent-red)]"
|
||||
>
|
||||
{/* Text Behind - Fades out first */}
|
||||
<div
|
||||
className="absolute inset-0 z-0 flex items-center justify-center"
|
||||
style={{ opacity: textOpacity, transform: 'translateY(-200px)' }}
|
||||
>
|
||||
<h1 className="whitespace-nowrap text-[35vw] font-bold leading-[0.8] tracking-tighter text-black">
|
||||
{word.split("").map((letter, index) => (
|
||||
<span
|
||||
key={index}
|
||||
className="inline-block animate-[slideUp_0.8s_ease-out_forwards] opacity-0"
|
||||
style={{
|
||||
animationDelay: `${index * 0.08}s`,
|
||||
transition: 'all 1.5s',
|
||||
transitionTimingFunction: 'cubic-bezier(0.86, 0, 0.07, 1)',
|
||||
}}
|
||||
>
|
||||
{letter}
|
||||
</span>
|
||||
))}
|
||||
</h1>
|
||||
Bygg en beställning
|
||||
</Link>
|
||||
<Link
|
||||
href="/order/SX-4471/validera"
|
||||
className="border border-foreground px-6 py-3.5 font-mono text-[12px] uppercase tracking-[0.1em] transition-colors hover:bg-foreground hover:text-background"
|
||||
>
|
||||
Se ett levererat dataset
|
||||
</Link>
|
||||
</div>
|
||||
|
||||
<div className="reveal delay-5 mt-12 grid grid-cols-2 gap-px border border-border bg-border sm:grid-cols-4">
|
||||
{STATS.map(([n, l]) => (
|
||||
<div key={l} className="bg-background p-4">
|
||||
<p className="text-2xl font-semibold tracking-[-0.02em]">{n}</p>
|
||||
<p className="mt-1 font-mono text-[10px] uppercase tracking-[0.08em] text-muted-foreground">{l}</p>
|
||||
</div>
|
||||
|
||||
<Image
|
||||
src="/images/hero-mono.png"
|
||||
alt="Modern architectural structure with reflection"
|
||||
fill
|
||||
className="absolute inset-0 z-10 object-cover"
|
||||
priority
|
||||
/>
|
||||
</div>
|
||||
))}
|
||||
</div>
|
||||
</div>
|
||||
|
||||
{/* Right Column */}
|
||||
<div
|
||||
className="flex h-full flex-row will-change-transform"
|
||||
style={{
|
||||
width: `${sideWidth}%`,
|
||||
gap: `${gap}px`,
|
||||
transform: `translateX(${sideTranslateRight}%) translateY(${sideTranslateY}%)`,
|
||||
opacity: sideOpacity,
|
||||
}}
|
||||
>
|
||||
{sideImages.filter(img => img.position === "right").map((img, idx) => (
|
||||
<div
|
||||
key={idx}
|
||||
className="relative h-full overflow-hidden will-change-transform"
|
||||
style={{
|
||||
flex: img.span,
|
||||
borderRadius: `${borderRadius}px`,
|
||||
}}
|
||||
>
|
||||
<Image
|
||||
src={img.src || "/placeholder.svg"}
|
||||
alt={img.alt}
|
||||
fill
|
||||
className="object-cover"
|
||||
/>
|
||||
{/* the artifact */}
|
||||
<div className="reveal-fade flex items-center">
|
||||
<div className="w-full border border-border bg-background">
|
||||
<div className="flex items-center justify-between border-b border-border px-4 py-2">
|
||||
<span className="font-mono text-[10px] uppercase tracking-[0.12em] text-muted-foreground">dim_company · exempel</span>
|
||||
<span className="font-mono text-[10px] uppercase tracking-[0.12em] text-muted-foreground">härkomst</span>
|
||||
</div>
|
||||
{ROWS.map((r, i) => (
|
||||
<div key={i} className={`reveal delay-${i + 1} flex items-center justify-between border-b border-border px-4 py-3 last:border-b-0`}>
|
||||
<div className="min-w-0">
|
||||
<p className="truncate text-sm">{r[1]}</p>
|
||||
<p className="font-mono text-[10.5px] text-muted-foreground">{r[0]}</p>
|
||||
</div>
|
||||
))}
|
||||
</div>
|
||||
|
||||
<span className="ml-3 inline-flex items-center gap-1.5 border border-border px-1.5 py-0.5 font-mono text-[10px] uppercase text-muted-foreground">
|
||||
<span
|
||||
className="inline-block h-[7px] w-[7px]"
|
||||
style={{ backgroundColor: r[3] === 0 ? "transparent" : PROV[r[3]], border: r[3] === 0 ? `1px solid ${PROV[0]}` : "none" }}
|
||||
/>
|
||||
{r[2]}
|
||||
</span>
|
||||
</div>
|
||||
))}
|
||||
</div>
|
||||
</div>
|
||||
</div>
|
||||
|
||||
{/* Tagline Section - Fixed at bottom */}
|
||||
<div
|
||||
className="pointer-events-none fixed bottom-0 left-0 right-0 z-10 px-6 pb-12 md:px-12 md:pb-16 lg:px-20 lg:pb-20"
|
||||
style={{ opacity: textOpacity }}
|
||||
>
|
||||
<p className="mx-auto max-w-2xl text-center text-2xl leading-relaxed text-white md:text-3xl lg:text-[2.5rem] lg:leading-snug">
|
||||
Lightweight, durable
|
||||
<br />
|
||||
and adventure-ready.
|
||||
</p>
|
||||
</div>
|
||||
|
||||
{/* Scroll space to enable animation */}
|
||||
<div className="h-[200vh]" />
|
||||
</section>
|
||||
);
|
||||
}
|
||||
|
||||
@@ -0,0 +1,49 @@
|
||||
# 01 · Product
|
||||
|
||||
## Vision
|
||||
|
||||
0pt is the most flexible, honest data-services platform for the Nordic + European B2B market. Customers order any kind of data work — from ready-made datasets to bespoke pipelines — and get full visibility into where every value came from and how confident it is. The differentiator is **honest accuracy**: provenance per field, freshness tracking, source transparency. Incumbents (Apollo, Cognism, ZoomInfo) overclaim; 0pt shows its work.
|
||||
|
||||
## The three flows (everything reduces to these)
|
||||
|
||||
Every service maps to one of three intake/delivery flows. The nine "categories" (scraping, structuring, datasets, ICP/leads, BI, ML, documents, subscriptions, custom) are **tags/filters within these flows**, not separate machinery.
|
||||
|
||||
1. **Catalog product** — ready-made, browse → sample → buy → deliver. Exists in Postgres or produced fast from the pipeline. Margin + scale.
|
||||
2. **Custom job** — does not exist yet; must be scoped. `configure (AI-assist) → estimate → manual scope/quote confirm → produce → QA → validate → deliver`. The hero flow. Consulting in a portal.
|
||||
3. **Subscription / feed** — recurring product or job: lead feeds, freshness monitoring, ongoing enrichment. `configure → recurring payment → recurring delivery/webhook`.
|
||||
|
||||
## Order lifecycle (state machine)
|
||||
|
||||
```
|
||||
DRAFT → QUOTED → DEPOSIT_PENDING → IN_PRODUCTION → REVIEW
|
||||
→ (REVISION → IN_PRODUCTION)* → ACCEPTED → DELIVERED → [SUBSCRIPTION_ACTIVE]
|
||||
```
|
||||
|
||||
- **DRAFT**: configurator in progress.
|
||||
- **QUOTED**: estimate produced (catalog = instant; custom = after scope confirm).
|
||||
- **DEPOSIT_PENDING**: awaiting 50 % deposit.
|
||||
- **IN_PRODUCTION**: deposit captured → workers run. Customer watches live.
|
||||
- **REVIEW**: QA passed threshold → customer validates (table/map/schema/quality).
|
||||
- **REVISION**: customer requested changes → escrow untouched → back to production.
|
||||
- **ACCEPTED**: customer approves → remaining 50 % released from escrow.
|
||||
- **DELIVERED**: exports + destinations unlocked; provenance log + quality report attached.
|
||||
- **SUBSCRIPTION_ACTIVE**: if converted to a feed.
|
||||
|
||||
## Business / escrow model
|
||||
|
||||
- **50 / 50.** Deposit (50 %) captured to start production. Balance (50 %) **held** and only captured/released on **ACCEPTED**. Revision never moves the held amount.
|
||||
- Pricing models: pay-per-job, volume, subscription (monthly), custom quote.
|
||||
- Estimate is a function of: selected field weights × markets × volume × entity multiplier × confidence-target multiplier + enrichment. Custom jobs get a manual scope/quote gate before DEPOSIT_PENDING.
|
||||
- Stripe: deposit = capture now; balance = either a held authorization released on accept, or a second invoice on accept (implement as a second Checkout/invoice keyed to the order; see `docs/04-api.md`).
|
||||
|
||||
## Customer DB access (designed, not a checkbox)
|
||||
|
||||
Default: the customer **pushes** data (upload/export) — we rarely need to read their DB. If we must: a **read-only, table-scoped, time-boxed** role the customer creates, IP-allowlisted, ideally against a replica. Creds encrypted, deleted after the job. Everything logged. This is a selling point to compliance-heavy buyers, and it requires a DPA.
|
||||
|
||||
## Personas (brief — for copy + prioritization)
|
||||
|
||||
SaaS-Martin (RevOps, wants clean ICP/leads), Bilhandlar-Bengt (vehicle + company data via VehIQ), Rekryterar-Rebecka (people/firmographics), Investor-Ivar (diligence, sanctions/UBO, scrutinizes compliance), Ekonomi-Erik (financials, scrutinizes compliance), Data-David (raw datasets + API). Ivar and Erik are *why* honest sourcing + DPA must be airtight.
|
||||
|
||||
## First-launch scope (de-risked)
|
||||
|
||||
Do **not** ship "20 markets × 9 categories." Ship the thinnest sellable slice: **Nordic company data** (the open stuff — GLEIF + Brreg + verified sources) + **vehicle data** (VehIQ synergy) as a couple of catalog datasets, **plus the custom-job intake**, Stripe, and delivery to download + into FiveCRM/Data Loader. First customers are Protosell internally + existing dealer relationships — services you already deliver, productized. Then expand markets as the sourcing map fills in.
|
||||
@@ -0,0 +1,70 @@
|
||||
# 02 · Architecture
|
||||
|
||||
## Principle: one database, many lenses
|
||||
|
||||
Postgres is the source of truth. Everything else is a **lens** on it. A tool enters the stack only if it (a) reads that Postgres and (b) sits behind the same auth. This is the difference between an app and a pile of separate logins.
|
||||
|
||||
```
|
||||
┌──────────────────────────────────────────────┐
|
||||
COLLECTION │ POSTGRES │ LENSES (customer + admin)
|
||||
(legal engine) │ dim_company · dim_contact · dim_vehicle │
|
||||
─────────────── │ fact_financials · dim_ubo │ ─────────────────────────────
|
||||
Official APIs │ dim_source · bridge_provenance │ Next.js portal (Vercel)
|
||||
/bulk ───────▶│ orders · order_items · jobs · deliveries │◀── Catalog + order lifecycle
|
||||
Authorized web │ subscriptions · users · orgs · files │ NocoDB (embedded "Min data")
|
||||
Licensed data │ db_grants · audit_log · pgvector embeddings │ Metabase (embedded dashboards)
|
||||
Customer upload│ │ Export: CSV/Parquet/JSON/API/FiveCRM
|
||||
└───────────────────▲──────────────────────────┘
|
||||
│
|
||||
FastAPI orchestrator + workers
|
||||
(estimates, orders, jobs, AI-assist, webhooks, pipeline)
|
||||
```
|
||||
|
||||
## Components & responsibilities
|
||||
|
||||
- **Next.js (Vercel)** — all customer/admin UI: marketing site, catalog, configurator, checkout, production tracker, validation, portal, admin. Calls FastAPI over an internal token. Embeds NocoDB/Metabase via signed URLs (no separate login shown).
|
||||
- **FastAPI orchestrator** (`src/orchestrator`) — REST contract in `docs/04-api.md`: estimates, orders, AI-assist, uploads, deliveries, webhooks, catalog, sources, job status/SSE. **Not on Vercel** (serverless timeouts kill long jobs).
|
||||
- **Workers** (`src/workers`) — adaptive router + pipeline stages (sourcing → extraction/OCR → structuring → enrichment/fusion → QA). Write to Postgres with provenance. See `docs/06-data-engine.md`.
|
||||
- **Postgres** — backbone (`docs/03-data-model.md`) incl. `pgvector` for matching/embeddings.
|
||||
- **NocoDB** — Airtable-style views over existing tables (no data duplication). Embedded into portal as "Min data".
|
||||
- **Metabase** — customer-facing dashboards. Embedded.
|
||||
- **Cloudflare R2** — file storage (uploads + deliverables) via presigned URLs.
|
||||
- **Stripe** — deposit capture + escrow balance release.
|
||||
- **Resend** — transactional email (order confirmation, "din data är klar", feedback).
|
||||
- **Auth.js** (NextAuth) with a Postgres adapter — single auth for the whole app; NocoDB/Metabase hidden behind it.
|
||||
|
||||
## Stack decisions — IN
|
||||
|
||||
| Concern | Choice | Why |
|
||||
|---|---|---|
|
||||
| Frontend | Next.js App Router + TS + Tailwind + shadcn/ui | Vercel-native, matches existing skills |
|
||||
| Backend/jobs | FastAPI + workers (existing) | already built; long-running jobs |
|
||||
| DB | Postgres (+ pgvector) | backbone; Neon free tier or on-VPS |
|
||||
| Airtable layer | **NocoDB** | sits on existing tables, no duplication |
|
||||
| BI | **Metabase** | customer-facing embedding, simple |
|
||||
| Files | Cloudflare R2 + presigned URLs | cheapest S3-compatible |
|
||||
| Payments | Stripe Checkout + webhook → FastAPI | ~1 day, no SaaS boilerplate needed |
|
||||
| Auth | Auth.js + Postgres adapter | one system |
|
||||
| Email | Resend | best Next.js integration, free tier |
|
||||
| Quality | Great Expectations / checks **inside workers** | scores shipped with each order |
|
||||
|
||||
## Stack decisions — OUT (do not add)
|
||||
|
||||
| Rejected | Reason |
|
||||
|---|---|
|
||||
| Superset | Metabase already covers BI — one tool, not two |
|
||||
| DataHub / OpenMetadata | enterprise-heavy; catalog = a Postgres table in Next.js + NocoDB. Add lineage only if an enterprise customer demands it |
|
||||
| Baserow / Teable | NocoDB chosen — not three Airtable clones |
|
||||
| Meilisearch (v1) | Postgres full-text suffices to a few thousand rows; add later |
|
||||
| Delta Sharing (v1) | unnecessary; customer pull via scoped role + exports |
|
||||
| Trigger.dev | the existing queue already schedules jobs; Resend covers mail |
|
||||
| FlareSolverr / stealth browsers / undetected-* / proxy-fingerprint rotation / fake-account creation | **prohibited** — see CLAUDE.md §1. Replace with compliant fetching |
|
||||
|
||||
## Hosting (cheapest coherent)
|
||||
|
||||
- **Vercel** (Hobby/free) — Next.js + custom domain `0pt.app`.
|
||||
- **One small VPS** (e.g. Hetzner ~€5/mo) running `docker-compose`: FastAPI, workers, NocoDB, Metabase, (Postgres or use Neon), optional Meilisearch later. Per-service managed platforms (Railway) get expensive once you have 4–5 always-on services; a single VPS + Vercel + R2 lands ≈ <€10/mo to start.
|
||||
- **Cloudflare R2** — object storage.
|
||||
- **Postgres** — on the VPS, or **Neon** free tier if you want it managed.
|
||||
|
||||
Full config, env, security, CI/CD: `docs/08-infrastructure.md`.
|
||||
@@ -0,0 +1,201 @@
|
||||
# 03 · Data model
|
||||
|
||||
Postgres is the backbone. Two halves: **data entities** (what we sell) and **platform tables** (orders, users, jobs). Every fact links to `bridge_provenance`. Migrations via Alembic (Python) or `sql/` migrations checked into the repo — pick one and be consistent; align with existing `src/workers/db.py`.
|
||||
|
||||
## Conventions
|
||||
|
||||
- `snake_case`, UTC timestamptz, soft-delete via `deleted_at` where useful.
|
||||
- Natural keys for registries (`org_nr`), surrogate `uuid` PKs for platform tables.
|
||||
- `created_at` / `updated_at` on every table.
|
||||
- All monetary amounts in **öre** (integer) to avoid float errors; currency `SEK` default.
|
||||
|
||||
## Data entities
|
||||
|
||||
```sql
|
||||
-- A company (the core registry entity)
|
||||
CREATE TABLE dim_company (
|
||||
org_nr text PRIMARY KEY, -- national reg. number (per market format)
|
||||
market text NOT NULL, -- ISO country code: SE/NO/DK/FI/...
|
||||
lei text, -- GLEIF cross-border key
|
||||
name text NOT NULL,
|
||||
status text, -- active/dissolved/...
|
||||
nace text, -- NACE/SNI code
|
||||
registered_at date,
|
||||
address jsonb,
|
||||
website text,
|
||||
updated_at timestamptz NOT NULL DEFAULT now(),
|
||||
created_at timestamptz NOT NULL DEFAULT now()
|
||||
);
|
||||
|
||||
CREATE TABLE fact_financials (
|
||||
org_nr text REFERENCES dim_company(org_nr),
|
||||
fiscal_year int NOT NULL,
|
||||
revenue bigint, -- öre
|
||||
result bigint,
|
||||
employees int,
|
||||
source_id text REFERENCES dim_source(source_id),
|
||||
PRIMARY KEY (org_nr, fiscal_year)
|
||||
);
|
||||
|
||||
CREATE TABLE dim_contact (
|
||||
contact_id uuid PRIMARY KEY DEFAULT gen_random_uuid(),
|
||||
org_nr text REFERENCES dim_company(org_nr),
|
||||
name text,
|
||||
role text,
|
||||
email text,
|
||||
phone text,
|
||||
linkedin text,
|
||||
-- PII: subject to GDPR; lawful basis recorded per delivery
|
||||
updated_at timestamptz NOT NULL DEFAULT now()
|
||||
);
|
||||
|
||||
CREATE TABLE dim_ubo ( -- ultimate beneficial owner
|
||||
ubo_id uuid PRIMARY KEY DEFAULT gen_random_uuid(),
|
||||
org_nr text REFERENCES dim_company(org_nr),
|
||||
owner text,
|
||||
share numeric(5,2)
|
||||
);
|
||||
|
||||
CREATE TABLE dim_vehicle ( -- VehIQ synergy
|
||||
vin text PRIMARY KEY,
|
||||
market text NOT NULL,
|
||||
reg_plate text,
|
||||
make text, model text, year int,
|
||||
owner_org text REFERENCES dim_company(org_nr),
|
||||
specs jsonb,
|
||||
updated_at timestamptz NOT NULL DEFAULT now()
|
||||
);
|
||||
```
|
||||
|
||||
## Provenance model (the signature — every fact links here)
|
||||
|
||||
```sql
|
||||
CREATE TABLE dim_source (
|
||||
source_id text PRIMARY KEY, -- 'gleif','brreg','companies_house',...
|
||||
name text NOT NULL,
|
||||
market text, -- or 'global'
|
||||
access text, -- 'api'|'bulk'|'authorized_web'|'licensed'
|
||||
licence text, -- short licence note
|
||||
cadence text, -- 'daily'|'weekly'|'annual'|...
|
||||
url text
|
||||
);
|
||||
|
||||
-- One row per (entity field) we deliver, linking value → where it came from + confidence
|
||||
CREATE TABLE bridge_provenance (
|
||||
field_id uuid PRIMARY KEY DEFAULT gen_random_uuid(),
|
||||
entity_type text NOT NULL, -- 'company'|'contact'|'vehicle'|'financials'
|
||||
entity_key text NOT NULL, -- org_nr / vin / contact_id
|
||||
field_name text NOT NULL, -- 'revenue','email',...
|
||||
source_id text REFERENCES dim_source(source_id),
|
||||
freshness date, -- when this value was confirmed
|
||||
confidence numeric(4,3) NOT NULL, -- 0.000–1.000
|
||||
level smallint NOT NULL, -- 0 unavailable/licensed, 1 single, 2 partial, 3 corroborated
|
||||
agreed_sources int DEFAULT 1, -- # independent sources that agree
|
||||
UNIQUE (entity_type, entity_key, field_name)
|
||||
);
|
||||
CREATE INDEX ON bridge_provenance (entity_type, entity_key);
|
||||
```
|
||||
|
||||
> Rule: **no value ships without a `bridge_provenance` row.** Level 0 = delivered as "not available / licensed", never a guessed value.
|
||||
|
||||
## Platform tables
|
||||
|
||||
```sql
|
||||
CREATE TABLE orgs (
|
||||
org_id uuid PRIMARY KEY DEFAULT gen_random_uuid(),
|
||||
name text NOT NULL, created_at timestamptz DEFAULT now()
|
||||
);
|
||||
CREATE TABLE users (
|
||||
user_id uuid PRIMARY KEY DEFAULT gen_random_uuid(),
|
||||
org_id uuid REFERENCES orgs(org_id),
|
||||
email text UNIQUE NOT NULL,
|
||||
role text NOT NULL DEFAULT 'member', -- 'owner'|'admin'|'member'|'siax_admin'
|
||||
created_at timestamptz DEFAULT now()
|
||||
);
|
||||
|
||||
CREATE TABLE orders (
|
||||
order_id uuid PRIMARY KEY DEFAULT gen_random_uuid(),
|
||||
org_id uuid REFERENCES orgs(org_id),
|
||||
flow text NOT NULL, -- 'catalog'|'custom'|'subscription'
|
||||
status text NOT NULL DEFAULT 'draft', -- see lifecycle in docs/01
|
||||
config jsonb NOT NULL, -- configurator state (markets, fields, ...)
|
||||
total_ore bigint, deposit_ore bigint, balance_ore bigint,
|
||||
est_days int, est_confidence numeric(4,3),
|
||||
created_at timestamptz DEFAULT now(), updated_at timestamptz DEFAULT now()
|
||||
);
|
||||
|
||||
CREATE TABLE order_items ( -- resolved line items / dataset refs
|
||||
item_id uuid PRIMARY KEY DEFAULT gen_random_uuid(),
|
||||
order_id uuid REFERENCES orders(order_id),
|
||||
kind text, ref text, qty int, meta jsonb
|
||||
);
|
||||
|
||||
CREATE TABLE jobs ( -- one production run per order
|
||||
job_id uuid PRIMARY KEY DEFAULT gen_random_uuid(),
|
||||
order_id uuid REFERENCES orders(order_id),
|
||||
stage text, -- sourcing|extraction|structuring|enrichment|qa
|
||||
pct numeric(5,2) DEFAULT 0,
|
||||
state text DEFAULT 'queued', -- queued|running|done|failed
|
||||
metrics jsonb, -- rows, confidence, per-market progress
|
||||
started_at timestamptz, finished_at timestamptz
|
||||
);
|
||||
|
||||
CREATE TABLE deliveries (
|
||||
delivery_id uuid PRIMARY KEY DEFAULT gen_random_uuid(),
|
||||
order_id uuid REFERENCES orders(order_id),
|
||||
formats text[], destinations text[],
|
||||
record_count int, avg_confidence numeric(4,3),
|
||||
quality_report jsonb, provenance_export_url text,
|
||||
files jsonb, -- [{format,url,bytes}]
|
||||
created_at timestamptz DEFAULT now()
|
||||
);
|
||||
|
||||
CREATE TABLE subscriptions (
|
||||
sub_id uuid PRIMARY KEY DEFAULT gen_random_uuid(),
|
||||
order_id uuid REFERENCES orders(order_id),
|
||||
cadence text, -- weekly|daily
|
||||
webhook_url text, status text DEFAULT 'active',
|
||||
next_run_at timestamptz
|
||||
);
|
||||
|
||||
CREATE TABLE files ( -- customer uploads
|
||||
file_id uuid PRIMARY KEY DEFAULT gen_random_uuid(),
|
||||
org_id uuid REFERENCES orgs(org_id),
|
||||
order_id uuid REFERENCES orders(order_id),
|
||||
name text, mime text, bytes bigint, r2_key text,
|
||||
created_at timestamptz DEFAULT now()
|
||||
);
|
||||
|
||||
CREATE TABLE db_grants ( -- customer read-only DB access (encrypted)
|
||||
grant_id uuid PRIMARY KEY DEFAULT gen_random_uuid(),
|
||||
org_id uuid REFERENCES orgs(org_id),
|
||||
order_id uuid REFERENCES orders(order_id),
|
||||
conn_encrypted bytea NOT NULL, -- encrypted connection string
|
||||
scope text, expires_at timestamptz, revoked_at timestamptz,
|
||||
created_at timestamptz DEFAULT now()
|
||||
);
|
||||
|
||||
CREATE TABLE audit_log (
|
||||
id bigserial PRIMARY KEY,
|
||||
actor text, action text, target text, meta jsonb,
|
||||
at timestamptz DEFAULT now()
|
||||
);
|
||||
|
||||
CREATE TABLE payments (
|
||||
payment_id uuid PRIMARY KEY DEFAULT gen_random_uuid(),
|
||||
order_id uuid REFERENCES orders(order_id),
|
||||
kind text, -- 'deposit'|'balance'
|
||||
amount_ore bigint, stripe_ref text,
|
||||
status text, -- 'pending'|'captured'|'released'|'failed'
|
||||
created_at timestamptz DEFAULT now()
|
||||
);
|
||||
```
|
||||
|
||||
## pgvector
|
||||
|
||||
`CREATE EXTENSION vector;` Use for entity matching/dedup and for the AI-assist (semantic match of NL request → catalog/config). Store embeddings in a `embeddings` table keyed by `(entity_type, entity_key)` or alongside catalog rows.
|
||||
|
||||
## Row-level security & catalog
|
||||
|
||||
- Catalog products: a `catalog` table (or view) over curated datasets with sample rows, counts, coverage, price.
|
||||
- Per-customer data isolation enforced by `org_id` filtering in the API and (where exposed via NocoDB/Metabase) by scoped views/connection params, plus Postgres RLS where practical.
|
||||
@@ -0,0 +1,97 @@
|
||||
# 04 · API contract (FastAPI)
|
||||
|
||||
Canonical REST contract. Reconcile the existing 25+ endpoints in `src/orchestrator/main.py` toward this. JSON in/out, `snake_case`. All amounts in **öre**. Auth: Next.js → FastAPI over a shared internal token (`X-Internal-Token`); customer identity passed as `org_id`/`user_id` resolved from the Auth.js session server-side.
|
||||
|
||||
## Conventions
|
||||
- Errors: `{ "error": { "code", "message" } }`, proper HTTP status. Interface-voice messages (Swedish) are produced in the frontend, not here.
|
||||
- Idempotency: accept `Idempotency-Key` on POSTs that create orders/payments.
|
||||
- Pagination: `?limit=&cursor=`.
|
||||
|
||||
## Catalog
|
||||
```
|
||||
GET /catalog -> [{id,title,markets,entity_type,count,updated,price_ore,confidence}]
|
||||
GET /catalog/{id} -> full dataset (overview, coverage, schema, quality summary)
|
||||
GET /catalog/{id}/sample?n=10 -> sample rows WITH per-field provenance
|
||||
```
|
||||
|
||||
## Estimate (configurator → price/time/confidence)
|
||||
```
|
||||
POST /estimates
|
||||
body: { flow, markets[], entity_type, fields[], enrichment[], volume, frequency, confidence_target, formats[] }
|
||||
-> { total_ore, deposit_ore, balance_ore, est_days, est_confidence, breakdown[] }
|
||||
```
|
||||
|
||||
## AI-assist (NL → config)
|
||||
```
|
||||
POST /ai/parse
|
||||
body: { text } -- "5000 svenska SaaS-bolag med e-post och omsättning"
|
||||
-> { config: <estimate body>, rationale } -- prefilled configurator; uses pgvector match to catalog/fields
|
||||
```
|
||||
|
||||
## Orders (lifecycle)
|
||||
```
|
||||
POST /orders body: { flow, config } -> { order_id, status:'draft'|'quoted' }
|
||||
GET /orders -> [orders for org]
|
||||
GET /orders/{id} -> order + current job + delivery (if any)
|
||||
POST /orders/{id}/scope (custom flow) body:{ notes } -> status:'quoted' (manual quote step)
|
||||
POST /orders/{id}/accept customer approves delivery -> triggers balance release; status:'accepted'
|
||||
POST /orders/{id}/revise body:{ notes } -> status:'revision'; escrow untouched
|
||||
```
|
||||
|
||||
## Payments (50/50 escrow)
|
||||
```
|
||||
POST /payments/{order_id}/deposit -> { checkout_url } (Stripe Checkout for 50%; on success → IN_PRODUCTION)
|
||||
POST /payments/{order_id}/release -> captures/charges the held balance (called on accept)
|
||||
POST /webhooks/stripe (Stripe → here) verify signature; update payments + order status
|
||||
```
|
||||
> Implement balance as either a manual capture on a held PaymentIntent, or a second Checkout/invoice keyed to `order_id`. Never charge balance before `accept`.
|
||||
|
||||
## Jobs / live status
|
||||
```
|
||||
GET /orders/{id}/status -> { stage, pct, per_market[], confidence, eta, rows }
|
||||
GET /orders/{id}/events -> Server-Sent Events stream (preferred for the production tracker)
|
||||
```
|
||||
|
||||
## Uploads (R2 presigned)
|
||||
```
|
||||
POST /uploads body:{ order_id, name, mime, bytes } -> { file_id, put_url } (client PUTs to R2)
|
||||
GET /orders/{id}/files -> [files]
|
||||
```
|
||||
|
||||
## Customer DB access (read-only grant)
|
||||
```
|
||||
POST /connections/db-grant
|
||||
body: { order_id, conn_string, scope, expires_at } -- stored ENCRYPTED; validated read-only
|
||||
-> { grant_id, status:'pending'|'verified' }
|
||||
DELETE /connections/db-grant/{id} -> revoke
|
||||
```
|
||||
|
||||
## Deliveries
|
||||
```
|
||||
GET /orders/{id}/delivery -> { formats, destinations, record_count, avg_confidence, quality_report, files[], provenance_export_url }
|
||||
POST /orders/{id}/delivery/destinations body:{ destinations[] } -> updates targets (CSV/Parquet/JSON/API/NocoDB/Metabase/FiveCRM/Sheets/webhook)
|
||||
```
|
||||
|
||||
## Sources (the transparency moat)
|
||||
```
|
||||
GET /sources -> [{source_id,name,market,access,licence,cadence,confidence}]
|
||||
GET /sources/{market} -> sources + freshness for a market
|
||||
```
|
||||
|
||||
## Subscriptions / feeds
|
||||
```
|
||||
POST /subscriptions body:{ order_id, cadence, webhook_url } -> { sub_id }
|
||||
GET /subscriptions -> [subs]
|
||||
POST /subscriptions/{id}/pause | /resume
|
||||
```
|
||||
|
||||
## Webhooks out (to customers)
|
||||
- On delivery / feed update: POST to `subscriptions.webhook_url` with `{ order_id, delta_url, summary }`.
|
||||
|
||||
## Health
|
||||
```
|
||||
GET /health -> { ok, db, workers, queue }
|
||||
```
|
||||
|
||||
## Existing-code reconciliation
|
||||
Map current endpoints (orders, estimates, ai-assistent, catalog, webhooks, health) onto the names above; rename for consistency, keep behavior. Add: `/orders/{id}/events` (SSE), `/payments/*` escrow split, `/connections/db-grant`, `/sources`, `/subscriptions`. Remove any endpoint that drives a bypass/stealth fetch path.
|
||||
@@ -0,0 +1,87 @@
|
||||
# 05 · Design system (mono)
|
||||
|
||||
Self-contained spec. Component-level v0 prompts live in `docs/design/0pt-v0-build-brief.md`; the full page/component inventory + visual rules in `docs/design/0pt-design-coverage-and-visuals.md`. This file is the source of truth for tokens + motion; build them once and reuse.
|
||||
|
||||
Derived from the **mono / EVASION** template (itself v0-generated). Light, editorial, monochrome, sharp, with a single red accent and animated grain + scroll-reveal motion. **No photography** — imagery is rendered data.
|
||||
|
||||
## Tokens
|
||||
|
||||
```
|
||||
THEME (light, default):
|
||||
--background #ffffff --foreground #0a0a0a --muted-foreground #737373
|
||||
--border #e5e5e5 --card #ffffff --input #f5f5f5
|
||||
THEME (dark sections — footer, one editorial block):
|
||||
--background #0a0a0a --foreground #fafafa --muted-foreground #a3a3a3 --border #262626
|
||||
ACCENT (single, sparing): red --accent #e40014 (one CTA / one number / one marker per screen)
|
||||
PROVENANCE squares (ONLY non-mono color besides the red CTA):
|
||||
level 0 #737373 (unavailable/licensed) · 1 #143aff (single source) · 2 #fcbb00 (partial) · 3 #00d294 (corroborated)
|
||||
RADIUS: 0 everywhere (max 2px on tiny chips). No drop shadows; structure via 1px hairlines.
|
||||
TYPE: Inter (everything; headlines weight 500–600, letter-spacing -0.02em, large) · JetBrains Mono (eyebrows, labels, numbers, codes, table headers — UPPERCASE, letter-spacing 0.12em, ~11px, #737373)
|
||||
GRID: 8px base; graph-paper hairlines; generous whitespace; left-aligned headlines.
|
||||
SECTION SPINE: real ordered indices 00–NN as mono eyebrows.
|
||||
```
|
||||
|
||||
## globals.css (add verbatim — guarantees identical motion + grain)
|
||||
|
||||
```css
|
||||
@keyframes reveal-up{0%{opacity:0;transform:translateY(60px)}to{opacity:1;transform:translateY(0)}}
|
||||
@keyframes reveal-left{0%{opacity:0;transform:translateX(-60px)}to{opacity:1;transform:translateX(0)}}
|
||||
@keyframes reveal-right{0%{opacity:0;transform:translateX(60px)}to{opacity:1;transform:translateX(0)}}
|
||||
@keyframes scale-in{0%{opacity:0;transform:scale(.9)}to{opacity:1;transform:scale(1)}}
|
||||
@keyframes fadeIn{0%{opacity:0;transform:scale(1.02)}to{opacity:1;transform:scale(1)}}
|
||||
@keyframes slideUp{0%{opacity:0;transform:translateY(100%)}to{opacity:1;transform:translateY(0)}}
|
||||
@keyframes float{0%,100%{transform:translateY(0)}50%{transform:translateY(-20px)}}
|
||||
@keyframes grain{0%,100%{transform:translate(0,0)}10%{transform:translate(-5%,-10%)}20%{transform:translate(-15%,5%)}30%{transform:translate(7%,-25%)}40%{transform:translate(-5%,25%)}50%{transform:translate(-15%,10%)}60%{transform:translate(15%,0)}70%{transform:translateY(15%)}80%{transform:translate(3%,35%)}90%{transform:translate(-10%,10%)}}
|
||||
.reveal{opacity:0}
|
||||
.reveal.in-view{animation:.8s ease-out forwards reveal-up}
|
||||
.reveal-left.in-view{animation:.8s ease-out forwards reveal-left}
|
||||
.reveal-right.in-view{animation:.8s ease-out forwards reveal-right}
|
||||
.reveal-scale.in-view{animation:.6s ease-out forwards scale-in}
|
||||
.reveal-fade.in-view{animation:.7s ease-out forwards fadeIn}
|
||||
.delay-1{animation-delay:.1s}.delay-2{animation-delay:.2s}.delay-3{animation-delay:.3s}.delay-4{animation-delay:.4s}.delay-5{animation-delay:.5s}.delay-6{animation-delay:.6s}
|
||||
.animate-float{animation:6s ease-in-out infinite float}
|
||||
.grain-overlay::before{content:"";position:absolute;inset:-100%;opacity:.03;pointer-events:none;background-image:url("data:image/svg+xml,%3Csvg viewBox='0 0 200 200' xmlns='http://www.w3.org/2000/svg'%3E%3Cfilter id='n'%3E%3CfeTurbulence type='fractalNoise' baseFrequency='0.9' numOctaves='4' stitchTiles='stitch'/%3E%3C/filter%3E%3Crect width='100%25' height='100%25' filter='url(%23n)'/%3E%3C/svg%3E");animation:8s steps(10,end) infinite grain}
|
||||
@media(prefers-reduced-motion:reduce){.reveal,.reveal-left,.reveal-right,.reveal-scale,.reveal-fade{opacity:1;animation:none}.animate-float,.grain-overlay::before{animation:none}}
|
||||
```
|
||||
|
||||
## useReveal hook
|
||||
|
||||
```tsx
|
||||
"use client";
|
||||
import { useEffect, useRef } from "react";
|
||||
export function useReveal<T extends HTMLElement = HTMLElement>() {
|
||||
const ref = useRef<T>(null);
|
||||
useEffect(() => {
|
||||
const el = ref.current; if (!el) return;
|
||||
const io = new IntersectionObserver((es) => es.forEach((e) => {
|
||||
if (e.isIntersecting) {
|
||||
el.querySelectorAll(".reveal,.reveal-left,.reveal-right,.reveal-scale,.reveal-fade")
|
||||
.forEach((n) => n.classList.add("in-view"));
|
||||
io.unobserve(el);
|
||||
}
|
||||
}), { threshold: 0.15, rootMargin: "0px 0px -10% 0px" });
|
||||
io.observe(el); return () => io.disconnect();
|
||||
}, []);
|
||||
return ref;
|
||||
}
|
||||
```
|
||||
Every section wraps in a tag using `useReveal()`; children get `reveal[/-left/-right/-scale/-fade]` + `delay-N`.
|
||||
|
||||
## Buttons & labels
|
||||
- Primary: solid `#0a0a0a` bg, white text, square, mono uppercase 12px / spacing 0.1em, padding 14px 26px; hover → bg `#e40014`.
|
||||
- Secondary: transparent, 1px `#0a0a0a` border; hover → bg `#0a0a0a`, white text.
|
||||
- Eyebrow: mono uppercase 11px, spacing 0.14em, `#737373`, with a 2-digit index ("01 — KATALOG").
|
||||
- Headlines: sentence case, never uppercase.
|
||||
|
||||
## Visual system — no photos (`DataArtifact`)
|
||||
Build one generative primitive; every "image" surface uses it, animating with the effect it replaces:
|
||||
- `table` (rows cascade in, provenance square per fact cell) — hero/cards.
|
||||
- `map` (country-code cells, fill bars animate width on view) — coverage.
|
||||
- `schema` (ER mini-diagram, key boxes red border) — schema previews.
|
||||
- `chart` (black bars/lines, exactly ONE red series, grow on view) — dashboards.
|
||||
- `grid` (hairline graph-paper + faint oversized numeral + one floating red square) — pure texture.
|
||||
|
||||
Mapping of template image elements → replacement + effect kept is in the coverage doc. Rules: code-only (CSS/SVG/canvas), color discipline inside visuals (mono + provenance squares + one red series), grain over any "image" surface, `prefers-reduced-motion` settles to static. Optional: add **Lenis** smooth scroll on top — preserves all effects.
|
||||
|
||||
## The signature (carry everywhere)
|
||||
Provenance + confidence is the identity: the provenance square on every fact, one red, mono numerals as texture, section spine, identical motion. A screen that lacks these is off-brand.
|
||||
@@ -0,0 +1,70 @@
|
||||
# 06 · Data engine
|
||||
|
||||
The engine turns sources into delivered, provenance-tagged data. It is the legal motor — the whole product positioning depends on it. Build on existing `src/workers/` (`adaptive_router`, `browser_worker`, `ocr_worker`, `db`) but enforce the sourcing boundary below.
|
||||
|
||||
## Sourcing chain (strict priority — this is the law)
|
||||
|
||||
For each field × market, acquire in this order; stop at the first that works:
|
||||
|
||||
1. **Official API / bulk** — registries with programmatic/bulk access (highest confidence).
|
||||
2. **Authorized public web** — clearly public pages, **respecting `robots.txt` and rate limits**, no auth walls.
|
||||
3. **Licensed partner data** — contractually obtained.
|
||||
4. **Otherwise → "not available / licensed only"** — delivered honestly as level 0. **Never guessed.**
|
||||
|
||||
### Prohibited (do not build, keep, or extend)
|
||||
- WAF/bot-detection bypass — **including FlareSolverr**, "stealth"/patched browsers, undetected-* used to defeat detection.
|
||||
- Proxy/fingerprint rotation to evade bans; scraping behind login walls; automated fake-account creation (Apollo etc.).
|
||||
- **Reconciliation:** if `browser_worker.py` has an `HTTP → FlareSolverr → Playwright` chain, **drop the FlareSolverr stage** and keep only: polite HTTP fetch (identifies itself, obeys robots + rate limits) → optional Playwright for genuinely public JS-rendered pages (same politeness). The `adaptive_router` then chooses among **compliant** methods only (API > bulk > authorized fetch), not bypass tiers.
|
||||
|
||||
## Pipeline stages (per job)
|
||||
|
||||
```
|
||||
sourcing → extraction/OCR → structuring → enrichment/fusion → QA
|
||||
```
|
||||
|
||||
1. **Sourcing** — `adaptive_router` picks the compliant method per source; "source-racing" = query several *authorized* sources and take the fastest/freshest. Writes raw payloads + which source answered.
|
||||
2. **Extraction / OCR** — API JSON parsed directly; documents (PDF/HTML/images) via **Unstructured** + OCR cascade (PaddleOCR → Tesseract → LLM fallback) in `ocr_worker`. Output: candidate fields.
|
||||
3. **Structuring** — NER + entity resolution (spaCy / transformers, sv/no/da/fi/de/fr), normalize to schema, link entities across markets via **LEI** (GLEIF golden key). Dedup with **RapidFuzz**.
|
||||
4. **Enrichment / fusion** — combine fields from multiple sources; compute **multi-source agreement** → confidence + level; append optional enrichment (NACE/SNI, sanctions/PEP flag from official lists, website/tech).
|
||||
5. **QA** — quality checks (**Great Expectations** or equivalent, run inside the worker): validity (format/range), completeness, freshness, uniqueness (post-dedup), corroboration. Drop rows below the order's `confidence_target` (deliver fewer, never fabricate). Emit the quality report stored on `deliveries`.
|
||||
|
||||
Each stage updates `jobs` (`stage`, `pct`, `metrics` incl. per-market progress, running counts, climbing confidence) so the production tracker / SSE can render it live.
|
||||
|
||||
## Confidence & provenance scoring
|
||||
|
||||
- Per field, write a `bridge_provenance` row: `source_id`, `freshness`, `confidence` (0–1), `level` (0–3), `agreed_sources`.
|
||||
- Level: 0 = unavailable/licensed; 1 = single source; 2 = partial corroboration; 3 = ≥2 independent sources agree.
|
||||
- `est_confidence` on the order = aggregate target; delivered `avg_confidence` = actual.
|
||||
|
||||
## `sources.yaml` (machine-readable registry)
|
||||
|
||||
Single registry that drives the router, the `/sources` API, and the public sourcing-map page. Schema per entry:
|
||||
|
||||
```yaml
|
||||
- source_id: gleif
|
||||
name: GLEIF (LEI)
|
||||
market: global
|
||||
access: api # api | bulk | authorized_web | licensed
|
||||
cadence: daily
|
||||
licence: "open (CC0) — VERIFY current terms"
|
||||
entities: [company]
|
||||
fields: [lei, legal_name, address]
|
||||
base_url: https://api.gleif.org
|
||||
robots_ok: true # for authorized_web only; must be honored
|
||||
rate_limit: "per provider terms"
|
||||
notes: "cross-border key"
|
||||
```
|
||||
|
||||
### Starter set (VERIFY current access/licence per source before relying — terms change)
|
||||
- **GLEIF / LEI** — typically open; cross-border key. *(verify)*
|
||||
- **Brønnøysund (Brreg, NO)** — typically open API. *(verify)*
|
||||
- **Companies House (UK)** — free API (key required). *(verify)*
|
||||
- **CVR / Virk (DK)** — API access with terms. *(verify)*
|
||||
- **PRH (FI)** — varies. *(verify)*
|
||||
- **Bolagsverket (SE)** — more restricted for bulk; plan licensed/partner route. *(verify)*
|
||||
- Vehicle: **RDW (NL)** open-ish, **Traficom (FI)**, **Transportstyrelsen (SE)**, **DVLA (UK)** — *(verify each; vehicle/PII rules differ)*
|
||||
|
||||
> Do not hardcode access terms as fact. When building each connector, confirm the current API/licence/robots status and record it in `sources.yaml`. If unclear or disallowed for bulk, route to licensed/partner or mark fields level 0.
|
||||
|
||||
## Libraries used here
|
||||
Crawlee or Scrapy (orchestration of **authorized** fetches) · Playwright (public JS pages, politely) · Unstructured (document parsing) · PaddleOCR/Tesseract (OCR) · spaCy / HF transformers (NER, multilingual) · Polars (fast tabular) · RapidFuzz (dedup/matching) · Great Expectations (quality) · dbt (optional transform layer). Versions/install: `docs/07-dependencies.md`.
|
||||
@@ -0,0 +1,65 @@
|
||||
# 07 · Dependencies & repos to import
|
||||
|
||||
Curated. Each entry: what it is, where it slots in, how to install. **Do not import on reputation/star-count** — several items in earlier brainstorming were unverified or hallucinated (see Exclusions). Verify each repo exists and is maintained before adopting.
|
||||
|
||||
## Frontend (Next.js on Vercel)
|
||||
| Package | Purpose | Install |
|
||||
|---|---|---|
|
||||
| next, react, typescript | App Router base | `npx create-next-app@latest --ts` |
|
||||
| tailwindcss | styling (mono tokens) | per Next setup |
|
||||
| shadcn/ui | base components | `npx shadcn@latest init` |
|
||||
| @tanstack/react-table **or** ag-grid-react | data tables / catalog grid | `npm i @tanstack/react-table` |
|
||||
| @uppy/core @uppy/react @uppy/dashboard @uppy/aws-s3 | file upload → R2 (presigned) | `npm i @uppy/core @uppy/react @uppy/dashboard @uppy/aws-s3` |
|
||||
| next-auth (Auth.js) + @auth/pg-adapter | single auth, Postgres-backed | `npm i next-auth @auth/pg-adapter` |
|
||||
| stripe / @stripe/stripe-js | checkout + portal | `npm i stripe @stripe/stripe-js` |
|
||||
| resend / react-email | transactional email | `npm i resend @react-email/components` |
|
||||
| lenis *(optional)* | smooth scroll (keeps all effects) | `npm i lenis` |
|
||||
| recharts *(only if needed)* | charts inside DataArtifact/dashboards | `npm i recharts` |
|
||||
|
||||
> Catalog/search in v1 = Postgres full-text. Add Meilisearch later only past a few thousand rows.
|
||||
|
||||
## Backend / workers (Python, FastAPI — existing)
|
||||
| Package | Purpose | Install |
|
||||
|---|---|---|
|
||||
| fastapi, uvicorn, pydantic | API (existing) | `pip install fastapi uvicorn pydantic` |
|
||||
| sqlalchemy / asyncpg | Postgres access (align `db.py`) | `pip install sqlalchemy asyncpg` |
|
||||
| alembic | migrations | `pip install alembic` |
|
||||
| celery + redis **or** existing adaptive_router/queue | job queue/concurrency | `pip install celery redis` |
|
||||
| httpx | polite HTTP fetch (identifies itself, obeys rate limits) | `pip install httpx` |
|
||||
| scrapy **or** crawlee | orchestrate **authorized** fetches | `pip install scrapy` |
|
||||
| playwright | public JS-rendered pages, politely | `pip install playwright && playwright install chromium` |
|
||||
| unstructured | parse PDF/HTML/DOCX/images → elements | `pip install "unstructured[all-docs]"` |
|
||||
| paddleocr / pytesseract | OCR cascade | `pip install paddleocr pytesseract` (+ tesseract bin) |
|
||||
| spacy (+ models) / transformers | NER, entity resolution (multilingual) | `pip install spacy transformers` |
|
||||
| polars | fast tabular processing | `pip install polars` |
|
||||
| rapidfuzz | dedup / fuzzy matching | `pip install rapidfuzz` |
|
||||
| great-expectations | data quality checks in pipeline | `pip install great-expectations` |
|
||||
| dbt-postgres *(optional)* | transform layer | `pip install dbt-postgres` |
|
||||
| pgvector (Postgres ext) | embeddings/matching | `CREATE EXTENSION vector;` |
|
||||
| boto3 *(or aioboto3)* | R2 (S3-compatible) | `pip install boto3` |
|
||||
| stripe (python) | webhook handling | `pip install stripe` |
|
||||
|
||||
## Self-hosted services (docker-compose on the VPS)
|
||||
| Service | Purpose | Image |
|
||||
|---|---|---|
|
||||
| **NocoDB** | Airtable-style views over existing Postgres tables; embedded "Min data" | `nocodb/nocodb` |
|
||||
| **Metabase** | customer-facing dashboards; embedded | `metabase/metabase` |
|
||||
| **Postgres** *(or Neon managed)* | backbone (+ pgvector) | `pgvector/pgvector:pg16` |
|
||||
| **Redis** *(if Celery)* | queue | `redis` |
|
||||
| **Meilisearch** *(later)* | catalog search at scale | `getmeili/meilisearch` |
|
||||
|
||||
## External managed (free/cheap tiers)
|
||||
- **Vercel** — Next.js hosting.
|
||||
- **Cloudflare R2** — object storage (presigned URLs).
|
||||
- **Stripe** — payments (test mode first).
|
||||
- **Resend** — email.
|
||||
- **Neon** *(optional)* — managed Postgres.
|
||||
|
||||
## EXCLUSIONS — do not import
|
||||
|
||||
- **Bypass/stealth/evasion**: FlareSolverr, "CloakBrowser", undetected-chromedriver/undetected-playwright (as bypass), proxy/fingerprint-rotation engines, any "WAF bypass" repo. Prohibited by CLAUDE.md §1 — they destroy the honest-accuracy positioning and create legal exposure.
|
||||
- **Fake-account automation** on Apollo/Cognism/etc.
|
||||
- **Unverified / likely-hallucinated** from earlier brainstorming: "CloakBrowser/CloakHQ", "Rota proxy engine", "OpenClaw". Do not adopt unless you confirm a real, maintained project — and even then, not if it falls in the bypass category.
|
||||
- **Overlapping tools** already decided against (Superset, DataHub/OpenMetadata, Baserow/Teable, Trigger.dev, Delta Sharing in v1) — see `docs/02-architecture.md`.
|
||||
|
||||
> Note on real-but-overhyped tools (Firecrawl, browser-use, etc.): they exist, but ignore inflated star-counts and only adopt if they fit a *compliant* role and don't duplicate the stack.
|
||||
@@ -0,0 +1,118 @@
|
||||
# 08 · Infrastructure
|
||||
|
||||
## Repo structure (monorepo)
|
||||
|
||||
```
|
||||
0pt/
|
||||
├─ CLAUDE.md
|
||||
├─ docs/ # this documentation set
|
||||
├─ apps/
|
||||
│ └─ web/ # Next.js (Vercel): marketing, catalog, portal, admin
|
||||
├─ services/
|
||||
│ └─ api/ # FastAPI orchestrator + workers (existing src/* migrates here)
|
||||
│ ├─ orchestrator/ # main.py + routers (docs/04-api.md)
|
||||
│ ├─ workers/ # adaptive_router, fetch_worker, ocr_worker, pipeline, db
|
||||
│ └─ sources/ # sources.yaml + connectors (docs/06)
|
||||
├─ packages/
|
||||
│ └─ shared/ # shared TS types (mirror API contract), design tokens
|
||||
├─ sql/ (or services/api/alembic/) # migrations (docs/03)
|
||||
├─ infra/
|
||||
│ ├─ docker-compose.yml # VPS always-on services
|
||||
│ └─ Caddyfile # reverse proxy + TLS
|
||||
└─ .github/workflows/ # CI/CD
|
||||
```
|
||||
|
||||
## Hosting topology
|
||||
|
||||
- **Vercel** — `apps/web` → `0pt.app`. Free Hobby until paying customers.
|
||||
- **One VPS** (Hetzner ~€5/mo) — `docker-compose`: FastAPI, workers, NocoDB, Metabase, Postgres (or Neon), Redis (if Celery), Caddy. Caddy terminates TLS and proxies `api.0pt.app`, `data.0pt.app` (NocoDB), `dash.0pt.app` (Metabase).
|
||||
- **Cloudflare R2** — files.
|
||||
- **Postgres** — on VPS (`pgvector/pgvector:pg16`) or **Neon** managed.
|
||||
|
||||
## docker-compose (infra/docker-compose.yml — sketch)
|
||||
|
||||
```yaml
|
||||
services:
|
||||
postgres:
|
||||
image: pgvector/pgvector:pg16
|
||||
environment: { POSTGRES_USER: opt, POSTGRES_PASSWORD: ${PG_PASSWORD}, POSTGRES_DB: opt }
|
||||
volumes: ["pgdata:/var/lib/postgresql/data"]
|
||||
redis:
|
||||
image: redis:7
|
||||
api:
|
||||
build: ../services/api
|
||||
command: uvicorn orchestrator.main:app --host 0.0.0.0 --port 8000
|
||||
environment: [DATABASE_URL, R2_*, STRIPE_*, RESEND_API_KEY, INTERNAL_TOKEN]
|
||||
depends_on: [postgres, redis]
|
||||
worker:
|
||||
build: ../services/api
|
||||
command: python -m workers.run # or celery -A workers worker
|
||||
environment: [DATABASE_URL, R2_*]
|
||||
depends_on: [postgres, redis]
|
||||
nocodb:
|
||||
image: nocodb/nocodb
|
||||
environment: { NC_DB: "pg://postgres:5432?u=opt&p=${PG_PASSWORD}&d=opt", NC_PUBLIC_URL: "https://data.0pt.app" }
|
||||
depends_on: [postgres]
|
||||
metabase:
|
||||
image: metabase/metabase
|
||||
environment: { MB_DB_TYPE: postgres, MB_DB_HOST: postgres, MB_DB_USER: opt, MB_DB_PASS: ${PG_PASSWORD}, MB_DB_DBNAME: opt }
|
||||
depends_on: [postgres]
|
||||
caddy:
|
||||
image: caddy:2
|
||||
ports: ["80:80","443:443"]
|
||||
volumes: ["./Caddyfile:/etc/caddy/Caddyfile", "caddydata:/data"]
|
||||
volumes: { pgdata: {}, caddydata: {} }
|
||||
```
|
||||
|
||||
## Environment variables (full inventory)
|
||||
|
||||
```
|
||||
# Postgres
|
||||
DATABASE_URL=postgres://opt:***@host:5432/opt
|
||||
PG_PASSWORD=***
|
||||
# Internal auth (Next ↔ FastAPI)
|
||||
INTERNAL_TOKEN=*** # sent as X-Internal-Token
|
||||
# Auth.js
|
||||
AUTH_SECRET=***
|
||||
AUTH_URL=https://0pt.app
|
||||
# Stripe
|
||||
STRIPE_SECRET_KEY=sk_***
|
||||
STRIPE_WEBHOOK_SECRET=whsec_***
|
||||
NEXT_PUBLIC_STRIPE_PUBLISHABLE_KEY=pk_***
|
||||
# Cloudflare R2 (S3-compatible)
|
||||
R2_ACCOUNT_ID=*** R2_ACCESS_KEY_ID=*** R2_SECRET_ACCESS_KEY=*** R2_BUCKET=opt-files R2_ENDPOINT=https://<acct>.r2.cloudflarestorage.com
|
||||
# Email
|
||||
RESEND_API_KEY=re_***
|
||||
# Embeds (signed URLs)
|
||||
METABASE_SITE_URL=https://dash.0pt.app METABASE_SECRET_KEY=***
|
||||
NOCODB_BASE_URL=https://data.0pt.app NOCODB_API_TOKEN=***
|
||||
# Encryption for db_grants
|
||||
GRANT_ENC_KEY=*** # 32-byte key, AES-GCM
|
||||
# App
|
||||
NEXT_PUBLIC_API_URL=https://api.0pt.app
|
||||
```
|
||||
|
||||
Never hardcode. Vercel env for `apps/web`; VPS `.env` (git-ignored) for compose. Provide `.env.example` with empty values.
|
||||
|
||||
## Security & compliance (build in, not bolt on)
|
||||
|
||||
- **Customer DB grants**: customer creates a read-only, table-scoped, time-boxed DB role; we store the connection string **AES-GCM encrypted** (`GRANT_ENC_KEY`), validate it's read-only, and **delete after the job**. IP-allowlist; prefer a replica. Log access in `audit_log`.
|
||||
- **File delivery**: short-lived **presigned R2 URLs**; never public buckets.
|
||||
- **Isolation**: API filters by `org_id`; Postgres **RLS** where practical; NocoDB/Metabase exposed only via **signed embed URLs** scoped per customer/order (no separate login surfaced).
|
||||
- **PII (contacts/UBO)**: lawful basis recorded per delivery; honor deletion; minimize retention.
|
||||
- **DPA**: ship a Data Processing Agreement; provenance log + audit log are part of the offering.
|
||||
- **Secrets**: rotate; least privilege; no secrets in client bundles (only `NEXT_PUBLIC_*`).
|
||||
|
||||
## Embedding NocoDB / Metabase (so it feels like one app)
|
||||
- Generate a **signed embed URL** server-side (Metabase signed embedding with `METABASE_SECRET_KEY`; NocoDB shared/locked view + token) scoped to the customer's data, and render it in an iframe inside the mono chrome. The customer never sees a NocoDB/Metabase login.
|
||||
|
||||
## CI/CD (.github/workflows)
|
||||
- **web**: lint + typecheck + build → deploy to Vercel on push to `main` (Vercel Git integration).
|
||||
- **api**: lint (ruff) + tests (pytest) → build Docker image → deploy to VPS (SSH `docker compose pull && up -d`, or a small runner).
|
||||
- Run DB migrations as a deploy step.
|
||||
|
||||
## Monitoring (cheap)
|
||||
- Vercel Analytics + Speed Insights (web).
|
||||
- **Sentry** free tier (web + api).
|
||||
- Uptime ping on `/health`.
|
||||
- Worker/job health visible in admin (reads `jobs`).
|
||||
@@ -0,0 +1,75 @@
|
||||
# 09 · Build plan
|
||||
|
||||
Phased, in dependency order. **Finish a phase to its acceptance criteria before the next.** Each phase = small, reviewable commits. Reconcile with existing code rather than rewriting wholesale.
|
||||
|
||||
## Phase 0 — Foundation & repo
|
||||
- Monorepo structure (`docs/08`). Migrate existing `src/orchestrator` + `src/workers` into `services/api`.
|
||||
- Postgres schema + migrations (`docs/03`); `pgvector` enabled. `.env.example`.
|
||||
- `docker-compose` up (postgres, redis, api, worker, nocodb, metabase, caddy). `/health` green.
|
||||
- CI skeleton (lint/typecheck/build).
|
||||
- **Accept:** `docker compose up` runs the whole stack locally; migrations apply; `/health` returns ok for db + queue.
|
||||
|
||||
## Phase 1 — Design system in code
|
||||
- Tailwind + shadcn; `globals.css` (grain + reveal keyframes) and `useReveal` verbatim (`docs/05`).
|
||||
- Build **DataArtifact** primitive (table/map/schema/chart/grid) and **ProvenanceBadge** (`docs/design` prompts 5.1, 2.11).
|
||||
- **Accept:** a sample page renders mono tokens, a DataArtifact animates on scroll, reduced-motion respected, focus rings visible.
|
||||
|
||||
## Phase 2 — Marketing + catalog (placeholder data)
|
||||
- Landing components top-to-bottom (build-brief 2.1–2.10): Header, Hero, Philosophy, Catalog teaser, Engine, Visualisation gallery, Coverage, Editorial, Proof, Footer.
|
||||
- Full **Catalog** page + **Dataset detail** + **Sourcing-map** page (coverage doc 5.4). Postgres full-text search.
|
||||
- All with loading/empty/error states + mobile.
|
||||
- **Accept:** marketing site + catalog browse work end-to-end on placeholder catalog rows; one continuous scroll reads well; DoD met per screen.
|
||||
|
||||
## Phase 3 — Auth & portal shell
|
||||
- Auth.js + Postgres adapter; sign in/up/verify/forgot. `orgs`/`users`.
|
||||
- Portal shell + Dashboard ("Mina beställningar") with empty state.
|
||||
- **Accept:** a user can sign up, land in an empty dashboard, sign out; routes protected.
|
||||
|
||||
## Phase 4 — Order lifecycle end-to-end (test mode)
|
||||
- **Configurator** (2.12) wired to `POST /estimates` (real estimate math in FastAPI).
|
||||
- **Deposit checkout** (2.13) → Stripe **test** Checkout → webhook → order `IN_PRODUCTION`.
|
||||
- **Production tracker** (2.14) reading `GET /orders/{id}/events` (SSE) — pipeline may be **simulated** at this phase.
|
||||
- **Validation & delivery** (2.15): tabs render from a sample delivery; **accept** → `POST /payments/release` (test) → `DELIVERED`; **revise** leaves escrow untouched.
|
||||
- AI-assist (`POST /ai/parse`) prefilling the configurator.
|
||||
- **Accept:** a full custom order can be configured, deposit paid (test), watched, validated, accepted, and "delivered" — state machine in `docs/01` traversed; escrow split correct.
|
||||
|
||||
## Phase 5 — Real data engine (Nordic slice)
|
||||
- Compliant connectors first: **GLEIF + Brreg + Companies House** (verify access terms; record in `sources.yaml`). Remove any FlareSolverr/bypass path (`docs/06`).
|
||||
- Pipeline stages produce real `dim_company` + `fact_financials` with `bridge_provenance`; QA threshold drops low-confidence rows.
|
||||
- Production tracker now reflects **real** job metrics.
|
||||
- **Accept:** a Nordic company dataset is produced for a real order, every field has provenance, quality report generated, level-0 fields shown honestly (not fabricated).
|
||||
|
||||
## Phase 6 — Delivery surfaces & integrations
|
||||
- Exports: CSV / Parquet / JSON via R2 presigned URLs.
|
||||
- Embedded **NocoDB** ("Min data") and **Metabase** dashboards via signed URLs in mono chrome.
|
||||
- Integrations: **FiveCRM push**, webhooks, Google Sheets.
|
||||
- File **uploads** (Uppy → R2) for matching/enrichment.
|
||||
- **Accept:** a delivered dataset is downloadable, viewable in embedded NocoDB + Metabase (no separate login), and pushable to FiveCRM; uploads work.
|
||||
|
||||
## Phase 7 — Subscriptions / feeds
|
||||
- `subscriptions`; recurring jobs (weekly/daily) delivering **deltas**; webhooks on change.
|
||||
- Freshness monitoring; Resend emails (confirmation, "din data är klar", feedback).
|
||||
- **Accept:** an order converts to a feed; a scheduled run produces a delta + fires a webhook + email.
|
||||
|
||||
## Phase 8 — Admin, hardening, compliance
|
||||
- Admin: order queue + scope/quote + escrow release; **source registry & freshness** monitoring; worker/job monitoring; customer + catalog management.
|
||||
- Customer **DB-access grant** flow (encrypted, read-only, time-boxed) + revoke.
|
||||
- Security pass: RLS, presigned-only files, signed embeds, secret rotation, `audit_log` everywhere; **DPA** published; legal pages.
|
||||
- Finalize all loading/empty/error states + mobile across the app.
|
||||
- **Accept:** admin can run the business; DB-grant flow works and self-revokes; security checklist passes; DoD met app-wide.
|
||||
|
||||
## Phase 9 — Launch
|
||||
- Domain `0pt.app` (Vercel) + subdomains (api/data/dash via Caddy). Sentry + uptime live.
|
||||
- Soft launch: Protosell internally + a handful of existing dealer relationships.
|
||||
- **Accept:** a real external customer completes an order start→delivery in production; feedback loop running.
|
||||
|
||||
---
|
||||
|
||||
## Working conventions (for Claude Code)
|
||||
- Tech docs + identifiers **English**; UI copy **Swedish**.
|
||||
- Conventional Commits; one concern per PR; keep diffs reviewable.
|
||||
- No secrets in code; update `.env.example` when adding env.
|
||||
- Add/extend tests for engine logic (estimate math, fusion/confidence, dedup) and API contract.
|
||||
- When a source's access/licence is unclear, **verify and record in `sources.yaml`** before relying; if disallowed for bulk, route to licensed/partner or mark level 0 — never bypass, never fabricate.
|
||||
- Touch the design tokens/motion only via `docs/05`; keep red ≤ once per screen; no photography.
|
||||
- If anything here conflicts with existing code, **flag it** in the PR description and propose the reconciliation rather than guessing.
|
||||
Reference in New Issue
Block a user