CI (SIAX Cloud) / security (push) Successful in 31s
CI (SIAX Cloud) / integration (push) Successful in 47s
CI (SIAX Cloud) / quality (push) Successful in 1m6s
Security Scan (SIAX Cloud) / security (push) Successful in 29s
CI (SIAX Cloud) / security (pull_request) Successful in 26s
Security Scan (SIAX Cloud) / security (pull_request) Successful in 26s
CI (SIAX Cloud) / integration (pull_request) Successful in 41s
CI (SIAX Cloud) / quality (pull_request) Successful in 56s
The previous commit said "nobody has probed the running site". I then probed it, so the docs should not keep hedging. curl -sI https://0pt.app/ HTTP/2 402 server: Vercel x-vercel-error: DEPLOYMENT_DISABLED strict-transport-security: max-age=63072000 https://0pt.app/api/health -> HTTP 402, body "Payment required / DEPLOYMENT_DISABLED" Two facts hold simultaneously and the docs now state both: the Coolify deploy hook fires for real on every push to main, AND the public product domain is dark behind a disabled Vercel deployment with DNS never cut over. Neither cancels the other. Still not established, and still marked as such: whether the Coolify app builds and starts. A deploy endpoint returning 200 means the request was accepted, not that a container came up healthy. Side note worth recording for PR #19: the HSTS header on that 402 response is Vercel's platform default, which is exactly the disappearing-on-cutover risk next.config.mjs in PR #19 was written to remove. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
237 lines
26 KiB
Markdown
237 lines
26 KiB
Markdown
# 02 · Architecture
|
||
|
||
> **Known issues affecting this doc, in brief** — full detail + evidence in [`README.md`](../README.md)'s
|
||
> "Known issues" section: (1) no auth/session system exists yet, so several newer routes trust
|
||
> client-supplied `tenant_id` (see "Known gaps" below); (2) the CL0UD/N0D/AUD0/ST0RE platform clients are
|
||
> real code but mock-only in this repo; (3) `docs/ICEBERG_WRITE_BOUNDARY.md` / `docs/11-lakehouse-write-plan.md`
|
||
> / `docs/P0_QUERY_SOURCE_FLOW.md` reference several `src/services/quality-*` and `sql/002`/`sql/003` files
|
||
> that were never actually committed to this repo (out of this doc's file scope to fix — flagged here so it
|
||
> isn't silently repeated); (4) two git remotes (`github.com/sax3l/0pt` and `git.siax.io/sax3l/0pt`) have
|
||
> diverged and it is unresolved which is authoritative — Simon's call, not assumed here.
|
||
|
||
## Principle: one database, many lenses
|
||
|
||
Postgres is the source of truth. Everything else is a **lens** on it. A tool enters the stack only if it (a) reads that Postgres and (b) sits behind the same auth. This is the difference between an app and a pile of separate logins.
|
||
|
||
This principle describes — and is largely realized by — the **original edge + engine plane**: marketing, catalog, order lifecycle, and the FastAPI engine. It does **not** describe the newer **P0 data-platform surface** (`app/api/0pt/*`), which is real, additive, and deliberately does not go through this same path. See "Two API surfaces" below.
|
||
|
||
```
|
||
┌──────────────────────────────────────────────┐
|
||
COLLECTION │ POSTGRES │ LENSES (customer + admin)
|
||
(legal engine) │ dim_company · dim_contact · dim_vehicle │
|
||
─────────────── │ fact_financials · dim_ubo │ ─────────────────────────────
|
||
Official APIs │ dim_source · bridge_provenance │ Next.js portal (Vercel)
|
||
/bulk ───────▶│ orders · order_items · jobs · deliveries │◀── Catalog + order lifecycle
|
||
Authorized web │ subscriptions · users · orgs · files │ NocoDB (embedded "Min data")
|
||
Licensed data │ db_grants · audit_log · pgvector embeddings │ Metabase (embedded dashboards)
|
||
Customer upload│ │ Export: CSV/Parquet/JSON/API/FiveCRM
|
||
└───────────────────▲──────────────────────────┘
|
||
│
|
||
services/api — FastAPI orchestrator + workers
|
||
(estimates, orders, jobs, AI-assist, webhooks, pipeline)
|
||
```
|
||
|
||
## Components & responsibilities
|
||
|
||
- **Next.js (Vercel)** — all customer/admin UI: marketing site, catalog, configurator, checkout, production tracker, validation, portal, admin. It also hosts **two distinct families** of API route handlers under `app/api/`: the original order/catalog/estimate surface (some of which defer to the FastAPI engine), and the newer P0 data-platform surface under `app/api/0pt/*` (does not defer to the engine at all — see below).
|
||
- **FastAPI orchestrator** (`services/api/orchestrator/main.py`) — REST contract in `docs/04-api.md`: estimates, orders, sources, vehicle lookup, sanctions screening, job run/status. **Not on Vercel** (serverless timeouts kill long jobs). *Correction: earlier drafts of this doc pointed at `src/orchestrator`; that was the pre-migration location. `docs/09-build-plan.md` Phase 0 called for moving it into `services/api`, and that move has happened — the code lives at `services/api/orchestrator` today.*
|
||
- **Workers** (`services/api/workers/`) — adaptive router (`adaptive_router.py`, verified: chooses only among compliant methods — official API → bulk → authorized web → licensed, no bypass/stealth/proxy-rotation tier exists in it, by design) + pipeline stages (`pipeline.py`, `queue.py`, `resolve.py`, `run.py`). See `docs/06-data-engine.md`.
|
||
- **Postgres** — backbone (`docs/03-data-model.md`) incl. `pgvector` for matching/embeddings, for the original edge + engine plane. Wired via `lib/db.ts` (env-guarded: `DATABASE_URL`/`POSTGRES_URL`; falls back to an in-memory, non-durable store when unset, e.g. `lib/orders-store.ts`).
|
||
- **NocoDB / Metabase** — both are defined as services in `infra/docker-compose.yml`; embedding them into the portal as "Min data" / customer dashboards was the original target, and neither is confirmed wired into a portal page. *Correction 2026-09-06: Metabase is no longer the canonical BI lens — Apache Superset is (see "Stack decisions" below). Metabase may stay as an optional App Catalog app; it must not be treated as the platform default.*
|
||
- **Cloudflare R2** — file storage (uploads + deliverables) via presigned URLs. Env-guarded (`app/api/uploads/route.ts` checks `R2_ACCESS_KEY_ID`/`R2_BUCKET`/`R2_ENDPOINT` and returns `501` until set); the presign call itself is still a stub (`put_url: null`) even when the env vars are present — not wired end-to-end yet.
|
||
- **Stripe** — deposit capture + escrow balance release. This one **is** wired: `app/api/payments/[order_id]/deposit/route.ts` and `.../release/route.ts` are real Stripe Checkout calls (env-guarded on `STRIPE_SECRET_KEY`, `501` until configured). See the correction in `docs/01-product.md`'s business model section for exactly which escrow mechanism is implemented.
|
||
- **Resend** — documented as the transactional-email provider (`docs/08-infrastructure.md`) but has no call site anywhere in this repo yet. Treat "+ email" as not-yet-implemented, not merely unconfigured.
|
||
- **Auth.js (NextAuth) with a Postgres adapter** — this is the **target**, listed here as a stack decision, not a built component. What exists today is a lightweight signed-session scaffold (`lib/auth.ts`: HMAC-SHA256 via Web Crypto, no external deps) behind a guarded dev-only login gate (`DEV_LOGIN_KEY`). There is **no** real session-to-org/tenant binding anywhere in this repo. Full Auth.js + OAuth is Phase 3 of `docs/09-build-plan.md` and has not been built. This matters directly for the tenant-scoping gap described below.
|
||
|
||
## Two API surfaces
|
||
|
||
This repo currently contains **two real, distinct API surfaces**, not one. Earlier versions of this doc described the edge as a uniform thin proxy to the FastAPI engine — that was true for part of the original surface and remains true for that part, but a second, newer surface was added that is a deliberate, ADR-documented exception to it. Conflating the two is the single biggest way to misread this codebase's architecture.
|
||
|
||
### 1. The original edge → engine surface
|
||
|
||
A subset of the original `app/api/*` route handlers defer to the FastAPI engine (`services/api`) via `lib/engine.ts`'s `engineFetch()`/`hasEngine()` bridge when `ENGINE_URL` is set, and fall back to local computation when it isn't:
|
||
|
||
- `app/api/estimates/route.ts` — defers to `POST {ENGINE_URL}/estimates`, falls back to `lib/data.ts`'s `computeEstimate`.
|
||
- `app/api/vehicles/route.ts`, `app/api/screen/route.ts`, `app/api/ai/parse/route.ts` — same pattern.
|
||
|
||
Other original routes are **not** engine-backed at all today: `app/api/orders/*` reads/writes `lib/orders-store.ts` directly (Postgres when configured, else in-memory); `app/api/catalog/route.ts` returns a hardcoded static array. Both of these are entirely edge-local — there is no engine call in either path currently, not "thin proxy with a local fallback."
|
||
|
||
This surface is documented in full in `docs/04-api.md` and matches the "one Postgres, many lenses" principle above (once `DATABASE_URL` is set).
|
||
|
||
### 2. The P0 data-platform surface (`app/api/0pt/*`)
|
||
|
||
A second, newer surface implements dataset registration, Iceberg lakehouse table mapping/writes, per-source rate-policy management, bounded synchronous source querying, and mailbox-based document capture — **directly in the edge app**, backed by `src/services/*` and `src/clients/*`. It does **not** call `lib/engine.ts`/`ENGINE_URL` and does **not** go through the FastAPI engine at all. This is deliberate: it is the concrete implementation of **ADR-0030** (Apache Iceberg as SIAX's canonical lakehouse table format, authored in the `n0d` repo) and the ownership split it defines. See `docs/10-lakehouse-decision.md` (the pointer to the ecosystem decision), `docs/11-lakehouse-write-plan.md` (the plan), and `docs/ICEBERG_WRITE_BOUNDARY.md` (the implementation record) for the full boundary — this doc only summarizes it.
|
||
|
||
Real, existing routes today (verified against the filesystem, not against `app/api/0pt/route.ts`'s own advertised list — see the note below):
|
||
|
||
| Route | Backed by |
|
||
|---|---|
|
||
| `GET /api/0pt` | static index (see note below) |
|
||
| `POST/GET /api/0pt/documents/capture` | `src/services/document-capture-service.ts` |
|
||
| `.../documents/{documentId}/extraction` | `src/services/document-extraction-service.ts` |
|
||
| `POST/GET /api/0pt/lakehouse/mappings` | `src/services/dataset-iceberg-mapping-service.ts`, `iceberg-write-service.ts` |
|
||
| `.../lakehouse/mappings/{mappingId}` | same |
|
||
| `POST .../lakehouse/mappings/{mappingId}/writes` | `src/services/iceberg-write-service.ts#requestIcebergWrite` |
|
||
| `POST/GET /api/0pt/sources/rate-policies` | `src/services/source-rate-policy-service.ts` |
|
||
| `.../sources/rate-policies/{sourceRatePolicyId}` | same |
|
||
| `POST /api/0pt/sources/{sourceId}/query` | `src/services/source-query-service.ts` (bounded, timeout-guarded ad-hoc SQL against a registered `database` source — see `docs/P0_QUERY_SOURCE_FLOW.md`) |
|
||
|
||
**Code-level inconsistency worth knowing about**: `app/api/0pt/route.ts`'s own `GET` handler advertises an `endpoints` list including `/api/0pt/sources`, `/api/0pt/datasets`, `/api/0pt/data-contracts`, `/api/0pt/quality/checks`, `/api/0pt/quality/reports`, `/api/0pt/features/views` — **none of these exist as routes** (confirmed by listing `app/api/0pt/` on disk); the lakehouse-mapping and rate-policy routes that *do* exist aren't listed there either. This is the same "described but never built" pattern as the `quality-check-service.ts` fabrication flagged below — this doc's route table above reflects what is actually on disk, not what the index endpoint claims.
|
||
|
||
**In-memory scope**: every P0 service (`DatasetService`, `SourceService`, `DatasetIcebergMappingService`, `SourceRatePolicyService`, `DocumentCaptureService`, …) holds its state in a plain `Map` inside the Next.js route module that instantiates it. Most route modules construct a fresh instance per module, so a record created via one route is not visible to a different route module in the same running process (each file's own header comment documents this; `document-capture-service.ts` worked around it for its own two routes with a shared singleton export). A Postgres migration exists (`sql/004_iceberg_write_boundary.sql`) but is not wired into any route handler yet. So the "one database, many lenses" principle above does not yet hold for this surface.
|
||
|
||
## Six-surface capability model & 0pt's current coverage
|
||
|
||
SIAX Cloud's target model (defined in `cl0ud`, `docs/architecture/SIAX_CLOUD_UNIFIED_ARCHITECTURE.md`) is that one capability can expose up to **six surfaces**: REST **api**, **cli** (`siax 0pt ...`), **mcp** (an agent tool), **ui** (a page/panel), **job** (an async N0D job type), and **sdk** (a generated client method). The capability registry's own `Surface` type (`cl0ud` `packages/capability-registry/src/types.ts`) currently encodes five of those six as a declarative field (`'api' | 'cli' | 'mcp' | 'ui' | 'job'`); `sdk` is generated separately, not yet a registry field.
|
||
|
||
**0pt's actual coverage today** (verified against `cl0ud packages/capability-registry/src/registry.ts`): 14 distinct `0pt.*` capabilities are registered —
|
||
|
||
`0pt.query_source`, `0pt.document_ocr`, `0pt.write_external_record`, `0pt.lakehouse_mapping_manage`, `0pt.lakehouse_write`, `0pt.source_rate_policy_manage`, `0pt.screen_sanctions`, `0pt.estimate_compute`, `0pt.vehicle_lookup`, `0pt.catalog_list`, `0pt.source_registry_list`, `0pt.order_create`, `0pt.order_payment`, `0pt.write_iceberg_table`
|
||
|
||
— and, checked entry by entry:
|
||
|
||
- **11** declare `surfaces: ['api']` only.
|
||
- **2** (`0pt.lakehouse_write`, `0pt.write_iceberg_table`) declare `surfaces: ['api', 'job']`.
|
||
- **1** (`0pt.write_external_record`) declares `surfaces: ['job']` **only** — `apiEnabled: false`, and its own comment states it has "NO dedicated HTTP route in the 0pt app yet"; it exists as a tested TS service (`src/services/external-writeback-service.ts`) invoked only via direct import today, not a live callable path.
|
||
- **Every single one** has `cliEnabled: false` and `mcpEnabled: false`. None has a `ui` surface entry either.
|
||
|
||
In plain terms: **0pt is API-only today** (plus two job-backed writes, and one job-only capability with no live route at all). If any doc — inside or outside this repo — describes a live `siax 0pt ...` CLI command or an `0pt.*` MCP tool, it is describing a target, not current reality; correct it on sight.
|
||
|
||
## Platform clients: CL0UD / N0D / AUD0 / ST0RE
|
||
|
||
`src/clients/{cl0ud,n0d,aud0,st0re,st0re-lakehouse}-client.ts` are real, typechecked TypeScript with accompanying tests (`tests/p0/*`), each following the same shape: a thin interface, an HTTP implementation written against the target service's real, committed route contract, and a deterministic in-memory mock used whenever no `baseUrl` is supplied to the constructor.
|
||
|
||
**SUPERSEDED 2026-08-31 (SIAX Platforms P0 wave 2)** — the paragraph that
|
||
stood here ("every one of these clients runs against its in-memory mock
|
||
today") was accurate when written and is now fixed, twice over:
|
||
|
||
1. every factory reads its `*_URL` from `process.env` when no constructor
|
||
`baseUrl` is given, and under `NODE_ENV=production` an unset URL is a hard
|
||
boot failure (`SiaxError DEPENDENCY_UNAVAILABLE`, `src/platform/dependency-mode.ts`)
|
||
— production can no longer fall back to a mock (Master-DoD §3/§10);
|
||
2. the HTTP wire shapes were verified against the sibling repos' actual
|
||
`main` branches and THREE of four were wrong (cl0ud path never existed,
|
||
st0re body was snake_case against a camelCase contract, aud0 lineage path
|
||
never existed). All rewritten and pinned by `tests/contracts/*.contract.test.ts`.
|
||
|
||
Current per-capability status lives in [`STATUS.md`](../STATUS.md) — that
|
||
file, not this doc, is the status source of truth (§56).
|
||
|
||
What each client is for, concretely:
|
||
|
||
- **CL0UD** — policy evaluation (`evaluatePolicy`). `iceberg-write-service.ts` and `external-writeback-service.ts` call it before acting; the mock always returns `"allow"`.
|
||
- **N0D** — job requests (`createJob`). `document-capture-service.ts` requests document-extraction jobs; `iceberg-write-service.ts` requests heavy-write/compaction jobs. 0pt only *requests* N0D jobs — it never executes them, per `docs/ICEBERG_WRITE_BOUNDARY.md`'s "N0D executes" boundary. **No evidence supports a claim that N0D has a dedicated-node ("`t_0pt`") auth pattern already built for this**: `n0d`'s repository was searched in full (all branches, all commits, all files) and contains zero references to `t_0pt`. Treat any such claim as unconfirmed until proven otherwise.
|
||
- **AUD0** — audit/lineage events (`ingestEvent`, `ingestLineageEvent`). Every mutating P0 service emits these; the mock is a true no-op, matching AUD0's own documented stance that it has no live ingestion endpoint yet.
|
||
- **ST0RE / ST0RE-lakehouse** — artifact-metadata registration and Iceberg table/namespace/snapshot registration, respectively. `st0re-lakehouse-client.ts`'s HTTP path is written against `sax3l/st0re`'s actual shipped route contract (commit `9ea0abb`, read directly during a prior pass) — real code, but not executed against a running ST0RE in this repo's test environment either.
|
||
|
||
## Known gaps
|
||
|
||
### tenant_id is not server-bound anywhere in the P0 surface
|
||
|
||
The platform-wide rule is that `tenant_id` must be bound to an authenticated session server-side and never trusted from client input. This repo violates that rule today, and it is **not** a deliberate two-tier design — it's a direct consequence of there being no auth/session system at all yet (see the Auth.js note above; `app/api/orders/route.ts`'s own `GET` handler literally comments `// org scoping added with auth`, i.e. not yet).
|
||
|
||
Concretely: `tenant_id`/`project_id` are accepted straight from the client-supplied request body (or query string) at every P0 mutation entry point —
|
||
|
||
- `app/api/0pt/lakehouse/mappings/route.ts` (POST)
|
||
- `app/api/0pt/sources/rate-policies/route.ts` (POST)
|
||
- `app/api/0pt/sources/[sourceId]/query/route.ts` (POST)
|
||
- `app/api/0pt/documents/capture/route.ts` → `src/services/document-capture-service.ts#captureFromMailbox`
|
||
|
||
`app/api/0pt/lakehouse/mappings/[mappingId]/writes/route.ts` doesn't read `tenant_id` from its own body — but it resolves the tenant by looking up `mapping.tenant_id` from a mapping that was itself created via one of the unauthenticated routes above, so the trust boundary problem is inherited, not avoided.
|
||
|
||
Do not expose any of these routes to untrusted callers before Phase 3 of `docs/09-build-plan.md` (Auth.js + org/session binding) lands. This is Simon's call on sequencing, not something to silently patch in a docs pass.
|
||
|
||
### 0pt's data-quality/QA stage is a fabrication in sibling docs, not built
|
||
|
||
Flagged so it is not repeated: `src/services/quality-check-service.ts`, `src/services/quality-report-service.ts`, `src/types/quality-check.ts`, `src/types/quality-report.ts`, `src/data-platform/services/catalog-service.ts`, `sql/002_p0_dataset_quality.sql`, and `sql/003_*.sql` were at one point referenced in `docs/ICEBERG_WRITE_BOUNDARY.md`, `docs/11-lakehouse-write-plan.md`, and `docs/P0_QUERY_SOURCE_FLOW.md` as if already built and tested. Verified this pass: `git log --all` for every one of those paths returns zero commits, on any branch, ever — `sql/` on disk today contains only `001_init.sql` and `004_iceberg_write_boundary.sql`. Those files have never existed in this repository. 0pt's data-quality/QA enforcement is **spec/target only** — a real, open build gap, not a doc typo. **All three of the docs above have since been corrected** (verified this pass by reading each in full) — none currently asserts those files as real; each carries its own explicit "fabricated, not built" callout, and `docs/P0_DATASET_QUALITY_FLOW.md` is the honest target-spec replacement.
|
||
|
||
### Two diverged git remotes
|
||
|
||
`origin` in local checkouts of this repo is `github.com/sax3l/0pt`. A separate Gitea remote, `git.siax.io/sax3l/0pt`, shares at least part of the same history (`services/api/workers/adaptive_router.py` is byte-for-byte identical between GitHub's current `main` and a Gitea post-fix PR branch) but independently accumulated commits GitHub's `main` never received — including, at one point, a WAF-bypass/proxy-rotation/fake-identity scraper stack that directly violated the CLAUDE.md §1 sourcing rule (since removed on Gitea via a separate PR). **Which remote is authoritative, which is actually deployed to `0pt.app`, and whether the two should be kept in sync is unresolved and is Simon's decision** — do not assume `origin`/GitHub is the sole copy of this codebase, here or in any other doc.
|
||
|
||
## Stack decisions — IN
|
||
|
||
| Concern | Choice | Why |
|
||
|---|---|---|
|
||
| Frontend | Next.js App Router + TS + Tailwind + shadcn/ui | Vercel-native, matches existing skills |
|
||
| Backend/jobs | FastAPI + workers (`services/api`) | already built; long-running jobs |
|
||
| DB | Postgres (+ pgvector) | backbone; Neon free tier or on-VPS |
|
||
| Airtable layer | **NocoDB** | sits on existing tables, no duplication |
|
||
| BI | **Apache Superset** | canonical generic BI for SIAX Cloud: Apache-2.0 with an official embed SDK + guest tokens, so embedding never needs a commercial tier. Already defined as a service in `services/infra/docker-compose.data-stack.yml` and inventoried in `docs/compliance/OSS-INVENTORY.md`. **Corrected 2026-09-06** — this row previously said Metabase; see the correction under "OUT" below |
|
||
| Files | Cloudflare R2 + presigned URLs | cheapest S3-compatible |
|
||
| Payments | Stripe Checkout + webhook → Next.js API | ~1 day, no SaaS boilerplate needed |
|
||
| Auth | Auth.js + Postgres adapter | one system — **target, not yet built** (see Components above) |
|
||
| Email | Resend | best Next.js integration, free tier — **not yet wired** (no call site in this repo) |
|
||
| Quality | Great Expectations / checks **inside workers** | scores shipped with each order — **not yet built** for the P0 surface (see "Known gaps" above) |
|
||
|
||
## Stack decisions — OUT (do not add)
|
||
|
||
> **CORRECTED 2026-09-06 — two rows in this table were wrong and are withdrawn below.**
|
||
> This section was written when 0pt was planned as a standalone SaaS product. Under the SIAX
|
||
> Cloud analytics decision, **0pt is the canonical Analytics/Data/Semantic owner** — there is
|
||
> no `BI0`/`Analytics0` repo, no separate Superset platform, and no product-specific BI
|
||
> backend acting as a parallel capability owner. Third-party BI tools are interchangeable
|
||
> providers or App Catalog apps. Two consequences contradict the table as it stood:
|
||
>
|
||
> - **Apache Superset is canonical**, not rejected (Apache-2.0, official embed SDK + guest
|
||
> tokens). Metabase moves to the optional App Catalog.
|
||
> - **OpenMetadata is the canonical catalog** (default *before* DataHub), not rejected.
|
||
> DataHub remains optional.
|
||
>
|
||
> This is not a paper change — **this repository already shipped both**, which is what makes
|
||
> the old rows self-contradicting rather than merely out of date:
|
||
>
|
||
> - `src/metadata/openmetadata-adapter.ts` + `tests/contracts/metadata-features.contract.test.ts`
|
||
> (OpenMetadata projection; `STATUS.md` classifies it `IMPLEMENTED`, projection-never-canonical),
|
||
> - `services/infra/docker-compose.data-stack.yml` defines a `superset` service
|
||
> (`apache/superset:latest`), and `docs/compliance/OSS-INVENTORY.md` already inventories
|
||
> `superset | Apache-2.0 | services/infra/data-stack`.
|
||
>
|
||
> The governing principle behind the change is **one definition of every metric**: the 0pt
|
||
> Metric Catalog feeds MetricFlow, which feeds Superset / Cube / the native API / INF0. Point a
|
||
> product at a BI engine directly and each surface invents its own "average price".
|
||
|
||
| Rejected | Reason |
|
||
|---|---|
|
||
| ~~Superset~~ | **WITHDRAWN 2026-09-06 — see the correction note above.** Superset is now the canonical BI engine, not a rejected one |
|
||
| ~~DataHub / OpenMetadata~~ | **WITHDRAWN 2026-09-06 — see the correction note above.** OpenMetadata is the canonical catalog; only DataHub stays non-canonical (optional App Catalog) |
|
||
| Metabase / Lightdash / Rill / Grafana / DataHub as *canonical* BI or catalog | optional App Catalog apps only, never the platform default. Metabase's OSS build is AGPL and its full app-/SSO-embedding sits behind Pro/Enterprise, which does not work for SIAX as a platform provider; Lightdash gates embedding/caching behind an enterprise self-host licence; Rill's documented deployment path is bound to Rill Cloud (breaks zero-SaaS); Grafana (AGPLv3) is the right tool for **operational** analysis and is AUD0-owned, not business BI |
|
||
| Baserow / Teable | NocoDB chosen — not three Airtable clones |
|
||
| Meilisearch (v1) | Postgres full-text suffices to a few thousand rows; add later |
|
||
| Delta Sharing (v1) | unnecessary; customer pull via scoped role + exports |
|
||
| Trigger.dev | the existing queue already schedules jobs; Resend covers mail |
|
||
| FlareSolverr / stealth browsers / undetected-* / proxy-fingerprint rotation / fake-account creation | **prohibited** — see CLAUDE.md §1. Replace with compliant fetching. (The Gitea-only scraper stack described under "Two diverged git remotes" above is exactly this — removed there, never present on GitHub `main`.) |
|
||
|
||
## Hosting (cheapest coherent) — **SUPERSEDED** (BACKLOG P2-5)
|
||
|
||
> **SUPERSEDED 2026-08-11.** This section describes the original Vercel+Neon
|
||
> hosting plan. `siax.config.json` flipped `deploy.target` from `"vercel"` to
|
||
> `"coolify"` on 2026-08-09 to match every other SIAX Cloud OS repo — that
|
||
> decision supersedes the Vercel prescription below. ~~The live instance is
|
||
> **still actually running on Vercel today** (no Coolify app has been
|
||
> provisioned yet — see `README.md`'s "Deploy" section and
|
||
> `docs/architecture/ARCHITECTURE.md` §5/§8 for the current, verified state)~~;
|
||
> this section is kept for history, not as the current target. Do not read
|
||
> "Vercel (Hobby/free)" below as still-current guidance.
|
||
>
|
||
> **CORRECTED 2026-09-06 — the struck sentence above is no longer true.** The Coolify
|
||
> deploy hook is live: repo secret `COOLIFY_TOKEN` exists (created 2026-08-31T11:34:34Z)
|
||
> and repo Actions variable `COOLIFY_APP_ID` holds a real 24-character UUID, not the
|
||
> `TODO-no-coolify-app-provisioned-yet` literal, so both guards in
|
||
> `.gitea/workflows/cd.yml` pass. On the push to `main` at `1ff34410` (2026-09-06) the
|
||
> `Deploy to Coolify / deploy` job finished **success** with `coolify deploy HTTP 200`.
|
||
> **Every merge into `main` fires a real redeploy against cloud.siax.io.**
|
||
>
|
||
> Probed 2026-09-06 08:00 UTC: `0pt.app` still answers from **Vercel** (`server: Vercel`)
|
||
> with **HTTP 402 `DEPLOYMENT_DISABLED`** on both `/` and `/api/health`. The public domain
|
||
> is therefore *down*, DNS has not been cut over, and the Coolify app serves no `0pt.app`
|
||
> traffic. Still unestablished: whether the Coolify app builds and starts at all — an
|
||
> accepted deploy request is not a healthy container.
|
||
|
||
- **Vercel** (Hobby/free) — Next.js + custom domain `0pt.app`.
|
||
- **One small VPS** (e.g. Hetzner ~€5/mo) running `docker-compose` (`infra/docker-compose.yml`): FastAPI, workers, NocoDB, Metabase, (Postgres or use Neon), optional Meilisearch later. Per-service managed platforms (Railway) get expensive once you have 4–5 always-on services; a single VPS + Vercel + R2 lands ≈ <€10/mo to start.
|
||
- **Cloudflare R2** — object storage (env-guarded, presign not yet wired — see Components above).
|
||
- **Postgres** — on the VPS, or **Neon** free tier if you want it managed.
|
||
|
||
Full config, env, security, CI/CD: `docs/08-infrastructure.md`. Env var inventory: [`.env.example`](../.env.example) (present in this repo today — the linked GitHub PR that originally tracked it has long since merged; removed the stale cross-repo link per BACKLOG P3-5).
|