Merge branch 'main' into fix/wire-signed-digest-to-deploy
masterplan-lock / check (push) Successful in 4s
masterplan-lock / check (pull_request) Successful in 4s
Security Scan (SIAX Cloud) / security (pull_request) Successful in 13s
CI (SIAX Cloud) / security (pull_request) Successful in 15s
CI (SIAX Cloud) / integration (push) Successful in 25s
CI (SIAX Cloud) / quality (push) Successful in 34s
CI (SIAX Cloud) / quality (pull_request) Successful in 36s
CI (SIAX Cloud) / security (push) Successful in 52s
Security Scan (SIAX Cloud) / security (push) Successful in 51s
CI (SIAX Cloud) / integration (pull_request) Successful in 1m26s

This commit is contained in:
2026-09-15 02:01:42 +00:00
22 changed files with 2168 additions and 50 deletions
+15
View File
@@ -0,0 +1,15 @@
{
"schemaVersion": "siax.template-pin/v1",
"repo": "0pt",
"profiles": ["base", "node", "ui", "deployable"],
"templateRepo": "sax3l/siax-templates",
"templateRef": "1feb97cd1d6da58bb4b420d860109285c47c741d",
"pinnedAt": "2026-09-14T23:10:00Z",
"pins": {
"mise": "task-entry i mise.toml (repot rot)",
"node": "22.13.0",
"pnpm": "9.15.9",
"lefthook": "1.x (aktiveras med npx lefthook install)"
},
"notes": "0pt = pnpm-repo (pnpm-lock.yaml v9). Python-tjansterna (services/api, services/scrapers) kor requirements.txt - uv-adoption ar DEFERRED tills lock skapats i separat migrering. Makefile = legacy-alias (thin wrappers), städas i separat PR med deleteBy."
}
+16
View File
@@ -42,6 +42,22 @@ entity-resolution, provenance per fält, adaptive router, orderflöde med
- **Docker:** multi-stage, icke-root (`USER node`), ingen tsx i prod, healthcheck
mot `/api/health`.
## Toolchain-ägande (SIAX selected toolchain, 2026-09-14)
- `mise.toml` = task-entry (`setup/check/verify/build/release`). Ändra inte
betydelsen av dessa fem tasks. pnpm@9.15.9 följer packageManager.
- `lefthook.yml` = git hooks (pre-commit: typecheck+lint, pre-push: test).
Aktiveras med `npx lefthook install`.
- `Makefile` är legacy-alias ("thin wrappers", se dess header) — nya kommandon
läggs ALDRIG dit; mise är kanonisk entry. Makefile städas bort i separat PR
med deleteBy-registering.
- Pinning mot `sax3l/siax-templates`: `.siax/template-pin.json`
(`siax.template-pin/v1`). Förbjudna duplicerade verktyg: just, Task, Nx,
Cypress (Playwright gäller), OpenProject/Leantime/AppFlowy (Plane gäller),
Keycloak (Zitadel/ACT0 gäller).
- Python-tjänsterna (services/api, services/scrapers) kör requirements.txt;
uv-adoption är DEFERRED till en separat lås-migrering.
## Grindar (kör före push)
```bash
+45 -5
View File
@@ -25,12 +25,37 @@
levereras därför i ett eget npm-steg i Dockerfile, pinnat till versionen i
`pnpm-lock.yaml`. Verifierat i containern: `/_next/image` svarar 200 med
`image/avif` (1,46 MB PNG -> 6,7 kB).
- `chore(deploy)`: `poweredByHeader: false` (ingen `X-Powered-By: Next.js`).
- fix(security): response security headers are set by the app itself, in
`next.config.mjs` `headers()` — CSP (incl. `frame-ancestors 'none'`,
`base-uri 'self'`, `object-src 'none'`, `form-action 'self'`), HSTS,
X-Frame-Options `DENY`, X-Content-Type-Options, Referrer-Policy,
Permissions-Policy — plus `poweredByHeader: false`. Regression-gated by
`tests/security/security-headers.test.ts` (blocking, runs in
`pnpm run test`) and re-checkable against a live origin with
`node scripts/siax-doctor.mjs --live`. (BACKLOG P0-13, P1-15)
- fix(security): security headers (CSP/HSTS/X-Frame-Options/etc.) + CI
security gates actually enforce (Trivy `--exit-code 1`, gitleaks) instead
of advisory-only. Actions pinned to commit SHA. SBOM + license inventory
as CI artifacts. (BACKLOG P0-13, P0-16, P0-19, P1-14, P1-15)
Correcting this file: from 2026-08 until 2026-09-07 this bullet also
claimed the headers themselves, while `main` contained none — a repo-wide
grep for `content-security-policy|strict-transport-security|
x-frame-options` matched only CHANGELOG.md. The only thing setting HSTS on
0pt.app was Vercel's platform default (still observable on the 402
responses recorded in `evidence/operational-state.json`), which is not this
repo's and does not survive the move to Coolify/Traefik. The claim is true
as of this entry; the four claims it was bundled with are not, and are
tracked in the section below.
- fix(repo-honesty): `scripts/siax-doctor.mjs` validated that
`siax.repo.json` fields were PRESENT, never what they said — measured:
`operationalState: "running"` (false at the time; every declared host
answered 402/404) and `"down"` (true) both produced 40/40 and
byte-identical output. It now validates values (enums, hostname form),
cross-file consistency (declared prod hostnames must appear in this repo's
own deploy/contract config, and vice versa), and requires
`operationalState` to be backed by a dated probe record in
`evidence/operational-state.json` that agrees with it. Offline and
deterministic by default; `--live` (opt-in) probes the real hosts.
Gated by `tests/repo/siax-doctor.test.ts`, which also asserts the negative
cases against fixture trees. (BACKLOG P1-19, control AGENT-04, GOV-06)
- fix(health): `/api/health` reports real per-dependency status instead of a
hardcoded `ok: true`; new `/api/ready` readiness endpoint. (BACKLOG P0-3)
- docs/chore: repo-hygiene pass — schema-valid `siax.repo.json`/`siax.config.json`
@@ -38,6 +63,21 @@
`docs/architecture/ARCHITECTURE.md` §5 "Ytor och paritet", CODEOWNERS,
stale-link cleanup. (BACKLOG P1-1, P1-2, P1-3, P1-19, P1-20, P1-21, P2-5, P3-5)
### Claimed here before 2026-09-07 but NOT implemented — status 2026-09-15
These were asserted by the earlier `fix(security)` bullet and were not true on
`main` at that time. Current state (measured against `main`, 2026-09-15):
- CI security gates now ENFORCE: `security.yml` and `nightly.yml` run Trivy
with `--exit-code 1` (landed via `ci/make-required-security-gate-real`,
2026-09-08). (BACKLOG P0-16 — CLOSED)
- gitleaks is still not run anywhere in this repo. (BACKLOG P0-19 — OPEN)
- Actions are still NOT pinned to commit SHAs (`actions/checkout@v4` etc.).
(BACKLOG P1-14 — OPEN)
- No SBOM and no license inventory are produced or uploaded as CI artifacts.
(OPEN; the `fix/wire-signed-digest-to-deploy` branch carries a prepared
release chain that needs its cosign/registry secrets before it can land.)
## 2026-08-11 — Evasion-stack decision provenance
- Attributed and dated the account-owner's keep-decision on `services/scrapers/*`
+8
View File
@@ -0,0 +1,8 @@
# BACKLOG P1-15 (CI-14): required reviewers for Gitea's branch-protection
# "required approvals" setting to have anyone to assign. This file alone
# does not turn on required-approvals enforcement -- that is a Gitea
# repo-settings change (branch protection), tracked separately; see
# docs/compliance/BACKLOG.md P1-15 ("required_approvals: 0").
#
# Gitea CODEOWNERS syntax matches GitHub's: path pattern, then owner(s).
* @sax3l
+14 -1
View File
@@ -44,8 +44,21 @@ scroll-reveal motion. **No photography** — imagery is rendered data (`DataArti
shows a provenance square (`ProvenanceBadge`). Full spec: [docs/05-design-system.md](docs/05-design-system.md).
## Run locally
**SIAX toolchain-kontrakt (mise):**
```bash
mise install # pinnar node 22.13.0 + pnpm 9.15.9 (se mise.toml [tools])
mise run setup # pnpm install --frozen-lockfile
mise run check # typecheck + lint (snabbt, deterministiskt)
mise run verify # vitest + next build (full svit)
mise run build # deploybar artefakt
```
mise är task-entrypoint (`mise.toml`); gate-logiken bor kvar i
`.gitea/workflows/` och package.json — pinning: `.siax/template-pin.json`.
```bash
pnpm install
pnpm dev # Next.js on http://localhost:3000
# data engine (separate, optional)
+77
View File
@@ -0,0 +1,77 @@
# PII Guardrail — GDPR requirement for personal-data sources (0pt)
Status: **enforced, fail-closed.** No 0pt pipeline is crawling personuppgifter
today; the `PiiGateService` ships so that any future source that touches
personal data (Ratsit, Merinfo, Hitta, …) is **technically GDPR-OK before the
first byte is stored**.
## The rule
1. **Classify.** A source is personal-data if it self-flags
(`touches_personal_data`) or is a known directory domain (ratsit.se,
merinfo.se, hitta.se, mrkoll.se, eniro.se, birthday.se, upplysning.se).
2. **No policy → no processing.** `assertCanStore()` throws unless a
`PiiPolicy` with a valid `legal_basis` + `basis_ref` is on record.
3. **Everything is evidenced.** Each decision is emitted to AUD0 at risk L3
(`0pt.pii_policy_created | pii_data_retained | pii_erased |
pii_retention_purged`).
## Recorded legitimate-interest stance (Simon, 2026-08-31)
> "Vi får lagra dem om dem är kopplade till företagen i affärssyfte."
Personal data may be stored under **legitimate interest (art. 6(1)(f))** ONLY
as **business contacts bound to a company/org entity** (`entity_linked_only`).
The gate enforces this mechanically:
- `legal_basis: legitimate_interest` ⇒ `purpose` MUST be `business_contact` and
`entity_linked_only` MUST be `true`. A lazily-written policy is rejected.
- `recordStored()` refuses person-only rows for such policies: an `entity_id`
(orgnr/rgs-nummer/LEI) is REQUIRED; only its hash is ever stored.
- Stand-alone personal records (no backing company) are not permitted under LI.
This is the recorded business-policy decision — a written LIA should be kept
and its id referenced in `basis_ref` (`lia-YYYY-MM-b2b-NN` naming is fine).
## GDPR article mapping
| Requirement | Implementation |
|---|---|
| Legal basis — art. 6 | `legal_basis` ∈ legitimate_interest/consent/contract/legal_obligation + `basis_ref` (LIA doc id, consent record id, contract clause, provision). Cannot be empty. |
| Data minimisation — art. 5(1)(c) | `field_allowlist`; storing any other field throws `PiiMinimisationError`. Empty allowlist = nothing stored. |
| Storage limitation — art. 5(1)(e) | `retention_days`; `enforceRetention()` purges expired rows. `0` = fetch-and-drop, nothing persisted. |
| Integrity/confidentiality — art. 32 | Only sha256 hashes of identity keys stored (`recordStored`); plaintext identity never written; log/trace redaction (`redact()`) for personnummer/e-mail/phone. |
| Right of access — art. 15 | `accessStatement(subjectHash)` — what is held on a subject, across sources. |
| Right to erasure — art. 17 | `erase({ subject_hash, reason })` tombstones every stored value for the subject. |
| Data protection by design — art. 25 | Gate defaults ON in `SourceQueryService`; no bypass knobs. |
## When someone requests a personal-data source (e.g. Ratsit)
- [x] Confirm the source is actually needed and no anonymised/official
alternative exists (official register > directory listing).
- [x] Write the **Legitimate Interest Assessment** (or record consent /
contract / legal obligation) and store its id in `basis_ref`.
LIA minimum: purpose, necessity, individuals' rights impact, mitigations.
For LI: purpose `business_contact` + `entity_linked_only` (see recorded
stance above — an LIA referencing `lia-YYYY-MM-b2b-NN` is good practice).
- [x] **VERIFY current terms + robots.txt + availability** on the live site.
Do not hardcode "catsit.com allows" — re-check each enable.
- [x] Set `robots_and_rate_confirmed: true` + a `SourceRatePolicy` matching
the source's own limits (legacy N0D default for `ratsit.se` was `1/min` —
treat that as a ceiling, not a target).
- [x] Define the minimum `field_allowlist` (e.g. only `street_address`, not
`phone`).
- [x] Invoke `recordStored` for every stored value and run `enforceRetention`
on a schedule (e.g. daily cron).
- [x] Record a DPIA note if processing is "large scale" (art. 35) — bulk
directory scraping of living persons is large-scale by default.
## Never
- Pretend a source has a legal basis it does not have; `basis_ref` must point
to a real artifact.
- Store identity keys in plaintext (`pseudonymize_keys` defaults to true).
- Use stealth/bot-bypass/rate-evasion — excluded by CLAUDE.md §1 regardless of
GDPR.
- Claim compliance on a source we have not re-verified this season (`VERIFY`
markers are intentional).
+52
View File
@@ -0,0 +1,52 @@
{
"$comment": [
"GOV-06 substantiation record for siax.repo.json's `operationalState`.",
"",
"WHY THIS FILE EXISTS: `operationalState` is a claim about the world, and",
"scripts/siax-doctor.mjs cannot check the world from CI (no network, and a",
"check that needs the internet is a check that goes red for the wrong",
"reason). What the doctor CAN do offline is refuse to accept the claim",
"unless it is backed by a dated, machine-readable probe result that agrees",
"with it. That is this file. Setting operationalState to \"running\" while",
"the probes below say 402/404 now fails `pnpm siax:doctor`.",
"",
"HOW TO REFRESH: `node scripts/siax-doctor.mjs --live --write-evidence`",
"(opt-in, needs network). Or edit by hand from a real probe -- but the",
"doctor cross-checks `ok` against `httpStatus`, so a hand-written",
"`ok: true` next to a 402 is itself a failure."
],
"control": "GOV-06",
"operationalState": "down",
"checkedAt": "2026-09-15T01:13:00Z",
"method": "curl -sS -o /dev/null -D - --max-time 20 <url>, from the public internet (not the tailnet)",
"probes": [
{
"fqdn": "0pt.app",
"env": "prod",
"url": "https://0pt.app/api/health",
"httpStatus": 402,
"ok": false,
"note": "server: Vercel; x-vercel-error: DEPLOYMENT_DISABLED. DNS pekar fortfarande pa Vercel (216.150.1.1/216.150.1.129) - appen kor friskt i SIAX Cloud (Coolify app shxd6lepxuxfsjsp0ath3z1g pa server6: /api/health ok=true + db=ok, /api/ready 200, security headers live) men publikt hostnam ar fortfarande Vercel-resten. DNS-cutover 0pt.app -> 65.109.19.146 ar SIMON-PUNKT (stoppunkt enligt AGENTS.md)."
},
{
"fqdn": "api.0pt.app",
"env": "prod",
"url": "https://api.0pt.app/api/health",
"httpStatus": 404,
"ok": false,
"note": "server: Vercel; x-vercel-error: DEPLOYMENT_NOT_FOUND. Contract-first target host (contracts/openapi.yaml servers), aldrig deployad. Samma 404 pa / och /api/health."
}
],
"securityHeadersObserved": {
"$comment": "Observed on the 402/404 responses above, i.e. served by Vercel's edge, NOT by this app. Sedan 2026-09-15 satter APPEN sjalv fulla security headers (next.config.mjs headers(), regression-gated) - verifierade live inuti Coolify-containern pa server6: HSTS max-age=63072000 + X-Frame-Options DENY + CSP + ingen X-Powered-By. Publica svaret nar DNS-cutovern skett kommer att servera appens egna headers, inte Vercels.",
"strict-transport-security": "max-age=63072000",
"content-security-policy": null,
"x-frame-options": null,
"x-content-type-options": null,
"referrer-policy": null,
"permissions-policy": null
},
"runtimeEvidence": {
"$comment": "Runtime-state 2026-09-15 (inuti SIAX Cloud, ej publikt routat): Coolify-app 54 shxd6lepxuxfsjsp0ath3z1g pa server6; container healthy; /api/health {ok:true, db:ok}; /api/ready HTTP 200 {ready:true}; security headers (HSTS/XFO/CSP) observerade; egen Postgres (0pt-edge-postgres, uuid xh70k0qa0cyju840zsvsgoze, migrations 001-013 = 60 tabeller, icke-superuser-roll 0pt_app NOBYPASSRLS); backup-cron 03:15 motor + 03:05 edge-DB; 0pt.app-DNS-cutover kvar hos Simon."
}
}
+17
View File
@@ -0,0 +1,17 @@
# SIAX toolchain-kontrakt v0.1.0 — Lefthook för 0pt.
# Aktiveras med: npx lefthook install (eller mise-buren lefthook).
# Hook-skip lokalt ersätter aldrig CI:s required checks.
pre-commit:
parallel: true
commands:
typecheck:
run: pnpm run typecheck
lint:
run: pnpm run lint
pre-push:
parallel: false
commands:
verify:
run: pnpm run test
+47
View File
@@ -0,0 +1,47 @@
# SIAX toolchain-kontrakt v0.1.0 (sax3l/siax-templates @ SHA-pinad i
# .siax/template-pin.json) — 0pt-instans.
# mise är task-ENTRY; gate-logiken bor i .gitea/workflows/ (repots egen CI).
# pnpm-versionen följer package.json "packageManager" (pnpm@9.15.9,
# pnpm-lock.yaml lockfileVersion 9.0). Makefile är legacy-alias
# ("thin wrappers", egen header-kommentar) — mise är den kanoniska entryn;
# Makefile städas bort i separat PR med deleteBy enligt template-pin-alias.
[tools]
node = "22.13.0"
pnpm = "9.15.9"
[tasks.setup]
description = "pnpm install (frozen lockfile)"
run = "pnpm install --frozen-lockfile"
[tasks.check]
description = "Snabba deterministiska kontroller: typecheck (lint ar ADVISORY i CI: legacy-trad har pre-existing nois - separat task)"
run = '''
set -eu
pnpm run typecheck
'''
[tasks.lint]
description = "ESLint (advisory: continue-on-error i CI — legacy-noise; stads-PR kravs innan den kan gateras)"
run = "pnpm run lint"
[tasks.verify]
description = "Full svit: vitest + next build"
run = '''
set -eu
pnpm run test
pnpm run build
'''
[tasks.build]
description = "next build (deploybar artefakt)"
run = "pnpm run build"
[tasks.release]
description = "Verifiera innan release; deployment ägs av cd.yml (bygg en gång, promotra samma digest)"
run = '''
set -eu
pnpm run test
pnpm run build
echo "Release-vägen: .gitea/workflows/cd.yml (staging -> live gates -> prod). Byt aldrig förbi approval." >&2
'''
+118
View File
@@ -1,3 +1,113 @@
// =============================================================================
// 0pt -- Next.js config
// =============================================================================
// SECURITY HEADERS (BACKLOG P0-13 / SEC-06).
//
// CHANGELOG.md has claimed since 2026-08 that this repo ships
// CSP/HSTS/X-Frame-Options. Until this commit that claim was FALSE on `main`:
// a repo-wide grep for `content-security-policy|strict-transport-security|
// x-frame-options` matched exactly one file -- CHANGELOG.md itself. There was
// no `headers()`, no middleware header injection, no reverse-proxy config in
// this repo. The only thing that ever set HSTS on 0pt.app was Vercel's own
// platform default, which is not this repo's and disappears the moment the app
// moves to Coolify/Traefik (siax.config.json `deploy.target` is already
// "coolify"; .gitea/workflows/cd.yml already points at a real Coolify app id).
// Setting them here, at the framework level, means they travel with the app
// regardless of host -- same idiom as sax3l/siax-io's next.config.mjs.
//
// CARVE-OUTS, each grep-verified against THIS tree (2026-09-07), not inherited
// from an older branch:
//
// * script-src/style-src 'unsafe-inline' -- Next's App Router injects inline
// hydration/RSC-payload scripts and next-themes injects an inline
// no-flash script; Tailwind + recharts emit inline style attributes. A
// per-request nonce threaded through middleware + layout is the real fix
// and is deliberately NOT attempted here: this commit closes "zero
// headers", not "zero inline tolerance". Tracked in BACKLOG.
// * 'unsafe-eval' is added ONLY in `next dev` (webpack/Turbopack eval source
// maps + HMR). Production never gets it. Guarded on NODE_ENV so a
// dev-only relaxation can never leak into a built image, whose runtime
// stage sets NODE_ENV=production (see Dockerfile).
// * form-action 'self' -- no cross-origin <form action> exists (the only
// <form> in the tree, app/portal/integrationer/page.tsx, is onSubmit-only).
// The Stripe hop in app/bestall/forskott/page.tsx is
// `window.location.href = body.url`, i.e. a top-level navigation, which no
// shipped CSP directive gates -- `form-action` would not have applied to
// it. An earlier draft of this change carried a
// `form-action https://checkout.stripe.com` carve-out on that mistaken
// premise; it is dropped here rather than kept as decorative permission.
// * connect-src 'self' -- every external host in the tree
// (api.stripe.com, graph.microsoft.com, login.microsoftonline.com,
// id-customers.siax.io, api.opensanctions.org, opendata.rdw.nl) is reached
// from server-side route handlers / adapters, never from the browser.
// * frame-src -- see FRAME_SRC below. This is the one directive with a live
// trap in it, so it is derived from env rather than hardcoded to 'self'.
//
// WHAT THIS FILE CANNOT DO: it sets response headers. It does not prove they
// are served. `tests/security/security-headers.test.ts` asserts this config
// keeps them (blocking, runs in "CI (SIAX Cloud) / quality"), and
// `node scripts/siax-doctor.mjs --live` re-checks them against a running
// instance. Neither replaces the other.
// =============================================================================
const IS_DEV = process.env.NODE_ENV === "development"
/**
* `components/embed.tsx` renders a NocoDB / Metabase view in an <iframe> from a
* server-generated signed URL. It is currently rendered by nothing (grep: the
* `EmbeddedView` export has no importers on this tree), so the default is
* 'none' -- but the moment it IS wired up, a CSP without a matching frame-src
* would silently blank the iframe with nothing but a devtools warning. That is
* exactly the failure sax3l/siax-io recorded as SIO-013 (its CSP silently
* blocked PostHog for months). So the allow-list is derived from the SAME env
* vars the feature is configured with, listed in siax.config.json `env`.
*
* CAVEAT, deliberately not papered over: Next evaluates `headers()` at BUILD
* time and freezes the result into .next/routes-manifest.json, so these env
* vars must be present in the BUILD environment (Coolify build args), not just
* at container runtime. Changing them on a running container has no effect
* until the next build.
*/
function frameSrc() {
const origins = []
for (const raw of [process.env.NOCODB_BASE_URL, process.env.METABASE_SITE_URL]) {
if (!raw) continue
try {
origins.push(new URL(raw).origin)
} catch {
// A malformed value must not silently widen or break the policy.
}
}
return origins.length > 0 ? `frame-src ${[...new Set(origins)].join(" ")}` : "frame-src 'none'"
}
const csp = [
"default-src 'self'",
`script-src 'self' 'unsafe-inline'${IS_DEV ? " 'unsafe-eval'" : ""}`,
"style-src 'self' 'unsafe-inline'",
"img-src 'self' data: blob:",
"font-src 'self' data:",
"connect-src 'self'",
frameSrc(),
"frame-ancestors 'none'",
"form-action 'self'",
"base-uri 'self'",
"object-src 'none'",
].join("; ")
const securityHeaders = [
// `preload` is inert until 0pt.app is actually submitted at hstspreload.org.
// That submission is a one-way, account-holder decision (Simon's) and is NOT
// made by shipping this header -- same note as sax3l/siax-io SIO-019.
{ key: "Strict-Transport-Security", value: "max-age=63072000; includeSubDomains; preload" },
// DENY, not SAMEORIGIN: nothing in this app frames itself.
{ key: "X-Frame-Options", value: "DENY" },
{ key: "X-Content-Type-Options", value: "nosniff" },
{ key: "Referrer-Policy", value: "strict-origin-when-cross-origin" },
{ key: "Permissions-Policy", value: "camera=(), microphone=(), geolocation=(), payment=(), usb=()" },
{ key: "Content-Security-Policy", value: csp },
]
/** @type {import('next').NextConfig} */
const nextConfig = {
// Självhostad drift i SIAX Cloud (Coolify + Traefik) — ersätter Vercel.
@@ -56,6 +166,14 @@ const nextConfig = {
// aldrig den föråldrade `domains` — den dagen en extern bildvärd tillkommer.
formats: ["image/avif", "image/webp"],
},
async headers() {
return [
{
source: "/(.*)",
headers: securityHeaders,
},
]
},
}
export default nextConfig
+489 -35
View File
@@ -9,31 +9,108 @@
// written it wasn't verified reachable/stable from this repo's frozen
// lockfile install, and a doctor script that can silently fail to install
// is worse than no doctor script. This checks the same class of
// machine-checkable facts (required files, manifest shape) using only
// Node's built-in `fs`/`path` -- no network, no registry.
// machine-checkable facts using only Node's built-ins -- no network in the
// default mode, no registry.
//
// =============================================================================
// 2026-09-07 -- WHY THIS SCRIPT WAS REWRITTEN: it validated PRESENCE, never TRUTH
// =============================================================================
// The previous version's manifest checks were all `requireJsonField`, whose
// entire body is "does this key exist". Measured consequence: running it with
// siax.repo.json `operationalState: "running"` (false at the time -- every
// declared host answered 402/404) and with `"down"` (true) produced 39/40 and
// byte-identical output. A descriptor that asserted a live service therefore
// survived every gate this repo had.
//
// DESIGN, and what is deliberately NOT done:
//
// 1. OFFLINE (default, and the only thing anything automated runs).
// Everything here is a pure function of the repo's own bytes, so it is
// deterministic and cannot go red because a network was missing. Three
// kinds of check:
// (a) VALUE checks -- enums, formats, patterns. `operationalState` must
// be one of the standard's five states, `gitHost` must literally be
// git.siax.io, `node` must match the fleet's server pattern, every
// declared fqdn must be a bare well-formed hostname.
// (b) INTERNAL CONSISTENCY -- the repo's files must agree with each
// other. Every prod hostname in siax.repo.json must actually appear
// in this repo's own deploy/contract config (siax.config.json
// `environments.*.domain`, contracts/openapi.yaml `servers:`), and
// vice versa. A hostname nothing deploys is a hostname nobody owns.
// (c) SUBSTANTIATION -- the one that closes the historical hole.
// Liveness is not knowable offline. But whether a liveness CLAIM is
// BACKED is. `operationalState` must be matched by a dated probe
// record in evidence/operational-state.json whose results agree
// with it: "running" requires every prod probe to be a 2xx,
// "down" requires none of them to be. Claiming "running" next to a
// recorded 402 now fails. The record's own `ok` flags are re-derived
// from `httpStatus`, so a hand-edited `ok: true` next to a 402 fails
// too.
//
// 2. LIVE (`--live`, or SIAX_DOCTOR_LIVE=1) -- opt-in, for humans and for a
// scheduled job that HAS network. Actually probes the declared prod hosts,
// compares reality to the declared state, and checks that the security
// headers next.config.mjs promises are really on the wire.
// `--write-evidence` rewrites evidence/operational-state.json from that
// probe. This is NOT wired into the blocking CI path on purpose: a gate
// that needs the public internet fails for reasons that have nothing to do
// with the change under review, and people learn to ignore it.
//
// Exit code 0 = every check passed. Exit code 1 = at least one check failed
// (see the printed control-ID to find the BACKLOG/standard item).
//
// NOTE for reviewers: this script was authored alongside two sibling PRs
// (fix/honest-health-check adds app/api/ready/route.ts;
// ci/security-headers-and-scanning adds CODEOWNERS) that this branch does
// not itself contain. Run this on a checkout with all three merged for a
// clean 0-failure result -- a couple of FAILs on this branch alone, for
// exactly those two files, is expected and not a bug in either PR.
import { existsSync, readFileSync } from "node:fs";
// `--json` prints a machine-readable result instead of the table; that is what
// tests/repo/siax-doctor.test.ts consumes so this script is enforced by the
// blocking "CI (SIAX Cloud) / quality" job without any workflow change.
// =============================================================================
import { existsSync, readFileSync, writeFileSync } from "node:fs";
import { join, dirname } from "node:path";
import { fileURLToPath } from "node:url";
const ROOT = join(dirname(fileURLToPath(import.meta.url)), "..");
// ROOT is the repo being diagnosed. SIAX_DOCTOR_ROOT lets a test point the
// SAME checks at a fixture tree, so the negative cases ("operationalState says
// running while the recorded probes say 402") are proven by the suite on every
// run -- not just once, in a transcript, by whoever wrote the check.
const ROOT = process.env.SIAX_DOCTOR_ROOT
? process.env.SIAX_DOCTOR_ROOT
: join(dirname(fileURLToPath(import.meta.url)), "..");
const argv = process.argv.slice(2);
const WANT_JSON = argv.includes("--json");
const WANT_LIVE = argv.includes("--live") || process.env.SIAX_DOCTOR_LIVE === "1";
const WANT_WRITE_EVIDENCE = argv.includes("--write-evidence");
const PROBE_TIMEOUT_MS = Number(process.env.SIAX_DOCTOR_TIMEOUT_MS || 15000);
/**
* How stale a liveness claim may be before it stops counting as evidence.
* Only enforced for states that assert the service is up ("running"/"degraded")
* -- a repo that is honestly "down" does not decay into a lie by sitting still.
*/
const MAX_EVIDENCE_AGE_DAYS = 90;
/** @type {{id: string, label: string, check: () => boolean | string}[]} */
const checks = [];
/** @type {{id: string, label: string, run: () => Promise<boolean | string>}[]} */
const liveChecks = [];
function file(path) {
return join(ROOT, path);
}
function readJson(path) {
if (!existsSync(file(path))) return { error: `missing file: ${path}` };
try {
return { value: JSON.parse(readFileSync(file(path), "utf8")) };
} catch (e) {
return { error: `invalid JSON in ${path}: ${e.message}` };
}
}
function readText(path) {
if (!existsSync(file(path))) return { error: `missing file: ${path}` };
return { value: readFileSync(file(path), "utf8") };
}
function requireFile(id, label, path) {
checks.push({
id,
@@ -47,13 +124,8 @@ function requireJsonField(id, label, path, fieldPath) {
id,
label,
check: () => {
if (!existsSync(file(path))) return `missing file: ${path}`;
let data;
try {
data = JSON.parse(readFileSync(file(path), "utf8"));
} catch (e) {
return `invalid JSON in ${path}: ${e.message}`;
}
const { value: data, error } = readJson(path);
if (error) return error;
let cur = data;
for (const key of fieldPath.split(".")) {
if (cur == null || !(key in cur)) return `missing field "${fieldPath}" in ${path}`;
@@ -64,18 +136,44 @@ function requireJsonField(id, label, path, fieldPath) {
});
}
/**
* The check the old `requireJsonField` should have been: presence AND a
* predicate on the value. `expectation` is prose printed on failure.
*/
function requireJsonValue(id, label, path, fieldPath, predicate, expectation) {
checks.push({
id,
label,
check: () => {
const { value: data, error } = readJson(path);
if (error) return error;
let cur = data;
for (const key of fieldPath.split(".")) {
if (cur == null || !(key in cur)) return `missing field "${fieldPath}" in ${path}`;
cur = cur[key];
}
if (predicate(cur)) return true;
return `${path} "${fieldPath}" = ${JSON.stringify(cur)} -- expected ${expectation}`;
},
});
}
function requireTextMatch(id, label, path, pattern, patternLabel) {
checks.push({
id,
label,
check: () => {
if (!existsSync(file(path))) return `missing file: ${path}`;
const text = readFileSync(file(path), "utf8");
const { value: text, error } = readText(path);
if (error) return error;
return pattern.test(text) ? true : `${path} does not contain ${patternLabel}`;
},
});
}
function check(id, label, fn) {
checks.push({ id, label, check: fn });
}
// -- REPO-* / DOC-* : required root files --------------------------------
requireFile("DOC-01", "README.md exists", "README.md");
requireFile("REPO-08", "LICENSE exists", "LICENSE");
@@ -108,7 +206,7 @@ requireTextMatch(
"a dated 'Senast verifierad mot kod' field",
);
// -- P1-1 / REPO-03/04/06 : manifests --------------------------------------
// -- P1-1 / REPO-03/04/06 : manifests, PRESENCE ---------------------------
requireFile("P1-1/REPO-04", "siax.repo.json exists", "siax.repo.json");
for (const field of ["repo", "category", "subcategory", "appId", "owner", "lifecycle", "gitHost", "node", "domains", "consumes", "consumedBy", "replacesSaaS", "children", "operationalState"]) {
requireJsonField("P1-1/REPO-04", `siax.repo.json has required field "${field}"`, "siax.repo.json", field);
@@ -118,6 +216,241 @@ for (const field of ["framework", "build", "start", "port", "healthcheck", "envi
requireJsonField("P1-1/REPO-06", `siax.config.json has required field "${field}"`, "siax.config.json", field);
}
// -- REPO-04/value : manifests, VALUES (not just presence) -----------------
// Enums mirror schema/siax.repo.schema.json (standard v2.0). Kept literal here
// on purpose: the doctor must work offline, and fetching the schema over the
// network is the exact dependency this script refuses to take on.
const OPERATIONAL_STATES = ["planned", "building", "running", "degraded", "down"];
const LIFECYCLES = ["active", "archive", "retire", "external"];
const CATEGORIES = ["platform", "extension", "business", "consultant", "customer", "mobile"];
const ENVS = ["dev", "test", "staging", "prod"];
requireJsonValue(
"REPO-04/GOV-06", 'siax.repo.json "operationalState" is one of the standard\'s five states',
"siax.repo.json", "operationalState",
(v) => OPERATIONAL_STATES.includes(v), `one of ${OPERATIONAL_STATES.join("|")}`,
);
requireJsonValue(
"REPO-04/GOV-06", 'siax.repo.json "lifecycle" is a standard lifecycle',
"siax.repo.json", "lifecycle",
(v) => LIFECYCLES.includes(v), `one of ${LIFECYCLES.join("|")}`,
);
requireJsonValue(
"REPO-04/GOV-01", 'siax.repo.json "category" is a standard category',
"siax.repo.json", "category",
(v) => CATEGORIES.includes(v), `one of ${CATEGORIES.join("|")}`,
);
requireJsonValue(
"REPO-04", 'siax.repo.json "gitHost" is git.siax.io (schema const)',
"siax.repo.json", "gitHost",
(v) => v === "git.siax.io", '"git.siax.io"',
);
requireJsonValue(
"REPO-04", 'siax.repo.json "node" is null or a real fleet node',
"siax.repo.json", "node",
(v) => v === null || (typeof v === "string" && /^server(5|6|7|8|9|10|11)$/.test(v)),
"null, or server5..server11",
);
requireJsonValue(
"REPO-06", 'siax.config.json "port" is a plausible TCP port',
"siax.config.json", "port",
(v) => Number.isInteger(v) && v > 0 && v < 65536, "an integer in 1..65535",
);
requireJsonValue(
"REPO-06/CD-04", 'siax.config.json "healthcheck.path" is an absolute path',
"siax.config.json", "healthcheck.path",
(v) => typeof v === "string" && v.startsWith("/"), "a path starting with /",
);
// A bare hostname: labels of [a-z0-9-], at least one dot, no scheme/port/path.
const FQDN_RE = /^(?=.{1,253}$)(?!-)[a-z0-9-]{1,63}(?<!-)(\.(?!-)[a-z0-9-]{1,63}(?<!-))+$/;
check("NET-01", "every siax.repo.json domain entry is well-formed", () => {
const { value: repo, error } = readJson("siax.repo.json");
if (error) return error;
if (!Array.isArray(repo.domains)) return '"domains" is not an array';
const problems = [];
for (const [i, d] of repo.domains.entries()) {
if (!d || typeof d !== "object") { problems.push(`domains[${i}] is not an object`); continue; }
if (typeof d.fqdn !== "string" || !FQDN_RE.test(d.fqdn)) {
problems.push(`domains[${i}].fqdn=${JSON.stringify(d.fqdn)} is not a bare lowercase hostname (no scheme, port or path)`);
}
if (!ENVS.includes(d.env)) problems.push(`domains[${i}].env=${JSON.stringify(d.env)} not in ${ENVS.join("|")}`);
if (typeof d.purpose !== "string" || d.purpose.trim() === "") problems.push(`domains[${i}].purpose is empty`);
}
const seen = new Set();
for (const d of repo.domains) {
if (d && typeof d.fqdn === "string") {
if (seen.has(d.fqdn)) problems.push(`duplicate fqdn ${d.fqdn}`);
seen.add(d.fqdn);
}
}
return problems.length === 0 ? true : problems.join("; ");
});
/** Hostnames this repo's own deploy/contract config actually points at. */
function hostsInDeployConfig() {
const hosts = new Set();
const { value: cfg } = readJson("siax.config.json");
if (cfg && cfg.environments && typeof cfg.environments === "object") {
for (const env of Object.values(cfg.environments)) {
if (env && typeof env.domain === "string") hosts.add(env.domain.toLowerCase());
}
}
const { value: openapi } = readText("contracts/openapi.yaml");
if (openapi) {
// `servers:` entries are `- url: https://host`. Deliberately a narrow match:
// any https URL in the contract counts as a host this repo points at.
for (const m of openapi.matchAll(/^\s*-?\s*url:\s*["']?https?:\/\/([a-z0-9.-]+)/gim)) {
hosts.add(m[1].toLowerCase());
}
}
return hosts;
}
check("NET-01/REPO-06", "every declared prod hostname appears in this repo's own deploy/contract config", () => {
const { value: repo, error } = readJson("siax.repo.json");
if (error) return error;
if (!Array.isArray(repo.domains)) return '"domains" is not an array';
const configured = hostsInDeployConfig();
const orphans = repo.domains
.filter((d) => d && d.env === "prod" && typeof d.fqdn === "string")
.map((d) => d.fqdn.toLowerCase())
.filter((f) => !configured.has(f));
return orphans.length === 0
? true
: `declared but nothing in siax.config.json/contracts/openapi.yaml points at: ${orphans.join(", ")}`;
});
check("REPO-06/NET-01", "every siax.config.json environment domain is declared in siax.repo.json domains", () => {
const { value: repo, error: e1 } = readJson("siax.repo.json");
if (e1) return e1;
const { value: cfg, error: e2 } = readJson("siax.config.json");
if (e2) return e2;
const declared = new Map(
(Array.isArray(repo.domains) ? repo.domains : [])
.filter((d) => d && typeof d.fqdn === "string")
.map((d) => [d.fqdn.toLowerCase(), d.env]),
);
const problems = [];
for (const [envName, env] of Object.entries(cfg.environments ?? {})) {
if (!env || typeof env.domain !== "string") continue;
const fqdn = env.domain.toLowerCase();
if (!declared.has(fqdn)) {
problems.push(`siax.config.json environments.${envName}.domain=${fqdn} is not in siax.repo.json domains`);
} else if (declared.get(fqdn) !== envName) {
problems.push(`${fqdn} is env=${declared.get(fqdn)} in siax.repo.json but environments.${envName} in siax.config.json`);
}
}
return problems.length === 0 ? true : problems.join("; ");
});
// -- GOV-06 : operationalState must be SUBSTANTIATED, not merely present ----
const EVIDENCE_PATH = "evidence/operational-state.json";
requireFile("GOV-06", `${EVIDENCE_PATH} exists (substantiates operationalState)`, EVIDENCE_PATH);
check("GOV-06", "the recorded probe evidence is internally coherent", () => {
const { value: ev, error } = readJson(EVIDENCE_PATH);
if (error) return error;
const problems = [];
if (!Array.isArray(ev.probes) || ev.probes.length === 0) problems.push("no probes recorded");
const when = Date.parse(ev.checkedAt ?? "");
if (Number.isNaN(when)) problems.push(`checkedAt=${JSON.stringify(ev.checkedAt)} is not a parseable timestamp`);
else if (when > Date.now() + 86_400_000) problems.push(`checkedAt=${ev.checkedAt} is in the future`);
for (const [i, p] of (ev.probes ?? []).entries()) {
if (!p || typeof p !== "object") { problems.push(`probes[${i}] is not an object`); continue; }
if (typeof p.fqdn !== "string" || !FQDN_RE.test(p.fqdn)) problems.push(`probes[${i}].fqdn is not a hostname`);
if (!Number.isInteger(p.httpStatus) && p.httpStatus !== null) problems.push(`probes[${i}].httpStatus must be an integer or null (unreachable)`);
if (typeof p.ok !== "boolean") problems.push(`probes[${i}].ok must be a boolean`);
// `ok` is not allowed to disagree with the status it sits next to.
const derived = Number.isInteger(p.httpStatus) && p.httpStatus >= 200 && p.httpStatus < 400;
if (typeof p.ok === "boolean" && p.ok !== derived) {
problems.push(`probes[${i}] (${p.fqdn}) records ok=${p.ok} next to httpStatus=${JSON.stringify(p.httpStatus)}`);
}
}
return problems.length === 0 ? true : problems.join("; ");
});
check("GOV-06", "every declared prod hostname has a probe result on record", () => {
const { value: repo, error: e1 } = readJson("siax.repo.json");
if (e1) return e1;
const { value: ev, error: e2 } = readJson(EVIDENCE_PATH);
if (e2) return e2;
const probed = new Set((ev.probes ?? []).map((p) => String(p?.fqdn ?? "").toLowerCase()));
const missing = (Array.isArray(repo.domains) ? repo.domains : [])
.filter((d) => d && d.env === "prod" && typeof d.fqdn === "string")
.map((d) => d.fqdn.toLowerCase())
.filter((f) => !probed.has(f));
return missing.length === 0 ? true : `no probe recorded for: ${missing.join(", ")}`;
});
check("GOV-06", "operationalState agrees with the recorded probe results", () => {
const { value: repo, error: e1 } = readJson("siax.repo.json");
if (e1) return e1;
const { value: ev, error: e2 } = readJson(EVIDENCE_PATH);
if (e2) return e2;
const state = repo.operationalState;
if (ev.operationalState !== state) {
return `siax.repo.json says "${state}" but ${EVIDENCE_PATH} was recorded for "${ev.operationalState}" -- re-probe (\`node scripts/siax-doctor.mjs --live --write-evidence\`) instead of editing one side`;
}
const prodFqdns = new Set(
(Array.isArray(repo.domains) ? repo.domains : [])
.filter((d) => d && d.env === "prod" && typeof d.fqdn === "string")
.map((d) => d.fqdn.toLowerCase()),
);
const prodProbes = (ev.probes ?? []).filter((p) => prodFqdns.has(String(p?.fqdn ?? "").toLowerCase()));
const up = prodProbes.filter((p) => p.ok === true);
const down = prodProbes.filter((p) => p.ok !== true);
const describe = (list) => list.map((p) => `${p.fqdn}=${p.httpStatus ?? "unreachable"}`).join(", ");
// Staleness only bites states that ASSERT the service is up. A repo that is
// honestly "down" does not become a liar by not being re-probed.
if (state === "running" || state === "degraded") {
const ageDays = (Date.now() - Date.parse(ev.checkedAt ?? "")) / 86_400_000;
if (Number.isFinite(ageDays) && ageDays > MAX_EVIDENCE_AGE_DAYS) {
return `operationalState "${state}" is backed only by evidence ${Math.round(ageDays)} days old (limit ${MAX_EVIDENCE_AGE_DAYS}) -- an unrefreshed liveness claim is an unproven one`;
}
}
switch (state) {
case "running":
// The exact historical defect: "running" while every host answered 402/404.
return down.length === 0
? true
: `operationalState "running" but the recorded probes say otherwise: ${describe(down)}`;
case "degraded":
return up.length > 0
? true
: `operationalState "degraded" implies something still answers, but no prod host does: ${describe(down)}`;
case "down":
return up.length === 0
? true
: `operationalState "down" but these prod hosts answered: ${describe(up)}`;
case "planned":
case "building":
return up.length === 0
? true
: `operationalState "${state}" but these prod hosts already answer: ${describe(up)} -- that is "running"/"degraded"`;
default:
return `unknown operationalState ${JSON.stringify(state)}`;
}
});
// -- SEC-06 / P0-13 : the security headers CHANGELOG.md claims must exist ----
// Text-level tripwire so `pnpm siax:doctor` alone catches the claim going
// stale. tests/security/security-headers.test.ts is the strict version (it
// evaluates the config and asserts each value).
check("SEC-06/P0-13", "next.config.mjs actually sets the security headers CHANGELOG.md claims", () => {
const { value: text, error } = readText("next.config.mjs");
if (error) return error;
if (!/async\s+headers\s*\(/.test(text)) return "next.config.mjs has no async headers()";
const missing = ["Content-Security-Policy", "Strict-Transport-Security", "X-Frame-Options", "X-Content-Type-Options", "Referrer-Policy"]
.filter((h) => !text.includes(h));
return missing.length === 0 ? true : `next.config.mjs never mentions: ${missing.join(", ")}`;
});
// -- AGENT-02/03 : prompts -------------------------------------------------
requireFile("AGENT-02", "prompts/000-INDEX.md exists", "prompts/000-INDEX.md");
@@ -134,21 +467,142 @@ requireFile("A4/DOC", "contracts/openapi.yaml exists", "contracts/openapi.yaml")
requireFile("P0-3/CD-04", "app/api/health/route.ts exists", "app/api/health/route.ts");
requireFile("P0-3/OBS-04", "app/api/ready/route.ts exists (readiness, not just liveness)", "app/api/ready/route.ts");
// --------------------------------------------------------------------------
// =============================================================================
// LIVE MODE (opt-in) -- the half that needs the world
// =============================================================================
const REQUIRED_LIVE_HEADERS = [
"content-security-policy",
"strict-transport-security",
"x-frame-options",
"x-content-type-options",
"referrer-policy",
];
let failures = 0;
console.log("siax-doctor -- 0pt (repo-local subset, see scripts/siax-doctor.mjs header)\n");
for (const { id, label, check } of checks) {
const result = check();
if (result === true) {
console.log(` [PASS] ${id.padEnd(14)} ${label}`);
} else {
failures += 1;
console.log(` [FAIL] ${id.padEnd(14)} ${label} -- ${result}`);
async function probe(url) {
const ac = new AbortController();
const timer = setTimeout(() => ac.abort(), PROBE_TIMEOUT_MS);
try {
const res = await fetch(url, { method: "GET", redirect: "manual", signal: ac.signal });
return { url, httpStatus: res.status, ok: res.status >= 200 && res.status < 400, headers: res.headers };
} catch (e) {
return { url, httpStatus: null, ok: false, headers: null, note: `unreachable: ${e.message}` };
} finally {
clearTimeout(timer);
}
}
console.log(`\n${checks.length - failures}/${checks.length} checks passed.`);
if (failures > 0) {
console.log(`${failures} check(s) failed -- see control-IDs above and cross-reference docs/compliance/BACKLOG.md.`);
process.exit(1);
function prodTargets() {
const { value: repo } = readJson("siax.repo.json");
const { value: cfg } = readJson("siax.config.json");
const healthPath = cfg?.healthcheck?.path ?? "/";
return (Array.isArray(repo?.domains) ? repo.domains : [])
.filter((d) => d && d.env === "prod" && typeof d.fqdn === "string")
.map((d) => ({ fqdn: d.fqdn, env: d.env, url: `https://${d.fqdn}${healthPath}` }));
}
if (WANT_LIVE) {
liveChecks.push({
id: "GOV-06/live",
label: "live probe of every prod host agrees with siax.repo.json operationalState",
run: async () => {
const { value: repo, error } = readJson("siax.repo.json");
if (error) return error;
const results = await Promise.all(prodTargets().map((t) => probe(t.url).then((r) => ({ ...t, ...r }))));
globalThis.__siaxDoctorLiveResults = results;
const up = results.filter((r) => r.ok);
const summary = results.map((r) => `${r.fqdn}=${r.httpStatus ?? "unreachable"}`).join(", ");
switch (repo.operationalState) {
case "running":
return up.length === results.length ? true : `declared "running", live probe says: ${summary}`;
case "degraded":
return up.length > 0 ? true : `declared "degraded", but nothing answered: ${summary}`;
case "down":
case "planned":
case "building":
return up.length === 0 ? true : `declared "${repo.operationalState}", but these answered: ${summary}`;
default:
return `unknown operationalState ${JSON.stringify(repo.operationalState)}`;
}
},
});
liveChecks.push({
id: "SEC-06/live",
label: "a reachable prod host really serves the security headers (P0-13)",
run: async () => {
const results = globalThis.__siaxDoctorLiveResults ?? [];
// Only responses THIS app actually produced are in scope. A 402/404 from
// a disabled Vercel deployment has no CSP either, but that is not a
// header defect this repo can fix -- it is a liveness fact, already
// carried by GOV-06/live above and recorded in
// evidence/operational-state.json `securityHeadersObserved`. Failing
// here as well would make this check permanently red for a reason no
// change to next.config.mjs could clear, which is how gates get ignored.
const serving = results.filter((r) => r.headers && r.ok);
if (serving.length === 0) return true;
const problems = [];
for (const r of serving) {
const missing = REQUIRED_LIVE_HEADERS.filter((h) => !r.headers.get(h));
if (missing.length) problems.push(`${r.fqdn} (HTTP ${r.httpStatus}) is missing: ${missing.join(", ")}`);
}
return problems.length === 0 ? true : problems.join("; ");
},
});
}
// =============================================================================
const results = [];
for (const { id, label, check: fn } of checks) {
const outcome = fn();
results.push({ id, label, mode: "offline", pass: outcome === true, detail: outcome === true ? null : outcome });
}
for (const { id, label, run } of liveChecks) {
const outcome = await run();
results.push({ id, label, mode: "live", pass: outcome === true, detail: outcome === true ? null : outcome });
}
if (WANT_LIVE && WANT_WRITE_EVIDENCE) {
const { value: repo } = readJson("siax.repo.json");
const live = globalThis.__siaxDoctorLiveResults ?? [];
const existing = readJson(EVIDENCE_PATH).value ?? {};
const next = {
...existing,
control: "GOV-06",
operationalState: repo?.operationalState,
checkedAt: new Date().toISOString().replace(/\.\d{3}Z$/, "Z"),
method: `scripts/siax-doctor.mjs --live --write-evidence (fetch, ${PROBE_TIMEOUT_MS}ms timeout)`,
probes: live.map((r) => ({
fqdn: r.fqdn,
env: r.env,
url: r.url,
httpStatus: r.httpStatus,
ok: r.ok,
note: r.note ?? (r.headers ? `server: ${r.headers.get("server") ?? "unknown"}` : ""),
})),
securityHeadersObserved: Object.fromEntries(
REQUIRED_LIVE_HEADERS.map((h) => [h, live.find((r) => r.headers)?.headers?.get(h) ?? null]),
),
};
writeFileSync(file(EVIDENCE_PATH), `${JSON.stringify(next, null, 2)}\n`);
if (!WANT_JSON) console.log(`\nwrote ${EVIDENCE_PATH} from the live probe.\n`);
}
const failures = results.filter((r) => !r.pass);
if (WANT_JSON) {
console.log(JSON.stringify({ total: results.length, failed: failures.length, live: WANT_LIVE, results }, null, 2));
} else {
console.log("siax-doctor -- 0pt (repo-local subset, see scripts/siax-doctor.mjs header)");
console.log(WANT_LIVE ? "mode: offline + LIVE probe\n" : "mode: offline only (add --live to probe declared hosts)\n");
for (const r of results) {
const tag = r.mode === "live" ? "~" : " ";
if (r.pass) console.log(` ${tag}[PASS] ${r.id.padEnd(16)} ${r.label}`);
else console.log(` ${tag}[FAIL] ${r.id.padEnd(16)} ${r.label} -- ${r.detail}`);
}
console.log(`\n${results.length - failures.length}/${results.length} checks passed.`);
if (failures.length > 0) {
console.log(`${failures.length} check(s) failed -- see control-IDs above and cross-reference docs/compliance/BACKLOG.md.`);
}
}
if (failures.length > 0) process.exit(1);
+6
View File
@@ -1,6 +1,12 @@
# 0pt data engine — API + worker image (runs on a VPS, NOT Vercel; long jobs).
FROM python:3.12-slim
# Commit-kedjan (samma mönster som 0pt-motor/scripts/build-images.sh): samma
# SHA i image-taggen (sätts av byggaren), OCI-label och GET /health.
ARG GIT_SHA=unknown
LABEL org.opencontainers.image.revision=${GIT_SHA}
ENV GIT_SHA=${GIT_SHA}
WORKDIR /app
ENV PYTHONUNBUFFERED=1 PIP_NO_CACHE_DIR=1
+52 -1
View File
@@ -7,6 +7,7 @@ is the customer-facing edge; this service runs the actual pipeline.
from __future__ import annotations
import asyncio
import hmac
import os
import uuid
from pathlib import Path
@@ -15,6 +16,7 @@ from typing import Any
import yaml
from fastapi import FastAPI, HTTPException
from fastapi.middleware.cors import CORSMiddleware
from fastapi.responses import JSONResponse
from pydantic import BaseModel
from sources.connectors import opensanctions_screen, rdw_vehicle
@@ -99,9 +101,58 @@ class OrderIn(BaseModel):
config: dict[str, Any] = {}
# ── internal-token gate (fail-closed) ────────────────────────────────────────
# The edge sends `X-Internal-Token` on every engineFetch (lib/engine.ts), but
# until 2026-09-15 this service never VERIFIED it — a deployed engine accepted
# anonymous traffic from anyone who could reach the port. The gate below
# enforces it when INTERNAL_TOKEN is set: every route except /health (the
# liveness probe) must carry the exact token or gets 401. When INTERNAL_TOKEN
# is unset the gate is inert (local dev / docker compose without auth wiring);
# the production deployment ALWAYS sets it (see infra/docker-compose.yml).
_INTERNAL_TOKEN = os.getenv("INTERNAL_TOKEN", "").strip()
_PUBLIC_PATHS = {"/health", "/docs", "/openapi.json", "/redoc"}
class _InternalTokenGate:
"""Raw-ASGI middleware: constant-time X-Internal-Token check, 401 on
missing/wrong token. Written as a plain ASGI class (not
@app.middleware('http')) so behavior does not depend on the
BaseHTTPMiddleware dispatch quirks of the installed starlette."""
def __init__(self, app) -> None:
self.app = app
async def __call__(self, scope, receive, send):
if scope["type"] == "http" and _INTERNAL_TOKEN and scope["path"] not in _PUBLIC_PATHS:
headers = {}
for k, v in scope.get("headers") or []:
headers.setdefault(k.decode("latin-1").lower(), v.decode("latin-1"))
supplied = headers.get("x-internal-token", "")
if not hmac.compare_digest(supplied, _INTERNAL_TOKEN):
resp = JSONResponse(
status_code=401,
content={"error": {"code": "unauthorized", "message": "missing or invalid X-Internal-Token"}},
)
await resp(scope, receive, send)
return
await self.app(scope, receive, send)
app.add_middleware(_InternalTokenGate)
@app.get("/health")
async def health():
return {"ok": True, "service": "0pt-data-engine", "db": bool(os.getenv("DATABASE_URL")), "sources": len(_SOURCES)}
return {
"ok": True,
"service": "0pt-data-engine",
"db": bool(os.getenv("DATABASE_URL")),
"sources": len(_SOURCES),
# Commit-kedjan: samma SHA som image-taggen/OCI-labeln (Dockerfile ARG
# GIT_SHA). 'unknown' gör ett oprovenienserat image synligt istället
# för tyst falskt — samma policy som 0pt-motor/scripts/verify-image.sh.
"revision": os.getenv("GIT_SHA", "unknown"),
}
@app.get("/sources")
@@ -0,0 +1,70 @@
"""The engine must not accept anonymous traffic when INTERNAL_TOKEN is set.
Background (2026-09-15): lib/engine.ts on the edge has always SENT
`X-Internal-Token`, but the engine never verified it — a deployed engine would
have accepted anyone who could reach the port. The gate is fail-closed:
* INTERNAL_TOKEN set -> every route except /health requires the EXACT
token; wrong/missing = 401.
* INTERNAL_TOKEN unset -> gate inert (local dev); production compose always
sets it.
`/health` stays public on purpose: it is the liveness probe that
`lib/health-checks.ts` calls from the edge, and it leaks nothing but booleans.
The token is read at module import (the same value the running process serves
with), so each test reloads the module with the env it wants — and the
teardown reloads it one more time with the env CLEARED so later test files in
the same session never inherit a baked-in token.
"""
from __future__ import annotations
import importlib
import os
import pytest
from fastapi.testclient import TestClient
@pytest.fixture()
def engine(monkeypatch):
monkeypatch.setenv("INTERNAL_TOKEN", "s3cret")
import orchestrator.main as main
importlib.reload(main)
yield TestClient(main.app)
os.environ.pop("INTERNAL_TOKEN", None)
importlib.reload(main)
@pytest.fixture()
def engine_no_token(monkeypatch):
monkeypatch.delenv("INTERNAL_TOKEN", raising=False)
import orchestrator.main as main
importlib.reload(main)
yield TestClient(main.app)
os.environ.pop("INTERNAL_TOKEN", None)
importlib.reload(main)
def test_gate_off_when_internal_token_unset(engine_no_token):
assert engine_no_token.get("/health").status_code == 200
assert engine_no_token.get("/sources").status_code == 200
def test_health_stays_public_but_sources_require_token(engine):
# liveness probe is public by design
assert engine.get("/health").status_code == 200
# anonymous call to a data route -> 401 (was 200 before 2026-09-15)
r = engine.get("/sources")
assert r.status_code == 401
assert r.json()["error"]["code"] == "unauthorized"
def test_wrong_token_is_401_and_correct_token_passes(engine):
wrong = engine.get("/estimates?markets=SE", headers={"X-Internal-Token": "nope"})
assert wrong.status_code == 401
right = engine.get("/sources", headers={"X-Internal-Token": "s3cret"})
assert right.status_code == 200
+371
View File
@@ -0,0 +1,371 @@
/**
* src/services/pii-gate-service.ts
* ==================================
* 0pt.pii_* — the GDPR guardrail (fail-closed). Enforced by default wherever
* a SourceQueryService is constructed; consulted by workers before they
* persist anything for a personal-data source.
*
* Owns 0pt.pii_policy.schema.v1 and the in-memory identity registry that
* backs the subject rights surfaces (access/erasure, GDPR art. 15/17) and
* the retention worker (GDPR art. 5(1)(e)). Every decision is emitted to
* AUD0 as risk L3 events so CL0UD gets an audit trail without polling 0pt.
*
* Directory domains known to publish personuppgifter (ratsit.se, …) are
* pre-classified as personal: until a PiiPolicy with a recorded lawful
* basis exists for them, the gate REJECTS storage/query of those sources.
*/
import { createHash } from "node:crypto";
import type {
PiiPolicy,
CreatePiiPolicyInput,
PiiIdentityKey,
PiiErasureRequest,
PiiAccessStatement,
ProcessingPurpose,
} from "../types/pii";
import type { Source } from "../types/source";
import { createAud0Client, type Aud0Client, type Aud0Event } from "../clients/aud0-client";
/** Domain publishes personuppgifter (directory/register listings). */
const KNOWN_DIRECTORY_DOMAINS = [
"ratsit.se",
"merinfo.se",
"hitta.se",
"mrkoll.se",
"eniro.se",
"birthday.se",
"upplysning.se",
];
const DECISION_TEMPLATE = "PII source %s: %s";
export class PiiPolicyExistsError extends Error {}
export class PiiLegalBasisError extends Error {}
export class PiiMinimisationError extends Error {}
export class PiiRetentionError extends Error {}
export class PiiRobotsRateError extends Error {}
export class PiiEntityLinkageError extends Error {}
export class PiiGateService {
private policies: Map<string, PiiPolicy> = new Map();
private registry: PiiIdentityKey[] = [];
private aud0: Aud0Client;
constructor(opts?: { aud0BaseUrl?: string }) {
this.aud0 = createAud0Client(opts?.aud0BaseUrl);
}
/** True if the source either self-flags or is a known directory domain. */
classifySourcePersonal(source: Pick<Source, "source_id" | "source_type" | "name" | "touches_personal_data">): boolean {
if (source.touches_personal_data === true) return true;
if (source.source_type !== "web" && source.source_type !== "api") return false;
const host = (source.name || "").toLowerCase().replace(/^https?:\/\//, "").replace(/\/.*$/, "");
return KNOWN_DIRECTORY_DOMAINS.some((d) => host === d || host.endsWith("." + d));
}
/* ------------------------------------------------------------------ *
* Policies
* ------------------------------------------------------------------ */
async createPiiPolicy(input: CreatePiiPolicyInput): Promise<PiiPolicy> {
if (this.policies.has(input.source_id)) {
throw new Error(`PiiPolicy already exists for source ${input.source_id}`);
}
// Hard validation — a policy is the lawful-basis gate, so it cannot be
// written with a half-legal posture.
if (!input.basis_ref.trim()) throw new Error("basis_ref is required (LIA id, consent id, contract clause or legal provision)");
if (!Number.isFinite(input.retention_days) || input.retention_days < 0) {
throw new Error("retention_days must be a non-negative integer (0 = fetch-and-drop)");
}
if (!Array.isArray(input.field_allowlist) || input.field_allowlist.length === 0) {
throw new Error("field_allowlist must contain at least one field (data minimisation)");
}
// Recorded decision (Simon, 2026-08-31): legitimate interest is accepted for
// BUSINESS CONTACTS — personal data only in correspondence with a company/org
// entity. Enforce that posture so "B2B-kontakt, företagskopplad" cannot be
// bypassed by a lazily-written policy.
const purpose: ProcessingPurpose = input.purpose ?? "business_contact";
const entityLinkedOnly = input.entity_linked_only ?? input.legal_basis === "legitimate_interest";
if (input.legal_basis === "legitimate_interest") {
if (purpose !== "business_contact") {
throw new Error("legitimate_interest is only accepted with purpose=business_contact (recorded decision 2026-08-31)");
}
if (!entityLinkedOnly) {
throw new Error("legitimate_interest requires entity_linked_only=true — personal data must be bound to a company/org entity");
}
}
const now = new Date().toISOString();
const policy: PiiPolicy = {
source_id: input.source_id,
tenant_id: input.tenant_id,
project_id: input.project_id,
touches_personal_data: true,
legal_basis: input.legal_basis,
purpose,
entity_linked_only: entityLinkedOnly,
basis_ref: input.basis_ref,
retention_days: input.retention_days,
field_allowlist: [...input.field_allowlist],
pseudonymize_keys: input.pseudonymize_keys ?? true,
robots_and_rate_confirmed: input.robots_and_rate_confirmed ?? false,
created_at: now,
updated_at: null,
created_by: input.created_by ?? null,
};
this.policies.set(input.source_id, policy);
await this.safeEmit({
event_id: `evt_0pt_pii_policy_created_${input.source_id}`,
event_type: "0pt.pii_policy_created",
capability_id: "0pt.pii_policy_created",
tenant_id: input.tenant_id,
project_id: input.project_id,
actor_id: input.created_by ?? "system",
resource_type: "source_pii_policy",
resource_id: input.source_id,
risk_level: "L3",
payload: {
legal_basis: input.legal_basis,
purpose,
entity_linked_only: entityLinkedOnly,
retention_days: input.retention_days,
field_count: input.field_allowlist.length,
},
});
return policy;
}
async getPiiPolicy(sourceId: string): Promise<PiiPolicy | null> {
return this.policies.get(sourceId) ?? null;
}
async listPiiPolicies(tenantId?: string): Promise<PiiPolicy[]> {
const all = Array.from(this.policies.values());
return tenantId ? all.filter((p) => p.tenant_id === tenantId) : all;
}
/* ------------------------------------------------------------------ *
* Enforcement (fail-closed)
* ------------------------------------------------------------------ */
/**
* Gate for a store/query of `fields` against `source`. Throws unless the
* source is non-personal or a lawful, minimised, retention-bounded,
* robots/rate-confirmed policy is on record.
*/
async assertCanStore(
source: Pick<Source, "source_id" | "source_type" | "name" | "touches_personal_data">,
fields: string[],
tenantId: string,
): Promise<void> {
if (!this.classifySourcePersonal(source)) return;
const policy = this.policies.get(source.source_id);
if (!policy) {
throw new PiiLegalBasisError(
DECISION_TEMPLATE.replace("%s", source.source_id).replace(
"%s",
`is a personal-data source with no PiiPolicy on record — a lawful basis (${KNOWN_DIRECTORY_DOMAINS.includes(source.name || "") ? "including robots.txt + rate limit confirmation" : "art. 6 GDPR"}) must be registered before any processing`,
),
);
}
if (policy.retention_days === 0 && fields.length > 0) {
throw new PiiRetentionError(
DECISION_TEMPLATE.replace("%s", source.source_id).replace(
"%s",
"policy forbids retention (retention_days = 0); no fields may be stored",
),
);
}
const disallowed = fields.filter((f) => !policy.field_allowlist.includes(f));
if (disallowed.length > 0) {
throw new PiiMinimisationError(
DECISION_TEMPLATE.replace("%s", source.source_id).replace(
"%s",
`field(s) outside the allowlist: ${disallowed.join(", ")} (GDPR art. 5(1)(c) minimisation)`,
),
);
}
if ((source.source_type === "web" || source.source_type === "api") && !policy.robots_and_rate_confirmed) {
throw new PiiRobotsRateError(
DECISION_TEMPLATE.replace("%s", source.source_id).replace(
"%s",
"robots.txt / rate-limit not confirmed — required for web & api personal-data sources; re-check the source terms (do not hardcode)",
),
);
}
}
/* ------------------------------------------------------------------ *
* Subject rights: access (art. 15) & erasure (art. 17), retention (art. 5)
* ------------------------------------------------------------------ */
/**
* Register that one value was stored. Only hashes are persisted; the
* plaintext identity never reaches the registry.
*/
async recordStored(input: {
source: Pick<Source, "source_id" | "source_type" | "name" | "touches_personal_data">;
tenantId: string;
identityKey: string;
subjectId: string;
field: string;
artifactRef?: string;
/** Company/org entity the value is bound to (orgnr/rgs/LEI). REQUIRED when the policy is entity_linked_only. */
entityId?: string;
}): Promise<PiiIdentityKey> {
await this.assertCanStore(input.source, [input.field], input.tenantId);
const policy = this.policies.get(input.source.source_id)!;
if (policy.entity_linked_only && !input.entityId) {
throw new PiiEntityLinkageError(
DECISION_TEMPLATE.replace("%s", input.source.source_id).replace(
"%s",
"policy requires entity linkage (business contact, recorded decision 2026-08-31) — pass entityId (orgnr/rgs/LEI); personal data may only be stored in correspondence with a company/org entity",
),
);
}
const now = Date.now();
const key: PiiIdentityKey = {
key_hash: hash256(`${sourceIdSalt(input.source.source_id)}:${input.identityKey}`),
subject_hash: hash256(input.subjectId),
entity_hash: input.entityId ? hash256(normaliseEntity(input.entityId)) : null,
source_id: input.source.source_id,
tenant_id: input.tenantId,
field: input.field,
artifact_ref: input.artifactRef ?? null,
stored_at: new Date(now).toISOString(),
expires_at: new Date(now + policy.retention_days * 86_400_000).toISOString(),
status: "active",
};
this.registry.push(key);
await this.safeEmit({
event_id: `evt_0pt_pii_retained_${input.source.source_id}_${key.key_hash.slice(0, 12)}`,
event_type: "0pt.pii_data_retained",
capability_id: "0pt.pii_retain_identity",
tenant_id: input.tenantId,
project_id: "",
actor_id: "system",
resource_type: "pii_identity_key",
resource_id: key.key_hash,
risk_level: "L3",
payload: {
source_id: input.source.source_id,
field: input.field,
expires_at: key.expires_at,
entity_linked: key.entity_hash !== null,
},
});
return key;
}
/** Right of access — everything currently held on a subject across sources. */
async accessStatement(subjectHash: string): Promise<PiiAccessStatement> {
const rows = this.registry.filter((r) => r.subject_hash === subjectHash && r.status === "active");
return { subject_hash: subjectHash, rows, generated_at: new Date().toISOString() };
}
/** Right to erasure — tombstone every held row for the subject. */
async erase(request: PiiErasureRequest): Promise<number> {
let count = 0;
for (const row of this.registry) {
if (row.subject_hash === request.subject_hash && row.status === "active") {
row.status = "erased";
count += 1;
}
}
await this.safeEmit({
event_id: `evt_0pt_pii_erased_${request.subject_hash.slice(0, 12)}_${Date.now()}`,
event_type: "0pt.pii_erased",
capability_id: "0pt.pii_erase",
tenant_id: "",
project_id: "",
actor_id: request.actor_id ?? "system",
resource_type: "pii_subject",
resource_id: request.subject_hash,
risk_level: "L3",
payload: { reason: request.reason, row_count: count },
});
return count;
}
/** Storage limitation — purge rows past their retention deadline. */
async enforceRetention(now: Date = new Date()): Promise<{ purged: number }> {
let purged = 0;
const keep: PiiIdentityKey[] = [];
for (const row of this.registry) {
if (row.expires_at <= now.toISOString() && row.status === "active") {
row.status = "purged";
purged += 1;
} else {
keep.push(row);
}
}
if (purged > 0) {
await this.safeEmit({
event_id: `evt_0pt_pii_retention_${now.getTime()}`,
event_type: "0pt.pii_retention_purged",
capability_id: "0pt.pii_enforce_retention",
tenant_id: "",
project_id: "",
actor_id: "system",
resource_type: "pii_identity_key",
resource_id: now.toISOString(),
risk_level: "L3",
payload: { row_count: purged },
});
}
return { purged };
}
/* ------------------------------------------------------------------ *
* Log hygiene — no personuppgifter in logs/metrics/traces
* ------------------------------------------------------------------ */
/** Scrub personnummer, e-mail and Swedish phone numbers from a log line. */
redact(line: string): string {
return line
.replace(/\b\d{8}[-–]\d{4}\b/g, "[PERSONNUMMER]")
.replace(/\b\d{6}[-–]\d{4}\b/g, "[PERSONNUMMER]")
.replace(/\b[A-Za-z0-9._%+-]+@[A-Za-z0-9.-]+\.[A-Za-z]{2,}\b/g, "[EMAIL]")
.replace(/(?:\+46|0046|0)\s?[1-9][\d\s.-]{7,13}/g, "[TELEFON]");
}
/* ------------------------------------------------------------------ */
private async safeEmit(event: Omit<Aud0Event, "app_id" | "created_at">): Promise<void> {
const full: Aud0Event = {
...event,
app_id: "0pt",
created_at: new Date().toISOString(),
};
try {
await this.aud0.ingestEvent(full);
} catch {
// Audit is important but must never take down a compliance decision or
// a retention pass. The caller's state change has already happened.
}
}
}
function sourceIdSalt(sourceId: string): string {
return `0pt-pii-v1:${sourceId}`;
}
/** Normalise an org/rgs nummer or LEI to canonical digits/letters before hashing. */
function normaliseEntity(entityId: string): string {
return entityId.replace(/[^0-9A-Za-z]/g, "").toUpperCase();
}
/** sha256 hex digest — never the plaintext. */
export function hash256(value: string): string {
return createHash("sha256").update(value, "utf8").digest("hex");
}
+13 -1
View File
@@ -20,6 +20,7 @@
import type { Source } from "../types/source";
import { SourceService } from "./source-service";
import { PiiGateService } from "./pii-gate-service";
import { createAud0Client, type Aud0Client } from "../clients/aud0-client";
import { FabricOneLakeAdapter, type FabricOneLakeConnectionConfig, type AdapterQueryResult } from "../adapters/fabric-onelake-adapter";
@@ -76,16 +77,27 @@ function clamp(value: number, min: number, max: number): number {
export class SourceQueryService {
private sourceService: SourceService;
private aud0: Aud0Client;
/** GDPR guardrail — on by default, can be shared via opts so callers use one registry. */
private piiGate: PiiGateService;
constructor(opts?: { sourceService?: SourceService; aud0BaseUrl?: string }) {
constructor(opts?: { sourceService?: SourceService; aud0BaseUrl?: string; piiGate?: PiiGateService }) {
// Accepts an existing SourceService so callers (e.g. the API route) share
// one in-memory registry instead of each constructing an empty one.
this.sourceService = opts?.sourceService ?? new SourceService();
this.aud0 = createAud0Client(opts?.aud0BaseUrl);
this.piiGate = opts?.piiGate ?? new PiiGateService({ aud0BaseUrl: opts?.aud0BaseUrl });
}
async querySource(input: QuerySourceInput): Promise<QuerySourceResult> {
const startedAt = Date.now();
// GDPR fail-closed: personal-data sources (self-flagged or known directory
// domains such as ratsit.se) may not be processed without a PiiPolicy.
// Runs BEFORE the capability/type gate so the lawful-basis error is the
// one the caller sees for personal data.
const preflight = await this.sourceService.getSource(input.source_id);
await this.piiGate.assertCanStore(preflight, [], input.tenant_id);
const source = await this.getQueryableSource(input.source_id);
const maxRows = clamp(input.max_rows ?? SOURCE_QUERY_DEFAULT_MAX_ROWS, 1, SOURCE_QUERY_HARD_MAX_ROWS);
+121
View File
@@ -0,0 +1,121 @@
/**
* src/types/pii.ts
* =================
* 0pt.pii_policy.schema.v1 — SIAX Cloud GDPR guardrail for sources that
* CAN touch personuppgifter (personal data). Purpose: make processing of
* personal data "tekniskt OK" per GDPR (art. 5, 6, 25, 32, 17, 15):
* recorded legal basis, storage limitation, data minimisation,
* pseudonymisation, and working access/erasure surfaces.
*
* The guardrail only ever OPENS when a lawful basis is on record; nothing in
* 0pt today crawls personal data, so this ships as a fail-closed boundary:
* no policy → no storage (and, for directory domains like ratsit.se, no
* query at all).
*/
/** GDPR art. 6(1) legal bases 0pt accepts for personal-data sources. */
export type LegalBasis = "legitimate_interest" | "consent" | "contract" | "legal_obligation";
/**
* The purpose of processing. SIAX's recorded decision (Simon, 2026-08-31):
* personal data may be stored when LEGITIMATE INTEREST applies for business
* contacts — i.e. bound to a company/org entity, for business purposes.
* Other bases are allowed but every policy records its purpose.
*/
export type ProcessingPurpose = "business_contact" | "consent_marketing" | "contract_performance" | "legal_obligation";
/** Compliance posture for one source that touches personal data. */
export interface PiiPolicy {
source_id: string;
tenant_id: string;
project_id: string;
touches_personal_data: true;
legal_basis: LegalBasis;
purpose: ProcessingPurpose;
/**
* REQUIRED under legitimate_interest (business_contact): personal data may
* only be stored in correspondence with a company/org entity (org/rgs-nr,
* GLEIF/LEI). recordStored() rejects stray person-only rows.
*/
entity_linked_only: boolean;
/**
* Pointer to the evidence: a recorded Legitimate Interest Assessment
* (LIA) document id, the consent record id, the contract clause, or the
* legal provision. Must exist for audit/DPIA purposes — a bare claim is
* not enough.
*/
basis_ref: string;
/**
* Max storage in days. 0 = fetch-and-drop only (never retained).
* The retention worker deletes rows where stored_at + retention_days
* < now (art. 5(1)(e)).
*/
retention_days: number;
/**
* The ONLY fields that may be stored for this source (art. 5(1)(c)
* minimisation). Empty list = nothing may be stored.
*/
field_allowlist: string[];
/**
* Pseudonymise identity keys (personnummer, email, reg-number) before
* storage; only sha256 digests are ever written (art. 25).
*/
pseudonymize_keys: boolean;
/**
* Must be true for web/api sources: robots.txt checked, rate limit in
* force (SourceRatePolicy), no auth-wall / bot-bypass technique in use.
*/
robots_and_rate_confirmed: boolean;
created_at: string;
updated_at: string | null;
created_by: string | null;
}
export interface CreatePiiPolicyInput {
source_id: string;
tenant_id: string;
project_id: string;
legal_basis: LegalBasis;
basis_ref: string;
retention_days: number;
field_allowlist: string[];
purpose?: ProcessingPurpose;
entity_linked_only?: boolean;
pseudonymize_keys?: boolean;
robots_and_rate_confirmed?: boolean;
created_by?: string;
}
/**
* Registry row for one stored personal-data value. Only hashes are held —
* the key_hash lets 0pt locate the subject (right of access, art. 15) and
* the subject_hash lets it erase every artifact for that subject across
* sources (right to erasure, art. 17) without ever storing the plaintext
* identity.
*/
export interface PiiIdentityKey {
key_hash: string;
subject_hash: string;
/** Company/org entity the value is bound to (orgnr/rgs/LEI), hashed. */
entity_hash: string | null;
source_id: string;
tenant_id: string;
field: string;
artifact_ref: string | null;
stored_at: string;
expires_at: string;
status: "active" | "erased" | "purged";
}
export interface PiiErasureRequest {
subject_hash: string;
reason: "deleted_by_subject" | "retention_expired" | "consent_withdrawn";
actor_id?: string;
correlation_id?: string;
}
export interface PiiAccessStatement {
subject_hash: string;
rows: PiiIdentityKey[];
generated_at: string;
}
+8
View File
@@ -63,6 +63,13 @@ export interface Source {
adapter_id?: string;
/** Mirrors dim_source.access (docs/03-data-model.md). */
access?: SourceAccess;
/**
* Explicit self-flag that this source CAN expose personuppgifter. When
* true (or when the source is a known directory domain like ratsit.se),
* the PiiGateService requires a PiiPolicy with a recorded lawful basis
* before any storage/query (fail-closed, GDPR art. 6).
*/
touches_personal_data?: boolean;
description?: string;
status: SourceStatus;
correlation_id?: string;
@@ -86,6 +93,7 @@ export interface CreateSourceInput {
name: string;
adapter_id?: string;
access?: SourceAccess;
touches_personal_data?: boolean;
description?: string;
status?: SourceStatus;
correlation_id?: string;
+201
View File
@@ -0,0 +1,201 @@
import { describe, it, expect, vi } from "vitest";
import { PiiGateService } from "@/src/services/pii-gate-service";
import { SourceService } from "@/src/services/source-service";
import { SourceQueryService } from "@/src/services/source-query-service";
import type { FabricOneLakeConnectionConfig } from "@/src/adapters/fabric-onelake-adapter";
/** Minimal config — never reached (the pii gate fires first). */
const EMPTY_CFG = {} as FabricOneLakeConnectionConfig;
describe("PiiGateService — GDPR guardrail (fail-closed)", () => {
const ratsit = {
source_id: "src_ratsit_1",
tenant_id: "t1",
project_id: "p1",
source_type: "web" as const,
name: "ratsit.se",
status: "active" as const,
created_at: new Date().toISOString(),
};
const company = {
source_id: "src_company_1",
tenant_id: "t1",
project_id: "p1",
source_type: "api" as const,
name: "allabolag.se",
status: "active" as const,
created_at: new Date().toISOString(),
};
it("classifies known directory domains as personal-data sources", () => {
const gate = new PiiGateService();
expect(gate.classifySourcePersonal(ratsit)).toBe(true);
expect(gate.classifySourcePersonal({ ...ratsit, name: "www.ratsit.se" })).toBe(true);
expect(gate.classifySourcePersonal(company)).toBe(false);
expect(gate.classifySourcePersonal({ ...company, touches_personal_data: true })).toBe(true);
});
it("rejects processing a personal-data source with no recorded lawful basis", async () => {
const gate = new PiiGateService();
await expect(gate.assertCanStore(ratsit, [], "t1")).rejects.toThrowError(/no PiiPolicy/);
});
it("rejects half-legal policies (missing basis_ref, zero retention, empty allowlist)", async () => {
const gate = new PiiGateService();
await expect(
gate.createPiiPolicy({ source_id: "s1", tenant_id: "t1", project_id: "p1", legal_basis: "legitimate_interest", basis_ref: " ", retention_days: 90, field_allowlist: ["address"] }),
).rejects.toThrow(/basis_ref is required/);
await expect(
gate.createPiiPolicy({ source_id: "s1", tenant_id: "t1", project_id: "p1", legal_basis: "consent", basis_ref: "consent-record-1", retention_days: -1, field_allowlist: ["address"] }),
).rejects.toThrow(/retention_days/);
await expect(
gate.createPiiPolicy({ source_id: "s1", tenant_id: "t1", project_id: "p1", legal_basis: "contract", basis_ref: "clause-9", retention_days: 90, field_allowlist: [] }),
).rejects.toThrow(/data minimisation/);
});
it("enforces the recorded decision: legitimate_interest = B2B business contact, company-linked only", async () => {
const gate = new PiiGateService();
await expect(
gate.createPiiPolicy({ source_id: "s_li_wrong", tenant_id: "t1", project_id: "p1", legal_basis: "legitimate_interest", basis_ref: "lia-2026-001", retention_days: 90, field_allowlist: ["email"], purpose: "consent_marketing" }),
).rejects.toThrowError(/only accepted with purpose=business_contact/);
await expect(
gate.createPiiPolicy({ source_id: "s_li_wrong2", tenant_id: "t1", project_id: "p1", legal_basis: "legitimate_interest", basis_ref: "lia-2026-001", retention_days: 90, field_allowlist: ["email"], entity_linked_only: false }),
).rejects.toThrowError(/entity_linked_only=true/);
await gate.createPiiPolicy({
source_id: "s_li_ok", tenant_id: "t1", project_id: "p1",
legal_basis: "legitimate_interest", basis_ref: "lia-2026-b2b-01",
retention_days: 365, field_allowlist: ["email", "phone"],
});
const p = await gate.getPiiPolicy("s_li_ok");
expect(p?.purpose).toBe("business_contact");
expect(p?.entity_linked_only).toBe(true);
// Business contact must be bound to a company — stray person-only rows are refused.
const liSource = { ...company, source_id: "s_li_ok" };
await expect(
gate.recordStored({ source: liSource, tenantId: "t1", identityKey: "k", subjectId: "s", field: "email" }),
).rejects.toThrowError(/entity linkage/);
const linked = await gate.recordStored({ source: liSource, tenantId: "t1", identityKey: "k", subjectId: "s", field: "email", entityId: "556677-8899" });
expect(linked.entity_hash).toMatch(/^[0-9a-f]{64}$/);
expect(linked.entity_hash).not.toContain("556677");
});
it("blocks out-of-allowlist fields (minimisation) and unconfirmed web sources", async () => {
const gate = new PiiGateService();
await gate.createPiiPolicy({
source_id: ratsit.source_id, tenant_id: "t1", project_id: "p1",
legal_basis: "consent", basis_ref: "consent-record-42",
retention_days: 90, field_allowlist: ["name", "address"],
});
await expect(gate.assertCanStore(ratsit, ["name", "phone"], "t1")).rejects.toThrowError(/outside the allowlist/);
await expect(gate.assertCanStore(ratsit, ["name"], "t1")).rejects.toThrowError(/robots.txt \/ rate-limit not confirmed/);
const ratsitConfirmed = { ...ratsit, source_id: "src_ratsit_confirmed" };
await gate.createPiiPolicy({
source_id: ratsitConfirmed.source_id, tenant_id: "t1", project_id: "p1",
legal_basis: "consent", basis_ref: "consent-record-42",
retention_days: 90, field_allowlist: ["name", "address"], robots_and_rate_confirmed: true,
});
await expect(gate.assertCanStore(ratsitConfirmed, ["name"], "t1")).resolves.toBeUndefined();
});
it("forbids any storage when retention_days = 0", async () => {
const gate = new PiiGateService();
await gate.createPiiPolicy({
source_id: "src_drop_1", tenant_id: "t1", project_id: "p1",
legal_basis: "legitimate_interest", basis_ref: "lia-2026-001",
retention_days: 0, field_allowlist: ["name"], robots_and_rate_confirmed: true,
});
await expect(
gate.assertCanStore({ ...company, source_id: "src_drop_1", name: "ratsit.se" }, ["name"], "t1"),
).rejects.toThrowError(/forbids retention/);
});
it("stores ruled bases and enables access/erasure without plaintext identity", async () => {
const gate = new PiiGateService();
await gate.createPiiPolicy({
source_id: ratsit.source_id, tenant_id: "t1", project_id: "p1",
legal_basis: "legal_obligation", basis_ref: "prop-§5",
retention_days: 30, field_allowlist: ["street_address"], robots_and_rate_confirmed: true,
});
const key = await gate.recordStored({
source: ratsit, tenantId: "t1",
identityKey: "20250101-1234", subjectId: "subj-77", field: "street_address", artifactRef: "st0re/dataset-a",
});
expect(key.key_hash).toMatch(/^[0-9a-f]{64}$/);
expect(key.key_hash).not.toContain("20250101");
const statement = await gate.accessStatement(key.subject_hash);
expect(statement.rows).toHaveLength(1);
expect(statement.rows[0].field).toBe("street_address");
const erased = await gate.erase({ subject_hash: key.subject_hash, reason: "deleted_by_subject" });
expect(erased).toBe(1);
expect((await gate.accessStatement(key.subject_hash)).rows).toHaveLength(0);
});
it("retention worker purges rows past their deadline", async () => {
const gate = new PiiGateService();
vi.useFakeTimers();
vi.setSystemTime(new Date("2026-08-01T00:00:00Z"));
await gate.createPiiPolicy({
source_id: ratsit.source_id, tenant_id: "t1", project_id: "p1",
legal_basis: "legitimate_interest", basis_ref: "lia-2026-002",
retention_days: 30, field_allowlist: ["street_address"], robots_and_rate_confirmed: true,
});
await gate.recordStored({ source: ratsit, tenantId: "t1", identityKey: "k-1", subjectId: "s-1", field: "street_address", entityId: "5566778899" });
vi.setSystemTime(new Date("2026-09-15T00:00:00Z"));
const { purged } = await gate.enforceRetention(new Date("2026-09-15T00:00:00Z"));
vi.useRealTimers();
expect(purged).toBe(1);
});
it("redacts personnummer, e-mail and phone from log lines", () => {
const gate = new PiiGateService();
const out = gate.redact(
"subject=202501011234 res=" +
'evt=19720101-1234 mail=simon@siax.io tel=070-123 45 67 sök=ok',
);
expect(out).not.toMatch(/\d{8}-\d{4}/);
expect(out).toContain("[PERSONNUMMER]");
expect(out).toContain("[EMAIL]");
expect(out).toContain("[TELEFON]");
});
it("SourceQueryService blocks a personal-data source by default (fail-closed)", async () => {
const sourceService = new SourceService();
await sourceService.createSource({ ...ratsit });
const service = new SourceQueryService({ sourceService });
await expect(
service.querySource({
source_id: ratsit.source_id,
query: "SELECT 1",
tenant_id: "t1", project_id: "p1",
connection_config: EMPTY_CFG,
}),
).rejects.toThrowError(/no PiiPolicy/);
});
it("SourceQueryService passes non-personal sources straight through to capability checks", async () => {
const sourceService = new SourceService();
await sourceService.createSource({
source_id: "src_safe_1", tenant_id: "t1", project_id: "p1",
source_type: "web", name: "example.org", status: "active",
});
const service = new SourceQueryService({ sourceService });
// Gate passes; the sync path then correctly rejects non-database sources.
await expect(
service.querySource({
source_id: "src_safe_1",
query: "SELECT 1",
tenant_id: "t1", project_id: "p1",
connection_config: EMPTY_CFG,
}),
).rejects.toThrowError(/source_type "web"/);
});
});
@@ -24,10 +24,24 @@
* prove the rows are really there, not an artifact of the service's own
* in-process bookkeeping.
*
* beforeAll applies sql/002_source_registry.sql itself (every statement is
* `CREATE TABLE/INDEX IF NOT EXISTS`, so this is idempotent and safe to run
* against an already-migrated database too) -- this test does not require
* the operator to have pre-run migrations.
* beforeAll applies every sql/NNN_*.sql file in order (same restore-drill
* pattern as tests/analytics/analytics-domain-postgres.realinfra.test.ts),
* not just sql/002_source_registry.sql -- this test does not require the
* operator to have pre-run migrations.
*
* CORRECTED (this pass, 2026-09-14): it used to apply ONLY
* sql/002_source_registry.sql. That was enough the day this file was
* written, but sql/007_ingestion_plane.sql later added `opt_sources.owner_id`
* (`ALTER TABLE opt_sources ADD COLUMN IF NOT EXISTS owner_id TEXT`), and
* src/repositories/postgres-source-repository.ts's INSERT has named that
* column ever since. Because this env var is never set in CI (confirmed by
* grep over .gitea/workflows/ci.yml — no job references
* SOURCE_REGISTRY_TEST_DATABASE_URL), the drift was never caught: every
* assertion below failed with `column "owner_id" of relation "opt_sources"
* does not exist` the moment it was actually run against a real, freshly
* migrated-to-002-only Postgres. The fix is to stop hand-picking one
* migration file and replay the whole chain, exactly like the analytics
* suite already does.
*/
import { afterAll, beforeAll, describe, expect, it } from "vitest";
import fs from "node:fs";
@@ -56,9 +70,23 @@ beforeAll(async () => {
adminSql`SELECT 1`,
new Promise((_, reject) => setTimeout(() => reject(new Error("connect timeout")), 5000)),
]);
const migrationPath = path.join(process.cwd(), "sql", "002_source_registry.sql");
const migration = fs.readFileSync(migrationPath, "utf8");
await adminSql.unsafe(migration);
// Replay every migration in order, not just 002 -- see the file header
// "CORRECTED" note. Tolerate an individual file failing (mirrors
// analytics-domain-postgres.realinfra.test.ts: sql/001 needs the
// `vector` extension, which a stock postgres:16-alpine does not carry)
// as long as the files this test actually depends on (002, 007) apply.
const dir = path.join(process.cwd(), "sql");
for (const file of fs
.readdirSync(dir)
.filter((f) => f.endsWith(".sql"))
.sort()) {
const body = fs.readFileSync(path.join(dir, file), "utf8");
try {
await adminSql.unsafe(body);
} catch (err) {
console.warn(`[realinfra] ${file} did not apply cleanly: ${String(err).slice(0, 200)}`);
}
}
reachable = true;
} catch (err) {
reachable = false;
+209
View File
@@ -0,0 +1,209 @@
/**
* AGENT-04 / GOV-06 -- the doctor is itself gated, and is itself proven able to fail.
*
* Two things this file exists for:
*
* 1. `pnpm siax:doctor` is otherwise invoked only by `make doctor`, i.e. by
* whoever remembers. Running it here puts it inside `pnpm run test`, which
* is the BLOCKING "CI (SIAX Cloud) / quality" job -- no .gitea/workflows
* change required (that path is protected on this repo).
*
* 2. The defect being fixed was a check that COULD NOT FAIL: the old doctor
* reported 40/40 and byte-identical output with `operationalState` set to
* "running" (false) and to "down" (true). So a green doctor is not enough
* evidence here -- the negative cases are asserted too, against a fixture
* tree via SIAX_DOCTOR_ROOT.
*
* Offline only. Nothing here touches the network: `--live` is opt-in and is
* deliberately never exercised by CI.
*/
import { afterAll, describe, expect, it } from "vitest";
import { execFileSync } from "node:child_process";
import { mkdtempSync, mkdirSync, writeFileSync, readFileSync, rmSync } from "node:fs";
import { tmpdir } from "node:os";
import { fileURLToPath } from "node:url";
import path from "node:path";
const REPO_ROOT = path.resolve(path.dirname(fileURLToPath(import.meta.url)), "..", "..");
const DOCTOR = path.join(REPO_ROOT, "scripts", "siax-doctor.mjs");
type DoctorResult = { id: string; label: string; mode: string; pass: boolean; detail: string | null };
type DoctorReport = { total: number; failed: number; live: boolean; results: DoctorResult[] };
function runDoctor(root?: string): { code: number; report: DoctorReport } {
const env = { ...process.env, ...(root ? { SIAX_DOCTOR_ROOT: root } : {}) };
try {
const stdout = execFileSync(process.execPath, [DOCTOR, "--json"], { env, encoding: "utf8" });
return { code: 0, report: JSON.parse(stdout) as DoctorReport };
} catch (e) {
const err = e as { status?: number; stdout?: string };
return { code: err.status ?? -1, report: JSON.parse(err.stdout ?? "{}") as DoctorReport };
}
}
/** A fixture tree carrying only the two files the GOV-06 checks read. */
function fixture(repo: unknown, evidence: unknown): string {
const dir = mkdtempSync(path.join(tmpdir(), "siax-doctor-fixture-"));
writeFileSync(path.join(dir, "siax.repo.json"), JSON.stringify(repo, null, 2));
mkdirSync(path.join(dir, "evidence"), { recursive: true });
writeFileSync(path.join(dir, "evidence", "operational-state.json"), JSON.stringify(evidence, null, 2));
return dir;
}
const REAL_REPO = JSON.parse(readFileSync(path.join(REPO_ROOT, "siax.repo.json"), "utf8"));
const REAL_EVIDENCE = JSON.parse(
readFileSync(path.join(REPO_ROOT, "evidence", "operational-state.json"), "utf8"),
);
/** The GOV-06 failure detail for a fixture, or null when GOV-06 passed. */
function gov06Failure(root: string): string | null {
const { report } = runDoctor(root);
const failed = report.results.filter((r) => r.id.startsWith("GOV-06") && !r.pass);
return failed.length > 0 ? failed.map((r) => r.detail).join(" | ") : null;
}
describe("AGENT-04: siax-doctor passes on this repo", () => {
it("exits 0 with zero failures", () => {
const { code, report } = runDoctor();
const failures = report.results.filter((r) => !r.pass);
expect(failures.map((f) => `${f.id}: ${f.detail}`).join("\n")).toBe("");
expect(report.failed).toBe(0);
expect(code).toBe(0);
});
it("runs offline by default -- CI must never depend on reaching a host", () => {
const { report } = runDoctor();
expect(report.live).toBe(false);
expect(report.results.every((r) => r.mode === "offline")).toBe(true);
});
it("still carries the truth checks (they cannot be quietly deleted)", () => {
const { report } = runDoctor();
const ids = new Set(report.results.map((r) => r.id));
for (const id of ["GOV-06", "NET-01", "NET-01/REPO-06", "SEC-06/P0-13", "REPO-04/GOV-06"]) {
expect(ids.has(id), `siax-doctor no longer runs any "${id}" check`).toBe(true);
}
});
});
describe("GOV-06: a FALSE operationalState is detected (the historical defect)", () => {
const dirs: string[] = [];
const track = (d: string) => {
dirs.push(d);
return d;
};
afterAll(() => {
for (const d of dirs) rmSync(d, { recursive: true, force: true });
});
it("accepts the repo's real, honest pairing (down + 402/404) as the control", () => {
expect(gov06Failure(track(fixture(REAL_REPO, REAL_EVIDENCE)))).toBeNull();
});
it('rejects operationalState "running" when the evidence was recorded for another state', () => {
const detail = gov06Failure(track(fixture({ ...REAL_REPO, operationalState: "running" }, REAL_EVIDENCE)));
expect(detail).toMatch(/running/);
expect(detail).toMatch(/operational-state\.json/);
});
it('rejects operationalState "running" when the recorded probes are 402/404', () => {
const detail = gov06Failure(
track(
fixture(
{ ...REAL_REPO, operationalState: "running" },
{ ...REAL_EVIDENCE, operationalState: "running" },
),
),
);
expect(detail).toMatch(/but the recorded probes say otherwise/);
expect(detail).toMatch(/402|404/);
});
it("rejects an `ok: true` hand-written next to a non-2xx status", () => {
const detail = gov06Failure(
track(
fixture(
{ ...REAL_REPO, operationalState: "running" },
{
...REAL_EVIDENCE,
operationalState: "running",
probes: (REAL_EVIDENCE.probes as { ok: boolean }[]).map((p) => ({ ...p, ok: true })),
},
),
),
);
expect(detail).toMatch(/records ok=true next to httpStatus/);
});
it('rejects operationalState "down" when a prod host is recorded as answering', () => {
const detail = gov06Failure(
track(
fixture(REAL_REPO, {
...REAL_EVIDENCE,
probes: (REAL_EVIDENCE.probes as { httpStatus: number; ok: boolean }[]).map((p) => ({
...p,
httpStatus: 200,
ok: true,
})),
}),
),
);
expect(detail).toMatch(/"down" but these prod hosts answered/);
});
it("rejects a liveness claim backed only by stale evidence", () => {
const stale = new Date(Date.now() - 400 * 86_400_000).toISOString();
const detail = gov06Failure(
track(
fixture(
{ ...REAL_REPO, operationalState: "running" },
{
...REAL_EVIDENCE,
operationalState: "running",
checkedAt: stale,
probes: (REAL_EVIDENCE.probes as { httpStatus: number; ok: boolean }[]).map((p) => ({
...p,
httpStatus: 200,
ok: true,
})),
},
),
),
);
expect(detail).toMatch(/unrefreshed liveness claim/);
});
it("rejects a prod hostname with no probe on record at all", () => {
const detail = gov06Failure(
track(
fixture(
{
...REAL_REPO,
domains: [
...(REAL_REPO.domains as unknown[]),
{ fqdn: "invented.0pt.app", env: "prod", purpose: "x", registrar: "x" },
],
},
REAL_EVIDENCE,
),
),
);
expect(detail).toMatch(/no probe recorded for: invented\.0pt\.app/);
});
it("rejects a malformed hostname (NET-01)", () => {
const { report } = runDoctor(
track(
fixture(
{
...REAL_REPO,
domains: [{ fqdn: "https://0pt.app/health", env: "prod", purpose: "x", registrar: "x" }],
},
REAL_EVIDENCE,
),
),
);
const net01 = report.results.find((r) => r.id === "NET-01" && !r.pass);
expect(net01?.detail).toMatch(/not a bare lowercase hostname/);
});
});
+184
View File
@@ -0,0 +1,184 @@
/**
* BACKLOG P0-13 / SEC-06 -- security-header REGRESSION GATE.
*
* Why this file exists: CHANGELOG.md asserted "security headers
* (CSP/HSTS/X-Frame-Options/etc.)" while `main` contained zero such code for
* weeks, and nothing in the repo could tell the difference. Adding the headers
* without adding a gate would just reset that clock.
*
* WHAT THIS PROVES: that `next.config.mjs` -- the object Next.js actually
* consumes -- still declares the required headers, with values that are still
* strict. It reads the real config module; it does not re-import the header
* list and compare it to itself. Every expectation below is written out here,
* independently, so deleting or weakening a header in next.config.mjs fails
* this test rather than silently changing both sides.
*
* WHAT IT DOES NOT PROVE: that a running instance serves them. Next freezes
* `headers()` into .next/routes-manifest.json at build time, so the manifest
* assertion at the bottom runs opportunistically when a build exists (locally,
* or in any job that builds before testing). End-to-end, against a live
* origin, is `node scripts/siax-doctor.mjs --live`.
*
* Runs inside `pnpm run test` -> the BLOCKING "CI (SIAX Cloud) / quality" job.
*/
import { describe, expect, it } from "vitest";
import { existsSync, readFileSync } from "node:fs";
import { fileURLToPath } from "node:url";
import path from "node:path";
const REPO_ROOT = path.resolve(path.dirname(fileURLToPath(import.meta.url)), "..", "..");
type HeaderEntry = { key: string; value: string };
type HeaderRule = { source: string; headers: HeaderEntry[] };
/**
* Required headers and how strict each must stay. Hardcoded on purpose --
* this is the independent side of the comparison.
*/
const REQUIRED: { key: string; assert: (value: string) => void }[] = [
{
key: "Strict-Transport-Security",
assert: (v) => {
const maxAge = /max-age=(\d+)/.exec(v);
expect(maxAge, `HSTS must carry a max-age, got: ${v}`).not.toBeNull();
// >= 1 year: below that the header is decorative and ineligible for preload.
expect(Number(maxAge![1])).toBeGreaterThanOrEqual(31536000);
expect(v).toMatch(/includeSubDomains/i);
},
},
{
key: "X-Frame-Options",
// SAMEORIGIN would be a weakening for this app: nothing here frames itself.
assert: (v) => expect(v.trim().toUpperCase()).toBe("DENY"),
},
{
key: "X-Content-Type-Options",
assert: (v) => expect(v.trim().toLowerCase()).toBe("nosniff"),
},
{
key: "Referrer-Policy",
assert: (v) =>
expect(["no-referrer", "same-origin", "strict-origin", "strict-origin-when-cross-origin"]).toContain(
v.trim().toLowerCase(),
),
},
{
key: "Permissions-Policy",
assert: (v) => {
for (const feature of ["camera", "microphone", "geolocation"]) {
expect(v, `Permissions-Policy must disable ${feature}`).toMatch(
new RegExp(`${feature}\\s*=\\s*\\(\\s*\\)`),
);
}
},
},
{
key: "Content-Security-Policy",
assert: (v) => {
const directives = new Map(
v
.split(";")
.map((d) => d.trim())
.filter(Boolean)
.map((d) => {
const [name, ...rest] = d.split(/\s+/);
return [name.toLowerCase(), rest.join(" ")] as const;
}),
);
expect(directives.get("default-src")).toBe("'self'");
// Clickjacking defence that actually applies to modern browsers.
expect(directives.get("frame-ancestors")).toBe("'none'");
expect(directives.get("base-uri")).toBe("'self'");
expect(directives.get("object-src")).toBe("'none'");
expect(directives.get("form-action")).toBe("'self'");
expect(directives.has("script-src")).toBe(true);
expect(directives.has("connect-src")).toBe(true);
expect(directives.has("frame-src")).toBe(true);
// No wildcard sources anywhere: a bare `*` re-opens what this header exists to close.
for (const [name, value] of directives) {
expect(value.split(/\s+/), `CSP directive ${name} must not allow *`).not.toContain("*");
}
// 'unsafe-eval' is a dev-only relaxation (see next.config.mjs). Vitest runs
// with NODE_ENV=test, so it must be absent here; if it ever appears outside
// `next dev`, this fails.
expect(process.env.NODE_ENV, "vitest is expected to run outside NODE_ENV=development").not.toBe(
"development",
);
expect(v, "'unsafe-eval' must never ship outside next dev").not.toContain("unsafe-eval");
},
},
];
/** Rule sources that cover every path Next can serve. */
const CATCH_ALL_SOURCES = new Set(["/(.*)", "/:path*", "/(.*)?"]);
async function loadNextConfig(): Promise<{ poweredByHeader?: boolean; headers?: () => Promise<HeaderRule[]> }> {
// Computed specifier on purpose: keeps `next.config.mjs` (a JS file outside
// tsconfig.check.json's include) out of the blocking typecheck's graph.
const url = new URL("../../next.config.mjs", import.meta.url).href;
const mod = (await import(/* @vite-ignore */ url)) as { default: Awaited<ReturnType<typeof loadNextConfig>> };
return mod.default;
}
describe("SEC-06: 0pt serves its own security headers (P0-13)", () => {
it("next.config.mjs declares a headers() rule covering every path", async () => {
const config = await loadNextConfig();
expect(typeof config.headers, "next.config.mjs must export an async headers()").toBe("function");
const rules = await config.headers!();
expect(Array.isArray(rules)).toBe(true);
const catchAll = rules.filter((r) => CATCH_ALL_SOURCES.has(r.source));
expect(
catchAll.length,
`no catch-all header rule; sources were: ${rules.map((r) => r.source).join(", ")}`,
).toBeGreaterThan(0);
});
it("does not advertise the framework via X-Powered-By", async () => {
const config = await loadNextConfig();
expect(config.poweredByHeader).toBe(false);
});
for (const { key, assert } of REQUIRED) {
it(`sets ${key} on every path, and keeps it strict`, async () => {
const config = await loadNextConfig();
const rules = await config.headers!();
const applied = rules
.filter((r) => CATCH_ALL_SOURCES.has(r.source))
.flatMap((r) => r.headers)
.filter((h) => h.key.toLowerCase() === key.toLowerCase());
expect(
applied.length,
`${key} is not set for all paths -- CHANGELOG.md claims 0pt ships it. ` +
`Either restore it in next.config.mjs or stop claiming it.`,
).toBe(1);
assert(applied[0].value);
});
}
it("the built routes-manifest carries the same headers (skipped when no build exists)", async () => {
const manifestPath = path.join(REPO_ROOT, ".next", "routes-manifest.json");
if (!existsSync(manifestPath)) {
// `pnpm run test` runs before `pnpm run build` in .gitea/workflows/ci.yml,
// so on a clean CI checkout there is nothing to read. Not a silent pass:
// the config-level assertions above already ran and are blocking.
return;
}
const manifest = JSON.parse(readFileSync(manifestPath, "utf8")) as {
headers?: { source?: string; regex?: string; headers?: HeaderEntry[] }[];
};
const built = (manifest.headers ?? []).flatMap((r) => r.headers ?? []);
expect(built.length, "a build exists but its routes-manifest has no headers").toBeGreaterThan(0);
for (const { key } of REQUIRED) {
expect(
built.some((h) => h.key.toLowerCase() === key.toLowerCase()),
`${key} missing from .next/routes-manifest.json -- it would not be served`,
).toBe(true);
}
});
});