From 12080fc5171201e17ed06c9bbb5c95f357cff997 Mon Sep 17 00:00:00 2001 From: Simon Date: Mon, 31 Aug 2026 01:15:37 +0200 Subject: [PATCH 1/4] feat(0pt): PII-Gate GDPR-guardrail for personal-data sources (fail-closed) PiiGateService makes any future personal-data source (ratsit.se and other directory domains) technically GDPR-OK before the first byte is stored: - recorded legal basis (art. 6) with basis_ref evidence, no policy => fail-closed - data minimisation via field allowlist (art. 5(1)(c)) - storage limitation + enforceRetention purge (art. 5(1)(e)) - sha256-only identity keys + log redaction (art. 25/32) - accessStatement / erase subject-rights surfaces (art. 15/17) - robots/rate confirmation required for web+api sources; no stealth/bypass - audit trail via AUD0 at risk L3 (0pt.pii_*) - gate enabled by default in SourceQueryService 10 new tests; full suite 166 pass | 1 skip; typecheck (blocking gate) clean. --- docs/compliance/PII_GUARDRAIL.md | 58 +++++ src/services/pii-gate-service.ts | 324 +++++++++++++++++++++++++++ src/services/source-query-service.ts | 14 +- src/types/pii.ts | 102 +++++++++ src/types/source.ts | 8 + tests/p0/pii-gate.test.ts | 173 ++++++++++++++ 6 files changed, 678 insertions(+), 1 deletion(-) create mode 100644 docs/compliance/PII_GUARDRAIL.md create mode 100644 src/services/pii-gate-service.ts create mode 100644 src/types/pii.ts create mode 100644 tests/p0/pii-gate.test.ts diff --git a/docs/compliance/PII_GUARDRAIL.md b/docs/compliance/PII_GUARDRAIL.md new file mode 100644 index 0000000..78c82df --- /dev/null +++ b/docs/compliance/PII_GUARDRAIL.md @@ -0,0 +1,58 @@ +# PII Guardrail — GDPR requirement for personal-data sources (0pt) + +Status: **enforced, fail-closed.** No 0pt pipeline is crawling personuppgifter +today; the `PiiGateService` ships so that any future source that touches +personal data (Ratsit, Merinfo, Hitta, …) is **technically GDPR-OK before the +first byte is stored**. + +## The rule + +1. **Classify.** A source is personal-data if it self-flags + (`touches_personal_data`) or is a known directory domain (ratsit.se, + merinfo.se, hitta.se, mrkoll.se, eniro.se, birthday.se, upplysning.se). +2. **No policy → no processing.** `assertCanStore()` throws unless a + `PiiPolicy` with a valid `legal_basis` + `basis_ref` is on record. +3. **Everything is evidenced.** Each decision is emitted to AUD0 at risk L3 + (`0pt.pii_policy_created | pii_data_retained | pii_erased | + pii_retention_purged`). + +## GDPR article mapping + +| Requirement | Implementation | +|---|---| +| Legal basis — art. 6 | `legal_basis` ∈ legitimate_interest/consent/contract/legal_obligation + `basis_ref` (LIA doc id, consent record id, contract clause, provision). Cannot be empty. | +| Data minimisation — art. 5(1)(c) | `field_allowlist`; storing any other field throws `PiiMinimisationError`. Empty allowlist = nothing stored. | +| Storage limitation — art. 5(1)(e) | `retention_days`; `enforceRetention()` purges expired rows. `0` = fetch-and-drop, nothing persisted. | +| Integrity/confidentiality — art. 32 | Only sha256 hashes of identity keys stored (`recordStored`); plaintext identity never written; log/trace redaction (`redact()`) for personnummer/e-mail/phone. | +| Right of access — art. 15 | `accessStatement(subjectHash)` — what is held on a subject, across sources. | +| Right to erasure — art. 17 | `erase({ subject_hash, reason })` tombstones every stored value for the subject. | +| Data protection by design — art. 25 | Gate defaults ON in `SourceQueryService`; no bypass knobs. | + +## When someone requests a personal-data source (e.g. Ratsit) + +- [x] Confirm the source is actually needed and no anonymised/official + alternative exists (official register > directory listing). +- [x] Write the **Legitimate Interest Assessment** (or record consent / + contract / legal obligation) and store its id in `basis_ref`. + LIA minimum: purpose, necessity, individuals' rights impact, mitigations. +- [x] **VERIFY current terms + robots.txt + availability** on the live site. + Do not hardcode "catsit.com allows" — re-check each enable. +- [x] Set `robots_and_rate_confirmed: true` + a `SourceRatePolicy` matching + the source's own limits (legacy N0D default for `ratsit.se` was `1/min` — + treat that as a ceiling, not a target). +- [x] Define the minimum `field_allowlist` (e.g. only `street_address`, not + `phone`). +- [x] Invoke `recordStored` for every stored value and run `enforceRetention` + on a schedule (e.g. daily cron). +- [x] Record a DPIA note if processing is "large scale" (art. 35) — bulk + directory scraping of living persons is large-scale by default. + +## Never + +- Pretend a source has a legal basis it does not have; `basis_ref` must point + to a real artifact. +- Store identity keys in plaintext (`pseudonymize_keys` defaults to true). +- Use stealth/bot-bypass/rate-evasion — excluded by CLAUDE.md §1 regardless of + GDPR. +- Claim compliance on a source we have not re-verified this season (`VERIFY` + markers are intentional). \ No newline at end of file diff --git a/src/services/pii-gate-service.ts b/src/services/pii-gate-service.ts new file mode 100644 index 0000000..da84d99 --- /dev/null +++ b/src/services/pii-gate-service.ts @@ -0,0 +1,324 @@ +/** + * src/services/pii-gate-service.ts + * ================================== + * 0pt.pii_* — the GDPR guardrail (fail-closed). Enforced by default wherever + * a SourceQueryService is constructed; consulted by workers before they + * persist anything for a personal-data source. + * + * Owns 0pt.pii_policy.schema.v1 and the in-memory identity registry that + * backs the subject rights surfaces (access/erasure, GDPR art. 15/17) and + * the retention worker (GDPR art. 5(1)(e)). Every decision is emitted to + * AUD0 as risk L3 events so CL0UD gets an audit trail without polling 0pt. + * + * Directory domains known to publish personuppgifter (ratsit.se, …) are + * pre-classified as personal: until a PiiPolicy with a recorded lawful + * basis exists for them, the gate REJECTS storage/query of those sources. + */ + +import { createHash } from "node:crypto"; +import type { + PiiPolicy, + CreatePiiPolicyInput, + PiiIdentityKey, + PiiErasureRequest, + PiiAccessStatement, +} from "../types/pii"; +import type { Source } from "../types/source"; +import { createAud0Client, type Aud0Client, type Aud0Event } from "../clients/aud0-client"; + +/** Domain publishes personuppgifter (directory/register listings). */ +const KNOWN_DIRECTORY_DOMAINS = [ + "ratsit.se", + "merinfo.se", + "hitta.se", + "mrkoll.se", + "eniro.se", + "birthday.se", + "upplysning.se", +]; + +const DECISION_TEMPLATE = "PII source %s: %s"; + +export class PiiPolicyExistsError extends Error {} +export class PiiLegalBasisError extends Error {} +export class PiiMinimisationError extends Error {} +export class PiiRetentionError extends Error {} +export class PiiRobotsRateError extends Error {} + +export class PiiGateService { + private policies: Map = new Map(); + private registry: PiiIdentityKey[] = []; + private aud0: Aud0Client; + + constructor(opts?: { aud0BaseUrl?: string }) { + this.aud0 = createAud0Client(opts?.aud0BaseUrl); + } + + /** True if the source either self-flags or is a known directory domain. */ + classifySourcePersonal(source: Pick): boolean { + if (source.touches_personal_data === true) return true; + if (source.source_type !== "web" && source.source_type !== "api") return false; + const host = (source.name || "").toLowerCase().replace(/^https?:\/\//, "").replace(/\/.*$/, ""); + return KNOWN_DIRECTORY_DOMAINS.some((d) => host === d || host.endsWith("." + d)); + } + + /* ------------------------------------------------------------------ * + * Policies + * ------------------------------------------------------------------ */ + + async createPiiPolicy(input: CreatePiiPolicyInput): Promise { + if (this.policies.has(input.source_id)) { + throw new Error(`PiiPolicy already exists for source ${input.source_id}`); + } + // Hard validation — a policy is the lawful-basis gate, so it cannot be + // written with a half-legal posture. + if (!input.basis_ref.trim()) throw new Error("basis_ref is required (LIA id, consent id, contract clause or legal provision)"); + if (!Number.isFinite(input.retention_days) || input.retention_days < 0) { + throw new Error("retention_days must be a non-negative integer (0 = fetch-and-drop)"); + } + if (!Array.isArray(input.field_allowlist) || input.field_allowlist.length === 0) { + throw new Error("field_allowlist must contain at least one field (data minimisation)"); + } + + const now = new Date().toISOString(); + const policy: PiiPolicy = { + source_id: input.source_id, + tenant_id: input.tenant_id, + project_id: input.project_id, + touches_personal_data: true, + legal_basis: input.legal_basis, + basis_ref: input.basis_ref, + retention_days: input.retention_days, + field_allowlist: [...input.field_allowlist], + pseudonymize_keys: input.pseudonymize_keys ?? true, + robots_and_rate_confirmed: input.robots_and_rate_confirmed ?? false, + created_at: now, + updated_at: null, + created_by: input.created_by ?? null, + }; + this.policies.set(input.source_id, policy); + + await this.safeEmit({ + event_id: `evt_0pt_pii_policy_created_${input.source_id}`, + event_type: "0pt.pii_policy_created", + capability_id: "0pt.pii_policy_created", + tenant_id: input.tenant_id, + project_id: input.project_id, + actor_id: input.created_by ?? "system", + resource_type: "source_pii_policy", + resource_id: input.source_id, + risk_level: "L3", + payload: { legal_basis: input.legal_basis, retention_days: input.retention_days, field_count: input.field_allowlist.length }, + }); + return policy; + } + + async getPiiPolicy(sourceId: string): Promise { + return this.policies.get(sourceId) ?? null; + } + + async listPiiPolicies(tenantId?: string): Promise { + const all = Array.from(this.policies.values()); + return tenantId ? all.filter((p) => p.tenant_id === tenantId) : all; + } + + /* ------------------------------------------------------------------ * + * Enforcement (fail-closed) + * ------------------------------------------------------------------ */ + + /** + * Gate for a store/query of `fields` against `source`. Throws unless the + * source is non-personal or a lawful, minimised, retention-bounded, + * robots/rate-confirmed policy is on record. + */ + async assertCanStore( + source: Pick, + fields: string[], + tenantId: string, + ): Promise { + if (!this.classifySourcePersonal(source)) return; + + const policy = this.policies.get(source.source_id); + if (!policy) { + throw new PiiLegalBasisError( + DECISION_TEMPLATE.replace("%s", source.source_id).replace( + "%s", + `is a personal-data source with no PiiPolicy on record — a lawful basis (${KNOWN_DIRECTORY_DOMAINS.includes(source.name || "") ? "including robots.txt + rate limit confirmation" : "art. 6 GDPR"}) must be registered before any processing`, + ), + ); + } + + if (policy.retention_days === 0 && fields.length > 0) { + throw new PiiRetentionError( + DECISION_TEMPLATE.replace("%s", source.source_id).replace( + "%s", + "policy forbids retention (retention_days = 0); no fields may be stored", + ), + ); + } + + const disallowed = fields.filter((f) => !policy.field_allowlist.includes(f)); + if (disallowed.length > 0) { + throw new PiiMinimisationError( + DECISION_TEMPLATE.replace("%s", source.source_id).replace( + "%s", + `field(s) outside the allowlist: ${disallowed.join(", ")} (GDPR art. 5(1)(c) minimisation)`, + ), + ); + } + + if ((source.source_type === "web" || source.source_type === "api") && !policy.robots_and_rate_confirmed) { + throw new PiiRobotsRateError( + DECISION_TEMPLATE.replace("%s", source.source_id).replace( + "%s", + "robots.txt / rate-limit not confirmed — required for web & api personal-data sources; re-check the source terms (do not hardcode)", + ), + ); + } + } + + /* ------------------------------------------------------------------ * + * Subject rights: access (art. 15) & erasure (art. 17), retention (art. 5) + * ------------------------------------------------------------------ */ + + /** + * Register that one value was stored. Only hashes are persisted; the + * plaintext identity never reaches the registry. + */ + async recordStored(input: { + source: Pick; + tenantId: string; + identityKey: string; + subjectId: string; + field: string; + artifactRef?: string; + }): Promise { + await this.assertCanStore(input.source, [input.field], input.tenantId); + const policy = this.policies.get(input.source.source_id)!; + + const now = Date.now(); + const key: PiiIdentityKey = { + key_hash: hash256(`${sourceIdSalt(input.source.source_id)}:${input.identityKey}`), + subject_hash: hash256(input.subjectId), + source_id: input.source.source_id, + tenant_id: input.tenantId, + field: input.field, + artifact_ref: input.artifactRef ?? null, + stored_at: new Date(now).toISOString(), + expires_at: new Date(now + policy.retention_days * 86_400_000).toISOString(), + status: "active", + }; + this.registry.push(key); + + await this.safeEmit({ + event_id: `evt_0pt_pii_retained_${input.source.source_id}_${key.key_hash.slice(0, 12)}`, + event_type: "0pt.pii_data_retained", + capability_id: "0pt.pii_retain_identity", + tenant_id: input.tenantId, + project_id: "", + actor_id: "system", + resource_type: "pii_identity_key", + resource_id: key.key_hash, + risk_level: "L3", + payload: { source_id: input.source.source_id, field: input.field, expires_at: key.expires_at }, + }); + return key; + } + + /** Right of access — everything currently held on a subject across sources. */ + async accessStatement(subjectHash: string): Promise { + const rows = this.registry.filter((r) => r.subject_hash === subjectHash && r.status === "active"); + return { subject_hash: subjectHash, rows, generated_at: new Date().toISOString() }; + } + + /** Right to erasure — tombstone every held row for the subject. */ + async erase(request: PiiErasureRequest): Promise { + let count = 0; + for (const row of this.registry) { + if (row.subject_hash === request.subject_hash && row.status === "active") { + row.status = "erased"; + count += 1; + } + } + await this.safeEmit({ + event_id: `evt_0pt_pii_erased_${request.subject_hash.slice(0, 12)}_${Date.now()}`, + event_type: "0pt.pii_erased", + capability_id: "0pt.pii_erase", + tenant_id: "", + project_id: "", + actor_id: request.actor_id ?? "system", + resource_type: "pii_subject", + resource_id: request.subject_hash, + risk_level: "L3", + payload: { reason: request.reason, row_count: count }, + }); + return count; + } + + /** Storage limitation — purge rows past their retention deadline. */ + async enforceRetention(now: Date = new Date()): Promise<{ purged: number }> { + let purged = 0; + const keep: PiiIdentityKey[] = []; + for (const row of this.registry) { + if (row.expires_at <= now.toISOString() && row.status === "active") { + row.status = "purged"; + purged += 1; + } else { + keep.push(row); + } + } + if (purged > 0) { + await this.safeEmit({ + event_id: `evt_0pt_pii_retention_${now.getTime()}`, + event_type: "0pt.pii_retention_purged", + capability_id: "0pt.pii_enforce_retention", + tenant_id: "", + project_id: "", + actor_id: "system", + resource_type: "pii_identity_key", + resource_id: now.toISOString(), + risk_level: "L3", + payload: { row_count: purged }, + }); + } + return { purged }; + } + + /* ------------------------------------------------------------------ * + * Log hygiene — no personuppgifter in logs/metrics/traces + * ------------------------------------------------------------------ */ + + /** Scrub personnummer, e-mail and Swedish phone numbers from a log line. */ + redact(line: string): string { + return line + .replace(/\b\d{8}[-–]\d{4}\b/g, "[PERSONNUMMER]") + .replace(/\b\d{6}[-–]\d{4}\b/g, "[PERSONNUMMER]") + .replace(/\b[A-Za-z0-9._%+-]+@[A-Za-z0-9.-]+\.[A-Za-z]{2,}\b/g, "[EMAIL]") + .replace(/(?:\+46|0046|0)\s?[1-9][\d\s.-]{7,13}/g, "[TELEFON]"); + } + + /* ------------------------------------------------------------------ */ + + private async safeEmit(event: Omit): Promise { + const full: Aud0Event = { + ...event, + app_id: "0pt", + created_at: new Date().toISOString(), + }; + try { + await this.aud0.ingestEvent(full); + } catch { + // Audit is important but must never take down a compliance decision or + // a retention pass. The caller's state change has already happened. + } + } +} + +function sourceIdSalt(sourceId: string): string { + return `0pt-pii-v1:${sourceId}`; +} + +/** sha256 hex digest — never the plaintext. */ +export function hash256(value: string): string { + return createHash("sha256").update(value, "utf8").digest("hex"); +} \ No newline at end of file diff --git a/src/services/source-query-service.ts b/src/services/source-query-service.ts index 4746963..e5f9ee1 100644 --- a/src/services/source-query-service.ts +++ b/src/services/source-query-service.ts @@ -20,6 +20,7 @@ import type { Source } from "../types/source"; import { SourceService } from "./source-service"; +import { PiiGateService } from "./pii-gate-service"; import { createAud0Client, type Aud0Client } from "../clients/aud0-client"; import { FabricOneLakeAdapter, type FabricOneLakeConnectionConfig, type AdapterQueryResult } from "../adapters/fabric-onelake-adapter"; @@ -76,16 +77,27 @@ function clamp(value: number, min: number, max: number): number { export class SourceQueryService { private sourceService: SourceService; private aud0: Aud0Client; + /** GDPR guardrail — on by default, can be shared via opts so callers use one registry. */ + private piiGate: PiiGateService; - constructor(opts?: { sourceService?: SourceService; aud0BaseUrl?: string }) { + constructor(opts?: { sourceService?: SourceService; aud0BaseUrl?: string; piiGate?: PiiGateService }) { // Accepts an existing SourceService so callers (e.g. the API route) share // one in-memory registry instead of each constructing an empty one. this.sourceService = opts?.sourceService ?? new SourceService(); this.aud0 = createAud0Client(opts?.aud0BaseUrl); + this.piiGate = opts?.piiGate ?? new PiiGateService({ aud0BaseUrl: opts?.aud0BaseUrl }); } async querySource(input: QuerySourceInput): Promise { const startedAt = Date.now(); + + // GDPR fail-closed: personal-data sources (self-flagged or known directory + // domains such as ratsit.se) may not be processed without a PiiPolicy. + // Runs BEFORE the capability/type gate so the lawful-basis error is the + // one the caller sees for personal data. + const preflight = await this.sourceService.getSource(input.source_id); + await this.piiGate.assertCanStore(preflight, [], input.tenant_id); + const source = await this.getQueryableSource(input.source_id); const maxRows = clamp(input.max_rows ?? SOURCE_QUERY_DEFAULT_MAX_ROWS, 1, SOURCE_QUERY_HARD_MAX_ROWS); diff --git a/src/types/pii.ts b/src/types/pii.ts new file mode 100644 index 0000000..606b07b --- /dev/null +++ b/src/types/pii.ts @@ -0,0 +1,102 @@ +/** + * src/types/pii.ts + * ================= + * 0pt.pii_policy.schema.v1 — SIAX Cloud GDPR guardrail for sources that + * CAN touch personuppgifter (personal data). Purpose: make processing of + * personal data "tekniskt OK" per GDPR (art. 5, 6, 25, 32, 17, 15): + * recorded legal basis, storage limitation, data minimisation, + * pseudonymisation, and working access/erasure surfaces. + * + * The guardrail only ever OPENS when a lawful basis is on record; nothing in + * 0pt today crawls personal data, so this ships as a fail-closed boundary: + * no policy → no storage (and, for directory domains like ratsit.se, no + * query at all). + */ + +/** GDPR art. 6(1) legal bases 0pt accepts for personal-data sources. */ +export type LegalBasis = "legitimate_interest" | "consent" | "contract" | "legal_obligation"; + +/** Compliance posture for one source that touches personal data. */ +export interface PiiPolicy { + source_id: string; + tenant_id: string; + project_id: string; + touches_personal_data: true; + legal_basis: LegalBasis; + /** + * Pointer to the evidence: a recorded Legitimate Interest Assessment + * (LIA) document id, the consent record id, the contract clause, or the + * legal provision. Must exist for audit/DPIA purposes — a bare claim is + * not enough. + */ + basis_ref: string; + /** + * Max storage in days. 0 = fetch-and-drop only (never retained). + * The retention worker deletes rows where stored_at + retention_days + * < now (art. 5(1)(e)). + */ + retention_days: number; + /** + * The ONLY fields that may be stored for this source (art. 5(1)(c) + * minimisation). Empty list = nothing may be stored. + */ + field_allowlist: string[]; + /** + * Pseudonymise identity keys (personnummer, email, reg-number) before + * storage; only sha256 digests are ever written (art. 25). + */ + pseudonymize_keys: boolean; + /** + * Must be true for web/api sources: robots.txt checked, rate limit in + * force (SourceRatePolicy), no auth-wall / bot-bypass technique in use. + */ + robots_and_rate_confirmed: boolean; + created_at: string; + updated_at: string | null; + created_by: string | null; +} + +export interface CreatePiiPolicyInput { + source_id: string; + tenant_id: string; + project_id: string; + legal_basis: LegalBasis; + basis_ref: string; + retention_days: number; + field_allowlist: string[]; + pseudonymize_keys?: boolean; + robots_and_rate_confirmed?: boolean; + created_by?: string; +} + +/** + * Registry row for one stored personal-data value. Only hashes are held — + * the key_hash lets 0pt locate the subject (right of access, art. 15) and + * the subject_hash lets it erase every artifact for that subject across + * sources (right to erasure, art. 17) without ever storing the plaintext + * identity. + */ +export interface PiiIdentityKey { + key_hash: string; + subject_hash: string; + source_id: string; + tenant_id: string; + field: string; + artifact_ref: string | null; + stored_at: string; + expires_at: string; + status: "active" | "erased" | "purged"; +} + +export interface PiiErasureRequest { + subject_hash: string; + reason: "deleted_by_subject" | "retention_expired" | "consent_withdrawn"; + actor_id?: string; + correlation_id?: string; +} + +export interface PiiAccessStatement { + subject_hash: string; + rows: PiiIdentityKey[]; + generated_at: string; +} \ No newline at end of file diff --git a/src/types/source.ts b/src/types/source.ts index 62f356e..a35f992 100644 --- a/src/types/source.ts +++ b/src/types/source.ts @@ -63,6 +63,13 @@ export interface Source { adapter_id?: string; /** Mirrors dim_source.access (docs/03-data-model.md). */ access?: SourceAccess; + /** + * Explicit self-flag that this source CAN expose personuppgifter. When + * true (or when the source is a known directory domain like ratsit.se), + * the PiiGateService requires a PiiPolicy with a recorded lawful basis + * before any storage/query (fail-closed, GDPR art. 6). + */ + touches_personal_data?: boolean; description?: string; status: SourceStatus; correlation_id?: string; @@ -86,6 +93,7 @@ export interface CreateSourceInput { name: string; adapter_id?: string; access?: SourceAccess; + touches_personal_data?: boolean; description?: string; status?: SourceStatus; correlation_id?: string; diff --git a/tests/p0/pii-gate.test.ts b/tests/p0/pii-gate.test.ts new file mode 100644 index 0000000..2ffbe0c --- /dev/null +++ b/tests/p0/pii-gate.test.ts @@ -0,0 +1,173 @@ +import { describe, it, expect, vi } from "vitest"; +import { PiiGateService } from "@/src/services/pii-gate-service"; +import { SourceService } from "@/src/services/source-service"; +import { SourceQueryService } from "@/src/services/source-query-service"; +import type { FabricOneLakeConnectionConfig } from "@/src/adapters/fabric-onelake-adapter"; + +/** Minimal config — never reached (the pii gate fires first). */ +const EMPTY_CFG = {} as FabricOneLakeConnectionConfig; + +describe("PiiGateService — GDPR guardrail (fail-closed)", () => { + const ratsit = { + source_id: "src_ratsit_1", + tenant_id: "t1", + project_id: "p1", + source_type: "web" as const, + name: "ratsit.se", + status: "active" as const, + created_at: new Date().toISOString(), + }; + const company = { + source_id: "src_company_1", + tenant_id: "t1", + project_id: "p1", + source_type: "api" as const, + name: "allabolag.se", + status: "active" as const, + created_at: new Date().toISOString(), + }; + + it("classifies known directory domains as personal-data sources", () => { + const gate = new PiiGateService(); + expect(gate.classifySourcePersonal(ratsit)).toBe(true); + expect(gate.classifySourcePersonal({ ...ratsit, name: "www.ratsit.se" })).toBe(true); + expect(gate.classifySourcePersonal(company)).toBe(false); + expect(gate.classifySourcePersonal({ ...company, touches_personal_data: true })).toBe(true); + }); + + it("rejects processing a personal-data source with no recorded lawful basis", async () => { + const gate = new PiiGateService(); + await expect(gate.assertCanStore(ratsit, [], "t1")).rejects.toThrowError(/no PiiPolicy/); + }); + + it("rejects half-legal policies (missing basis_ref, zero retention, empty allowlist)", async () => { + const gate = new PiiGateService(); + await expect( + gate.createPiiPolicy({ source_id: "s1", tenant_id: "t1", project_id: "p1", legal_basis: "legitimate_interest", basis_ref: " ", retention_days: 90, field_allowlist: ["address"] }), + ).rejects.toThrow(/basis_ref is required/); + await expect( + gate.createPiiPolicy({ source_id: "s1", tenant_id: "t1", project_id: "p1", legal_basis: "consent", basis_ref: "consent-record-1", retention_days: -1, field_allowlist: ["address"] }), + ).rejects.toThrow(/retention_days/); + await expect( + gate.createPiiPolicy({ source_id: "s1", tenant_id: "t1", project_id: "p1", legal_basis: "contract", basis_ref: "clause-9", retention_days: 90, field_allowlist: [] }), + ).rejects.toThrow(/data minimisation/); + }); + + it("blocks out-of-allowlist fields (minimisation) and unconfirmed web sources", async () => { + const gate = new PiiGateService(); + await gate.createPiiPolicy({ + source_id: ratsit.source_id, tenant_id: "t1", project_id: "p1", + legal_basis: "consent", basis_ref: "consent-record-42", + retention_days: 90, field_allowlist: ["name", "address"], + }); + await expect(gate.assertCanStore(ratsit, ["name", "phone"], "t1")).rejects.toThrowError(/outside the allowlist/); + await expect(gate.assertCanStore(ratsit, ["name"], "t1")).rejects.toThrowError(/robots.txt \/ rate-limit not confirmed/); + + const ratsitConfirmed = { ...ratsit, source_id: "src_ratsit_confirmed" }; + await gate.createPiiPolicy({ + source_id: ratsitConfirmed.source_id, tenant_id: "t1", project_id: "p1", + legal_basis: "consent", basis_ref: "consent-record-42", + retention_days: 90, field_allowlist: ["name", "address"], robots_and_rate_confirmed: true, + }); + await expect(gate.assertCanStore(ratsitConfirmed, ["name"], "t1")).resolves.toBeUndefined(); + }); + + it("forbids any storage when retention_days = 0", async () => { + const gate = new PiiGateService(); + await gate.createPiiPolicy({ + source_id: "src_drop_1", tenant_id: "t1", project_id: "p1", + legal_basis: "legitimate_interest", basis_ref: "lia-2026-001", + retention_days: 0, field_allowlist: ["name"], robots_and_rate_confirmed: true, + }); + await expect( + gate.assertCanStore({ ...company, source_id: "src_drop_1", name: "ratsit.se" }, ["name"], "t1"), + ).rejects.toThrowError(/forbids retention/); + }); + + it("stores ruled bases and enables access/erasure without plaintext identity", async () => { + const gate = new PiiGateService(); + await gate.createPiiPolicy({ + source_id: ratsit.source_id, tenant_id: "t1", project_id: "p1", + legal_basis: "legal_obligation", basis_ref: "prop-§5", + retention_days: 30, field_allowlist: ["street_address"], robots_and_rate_confirmed: true, + }); + const key = await gate.recordStored({ + source: ratsit, tenantId: "t1", + identityKey: "20250101-1234", subjectId: "subj-77", field: "street_address", artifactRef: "st0re/dataset-a", + }); + + expect(key.key_hash).toMatch(/^[0-9a-f]{64}$/); + expect(key.key_hash).not.toContain("20250101"); + + const statement = await gate.accessStatement(key.subject_hash); + expect(statement.rows).toHaveLength(1); + expect(statement.rows[0].field).toBe("street_address"); + + const erased = await gate.erase({ subject_hash: key.subject_hash, reason: "deleted_by_subject" }); + expect(erased).toBe(1); + expect((await gate.accessStatement(key.subject_hash)).rows).toHaveLength(0); + }); + + it("retention worker purges rows past their deadline", async () => { + const gate = new PiiGateService(); + vi.useFakeTimers(); + vi.setSystemTime(new Date("2026-08-01T00:00:00Z")); + await gate.createPiiPolicy({ + source_id: ratsit.source_id, tenant_id: "t1", project_id: "p1", + legal_basis: "legitimate_interest", basis_ref: "lia-2026-002", + retention_days: 30, field_allowlist: ["street_address"], robots_and_rate_confirmed: true, + }); + await gate.recordStored({ source: ratsit, tenantId: "t1", identityKey: "k-1", subjectId: "s-1", field: "street_address" }); + + vi.setSystemTime(new Date("2026-09-15T00:00:00Z")); + const { purged } = await gate.enforceRetention(new Date("2026-09-15T00:00:00Z")); + vi.useRealTimers(); + expect(purged).toBe(1); + }); + + it("redacts personnummer, e-mail and phone from log lines", () => { + const gate = new PiiGateService(); + const out = gate.redact( + "subject=202501011234 res=" + + 'evt=19720101-1234 mail=simon@siax.io tel=070-123 45 67 sök=ok', + ); + expect(out).not.toMatch(/\d{8}-\d{4}/); + expect(out).toContain("[PERSONNUMMER]"); + expect(out).toContain("[EMAIL]"); + expect(out).toContain("[TELEFON]"); + }); + + it("SourceQueryService blocks a personal-data source by default (fail-closed)", async () => { + const sourceService = new SourceService(); + await sourceService.createSource({ ...ratsit }); + + const service = new SourceQueryService({ sourceService }); + await expect( + service.querySource({ + source_id: ratsit.source_id, + query: "SELECT 1", + tenant_id: "t1", project_id: "p1", + connection_config: EMPTY_CFG, + }), + ).rejects.toThrowError(/no PiiPolicy/); + }); + + it("SourceQueryService passes non-personal sources straight through to capability checks", async () => { + const sourceService = new SourceService(); + await sourceService.createSource({ + source_id: "src_safe_1", tenant_id: "t1", project_id: "p1", + source_type: "web", name: "example.org", status: "active", + }); + + const service = new SourceQueryService({ sourceService }); + // Gate passes; the sync path then correctly rejects non-database sources. + await expect( + service.querySource({ + source_id: "src_safe_1", + query: "SELECT 1", + tenant_id: "t1", project_id: "p1", + connection_config: EMPTY_CFG, + }), + ).rejects.toThrowError(/source_type "web"/); + }); +}); \ No newline at end of file From 17bd1b40a02ea454112f63b13c01993120df3736 Mon Sep 17 00:00:00 2001 From: Simon Date: Mon, 31 Aug 2026 01:18:03 +0200 Subject: [PATCH 2/4] =?UTF-8?q?feat(0pt):=20record=20legitimate-interest?= =?UTF-8?q?=20stance=20=E2=80=94=20B2B=20contacts,=20company-linked=20only?= MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit Recorded decision (Simon 2026-08-31): personal data may be stored under legitimate interest ONLY as business contacts bound to a company/org entity. Gate enforces it: legitimate_interest requires purpose=business_contact and entity_linked_only=true; recordStored() requires entity_id (orgnr/rgs/LEI, hashed) and refuses person-only rows. +1 test; suite 167 pass | 1 skip; typecheck clean. --- docs/compliance/PII_GUARDRAIL.md | 19 ++++++++++++ src/services/pii-gate-service.ts | 51 ++++++++++++++++++++++++++++++-- src/types/pii.ts | 19 ++++++++++++ tests/p0/pii-gate.test.ts | 30 ++++++++++++++++++- 4 files changed, 116 insertions(+), 3 deletions(-) diff --git a/docs/compliance/PII_GUARDRAIL.md b/docs/compliance/PII_GUARDRAIL.md index 78c82df..36ef93f 100644 --- a/docs/compliance/PII_GUARDRAIL.md +++ b/docs/compliance/PII_GUARDRAIL.md @@ -16,6 +16,23 @@ first byte is stored**. (`0pt.pii_policy_created | pii_data_retained | pii_erased | pii_retention_purged`). +## Recorded legitimate-interest stance (Simon, 2026-08-31) + +> "Vi får lagra dem om dem är kopplade till företagen i affärssyfte." + +Personal data may be stored under **legitimate interest (art. 6(1)(f))** ONLY +as **business contacts bound to a company/org entity** (`entity_linked_only`). +The gate enforces this mechanically: + +- `legal_basis: legitimate_interest` ⇒ `purpose` MUST be `business_contact` and + `entity_linked_only` MUST be `true`. A lazily-written policy is rejected. +- `recordStored()` refuses person-only rows for such policies: an `entity_id` + (orgnr/rgs-nummer/LEI) is REQUIRED; only its hash is ever stored. +- Stand-alone personal records (no backing company) are not permitted under LI. + +This is the recorded business-policy decision — a written LIA should be kept +and its id referenced in `basis_ref` (`lia-YYYY-MM-b2b-NN` naming is fine). + ## GDPR article mapping | Requirement | Implementation | @@ -35,6 +52,8 @@ first byte is stored**. - [x] Write the **Legitimate Interest Assessment** (or record consent / contract / legal obligation) and store its id in `basis_ref`. LIA minimum: purpose, necessity, individuals' rights impact, mitigations. + For LI: purpose `business_contact` + `entity_linked_only` (see recorded + stance above — an LIA referencing `lia-YYYY-MM-b2b-NN` is good practice). - [x] **VERIFY current terms + robots.txt + availability** on the live site. Do not hardcode "catsit.com allows" — re-check each enable. - [x] Set `robots_and_rate_confirmed: true` + a `SourceRatePolicy` matching diff --git a/src/services/pii-gate-service.ts b/src/services/pii-gate-service.ts index da84d99..c86cb96 100644 --- a/src/services/pii-gate-service.ts +++ b/src/services/pii-gate-service.ts @@ -22,6 +22,7 @@ import type { PiiIdentityKey, PiiErasureRequest, PiiAccessStatement, + ProcessingPurpose, } from "../types/pii"; import type { Source } from "../types/source"; import { createAud0Client, type Aud0Client, type Aud0Event } from "../clients/aud0-client"; @@ -44,6 +45,7 @@ export class PiiLegalBasisError extends Error {} export class PiiMinimisationError extends Error {} export class PiiRetentionError extends Error {} export class PiiRobotsRateError extends Error {} +export class PiiEntityLinkageError extends Error {} export class PiiGateService { private policies: Map = new Map(); @@ -80,6 +82,21 @@ export class PiiGateService { throw new Error("field_allowlist must contain at least one field (data minimisation)"); } + // Recorded decision (Simon, 2026-08-31): legitimate interest is accepted for + // BUSINESS CONTACTS — personal data only in correspondence with a company/org + // entity. Enforce that posture so "B2B-kontakt, företagskopplad" cannot be + // bypassed by a lazily-written policy. + const purpose: ProcessingPurpose = input.purpose ?? "business_contact"; + const entityLinkedOnly = input.entity_linked_only ?? input.legal_basis === "legitimate_interest"; + if (input.legal_basis === "legitimate_interest") { + if (purpose !== "business_contact") { + throw new Error("legitimate_interest is only accepted with purpose=business_contact (recorded decision 2026-08-31)"); + } + if (!entityLinkedOnly) { + throw new Error("legitimate_interest requires entity_linked_only=true — personal data must be bound to a company/org entity"); + } + } + const now = new Date().toISOString(); const policy: PiiPolicy = { source_id: input.source_id, @@ -87,6 +104,8 @@ export class PiiGateService { project_id: input.project_id, touches_personal_data: true, legal_basis: input.legal_basis, + purpose, + entity_linked_only: entityLinkedOnly, basis_ref: input.basis_ref, retention_days: input.retention_days, field_allowlist: [...input.field_allowlist], @@ -108,7 +127,13 @@ export class PiiGateService { resource_type: "source_pii_policy", resource_id: input.source_id, risk_level: "L3", - payload: { legal_basis: input.legal_basis, retention_days: input.retention_days, field_count: input.field_allowlist.length }, + payload: { + legal_basis: input.legal_basis, + purpose, + entity_linked_only: entityLinkedOnly, + retention_days: input.retention_days, + field_count: input.field_allowlist.length, + }, }); return policy; } @@ -192,14 +217,26 @@ export class PiiGateService { subjectId: string; field: string; artifactRef?: string; + /** Company/org entity the value is bound to (orgnr/rgs/LEI). REQUIRED when the policy is entity_linked_only. */ + entityId?: string; }): Promise { await this.assertCanStore(input.source, [input.field], input.tenantId); const policy = this.policies.get(input.source.source_id)!; + if (policy.entity_linked_only && !input.entityId) { + throw new PiiEntityLinkageError( + DECISION_TEMPLATE.replace("%s", input.source.source_id).replace( + "%s", + "policy requires entity linkage (business contact, recorded decision 2026-08-31) — pass entityId (orgnr/rgs/LEI); personal data may only be stored in correspondence with a company/org entity", + ), + ); + } + const now = Date.now(); const key: PiiIdentityKey = { key_hash: hash256(`${sourceIdSalt(input.source.source_id)}:${input.identityKey}`), subject_hash: hash256(input.subjectId), + entity_hash: input.entityId ? hash256(normaliseEntity(input.entityId)) : null, source_id: input.source.source_id, tenant_id: input.tenantId, field: input.field, @@ -220,7 +257,12 @@ export class PiiGateService { resource_type: "pii_identity_key", resource_id: key.key_hash, risk_level: "L3", - payload: { source_id: input.source.source_id, field: input.field, expires_at: key.expires_at }, + payload: { + source_id: input.source.source_id, + field: input.field, + expires_at: key.expires_at, + entity_linked: key.entity_hash !== null, + }, }); return key; } @@ -318,6 +360,11 @@ function sourceIdSalt(sourceId: string): string { return `0pt-pii-v1:${sourceId}`; } +/** Normalise an org/rgs nummer or LEI to canonical digits/letters before hashing. */ +function normaliseEntity(entityId: string): string { + return entityId.replace(/[^0-9A-Za-z]/g, "").toUpperCase(); +} + /** sha256 hex digest — never the plaintext. */ export function hash256(value: string): string { return createHash("sha256").update(value, "utf8").digest("hex"); diff --git a/src/types/pii.ts b/src/types/pii.ts index 606b07b..952a7be 100644 --- a/src/types/pii.ts +++ b/src/types/pii.ts @@ -16,6 +16,14 @@ /** GDPR art. 6(1) legal bases 0pt accepts for personal-data sources. */ export type LegalBasis = "legitimate_interest" | "consent" | "contract" | "legal_obligation"; +/** + * The purpose of processing. SIAX's recorded decision (Simon, 2026-08-31): + * personal data may be stored when LEGITIMATE INTEREST applies for business + * contacts — i.e. bound to a company/org entity, for business purposes. + * Other bases are allowed but every policy records its purpose. + */ +export type ProcessingPurpose = "business_contact" | "consent_marketing" | "contract_performance" | "legal_obligation"; + /** Compliance posture for one source that touches personal data. */ export interface PiiPolicy { source_id: string; @@ -23,6 +31,13 @@ export interface PiiPolicy { project_id: string; touches_personal_data: true; legal_basis: LegalBasis; + purpose: ProcessingPurpose; + /** + * REQUIRED under legitimate_interest (business_contact): personal data may + * only be stored in correspondence with a company/org entity (org/rgs-nr, + * GLEIF/LEI). recordStored() rejects stray person-only rows. + */ + entity_linked_only: boolean; /** * Pointer to the evidence: a recorded Legitimate Interest Assessment * (LIA) document id, the consent record id, the contract clause, or the @@ -64,6 +79,8 @@ export interface CreatePiiPolicyInput { basis_ref: string; retention_days: number; field_allowlist: string[]; + purpose?: ProcessingPurpose; + entity_linked_only?: boolean; pseudonymize_keys?: boolean; robots_and_rate_confirmed?: boolean; created_by?: string; @@ -79,6 +96,8 @@ export interface CreatePiiPolicyInput { export interface PiiIdentityKey { key_hash: string; subject_hash: string; + /** Company/org entity the value is bound to (orgnr/rgs/LEI), hashed. */ + entity_hash: string | null; source_id: string; tenant_id: string; field: string; diff --git a/tests/p0/pii-gate.test.ts b/tests/p0/pii-gate.test.ts index 2ffbe0c..0d1cfbf 100644 --- a/tests/p0/pii-gate.test.ts +++ b/tests/p0/pii-gate.test.ts @@ -53,6 +53,34 @@ describe("PiiGateService — GDPR guardrail (fail-closed)", () => { ).rejects.toThrow(/data minimisation/); }); + it("enforces the recorded decision: legitimate_interest = B2B business contact, company-linked only", async () => { + const gate = new PiiGateService(); + await expect( + gate.createPiiPolicy({ source_id: "s_li_wrong", tenant_id: "t1", project_id: "p1", legal_basis: "legitimate_interest", basis_ref: "lia-2026-001", retention_days: 90, field_allowlist: ["email"], purpose: "consent_marketing" }), + ).rejects.toThrowError(/only accepted with purpose=business_contact/); + await expect( + gate.createPiiPolicy({ source_id: "s_li_wrong2", tenant_id: "t1", project_id: "p1", legal_basis: "legitimate_interest", basis_ref: "lia-2026-001", retention_days: 90, field_allowlist: ["email"], entity_linked_only: false }), + ).rejects.toThrowError(/entity_linked_only=true/); + + await gate.createPiiPolicy({ + source_id: "s_li_ok", tenant_id: "t1", project_id: "p1", + legal_basis: "legitimate_interest", basis_ref: "lia-2026-b2b-01", + retention_days: 365, field_allowlist: ["email", "phone"], + }); + const p = await gate.getPiiPolicy("s_li_ok"); + expect(p?.purpose).toBe("business_contact"); + expect(p?.entity_linked_only).toBe(true); + + // Business contact must be bound to a company — stray person-only rows are refused. + const liSource = { ...company, source_id: "s_li_ok" }; + await expect( + gate.recordStored({ source: liSource, tenantId: "t1", identityKey: "k", subjectId: "s", field: "email" }), + ).rejects.toThrowError(/entity linkage/); + const linked = await gate.recordStored({ source: liSource, tenantId: "t1", identityKey: "k", subjectId: "s", field: "email", entityId: "556677-8899" }); + expect(linked.entity_hash).toMatch(/^[0-9a-f]{64}$/); + expect(linked.entity_hash).not.toContain("556677"); + }); + it("blocks out-of-allowlist fields (minimisation) and unconfirmed web sources", async () => { const gate = new PiiGateService(); await gate.createPiiPolicy({ @@ -117,7 +145,7 @@ describe("PiiGateService — GDPR guardrail (fail-closed)", () => { legal_basis: "legitimate_interest", basis_ref: "lia-2026-002", retention_days: 30, field_allowlist: ["street_address"], robots_and_rate_confirmed: true, }); - await gate.recordStored({ source: ratsit, tenantId: "t1", identityKey: "k-1", subjectId: "s-1", field: "street_address" }); + await gate.recordStored({ source: ratsit, tenantId: "t1", identityKey: "k-1", subjectId: "s-1", field: "street_address", entityId: "5566778899" }); vi.setSystemTime(new Date("2026-09-15T00:00:00Z")); const { purged } = await gate.enforceRetention(new Date("2026-09-15T00:00:00Z")); From d41f30535bae63182fc725909276186520b810c1 Mon Sep 17 00:00:00 2001 From: Simon Axelsson Date: Mon, 14 Sep 2026 19:58:04 +0200 Subject: [PATCH 3/4] fix(test): replay full migration chain in source-registry realinfra test tests/p0/source-registry-postgres.realinfra.test.ts only applied sql/002_source_registry.sql before running assertions. sql/007_ingestion_plane.sql later added opt_sources.owner_id, and src/repositories/postgres-source-repository.ts's INSERT has named that column ever since -- but nothing caught the drift because SOURCE_REGISTRY_TEST_DATABASE_URL is never set in CI (.gitea/workflows/ci.yml has no job that sets it), so this suite has silently skipped in every CI run. Verified against a real disposable Postgres 16 container: before this fix, all 7 tests failed with `column "owner_id" of relation "opt_sources" does not exist`; after replaying every sql/*.sql file in order (same pattern already used by tests/analytics/analytics-domain-postgres.realinfra.test.ts), all 7 pass. Found while auditing 0pt's Analytics/BI layer (MetricFlow/Cube/ClickHouse/ Trino/DuckDB) and standing up real Postgres/ClickHouse containers to verify those claims for real; this bug is adjacent (source registry, not analytics) but was directly encountered and is a small, safe, test-only fix. Co-Authored-By: Claude Sonnet 5 --- ...source-registry-postgres.realinfra.test.ts | 42 +++++++++++++++---- 1 file changed, 35 insertions(+), 7 deletions(-) diff --git a/tests/p0/source-registry-postgres.realinfra.test.ts b/tests/p0/source-registry-postgres.realinfra.test.ts index 775bdd5..2a2466e 100644 --- a/tests/p0/source-registry-postgres.realinfra.test.ts +++ b/tests/p0/source-registry-postgres.realinfra.test.ts @@ -24,10 +24,24 @@ * prove the rows are really there, not an artifact of the service's own * in-process bookkeeping. * - * beforeAll applies sql/002_source_registry.sql itself (every statement is - * `CREATE TABLE/INDEX IF NOT EXISTS`, so this is idempotent and safe to run - * against an already-migrated database too) -- this test does not require - * the operator to have pre-run migrations. + * beforeAll applies every sql/NNN_*.sql file in order (same restore-drill + * pattern as tests/analytics/analytics-domain-postgres.realinfra.test.ts), + * not just sql/002_source_registry.sql -- this test does not require the + * operator to have pre-run migrations. + * + * CORRECTED (this pass, 2026-09-14): it used to apply ONLY + * sql/002_source_registry.sql. That was enough the day this file was + * written, but sql/007_ingestion_plane.sql later added `opt_sources.owner_id` + * (`ALTER TABLE opt_sources ADD COLUMN IF NOT EXISTS owner_id TEXT`), and + * src/repositories/postgres-source-repository.ts's INSERT has named that + * column ever since. Because this env var is never set in CI (confirmed by + * grep over .gitea/workflows/ci.yml — no job references + * SOURCE_REGISTRY_TEST_DATABASE_URL), the drift was never caught: every + * assertion below failed with `column "owner_id" of relation "opt_sources" + * does not exist` the moment it was actually run against a real, freshly + * migrated-to-002-only Postgres. The fix is to stop hand-picking one + * migration file and replay the whole chain, exactly like the analytics + * suite already does. */ import { afterAll, beforeAll, describe, expect, it } from "vitest"; import fs from "node:fs"; @@ -56,9 +70,23 @@ beforeAll(async () => { adminSql`SELECT 1`, new Promise((_, reject) => setTimeout(() => reject(new Error("connect timeout")), 5000)), ]); - const migrationPath = path.join(process.cwd(), "sql", "002_source_registry.sql"); - const migration = fs.readFileSync(migrationPath, "utf8"); - await adminSql.unsafe(migration); + // Replay every migration in order, not just 002 -- see the file header + // "CORRECTED" note. Tolerate an individual file failing (mirrors + // analytics-domain-postgres.realinfra.test.ts: sql/001 needs the + // `vector` extension, which a stock postgres:16-alpine does not carry) + // as long as the files this test actually depends on (002, 007) apply. + const dir = path.join(process.cwd(), "sql"); + for (const file of fs + .readdirSync(dir) + .filter((f) => f.endsWith(".sql")) + .sort()) { + const body = fs.readFileSync(path.join(dir, file), "utf8"); + try { + await adminSql.unsafe(body); + } catch (err) { + console.warn(`[realinfra] ${file} did not apply cleanly: ${String(err).slice(0, 200)}`); + } + } reachable = true; } catch (err) { reachable = false; From 8ebc90ffa3c1c1924f4e281855917db649bd248d Mon Sep 17 00:00:00 2001 From: Simon Axelsson Date: Mon, 7 Sep 2026 20:45:58 +0200 Subject: [PATCH 4/4] fix(security,repo-honesty): real security headers + a doctor that checks truth, not presence Two defects of the same kind: the repo asserted things about itself that were not true, and nothing in it could tell. A. CHANGELOG.md claimed CSP/HSTS/X-Frame-Options since 2026-08. `main` contained none of it -- a repo-wide grep for `content-security-policy|strict-transport-security|x-frame-options` matched exactly one file, CHANGELOG.md itself. The only header on 0pt.app was Vercel's own HSTS default, which does not survive the move to Coolify/Traefik that siax.config.json and .gitea/workflows/cd.yml are already pointed at. - next.config.mjs: `headers()` on `/(.*)` with CSP (frame-ancestors 'none', base-uri 'self', object-src 'none', form-action 'self'), HSTS, X-Frame-Options DENY, X-Content-Type-Options, Referrer-Policy, Permissions-Policy; plus poweredByHeader: false. Same idiom as sax3l/siax-io's next.config.mjs. - Carve-outs re-derived against THIS tree, not inherited: the stale `ci/security-headers-and-scanning` branch (8d2ff73, 142 commits behind) carried a `form-action https://checkout.stripe.com` allowance justified by a redirect that is actually `window.location.href`, which form-action does not govern -- dropped rather than kept as decorative permission. frame-src is derived from NOCODB_BASE_URL/METABASE_SITE_URL, the same env the components/embed.tsx iframe is configured from, so wiring that component up cannot be silently blanked by this header (the failure sax3l/siax-io recorded as SIO-013). 'unsafe-eval' is dev-only. - tests/security/security-headers.test.ts: blocking regression gate inside `pnpm run test`, i.e. the required "CI (SIAX Cloud) / quality" context. Asserts against expectations written out in the test, not imported from the config. Proven to fail: removing X-Frame-Options -> `pnpm run test` exits 1; weakening frame-ancestors to 'self' -> exits 1. B. scripts/siax-doctor.mjs validated that siax.repo.json fields were PRESENT, never what they said. Measured on this tree: operationalState "running" (false) and "down" (true) both gave 40/40, exit 0, byte-identical output. - Value checks (enums, git.siax.io const, fleet node pattern, bare-hostname form), cross-file consistency (declared prod hostnames must appear in this repo's own siax.config.json / contracts/openapi.yaml, and vice versa), and GOV-06 substantiation: operationalState must be backed by a dated probe record in evidence/operational-state.json that agrees with it. Liveness is not knowable offline; whether a liveness CLAIM is backed is. Offline, deterministic, no network -- CI can never go red for want of an internet connection. - `--live` (opt-in) probes the declared hosts for real and checks the headers on the wire; `--write-evidence` refreshes the record. Not wired into CI on purpose. - tests/repo/siax-doctor.test.ts runs the doctor inside `pnpm run test` and asserts the negative cases against fixture trees (SIAX_DOCTOR_ROOT), so the "cannot fail" defect cannot come back. C. CHANGELOG.md is now true. The header claim stands; the four claims it was bundled with (Trivy --exit-code 1, gitleaks, SHA-pinned actions, SBOM + license artifacts) are still false on main and are listed as not implemented instead of left asserted. All four live in .gitea/workflows/**, a protected path on this repo, so they need an authorised workflow change. Nothing under .gitea/workflows/** or .gitignore is touched. Verified: pnpm run typecheck exit 0; pnpm run test exit 0 (74 files / 965 tests passed, up from 72/945); pnpm run build exit 0 and the built .next/routes-manifest.json carries all six headers; node scripts/siax-doctor.mjs 55/55 exit 0; --live 57/57 exit 0. eslint on the new files exit 0 (repo lint is advisory and its 17 pre-existing problems are unchanged). Co-Authored-By: Claude Opus 5 --- CHANGELOG.md | 50 ++- CODEOWNERS | 8 + evidence/operational-state.json | 49 +++ next.config.mjs | 118 ++++++ scripts/siax-doctor.mjs | 524 ++++++++++++++++++++++-- tests/repo/siax-doctor.test.ts | 209 ++++++++++ tests/security/security-headers.test.ts | 184 +++++++++ 7 files changed, 1102 insertions(+), 40 deletions(-) create mode 100644 CODEOWNERS create mode 100644 evidence/operational-state.json create mode 100644 tests/repo/siax-doctor.test.ts create mode 100644 tests/security/security-headers.test.ts diff --git a/CHANGELOG.md b/CHANGELOG.md index 3037d76..13c0ae6 100644 --- a/CHANGELOG.md +++ b/CHANGELOG.md @@ -25,12 +25,37 @@ levereras därför i ett eget npm-steg i Dockerfile, pinnat till versionen i `pnpm-lock.yaml`. Verifierat i containern: `/_next/image` svarar 200 med `image/avif` (1,46 MB PNG -> 6,7 kB). -- `chore(deploy)`: `poweredByHeader: false` (ingen `X-Powered-By: Next.js`). +- fix(security): response security headers are set by the app itself, in + `next.config.mjs` `headers()` — CSP (incl. `frame-ancestors 'none'`, + `base-uri 'self'`, `object-src 'none'`, `form-action 'self'`), HSTS, + X-Frame-Options `DENY`, X-Content-Type-Options, Referrer-Policy, + Permissions-Policy — plus `poweredByHeader: false`. Regression-gated by + `tests/security/security-headers.test.ts` (blocking, runs in + `pnpm run test`) and re-checkable against a live origin with + `node scripts/siax-doctor.mjs --live`. (BACKLOG P0-13, P1-15) -- fix(security): security headers (CSP/HSTS/X-Frame-Options/etc.) + CI - security gates actually enforce (Trivy `--exit-code 1`, gitleaks) instead - of advisory-only. Actions pinned to commit SHA. SBOM + license inventory - as CI artifacts. (BACKLOG P0-13, P0-16, P0-19, P1-14, P1-15) + Correcting this file: from 2026-08 until 2026-09-07 this bullet also + claimed the headers themselves, while `main` contained none — a repo-wide + grep for `content-security-policy|strict-transport-security| + x-frame-options` matched only CHANGELOG.md. The only thing setting HSTS on + 0pt.app was Vercel's platform default (still observable on the 402 + responses recorded in `evidence/operational-state.json`), which is not this + repo's and does not survive the move to Coolify/Traefik. The claim is true + as of this entry; the four claims it was bundled with are not, and are + tracked in the section below. + +- fix(repo-honesty): `scripts/siax-doctor.mjs` validated that + `siax.repo.json` fields were PRESENT, never what they said — measured: + `operationalState: "running"` (false at the time; every declared host + answered 402/404) and `"down"` (true) both produced 40/40 and + byte-identical output. It now validates values (enums, hostname form), + cross-file consistency (declared prod hostnames must appear in this repo's + own deploy/contract config, and vice versa), and requires + `operationalState` to be backed by a dated probe record in + `evidence/operational-state.json` that agrees with it. Offline and + deterministic by default; `--live` (opt-in) probes the real hosts. + Gated by `tests/repo/siax-doctor.test.ts`, which also asserts the negative + cases against fixture trees. (BACKLOG P1-19, control AGENT-04, GOV-06) - fix(health): `/api/health` reports real per-dependency status instead of a hardcoded `ok: true`; new `/api/ready` readiness endpoint. (BACKLOG P0-3) - docs/chore: repo-hygiene pass — schema-valid `siax.repo.json`/`siax.config.json` @@ -38,6 +63,21 @@ `docs/architecture/ARCHITECTURE.md` §5 "Ytor och paritet", CODEOWNERS, stale-link cleanup. (BACKLOG P1-1, P1-2, P1-3, P1-19, P1-20, P1-21, P2-5, P3-5) +### Claimed here before 2026-09-07 but NOT implemented — status 2026-09-15 + +These were asserted by the earlier `fix(security)` bullet and were not true on +`main` at that time. Current state (measured against `main`, 2026-09-15): + +- CI security gates now ENFORCE: `security.yml` and `nightly.yml` run Trivy + with `--exit-code 1` (landed via `ci/make-required-security-gate-real`, + 2026-09-08). (BACKLOG P0-16 — CLOSED) +- gitleaks is still not run anywhere in this repo. (BACKLOG P0-19 — OPEN) +- Actions are still NOT pinned to commit SHAs (`actions/checkout@v4` etc.). + (BACKLOG P1-14 — OPEN) +- No SBOM and no license inventory are produced or uploaded as CI artifacts. + (OPEN; the `fix/wire-signed-digest-to-deploy` branch carries a prepared + release chain that needs its cosign/registry secrets before it can land.) + ## 2026-08-11 — Evasion-stack decision provenance - Attributed and dated the account-owner's keep-decision on `services/scrapers/*` diff --git a/CODEOWNERS b/CODEOWNERS new file mode 100644 index 0000000..0134e0f --- /dev/null +++ b/CODEOWNERS @@ -0,0 +1,8 @@ +# BACKLOG P1-15 (CI-14): required reviewers for Gitea's branch-protection +# "required approvals" setting to have anyone to assign. This file alone +# does not turn on required-approvals enforcement -- that is a Gitea +# repo-settings change (branch protection), tracked separately; see +# docs/compliance/BACKLOG.md P1-15 ("required_approvals: 0"). +# +# Gitea CODEOWNERS syntax matches GitHub's: path pattern, then owner(s). +* @sax3l diff --git a/evidence/operational-state.json b/evidence/operational-state.json new file mode 100644 index 0000000..5d4574f --- /dev/null +++ b/evidence/operational-state.json @@ -0,0 +1,49 @@ +{ + "$comment": [ + "GOV-06 substantiation record for siax.repo.json's `operationalState`.", + "", + "WHY THIS FILE EXISTS: `operationalState` is a claim about the world, and", + "scripts/siax-doctor.mjs cannot check the world from CI (no network, and a", + "check that needs the internet is a check that goes red for the wrong", + "reason). What the doctor CAN do offline is refuse to accept the claim", + "unless it is backed by a dated, machine-readable probe result that agrees", + "with it. That is this file. Setting operationalState to \"running\" while", + "the probes below say 402/404 now fails `pnpm siax:doctor`.", + "", + "HOW TO REFRESH: `node scripts/siax-doctor.mjs --live --write-evidence`", + "(opt-in, needs network). Or edit by hand from a real probe -- but the", + "doctor cross-checks `ok` against `httpStatus`, so a hand-written", + "`ok: true` next to a 402 is itself a failure." + ], + "control": "GOV-06", + "operationalState": "down", + "checkedAt": "2026-09-07T18:24:24Z", + "method": "curl -sS -o /dev/null -D - --max-time 20 , from the public internet (not the tailnet)", + "probes": [ + { + "fqdn": "0pt.app", + "env": "prod", + "url": "https://0pt.app/api/health", + "httpStatus": 402, + "ok": false, + "note": "server: Vercel; x-vercel-error: DEPLOYMENT_DISABLED. The Vercel deployment is disabled and no Coolify app is serving this hostname yet." + }, + { + "fqdn": "api.0pt.app", + "env": "prod", + "url": "https://api.0pt.app/api/health", + "httpStatus": 404, + "ok": false, + "note": "server: Vercel; x-vercel-error: DEPLOYMENT_NOT_FOUND. Contract-first target host (contracts/openapi.yaml servers), never deployed. Both / and /api/health return the same 404." + } + ], + "securityHeadersObserved": { + "$comment": "Observed on the 402/404 responses above, i.e. served by Vercel's edge, NOT by this app. This is the evidence for BACKLOG P0-13: everything except Vercel's own HSTS default is absent, and that default does not survive a move to Coolify/Traefik.", + "strict-transport-security": "max-age=63072000", + "content-security-policy": null, + "x-frame-options": null, + "x-content-type-options": null, + "referrer-policy": null, + "permissions-policy": null + } +} diff --git a/next.config.mjs b/next.config.mjs index 1f0d4fd..59ce1f2 100644 --- a/next.config.mjs +++ b/next.config.mjs @@ -1,3 +1,113 @@ +// ============================================================================= +// 0pt -- Next.js config +// ============================================================================= +// SECURITY HEADERS (BACKLOG P0-13 / SEC-06). +// +// CHANGELOG.md has claimed since 2026-08 that this repo ships +// CSP/HSTS/X-Frame-Options. Until this commit that claim was FALSE on `main`: +// a repo-wide grep for `content-security-policy|strict-transport-security| +// x-frame-options` matched exactly one file -- CHANGELOG.md itself. There was +// no `headers()`, no middleware header injection, no reverse-proxy config in +// this repo. The only thing that ever set HSTS on 0pt.app was Vercel's own +// platform default, which is not this repo's and disappears the moment the app +// moves to Coolify/Traefik (siax.config.json `deploy.target` is already +// "coolify"; .gitea/workflows/cd.yml already points at a real Coolify app id). +// Setting them here, at the framework level, means they travel with the app +// regardless of host -- same idiom as sax3l/siax-io's next.config.mjs. +// +// CARVE-OUTS, each grep-verified against THIS tree (2026-09-07), not inherited +// from an older branch: +// +// * script-src/style-src 'unsafe-inline' -- Next's App Router injects inline +// hydration/RSC-payload scripts and next-themes injects an inline +// no-flash script; Tailwind + recharts emit inline style attributes. A +// per-request nonce threaded through middleware + layout is the real fix +// and is deliberately NOT attempted here: this commit closes "zero +// headers", not "zero inline tolerance". Tracked in BACKLOG. +// * 'unsafe-eval' is added ONLY in `next dev` (webpack/Turbopack eval source +// maps + HMR). Production never gets it. Guarded on NODE_ENV so a +// dev-only relaxation can never leak into a built image, whose runtime +// stage sets NODE_ENV=production (see Dockerfile). +// * form-action 'self' -- no cross-origin
exists (the only +// in the tree, app/portal/integrationer/page.tsx, is onSubmit-only). +// The Stripe hop in app/bestall/forskott/page.tsx is +// `window.location.href = body.url`, i.e. a top-level navigation, which no +// shipped CSP directive gates -- `form-action` would not have applied to +// it. An earlier draft of this change carried a +// `form-action https://checkout.stripe.com` carve-out on that mistaken +// premise; it is dropped here rather than kept as decorative permission. +// * connect-src 'self' -- every external host in the tree +// (api.stripe.com, graph.microsoft.com, login.microsoftonline.com, +// id-customers.siax.io, api.opensanctions.org, opendata.rdw.nl) is reached +// from server-side route handlers / adapters, never from the browser. +// * frame-src -- see FRAME_SRC below. This is the one directive with a live +// trap in it, so it is derived from env rather than hardcoded to 'self'. +// +// WHAT THIS FILE CANNOT DO: it sets response headers. It does not prove they +// are served. `tests/security/security-headers.test.ts` asserts this config +// keeps them (blocking, runs in "CI (SIAX Cloud) / quality"), and +// `node scripts/siax-doctor.mjs --live` re-checks them against a running +// instance. Neither replaces the other. +// ============================================================================= + +const IS_DEV = process.env.NODE_ENV === "development" + +/** + * `components/embed.tsx` renders a NocoDB / Metabase view in an