Parallellsessions-merge lämnade 12 eslint-errors på main (no-explicit-any i
graph-mailbox-adapter + 2 p0-tester). Graph-JSON:n är genuint ostrukturerad
— medveten, avgränsad eslint-undantag med skäl (DoD §13); p0-testerna
typade med Record/cast. Lint 0 errors; tsc grön; 1141/1141.
Implements Simon's specified pattern (dealer_stock_entry/deal ->
watermark/updated_at -> 0pt ingestion -> ClickHouse) as a P061 connector
pair (PostgresWatermarkConnector, PostgresKeyedLookupConnector) plugged
into the existing P062 IngestionRunner, writing to opt_dataset_projection
via SiaxAnalyticsEngine.ingestProjection.
Built here, not in admin/vehiq: that repo's own ADR-0035 (decisions 1/2/10,
CI-enforced by validate_engine_firewall.py) forbids VehIQ code from ever
holding a ClickHouse credential or hammering OLTP directly for analytics.
See docs/P070_VEHIQ_OLTP_EXTRACTOR.md for the full rationale.
Also corrects a real schema mismatch found while building this: production
has no generic `updated_at` on any of the three target tables (verified
live, read-only, 2026-09-25). dealer_stock_entry/deal use listed_at/
created_at as insert-time watermarks instead (documented caveat: in-place
updates are not caught without a schema change); app.listing has no
timestamp column at all and is deliberately not covered yet.
Vehicle dimension lookup is narrow by construction: keyed on exactly the
vehicle_id set referenced by the batch just read, never a scan of the
98M-row vehiq.vehicle table (connector-level validate() refuses that table
by name; runner-level tests assert the query only ever touches referenced
IDs).
Verified: fixture-backed connector/runner tests (watermark+limit,
same-timestamp tie at a page boundary, crash/resume, empty-table case,
narrow-lookup query-count assertions) + a real disposable Postgres
container (caught two real Date-handling bugs fixtures could not) + a
read-only proof against actual production (server10, vehiq-db): the
extractor's exact SQL returns zero rows, no error, confirming it is safe
to schedule with zero current effect.
.gitea/workflows/vehiq-oltp-ingest.yml is present, cron-scheduled, and
defaults to VEHIQ_INGEST_MODE=preview (read-only, never writes) with no
secrets configured — safe to merge with zero production effect until
explicitly activated per the doc's "what activating this requires".
Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>
Recorded decision (Simon 2026-08-31): personal data may be stored under
legitimate interest ONLY as business contacts bound to a company/org entity.
Gate enforces it: legitimate_interest requires purpose=business_contact and
entity_linked_only=true; recordStored() requires entity_id (orgnr/rgs/LEI,
hashed) and refuses person-only rows. +1 test; suite 167 pass | 1 skip;
typecheck clean.
PiiGateService makes any future personal-data source (ratsit.se and other
directory domains) technically GDPR-OK before the first byte is stored:
- recorded legal basis (art. 6) with basis_ref evidence, no policy => fail-closed
- data minimisation via field allowlist (art. 5(1)(c))
- storage limitation + enforceRetention purge (art. 5(1)(e))
- sha256-only identity keys + log redaction (art. 25/32)
- accessStatement / erase subject-rights surfaces (art. 15/17)
- robots/rate confirmation required for web+api sources; no stealth/bypass
- audit trail via AUD0 at risk L3 (0pt.pii_*)
- gate enabled by default in SourceQueryService
10 new tests; full suite 166 pass | 1 skip; typecheck (blocking gate) clean.
connection_config.sqlEndpoint on POST /api/0pt/sources/{sourceId}/query
was passed straight from the request body to the mssql TDS driver as the
outbound connection target, with only its http(s):// scheme stripped —
no hostname allowlist or private/loopback/link-local/cloud-metadata
guard. This route already requires a Zitadel bearer token + tenant match
(P014 tenant guard), so this was not anonymously exploitable, but it is
the exact SSRF shape found and exploited-in-principle elsewhere in this
estate tonight (press0), so it's worth closing as defense-in-depth.
Adds src/lib/security/ssrf-guard.ts: resolves the endpoint's hostname
and rejects any address in a private/loopback/link-local/cloud-metadata/
unspecified range (RFC1918, 127.0.0.0/8, 169.254.0.0/16 incl. the
169.254.169.254 metadata address, ::1, fc00::/7, fe80::/10, IPv4-mapped
IPv6), failing closed on DNS resolution errors. The range-check logic is
ported 1:1 from press0's src/lib/security/ssrf-guard.ts built the same
night for the same bug class against a fetch()-based URL there; this
module's surface differs because sqlEndpoint is a bare host[:port], not
a fetchable URL. The DNS lookup is injectable (mirrors this adapter's
existing fetchImpl pattern) so tests are hermetic.
FabricOneLakeAdapter.getPool() now calls the guard before loadDriver()/
getToken()/opening the TDS pool, so a blocked target never reaches a
real outbound connection attempt or spends an AAD token exchange.
Tests: new tests/p0/fabric-onelake-ssrf-guard.test.ts unit-tests the
guard module directly (range boundaries, IPv4-mapped-IPv6 bypass
attempt, fail-closed on DNS failure/empty answer). New adapter-level
tests in tests/p0/fabric-onelake-adapter.test.ts prove (a) a
private-IP-resolving sqlEndpoint is rejected before fetch/connect is
ever called, (b) a literal loopback sqlEndpoint is rejected the same
way, and (c) a normal external Fabric hostname still passes the guard
and completes the query — the guard doesn't false-block legitimate
targets. Existing adapter tests are updated to inject a deterministic
DNS resolver so they stay hermetic instead of depending on real DNS.
Full suite: 997 passed, 0 failed, 184 skipped (self-skipping realinfra
tests with no live backend this run). typecheck and lint clean on all
touched files.
Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>
The `resolveStoreMode` kill switch (Master-DoD §3/§10) covered the newer
`src/repositories/*` stores but not two older paths, both of which are
live production execution paths today.
1. LEGACY NEXT.JS STORES. `lib/orders-store.ts` and
`lib/subscriptions-store.ts` predate `src/platform/dependency-mode.ts`
and called `getSql()` directly, silently falling back to module-global
`Map`s when no database is configured. For orders, payments and
subscriptions that is not a degraded mode, it is invisible loss of
money-bearing state: the API returns 201 and the row evaporates on the
next restart or on any other replica. The live 0pt deployment
(Coolify app 54) runs NODE_ENV=production with DATABASE_URL,
POSTGRES_URL and POSTGRES_PRISMA_URL all unset — so this is the
behavior in production right now, across 7 API routes.
These now go through `getSqlForDurableStore()`, which reuses the SAME
`resolveStoreMode` decision every `src/repositories/*` store already
uses rather than re-implementing a second production check. The reuse
is load-bearing: `isProductionRuntime` deliberately exempts
`NEXT_PHASE=phase-production-build`, because `next build` runs
page-data collection under NODE_ENV=production while serving nothing.
A naive `process.env.NODE_ENV === "production"` test here would have
failed the BUILD instead of protecting a request path.
2. LOCALBLOBSTORE. It writes document bytes to a container-local disk
while `document-capture-service` registers an artifact pointer
claiming the bytes are stored — durable object storage silently
swapped for ephemeral disk. It is the default blob store for both the
capture and extraction services. In production it now raises
DEPENDENCY_UNAVAILABLE (capability blocked: no ST0RE bytes path
exists yet) instead of fabricating success. Asserted per-operation,
not in the constructor, so merely wiring the services is not fatal —
only actually persisting or reading bytes is.
Evidence: full suite 956 passed / 184 skipped, typecheck clean,
dependency-cruiser 0 errors, lint identical to main's pre-existing
baseline (17 problems, none in changed files). Both guards asserted in
both directions and mutation-tested: bypassing resolveStoreMode -> 2
failed; dropping the write assert -> 2 failed; dropping the read assert
-> 1 failed; all restored -> 8 passed. 7 non-test call sites.
DEPLOYMENT NOTE: merging changes no runtime (no Gitea webhook exists on
any repo; deploys are manual). Deploying this build REQUIRES
provisioning DATABASE_URL for 0pt first, or the order/subscription
routes will correctly return DEPENDENCY_UNAVAILABLE instead of silently
accepting orders into memory. That provisioning is a Simon action.
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Certifieringslanan (BI-P147) hittade det; jag verifierade det oberoende mot
LICENSE-filen vid respektive git-tagg 2026-09-07:
v0.208.1 Business Source License 1.1 — source-available, INTE öppen källkod
v0.209.0 Apache License 2.0
dbt Labs licensierade om MetricFlow vid 0.209.0. Den gamla pinnen låg alltså
exakt en patch under gränsen. För en självhostad plattform vars hela premiss
är att inte ha ett proprietärt runtime-beroende är det en licensdefekt, inte
en smaksak: BUSL begränsar produktionsanvändning fram till sitt change date.
Notera metodpunkten: PyPI redovisar INGET licensfält för någondera versionen,
så en metadatakontroll hade inte hittat något. Skillnaden syns bara i
LICENSE-filen vid taggen.
Ny grind: `tests/unit/metricflow-licence.test.ts` asserterar att pinnen ligger
på eller över `METRICFLOW_FIRST_OPEN_SOURCE_VERSION`, och — som negativ
kontroll — att alla tre kända BUSL-versionerna (0.207.3, 0.208.0, 0.208.1)
faktiskt förkastas av samma jämförelse. Utan den andra assertionen hade den
första kunnat passera med en jämförelse som inte diskriminerar.
Detta gör INTE MetricFlow verifierad: `providerReadiness()` returnerar
fortfarande `verified: false`, ingen supply-chain-granskning är gjord och
providern har aldrig körts (ingen Python-runtime är provisionerad). Att laga
licensen är inte samma sak som att bevisa komponenten.
typecheck rent, 2/2 tester gröna.
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Same shape as the BI-P029 finding in the previous commit. `bi-slo.ts` said
"evidence/bi-p103 records the first baselines". There was no evidence/bi-p103.
The catalogue and the evaluator were both real and both well covered — against
SYNTHETIC events. A synthetic event carries whatever duration the test author
typed, so that suite proves the arithmetic and nothing about the system: it
cannot tell you whether a dashboard query takes 8 ms or 8 s. BI-P103's DoD is
one sentence, "measured baselines exist", and it was the part not done.
tests/integration/bi-p103-slo-baseline.integration.test.ts runs 460 real
queries against a live ClickHouse over a 400 000-row fact table -- 120 cold
dashboard queries (each a different bucket so the BI-P020 cache genuinely
misses), the same 120 again through the same shipped cache now hitting, and
220 exploration queries, that last number chosen so explore.p99's min_samples
of 200 is MET rather than reported low-confidence. Every duration is
wall-clock around the real call, and the telemetry goes through the shipped
producer and the shipped evaluator, so the evidence is written by the code
under test.
Measured: 7 of 11 objectives produced a number, `fully_measured` false,
measured_fraction 0.64. The four that did not each say why -- dashboard
freshness is a property of the dataset, the exploration timeout is a
configured ceiling, report completion has no producer in 0pt at all, and
report delivery is N0TIFY's receipt. A baseline table listing only the rows
that worked is how an estate comes to believe it measures more than it does.
The evidence also records what is NARROWER than production, because these
numbers will be quoted: the cached path is the in-process reference cache and
not Valkey, every latency is the engine round trip with no HTTP or
authorization layer, and there is no concurrency.
The suite is adversarial about its own inputs: a "cold" sample that came from
the cache fails, a cache "hit" that missed fails, a cached p95 that is not
faster than the cold p95 fails, and dropping the exploration sample to 60
fails on explore.p99 being low-confidence rather than quietly recording a
p99 over 60 requests.
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
`semantic-provider.ts` cited tests/integration/semantic-parity.integration.test.ts
as the proof of the one BI-P029 DoD surface it claimed was closed, and
`providerReadiness().proven` asserted the comparison had been made against live
ClickHouse. The file did not exist. The citation was the evidence.
Writing it exposed a divergence that had been shipping:
`compileRequestToClickHouse` had no notion of a metric's OWN filters
(BI-P028 `MetricSpec.filters`). `ResolvedMetric` did not carry them and the
compiler never asked, so a metric defined as "invoiced revenue EXCLUDING
internal transfers" executed as "invoiced revenue". On the committed fixture
that is 11 249 instead of 1 250 for one country -- a plausible-looking number,
9x too large, under the correct canonical metric id. The registry, the
governance checks and the MetricFlow manifest all carried the filter; only the
thing that actually ran did not.
* `ResolvedMetric` gains `definition_filters`, already resolved onto the
physical dim_names key so the compilers stay dumb.
* The ClickHouse compiler folds them into THAT metric's conditional
aggregate, never into the shared WHERE -- two metrics in one request may
exclude different rows, and a shared predicate would apply one metric's
meaning to the other. `renderFilter`'s uniquifier becomes a string so
request filter `0` and metric filter `0_0m` cannot collide in the
parameter map.
* `compileRequestToSemantic` carries them through, and
`MetricFlowProvider.compile` emits them per metric.
The test runs both compile paths against one live ClickHouse: 0pt's `sumIf`
compiler, and a renderer driven only by the provider's structured query that
shares no SQL-generation code and emits `sum(if(...))` with inlined literals.
It asserts the two SQL strings differ, so parity is not vacuous. Backing the
fix out reproduces the pre-fix numbers, and the suite records them itself
rather than quoting a console: evidence/bi-p029/semantic-parity.json.
Three of BI-P029's four DoD surfaces stay OPEN and still say so: MetricFlow
itself is not executed (no Python runtime is provisioned here), Superset is not
implemented, and the AI Analyst path has not been compared.
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
admin
siax-bot