Commit Graph
7 Commits
Author SHA1 Message Date
Claude CodeandClaude Sonnet 5 07158099ee feat(ingestion): VehIQ OLTP -> ClickHouse watermark extractor (P070)
CI (SIAX Cloud) / integration (push) Skipped
masterplan-lock / check (push) Successful in 7s
CI (SIAX Cloud) / security (push) Successful in 17s
masterplan-lock / check (pull_request) Successful in 9s
Security Scan (SIAX Cloud) / security (push) Successful in 38s
CI (SIAX Cloud) / security (pull_request) Successful in 26s
Security Scan (SIAX Cloud) / security (pull_request) Successful in 26s
CI (SIAX Cloud) / quality (push) Successful in 57s
CI (SIAX Cloud) / integration (pull_request) Successful in 39s
CI (SIAX Cloud) / quality (pull_request) Successful in 59s
Implements Simon's specified pattern (dealer_stock_entry/deal ->
watermark/updated_at -> 0pt ingestion -> ClickHouse) as a P061 connector
pair (PostgresWatermarkConnector, PostgresKeyedLookupConnector) plugged
into the existing P062 IngestionRunner, writing to opt_dataset_projection
via SiaxAnalyticsEngine.ingestProjection.

Built here, not in admin/vehiq: that repo's own ADR-0035 (decisions 1/2/10,
CI-enforced by validate_engine_firewall.py) forbids VehIQ code from ever
holding a ClickHouse credential or hammering OLTP directly for analytics.
See docs/P070_VEHIQ_OLTP_EXTRACTOR.md for the full rationale.

Also corrects a real schema mismatch found while building this: production
has no generic `updated_at` on any of the three target tables (verified
live, read-only, 2026-09-25). dealer_stock_entry/deal use listed_at/
created_at as insert-time watermarks instead (documented caveat: in-place
updates are not caught without a schema change); app.listing has no
timestamp column at all and is deliberately not covered yet.

Vehicle dimension lookup is narrow by construction: keyed on exactly the
vehicle_id set referenced by the batch just read, never a scan of the
98M-row vehiq.vehicle table (connector-level validate() refuses that table
by name; runner-level tests assert the query only ever touches referenced
IDs).

Verified: fixture-backed connector/runner tests (watermark+limit,
same-timestamp tie at a page boundary, crash/resume, empty-table case,
narrow-lookup query-count assertions) + a real disposable Postgres
container (caught two real Date-handling bugs fixtures could not) + a
read-only proof against actual production (server10, vehiq-db): the
extractor's exact SQL returns zero rows, no error, confirming it is safe
to schedule with zero current effect.

.gitea/workflows/vehiq-oltp-ingest.yml is present, cron-scheduled, and
defaults to VEHIQ_INGEST_MODE=preview (read-only, never writes) with no
secrets configured — safe to merge with zero production effect until
explicitly activated per the doc's "what activating this requires".

Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>
2026-09-25 19:11:15 +02:00
admin 655a71ebd7 feat(HSP-085): migration-studio — T01 ej importerbart döljs EJ ur nämnaren (ratio = importable/total), T02 session-resume med tenant-check + checkpoint + resolved conflicts, T03 coverage-claim SCOPED (aldrig komplett M365-ersättning), 8 tester
masterplan-lock / check (push) Successful in 7s
masterplan-lock / check (pull_request) Successful in 5s
Security Scan (SIAX Cloud) / security (push) Successful in 15s
CI (SIAX Cloud) / security (push) Successful in 17s
Security Scan (SIAX Cloud) / security (pull_request) Successful in 13s
CI (SIAX Cloud) / security (pull_request) Successful in 15s
CI (SIAX Cloud) / quality (push) Successful in 36s
CI (SIAX Cloud) / quality (pull_request) Successful in 37s
CI (SIAX Cloud) / integration (pull_request) Successful in 43s
CI (SIAX Cloud) / integration (push) Has been skipped
2026-09-22 22:18:01 +02:00
admin 85b0e6cd72 feat(HSP-084): cutover — T01 detectDualWriters (båda aktiva = STOPPAD), T02 handleLateSourceChanges (carry eller stopp, aldrig tyst förlorad), T03 buildRollbackPlan (targetWritesCarriedBack + käll-writer återaktiveras + konsistensverifiering), 8 tester
CI (SIAX Cloud) / quality (pull_request) Failing after 38s
CI (SIAX Cloud) / integration (push) Has been skipped
masterplan-lock / check (push) Successful in 4s
CI (SIAX Cloud) / security (push) Successful in 15s
Security Scan (SIAX Cloud) / security (push) Successful in 14s
masterplan-lock / check (pull_request) Successful in 5s
CI (SIAX Cloud) / security (pull_request) Successful in 17s
Security Scan (SIAX Cloud) / security (pull_request) Successful in 16s
CI (SIAX Cloud) / integration (pull_request) Successful in 26s
CI (SIAX Cloud) / quality (push) Failing after 29s
2026-09-22 21:00:19 +02:00
admin 45dd7dc6e2 feat(HSP-067): exit-manifest — T01 open formats only (portabel till icke-SIAX-mål), T02 notice-krav följer tillåtna artefakter, T03 nonExportable EXPLIKIT med skäl (aldrig tyst utelämnat), 7 tester
CI (SIAX Cloud) / integration (push) Has been skipped
masterplan-lock / check (push) Successful in 4s
CI (SIAX Cloud) / security (push) Successful in 15s
masterplan-lock / check (pull_request) Successful in 12s
Security Scan (SIAX Cloud) / security (pull_request) Successful in 14s
CI (SIAX Cloud) / security (pull_request) Successful in 14s
CI (SIAX Cloud) / quality (pull_request) Successful in 32s
Security Scan (SIAX Cloud) / security (push) Successful in 1m21s
CI (SIAX Cloud) / quality (push) Successful in 1m34s
CI (SIAX Cloud) / integration (pull_request) Successful in 1m46s
2026-09-22 20:14:20 +02:00
admin 57d5ff36fe feat(HSP-083): migration-runner — InMemoryCheckpointStore (T02 cursor commit EFTER durable target-ack), idempotent-sink replay-säkerhet (T01 krasch → replay räknad, inga dubbletter), T03 VERIFIED kräver per-partition content-hash (count-only = UNVERIFIED), 6 tester
CI (SIAX Cloud) / integration (push) Has been skipped
masterplan-lock / check (push) Successful in 4s
CI (SIAX Cloud) / security (push) Successful in 12s
CI (SIAX Cloud) / security (pull_request) Successful in 19s
Security Scan (SIAX Cloud) / security (pull_request) Successful in 18s
CI (SIAX Cloud) / integration (pull_request) Successful in 26s
masterplan-lock / check (pull_request) Successful in 31s
Security Scan (SIAX Cloud) / security (push) Successful in 38s
CI (SIAX Cloud) / quality (push) Successful in 1m15s
CI (SIAX Cloud) / quality (pull_request) Successful in 1m17s
2026-09-22 16:55:08 +02:00
admin 23d05d9379 feat(HSP-082): migration-mapping — decideEntityMerge (T01 identiska namn EJ ihopslagna utan bevis), transformRecord (T02 åäö byte-exakt + UTC-normalisering med original-offset), aclBlockingCheck (T03 saknad ACL-mappning BLOCKERANDE), 10 tester
CI (SIAX Cloud) / quality (push) Successful in 1m7s
CI (SIAX Cloud) / integration (push) Has been skipped
masterplan-lock / check (push) Successful in 4s
masterplan-lock / check (pull_request) Successful in 5s
CI (SIAX Cloud) / security (pull_request) Successful in 15s
Security Scan (SIAX Cloud) / security (push) Successful in 23s
Security Scan (SIAX Cloud) / security (pull_request) Successful in 18s
CI (SIAX Cloud) / integration (pull_request) Successful in 26s
CI (SIAX Cloud) / security (push) Successful in 39s
CI (SIAX Cloud) / quality (pull_request) Successful in 36s
2026-09-22 15:37:35 +02:00
admin c1c7a33d65 feat(HSP-081): discovery-inventory — inventory-report (T01 write-scoped sources EXCLUDED från inventeringen, T02 persondata REDACTED till referenser rekursivt, T03 otillgängliga källor = okända med real fel — aldrig tyst droppade), 6 tester
CI (SIAX Cloud) / integration (push) Has been skipped
masterplan-lock / check (push) Successful in 5s
Security Scan (SIAX Cloud) / security (push) Successful in 15s
CI (SIAX Cloud) / security (push) Successful in 17s
masterplan-lock / check (pull_request) Successful in 4s
CI (SIAX Cloud) / security (pull_request) Successful in 15s
Security Scan (SIAX Cloud) / security (pull_request) Successful in 13s
CI (SIAX Cloud) / quality (push) Successful in 51s
CI (SIAX Cloud) / integration (pull_request) Successful in 30s
CI (SIAX Cloud) / quality (pull_request) Successful in 37s
2026-09-22 15:05:29 +02:00