a0dd6b76532af28fa76ebffc013d6f00ba6d10da
9
Commits
| Author | SHA1 | Message | Date | |
|---|---|---|---|---|
|
|
b3239487b7 |
feat(document-workbench): wire customer upload, structuring, credits and readiness
CI (SIAX Cloud) / integration (push) Skipped
CI (SIAX Cloud) / sonar (push) Skipped
masterplan-lock / check (push) Successful in 10s
CI (SIAX Cloud) / sonar (pull_request) Skipped
Security Scan (SIAX Cloud) / security (push) Successful in 13s
masterplan-lock / check (pull_request) Successful in 7s
CI (SIAX Cloud) / security (push) Successful in 26s
Security Scan (SIAX Cloud) / security (pull_request) Successful in 17s
CI (SIAX Cloud) / security (pull_request) Successful in 19s
CI (SIAX Cloud) / quality (push) Successful in 1m9s
CI (SIAX Cloud) / quality (pull_request) Successful in 2m26s
CI (SIAX Cloud) / integration (pull_request) Successful in 2m33s
- POST /api/0pt/documents/upload: real multipart bytes into DOC0 through the durable attempt journal and the existing coordinator; verified tenant/actor only, project authorization fail-closed. - POST /api/0pt/documents/structure: DOC0 segments -> INF0 -> evidence-bound validation -> optional JSON/CSV export, with credit reserve/settle. - Credits: balance, server-owned price book, Stripe Checkout and idempotent webhook reconciliation via a Stripe re-read of session + payment intent. - /portal/dokument and /portal/krediter: SIAX-styled workspace. - @siax/access: universal, framework-neutral OpenFGA project relationship primitive reusable by DOC0/N0D/other products. - Readiness: real Redis PING probe instead of presence-only; queue is now a live dependency when configured. - Dockerfile: copy every workspace manifest so --frozen-lockfile works. - Format/build hygiene: .prettierignore, .worktrees ignored, valid manifest YAML. |
||
|
|
b212a11870 |
Merge pull request 'chore: untrack committed __pycache__ and scraper output dumps (P4-4)' (#22) from chore/untrack-pycache-and-scraper-output into main
Deploy to Coolify / deploy (push) Failing after 1s
CI (SIAX Cloud) / security (push) Successful in 15s
Security Scan (SIAX Cloud) / security (push) Successful in 15s
CI (SIAX Cloud) / quality (push) Successful in 37s
CI (SIAX Cloud) / integration (push) Failing after 47s
|
||
|
|
9740e20818 |
feat(data-os): TS/Python quality bridge + provenance fields + fault-injection tests
CI (SIAX Cloud) / security (push) Successful in 23s
Security Scan (SIAX Cloud) / security (push) Successful in 22s
CI (SIAX Cloud) / integration (push) Successful in 29s
CI (SIAX Cloud) / quality (push) Successful in 45s
CI (SIAX Cloud) / security (pull_request) Successful in 27s
CI (SIAX Cloud) / integration (pull_request) Successful in 31s
Security Scan (SIAX Cloud) / security (pull_request) Successful in 29s
CI (SIAX Cloud) / quality (pull_request) Successful in 48s
Continues P010 (branch renamed from feat/p010-source-registry-persistence):
closes the three remaining BEFORE_STATE.md gaps (§2 TS/Python split-brain,
§3 missing provenance fields, §4 untested failure modes), all additive to
the production pipeline — the Python QA filter's delivery decision is
unchanged by any of this.
TS/Python bridge (§2):
- New route surface app/api/0pt/quality/* (rules/runs/issues CRUD +
POST /api/0pt/quality/bridge-run) over the shared QualityService
singleton (src/services/quality.ts), fixing the same "no shared
instance" defect the Source Registry had.
- src/quality/quality-bridge.ts auto-provisions a deterministic,
tenant-scoped baseline rule set on first use so the Python engine never
pre-registers rules through the CRUD surface.
- services/api/workers/quality_bridge.py: the Python-side HTTP client.
Feature-flagged off by default (OPT_QUALITY_BRIDGE_ENABLED), fails open
on any error (never raises into run_job, never changes `delivered`).
Advisory only — real per-row gating against the TS engine isn't
implementable yet (ExpectationEngine returns dataset-level pass/fail
counts + samples, not a per-row boolean array); documented, not faked.
- Live-verified end-to-end: `next dev` (AUTH_ENFORCE=false) + the real
Python pipeline hitting the real brreg.no API, round-tripping through
the real bridge route, receiving a real QualityService score.
Provenance (§3): sources/connectors.py's Provenance dataclass gains
retrieved_at, transform, connector_version, producer, quality,
entity_resolution, artifact_ref (all previously absent from the real
dataflow entirely) via a make_provenance() factory used by every connector.
resolve.py stamps entity_resolution on merged records' provenance (never on
unmerged ones). pipeline.py stamps artifact_ref at delivery and quality
once the bridge succeeds.
Failure tests (§4), real fault injection against real code (services/api,
new pytest + respx suite, not vitest mocks):
- dedup-determinism + duplicate-input + out-of-order: resolve.py under
shuffled/interleaved/exhaustive-permutation input, plus an explicit,
verified test of the one case that IS legitimately order-sensitive
(non-transitive similarity in a greedy clusterer) instead of hiding it.
- schema-evolution: respx-mocked upstream responses with added/removed/
alien-shaped fields through the real brreg connector + pipeline.
- restart: connector exceptions mid-job (including one from respx itself)
never crash run_job.
- backfill: the real property that holds given no persistence layer
exists (replay is compute-idempotent) — NOT a fabricated dedup-store
test.
- CDC-interruption and true backfill/delivery-store idempotency:
explicitly documented as not implementable against this codebase
(no CDC pipeline, no delivery store exist anywhere) via skip markers
with reasons, in tests/test_known_gaps.py, rather than silently
dropped or faked.
Testing: 256/256 vitest (249 pre-existing + 7 new bridge route/service
tests) and 28/28 pytest (2 documented skips) green locally. tsc -p
tsconfig.check.json has zero new errors (14 pre-existing errors, all in
the untracked/unaudited src/fleet, src/graph, src/metrics, src/projections
paths this pass does not touch, per BEFORE_STATE.md).
No PR opened, per instructions.
Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>
|
||
|
|
6c17f0a0d9 |
chore: untrack committed __pycache__ and scraper output dumps (P4-4)
Security Scan (SIAX Cloud) / security (push) Successful in 18s
CI (SIAX Cloud) / security (push) Successful in 21s
CI (SIAX Cloud) / quality (push) Successful in 42s
CI (SIAX Cloud) / security (pull_request) Successful in 20s
CI (SIAX Cloud) / quality (pull_request) Successful in 41s
Security Scan (SIAX Cloud) / security (pull_request) Successful in 16s
13 compiled .pyc files under services/{api,scrapers}/**/__pycache__/ and 50
scraped brreg.no JSON pages under services/scrapers/output/ were committed to
git despite __pycache__/*.pyc already being covered by .gitignore -- they
predate that rule and git does not retroactively drop already-tracked files
that start matching an ignore pattern.
Neither set is referenced by any source, test, or doc as a fixture (verified
via repo-wide grep); both are pure runtime artifacts regenerated on demand.
Untrack them with git rm --cached, delete the local copies, and add
services/scrapers/output/ to .gitignore so scraper runs stop reintroducing
this. No source or test file touched; pnpm test still 156 passed / 1 honestly
skipped (realinfra ST0RE check) after the change.
Matches docs/compliance/BACKLOG.md P4-4, explicitly flagged as hygiene doable
without a product decision. Picks up unfinished work from an earlier,
never-merged branch of the same name.
|
||
|
|
48d31d0081 | ci(workflows): split CD/security/nightly under .gitea/workflows + gitignore hygiene (protected-path PR) | ||
|
|
bcf285f057 |
merge: unify GitHub main into Gitea main (0pt)
Merges github/main (6 commits: 17-bug audit fix, CI quality gate,
.env.example + .gitignore fix, missing platform clients + registry
services, docs accuracy overhaul) into gitea/main (8 commits: Iceberg
write boundary + live ST0RE verification, lakehouse mapping routes,
Waves 1-4 foundational hardening).
16 conflicts. Both sides had independently written the four
src/clients/*.ts and the two registry services, so those are combined
rather than picked:
- st0re-client.ts: GitHub's endpoint wins on evidence. POST goes to
/v1/st0re/artifacts/register, not /v1/st0re/artifacts -- verified in
sax3l/st0re src/api/routes/artifacts.ts, where the bare path is a
GET-only list route. Gitea's URL normalisation, error-body text and
mock kept.
- n0d-client.ts: field is `status`, not `state`, with N0D's real 13-state
union -- verified in sax3l/n0d packages/schemas/n0d.job.schema.json,
which says explicitly not to add "succeeded". Gitea's mock echo,
URL normalisation and error-body text kept.
- cl0ud-client.ts: Gitea's fail-closed HTTP normalisation kept (a 200
with no `decision` reads as "deny", not as a silent allow). GitHub's
PolicyDecision/PolicyDecisionValue naming kept; PolicyResult retained
as an alias.
- aud0-client.ts: permissive union of both input types (optional
payload/metadata/correlation_id), GitHub's echo return, Gitea's URL
normalisation and error-body text, plus a JSON-parse fallback so a
successful POST with an empty body does not throw.
- dataset-service.ts / source-service.ts: GitHub's record/DTO split and
event naming (0pt.source_created / 0pt.create_source, which the merged
docs already reference) plus Gitea's name validation, description /
created_by fields and actor_id derivation. List filters are the union.
- lakehouse mapping routes: GitHub's shared lakehouse-mapping-registry
singletons (without which every write 400'd "mapping not found") AND
Gitea's Zod parseBody validation. Both, not either.
- src/types/source.ts, src/types/dataset.ts: unions of both vocabularies;
GitHub's `access` legal-sourcing field kept.
- .env.example: GitHub's audited inventory plus Gitea's Zitadel /
AUTH_ENFORCE block.
- CLAUDE.md, docs/ICEBERG_WRITE_BOUNDARY.md: GitHub's corrections kept,
Gitea's live-verification kept and marked as superseding the
"would 400 outright" finding (Gitea had already added the required
X-Tenant-Id/X-Actor-Id headers).
- pnpm-lock.yaml: regenerated with pnpm 9.15.9, not merged.
Also corrects CLAUDE.md's claim that the banned services/scrapers stack
was "since removed" -- the removal PR is open, not merged, and this
branch inherits that code from Gitea main. Nothing here resurrects,
extends or re-wires any of it; services/scrapers/ is byte-identical to
Gitea main.
Verified on the merged tree: typecheck (src/ + tests/) clean;
17 test files, 121 passed, 1 skipped (the realinfra test skipping
honestly on ST0RE HTTP 401), 0 failures. Gitea main baseline was the
same 121/1; GitHub main baseline could not even `pnpm install`
("packages field missing or empty") -- Gitea's pnpm-workspace.yaml fix
resolves that. Whole-repo `typecheck:app` reports 9 errors, an identical
set to Gitea main's baseline, so it is advisory in both CI workflows
rather than deleted. Lint: 16 problems, identical to Gitea baseline.
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
|
||
|
|
844220d9f5 | feat(foundation): Wave 3 integrity/persistence/build fixes | ||
|
|
89e180fc49 |
chore: add .env.example, fix .gitignore blanket .env* rule (#3)
.env.example was referenced by README.md and required by docs/09-build-plan.md (Phase 0) but never existed in the repo. Root cause: .gitignore's `.env*` rule silently ignored it too, so `git add .env.example` was always a no-op even if someone created it locally. - Add .gitignore exception: `!.env.example`. - Add .env.example covering every process.env.* read found in this pass (grep-verified across app/, lib/, src/), organized by concern, each section marked "wired in code today" or "documented target, not yet read by code" per docs/08-infrastructure.md. No real values; matches repo convention. Excluded ODOO_BASE_URL: only appears inside a comment describing a past refactor away from module-level env reads, not an actual process.env read. pnpm test: 88 passed, 1 todo, 0 failures (13 files, tests/**/*.test.ts). tsc --noEmit: clean. |
||
|
|
59ebafd993 | Initial commit from v0 |