Merges `salvage/20260925/siax-arkiv/siax-work/0pt-stash-stash0` (7 commit(s) not yet in `main`) via PR so CI can gate it.
Latest commit: WIP on feature/0pt-remove-noncompliant-scrapers: 6a6a96e workers: ixbrl_lite — lxml-baserad iXBRL-extraktor (60x snabbar
Generated with Claude Code
Three commits landed directly on main (afd301f, bb6be0f, 4a164c5, all
2026-07-13) that violate CLAUDE.md §1 ("Non-negotiable constraints") and
this repo's own services/api/README.md ("Deliberately excluded — never
add here"):
- services/scrapers/waf_bypass.py: FlareSolverr, cloudscraper, Tor SOCKS5
rotation, free-proxy-pool harvesting, fake identity generation, TLS
fingerprint impersonation (curl_cffi) — the exact bypass/stealth taxonomy
the constraint enumerates.
- services/infra/clickhouse/* + docker-compose.data-stack.yml: pipeline
built solely to ingest that scraper's output.
- services/scrapers/engine.py, engine_v2.py, run_pipeline.py, s3_sync.py,
run_benchmark.py, Dockerfile, docker-compose.yml, requirements.txt,
__pycache__/*.pyc, output/*.json fixtures: the rest of the standalone
scraper stack.
- services/scrapers/README.md: pointed at an external "b0ts" repo for
"deep scraping" and committed a live Hetzner Server2 IP + SSH key path
in plaintext — a confidentiality-rule violation independent of the
compliance one.
- .gitea/workflows/ci.yml: a no-op "Scraper CI" stub added solely for
this code (echoes a string, checks nothing).
- services/api/workers/adaptive_router.py: reverted to the pre-4a164c5
compliant version. The removed version dropped the ACCESS_RANK /
compliant-only docstring and replaced MARKET_SOURCES with ~25 source
ids (proff, gulesider, personensuche, whitepages, northdata, ...) none
of which exist in sources.yaml or connectors.py — dangling references
to sources with no registered licence/robots/access record.
None of the removed source ids were ever registered as compliant
connectors, and no doc (README.md, CLAUDE.md, docs/**) references
services/scrapers, services/infra, clickhouse, or b0ts — this stack was
wired in from outside the documented architecture, not part of any
planned build-plan phase.
Verified:
- Repo-wide grep for flaresolverr|stealth|undetected|proxy.rotat|
waf_bypass|cloudscraper|fake.identity|b0ts now only matches the
compliant docs/code that document the exclusion (CLAUDE.md,
services/api/README.md, services/api/sources/connectors.py,
adaptive_router.py's own docstring).
- python3 -m py_compile services/api/workers/adaptive_router.py: OK.
- adaptive_router.py diffed byte-for-byte against git show 4a164c5^ —
identical restore.
NOT verified (environment limitation, not introduced by this change):
- `pnpm install` cannot complete in this sandbox — no outbound network
access to registry.npmjs.org (all package fetches ETIMEDOUT). No
tsc/next build/vitest could therefore be run. This is a pre-existing
environment constraint: this change touches zero .ts/.tsx files, and
tests/p0/*.test.ts already import `vitest`, which is not present in
package.json or pnpm-lock.yaml (0 occurrences) — that test suite was
not runnable before this change either.
Not in scope of this change: services/api's own __pycache__/*.pyc files
predate this commit and are a separate, pre-existing hygiene issue.
- arelle-release + lxml for taxonomi-generisk extraction
- alla tаggade fakta (concept+context+unit+varde+dimensioner) per bolag
- union = hela svenska bastaxonomin (tusentals variabler)
- iter_corpus() for direkt batch over ytterzip->innerzip->XHTML
- batch-runner i workers/ixbrl_batch.py
Stängd i slutstädning 2026-10-30 (Simon: avsluta och städa allt). Detta är en orörd WIP-/salvage-snapshot som aldrig granskats eller varit avsedd att mergas. Grenen salvage/20260925/siax-arkiv/siax-work/0pt-stash-stash0 finns kvar i repot som säkerhetskopia — återöppna om arbetet ska slutföras.
Stängd i slutstädning 2026-10-30 (Simon: avsluta och städa allt). Detta är en orörd WIP-/salvage-snapshot som aldrig granskats eller varit avsedd att mergas. Grenen `salvage/20260925/siax-arkiv/siax-work/0pt-stash-stash0` finns kvar i repot som säkerhetskopia — återöppna om arbetet ska slutföras.
Blocking a user prevents them from interacting with repositories, such as opening or commenting on pull requests or issues. Learn more about blocking a user.
sax3l
Merges
salvage/20260925/siax-arkiv/siax-work/0pt-stash-stash0(7 commit(s) not yet inmain) via PR so CI can gate it.Latest commit: WIP on feature/0pt-remove-noncompliant-scrapers:
6a6a96eworkers: ixbrl_lite — lxml-baserad iXBRL-extraktor (60x snabbarGenerated with Claude Code
6a6a96eworkers: ixbrl_lite — lxml-baserad iXBRL-extraktor (60x snabbare an Arelle) f64ce2e6346a6a96eworkers: ixbrl_lite — lxml-baserad iXBRL-extraktor (60x snabbare an Arelle) 2f133891a5Stängd i slutstädning 2026-10-30 (Simon: avsluta och städa allt). Detta är en orörd WIP-/salvage-snapshot som aldrig granskats eller varit avsedd att mergas. Grenen
salvage/20260925/siax-arkiv/siax-work/0pt-stash-stash0finns kvar i repot som säkerhetskopia — återöppna om arbetet ska slutföras.Pull request closed