Files
accounted/scripts
Jakob Wennberg 723a0f537b feat(parties): shadow evaluation of the key pre-classifier (#2161)
* feat(parties): shadow evaluation of the key pre-classifier

Scores a deterministic rule router and the Bedrock model router (zero-shot
and with twenty founder examples) against the founder-labelled golden set,
on the same held-out rows, reporting strict agreement plus party TPR/TNR.
Read-only: reads the gitignored JSONL, calls getAiService(), writes a report
next to the input, never opens a database connection.

Results and the definitional disagreements are recorded in the README.

Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>

* docs(parties): settle the four edge rules from the first labelling round

Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>

---------

Co-authored-by: Jakob Wennberg <311770904+jakobwennberg-oss@users.noreply.github.com>
Co-authored-by: Claude Fable 5.1 <noreply@anthropic.com>
2026-09-02 17:13:52 +02:00
..