The weights in the matcher are chosen by hand, and two of them were
adjusted last week because they produced a wrong answer on real data. A
pair agreeing to within 1%, from a merchant the matcher recognised, still
scored 0.62 because a date had drifted. That says more about the
constants than about the receipt.
So the formula keeps what it is good at and hands over what it is not.
Above 0.80 it decides alone: an exact amount from a recognised merchant
needs no second opinion, and buying one for every pair would be latency
and cost for a verdict nobody doubts. Below 0.60 nothing is asked either,
because inviting a model to rescue a pair the evidence does not support
is how plausible wrong answers get made. Only the band between goes for
adjudication, which on a real ledger was five pairs against one the
formula had already settled.
The question is a yes or no with a reason, never a score. An earlier
design in this feature asked a model to rate its own certainty and it
anchored on round numbers, which is what the calibration literature
predicts. Judging concrete evidence and explaining the judgement is a
different task.
A verdict is checked, not trusted: a key nobody asked about is dropped, a
pair answered twice keeps its first answer, and a failed call accepts
nothing, leaving the run exactly where the arithmetic left it. The
formula's own score is stored unflattered next to the verdict, because
dressing it up would hide the uncertainty that sent the pair for a second
opinion in the first place, and agent_metadata records which instrument
decided.
Adjudication runs on a dry run too. A provkörning is meant to show what a
real run would propose, and skipping it would show a smaller, different
answer than the one that lands.
Measured on a real ledger: five uncertain pairs asked, two accepted with
reasons a human can check ("Samma leverantör och belopp stämmer inom
rimlig valutamarginal"), three rejected. Proposals went from one to three.
Co-authored-by: Jakob Wennberg <311770904+jakobwennberg-oss@users.noreply.github.com>
Co-authored-by: Claude Opus 5 (1M context) <noreply@anthropic.com>
Jakob Wennberg
88140e5774