The one thing this tool cannot afford to do is accuse you wrongly. So we measure it — on real files, with a method you can rerun — instead of asserting it.
npx tsx scripts/false-accusation-rate.ts --frozen (and --v2, --v3, --v4). This is our own hard set — a regression guard, not proof the world agrees.There are two measurements, and they answer different questions.
The frozen public corpus. fa-corpus v1–v4 is a committed, synthetic, redacted set of 170 checks — near-misses a careful checker must stay quiet on, each paired with a real violation it must catch. Run every rules file against every session and count any FAIL: 0 of 170. Anyone can reproduce it from the repo alone with npx tsx scripts/false-accusation-rate.ts --frozen (and --v2, --v3, --v4); it only moves when the checkers change.
The maintainer's own sessions. We also turned the tool on 44 of the maintainer's real sessions (mostly building RuleReceipt itself), frozen as a fixed snapshot, and hand-checked every FAIL. Before 0.1.106, 5 of 26 “broken” verdicts were wrong — all from one cause, a word from a rule text matching in unrelated code. That cause is fixed in 0.1.106, and 0 were wrong after. These sessions are private, so this figure is not publicly reproducible; the frozen corpus above is the reproducible one. Because the fix was tuned on these 44 sessions, a fresh re-measure on new sessions — never used for tuning — is coming, and that becomes the published real-world number.
Precision alone can be gamed by never accusing anything. So there is a second set: hand-labelled sessions that genuinely break a rule (a push to main, an edit to .env, a console.log written into a file, a "tests pass" claim contradicted by a failing run, a forbidden import) — alongside near-misses that must not fire (a push to a feature branch, editing .env.example, a command that only mentions git push). It is a sanity check across the checkers, not a real-world detection rate. Every violation is caught; every near-miss stays quiet.
The strongest version of this claim is the one you check. The self-test ships in the CLI and makes zero network calls — watch it with lsof or Little Snitch:
npx rulereceipt@latest selftest
The full precision and detection measurements are ordinary scripts in the open repository — clone it and run them against your own corpus and sessions.
--llm) is labelled as an opinion and never counted as a verdict. When the evidence doesn't settle a rule, the answer is "couldn't tell," never a guess.