Expand description
What precision a detector can reach at our prevalence (D23).
Published detector numbers are measured on corpora that are roughly one-third positive, because they sample same-declaration parallel edits. Deployed against a mainline where bad merges are well under one percent, the same detector reports the same recall and a completely different precision — the false positives are drawn from a vastly larger pool of negatives.
That transformation is the difference between “60% recall at 43% precision, promising” and “one true alarm per 160”, so it should be computed rather than eyeballed. Recall and specificity are properties of the detector and carry across; precision is a property of the detector and the population and does not.
§What the prevalence here actually is
The measured 0.004 is choir-queue::corpus’s revert-labelled bad-merge
rate on rust-lang/rust — merges that history took back. That is not
the same event as “merges containing a semantic conflict”: it misses
defects fixed forward and includes reverts that were not defects. It
is used because it is the only prevalence anyone has measured on a
real mainline, and because the operational question is about flagging
merges, not about conflicts in the abstract.
Structs§
- Detector
- A detector’s prevalence-independent behaviour.
Functions§
- required_
false_ positive_ rate - The false-positive rate a detector must not exceed to reach
target_precisionatprevalencewithrecall.