Reviews · ConceptsAvailable

Risk routing

The core claim is a capture curve: reading only the top-scored N% of PRs (the “budget”), what share of defect-inducing PRs land inside that band?

The capture curve is measured on the backtest CLI: scoring there is deterministic, and the holdout half learns hotspot path segments from the repo’s own past defects. Live GitHub App reviews use static hotspot segments plus, when enabled, an LLM diff-reader (ADR-0004) — not the rolling-window learner, and not a model-free hot path.

capture@budget — headline routing metric: % of defect PRs caught at a given flag rate
AUC — budget-independent summary across all flag rates
size-only baseline — every report shows it; if Coldworks can’t beat “biggest diff first,” you should not pay for Coldworks

Per-rule precision (sentry backtest, train half)

rule                    fired   hit   lift
hotspot_path              581    19  2.34x
refactor_pure_mod         332     9  1.94x
deletion_leaning          214     4  0.82x  # dead — kept honest

# every rule ships with its receipts;
# rules that don't pay get reported, not hidden