Reviews · ReferenceAvailable

CLI · doug-backtest

Replay a repository’s merged-PR history and print the full scoreboard. Harvests are cached and checkpointed — kill it and relaunch any time; it resumes.

repopositional · required: owner/repo, e.g. astral-sh/ruff
--limitint · default 300: Merged PRs to harvest, newest-first within the window.
--beforeYYYY-MM-DD: Only harvest PRs created before this date — the right-censoring guard.
--labelsgit | api | both · default git: Defect label source. See Defect labels.
--tokenstring: GitHub token. Default: environment, then the gh CLI.
--cache-dirpath · default .backtest-cache: Harvest cache location. Safe to reuse across runs; seeds bigger runs from smaller ones.
--outputpath: Write the full JSON report here as well as printing tables.
--backfill-detailsflag: Fetch per-file stats + patch text for cached PRs missing them (~1 request/PR; checkpointed, resumable).

Examples

# quick look, 300 PRs
$ doug-backtest astral-sh/ruff

# serious run: deep window, censoring guard,
# patch-level detail, JSON report
$ doug-backtest grafana/grafana \
    --limit 12000 \
    --before 2026-06-15 \
    --labels git \
    --backfill-details \
    --output report.json

# resume after a kill — same command, cache does the rest
$ doug-backtest grafana/grafana --limit 12000 \
    --before 2026-06-15