A Working Checklist Against Backtest Overfitting

Placeholder article. This is seed content so the site renders populated. Replace with real research before launch.

Every quant has a strategy graveyard, and most of the tombstones read the same way: beautiful in-sample, dead on arrival live. Overfitting is rarely one catastrophic mistake — it is an accumulation of small, defensible-looking choices. This post publishes the actual checklist run against every strategy in this research program before it gets paper capital, let alone real capital.

The checks split into three groups. Statistical: deflated Sharpe ratio accounting for the number of trials, probabilistic Sharpe against a 0.5 benchmark, and a minimum backtest length derived from the strategy's autocorrelation. Structural: parameter plateau analysis (a strategy that only works at lookback=20 but not 15 or 25 is a coincidence, not an edge), walk-forward degradation ratio, and sub-period consistency. Boring: point-in-time universe construction, dividend and split handling, and timestamp alignment between signal and execution.

The boring group catches the most failures. Survivorship bias in the universe alone routinely adds 2-4% annualized to equity strategy backtests. Look-ahead in fundamental data adds more. Before running a single statistical test, verify the data could actually have been known at the timestamp the backtest trades on — most "alpha" dies right there.