Three Ways Your Backtest Is Lying to You

There is a particular kind of chart that should make you cautious: the backtest equity curve that rises in a near straight line, with shallow drawdowns and returns that would embarrass a professional fund. It looks like an edge. It is usually an artifact.

A backtest replays a strategy against historical data. Done carefully, it is one of the most useful things you can do before committing capital. Done carelessly, which is how most are done, it becomes an elaborate way to mislead yourself with arithmetic.

Three failures account for the majority of the damage.

Overfitting

Overfitting occurs when you tune a strategy until it fits the past precisely. You test a 50-day moving average and the result is acceptable. You try 47 days and it improves. You add a filter excluding Tuesdays because Tuesdays performed poorly in your sample. You continue adjusting until the equity curve looks impressive.

What you have built is a model of the noise in one specific slice of history. Markets contain substantial randomness, and with enough parameters and enough adjustment you will always find a combination that would have produced returns in the past. That combination has learned the coin flips rather than a durable pattern. Applied to data it has not seen, it fails, because the next set of coin flips is different.

The diagnostic is fragility. A genuine edge tolerates small changes to its parameters, so 50 days and 55 days should produce broadly similar results. An overfit strategy collapses the moment anything shifts, because it was balanced on the specific quirks of the sample.

The fix: partition the data. Build and tune on one portion, the in-sample period, then test once on a portion you never examined during development. If performance holds on unseen data, the result may be real. If it deteriorates sharply, you overfit. Walk-forward testing is stronger still, repeatedly optimizing on a rolling window and testing on the period that follows, which approximates how you would retune a system in practice. The underlying rule: a strategy that has not been tested on data it was not built on has not been tested.

Look-ahead bias

Look-ahead bias means using information the strategy could not have possessed at the moment of the trade. It is subtle, common, and manufactures returns from nothing.

The standard example: a rule buys when the daily close exceeds some level, and the code executes that purchase at the same day's close. But the closing price is only known after the close. In practice you could not have acted on it. In the backtest you did, capturing a move that was never available.

It appears in less obvious places. Using restated earnings figures, the corrected numbers published months later, to drive a decision the strategy supposedly made on the original release date. Or its close relative, survivorship bias: testing an equity strategy only on companies that still exist, silently excluding every one that failed. The universe becomes stacked with survivors, and the strategy appears far safer than trading the real market would have been.

Each of these leaks future knowledge into a past decision, and future knowledge is worth a fortune your live account will never have.

The fix: for every rule, ask whether you would actually have held that information at the moment the trade fires. If a signal is generated on the close, the earliest execution is the following open. If you use fundamental data, use the values as originally reported on the date they were reported, not as later revised. And test on a universe that includes the companies that failed.

Ignored transaction costs

This is the quietest of the three, because the strategy genuinely worked until the market took its share.

Backtests frequently assume execution at the exact price on the chart, at no cost. Live trading is not free. There is the spread between bid and ask, paid on every entry and exit. There is slippage, the gap between the expected price and the realized fill, which widens precisely when markets move quickly and you most want out. There are commissions. At size there is market impact, your own order moving the price against you.

For a strategy trading a few times a year, none of this matters much. For a high-turnover strategy it is fatal. A system capturing a small edge across hundreds of trades can look excellent on paper and lose money live, because each round trip surrenders a few cents to the spread and slippage. A few cents multiplied by several hundred trades is the entire edge.

The fix: model costs explicitly. Subtract a realistic spread and slippage estimate from every fill, add commissions, and for anything trading frequently, use pessimistic assumptions rather than optimistic ones. If the edge survives only when trading is free, you do not have an edge. You have a cost you have not yet paid.

The honest backtest

Combine the three corrections and you have a discipline: test out-of-sample, refuse to use information you would not have held, and charge yourself realistic costs. Most strategies that looked exceptional will stop looking exceptional. That is not a failure of the method. It is the method working, showing you the truth before the market charges you to learn it.

A backtest is not there to build confidence in a strategy. It is there to break the strategy while breaking it is still free. The systems that survive a serious attempt to kill them are the only ones worth trading.

We built a checklist for running an honest backtest, turning every trap above into a set of checks you can run before trusting a result. It is free, and the link is in the newsletter.