Data Leakage in Backtesting: The Silent Killer
The most common reason a strategy looks brilliant in testing and dies in live markets is not bad luck. It is information from the future quietly reaching your model.
A backtest showing 85% accuracy is far more likely to contain a bug than a discovery. The bug is usually leakage: your model saw information it could not have had at the time it was supposedly making the prediction.
The five most common sources
1. Scaling before splitting
You normalise your whole dataset, then split it into train and test. The scaler's mean and standard deviation were computed using test-period data, so every training row now carries a whisper of the future. This single mistake accounts for an enormous share of unrealistic backtests.
2. Shifted targets done wrong
If your target is tomorrow's return, every feature must be computed strictly from data available today. An indicator that uses a centred moving average — one that looks both backwards and forwards — is leakage by construction.
3. Survivorship in the universe
Testing a stock strategy on today's index constituents means testing only on companies that survived. The ones that were delisted, acquired or collapsed are missing, and they were exactly the losers.
4. Restated fundamentals
Financial data providers revise figures. If your database holds the corrected version rather than what was actually published at the time, your model is trading on numbers nobody had.
5. Repeated tuning on the same test set
Technically not leakage, but the same outcome. If you adjust hyperparameters until the test score improves, the test set has become part of training. You have optimised for a specific stretch of history.
How to catch it
- Suspicion is a tool. Any equity curve that rises smoothly with shallow drawdowns deserves scepticism before celebration.
- Shuffle the target. Randomise your labels and re-run. Performance should collapse to chance. If it does not, information is flowing where it should not.
- Walk forward. Train on a window, test on the next, roll, repeat. This mirrors reality far better than a single split.
- Add friction. Include spread, slippage and commission. Leaked edges are usually so thin that realistic costs erase them entirely.
A backtest is not evidence that a strategy works. It is evidence that it did not obviously fail on one particular slice of history.
The honest framing
Leakage is not a beginner's mistake. It appears in published academic work and in professional systems. The defence is not cleverness but process: strict temporal separation, walk-forward validation, and treating any exceptional result as a bug report until proven otherwise.
Educational content only. Not financial advice. Historical performance does not guarantee future results.
See the models in action
Reading about forecasting is one thing. Create a free account and get 5 AI predictions a month — no card required.
Start free
