Why I Stopped Trusting Backtests — Even Good Ones

Walk-forward optimisation fixed the curve-fitting problem. It didn't fix the fact that I still hadn't watched the strategy trade a single real signal before trusting it.

Walk-forward testing was a real improvement, and I want to be honest that it wasn't the end of the story — because it's tempting to write "and then I fixed it" and move on, and that isn't quite what happened.

A walk-forward result is still a backtest. It's a much more honest backtest than the curve-fitted version I started with, but it's still built entirely from historical data, replayed against rules that assume things: that a signal fills at the price it says it does, that slippage behaves the way you modelled it, that the market's character over the next month resembles its character over the window you just tested. Those assumptions are usually close enough to true. Occasionally they aren't, and a backtest — even a rigorous one — has no way to tell you which case you're in.

The step I'd skipped

The step I was missing was forward testing: taking a strategy that had passed walk-forward validation and running it live — on paper, with no real money — against the actual market, going forward, one real signal at a time. Not replayed history. Genuinely new candles, arriving in real time, that the strategy had never seen and that I hadn't tuned anything against.

I already had the webhook infrastructure from the very first version of this, so wiring the walk-forward survivors into paper trades cost almost nothing extra to build. What it bought me was completely different from what any backtest could: a real forward track record, accumulating trade by trade, that I could point to and say this is what actually happened, not this is what the numbers say would have happened.

A backtest tells you what a strategy would have done. Forward testing tells you what it does.

Once I had a handful of strategies accumulating forward results this way, I finally had something worth trusting: not one number from one test, but a strategy behaving the way it claimed to across three different kinds of scrutiny — a fitted window, an honest walk-forward, and now real time. Strategies that survived all three were the ones I'd actually consider for real capital. The ones that only survived the first were exactly the ones curve fitting produces, and now I had a way to catch them before they cost anything.

That forward-testing pipeline — strategies firing into paper trades automatically, building a track record I could use to filter and select — is also where this stopped being something I could run by hand. The next post is about the point where I gave up doing walk-forward optimisation manually in TradingView and wrote code to do it instead.

Follow bots built the way this post describes

Walk-forward tested, forward-tested on paper, then run with real risk limits. 15-day free Platinum trial, no card required.