Adrian Reid explains when a backtested trading system is ready to trade live. He warns against over-optimizing, which causes overfitting, and argues the real test is whether the system adds value to your overall portfolio through better returns or lower risk, not whether it performs perfectly on its own.
How do you know after you've back tested a system, is it good enough to go live? This is probably one of the harder questions in trading to answer. And the reason it's hard is because we always have this temptation to just make it a little better, just work on a little more, just optimize it, just get it a little bit better. I just want the equity curve to be a bit smaller, I just want to reduce that draw. I just want a slightly higher return. Right.
This is a never ending challenge of perfectionism. And the trouble is perfectionism in back testing where you keep going and going, trying to improve it, trying to improve it, trying to improve it, actually ends up doing the opposite. Because the more we optimize and the more we try and improve our strategies.
The more overfit we end up being to pass data, the worse they perform in real life. So I have a process for backtesting and again, I'm not going to go into the whole process here. It's proprietary, it's what I teach all my students in the mentoring program. But it's a once through process. You get the system, you do follow the steps. One of those steps is optimization. We optimize once, go through each parameter, optimize them once. We brainstorm for improvement ideas once, we test those improvement ideas once. And when you get to the end of the process, you validate that the system passes or fails out of sample and you make a go or no go decision. The process is linear and if you get through the process at the end, you make a decision. Did it pass in sample, out of sample? What that means is did it hold up on unseen data? Now passing doesn't mean it's great. It doesn't mean it's going to make 30% return per year with 5% drawdown. Doesn't mean any of that. It just means you develop the system on your in sample data. And when you test validated on out of sample data, the performance was similar enough that you can pass it. That's the first stage gate. Okay, this system held up great.
Should I trade it? It's a completely different question. It's unrelated to back testing and this is why it trips people up so much. The decision to trade a system is nothing to do with the backtesting process. It's everything to do with capital allocation.
So if you've got a strategy already and you're trading it and you develop a new strategy and it passes around a sample test and you get to the go, no go decision, your job is not to Say, is this system good enough to trade? That's wrong.
Instead what you want to do is say, if I add this system to my portfolio, does it make my portfolio better? It's a totally different question.
Can you see how it's different?
Because you might have a really great smooth equity curve, but it's so highly correlated to what you're already doing that it doesn't add any value to your portfolio. But you might have a really ugly equity curve that doesn't make much money, but it makes money. Exactly. When your other system loses money, and when you combine it into the portfolio, it smooths things out.
So your portfolio level return goes up. Then you can say, ah, this system is good enough to trade and allocate money to.
Now this is the nuance. The people who think at the system level get trapped in this never ending cycle of optimize, improve and optimize and improve and can I make it better? And how much return can I get out of this system? But we don't care about each system, we care about the portfolio of systems. Because we're never going to trade just one system, we need several working together in a portfolio.
So once you develop a system that passes, if it adds value at the portfolio level and improves your returns, then it's good enough to trade.
Does that make sense? It's a totally different mindset and it really frees you up because you're not agonizing over generating a system that has certain stats. You've just got to get a system that passes the other sample tests. Now, all of those tests in the process, if it passes all of that, then you do capital allocation. So freeing means you can get through the process much more quickly, build your portfolio more quickly. And a portfolio of good systems is way better than one great system because you get good diversification.
