Optimization in Backtesting

What's the first thing you want to do when you crack open your backtesting software and you put in your trading system? I bet you it's optimization because everyone wants to optimize their trading system to maximize the profit that you can make from your trading rules. But is optimization the be all and end all? Is that really what you should be focusing on? Well, it's a super important tool, but it's not everything and it's probably not what you think. My name is Adrian Read. This is enlightened stock trading. And in this video series I'm focusing on backtesting. And today or in this session, what I'm talking about is optimization in backtesting. And I want to talk about three styles of optimization. Individual parameter optimization, brute force and walk forward. Three really important styles of optimization that you might use in different circumstances. But first of all, what is the purpose of optimization and why do we want to do it? But before we go any further, make sure you hit the subscribe button and give this video a thumbs up so that you get notified of all of my future videos. Because I've got new trading content coming out all the time and I don't want you to miss a thing. Well, I've written a whole blog post on this, so you can have a read of that at the link below. But let me just summarize it real quick for you. Optimization is about maximizing your probability of future profit. It's not about maximizing your future profit. Because when we optimize and backtest, we're working on past data and if we optimize to find the most profitable parameter combination, what we end up doing is curve fitting and cherry picking the very best combination. And in the past that's going to look great. We're going to get an amazing equity curve that's nice and smooth. But in future real time trading we that system is going to fall apart because you've curve fit to the very best parameter values. So instead what you want to do, instead of focusing on the maximum profit that you can get from your past backtest, you want to focus on maximizing your chances of future profitability. So how do we do that? Well, we run optimizations to find areas of stability within the parameter space. If your system is.

Profitable with a 50 day moving average, but it loses money with a 45 day moving average, and it loses money with a 55 day moving average, it's not a very stable system. But if you run an optimization, you find that actually with a 70 day moving average, the System is profitable, but not quite as profitable as maybe it was at 50, but at 60 it's profitable and at 80 it's profitable and at 70, 55 it's profitable and at 85 it's profitable. It's a nice stable area of good performance. That's a much more powerful discovery because that broad area of strong performance is likely to give you profitable future trading. Because if the market shifts a little bit, behavior shifts in the future, which it will, then that parameter value is more likely to be profitable. Whereas compare that to the profit peak, the that you found when you just did an optimization and chose the very best combination. That profit peak was unstable and therefore in future trading probably not going to actually be profitable. So in back testing in optimization rather, the first thing you want to do is get your focus, your mindset right. We're looking at maximizing probability of future profit, not maximizing future profit, dollars, probability, not dollars. That mistake probably costs more traders losses than anything else in systematic trading. Let's talk about the three styles of optimization right now. Individual parameter, brute force and walk forward. Individual parameter optimization is first. So what is individual parameter optimization? That's when you code your system up and then you take each parameter one at a time and you vary it from some low number to some high number and you choose the value for that parameter that gives you the performance that you're most happy with. As I said earlier, that parameter value should be selected based on a broad area of stable performance rather than a narrow spike of very high performance. We want stability and profit, not just maximum profit. Individual parameter optimization is you take them one at a time, you vary it, you choose a value and you input that into the system and then you move on to the next parameter, you vary it, you choose a value and you put that into the system. And so on and so on down through all of the parameters in the system. The great thing about individual parameter optimization is first of all, it's easy on the computer, doesn't take a whole lot of computing power, so it's pretty quick. Second thing is, it's easy to visualize. If you're a visual person, you like to see the data. When you're just optimizing one parameter at a time, you get a two dimensional chart, XYZ sort of axis, and you can see the profitability as you vary the value of that parameter. And you can, using that visual illustration, choose the value that you're most interested in. If you try and optimize three, four or five variables or parameters simultaneously, you can no longer visualize that performance because there's too many dimensions. So it becomes much harder. So individual parameter optimization is quick, it's easy, and it's visual. That's a really great set of advantages. If you're a beginner trader and you're just starting to optimize trading systems, optimizing parameters individually is by far the best way to do it. The next way is brute force optimization. And brute force optimization is something that a lot of traders come into the market and just try and do straight away. They get their trading rules, let's say it's a triple moving average system. They, they have three parameters, one for each moving average. They put a broad range around each of them, they optimize everything simultaneously. They test 365,000 combinations, they choose the best one and they start trading with it. And then they wonder why that they lose money in real time trading. The reason why that loses money in real time trading is that you've absolutely curve fit your trading system to past data. You've tested every single possible combination, you've chosen the very best combination out of all 365,000 of them, and you've expected that to perform equally well in the future, which it absolutely won't. Brute force optimization is a real trap. Trading software is also set up to make you inclined to do brute force optimization. Look how powerful and fast our trading software is. You can just put all your parameters in and optimize them and get the best one. And not only that, if there's too many combinations, we can use fancy optimization functions like particle swarm and all these fancy mathematical sounding terms, which are just fancy complex ways of saying we just optimize a little bit less of the data. Instead of looking at the entire parameter space of all combinations, we start with a few different combinations and then we gradually zero in to find the best one. But they all have the same problem. They're still trying to find the best combination out of all parameter combinations in the optimization. And that's the problem, because that leads to curve fitting. If you want to do brute force optimization, you need to be really clever about it. And in the trader success system, I teach this fantastic method of optimization called the robust brute force optimization method. It's something that I developed to improve my own trading and optimizations and I teach it to all of my students because what it does is it allows you to find the areas of stability within the multidimensional space of the entire parameter set. So all you have to do is set up your entire optimization to run Brute force, you leave your computer to run for a couple of hours or overnight, you get thousands and thousands of rows of data. And the method I use to manipulate that data to find the areas of stability gives you the right parameter values to use that are more likely to hold up in real time trading. And the cool thing is I've evaluated the effectiveness of this strategy using walk forward techniques. So I've run an optimization using standard individual parameter optimization and then ran it forward into the future for several years and noted the performance and seen how much the performance varies. And I've compared that with my robust brute force optimization method, which is a proprietary process by the way. Only members of the traded success system get taught that. I've run that robust brute force optimization method and then played it forward several years into the future. And I've done this many, many times. And the robust brute force optimization method is by far the most stable and reliable way of optimizing a trading system that I've found in my 25 years of traditional trading. So brute force optimization is very risky if you just do it the blunt way that the trading software is set up for. But if you use robust brute force optimization like I teach in the trader Success system, it is the most powerful tool you can ever add to your systematic trading toolbox. Now the third method of optimization is walk forward optimization. And walk forward optimization was developed and promoted by Robert Pardo, who is a pioneer in the systematic trading space. Very, very clever guy. And what warp forward optimization does is takes a slice of the data, optimizes the system, all of the parameters, selects the best parameter value based on that slice of the data and then walks it forward with that chosen parameter and tests this out of sample set. Then it does the next time period and re optimizes and then chooses the best value and walks that forward. Then it does the next.

Period of time, reoptimizes and walks that forward. And the results that you get back from your walk forward optimization is the pieced together out of sample backtests from all of those steps. Now, walk forward optimization is held up as the gold standard in optimization, which I agree it is under certain circumstances. Here's where it works really really well. Short term systems, high frequency systems and systems that have a relatively standardized distribution like narrow distribution of trade outcomes. The reason it works well there is because in each optimization window, whether it's three months, six months, 12 months, 24 months, you, you get a lot of fully developed trades that the optimization can make its decision on and then you walk it forward. But where it doesn't work so well is longer term systems systems like long term trend following. Long term trend following is characterized by a large number of small losing trades which happen very quickly, short term, and then a small number of large winning trades that can go on for months and months and months. If you try and do walk forward optimization for a long term trend following system, what happens is all of the losing trades are fully developed in the optimization window, but the winning trades are not fully developed in the optimization window. The winning trades are truncated because the optimization finishes and it cuts them off. So what that means is you're making suboptimal optimization decisions because too many of the trades, and particularly the winning trades are truncated. So I found walk forward to work really well for short term high frequency, high volume systems, high turnover systems. But for long term slower moving systems like trend following does not work so well at all. So choose the right optimization method for your systems. For long term trend following and medium term stock trading systems, I use my robust brute force optimization method. You might want to use the individual parameter optimization or very careful brute force optimization, making sure that you look for stability. For short term systems mean reversion that trade very frequently. Walk forward optimization is typically the best way forward. So optimization and backtesting, you have three choices. It's not a clear winner in all circumstances. You've got to choose the right tool for the job. So that's all about optimization. In backtesting we've covered individual parameter optimization, brute force optimization and walk forward optimization. And I give you a little sneak peek into something that my members of the trader success system get, which is my robust brute force optimization method. In the next video down below, if you're on my blog, or in the next video in this series if you're on YouTube, I'm going to talk about backtesting software and give you an overview of the landscape for backtesting software. So stick around for that. My name is Adrian Reid. This is Enlightened Stock Trading. I'll see you in the the next video. Bye for now.