Are You Interpreting Your Backtests Correctly? (RealTest Deep Dive)

What I want to do is run a backtest and show you how to A, run the backtest and then B, find and interpret the results. Real Test produces a lot of great information after you run a backtest. So I'm going to walk you through that, show you what it all means, how to interpret it, and then also how to adjust it if you need to. So let's jump in, go across to RealTest now, and we'll have a look at our script. Okay, so here we are in our illustrative system that we've been working up until now. So it's the NASDAQ 100 current and past, and it's a simple trend following system. So the code hasn't changed since the last video where you saw it. So if we're going to run a backtest, the first thing we need to do is hit the apply button to apply this script and then import the data if we haven't done that already. So let's do that. It should just take a couple of seconds because we're only importing the data for the NASDAQ 100 current and past, so that's pretty quick. Now, I've just made my Windows application font sizes a bit bigger so that you can see it. Usually it looks a little bit smaller than this, but hopefully this is large enough for you to read on the video. Okay, so as soon as you run the test, the results tab will open up. Inside the results tab, you'll get one row of data for each test that you run. So if you press back test again, you'll get another row. If you change some parameters or some rules and run a test, you'll get another row. So the performance measures and columns that are in this test window are completely customizable. I'll show you how to do that in a second. However, I want to encourage you to not get bogged down and sidetracked by customizing all sorts of fancy performance measures early in your trading journey. The standard out of the box performance measures that come with Real Test have more than enough insight for you to test and design systems that work. So don't get too distracted. Let's just focus on the main game of building and implementing trading systems that work, rather than overanalyzing and producing lots of complex performance metrics. Hey, if you're finding this video helpful and you want to go deeper with Real Test, I, I've put together a complete free program called RealTest Launchpad. The best part is it's completely free and you get lifetime access. So all you need to do is click the link in the description below and register to get access to Real Test launchpad Now. So if you want to learn Real Test fast and get up and running with confidence in the next couple of days, click the link in the description below and get access to RealTest Launchpad. You can just pause this video, click the link, get registered, and then come back. You'll get instant access to over 30 different tutorial lessons, showing you how to do all of the important functions. Real Test. So click the link below and I'll see you on the inside. Now, back to the video. Okay, so having said that, let's have a look at what we've got here. We've got the test number in this session that you've been using Real Test. What number test is it? So this is the first one that we've run since I opened up Real Test. You've got the name of the, of the test, and that name is defined in the backtest settings. The settings here. See, we've got test name and I've called it NASDAQ 100 illustrative system with an underscore. And that's exactly what appears here. Then we've got the from to dates where the first day of the backtest all the way through to the last day of the backtest. How many periods that covered? That's pretty obvious. So how many in this case? It's a daily system. So this is daily bars, total net profit. This column is comp or compounding, and this is telling you whether the system has compounding turned on or not. In this case, compounding is true rate of return. This is the compound annual growth rate. 16.47% max drawdown. Obviously the biggest dip in the account had you followed these rules, ma, which is basically the rate of return divided by the max drawdown. This is a measure of smoothness. The higher this number, the better. Typically, one is a pretty good benchmark to aim for, although it doesn't have to be one to be tradable. You can trade systems that are well below that, particularly when you're combining systems together into a portfolio. So again, don't get too hung up on any individual performance performance measure. We've got the number of trades in the backtest, the percentage of those trades that are winners, the expectancy of the trading systems. This is similar to average profit per trade. Then we've got the average size of a win, which in this case is 33%. The average size of a loss, which in this case is 10.98. This is pretty typical of a trend following system, right? Because in trend following you expect to have lots of small losses and the occasional large win. In this system we've got average win of 33%, which is quite big, and an average loss of just under 11%, which is pretty small. Then we've got the win length. This is how many bars on average the winners are held for. Loss length, same thing, number of bars that the losing trades are held for. And you can see here that the winning trades are held for much longer than the losing trades, which again is exactly what you'd expect for a trend following system. Profit factor, which is the total profits divided by total losses. Sharpe ratio, which is a measure of smoothness. Then we've got average exposure, which is the average exposure over the entire life of the backtest. Maximum exposure, maximum exposure of the system at any one point in time. And then we have the parameter values that were used during this test. And in this case we just pressed backtest, not optimize. So these are all of the default parameters from our system. So you can see here 200200 2010. If we go back to our NASDAQ system and look at the parameters, section 2002002010 are the different default parameter values for each parameter in our backtest. Okay, so that's the generic performance report that we get when we press test. But of course, real test gives us access to a huge amount more information and insights into the backtest performance of each strategy that we run. So the way we get all of that additional information is by right clicking on the row of the backtest that we're interested in and then selecting some of these other options. The first and most obvious one is the stats graphs. So let's have a look at that. So if we open up the stats graphs, you can see immediately there's a whole range of standard default graphs to show the performance of the system. The first one is the equity curve, starting at the 100,000 of our base equity, showing how that system evolved over time. You can see the flat spots when it was out of the market, and so on. So this chart is currently a log scale. You can change that to a linear scale if you'd prefer, although typically a log scale is a better way to look at that. The other thing you can do, which is quite neat, if you've got a trading journal, if you're keeping notes on your systems, you can copy this image to the clipboard, paste it into your journal. So that's the equity curve. The next One here is the time weighted equity. And this is essentially what $1 invested at the start of the backtest would look like over the life of the backtest. So it's very similar to the equity curve, but it's just adjusted down to a starting value of $1. There's the drawdown profile, which shows you how deep the drawdown has been at any one point in time in the backtest. The deeper this green shading goes, the deeper the drawdown. If the shading is right up here and there's no green at all, and that means that system's at a new equity high. So you can see that this system spends a lot of time in drawdown. Again, this is just an illustrative system to show you how real test works, so don't get too excited or worried about that. Now, obviously what we want to see in a drawdown profile is small and consistent drawdowns over time with lots of new equity highs. So lots of times where the drawdown profile comes back up to the zero mark. And we want to see that these drawdown dips are somewhat even over time and not getting bigger and bigger and bigger. But that's a whole other story about interpreting a trading system's performance and stability.

Okay, draw down bars. This chart shows how many days the system has been in drawdown, or essentially how many days between equity highs. So you can see there was a long stretch following 2000 when the tech bubble burst. It took several years for this system to get back to a new equity high, and then this dropped down to zero again. We can also see the daily returns or the distribution of daily returns. Weekly returns, monthly, quarterly and yearly. Maximum adverse excursion, maximum favorable excursion, respectively are how far against you or in your favor or all open positions have moved in dollar terms on that day. In the backtest setups is how many setups the system generated. Orders is how many orders were placed. Entries is how many entries were taken. Exits how many exits were taken on that day. Position shows how many positions were open. So you can see that this system has up to 10 positions open any one time. But there's plenty of times where there's not 10 full positions open, like here, here, and here, and so on. This shows how much money is invested and what your exposure is in this strategy over time. So they're the default charts that you get when you open up the stats graphs after running a backtest. Now, if you want to know how any of those charts are calculated to understand better the formulas of how they're built up all you have to do is come into one of these charts, right click and select Edit Graph, Graph formulas, or press F9. So we can do that, and it's going to open up the real test script that defines the graphs. And here you can see the name of the graph and the formula behind that graph. Now, again, I want to encourage you to not get too hung up on defining your own performance measures and so on. I have found the default ones more than sufficient to get me started and to understand the performance of a system. Of course, as you go on in your trading journey, you might want to add your own special charts and make adjustments to formulas and so on. But in the initial stages at least, I would encourage you to avoid it. Just focus on the default performance measurements and charts, and that'll be more than enough to get you going and allow you to make progress without getting bogged down in all of the detail. So I'll close this up. Okay, so these are the charts. Let's go back to the results. And if we right click here, we can also look at trade plots. This is really interesting. So trade plots will show you all of the closed trades over time, how big the winners and losers were. The red ones obviously losers, the green ones are winners. And if you right click on this, you can look at a couple of different types of charts. So we can look at the plots in terms of dollars, which is not very useful when you've got compounding, because over time, when you've got compounding, the trades get bigger and bigger and bigger because your equity is getting bigger and bigger and bigger. So I actually much prefer to look at them in terms of percentages. So we'll stick to percentages. Right click here. You can also look at profit distribution. This is pretty neat. It shows you how many trades were closed out at different magnitudes of wins. So you can see this system had a small number of very large winners, a large number of small winners, and a large number of small losses, and no large losses or very few large losses. So that's the scatter plot. We can look at the Mae MFE distribution, and that shows us how far the trades have moved in our favor or against us. We can look at cumulative profit. So this shows us when we made new equity highs. And it puts a line of best through that, line of best, fit through that on a log chart so that we can see whether or not the edge is eroding.

If we look at the Monte Carlo plot, what this shows us is a randomization of the equity curve. So it Takes the base backtest equity curve and then randomly reshuffles it many, many times and shows you the outcomes of that. Now, this is only showing you a small number of equity curves here. You can set that to be a large number. It's showing the best five and the worst five. At the moment, you can set that to run a lot more. Real test, by default runs 100 samples. So 100 different backtest reshuffles and it shows the best and worst five. In my experience, 100 is probably not quite enough to get a really solid understanding of how good or bad the system could be. So I would tend to run a thousand. But again, this gives you a better idea than just looking at one single equity curve. We can also look at the Monte Carlo drawdowns. This is very useful because if we run our Monte Carlo simulation many times, the drawdown profile will be a little bit different and we want to understand how bad that could get. So this shows you again the worst five drawdowns out of 100 samples. I would prefer to run again many, many samples, so maybe a thousand and see how bad the drawdown could get. But we can see that our black equity curve, which is the one that we ran the first default one, had the worst case drawdown here of just under 28%. The next one here is equal count bins. So what this does is it groups all trades in the backtest into an equal number of trades per bin and then calculates the average profit for that group of trades. So you can see here, over time, the average profit per bin has been decreasing, which is not what we want to see. Ideally, we want to see that horizontal and stable over time. The first 60 trades had an average profit per trade of just under 7.5%. The next 60 trades have an average profit of around 20% or a bit over. And third group of 60 trades somewhere between 12.5% and 15%. Okay, let's come back to the results tab and see what else we've got access to in there. So again, let's right click. We've looked at the stats graphs, we've looked at the trade plots. Let's look at the trade list. So the trade list is every trade that the system took. And you can see there's a bunch of useful information in here to analyze the system at a trade level. So we've got obviously the trade number, the strategy, the symbol or the ticker, whether it was a long or a short trade, the entry date, the time it was entered. So the open, the close Or a limit price or a stop price. The quantity we bought, the price we entered, the date we exited, the price we exited, the quantity we exited with.264.264. That matches because the number of shares we opened and closed with were the same. The exit price. We also have this reason column end of data Close less than moving average, low less than trailing stop. If you remember in our code, if we look down in the strategy section, we had exit rule and we said select is Exit 1 and we labeled it with close less than moving average with close is exit 2 and we labeled it with low less than trailing stop. And this is where you can see that in this reason column here, the number of bars the trade was open for. The percentage gain, the profit in dollars, the percent maximum favorable excursion for this trade. The maximum adverse excursion for this trade. The maximum favorable excursion is the largest distance the trade moves in our favor from the open to the highest price of the trade in a long trade or the lowest price in a short trade before closing the trade out. The maximum adverse excursion is the furthest distance trade moves against you from the open before closing out. So MFE and Mae Fraction. This is the fraction of our account that is invested in that trade, the dollar size and any dividends that were received during the backtest. So this trade list, for example, could allow us to look for outliers. We can sort on percent gain. We can see the biggest winner and we can say, okay, is that an outlier? Do we want to exclude that ticker or not? We can look at the biggest losers and try and learn something from that. Now the best way to learn something from a winning or losing trade is actually to look at how that trade played out bar by bar. We can do that super easily in the trades list just by right clicking on it and clicking show chart. This shows us the buy price and the sell price. So we can see, all right, this is where we got in, what happened in the trade and then where did we get out? So this is super useful for generating hypotheses to try and improve the system. The more you look at trades that the system generates, the better able you are to come up with a lot ideas. But also the more you look at the trades the system generates, the more aware you're going to be of what could happen when you're trading the system. So I would encourage you to look at many, many trades in your backtest and see how exactly did they play out over the life of the trade to get used to what could happen following this system. So let's have a look at another couple of trades in the trade log. We'll come back here and I'll just sort it on best to worst and let's look at some good trades here. Let's look at. But this one will right click hit show chart. You can also just click on it and hit Enter. So we'll click that. You can see we bought the trade down here and it had a nice smooth uptrend, hit two big tops, it came down, hit the trailing stop and we got out here. So that's a pretty typical trend trade. Let's look at another one, maybe one that has a different type of exit. So let's click on that, hit enter. That'll show us the chart. You can see we bought here held it all away, some long sideways bits, but it was clearly an uptrend and then it closed below the moving average here and then we got out.

So again you want to look at lots of different trades in your backtest just to get an understanding of how the system works to make sure it's doing exactly what you expect. Coming back to the results tab, the next thing you can see is the show summary report that just pulls up a HTML version of the backtest report. I typically don't use that. I much prefer to look at it a row of data like this, I find that much easier. So I'm just going to stick with that. What else have we got? One that's particularly useful is this Edit column formulas. If you ever want to know how any of these formulas are calculated, all you need to do is right click and select Edit Column Formulas or just press F9. And what that's going to do is show you the formulas used in each column of the data. Now again, I want to encourage you, don't get bogged down in this at the beginning of your trading journey. If you want to create your own performance measures and everything, that's fantastic. But I would suggest that that's an advanced level skill and at the beginning of your trading journey is probably not something that you want to do. I typically use mostly the default performance measures and have found them to be a pretty useful set of measures for analyzing trading systems. So having said all that, if you come in here and you want to know what these formulas, how they're calculated, you can look at periods and then you can decipher this formula and understand what that is. I'm not going to go through every one here, that'll take way too long. But typically what you'll do if you don't understand it, just click on it, press F1, it'll bring up an explanation, you can read it and start to understand and deconstruct how these formulas are actually made. So that can be a useful exercise at some point in your trading journey. Another couple of things that are useful in the Results tab. If you right click on them, you'll see you've got some options to save as CSV files for different things. The list, the stats data or the trade list. So you can export it to CSV, analyze it in Excel to your heart's content. Okay, so that's an overview of what you get when you run a backtest in real test. There's a lot of great statistics, there's some charts that are really useful and a lot of information to help you analyze and understand your trading system. If you want more help with any of that, obviously the in context help by pressing F1 is the first place you should go. The second thing second place is the user manual or the real test form forum to get your questions answered. So that's all for this video. I'll see you in the next one. Bye for now.