Backtesting a Complete Trading System in RealTest (NASDAQ 100 Trend Following Example)

What I'd like to do in this video is give you a walkthrough of a complete system that actually works so you can see how all of this fits together. So this system is a simple NASDAQ 100 trend following system. I want to be clear, I'm not recommending you trade this system. It's just a simple trend following system example. There's certainly way better systems out there. This is not trading ready, but it is backtesting ready. So it's going to show you how all of the elements of the script fit together and how the syntax works to create an entire system. And then you can take this system, build on it, or adjust it to make it your own to build something that's actually tradable. So let's jump into real test and have a look. So you'll see. I'll just give you a quick run through. We've got the notes section, import settings, order settings, we've got parameters, library, data, template strategy and charts. As I said in a previous video, you don't necessarily need to have every single section in every single script. It depends on what you need your scripts to do. So what I'll do is I'll run through this script, I'll tell you what I've got there, how it works, how it fits together, I'll explain all of the syntax and that way hopefully it'll really start to bring all of this to life. So let's jump in and have a look. So right at the top we've got notes. And remember, this is all green, these are just comments. And this is an example of how I like to document. I explain what the system is, the data source, time frame, market padding adjustments, the index filter, and so on. So I document all of this in all of my systems and I think that's good practice because then when you open the script, you'll know exactly what it's all about. So put whatever comments you want in there.

The first real section here is import and the data source I'm using is Nor Gate. For all of my Australia, US and Canada systems, I use Nor Gate data. For everything else, I use Meta stock data. Our include list is NASDAQ 100 current and past. This system only trades stocks that are in the NASDAQ 100 index at the time of the signal. So we need the NASDAQ 100 current and past watch list from Nor Gate to make this work. In addition to the stock data, we also need to bring in the NDX. So we've got include list NASDAQ 100 current and pass. We also have include list $MDX. What that's going to do is bring the price history for the NASDAQ 100 index into the database as well as the price history for all of the NASDAQ 100 stocks. So we can refer to the index in our backtest because we can use it later on down, further down the code as an index filter. Hey, if you're finding this video helpful and you want to go deeper with realtest, I've put together a complete free program called realtest Launchpad. The best part is it's completely free and you get lifetime access. So all you need to do is click the link in the description below and register to get access to Real Test Launchpad now. So if you want to learn Real Test fast and get up and running with confidence in the next couple of days, click the link in the description below and get access to Real Test Launch Launchpad. You just pause this video, click the link, get registered, and then come back. You'll get instant access to over 30 different tutorial lessons showing you how to do all of the important functions in Real Test. So click the link below and I'll see you on the inside. Now back to the video. Okay, start date. I just want the earliest date of data that we've got available. End date. I want the latest date. The reason we have latest date is because I want it to bring in everything so that we can generate signals and trade this system in real life. Obviously this is an example system. You're not going to trade this actual one, but you don't want to have an end date as a particular date in your script. If it's a system you're going to use, if you're going to actually use it, you need to make sure the end date is latest. I'm going to save this data file. I'm going to call it NASDAQ 100 current and pass RTD. That way I can refer to it in other systems and I don't have to reload it every single time. Padding is all market days and adjustments is capital. Ok, let's talk about the settings. First of all, I'm going to specify the data file which I've done up here. Don't really necessarily need to do this, but I like to have it in both places. Account size. I'm just going to start with $100,000 for this backtest. Bar size is daily. This is a system that trades every single day. The start date for the for the analysis is 1st of January 95, end date is latest. Use available bars this is set to false. What news available bars means is let's say for example, you've got a 200 day moving average and right at the beginning of history, if a stock is new on the first day that stock trades, it's only got one bar of history. So you shouldn't be able to calculate a 200 bar moving average. If it's only got 199 days of history, you still shouldn't be able to calculate a 200 day moving average. However, RealTest has a function that will allow indicators to calculate using the available bars to so if there's only 100 bars available and you've got a 200 bar moving average, it'll average the hundred of them and then as the number of bars expands, it'll use more and more until it gets to 200 and then it will be a proper 200 day moving average. In this system, I don't want that to happen. In most cases you won't want that to happen because there can be some unexpected consequences. So I've got use available bars set to false. I'm keeping trades for the strategy. So this means that every trade that's generated using the strategy down here will be kept in the backtest so it doesn't throw any trades away. And when I run a backtest it's going to have this name. So that way if I generate a whole bunch of backtests in a session, I'll know which system they're all for. I don't have scan settings or test settings in this script. I'm just using the default options. I do have some order settings though. There's a couple different ways of doing orders. And later in the course I'm going to go through a whole section on order generation. But in this script, what I've got is the simplest sort of order generation I would say is most useful for traders using interactive brokers. And it's an order generation script that generates a basket order file so you can upload that basket into the basket trader and place all your trades in one go. That's the simplest way. Obviously. You can also use Order Clerk, which I'll explain later on in this course. Okay, so how do the order settings work? We specify the data file, which is the same as up above. When we're generating orders, we don't need to go all the way back to the beginning of time. We just need to go back a little way so that realtors can calculate what the open trades are and what the new trades are. So I've just set that to the 1st of January 2023. If you want to go further back, it doesn't really matter, it just takes a little longer. End date is latest because we want the latest orders account size. Now you can specify the account size for order generation. Here I've put 20,000. You can put whatever you want. If you'd left that out, it would just use this account size from the settings by default. But because this is Order settings, when we press the order button, it's going to override it and generate the orders with an account size of 20,000. So this number can be whatever you need it to be. And this would typically be the amount of capital that you've allocated to that strategy on that day. So you have to recalculate that each day if you're doing this manually so that you get the right position size for your account. The orders mode I'm using is template mode. Let's have a quick look at what the other modes are here. If I just press F2 you can see order modes. You can do autoclerk, Autoclerk, Stats, Order Clerk, Sync Rebalance Template or Text Template is what I'm using for this one. And I'll go through those other order types later in the program. The template we're using is the IB Basket Trader template that comes with Real Test. You should see that in your Scripts Examples folder. So you can just pick this up. It's a sample template that has all of the columns that you need for the Basket Trader optimized settings. I don't have any specific optimized settings for this system. It's just using all of the built in defaults parameters. I've got several different parameters for this system. Notice I've got this is and then ltma bars. The way I code systems is I want every single system to have unique parameter names and unique rules. That means in order to achieve that, the easiest way to do it is to put a prefix on every parameter and every rule so that if I have the same ltma bars, for example, in one system and another system, they've got different prefixes so they're all unique. And that's really useful for when we combine multiple systems together in a portfolio of systems. Back test so that every system has unique parameters and rule names so nothing gets confused. So that's why I've got these kind of funny looking prefixes beforehand. Is just stands for illustrative system. But every other system gets A unique prefix like this after the prefix, then I've got the name, which I understand what this means. Lt, long term Massachusetts. Moving average bars. Number of bars in the moving average. You can have whatever nomenclature you want here. This is just the nomenclature I'm using for this system. And these are all set up. Well, the first three anyway are set up for optimization. So the long term moving average bars I've set up to optimize from 100 to 300 in steps of 100. And the default value is 200. So if I press the optimize button, it would go 1, 100, 200, 300. It will run all of those three values. The second parameter value is the highest high value bars, the number of bars in the highest high value. Look back and you'll see this further down in the formula. And I've set that up to optimize from 100 to 400 in steps of 100. And the default is also 200. I've got a trailing stop parameter. This is just the parameter that defines the width of the trailing stop in percentage terms. So that's set up to optimize from 10 to 50 in steps of 10. And the default is a 20% trailing stop and then percent equity per trade. I've just hard plugged that as 10 because this system has 10 equally sized positions. Okay, so that's the parameter section. Let's look at the library section. So remember, the library section are the formulas that are calculated on every single run on every single ticker. So these are recalculated as needed in every single ticker that you backtest. The library section formulas are the slowest to run. Okay, so I've got two formulas in the library section. The first one is exit rule one, second one is exit rule two. And again, they've got the same prefix to make it unique, and then exit 1 and exit 2. So let's look at the first exit rule. It's called Exit 1 and it's got the same prefix. We're using the cross function and the cross function. The way it works is the cross function is true when the first thing crosses above the second second thing. Another way of saying that is when the second thing crosses below the first thing. So looking at the cross function, we've got a moving average and we've got the close. What sort of moving average is it? That's a simple moving average. If you weren't sure, just highlight it and press F1 and it tells you it's a simple moving average, which is great. What are we averaging? We're averaging the close and over. How many bars are we averaging it? Double click on this, you'll see it highlighted up here and we can come across and see that the default is 200. So if I just press the backtest, this is going to use a 200 day moving average. If I press optimize, it's going to vary the length of this moving average from 100 to 300 in steps of 100. So this exit rule is true when the closing price crosses below the 200 day moving average. Now, I might hear you saying, hang on a minute, Adrian, surely you could pre calculate that formula because the moving average for every stock is known in advance and when the price crosses below the moving average, it's known in advance. It doesn't have to be calculated in context. So therefore, does that really need to be in the library section? Couldn't we put it down in the data section? And yes, you're absolutely right, if that's what you're thinking, because this could go in the data section. I've done this intentionally because I want to show you the difference in time a backtest takes when you've got a formula in the library section versus when you've got a formula in the data section. So I'm going to show you that at the end of this video. So let's jump back in. Okay. Exit rule 2. This is exit 2 again with this unique prefix. This unique prefix is just my coding style. You don't have to do that. But I think it's good practice to have unique names for every parameter and every rule across all of your systems. Okay, let's have a look at the second exit rule. Exit rule 2. So this is a trailing stop exit. And this trailing stop is triggered by the low of the day. And it's triggered when the low of the day is less than the trailing stop value. And the number of bars that the trade has been held for is more than one. Now if you didn't know what bars held is, you can just highlight it, press F1 and it'll explain here the number of bars since the position was entered, not including the entry date. So it's basically how long the trade has been open for. And this is the calculation for the trailing stop value. Let's break that down one piece at a time. First of all, this first bit, 1 minus our trailing stop percent. Our trailing stop percent up here by default is 20. We've got 20 divided by 100. So it's 1 minus 20 divided by 100 or 1 minus 0.2 or 0.8. So this means 0.8 times the highest high value of the closing price since the trade was entered. So what we're doing is every single day in the backtest, we're looking at the closing price. We're saying, what's the highest close since the trade was entered? And then each day, as that trade moves in our favor, the highest close gets higher and higher, and then maybe it dips down a little bit, and then the highest close gets higher and higher. At every point in the backtest, we're looking back the number of days that trade has been open, finding the highest close, multiplying it by 0.8. And that's our trailing stop value. What that does is as the trade moves up and moves up and moves up, the trailing stop follows behind it, a distance of 20% below the highest close since entry. There's a couple different ways you could do this. We're triggering it by the low of the day. Touching this trailing stop level, you could trigger it by the close of the day or the open of the day, or the high of the day. We're also calculating the highest high value based on the close. You can calculate the highest high value based on the high of the day since entry. There's many different ways of doing this. This is just the formula that I happen to have chosen for this system. Okay, so this is the library section. Now we'll go to the data section.

In the data section, I've got a rule here called the highest high value close filter. And this rule is calculating is the close equal to the highest high value of the close over the last.

Looking up here, 200 bars. So this is true when today's close is the highest close out of the last 200 bars. So this is a breakout entry rule. If today's close is the highest, then this rule will be true. Our next rule is the index filter. Now, in the index filter, we're going to call an external ticker. An external ticker means we're referring to some data that's not the ticker that we're actually backtesting on in that instance. So for our index filter, we want to refer to the NASDAQ 100 index, the dollar NDX. And so the way we do that is we use the extern function. And xturn is a really powerful function. Again, just click F1 here. It'll explain all of it. This evaluates a calculation for a different stock or contract or bid or bar size. So what that means is instead of calculating this formula. If we just typed this formula here, if I just put that here and then commented that out, if I just put this formula here, that would calculate the close is greater than the moving average of the close using a 200 bar moving average for the tickerware backtesting. But I don't want to use the tickerware backtesting. I want to use the index because I want to know if the index is above the 200 day moving average because I only want to trade in a bull market. So let's put this back. So instead of using this formula on its own, we're going to wrap it in extern and we're going to tell it refer to the NDX. Now, $ndx is the ticker symbol and we need to also tell real test that it's an external ticker. So we need another dollar sign. Just take my word for it, it's dollar and then the name of the ticker, the name of the ticker is $NDX. So you just end up with two dollar signs. So XTurn refer to this ticker and calculate this rule. So if the NDX is above its 200 day moving average, this rule will be true. Okay, let's keep going. Okay, so let's go down to the next rule. The next rule is called isentry and this is basically combining the different entry rules together. What I want to do is create a rule that combines each of the entry rules we've talked about already. The first entry rule is the highest high value close filter. So if that is true and the index filter is true and in NDX, what is in NDX? Let's again press F1. In NDX is a rule that says the ticker that we're back testing must be in the NDX index. If you wanted to test on stocks that were in The S&P 500 index, it would be in SPX, but we're using the NASDAQ so it's in NDX. So this is just a neat way of saying this stock must be an index constituent on the day of the entry signal. Okay, so you can see we've got three entry conditions in this rule. We've got the 200 day highest close, we've got the index regime filter which is that the index must be above its 200 day moving average. And we've got in IndX, which is the index constituency. And you can perfectly well write it like this, this and this and this. But I actually don't like to write it that way. I'd rather do it slightly differently. So let me show you how I would prefer to do it.

I prefer to do it like this because if they're on separate lines, this still is interpreted as one rule. Except what you can do is you can comment out that one and see whether it has an impact, or you can comment out that one and see if it has any impact and you can comment out that one and see if it has any impact and you can do that individually. So it's much easier to put the rules on separate lines like this so that you can comment rules in and out and in order to have them all on separate lines, because you obviously can't comment out this, you need to name the rule. If we comment out that, the thing won't work. So what I do is we have that name and then I put a one here because one is always true. So this is true. If this is true and this is true and this is true and this is true. So this is our compound rule that combines all these things together to tell us when isentry is true. Alright, you'll see here I've got isexit1 the same rule up here, but I've got it commented out. Because what we're going to do again later is I'm going to test it in the library section versus in the data section and we're going to see what the speed difference is. Okay, then we've got template. So template is how we keep our code simple if we've got multiple strategies in the one system. So in template you've got to give it a name. So I've given it a name called NASDAQ 100 Illustrative System Base. Again, you can make this whatever you want. I'd suggest just make it human readable so you remember what it is. Calendar SIM is the symbol used as the market date line list for this strategy. And we're going to use NDX because we're trading stocks in the NASDAQ 100 index. So we want to align everything to the NASDAQ index. Our capital allocation, we're just going to use combined equity. Our max positions is 100 divided by the percentage of equity per trade. Now you can define this either way, right? You can specify the maximum number of positions and then calculate the percentage of equity per trade. Or you can specify the percentage of equity per trade and then calculate the maximum of positions. Mathematically it's the same thing. I wouldn't get too hung up on it. Okay, then we define commission and here is the commission formula to mimic the Interactive Brokers Commission calculation. So as an Australian trading US stocks on Interactive Brokers, for me the minimum commission is $1 per trade. The commission is 0.5 cents per share. So 0.05 times shares. So we need to calculate the max of those two. However, the maximum commission that we're going to pay is 1% of the position size. That's also specified in the IB Commission structure. So what we're going to do is calculate the minimum of 1% of our position value or this calculation here. And that's going to give us a commission in our backtest which matches the Interactive Brokers commission. We're going to calculate slippage separately and our slippage here is specified per share or per contract. And so we're going to specify the fill price times some percentage. And I've just got a quarter of a percent here for slippage. These are pretty liquid stocks, so our slippage is going to be pretty low. What quantity are we going to buy in each trade? Well, I've specified percent equity per trade. If I double click on that, you can scroll up here, you can see it's highlighted and the value is 10. So it's 10% of equal equity. Now when we specify our quantity, we have to say what the quantity type is. So that Quantity type is percent. Remember we can specify percent shares or value. This is 10%. So that's our quantity type. So that's our template. And we could reuse that template multiple times in different strategies into one script. Only got one strategy in this script and the strategy is called NASDAQ 100 illustrative system. This system, this strategy is going to use the NASDAQ 100 Illustrative System base which we defined up here in the template. So if we're using a template you say using and then the name of the template. What side? This is a long only strategy. If it was long short, it would be both. If it was short only, it would say short again. You can do that by pressing F2 and you can see those options here.

Okay, the next thing we need to define is the setup score. How are we going to rank the trades when we've got more signals and we've got capital available to take. So what I'm going to do is define this score function here, isscore. And you'll see isscore is defined up here. And I've just defined is score as the close divided by the moving average of the ticker, which moving average double Click on here, scroll up, you'll see it's a 200 bar moving average. So what we're doing is ranking the trades based on which trade is the furthest above its 200 day moving average. The idea being there, the further above the 200 day moving average it is, the stronger the trend is. Therefore the more likely we want to own that stock. So that's the position score formula. So coming back here, setup score is that formula which is defined up here. Our entry setup, which is the trigger is the rule is entry. We defined that up here and went through it. It's this rule here.

The entry time we're going to enter next bar on open. Okay, the next rule is the exit rule. And this system has two exit rules. The cross below the long term moving average and the cross below the trailing stop. When we're in a backtest, it's valuable to know which of those two rules trigger the exit for each trade. So what we do in the code here is we use this select function. What this says is if exit rule 1 is true, then call the exit this value. If exit rule 2 is true, then call the exit that value. That way when we run the backtest, all the exits are going to be labeled and so we know which rule triggered the exit. We can say which rule is doing the work most of the time. We can look at which rules are effective, which rules are not effective, and so on. You don't have to do that. You could just say exit rule one or exit rule two. But I like this because it gives us a lot more insight into what the system's actually doing. And then exit time next bar on open because we're going to get the signal and then place the trade to exit next bar on open. And then we've got the charts charting the moving average. So I'm plotting a moving average of the closing price with 200 bars, the highest high value. It's the HHV formula of the close over last. Using that formula, which is also 200 bars defined up here.

Then I'm also going to plot the index. I'm going to use the external function, call the ndx and I want to plot the closing price. And I'm going to plot the index filter and I'm going to use the external function. I'm going to call the ndx and I'm going to plot the moving average of the ndx. You'll notice that for the index I'm plotting that in the upper pane. So I've specified In curly brackets, the little hat indicating the upper pane. And I'm also going to plot the moving average in the upper pane. Now if we look at this, let's click the Apple chart. You'll see we've got the price chart of Apple, we've got the moving average of Apple, we've got the highest high breakout of Apple and we've got the NDX as a line chart and we've got the 200 day moving average as a line chart. So this way we can see what's going on with the main indicators in the system. Ok, so we've got a complete system ready to go. What we're going to do now is run our first. So what we're going to do first is hit apply and then if we haven't already imported, we're going to import. So let's do that. It's the NASDAQ 100, so it happens pretty quickly. You can see it's working down the bottom here. Importing the data took a total of five seconds. Now we can hit test. So if we hit test now we can see some performance stats. So this row shows a whole bunch of performance stats. We can click into that and see more details, which I don't want to do just yet, but I'll come back to this idea of the library section versus the data section. So let's have a look at this. The backtest you can see, you might be able to see right down here in really small letters. The backtest says it took 1.43 seconds, not very long. But now what I want to do is I want to comment out the exit rule from the library section and I want to add it to the data section. So it's down here now. So I'll save the script, I'll apply it. Now I'm going to run the test and let's watch what happens now. It was 0.625 seconds. So it was actually, I know both were fast, but it was dramatically faster with that rule in the data section compared to in the library section because it doesn't have to recalculate for every single ticker for every single bar of the backtest. If we pre calculate the formulas we the backtest runs quicker. This is obviously a super simple system so we don't need to worry too much about the speed. But the more complex your systems get, or if you're running multiple systems at a time, running a big backtest on a much bigger data set, this becomes a really significant deal. And so structuring the scripts correctly, putting the right things in the data section and only what's absolutely required in the library section up in there will make your backtest run a whole lot quicker. So now we've got a complete set system example that we can work with. What I'll do in subsequent videos is we're going to walk through the back testing process and functionality within real test. We'll run some optimizations and get you familiar with all of that. So in this video what we've managed to achieve is a complete walkthrough from start to finish of a system code that actually works. It's not an amazing tradable system, but it's a good start and it's certainly profitable. In subsequent videos what we'll do is deep dive into the backtesting process and functionality, run some optimizations and get you familiar with all of that. But that's all for this video. I'll see you in the next one. Bye for now.