Amibroker Tutorial: How to Read Amibroker Backtest Report and Optimization Report Correctly

Are you an amibroker user who wants to improve the way you do backtests and optimizations? In this video taken from a one on one mentoring session from one of my students with the Trader Success System, I take you through exactly how to use the backtest report, the trade log that you get during the backtest, and also the optimization report to make better decisions about your trading systems in amibroker.

So stick around.

So I've run a backtest. The system is called Formula 112. That's how random it is.

And what you can see on the screen here is the trade database that the backtest generates. So first we'll do a normal backtest, then we'll do an optimization and we compare the columns and so on. Okay.

So if we look at the, the trade database that we get when we run a standard backtest with no optimization basically tells you everything amibroker needs to know about that trade in order to assemble the equity curve. So you get the entry date, the exit date, the entry price, the exit price and more or less Everything else is calculated based on those things. So we know for each row it'll tell us the price change from start to finish. The percent profit, the number of shares you bought according to your position sizing formula which is defined in the system, the value of that position as in how many dollars that was worth.

This cumulative profit. I don't actually understand that. I think it's the system's cumulative profit on the day that signal that stock was triggered. But I don't look at that one. Then there's number of bars which is how long the trade was open for. So you can see here in this system that ranges from about six bars up to over 200 bars because this is a trend following system. So the losing trades get stopped out pretty quickly, but the winning trades can go on for long. So see this trade here was open for 274 bar and it went up 61% near enough.

The profit per bar is really quite useful when you're optimizing a system. One of the things that you want to make sure of is that your capital is effectively utilized. So you can increase the average profit per trade by using a longer term exit. But let's say, you know, for example. What's an example? Let's say you're using a trend following system and the price Crossing below the 200 day moving average is your long term exit. If you made it price crossing below the 300 day moving average, you'd probably get a higher average profit Per trade, which sounds like a good thing. But the profit per bar, the profit per day you're in, the trade may actually go down because it's taking so much longer to get your exit. And so you've got to be a bit mindful. Not that we're in a rush, but you've got to be a bit mindful of how quickly you the system makes the money. Because.

If two systems have the same average profit per trade, but one has shorter trades, that second system will compound more quickly, and so your account will grow more quickly. So that's quite useful.

Maximum adverse excursion and maximum favorable excursion are the last columns I've got here. There's some other columns you can probably add, but these are the only ones I see. So maximum adverse excursion is the worst distance in percentage terms. The stock moves against you in the life of the trade.

So if you enter at $10 and the stock goes down to $5 before closing out at 15.

The drop from 10 to 5 is a 50% adverse excursion. That's the maximum adverse excursion.

Does that make sense?

And then favorable excursion mfv, Maximum favorable excursion is how far it goes in your favor before it ultimately closes out. So if you buy a stock at $10 and it goes to 20 before closing out at 15.

The favorable excursion is from 10 to 20, which is 100% maximum favorable excursion. And the term maximum just means in the life of the trade, what was the very best or very worst that it got to.

And I don't use adverse excursion all that much. It's useful for setting your stop losses a little bit. But favorable excursion is really interesting because if you have a, let's say you got an average profit per trade of like 5%, the system could be quite profitable. But if heaps of trades have a favorable excursion of 20, 30, 50, 100% before closing out on average at 5, you may be missing out on a lot of profit because your exit is no good.

Yeah. So if your favorite excursion is super high or much higher than your average profit per trade, sometimes it's worth looking at a bunch of example trades and trying to find a better exit that captures more of that favorable move because you're giving up too much profit from between the high point and your exit point. So this is for. This is the trade database. The next thing you see when you run a normal backtest is the trade report. Now, can you see my trade report here? Okay, so this backtest report and.

The results you get when you do an optimization, the columns you get in optimization are Very similar, almost identical. Okay, so if you understand one, you can you understand the other. So in the backtest report, this is basically giving you all of the statistics of the system.

So that you can evaluate, you know, whether or not it's good enough or whether you've improved it or made it worse. Now, I look at a few things.

The most important ones we can go through. Now.

First thing I generally look at is the compound annual return.

For obvious reasons, because we want to make money. I'm also interested in the exposure.

Exposure is the percentage of the time that your money is invested. So let's say.

You'Ve got $100,000 in your account, and on average, over the life of the back test, it was only 50% invested.

So half the time you're fully in the market, half the time you're in cash, your exposure would be 50%. If you are 100% invested all of the time, your exposure would be 100%.

So why is that interesting?

Well, if you've got two systems, they both make 20% return per year, but one has an exposure of 50%, and one has an average exposure of 80%. Which one has the higher risk? 80%. Right. Because your cash is exposed to the market moves more of the time.

So for the same level of return, you want a lower exposure.

The other thing that exposure is useful for is to see how well you can use that system to layer on top of your other systems. Like, I have a few systems where the exposure is very low, like 5 or 6%, because they don't trade very often. And if you could make 10% return per year on average, with an average of 5% exposure, that's pretty amazing because you're only putting a teeny bit of money into the market every now and then, 5% of the time, and you're making 10% return per year. And the great thing about that is you could take that system because the exposure is low, and you can combine it with another system with low exposure and another system with low exposure, another system with low exposure, and pretty soon you've got some magic happening in your portfolio.

Does that make sense?

Okay, so let's say you've got $10,000 in your account. Let's say you've got one system that only signals every now and then what it takes. One trade at a time. Each trade is 500 bucks.

You know, and it only signals one trade at a time on average. So the maximum cash you're using is $500 out of 10,000.

Right? 5%. Yeah. Yeah. So Even if you've got a signal and you've taken a trade, you've still got $9,500 cash sitting around doing nothing. So you could take another system.

And trade it as well using your whole $10,000 of equity. And when you if you're making 5% or 10% per year on that first system and then another 10% per year on the second system, those 10% are additive, so you'll be making 20% on your portfolio. So this system here has a 68% exposure which is fairly high but typical for trend following.

So you can't really add two of these together without taking on leverage. You have to sort of give a bit of capital to one bit of capital to another.

But in a mean reversion system you might. It's more often like the other example I gave you where it's you make 10% return per year, but you're only using 5 or 10% exposure. So you can add two of them together pretty easily. Or you could add a mean reversion to a trend following system and not really use up any extra capital.

I also look at the average profit or loss percentage. I don't really look at the average profit or loss in dollar terms because in dollar terms that depends on how much capital you start with and it depends on how long the back test is, how big your account grows to. I'm much more interested in the percentage number because it's normalized over time.

So 15% at the start of your backtest is 15% at the end of your backtest. It doesn't matter about the equity. So I'll look at that. I look at the average bars held.

Because I want to make my money more efficiently if I can, or if there's a reason to hold for long term, like tax considerations. I want to know that my average profit is held for a long time.

And these two stats, the average profit loss and the average bars are broken down for winners and losers.

So a little further down you've got the.

Average profit percent and average bars held for the winners, the percentage of winners. So I'll look at those and then same underneath for the losers.

And then down the bottom. There's a couple of things I look at which are the most interesting. One is the max system drawdown.

The compound annual return divided by the max system drawdown I quite like as a measure. The reason I like that is if this is your equity curve, the squiggly line in red.

Let'S say you make 20% per year.

And you Suffer a worst case drawdown of 20%.

The compound annual return divided by max drawdown is 20 over 20 equals 1.

A ratio of 1 is pretty phenomenal, particularly for trend following.

If your maximum drawdown on another system.

Was.

Let'S say it did this.

This other system made 20%, but it had a max drawdown of 40.

That system is much harder to tolerate.

Right? Yeah. And too painful is a term for a judgment that is relevant for you. Someone else may be fine with it and someone else may be freaking out at the 20% drawdown of the first system. Yeah, right. So this is why Michael, this is why the T6 trader profile. The drawdown tolerance is also useful because based on all that, you can decide whether a system is right for you.

And you know, if the drawdown is too high, it won't be and you need to modify it before you can trade it. So car over max, max drawdown, comparative return divided by max drawdown is quite useful because you always want to be Basically the objective of your optimization and your system testing and everything is to push that up to make it better.

Now I've traded systems with car over max drawdown as low as about 0.3.

But I prefer sort of 0.6 or above for the individual system. But if I'm combining systems into a portfolio, you don't have to evaluate the system individually. So you know, let's say you've got a bull market system that makes money when the market's going up and you've got a short selling system that makes money when the market's going.

The stats of the bear market system that makes money when the market's going down don't have to be great on their own. It can have a fairly big drawdown and a fairly modest return. Like big drawdown, small return. Because when you add it to your other system, the combined effect is really good because they're diversified, they run counter to each other.

These are the stats that I focus on most. There's a lot of other stuff in there, but a lot of it is unnecessary.

Okay. And so the same thing goes in your optimization. I'll just, I have to go into the code of this system and just run a quick. Change something to it and optimize so we can look at that.

So I just made one variable, an optimization variable. So I'm just going to press optimize now so we can get all of those columns up. And this might take a little while because I haven't done an optimization in the system before. Here we go. So you'll see, the first thing you should notice is that the columns are pretty much the same as in that backtest report that we just saw. Net profit, percent, net profit, dollars, compound annual return, risk adjusted return, exposure.

Maximum system drawdown, percent. They're all basically the report for the backtest gives it to you in a vertical format. And when you run an optimization, it gives you all those same numbers in a horizontal format.

That makes sense.

See the columns that you've got here?

So net profit, car risk, adjusted return, exposure, and everything, they're exactly the same headings as were down the side in the backtest report we just looked at. So if you understand the backtest report, then you understand the optimization.

But all you're doing in the optimization is comparing one row to the next to make a judgment about which is better.

So it's lining them all up for you so you can make that comparison. So when you look at the optimization report, you want to look at the same columns that I highlighted in the backtest report because everything is a trade off. You know, one might come with a really high percentage profit, but a really high drawdown. One might come with a lower profit, but a nice smooth equity curve. A high win rate means smaller winners typically. Low win rate means bigger winners, typically. So you kind of want to understand the trade offs and find the one that best suits you. Now generally what we're doing is trying to make smooth equity curve. So I usually start.

With compound and return over max drawdown.

So comparing that first to see if one if there's a clear winner or you know, if there's a clear pattern of improvement or deterioration in the smoothness of the equity curve as we change that optimization value. So on this, in this system, we change the short term moving average. That's what I optimized. And we changed it from.

Actually, I don't know what we changed it from. Let me just go over here and have a look.

We changed it from 20 up to 100 in steps of 10. Okay. And what we can see in the compound and return of a max drawdown is the shape of the equity curve didn't change much.

They're basically 0.4 to 0.44 car of max drawdown for every value of that short term moving average. So compound annual return is the growth, the percent. It's like interest rate. It's the equivalent interest rate from the start of your backtest to the end of your backtest.

That's what compound annual return is. And then max drawdown is the biggest percentage drop at any one point in your backtest.

So compound annual return.

Over max drawdown.

So let's take a simple example. If we made 20% return per year and we had a 40% drawdown.

Then car over MDD will be 0.5.

And so the bigger that number, the smoother the system.

If you like. Let's say we, let's say we did this backtest and we got 20% error return and 40% drawdown. Let's say we change the system, make it better, and all of a sudden we don't have 40% drawdown, we've got 10.

But you're still making 20% return per year. Is the system better.

A lot. Right. Yes. What is the car over max drawdown now? So we get two. So that's substantially better than the previous example. Yeah. Now often it's not so stark, but, you know, if you add a new rule or take a rule away, the chirogrmax drawdown will tend to change.

Yeah. And so when you do your optimization, you want to look at this, because if it gets materially smoother, that's a good thing. So, and the current, that will mean the caravan max drawdown will go up.

Okay, so that's one thing that you can look at in your backtest in your optimization caravan max drawdown. But you can't just look at that. Sadly, there's no one magic number that we can, we can look at. But it's not just the bad numbers you're looking at. You're also looking for like, step changes or spikes or instability.

Like, you know, if you're. If I let me draw on the screen again for a sec here.

Because basically with optimization system design, one of the most important principles we're trying to adhere to is to find a system which is stable over time.

And stable across different parameter values. But if you've got, let's say this is the compound annual return on the vertical axis, and on the horizontal axis is all the different values of the parameter that you're testing. Right. Let's imagine the chart was like this. You know, it's kind of all over the place.

It doesn't give you any confidence that that system is.

Working. It's kind of random. But if instead the profile looked kind of like this.

Well, then it's pretty stable. Right. And the answer is you choose the value for the parameters in the middle of the area of stability.

And so you've kind of got to look at the performance stats in optimization and the parameter, how they vary as you change the Parameters to make your judgment about where you should put your parameter value.

And so I do this in my head. Like I sort of visualize looking at the numbers. Some people put it into Excel and draw the charts. I used to do that. To be honest, I find it a little slow now. So I really just kind of of look at the trend in the numbers and you know, pick, pick the value. But I do that for compound annual return. I do it for max system drawdown.

I do it for the car over max drawdown which is further to the right and all the stats I'm interested in. Right. So I'll look at the car over next drawdown which is here.

I look at the number of trades which, which is actually quite important.

If you do an optimization and the number of trades falls off dramatically and ends up you don't want to choose an optimum value that produces very few trades because it's probably not reliable.

So at one end of the optimization if you're producing a thousand trades and at the other end of the optimization you're producing 50, you know that this end of the optimization is kind of suspicious. You can't really rely on it. It's not enough data. Yeah.

I also look at the average percent profit loss, average bars held.

All the same things I mentioned earlier, the percent winners.

Percent winners, average win, average loss. All of the same things are highlighted on the performance report. I'll look at in the optimization report and I'll choose the row that represents the best trade off. So one of the most, one of the most challenging things I guess to get our heads around for optimization is there's not a right answer. But we're looking for things that we clearly want to avoid. And we're looking for areas of better performance.

Areas of worst performance. And we obviously going to position our parameter values around the areas of better performance as long as they're stable.

The mistake that most people make when optimizing a trading system or working with a trading system is pressing optimize and using one performance measure to tell them which is the best value.

But it's too simplistic like it doesn't really work that way because we're not after best, we're after most stable and profitable. If we can improve the performance and get most stable, great.

But if the best performance is an unstable area, like if you vary the parameter value just a little bit and the performance falls, crap, then that's no good. We don't want that.

So what I would, what I would suggest as a next step is to practice this.

So run an optimization, get this table and say, okay, I think this is the best value. This is the best trade off.

Sam.