Financial markets are driven by emotion and noise, making past returns unreliable guides to the future. The tutorial warns against overfitting trading strategies to historical data: a three-parameter grid search creates 125 possible configurations, most of which fail in real trading. Robust strategies should produce similar results when parameters shift slightly, like a moving-average crossover system with a 50-day and 200-day window.
Chapters
Designing a trading system is not easy. Financial market data is the amalgamation of fear, greed, noise, and generally a whole salad of millions of people across the world taking bets practically simultaneously on the price of financial instruments. Given this fact, it is plain to see why so many financial advisors say past performance is not indicative of future results. While there is a temptation to try and create trading strategies that have wonderful performance on past data, also known as in sample data, there have been numerous studies showing that when too much effort is placed into this activity that the system fails going forward on unseen data, or what is more formally referred to as out of sample data. This is more commonly known as overfitting, which is the act of trying to tailor a backtest to look as good as possible, smoothest line, highest profit, whatever the criteria may be, rather than seeing a backtest as an evaluation of a trading system. In order to avoid such phenomena, trading systems should strive to have as few parameters, essentially the knobs you can turn to change settings as possible. Why? Because the number of system configurations increases exponentially with parameters. That is, consider a grid search that tests five separate settings for each parameter. A one parameter system would need only five runs, a two parameter system would need 25, a three parameter system would already need 125, and beyond that the numbers start to get very high very fast. Furthermore, a trading strategy should not perform much differently when the parameters are slightly altered. For instance, a trend following system should have similar results for similar settings of its trend filter. For instance, a trend following system that buys a stock when its 50 day moving average crosses above its 200 day moving average should behave similarly if those numbers are 45 and 210 respectively. Lastly, one can conduct statistical hypothesis tests to determine whether or not the following properties are true. First of all, is there a relationship between an indicator and future returns? Secondly, is there statistical evidence for the signal process to generate outperformance against some benchmark? While Quantstrat's built in mechanics to protect against overfitting are beyond the scope of this class, these are nevertheless important things to keep in mind. Let's move on to some exercises.
