Adrian Reid explains two distinct survivorship-bias problems that affect backtests: (1) using today's index constituents for historical backtests (a major issue requiring historically accurate index membership data), and (2) company-survival bias caused by delistings (bankruptcy, acquisitions, M&A). He argues that delisted-stock data is essential for index-limited backtests but is often less critical for well-designed trend-following systems with entry filters and stop losses, because such systems rarely hold securities long enough to reach bankruptcy. He also notes acquisitions often produce positive jumps that, if missing, would understate historical returns, and reports from his own testing that including delisted stocks usually improved performance by providing more opportunities.
Chapters
If you've developed trading systems for more than five or 10 minutes, you've probably heard of survivorship bias as a massive issue that you absolutely have to overcome. Otherwise you're deluding yourself and your system will not work right now. This is true sometimes, but not all the time. Some people in trading industry make a way bigger deal of this than it needs to. In this video I'm going to talk about when survivorship bias is is a real problem. When you need the delisted stock data.
To make sure that your backtest is.
Super accurate and when maybe it doesn't matter quite as much as you think.
So stick around, check out this video.
But before we go any further, make.
Sure you hit the subscribe button and.
Give this video a thumbs up so that you get notified of all of my future videos. Because I've got new trading content coming out all the time and I don't.
Want you to miss a thing. There's two main areas where survivorship bias is a problem. One of them is massive. Okay, let's cover the massive one first. Massive survivorship bias.
Concern is when you're back testing on stocks in an index. So let's say you have a system and you want to trade just stocks in the S&P 500.
If you test on today's S&P 500 stocks and do a 20 year back test, the results are going to look amazing if it's a long side system. Because Today's S&P 500 stocks are the stocks that got into the S and P grew. Well, the stocks that grew to get into the S and P today. So by choosing today's list and back testing on those, you're pre selecting today's winners, which is a massive problem. So if you're going to back test a system that is on that trades just stocks within an index, given that the constituents of that index change over time, you must have historically accurate index constituents. What that means is on every bar of the backtest, before you take a trade, your backtest has to check on this bar. Was this stock a member of the index or not? If it was, you can take the trade. If it wasn't, you skip the trade. In order to do that, you need the nor gate data because it's the only place that you can get the historically accurate index constituents. The second type of survivorship bias is did the company survive? Right. And that's, you know, delisting bankruptcies, takeovers, mergers and acquisitions, or cause a company to disappear. So you would think that by testing on Today's listed stocks, you have survivorship bias because the stock survived until today. And you do. There is some in there, but it can work both for you and against you. And so depending on the type of system you don't like, you may not your back test may be quite fine. Right? So mention a trend following system. Now to get a signal to buy a stock in a trend following system, the stock has to be going up.
A stock that's going up typically doesn't just explode in a fury of bankruptcy and disappear. It can happen. It's happened to me twice or three times in 20 something years and thousands and thousands of trades. Neither of those times were in Australia. They were both on less regulated markets, let's say.
So if we're trend following, we're only buying stocks that are going up. We reduce the risk of a stock disappearing in our backtest. Now here's why this is a problem. Because if your system might hold a stock that goes to bankruptcy, but your data doesn't include delisted stocks, then your your backtest is optimistic. But if your system would never hold a stock that goes to bankruptcy, then your backtest won't be optimistic because it wouldn't have held those stocks anyway, even though they're not in the backtest. So we've got to ask ourselves why do stocks get delisted and stocks get delisted for a couple of reasons. One is bankruptcy. And in a, in a system that has a trend filter on the entry and a stop loss on the exit, you're generally not going to hold a stock all the way to bankruptcy. Just won't happen. Second reason why stocks disappeared get delisted is acquisitions. Now if the share price is $10 per share and a company comes in, another company comes to acquire that stock and they offer $5 per share, guess what? Nothing happens. Because the shareholders aren't stupid enough to take an acquisition price below the current offer. So for an acquisition to go through, it has to be at a premium. So if the price of Your stock is $10 a share and the acquiring company comes in at $12 a share and say, well, it's an on market market takeover, we're acquiring all of the shares at $12 a share and the board of directors recommends it and shareholders vote in favor of it, share price jumps to $12. Basically everyone gets out around $12. So in that case, that stock that has been delisted due to acquisition is no longer in your database. That doesn't have your historic, your delisted data. But if you were holding that stock in the past, what would have happened? You would have made a profit because you were holding a stock at 10 and then the acquisition came through and it jumped to 12 and then it went flatlined, basically until the merger went through and you got $12 a share. So that's actually a positive uplift that you don't see in your backtest. So the survivorship bias issue for.
Systems that have a trend filter and a stop loss and sensible exits, basically sensibly designed systems isn't that much of a problem for acquisitions, and it isn't that much of a problem for bankruptcies, because most of the time you won't be holding them. Might happen sometimes, so you might miss a few outliers, but in thousands of trades is not going to be a big deal.
So in terms of delisted stock data, I don't get as excited about it as some people. What I've found, in most cases, actually all cases that I can remember with my systems, when I've designed the system on data without the D listed stocks and then tested it later on data with delisted stocks, the system performance gets substantially better.
With the delisted stocks. Why? Because you got more opportunity. Because there were more stocks to give signals and so it made more money. The stats didn't degrade.
So therefore I don't lose too much sleep over it. But you absolutely cannot design a system that trades and stocks in the index without the historically accurate data. That's very misleading and dangerous. It.
