Survivorship Bias – How does it work?
I was recently lurking around an online investment forum when the following post came up –
“I recently ran an experiment where I generated random 7 to 50-stock portfolios from the 500 largest U.S. traded companies and measured their performance over the past 10 years. All of the randomly selected portfolios outperformed the S&P500 and BRK.B in terms of compound annual rate of return. Has anyone else tried this experiment? I then convinced myself that it worked by looking at the Guggenheim ETF RSP that holds an equal weight portfolio of the S&P500 stocks. It too beat the S&P500 over the past 10 years. I posted a video on YouTube of the experiment and the results”
Before I dig any deeper into this article, take a moment and see what the problem the above statement runs into.
The answer, survivorship bias. By purchasing the top companies of today and backtesting them over the past ten years, you are just taking stocks which have already won and bought them at some point in the past – completely skewing your returns. Survivorship bias is a big problem in the mutual fund industry. Many mutual fund companies will eliminate poorly performing mutual fund products, or push poor performing funds into a larger, better performing fund to show better results.
How does survivorship bias work?
I would rather not have to go through the process of building tons of portfolios to illustrate this, so we are going to do it in one easy screen.
The first thing we want to do is build a screen. When backtested, this screen will find the top ranked companies in terms of market cap for each rebalance point in the past – eliminating the above bias.
This returns quite a few companies, so let’s go ahead and knock that number down to something more manageable. You’ll see why in a second.
By adding max holdings of 5, and sorting by Piotroski Score T12M, we get a nice little group of recognizable stocks.
Going back to the original quote, it appears that they are taking the top companies of today and backtesting them over the past ten years. Therefore, to mirror their process, we need to create a separate tab and backtest our results over the same period as the rest of the screener.
How does it do?
We run the backtest a little further back than the original quote by going back twenty years rather than ten. Without looking at the legend, can you tell which line is affected by survivorship bias? This sort of thing is why you, as an investor, have to be diligent and make sure the math is right prior to investing. Yes these companies did well in the past, but remember, that is no indication of future performance. On the other hand, the real strategy that isn’t affected by this sort of bias return just about the exact same as the market over the same time period which is to be expected, since the largest companies share correlation with the overall market.