I was recently lurking around an online investment forum when the following post came up related to survivorship bias –

I recently ran an experiment where I generated random 7 to 50-stock portfolios from the 500 largest U.S. traded companies and measured their performance over the past 10 years. All of the randomly selected portfolios outperformed the S&P500 and BRK.B in terms of compound annual rate of return. Has anyone else tried this experiment? I then convinced myself that it worked by looking at the Guggenheim ETF RSP that holds an equal weight portfolio of the S&P500 stocks. It too beat the S&P500 over the past 10 years. I posted a video on YouTube of the experiment and the results”

Before I dig deeper into this article, take a moment to see what problem the above statement runs into.

The Problem with Survivorship Bias.

The answer is survivorship bias. By purchasing the top companies of today and backtesting them over the past ten years, you are just taking stocks that have already won and bought them at some point in the past – completely skewing your returns.

Is survivorship bias relevant? Survivorship bias is a big problem in the mutual fund industry. Many mutual fund companies will eliminate poorly performing mutual fund products or push poor-performing funds into a larger, better-performing fund to show better results.

How does survivorship bias work?

I would rather not have to go through the process of building tons of portfolios to illustrate this, so let’s do it in one easy screen.

survivorship bias

The first thing we want to do is build a screen. When backtested, this screen will find the top ranked companies in terms of market cap for each rebalance point in the past – eliminating the above bias.

survivorship bias

This returns quite a few companies, so let’s go ahead and knock that number down to something more manageable. You’ll see why in a second.

By adding max holdings of 5, and sorting by Piotroski Score T12M, we get a nice little group of recognizable stocks.

walmart, coca-cola

Going back to the original quote, it appears that they are taking the top companies of today and then backtesting them over the past ten years, aka survivorship bias. Therefore, to mirror their process, we need to create a separate tab and backtest our results over the same period as the rest of the screener.

survivorship bias

The Results

survivorship bias

We run the backtest a little further back than the original quote by going back twenty years rather than ten. Without looking at the legend, can you tell which line is affected by survivorship bias? This sort of thing is why you, as an investor, have to be diligent and make sure the math is right before investing.

Takeaways

The companies did well in the past, but remember, that is no indication of future performance. On the other hand, the real strategy that isn’t affected by this sort of bias return just about the exact same as the market over the same time period which is to be expected, since the largest companies share correlation with the overall market.