An excerpt from "What Are You Really Getting When You Invest in a Hedge Fund?" from the AAII Journal.
The hedge fund industry asks current hedge funds to report their historical returns, and individual hedge funds choose whether they want to provide those returns. This procedure ensures that the reported returns exaggerate the returns earned by the typical investor. The biases include survivorship bias, non-reporting bias, questionable-numbers bias, and instant-history bias.
Suppose there were 1,000 hedge funds in existence five years ago, but only 550 of these remain today. Survivorship bias refers to the fact that the returns on the 450 non-surviving funds are not reported. Naturally, on average, these non-survivors' returns were poor. Yet, their returns and the returns earned by their investors are unrecorded. Thus, the reported returns exaggerate the returns earned by the average investor.
Survivorship bias also exists in the mutual fund industry. For 1971–1991, one study estimated that average reported returns in the mutual fund industry were 1.5% higher than the returns actually earned by the average investor. However, the survivorship bias in the hedge fund industry is probably larger than this.
The second and third biases exist because the current hedge funds can decide whether or not to report their current numbers, and their numbers are not audited. To continue with the example above, of the 550 survivors, 400 may provide their returns. Excluding the returns from the 150 non-reporting hedge funds is the non-reporting bias. In addition, the returns provided by the 400 hedge funds lack external verification. This is the questionable-numbers bias.
The last bias is the instant-history bias. A manager may begin several hedge funds, each one based on a different strategy with a small amount of the seed money in each. After a couple of years, the successful strategies will go public and their historical records are made public. The records of the unsuccessful strategies are not made public. Moreover, the historical returns earned by the successful strategies are not returns earned by typical investors.
It should be obvious that the industry-reported returns should be viewed skeptically. One study ["A Critical Look at the Case for Hedge Funds," by Richard M. Ennis and Michael D. Sebastian, Summer 2003 Journal of Portfolio Management] examined the overall bias in numbers for the 1992–2002 period by comparing the reported 7.1% average return on the Hedge Fund Research Fund of Funds Index, which is the return earned by actual investors in funds of hedge funds, to the Hedge Fund Research Composite Index's reported average return of 11.3%. The 4.2% difference suggests a large bias in the industry's numbers. Moreover, the 7.1% return earned by actual investors was less than the 8.5% return on the S&P 500 and the 7.3% return on the Lehman Aggregate Bond Index over the same period.