How to lie with statistics: Investment performance charts – A Tip from The Skilled Investor
Darrell Huff wrote a short and very informative book, “How to Lie with Statistics,” which was first published in 1954 and was amusingly illustrated by Irving Geis. This book is still in print and remains very popular (Amazon book rank #2,040 in June 2007). It plainly and humorously discusses how statistics can be distorted and misused to serve the self-interest of the presenter.
Historical investment performance charts for investment funds are a case in point. While the numbers they present might be historically accurate, their presentation in advertising, on line, and in printed materials can amount to lies from several perspectives. Performance charts are designed to lure gullible individual investors with an implied promise that superior past performance will continue. The financial research literature tells us clearly that on average this is a promise that cannot be kept. In other words, historical performance charts are a veiled lie. They may report factual information, but their purpose is to deceive.
These lies or deceptions include:
1) SELECTING ONLY “WINNERS” TO PROMOTE. When selling to you, securities industry sales people and the fund companies that advertise performance select only those historical investment fund performance charts that show superior historical performance. The industry sells its winners, and it ignores or hides its losers. Charts for their loser funds are available, but sales representatives are not eager to present them. You have to dig them out yourself on the web. Or, these inferior or average performance charts will be mailed to you AFTER you have bought what you thought was a “superior” fund, but, gosh, things just did not remain superior.
Except for very, very poor historical performance, which tends to be an indicator of excessive costs, the financial research literature tells us that historical fund performance is meaningless. The industry knows that many investors naively project past performance into the future. Yet the scientific finance literature simply does not support such assumptions. If investing were this easy, then those who buy based on past performance would be consistent winners in the future and would grow relatively richer and richer. The opposite turns out to be true. [See articles in these categories on The Skilled Investor website: Luck versus Skill (4 articles) and Selecting Investment Funds (16 articles) ]
For your amusement when you are being sold to by a securities industry sales person, ask to see an asset-weighted chart that combines the entire historical performance of all the funds for a fund family. Good luck in getting to see that one! I could list a dozen reasons why you will be told that such a fund family chart does not exist. However, the real reason is that this aggregate historical performance chart would likely show that the entire fund family trails a very broad market index by almost as much as the fund family charges in fees.
I use the word “almost,” because professionally managed funds have shown a slightly positive ability to pick individual securities. Unfortunately, this slightly positive gross returns advantage is more than wiped out by management fees and transactions costs, which are several times greater than this small gross returns gain. Then, of course, there are the sales loads and 12b-1 marketing fees and the percent-of-assets management fees that you pay to your broker or investment advisor. In return, your broker and investment advisor will do you a dis-service by only pushing selected funds with “superior” performance charts and higher costs. These sales and asset management fees just drag your returns down even more year after year after year. (See this article on The Skilled Investor website: “Pay less to get more” and this category of 15 articles: Controlling Investment Costs )
2) EASY INDEX BENCHMARKING. Historical performance charts will compare a particular fund’s performance against some index benchmark. An index is an index, isn’t it? The question that individual investors should ask is whether the index benchmark really is appropriate. All index benchmarks are not the same, and there can be very significant differences between index benchmarks — even when indexes seem to match the particular investment style of the fund in question.
When you look at a performance chart, do you investigate whether the fund company picked a challenging index or an easy hurdle that they could more easily stumble over? For more about the variations between index benchmarks, see Craig L. Israelsen’s article, “Variance Among Indexes: Don’t judge an index by its title” in the May/June 2007 issue of the Journal of Indexes (Pages 26 to 29) Dr. Israelsen analyzes the various indexes published by the six major U.S. index providers (Standard & Poors, Russell, MSCI, Morningstar, Lipper, and Dow Jones). He finds very wide performance variations even with indexes that supposedly represent the same “style” of investing.
Dr. Israelsen concluded his article by commenting: “It is important to recognize that significant performance differentials among prominent indexes can lead to misleading conclusions about mutual fund performance. Funds with mediocre performance histories can be made to look better by being compared to a prominent benchmark with a weaker performance history. At the very least, the industry needs to recognize the existence of potentially sizable performance differentials among various U.S. equity indexes, and therefore view performance comparisons between Mutual Fund A and Index B for what they are: marketing materials.”
3) HARD TO INTERPRET CUMULATIVE HISTORICAL PERFORMANCE CHARTS. For humor, let’s assume for a moment that historical performance charts actually do have some useful information for individual investors.
(This might not actually be very funny to many investors who have been lured into lousy and expensive investments because of historical performance charts. It can be hard to see humor, when the securities industry siphons away your assets through high fees using the siren song of superior historical performance charts. The cover-your-ass small legal print in the footnote of the performance chart is actually right. Essentially, it says, “Don’t count on it.” And, you should not.)
Interpreting rates of change from a cumulative performance chart is a challenge for many people. Visually, cumulative historical performance charts are just very difficult to interpret. Most people would only look at the most recent values to see whether the fund’s cumulative performance to date is above or below the index. Well, of course, if you are being sold to or advertised to, then the most recent cumulative performance will always be above the benchmark, because of selectivity (#1 above, “Selecting only “winners” to promote). This is the easiest kind of fund to sell to naive individual investors — you know, “good” funds with “better” performance.
However, a fund’s performance history that would truly exhibit investment management skill (or just a sting of good luck) is the relative rate of change in fund versus benchmark asset valuation. The rate of change between the fund’s historical performance and the benchmark index is what counts. A consistently superior fund would have a cumulative performance line that increasingly and consistently diverges from the benchmark index. Visually, the wedge between the two lines should just keep widening. (On the other hand, a widening wedge could also describe the situation of an overly easy benchmark comparison and mediocre fund performance. (#2 above, “Easy index benchmarking)
Rarely do you see historical performance charts with increasingly widening lines — particularly since luck is a major factor and high fees and high trading costs tend to drag fund performance down relative to appropriate benchmarks. If, for example, the lines diverged quickly ten years ago and then they maintained a relatively constant gap thereafter, that could mean that a very small and immature fund got lucky and/or it had a riskier investment portfolio profile. Then, money from performance chasing individual investors flowed in, and the fund got much larger. If the gap between the lines on the chart does not increasingly widen, then this means that subsequent performance has just been mediocre. If the lines tend to narrow that demonstrates subsequent inferior performance. Cumulative performance could still be above the index due to a selectivity bias and/or an easy index benchmark, but the fund might really have been exhibiting mediocre or inferior performance for years.
4) INDUSTRY CYNICISM ABOUT UNSOPHISTICATED INDIVIDUAL INVESTORS. The securities industry knows that chasing historical performance is bad for individual investors, but they encourage this behavior by publishing historical performance charts and 4 star and 5 star Morningstar Ratings, which are also largely meaningless. For the industry not to know would imply that many very smart professional investment managers have had their heads in the sand about decades of financial research. (Concerning Morningstar Ratings, see articles in this category on The Skilled Investor website: Ratings Services: Morningstar (16 articles) and “Investment astrology – should you pick investments according to the Morningstars?” )
The securities industry and many of its brokers and investment advisors know that low cost index strategies are better for individual investors. However, the “active-management-beat-the-market” industry crowd will not make any money off of you, if they tell you that. They have to push the “we deliver superior performance” mantra, because that is the justification for their excessively high and performance killing fees. Since market realities make it virtually impossible for actively managed funds to consistently beat the market after their fees, they have to resort to promises, deceptions, and what Darrel Huff would call “statistical” lies. These lies include: #1 selecting only winners to promote, #2 easy index benchmarking, and #3 hard to interpret cumulative historical performance charts.
Those in the industry who do not understand this have not bothered to do their homework. And, why should they? If these superior performance hustlers learned what is good for individual investors, they might also realize that they should find another career that adds some genuine value to our society.
CONCLUSION: How one fund family solves this problem — They refuse to play the game.
In the same issue of the Journal of Indexes, which published the Israelsen article referenced above about variance among index benchmarks, there was a “Straight Talk” interview with John Brennan, CEO of the Vanguard Group, who succeeded John Bogle in 1996. (Pages 24-25, 50) When asked about performance chasing, Brennan said the following: “The way(s) you mitigate against it are several. One, you never — in our view — never promote performance. You just never run a performance ad. I think that is endemic to our business, and I think it’s a shame for our industry. When you read a performance ad, there is an assumption that the strong performance will continue. And that is not necessarily true. The second thing is … when you call Vanguard to talk about our funds, or when you read our literature, you won’t find a Morningstar Star Rating. … The third way to mitigate is with communication. When you read our annual reports after a terrific year, you can be sure that we will tell investors, “please don’t assume that this will continue.” … Finally, you also have to be willing and able to close a fund and have a cooling off period. That’s bad for business, and it always inspires some nasty letters. But it’s how you build a performance track record. I have never once regretted closing a fund. At the end of the day, firms that promote performance do so at their own peril.” (And, The Skilled Investor would add — at the peril of their clients, i.e. you!)
(Note that there is no business relationship between The Skilled Investor Blog or website and the Vanguard Group. The Skilled Investor has not received any kind of compensation for this article. Note, also, that some of The Skilled Investor’s personal family assets have been invested in Vanguard’s index funds for many years.)
