The quality of the old mutual fund Morningstar Ratings prior to mid-2002

From 1985 to 2002, increasingly large numbers of individual investors directed billions of their investments dollars into funds that scored higher according to the old Morningstar Rating system. Morningstar has said that its old star ratings system had serious flaws.

Prior to mid-2002, Morningstar made relatively few changes to its Morningstar Rating* methodology, since their introduction in 1985. This article does not attempt an exhaustive history of Morningstar’s prior methods or adjustments, but rather highlights key aspects of the star rating system prior to mid-2002. Two of the most notable differences between the “old” and “new” star ratings relate to how Morningstar formerly defined star rating categories and how it estimated and adjusted for risk.

Now that the stars have been redefined, what rational basis do investors have for following the “new” star ratings? The Skilled Investor is concerned that star ratings greatly over-simplify investment decision-making to the potential detriment of those who blindly follow them. (See: Morningstar Ratings should be used with caution)

Prior to mid-2202, Morningstar used four broad categories to group mutual funds and develop its stars prior to mid-2002. These categories were 1) US stocks, 2) international stocks, 3) taxable bonds, and 4) municipal bonds. Commenting on its own star categorization methods that it had used for the previous 17 years, Morningstar documents from 2002 bill the more numerous categories of the new star rating system as a significant enhancement that obviates certain shortcomings of the old system.

Morningstar’s comments on these prior shortcomings include “That had an unfortunate side effect: When a particular style of investing was hot – like growth was in the late 1990s or deep-value right now – a disproportionate share of funds within that style received 4 or 5 stars. It did not matter if the manager was good or bad. By being in the right place at the right time, they were able to pick up stars.”¹

“Under our original star-rating system, though, greater specialization meant that a fund’s investment universe would frequently overwhelm management’s skill. A few years ago, most high-yield bond funds were awarded 4 or 5 stars, because high-yield bonds produced higher returns than the rest of the bond universe. Likewise, large-growth funds nearly all had 4 or 5 stars at the end of 1999, while only a couple small-value funds had 5 stars. Large-growth managers were being rewarded more for luck than smarts. Today, a value rally has slanted things toward small value. Obviously, small-value managers didn’t become geniuses over night.”²

The scientific investment literature has suggested for decades that the returns of certain stock sub-groups behave differently than the overall market. By the early 1990s, “multi-factor” investment models were widely accepted in academia, because they did a superior job of explaining equity returns.

Morningstar introduced a separate “category rating” system in 1996. In an analysis, William Sharpe, Professor Emeritus at Stanford and Nobel Prize winner in economics, reported that Morningstar’s category rating system had 44 categories in mid-1997, including 20 US equity categories.³ Morningstar did not integrate its category rating system with its proprietary star rating system until mid-2002. (In mid-2002, Morningstar discontinued its category rating system, when it introduced the revised star rating system.)

Concerning Morningstar’s methods for adjusting returns for risk, the company has now adopted methods that are more similar to those that have been standard in the scientific investment literature for many years. Prior to mid-2002, Morningstar’s stars were adjusted for risk using a proprietary measure of a fund’s monthly return shortfall, if any, compared to the US T-bill interest rate. Such percentage shortfalls were totaled and then averaged across all months of the 3-, 5, or 10-year period being used to generate an old star rating. The resulting number was then was subtracted from the fund’s average monthly return over the same period to develop the Morningstar Risk-Adjusted Return (MRAR). Morningstar’s older risk measurement method contrasted with the more widely accepted scientific investment methods of measuring investment risk, which used price volatility or more specifically the statistical standard deviation of returns for a security.

In 2002, Morningstar was frank about the shortcomings of its prior risk adjustment methods used to calculate its old stars for 17 years. For example, when commenting on its former risk adjustment methods, Morningstar said, “The problem with that methodology, however, was that it failed to flag funds whose huge returns indicated they were engaging in risky behavior. For example, the old star rating system failed to adequately penalize many growth- and tech-stock-laden funds in the late 1990s. Although many such funds had posted few losses over that stretch, their huge returns suggested they were taking a fair amount of risk – a huge amount, in some cases.”⁴ 

One suspects that all those investors who were influenced by Morningstar’s star ratings to choose particular mutual funds in the 1990s would have preferred to have had the benefits of this revised risk adjustment schema much earlier. By July 1, 2002 when the new star rating system was introduced, the tech stock bubble had already collapsed. From its intraday peak of 5,132 on March 10, 2000, the NASDAQ Composite Index fell 3,728 points to close at 1404 on July 1, 2002. Yes, there had been substantial risk in many funds that had 4- and 5-star ratings in the late 1990s. 

Morningstar’s revised, risk adjustment method brings it closer to the analytical methods of the scientific investment community. It is encouraging that the star rating system after mid-2002 reflects a more standard economic expected utility model. Still, individual investors need to understand that to summarize risk and return in a simplified one-to-five star rating system, certain assumptions must be made. In particular, Morningstar has calibrated its risk model to reflect what it believes is the average risk preference of typical risk-averse retail investors.

Morningstar has characterized its average risk calibration as “a better risk measure” that “was designed by finance PhDs, but you won’t have to go back to school to make use of it.”⁵ Morningstar says that “the new Morningstar Rating(tm) is based on “expected utility theory which recognizes that investors are: a) more concerned about a possible poor outcome than an unexpectedly good outcome and b) willing to give up some portion of their expected return in exchange for greater certainty of return.”⁶ 

When evaluating whether to pay attention to Morningstar’s new mutual fund star ratings, an individual investor will need to assess whether he is a “typical risk-averse retail investor” or whether he has a substantially greater or lesser risk aversion than such an average investor. This must be a subjective decision. Morningstar appears to provide no quantitative guideposts for an individual investor to measure his relative risk preferences. Investors who judge that their risk preferences may differ significantly from that of a “typical risk-averse retail investor” need to realize that mutual fund star ratings calculated to reflect his particular preferences could differ from Morningstar’s published star ratings. This could possibly mean, for example, that a two-star rating calculated for one person could perhaps be a four-star rating for another (or the converse) depending upon the differences in their risk preferences.

Also, see these related rating service articles about Morningstar:

• Morningstar Ratings should be used with caution
• The quality of the old Morningstar Ratings prior to mid-2002
• What does Morningstar, Inc. say its mutual fund stars can do?
• What does Morningstar, Inc. say its mutual fund stars cannot do?
• Simplifying investment decision making can be taken too far
• High Morningstar Ratings can lure you into funds with costly sales loads
• How stable have Morningstar Ratings for mutual funds been over time?
• What the instability of mutual fund Morningstar Ratings means for long-term investors – Commentary
• Does it pay to trade when the Morningstar Rating of a mutual fund changes?
• What might be wrong with buying a mutual fund with a 4 or 5 star Morningstar Rating?
• Do Morningstar Ratings predict risk-adjusted equity mutual fund performance?
• Do mutual fund Morningstar Ratings changes influence individual investors?
• Investment astrology – should you pick investments according to the Morningstars?
• How the new Morningstar Ratings for mutual funds have been determined since mid-2002
• How Morningstar Ratings for mutual funds are used as a marketing tool

* The Morningstar Rating is a trademark of Morningstar, Inc. The Morningstar Rating has also been referred to in the media as the Morningstar stars, the star rating, the star rating system, etc.

1) Benz, Christine. “Special Report: Introducing Morningstar’s New Star Rating.” July 03, 2002.
http://news.morningstar.com/article/article.asp?id=77455&_QSBPA=Y
2) Kinnel, Russel. “Morningstar’s New Star Rating." April 22, 2002.
http://news.morningstar.com/article/article.asp?id=14071&_QSBPA=Y
http://news.morningstar.com/doc/article/0,1,14071,00.html
3) Sharpe, William F. “Morningstar’s Risk-Adjusted Ratings.” Financial Analysts Journal, July/August 1998: 22
4) Benz, op cit., July 03, 2002
5) Kinnel, op cit., April 22, 2002
6) “Fact Sheet: The New Morningstar Rating for Funds.”
http://corporate.morningstar.com/US/documents/MethodologyDocuments/FactSheets/MorningstarRatingForFunds_FactSheet.pdf

VeriPlan Financial Planning Excel Software