Do the New Morningstar Mutual Fund Ratings Predict Top Mutual Fund Performance?

Compared to its old system, early data indicates that Morningstar’s new star mutual fund ratings system may have some mutual funds performance predictability. However, this new 5 star mutual rating system may just be a proxy for fund costs. Instead of using the stars, investors should use costs to screen funds, which are a more direct and proven indicator of future top mutual fund performance.

For two decades, the Morningstar Ratings* have had a very significant impact on individual investors’ mutual fund purchase decisions. Because of its prominence, numerous academics and investment professionals have studied and published scientific investment papers about the Morningstar Rating system. In mid-2002, Morningstar made substantial changes to its star rating system. (See: How the new Morningstar Ratings for mutual funds have been determined since mid-2002)

In late 2004, Morningstar issued a brief analysis of the first two years of data from the new mutual fund rankings.¹ In “Studying the New Star Rating for Funds,” Russel Kinnel cautioned that two years of data were not sufficient to determine whether what he was commenting on were facts or just statistical noise. However, he suggested that the new Morningstar Ratings might have some predictive abilities regarding future top performing mutual funds – particularly with bond mutual funds. (See: Is it worth paying higher bond mutual fund management fees?)

Mr. Kinnel’s comments about the ability of the new Morningstar Ratings to predict superior future mutual fund performance are a departure from previous Morningstar commentary. In the past, the company rather studiously avoided making performance prediction claims about the old Morningstar ratings. (See: What does Morningstar, Inc. say its mutual fund stars can do? and What does Morningstar, Inc. say its mutual fund stars cannot do?)

The scientific investment literature indicates that “superior” historical mutual fund performance net of costs and taxes is largely a random affair driven by stock market volatility and luck. 

On average, the scientific data have not demonstrated that mutual fund managers can persistently and predictively deliver superior performance. Instead, only very inferior past performance seems to be indicative of likely inferior future management performance. 

Some of the research literature does demonstrate that the average professionally managed mutual fund exceeds market performance benchmarks modestly - before their higher investment costs and increased taxes are considered. However, this slight increase in gross performance is negated several times over by the excessive fees and higher taxes of actively managed mutual funds. 

In addition, the range of performance variation is very wide across actively managed mutual funds. Furthermore, there are no reliable ways to identify beforehand mutual funds that will beat their benchmarks and exceed their added costs. If you ignore investment fees and shop only the basis of 4 star and 5 star mutual fund ratings, you are more likely to pick an inferior and expensive actively managed mutual fund.   

In his widely cited paper, “On the Persistence in Mutual Fund Performance,” Professor Mark Carhart of USC stated, “persistence in mutual fund performance does not reflect superior stock-picking skill. Rather, common factors in stock returns and persistent differences in mutual fund expenses and transaction costs explain almost all of the predictability in mutual fund returns. Only the strong, persistent underperformance of the worst-return mutual funds remains anomalous.”²

It makes sense economically that the lowest mutual fund costs and best mutual fund performance are positively related. 

Individual mutual funds tend to maintain relatively stable expense ratios over time. Some have higher charges and some have lower charges. One would expect that funds with lower costs could generally have higher performance. 

In his article, Mr. Kinnel presented a table showing the two-year average performance rankings of funds within each new star rating from 1-star to 5-stars. Data were provided for four types of funds: 1) domestic equity, 2) international equity, 3) taxable bond, and 4) muni bond. Star ranking and performance appeared to be correlated – the higher the star ranking the better the performance ranking. At first glance, it could seem that the new star ranking system might somehow predict superior future performance.

However, in the same chart Mr. Kinnel also presented data on the average expense ratio, front-end load, and deferred load for each star rating category. Clearly, across all fund types, there also appeared to be a very strong correlation between lower expenses and no loads with higher four star and five star mutual fund ratings. Is this newfound predictive ability of the star ratings system just a proxy for fund expenses?

If that were to be the case, then why bother to pay attention to the Morningstar Rating, which is subject to very significant random variations across funds? Instead, The Skilled Investor suggests that you should simply pay direct attention to expense ratios, loads, and trading costs, which are economically based factors that clearly can drag down performance. Slash your costs and avoid the trading costs associated with portfolio turnover. Buy broad market mutual funds directly and avoid paying sales loads and 12b1 fees. (See: Choose mutual funds with lower investment management expenses and Avoid mutual funds with sales commissions and 12b-1 fees)

The inverse relationship between mutual fund performance and investment costs has been widely established in the scientific investment literature.

Do investors really need a 4star and 5star based proxy for fund expenses that is subject to significant random distortions? If an investor selects a mutual fund based on lower costs, he will still be subject to volatility. However, his cost-based selection criteria would be tied directly to an economic factor that has been established in the scientific investment literature. If instead he selects a fund based on a star rating that just may be temporarily high, he could be accepting unnecessarily high expenses because of that star rating. In the end, he would be much more likely to obtain inferior results and end up poorer. (See: How stable have Morningstar Ratings for mutual funds been over time? and What the instability of mutual fund Morningstar Ratings means for long-term investors – Commentary)

Millions of investors have used the Morningstar Rating system in the past two decades. Mutual fund companies have selectively and heavily promoted only their 4 star and 5 star mutual fund ratings to the detriment of investors. 

Yet, we still do not really know what the stars actually mean. The Skilled Investor would suggest that Morningstar’s analysts conduct a statistical study, which controls for and extracts the influence of investment costs. Then, perhaps they might be able to tell investors whether the new star rating system contains any useful predictive information and what it might pertain to beyond the well-known and predictable drag of expenses on returns.

The fact that the Morningstar star rating system changed significantly in mid-2002 could be particularly frustrating to those researchers who have tried to apply the analytical methods of investment science to evaluate the informational content, validity, and predictability of the Morningstar star rating system. It is certainly challenging to make definitive statements about the usefulness of a star rating system that is a moving target, and as a result is less subject to scientific verification.

When Morningstar refined its new star mutual fund rating system in 2002, it brought the stars more into line with the practices of the scientific investment literature. To paraphrase a famous domestic maven who has lived in several big houses, “this is a good thing.” However, it remains unclear what the impact of these new changes to the star rating system will be. Will the changes make the new stars more or less useful to rational mutual fund purchase decision making by individual investors?

The answers are not obvious. It will take more time for data on the new Morningstar star system to be collected, and objective statistical studies must be conducted and published. Meanwhile, in effect, individual investors will be following the new stars blindly - just like some many of them and their advisors have done for some many years in following the old mutual fund star ratings.

Also, see these related articles:

• Investment astrology – should you pick investments according to the Morningstars?
• Morningstar Ratings should be used with caution
• The quality of the “old” Morningstar Ratings prior to mid-2002
• Simplifying investment decision making can be taken too far
• High Morningstar Ratings can lure you into funds with costly sales loads
• How Morningstar Ratings for mutual funds are used as a marketing tool
• Do mutual fund Morningstar Ratings changes influence individual investors?
• Does it pay to trade when the Morningstar Rating of a mutual fund changes?
• Do Morningstar Ratings predict risk-adjusted equity mutual fund performance?
• What might be wrong with buying a mutual fund with a 4 or 5 star Morningstar Rating?

* The Morningstar Rating is a trademark of Morningstar, Inc. 

1) Russel Kinnel, “Studying the New Star Rating for Funds.”. http://www.morningstar.com November 22, 2004
2) Mark M. Carhart, “On the Persistence in Mutual Fund Performance.” The Journal of Finance, Vol. LII, No. 1, March 1997, p. 1