From the August 2010 newsletter:
The financial industry certainly places more emphasis on style than substance. So when their work is actually evaluated, it tends to be disappointing. Wall Street’s earnings forecasts? They suck. Performance of mutual fund managers? Quite embarrassing. Do Morningstar ratings also belong in the same category? You probably see them all the time; mutual fund companies love using Morningstar ratings in their marketing materials. But is there any value in a 5-star rating?
Luckily for us, researchers recently looked into these ratings and published their results. They compared Morningstar ratings vs. fund expense ratios as a predictor of future performance. The expense ratio is one of my favorite metrics. If you assume that mutual fund managers have no value, which I find to be a very good approximation, you would expect lower costs to predict better performance. The report found just that:
Expense ratios are strong predictors of performance. In every asset class over every time period, the cheapest quintile produced higher total returns than the most expensive quintile.
And how about Morningstar ratings? 5-star ratings predicted better performance vs. 1-star ratings in 13 of 20 observations, a success rate of just 65%. That sounds pretty good on its own, but it’s still worse than a metric that anyone can look up in seconds.
Since Morningstar uses prior performance (after fees) to calculate its ratings, the ratings already include information about expense ratios indirectly. So what is Morningstar adding with its fancy algorithm? Let’s use a little high-school algebra to find out (Warning: Geek Alert!):
Rating = Expense Ratio + Morningstar's Additional Analytics
And we just found out that:
Expense Ratio > Rating
Finally, using my graduate degree in math, I get this:
Morningstar's Additional Analytics < 0
Yes, its algorithm is horrible. And that’s not all. Morningstar reserves its 1- and 5-star ratings for the top and bottom 10% of funds. However, expense ratios were split into quintiles, or as normal people would say, 20% buckets. That’s just so sneaky. So expense ratios were handicapped by using 20% buckets instead of 10%, and still beat Morningstar ratings. Ouch.
Why would the researchers do that? Well, there’s one thing I forgot to tell you. People have done this evaluation many times with similar results, so it’s not news to serious students of investing. The interesting part of the report I quoted is the publisher: Morningstar. If you read its report, it sounds like a politician answering a tough question — uncomfortable. Independent thinkers can go directly to the results here (pdf).
After writing this, I noticed that Morningstar clarified that ratings are indicators of past performance, and should not be used to predict future performance. If Morningstar were concerned about substance, it would tailor its ratings to how investors actually use them — as an indicator of a good investment. If it did that, most 5-star rated funds would just be index funds. But Morningstar unfortunately emphasizes style (and money), so it ends up with an imperfect rating system that benefits one of its biggest clients, mutual funds.