Thank you for your comment. You are correct. It would be difficult to implement my test, because it would require verification of explicitly stated predictions over time. Few individual investors try to do this at all, yet they listen to the predictions of others and assume others have skill. Individual investors tend to be happy, if the prices of the particular securities in their portfolios go up rather than down. They rarely do accurate performance comparisons with appropriate index benchmarks.

Another way to look at this would be to analyze the accuracy of predictions over time made by professional analysts who publish reports based on fundamental business analysis. This would have to be a statistical study, because the number of analysts who are correct on the logic of ALL their predictions would go to zero is just a quarter or two.

This all leads to statistical studies as the only solution to distinguishing luck and skill, and there are many financial academics and “buy side” money management professionals who do analyze analyst accuracy. The theory being that there are a few analysts in the pack who have a clue and most of the rest are followers who write history rather than predict with any accuracy. If you could sort out true skill from among the analyst herd, then you could make more money (in theory) by following only the skilled analysts and ignoring the rest.

The point, however, about such statistical studies of professional analysts is that it takes a very large amount of data over a very long time to “prove” statistically that a particular analyst is skilled rather than lucky. In practice, you cannot tell the difference between luck and skill — even over a long period of time. If you cannot reliably identify skill and there is an inexhaustible supply of others who claim they have it but don’t (most professional analysts, stock newsletters, drunks at parties), then is it worth paying ANY of them extra to follow their advice? I don’t think so. Studies of active management show a slight bit of skill which is swamped by all the extra costs.

To extend this a bit further, a very large amount of professional money management and trading activity is simply “machine driven.” With machine driven money management, it does not matter if a human analyst is right or wrong. You cull through huge amounts of data looking for what seem to be superior trends and trade that information. The problem of “skill and luck” is still there, but in just a different form. Is a machine trader’s strategy based on something economically real or just random patterns in the data that appear to be insightful? You can only tell AFTER you have placed your bets.

On this subject, I just read a great quote from William Sharpe in the November 2007 Stanford Business School magazine (p.25). He said, “A lot of people have a strong vested financial interest in saying ‘I know how to beat an index fund,’ and if you torture a body of data long enough it will confess to anything you want. I’d be skeptical of anyone who assumes that there is a simple formula to get something for nothing.

In summary, there is not any good way to tell investment skill from luck. On their own individual investors are completely out gunned by the industry. When they pay professionals more to get more, instead they are more likely to get less. The only smart strategy is to run as fast as you can to the very low cost end of the investment products spectrum. If you do, you end up in “passive index land” rather “active fantasy land.”

The Skilled Investor

To be a “genius” investor you only need to get your decisions right more often than the other guy. You wouldn’t necessarily have to get all of them right. A typical investor makes many, many investment decisions - which ones would be the ones that you would use to test his success, and what population of decision makers would you test these decisions against?

No, the statistical method of distinguishing is probably better, even though it too is flawed. If you take a look at the population of fund managers - this is a much smaller population - say 50,000 (which is probably an over estimate). If someone consistently beats the average, a reasonable guess would be to assume that he knows what he is doing, although that is not necessarily so.

BTW, the chance of flipping H 10 times in a row is 1 in 1024… if the population of investors in the U.S. all flipped coins, there would be tens of thousands of them that would get heads 10 time in a row…