Even if an investor has obtained superior results over an extended period, is this sufficient proof that these investment results were actually due to skill rather than just a lucky streak?
No, these investment results could still be due to chance. For example, take a large population, such as all individual investors in the U.S., and have each person perform a random chance operation like flipping a coin repeatedly. After many repetitions, a small portion of the population could appear to have remarkable results.
After 10 consecutive coin flips, some investors probably would have flipped 10 heads in a row (HHHHHHHHHH). If each head were to indicate a consecutive win in the stock market, that person might seem like an investment genius. However, ten heads in a row is no more or less likely than any other particular sequence of heads and tails. The person who flipped THTTHTTHHH has a sequence that is just as unique as 10 heads in a row. The person who flipped THTTHHTHHH, which is different only on the sixth coin toss, also has an equally unique sequence. However, to many people, these sequences of heads and tails would not appear to be nearly as exceptional as 10 heads in a row, even though the really are.
Repeated investment success may still be the result of luck rather than skill.
While it may appear that the person who got 10 heads in a row did something remarkable, it just appears to be more unusual, because the results were all the same. In a similar vein, if a large population of investors were each to choose investment securities at random and were to do this repeatedly, the results will be purely accidental. However, those who happened to pick securities with better results might seem like geniuses.
Given human propensities, you could almost guarantee that the longer the sequence of their accidental winners, the more the chests of these lucky winners would inflate with pride. They would probably also become increasingly vocal. The fact that they are no more or less likely to be a winner in the next round than any other participant, might be lost on them and anyone who hears their story, until the results of the next round are known. In addition, even if they lose the next round, they may conveniently not report their loss. Too many lucky throws of the investment dice and these persons could be absolutely insufferable at cocktail parties. (See: Can you really beat the securities markets? and Chance creates the illusion that investors can beat the stock market)
Only if an investor makes numerous specific predictions over time about WHY the prices of various securities will move in particular directions and those predictions come true far more often than not, can investment skill rather than dumb luck be demonstrated.
Because long-term success may still be due to random chance, there needs to be more stringent criteria for judging true investment skill. True skill might be demonstrated by two methods:
- An investor with superior skill could have a better understanding of what known information means and could use trading strategies to capitalize upon that better understanding. For example, such an investor might be better at using published accounting information to identify companies whose default risks would in fact turn out to be different from what others judge them to be. If such an investor made a large majority of such accurate selections, then it would be more probable that this analyst demonstrated superior skill.
- An investor with superior skill might be a better prognosticator of what is likely to happen in the future. In this case, the investor would make specific prior predictions about things that would happen. A higher degree of accuracy on specific predictions would be a way to identify an investor with skill.
Both of these potential sources of verifiable skill have a common element, which is the specificity of their predictions. Not only must an investor with skill call the direction of price movements, he must also predict why the price will move with a reasonably high degree of factual precision. If an investor predicts that a security will perform exceptionally well for specific reasons, and it does perform well, but for reasons that have nothing to do with his predictions, then this is still a chance result and not skill. (See: The illusion of superior professional investment manager performance)
The future unfolds unpredictably, and just because it turns out one way or another does not make an individual investor a genius or a fool.
An investor can only properly claim to be an investment genius when the proof is indisputable over a very long history of precise predictions of what would happen. Moreover, as with soothsayers and charlatans, it does not count to predict accurately a few times in the midst of a very large number of inaccurate predictions. Certain investors have received substantial acclaim because they once accurately called and acted upon a turn in the market, while their other erroneous predictions were forgotten.
Note also that an investor is not necessarily a fool if the future happens to unfold differently than he predicted. His assessment of opportunities and risks may have been reasonable at the time he made his judgment. However, what he may have been concerned about might not have materialized because other factors became more dominant. An investor is only a fool, if he convinces himself that he can predict the future, when he does not have a specific record of accomplishment as evidence. Investors who have been lucky and become over confident because of their past luck can become very dangerous to their own future financial welfare.
Comment by shadox
1 November 10, 2007, 1:22 pm o'clock |
This is a very preceptive post, however, I disagree with you on the test for identifying true success. It is simply not a usable test. How would you implement it?
To be a “genius” investor you only need to get your decisions right more often than the other guy. You wouldn’t necessarily have to get all of them right. A typical investor makes many, many investment decisions - which ones would be the ones that you would use to test his success, and what population of decision makers would you test these decisions against?
No, the statistical method of distinguishing is probably better, even though it too is flawed. If you take a look at the population of fund managers - this is a much smaller population - say 50,000 (which is probably an over estimate). If someone consistently beats the average, a reasonable guess would be to assume that he knows what he is doing, although that is not necessarily so.
BTW, the chance of flipping H 10 times in a row is 1 in 1024… if the population of investors in the U.S. all flipped coins, there would be tens of thousands of them that would get heads 10 time in a row…
Comment by The Skilled Investor
2 November 10, 2007, 3:50 pm o'clock |
Hi Shadox,
Thank you for your comment. You are correct. It would be difficult to implement my test, because it would require verification of explicitly stated predictions over time. Few individual investors try to do this at all, yet they listen to the predictions of others and assume others have skill. Individual investors tend to be happy, if the prices of the particular securities in their portfolios go up rather than down. They rarely do accurate performance comparisons with appropriate index benchmarks.
Another way to look at this would be to analyze the accuracy of predictions over time made by professional analysts who publish reports based on fundamental business analysis. This would have to be a statistical study, because the number of analysts who are correct on the logic of ALL their predictions would go to zero is just a quarter or two.
This all leads to statistical studies as the only solution to distinguishing luck and skill, and there are many financial academics and “buy side” money management professionals who do analyze analyst accuracy. The theory being that there are a few analysts in the pack who have a clue and most of the rest are followers who write history rather than predict with any accuracy. If you could sort out true skill from among the analyst herd, then you could make more money (in theory) by following only the skilled analysts and ignoring the rest.
The point, however, about such statistical studies of professional analysts is that it takes a very large amount of data over a very long time to “prove” statistically that a particular analyst is skilled rather than lucky. In practice, you cannot tell the difference between luck and skill — even over a long period of time. If you cannot reliably identify skill and there is an inexhaustible supply of others who claim they have it but don’t (most professional analysts, stock newsletters, drunks at parties), then is it worth paying ANY of them extra to follow their advice? I don’t think so. Studies of active management show a slight bit of skill which is swamped by all the extra costs.
To extend this a bit further, a very large amount of professional money management and trading activity is simply “machine driven.” With machine driven money management, it does not matter if a human analyst is right or wrong. You cull through huge amounts of data looking for what seem to be superior trends and trade that information. The problem of “skill and luck” is still there, but in just a different form. Is a machine trader’s strategy based on something economically real or just random patterns in the data that appear to be insightful? You can only tell AFTER you have placed your bets.
On this subject, I just read a great quote from William Sharpe in the November 2007 Stanford Business School magazine (p.25). He said, “A lot of people have a strong vested financial interest in saying ‘I know how to beat an index fund,’ and if you torture a body of data long enough it will confess to anything you want. I’d be skeptical of anyone who assumes that there is a simple formula to get something for nothing.
In summary, there is not any good way to tell investment skill from luck. On their own individual investors are completely out gunned by the industry. When they pay professionals more to get more, instead they are more likely to get less. The only smart strategy is to run as fast as you can to the very low cost end of the investment products spectrum. If you do, you end up in “passive index land” rather “active fantasy land.”
The Skilled Investor