Chance creates the illusion that investors can beat the stock market

"Market efficiency" makes it very difficult for individual investors to "beat the market." 

Making their own decisions, individual investors perform so poorly that on average their investment returns lag behind the returns that one would expect from a completely random stock selection process. 

The average professional trader does somewhat better than amateurs do, and professionals probably do so, in part, at the expense of the amateurs. However, on average, any actual performance advantage delivered by professionals is significantly less than the average fees they charge for their services. (See: The illusion of superior professional investment manager performance)

Securities markets in modern industrialized countries tend to price traded assets relatively close to their risk-adjusted expected values. When market prices tend to reflect fully all available information about a security, then those markets are considered "efficient." This is particularly true in the United States with its broad, real-time, and relatively transparent or relatively honest financial markets.

Of course, the media has been filled with stories of industry misconduct and fraud since the market crash. Nevertheless, U.S. markets still do a reasonably good job of providing a highly liquid forum where knowledgeable and willing buyers and sellers can trade at arms length at prices that are determined by supply and demand rather than manipulation.

This relative "market efficiency" makes it very difficult for individual investors to "beat the market." In fact, academic studies have shown that, left to their own decisions, individual investors perform so poorly that on average their investment returns lag behind the returns that one would expect from a completely random stock selection process. (See: What is the cost to individual investors of sub-optimal portfolio diversification?)

When higher investment costs, increased taxation, and the value of one’s personal time are considered, the efforts of amateur individual investors to "beat the market" become even more clearly uneconomical. 

Nevertheless, the illusion that one can beat the market persists very widely. The securities industry keeps telling them it is possible. Brokers push individual security recommendations and individuals buy them hoping to outperform the market.  (See: Can you really beat the securities markets?)

Rarely are individual investors sufficiently disciplined to compare their personal portfolio performance against an appropriate market index benchmark. Instead, as a herd, they to move their money into the most recent hot sector, after securities prices have already run up. Many do not really understand their personal performance and substitute costly activism for a more passive approach.

The average professional trader does somewhat better than amateurs do and, in part, probably does so at the expense of the amateurs. Professionals seem able to capture some of a very slight persistence that equity price movements demonstrate over time, while amateurs are ill equipped and insufficiently capitalized to do the same.

The elusiveness of the very slight price persistence demonstrated by securities cannot be overstated. 

Professional money managers are more disciplined and scientific about portfolio management, and their methods allow them to capture some of the persistence in price movements from the securities in portfolios that they already hold. Conversely, amateurs tend to sell winners too quickly and hold on to losing securities too long.

It is noteworthy, however, that professionals seem unable to select securities in the first place that will prove to have persistence in their price movements. Professionals just seem better able to manage the situation after they happen to have acquired securities that do exhibit price persistence. Of course, professionals will claim their performance is the result of skill and not luck, but academics have simply not found such skill to be borne out in the manager performance numbers.  (See: Distinguishing between true investment skill and luck)

Could individuals invest in a fund to share some of these small excess returns with professionals who capture them? The answer unfortunately is no. On average, any actual performance advantage delivered by professionals is significantly less than the average fees they charge for their services.