How does the illusion of validity make you overconfident in your ability to predict the future? - Chapter 20

System 1 is known for jumping to conclusions from limited evidence (WYSIATI). The coherence of the story created by System 1 and System 2 makes us confident about our opinions. The quality and amount of the evidence are less important, because poor evidence created a good story. We even have beliefs without any evidence for them, we just believe something because our partner or friend does. It is ridiculous how confident we are in our beliefs when we know so little.

The illusion of validity can be explained by the following experiment. Two psychology students watched soldiers in officer-training trying to solve a problem. They decided to evaluate them: who would make a great army leader? The exercise was exhausting and required several attempts. Some soldiers seemed arrogant, persistent, submissive, hot-tempered, stubborn, patient or quitters. Some were expected to become great leaders, others were ruled out for the officer rank. The students were convinced the exercise revealed the true nature of the soldiers and were very confident in their evaluations. Their impressions were coherent and did not conflict. After some time, they learned how the soldiers were doing and compared their evaluations against the reports by their commanders. This showed that the predictions about their performance at the officer-training were hardly any better than blind guesses. The students decided to repeat the experiment with a new group of soldiers. The shocking truth about the quality of their previous predictions had no effect on how they evaluated the soldiers and very little effect on how confident they felt in their judgments and predictions. The evidence of their prior failure did not damage their confidence and did not cause a moderation of the predictions, which it should have. They still felt like their predictions were valid. Another example of the illusion of validity is the Müller-Lyer illusion.

The predictions of the future performances of the soldiers are instances of the representativeness heuristic and of substitution. The predictions were based on 60 minutes of the soldiers behavior in artificial circumstances. The predictions were totally nonregressive and supported by weak evidence (WYSIATI). The experiment demonstrates that confidence reflects the coherence of the information and the cognitive ease of processing the information. Remember that a very confident person has formed a coherent story in his mind, which does not necessarily mean it’s the truth.

What is the illusion of stock-picking skill?

Each day, billions of shares are traded, which involves many buyers and sellers. They have the same information, their exchange is based on a difference in opinions. A buyer thinks the price will rise, while the seller expects it to drop. Something makes them believe they know more about the future price than others, but for many, that belief is merely an illusion. The price of stock is based on the available information about the company value and the best prediction about the future of the stock. If someone believes the price will rise, he will buy the stock, which causes the price to rise.

A lot of individual investors suffer constant trading losses. 10.000 investors who sold stock and immediately bought new stock were part of study. They expected that the stock that was bought would do better than the stock that was sold. The findings showed that, on average, the shares that were sold did much better than the bought ones. A later study showed that the most active investors gained the lowest returns and the least trading investors had the best results. Other research indicates that men acted on their bad predictions more often than women, which is why women had better investments outcomes. On the other side of a transaction are professional investors and financial institutions, waiting for individual investors to make a mistake. They tend to sell stock that got more value (winners) and hang on to stock that dropped in price (losers). In the short run, winners do better than losers, so they sell and buy the wrong stock. Individual investors are more influenced by companies being in the news than professional investors. Only a few stock pickers have the skill to beat the market repeatedly. Even professional investors are no persistent achievers. The persistence of individual differences in achievements is proof of having a skill. Research shows that most fund managers select stock like rolling dice: they play a game of chance, not of skill. A fund having a good year is mostly due to luck.

Visual illusions tend to be less stubborn than cognitive illusions. The Müller-Lyer illusion changed your behavior, but not how you see the lines. You know you cannot trust what you saw. Investors who are told good outcomes are the result of luck and skill still believe they are doing better than the market, despite the statistical facts proving otherwise. They accept the information intellectually, but it has no effect on their feelings. The illusion of skill is persistent in the financial world, but why? Stock pickers are highly skilled when it comes to consulting data, examining balance sheets and assessing the competition. Their work requires a lot of training and experience in using these skills. However, they lack the skill of knowing whether the information about a company is already incorporated in the price of their shares and seem unaware of their ignorance. Subjective confidence is a feature of System 1. Moreover, the illusions of skill and validity are supported by the powerful professional culture of the financial community. A lot of members believe they can do something that someone else cannot.

It is for people hard to accept that the future cannot be predicted, due to the ease with which they can explain the past. In hindsight, many things make sense. This leads to the intuition that something that makes sense in hindsight today could be predicted yesterday. The illusion that we understand the past makes us overconfident in our ability to predict the future. We think the past can be explained by focusing on the abilities and intentions of a few great leaders, social movements, or technological and cultural developments. We cannot believe that big historical events are determined by luck.

The illusion of valid prediction gets exploited by pundits in politics, business, media or the financial world. Newspapers and television stations hire experts to evaluate the past and predict the future. Readers/viewers think they receive insightful information, which the experts think they are offering. Expert predictions about economic and political trends were the subject of a landmark study by psychologist Tetlock. He collected over 80.000 predictions and the outcome was shocking: they performed worse than blindfolded golfers. Even in the field of their expertise, they did not better than non-experts. People with the most knowledge are frequently less reliable, because they develop an illusion of their skill and become overly confident. The more famous the expert is, the more overconfident he is and the more outrageous his predictions are. The study also showed how these experts are less willing to admit they had been wrong and came with several excuses.

The lessons to be learned from this chapter is firstly that predictions error are inevitable, because our world is unpredictable. Secondly, high subjective confidence is not trustworthy as an indicator of accuracy. Short-term trends can be predicted, and achievements and behavior can be forecast from previous achievements and behaviors, but you should not rely on long-term predictions made by pundits.