Some years ago Daniel Kahneman and Gary Klein wrote a paper titled – Conditions for intuitive expertise: a failure to disagree, in which they explored the differences between two approaches to intuition and expertise that are often viewed as conflicting: Heuristics and Biases (HB) and Naturalistic Decision Making (NDM).
Starting from the obvious fact that professional intuition is sometimes marvelous and sometimes flawed, the authors attempted to map the boundary conditions that separate true intuitive skill from overconfident and biased impressions.
They concluded that evaluating the likely quality of an intuitive judgment requires an assessment of the predictability of the environment in which the judgment is made and of the individual’s opportunity to learn the regularities of that environment. Subjective experience is not a reliable indicator of judgment accuracy.
What is notable is that finance professionals, psychotherapists, and intelligence analysts are more conducive to overconfidence than weather forecasters, engineers, and logistics specialists because the later group who are marked by standard methods, clear feedback, and direct consequences for error, appear to appreciate the boundaries of their expertise. Here’s an excerpt from the paper:
DK [Daniel Kahneman] is particularly interested in cases in which professionals who know how to use their knowledge for some purposes attempt to use the same knowledge for other purposes. He views the fractionation of expertise as one element in the explanation of the illusion of validity: the overconfidence that professionals sometimes experience in dealing with problems in which they have little or no skill.
Finance professionals, psychotherapists, and intelligence analysts may know a great deal about a particular company, patient, or international conflict, and they may have received ample feedback supporting their confidence in the performance of some tasks—typically those that deal with the short term— but the feedback they receive from their failures in long-term judgments is delayed, sparse, and ambiguous. The experience of the professionals that DK has thought about is therefore conducive to overconfidence.
These professionals may have strong subjective confidence in their judgments, but we do not believe that subjective confidence reliably indicates whether intuitive judgments or decisions are valid. When experts recognize anomalies, using judgments of typicality and familiarity, they are detecting violations of patterns in the external situation.
In contrast, people do not have a strong ability to distinguish correct intuitions from faulty ones. People, even experts, do not appear to be skilled in detecting patterns in the internal situation in order to identify the basis for their judgments. Therefore, reliance on subjective confidence may contribute to overconfidence.
The experts that GK (Gary Klein) has studied seem less susceptible to overconfidence, perhaps in part because of the direct personal risks it poses. Weather forecasters, engineers, and logistics specialists typically resist requests to make judgments about matters that fall outside their area of competence. People in professions marked by standard methods, clear feedback, and direct consequences for error appear to appreciate the boundaries of their expertise.
These experts know more knowledgeable experts exist. Weather forecasters know there are people in another location who better understand the local dynamics. Structural engineers know that chemical engineers, or even structural engineers working with different types of models or materials, are the true experts who should be consulted.
As in the other topics that we have considered, we find no reason to disagree about either fractionation of expertise or overconfidence. As usual, different rules apply to different tasks.
—
We describe task environments as “high-validity” if there are stable relationships between objectively identifiable cues and subsequent events or between cues and the outcomes of possible actions. Medicine and firefighting are practiced in environments of fairly high validity. In contrast, outcomes are effectively unpredictable in zero-validity environments. To a good approximation, predictions of the future value of individual stocks and long-term forecasts of political events are made in a zero-validity environment.
The situation that we have labeled fractionation of skill is another source of overconfidence. Professionals who have expertise in some tasks are sometimes called upon to make judgments in areas in which they have no real skill. (For example, financial analysts may be skilled at evaluating the likely commercial success of a firm, but this skill does not extend to the judgment of whether the stock of that firm is underpriced.) It is difficult both for the professionals and for those who observe them to determine the boundaries of their true expertise.
You can read the entire paper here:
For all the latest news and podcasts, join our free newsletter here.
Don’t forget to check out our FREE Large Cap 1000 – Stock Screener, here at The Acquirer’s Multiple:
