We have been oversold on the base rate fallacy in probabilistic judgment from an empirical, normative, and methodological standpoint. At the empirical level, a thorough examination of the base rate literature (including the famous lawyer–engineer problem) does not support the conventional wisdom that people routinely ignore base rates. Quite the contrary, the literature shows that base rates are almost always used and that their degree of use depends on task structure and representation. Specifically, base rates play a relatively larger role (...) in tasks where base rates are implicitly learned or can be represented in frequentist terms. Base rates are also used more when they are reliable and relatively more diagnostic than available individuating information. At the normative level, the base rate fallacy should be rejected because few tasks map unambiguously into the narrow framework that is held up as the standard of good decision making. Mechanical applications of Bayes's theorem to identify performance errors are inappropriate when (1) key assumptions of the model are either unchecked or grossly violated, and (2) no attempt is made to identify the decision maker's goals, values, and task assumptions. Methodologically, the current approach is criticized for its failure to consider how the ambiguous, unreliable, and unstable base rates of the real world are and should be used. Where decision makers' assumptions and goals vary, and where performance criteria are complex, the traditional Bayesian standard is insufficient. Even where predictive accuracy is the goal in commonly defined problems, there may be situations (e.g., informationally redundant environments) in which base rates can be ignored with impunity. A more ecologically valid research program is called for. This program should emphasize the development of prescriptive theory in rich, realistic decision environments. (shrink)

Base rates have no necessary relation to judgments that are not themselves probabilities. There is no logical imperative, for instance, that behavioral base rates must affect causal attributions or that base rate information should affect judgments of legal liability. Decision theorists should be cautious in arguing that base rates place normative constraints on judgments of anything other than posterior probabilities.

Abstract(1) The miscitations of seminal experiments in the base rate literature adds to the existing database of systematic miscitations of wellknown psychological experiments. These miscitations may be caused by a process of reconstructive remembering. (2) Representative design should be the methodological core of Koehler's call for ecologically valid research. This approach can benefit both basic and applied research.

Koehler's summary and critique of research on the base rate fallacy is cogent and persuasive. However, he may have overstated the case, and his suggestions for future research may be too restrictive. We agree that methodological approaches to this topic should be broadened, but we argue that experimental laboratory research and the Bayesian normative standard are useful and should not be abandoned.

This commentary presents a self-assessment inventory that will allow readers to determine their own attitude toward the base rate fallacy and its literature. The inventory is scientifically valid but not Medicare/Medicaid reimbursable.

When base rates are learned and used in an experiential manner subjects show better base rate use, perhaps because the implicit learning system is engaged. A causal framework in which base rates are relevant might also be necessary. Humans might thus perform better on more ecologically valid tasks, which are likely to contain those three components.

Base rates are vital in predicting violent criminal recidivism. However, both lay people given simulated prediction tasks and professionals milking real life predictions appear insensitive to variations in the base rate of violent recidivism. Although there are techniques to help decision makers attend to base rates, increased decision accuracy is better sought in improved actuarial models as opposed to improved clinicians.

Consistent with Koehler's position, we propose a generalization of the base rate fallacy and earlier conservatism literatures. In studies using both traditional tasks and new tasks based on ecologically valid base rates, our subjects typically underweight individuating information at least as much as they underweight base rates. The implications of cue consistency for averaging heuristics are discussed.

Koehler is right that base rate information is used, to various degrees, both in laboratory tasks and in everyday life. However, it is not time to turn our backs on laboratory tasks and focus solely on ecologically valid decision making. Tightly controlled experimental data are still needed to understandhowbase rate information is used, and how this varies among groups.

Any instance (i.e., event, behavior, trait) belongs to infinitely many reference classes, hence there are infinitely many base rates from which to choose. People clearly do not entertain all possible reference classes, however, so something must be limiting the search space. We suggest some possible mechanisms that determine which reference class is evoked for the purpose of judgment and decision.

The fallacy beneath the base rate fallacy is that we know what a base rate is. We talk as if base rates and individuating information were two different kinds of information. From a Bayesian perspective, however, the only difference between base rate and individuating information is – which comes first.

Koehler is right to argue for more nuanced interpretation of base rate anomalies. These anomalies are best understood in relation to a broader class of cognitive anomalies, which are important for theory and practice. Recognizing a need for more nuanced analysis should not be taken as a license for treating the effects as “explained away.”.

Koehler's target article attempts a balanced view of the relevance of knowledge of base rates to judgments of subjective or credal probability, but he is not sensitive enough to the difference between requiring and permitting the equation of probability judgments with base rates, the interaction between precision of base rate and reference class information, and the possibility of indeterminate probability judgment.

Two distinct issues are sometimes confused in the base rate literature: Why do people make logical mistakes in the assessment of probabilities? and why do subjects not use base rates the way experimenters do? The latter problem may often reflect differences in an implicit reference class rather than a disinclination to update a base rate by Bayes' theorem. Also important are considerations concerning the interaction of several potentially relevant base rates.

The underutilization of base rates is a consistent finding. The strong claim that base rates are ignored has been rejected and this needs no further emphasis. Following the path of “normal science,” research examines the conditions predicting changes in the degree of underutilization. A scientific revolution that might dethrone the heuristics and biases paradigm is not in sight.

Koehler's call for a reanalysis of the base rate fallacy is particularly important in the applied domain of accounting, since base rate data appear to be an important input for many accounting tasks. In this commentary I discuss the use of base rates in accounting and explain why more flexible standards of performance are important when judging the use of base rates.

This commentary discusses three points: (1) The implications of the fact that it is rational to ignore base rates if probabilities are estimated by frequencies from samples without missing data (natural sampling); (2) second order probabilities distributions are a plausible way to model imprecise probabilities; and (3) Bayesian networks represent a normative reference for multi-cue models of probabilistic inference.

Failure to consider base rate is regarded as potentially hazardous, mainly because its consideration is assumed to be determined solely by P(H/D), the probability of the individuating data if the hypothesis is true, and not at all by P(D/˜H), the probability if the hypothesis is false. However, when P(D/˜H) is unconfounded from P(D/H), it turns out to be the stronger determinant of base rate consideration.

Setting the two hypotheses of complete neglect and full use of base rates against each other is inappropriate. The proper question concerns the degree to which base rates are used (or neglected), and under what conditions. We outline alternative approaches and recommend regression analysis. Koehler's conclusion that we have been oversold on the base rate fallacy seems to be premature.

A recent study showed physicians' reasoning about a realistic case to be ignorant of base rate. It also showed physicians interpreting information pertinent to base rate differently, depending on whether it was presented early or late in the case. Although these adult reasoners might do better if given hints through talk of relative frequencies, this would not prove that they had no problem of base rate neglect.

The environment in which humans evolved is strongly and positively autocorrelated in space and time. Probabilistic judgments based on the assumption of independence may not yield evolutionarily adaptive behavior. A number of “faults” of human reasoning are not faulty under fuzzy arithmetic, a nonprobabilistic calculus of reasoning under uncertainty that may be closer to that underlying human decision making.

In contrast to traditional research on base-rate neglect, an ecologically-oriented research program would analyze the correspondence between cognitive algorithms and the nature of information in the environment. Bayesian computations turn out to be simpler when information is represented in frequency formats as opposed to the probability formats used in previous research. Frequency formats often enable even uninstructed subjects to perform Bayesian reasoning.

The base rate literature has an opposite twin in the social psychological literature on stereotypes, which concludes that people use their preexisting beliefs about probabilistic category attributes too much, rather than not enough. This ironic discrepancy arises because beliefs about category attributes enhance accuracy when the beliefs are accurate and diminish accuracy when they are not. To determine the accuracy of base rate/stereotype beliefs requires research that addresses specific content.

Abstract(1) There is enough contradictory evidence regarding the role of base rates in category learning to confirm the nonexistence of biases in such learning. (2) It is not always possible to activate statistical reasoning through frequentist representation. (3) It is necessary to use the concept of systematic processing in reconsidering the published work on biases.

The important question is how people process probabilistic information, not whether they process it in accordance with a normative model that we never should have expected them to be capable of following. Experience is not the cure, as widely thought, to problems with utilizing base rate information.

As illustrated by research Koehler himself cites (Dawes et al. 1993), the purpose of experiments is to choose between contrasting explanations of past observations – rather than to seek statistical generalizations about the prevalence of effects. True external validity results not from sampling various problems that are representative of “real world” decision making, but from reproducing an effect in the laboratory with minimal contamination (including from real world factors).

The base rate fallacy is directly dependent on a particular judgment paradigm in which information may be unambiguously designated as either “base rate” or “individuating,” and in which subjects make two-stage sequential judgments. The paradigm may be a poor match for real world settings, and the fallacy may thus be undefined for natural ecologies of judgment.

There is a familiar risk of antinomy if fromxisEand p(xisH/xisE) =rit is permissible to infer p(xisH) =r, and what Carnap (1950) called “The requirement of total evidence” will not prevent such antinomies satisfactorily. What is needed instead is a properly developed theory of evidential weight.

Koehler's work will assist the effort to understand legal fact finding. It leaves two questions somewhat open: (i) the extent to which empirical research can measure correctness of fact-finding, a function that involves the resolution of normative questions and (ii) the standards judges should use in the absence of the research he advocates.

Cognitive algebra strongly disproved the representativeness heuristic almost before it was published; and therewith it also disproved the base rate fallacy. Cognitive algebra provides a theoretical foundation for judgment-decision theory through its joint solution to the two fundamental problems – true measurement of subjective values, and cognitive rules for integration of multiple determinants.

Koriat & Goldsmith's distinction between the correspondence and storehouse metaphors is valuable for both memory theory and methodology. It is questionable, however, whether this distinction underlies the heated debate about so called “everyday memory” research. The distinction between experimental and naturalistic methodologies better characterizes this debate. I compare these distinctions and discuss how the methodological distinction, between experimental and naturalistic designs, could give rise to different theoretical approaches.

Koriat & Goldsmith provide an excellent analysis of the flexibility of retrieval processes and how they are situationally dependent. I agree with their emphasis on functional considerations and argue that the traditional laboratory experiment motivates the subject to be accurate. However, I disagree with their strong claim that the quantity–accuracy distinction implies an essential discontinuity between traditional and naturalistic approaches to the study of memory.

The correspondence view is consistent with a theory of direct remembering that assumes continuity between perception and memory. Two implications of direct remembering for correspondence are suggested. It is assumed that forgetting is exponential, and that remembering at one time is independent of factors influencing remembering at another. Elaboration of the correspondence view in the same terms as perception offers a novel approach to the study of memory.

This article is concerned with how we make decisions based on how problems are presented to us and the effect that the framing of the problem might have on our choices. Current philosophical and psychological accounts of the framing effect in experiments such as the Asian Disease Problem concern reference points and domains. We question the importance of reference points and domains. Instead, we adopt a linguistic perspective focussing on the role of the evaluative polarity evoked by the words - (...) negativeand positive - used to describe the options in the decision problem. We show that the evaluative polarity of the different wordings in the ADP better explain participants’ behaviour than reference points and domains. We propose two models in which the values given to evaluative polarity words directly influence the strength of framing. The results indicate that linguistic considerations regarding evaluative polarity have to be considered in relation to the ADP. The account resembles Fuzzy-Trace-Theory but allows for thestrength of evaluative polarity to directly affect behaviour. In the discussion, we also assess how evaluative polarity relates to negation, antonyms and the communicative frame within which the choices are presented. (shrink)

If reason is a real causal force,operative in some, but not all ofour cognition and conation, then itought to be possible to tell anaturalistic story that distinguishes themind which is moved byreason from the mind which is movedby forces other than reason.This essay proposes some steps towardthat end. I proceed by showingthat it is possible to reconcile certainemerging psychological ideasabout the causal powers of themind/brain with a venerablephilosophical vision of reason as the facultyof norms. My accountof reason is psychologistic, social, (...) and consistent with anevolutionary approach to mind. The account preserves thenormativity by deflating it. But I argue that onlysuch deflated normativity has any chance of beingmade naturalistically respectable. (shrink)

This work examines the influence of reference dependence, including value selection bias and congruence effects, on diagnostic reasoning. Across two studies, we explored how dependence on the initial problem structure influences the ability to solve simplified precursors to the more traditional Bayesian reasoning problems. Analyses evaluated accuracy and types of response errors as a function of congruence between the problem presentation and question of interest, amount of information, need for computation, and individual differences in numerical abilities. Across all problem variations, (...) there was consistent and strong evidence of a value selection bias in that incorrect responses almost always conformed to values that were provided in the problem rather than other errors including those related to computation. The most consistent and unexpected error across all conditions in the first experiment was that people were often more likely to utilize the superordinate value as part of their solution rather than the anticipated reference class values. This resulted in a weakened effect of congruence, with relatively low accuracy even in congruent conditions, and a dominant response error of the superordinate value. Experiment 2 confirmed that the introduction of a new sample drew attention away from the provided reference class, increasing reliance on the overall sample size. This superordinate preference error, along with the benefit of repeating the PPV reference class within the question, demonstrated the importance of reference dependence based on the salience of information within the response prompt. Throughout, higher numerical skills were generally associated with higher accuracy, whether calculations were required or not. (shrink)

The relatively new discipline of risk analysis promises to provide objective guidance in some of the most controversial issues in modern high‐technology societies. Four conditions are discussed that must be satisfied for this promise to be fulfilled. Since none of these conditions is satisfied, risk analysis does not keep its promise. In its attempts to reduce genuinely political issues to technocratic calculations, it neglects many of the factors that should influence decisions on risk acceptance. A list of tentative guidelines is (...) given for numerical decision support that encourages democratic decision‐making instead of trying to evade it. (shrink)

Decision-making competence in later adulthood is affected by declines in cognitive skills, and age-related changes in affect and experience can sometimes compensate. However, recent findings suggest that age-related changes in motivation also affect the extent to which adults draw from experience, affect, and deliberative skills when making decisions. To date, relatively little attention has been given to strategies for addressing age-related changes in motivation to promote better decisions in older adults. To address this limitation, we draw from diverse literatures to (...) suggest promising intervention strategies for motivating older recipients’ motivation to make better decisions. We start by reviewing the life-span developmental literature, which suggests that older adults’ motivation to put effort into decisions depends on the perceived personal relevance of decisions as well as their self-efficacy (i.e., confidence in applying their ability and knowledge). Next, we discuss two approaches from the health intervention design literature, the mental models approach and the patient activation approach, which aim to improve motivation for decision making by improving personal relevance or by building self-efficacy or confidence to use new information and skills. Using examples from these literatures, we discuss how to construct interventions to motivate good decisions in later adulthood. (shrink)

Operational psychology is an emerging subdiscipline that has enhanced the U.S. military's combat capabilities during the Global War on Terrorism. What makes this subdiscipline unique is its use of psychological principles and skills to improve a commander's decision making as it pertains to conducting combat (or related operations). Due to psychology's expanding role in combat support, psychologists are being confronted with challenges that require the application of their professional ethics in areas in which little if any guidance has been provided. (...) Operational psychologists are at the forefront of this expansion. Accordingly, they need a decision model to assist them in this complex dynamic environment. To this end, this article reviews various decision models and ethical frameworks, selects the most appropriate, and then applies it to the challenges faced by operational psychologists. A naturalistic decision model that integrates rational and intuitive elements is recommended. (shrink)

In this response, we clarify several misunderstandings of the understanding/acceptance principle and defend our specific operationalization of that principle. We reiterate the importance of addressing the problem of rational task construal and we elaborate the notion of computational limitations contained in our target article. Our concept of thinking dispositions as variable intentional-level styles of epistemic and behavioral regulation is explained, as is its relation to the rationality debate. Many of the suggestions of the commentators for elaborating two-process models are easily (...) integrated into our generic dual-process account. We further explicate how we view the relation between System 1 and System 2 and evolutionary and normative rationality. We clarify our attempt to fuse the contributions of the cognitive ecologists with the insights of the original heuristics and biases researchers. (shrink)

Classical antiquity provides not just the storehouse metaphor, which postdates Plato, but also parts of the correspondence metaphor. In the fifth century B.C., Thucydides (1.22) considered the role of gist and accuracy in writing history, and Aristotle (Poetics1451b, 1460b 8–11) offered an explanation. Finally, the Greek for truth (alêtheia) means “that which is not forgotten.”.

In two experiments, we investigated the relationship between confidence and accuracy in syllogistic reasoning. Participants judged the validity of conclusions and provided confidence ratings twice for each problem: once quickly and again after further deliberation. Correlations between confidence and accuracy were small or nonexistent. In addition, confidence and accuracy were mediated by different variables. Confidence judgments appeared to reflect external cues, so that confidence was greater when the participants were allowed additional time to think about the problem, as well as (...) when the conclusion was either believable or unbelievable, rather than neutral. In contrast, accuracy changed little as a function of the amount of time available and did not differ for believable and neutral problems. These data support a model in which initial decisions are made quickly, on the basis of heuristic cues, and analytic processes are used to justify or rationalize the earlier decision. (shrink)

Previous studies have shown that people often use heuristics in making inferences and that subjective memory experiences, such as recognition or familiarity of objects, can be valid cues for inferences. So far, many researchers have used the binary choice task in which two objects are presented as alternatives (e.g., “Which city has the larger population, city A or city B?”). However, objects can be presented not only as alternatives but also in a question (e.g., “Which country is city X in, (...) country A or country B?”). In such a situation, people can make inferences based on the relationship between the object in the question and each object given as an alternative. In the present study, we call this type of task a “relationships‐comparison task.” We modeled the three inference strategies that people could apply to solve it (familiarity‐matching [FM; a new heuristic we propose in this study], familiarity heuristic [FH], and knowledge‐based inference [KI]) to examine people's inference processes. Through Studies 1, 2, and 3, we found that (a) people tended to rely on heuristics, and that FM (inferences based on similarity in familiarity between objects) well explained participants' inference patterns; (b) FM could work as an ecologically rational strategy for the relationships–comparison task since it could effectively reflect environmental structures, and that the use of FM could be highly replicable and robust; and (c) people could sometimes use a decision strategy like FM, even in their daily lives (consumer behaviors). The nature of the relationships–comparison task and human cognitive processes is discussed. (shrink)

Both the fast and frugal heuristics and the naturalistic decision making research programmes have identified important areas of inquiry previously neglected in the traditional study of human judgment and decision making, and have greatly contributed to the understanding of people's real-world decision making under environmental constraints. The two programmes share similar theoretical arguments regarding the rationality, optimality, and role of experience in decision making. Their commonalities have made them appealing to each other, and efforts have been made, by their leading (...) academics, to promote synergy and integration. However, there has been little progress towards this during the last decade. This paper seeks to address this gap by seeking to better understand their commonalities and differences. To do so, literature relating to the two programmes is reviewed. The findings of the review indicated that an integration of the two could enhance FFHs' field research in applied settings, facilitate its investigation on boundary conditions of people's decision strategy selection, enable NDM to embrace emerging research opportunities in the age of big data, as well as permit each programme to enlighten the research topics and to validate the research findings of the other. (shrink)

The correspondence metaphor is useful in developing functional models of memory. However, the storehouse metaphor is still useful in developing structural and process models of memory. Traditional research techniques explore the structure of memory; everyday techniques explore the function of memory. We illustrate this point with two examples: amnesia and metamemory. In each phenomenon, both metaphors are useful.

Samuels and Stich explore the debate over the extent to which ordinary human reasoning and decision making is rational. One prominent cluster of views, often associated with the heuristics and biases tradition in psychology, maintains that human reasoning is, in important respects, normatively problematic or irrational. Samuels and Stich start by sketching some key experimental findings from this tradition and describe a range of pessimistic claims about the rationality of ordinary people that these and related findings are sometimes taken to (...) support. Such pessimistic interpretations of the experimental findings have not gone unchallenged however: Samuels and Stich outline some of the research on reasoning that has been done by evolutionary psychologists and sketch a cluster of more optimistic theses about ordinary reasoning that such psychologists defend. Although Samuels and Stich think that the most dire pronouncements made by writers in the heuristics and biases tradition are unwarranted, they also maintain that the situation is rather more pessimistic than sometimes suggested by evolutionary psychologists. They conclude by defending this “middle way” and sketch a family of “dual processing” theories of reasoning which, they argue, offer some support for the moderate interpretation they advocate. (shrink)