We believe that when assessing the likelihood of uncertain events, statistically unsophisticated people utilize a coarse internal scale that only has a limited number of categories. The success of the nested sets hypothesis may lie in its ability to provide an appropriate set structure of the problem by reducing the computational demands.

We agree that current evolutionary accounts of base-rate neglect are unparsimonious, but we dispute the authors' account of the effect in terms of parallel associative and rule-based processes. We also question their assumption that cueing of nested set relations facilitates performance due to recruitment of explicit reasoning processes. In our account, such reasoning is always involved, but usually unsuccessful.

Barbey & Sloman (B&S) claim that frequency formats and other task manipulations induce people to substitute associative thinking for rule-based thinking about nested sets. My critique focuses on the substitution assumption. B&S demonstrate that nested sets are important to solve base-rate problems but they do not show that thinking about these nested sets relies on a different type of reasoning.

Are there ever good epistemic reasons to neglect base rates? Assuming an empiricist modal epistemology, I argue that we face an interesting tension between some very plausible epistemic norms: a norm requiring us to proportion our beliefs to the evidence may facilitate knowledge of the actual world, whilst inhibiting our acquisition of modal knowledge—knowledge of how things could be, but are not. The potential for this tension in our epistemic norms is a significant result in its own right. (...) It can also rationalize certain forms of demographic base-rate neglect. (shrink)

While improving the theoretical account of base-rate neglect, Barbey & Sloman's (B&S's) target article suffers from affect neglect by failing to consider the fundamental role of emotional processes in decisions. We illustrate how affective influences are fundamental to decision making, and discuss how the dual process model can be a useful framework for understanding hot and cold cognition in reasoning.

The phenomenon of base-rate neglect has elicited much debate. One arena of debate concerns how people make judgments under conditions of uncertainty. Another more controversial arena concerns human rationality. In this target article, we attempt to unpack the perspectives in the literature on both kinds of issues and evaluate their ability to explain existing data and their conceptual coherence. From this evaluation we conclude that the best account of the data should be framed in terms of a (...) dual-process model of judgment, which attributes base-rate neglect to associative judgment strategies that fail to adequately represent the set structure of the problem. Base-rate neglect is reduced when problems are presented in a format that affords accurate representation in terms of nested sets of individuals. (shrink)

A recent study showed physicians' reasoning about a realistic case to be ignorant of base rate. It also showed physicians interpreting information pertinent to base rate differently, depending on whether it was presented early or late in the case. Although these adult reasoners might do better if given hints through talk of relative frequencies, this would not prove that they had no problem of base rate neglect.

Three experiments examine the effect of base rate consistency under direct experience. Base rate consistency was manipulated by blocking trials and setting base rate choice reinforcement to be either consistent or inconsistent across trial blocks. Experiment 1 shows that, contrary to the usual finding, participants use base rate information more than individuating information when it is consistent, but less when it is inconsistent. In Experiment 2, this effect was replicated, and transferred in (...) verbal questions posed subsequently. Despite experience with consistent base rates increasing sensitivity to base rates in word problems, verbal responses were far from normative. In Experiment 3, participants’ use of base rates was once again moderated by its consistency, but this effect was itself moderated by the diagnosticity of base rate information. Participants were highly accurate in estimating experienced base rates. These studies demonstrate that base rate usage is complex and a function of how base rates are presented (experienced versus summary statistics) and response format (choice proportions versus probability estimates). Knowledge of base rates was insufficient for proper usage in verbal word problems. Although choice proportions showed a sophisticated sensitivity to experienced base rate information, participants seemed unable to demonstrate a similar sophistication when given typical word problems indicating that base rate neglect is a function of information representation and not an inherent processing bias. (shrink)

Barbey & Sloman's (B&S's) base-rate respect model is anticipated by Reyna's denominator neglect model. There are parallels at three levels: (a) explanations are grounded in a general cognitive theory (rather than in domain-specific ideas); (b) problem structure is treated as a key source of reasoning errors; and most importantly, (c) nested set relations are seen as the cause of base-rate neglect.

The commentaries indicate a general agreement that one source of reduction of base-rate neglect involves making structural relations among relevant sets transparent. There is much less agreement, however, that this entails dual systems of reasoning. In this response, we make the case for our perspective on dual systems. We compare and contrast our view to the natural frequency hypothesis as formulated in the commentaries.

Homo sapiens have evolved a dual-process cognitive architecture that is adaptive but prone to systematic errors. Fuzzy-trace theory predicts that nested or overlapping class-inclusion relations create processing interference, resulting in denominator neglect: behaving as if one ignores marginal denominators in a 2 × 2 table. Ignoring marginal denominators leads to fallacies in base-rate problems and conjunctive and disjunctive probability estimates.

Ego-justifying, group-justifying, and system-justifying motivations contribute to base-rate respect. People tend to neglect (and use) base rates when doing so allows them to draw desired conclusions about matters such as their health, the traits of their in-groups, and the fairness of the social system. Such motivations can moderate whether people rely on the rule-based versus associative strategies identified by Barbey & Sloman (B&S).

The tendency to neglect base-rates in judgment under uncertainty may be as Barbey & Sloman (B&S) suggest, but it is neither inevitable (as they document; see also Koehler 1996) nor unique. Here we would like to point out another line of evidence connecting ecological rationality to dual processes, the failure of individuals to appropriately judge cumulative probability.

Setting the two hypotheses of complete neglect and full use of base rates against each other is inappropriate. The proper question concerns the degree to which base rates are used (or neglected), and under what conditions. We outline alternative approaches and recommend regression analysis. Koehler's conclusion that we have been oversold on the base rate fallacy seems to be premature.

The no miracles argument is one of the main arguments for scientific realism. Recently it has been alleged that the no miracles argument is fundamentally flawed because it commits the base rate fallacy. The allegation is based on the idea that the appeal of the no miracles argument arises from inappropriate neglect of the base rate of approximate truth among the relevant population of theories. However, the base rate fallacy allegation relies on an (...) assumption of random sampling of individuals from the population which cannot be made in the case of the no miracles argument. Therefore the base rate fallacy objection to the no miracles argument fails. I distinguish between a “local” and a “global” form of the no miracles argument. The base rate fallacy objection has been leveled at the local version. I argue that the global argument plays a key role in supporting a base-rate-fallacy-free formulation of the local version of the argument. (shrink)

We present evidence supporting the target article's assertion that while the presentation of base-rate information in a natural frequency format can be helpful in enhancing sensitivity to base rates, method of presentation is not a panacea. Indeed, we review studies demonstrating that when subjects directly experience base rates as natural frequencies in a trial-by-trial setting, they evince large base-rate neglect.

Barbey & Sloman (B&S) attribute base-rate neglect to associative processes (like retrieval from memory) that fail to adequately represent the set structure of the problem. This commentary notes that associative responses can also lead to base-rate overweighting. We suggest that the difference between the two patterns is related to the distinction between decisions from experience and decisions from description.

Does the most common response given by participants presented with Tversky and Kahneman’s famous taxi cab problem amount to a violation of Bayes’ theorem? In other words, do they fall victim to so-called base-rate fallacy? In the present paper, following an earlier suggestion by Crupi and Girotto, we will identify the logical arguments underlying both the original diagnosis of irrationality in this reasoning task under uncertainty and a number of objections that have been raised against such a diagnosis. (...) This will enable us to show firstly that, contrary to the dismissive arguments recently put forward by Elqayam and Evans, empirical evidence turns out to be quite useful in addressing questions of this kind. Therefore, it can make a significant contribution to moving the base-rate fallacy controversy forward. Secondly, the available empirical evidence (though limited and sometimes inconclusive) seems to support the charge of irrationality levelled against the majority of participants presented with the taxi cab problem, and hence suggests that base-rate neglect is a real fallacy − that is the conclusion of our analysis. (shrink)

Is science in the midst of a crisis of replicability and false discoveries? In a recent article, Alexander Bird offers an explanation for the apparent lack of replicability in the biomedical sciences. Bird argues that the surprise at the failure to replicate biomedical research is a result of the fallacy of neglecting the base rate. The base-rate fallacy arises in situations in which one ignores the base rate—or prior probability—of an event when assessing the (...) probability of this event in the light of some observed evidence. By extension, the replication crisis would result from ignoring the low prior probability of biomedical hypotheses. In this paper, my response to Bird’s claim is twofold. First, I show that the argument according to which the replication crisis is due to the low prior of biomedical hypotheses is incomplete. Second, I claim that a simple base-rate fallacy model does not account for some important methodological insights that have emerged in discussions of the replication crisis. (shrink)

Manipulations that draw attention to extensional or set-based considerations are neither sufficient nor necessary for enhanced use of base rates in intuitive judgments. Frequency formats are only one part of the puzzle of base-rate use and neglect. The conditions under which these and other manipulations promote base-rate use may be more parsimoniously organized under the broader notion of case-based judgment.

We examine the role of need satisfaction in non-comparative justice ratings about endowments with goods. As normative approaches, we discuss utilitarianism, prioritarianism, and sufficientarianism. Using a vignette experiment, we show that a need context increases the prevalence of prioritarianistic and sufficientarianistic justice ratings, which leads to an ethically problematic sigmoid shape of the justice evaluation function.

Recent evidence suggests that people are highly efficient at detecting conflicting outputs produced by competing intuitive and analytic reasoning processes. Specifically, De Neys and Glumicic demonstrated that participants reason longer about problems that are characterized by conflict between stereotypical personality descriptions and base-rate probabilities of group membership. However, this finding comes from problems involving probabilities much more extreme than those used in traditional studies of base-rate neglect. To test the degree to which these findings depend (...) on such extreme probabilities, we varied base-rate probabilities over five experiments and compared participants’ response time for conflict problems with non-conflict problems. Longer response times for stereotypical responses to conflict versus non-conflict problems were found only in the presence of extreme probabilities. Our results suggest that humans may not be consistently efficient at detecting conflicts during reasoning. (shrink)

Koehler is right to argue for more nuanced interpretation of base rate anomalies. These anomalies are best understood in relation to a broader class of cognitive anomalies, which are important for theory and practice. Recognizing a need for more nuanced analysis should not be taken as a license for treating the effects as “explained away.”.

The concept of sustainable and responsible (SR) investments expresses that every investment should be based on the SR investor’s code of ethics. To a large extent the allocation of SR investments to more sustainable companies and ethical practices is based on the environmental, social, and corporate governance (ESG) scores provided by rating agencies. However, a thorough investigation of ESG scores is a neglected topic in the literature. This paper uses Thomson Reuters ASSET4 ESG ratings to analyze the influence of firm (...) size, a company’s available resources for providing ESG data, and the availability of a company’s ESG data on the company’s sustainability performance. We find a significant positive correlation between the stated variables, which can be explained by organizational legitimacy. The results raise the question of whether the way the ESG score measures corporate sustainability gives an advantage to larger firms with more resources while not providing SR investors with the information needed to make decisions based on their beliefs. Due to our results, SR investors and scholars should reopen the discussion about: what sustainability rating agencies measure with ESG scores, what exactly needs to be measured, and if the sustainable finance community can reach their self-imposed objectives with this measurement. (shrink)

We have been oversold on the base rate fallacy in probabilistic judgment from an empirical, normative, and methodological standpoint. At the empirical level, a thorough examination of the base rate literature (including the famous lawyer–engineer problem) does not support the conventional wisdom that people routinely ignore base rates. Quite the contrary, the literature shows that base rates are almost always used and that their degree of use depends on task structure and representation. Specifically, (...) class='Hi'>base rates play a relatively larger role in tasks where base rates are implicitly learned or can be represented in frequentist terms. Base rates are also used more when they are reliable and relatively more diagnostic than available individuating information. At the normative level, the base rate fallacy should be rejected because few tasks map unambiguously into the narrow framework that is held up as the standard of good decision making. Mechanical applications of Bayes's theorem to identify performance errors are inappropriate when (1) key assumptions of the model are either unchecked or grossly violated, and (2) no attempt is made to identify the decision maker's goals, values, and task assumptions. Methodologically, the current approach is criticized for its failure to consider how the ambiguous, unreliable, and unstable base rates of the real world are and should be used. Where decision makers' assumptions and goals vary, and where performance criteria are complex, the traditional Bayesian standard is insufficient. Even where predictive accuracy is the goal in commonly defined problems, there may be situations (e.g., informationally redundant environments) in which base rates can be ignored with impunity. A more ecologically valid research program is called for. This program should emphasize the development of prescriptive theory in rich, realistic decision environments. (shrink)

ABSTRACT Most of the world's health problems afflict poor countries and their poorest inhabitants. There are many reasons why so many people die of poverty‐related causes. One reason is that the poor cannot access many of the existing drugs and technologies they need. Another, is that little of the research and development (R&D) done on new drugs and technologies benefits the poor. There are several proposals on the table that might incentivize pharmaceutical companies to extend access to essential drugs and (...) technologies to the global poor.1 Still, the problem remains – the poor are suffering and dying from lack of access to essential medicines. So, it is worth considering a new alternative. This paper suggests rating pharmaceutical and biotechnology companies based on how some of their policies impact poor people's health. It argues that it might be possible to leverage a rating system to encourage companies to extend access to essential drugs and technologies to the poor. (shrink)

Unilateral spatial neglect is a disorder characterized by the failure to report, respond to, or orient toward the contralateral side of space to a brain lesion. Current assessment methods often fail to discover milder forms, cannot differentiate between unilateral spatial neglect subtypes and lack ecological validity. There is also a need for treatment methods that target subtypes. Immersive virtual reality systems in combination with eye-tracking have the potential to overcome these shortcomings, by providing more naturalistic environments and tasks, (...) with sensitive and detailed measures. This systematic review examines the state of the art of research on these technologies as applied in the assessment and treatment of USN. As we found no studies that combined immersive VR and ET, we reviewed these approaches individually. The review of VR included seven articles, the ET review twelve. The reviews revealed promising results. All included studies found significant group-level differences for several USN measures. In addition, several studies found asymmetric behavior in VR and ET tasks for patients who did not show signs of USN in conventional tests. Particularly promising features were multitasking in complex VR environments and detailed eye-movement analysis. No VR and only a few ET studies attempted to differentiate USN subtypes, although the technologies appeared appropriate. One ET study grouped USN participants using individual heatmaps, and another differentiated between subtypes on drawing tasks. Regarding ecological validity, although no studies tested the prognostic validity of their assessment methods, VR and ET studies utilized naturalistic tasks and stimuli reflecting everyday situations. Technological characteristics, such as the field of view and refresh rate of the head-mounted displays, could be improved, though, to improve ecological validity. We found no studies that utilized VR or ET technologies for USN treatment up until the search date of the 26th of February 2020. In conclusion, VR-ET-based systems show great potential for USN assessment. VR-ET holds great promise for treatment, for example, by monitoring behavior and adapting and tailoring to the individual person’s needs and abilities. Future research should consider developing methods for individual subtypes and differential diagnostics to inform individual treatment programs. (shrink)

Experience from a three-year Home Office funded evaluation of a project intended to reduce school exclusions is used to explore methodological dilemmas raised by the current emphasis upon 'evidence-based' policy formation. The social construction of school exclusion rates poses problems of reliability and validity, especially when such rates are simultaneously being used for target setting. In principle, the concept of 'evidence-based' can refer to a wide variety of research questions and appropriate research methodologies. Despite this, moves towards interpreting 'evidence-based' as (...) predominantly measurement and outcomes oriented can be found both in government evaluation guidelines and in procedures for systematic reviews of research. Given the complexity of educational innovations, any neglect of research into the processes of change in naturalistic settings will not only lead to a restricted awareness of a project's impact but also to a failure to understand what certain apparent outcomes actually mean. (shrink)

ABSTRACTResearch has established that human thinking is often biased by intuitive judgement. The base-rate neglect effect provides such an example, so named because people often support their decisions in stereotypical individuating information, neglecting base-rates. Here, we test the hypothesis that reasoners acknowledge information provided by base-rates and may use individuating information in support of a “rational” decision process. Results from four experiments show that “base-rate neglecting” occurs when participants acknowledge sample distributions; participants who (...) prefer individuating over base-rate information perceive base-rates as less diagnostic and are more confident in their individuating-based responses; and that posterior probabilities predict more individuating-based responses for individuating-preference participants. However, data also show a deeper form of base-rate neglect: even when some partic... (shrink)

When people are asked to estimate the probabilities of uncertain events, they often neglect the additivity principle, which requires that the probabilities assigned to an exhaustive set of outcomes should add up to 100%. Previous studies indicate that additivity neglect is dependent on response format, self-generated probability estimates being more coherent than estimates on rating scales. The present study made use of eye-tracking methodology, recording the movement, frequency and duration of fixations during the solution of ten additivity problems (...) and two control tasks. Participants produced more non-additive estimates in the Scale format than in the Self-generated format. Self-generated estimates also led to longer decision time and a higher number of repeated inspections, suggesting a deliberate comparison process. In contrast, the Scale format seemed to encourage a case-based approach where each outcome is evaluated in isolation. (shrink)

The base rate fallacy is a fallacy that occurs in probabilistic reasoning when available general information is omitted from the calculations and attention is given to specific information only. To illustrate this concept, this chapter discusses the scenario in Philip K. Dick's short story “Minority Report”, with some details augmented by the 2002 movie daptation by Steven Spielberg. The legislative approach seems to be based not just on the base rate fallacy but on assuming a (...) class='Hi'>base rate that is divorced from reality, that the drug use rate among welfare applicants is significantly higher than the national rate. Once the accurate base rate is applied (be it the proportions between the races in the US population, or the percentage of drug users among the people receiving government welfare, etc.), one can clearly see that the original conclusion does not follow. (shrink)

The important question is how people process probabilistic information, not whether they process it in accordance with a normative model that we never should have expected them to be capable of following. Experience is not the cure, as widely thought, to problems with utilizing base rate information.

The base rate literature has an opposite twin in the social psychological literature on stereotypes, which concludes that people use their preexisting beliefs about probabilistic category attributes too much, rather than not enough. This ironic discrepancy arises because beliefs about category attributes enhance accuracy when the beliefs are accurate and diminish accuracy when they are not. To determine the accuracy of base rate/stereotype beliefs requires research that addresses specific content.

Base rates have no necessary relation to judgments that are not themselves probabilities. There is no logical imperative, for instance, that behavioral base rates must affect causal attributions or that base rate information should affect judgments of legal liability. Decision theorists should be cautious in arguing that base rates place normative constraints on judgments of anything other than posterior probabilities.

In base rate problems the estimated probability must equal the base rate only where random sampling is assumed. Otherwise there is uncertainty over and above that which can be included in any probability model and inductive inference is involved. Subjects should use base rates to the extent that the problem suggests a simple random sampling model.

We investigated the influence of stimulus base rates on the Implicit Association Test . Using an East/West-German attitude-IAT, we demonstrated that both overall response speed and differential response speed underlying IAT effects depend on the relative frequencies of the stimulus categories. First, when those stimuli that are more common in reality also occurred more frequently in the stimulus list, response speed generally increased. Second, IAT effects increased when congruent blocks profited from the compatibility of frequency-based response biases , whereas (...) IAT effects decreased when incongruent trial blocks profited from response compatibility. These findings demonstrate that the stimulus context moderates the magnitude of the IAT effect. Simultaneously, they highlight the need to explore the extent to which implicit measures reflect properties of the task or the environment rather than attributes of test-takers. (shrink)

The base rate fallacy is directly dependent on a particular judgment paradigm in which information may be unambiguously designated as either “base rate” or “individuating,” and in which subjects make two-stage sequential judgments. The paradigm may be a poor match for real world settings, and the fallacy may thus be undefined for natural ecologies of judgment.

There is a theorem that shows that it is impossible for an algorithm to jointly satisfy the statistical fairness criteria of Calibration and Equalised Odds non-trivially. But what about the recently advocated alternative to Calibration, Base Rate Tracking? Here, we show that Base Rate Tracking is strictly weaker than Calibration, and then take up the question of whether it is possible to jointly satisfy Base Rate Tracking and Equalised Odds in non-trivial scenarios. We show (...) that it is not, thereby establishing an even more general impossibility theorem. (shrink)

The no-miracles argument and the pessimistic induction are arguably the main considerations for and against scientific realism. Recently these arguments have been accused of embodying a familiar, seductive fallacy. In each case, we are tricked by a base rate fallacy, one much-discussed in the psychological literature. In this paper we consider this accusation and use it as an explanation for why the two most prominent `wholesale' arguments in the literature seem irresolvable. Framed probabilistically, we can see very clearly (...) why realists and anti-realists have been talking past one another. We then formulate a dilemma for advocates of either argument, answer potential objections to our criticism, discuss what remains (if anything) of these two major arguments, and then speculate about a future philosophy of science freed from these two arguments. In so doing, we connect the point about base rates to the wholesale/retail distinction; we believe it hints at an answer of how to distinguish profitable from unprofitable realism debates. In short, we offer a probabilistic analysis of the feeling of ennui afflicting contemporary philosophy of science. (shrink)

Decision-makers in institutional and non-institutional contexts are sometimes confronted with the issue of whether to use generalisations expressing the statistical incidence of a negative trait in a disadvantaged and discriminated-against social group in order to draw an inference concerning a member of that group. If a criminal court were confronted with such a question, what answer should it give? First, the article argues that, our qualms notwithstanding, morality does not demand that these generalisations be disregarded. In doing so, the article (...) addresses the relationship between factual accuracy and the demands of morality in criminal trials. Second, the article considers the implications of this conclusion for the legal question as to whether the evidence at issue should be excluded, in particular, on grounds of unfairness — pursuant to section 78 of the Police and Criminal Evidence Act 1984. (shrink)

Koehler's call for a reanalysis of the base rate fallacy is particularly important in the applied domain of accounting, since base rate data appear to be an important input for many accounting tasks. In this commentary I discuss the use of base rates in accounting and explain why more flexible standards of performance are important when judging the use of base rates.

Howson famously argues that the no-miracles argument, stating that the success of science indicates the approximate truth of scientific theories, is a base rate fallacy: it neglects the possibility of an overall low rate of true scientific theories. Recently a number of authors has suggested that the corresponding probabilistic reconstruction is unjust, as it concerns only the success of one isolated theory. Dawid and Hartmann, in particular, suggest to use the frequency of success in some field of (...) research \ to infer a probability of truth for a new theory from \. I here shed doubts on the justification of this and similar moves and suggest a way to directly bound the probability of truth. As I will demonstrate, my bound can become incompatible with the assumption specific testing and Dawid and Hartmann’s estimate for success. (shrink)