For more than 20 years, research has proven the beneficial effect of natural frequencies when it comes to solving Bayesian reasoning tasks (Gigerenzer & Hoffrage, 1995). In a recent meta-analysis, McDowell & Jacobs (2017) showed that presenting a task in natural frequency format increases performance rates to 24% compared to only 4% when the same task is presented in probability format. Nevertheless, on average three quarters of participants in their meta-analysis failed to obtain the correct solution for such a task (...) in frequency format. In this paper, we present an empirical study on what participants typically do wrong when confronted with natural frequencies. We found that many of them did not actually use natural frequencies for their calculations, but translated them back into complicated probabilities instead. This switch from the intuitive presentation format to a less intuitive calculation format will be discussed within the framework of psychological theories (e.g., the Einstellung effect). (shrink)

Theoretical arguments and empirical investigations indicate that a high proportion of published findings do not replicate and are likely false. The current position paper provides a broad perspective on scientific error, which may lead to replication failures. This broad perspective focuses on reform history and on opportunities for future reform. We organize our perspective along four main themes: institutional reform, methodological reform, statistical reform and publishing reform. For each theme, we illustrate potential errors by narrating the story of a fictional (...) researcher during the research cycle. We discuss future opportunities for reform. The resulting agenda provides a resource to usher in an era that is marked by a research culture that is less error-prone and a scientific publication landscape with fewer spurious findings. (shrink)

Over the last decades, understanding the sources of the difficulty of Bayesian problem solving has been an important research goal, with the effects of numerical format and individual numeracy being widely studied. However, the focus on the comprehension of probability numbers has overshadowed the relational reasoning demand of the Bayesian task. This is particularly the case when the statistical data are verbally described since the requested quantitative relation (posterior ratio) is misaligned with the presented ones (prior and likelihood ratios). In (...) this regard, here I develop the proposal that research on Bayesian reasoning might improve by considering the notational alignment framework of mathematical problem-solving. Specifically, this framework can help to understand the sources of the main difficulties underlying Bayesian inferences based on verbal descriptions. In essence, the present proposal supports the general claim in math education regarding the need to foster relational comprehension to avoid misleading alignments and improve problem solving. (shrink)

:Behavioural public policy has come under fire by critics who claim that it is illiberal. Some authors recently suggest that there is a type of BPP – boosting – that is not as vulnerable to this normative critique. Our paper challenges this claim: there's no non-circular way to draw the distinction between nudge and boost that would make the normative difference required to infer the permissibility of a policy intervention from its type-membership. We consider two strategies: paradigmatic examples and causal (...) mechanisms. We conclude by sketching some suggestions about the right way to approach the normative issues. (shrink)

Cognition is both empowered and limited by representations. The matrix lens model explicates tasks that are based on frequency counts, conditional probabilities, and binary contingencies in a general fashion. Based on a structural analysis of such tasks, the model links several problems and semantic domains and provides a new perspective on representational accounts of cognition that recognizes representational isomorphs as opportunities, rather than as problems. The shared structural construct of a 2 × 2 matrix supports a set of generic tasks (...) and semantic mappings that provide a unifying framework for understanding problems and defining scientific measures. Our model's key explanatory mechanism is the adoption of particular perspectives on a 2 × 2 matrix that categorizes the frequency counts of cases by some condition, treatment, risk, or outcome factor. By the selective steps of filtering, framing, and focusing on specific aspects, the measures used in various semantic domains negotiate distinct trade-offs between abstraction and specialization. As a consequence, the transparent communication of such measures must explicate the perspectives encapsulated in their derivation. To demonstrate the explanatory scope of our model, we use it to clarify theoretical debates on biases and facilitation effects in Bayesian reasoning and to integrate the scientific measures from various semantic domains within a unifying framework. A better understanding of problem structures, representational transparency, and the role of perspectives in the scientific process yields both theoretical insights and practical applications. (shrink)

An experiment was conducted to test the effectiveness of brief instruction in information structuring (i.e., representing and integrating information) for improving the coherence of probability judgments and binary choices among intelligence analysts. Forty-three analysts were presented with comparable sets of Bayesian judgment problems before and immediately after instruction. After instruction, analysts’ probability judgments were more coherent (i.e., more additive and compliant with Bayes theorem). Instruction also improved the coherence of binary choices regarding category membership: after instruction, subjects were more likely (...) to invariably choose the category to which they assigned the higher probability of a target’s membership. The research provides a rare example of evidence-based validation of effectiveness in instruction to improve the statistical assessment skills of intelligence analysts. Such instruction could also be used to improve the assessment quality of other types of experts who are required to integrate statistical information or make probabilistic assessments. (shrink)

Humans have long been characterized as poor probabilistic reasoners when presented with explicit numerical information. Bayesian word problems provide a well-known example of this, where even highly educated and cognitively skilled individuals fail to adhere to mathematical norms. It is widely agreed that natural frequencies can facilitate Bayesian reasoning relative to normalized formats (e.g. probabilities, percentages), both by clarifying logical set-subset relations and by simplifying numerical calculations. Nevertheless, between-study performance on “transparent” Bayesian problems varies widely, and generally remains rather unimpressive. (...) We suggest there has been an over-focus on this representational facilitator (i.e. transparent problem structures) at the expense of the specific logical and numerical processing requirements and the corresponding individual abilities and skills necessary for providing Bayesian-like output given specific verbal and numerical input. We further suggest that understanding this task-individual pair could benefit from considerations from the literature on mathematical cognition, which emphasizes text comprehension and problem solving, along with contributions of online executive working memory, metacognitive regulation, and relevant stored knowledge and skills. We conclude by offering avenues for future research aimed at identifying the stages in problem solving at which correct versus incorrect reasoners depart, and how individual difference might influence this time point. (shrink)

This paper aims to make explicit the methodological conditions that should be satisfied for the Bayesian model to be used as a normative model of human probability judgment. After noticing the lack of a clear definition of Bayesianism in the psychological literature and the lack of justification for using it, a classic definition of subjective Bayesianism is recalled, based on the following three criteria: an epistemic criterion, a static coherence criterion and a dynamic coherence criterion. Then it is shown that (...) the adoption of this framework has two kinds of implications. The first one regards the methodology of the experimental study of probability judgment. The Bayesian framework creates pragmatic constraints on the methodology that are linked to the interpretation of, and the belief in, the information presented, or referred to, by an experimenter in order for it to be the basis of a probability judgment by individual participants. It is shown that these constraints have not been satisfied in the past, and the question of whether they can be satisfied in principle is raised and answered negatively. The second kind of implications consists of two limitations in the scope of the Bayesian model. They regard (1) the background of revision (the Bayesian model considers only revising situations but not updating situations), and (2) the notorious case of the null priors. In both cases Lewis’ rule is an appropriate alternative to Bayes’ rule, but its use faces the same operational difficulties. (shrink)

If citizens’ behavior threatens to harm others or seems not to be in their own interest, it is not uncommon for governments to attempt to change that behavior. Governmental policy makers can apply established tools from the governmental toolbox to this end. Alternatively, they can employ new tools that capitalize on the wealth of knowledge about human behavior and behavior change that has been accumulated in the behavioral sciences. Two contrasting approaches to behavior change are nudge policies and boost policies. (...) These policies rest on fundamentally different research programs on bounded rationality, namely, the heuristics and biases program and the simple heuristics program, respectively. This article examines the policy–theory coherence of each approach. To this end, it identifies the necessary assumptions underlying each policy and analyzes to what extent these assumptions are implied by the theoretical commitments of the respective research program. Two key results of this analysis are that the two policy approaches rest on diverging assumptions and that both suffer from disconnects with the respective theoretical program, but to different degrees: Nudging appears to be more adversely affected than boosting does. The article concludes with a discussion of the limits of the chosen evaluative dimension, policy–theory coherence, and reviews some other benchmarks on which policy programs can be assessed. (shrink)

The terms nested sets, partitive frequencies, inside-outside view, and dual processes add little but confusion to our original analysis (Gigerenzer & Hoffrage 1995; 1999). The idea of nested set was introduced because of an oversight; it simply rephrases two of our equations. Representation in terms of chances, in contrast, is a novel contribution yet consistent with our computational analysis System 1.dual process theory” is: Unless the two processes are defined, this distinction can account post hoc for almost everything. In contrast, (...) an ecological view of cognition helps to explain how insight is elicited from the outside (the external representation of information) and, more generally, how cognitive strategies match with environmental structures. (shrink)

Can the general public learn to deal with risk and uncertainty, or do authorities need to steer people’s choices in the right direction? Libertarian paternalists argue that results from psychological research show that our reasoning is systematically flawed and that we are hardly educable because our cognitive biases resemble stable visual illusions. For that reason, they maintain, authorities who know what is best for us need to step in and steer our behavior with the help of “nudges.” Nudges are nothing (...) new, but justifying them on the basis of a latent irrationality is. In this article, I analyze the scientific evidence presented for such a justification. It suffers from narrow logical norms, that is, a misunderstanding of the nature of rational thinking, and from a confirmation bias, that is, selective reporting of research. These two flaws focus the blame on individuals’ minds rather than on external causes, such as industries that spend billions to nudge people into unhealthy behavior. I conclude that the claim that we are hardly educable lacks evidence and forecloses the true alternative to nudging: teaching people to become risk savvy. (shrink)

Visual aids can improve comprehension of risks associated with medical treatments, screenings, and lifestyles. Do visual aids also help decision makers accurately assess their risk comprehension? That is, do visual aids help them become well calibrated? To address these questions, we investigated the benefits of visual aids displaying numerical information and measured accuracy of self-assessment of diagnostic inferences (i.e., metacognitive judgment calibration) controlling for individual differences in numeracy. Participants included 108 patients who made diagnostic inferences about three medical tests on (...) the basis of information about the sensitivity and false-positive rate of the tests and disease prevalence. Half of the patients received the information in numbers without a visual aid, while the other half received numbers along with a grid representing the numerical information. In the numerical condition, many patients --especially those with low numeracy-- misinterpreted the predictive value of the tests and profoundly overestimated the accuracy of their inferences. Metacognitive judgment calibration mediated the relationship between numeracy and accuracy of diagnostic inferences. In contrast, in the visual aid condition, patients at all levels of numeracy showed high-levels of inferential accuracy and metacognitive judgment calibration. Results indicate that accurate metacognitive assessment may explain the beneficial effects of visual aids and numeracy --a result that accords with theory suggesting that metacognition is an essential part of risk literacy. We conclude that well-designed risk communications can inform patients about health-relevant numerical information while helping them assess the quality of their own risk comprehension. (shrink)

We hypothesised that belief in conspiracy theories would be predicted by the general tendency to attribute agency and intentionality where it is unlikely to exist. We further hypothesised that this tendency would explain the relationship between education level and belief in conspiracy theories, where lower levels of education have been found to be associated with higher conspiracy belief. In Study 1 participants were more likely to agree with a range of conspiracy theories if they also tended to attribute intentionality and (...) agency to inanimate objects. As predicted, this relationship accounted for the link between education level and belief in conspiracy theories. We replicated this finding in Study 2, whilst taking into account beliefs in paranormal phenomena. These results suggest that education may undermine the reasoning processes and assumptions that are reflected in conspiracy belief. (shrink)

Does the most common response given by participants presented with Tversky and Kahneman’s famous taxi cab problem amount to a violation of Bayes’ theorem? In other words, do they fall victim to so-called base-rate fallacy? In the present paper, following an earlier suggestion by Crupi and Girotto, we will identify the logical arguments underlying both the original diagnosis of irrationality in this reasoning task under uncertainty and a number of objections that have been raised against such a diagnosis. This will (...) enable us to show firstly that, contrary to the dismissive arguments recently put forward by Elqayam and Evans, empirical evidence turns out to be quite useful in addressing questions of this kind. Therefore, it can make a significant contribution to moving the base-rate fallacy controversy forward. Secondly, the available empirical evidence (though limited and sometimes inconclusive) seems to support the charge of irrationality levelled against the majority of participants presented with the taxi cab problem, and hence suggests that base-rate neglect is a real fallacy − that is the conclusion of our analysis. (shrink)

Causal reasoning is one of our most central cognitive competencies, enabling us to adapt to our world. Causal knowledge allows us to predict future events, or diagnose the causes of observed facts. We plan actions and solve problems using knowledge about cause-effect relations. Without our ability to discover and empirically test causal theories, we would not have made progress in various empirical sciences. In the past decades, the important role of causal knowledge has been discovered in many areas of cognitive (...) psychology. Despite the ubiquity of causal reasoning, textbooks of cognitive psychology have neglected this growing field. The goal of The Oxford Handbook of Causal Reasoning is to fill this gap. The handbook brings together the leading researchers in the field of causal reasoning and offers state-of-the-art presentations of theories and research. It provides introductions of competing theories of causal reasoning, and discusses its role in various cognitive functions and domains. The final section presents research from neighboring fields. 1. Causal Reasoning: An Introduction Michael R. Waldmann Part I: Theories of Causal Cognition 2. Associative Accounts of Causal Cognition Mike E. Le Pelley, Oren Griffiths, and Tom Beesley 3. Rules of Causal Judgment: Mapping Statistical Information Onto Causal Beliefs José C. Perales, Andrés Catena, Antonio Cándido, and Antonio Maldonado 4. The Inferential Reasoning Theory of Causal Learning: Toward a Multi- Process Propositional Account Yannick Boddez, Jan De Houwer, and Tom Beckers 5. Causal Invariance as an Essential Constraint for Creating a Causal Representation of the World: Generalizing the Invariance of Causal Power Patricia W. Cheng and Hongjing Lu 6. The Acquisition and Use of Causal Structure Knowledge Benjamin Margolin Rottman 7. Formalizing Prior Knowledge in Causal Induction Thomas L. Griffiths 8. Causal Mechanisms Samuel G. B. Johnson and Woo-kyoung Ahn 9. Force Dynamics and Causation Phillip Wolff and Robert Thorstad 10. Mental Models and Causation P. N. Johnson- Laird and Sangeet S. Khemlani 11. Pseudocontingencies Klaus Fiedler and Florian Kutzner 12. Singular Causation David Danks 13. Cognitive Neuroscience of Causal Reasoning Joachim T. Operskalski and Aron K. Barbey Part II: Basic Cognitive Functions 14. Visual Impressions of Causality Peter White 15. Goal-Directed Actions Bernhard Hommel 16. Planning and Control Magda Osman 17. Reinforcement Learning and Causal Models Samuel J. Gershman 18. Causation and the Probability of Causal Conditionals David E. Over 19. Causal Models and Conditional Reasoning Mike Oaksford and Nick Chater 20. Concepts as Causal Models: Categorization Bob Rehder 21. Concepts as Causal Models: Induction Bob Rehder 22. Causal Explanation Tania Lombrozo and Nadya Vasilyeva 23. Diagnostic Reasoning Björn Meder and Ralf Mayrhofer 24. Inferring Causal Relations by Analogy Keith J. Holyoak and Hee-Seung Lee 25. Causal Argument Ulrike Hahn, Roland Bluhm, and Frank Zenker 26. Causality in Decision- Making York Hagmayer and Philip M. Fernbach Part III: Domains of Causal Reasoning 27. Intuitive Theories Tobias Gerstenberg and Joshua B. Tenenbaum 28. Space, Time, and Causality Marc J. Buehner 29. Causation in Legal and Moral Reasoning David A. Lagnado and Tobias Gerstenberg 30. The Role of Causal Knowledge in Reasoning About Mental Disorders Woo-kyoung Ahn, Nancy S. Kim, and Matthew S. Lebowitz 31. Causality and Causal Reasoning in Natural Language Torgrim Solstad and Oliver Bott 32. Social Attribution and Explanation Denis Hilton Part IV: Development, Phylogeny, and Culture 33. The Development of Causal Reasoning Paul Muentener and Elizabeth Bonawitz 34. Causal Reasoning in Non-Human Animals Christian Schloegl and Julia Fischer 35. Causal Cognition and Culture Andrea Bender, Sieghard Beller, and Douglas L. Medin. (shrink)

In the present paper we empirically investigate the psychometric properties of some of the most famous statistical and logical cognitive illusions from the “heuristics and biases” research program by Daniel Kahneman and Amos Tversky, who nearly 50 years ago introduced fascinating brain teasers such as the famous Linda problem, the Wason card selection task, and so-called Bayesian reasoning problems (e.g., the mammography task). In the meantime, a great number of articles has been published that empirically examine single cognitive illusions, theoretically (...) explaining people’s faulty thinking, or proposing and experimentally implementing measures to foster insight and to make these problems accessible to the human mind. Yet these problems have thus far usually been empirically analyzed on an individual-item level only (e.g., by experimentally comparing participants’ performance on various versions of one of these problems). In this paper, by contrast, we examine these illusions as a group and look at the ability to solve them as a psychological construct. Based on an sample ofN= 2,643 Luxembourgian school students of age 16–18 we investigate the internal psychometric structure of these illusions (i.e., Are they substantially correlated? Do they form a reflexive or a formative construct?), their connection to related constructs (e.g., Are they distinguishable from intelligence or mathematical competence in a confirmatory factor analysis?), and the question of which of a person’s abilities can predict the correct solution of these brain teasers (by means of a regression analysis). (shrink)

In teaching statistics in secondary schools and at university, two visualizations are primarily used when situations with two dichotomous characteristics are represented: 2×2 tables and tree diagrams. Both visualizations can be depicted either with probabilities or with frequencies. Visualizations with frequencies have been shown to help students significantly more in Bayesian reasoning problems than probability visualizations do. Because tree diagrams or double-trees (which are largely unknown in school) are node-branch-structures, these two visualizations (compared to the 2×2 table) can even simultaneously (...) display probabilities on branches and frequencies inside the nodes. This is a teaching advantage as it allows the frequency concept to be used to better understand probabilities. However, 2×2 tables and (double-) trees have a decisive disadvantage: While joint probabilities (e.g., P(AB)) are represented in 2×2 tables but no conditional probabilities (e.g., P(A|B)), it is exactly the other way around with (double-) trees. Therefore, a visualization that is equally suitable for the representation of joint probabilities and conditional probabilities is desirable. In this article, we present a new visualization—the frequency net—in which all absolute frequencies and all types of probabilities can be depicted. In addition to a detailed theoretical analysis of the frequency net, we report the results of a study with 249 university students that shows that “net diagrams” can improve reasoning without previous instruction to a similar extent as 2×2 tables and double-trees. Regarding questions about conditional probabilities, frequency visualizations (2×2 table, double-tree, or net diagram with natural frequencies) are consistently superior to probability visualizations. The frequency net performs as well as the frequency double-tree. Only the 2×2 table with frequencies—the only visualization that participants were already familiar with —led to higher performance rates. If, on the other hand, a question about a joint probability had to be answered, all implemented visualizations clearly supported participants’ performance, and no uniform format effect becomes visible. Participants reached the highest performance in the versions with probability 2×2 tables and probability net diagrams. Furthermore, after conducting a detailed error analysis, we report interesting error shifts between the two information formats and the different visualizations and give recommendations for teaching probability. (shrink)

The phenomenon of base-rate neglect has elicited much debate. One arena of debate concerns how people make judgments under conditions of uncertainty. Another more controversial arena concerns human rationality. In this target article, we attempt to unpack the perspectives in the literature on both kinds of issues and evaluate their ability to explain existing data and their conceptual coherence. From this evaluation we conclude that the best account of the data should be framed in terms of a dual-process model of (...) judgment, which attributes base-rate neglect to associative judgment strategies that fail to adequately represent the set structure of the problem. Base-rate neglect is reduced when problems are presented in a format that affords accurate representation in terms of nested sets of individuals. (shrink)