Commentators agree that simple conclusions about a general base rate fallacy are not appropriate. It is more constructive to identify conditions under which base rates are differentially weighted. Commentators also agree that improving the ecological validity of the research is desirable, although this is less important to those interested exclusively in psychological processes. The philosophers and ecologists among the commentators offer a kinder perspective on base rate reasoning than the psychologists. My own perspective is that the interesting questions (both psychological (...) and normative/prescriptive) are best answered through empirical studies that use real-world performance standards. (shrink)

We have been oversold on the base rate fallacy in probabilistic judgment from an empirical, normative, and methodological standpoint. At the empirical level, a thorough examination of the base rate literature (including the famous lawyer–engineer problem) does not support the conventional wisdom that people routinely ignore base rates. Quite the contrary, the literature shows that base rates are almost always used and that their degree of use depends on task structure and representation. Specifically, base rates play a relatively larger role (...) in tasks where base rates are implicitly learned or can be represented in frequentist terms. Base rates are also used more when they are reliable and relatively more diagnostic than available individuating information. At the normative level, the base rate fallacy should be rejected because few tasks map unambiguously into the narrow framework that is held up as the standard of good decision making. Mechanical applications of Bayes's theorem to identify performance errors are inappropriate when (1) key assumptions of the model are either unchecked or grossly violated, and (2) no attempt is made to identify the decision maker's goals, values, and task assumptions. Methodologically, the current approach is criticized for its failure to consider how the ambiguous, unreliable, and unstable base rates of the real world are and should be used. Where decision makers' assumptions and goals vary, and where performance criteria are complex, the traditional Bayesian standard is insufficient. Even where predictive accuracy is the goal in commonly defined problems, there may be situations (e.g., informationally redundant environments) in which base rates can be ignored with impunity. A more ecologically valid research program is called for. This program should emphasize the development of prescriptive theory in rich, realistic decision environments. (shrink)

Base rates have no necessary relation to judgments that are not themselves probabilities. There is no logical imperative, for instance, that behavioral base rates must affect causal attributions or that base rate information should affect judgments of legal liability. Decision theorists should be cautious in arguing that base rates place normative constraints on judgments of anything other than posterior probabilities.

Abstract(1) The miscitations of seminal experiments in the base rate literature adds to the existing database of systematic miscitations of wellknown psychological experiments. These miscitations may be caused by a process of reconstructive remembering. (2) Representative design should be the methodological core of Koehler's call for ecologically valid research. This approach can benefit both basic and applied research.

Koehler's summary and critique of research on the base rate fallacy is cogent and persuasive. However, he may have overstated the case, and his suggestions for future research may be too restrictive. We agree that methodological approaches to this topic should be broadened, but we argue that experimental laboratory research and the Bayesian normative standard are useful and should not be abandoned.

This commentary presents a self-assessment inventory that will allow readers to determine their own attitude toward the base rate fallacy and its literature. The inventory is scientifically valid but not Medicare/Medicaid reimbursable.

When base rates are learned and used in an experiential manner subjects show better base rate use, perhaps because the implicit learning system is engaged. A causal framework in which base rates are relevant might also be necessary. Humans might thus perform better on more ecologically valid tasks, which are likely to contain those three components.

Base rates are vital in predicting violent criminal recidivism. However, both lay people given simulated prediction tasks and professionals milking real life predictions appear insensitive to variations in the base rate of violent recidivism. Although there are techniques to help decision makers attend to base rates, increased decision accuracy is better sought in improved actuarial models as opposed to improved clinicians.

Consistent with Koehler's position, we propose a generalization of the base rate fallacy and earlier conservatism literatures. In studies using both traditional tasks and new tasks based on ecologically valid base rates, our subjects typically underweight individuating information at least as much as they underweight base rates. The implications of cue consistency for averaging heuristics are discussed.

Koehler is right that base rate information is used, to various degrees, both in laboratory tasks and in everyday life. However, it is not time to turn our backs on laboratory tasks and focus solely on ecologically valid decision making. Tightly controlled experimental data are still needed to understandhowbase rate information is used, and how this varies among groups.

Any instance (i.e., event, behavior, trait) belongs to infinitely many reference classes, hence there are infinitely many base rates from which to choose. People clearly do not entertain all possible reference classes, however, so something must be limiting the search space. We suggest some possible mechanisms that determine which reference class is evoked for the purpose of judgment and decision.

The fallacy beneath the base rate fallacy is that we know what a base rate is. We talk as if base rates and individuating information were two different kinds of information. From a Bayesian perspective, however, the only difference between base rate and individuating information is – which comes first.

Koehler is right to argue for more nuanced interpretation of base rate anomalies. These anomalies are best understood in relation to a broader class of cognitive anomalies, which are important for theory and practice. Recognizing a need for more nuanced analysis should not be taken as a license for treating the effects as “explained away.”.

Koehler's target article attempts a balanced view of the relevance of knowledge of base rates to judgments of subjective or credal probability, but he is not sensitive enough to the difference between requiring and permitting the equation of probability judgments with base rates, the interaction between precision of base rate and reference class information, and the possibility of indeterminate probability judgment.

Two distinct issues are sometimes confused in the base rate literature: Why do people make logical mistakes in the assessment of probabilities? and why do subjects not use base rates the way experimenters do? The latter problem may often reflect differences in an implicit reference class rather than a disinclination to update a base rate by Bayes' theorem. Also important are considerations concerning the interaction of several potentially relevant base rates.

The underutilization of base rates is a consistent finding. The strong claim that base rates are ignored has been rejected and this needs no further emphasis. Following the path of “normal science,” research examines the conditions predicting changes in the degree of underutilization. A scientific revolution that might dethrone the heuristics and biases paradigm is not in sight.

Koehler's call for a reanalysis of the base rate fallacy is particularly important in the applied domain of accounting, since base rate data appear to be an important input for many accounting tasks. In this commentary I discuss the use of base rates in accounting and explain why more flexible standards of performance are important when judging the use of base rates.

This commentary discusses three points: (1) The implications of the fact that it is rational to ignore base rates if probabilities are estimated by frequencies from samples without missing data (natural sampling); (2) second order probabilities distributions are a plausible way to model imprecise probabilities; and (3) Bayesian networks represent a normative reference for multi-cue models of probabilistic inference.

Failure to consider base rate is regarded as potentially hazardous, mainly because its consideration is assumed to be determined solely by P(H/D), the probability of the individuating data if the hypothesis is true, and not at all by P(D/˜H), the probability if the hypothesis is false. However, when P(D/˜H) is unconfounded from P(D/H), it turns out to be the stronger determinant of base rate consideration.

Setting the two hypotheses of complete neglect and full use of base rates against each other is inappropriate. The proper question concerns the degree to which base rates are used (or neglected), and under what conditions. We outline alternative approaches and recommend regression analysis. Koehler's conclusion that we have been oversold on the base rate fallacy seems to be premature.

A recent study showed physicians' reasoning about a realistic case to be ignorant of base rate. It also showed physicians interpreting information pertinent to base rate differently, depending on whether it was presented early or late in the case. Although these adult reasoners might do better if given hints through talk of relative frequencies, this would not prove that they had no problem of base rate neglect.

The environment in which humans evolved is strongly and positively autocorrelated in space and time. Probabilistic judgments based on the assumption of independence may not yield evolutionarily adaptive behavior. A number of “faults” of human reasoning are not faulty under fuzzy arithmetic, a nonprobabilistic calculus of reasoning under uncertainty that may be closer to that underlying human decision making.

In contrast to traditional research on base-rate neglect, an ecologically-oriented research program would analyze the correspondence between cognitive algorithms and the nature of information in the environment. Bayesian computations turn out to be simpler when information is represented in frequency formats as opposed to the probability formats used in previous research. Frequency formats often enable even uninstructed subjects to perform Bayesian reasoning.

The base rate literature has an opposite twin in the social psychological literature on stereotypes, which concludes that people use their preexisting beliefs about probabilistic category attributes too much, rather than not enough. This ironic discrepancy arises because beliefs about category attributes enhance accuracy when the beliefs are accurate and diminish accuracy when they are not. To determine the accuracy of base rate/stereotype beliefs requires research that addresses specific content.

Abstract(1) There is enough contradictory evidence regarding the role of base rates in category learning to confirm the nonexistence of biases in such learning. (2) It is not always possible to activate statistical reasoning through frequentist representation. (3) It is necessary to use the concept of systematic processing in reconsidering the published work on biases.

The important question is how people process probabilistic information, not whether they process it in accordance with a normative model that we never should have expected them to be capable of following. Experience is not the cure, as widely thought, to problems with utilizing base rate information.

As illustrated by research Koehler himself cites (Dawes et al. 1993), the purpose of experiments is to choose between contrasting explanations of past observations – rather than to seek statistical generalizations about the prevalence of effects. True external validity results not from sampling various problems that are representative of “real world” decision making, but from reproducing an effect in the laboratory with minimal contamination (including from real world factors).

The base rate fallacy is directly dependent on a particular judgment paradigm in which information may be unambiguously designated as either “base rate” or “individuating,” and in which subjects make two-stage sequential judgments. The paradigm may be a poor match for real world settings, and the fallacy may thus be undefined for natural ecologies of judgment.

There is a familiar risk of antinomy if fromxisEand p(xisH/xisE) =rit is permissible to infer p(xisH) =r, and what Carnap (1950) called “The requirement of total evidence” will not prevent such antinomies satisfactorily. What is needed instead is a properly developed theory of evidential weight.

Koehler's work will assist the effort to understand legal fact finding. It leaves two questions somewhat open: (i) the extent to which empirical research can measure correctness of fact-finding, a function that involves the resolution of normative questions and (ii) the standards judges should use in the absence of the research he advocates.

Cognitive algebra strongly disproved the representativeness heuristic almost before it was published; and therewith it also disproved the base rate fallacy. Cognitive algebra provides a theoretical foundation for judgment-decision theory through its joint solution to the two fundamental problems – true measurement of subjective values, and cognitive rules for integration of multiple determinants.

The present experiment investigated the development of implicit and explicit knowledge during concept learning. According to Cleeremans and Jiménez , the content of a representation can be conscious only when the representation is of a sufficiently good quality; on this theory, increasing explicit and decreasing implicit knowledge might be expected with training. The view that implicit knowledge arises from compilation of explicit knowledge makes the opposite prediction. The present research tested these possibilities using subjective measures based on confidence ratings. One (...) group of participants was presented with blocks of category exemplars that activated prior knowledge, whereas another group was presented with blocks of categories that did not elicit any useful prior knowledge. The results showed that, irrespective of the knowledge group participants were allocated to, explicit knowledge increased over the course of learning, whereas implicit knowledge either stayed the same or decreased, consistent with Cleeremans and Jiménez’s prediction. (shrink)

Using the Chinese Ring Puzzle (Kotovsky & Simon, Citation1990; P. J. Reber & Kotovsky, Citation1997), we studied the effect on rule discovery of having to plan actions or not in order to reach a goal state. This was done by asking participants to predict legal moves as in implicit learning tasks (Experiment 1) and by asking participants to make legal moves as in problem-solving tasks (Experiment 2). Our hypothesis was that having a specific goal state to reach has a dual (...) effect on rule discovery. The first effect is positive and related to feedback from moves done in order to attain the goal: generalising the results of action and associating them to the conditions in which they were obtained allows discovery of the rule and learning it. The second effect is negative. In attempting to reach a specific goal, participants first tend to reduce the distance that separates the current state from the goal state (hill climbing) and so neglect the kind of exploration that facilitates rule and procedure discovery because this would seem to be a detour from the goal. Results show that having to plan actions improved performance in implicit learning tasks (Experiment 1), yet it impaired performance in problem-solving tasks (Experiment 2). Although implicit learning and problem solving are based on rule discovery, and entail noticing regularities in the material, in both cases, rule discovery processes appear to be task-dependent. (shrink)

Transitive inference (TI) plays a part in many aspects of reasoning, and is usually assessed using variants on a particular task dubbed the “IP-paradigm”. Advocates of this paradigm assume it ensures that subjects must use deduction to solve the inferential questions. The present task with 63 adults strengthened this claim by removing all possible perceptual cues and limiting as far as possible all cues from the training procedure itself. Response speed and accuracy were measured as premises were learned. Findings show (...) that when over-training and non-logical cues are avoided, adults nevertheless reach high levels of overall performance. However, key indexes of Lexical Marking, Relative-End-Anchoring, and Symbolic Distance are largely absent or in the opposite direction to the generally accepted view at the start of training, and are present only for response time (RT) at the end of training. The widely accepted strategy of retrieval from an integrated mental representation does emerge; but only after an initial associative process and a deductive process. Regarding three comparisons critical for demonstrating deductive TI, the inference was more demanding than its antecedents both in terms of accuracy and RT, and both after initial and final training. Existing theories of TI account for some of these findings, but what is required is a theory accommodating them all. (shrink)

Using a classical conditioning technique, this study investigated whether nonconscious associative learning could be indexed by event-related brain activity . There were three phases. In a preconditioning baseline phase, pleasant and unpleasant facial schematics were presented in awareness . A conditioning phase followed, in which stimuli were presented outside awareness , with an unpleasant face linked to an aversive shock and a pleasant face not linked to a shock. The third, postconditioning phase, involved stimulus presentations in awareness . Evidence for (...) acquisition of a conditional response was sought by comparing suprathreshold pre- and postconditioning phases, as well as in the subthreshold conditioning phase itself. For the pre-postconditioning phase analyses, significant ERP component differences differentiating CS+ and CS− were observed for N1, P2, and especially P3. For the conditioning phase, significant differences were observed in the 100–400 ms. post-stimulus region reflecting a CS+ processing negativity. Brain activity does indeed index the acquisition of a conditional response to subthreshold stimuli. Associative learning can occur outside awareness. (shrink)

Using a classical conditioning technique, this study investigated whether nonconscious associative learning could be indexed by event-related brain activity . There were three phases. In a preconditioning baseline phase, pleasant and unpleasant facial schematics were presented in awareness . A conditioning phase followed, in which stimuli were presented outside awareness , with an unpleasant face linked to an aversive shock and a pleasant face not linked to a shock. The third, postconditioning phase, involved stimulus presentations in awareness . Evidence for (...) acquisition of a conditional response was sought by comparing suprathreshold pre- and postconditioning phases, as well as in the subthreshold conditioning phase itself. For the pre-postconditioning phase analyses, significant ERP component differences differentiating CS+ and CS− were observed for N1, P2, and especially P3. For the conditioning phase, significant differences were observed in the 100–400 ms. post-stimulus region reflecting a CS+ processing negativity. Brain activity does indeed index the acquisition of a conditional response to subthreshold stimuli. Associative learning can occur outside awareness. (shrink)

Consciousness can be measured in various ways, but different measures often yield different conclusions about the extent to which awareness relates to performance. Here, we compare five different subjective measures of awareness in the context of an artificial grammar learning task. Participants expressed their subjective awareness of rules using one of five different scales: confidence ratings , post-decision wagering , feeling of warmth , rule awareness , and continuous scale . All scales were equally sensitive to conscious knowledge. PDW, however, (...) was affected by risk aversion, and both RAS and SDS applied different minimal criteria for rule awareness. CR seems to capture the largest range of consciousness, but failed to indicate unconscious knowledge with the guessing criterion. We close by discussing the theoretical implications of scale sensitivity and propose that CR’s unique features enable a finer assessment of subjective states of awareness. (shrink)

Participants were unknowingly exposed to complex regularities in a working memory task. The existence of implicit knowledge was subsequently inferred from a preference for stimuli with similar grammatical regularities. Several affective traits have been shown to influence AGL performance positively, many of which are related to a tendency for automatic responding. We therefore tested whether the mindfulness trait predicted a reduction of grammatically congruent preferences, and used emotional primes to explore the influence of affect. Mindfulness was shown to correlate negatively (...) with grammatically congruent responses. Negative primes were shown to result in faster and more negative evaluations. We conclude that grammatically congruent preference ratings rely on habitual responses, and that our findings provide empirical evidence for the non-reactive disposition of the mindfulness trait. (shrink)

Although many studies have provided evidence that abstract knowledge can be acquired in artificial grammar learning, it remains unclear how abstract knowledge can be attained in sequence learning. To address this issue, we proposed a dual simple recurrent network model that includes a surface SRN encoding and predicting the surface properties of stimuli and an abstract SRN encoding and predicting the abstract properties of stimuli. The results of Simulations 1 and 2 showed that the DSRN model can account for learning (...) effects in the serial reaction time task under different conditions, and the manipulation of the contribution weight of each SRN accounted for the contribution of conscious and unconscious processes in inclusion and exclusion tests in previous studies. The results of human performance in Simulation 3 provided further evidence that people can implicitly learn both chunking and abstract knowledge in sequence learning, and the results of Simulation 3 confirmed that the DSRN model can account for how people implicitly acquire the two types of knowledge in sequence learning. These findings extend the learning ability of the SRN model and help understand how different types of knowledge can be acquired implicitly in sequence learning. (shrink)

The question of whether studies of human learning provide evidence for distinct conscious and unconscious influences remains as controversial today as ever. Much of this controversy arises from the use of the logic of dissociation. The controversy has prompted the use of an alternative approach that places conscious and unconscious influences on memory retrieval in opposition. Here we ask whether evidence acquired via the logic of opposition requires a dual-process account or whether it can be accommodated within a single similarity-based (...) account. We report simulations using a simple neural network model of two artificial grammar learning experiments reported by Higham, Vokey, and Pritchard that dissociated conscious and unconscious influences on classification. The simulations demonstrate that opposition logic is insufficient to distinguish between single- and multiple-system models. (shrink)

This article explicates the interaction between implicit and explicit processes in skill learning, in contrast to the tendency of researchers to study each type in isolation. It highlights various effects of the interaction on learning (including synergy effects). The authors argue for an integrated model of skill learning that takes into account both implicit and explicit processes. Moreover, they argue for a bottom-up approach (first learning implicit knowledge and then explicit knowledge) in the integrated model. A variety of qualitative data (...) can be accounted for by the approach. A computational model, CLARION, is then used to simulate a range of quantitative data. The results demonstrate the plausibility of the model, which provides a new perspective on skill learning. (PsycINFO Database Record (c) 2012 APA, all rights reserved). (shrink)