Why do people who disagree about one subject tend to disagree about other subjects as well? In this paper, we introduce a model to explore this phenomenon of ‘epistemic factionization’. Agents attempt to discover the truth about multiple propositions by testing the world and sharing evidence gathered. But agents tend to mistrust evidence shared by those who do not hold similar beliefs. This mistrust leads to the endogenous emergence of factions of agents with multiple, highly correlated, polarized beliefs.

This article analyses the consequences that disinformation phenomena have for a model of communication, focusing on the dangers that disinformation poses to democratic societies, especially when it is disseminated by the media. Disinformation is examined here from the perspective of social cognitive psychology, with special attention to the role played by motivated reasoning and confirmation bias in human cognition. From this perspective, disinformation phenomena should be studied not only through an analysis of how the media operate, but also through an (...) understanding of how we process information and what we use it for from a social cognitive point of view. This article emphasizes the role that intuition and affective persuasion play in communication processes, as key elements of motivated reasoning, and argues that once this cognitive dimension is integrated into communication theory, preventive strategies can be designed to protect democracies from the dangers caused by disinformation. Ideological polarization and a lack of consensus are highlighted here as being among the biggest dangers, preventing agreement on issues that affect the proper functioning of democracy. While a certain conception of communication posits reasoning, the media and education as the tools for resolving conflicts and preventing disagreements, this article concludes that the success of disinformation phenomena points to the need for a model that includes the cognitive elements mentioned above. (shrink)

In this paper we present the results of a simulation study of credence developments in groups of communicating Bayesian agents, as they update their beliefs about a given proposition p. Based on the empirical literature, one would assume that these groups of rational agents would converge on a view over time, or at least that they would not polarize. This paper presents and discusses surprising evidence that this is not true. Our simulation study shows that these groups of Bayesian agents (...) show group polarization behavior under a broad range of circumstances. This is, we think, an unexpected result, that raises deeper questions about whether the kind of polarization in question is irrational. If one accepts Bayesian agency as the hallmark of epistemic rationality, then one should infer that the polarization we find is also rational. On the other hand, if we are inclined to think that there is something epistemically irrational about group polarization, then something must be off in the model employed in our simulation study. We discuss several possible interfering factors, including how epistemic trust is defined in the model. Ultimately, we propose that the notion of Bayesian agency is missing something in general, namely the ability to respond to higher-order evidence. (shrink)

Contemporary societies are often “polarized”, in the sense that sub-groups within these societies hold stably opposing beliefs, even when there is a fact of the matter. Extant models of polarization do not capture the idea that some beliefs are true and others false. Here we present a model, based on the network epistemology framework of Bala and Goyal, 784–811 1998), in which polarization emerges even though agents gather evidence about their beliefs, and true belief yields a pay-off advantage. As we (...) discuss, these results are especially relevant to polarization in scientific communities, for these reasons. The key mechanism that generates polarization involves treating evidence generated by other agents as uncertain when their beliefs are relatively different from one’s own. (shrink)

Science strives for coherence. For example, the findings from climate science form a highly coherent body of knowledge that is supported by many independent lines of evidence: greenhouse gas emissions from human economic activities are causing the global climate to warm and unless GHG emissions are drastically reduced in the near future, the risks from climate change will continue to grow and major adverse consequences will become unavoidable. People who oppose this scientific body of knowledge because the implications of cutting (...) GHG emissions—such as regulation or increased taxation—threaten their worldview or livelihood cannot provide an alternative view that is coherent by the standards of conventional scientific thinking. Instead, we suggest that people who reject the fact that the Earth’s climate is changing due to greenhouse gas emissions oppose whatever inconvenient finding they are confronting in piece-meal fashion, rather than systematically, and without considering the implications of this rejection to the rest of the relevant scientific theory and findings. Hence, claims that the globe “is cooling” can coexist with claims that the “observed warming is natural” and that “the human influence does not matter because warming is good for us.” Coherence between these mutually contradictory opinions can only be achieved at a highly abstract level, namely that “something must be wrong” with the scientific evidence in order to justify a political position against climate change mitigation. This high-level coherence accompanied by contradictory subordinate propositions is a known attribute of conspiracist ideation, and conspiracism may be implicated when people reject well-established scientific propositions. (shrink)

The 11 articles in this issue explore how people respond to climate change and other global challenges. The articles pursue three broad strands of enquiry that relate to the effects and causes of “skepticism” about climate change, the purely cognitive challenges that are posed by a complex scientific issue, and the ways in which climate change can be communicated to a wider audience. Cognitive science can contribute to understanding people's responses to global challenges in many ways, and it may also (...) contribute to implementing solutions to those problems. (shrink)

The participation of physicians in the atrocities of the Holocaust exposed vulnerabilities in medicine’s moral commitment to patients’ best interests that every health professional should recognize. Teaching about this history is challenging, as it is extremely complex and there are no common standards for what basic historical facts students in health professions training programs should learn. Nor is there guidance on how these historical facts can or should be related to contemporary ethical issues facing health professionals. To address these problems, (...) we propose a set of core historical facts about health professional involvement in the Holocaust that every student in a health professional training program should learn. We then identify three ethical lessons from the Holocaust that are pertinent today as physicians struggle to maintain their moral compass and earn the trust of patients and the public: 1) The lesson of commitment to science; maintaining balance between reason and skepticism in the search for truth, The lesson of clinical detachment; maintaining balance between necessary professional distance with a commitment to humanism and intimacy with patients, and 3) The lesson of competing loyalties; maintaining balance in upholding medicine’s multiple responsibilities, including to individual patients and the larger community. Embedding these facts and lessons into the education of health professionals is challenging yet critically important. Today’s physicians struggle with some of the same ethical tensions as did German physicians in the Nazi era, albeit in a much-attenuated fashion. Awareness of these tensions and taking active measures to maintain them in balance are necessary components of humanistic health care, which should be an integral part of health professional training programs. (shrink)

There is a robust scientific consensus concerning climate change and evolution. But many people reject these expert views, in favour of beliefs that are strongly at variance with the evidence. It is tempting to try to explain these beliefs by reference to ignorance or irrationality, but those who reject the expert view seem often to be no worse informed or any less rational than the majority of those who accept it. It is also tempting to try to explain these beliefs (...) by reference to epistemic overconfidence. However, this kind of overconfidence is apparently ubiquitous, so by itself it cannot explain the difference between those who accept and those who reject expert views. Instead, I will suggest that the difference is in important part explained by differential patterns of epistemic deference, and these patterns, in turn, are explained by the cues that we use to filter testimony. We rely on cues of benevolence and competence to distinguish reliable from unreliable testifiers, but when debates become deeply politicized, asserting a claim may itself constitute signalling lack of reliability. (shrink)

We often need to have beliefs about things on which we are not experts. Luckily, we often have access to expert judgements on such topics. But how should we form our beliefs on the basis of expert opinion when experts conflict in their judgments? This is the core of the novice/2-expert problem in social epistemology. A closely related question is important in the context of policy making: how should a policy maker use expert judgments when making policy in domains in (...) which she is not herself an expert? This question is more complex, given the messy and strategic nature of politics. In this paper we argue that the prediction with expert advice framework from machine learning provides helpful tools for addressing these problems. We outline conditions under which we should expert PWEA to be helpful and those under which we should not expect these methods to perform well. (shrink)

Psychological studies show that the beliefs of two agents in a hypothesis can diverge even if both agents receive the same evidence. This phenomenon of belief polarisation is often explained by invoking biased assimilation of evidence, where the agents’ prior views about the hypothesis affect the way they process the evidence. We suggest, using a Bayesian model, that even if such influence is excluded, belief polarisation can still arise by another mechanism. This alternative mechanism involves differential weighting of the evidence (...) arising when agents have different initial views about the reliability of their sources of evidence. We provide a systematic exploration of the conditions for belief polarisation in Bayesian models which incorporate opinions about source reliability, and we discuss some implications of our findings for the psychological literature. (shrink)

Predictable polarization is everywhere: we can often predict how people’s opinions, including our own, will shift over time. Extant theories either neglect the fact that we can predict our own polarization, or explain it through irrational mechanisms. They needn’t. Empirical studies suggest that polarization is predictable when evidence is ambiguous, that is, when the rational response is not obvious. I show how Bayesians should model such ambiguity and then prove that—assuming rational updates are those which obey the value of evidence—ambiguity (...) is necessary and sufficient for the rationality of predictable polarization. The main theoretical result is that there can be a series of such updates, each of which is individually expected to make you more accurate, but which together will predictably polarize you. Polarization results from asymmetric increases in accuracy. This mechanism is not only theoretically possible, but empirically plausible. I argue that cognitive search—searching a cognitively accessible space for a particular item—often yields asymmetrically ambiguous evidence, I present an experiment supporting its polarizing effects, and I use simulations to show how it can explain two of the core causes of polarization: confirmation bias and the group polarization effect. (shrink)

We present three experiments using a novel problem in which participants update their estimates of propensities when faced with an uncertain new instance. We examine this using two different causal structures (common cause/common effect) and two different scenarios (agent‐based/mechanical). In the first, participants must update their estimate of the propensity for two warring nations to successfully explode missiles after being told of a new explosion on the border between both nations. In the second, participants must update their estimate of the (...) accuracy of two early warning tests for cancer when they produce conflicting reports about a patient. Across both experiments, we find two modal responses, representing around one‐third of participants each. In the first, “Categorical” response, participants update propensity estimates as if they were certain about the single event, for example, certain that one of the nations was responsible for the latest explosion, or certain about which of the two tests is correct. In the second, “No change” response, participants make no update to their propensity estimates at all. Across the three experiments, the theory is developed and tested that these two responses in fact have a single representation of the problem: because the actual outcome is binary (only one of the nations could have launched the missile; the patient either has cancer or not), these participants believe it is incorrect to update propensities in a graded manner. They therefore operate on a “certainty threshold” basis, whereby, if they are certain enough about the single event, they will make the “Categorical” response, and if they are below this threshold, they will make the “No change” response. Ramifications are considered for the “categorical” response in particular, as this approach produces a positive‐feedback dynamic similar to that seen in the belief polarization/confirmation bias literature. (shrink)

Belief polarization is widely seen to threaten havoc on our shared political lives. It is often assumed that BP is the product of epistemically irrational behaviors at the individual level. After distinguishing between BP as it occurs in intra-group and inter-group settings, this paper argues that neither process necessarily reflects individual epistemic irrationality. It is true that these processes can work in tandem to produce so-called “echo chambers.” But while echo chambers are often problematic from the point of view of (...) collective rationality, it doesn't follow that individuals are doing anything wrong, epistemically speaking, in seeking them out. In non-ideal socio-epistemic contexts, echo chamber construction might provide one's best defense against systematic misinformation and deception. (shrink)

When public opinion gets polarized, the population’s beliefs can experience two different changes: they can become more extreme in their contents or they can be held with greater confidence. These two possibilities point to two different understandings of the rupture that characterizes political polarization: extremism and radicalism. In this article, I show that from the close examination of the best available evidence regarding how we get polarized, it follows that the pernicious type of political polarization has more to do with (...) radicalism than with extremism. Reinforcing the confidence in the core beliefs of the group we identify with makes our beliefs immune to the reasons coming from the other political side. Finally, I also suggest that the rise of political polarization is not necessarily the result of an irrational process. (shrink)

YouTube has been implicated in the transformation of users into extremists and conspiracy theorists. The alleged mechanism for this radicalizing process is YouTube’s recommender system, which is optimized to amplify and promote clips that users are likely to watch through to the end. YouTube optimizes for watch-through for economic reasons: people who watch a video through to the end are likely to then watch the next recommended video as well, which means that more advertisements can be served to them. This (...) is a seemingly innocuous design choice, but it has a troubling side-effect. Critics of YouTube have alleged that the recommender system tends to recommend extremist content and conspiracy theories, as such videos are especially likely to capture and keep users’ attention. To date, the problem of radicalization via the YouTube recommender system has been a matter of speculation. The current study represents the first systematic, pre-registered attempt to establish whether and to what extent the recommender system tends to promote such content. We begin by contextualizing our study in the framework of technological seduction. Next, we explain our methodology. After that, we present our results, which are consistent with the radicalization hypothesis. Finally, we discuss our findings, as well as directions for future research and recommendations for users, industry, and policy-makers. (shrink)

YouTube has been implicated in the transformation of users into extremists and conspiracy theorists. The alleged mechanism for this radicalizing process is YouTube’s recommender system, which is optimized to amplify and promote clips that users are likely to watch through to the end. YouTube optimizes for watch-through for economic reasons: people who watch a video through to the end are likely to then watch the next recommended video as well, which means that more advertisements can be served to them. This (...) is a seemingly innocuous design choice, but it has a troubling side-effect. Critics of YouTube have alleged that the recommender system tends to recommend extremist content and conspiracy theories, as such videos are especially likely to capture and keep users’ attention. To date, the problem of radicalization via the YouTube recommender system has been a matter of speculation. The current study represents the first systematic, pre-registered attempt to establish whether and to what extent the recommender system tends to promote such content. We begin by contextualizing our study in the framework of technological seduction. Next, we explain our methodology. After that, we present our results, which are consistent with the radicalization hypothesis. Finally, we discuss our findings, as well as directions for future research and recommendations for users, industry, and policy-makers. (shrink)

Theories of rational decision making often abstract away from computational and other resource limitations faced by real agents. An alternative approach known as resource rationality puts such matters front and center, grounding choice and decision in the rational use of finite resources. Anticipated by earlier work in economics and in computer science, this approach has recently seen rapid development and application in the cognitive sciences. Here, the theory of rationality plays a dual role, both as a framework for normative assessment (...) and as a source of scientific hypotheses about how mental processes in fact work. The latter project, often called rational analysis, depends for its success on a fine-grained characterization of the computational problem facing a decision maker, which may in turn depend on realistic assumptions about what the relevant agent is like. As a consequence, resource rationality involves a delicate, but often fruitful interplay between the normative and the descriptive. (shrink)

Resource rationality may explain suboptimal patterns of reasoning; but what of “anti-Bayesian” effects where the mind updates in a direction opposite the one it should? We present two phenomena — belief polarization and the size-weight illusion — that are not obviously explained by performance- or resource-based constraints, nor by the authors’ brief discussion of reference repulsion. Can resource rationality accommodate them?