Formal Social Epistemology, Misc

Peer review is often taken to be the main form of quality control on academic research. Usually journals carry this out. However, parts of maths and physics appear to have a parallel, crowd-sourced model of peer review, where papers are posted on the arXiv to be publicly discussed. In this paper we argue that crowd-sourced peer review is likely to do better than journal-solicited peer review at sorting papers by quality. Our argument rests on two key claims. First, crowd-sourced peer (...) review will lead on average to more reviewers per paper than journal-solicited peer review. Second, due to the wisdom of the crowds, more reviewers will tend to make better judgments than fewer. We make the second claim precise by looking at the Condorcet Jury Theorem as well as two related jury theorems developed specifically to apply to peer review. (shrink)

This paper investigates how the credit incentive to engage in questionable research practices interacts with cumulative advantage, the process whereby high-status academics more easily increase their status than low-status academics. I use a mathematical model to highlight two dynamics that have not yet received much attention. First, due to cumulative advantage, questionable research practices may pay off over the course of an academic career even if they are not attractive at the level of individual publications. Second, because of the role (...) of bottleneck moments in academic careers, questionable research practices may be selected for even if they do not provide a benefit in expectation. I also observe that, within the model, the most successful academics are the most likely to have benefited from fraud. (shrink)

In this paper, we explain and showcase the promising methodology of testimonial network analysis and visualization for experimental epistemology, arguing that it can be used to gain insights and answer philosophical questions in social epistemology. Our use case is the epistemic community that discusses vaccine safety primarily in English on Twitter. In two studies, we show, using both statistical analysis and exploratory data visualization, that there is almost no neutral or ambivalent discussion of vaccine safety on Twitter. Roughly half the (...) accounts engaging with this topic are pro-vaccine, while the other half are con-vaccine. We also show that these two camps rarely engage with one another, and that the con-vaccine camp has greater epistemic reach and receptivity than the pro-vaccine camp. In light of these findings, we question whether testimonial networks as they are currently constituted on popular fora such as Twitter are living up to their promise of delivering the wisdom of crowds. We conclude by pointing to directions for further research in digital social epistemology. (shrink)

Supra-Bayesianism is the Bayesian response to learning the opinions of others. Probability pooling constitutes an alternative response. One natural question is whether there are cases where probability pooling gives the supra-Bayesian result. This has been called the problem of Bayes-compatibility for pooling functions. It is known that in a common prior setting, under standard assumptions, linear pooling cannot be nontrivially Bayes-compatible. We show by contrast that geometric pooling can be nontrivially Bayes-compatible. Indeed, we show that, under certain assumptions, geometric and (...) Bayes-compatible pooling are equivalent. Granting supra-Bayesianism its usual normative status, one upshot of our study is thus that, in a certain class of epistemic contexts, geometric pooling enjoys a normative advantage over linear pooling as a social learning mechanism. We discuss the philosophical ramifications of this advantage, which we show to be robust to variations in our statement of the Bayes-compatibility problem. (shrink)

What differentiates scientific research from non-scientific inquiry? Philosophers addressing this question have typically been inspired by the exalted social place and intellectual achievements of science. They have hence tended to point to some epistemic virtue or methodological feature of science that sets it apart. Our discussion on the other hand is motivated by the case of commercial research, which we argue is distinct from (and often epistemically inferior to) academic research. We consider a deflationary view in which science refers to (...) whatever is regarded as epistemically successful, but find that this does not leave room for the important notion of scientific error and fails to capture distinctive social elements of science. This leads us to the view that a demarcation criterion should be a widely upheld social norm without immediate epistemic connotations. Our tentative answer is the communist norm, which calls on scientists to share their work widely for public scrutiny and evaluation. (shrink)

Predictable polarization is everywhere: we can often predict how people’s opinions, including our own, will shift over time. Extant theories either neglect the fact that we can predict our own polarization, or explain it through irrational mechanisms. They needn’t. Empirical studies suggest that polarization is predictable when evidence is ambiguous, that is, when the rational response is not obvious. I show how Bayesians should model such ambiguity and then prove that—assuming rational updates are those which obey the value of evidence—ambiguity (...) is necessary and sufficient for the rationality of predictable polarization. The main theoretical result is that there can be a series of such updates, each of which is individually expected to make you more accurate, but which together will predictably polarize you. Polarization results from asymmetric increases in accuracy. This mechanism is not only theoretically possible, but empirically plausible. I argue that cognitive search—searching a cognitively accessible space for a particular item—often yields asymmetrically ambiguous evidence, I present an experiment supporting its polarizing effects, and I use simulations to show how it can explain two of the core causes of polarization: confirmation bias and the group polarization effect. (shrink)

Several philosophers and psychologists have characterized belief in conspiracy theories as a product of irrational reasoning. Proponents of conspiracy theories apparently resist revising their beliefs given disconfirming evidence and tend to believe in more than one conspiracy, even when the relevant beliefs are mutually inconsistent. In this paper, we bring leading views on conspiracy theoretic beliefs closer together by exploring their rationality under a probabilistic framework. We question the claim that the irrationality of conspiracy theoretic beliefs stems from an inadequate (...) response to disconfirming evidence and internal incoherence. Drawing analogies to Lakatosian research programs, we argue that maintaining a core conspiracy belief can be Bayes-rational when it is embedded in a network of auxiliary beliefs, which can be revised to protect the more central belief from disconfirmation. We propose that the (ir)rationality associated with conspiracy belief lies not in a flawed updating method, but in their failure to converge toward well-confirmed, stable belief networks in the long run. This approach not only reconciles previously disjointed views, but also points toward more specific hypotheses explaining why some agents may be prone to adopting beliefs in conspiracy theories. (shrink)

Theoretical arguments and empirical investigations indicate that a high proportion of published findings do not replicate and are likely false. The current position paper provides a broad perspective on scientific error, which may lead to replication failures. This broad perspective focuses on reform history and on opportunities for future reform. We organize our perspective along four main themes: institutional reform, methodological reform, statistical reform and publishing reform. For each theme, we illustrate potential errors by narrating the story of a fictional (...) researcher during the research cycle. We discuss future opportunities for reform. The resulting agenda provides a resource to usher in an era that is marked by a research culture that is less error-prone and a scientific publication landscape with fewer spurious findings. (shrink)

There are many reasons we might want to take the opinions of various individuals and pool them to give the opinions of the group they constitute. If all the individuals in the group have probabilistic opinions about the same propositions, there is a host of pooling functions we might deploy, such as linear or geometric pooling. However, there are also cases where different members of the group assign probabilities to different sets of propositions, which might overlap a lot, a little, (...) or not at all. There are far fewer proposals for how to proceed in these cases, and those there are have undesirable features. I begin by considering four proposals and arguing that they don't work. Then I'll describe my own proposal, which is intended to cover the situation in which we want to pool the individual opinions in order to ascribe an opinion to the group considered as an agent in its own right. (shrink)

We argue that the appraisal of models in social epistemology requires conceiving of them as argumentative devices, taking into account the argumentative context and adopting a family-of-models perspective. We draw up such an account and show how it makes it easier to see the value and limits of the use of models in social epistemology. To illustrate our points, we document and explicate the argumentative role of epistemic landscape models in social epistemology and highlight their limitations. We also claim that (...) our account could be fruitfully used in appraising other models in philosophy and science. (shrink)

Around 97% of climate scientists endorse anthropogenic global warming (AGW), the theory that human activities are partly responsible for recent increases in global average temperatures. Clearly, this widespread endorsement of AGW is a reason for non-experts to believe in AGW. But what is the epistemic significance of the fact that some climate scientists do not endorse AGW? This paper contrasts expert unanimity, in which virtually no expert disagrees with some theory, with expert consensus, in which some non-negligible proportion either rejects (...) or is uncertain about the theory. It is argued that, from a layperson’s point of view, an expert consensus is often stronger evidence for a theory’s truth than unanimity. Several lessons are drawn from this conclusion, e.g. concerning what laypeople should infer from expert pronouncements, how journalists should report on scientific theories, and how working scientists should communicate with the public. (shrink)

When members of a group doxastically disagree with each other, decisions in the group are often hard to make. The members are supposed to find an epistemic compromise. How do members of a group reach a rational epistemic compromise on a proposition when they have different (rational) credences in the proposition? I answer the question by suggesting the Fine-Grained Method of Aggregation, which is introduced in Brössel and Eder 2014 and is further developed here. I show how this method faces (...) challenges of the standard method of aggregation, Weighted Straight Averaging, in a successful way. One of the challenges concerns the fact that Weighted Straight Averaging does not respect the evidential states of agents. Another challenge arises because Weighted Straight Averaging does not account for synergetic effects. (shrink)

Peer reviewers at many funding agencies and scientific journals are asked to score submissions both on individual criteria and overall. The overall scores should be some kind of aggregate of the criteria scores. Carole Lee identifies this as a potential locus for bias to enter the peer review process, which she calls commensuration bias. Here I view the aggregation of scores through the lens of social choice theory. I argue that, when reviewing grant proposals, it is in many cases impossible (...) to avoid commensuration bias. (shrink)

Prepublication peer review should be abolished. We consider the effects that such a change will have on the social structure of science, paying particular attention to the changed incentive structure and the likely effects on the behaviour of individual scientists. We evaluate these changes from the perspective of epistemic consequentialism. We find that where the effects of abolishing prepublication peer review can be evaluated with a reasonable level of confidence based on presently available evidence, they are either positive or neutral. (...) We conclude that on present evidence abolishing peer review weakly dominates the status quo. (shrink)

Evidential Uniqueness is the thesis that, for any batch of evidence, there’s a unique doxastic state that a subject with that evidence should have. One of the most common kinds of objections to views that violate Evidential Uniqueness are arbitrariness objections – objections to the effect that views that don’t satisfy Evidential Uniqueness lead to unacceptable arbitrariness. The goal of this paper is to examine a variety of arbitrariness objections that have appeared in the literature, and to assess the extent (...) to which these objections bolster the case for Evidential Uniqueness. After examining a number of different arbitrariness objections, I’ll conclude that, by and large, these objections do little to bolster the case for Evidential Uniqueness. (shrink)

For two ideally rational agents, does learning a finite amount of shared evidence necessitate agreement? No. But does it at least guard against belief polarization, the case in which their opinions get further apart? No. OK, but are rational agents guaranteed to avoid polarization if they have access to an infinite, increasing stream of shared evidence? No.

People don't always speak the truth. When they don't, we do better not to trust them. Unfortunately, that's often easier said than done. People don't usually wear a ‘Not to be trusted!’ badge on their sleeves, which lights up every time they depart from the truth. Given this, what can we do to figure out whom to trust, and whom not? My aim in this paper is to offer a partial answer to this question. I propose a heuristic—the “Humility Heuristic”—which (...) is meant to help guide our search for trustworthy advisors. In slogan form, the heuristic says: people worth trusting admit to what they don't know. I give this heuristic a precise probabilistic interpretation, offer a simple argument for it, defend it against some potential worries, and demonstrate its practical worth by showing how it can help address some difficult challenges in the relationship between experts and laypeople. (shrink)

This paper introduces a generic agent-based model simulating the exchange and the diffusion of pro and con arguments. It is applied to the case of the diffusion of vegetarian diets in the context of a potential emergence of a second nutrition transition. To this day, agent-based simulation has been extensively used to study opinion dynamics. However, the vast majority of existing models have been limited to extremely abstract and simplified representations of the diffusion process. These simplifications impairs the realism of (...) the simulations and disables the understanding of the reasons for the shift of an actor's opinion. The generic model presented here explicitly represents exchanges of arguments between actors in the context of an opinion dynamic model. In particular, the inner attitude towards an opinion of each agent is formalized as an argumentation graph and each agent can share arguments with other agents. Simulation experiments show that introducing attacks between arguments and a limitation of the number of arguments mobilized by agents has a strong impact on the evolution of the agents' opinion. We also highlight that when a new argument is introduced into the system, the quantity and the profile of the agents receiving the new argument will impact the evolution of the overall opinion. Finally, the application of this model to vegetarian diet adoption seems consistent with historical food behaviour dynamics observed during crises. (shrink)

Scientific research that could cause grave harm, either through accident or intentional malevolence, is known as dual-use research. Recent high-profile cases of dual-use research in the life sciences have led to debate about the extent to which restrictions on the conduct and dissemination of such research may impede scientific progress. We adapt formal models of scientific networks to systematically explore the effects that different regulatory schemes may have on a community’s ability to learn about the world. Our results suggest that, (...) contrary to common wisdom, some restrictions on the conduct and dissemination of dual-use research do not inhibit scientific progress and may actually aid communities in learning. (shrink)

For millennia, knowledge has eluded a precise definition. The industrialization of knowledge (IoK) and the associated proliferation of the so-called knowledge communities in the last few decades caused this state of affairs to deteriorate, namely by creating a trio composed of data, knowledge, and information (DIK) that is not unlike the aporia of the trinity in philosophy. This calls for a general theory of knowledge (ToK) that can work as a foundation for a science of knowledge (SoK) and additionally distinguishes (...) knowledge from both data and information. In this paper, I attempt to sketch this generality via the establishing of both knowledge structures and knowledge systems that can then be adopted/adapted by the diverse communities for the respective knowledge technologies and practices. This is achieved by means of a formal–indeed, mathematical–approach to epistemological matters a.k.a. formal epistemology. The corresponding application focus is on knowledge systems implementable as computer programs. (shrink)

How should we adjust our beliefs in light of the testimony of those who are in a better epistemic position than ourselves, such as experts and other epistemic superiors? In this paper, I develop and defend a deference model of epistemic authority. The paper attempts to resolve the debate between the preemption view and the total evidence view of epistemic authority by taking an accuracy-first approach to the issue of how we should respond to authoritative and expert testimony. I argue (...) that when we look at the debate through the lens of accuracy, it becomes clear that matters are more complicated than either the preemption view or the total evidence view are able to account for. Consequently, a deference model, outlined within a credence-based framework, does a better job of capturing the relevant phenomena, and explaining how we should update our beliefs in response to epistemically authoritative testimony. (shrink)

Introduction to the Synthese Special Issue on Disagreement in Science.

Scholars belong to multiple communities of credit simultaneously. When these communities disagree about a scholarly achievement’s credit assignment, this raises a puzzle for decision and game theor...

Testimonial evidence that is particularly helpful to confirmation combines agreement on some content with variation of detail. I examine the phenomenon of “undesigned coincidences” from a probabilistic point of view to explain how varied reports, including those that dovetail in detail, assist confirmation of an hypothesis. The formal analysis uses recent work in probability theory surrounding the concepts of dependence, independence, and varied evidence. I also discuss the connection between these types of report connections and an hypothesis about the reliability (...) of the sources involved. (shrink)

I present a puzzle about the epistemic role beliefs about experts' beliefs play in a rational agent's system of beliefs. It is shown that accepting the claim that an expert's degree of belief in a proposition, A, screens off the evidential support another proposition, B, gives to A in case the expert knows and is certain about whether B is true, leads in some cases to highly unintuitive conclusions. I suggest a solution to the puzzle according to which evidential screening (...) off is rejected, but show that the price of this solution is either giving up on the mere idea of deferring to expert's opinion or giving up on Bayesian conditionalization. (shrink)

Recent impossibility theorems for fair risk assessment extend to the domain of epistemic justice. We translate the relevant model, demonstrating that the problems of fair risk assessment and just credibility assessment are structurally the same. We motivate the fairness criteria involved in the theorems as also being appropriate in the setting of testimonial justice. Any account of testimonial justice that implies the fairness/justice criteria must be abandoned, on pain of triviality.

Recent studies of scientific interaction based on agent-based models suggest that a crucial factor conducive to efficient inquiry is what Zollman has dubbed ‘transient diversity’. It signifies a process in which a community engages in parallel exploration of rivaling theories lasting sufficiently long for the community to identify the best theory and to converge on it. But what exactly generates transient diversity? And is transient diversity a decisive factor when it comes to the efficiency of inquiry? In this paper we (...) examine the impact of different conditions on the efficiency of inquiry, as well as the relation between diversity and efficiency. This includes certain diversity-generating mechanisms previously proposed in the literature, as well as some factors that have so far been neglected. This study is obtained via an argumentation-based ABM. Our results suggest that cautious decision-making does not always have a significant impact on the efficiency of inquiry while different evaluations underlying theory-choice and different social networks do. Moreover, we find a correlation between diversity and a successful performance of agents only under specific conditions, which indicates that transient diversity is sometimes not the primary factor responsible for efficiency. Altogether, when comparing our results to those obtained by structurally different ABMs based on Zollman’s work, the impact of specific factors on efficiency of inquiry, as well as the role of transient diversity in achieving efficiency, appear to be highly dependent on the underlying model. (shrink)

For over a decade now epistemologists have been thinking about the peer disagreement problem of whether a person is reasonable in not lowering her confidence in her belief P when she comes to accept that she has an epistemic peer on P who disbelieves P. However, epistemologists have overlooked a key realistic way how epistemic peers can, or even have to, differ epistemically—a way that reveals the inadequacy of both conformist and non-conformist views on peer disagreement by uncovering how the (...) causes of peer disagreement bear on the debate’s core philosophical issue. Part of our argument for this thesis will involve giving a thorough yet entirely informal presentation of mathematical theorems in economics by Robert Aumann :1236–1239,1976) and Polemarchakis and Geneakoplos which represent a formally precise description of how two rational agents must deal with disagreement under certain epistemically interesting circumstances. (shrink)

Suppose one hundred prisoners are in a yard under the supervision of a guard, and at some point, ninety-nine of them collectively kill the guard. If, after the fact, a prisoner is picked at random and tried, the probability of his guilt is 99%. But despite the high probability, the statistical chances, by themselves, seem insufficient to justify a conviction. The question is why. Two arguments are offered. The first, decision-theoretic argument shows that a conviction solely based on the statistics (...) in the prisoner scenario is unacceptable so long as the goal of expected utility maximization is combined with fairness constraints. The second, risk-based argument shows that a conviction solely based on the statistics in the prisoner scenario lets the risk of mistaken conviction surge potentially too high. The same, by contrast, cannot be said of convictions solely based on DNA evidence or eyewitness testimony. A noteworthy feature of the two arguments in the paper is that they are not confined to criminal trials and can in fact be extended to civil trials. (shrink)

In everyday life and in science we acquire evidence of evidence and based on this new evidence we often change our epistemic states. An assumption underlying such practice is that the following EEE Slogan is correct: 'evidence of evidence is evidence' (Feldman 2007, p. 208). We suggest that evidence of evidence is best understood as higher-order evidence about the epistemic state of agents. In order to model evidence of evidence we introduce a new powerful framework for modelling epistemic states, Dyadic (...) Bayesianism. Based on this framework, we then discuss characterizations of evidence of evidence and argue for one of them. Finally, we show that whether the EEE Slogan holds, depends on the specific kind of evidence of evidence. (shrink)

Review of the volume "Scientific Collaboration and Collective Knowledge: New Essays", edited by Thomas Boyer-Kassem, Conor Mayo-Wilson, and Michael Weisberg.

There is a commonly made distinction between two types of scientists: risk-taking, trailblazing mavericks and detail-oriented followers. A number of recent papers have discussed the question what a desirable mixture of mavericks and followers looks like. Answering this question is most useful if a scientific community can be steered toward such a desirable mixture. One attractive route is through credit incentives: manipulating rewards so that reward-seeking scientists are likely to form the desired mixture of their own accord. Here I argue (...) that this idea is less straightforward than it may seem. Interpreting mavericks as scientists who prioritize rewards over speed and risk, I show in a deliberatively simple model that there is a fixed mixture which is not particularly likely to be desirable and which credit incentives cannot alter. I consider a way around this result, but this has some major drawbacks. I conclude that credit incentives are not as promising a way to create a desirable mixture of mavericks and followers as one might have thought. (shrink)

This paper offers a new angle on the common idea that the process of science does not support epistemic diversity. Under minimal assumptions on the nature of journal editing, we prove that editorial procedures, even when impartial in themselves, disadvantage less prominent research programs. This purely statistical bias in article selection further skews existing differences in the success rate and hence attractiveness of research programs, and exacerbates the reputation difference between the programs. After a discussion of the modeling assumptions, the (...) paper ends with a number of recommendations that may help promote scientific diversity through editorial decision making. (shrink)

We often ask for the opinion of a group of individuals. How strongly does the scientific community believe that the rate at which sea levels are rising has increased over the last 200 years? How likely does the UK Treasury think it is that there will be a recession if the country leaves the European Union? What are these group credences that such questions request? And how do they relate to the individual credences assigned by the members of the particular (...) group in question? According to the credal judgement aggregation principle, linear pooling, the credence function of a group should be a weighted average or linear pool of the credence functions of the individuals in the group. In this chapter, I give an argument for linear pooling based on considerations of accuracy. And I respond to two standard objections to the aggregation principle. (shrink)

We motivate a picture of social epistemology that sees forgetting as subject to epistemic evaluation. Using computer simulations of a simple agent-based model, we show that how agents forget can have as large an impact on group epistemic outcomes as how they share information. But, how we forget, unlike how we form beliefs, isn’t typically taken to be the sort of thing that can be epistemically rational or justified. We consider what we take to be the most promising argument for (...) this claim and find it lacking. We conclude that understanding how agents forget should be as central to social epistemology as understanding how agents form beliefs and share information with others. (shrink)

Spatially situated opinions that can be held with different degrees of conviction lead to spatiotemporal patterns such as clustering (homophily), polarization, and deadlock. Our goal is to understand how sensitive these patterns are to changes in the local nature of interactions. We introduce two different mixing mechanisms, spatial relocation and nonlocal interaction (“telephoning”), to an earlier fully spatial model (no mixing). Interestingly, the mechanisms that create deadlock in the fully spatial model have the opposite effect when there is a sufficient (...) amount of mixing. With telephoning, not only is polarization and deadlock broken up, but consensus is hastened. The effects of mixing by relocation are even more pronounced. Further insight into these dynamics is obtained for selected parameter regimes via comparison to the mean-field differential equations. (shrink)

This paper addresses the problem of judgment aggregation in science. How should scientists decide which propositions to assert in a collaborative document? We distinguish the question of what to write in a collaborative document from the question of collective belief. We argue that recent objections to the application of the formal literature on judgment aggregation to the problem of judgment aggregation in science apply to the latter, not the former question. The formal literature has introduced various desiderata for an aggregation (...) procedure. Proposition-wise majority voting emerges as a procedure that satisfies all desiderata which represent norms of science. An interesting consequence is that not all collaborating scientists need to endorse every proposition asserted in a collaborative document. (shrink)

It has been argued that an epistemically rational agent’s evidence is subjectively mediated through some rational epistemic standards, and that there are incompatible but equally rational epistemic standards available to agents. This supports Permissiveness, the view according to which one or multiple fully rational agents are permitted to take distinct incompatible doxastic attitudes towards P (relative to a body of evidence). In this paper, I argue that the above claims entail the existence of a unique and more reliable epistemic standard. (...) My strategy relies on Condorcet’s Jury Theorem. This gives rise to an important problem for those who argue that epistemic standards are permissive, since the reliability criterion is incompatible with such a type of Permissiveness. (shrink)

It is often suggested that disagreement among scientific experts is a reason not to trust those experts, even about matters on which they are in agreement. In direct opposition to this view, I argue here that the very fact that there is disagreement among experts on a given issue provides a positive reason for non-experts to trust that the experts really are justified in their attitudes towards consensus theories. I show how this line of thought can be spelled out in (...) three distinct frameworks for non-deductive reasoning: namely, Bayesian Confirmation Theory, Inference to the Best Explanation, and Inferential Robustness Analysis. (shrink)

According to an influential Enlightenment ideal, one shouldn't rely epistemically on other people's say-so, at least not if one is in a position to evaluate the relevant evidence for oneself. However, in much recent work in social epistemology, we are urged to dispense with this ideal, which is seen as stemming from a misguided focus on isolated individuals to the exclusion of groups and communities. In this paper, I argue that that an emphasis on the social nature of inquiry should (...) not lead us to entirely abandon the Enlightenment ideal of epistemically autonomous agents. Specifically, I suggest that it is an appropriate ideal for those who serve as experts in a given epistemic community, and develop a notion of expert acceptance to make sense of this. I go on to show that, all other things being equal, this kind of epistemic autonomy among experts makes their joint testimony more reliable, which in turn brings epistemic benefits both to laypeople and to experts in other fields. (shrink)

There are various ways to reach a group decision on a factual yes–no question. One way is to vote and decide what the majority votes for. This procedure receives some epistemological support from the Condorcet Jury Theorem. Alternatively, the group members may prefer to deliberate and will eventually reach a decision that everybody endorses—a consensus. While the latter procedure has the advantage that it makes everybody happy, it has the disadvantage that it is difficult to implement, especially for larger groups. (...) Besides, the resulting consensus may be far away from the truth. And so we ask: Is deliberation truth-conducive in the sense that majority voting is? To address this question, we construct a highly idealized model of a particular deliberation process, inspired by the movie Twelve Angry Men, and show that the answer is ‘yes’. Deliberation procedures can be truth-conducive just as the voting procedure is. We then explore, again on the basis of our model and using agent-based simulations, under which conditions it is better epistemically to deliberate than to vote. Our analysis shows that there are contexts in which deliberation is epistemically preferable and we will provide reasons for why this is so. (shrink)

Should editors of scientific journals practice triple-anonymous reviewing? I consider two arguments in favor. The first says that insofar as editors’ decisions are affected by information they would not have had under triple-anonymous review, an injustice is committed against certain authors. I show that even well-meaning editors would commit this wrong and I endorse this argument. The second argument says that insofar as editors’ decisions are affected by information they would not have had under triple-anonymous review, it will negatively affect (...) the quality of published papers. I distinguish between two kinds of biases that an editor might have. I show that one of them has a positive effect on quality and the other a negative one, and that the combined effect could be either positive or negative. Thus I do not endorse the second argument in general. However, I do endorse this argument for certain fields, for which I argue that the positive effect does not apply. (shrink)

Recent philosophical work has praised the reward structure of science, while recent empirical work has shown that many scientific results may not be reproducible. I argue that the reward structure of science incentivizes scientists to focus on speed and impact at the expense of the reproducibility of their work, thus contributing to the so-called reproducibility crisis. I use a rational choice model to identify a set of sufficient conditions for this problem to arise, and I argue that these conditions plausibly (...) apply to a wide range of research situations. Currently proposed solutions will not fully address this problem. Philosophical commentators should temper their optimism about the reward structure of science. (shrink)

Recent empirical work has shown that many scientific results may not be reproducible. By itself, this does not entail that there is a problem. However, I argue that there is a problem: the reward structure of science incentivizes scientists to focus on speed and impact at the expense of the reproducibility of their work. I illustrate this using a well-known failure of reproducibility: Fleischmann and Pons' work on cold fusion. I then use a rational choice model to identify a set (...) of sufficient conditions for this problem to arise, and I argue that these conditions plausibly apply to a wide range of research situations. In the conclusion I consider possible solutions and implications for how Fleischmann and Pons' work should be evaluated. (shrink)

Weisberg and Muldoon’s epistemic landscape model (ELM) has been one of the most significant contributions to the use of agent-based models in philosophy. The model provides an innovative approach to establishing the optimal distribution of cognitive labor in scientific communities, using an epistemic landscape. In the paper, we provide a critical examination of ELM. First, we show that the computing mechanism for ELM is correct insofar as we are able to replicate the results using another programming language. Second, we show (...) that small changes in the rules that determine the behavior of individual agents can lead to important changes in simulation results. Accordingly, we claim that ELM results are robust with respect to the computing mechanism, but not necessarily across parameter space. We conclude by reflecting on the possible lessons to be gained from ELM as a class of simulations or cluster of models. (shrink)

While many philosophers have agreed that evidence of disagreement is a kind of higher-order evidence, this has not yet resulted in formally precise higher-order approaches to the problem of disagreement. In this paper, we outline a simple formal framework for determining the epistemic significance of a body of higher-order evidence, and use this framework to motivate a novel interpretation of the popular “equal weight view” of peer disagreement—we call it the Variably Equal Weight View (VEW). We show that VEW differs (...) from the standard Split the Difference (SD) interpretation of the equal weight view in almost all cases of peer disagreement, and use our formal framework to explain why SD has seemed attractive but is in fact misguided. A desirable feature of VEW, we argue, is that it gives rise to plausible instances of synergy—an effect whereby the parties to a disagreement should become more (or less) confident in the disputed proposition than any of them were prior to disagreement. Lastly, we show how VEW may be generalized to cases of non-peer disagreement. (shrink)

Merging of opinions results underwrite Bayesian rejoinders to complaints about the subjective nature of personal probability. Such results establish that sufficiently similar priors achieve consensus in the long run when fed the same increasing stream of evidence. Initial subjectivity, the line goes, is of mere transient significance, giving way to intersubjective agreement eventually. Here, we establish a merging result for sets of probability measures that are updated by Jeffrey conditioning. This generalizes a number of different merging results in the literature. (...) We also show that such sets converge to a shared, maximally informed opinion. Convergence to a maximally informed opinion is a (weak) Jeffrey conditioning analogue of Bayesian “convergence to the truth” for conditional probabilities. Finally, we demonstrate the philosophical significance of our study by detailing applications to the topics of dynamic coherence, imprecise probabilities, and probabilistic opinion pooling. (shrink)

The question of how the probabilistic opinions of different individuals should be aggregated to form a group opinion is controversial. But one assumption seems to be pretty much common ground: for a group of Bayesians, the representation of group opinion should itself be a unique probability distribution, 410–414, [45]; Bordley Management Science, 28, 1137–1148, [5]; Genest et al. The Annals of Statistics, 487–501, [21]; Genest and Zidek Statistical Science, 114–135, [23]; Mongin Journal of Economic Theory, 66, 313–351, [46]; Clemen and (...) Winkler Risk Analysis, 19, 187–203, [7]; Dietrich and List [14]; Herzberg Theory and Decision, 1–19, [28]). We argue that this assumption is not always in order. We show how to extend the canonical mathematical framework for pooling to cover pooling with imprecise probabilities by employing set-valued pooling functions and generalizing common pooling axioms accordingly. As a proof of concept, we then show that one IP construction satisfies a number of central pooling axioms that are not jointly satisfied by any of the standard pooling recipes on pain of triviality. Following Levi, 3–11, [39]), we also argue that IP models admit of a much better philosophical motivation as a model of rational consensus. (shrink)

Recent years have seen a surge in online collaboration between experts and amateurs on scientific research. In this article, we analyse the epistemological implications of these crowdsourced projects, with a focus on Zooniverse, the world’s largest citizen science web portal. We use quantitative methods to evaluate the platform’s success in producing large volumes of observation statements and high impact scientific discoveries relative to more conventional means of data processing. Through empirical evidence, Bayesian reasoning, and conceptual analysis, we show how information (...) and communication technologies enhance the reliability, scalability, and connectivity of crowdsourced e-research, giving online citizen science projects powerful epistemic advantages over more traditional modes of scientific investigation. These results highlight the essential role played by technologically mediated social interaction in contemporary knowledge production. We conclude by calling for an explicitly sociotechnical turn in the philosophy of science that combines insights from statistics and logic to analyse the latest developments in scientific research. (shrink)

I argue elsewhere (Roche 2014) that evidence of evidence is evidence under screening-off. Tal and Comesaña (2017) argue that my appeal to screening-off is subject to two objections. They then propose an evidence of evidence thesis involving the notion of a defeater. There is much to learn from their very careful discussion. I argue, though, that their objections fail and that their evidence of evidence thesis is open to counterexample.