Judgment Aggregation

Much of our information comes to us indirectly, in the form of conclusions others have drawn from evidence they gathered. When we hear these conclusions, how can we modify our own opinions so as to gain the benefit of their evidence? In this paper we study the method known as geometric pooling. We consider two arguments in its favour, raising several objections to one, and proposing an amendment to the other.

Judgment-aggregation theory has always focused on the attainment of rational collective judgments. But so far, rationality has been understood in static terms: as coherence of judgments at a given time, defined as consistency, completeness, and/or deductive closure. This paper asks whether collective judgments can be dynamically rational, so that they change rationally in response to new information. Formally, a judgment aggregation rule is dynamically rational with respect to a given revision operator if, whenever all individuals revise their judgments in light (...) of some information (a learnt proposition), then the new aggregate judgments are the old ones revised in light of this information, i.e., aggregation and revision commute. We prove an impossibility theorem: if the propositions on the agenda are non-trivially connected, no judgment aggregation rule with standard properties is dynamically rational with respect to any revision operator satisfying some basic conditions on revision. Our theorem is the dynamic-rationality counterpart of some well-known impossibility theorems for static rationality. We also explore how dynamic rationality might be achieved by relaxing some of the conditions on the aggregation rule and/or the revision operator. Notably, premise-based aggregation rules are dynamically rational with respect to so-called premise-based revision operators. (shrink)

Does pre-voting group deliberation increase majority competence? To address this question, we develop a probabilistic model of opinion formation and deliberation. Two new jury theorems, one pre-deliberation and one post-deliberation, suggest that deliberation is beneficial. Successful deliberation mitigates three voting failures: (1) overcounting widespread evidence, (2) neglecting evidential inequality, and (3) neglecting evidential complementarity. Simulations and theoretic arguments confirm this. But there are five systematic exceptions where deliberation reduces majority competence, always by increasing failure (1). Our analysis recommends deliberation that (...) is 'participatory', 'even', but possibly 'unequal', i.e., that involves substantive sharing, privileges no evidences, but possibly privileges some persons. (shrink)

This paper provides an introductory review of the theory of judgment aggregation. It introduces the paradoxes of majority voting that originally motivated the field, explains several key results on the impossibility of propositionwise judgment aggregation, presents a pedagogical proof of one of those results, discusses escape routes from the impossibility and relates judgment aggregation to some other salient aggregation problems, such as preference aggregation, abstract aggregation and probability aggregation. The present illustrative rather than exhaustive review is intended to give readers (...) new to the field of judgment aggregation a sense of this rapidly growing research area. (shrink)

How should your opinion change in light of an epistemic peer's? We show that the pooling rule known as “upco” is the unique answer satisfying some natural desiderata. If your revised opinion will impact your other views by Jeffrey conditionalization, then upco is the only standard pooling rule that ensures the order in which peers are consulted makes no difference. Popular alternatives like linear pooling, geometric pooling, and harmonic pooling cannot boast the same. In fact, no alternative can that possesses (...) four minimal properties these proposals all share. (shrink)

In the first half of this paper, I argue that group belief ascriptions are highly ambiguous. What's more, in many cases, neither the available contextual factors nor known pragmatic considerations are sufficient to allow the audience to identify which of the many possible meanings is intended. In the second half, I argue that this ambiguity often has bad consequences when a group belief ascription is heard and taken as testimony. And indeed it has these consequences even when the ascription is (...) true on the speaker's intended interpretation, when the speaker does not intend to mislead and indeed intends to cooperatively inform, and when the audience incorporates the evidence from the testimony as they should. I conclude by arguing that these consequences should lead us to stop using such ascriptions. (shrink)

We often learn the opinions of others without hearing the evidence on which they're based. The orthodox Bayesian response is to treat the reported opinion as evidence itself and update on it by conditionalizing. But sometimes this isn't feasible. In these situations, a simpler way of combining one's existing opinion with opinions reported by others would be useful, especially if it yields the same results as conditionalization. We will show that one method---upco, also known as multiplicative pooling---is specially suited to (...) this role when the opinions you wish to pool concern hypotheses about chances. The result has interesting consequences: it addresses the problem of disagreement between experts; and it sheds light on the social argument for the uniqueness thesis. (shrink)

The current literature standardly conceives of voting power in terms of decisiveness: the ability to change the voting outcome by unilaterally changing one’s vote. I argue that this classic conception of voting power, which fails to account for partial decisiveness or efficacy, produces erroneous results because it saddles the concept of voting power with implausible microfoundations. This failure in the measure of voting power in turn reflects a philosophical mistake about the concept of social power in general: a failure to (...) recognize that an agent can exercise individual social power with others’ assistance, in virtue of the group’s collective power, sometimes even when she could not unilaterally scuttle the group’s collective power. I therefore develop a conception of efficacy that admits of degrees and defend a Recursive Measure of voting power that takes partial efficacy into account. (shrink)

This essay replies to Michael Morreau and Erik J. Olsson’s ‘Learning from Ranters: The Effect of Information Resistance on the Epistemic Quality of Social Network Deliberation’. Morreau and Olsson use simulations to suggest that false ranters—agents who do not update their beliefs and only ever assert false claims—do not diminish the epistemic value of deliberation for other agents and can even be epistemically valuable. They argue conclude that “Our study suggests that including [false] ranters has little or no negative effect (...) on the epistemic value of social deliberation. Including them can even be epistemically beneficial. -/- I present concerns about their model and I enumerate some epistemic values that their discussion omits. I first raise two interrelated worries: (i.) The model is too simple; real-life irresponsible assertors do not speak uniformly falsely, for example. And tracking testifiers’ reliability demands cognitive resources. (ii.) In reality, we do not treat a person’s assertions that p as evidence that not p, especially not for all their assertions. I argue that these simplifications threat Morreau and Olsson conclusion. -/- I then enumerate epistemic values and disvalues that Morreau and Olsson do not discuss, but which bear on the value of false assertions and false ranters in their simulation and in real life. Candidate disvalues include: False beliefs of ranters; cognitive costs of discerning and recalling testifiers’ reliability; distrust; epistemic polarisatin; a prevalence of false assertion can impugn all testimony, including that of reliable testifiers. And candidate values include: a prevalence of true beliefs and assertions; attention being directed towards the right things; well-functioning trust relationships; believing people’s assertions; healthy discussion; epistemic respect; learning together. -/- . (shrink)

Multidimensional concepts are everywhere, and they are important. Examples include moral value, welfare, scientific confirmation, democracy, and biodiversity. How, if at all, can we aggregate the underlying dimensions of a multidimensional concept F to yield verdicts about which things are Fer than which overall? Social choice theory can be used to model and investigate this aggregation problem. Here, we focus on a particularly thorny problem made salient by this social choice-theoretic framework: the underlying dimensions of a given concept might be (...) measurable on different types of scales—e.g., some ordinal and some cardinal. An underappreciated impossibility theorem due to Anna Khmelnitskaya shows that seemingly plausible constraints on aggregation across scale types are inconsistent. This impossibility threatens to render the notion of overall Fness incoherent. We attempt to defuse this threat, arguing that the impossibility depends on an overly restrictive conception of measurement and of how measurement constrains aggregation. Adopting a more flexible—and, we think, more perspicuous—conception of measurement opens an array of possibilities for aggregation across disparate scale types. (shrink)

Models of collective deliberation often assume that the chief aim of a deliberative exchange is the sharing of information. In this paper, we argue that an equally important role of deliberation is to draw participants’ attention to pertinent questions, which can aid the assembly and processing of distributed information by drawing deliberators’ attention to new issues. The assumption of logical omniscience renders classical models of agents’ informational states unsuitable for modelling this role of deliberation. Building on recent insights from psychology, (...) linguistics and philosophy about the role of questions in speech and thought, we propose a different model in which beliefs are treated as answers directed at specific questions. Here, questions are formally represented as partitions of the space of possibilities and individuals’ information states as sets of questions and corresponding partial answers to them. The state of conversation is then characterised by individuals’ information together with the questions under discussion, which can be steered by various deliberative inputs. Using this model, deliberation is then shown to shape collective decisions in ways that classical models cannot capture, allowing for novel explanations of how group consensus is achieved. (shrink)

Public reason liberals expect individuals to have justificatory reasons for their views of certain political issues. This paper considers how groups can, and whether they should, give collective public reasons for their political decisions. A problem is that aggregating individuals’ consistent judgments on reasons and a decision can produce inconsistent collective judgments. The group will then fail to give a reason for its decision. The paper considers various solutions to this problem and defends a deliberative procedure by showing how it (...) incentivizes information sharing and leads to outcomes most acceptable to the group members. (shrink)

This chapter briefly reviews the present state of judgment aggregation theory and tentatively suggests a future direction for that theory. In the review, we start by emphasizing the difference between the doctrinal paradox and the discursive dilemma, two idealized examples which classically serve to motivate the theory, and then proceed to reconstruct it as a brand of logical theory, unlike in some other interpretations, using a single impossibility theorem as a key to its technical development. In the prospective part, having (...) mentioned existing applications to social choice theory and computer science, which we do not discuss here, we consider a potential application to law and economics. This would be based on a deeper exploration of the doctrinal paradox and its relevance to the functioning of collegiate courts. On this topic, legal theorists have provided empirical observations and theoretical hints that judgment aggregation theorists would be in a position to clarify and further elaborate. As a general message, the chapter means to suggest that the future of judgment aggregation theory lies with its applications rather than its internal theoretical development. (shrink)

The purpose of this paper is to illustrate, formally, an ambiguity in the exercise of political influence. To wit: A voter might exert influence with an eye toward maximizing the probability that the political system (1) obtains the correct (e.g. just) outcome, or (2) obtains the outcome that he judges to be correct (just). And these are two very different things. A variant of Condorcet's Jury Theorem which incorporates the effect of influence on group competence and interdependence is developed. Analytic (...) and numerical results are obtained, the most important of which is that it is never optimal--from the point-of-view of collective accuracy--for a voter to exert influence without limit. He ought to either refrain from influencing other voters or else exert a finite amount of influence, depending on circumstance. Philosophical lessons are drawn from the model, to include a solution to Wollheim's "Paradox in the Theory of Democracy". (shrink)

Procedural justifications of democracy emphasize inclusiveness and respect and by doing so come into conflict with instrumental justifications that depend on voters’ competence. This conflict raises questions about jury theorems and makes their standing in democratic theory contested. We show that a type of no-regret learning called meta-induction can help to satisfy the competence assumption without excluding voters or diverse opinion leaders on an a priori basis. Meta-induction assigns weights to opinion leaders based on their past predictive performance to determine (...) the level of their inclusion in recommendations for voters. The weighting minimizes the difference between the performance of meta-induction and the best opinion leader in hindsight. The difference represents the regret of meta-induction whose minimization ensures that the recommendations are optimal in supporting voters’ competence. Meta-induction has optimal truth-tracking properties that support voters’ competence even if it is targeted by mis/disinformation and should be considered a tool for supporting democracy in hyper-plurality. (shrink)

Majority rule is often regarded as an important democratic principle. But modern democracies divide voters into districts. So if the majority should rule, which majority should rule? Should it be the popular majority, or an electoral majority (i.e., either the majority of voters in the majority of districts, or the majority of voters in districts that contain the majority of the population)? I argue that majority rule requires rule by the popular majority. This view is not novel and may seem (...) obvious. But it is rarely defended, and often rejected. I defend it by arguing that this view about which majority should rule is best supported by the arguments for why the majority should rule, and best avoids the arbitrary and perverse verdicts of rule by electoral majorities. I close by considering the significant practical implications of the view. The Electoral College is often criticized on the basis that the winner of the popular vote doesn’t always win the election. But all district-based democracies have this feature, so all district-based democracies must be reformed if they are to satisfy the principle of majority rule. (shrink)

Supra-Bayesianism is the Bayesian response to learning the opinions of others. Probability pooling constitutes an alternative response. One natural question is whether there are cases where probability pooling gives the supra-Bayesian result. This has been called the problem of Bayes-compatibility for pooling functions. It is known that in a common prior setting, under standard assumptions, linear pooling cannot be nontrivially Bayes-compatible. We show by contrast that geometric pooling can be nontrivially Bayes-compatible. Indeed, we show that, under certain assumptions, geometric and (...) Bayes-compatible pooling are equivalent. Granting supra-Bayesianism its usual normative status, one upshot of our study is thus that, in a certain class of epistemic contexts, geometric pooling enjoys a normative advantage over linear pooling as a social learning mechanism. We discuss the philosophical ramifications of this advantage, which we show to be robust to variations in our statement of the Bayes-compatibility problem. (shrink)

There have been many discussions recently from philosophers, cognitive scientists, and psychologists about group polarization, with online and social media environments in particular receiving a lot of attention, both because of people's increasing reliance on such environments for receiving and exchanging information and because such environments often allow individuals to selectively interact with those who are like-minded. My goal here is to argue that the group epistemologist can facilitate understanding the kinds of factors that drive group polarization in a way (...) that has been overlooked by the existing research. Specifically, I argue that polarization can occur in part because of the ways that members of a group treat the group itself (as opposed to an individual member within that group) as a source of information, and in doing so makes their own position, as well as that of the group, more extreme. I refer to this as a structural factor in driving polarization, and argue that such factors can contribute to explanations of polarization that occurs specifically in an important subset of groups, namely online and social media groups in which little is known about other members within the group, what I refer to as anonymous, semi-anonymous and pseudonymous (ASAP) groups. (shrink)

We tend to applaud those who think for themselves: the ever-curious student, for example, or the grownup who does their own research. Even as we’re applauding, however, we ourselves often don’t think for ourselves. This book argues that’s completely OK. -/- In fact, it’s often best just to take other folks’ word for it, allowing them to do the hard work of gathering and evaluating the relevant evidence. In making this argument, philosopher Jonathan Matheson shows how 'expert testimony' and 'the (...) wisdom of crowds' are tested and provides convincing ideas that make it rational to believe something simply because other people believe it. Matheson then takes on philosophy’s best arguments against his thesis, including the idea that non-self-thinkers are free-riding on the work of others, Socrates’ claim that 'the unexamined life isn’t worth living,' and that outsourcing your intellectual labor makes you vulnerable to errors and manipulation. Matheson shows how these claims and others ultimately fail -- and that when it comes to thinking, we often need not be sheepish about being sheep. (shrink)

This paper is about a problem that can arise when we try to harness the “wisdom of the crowd” from groups comprised of individuals who exhibit a certain kind of epistemic humility in the way they respond to testimonial evidence. I begin by setting out the problem and then make some initial steps toward solving it. The solution I develop is tentative and may not apply in all circumstances, but it promises to alleviate what seems to me to be a (...) problem of both theoretical interest and practical importance. (shrink)

We study the logic of so-called lexicographic or priority merge for multi-agent plausibility models. We start with a systematic comparison between the logical behavior of priority merge and the more standard notion of pooling through intersection, used to define, for instance, distributed knowledge. We then provide a sound and complete axiomatization of the logic of priority merge, as well as a proof theory in labeled sequents that admits cut. We finally study Moorean phenomena and define a dynamic resolution operator for (...) priority merge for which we also provide a complete set of reduction axioms. (shrink)

Many epistemologists have argued that there is some degree of independence between individual and collective reliability (e.g., Kitcher 1990; Mayo-Wilson, Zollman, and Danks 2011; Dunn 2018). The question, then, is: To what extent are the two independent of each other? And in which contexts do they come apart? In this paper, I present a new case confirming the independence between individual and collective reliability optimization. I argue that, in voting groups, optimizing individual reliability can conflict with optimizing collective reliability. This (...) can happen even if various conditions are held constant, such as: the evidence jurors have access to, the voting system, the number of jurors, some independence conditions between voters, and so forth. This observation matters in many active debates on, e.g., epistemic dilemmas, the wisdom of crowds, independence theses, epistemic democracy, and the division of epistemic labour. (shrink)

Objectives To explore how factors relating to grades and grading affect the correctness of choices that grant-review panels make among submitted proposals. To identify interventions in panel design that may be expected to increase the correctness of choices. -/- Method Experimentation with an empirically-calibrated computer simulation model of panel review. Model parameters are set in accordance with procedures at a national science funding agency. Correctness of choices among research proposals is operationalized as agreement with the choices of an elite panel. (...) -/- Conclusions The simulation model generates several hypotheses to guide further research. Increasing the number of grades used by panel members increases the correctness of simulated choices among submitted proposals. Collective decision procedures giving panels a greater capacity for discriminating among proposals also increase correctness. Surprisingly, differences in grading standards among panel members do not appreciably decrease correctness. (shrink)

There are many reasons we might want to take the opinions of various individuals and pool them to give the opinions of the group they constitute. If all the individuals in the group have probabilistic opinions about the same propositions, there is a host of pooling functions we might deploy, such as linear or geometric pooling. However, there are also cases where different members of the group assign probabilities to different sets of propositions, which might overlap a lot, a little, (...) or not at all. There are far fewer proposals for how to proceed in these cases, and those there are have undesirable features. I begin by considering four proposals and arguing that they don't work. Then I'll describe my own proposal, which is intended to cover the situation in which we want to pool the individual opinions in order to ascribe an opinion to the group considered as an agent in its own right. (shrink)

In this paper, we argue that computer simulations can provide valuable insights into the performance of voting methods on different collective decision problems. This could improve institutional design, even when there is no general theoretical result to support the optimality of a voting method. To support our claim, we first describe a decision problem that has not received much theoretical attention in the literature. We outline different voting methods to address that collective decision problem. Under certain criteria of assessment akin (...) to extensions of the Condorcet Jury Theorem, we run simulations for the methods using MATLAB, in order to compare their performance under various conditions. We consider and respond to concerns about the use of simulations in the assessment of voting procedures for policymaking. (shrink)

Jury theorems are mathematical theorems about the ability of collectives to make correct decisions. Several jury theorems carry the optimistic message that, in suitable circumstances, ‘crowds are wise’: many individuals together (using, for instance, majority voting) tend to make good decisions, outperforming fewer or just one individual. Jury theorems form the technical core of epistemic arguments for democracy, and provide probabilistic tools for reasoning about the epistemic quality of collective decisions. The popularity of jury theorems spans across various disciplines such (...) as economics, political science, philosophy, and computer science. This entry reviews and critically assesses a variety of jury theorems. It first discusses Condorcet's initial jury theorem, and then progressively introduces jury theorems with more appropriate premises and conclusions. It explains the philosophical foundations, and relates jury theorems to diversity, deliberation, shared evidence, shared perspectives, and other phenomena. It finally connects jury theorems to their historical background and to democratic theory, social epistemology, and social choice theory. (shrink)

There are many things—call them ‘experts’—that you should defer to in forming your opinions. The trouble is, many experts are modest: they’re less than certain that they are worthy of deference. When this happens, the standard theories of deference break down: the most popular (“Reflection”-style) principles collapse to inconsistency, while their most popular (“New-Reflection”-style) variants allow you to defer to someone while regarding them as an anti-expert. We propose a middle way: deferring to someone involves preferring to make any decision (...) using their opinions instead of your own. In a slogan, deferring opinions is deferring decisions. Generalizing the proposal of Dorst (2020a), we first formulate a new principle that shows exactly how your opinions must relate to an expert’s for this to be so. We then build off the results of Levinstein (2019) and Campbell-Moore (2020) to show that this principle is also equivalent to the constraint that you must always expect the expert’s estimates to be more accurate than your own. Finally, we characterize the conditions an expert’s opinions must meet to be worthy of deference in this sense, showing how they sit naturally between the too-strong constraints of Reflection and the too-weak constraints of New Reflection. (shrink)

When members of a group doxastically disagree with each other, decisions in the group are often hard to make. The members are supposed to find an epistemic compromise. How do members of a group reach a rational epistemic compromise on a proposition when they have different (rational) credences in the proposition? I answer the question by suggesting the Fine-Grained Method of Aggregation, which is introduced in Brössel and Eder 2014 and is further developed here. I show how this method faces (...) challenges of the standard method of aggregation, Weighted Straight Averaging, in a successful way. One of the challenges concerns the fact that Weighted Straight Averaging does not respect the evidential states of agents. Another challenge arises because Weighted Straight Averaging does not account for synergetic effects. (shrink)

It is often suggested that when opinions differ among individuals in a group, the opinions should be aggregated to form a compromise. This paper compares two approaches to aggregating opinions, linear pooling and what I call opinion agglomeration. In evaluating both strategies, I propose a pragmatic criterion, No Regrets, entailing that an aggregation strategy should prevent groups from buying and selling bets on events at prices regretted by their members. I show that only opinion agglomeration is able to satisfy the (...) demand. I then proceed to give normative and empirical arguments in support of the pragmatic criterion for opinion aggregation, and that ultimately favor opinion agglomeration. (shrink)

Peer reviewers at many funding agencies and scientific journals are asked to score submissions both on individual criteria and overall. The overall scores should be some kind of aggregate of the criteria scores. Carole Lee identifies this as a potential locus for bias to enter the peer review process, which she calls commensuration bias. Here I view the aggregation of scores through the lens of social choice theory. I argue that, when reviewing grant proposals, it is in many cases impossible (...) to avoid commensuration bias. (shrink)

Frente a problemas de decisión colectiva de cierta complejidad, distintos métodos de votación pueden considerarse igualmente democráticos. Ante esta situación, argumento que es posible investigar cuáles de esos métodos producen mejores resultados epistémicos sobre asuntos fácticos. Comienzo ilustrando la relación entre democracia y métodos de votación con un sencillo ejemplo. Muestro cómo el uso de modelos idealizados permite descubrir algunas propiedades de los métodos de votación; varios de estos descubrimientos muestran que, frente a problemas de cierta complejidad, no hay una (...) respuesta clara acerca de cuál es el resultado de una elección democrática. Frente a esto, sugiero que deberíamos tomar en cuenta un rasgo epistémico instrumental de varios métodos de votación: su capacidad para generar respuestas correctas ante varias situaciones. Esta intuición ofrece lecciones importantes para el diseño de instituciones electorales. (shrink)

One of the best known approaches to the logic of agency are the ‘stit’ logics. Often, it is not the actions of an individual agent that bring about a certain outcome but the joint actions of a set of agents, collectively. Collective agency has received comparatively little attention in ‘stit’. The paper maps out several different forms, several different senses in which a particular set of agents, collectively, can be said to bring about a certain outcome, and examines how these (...) forms can be expressed in ‘stit’ and stit-like logics. The outcome that is brought about may be unintentional, and perhaps even accidental; the account deliberately ignores aspects such as joint intention, communication between agents, awareness of other agents’ intentions and capabilities, even the awareness of another agent’s existence. The aim is to investigate what can be said about collective agency when all such considerations are ignored, besides mere consequences of joint actions. The account will be related to the ‘strictly stit’ of Belnap and Perloff, 25–48 1993) and their suggestions concerning ‘inessential members’ and ‘mere bystanders’. We will adjust some of those conjectures and distinguish further between ‘potentially contributing bystanders’ and ‘impotent bystanders’. (shrink)

Our aim in this chapter is to draw attention to what we see as a disturbing feature of conciliationist views of disagreement. Roughly put, the trouble is that conciliatory responses to in-group disagreement can lead to the frustration of a group's epistemic priorities: that is, the group's favoured trade-off between the "Jamesian goals" of truth-seeking and error-avoidance. We show how this problem can arise within a simple belief aggregation framework, and draw some general lessons about when the problem is most (...) pronounced. We close with a tentative proposal for how to solve the problem raised without rejecting conciliationism. (shrink)

We analyse the incentives of individuals to misrepresent their truthful judgments when engaged in collective decision-making. Our focus is on scenarios in which individuals reason about the incentives of others before choosing which judgments to report themselves. To this end, we introduce a formal model of strategic behaviour in logic-based judgment aggregation that accounts for such higher-level reasoning as well as the fact that individuals may only have partial information about the truthful judgments and preferences of their peers. We find (...) that every aggregation rule must belong to exactly one of three possible categories: it is either (i) immune to strategic manipulation for every level of reasoning, or (ii) manipulable for every level of reasoning, or (iii) immune to manipulation only for everykth level of reasoning, for some natural number kgreater than 1. (shrink)

Epistemically immodest agents take their own epistemic standards to be among the most truth-conducive ones available to them. Many philosophers have argued that immodesty is epistemically required of agents, notably because being modest entails a problematic kind of incoherence or self-distrust. In this paper, I argue that modesty is epistemically permitted in some social contexts. I focus on social contexts where agents with limited cognitive capacities cooperate with each other (like juries).

We give a review and critique of jury theorems from a social-epistemology perspective, covering Condorcet’s (1785) classic theorem and several later refinements and departures. We assess the plausibility of the conclusions and premises featuring in jury theorems and evaluate the potential of such theorems to serve as formal arguments for the ‘wisdom of crowds’. In particular, we argue (i) that there is a fundamental tension between voters’ independence and voters’ competence, hence between the two premises of most jury theorems; (ii) (...) that the (asymptotic) conclusion that ‘huge groups are infallible’, reached by many jury theorems, is an artifact of unjustified premises; and (iii) that the (nonasymptotic) conclusion that ‘larger groups are more reliable’, also reached by many jury theorems, is not an artifact and should be regarded as the more adequate formal rendition of the ‘wisdom of crowds’. (shrink)

Self-authorship is fundamental to respecting the dignity of persons, and epistemic credibility depends upon impartial review. While these claims may seem obviously true, they arguments for them are rarely given. In a supposedly "post-truth" world in which respect for individual rights is under attack, the obvious must be argued for and reiterated. To that end, I mine sources from the European Enlightenment (Bacon, Hume, Kant, and Mill) to make the case for self-authorship and impartial review.

Scholars belong to multiple communities of credit simultaneously. When these communities disagree about a scholarly achievement’s credit assignment, this raises a puzzle for decision and game theor...

Panels, boards, and committees throughout society evaluate all manner of things by grading them, first individually and then collectively. Thus risks are prioritized, research proposals are funded, and candidates are shortlisted for jobs. It is not usual to pick winners in political elections by grading the candidates, but there are examples in history. This article takes up a question about the quality of judgments and decisions made by grading: under which conditions are they likely to be right? An answer comes (...) in the form of a jury theorem for median grading. Here, the collective grade for a thing is the median of its individually assigned grades—the one in the middle, when all of them are listed from "top" to "bottom." A second objective of this article is to suggest a solution to problems of voter ignorance in democracies. The idea is for democratic assemblies to use voting methods that make more of people's limited knowledge than do commonly used methods, such as majority voting. It turns out that in theory anyway, and perhaps also in practice, median grading can enable unenlightened assemblies to “track the truth”—even as majority voting would run them off the rails. (shrink)

The method of supergrading is introduced for deriving a ranking of items from scores or grades awarded by several people. Individual inputs may come in different languages of grades. Diversity in grading standards is an advantage, enabling rankings derived by this method to separate more items from one another. A framework is introduced for studying grading on the basis of observations. Measures of accuracy, reliability and discrimination are developed within this framework. Ability in grading is characterized for individuals and groups (...) as the capacity to grade reliably, accurately and at a high level of discrimination. It is shown that the collective ability of a supergrading group with diverse standards can be greater than that of a less diverse group whose members have greater ability. (shrink)

Sometimes citizens disagree about political matters, but a decision must be made. We have two theoretical frameworks for resolving political disagreement. The first is the framework of social choice. In it, our goal is to treat parties to the dispute fairly, and there is no sense in which some are right and the others wrong. The second framework is that of collective decision-making. Here, we do believe that preferences are truth apt, and our moral consideration is owed not to those (...) who disagree but to the community that stands to benefit or suffer from the decision. In this chapter, I describe and explore these two frameworks. I conclude two things. First, analysis of real-world disagreement suggests that collective decision-making is the right way to model politics. In most, possibly even all, political disagreement, all parties believe (if implicitly) that there is an objective standard of correctness. Second, this matter is connected to the concept of pluralism. If pluralism is true, then collective decision-making cannot be applied to some political disagreements. More surprisingly, pluralism may rule out the applicability of social choice theory as well. (shrink)

Policymakers who seek to make scientifically informed decisions are constantly confronted by scientific uncertainty and expert disagreement. This thesis asks: how can policymakers rationally respond to expert disagreement and scientific uncertainty? This is a work of non-ideal theory, which applies formal philosophical tools developed by ideal theorists to more realistic cases of policymaking under scientific uncertainty. I start with Bayesian approaches to expert testimony and the problem of expert disagreement, arguing that two popular approaches— supra-Bayesianism and the standard model of (...) expert deference—are insufficient. I develop a novel model of expert deference and show how it can deal with many of these problems raised for them. I then turn to opinion pooling, a popular method for dealing with disagreement. I show that various theoretical motivations for pooling functions are irrelevant to realistic policymaking cases. This leads to a cautious recommendation of linear pooling. However, I then show that any pooling method relies on value judgements, that are hidden in the selection of the scoring rule. My focus then narrows to a more specific case of scientific uncertainty: multiple models of the same system. I introduce a particular case study involving hurricane models developed to support insurance decision-making. I recapitulate my analysis of opinion pooling in the context of model ensembles, confirming that my hesitations apply. This motivates a shift of perspective, to viewing the problem as a decision theoretic one. I rework a recently developed ambiguity theory, called the confidence approach, to take input from model ensembles. I show how it facilitates the resolution of the policymaker’s problem in a way that avoids the issues encountered in previous chapters. This concludes my main study of the problem of expert disagreement. In the final chapter, I turn to methodological reflection. I argue that philosophers who employ the mathematical methods of the prior chapters are modelling. Employing results from the philosophy of scientific models, I develop the theory of normative modelling. I argue that it has important methodological conclusions for the practice of formal epistemology, ruling out popular moves such as searching for counterexamples. (shrink)

Group beliefs in social networks are often construed as arising from individual beliefs through processes of update and aggregation. In this paper, we explore an alternative ‘arational’ perspective. More specifically, we focus on group attitudes as neutral tendencies toward alignment of opinions driven by influence patterns among agents modeled in a Markov dynamics. In addition, we investigate logical patterns in the resulting potential group beliefs or, in more neutral arational terminology: collective opinion structures.

This paper is about how to aggregate outside opinion. If two experts are on one side of an issue, while three experts are on the other side, what should a non-expert believe? Certainly, the non-expert should take into account more than just the numbers. But which other factors are relevant, and why? According to the view developed here, one important factor is whether the experts should have been expected, in advance, to reach the same conclusion. When the agreement of two (...) (or of twenty) thinkers can be predicted with certainty in advance, their shared belief is worth only as much as one of their beliefs would be worth alone. This expectational model of belief dependence can be applied whether we think in terms of credences or in terms of all-or-nothing beliefs. (shrink)

While social epistemologists have recently begun addressing questions about whether groups can possess beliefs or knowledge, little has yet been said about whether groups can properly be said to possess understanding. Here I want to make some progress on this question by considering two possible accounts of group understanding, modeled on accounts of group belief and knowledge: a deflationary account, according to which a group understands just in case most or all of its members understand, and an inflationary account, according (...) to which a group’s understanding does not depend solely on whether its members understand. I argue that both accounts face problems. The deflationary account has two such problems: aggregation problems that are familiar from discussions of group belief, and the problem of different bases, wherein members possess understanding for different but consistent reasons. The inflationary account faces what I call the problem of distributed grasping: while it is widely accepted that understanding requires a kind of “grasping”, it is hard to make sense of how this requirement could be met at the group level while not necessarily being met by any individual member. Despite its problems, I make a case for the inflationary account. This will require addressing the problem of distributed grasping: to do this, I propose a different way of thinking about the grasping relation at the group level, such that it is constituted by a dependency relationship between members. (shrink)

Review of the volume "Scientific Collaboration and Collective Knowledge: New Essays", edited by Thomas Boyer-Kassem, Conor Mayo-Wilson, and Michael Weisberg.

Social scientists use many different methods, and there are often substantial disagreements about which method is appropriate for a given research question. In response to this uncertainty about the relative merits of different methods, W. E. B. Du Bois advocated for and applied “methodological triangulation”. This is to use multiple methods simultaneously in the belief that, where one is uncertain about the reliability of any given method, if multiple methods yield the same answer that answer is confirmed more strongly than (...) it could have been by any single method. Against this, methodological purists believe that one should choose a single appropriate method and stick with it. Using tools from voting theory, we show Du Boisian methodological triangulation to be more likely to yield the correct answer than purism, assuming the scientist is subject to some degree of diffidence about the relative merits of the various methods. This holds even when in fact only one of the methods is appropriate for the given research question. (shrink)

Various social epistemologists employ what seem to be rather distinct notions of group rationality. In this essay, I offer an account of group rationality that is able to unify the dominant notions present in the literature under a single framework. I argue that if we employ a teleological account of epistemic rationality, and then allow that there are many different epistemic goals that are worth pursuing for various groups and individuals, we can then see how those seemingly divergent understandings of (...) group rationality are all intimately related. I close by showing how the view has the additional benefit of allowing us to generate practical, normative suggestions for groups in the real world. (shrink)