Foundation models such as GPT-4 are fine-tuned to avoid unsafe or otherwise problematic behavior, so that, for example, they refuse to comply with requests for help with committing crimes or with producing racist text. One approach to fine-tuning, called reinforcement learning from human feedback, learns from humans' expressed preferences over multiple outputs. Another approach is constitutional AI, in which the input from humans is a list of high-level principles. But how do we deal with potentially diverging input from humans? How (...) can we aggregate the input into consistent data about "collective" preferences or otherwise use it to make collective choices about model behavior? In this paper, we argue that the field of social choice is well positioned to address these questions, and we discuss ways forward for this agenda, drawing on discussions in a recent workshop on Social Choice for AI Ethics and Safety held in Berkeley, CA, USA in December 2023. (shrink)

A platitude that took hold with Kuhn is that there can be several equally good ways of balancing theoretical virtues for theory choice. Okasha recently modelled theory choice using technical apparatus from the domain of social choice: famously, Arrow showed that no method of social choice can jointly satisfy four desiderata, and each of the desiderata in social choice has an analogue in theory choice. Okasha suggested that one can avoid the Arrow analogue for theory choice by (...) employing a strategy used by Sen in social choice, namely, to enhance the information made available to the choice algorithms. I argue here that, despite Okasha’s claims to the contrary, the information-enhancing strategy is not compelling in the domain of theory choice. (shrink)

Sometimes citizens disagree about political matters, but a decision must be made. We have two theoretical frameworks for resolving political disagreement. The first is the framework of social choice. In it, our goal is to treat parties to the dispute fairly, and there is no sense in which some are right and the others wrong. The second framework is that of collective decision-making. Here, we do believe that preferences are truth apt, and our moral consideration is owed not to (...) those who disagree but to the community that stands to benefit or suffer from the decision. In this chapter, I describe and explore these two frameworks. I conclude two things. First, analysis of real-world disagreement suggests that collective decision-making is the right way to model politics. In most, possibly even all, political disagreement, all parties believe (if implicitly) that there is an objective standard of correctness. Second, this matter is connected to the concept of pluralism. If pluralism is true, then collective decision-making cannot be applied to some political disagreements. More surprisingly, pluralism may rule out the applicability of social choice theory as well. (shrink)

This book offers a comprehensive overview and critique of the most important political and philosophical interpretations of the basic results of social choice, assessing their plausibility and seeking to identify the links between the theory of social choice and the more traditional issues of political theory and philosophy. In this regard, the author eschews a strong methodological commitment or technical formalism; the approach is instead based on the presentation of political facts and illustrated via numerous real-life examples. This (...) allows the reader to get acquainted with the philosophical and political dispute surrounding voting and collective decision-making and its links to social choice theory. (shrink)

Social choice theory is the study of collective decision processes and procedures. It is not a single theory, but a cluster of models and results concerning the aggregation of individual inputs (e.g., votes, preferences, judgments, welfare) into collective outputs (e.g., collective decisions, preferences, judgments, welfare). Central questions are: How can a group of individuals choose a winning outcome (e.g., policy, electoral candidate) from a given set of options? What are the properties of different voting systems? When is a voting (...) system democratic? How can a collective (e.g., electorate, legislature, collegial court, expert panel, or committee) arrive at coherent collective preferences or judgments on some issues, on the basis of its members' individual preferences or judgments? How can we rank different social alternatives in an order of social welfare? Social choice theorists study these questions not just by looking at examples, but by developing general models and proving theorems. (shrink)

The two most influential traditions of contemporary theorizing about democracy, social choice theory and deliberative democracy, are generally thought to be at loggerheads, in that the former demonstrates the impossibility, instability or meaninglessness of the rational collective outcomes sought by the latter. We argue that the two traditions can be reconciled. After expounding the central Arrow and Gibbard-Satterthwaite impossibility results, we reassess their implications, identifying the conditions under which meaningful democratic decision making is possible. We argue that deliberation can (...) promote these conditions, and hence that social choice theory suggests not that democratic decision making is impossible, but rather that democracy must have a deliberative aspect. (shrink)

A major approach to the ethics of artificial intelligence is to use social choice, in which the AI is designed to act according to the aggregate views of society. This is found in the AI ethics of “coherent extrapolated volition” and “bottom–up ethics”. This paper shows that the normative basis of AI social choice ethics is weak due to the fact that there is no one single aggregate ethical view of society. Instead, the design of social choice (...) AI faces three sets of decisions: standing, concerning whose ethics views are included; measurement, concerning how their views are identified; and aggregation, concerning how individual views are combined to a single view that will guide AI behavior. These decisions must be made up front in the initial AI design—designers cannot “let the AI figure it out”. Each set of decisions poses difficult ethical dilemmas with major consequences for AI behavior, with some decision options yielding pathological or even catastrophic results. Furthermore, non-social choice ethics face similar issues, such as whether to count future generations or the AI itself. These issues can be more important than the question of whether or not to use social choice ethics. Attention should focus on these issues, not on social choice. (shrink)

Sports tournaments provide a procedure for producing a champion and ranking the contestants based on game results. As such, tournaments mirror aggregation methods in social choice theory, where diverse individual preferences are put together to form an overall social preference. This connection allows us a novel way of conceptualizing sports tournaments, their results, and significance. I argue that there are genuine intransitive dominance relationships in sports, that social choice theory provides a framework for understanding rankings in such (...) situations and that these considerations provide a new reason to endorse championship pluralism. (shrink)

In democracies citizens are supposed to have some control over the general direction of policy. According to a pretheoretical interpretation of this idea, the people have control if elections and other democratic institutions compel officials to do what the people want, or what the majority want. This interpretation of popular control fits uncomfortably with insights from social choice theory; some commentators—Riker, most famously—have argued that these insights should make us abandon the idea of popular rule as traditionally understood. This (...) article presents a formal theory of popular control that responds to the challenge from social choice theory. It makes precise a sense in which majorities may be said to have control even if the majority preference relation has an empty core. And it presents a simple game-theoretic model to illustrate how majorities can exercise control in this specified sense, even when incumbents are engaged in purely re-distributive policymaking and the majority rule core is empty. (shrink)

Amartya Sen has recently suggested that certain issues which arise in the application of the capability approach can be seen in terms of social choice. This article explores certain connections and tensions between Kenneth Arrow's celebrated discussion of social choice and the capability approach while focusing on one central link: pluralism. Given the variety of values people hold, substantive issues which arise in the application of the capability approach can be seen as social choice problems. Seeing them (...) in this way helps to explain some of Sen's suggestions about applying the approach in the light of an analogue of Arrow's theorem. However, it also poses a potential problem because of the focus on preferences in social choice theory, given that the capability approach is motivated in part by problems which `adaptive preferences' raise for `utility'-based views. In this article, it is argued that Sen's writings about public reasoning allow him to address this problem to some degree. The reading underlying this argument clarifies issues about the relationship between the individual and society in his approach. It also illuminates the extent of Sen's debt to John Rawls's writings on `public reason', while clarifying some points on which Sen and Rawls diverge. Key Words: social choice • capability • welfare • democracy. (shrink)

Arrow’s impossibility result stems chiefly from a combination of two requirements: independence and fixity. Independence says that the social choice is independent of individual preferences involving unavailable alternatives. Fixity says that the social choice is fixed by a social preference relation that is independent of what is available. Arrow found that requiring, further, that this relation be transitive yields impossibility. Here it is shown that allowing intransitive social indifference still permits only a vastly unsatisfactory system, a (...) liberum veto oligarchy. Arrow’s argument for independence, though, undermines any rationale for fixity. (shrink)

Most political liberals argue that only rules, policies and institutions that are part of society’s basic structure need to be justified with so-called public reasons. Laws enacted outside this set are legitimate if and when public reasons can justify the procedure that selects them. I argue that this view is susceptible to known problems from social choice theory. However, there are resources within political liberalism that could address them. If the scope of public reason is extended beyond the basic (...) structure it could order people’s preferences in a way that circumvents the identified issues. (shrink)

This article presents an approach to how to make reasonable social choices when independent criteria (e.g., prioritarianism, religious freedom) fail to fully determine what to do. The article outlines different explanations of why independent criteria sometimes fail to fully determine what to do and illustrates how they can still be used to eliminate ineligible alternatives, but it is argued that the independent criteria cannot ground a reasonable social choice in these situations. To complement independent criteria when they fail (...) to fully determine what to do, it is suggested that society must engage in public deliberation by way of generating new reasons that can determine how to rank the alternatives. It is suggested that the approach to social choice presented here reveals a way of accepting the relevance of independent criteria for social choice without letting go of the idea that the attitudes of affected parties matter. (shrink)

Extensive social choice theory is used to study the problem of measuring group fitness in a two-level biological hierarchy. Both fixed and variable group size are considered. Axioms are identified that imply that the group measure satisfies a form of consequentialism in which group fitness only depends on the viabilities and fecundities of the individuals at the lower level in the hierarchy. This kind of consequentialism can take account of the group fitness advantages of germ-soma specialization, which is not (...) possible with an alternative social choice framework proposed by Okasha, but which is an essential feature of the index of group fitness for a multicellular organism introduced by Michod, Viossat, Solari, Hurand, and Nedelcu to analyze the unicellular-multicellular evolutionary transition. The new framework is also used to analyze the fitness decoupling between levels that takes place during an evolutionary transition. (shrink)

Sen’s “The Impossibility of a Paretian Liberal” was meant to crystallize his fundamental criticism against the welfaristic basis of welfare economics in general, and social choice theory in particular. This paper vindicates Sen’s criticism, arguing that its logical relevance is not lost in light of recent criticisms against his method of articulating individual rights in terms of a person’s decisive power in social choice. We show that some recent proposals that Sen’s articulation failed to capture a strong libertarian (...) tradition of free contract and that an appropriate formulation of this tradition wipes off the Sen impossibility cannot be sustained. We then show that the game form articulation of rights casts serious doubts on the Sen articulation of liberty, but the Sen impossibility strenuously comes back in the context of realizing conferred game form rights as well as in the context of initial conferment of game form rights. (shrink)

This textbook provides a survey of the literature of social choice. It integrates the ethical aspects of the subject, with positive aspects of decision mechanisms that centre on the revelation of true preferences. The literature on the subject presently consists of a great many papers. This book draws them together in common notation and points out interpretations which are often missing in specialist papers. Applications in economics, electoral politics, and ethics are discussed. The book will be used by senior (...) undergraduate and graduate students of economics, political science and philosophy as a text book in the subject. (shrink)

Kirjasymposio: Social Choice and Democratic Values – kirjan esittely.

It has become accepted that social choice is impossible in the absence of interpersonal comparisons of well-being. This view is challenged here. Arrow obtained an impossibility theorem only by making unreasonable demands on social choice functions. With reasonable requirements, one can get very attractive possibilities and derive social preferences on the basis of non-comparable individual preferences. This new approach makes it possible to design optimal second-best institutions inspired by principles of fairness, while traditionally the analysis of optimal (...) second-best institutions was thought to require interpersonal comparisons of well-being. In particular, this approach turns out to be especially suitable for the application of recent philosophical theories of justice formulated in terms of fairness, such as equality of resources. (shrink)

The problem of merging several ontologies has important applications in the Semantic Web, medical ontology engineering and other domains where information from several distinct sources needs to be integrated in a coherent manner.We propose to view ontology merging as a problem of social choice, i.e. as a problem of aggregating the input of a set of individuals into an adequate collective decision. That is, we propose to view ontology merging as ontology aggregation. As a first step in this direction, (...) we formulate several desirable properties for ontology aggregators, we identify the incompatibility of some of these properties, and we define and analyse several simple aggregation procedures. Our approach is closely related to work in judgment aggregation, but with the crucial difference that we adopt an open world assumption, by distinguishing between facts not included in an agent’s ontology and facts explicitly negated in an agent’s ontology. (shrink)

In normative political theory, it is widely accepted that democracy cannot be reduced to voting alone, but that it requires deliberation. In formal social choice theory, by contrast, the study of democracy has focused primarily on the aggregation of individual opinions into collective decisions, typically through voting. While the literature on deliberation has an optimistic flavour, the literature on social choice is more mixed. It is centred around several paradoxes and impossibility results identifying conflicts between different intuitively plausible (...) desiderata. In recent years, there has been a growing dialogue between the two literatures. This paper discusses the connections between them. Important insights are that (i) deliberation can complement aggregation and open up an escape route from some of its negative results; and (ii) the formal models of social choice theory can shed light on some aspects of deliberation, such as the nature of deliberation-induced opinion change. (shrink)

This paper distinguishes an index ordering and a social ordering function as a simple way to formalize the indexing problem in the social choice framework. Two main conclusions are derived. First, the alleged dilemma between welfarism and perfectionism is shown to involve a third possibility, exemplified by the fairness approach to social choice. Second, the idea that an individual is better off than another whenever he has more (goods, functionings, etc.) in all dimensions, which is known to (...) enter in conflict with the Pareto principle, can be partly salvaged by adopting the fairness approach. (shrink)

This article provides a systematic analysis of social choice theory without the Pareto principle, by revisiting the method of Murakami Yasusuke. This article consists of two parts. The first part investigates the relationship between rationality of social preference and the axioms that make a collective choice rule either Paretian or anti-Paretian. In the second part, the results in the first part are applied to obtain impossibility results under various rationality requirements of social preference, such as S-consistency, quasi-transitivity, (...) semi-transitivity, the interval-order property, and acyclicity. (shrink)

Social choice theory and theories of deliberative discourse have deeply impacted on the way political scientists understand the dynamics of democratic politics and decision-making. _Deliberation, Social Choice and Absolutist Democracy_ addresses the dispute between these competing schools of thought. Deliberative democracy and social choice theorists offer the two dominant and competing conceptions of participation in contemporary democratic theory. With the former holding that theories of discourse tell us that through the democratic process we can arrive at consensus, (...) rational outcomes and even principles of justice, while the latter suggest that fair and equal participation is more likely to lead to instability and irrational outcomes. With an in-depth examination of social choice theory and deliberative democracy, David van Mill: presents two case studies on the American Continental Congress 1774-1789 provides an assessment of the types of institutions that will promote radical democracy and create stable outcomes with the minimum sacrifice of the freedom and equality of participants defends a more radical idea of absolutist democracy, gleaned from the writings of Hobbes, against the claims made in favour of limited constitutional government. This book will be of interest to students and researchers of political theory, particularly those with an interest in democracy and social choice theory. (shrink)

Jonathan Aldred shares our desire to promote a reconciliation between social choice theory and deliberative democracy in the interests of a more comprehensive and compelling account of democracy.1 His comments on some details of our analysis – specifically, our use of Arrow’s conditions of universal domain and independence of irrelevant alternatives – give us an opportunity to clarify our position. His discussion of the independence condition in particular identifies some ambiguity in our exposition, and as such is useful. We (...) are less impressed by the way Aldred characterizes the overall terms of the reconciliation we propose. We believe that his argument on this matter should be resisted because it provides deliberative democrats with a bad excuse to dismiss social choice theory altogether, which is surely not what he intends. (shrink)

This paper examines social choice theory with the strong Pareto principle. The notion of conditional decisiveness is introduced to clarify the underlying power structure behind strongly Paretian aggregation rules satisfying binary independence. We discuss the various degrees of social rationality: transitivity, semi-transitivity, the interval-order property, quasi-transitivity, and acyclicity.

This volume provides an overview of issues arising in work on the foundations of decision theory and social choice. The collection will be of particular value to researchers in economics with interests in utility or welfare, but also to any social scientist or philosopher interested in theories of rationality or group decision-making.

Distributed cognition refers to processes which are (i) cognitive and (ii) distributed across multiple agents or devices rather than performed by a single agent. Distributed cognition has attracted interest in several fields ranging from sociology and law to computer science and the philosophy of science. In this paper, I discuss distributed cognition from a social-choice-theoretic perspective. Drawing on models of judgment aggregation, I address two questions. First, how can we model a group of individuals as a distributed cognitive system? (...) Second, can a group acting as a distributed cognitive system be ‘rational’ and ‘track the truth’ in the outputs it produces? I argue that a group’s performance as a distributed cognitive system depends on its ‘aggregation procedure’ – its mechanism for aggregating the group members’ inputs into collective outputs – and I investigate the properties of an aggregation procedure that matter. (shrink)

Problems and choices -- Who shall live? -- The physician : the captain of the team -- The hospital : the house of hope -- Drugs : the key to modern medicine -- Paying for medical care.

Kuhn’s famous thesis that there is ‘no unique algorithm’ for choosing between rival scientific theories is analysed using the machinery of social choice theory. It is shown that the problem of theory choice as posed by Kuhn is formally identical to a standard social choice problem. This suggests that analogues of well-known results from the social choice literature, such as Arrow’s impossibility theorem, may apply to theory choice. If an analogue of Arrow’s theorem does hold for theory (...) choice this would refute Kuhn’s thesis, but it would also pose a threat to the rationality of science, a threat that is if anything more worrying than that posed by Kuhn. Various possible ‘escape routes’ from Arrow’s impossibility result are examined, in particular Amartya Sen’s idea of ‘enriching the informational basis’. It is shown that Sen’s idea can be applied to the problem of theory choice in science. This in turn sheds light on two well-known approaches to inductive inference in philosophy of science: Bayesianism and statistical model selection. (shrink)

The notion of justice is not some inwardly homogeneous, simple and objective one. Assumed the gains and losses on the one side and the relative levels of welfare on tiie other side play the cmcial role as criteria for being just, there are at least the four different, mutually exhaustive and irreducible conceptions of justice - cardinal and ordinal utilitarianism and moderate and radical egalitarianism. The first and fourth theories rely on just one criterion, whereas theories two and three rely (...) on two criteria each. These two theories are more refined but not more just: Their acceptability depends on normative persuasion. Thus, by introducing the intensity of preferences into analysis, the whole discussion of the concept of justice becomes more comprehensive but also more complicated. (shrink)

The notion of justice is not some inwardly homogeneous, simple and objective one. Assumed the gains and losses on the one side and the relative levels of welfare on tiie other side play the cmcial role as criteria for being just, there are at least the four different, mutually exhaustive and irreducible conceptions of justice - cardinal and ordinal utilitarianism and moderate and radical egalitarianism. The first and fourth theories rely on just one criterion, whereas theories two and three rely (...) on two criteria each. These two theories are more refined but not more just: Their acceptability depends on normative persuasion. Thus, by introducing the intensity of preferences into analysis, the whole discussion of the concept of justice becomes more comprehensive but also more complicated. (shrink)

This introductory text explores the theory of social choice. Written as a primer suitable for advanced undergraduates and graduates, this text will act as an important starting point for students grappling with the complexities of social choice theory. Rigorous yet accessible, this primer avoids the use of technical language and provides an up-to-date discussion of this rapidly developing field. This is the first in a series of texts published in association with the LSE.

Suppose it is possible to make approximate interpersonal comparisons of welfare gains and losses. Thus, if w\documentclass[12pt]{minimal} \usepackage{amsmath} \usepackage{wasysym} \usepackage{amsfonts} \usepackage{amssymb} \usepackage{amsbsy} \usepackage{mathrsfs} \usepackage{upgreek} \setlength{\oddsidemargin}{-69pt} \begin{document}$$w$$\end{document}, x\documentclass[12pt]{minimal} \usepackage{amsmath} \usepackage{wasysym} \usepackage{amsfonts} \usepackage{amssymb} \usepackage{amsbsy} \usepackage{mathrsfs} \usepackage{upgreek} \setlength{\oddsidemargin}{-69pt} \begin{document}$$x$$\end{document}, y\documentclass[12pt]{minimal} \usepackage{amsmath} \usepackage{wasysym} \usepackage{amsfonts} \usepackage{amssymb} \usepackage{amsbsy} \usepackage{mathrsfs} \usepackage{upgreek} \setlength{\oddsidemargin}{-69pt} \begin{document}$$y$$\end{document} and z\documentclass[12pt]{minimal} \usepackage{amsmath} \usepackage{wasysym} \usepackage{amsfonts} \usepackage{amssymb} \usepackage{amsbsy} \usepackage{mathrsfs} \usepackage{upgreek} \setlength{\oddsidemargin}{-69pt} \begin{document}$$z$$\end{document} are personal states, then it is sometimes possible to say “The welfare gain of the state change w⇝x\documentclass[12pt]{minimal} \usepackage{amsmath} \usepackage{wasysym} \usepackage{amsfonts} \usepackage{amssymb} \usepackage{amsbsy} \usepackage{mathrsfs} (...) \usepackage{upgreek} \setlength{\oddsidemargin}{-69pt} \begin{document}$$w\leadsto x$$\end{document} is greater than the welfare gain of the state change y⇝z\documentclass[12pt]{minimal} \usepackage{amsmath} \usepackage{wasysym} \usepackage{amsfonts} \usepackage{amssymb} \usepackage{amsbsy} \usepackage{mathrsfs} \usepackage{upgreek} \setlength{\oddsidemargin}{-69pt} \begin{document}$$y\leadsto z$$\end{document}”. We can represent this by the formula “⪰\documentclass[12pt]{minimal} \usepackage{amsmath} \usepackage{wasysym} \usepackage{amsfonts} \usepackage{amssymb} \usepackage{amsbsy} \usepackage{mathrsfs} \usepackage{upgreek} \setlength{\oddsidemargin}{-69pt} \begin{document}$$\succeq $$\end{document}”, where \documentclass[12pt]{minimal} \usepackage{amsmath} \usepackage{wasysym} \usepackage{amsfonts} \usepackage{amssymb} \usepackage{amsbsy} \usepackage{mathrsfs} \usepackage{upgreek} \setlength{\oddsidemargin}{-69pt} \begin{document}$$$$\end{document} is a difference preorder: an incomplete preorder on the space of all possible personal state changes. A social state change is a bundle of personal state changes. A social difference preorder is an incomplete preorder on the space of social state changes, which satisfies Pareto and Anonymity axioms. We characterize a family of SDPs which are roughly utilitarian in nature. We also apply the model to redistributive taxation. (shrink)

This book presents an exploration of the idea of the common or social good, extended so that alternatives with different populations can be ranked. The approach is, in the main, welfarist, basing rankings on the well-being, broadly conceived, of those who are alive. The axiomatic method is employed, and topics investigated include: the measurement of individual well-being, social attitudes toward inequality of well-being, the main classes of population principles, principles that provide incomplete rankings, principles that rank uncertain alternatives, (...) best choices from feasible sets, and applications. The chapters are divided, with mathematical arguments confined to the second part. The first part is intended to make the arguments accessible to a more general readership. Although the book can be read as a defense of the critical-level generalized-utilitarian class of principles, comprehensive examinations of other classes are included. (shrink)

We introduce a logic specifically designed to support reasoning about social choice functions. The logic includes operators to capture strategic ability, and operators to capture agent preferences. We establish a correspondence between formulae in the logic and properties of social choice functions, and show that the logic is expressively complete with respect to social choice functions, i.e., that every social choice function can be characterised as a formula of the logic. We prove that the logic is (...) decidable, and give a complete axiomatization. To demonstrate the value of the logic, we show in particular how it can be applied to the problem of determining whether a social choice function is strategy-proof. (shrink)

The social welfare functional approach to social choice theory fails to distinguish a genuine change in individual well-beings from a merely representational change due to the use of different measurement scales. A generalization of the concept of a social welfare functional is introduced that explicitly takes account of the scales that are used to measure well-beings so as to distinguish between these two kinds of changes. This generalization of the standard theoretical framework results in a more satisfactory (...) formulation of welfarism, the doctrine that social alternatives are evaluated and socially ranked solely in terms of the well-beings of the relevant individuals. This scale-dependent form of welfarism is axiomatized using this framework. The implications of this approach for characterizing classes of social welfare orderings are also considered. (shrink)

Extensive measurement is the standard measurement-theoretic approach for constructing a ratio scale. It involves the comparison of objects that can be concatenated in an additively representable way. This paper studies the implications of extensively measurable welfare for social choice theory. We do this in two frameworks: an Arrovian framework with a fixed population and no interpersonal comparisons, and a generalized framework with variable populations and full interpersonal comparability. In each framework we use extensive measurement to introduce novel domain restrictions, (...) independence conditions, and constraints on social evaluation. We prove a welfarism theorem for these domains and characterize the social welfare functions that satisfy the axioms of extensive measurement at both the individual and social levels. The main results are simple axiomatizations of strong dictatorship in the Arrovian framework and classical utilitarianism in the generalized framework. (shrink)