Humanity stands at a precipice. -/- Our species could survive for millions of generations — enough time to end disease, poverty, and injustice; to reach new heights of flourishing. But this vast future is at risk. With the advent of nuclear weapons, humanity entered a new age, gaining the power to destroy ourselves, without the wisdom to ensure we won’t. Since then, these dangers have only multiplied, from climate change to engineered pandemics and unaligned artificial intelligence. If we do not (...) act fast to reach a place of safety, it may soon be too late. -/- The Precipice explores the science behind the risks we face. It puts them in the context of the greater story of humanity: showing how ending these risks is among the most pressing moral issues of our time. And it points the way forward, to the actions and strategies we can take today to safeguard humanity’s future. (shrink)

Which is better: a guarantee of a modest amount of moral value, or a tiny probability of arbitrarily large value? To prefer the latter seems fanatical. But, as I argue, avoiding such fanaticism brings severe problems. To do so, we must decline intuitively attractive trade-offs; rank structurally identical pairs of lotteries inconsistently, or else admit absurd sensitivity to tiny probability differences; have rankings depend on remote, unaffected events ; and often neglect to rank lotteries as we already know we would (...) if we learned more. Compared to these implications, fanaticism is highly plausible. (shrink)

This paper presents a new solution to the problems for orthodox decision theory posed by the Pasadena game and its relatives. I argue that a key question raised by consideration of these gambles is whether evaluative compositionality (as I term it) is a requirement of rationality: is the value that an ideally rational agent places on a gamble determined by the values that she places on its possible outcomes, together with their mode of composition into the gamble (i.e. the probabilities (...) assigned to them)? The paper first outlines a certain simple response to the Pasadena game and identifies two problems with this response, the second of which is that it leads to a wholesale violation of evaluative compositionality. I then argue that rationality does not require decision makers to factor in outcomes of arbitrarily low probability. A method for making decisions which flows from this basic idea is then developed, and it is shown that this decision method (Truncation) leads to a limited — as opposed to wholesale — violation of evaluative compositionality. The paper then argues that the truncation method yields solutions to the problems posed by the Pasadena game and its relatives that are both attractive in themselves and superior to those yielded by alternative proposals in the literature. (shrink)

This paper examines the relationship between Savage's sure thing principle and the value of information. We present two classes of results. First, we show that, under a consequentialist axiom, the sure-thing principle is neither sufficient nor necessary for perfect information to be always desirable: specifically, under consequentialism, the sure thing principle is not implied by the condition that perfect information is always valuable; moreover, the joint imposition of the sure thing principle, consequentialism and either one of two state independence axioms (...) does not imply that perfect information is always desirable. Second, we demonstrate that, under consequentialism, the sure thing principle is necessary for a nonnegative value of possibly imperfect information (though of course the principle is still not sufficient). One implication of these results is that the sure thing principle, under consequentialism, plays a somewhat different role in ensuring dynamic consistency in decision making under uncertainty than does the independence axiom in decision making under risk. (shrink)

Abstract: This paper defends a constraint that any satisfactory decision theory must satisfy. I show how this constraint is violated by all of the decision theories that have been endorsed in the literature that are designed to deal with cases in which opinions or values are represented by a set of functions rather than a single one. Such a decision theory is necessary to account for the existence of what Ruth Chang has called “parity” (as well as for cases in (...) which agents have incomplete preferences or imprecise credences). The problem with the all of the decision theories that have been defended to account for parity is that they are committed to a claim I call unanimity: when all of the functions in the set agree that an agent ought to do A, then an agent ought to do A. A decision theory committed to unanimity violates the constraint I defend in this paper. Thus, if parity exists, a new approach to decision theory is necessary. (shrink)

People with the kind of preferences that give rise to the St. Petersburg paradox are problematic---but not because there is anything wrong with infinite utilities. Rather, such people cannot assign the St. Petersburg gamble any value that any kind of outcome could possibly have. Their preferences also violate an infinitary generalization of Savage's Sure Thing Principle, which we call the *Countable Sure Thing Principle*, as well as an infinitary generalization of von Neumann and Morgenstern's Independence axiom, which we call *Countable (...) Independence*. In violating these principles, they display foibles like those of people who deviate from standard expected utility theory in more mundane cases: they choose dominated strategies, pay to avoid information, and reject expert advice. We precisely characterize the preference relations that satisfy Countable Independence in several equivalent ways: a structural constraint on preferences, a representation theorem, and the principle we began with, that every prospect has a value that some outcome could have. (shrink)

Philosophy and Phenomenological Research, EarlyView.

The existence of a transitive, complete, and weakly independent relation on the full set of gambles implies the existence of a non-Ramsey set. Therefore, each transitive and weakly independent relation on the set of gambles either is incomplete or does not have an explicit description. Whatever tools decision theory makes available, there will always be decision problems where these tools fail us. In this sense, decision theory remains incomplete.

One result of this note is about the nonconstructivity of countably infinite lotteries: even if we impose very weak conditions on the assignment of probabilities to subsets of natural numbers we cannot prove the existence of such assignments constructively, i.e., without something such as the axiom of choice. This is a corollary to a more general theorem about large-small filters, a concept that extends the concept of free ultrafilters. The main theorem is that proving the existence of large-small filters requires (...) a nonconstructive axiom like AC. (shrink)

:We address the question, in decision theory, of how the value of risky options should be assessed when they have no finite standard expected value, that is, where the sum of the probability-weighted payoffs is infinite or not well defined. We endorse, combine and extend the proposal of Easwaran to evaluate options on the basis of their weak expected value, and the proposal of Colyvan to rank options on the basis of their relative expected value.

We show that it is not possible to extend the ranking of one-stage lotteries based on their weak-expectation to a reflexive and transitive relation on the collection of one- and two-stage lotteries that satisfies two basic axioms, the minimal value axiom and the reduction axiom. We propose an extension that satisfies only the first axiom. This ranking takes payoffs, their probabilities, and the tree structure into account.

In our 2004, we introduced two games in the spirit of the St Petersburg game, the Pasadena and Altadena games. As these latter games lack an expectation, we argued that they pose a paradox for decision theory. Terrence Fine has shown that any finite valuations for the Pasadena, Altadena, and St Petersburg games are consistent with the standard decision-theoretic axioms. In particular, one can value the Pasadena game above the other two, a result that conflicts with both our intuitions and (...) dominance reasoning. We argue that this result, far from resolving the Pasadena paradox, should serve as a reductio of the standard theory, and we consequently make a plea for new axioms for a revised theory. We also discuss a proposal by Kenny Easwaran that a gamble should be valued according to its 'weak expectation', a generalization of the usual notion of expectation. (shrink)

(No abstract is available for this citation).

The principle of Anteriority says that prospects that are identical from the perspective of every possible person’s welfare are equally good overall. The principle enjoys prima facie plausibility, and has been employed for various theoretical purposes. Here it is shown using an analogue of the St Petersburg Paradox that Anteriority is inconsistent with central principles of axiology.

We argue that all comparative expressions in natural language obey a principle that we call Comparability: if x and y are at least as F as themselves, then either x is at least as F as y or y is at least as F as x. This principle has been widely rejected among philosophers, especially by ethicists, and its falsity has been claimed to have important normative implications. We argue that Comparability is needed to explain the goodness of several patterns (...) of inference that seem manifestly valid, that the purported failures of Comparability would have absurd consequences, and that the influential arguments against Comparability are less compelling than they may have initially seemed. (shrink)

I reason: (1) For any x, if I knew that A contained x, then the odds are even that B contains either 2x or x/2, so the expected amount in B would be 5x/4. So (2) for all x, if I knew that A contained x, I would have an expected gain in switching to B. So (3) I should switch to B. But this seems clearly wrong, as my information about A and B is symmetrical.

This paper argues for the existence of a fourth positive generic value relation that can hold between two items beyond ‘better than’, ‘worse than’, and ‘equally good’: namely ‘on a par’.

We are often faced with choices that involve the weighing of people's lives against each other, or the weighing of lives against other good things. These are choices both for individuals and for societies. A person who is terminally ill may have to choose between palliative care and more aggressive treatment, which will give her a longer life but at some cost in suffering. We have to choose between the convenience to ourselves of road and air travel, and the lives (...) of the future people who will be killed by the global warming we cause, through violent weather, tropical disease, and heat waves. We also make choices that affect how many lives there will be in the future: as individuals we choose how many children to have, and societies choose tax policies that influence people's choices about having children. These are all problems of weighing lives. How should we weigh lives? Weighing Lives develops a theoretical basis for answering this practical question. It extends the work and methods of Broome's earlier book Weighing Goods to cover the questions of life and death. Difficult problems come up in the process. In particular, Weighing Lives tackles the well-recognized, awkward problems of the ethics of population. It carefully examines the common intuition that adding people to the population is ethically neutral - neither a good nor a bad thing - but eventually concludes this intuition cannot be fitted into a coherent theory of value. In the course of its argument, Weighing Lives examines many of the issues of contemporary moral theory: the nature of consequentialism and teleology; the transitivity, continuity, and vagueness of betterness; the quantitative conception of wellbeing; the notion of a life worth living; the badness of death; and others. This is a work of philosophy, but one of its distinctive features is that it adopts some of the precise methods of economic theory (without introducing complex mathematics). Not only philosophers, but also economists and political theorists concerned with the practical question of valuing life, should find the book's conclusions highly significant to their work. (shrink)

Risk-weighted expected utility theory permits preferences which violate the Sure-Thing Principle. But preferences that violate the STP can lead to bad decisions in sequential choice problems. In particular, they can lead decision-makers to adopt a strategy that is dominated – i.e. a strategy such that some available alternative leads to a better outcome in every possible state of the world.

With very advanced technology, a very large population of people living happy lives could be sustained in the accessible region of the universe. For every year that development of such technologies and colonization of the universe is delayed, there is therefore a corresponding opportunity cost: a potential good, lives worth living, is not being realized. Given some plausible assumptions, this cost is extremely large. However, the lesson for standard utilitarians is not that we ought to maximize the pace of technological (...) development, but rather that we ought to maximize its safety, i.e. the probability that colonization will eventually occur. This goal has such high utility that standard utilitarians ought to focus all their efforts on it. Utilitarians of a ‘person-affecting’ stripe should accept a modified version of this conclusion. Some mixed ethical views, which combine utilitarian considerations with other criteria, will also be committed to a similar bottom line. (shrink)

We begin by showing that every theory of the value of uncertain prospects must have one of three unpalatable properties. _Reckless_ theories recommend giving up a sure thing, no matter how good, for an arbitrarily tiny chance of enormous gain; _timid_ theories permit passing up an arbitrarily large potential gain to prevent a tiny increase in risk; _non-transitive_ theories deny the principle that, if A is better than B and B is better than C, then A must be better than (...) C. Having set up this trilemma, we study its horns. Non-transitivity has been much discussed; we focus on drawing out the costs and benefits of recklessness and timidity when it comes to axiology, decision theory, and normative uncertainty. (shrink)

Orthodox decision theory gives no advice to agents who hold two goods to be incommensurate in value because such agents will have incomplete preferences. According to standard treatments, rationality requires complete preferences, so such agents are irrational. Experience shows, however, that incomplete preferences are ubiquitous in ordinary life. In this paper, we aim to do two things: (1) show that there is a good case for revising decision theory so as to allow it to apply non-vacuously to agents with incomplete (...) preferences, and (2) to identify one substantive criterion that any such non-standard decision theory must obey. Our criterion, Competitiveness, is a weaker version of a dominance principle. Despite its modesty, Competitiveness is incompatible with prospectism, a recently developed decision theory for agents with incomplete preferences. We spend the final part of the paper showing why Competitiveness should be retained, and prospectism rejected. (shrink)

ABSTRACTThis paper addresses the problem of opaque sweetening and argues that one should use stochastic dominance in comparing lotteries even when dealing with incomplete orderings that allow for non-comparable outcomes.

I argue that standard decision theories, namely causal decision theory and evidential decision theory, both are unsatisfactory. I devise a new decision theory, from which, under certain conditions, standard game theory can be derived.

We pose and resolve several vexing decision theoretic puzzles. Some are variants of existing puzzles, such as 'Trumped' (Arntzenius and McCarthy 1997), 'Rouble trouble' (Arntzenius and Barrett 1999), 'The airtight Dutch book' (McGee 1999), and 'The two envelopes puzzle' (Broome 1995). Others are new. A unified resolution of the puzzles shows that Dutch book arguments have no force in infinite cases. It thereby provides evidence that reasonable utility functions may be unbounded and that reasonable credence functions need not be countably (...) additive. The resolution also shows that when infinitely many decisions are involved, the difference between making the decisions simultaneously and making them sequentially can be the difference between riches and ruin. Finally, the resolution reveals a new way in which the ability to make binding commitments can save perfectly rational agents from sure losses. (shrink)

In choosing between moral alternatives -- choosing between various forms of ethical action -- we typically make calculations of the following kind: A is better than B; B is better than C; therefore A is better than C. These inferences use the principle of transitivity and are fundamental to many forms of practical and theoretical theorizing, not just in moral and ethical theory but in economics. Indeed they are so common as to be almost invisible. What Larry Temkin's book shows (...) is that, shockingly, if we want to continue making plausible judgments, we cannot continue to make these assumptions. Temkin shows that we are committed to various moral ideals that are, surprisingly, fundamentally incompatible with the idea that "better than" can be transitive. His book develops many examples where value judgments that we accept and find attractive, are incompatible with transitivity. While this might seem to leave two options -- reject transitivity, or reject some of our normative commitments in order to keep it -- Temkin is neutral on which path to follow, only making the case that a choice is necessary, and that the cost either way will be high. Temkin's book is a very original and deeply unsettling work of skeptical philosophy that mounts an important new challenge to contemporary ethics. (shrink)

Lara Buchak sets out a new account of rational decision-making in the face of risk. She argues that the orthodox view is too narrow, and suggests an alternative, more permissive theory: one that allows individuals to pay attention to the worst-case or best-case scenario, and vindicates the ordinary decision-maker.

This study uses techniques from economics to illuminate fundamental questions in ethics, particularly in the foundations of utilitarianism. Topics considered include the nature of teleological ethics, the foundations of decision theory, the value of equality and the moral significance of a person's continuing identity through time.

Temkin's book is a very original and deeply unsettling work of skeptical philosophy that mounts an important new challenge to contemporary ethics.

It is possible that the world contains infinitely many agents that have positive and negative levels of well-being. Theories have been developed to ethically rank such worlds based on the well-being levels of the agents in those worlds or other qualitative properties of the worlds in question, such as the distribution of agents across spacetime. In this thesis I argue that such ethical rankings ought to be consistent with the Pareto principle, which says that if two worlds contain the same (...) agents and some agents are better off in the first world than they are in the second and no agents are worse off than they are in the second, then the first world is better than the second. I show that if we accept four axioms – the Pareto principle, transitivity, an axiom stating that populations of worlds can be permuted, and the claim that if the ‘at least as good as’ relation holds between two worlds then it holds between qualitative duplicates of this world pair – then we must conclude that there is ubiquitous incomparability between infinite worlds. I show that this is true even if the populations of infinite worlds are disjoint or overlapping, and that we cannot use any qualitative properties of world pairs to rank these worlds. Finally, I argue that this incomparability result generates puzzles for both consequentialist and non-consequentialist theories of objective and subjective permissibility. (shrink)

When real-valued utilities for outcomes are bounded, or when all variables are simple, it is consistent with expected utility to have preferences defined over probability distributions or lotteries. That is, under such circumstances two variables with a common probability distribution over outcomes – equivalent variables – occupy the same place in a preference ordering. However, if strict preference respects uniform, strict dominance in outcomes between variables, and if indifference between two variables entails indifference between their difference and the status quo, (...) then preferences over rich sets of unbounded variables, such as variables used in the St. Petersburg paradox, cannot preserve indifference between all pairs of equivalent variables. In such circumstances, preference is not a function only of probability and utility for outcomes. Then the preference ordering is not defined in terms of lotteries. (shrink)

We prove a representation theorem for preference relations over countably infinite lotteries that satisfy a generalized form of the Independence axiom, without assuming Continuity. The representing space consists of lexicographically ordered transfinite sequences of bounded real numbers. This result is generalized to preference orders on abstract superconvex spaces.

Naive versions of decision theory take probabilities and utilities as primitive and use expected value to give norms on rational decision. However, standard decision theory takes rational preference as primitive and uses it to construct probability and utility. This paper shows how to justify a version of the naive theory, by taking dominance as the most basic normatively required preference relation, and then extending it by various conditions under which agents should be indifferent between acts. The resulting theory can make (...) all the decisions of classical expected utility theory, plus more in cases where expected utilities are infinite or undefined. Although the theory requires similarly strong assumptions to classical expected utility theory, versions of the theory can be developed with slightly weaker assumptions, without having to prove a new representation theorem for the weaker theory. This alternate foundation is particularly useful if probability is prior to preference, as suggested by the recent program to base probabilism on accuracy and alethic considerations rather than pragmatic ones. (shrink)

The lesson to be learned from the paradoxical St. Petersburg game and Pascal’s Mugging is that there are situations where expected utility maximizers will needlessly end up poor and on death’s door, and hence we should not be expected utility maximizers. Instead, when it comes to decision-making, for possibilities that have very small probabilities of occurring, we should discount those probabilities down to zero, regardless of the utilities associated with those possibilities.