The thesis of instrumental convergence holds that a wide range of ends have common means: for instance, self preservation, desire preservation, self improvement, and resource acquisition. Bostrom contends that instrumental convergence gives us reason to think that "the default outcome of the creation of machine superintelligence is existential catastrophe". I use the tools of decision theory to investigate whether this thesis is true. I find that, even if intrinsic desires are randomly selected, instrumental rationality induces biases towards certain kinds of (...) choices. Firstly, a bias towards choices which leave less up to chance. Secondly, a bias towards desire preservation, in line with Bostrom's conjecture. And thirdly, a bias towards choices which afford more choices later on. I do not find biases towards any other of the convergent instrumental means on Bostrom's list. I conclude that the biases induced by instrumental rationality at best weakly support Bostrom's conclusion that machine superintelligence is likely to lead to existential catastrophe. (shrink)

We present a simple model to show the compossibility of middle knowledge, grounded truth, libertarian free will, predestination, evil, hell, a sin-free heaven, God being perfectly just, free, praiseworthy, and necessarily omnibenevolent, omniscient, and omnipotent, this world being both replete with injustice and the best of all possible worlds, heinous suffering, no-one unjustly suffering, God’s grace for the godly, the prospering of the godless, original sin, human responsibility, transworld depravity, irresistible grace, and Arminian human choice. The model is not intended (...) to be realistic, but its possibility argues for the possibility that a realistic model containing such compossibles could exist – and even be actual. (shrink)

Ruth Chang and Joseph Raz have both drawn far-reaching consequences for agency from the phenomenon of incommensurability. After criticizing their arguments, I outline an alternative view: if incommensurability is vagueness, then there are no substantial implications for agency, except perhaps a limited form of naturalistic voluntarism if our reasons are provided by desires.

Causalists and Evidentialists can agree about the right course of action in an (apparent) Newcomb problem, if the causal facts are not as initially they seem. If declining $1,000 causes the Predictor to have placed $1m in the opaque box, CDT agrees with EDT that one-boxing is rational. This creates a difficulty for Causalists. We explain the problem with reference to Dummett's work on backward causation and Lewis's on chance and crystal balls. We show that the possibility that the causal (...) facts might be properly judged to be non-standard in Newcomb problems leads to a dilemma for Causalism. One horn embraces a subjectivist understanding of causation, in a sense analogous to Lewis's own subjectivist conception of objective chance. In this case the analogy with chance reveals a terminological choice point, such that either (i) CDT is completely reconciled with EDT, or (ii) EDT takes precedence in the cases in which the two theories give different recommendations. The other horn of the dilemma rejects subjectivism, but now the analogy with chance suggests that it is simply mysterious why causation so construed should constrain rational action. (shrink)

"Five Questions on Formal Philosophy": Like the other authors in the volume, I was asked for my reflections on the character of philosophy by answering the following five questions : 1. Why were you initially drawn to formal methods? 2. What example from your work illustrates the role formal methods can play in philosophy? 3. What is the proper role of philosophy in relation to other disciplines? 4. What do you consider the most neglected topics and/or contributions in late 20th (...) century philosophy? 5. What are the most important problems in philosophy and which are the prospects for progress? (shrink)

The classic formulation of causal decision theory (CDT) appeals to counterfactuals. It says that you should aim to choose an option that would have a good outcome, were you to choose it. However, this version of CDT faces trouble if the laws of nature are deterministic. After all, the standard theory of counterfactuals says that, if the laws are deterministic, then if anything—including the choice you make—were different in the present, either the laws would be violated or the distant past (...) would be changed. And as several authors have shown, it's easy to transform this upshot of the standard theory of counterfactuals into full-blown counterexamples to CDT. In response to these counterexamples, I argue here that the problem lies, not so much with CDT's guiding idea—that it's the expected causal consequences of your actions that matter for rational decision-making—but with the fact that the classic formulation of CDT doesn't pay sufficient attention to the context-sensitivity of counterfactuals. I develop a contextualist version of CDT which better accounts for this context-sensitivity. And I show that my theory avoids the problems faced by the classic formulation of CDT in determinstic worlds. (shrink)

This volume concerns Rational Agents - humans, players in a game, software or institutions - which must decide the proper next action in an atmosphere of partial information and uncertainty. The book collects formal accounts of Uncertainty, Rationality and Agency, and also of their interaction. It will benefit researchers in artificial systems which must gather information, reason about it and then make a rational decision on which action to take.

The role of causation and counterfactuals in causal decision theory is vexed and disputed. Recently, Brian Hedden (2023) argues that we should abandon causal decision theory in favour of an alternative: counterfactual decision theory. I argue that, pace Hedden, counterfactual decision theory is not a competitor to, but rather a version of, causal decision theory – the most popular version by far. I provide textual evidence that the founding fathers of causal decision theory (Stalnaker, Gibbard, Harper, Lewis, Skyrms, Sobel, and (...) Joyce) all endorse counterfactual decision theories. I additionally discuss why these theories came to be called ‘causal’, rather than ‘counterfactual’. And I argue that, properly understood, causal decision theory escapes Hedden's objections. (shrink)

In Newcomb’s paradox you can choose to receive either the contents of a particular closed box, or the contents of both that closed box and another one. Before you choose though, an antagonist uses a prediction algorithm to accurately deduce your choice, and uses that deduction to fill the two boxes. The way they do this guarantees that you made the wrong choice. Newcomb’s paradox is that game theory’s expected utility and dominance principles appear to provide conflicting recommendations for what (...) you should choose. Here we show that the conflicting recommendations assume different probabilistic structures relating your choice and the algorithm’s prediction. This resolves the paradox: the reason there appears to be two conflicting recommendations is that the probabilistic structure relating the problem’s random variables is open to two, conflicting interpretations. We then show that the accuracy of the prediction algorithm in Newcomb’s paradox, the focus of much previous work, is irrelevant. We end by showing that Newcomb’s paradox is time-reversal invariant; both the paradox and its resolution are unchanged if the algorithm makes its ‘prediction’ after you make your choice rather than before. (shrink)

_ doi:10.1093/analys/anx072 _, published: 27 June 2017.

Anti-exceptionalism about logic takes logical theories to be continuous with scientific theories. Scientific theories are subject to criteria of theoretical equivalence. This article compares two types of theoretical equivalence – one syntactic and one semantic – in the context of logical anti-exceptionalism, and argues that the syntactic approach leads to undesirable consequences. The anti-exceptionalist should therefore take a semantic approach when evaluating whether logical theories, understood as scientific theories, are equivalent. This article argues for a particular semantic approach, in terms (...) of categorical equivalence, to determine whether logical theories are equivalent. (shrink)

The 'Why ain'cha rich?' argument for one-boxing in Newcomb's problem allegedly vindicates evidential decision theory and undermines causal decision theory. But there is a good response to the argument on behalf of causal decision theory. I develop this response. Then I pose a new problem and use it to give a new 'Why ain'cha rich?' argument. Unlike the old argument, the new argument targets evidential decision theory. And unlike the old argument, the new argument is sound.

Suppose that you have to take a test tomorrow but you do not want to study. Unfortunately you should study, since you care about passing and you expect to pass only if you study. Is there anything you can do to make it the case that you should not study? Is there any way for you to ‘rationalize’ slacking off? I suggest that such rationalization is impossible. Then I show that if evidential decision theory is true, rationalization is not only (...) possible but sometimes advisable. (shrink)

There are arguments which purport to rebut psychological determinism by appealing to its alleged incompatibility with rationality. I argue that they all fail. Against Davidson, I argue that rationality does not preclude the existence of psychological laws. Against Popper, I argue that rationality is compatible with the possibility of predicting human actions. Against Schlesinger, I claim that Newcomb's problem cannot be invoked to show that human actions are unpredictable. Having vindicated the possibility of a rationally-based theory of action, I consider (...) the form it might take. (shrink)

This article proposes a new theory of rational decision, distinct from both causal decision theory (CDT) and evidential decision theory (EDT). First, some intuitive counterexamples to CDT and EDT are presented. Then the motivation for the new theory is given: the correct theory of rational decision will resemble CDT in that it will not be sensitive to any comparisons of absolute levels of value across different states of nature, but only to comparisons of the differences in value between the available (...) options within states of nature; however, the correct theory will also resemble EDT in that it will rely on conditional probabilities (not unconditional probabilities). The new theory gives a prominent role to the notion of a “benchmark” for each state of nature, by comparison with which the value of the available options in that state of nature are measured, and so it has been called the Benchmark Theory (BT). It is argued that BT gives the right verdict on the cases that seem to be counterexamples to CDT and EDT. Finally, some objections to BT are considered and answered. (shrink)

Newcomb’s problem is viewed as a dynamic game with an agent and a superior being as players. Depending on whether or not a risk-neutral agent’s confidence in the superior being, as measured by a subjective probability assigned to the move order, exceeds a threshold or not, one obtains the one-box outcome or the two-box outcome, respectively. The findings are extended to an agent with arbitrary increasing utility, featuring in general two thresholds. All solutions require only minimal assumptions about the being’s (...) payoffs, and the being is always sure to predict the agent’s choice in equilibrium. The relevant Nash equilibria are subgame-perfect, except for risk-seeking agents where for intermediate beliefs, the being may be unable to ensure perfect prediction without relying on noncredible threats. Lastly, analogies of Newcomb’s problem to the commitment problem on a continuum are discussed. (shrink)

In this paper I explore a version of standard (expected utility) decision theory in which the probability parameter is interpreted as an objective chance believed by agents to obtain and values of this parameter are fixed by indicative conditionals linking possible actions with possible outcomes. After reviewing some recent developments centering on the common-cause counterexamples to the standard approach, I introduce and briefly discuss the key notions in my own approach. (This approach has essentially the same results as the causal (...) approach in common-cause cases.) I then discuss the Rule of Dominance and find, in the context of the present proposal, that it cannot serve as an independent source of action justification. Turning next to Newcomb''s Problem, I argue that the much discussed issue of back-tracking counterfactuals is something of a red herring for decision theory. Once the twin distractions of back-tracking counterfactuals and Dominance Reasoning are set aside the 1-box solution emerges as a natural consequence of the present proposal. It is of interest that this proposal agrees with the causal approach in the standard common-cause examples and the expected-utility approach in the Newcomb case: one can be smart and rich and keep on smoking. (shrink)

Causal eliminativists maintain that all causal talk is false. The prospects for such a view seem to be stymied by an indispensability argument, charging that any agent must distinguish between effective and ineffective strategies, and that such a distinction must commit that agent to causal notions. However, this argument has been under-explored. The contributions of this paper are twofold: first, I provide a thorough explication of the indispensability argument and the various ways it might be defended. Second, I point to (...) an important limitation in the argument and suggest that it does not give us sufficient reason to reject eliminativism. In support of this last claim, I show that the distinction between effective and ineffective strategies could perfectly well be grounded in a counterfactual rather than a causal decision theory and argue that there are fully adequate explanations of how we could come to make the requisite counterfactual judgments that need not invoke causal concepts. (shrink)

This commentary focuses on the parts of psychological game theory dealing with preference, as illustrated by team reasoning, and supports the conclusion that these theoretical notions do not contribute above and beyond existing theory in understanding social interaction. In particular, psychology and games are already bridged by a comprehensive, formal, and inherently psychological theory, interdependence theory (Kelley & Thibaut 1978; Kelley et al. 2003), which has been demonstrated to account for a wide variety of social interaction phenomena.

I describe a thought experiment in which an agent must choose between suffering a greater pain in the past or a lesser pain in the future. This case demonstrates that the ‘temporal value asymmetry’ – our disposition to attribute greater significance to future pleasures and pains than to past – can have consequences for the rationality of actions as well as attitudes. This fact, I argue, blocks attempts to vindicate the temporal value asymmetry as a useful heuristic tied to the (...) asymmetry of causation. Since the two standard arguments for the rationality of the temporal value asymmetry appeal to causal asymmetry and the passage of time respectively, the failure of the causal asymmetry explanation suggests that the B-theory, which rejects temporal passage, has substantial revisionary implications concerning our attitudes toward past and future experience. (shrink)

Examples involving common causes — most prominently, examples involving genetically influenced choices — are analytically equivalent not to standard Newcomb Problems — in which the Predictor genuinely predicts the agent's decision — but to non-standard Newcomb Problems — in which the Predictor guarantees the truth of her predictions by interfering with the agent's decision to make the agent choose as it was predicted she would. When properly qualified, causal and epistemic decision theories diverge only on standard — not on non-standard (...) — Newcomb Problems, and thus not on examples involving common causes. (shrink)

Does the desirability of a proposition depend on whether it is true? Not according to the Invariance assumption, held by several notable philosophers. The Invariance assumption plays an important role in David Lewis’ famous arguments against the so-called Desire-as-Belief thesis (DAB), an anti-Humean thesis according to which a rational agent desires a proposition exactly to the degree that she believes the proposition to be desirable. But the assumption is of interest independently of Lewis’ arguments, for instance since both Richard Jeffrey (...) and James Joyce make the assumption (or, strictly speaking, accept a thesis that implies Invariance) in their influential books on decision theory. The main claim to be defended in this paper is that Invariance is incompatible with certain assumptions of decision theory. I show that the assumption fails on the most common interpretations of desirability and/or choice-worthiness found in decision theory. I moreover show that Invariance is inconsistent with Richard Jeffrey’s decision theory, on which Lewis’ arguments against DAB are based. Finally, I show that Invariance contradicts how we in general do and should think about conditional desirability. (shrink)

The paper will show how one may rationalize one-boxing in Newcomb's problem and drinking the toxin in the Toxin puzzle within the confines of causal decision theory by ascending to so-called reflexive decision models which reflect how actions are caused by decision situations (beliefs, desires, and intentions) represented by ordinary unreflexive decision models.

The paper builds on the basically Humean idea that A is a cause of B iff A and B both occur, A precedes B, and A raises the metaphysical or epistemic status of B given the obtaining circumstances. It argues that in pursuit of a theory of deterministic causation this ‘status raising’ is best explicated not in regularity or counterfactual terms, but in terms of ranking functions. On this basis, it constructs a rigorous theory of deterministic causation that successfully deals (...) with cases of overdetermination and pre-emption. It finally indicates how the account's profound epistemic relativization induced by ranking theory can be undone. Introduction Variables, propositions, time Induction first Causation Redundant causation Objectivization. (shrink)

The crucial premise of the standard argument for two-boxing in Newcomb's problem, a causal dominance principle, is false. We present some counterexamples. We then offer a metaethical explanation for why the counterexamples arise. Our explanation reveals a new and superior argument for two-boxing, one that eschews the causal dominance principle in favor of a principle linking rational choice to guidance and actual value maximization.

Consequentialists often assume rational monism: the thesis that options are always made rationally permissible by the maximization of the selfsame quantity. This essay argues that consequentialists should reject rational monism and instead accept rational pluralism: the thesis that, on different occasions, options are made rationally permissible by the maximization of different quantities. The essay then develops a systematic form of rational pluralism which, unlike its rivals, is capable of handling both the Newcomb problems that challenge evidential decision theory and the (...) unstable problems that challenge causal decision theory. (shrink)

You receive a pink packet from Miss Lead, a notoriously deceptive truth-teller. You know that if you open the packet and do not find blank pages, then you will justifiably change your mind about the evidence being misleading. Indeed, you will infer that your previous fears about misleading evidence were themselves founded on misleading evidence. Should you open the pink packet? No, answers an advocate of self-censorship. Yes, answers an advocate of the principle that you should base conclusions on all (...) the available evidence. Your final deliberations will be guided by Gilbert Harman’s Thought. Ultimately, the case against opening the packet prevails. (shrink)

It has often been noted that there is some tension between engaging in decision-making and believing that one’s choices might be predetermined. The possibility that our choices are predetermined forces us to consider, in our decisions, act-state pairs which are inconsistent, and hence to which we cannot assign sensible utilities. But the reasoning which justifies two-boxing in Newcomb’s problem also justifies associating a non-zero causal probability with these inconsistent act-state pairs. Put together these undefined utilities and non-zero probabilities entail that (...) expected utilities are undefined whenever it is a possibility that our choices are predetermined. There are three ways to solve the problem, but all of them suffer serious costs: always assume that, contrary to our evidence, the outcome of our present decision-making is not predetermined; give up the reasoning that justifies unconditional two-boxing in Newcomb’s problem; or allow epistemically impossible outcomes to contribute to expected utility, leading to the wrong results in a series of cases introduced by Ahmed :665–685, 2014a, Evidence, decision and causality, Cambridge University Press, Cambridge, 2014b). However they choose to respond, causal decision theorists cannot remain silent: the intuitive tension between decision-making and the possibility of predetermination can be made precise, and resolving it will require giving up something. Causal decision theorists have a predetermination problem. (shrink)

I have maintained that some but not all prisoners' dilemmas are side-by-side Necomb problems. The present paper argues that, similarly, some but not all versions of Newcomb's Problem are prisoners' dilemmas in which Taking Two and Predicting Two make an equilibrium that is dispreferred by both the box-chooser and predictor to the outcome in which only one box is taken and this is predicted. I comment on what kinds of prisoner's dilemmas Newcomb's Problem can be, and on opportunities that results (...) reached may open for kinds of cooperative reasoning in versions of Newcomb's Problem. (shrink)

A person who does not have good intellectual reasons for believing in God can, depending on his probabilities and values for consequences of believing, have good practical reasons. Pascalian wagers founded on a variety of possible probability/value profiles are examined from a Bayesian perspective central to which is the idea that states and options are pragmatically reasonable only if they maximize subjective expected value. Attention is paid to problems posed by representations of values by Cantorian infinities. An appendix attends to (...) Robinsonian hyperreals. Another appendix presents for comparison Newcomb's Problem and a problem in some ways like it suggested, I think, by ideas of John Calvin. (shrink)

Responses to Newcomb-like challenges to evidential decision theories such as Jeffrey’s “logic of decision” range from allegations of incoherence and irrelevance; through stonewalling - “Just one box for me, thank you.“; to arguments that maintain that when properly applied by an ideal agent such theories get the right answers and, for example, prescribe the taking of both boxes, not just one; on to conservative revisions of evidential decision theories that are held to get these supposedly right answers while remaining true (...) to its evidential, Humean spirit; and finally to responses of radical revisionists who claim that only causal decision theories can get all Newcomb-like problem cases right. I discuss responses of the first sort in another paper - “Newcomb-like Problems for Jeffrey’s Logic of Decision.”. (shrink)

It is plausible that Newcomb problems in which causal maximizers and evidential maximizers would do different things would not be possible for ideal maximizers who are attentive to metatickles. An objection to Eells’s first argument for this makes welcome a second. Against it I argue that even ideal evidential and causal maximizers would do different things in some non-dominance Newcomb problems; and that they would hope for different things in some third-person and non-action problems, which is relevant if a good (...) theory of rational choices of acts should fit smoothly into a good theory of rational desires for facts. (shrink)

In response to widespread doubts among professional philosophers (Russell, Horwich, Dietrich, McGinn, Chalmers), Stoljar argues for a ‘reasonable optimism’ about progress in philosophy. He defends the large and surprising claim that ‘there is progress on all or reasonably many of the big questions.’ However, Stoljar’s caveats and admitted avoidance of historical evidence permits overlooking persistent controversies in philosophy of mind and cognitive science that are essentially unchanged since the 17th Century. Stoljar suggests that his claims are commonplace in philosophy departments (...) and, indeed, the evidence I adduce constitutes an indictment of the widely shared view among professional analytic philosophers. (shrink)

A fully adequate solution to Newcomb’s Problem (Nozick 1969) should reveal the source of its extraordinary elusiveness and persistent intractability. Recently, a few accounts have independently sought to meet this criterion of adequacy by exposing the underlying source of the problem’s profound puzzlement. Thus, Sorensen (1987), Slezak (1998), Priest (2002) and Maitzen and Wilson (2003) share the ‘no box’ view according to which the very idea that there is a right choice is misconceived since the problem is ill-formed or incoherent (...) in some way. Among proponents of this view, Richard Jeffrey (2004) recently declared that he renounces his earlier position that accepted Newcomb problems as genuine decision problems. Significantly, Jeffrey suggests that “Newcomb problems are like Escher’s famous staircase on which an unbroken ascent takes you back where you started” (Jeffrey (2004; 113)). Jeffrey’s analogy is apt for a puzzle whose specific logical features can be precisely articulated. Along the lines of these related approaches, I propose to improve and clarify them by providing such a deeper analysis that elucidates their essential, related insights. (shrink)

I want in this paper to do two things. First, I want to respond to some studies that argue that people are often not rational: that people regularly and systematically depart from rationality. The conclusion itself does not worry me. I pressed for the same in a recent book. But the arguments seem to me wrong, and wrong in an interesting way. There may be something to be learned from seeing how and why they fail.

I present a game-theoretic way to understand the situation describing Newcomb’s Problem (NP) which helps to explain the intuition of both one-boxers and two-boxers. David Lewis has shown that the NP may be modelled as a Prisoners Dilemma game (PD) in which ‘cooperating’ corresponds to ‘taking one box’. Adopting relevant results from game theory, this means that one should take just one box if the NP is repeated an indefinite number of times, but both boxes if it is a one-shot (...) game. Causal decision theorists thus give the right answer for the one-shot situation, whereas the one-boxers’ solution applies to the indefinitely iterated case. Because Nozick’s set-up of the NP is ambiguous between a one-shot and a repeated game, both of these solutions may appear plausible – depending on whether one conceives of the situation as one-off or repeated. If the players’ aim is to maximize their payoffs, the symmetric structure of the PD implies that the two players will behave alike both when the game is one-shot and when it is played repeatedly. Therefore neither the observed outcome of both players selecting the same strategy (in the PD) nor, correspondingly, the predictor’s accurate prediction of this outcome (in the NP) is at all surprising. There is no need for a supernatural predictor to explain the NP phenomena. (shrink)