We begin by showing that every theory of the value of uncertain prospects must have one of three unpalatable properties. _Reckless_ theories recommend giving up a sure thing, no matter how good, for an arbitrarily tiny chance of enormous gain; _timid_ theories permit passing up an arbitrarily large potential gain to prevent a tiny increase in risk; _non-transitive_ theories deny the principle that, if A is better than B and B is better than C, then A must be better than (...) C. Having set up this trilemma, we study its horns. Non-transitivity has been much discussed; we focus on drawing out the costs and benefits of recklessness and timidity when it comes to axiology, decision theory, and normative uncertainty. (shrink)

Person-affecting views in population ethics state that (in cases where all else is equal) we’re permitted but not required to create people who would enjoy good lives. In this paper, I present an argument against every possible variety of person-affecting view. The argument takes the form of a dilemma. Narrow person-affecting views must embrace at least one of three implausible verdicts in a case that I call ‘Expanded Non-Identity.’ Wide person-affecting views run into trouble in a case that I call (...) ‘Two-Shot Non-Identity.’ One plausible practical upshot of my argument is as follows: we individuals and our governments should be doing more to reduce the risk of human extinction this century. (shrink)

In response to cases that involve tiny probabilities of huge payoffs, some argue that we ought to discount small probabilities down to zero. However, this paper shows that doing so violates Independence and Continuity, and as a result of these violations, those who discount small probabilities can be exploited by money pumps. Various possible ways of avoiding exploitation will be discussed. This paper concludes that the money pump for Independence undermines the plausibility of discounting small probabilities. Much of the discussion (...) on Independence generalizes to other views that also violate Independence. (shrink)

This paper defends the claim that mere temporal proximity always and without exception strengthens certain moral duties, including the duty to save – call this view Robust Temporalism. Although almost all other moral philosophers dismiss Robust Temporalism out of hand, I argue that it is prima facie intuitively plausible, and that it is analogous to a view about special obligations that many philosophers already accept. I also defend Robust Temporalism against several common objections, and I highlight its relevance to a (...) number of practical policy debates, including longtermism. My conclusion is that Robust Temporalism is a moral live option, that deserves to be taken much more seriously in the future. (shrink)

This chapter starts from the claim that the state is plagued with problems of political short-termism: excessive priority given to near-term benefits at the expense of benefits further in thefuture. One possible mechanism to reduce short-termism involves apportioning greater relative political influence to the young, since younger citizens generally have greater additional life expectancy than older citizens and thus it looks reasonable to expect that they have preferences that are extended further into the future. But the chapter shows that this (...) is unlikely to make states significantly less short-termist: no empirical relationship has been found between age and willingness to support long-termist policies. Instead, the chapter proposes a more promising age-based mechanism. States should develop youth citizens’ assemblies that ensure accountability to future generations through a scheme of retrospective accountability. (shrink)

Longtermists have argued that humanity should significantly increase its efforts to prevent catastrophes like nuclear wars, pandemics, and AI disasters. But one prominent longtermist argument overshoots this conclusion: the argument also implies that humanity should reduce the risk of existential catastrophe even at extreme cost to the present generation. This overshoot means that democratic governments cannot use the longtermist argument to guide their catastrophe policy. In this paper, we show that the case for preventing catastrophe does not depend on longtermism. (...) Standard cost-benefit analysis implies that governments should spend much more on reducing catastrophic risk. We argue that a government catastrophe policy guided by cost-benefit analysis should be the goal of longtermists in the political sphere. This policy would be democratically acceptable, and it would reduce existential risk by almost as much as a strong longtermist policy. (shrink)

Is the overall value of a world just the sum of values contributed by each value-bearing entity in that world? Additively separable axiologies (like total utilitarianism, prioritarianism, and critical level views) say 'yes', but non-additive axiologies (like average utilitarianism, rank-discounted utilitarianism, and variable value views) say 'no'. This distinction is practically important: additive axiologies support 'arguments from astronomical scale' which suggest (among other things) that it is overwhelmingly important for humanity to avoid premature extinction and ensure the existence of a (...) large future population, while non-additive axiologies need not. We show, however, that when there is a large enough 'background population' unaffected by our choices, a wide range of non-additive axiologies converge in their implications with some additive axiology -- for instance, average utilitarianism converges to critical-level utilitarianism and various egalitarian theories converge to prioritiarianism. We further argue that real-world background populations may be large enough to make these limit results practically significant. This means that arguments from astronomical scale, and other arguments in practical ethics that seem to presuppose additive separability, may be truth-preserving in practice whether or not we accept additive separability as a basic axiological principle. (shrink)

Given the deep disagreement surrounding population axiology, one should remain uncertain about which theory is best. However, this uncertainty need not leave one neutral about which acts are better or worse. We show that, as the number of lives at stake grows, the Expected Moral Value approach to axiological uncertainty systematically pushes one toward choosing the option preferred by the Total View and critical-level views, even if one’s credence in those theories is low.

The notion of existential catastrophe is increasingly appealed to in discussion of risk management around emerging technologies, but it is not completely clear what this notion amounts to. Here, I provide an opinionated survey of the space of plausibly useful definitions of existential catastrophe. Inter alia, I discuss: whether to define existential catastrophe in ex post or ex ante terms, whether an ex ante definition should be in terms of loss of expected value or loss of potential, and what kind (...) of probabilities should be involved in any appeal to expected value. (shrink)

Longtermism is the view that the most urgent global priorities, and those to which we should devote the largest portion of our resources, are those that focus on (i) ensuring a long future for humanity, and perhaps sentient or intelligent life more generally, and (ii) improving the quality of the lives that inhabit that long future. While it is by no means the only one, the argument most commonly given for this conclusion is that these interventions have greater expected goodness (...) per unit of resource devoted to them than each of the other available interventions, including those that focus on the health and well-being of the current population. In this paper, I argue that, even if we grant the consequentialist ethics upon which this argument depends, and even if we grant one of the axiologies that are typically paired with that ethics to give the argument, we are not morally required to choose an option that maximises expected utility; indeed, we might not even be permitted to do so. Instead, I will argue, if the argument's consequentialism is correct, we should choose using a decision theory that is sensitive to risk, and allows us to give greater weight to worse-case outcomes than expected utility theory does. And, I will show, such decision theories do not always recommend longtermist interventions. Indeed, sometimes, they recommend exactly the opposite: sometimes, they recommend hastening human extinction. Many, though not all, will take this as a reductio of the consequentialism or the axiology of the argument. I remain agnostic on the conclusion we should draw. (shrink)

How ought you to evaluate your options if you're uncertain about what's fundamentally valuable? A prominent response is Expected Value Maximisation (EVM)—the view that under axiological uncertainty, an option is better than another if and only if it has the greater expected value across axiologies. But the expected value of an option depends on quantitative probability and value facts, and in particular on value comparisons across axiologies. We need to explain what it is for such facts to hold. Also, EVM (...) is by no means self-evident. We need an argument to defend that it’s true. This book introduces an axiomatic approach to answer these worries. It provides an explication of what EVM means by use of representation theorems: intertheoretic comparisons can be understood in terms of facts about which options are better than which, and mutatis mutandis for intratheoretic comparisons and axiological probabilities. And it provides a systematic argument to the effect that EVM is true: the theory can be vindicated through simple axioms. The result is a formally cogent and philosophically compelling extension of standard decision theory, and original take on the problem of axiological or normative uncertainty. (shrink)

Effective altruism has become a worldwide phenomenon. The movement combines empathy and reason in the attempt to improve the world. Adherents don’t let moral gut instincts dictate their altruistic efforts, but use evidence and reflection to do the most good they can. Effective altruism originated, and primarily grew, in strongly secular environments—such as philosophy departments or Silicon Valley. So far, a religious perspective on this movement has been lacking. What can people of faith learn from effective altruism? What may they (...) criticise? What can effective altruism in turn take from religion? This volume offers a first examination of these questions, covering various Christian as well as Jewish and Buddhist perspectives. (shrink)

This volume collects selected papers delivered at the 15th Conference of the International Society for Utilitarian Studies, which was held at Karlsruhe Institute of Technology in July 2018. It includes papers dealing with the past, present, and future of utilitarianism – the theory that human happiness is the fundamental moral value – as well as on its applications to animal ethics, population ethics, and the future of humanity, among other topics.

Which is better: a guarantee of a modest amount of moral value, or a tiny probability of arbitrarily large value? To prefer the latter seems fanatical. But, as I argue, avoiding such fanaticism brings severe problems. To do so, we must decline intuitively attractive trade-offs; rank structurally identical pairs of lotteries inconsistently, or else admit absurd sensitivity to tiny probability differences; have rankings depend on remote, unaffected events ; and often neglect to rank lotteries as we already know we would (...) if we learned more. Compared to these implications, fanaticism is highly plausible. (shrink)

How might we extend aggregative moral theories to compare infinite worlds? In particular, how might we extend them to compare worlds with infinite spatial volume, infinite temporal duration, and infinitely many morally valuable phenomena? When doing so, we face various impossibility results from the existing literature. For instance, the view we adopt can endorse the claim that worlds are made better if we increase the value in every region of space and time, or that they are made better if we (...) increase the value obtained by every person. But they cannot endorse both claims, so we must choose. In this paper I show that, if we choose the latter, our view will face serious problems such as generating incomparability in many realistic cases. Opting instead to endorse the first claim, I articulate and defend a spatiotemporal, expansionist view of infinite aggregation. Spatiotemporal views such as this do face some difficulties, but I show that these can be overcome. With modification, they can provide plausible comparisons in cases that we care about most. (shrink)

Longtermists claim that what we ought to do is mainly determined by how our actions might affect the very long-run future. A natural objection to longtermism is that these effects may be nearly impossible to predict — perhaps so close to impossible that, despite the astronomical importance of the far future, the expected value of our present actions is mainly determined by near-term considerations. This paper aims to precisify and evaluate one version of this epistemic objection to longtermism. To that (...) end, I develop two simple models for comparing ‘longtermist’ and ‘neartermist’ interventions, incorporating the idea that it is harder to make a predictable difference to the further future. These models yield mixed conclusions: if we simply aim to maximize expected value, and don’t mind premising our choices on minuscule probabilities of astronomical payoffs, the case for longtermism looks robust. But on some prima facie plausible empirical worldviews, the expectational superiority of longtermist interventions depends heavily on these ‘Pascalian’ probabilities. So the case for longtermism may depend either on plausible but non-obvious empirical claims or on a tolerance for Pascalian fanaticism. (shrink)

I argue that the use of a social discount rate to assess the consequences of climate policy is unhelpful and misleading. I consider two lines of justification for discounting: (i) ethical arguments for a "pure rate of time preference" and (ii) economic arguments that take time as a proxy for economic growth and the diminishing marginal utility of consumption. In both cases I conclude that, given the long time horizons, distinctive uncertainties, and particular costs and benefits at stake in the (...) climate context, discount rates are at best a poor proxy for the normative considerations they are meant to represent. (shrink)

Why, if at all, should we object to economic inequality? Some central arguments – the argument from decreasing marginal utility for example – invoke instrumental reasons and object to inequality because of its effects. Such instrumental arguments, however, often concern only the static effects of inequality and neglect its intertemporal consequences. In this article, we address this striking gap and investigate income inequality's intertemporal consequences, including its potential effects on humanity's (very) long-term future. Following recent arguments around future generations and (...) so-called longtermism, those effects might arguably matter more than inequality's short-term consequences. We assess whether we have instrumental reason to reduce economic inequality based on its intertemporal effects in the short, medium, and the very long term. We find a good short and medium-term instrumental case for lower economic inequality. We then argue, somewhat speculatively, that we have instrumental reasons for inequality reduction from a longtermist perspective too, primarily because greater inequality could increase existential risk. We thus have instrumental reasons to reduce inequality, regardless of which time-horizon we take. We then argue that from most consequentialist perspectives, this pro tanto reason also gives us all-things-considered reason. And even across most non-consequentialist views in philosophy, this argument gives us either an all-things-considered or at least weighty pro tanto reason against inequality. (shrink)

Longtermism is the view that positively influencing the long-term future is one of the key moral priorities of our time. Longtermists generally focus on humans, and neglect animals. This is a mistake. In this paper I will show that the basic argument for longtermism applies to animals at least as well as it does to humans, and that the reasons longtermists have given for ignoring animals do not withstand scrutiny. Because of their numbers, their capacity for suffering, and our ability (...) to influence their futures, animals ought to be a central concern of longtermists. Furthermore, I will suggest that longtermism is a fruitful framework for thinking about the wellbeing of animals, as it helps us to identify actions we can take now that have a reasonable chance of improving the wellbeing of animals over the very long term. (shrink)

Philosophy and Phenomenological Research, EarlyView.

I argue that many of the priority rankings that have been proposed by effective altruists seem to be in tension with apparently reasonable assumptions about the rational pursuit of our aims in the face of uncertainty. The particular issue on which I focus arises from recognition of the overwhelming importance and inscrutability of the indirect effects of our actions, conjoined with the plausibility of a permissive decision principle governing cases of deep uncertainty, known as the maximality rule. I conclude that (...) we lack a compelling decision theory that is consistent with a longtermist perspective and does not downplay the depth of our uncertainty, while also supporting orthodox effective altruist conclusions about cause prioritization. (shrink)

One widely used method for allocating health care resources involves the use of cost-effectiveness analysis (CEA) to rank treatments in terms of quality-adjusted life-years (QALYs) gained. CEA has been criticized for discriminating against people with disabilities by valuing their lives less than those of non-disabled people. Avoiding discrimination seems to lead to the ’QALY trap’: we cannot value saving lives equally and still value raising quality of life. This paper reviews existing responses to the QALY trap and argues that all (...) are problematic. Instead, we argue that adopting a moderate form of prioritarianism avoids the QALY trap and disability discrimination. (shrink)

This article is a critical survey of the debate over the value of the social discount rate, with a particular focus on climate change. The ma- jority of the material surveyed is from the economics rather than from the philosophy literature, but the emphasis of the survey itself is on founda- tions in ethical and other normative theory rather than highly technical details. I begin by locating the standard approach to discounting within the overall landscape of ethical theory, and explaining (...) the assumptions and simplifications that are needed in order to arrive at the model that is standard in the discounting literature. The article then covers the general theory of the Ramsey equation and its relationship to observed interest rates, arguments for and against a positive rate of pure time preference, the consumption elasticity of utility, and the effect of various sorts of uncertainty on the discount rate. Finally, it turns specifically to the ap- plication of this debate to the case of climate change, focussing on the recent controversy over the low discount rate used in the Stern Review of the Economics of Climate Change. (shrink)

Artificial general intelligence is a greatly anticipated technology with non-trivial existential risks, defined as machine intelligence with competence as great/greater than humans. To date, social scientists have dedicated little effort to the ethics of AGI or AGI researchers. This paper employs inductive discourse analysis of the academic literature of two intellectual groups writing on the ethics of AGI—applied and/or ‘basic’ scientific disciplines henceforth referred to as technicians (e.g., computer science, electrical engineering, physics), and philosophy-adjacent disciplines henceforth referred to as PADs (...) (e.g., philosophy, theology, anthropology). These groups agree that AGI ethics is fundamentally about mitigating existential risk. They highlight our moral obligation to future generations, demonstrate the ethical importance of better understanding consciousness, and endorse a hybrid of deontological/utilitarian normative ethics. Technicians favor technocratic AGI governance, embrace the project of ‘solving’ moral realism, and are more deontologically inclined than PADs. PADs support a democratic approach to AGI governance, are more skeptical of deontology, consider current AGI predictions as fundamentally imprecise, and are wary of using AGI for moral fact-finding. (shrink)

Plausibly, one important part of a good life is doing work that makes a contribution, or a positive difference to the world. In this paper, however, I explore contribution pessimism, the view that people will not always have adequate opportunities for making contributions. I distinguish between three interestingly different and at least initially plausible reasons why this view might be true: in slogan form, things might become too easy, they might become too good, or we might be too late. Now, (...) one response to these problems might be to deliberately attempt to undo them. However, I claim that this solution is intuitively misguided, and argue that explaining this supports a holistic approach to the value of contributions. Finally, I argue that if contribution pessimism is correct, it could provide an explanation of some widely held intuitions about issues in population ethics, with implications about the practical issue of how much priority we should give to addressing risks of human extinction. (shrink)

Ambitious value learning proposals to solve the AI alignment problem and avoid catastrophic outcomes from a possible future misaligned artificial superintelligence (such as Coherent Extrapolated Volition [CEV]) have focused on ensuring that an artificial superintelligence (ASI) would try to do what humans would want it to do. However, present and future sentient non-humans, such as non-human animals and possible future digital minds could also be affected by the ASI’s behaviour in morally relevant ways. This paper puts forward Sentientist Coherent Extrapolated (...) Volition, an alternative to CEV, that directly takes into account the interests of all sentient beings. This ambitious value learning proposal would significantly reduce the likelihood of risks of astronomical suffering from the ASI’s behaviour, and thus we have very strong pro-tanto moral reasons in favour of implementing it instead of CEV. This fact is crucial in conducting an adequate cost-benefit analysis between different ambitious value learning proposals. (shrink)

Many people assume that the extinction of humanity would be a bad thing. This article scrutinizes this apparent badness and demonstrates that on most plausible consequentialist frameworks, the extinction of humanity is not necessarily bad. The best accounts of the badness of the extinction of humanity focus on the loss of potential utility, but this loss can be offset if it is the result of sufficiently large gains by the present generation. Plausible means of calculating the goodness of outcomes accordingly (...) suggest hastening extinction even in some circumstances where the alternative is a long period of human existence at a high level. (shrink)

This thesis is about the moral reasons we have to do less than best. It consists of six chapters. Part I of the thesis proposes, extends, and defends reasons to do less than best. In Chapter One (“The Conditional Obligation”) I outline and reject two recent arguments from Joe Horton and Theron Pummer for the claim that we have a conditional obligation to bring about the most good. In Chapter Two (“Agglomeration and Agent-Relative Costs”) I argue that agent-relative costs can (...) justify permissions to less than best in a far greater range of decisions contexts than we currently believe. In Chapter Three (“Risk and Aggregation”) I discuss how anti-aggregative moral theories deal with risk, whilst proposing an argument in favour of ex-post moral theories. -/- In Part II of the thesis, I discuss longtermism, presenting a series of reasons to believe we have both significant permissions and obligations to prioritise those in the short-term. In Chapter Four (“Listening to Future People”) I criticise a recent argument presented by Andreas Mogensen for the claim we cannot have obligations grounded in the wellbeing of future people. I then show that, even permitting the existence of obligations to future people, we still have significant reasons to resist longtermism on the grounds of partiality. In Chapter Five (“The Complaints of Future People”) I argue that scepticism about aggregation is inconsistent with an obligation, or indeed even a permission, to help far future people. In Chapter Six (“Leaving the Present Behind”) I explore the moral significance of “dooming”, whilst arguing that we have a pro tanto reason of fairness to prioritise aiding those currently in existence. (shrink)

Longtermists claim that what we ought to do is mainly determined by how our actions might affect the very long-run future. A natural objection to longtermism is that these effects may be nearly impossible to predict -- perhaps so close to impossible that, despite the astronomical importance of the far future, the expected value of our present actions is mainly determined by near-term considerations. This paper aims to precisify and evaluate one version of this epistemic objection to longtermism. To that (...) end, I develop two simple models for comparing "longtermist" and "neartermist" interventions, incorporating the idea that it is harder to make a predictable difference to the further future. These models yield mixed conclusions: if we simply aim to maximize expected value, and don't mind premising our choices on minuscule probabilities of astronomical payoffs, the case for longtermism looks robust. But on some prima facie plausible empirical worldviews, the expectational superiority of longtermist interventions depends heavily on these "Pascalian" probabilities. So the case for longtermism may depend either on plausible but non-obvious empirical claims or on a tolerance for Pascalian fanaticism. (shrink)

Longtermism holds roughly that in many decision situations, the best thing we can do is what is best for the long-term future. The scope question for longtermism asks: how large is the class of decision situations for which longtermism holds? Although longtermism was initially developed to describe the situation of cause-neutral philanthropic decisionmaking, it is increasingly suggested that longtermism holds in many or most decision problems that humans face. By contrast, I suggest that the scope of longtermism may be more (...) restricted than commonly supposed. After specifying my target, swamping axiological strong longtermism (swamping ASL), I give two arguments for the rarity thesis that the options needed to vindicate swamping ASL in a given decision problem are rare. I use the rarity thesis to pose two challenges to the scope of longtermism: the area challenge that swamping ASL often fails when we restrict our attention to specific cause areas, and the challenge from option unawareness that swamping ASL may fail when decision problems are modified to incorporate agents’ limited awareness of the options available to them. GPI Working Paper No. 6-2021. (shrink)

I argue that moral philosophers have either misunderstood the problem of moral demandingness or at least failed to recognize important dimensions of the problem that undermine many standard assumptions. It has been assumed that utilitarianism concretely directs us to maximize welfare within a generation by transferring resources to people currently living in extreme poverty. In fact, utilitarianism seems to imply that any obligation to help people who are currently badly off is trumped by obligations to undertake actions targeted at improving (...) the value of the long-term future. Reflecting on the demands of beneficence in respect of the value of the far future forces us to view key aspects of the problem of moral demandingness in a very different light. GPI Working Paper No. 1-2020. (shrink)

I argue that many of the priority rankings that have been proposed by effective altruists seem to be in tension with apparently reasonable assumptions about the rational pursuit of our aims in the face of uncertainty. The particular issue on which I focus arises from recognition of the overwhelming importance and inscrutability of the indirect effects of our actions, conjoined with the plausibility of a permissive decision principle governing cases of deep uncertainty, known as the maximality rule. I conclude that (...) we lack a compelling decision theory that is consistent with a long-termist perspective and does not downplay the depth of our uncertainty while supporting orthodox effective altruist conclusions about cause prioritization. GPI Working Paper No. 2-2019. (shrink)

This paper considers the argument according to which, because we should regard it as a priori very unlikely that we are among the most important people who will ever exist, we should increase our confidence that the human species will not persist beyond the current historical era, which seems to represent a crucial juncture in human history and perhaps even the history of life on earth. The argument is a descendant of the Carter-Leslie Doomsday Argument, but I show that it (...) does not inherit the crucial flaw in its immediate ancestor. Nonetheless, we are not forced to follow the argument where it leads if we instead significantly decrease our confidence that we can affect the long-run future of humanity. GPI Working Paper No. 1-2019. (shrink)

Consequentialism is thought to be in significant conflict with animal rights theory because it does not regard activities such as confinement, killing, and exploitation as in principle morally wrong. Proponents of the “Logic of the Larder” argue that consequentialism results in an implausibly pro-exploitation stance, permitting us to eat farmed animals with positive well- being to ensure future such animals exist. Proponents of the “Logic of the Logger” argue that consequentialism results in an implausibly anti-conservationist stance, permitting us to exterminate (...) wild animals with negative well-being to ensure future such animals do not exist. We argue that this conflict is overstated. Once we have properly accounted for indirect effects, such as the role that our policies play in shaping moral attitudes and behavior and the importance of accepting policies that are robust against deviation, we can see that consequentialism may converge with animal rights theory significantly, even if not entirely. (shrink)

Suppose you want to do as much good as possible. What should you do? According to members of the effective altruism movement—which has produced much of the thinking on this issue and counts several moral philosophers as its key protagonists—we should prioritise among the world’s problems by assessing their scale, solvability, and neglectedness. Once we’ve done this, the three top priorities, not necessarily in this order, are (1) aiding the world’s poorest people by providing life-saving medical treatments or alleviating poverty (...) itself, (2) preventing global catastrophic risks, such as those posed by nuclear war or rogue artificial intelligence, and (3) ending factory farming. These claims are both plausible and striking. If correct, they should prompt a stark revision of how we approach our altruistic activities. However, the project of determining how to do the most good—as opposed to say, whether we should do the most good—has only recently, within the last ten years, become the subject of serious academic attention. Many key claims have not yet been carefully scrutinised. This is a cause for concern: are effective altruists doing good badly? In this thesis, I critique and develop some of the latest claims about how individuals can do the most good. I do this in three areas: the value of saving lives (preventing premature deaths), how best to improve lives (making people happier during their lives), and cause prioritisation methodology (frameworks for determining which problems are the highest priorities). In each case, I raise novel theoretical considerations that, when incorporated, change the analysis. Roughly speaking, my main conclusions are (1) saving lives is not as straightforwardly good we tend to suppose, may not be good at all, and is not clearly a priority; (2) happiness can be measured through self-reports and, based on the self-reported evidence, treating mental health stands out as an overlooked problem that may be an even more cost-effective way to improve lives than alleviating poverty; (3) the cause prioritisation methodology proposed by effective altruists needs to be moderately reconceptualised and, when it is, it turns out it is not as illuminating a tool as we might have thought and hoped. (shrink)

Purpose This paper aims to formalize long-term trajectories of human civilization as a scientific and ethical field of study. The long-term trajectory of human civilization can be defined as the path that human civilization takes during the entire future time period in which human civilization could continue to exist. -/- Design/methodology/approach This paper focuses on four types of trajectories: status quo trajectories, in which human civilization persists in a state broadly similar to its current state into the distant future; catastrophe (...) trajectories, in which one or more events cause significant harm to human civilization; technological transformation trajectories, in which radical technological breakthroughs put human civilization on a fundamentally different course; and astronomical trajectories, in which human civilization expands beyond its home planet and into the accessible portions of the cosmos. -/- Findings Status quo trajectories appear unlikely to persist into the distant future, especially in light of long-term astronomical processes. Several catastrophe, technological transformation and astronomical trajectories appear possible. -/- Originality/value Some current actions may be able to affect the long-term trajectory. Whether these actions should be pursued depends on a mix of empirical and ethical factors. For some ethical frameworks, these actions may be especially important to pursue. (shrink)

Given the deep disagreement surrounding population axiology, one should remain uncertain about which theory is best. However, this uncertainty need not leave one neutral about which acts are better or worse. We show that as the number of lives at stake grows, the Expected Moral Value approach to axiological uncertainty systematically pushes one towards choosing the option preferred by the Total and Critical Level views, even if one’s credence in those theories is low.

There are two types of artificial general intelligence (AGI) safety solutions: global and local. Most previously suggested solutions are local: they explain how to align or “box” a specific AI (Artificial Intelligence), but do not explain how to prevent the creation of dangerous AI in other places. Global solutions are those that ensure any AI on Earth is not dangerous. The number of suggested global solutions is much smaller than the number of proposed local solutions. Global solutions can be divided (...) into four groups: 1. No AI: AGI technology is banned or its use is otherwise prevented; 2. One AI: the first superintelligent AI is used to prevent the creation of any others; 3. Net of AIs as AI police: a balance is created between many AIs, so they evolve as a net and can prevent any rogue AI from taking over the world; 4. Humans inside AI: humans are augmented or part of AI. We explore many ideas, both old and new, regarding global solutions for AI safety. They include changing the number of AI teams, different forms of “AI Nanny” (non-self-improving global control AI system able to prevent creation of dangerous AIs), selling AI safety solutions, and sending messages to future AI. Not every local solution scales to a global solution or does it ethically and safely. The choice of the best local solution should include understanding of the ways in which it will be scaled up. Human-AI teams or a superintelligent AI Service as suggested by Drexler may be examples of such ethically scalable local solutions, but the final choice depends on some unknown variables such as the speed of AI progress. (shrink)

In recent years billions of philanthropic dollars have been deployed through a movement and philosophy known as Effective Altruism, notably through the organizations Open Philanthropy, GiveWell, Good Ventures, and the over 3,200 persons taking the ‘Giving What We Can’ pledge to limit their personal income and donate the remainder to charity. Effective Altruism, or EA, explicitly aims to ‘use evidence and reasoning to determine the most effective ways to benefit others.’ Within the EA community there are competing viewpoints on how (...) to evaluate effectiveness, which forms of evidence and reasoning are best, and several leading philosophers with their own adherents and value systems. The discussions surrounding the employment of these viewpoints can grow heated, with one commenter complaining that ‘Effective Altruism markets itself as being centered on addressing global poverty, when in fact it is centered on manipulating people into believing in the imaginary AI doomsday.’ This complaint highlights a major schism within EA. On one side is the empirical evaluation of existing altruistic interventions according to their own internal justifications - the attempts to judge the most effective way to solve current issues like global poverty, using hard data and statistics. On the other side is the a logically based rationalist effort to determine undervalued new interventions, bring them appropriate funding, and potentially solve issues which are ignored by mainstream altruism. From within the community, these two parts of the current EA community are often considered two sides of the same coin. Extrapolations of techniques used to evaluate current interventions often provide the justification for attempting new interventions, and can highlight areas worth a closer examination or the allocation of more funding. Over time, a single paradigm has come to dominate many of these extrapolations, leading to the complaint above. Nicholas Beckstead, a program officer at Open Philanthropy, laid out the argument underpinning this paradigm in a 2013 doctoral thesis titled ‘On the Overwhelming Importance of Shaping the Far Future.’ The argument in this thesis, which we will call the Far Future Trillions Argument, is central to many of the most heated discussions in how to best employ the billions of dollars of the Effective Altruism. After careful consideration it is our reasoned opinion that the Far Future Trillions Argument and its resulting recommendations, while internally coherent and defensible, are not within the remit of a movement characterized by the use of evidence in determining the effectiveness of altruistic interventions. This overall conclusion is based on seven separate arguments. The first argument is based on empirical evaluation of foreign aid interventions, and concludes that characteristics of the FFTA are inimical to effective interventions. Our second and third arguments conclude that the FFTA lacks certain characteristics of pragmatic models, and requires empirically fallacious methodological assumptions to connect far future outcomes with present interventions. Fourth and fifth, we argue that the purely rationalist, philosophical nature of the FFTA divorces it from evidence in a way that inherently foils the determination of the effectiveness of interventions based upon it. Our sixth and seventh arguments take issue with the ongoing privileged employ of the version of utilitarianism and moral impartiality toward future persons used in the FFTA and its offshoot justifications, asserting that the inclusion of a full and wider scope of normative justifications is necessary for pragmatic modeling. (shrink)