A small but growing number of studies have aimed to understand, assess and reduce existential risks, or risks that threaten the continued existence of mankind. However, most attention has been focused on known and tangible risks. This paper proposes a heuristic for reducing the risk of black swan extinction events. These events are, as the name suggests, stochastic and unforeseen when they happen. Decision theory based on a fixed model of possible outcomes cannot properly deal with this kind of event. (...) Neither can probabilistic risk analysis. This paper will argue that the approach that is referred to as engineering safety could be applied to reducing the risk from black swan extinction events. It will also propose a conceptual sketch of how such a strategy may be implemented: isolated, self-sufficient, and continuously manned underground refuges. Some characteristics of such refuges are also described, in particular the psychosocial aspects. Furthermore, it is argued that this implementation of the engineering safety strategy safety barriers would be effective and plausible and could reduce the risk of an extinction event in a wide range of possible scenarios. Considering the staggering opportunity cost of an existential catastrophe, such strategies ought to be explored more vigorously. (shrink)

As we learn more about the human brain, novel biotechnological means to modulate human behaviour and emotional dispositions become possible. These technologies could be used to enhance our morality. Moral bioenhancement, an instance of human enhancement, alters a person’s dispositions, emotions or behaviour in order to make that person more moral. I will argue that moral bioenhancement could be carried out in three different ways. The first strategy, well known from science fiction, is behavioural enhancement. The second strategy, favoured by (...) prominent defenders of moral bioenhancement, is emotional enhancement. The third strategy is the enhancement of moral dispositions, such as empathy and inequity aversion. I will argue that we ought to implement a combination of the second and third strategies. Furthermore, I will argue that the usual arguments against other instances of human enhancement do not apply to moral bioenhancement, or apply only to the first strategy, behavioural enhancement. (shrink)

Brain machine interface (BMI) technology makes direct communication between the brain and a machine possible by means of electrodes. This paper reviews the existing and emerging technologies in this field and offers a systematic inquiry into the relevant ethical problems that are likely to emerge in the following decades.

This article will explore a problem which is related to our moral obligations towards species. Although the re-creation of extinct animals has been discussed to some degree both in lay deliberations as well as by scientists, advocates tend to emphasize the technological and scientific value of such an endeavour, and the “coolness” factor, 32–33, 2013). This article will provide an argument in favour of re-creation based on normative considerations. The environmentalist community generally accepts that it is wrong to exterminate species, (...) for reasons beyond any instrumental value these species may have. It is often also claimed that humanity has a collective responsibility to either preserve or at least to not exterminate species. These two beliefs are here assumed to be correct. The argument presented here departs from and places these two ideas in a deontological framework, from which it is argued that when humanity causes the extinction of a species, this is a moral transgression, entailing a residual obligation. Such an obligation implies a positive duty to mitigate any harm caused by our moral failure. In light of recent scientific progress in the field of genetic engineering, it will be argued that humanity has a prima facie obligation to re-create species whose extinction mankind may have caused, also known as de-extinction. (shrink)

An intelligent machine surpassing human intelligence across a wide set of skills has been proposed as a possible existential catastrophe. Among those concerned about existential risk related to artificial intelligence, it is common to assume that AI will not only be very intelligent, but also be a general agent. This article explores the characteristics of machine agency, and what it would mean for a machine to become a general agent. In particular, it does so by articulating some important differences between (...) belief and desire in the context of machine agency. One such difference is that while an agent can by itself acquire new beliefs through learning, desires need to be derived from preexisting desires or acquired with the help of an external influence. Such influence could be a human programmer or natural selection. We argue that to become a general agent, a machine needs productive desires, or desires that can direct behavior across multiple contexts. However, productive desires cannot sui generis be derived from non-productive desires. Thus, even though general agency in AI could in principle be created by human agents, general agency cannot be spontaneously produced by a non-general AI agent through an endogenous process. In conclusion, we argue that a common AI scenario, where general agency suddenly emerges in a non-general agent AI, such as DeepMind’s superintelligent board game AI AlphaZero, is not plausible. (shrink)

The possibility of extraterrestrial intelligence (ETI) exists despite no observed evidence, and the risks and benefits of actively searching for ETI (Active SETI) have been debated. Active SETI has been criticized for potentially exposing humanity to existential risk, and a recent game-theoretical model highlights the Hobbesian trap that could occur following contact if mutual distrust leads to mutual destruction. We argue that observing a nearby ETI would suggest the existence of many unobserved ETI. This would expand the game and implies (...) that there may be a mechanism that prevents ETI from attacking us. As a result, the Hobbesian trap would be avoided if both parties assumed the other is not likely to attack. (shrink)

Disease prioritarianism is a principle that is often implicitly or explicitly employed in the realm of healthcare prioritization. This principle states that the healthcare system ought to prioritize the treatment of disease before any other problem. This article argues that disease prioritarianism ought to be rejected. Instead, we should adopt ‘the problem-oriented heuristic’ when making prioritizations in the healthcare system. According to this idea, we ought to focus on specific problems and whether or not it is possible and efficient to (...) address them with medical means. This has radical implications for the extension of the healthcare system. First, getting rid of the binary disease/no-disease dichotomy implicit in disease prioritarianism would improve the ability of the healthcare system to address chronic conditions and disabilities that often defy easy classification. Second, the problem-oriented heuristic could empower medical practitioners to address social problems without the need to pathologize these conditions. Third, the problem-oriented heuristic clearly states that what we choose to treat is a normative consideration. Under this assumption, we can engage in a discussion on de-medicalization without distorting preconceptions. Fourth, this pragmatic and de-compartmentalizing approach should allow us to reconsider the term ‘efficiency’. (shrink)

The last decade has seen significant improvements in Artificial Intelligence (AI) technologies, including robotics, machine vision, speech recognition and text generation. Increasing automation will undoubtedly affect the future of work, and discussions on how the development of AI in the workplace will impact labor markets often include two scenarios: (1) labor replacement and (2) labor enabling. The former involves replacing workers with machines, while the latter assumes that human-machine cooperation can significantly improve worker productivity. In this context, it is often (...) argued that (1) could lead to mass unemployment and that (2) therefore would be more desirable. We argue, however, that the labor-enabling scenario conflates two distinct possibilities. On the one hand, technology can increase productivity while also promoting “the goods of work,” such as the opportunity to pursue excellence, experience a sense of community and contribute to society (human augmentation). On the other hand, higher productivity can also be achieved in a way that reduces opportunities for the “goods of work” and/or increases “the bads of work,” such as injury, reduced physical and mental health, reduction of autonomy, privacy and human dignity (human stunting). We outline the differences of these outcomes and discuss the implications for the labor market in the context of contemporaneous discussions on the value of work and human wellbeing. (shrink)

The biosphere represents the global sum of all ecosystems. According to a prominent view in environmental ethics, ecocentrism, these ecosystems matter for their own sake, and not only because they contribute to human ends. As such, some ecocentrists are critical of the modern industrial civilization, and a few even argue that an irreversible collapse of the modern industrial civilization would be a good thing. However, taking a longer view and considering the eventual destruction of the biosphere by astronomical processes, we (...) argue that humans, a species with considerable technological know-how and industrial capacity could intervene to extend the lifespan of Earth’s biosphere, perhaps by several billion years. We argue that human civilization, despite its flaws and harmful impacts on many ecosystems, is the biosphere’s best hope of avoiding premature destruction. We argue that proponents of ecocentrism, even those who wholly disregard anthropocentric values, have a strong moral reason preserve the modern industrial civilization, for as long as needed to ensure biosphere survival. (shrink)

Machine learning algorithms are increasingly used to support decision-making in the exercise of public authority. Here, we argue that an important consideration has been overlooked in previous discussions: whether the use of ML undermines the democratic legitimacy of public institutions. From the perspective of democratic legitimacy, it is not enough that ML contributes to efficiency and accuracy in the exercise of public authority, which has so far been the focus in the scholarly literature engaging with these developments. According to one (...) influential theory, exercises of administrative and judicial authority are democratically legitimate if and only if administrative and judicial decisions serve the ends of the democratic law maker, are based on reasons that align with these ends and are accessible to the public. These requirements are not satisfied by decisions determined through ML since such decisions are determined by statistical operations that are opaque in several respects. However, not all ML-based decision support systems pose the same risk, and we argue that a considered judgment on the democratic legitimacy of ML in exercises of public authority need take the complexity of the issue into account. This paper outlines considerations that help guide the assessment of whether a ML undermines democratic legitimacy when used to support public decisions. We argue that two main considerations are pertinent to such normative assessment. The first is the extent to which ML is practiced as intended and the extent to which it replaces decisions that were previously accessible and based on reasons. The second is that uses of ML in exercises of public authority should be embedded in an institutional infrastructure that secures reason giving and accessibility. (shrink)

Essay I explores brain machine interface technologies. These make direct communication between the brain and a machine possible by means of electrical stimuli. This essay reviews the existing and emerging technologies in this field and offers a systematic inquiry into the relevant ethical problems that are likely to emerge in the following decades. Essay II, co-written with professor Sven-Ove Hansson, presents a novel procedure to engage the public in ethical deliberations on the potential impacts of brain machine interface technology. We (...) call this procedure a Convergence seminar, a form of scenario-based group discussion that is founded on the idea of hypothetical retrospection. The theoretical background of this procedure and the results of the five seminars are presented here. Essay III discusses moral enhancement, an instance of human enhancement that alters a person’s dispositions, emotions or behavior in order to make that person more moral. Moral enhancement could be carried out in three different ways. The first strategy is behavioral enhancement. The second strategy, favored by prominent defenders of moral enhancement, is emotional enhancement. The third strategy is the enhancement of moral dispositions, such as empathy and inequity aversion. I argue that we ought to implement a combination of the second and third strategies. (shrink)

Essay I explores brain machine interface technologies. These make direct communication between the brain and a machine possible by means of electrical stimuli. This essay reviews the existing and emerging technologies in this field and offers an inquiry into the ethical problems that are likely to emerge. Essay II, co-written with professor Sven-Ove Hansson, presents a novel procedure to engage the public in deliberations on the potential impacts of technology. This procedure, convergence seminar, is a form of scenario-based discussion that (...) is founded on the idea of hypothetical retrospection. The theoretical background and the results of the five seminars are presented. Essay III discusses moral bioenhancement, an instance of human enhancement that alters a person’s dispositions, emotions or behavior. Moral bioenhancement could be carried out in three different ways. The first strategy is behavioral enhancement. The second strategy, favored by prominent defenders of moral enhancement, is emotional enhancement. The third strategy is the enhancement of moral dispositions, such as empathy and inequity aversion. I argue that we ought to implement a combination of the second and third strategies. Essay IV considers the possibility and potential desirability of sensory enhancement. It is proposed that existing sensory modalities in vertebrate animals are proof of concept of what is biologically possible to create in humans. Three considerations on the normative aspects of sensory enhancement are also presented in this essay. Essay V rejects disease prioritarianism, the idea that the healthcare system ought to prioritize the treatment of diseases. Instead, an approach that focuses on what medicine can accomplish is proposed. Essay VI argues that from the idea that species have an intrinsic value and that humanity has a collective responsibility to protect animal species from extinction, the conclusion that we ought to recreate species follows. Essay VII argues that unknown existential risks have not been properly addressed. It proposes a heuristic for doing so, and a concrete strategy. This strategy consists in building refuges that could withstand a large number of catastrophic events. (shrink)

We present a novel procedure to engage the public in ethical deliberations on the potential impacts of brain machine interface technology. We call this procedure a convergence seminar, a form of scenario-based group discussion that is founded on the idea of hypothetical retrospection. The theoretical background of this procedure and the results of five seminars are presented.