Among ethicists and engineers within robotics there is an ongoing discussion as to whether ethical robots are possible or even desirable. We answer both of these questions in the positive, based on an extensive literature study of existing arguments. Our contribution consists in bringing together and reinterpreting pieces of information from a variety of sources. One of the conclusions drawn is that artifactual morality must come in degrees and depend on the level of agency, autonomy and intelligence of the machine. (...) Moral concerns for agents such as intelligent search machines are relatively simple, while highly intelligent and autonomous artifacts with significant impact and complex modes of agency must be equipped with more advanced ethical capabilities. Systems like cognitive robots are being developed that are expected to become part of our everyday lives in future decades. Thus, it is necessary to ensure that their behaviour is adequate. In an analogy with artificial intelligence, which is the ability of a machine to perform activities that would require intelligence in humans, artificial morality is considered to be the ability of a machine to perform activities that would require morality in humans. The capacity for artificial (artifactual) morality, such as artifactual agency, artifactual responsibility, artificial intentions, artificial (synthetic) emotions, etc., come in varying degrees and depend on the type of agent. As an illustration, we address the assurance of safety in modern High Reliability Organizations through responsibility distribution. In the same way that the concept of agency is generalized in the case of artificial agents, the concept of moral agency, including responsibility, is generalized too. We propose to look at artificial moral agents as having functional responsibilities within a network of distributed responsibilities in a socio-technological system. This does not take away the responsibilities of the other stakeholders in the system, but facilitates an understanding and regulation of such networks. It should be pointed out that the process of development must assume an evolutionary form with a number of iterations because the emergent properties of artifacts must be tested in real world situations with agents of increasing intelligence and moral competence. We see this paper as a contribution to the macro-level Requirement Engineering through discussion and analysis of general requirements for design of ethical robots. (shrink)

I explore physics implications of the External Reality Hypothesis (ERH) that there exists an external physical reality completely independent of us humans. I argue that with a sufficiently broad definition of mathematics, it implies the Mathematical Universe Hypothesis (MUH) that our physical world is an abstract mathematical structure. I discuss various implications of the ERH and MUH, ranging from standard physics topics like symmetries, irreducible representations, units, free parameters, randomness and initial conditions to broader issues like consciousness, parallel universes and (...) Gödel incompleteness. I hypothesize that only computable and decidable (in Gödel’s sense) structures exist, which alleviates the cosmological measure problem and may help explain why our physical laws appear so simple. I also comment on the intimate relation between mathematical structures, computations, simulations and physical systems. (shrink)

Artificial intelligence (AI) and robotics are digital technologies that will have significant impact on the development of humanity in the near future. They have raised fundamental questions about what we should do with these systems, what the systems themselves should do, what risks they involve, and how we can control these. - After the Introduction to the field (§1), the main themes (§2) of this article are: Ethical issues that arise with AI systems as objects, i.e., tools made and used (...) by humans. This includes issues of privacy (§2.1) and manipulation (§2.2), opacity (§2.3) and bias (§2.4), human-robot interaction (§2.5), employment (§2.6), and the effects of autonomy (§2.7). Then AI systems as subjects, i.e., ethics for the AI systems themselves in machine ethics (§2.8) and artificial moral agency (§2.9). Finally, the problem of a possible future AI superintelligence leading to a “singularity” (§2.10). We close with a remark on the vision of AI (§3). - For each section within these themes, we provide a general explanation of the ethical issues, outline existing positions and arguments, then analyse how these play out with current technologies and finally, what policy consequences may be drawn. (shrink)

In his 1993 article ‘The Coming Technological Singularity: How to survive in the posthuman era’ the computer scientist Virnor Vinge speculated that developments in artificial intelligence might reach a point where improvements in machine intelligence result in smart AI’s producing ever-smarter AI’s. According to Vinge the ‘singularity’, as he called this threshold of recursive self-improvement, would be a ‘transcendental event’ transforming life on Earth in ways that unaugmented humans are not equipped to envisage. In this paper I argue Vinge’s idea (...) of a technologically led intelligence explosion is philosophically important because it requires us to consider the prospect of a posthuman condition succeeding the human one. What is the ‘humanity’ to which the posthuman is ‘post’? Does the possibility of a posthumanity presuppose that there is a ‘human essence’, or is there some other way of conceiving the human-posthuman difference? I argue that the difference should be conceived as a historically emergent disconnection between individuals, not in terms of the presence or lack of essential properties. I also suggest that these individuals should not be conceived in narrow biological terms but in ‘wide’ terms permitting biological, cultural and technological relations of descent between human and posthuman. Finally, I consider the ethical implications of this metaphysics of the posthuman. If, as I claim, the posthuman difference is not one between kinds but between individuals, we cannot specify its nature a priori but only a posteriori. Thus the only way to evaluate the posthuman condition would be to witness the emergence of posthumans. The implications of this are somewhat paradoxical. We are not currently in a position to evaluate the posthuman condition. Since there are no posthumans, the condition of posthumanity is not defined. However, posthumans could result from some iteration of our current technical activity, so we have an interest in understanding what they might be like. It follows that we have an interest in making or becoming posthumans. (shrink)

John Searle distinguished between weak and strong artificial intelligence (AI). This essay discusses a third alternative, mild AI, according to which a machine may be capable of possessing a species of mentality. Using James Fetzer's conception of minds as semiotic systems, the possibility of what might be called ``mild AI'' receives consideration. Fetzer argues against strong AI by contending that digital machines lack the ground relationship required of semiotic systems. In this essay, the implementational nature of semiotic processes posited by (...) Charles S. Peirce's triadic sign relation is re-examined in terms of the underlying dispositional processes and the ontological levels they would span in an inanimate machine. This suggests that, if non-human mentality can be replicated rather than merely simulated in a digital machine, the direction to pursue appears to be that of mild AI. (shrink)

The challenge of designing computer systems and robots with the ability to make moral judgments is stepping out of science fiction and moving into the laboratory. Engineers and scholars, anticipating practical necessities, are writing articles, participating in conference workshops, and initiating a few experiments directed at substantiating rudimentary moral reasoning in hardware and software. The subject has been designated by several names, including machine ethics, machine morality, artificial morality, or computational morality. Most references to the challenge elucidate one facet or (...) another of what is a very rich topic. This paper will offer a brief overview of the many dimensions of this new field of inquiry. (shrink)

The thinkers who have reflected on the problem of a coming superintelligence have generally seen the issue as a technological problem, a problem of how to control what the superintelligence will do. I argue that this approach is probably mistaken because it is based on questionable assumptions about the behavior of intelligent agents and, moreover, potentially counterproductive because it might, in the end, bring about the existential catastrophe that it is meant to prevent. I contend that the problem posed by (...) a future superintelligence will likely be a political problem, that is, one of establishing a peaceful form of coexistence with other intelligent agents in a situation of mutual vulnerability, and not a technological problem of control. (shrink)

Almost all computational models of the mind and brain ignore details about neurotransmitters, hormones, and other molecules. The neglect of neurochemistry in cognitive science would be appropriate if the computational properties of brains relevant to explaining mental functioning were in fact electrical rather than chemical. But there is considerable evidence that chemical complexity really does matter to brain computation, including the role of proteins in intracellular computation, the operations of synapses and neurotransmitters, and the effects of neuromodulators such as hormones. (...) Neurochemical computation has implications for understanding emotions, cognition, and artificial intelligence. (shrink)

If the computational theory of mind is right, then minds are realized by machines. There is an ordered complexity hierarchy of machines. Some finite machines realize finitely complex minds; some Turing machines realize potentially infinitely complex minds. There are many logically possible machines whose powers exceed the Church–Turing limit (e.g. accelerating Turing machines). Some of these supermachines realize superminds. Superminds perform cognitive supertasks. Their thoughts are formed in infinitary languages. They perceive and manipulate the infinite detail of fractal objects. They (...) have infinitely complex bodies. Transfinite games anchor their social relations. (shrink)

It is remarkable how much robotics research is promoted by appealing to the idea that the only way to deal with a looming demographic crisis is to develop robots to look after older persons. This paper surveys and assesses the claims made on behalf of robots in relation to their capacity to meet the needs of older persons. We consider each of the roles that has been suggested for robots in aged care and attempt to evaluate how successful robots might (...) be in these roles. We do so from the perspective of writers concerned primarily with the quality of aged care, paying particular attention to the social and ethical implications of the introduction of robots, rather than from the perspective of robotics, engineering, or computer science. We emphasis the importance of the social and emotional needs of older persons—which, we argue, robots are incapable of meeting—in almost any task involved in their care. Even if robots were to become capable of filling some service roles in the aged-care sector, economic pressures on the sector would most likely ensure that the result was a decrease in the amount of human contact experienced by older persons being cared for, which itself would be detrimental to their well-being. This means that the prospects for the ethical use of robots in the aged-care sector are far fewer than first appears. More controversially, we believe that it is not only misguided, but actually unethical, to attempt to substitute robot simulacra for genuine social interaction. A subsidiary goal of this paper is to draw attention to the discourse about aged care and robotics and locate it in the context of broader social attitudes towards older persons. We conclude by proposing a deliberative process involving older persons as a test for the ethics of the use of robots in aged care. (shrink)

Bill Joy’s deep pessimism is now famous. Why the Future Doesn’t Need Us, his defense of that pessimism, has been read by, it seems, everyone—and many of these readers, apparently, have been converted to the dark side, or rather more accurately, to the future-is-dark side. Fortunately (for us; unfortunately for Joy), the defense, at least the part of it that pertains to AI and robotics, fails. Ours may be a dark future, but we cannot know that on the basis of (...) Joy’s reasoning. On the other hand, we ought to fear a good deal more than fear itself: we ought to fear not robots, but what some of us may do with robots. (shrink)

This essay explores how the human biological self is socially constructed, and rejects various truisms that define our character. Rather than being stand-alone entities, the human biological self forms what biologists call “superorganisms” and what I call “poly-super-organisms.” Thus, along with prokaryotes , viruses, and other entities, we are combined in an inseparable menagerie of species that is spread across multiple bodies. Biologists claim that only males and females are organisms. As described here, however, human sperm and eggs are equally (...) entitled to that rank. Much about human reproduction is also socially constructed. For example, contrary to scientific wisdom, humans have always reproduced both sexually and asexually. Moreover, human life does not begin between conception and birth, and neither event creates new life. In addition, some cancer cells are organisms; they are contagious entities formed by asexual human reproduction. Finally, while science universally describes humans as being living entities, accumulating evidence makes clear that the life concept is failing. In fact, much that originates from human bodies does not fall under current biological or cultural classifications.How we construct the human biological self has consequences for understanding our health, how we reproduce, and what we are. These social constructions greatly influence our views on abortion, stem-cell research, human cloning, and our post-human future. A more accurate construction of the human biological self will help us better navigate these contentious issues. (shrink)

This paper adopts a legal perspective to counter some exaggerations of today’s debate on the social understanding of robotics. According to a long and well-established tradition, there is in fact a relative strong consensus among lawyers about some key notions as, say, agency and liability in the current use of robots. However, dealing with a field in rapid evolution, we need to rethink some basic tenets of the contemporary legal framework. In particular, time has come for lawyers to acknowledge that (...) some acts of robots should be considered as a new source of legal responsibility for others’ behaviour. (shrink)

The paper focuses on processes of normalization through which dis/ability is simultaneously produced in specific collectives, networks, and socio-technological systems that enable the construction of such demarcations. Our point of departure is the cochlear implant, a neuroprosthetic device intended to replace and/or augment the function of the damaged inner ear. Unlike hearing aids, which amplify sounds, the CI does the work of damaged hair cells in the inner ear by providing sound signals to the brain. We examine the processes of (...) the CI’s genesis as well as its specific uses by and interrelations to the different and divergent actors that the CI assembles. We argue that the technological device and the implicated normalization process mobilize complex effects in varying socio-technical arrangements. The CI is conceived as a “boundary object” [89] or a “quasi-object” [49, 83], i.e., a metastabilized medium of translation that coordinates social, cultural, and technological action. Although intended to transform non-hearing or hard of hearing people into competent and “normal” hearing subjects, the CI system reproduces the asymmetrical structures of the disability discourse [14] through its function of “developing and maintaining coherence between intersecting social worlds” [89, 393]. Additionally, it initiates controversial discourses that have resulted in new forms of biosocial collectivities ranging from cochlear implantees with normal human hearing to human configurations who have passed through human enhancement. Our approach is thus situated at the intersection of disability and media studies and tackles the particular conditions technological media configurations impose upon the production of dis/ability. (shrink)

Technological imagination and actual technological achievements have always been two very different things. Sudden and unpredictable events always seem to intervene between our visions regarding possible futures and the subsequent concrete realizations. Thus, our ideas and projects are continually being redirected. In the field of naturoids—that is, artificial systems designed to simulate certain features of natural ones—the above-mentioned tendency is particularly frequent and illuminating, giving all prophecies a flavor of fiction rather than of reliable technological forecast. The dream of reproducing (...) complete natural systems—man included—by means of technological strategies clashes with the “nature” of artificial objects and also with that of natural exemplars. The only certainty is that the future will exhibit growing rates of variety and heterogeneity, and we cannot know, from our present standpoint, what this will imply for our species, or, at least, for our cultures. (shrink)

After 50 years, the fields of artificial intelligence and robotics capture the imagination of the general public while, at the same time, engendering a great deal of fear and skepticism. Isaac Asimov recognized this deep-seated misconception of technology and created the Three Laws of Robotics. The first part of this paper examines the underlying fear of intelligent robots, revisits Asimov’s response, and reports on some current opinions on the use of the Three Laws by practitioners. Finally, an argument against robotic (...) rebellion is made along with a call for personal responsibility and suggestions for implementing safety constraints in intelligent robots. (shrink)

To reassure those concerned about wholesale discontinuity between human existence and posthumanity, transhumanists assert shared ground with antiquity on vital challenges and aspirations. Because their claims reflect key misconceptions, there is no shared vision for transhumanists to invoke. Having exposed their misuses of Prometheus, Plato, and Aristotle, I show that not only do transhumanists and antiquity crucially diverge on our relation to ideals, contrast-dependent aspiration, and worthy endeavors but that illumining this divide exposes central weaknesses in transhumanist argumentation. What is (...) more, antiquity’s handling of these topics suggests a way through the impasse in current enhancement debates about human “nature” and helps to resolve a tension within transhumanists’ accounts of what our best moments signify about the ontological requirements for real flourishing. (shrink)

Increasingly, innovation in artificial intelligence technologies portends the re-conceptualization of human existentiality along the paradigm of posthumanism. An exposition of this through a critical culturo-historical methodology uncloaks the Eurocentric genitive basis of the philosophical anthropology that underpins this technological posthumanism, as well as its dystopian possibilities. As a contribution to obviating the latter, an Africanist civilizational humanism proclaimed by Pixley ka Isaka Seme is proffered as a plausible alternative paradigm for humanity’s technological advancement. Seme, a pan-Africanist thinker of the early (...) twentieth century, proclaimed humanistic-spirituality as the indispensable gift African Civilisation-in-its-renaissance is yet to offer global humanity. His postulation is being provided as a contribution to the archive on cross-cultural ethics of artificial intelligence. (shrink)

As the artificial intelligence of computers grows ever-more sophisticated and continues to surpass the capacities of human minds in many ways, people are forced to question alleged ontological categories that separate humans from machines. As we are entering the world which is populated by non-enhanced and enhanced humans, cyborgs, robots, androids, avatars, and clones among them, the desire for evolutionary mastery of the natural world has taken on the two main directions: merging with machines in disembodied forms or embodied forms. (...) As a path to breaking past the discontinuity between humans and machines and enter into a world beyond the “fourth” discontinuity, machines are viewed as an evolutionary step toward the “perfection” or “immortality” of humans. However, this popular, instrumental views of machines, stemming from the existential death anxiety and the hope for transcending mortality, reveals the karmic dilemma of desiring or grasping something. We shall discuss the possibility that machines can present, ultimately, a revolutionary step rather than an evolutionary step toward understanding “who we are.” The path toward a continuity with machines lies not in our desire for merging with the robots, but in recognizing the arbitrary nature of all such identity categories. This radical understanding of the self-identity can be seen as a facet of enlightened experience. (shrink)

Robot enthusiasts envision robots will become a “race unto themselves” as they cohabit with the humankind one day. Profound questions arise surrounding one of the major areas of research in the contemporary world—that concerning artificial intelligence. Fascination and anxiety that androids impose upon us hinges on how we come to conceive of the “Cultural Other.” Applying the notion of the “other” in multicultural research process, we will explore how the “Other” has been used to illustrate values and theories about robots, (...) as a mirror for the self. In this paper, we focus on the social, cultural, and religious implications of humans’ attitudes toward relationships between humans with robots. Six major views on humanoid robots are proposed: (1) robots as the “Frightening Other,” (2) robots as the “Subhuman Other,” (3) robots as the “Human Substitute,” (4) robots as the “Sentient Other,” (5) robots as the “Divine Other,” and (6) robots as the “Co-evolutionary Path to Immortality.” The likely and preferable scenario is the last one, which is compatible with an optimistic posthuman world in our evolutionary future. We imagine whether humans will meet the challenge of loving all living and non-living beings (including mechanical entities) might be the key to the co-evolution of both species and the ultimate happiness. (shrink)

Information management systems improve the retention of information in large collections. As such they act as memory prostheses, implying an ideal basis in human memory models. Since humans process information by association, and situate it in the context of space and time, systems should maximize their effectiveness by mimicking these functions. Since human attentional capacity is limited, systems should scaffold cognitive efforts in a comprehensible manner. We propose the Principles of Mnemonic Associative Knowledge (P-MAK), which describes a framework for semantically (...) identifying, organizing, and retrieving information, and for encoding episodic events by time and stimuli. Inspired by prominent human memory models, we propose associative networks as a preferred representation. Networks are ideal for their parsimony, flexibility, and ease of inspection. Networks also possess topological properties—such as clusters, hubs, and the small world—that aid analysis and navigation in an information space. Our cognitive perspective addresses fundamental problems faced by information management systems, in particular the retrieval of related items and the representation of context. We present evidence from neuroscience and memory research in support of this approach, and discuss the implications of systems design within the constraints of P-MAK’s principles, using text documents as an illustrative semantic domain. (shrink)

There is remarkable convergence between twentieth‐century interpretations of the image of God (imago Dei), what it means for human beings to be created in God's image, and approaches toward creating in our own image in the field of artificial intelligence (AI). Both fields have viewed the intersection between God and humanity or humanity and computers in terms of either (1) a property or set of properties such as intelligence, (2) the functions we engage in or are capable of, or (3) (...) the relationships we establish and maintain. Each of these three approaches reflects a different understanding of what stands at the core of our humanity. Functional and relational approaches were common in the late twentieth century, with a functional understanding the one most accepted by society at large. A relational view, however, gives new insights into human dignity in a computer age as well as new approaches to AI research. (shrink)

At the center of contemporary discourse on technology — or the digital discourse — is the assertion that network technology ushers in a new phase of capitalism which is more democratic, participatory, and de-alienating for individuals. Rather than viewing this discourse as a transparent description of the new realities of techno-capitalism and judging its claims as true (as the hegemonic view sees it) or false (a view expressed by few critical voices), this article offers a new framework which sees the (...) digital discourse as signaling a historical shift in the technological legitimation of capitalism, concurrent with the emergence of the post-Fordist phase of capitalism. Technology discourse legitimated the Fordist phase of capitalism by stressing the ability of technology and technique to mitigate exploitation. It hence legitimated the interventionist welfare state, the central planning in businesses and the economy, the hierarchized corporation, and the tenured worker. In contrast, contemporary technology discourse legitimates the post-Fordist phase of capitalism by stressing the ability of technology to mitigate alienation. It hence legitimates the withdrawal of the state from markets, the dehierarchization and decentralization of businesses, and the flexibilization of production and the labor process. (shrink)

What is humanity? I do not claim to answer this question; more simply, I will seek to bring to light the now-problematic character of the very concept of humanity. I will start from a basic established fact: today we cannot conceive the notion of humanity without starting from a dual tension, between a distant past and a future which is either more or less near or very remote. In whatever direction one turns, the concept of humanity is confused. It is (...) still more confused if we seek to embrace past and future together. (shrink)

The question of liability in the case of using intelligent agents is far from simple, and cannot sufficiently be answered by deeming the human user as being automatically responsible for all actions and mistakes of his agent. Therefore, this paper is specifically concerned with the significant difficulties which might arise in this regard especially if the technology behind software agents evolves, or is commonly used on a larger scale. Furthermore, this paper contemplates whether or not it is possible to share (...) the responsibility with these agents and what are the main objections surrounding the assumption of considering such agents as responsible entities. This paper, however, is not intended to provide the final answer to all questions and challenges in this regard, but to identify the main components, and provide some perspectives on how to deal with such issue. (shrink)

One is occasionally reminded of Foucault's proclamation in a 1970 interview that "perhaps, one day this century will be known as Deleuzian." Less often is one compelled to update and restart with a supplementary counter-proclamation of the mathematician, David Lindley: "the twenty-first century would be a Bayesian era..." The verb tenses of both are conspicuous. // To critically attend to what is today often feared and demonized, but also revered, deployed, and commonly referred to as algorithm(s), one cannot avoid the (...) mathematical and philosophical legacies of probability. // But attending to these probabilistic or Bayesian legacies must include an undeniable theological legacy in which they remain entangled. // We are not, today, discovering quirky theological metaphors in contemporary technics. It's the other way around. The technologies are mere metaphors of past theologies. (shrink)

The Turing Test is claimed by many to be a way to test for the presence, in computers, of such ``deep'' phenomena as thought and consciousness. Unfortunately, attempts to build computational systems able to pass TT have devolved into shallow symbol manipulation designed to, by hook or by crook, trick. The human creators of such systems know all too well that they have merely tried to fool those people who interact with their systems into believing that these systems really have (...) minds. And the problem is fundamental: the structure of the TT is such as to cultivate tricksters. A better test is one that insists on a certain restrictive epistemic relation between an artificial agent A, its output o, and the human architect H of A – a relation which, roughly speaking, obtains when H cannot account for how A produced o. We call this test the ``Lovelace Test'' in honor of Lady Lovelace, who believed that only when computers originate things should they be believed to have minds. (shrink)

How do engineering educators adequately and richly introduce to young engineers the perplexing ethical issues associated with the development of new technologies? Robotics, nanotechnology, cloning, cyberintelligence, and genetic engineering, for example, each hold the potential to radically alter the fundamental nature of human life. Senior citizens in our society have a lifetime of experience adopting new technologies into their lives. Through an intergenerational dialogue, undergraduate engineers can come to appreciate and understand what technological change can really mean, both in practical (...) and ethical terms. This article explores the use of intergenerational dialogue as a learning tool, with a focus on the experience of students enrolled in a required engineering ethics course. Reactions and thoughts recounted in this article from both undergraduate engineers and senior citizens signify the immeasurable educational value of intergenerational exploration and its effectiveness in examining the ethical questions connected to the development of new technologies. (shrink)

In this paper, the author aims to show that transhumanists are confused about their own conception of the posthuman: transhumanists anticipate radical transformation of the human through technology and at the same time assume that the criteria to determine what is “normal” and what is “enhanced” are univocal, both in our present time and in the future. Inspired by Nietzsche’s notion of the Overhuman, the author argues that the slightest “historical and phenomenological sense” discloses copious variations of criteria, both diachronic (...) and synchronic, for what can be considered “normal” and “enhanced.” Radical transformation through technology does not simply enable us to become “stronger,” “smarter,” or “healthier,” but it can and often will also change the very standard or yardstick with which we measure what counts as “stronger,” “smarter,” or “healthier.” Put yet differently: new and emerging technologies are not neutral means but often bring about different and, from our current perspective, foreign standards for determining what are “normal” and “enhanced” capacities. Since the qualitative meanings of these terms are themselves not fixed, it is unintelligible and too reassuring to simply predict that new technologies will enhance human beings. (shrink)

Michel Foucault's early works criticize the development of modern democratic institutions as creating a surveillance society, which functions to control bodies by making them feel watched and monitored full time. His later works attempt to recover private space by exploring subversive techniques of the body and language. Following Foucault, pragmatists like Richard Shusterman and Richard Rorty have also developed very rich approaches to this project, extending it deeper into the literary and somatic dimensions of self-stylizing. Yet, for a debate centered (...) on the re-creation of the vision of the self, these discussions have yet to fully engage—even while they set the conditions for this engagement with—issues of posthuman technologies, such as AI, robotics, and genetic engineering. These certainly will constitute the primary engine of public institutional surveillance in the years to come. With this surveillance, the two spheres of private self-fashioning and public institutions will continue to evolve in relation to one another. As they do so, democratic society will change considerably. Perhaps Rorty's own normative and linguistic ideal of conversational maintenance may provide something of a democratic limit case on any future posthuman self-fashioning. (shrink)

Because of accelerating technological progress, humankind may be rapidly approaching a critical phase in its career. In addition to well-known threats such as nuclear holocaust, the propects of radically transforming technologies like nanotech systems and machine intelligence present us with unprecedented opportunities and risks. Our future, and whether we will have a future at all, may well be determined by how we deal with these challenges. In the case of radically transforming technologies, a better understanding of the transition dynamics from (...) a human to a "posthuman" society is needed. Of particular importance is to know where the pitfalls are: the ways in which things could go terminally wrong. While we have had long exposure to various personal, local, and endurable global hazards, this paper analyzes a recently emerging category: that of existential risks. These are threats that could case our extinction or destroy the potential of Earth - originating intelligent life. Some of these threats are relatively well known while others, including some of the gravest, have gone almost unrecognized. Existential risks have a cluster of features that make ordinary risk management ineffective. A final section of this paper discusses several ethical and policy implications. A clearer understanding of the threat picture will enable us to formulate better strategies. (shrink)

Various kinds of human bioenhancement represent a major topic of contention in both bioethics and futures studies. Moral enhancement is one of them. It will be argued that voluntary moral bio-enhancement should be based on an opt out moral enhancement scheme. Such a scheme would avoid the challenges of a voluntary moral enhancement opt in scheme. The former has a proper place in a minimal state. It will be explained why such a state can be called Utopia. The concept of (...) voluntary opt out moral enhancement in Utopia will be highlighted in detail. (shrink)

It has already become a commonplace to discuss postbiological evolution in various contexts of futures studies, bioethics, cognitive sciences, philosophical anthropology, or even economics and SETI studies. The assumption is that technological/cultural evolution will soon entirely substitute for the biological processes which underlie human existence – and, by analogy, the existence of other independently evolved intelligent beings, if any. Various modes of postbiological evolution of humans have been envisioned in both fictional and discursive contexts. Little thought has been devoted so (...) far to the question whether these postbiological modes are truly final in both logical and conceptual terms. What lies beyond the postbiological realm? Clearly, only a few radical speculations on the topic can be offered. In this paper, it is argued that in a sufficiently broad subset of scenarios for postbiological evolution there will be a kind of reverse trend: the one of reintegration with the biological universe, by that point understood in a much wider and more inclusive sense. The argument for such reintegration could be understood through the metaphor of expanding Klein bottle as a symbolic image of the post-postbiological evolutionary trajectories. This kind of trajectory will lead to the state in which products of culture are indistinguishable from the natural environment, suggesting practical consequences for our SETI efforts. (shrink)

Because of accelerating technological progress, humankind may be rapidly approaching a critical phase in its career. In addition to well-known threats such as nuclear holocaust, the prospects of radically transforming technologies like nanotech systems and machine intelligence present us with unprecedented opportunities and risks. Our future, and whether we will have a future at all, may well be determined by how we deal with these challenges. In the case of radically transforming technologies, a better understanding of the transition dynamics from (...) a human to a “posthuman” society is needed. Of particular importance is to know where the pitfalls are: the ways in which things could go terminally wrong. While we have had long exposure to various personal, local, and endurable global hazards, this paper analyzes a recently emerging category: that of existential risks. These are threats that could cause our extinction or destroy the potential of Earth-originating intelligent life. Some of these threats are relatively well known while others, including some of the gravest, have gone almost unrecognized. Existential risks have a cluster of features that make ordinary risk management ineffective. A final section of this paper discusses several ethical and policy implications. A clearer understanding of the threat picture will enable us to formulate better strategies. (shrink)

Evolutionary development is sometimes thought of as exhibiting an inexorable trend towards higher, more complex, and normatively worthwhile forms of life. This paper explores some dystopian scenarios where freewheeling evolutionary developments, while continuing to produce complex and intelligent forms of organization, lead to the gradual elimination of all forms of being that we care about. We then consider how such catastrophic outcomes could be avoided and argue that under certain conditions the only possible remedy would be a globally coordinated policy (...) to control human evolution by modifying the fitness function of future intelligent life forms. (shrink)

A vision of future daily life is explored in Ambient Intelligence . It contains the assumption that intelligent technology should disappear into our environment to bring humans an easy and entertaining life. The mental, physical, methodical invisibility of AmI will have an effect on the relation between design and use activities of both users and designers. Especially the ethics discussions of AmI, privacy, identity and security are moved into the foreground. However in the process of using AmI, it will go (...) beyond these themes. The infiltration of AmI will cause the construction of new meanings of privacy, identity and security because the "visible" acting of people will be preceded, accompanied and followed by the invisible and visible acting of the AmI technology and their producers.A question in this paper is: How is it possible to create critical transformative rooms in which doubting will be possible under the circumstances that autonomous 'intelligent agents' surround humans? Are humans in danger to become just objects of artificial intelligent conversations? Probably the relation between mental, physical, methodical invisibility and visibility of AmI could give answers. (shrink)

What happens when machines become more intelligent than humans? One view is that this event will be followed by an explosion to ever-greater levels of intelligence, as each generation of machines creates more intelligent machines in turn. This intelligence explosion is now often known as the “singularity”. The basic argument here was set out by the statistician I.J. Good in his 1965 article “Speculations Concerning the First Ultraintelligent Machine”: Let an ultraintelligent machine be defined as a machine that can far (...) surpass all the intellectual activities of any man however clever. Since the design of machines is one of these intellectual activities, an ultraintelligent machine could design even better machines; there would then unquestionably be an “intelligence explosion”, and the intelligence of man would be left far behind. Thus the first ultraintelligent machine is the last invention that man need ever make. The key idea is that a machine that is more intelligent than humans will be better than humans at designing machines. So it will be capable of designing a machine more intelligent than the most intelligent machine that humans can design. So if it is itself designed by humans, it will be capable of designing a machine more intelligent than itself. By similar reasoning, this next machine will also be capable of designing a machine more intelligent than itself. If every machine in turn does what it is capable of, we should expect a sequence of ever more intelligent machines. This intelligence explosion is sometimes combined with another idea, which we might call the “speed explosion”. The argument for a speed explosion starts from the familiar observation that computer processing speed doubles at regular intervals. Suppose that speed doubles every two years and will do so indefinitely. Now suppose that we have human-level artificial intelligence 1 designing new processors. Then faster processing will lead to faster designers and an ever-faster design cycle, leading to a limit point soon afterwards. The argument for a speed explosion was set out by the artificial intelligence researcher Ray Solomonoff in his 1985 article “The Time Scale of Artificial Intelligence”.1 Eliezer Yudkowsky gives a succinct version of the argument in his 1996 article “Staring at the Singularity”: “Computing speed doubles every two subjective years of work.. (shrink)

Die Übernahme des KI-Unternehmens DeepMind durch Google für rund eine halbe Milliarde US-Dollar signalisierte vor einem Jahr, dass von der KI-Forschung vielversprechende Ergebnisse erwartet werden. Spätestens seit bekannte Wissenschaftler wie Stephen Hawking und Unternehmer wie Elon Musk oder Bill Gates davor warnen, dass künstliche Intelligenz eine Bedrohung für die Menschheit darstellt, schlägt das KI-Thema hohe Wellen. Die Stiftung für Effektiven Altruismus (EAS, vormals GBS Schweiz) hat mit der Unterstützung von Experten/innen aus Informatik und KI ein umfassendes Diskussionspapier zu den Chancen (...) und Risiken der künstlichen Intelligenz verfasst und geht darin auf aktuelle, mittel- und langfristige Herausforderungen im Bereich der KI-Entwicklung ein. (shrink)

Teilhard is among the first to seriously explore the future of human evolution. He advocates both bio-technologies (e.g. genetic engineering) and intelligence technologies. He discusses the emergence of a global computation - communication system (and is said by some to have been the first to have envisioned the Internet). He advocates the development of a global society. He is almost surely the first to discuss the acceleration of technological progress to a Singularity in which human intelligence will become super-intelligence. He (...) discusses the spread of human intelligence into the universe and its amplification into a cosmic-intelligence. His work has been taken up by Barrow and Tipler; Tipler; Moravec; and Kurzweil. Of course, Teilhard's Omega Point Theory is deeply Christian. For secular transhumanists, this may be difficult. But transhumanism cannot avoid a fateful engagement with Christianity. Christian institutions may support or oppose transhumanism. Since Christianity is an extremely powerful cultural force in the West, it is imperative for transhumanism to engage it carefully. A serious study of Teilhard can help that engagement and will thus be rewarding to both communities. (shrink)

In this essay I will explore an understanding of the potential moral agency of robots; arguing that the key characteristics of physical embodiment; adaptive learning; empathy in action; and a teleology toward the good are the primary necessary components for a machine to become a moral agent. In this context; other possible options will be rejected as necessary for moral agency; including simplistic notions of intelligence; computational power; and rule-following; complete freedom; a sense of God; and an immaterial soul. I (...) argue that it is likely that such moral machines may be able to be built; and that this does not diminish humanity or human personhood. (shrink)