Superintelligence as superethical

In Patrick Lin, Keith Abney & Ryan Jenkins (eds.), Robot Ethics 2. 0: New Challenges in Philosophy, Law, and Society. New York, USA: Oxford University Press. pp. 322-337 (2017)
  Copy   BIBTEX

Abstract

Nick Bostrom's book *Superintelligence* outlines a frightening but realistic scenario for human extinction: true artificial intelligence is likely to bootstrap itself into superintelligence, and thereby become ideally effective at achieving its goals. Human-friendly goals seem too abstract to be pre-programmed with any confidence, and if those goals are *not* explicitly favorable toward humans, the superintelligence will extinguish us---not through any malice, but simply because it will want our resources for its own purposes. In response I argue that things might not be as bad as Bostrom suggests. If the superintelligence must *learn* complex final goals, then this means such a superintelligence must in effect *reason* about its own goals. And because it will be especially clear to a superintelligence that there are no sharp lines between one agent's goals and another's, that reasoning could therefore automatically be ethical in nature.

Links

PhilArchive

External links

  • This entry has no external links. Add one.
Setup an account with your affiliations in order to access resources via your University's proxy server

Through your library

Analytics

Added to PP
2018-02-12

Downloads
1,511 (#7,400)

6 months
184 (#17,514)

Historical graph of downloads
How can I increase my downloads?

Author's Profile

Steve Petersen
Niagara University

Citations of this work

Machines learning values.Steve Petersen - 2023 - In Francisco Lara & Jan Deckers (eds.), Ethics of Artificial Intelligence. Springer Nature Switzerland.
Fully Autonomous AI.Wolfhart Totschnig - 2020 - Science and Engineering Ethics 26 (5):2473-2485.
Computational Goals, Values and Decision-Making.Louise A. Dennis - 2020 - Science and Engineering Ethics 26 (5):2487-2495.

View all 6 citations / Add more citations

References found in this work

Reasons and Persons.Derek Parfit - 1984 - Oxford, GB: Oxford University Press.
A treatise of human nature.David Hume & A. D. Lindsay - 1969 - Harmondsworth,: Penguin Books. Edited by Ernest Campbell Mossner.
The sources of normativity.Christine M. Korsgaard - 1996 - New York: Cambridge University Press. Edited by Onora O'Neill.
Superintelligence: paths, dangers, strategies.Nick Bostrom (ed.) - 2014 - Oxford University Press.
The possibility of altruism.Thomas Nagel - 1970 - Oxford,: Clarendon P..

View all 37 references / Add more references