Superintelligence and the Future of Governance: On Prioritizing the Control Problem at the End of History

In Yampolskiy Roman (ed.), Artificial Intelligence Safety and Security. CRC Press (2018)
  Copy   BIBTEX


This chapter argues that dual-use emerging technologies are distributing unprecedented offensive capabilities to nonstate actors. To counteract this trend, some scholars have proposed that states become a little “less liberal” by implementing large-scale surveillance policies to monitor the actions of citizens. This is problematic, though, because the distribution of offensive capabilities is also undermining states’ capacity to enforce the rule of law. I will suggest that the only plausible escape from this conundrum, at least from our present vantage point, is the creation of a “supersingleton” run by a friendly superintelligence, founded upon a “post-singularity social contract.” In making this argument, the present chapter offers a novel reason for prioritizing the “control problem,” i.e., the problem of ensuring that a greaterthan-human-level AI will positively enhance human well-being.



    Upload a copy of this work     Papers currently archived: 92,991

External links

Setup an account with your affiliations in order to access resources via your University's proxy server

Through your library


Added to PP

208 (#99,658)

6 months
20 (#138,999)

Historical graph of downloads
How can I increase my downloads?

Citations of this work

Fully Autonomous AI.Wolfhart Totschnig - 2020 - Science and Engineering Ethics 26 (5):2473-2485.

Add more citations

References found in this work

The extended mind.Andy Clark & David J. Chalmers - 1998 - Analysis 58 (1):7-19.
The law of peoples.John Rawls - 1999 - Cambridge, Mass.: Harvard University Press. Edited by John Rawls.
Superintelligence: paths, dangers, strategies.Nick Bostrom (ed.) - 2014 - Oxford University Press.
Unfit for the Future: The Need for Moral Enhancement.Ingmar Persson & Julian Savulescu - 2012 - Oxford, GB: Oxford University Press UK. Edited by Julian Savulescu.

View all 25 references / Add more references