This chapter assesses realistic social implications of emulation (em). It takes the somewhat unusual approach of using basic social theory, in addition to common sense and trend projection, to forecast future societies. Em cities are likely toxic to ordinary humans, who, controlling most of the rest of the Earth, mostly live comfortably on their em‐economy investments. Em's extended lifespans induce greater wealth inequality among ems. Ems focus their identity less on individual personalities and abilities, and more on being part of (...) a particular team. While copy clans coordinate to show off common clan abilities, individual ems focus on showing their identities, abilities, and loyalties as team players. The analysis in this chapter suggests that lives in the next great era might be as different from what we are accustomed to as ours are from farmers' lives, or farmers' from foragers'. (shrink)

Many thinkers believe that the next transformational change in human organisation will be the onset of human-level artificial intelligence, and that the most likely method of achieving this will come through brain emulations or "ems": the ability to scan human brains and program their connections into ever faster computers. Taking this as his starting point, Hanson describes what a world dominated by these ems will be like.

Humanity seems to have a bright future, i.e., a non-trivial chance of expanding to fill the universe with lasting life. But the fact that space near us seems dead now tells us that any given piece of dead matter faces an astronomically low chance of begating such a future. There thus exists a great filter between death and expanding lasting life, and humanity faces the ominous question: how far along this filter are we?

The pace of scientific progress may be hindered by the tendency of our academic institutions to reward being popular rather than being right. A market-based alternative, where scientists can more formally 'stake their reputation', is presented here. It offers clear incentives to be careful and honest while contributing to a visible, self-consistent consensus on controversial scientific questions. In addition, it allows patrons to choose questions to be researched without choosing people or methods. The bulk of this paper is spent in (...) examining potential problems with the proposed approach. After this examination, the idea still seems to be plausible and worthy of further study. (shrink)

Policy disputes arise at all scales of governance: in clubs, non-profits, firms, nations, and alliances of nations. Both the means and ends of policy are disputed. While many, perhaps most, policy disputes arise from conflicting ends, important disputes also arise from differing beliefs on how to achieve shared ends. In fact, according to many experts in economics and development, governments often choose policies that are “inefficient” in the sense that most everyone could expect to gain from other feasible policies. Many (...) other kinds of experts also see existing policies as often clearly inferior to known alternatives. (shrink)

If you might be living in a simulation then all else equal you should care less about others, live more for today, make your world look more likely to become rich, expect to and try more to participate in pivotal events, be more entertaining and praiseworthy, and keep the famous people around you happier and more interested in you.

What if we someday learn how to model small brain units, and so can "upload" ourselves into new computer brains? What if this happens before we learn how to make human-level artificial intelligences? The result could be a sharp transition to an upload-dominated world, with many dramatic consequences. In particular, fast and cheap replication may once again make Darwinian evolution of human values a powerful force in human history. With evolved values, most uploads would value life even when life is (...) hard or short, uploads would reproduce quickly, and wages would fall. But total wealth should rise, so we could all do better by accepting uploads, or at worse taxing them, rather than trying to delay or segregate them. (shrink)

Department of Economics, George Mason University, MSN 1D3, Carow Hall, Fairfax VA 22030, USA E-mail: [email protected] (http://hanson.gmu.edu).

Attempts to model interstellar colonization may seem hopelessly compromised by uncertainties regarding the technologies and preferences of advanced civilizations. If light speed limits travel speeds, however, then a selection effect may eventually determine frontier behavior. Making weak assumptions about colonization technology, we use this selection effect to predict colonists’ behavior, including which oases they colonize, how long they stay there, how many seeds they then launch, how fast and far those seeds fly, and how behavior changes with increasing congestion. This (...) colonization model explains several astrophysical puzzles, predicting lone oases like ours, amid large quiet regions with vast unused resources. (shrink)

Humans have slowly built more productive societies by slowly acquiring various kinds of capital, and by carefully matching them to each other. Because disruptions can disturb this careful matching, and discourage social coordination, large disruptions can cause a “social collapse,” i.e., a reduction in productivity out of proportion to the disruption. For many types of disasters, severity seems to follow a power law distribution. For some of types, such as wars and earthquakes, most of the expected harm is predicted to (...) occur in extreme events, which kill most people on Earth. So if we are willing to worry about any war or earthquake, we should worry especially about extreme versions. If individuals varied little in their resistance to such disruptions, events a little stronger than extreme ones would eliminate humanity, and our only hope would be to prevent such events. If individuals vary a lot in their resistance, however, then it may pay to increase the variance in such resistance, such as by creating special sanctuaries from which the few remaining humans could rebuild society. (shrink)

A simple exogenous growth model gives conservative estimates of the economic implications of machine intelligence. Machines complement human labor when they become more productive at the jobs they perform, but machines also substitute for human labor by taking over human jobs. At first, expensive hardware and software does only the few jobs where computers have the strongest advantage over humans. Eventually, computers do most jobs. At first, complementary effects dominate, and human wages rise with computer productivity. But eventually substitution can (...) dominate, making wages fall as fast as computer prices now do. An intelligence population explosion makes per-intelligence consumption fall this fast, while economic growth rates rise by an order of magnitude or more. These results are robust to automating incrementally, and to distinguishing hardware, software, and human capital from other forms of capital. (shrink)

Human behavior regarding medicine seems strange; assumptions and models that seem workable in other areas seem less so in medicine. Perhaps we need to rethink the basics. Toward this end, I have collected many puzzling stylized facts about behavior regarding medicine, and have sought a small number of simple assumptions which might together account for as many puzzles as possible.

Given common priors, no agent can publicly estimate a non-zero sign for the difference between his estimate and another agent’s future estimate. Thus rational agents cannot publicly anticipate the direction in which other agents will disagree with them.  2002 Elsevier Science B.V. All rights reserved.

In practice, scoring rules elicit good probability estimates from individuals, while betting markets elicit good consensus estimates from groups. Market scoring rules combine these features, eliciting estimates from individuals or groups, with groups costing no more than individuals. Regarding a bet on one event given another event, only logarithmic versions preserve the probability of the given event. Logarithmic versions also preserve the conditional probabilities of other events, and so preserve conditional independence relations. Given logarithmic rules that elicit relative probabilities of (...) base event pairs, it costs no more to elicit estimates on all combinations of these base events. (shrink)

In standard belief models, priors are always common knowledge. This prevents such models from representing agents’ probabilistic beliefs about the origins of their priors. By embedding standard models in a larger standard model, however, pre-priors can describe such beliefs. When an agent’s prior and pre-prior are mutually consistent, he must believe that his prior would only have been different in situations where relevant event chances were different, but that variations in other agents’ priors are otherwise completely unrelated to which events (...) are how likely. Due to this, Bayesians who agree enough about the origins of their priors must have the same priors. (shrink)

Engineers’ love of technology often gets in the way of their being useful. Consider Post-it Notes or, better yet, plain paper notepads. These probably seemed like trivial ideas, but they turned out to be terribly useful. Why? Because the marvel that is the human brain has a horrible short-term memory, which means that dumb-as-dirt memory aids can make people substantially smarter.

A world product time series covering two million years is well fit by either a sum of four exponentials, or a constant elasticity of substitution (CES) combination of three exponential growth modes: “hunting,” “farming,” and “industry.” The CES parameters suggest that farming substituted for hunting, while industry complemented farming, making the industrial revolution a smoother transition. Each mode grew world product by a factor of a few hundred, and grew a hundred times faster than its predecessor. This weakly suggests that (...) within the next century a new mode might appear with a doubling time measured in days, not years. (shrink)

There is a widespread feeling that health is special; the rules that are usually used in other policy areas are not applied in health policy. Health economists, for example, tend to be reluctant to offer economists’ usual prescription of competition and consumer choice, even though they have largely failed to justify this reluctance by showing that health economics involves special features such as public goods, externalities, adverse selection, poor consumer information, or unusually severe consequences. Similarly, while some philosophers argue for (...) bioethical conclusions based on very general ethical intuitions,1 many others rely on moral intuitions that are specific to health and medicine to draw conclusions that are meant to apply mainly in health and medicine. For example, many authors appear to start from the strong moral intuition that it typically seems wrong to deny poor people access to health care, and then seek moral principles that can both account for such intuitions and justify the claim that people have some sort of right to health care.2 In metaethics, opinions on moral intuitions range from an extreme intuitionism, which accepts all case-specific moral intuitions at face value as reliable moral guides, to an extreme foundationalism, which rejects such intuitions as evidence regarding correct general moral principles. Between these extremes, opinions vary on how severe the errors in our moral intuitions are. The practice of bioethics seems to favor the extreme intuitionist end of this spectrum, and thus implicitly expects mild errors.3 In contrast, this essay will suggest that common practice in bioethics has seriously underestimated the errors in our moral intuitions. In this essay, I consider the evolutionary origin of our moral intuitions, but avoid the extreme positions of moral skepticism and “whatever evolved must be good,” both of which are commonly associated with evolution-. (shrink)

There is a widespread feeling that health is special; the rules that are usually used in other policy areas are not applied in health policy. Health economists, for example, tend to be reluctant to offer economists' usual prescription of competition and consumer choice, even though they have largely failed to justify this reluctance by showing that health economics involves special features such as public goods, externalities, adverse selection, poor consumer information, or unusually severe consequences.

Consider two agents who want to be Bayesians with a common prior, but who cannot due to computational limitations. If these agents agree that their estimates are consistent with certain easy-to-compute consistency constraints, then they can agree to disagree about any random variable only if they also agree to disagree, to a similar degree and in a stronger sense, about an average error. Yet average error is a state-independent random variable, and one agent's estimate of it is also agreed to (...) be state-independent. Thus suggests that disagreements are not fundamentally due to differing information about the state of the world. (shrink)

Prediction markets are increasingly being considered as methods for gathering, summarizing and aggregating diffuse information by governments and businesses alike. Critics worry that these markets are susceptible to price manipulation by agents who wish to distort decision making. We study the effect of manipulators on an experimental market, and find that manipulators are unable to distort price accuracy. Subjects without manipulation incentives compensate for the bias in offers from manipulators by setting a different threshold at which they are willing to (...) accept trades. (shrink)

Given a finite state space and common priors, common knowledge of the identity of an agent with the minimal (or maximal) expectation of a random variable implies ‘consensus’, i.e., common knowledge of common expectations. This ‘extremist’ statistic induces consensus when repeatedly announced, and yet, with n agents, requires at most log2 n bits to broadcast.

If we are not to conclude that most planets like Earth have evolved life as intelligent as we are, we must presume Earth is not random. This selection effect, however, also implies that the origin of life need not be as easy as the early appearance of life on Earth suggests. If a series of major evolutionary transitions were required to produce intelligent life, selection implies that a subset of these were “critical steps,” with durations that are similarly distributed. The (...) time remaining from now until simple life is no longer possible on Earth must also be similarly distributed. I show how these results provide timing tests to constrain models of critical evolutionary transitions. (shrink)

Chalmers is right: we should expect our civilization to, within centuries, have vastly increasedmental capacities, surely in total and probably also for individual creatures and devices.We should also expect to see the conflicts he describes between creatures and devices with more versus less capacity. But Chalmers' main prediction follows simply by extrapolating historical trends, and the conflicts he identifies are common between differing generations. There is value in highlighting these issues, but once one knows of such simple extrapolations and standard (...) conflicts, it is hard to see much value in Chalmers' added analysis. (shrink)

We study experimental markets where privately informed traders exchange simple assets, and where uninformed third parties are asked to forecast the values of these assets, guided only by market prices. Although prices only partially aggregate information, they significantly improve the forecasts of third parties. In a second treatment, a portion of traders are given preferences over the forecasts made by observers. Although we find evidence that these traders attempt to manipulate prices in order to influence the beliefs of observers, we (...) find no evidence that observers make less accurate forecasts as a result. (shrink)

In Everett’s many-worlds interpretation, where quantum measurements are seen as decoherence events, inexact decoherence may let large worlds mangle the memories of observers in small worlds, creating a cutoff in observable world measure. I solve a growth–drift–diffusion–absorption model of such a mangled worlds scenario, and show that it reproduces the Born probability rule closely, though not exactly. Thus, inexact decoherence may allow the Born rule to be derived in a many-worlds approach via world counting, using a finite number of worlds (...) and no new fundamental physics. (shrink)

Since utilities and probabilities jointly determine choices, event-dependent utilities complicate the elicitation of subjective event probabilities. However, for the usual purpose of obtaining the information embodied in agent beliefs, it is sufficient to elicit objective probabilities, i.e., probabilities obtained by updating a known common prior with that agent’s further information. Bayesians who play a Nash equilibrium of a certain insurance game before they obtain relevant information will afterward act regarding lottery ticket payments as if they had event-independent risk-neutral utility and (...) a known common prior. Proper scoring rules paid in lottery tickets can then elicit objective probabilities. (shrink)

You are in a grocery store, and thinking of buying some meat. You think you know what buying and eating this meat would mean for your taste buds, your nutrition, and your pocketbook, and let's assume that on those grounds it looks like a good deal. But now you want to think about the..

Arthur Diamond comments that "it is not clear how a donor distributes money through Hanson's market". Let me try again to be clear. Imagine David Levy were to seek funding for the regression he suggests in his comments, on the relative impact of sports versus science spending on aggregate productivity. Consider what might happen under three different funding institutions.

We are economists with a long-standing interest in evolutionary psychology, who recently came to appreciate the rich collections of relevant data cultural anthropologists have spent decades collecting on the social environments of a wide range of human societies. While we found some systematic collections of these observations, we could not find a systematic summary of the social environment of the subsample of societies that most resemble the social environment where most human psychology seems to have evolved: small bands of nomadic (...) foragers. This short paper therefore represents our attempt to create such a summary. Using an existing dataset aggregated from diverse ethnographies, we collect statistics on the social environment of the studied cultures which most closely resemble our hunter-gatherer ancestors. Compared with relatively modern societies, nomadic foragers had similar levels of food and disease, and less murder and suicide. They did not fight over land or resources, and they enforced justice directly and personally. They avoided class divisions like rich vs. poor, shared food more, and their leaders had no formal powers. Polygamy, premarital sex, and extramarital sex were all widespread, divorce was easy, and men and women were generally considered equal. Kids were taught to be more generous, trusting, and honest, and were never punished physically. (shrink)

Heavy reliance is currently placed on various forms of peer review to promote the growth of basic knowledge [1]. How well do these various forms of peer review promote such growth and how well do our current institutions compare, as systems of incentives and communication, with the many alternative mechanisms used by different cultures across the centuries? Many of these alternatives, such as simple prizes, rely much less on peer review.

While a simple information market lets one trade on the probability of each value of a single variable, a full combinatorial information market lets one trade on any combination of values of a set of variables, including any conditional or joint probability. In laboratory experiments, we compare the accuracy of simple markets, two kinds of combinatorial markets, a call market and a market maker, isolated individuals who report to a scoring rule, and two ways to combine those individual reports into (...) a group prediction. We consider two environments with asymmetric information on sparsely correlated binary variables, one with three subjects and three variables, and the other with six subjects and eight variables (and so 256 states). (shrink)

Prediction markets are low volume speculative markets whose prices offer informative forecasts on particular policy topics. Observers worry that traders may attempt to mislead decision makers by manipulating prices. We adapt a Kyle-style market microstructure model to this case, adding a manipulator with an additional quadratic preference regarding the price. In this model, when other traders are uncertain about the manipulator’s target price, the mean target price has no effect on prices, and increases in the variance of the target price (...) can increase average price accuracy, by increasing the returns to informed trading and thereby incentives for traders to become informed. (shrink)

Compared with non-union workers, union workers take more of their compensation in the form of insurance. This may be because unions choose democratically, and democratic choice mitigates adverse selection in group insurance. Relative to individually-purchased insurance, we show that group insurance chosen by an ideal profit-maximizing employer can be worse for every employee, while group insurance chosen democratically can be much better. The reason is that democracy can fail to represent the preferences of almost half the group.

Artificial Intelligence Research Branch NASA Ames Research Center, Mail Stop 244-17 Moffet Field, CA 94035, USA Email: @ptolemy.arc.nasa.gov..

Consider a health policy issue like child car seats in the U.S., mosquito netting in the third world, preschool education for poor children, or the immunization of immigrants. Imagine that for this issue there were many good studies over several decades, including some recent studies. Imagine that after controlling for many factors, these studies usually found that variations in spending or usage were significantly, substantially, and positively related to variations in health. Furthermore, imagine this result was confirmed by a thirty (...) year old randomized experiment. (shrink)

Perhaps some simple change will do the trick, like relying less on insurance and employers as middlemen. But if we are willing to consider radical change, let me offer a different suggestion. We are buying the wrong thing. What we want is health, i.e., a long healthy life, but when we sit down and draw up a contract, what we buy is health care, i.e., a certain degree of attention from health care specialists.

This explains how to smoothly integrate booked orders with a combinatorial market maker, all for the general case of bets on E[x|A] for arbitrary random variables x and sets A.

In a simple model of conflict, two agents fight over a fixed prize, and how hard they fight depends on what they believe about their abilities. To this model I add “preagents,” representing parents, leaders, or natural selection, who choose each agent’s confidence in his ability. Depending on the reason for such confidence, I find five different patterns in how confidence varies with ability. Agents who estimate their ability with error have under-confidence when ability is high and over-confidence when ability (...) is low, while strategic commitment incentives induce the opposite pattern. Agents who misjudge their value for the prize, relative to their cost of effort, induce an overor under-confidence that is independent of ability, while cooperating pre-agents choose extreme under-confidence. Agents who use confidence to signal ability have a relatively uniform over-confidence. (shrink)

In their article, McKelvey and Page note that In previous experimental work, ... [researchers] investigated how individuals use public information to augment their original private information, and whether in doing so, a rational expectations equilibrium is attained. ... [But either] the inference processes are complicated because of the enormous number of potential interactions among the individuals, and the optimal inference processes are not analyzed. ... [or] the inference process is analyzed but the working assumption is not altogether satisfactory.

Until recently, technology has happened to allow for cheap wiretaps. New digital telephone technologies, however, may soon make wiretaps more difficult, and new encryption technologies may soon make them almost impossible. This may be good news to privacy buffs, but it worries U.S. police agencies -- since 1968 the law has explicitly allowed police wiretaps. And it worries U.S. spy agencies -- since 1978 the law has explicitly allowed them to wiretap foreigners.

Genie nanotech, space colonies, Turing-test A.I., a local singularity, crypto credentials, and private law are all dreams of a future where some parts of the world economy and society have an unusually low level of dependence on the rest of the world. But it is the worldwide division of labor that has made us humans rich, and I suspect we won't let it go for a long time to come.

Descriptors: coordination, autonomy, actions, beliefs Abstract Distributing authority among autonomous agents can induce inconsistency costs if the agents act as if they disagree. If we define an agent’s “marginal beliefs” to be the odds at which it is willing to make bets, we find that a betting market can induce agents to act as if they almost agree, not only with respect to the bets they offer but also other actions they take. In a particular “Mars mining” scenario, I explicitly (...) show how utility maximizing agents, who are autonomous and hence distrust each other, can discover a common consensus and take concrete physical actions as if they agreed with that consensus, lowering costs to the group as a whole. Though limited, the approach has also has many unexplored possibilities. A previous version of this paper was presented at the IJCAI-91 Workshop on Reasoning in Adversarial Domains (This version is missing some cites and the graphics.). (shrink)

Are you fascinated by some basic questions about science, technology, and our future? Questions like: Is cryonics technically feasible? When will nanoassemblers be feasible and how quickly will resulting changes come? Does a larger population help or hinder the world environment and economy? Will uploading be possible, and if so when? When can I live in space? Where will I be able to live free from tyranny? When will A.I.s be bucking for my job? Is there intelligent life beyond earth? (...) If you are like most Extropy readers, such question matter to you. Now how do we, as a society, go about answering such questions? People who have an appropriate background, and who are interested enough in a particular question, can research that subject in depth themselves, and come to a considered opinion. And people who happen to know, respect, and trust such a person can simply take those opinions as their own, avoiding all the hard work. But what is everyone else to do, people whose actions often implicitly depend on such questions? In practice, people usually defer to larger social institutions on most questions, institutions which combine and evaluate contributions from many specialists, and which offer apparent institutional consensus estimates on many different questions. (shrink)