This chapter analyzes the control problem, the unique principal‐agent problem that arises with the creation of an artificial superintelligent agent. It distinguishes two broad classes of potential methods for addressing this problem, capability control and motivation selection, and examines several specific techniques within each class. It also alludes to the esoteric possibility of “anthropic capture”. Capability control methods seek to prevent undesirable outcomes by limiting what the superintelligence can do. This might involve boxing methods (...) or incentive methods, stunting method. In addition, capability control methods might involve tripwires method. Motivation selection can involve explicitly formulating a goal or set of rules to be followed (direct specification) or setting up the system so that it can discover an appropriate set of values for itself by reference to some implicitly or indirectly formulated criterion (indirect normativity). (shrink)

This paper presents a new trilemma with respect to resolving the control and alignment problems in machine ethics. Section 1 outlines three possible types of artificial moral agents (AMAs): (1) 'Inhuman AMAs' programmed to learn or execute moral rules or principles without understanding them in anything like the way that we do; (2) 'Better-Human AMAs' programmed to learn, execute, and understand moral rules or principles somewhat like we do, but correcting for various sources of human moral error; and (3) (...) 'Human-Like AMAs' programmed to understand and apply moral values in broadly the same way that we do, with a human-like moral psychology. Sections 2–4 then argue that each type of AMA generates unique control and alignment problems that have not been fully appreciated. Section 2 argues that Inhuman AMAs are likely to behave in inhumane ways that pose serious existential risks. Section 3 then contends that Better-Human AMAs run a serious risk of magnifying some sources of human moral error by reducing or eliminating others. Section 4 then argues that Human-Like AMAs would not only likely reproduce human moral failures, but also plausibly be highly intelligent, conscious beings with interests and wills of their own who should therefore be entitled to similar moral rights and freedoms as us. This generates what I call the New Control Problem: ensuring that humans and Human-Like AMAs exert a morally appropriate amount of control over each other. Finally, Section 5 argues that resolving the New Control Problem would, at a minimum, plausibly require ensuring what Hume and Rawls term ‘circumstances of justice’ between humans and Human-Like AMAs. But, I argue, there are grounds for thinking this will be profoundly difficult to achieve. I thus conclude on a skeptical note. Different approaches to developing ‘safe, ethical AI’ generate subtly different control and alignment problems that we do not currently know how to adequately resolve, and which may or may not be ultimately surmountable. (shrink)

We study a joint pricing and inventory control problem for perishables with positive lead time in a finite horizon periodic-review system. Unlike most studies considering a continuous density function of demand, in our paper the customer demand depends on the price of current period and arrives according to a homogeneous Poisson process. We consider both backlogging and lost-sales cases, and our goal is to find a simultaneously ordering and pricing policy to maximize the expected discounted profit over the (...) planning horizon. When there is no fixed ordering cost involved, we design a deep reinforcement learning algorithm to obtain a near-optimal ordering policy and show that there are some monotonicity properties in the learned policy. We also show that our deep reinforcement learning algorithm achieves a better performance than tabular-based Q-learning algorithms. When a fixed ordering cost is involved, we show that our deep reinforcement learning algorithm is effective and efficient, under which the problem of “curse of dimension” is circumvented. (shrink)

This chapter argues that dual-use emerging technologies are distributing unprecedented offensive capabilities to nonstate actors. To counteract this trend, some scholars have proposed that states become a little “less liberal” by implementing large-scale surveillance policies to monitor the actions of citizens. This is problematic, though, because the distribution of offensive capabilities is also undermining states’ capacity to enforce the rule of law. I will suggest that the only plausible escape from this conundrum, at least from our present vantage point, is (...) the creation of a “supersingleton” run by a friendly superintelligence, founded upon a “post-singularity social contract.” In making this argument, the present chapter offers a novel reason for prioritizing the “control problem,” i.e., the problem of ensuring that a greaterthan-human-level AI will positively enhance human well-being. (shrink)

Spence develops and applies a normative model based on rationalist and virtue ethics as well as stoic philosophy to assess the impact of technology on wellbeing. Through developing this model, Spence offers a novel and important examination of the benefit of technology to our society as a whole.

The danger of human operators devolving responsibility to machines and failing to detect cases where they fail has been recognised for many years by industrial psychologists and engineers studying the human operators of complex machines. We call it “the control problem”, understood as the tendency of the human within a human–machine control loop to become complacent, over-reliant or unduly diffident when faced with the outputs of a reliable autonomous system. While the control problem has been (...) investigated for some time, up to this point its manifestation in machine learning contexts has not received serious attention. This paper aims to fill that gap. We argue that, except in certain special circumstances, algorithmic decision tools should not be used in high-stakes or safety-critical decisions unless the systems concerned are significantly “better than human” in the relevant domain or subdomain of decision-making. More concretely, we recommend three strategies to address the control problem, the most promising of which involves a complementary coupling between highly proficient algorithmic tools and human agents working alongside one another. We also identify six key principles which all such human–machine systems should reflect in their design. These can serve as a framework both for assessing the viability of any such human–machine system as well as guiding the design and implementation of such systems generally. (shrink)

A fundamental task for any prospective cognitive architecture is information control: routing information to the relevant mechanisms to support a variety of tasks. Jerry Fodor has argued that the Massive Modularity Hypothesis cannot account for flexible information control due to its architectural commitments and its reliance on heuristic information processing. I argue instead that the real trouble lies in its commitment to nativism—recent massive modularity models, despite incorporating mechanisms for learning and self-organization, still cannot learn to control (...) information flexibly enough. After defending this position, I close the paper by discussing wider implications for control and cognitive architecture. (shrink)

The aim of this study is to introduce a novel method to solve a class of two-dimensional fractional optimal control problems. Since there are some difficulties solving these problems using analytical methods, thus finding numerical methods to approximate their solution is a challenging topic. In this study, we use transcendental Bernstein series. In fact, for solving the problem, we generalize the Bernstein polynomials to a larger class of functions which can provide more accurate approximate solutions. The convergence theorem (...) is proved. Some examples are solved to demonstrate the validity and applicability of this technique. Comparing the results with other methods, we can find the efficiency and applicability of the scheme. (shrink)

This article investigates the ability of mental incapacity tests to account for problems of control, through a study of the approach to alcohol dependence and a comparison with the approach to anorexia nervosa, in England and Wales. The focus is on two areas of law where questions of legal and mental capacity arise for people who are alcohol dependent: decisions about treatment for alcohol dependence and diminished responsibility for a killing. The mental incapacity tests used in these legal contexts (...) are importantly different—one involves a ‘cognitive’ test, while the other includes an explicit impaired-control limb—and the comparison provides insight into a longstanding debate about the virtues of one type of test over the other. It is shown that both kinds of test can take control problems into account, but also that both can be interpreted in narrow and wide ways that significantly influence the outcome of the assessment. It is therefore argued that to a large extent, it is not the kind of mental incapacity test that matters, but how the test is interpreted. It is further proposed that value judgements are playing an unrecognised and inappropriate role in shaping this interpretation. This raises concerns about the current approach to assessing the impact of alcohol dependency on the capacity to make decisions about alcohol use or treatment, as well as broader concerns about flexibility within incapacity tests. (shrink)

Self-control has been described as the ability to master motivation that is contrary to one’s better judgement; that is, an ability that prevents such motivation from resulting in behaviour that is contrary to one’s overall better judgement (Mele, Irrationality: An essay on Akrasia, self-deception and self-control, p. 54, 1987). Recent discussions in philosophy have centred on the question of whether synchronic self-control, in which one exercises self-control whilst one is currently experiencing opposing motivation, is actional or (...) non-actional. The actional theorist maintains that such exercises are actions, whilst the non-actional theorist claims that synchronic self-control consists in the having of unmotivated thoughts which occur at times when agents experience recalcitrant motivations and which serve to attenuate the strength of those motivations. In this paper I discuss a class of cases which has been largely ignored in these discussions, cases which I argue are common synchronic self-control problems, but in which a lack of motivation is the characteristic symptom. I argue that such cases present problems for actional accounts and add some support to the non-actional approach. (shrink)

A crucial question for libertarians about free will and moral responsibility concerns how their accounts secure more control than compatibilism. This problem is particularly exasperating for event-causal libertarianism, as it seems that the only difference between these accounts and compatibilism is that the former require indeterminism. But how can indeterminism, a mere negative condition, enhance control? This worry has led many to conclude that the only viable form of libertarianism is agent-causal libertarianism. In this paper I show (...) that this conclusion is premature. I explain how event-causal libertarianism secures more control than compatibilism by offering a novel argument for incompatibilism. Part of the reason my solution has gone unnoticed is that it is often mistakenly assumed that an agent's control is wholly exhausted by the agent's powers and abilities. I argue, however, that control is constituted not just by what we have the ability to do, but also by what we have the opportunity to do. And it is by furnishing agents with new opportunities that event-causal libertarianism secures enhanced control. In order to defend this claim, I provide an analysis of opportunities and construct a novel incompatibilist argument to show that the opportunity to do otherwise is incompatible with determinism. (shrink)

We investigate flexibility and problem solving in skilled action. We conducted a field study of mountain bike riding that required a learner rider to cope with major changes in technique and equipment. Our results indicate that relatively inexperienced individuals can be capable of fairly complex 'on-the-fly' problem solving which allows them to cope with new conditions. This problem solving is hard to explain for classical theories of skill because the adjustments are too large to be achieved by (...) automatic mechanisms and too complex and rapid to be achieved by cognitive processes as they are usually understood. A recent theory, Mesh, can explain these results because it posits that skill-specific cognitive abilities develop during skill learning and that control typically involves an interplay between cognitive and automatic mechanisms. Here we develop Mesh further, providing a detailed explanation for these problem solving abilities. We argue that causal representation, metacognitive awareness and other forms of performance awareness combine in the formulation and control of action strategies. We also argue that the structure of control present in this case is inconsistent with Bratman's model of intentions, and that, in the face of high uncertainty and risk, intentions can be much more labile than Bratman recognises. In addition, we found limitations and flaws in problem solving which illuminate the representations involved. Finally, we highlight the crucial role of social and cultural learning in the development of complex skills. (shrink)

Translation abstractSummary: Control versus Complexity: Approaches to the Carbon Dioxide Problem at IIASA. In the 1970s and 1980s the International Institute for Applied Systems Analysis (IIASA) hosted several research projects, workshops and conferences in order to discuss the implications of rising carbon dioxide concentrations in the Earth's atmosphere. A number of distinguished scholars, some of whom later became prominent protagonists within the United Nations’ Intergovernmental Panel on Climate Change (IPCC) and sustainability communities more generally, participated in these debates. (...) Since those at IIASA did not engage in obviously related fields such as geophysics or climatology, there is a need to explain how, and via which contexts the issue entered the institute's agenda. This article examines this historical context and contrasts two competing paradigms that emerged at IIASA in order to assess and respond to the carbon dioxide question: The first approach was related to the organisation's research projects on the future of energy systems, which drew on physics, engineering, economics and applied system dynamical modeling; the second approach drew earlier research into ecological management and complex dynamical systems theory, and argued for the use of multiple methods to assess the carbon dioxide question. The first approach invoked ideas of techno‐economical control mechanisms, the second resulted in a more embedded framing of climate change as one of a larger complex of issues relating to sustainable development. Based on resources from IIASA's research repositories and institutional archive, this study retraces these competing discourse framings and outlines the specific research and modeling strategies, policies, and cultural and technological imaginaries related to them. (shrink)

The present paper presents a philosofical analysis of the concept of control based on a specific characterization of the structure of control systems. The paper is subdivided into four section: in the first and the second sections the control dynamics is described and interpreted, while the last two sections briefly copy with the relationships between control, teleonomy, teleology and freedom.

Central to Fischer and Ravizza's theory of moral responsibility is the concept of guidance control, which involves two conditions: (1) moderate reasons-responsiveness, and (2) mechanism ownership. We raise a worry for Fischer and Ravizza's account of (1). If an agent acts contrary to reasons which he could not recognize, this should lead us to conclude that he is not morally responsible for his behaviour; but according to Fischer and Ravizza's account, he satisfies the conditions for guidance control and (...) is therefore morally responsible. We consider ways in which the account of guidance control might be mended. (shrink)

We raise a problem of applicability of RCTs to validate nuclear diagnostic imaging tests. In spite of the wide application of PET and other similar techniques that use radiopharmaceuticals for diagnostic purposes, RCT-based evidence on their validity is sparse. We claim that this is due to a general conceptual problem that we call Prevalence of Treatment, which arises in connection with designing RCTs for testing any diagnostic procedure in the present context of medical research, and is particularly apparent (...) in this case. We also identify three practical reasons why RCTs do not qualify as the best option for PET validation, which have to do with specific characteristics of nuclear diagnostic imaging, and of radiopharmaceuticals. The paper is meant to contribute both to the philosophical discussion on the EBM hierarchy of evidence, and on the specific debate on radiopharmaceuticals in nuclear medicine. (shrink)

The primary argument against Rachlin's act-versus-pattern analysis of self-control is that it is wrong to think of a temptation as a solitary act while the alternative is conceived of as an element of a pattern. Either both are solitary acts or both are members of patterns, however different the patterns may be in their complexity and abstractness.

On the basis of critical discussions which have taken place in recent years in the Federal Republic of Germany, certain methodological, ethical and legal problems arising in relation to controlled trials are discussed. Because of methodological inconsistencies inherent in the experimental approach, the efficacy of a drug must in any case be judged by physicians. This leads to major ethical and even--at least in Germany--legal problems which impose considerable limits on the feasibility of controlled trials in Germany. Editor's note: This (...) paper is written at the invitation of the journal, following the considerable controversy on the ethics of clinical trials in the European Journal of Clinical Pharmacology (8-11). A critical commentary follows the paper with a short response from the authors and a further response from the commentator. (shrink)

I argue that when perception plays a guiding role in intentional bodily action, it is a necessary part of that action. The argument begins with a challenge that necessarily arises for embodied agents, what I call the Many-Many Problem. The Problem is named after its most common case where agents face too many perceptual inputs and too many possible behavioral outputs. Action requires a solution to the Many-Many Problem by selection of a specific linkage between input and (...) output. In bodily action the agent perceptually selects, and in this way perceptually attends to, relevant information so as to guide the execution of specific movements. Since perceptual attention is a necessary part of solving the Many-Many Problem, it is a necessary part of bodily action. Indeed, the process of implementing a solution to the Many-Many Problem, as constrained by the agent's motivational state, just is the agent's performing an intentional bodily action in the relevant way. (shrink)

본고의 목적은 전기 키레네학파가 아리스티포스를 충실히 따르고자 했음에도, 절제 및 인식의 문제와 관련해서는 자신들의 학설에 아리스티포스의 신념을 성공적으로 반영했다고 평가하기 어렵다는 점을 보이는 일이다. 이를 위해 본고는 아리스티포스의 일화들 속에서 절제력을 중요시하고 욕망의 무한확대를 경계하기도 하는 그의 모습을 확인한다. 이어서 전기 학파의 쾌락 개념과 반행복주의를 서술하고, 도덕의 본래적 가치를 부정하는 그들의 입장을 설명하면서, 전기 학파와 아리스티포스의 공통점을 찾는다. 다음으로 전기 학파가 절제력에 대해 침묵했다는 문제점과 인식론적 회의주의를 무리하게 도입했다는 문제점을 제시하고, 바로 이 때문에 그들이 아리스티포스로부터 멀어지게 되었다고 해석한다. 마지막으로 본고는 (...) 이 두 문제점이 에피쿠로스와 밀의 쾌락주의적 윤리학에서 어떻게 처리되었는지 살펴봄으로써, 동적 쾌락을 추구하는 쾌락주의가 염세적 색깔을 띠지 않으려면 인간의 고급 능력 사용을 권장하고 인식론적 회의주의를 포기하거나 완화시킬 필요가 있다고 제안한다. (shrink)

Introduction:Can nature be controlled?. Autonomous nature -- Greco-Roman concepts of nature -- Christianity and nature -- Nature personified : Renaissance ideas of nature -- Controlling nature. Vexing nature : Francis Bacon and the origins of experimentation -- Natural law : Spinoza on natura naturans and natura naturata -- Laws of nature :Lleibniz and Newton -- Epilogue : rambunctious nature in the twenty-first century.

This article offers a historical inquiry into behaviorism and its impact on standard of judgement concerning education policies. Drawing from Aldous Huxley’s reservation towards behaviorism as a scientific movement that naturalizes the role of control in human affairs, the paper maps the impact of behaviorism on economics of education. By tracing the influence of behaviorism in both rational (human capital theory) and quasi-rational (behavioral insight) economics, we draw attention to the activity of knowledge-making that describes and prescribes agency. The (...) paper demonstrates how policy instruments and outcomes are intimately linked to assumptions about personhood, and in the case of behavioral insight, they contribute to the scope of decision-making entrusted to government and business. Utilizing a genealogical approach, the paper invites ethical reflections on the long-term implications of behavioral economics in light of automation. (shrink)

Referring to a problem-oriented research mode, transdisciplinarity faces the challenge of dealing with the complexity of real-world problems in a methodologically controlled manner. As the first st...

Most of us have had the experience of resisting our currently strongest desire, for example, resisting the desire to eat another cookie when eating another cookie is what we most want to do. The puzzle of synchronic self‐control, however, says that this is impossible: an agent cannot ever resist her currently strongest desire. The paper argues that one prominent solution to this puzzle – the solution offered by Al Mele – faces a serious ‘mismatch problem’, which ultimately undermines (...) its plausibility. It is furthermore argued that this problem can be avoided if we adopt a ‘divided mind’ solution instead. (shrink)

According to event-causal libertarianism, an action is free in the sense relevant to moral responsibility when it is caused indeterministically by an agent’s beliefs, desires, intentions, or by their occurrences. This paper attempts to clarify one of the major objections to this theory: the objection that the theory cannot explain the relevance of indeterminism to this kind of freedom (known as free will). Christopher Evan Franklin (2011, 2018) has argued that the problem of explaining the relevance of indeterminism to (...) free will (which he calls “the problem of enhanced control”) arises because it is difficult to see how indeterminism could enhance our abilities, and disappears when we realize that beside the relevant abilities free will requires opportunities. In this paper, I argue that the problem occurs not because of the focus on abilities, but because of the difficulty to explain how indeterminism could contribute to the satisfaction of the sourcehood condition of free will in the framework of event-causal theory of action. (shrink)