The notion of “responsibility gap” with artificial intelligence (AI) was originally introduced in the philosophical debate to indicate the concern that “learning automata” may make more difficult or impossible to attribute moral culpability to persons for untoward events. Building on literature in moral and legal philosophy, and ethics of technology, the paper proposes a broader and more comprehensive analysis of the responsibility gap. The responsibility gap, it is argued, is not one problem but a set of at least four interconnected (...) problems – gaps in culpability, moral and public accountability, active responsibility—caused by different sources, some technical, other organisational, legal, ethical, and societal. Responsibility gaps may also happen with non-learning systems. The paper clarifies which aspect of AI may cause which gap in which form of responsibility, and why each of these gaps matter. It proposes a critical review of partial and non-satisfactory attempts to address the responsibility gap: those which present it as a new and intractable problem (“fatalism”), those which dismiss it as a false problem (“deflationism”), and those which reduce it to only one of its dimensions or sources and/or present it as a problem that can be solved by simply introducing new technical and/or legal tools (“solutionism”). The paper also outlines a more comprehensive approach to address the responsibility gaps with AI in their entirety, based on the idea of designing socio-technical systems for “meaningful human control", that is systems aligned with the relevant human reasons and capacities. (shrink)

In this paper, in line with the general framework of value-sensitive design, we aim to operationalize the general concept of “Meaningful Human Control” in order to pave the way for its translation into more specific design requirements. In particular, we focus on the operationalization of the first of the two conditions investigated: the so-called ‘tracking’ condition. Our investigation is led in relation to one specific subcase of automated system: dual-mode driving systems. First, we connect and compare meaningful human control with (...) a concept of control very popular in engineering and traffic psychology, and we explain to what extent tracking resembles and differs from it. This will help clarifying the extent to which the idea of meaningful human control is connected to, but also goes beyond, current notions of control in engineering and psychology. Second, we take the systematic analysis of practical reasoning as traditionally presented in the philosophy of human action and we adapt it to offer a general framework where different types of reasons and agents are identified according to their relation to an automated system’s behaviour. This framework is meant to help explaining what reasons and what agents play a role in controlling a given system, thereby enabling policy makers to produce usable guidelines and engineers to design systems that properly respond to selected human reasons. In the final part, we discuss a practical example of how our framework could be employed in designing automated driving systems. (shrink)

The paper has two goals. The first is presenting the main results of the recent report Ethics of Connected and Automated Vehicles: recommendations on road safety, privacy, fairness, explainability and responsibility written by the Horizon 2020 European Commission Expert Group to advise on specific ethical issues raised by driverless mobility, of which the author of this paper has been member and rapporteur. The second is presenting some broader ethical and philosophical implications of these recommendations, and using these to contribute to (...) the establishment of Ethics of Transportation as an independent branch of applied ethics. The recent debate on the ethics of Connected and Automated Vehicles presents a paradox and an opportunity. The paradox is the presence of a flourishing debate on the ethics of one very specific transportation technology without ethics of transportation being in itself a well-established academic discipline. The opportunity is that now that a spotlight has been switched on the ethical dimensions of CAVs it may be easier to establish a broader debate on ethics of transportation. While the 20 recommendations of the EU report are grouped in three macro-areas: road safety, data ethics, and responsibility, in this paper they will be grouped according to eight philosophical themes: Responsible Innovation, road justice, road safety, freedom, human control, privacy, data fairness, responsibility. These are proposed as the first topics for a new ethics of transportation. (shrink)

How does the use of military drones affect the legal, political, and moral responsibility of different actors involved in their deployment and design? This volume offers a fresh contribution to the ethics of drone warfare by providing, for the first time, a systematic interdisciplinary discussion of different responsibility issues raised by military drones. The book discusses four main sets of questions: First, from a legal point of view, we analyse the ways in which the use of drones makes the attribution (...) of criminal responsibility to individuals for war crimes more complicated and what adjustments may be required in international criminal law and in military practices to avoid ’responsibility gaps’ in warfare. From a moral and political perspective, the volume looks at the conditions under which the use of military drones by states is impermissible, permissible, or even obligatory and what the responsibilities of a state in the use of drones towards both its citizens and potential targets are. From a socio-technical perspective, what kind of new human machine interaction might drones bring and which new kinds of shared agency and responsibility? Finally, we ask how the use of drones changes our conception of agency and responsibility. The book will be of interest to scholars and students in ethics and to those in law, politics and the military involved in the design, deployment and evaluation of military drones. (shrink)

We argue that lack of direct and conscious control is not, in principle, a reason to be afraid of machines in general and robots in particular: in order to articulate the ethical and political risks of increasing automation one must, therefore, tackle the difficult task of precisely delineating the theoretical and practical limits of sustainable delegation to robots.

How should autonomous vehicles be programmed to behave in the event of an unavoidable accident in which the only choice open is one between causing different damages or losses to different objects or persons? This paper addresses this ethical question starting from the normative principles elaborated in the law to regulate difficult choices in other emergency scenarios. In particular, the paper offers a rational reconstruction of some major principles and norms embedded in the Anglo-American jurisprudence and case law on the (...) “doctrine of necessity”; and assesses which, if any, of these principles and norms can be utilized to find reasonable guidelines for solving the ethical issue of the regulation of the programming of autonomous vehicles in emergency situations. The paper covers the following topics: the distinction between “justification” and “excuse”, the legal prohibition of intentional killing outside self-defence, the incommensurability of goods, and the legal constrains to the use of lethal force set by normative positions: obligations, responsibility, rights, and authority. For each of these principles and constrains the possible application to the programming of autonomous vehicles is discussed. Based on the analysis, some practical suggestions are offered. (shrink)

When should we use care robots? In this paper we endorse the shift from a simple normative approach to care robots ethics to a complex one: we think that one main task of a care robot ethics is that of analysing the different ways in which different care robots may affect the different values at stake in different care practices. We start filling a gap in the literature by showing how the philosophical analysis of the nature of healthcare activities can (...) contribute to robot ethics. We rely on the nature-of-activities approach recently proposed in the debate on human enhancement, and we apply it to the ethics of care robots. The nature-of-activities approach will help us to understand why certain practice-oriented activities in healthcare should arguably be left to humans, but certain goal-directed activities in healthcare can be fulfilled with the assistance of a robot. In relation to the latter, we aim to show that even though all healthcare activities can be considered as practice-oriented, when we understand the activity in terms of different legitimate ‘fine-grained’ descriptions, the same activities or at least certain components of them can be seen as clearly goal-directed. Insofar as it allows us to ethically assess specific functionalities of specific robots to be deployed in well-defined circumstances, we hold the nature-of-activities approach to be particularly helpful also from a design perspective, i.e. to realize the Value Sensitive Design approach. (shrink)

The urban traffic environment is characterized by the presence of a highly differentiated pool of users, including vulnerable ones. This makes vehicle automation particularly difficult to implement, as a safe coordination among those users is hard to achieve in such an open scenario. Different strategies have been proposed to address these coordination issues, but all of them have been found to be costly for they negatively affect a range of human values (e.g. safety, democracy, accountability…). In this paper, we claim (...) that the negative value impacts entailed by each of these strategies can be interpreted as lack of what we call Meaningful Human Control over different parts of a sociotechnical system. We argue that Meaningful Human Control theory provides the conceptual tools to reduce those unwanted consequences, and show how “designing for meaningful human control” constitutes a valid strategy to address coordination issues. Furthermore, we showcase a possible application of this framework in a highly dynamic urban scenario, aiming to safeguard important values such as safety, democracy, individual autonomy, and accountability. Our meaningful human control framework offers a perspective on coordination issues that allows to keep human actors in control while minimizing the active, operational role of the drivers. This approach makes ultimately possible to promote a safe and responsible transition to full automation. (shrink)

Accountability and responsibility are key concepts in the academic and societal debate on Autonomous Weapon Systems, but these notions are often used as high-level overarching constructs and are not operationalised to be useful in practice. “Meaningful Human Control” is often mentioned as a requirement for the deployment of Autonomous Weapon Systems, but a common definition of what this notion means in practice, and a clear understanding of its relation with responsibility and accountability is also lacking. In this paper, we present (...) a definition of these concepts and describe the relations between accountability, responsibility, control and oversight in order to show how these notions are distinct but also connected. We focus on accountability as a particular form of responsibility—the obligation to explain one’s action to a forum—and we present three ways in which the introduction of Autonomous Weapon Systems may create “accountability gaps”. We propose a Framework for Comprehensive Human Oversight based on an engineering, socio-technical and governance perspective on control. Our main claim is that combining the control mechanisms at technical, socio-technical and governance levels will lead to comprehensive human oversight over Autonomous Weapon Systems which may ensure solid controllability and accountability for the behaviour of Autonomous Weapon Systems. Finally, we give an overview of the military control instruments that are currently used in the Netherlands and show the applicability of the comprehensive human oversight Framework to Autonomous Weapon Systems. Our analysis reveals two main gaps in the current control mechanisms as applied to Autonomous Weapon Systems. We have identified three first options as future work for the design of a control mechanism, one in the technological layer, one in the socio-technical layer and one the governance layer, in order to achieve comprehensive human oversight and ensure accountability over Autonomous Weapon Systems. (shrink)

David Shoemaker has claimed that a binary approach to moral responsibility leaves out something important, namely instances of marginal agency, cases where agents seem to be eligible for some responsibility responses but not others. In this paper we endorse and extend Shoemaker’s approach by presenting and discussing one more case of marginal agency not yet covered by Shoemaker or in the other literature on moral responsibility. Our case is that of Kenneth Parks, a Canadian man who drove a long way (...) to his mother-in-law’s and killed her in a state of somnambulism. We support our claim about Parks’ marginal responsibility in three steps: we first deny that Parks acts involuntarily as traditionally claimed in the legal literature; we then propose to extend Shoemaker’s analysis of marginal responsibility based on quality of will so as to include two other dimensions: the moral status of the agent and the actual causal effects of their actions; finally, we distinguish Parks’ marginal responsibility from four other existing concepts: “tracing”, diminished responsibility, causal responsibility, and moral disapproval without responsibility. (shrink)

This paper proposes that autonomous vehicles should be designed to reduce light pollution. In support of this specific proposal, a moral assessment of autonomous vehicles more comprehensive than the dilemmatic life-and-death questions of trolley problem-style situations is presented. The paper therefore consists of two interrelated arguments. The first is that autonomous vehicles are currently still a technology in development, and not one that has acquired its definitive shape, meaning the design of both the vehicles and the surrounding infrastructure is open-ended. (...) Design for values is utilized to articulate a path forward, by which engineering ethics should strive to incorporate values into a technology during its development phase. Second, it is argued that nighttime lighting—a critical supporting infrastructure—should be a prima facie consideration for autonomous vehicles during their development phase. It is shown that a reduction in light pollution, and more boldly a better balance of lighting and darkness, can be achieved via the design of future autonomous vehicles. Two case studies are examined through which autonomous vehicles may be designed for “driving in the dark.” Nighttime lighting issues are thus inserted into a broader ethics of autonomous vehicles, while simultaneously introducing questions of autonomous vehicles into debates about light pollution. (shrink)

The paper presents a framework to realise “meaningful human control” over Automated Driving Systems. The framework is based on an original synthesis of the results of the multidisciplinary research project “Meaningful Human Control over Automated Driving Systems” lead by a team of engineers, philosophers, and psychologists at Delft University of the Technology from 2017 to 2021. Meaningful human control aims at protecting safety and reducing responsibility gaps. The framework is based on the core assumption that human persons and institutions, not (...) hardware and software and their algorithms, should remain ultimately—though not necessarily directly—in control of, and thus morally responsible for, the potentially dangerous operation of driving in mixed traffic. We propose an Automated Driving System to be under meaningful human control if it behaves according to the relevant reasons of the relevant human actors (tracking), and that any potentially dangerous event can be related to a human actor (tracing). We operationalise the requirements for meaningful human control through multidisciplinary work in philosophy, behavioural psychology and traffic engineering. The tracking condition is operationalised via a proximal scale of reasons and the tracing condition via an evaluation cascade table. We review the implications and requirements for the behaviour and skills of human actors, in particular related to supervisory control and driver education. We show how the evaluation cascade table can be applied in concrete engineering use cases in combination with the definition of core components to expose deficiencies in traceability, thereby avoiding so-called responsibility gaps. Future research directions are proposed to expand the philosophical framework and use cases, supervisory control and driver education, real-world pilots and institutional embedding. (shrink)

When should humans enhance themselves? We try to answer this question by engaging in a conceptual analysis of the nature of different activities. We think that cognitive enhancement is morally impermissible in some practice-oriented activities, such as some educational activities, when it is the case both that cognitive enhancement would negatively affect the point of those activities and that we have good reasons to value that point. We then argue that cognitive enhancement should be allowed in two groups of cases, (...) namely in practice-oriented activities, such as recreational activities on which little moral value or social import hangs, and in some prominently goal-directed activities, such as high-responsibility professions, the goal of which has significant moral or social value. Finally, we argue that the use of efficacious and relatively safe cognitive enhancers may even be obligatory in those high-responsibility professions under certain special circumstances. (shrink)

We propose a multi-step evaluation schema designed to help procurement agencies and others to examine the ethical dimensions of autonomous systems to be applied in the security sector, including autonomous weapons systems.

The scope and mission of the history of science have been constant objects of reflection and debate within the profession. Recently, Lorraine Daston has called for a shift of focus: from the history of science to the history of knowledge. Such a move is an attempt at broadening the field and ridding it of the contradictions deriving from its modernist myth of origin and principle of demarcation. Taking the move from a pluralistic concept of medicine, the present paper explores the (...) actual and possible contributions that a history of knowledge can offer to the history of medicine in particular. As we will argue, the history of medicine has always been a history of knowledge, but for good reasons has always stuck to the concept of medicine as its object and problem throughout the ages, including the modern, scientific one. We argue that, in the history of medicine, the demarcation between scientific and non‐scientific represents an accident, but is not foundational as in the case of natural science. Furthermore, the history of medicine programmatically played a role in at least two academic domains (history proper and medical education), adjusting historical narratives of medical knowledge to its audience. Accordingly, we underscore that the history of both science and medicine, as traditionally defined, already provides room for almost the whole spectrum of approaches to history. Moreover, their different myths of origin can, and indeed must, be included in the reflexivity of the historical gaze. We argue that the position towards a history of science, medicine, or knowledge is not a question of narrative or theory, rather, it is a question of relevance and awareness of extant contexts. (shrink)

This is the short version, in French translation by Anne Querrien, of the originally jointly authored paper: Müller, Vincent C., ‘Autonomous killer robots are probably good news’, in Ezio Di Nucci and Filippo Santoni de Sio, Drones and responsibility: Legal, philosophical and socio-technical perspectives on the use of remotely controlled weapons. - - - L’article qui suit présente un nouveau système d’armes fondé sur des robots qui risque d’être prochainement utilisé. À la différence des drones qui sont manoeuvrés (...) à distance mais comportent une part importante de discernement humain, il s’agit de machines programmées pour défendre, attaquer, ou tuer de manière autonome. Les auteurs, philosophes, préfèrent prévenir de leur prochaine diffusion et obtenir des Nations Unies leur régulation. Une campagne internationale propose plutôt leur interdiction. (shrink)

In un saggio del 1984 destinato a una certa fortuna interpretativa, lo storico dell’architettura Jean-Louis Cohen presentava la figura dell’architetto-intellettuale come l’esito di condizioni specifiche che avevano permesso ad alcuni architetti italiani – in contrapposizione, per esempio, a quelli francesi – di affiancare alla propria attività professionale e di insegnamento una riflessione approfondita sulle condizioni del proprio operare, rafforzando un’interpretazione dell’architettura com...

In lieu of an abstract, here is a brief excerpt of the content:The "Thinking of Thinking" in MetaphysicsA.9 JOSEPH G. DE FILIPPO a+~6v &Qa voE[, e~eQ ~o~t ~6 XQ&~O~OV, xetl. I~o~tv ~1VdOloLgvo1]o~t0g v6"qotg. (A.9, 1o74b33-34) Therefore it thinks itself, if indeed it is most powerful, and its thinking is the thinking of thinking. Thus culminates Aristotle's treatment of God's activity in the twelfth book of the Metaphysics. The conclusion seems transparent. God is an intellect (vo~Sg); since he is also (...) the best thing, and it is absurd that he should think anything worse than himself, he therefore thinks himself. And since he is nothing other than a thinking (v6"qotg), what he thinks of is thinking (vo~loEe0g). "Thinking of thinking," however, is a notion that is as ambiguous in ancient Greek as it is in English. It is just not clear what sort of thing such a reflexively self-thinking mind would be, though it might seem natural to construe the activity of Aristotle's God as somehow an act of pure self-awareness, self-consciousness or self-knowledge. On such an understanding God would be a separately existing consciousness, with himself or his activity as the only object of that consciousness. In 196 9 R. Norman published a paper which disputed this sort of interpretation.' He argued that when one considers Aristotle's theory of, R. Norman, "Aristotle's Philosopher-God," Phronesis 14 (1969): 63-74 (reprinted in Articles on Aristotle, vol. 4, ed. J. Barnes, M. Schofield and R. Sorabji [London, 1979], 93-1o2). While I a-m indebted to Norman's basic approach, my treatment of God's self-thought differs substantially from his in two respects. First, although it isessential to recognize that in Aristotle human thought too thinks itself, this does not justify the conclusion that God's thinking is "of the same kind as our own normal abstract thought" (73). As 1shall argue below in section 4, the characterization of the Prime Mover as v6"qoLgvo~o~t0g is meant specifically to differentiate its activity from the selfthinking of human volSg;human thought could never be characterized as v6"qotgvofioEt0g.Secondly, Norman does not isolate the real source of A.9's argument, which, as l shall argue, is the potential conflict between Aristotle's own physical and psychological theories. M. Wedin, Mind and Imagination in Aristotle (New Haven, 1988), 229-45, also argues against Norman's thesis that human and divine thought are the same in kind. [543] 544 JOURNAL OF THE HISTORY OF PHILOSOPHY 33:4 OCTOBER 1995 volSg from De Anima III.4-8, it is clear that the conception of God as a selfthinking mind does not entail that he engages in self-contemplation in the normal sense of the term. On Norman's interpretation, God thinks himself because he thinks a definite content with which he is numerically identical, according to the well-known Aristotelian doctrine of the unity of vo~g with its object of thought (vo~lx6v). To describe this act simply as self-contemplation belies the fact that its object must have an intelligible content, which in the case of God is not adequately described as his "self," for this self would indeed be nothing more than an act of self-contemplation. While many commentators have followed Norman in recognizing that the description of God as "thinking himself" is drawn from the De Anima's characterization of self-thinking, there has persisted a tendency to interpret Aristotle 's God as engaged in an act of pure self-awareness or under some similar description.' The reason for this persistence is due, I believe, to a crucial difference between chapters 7 and 9 of book A. Whereas both chapters characterize the Prime Mover (PM) as a self-thinker, in chapter 9 he is also said to be ~1 v61]otg vo~loec0g V6tlOtg (the thinking of thinking), which understandably appears to be a much more specific delineation of his thought content. A. 7 tells us that the PM is a vofJg that thinks itself (to72b18-3o); according to De Anima III. 4 that simply is what voa)g does at the highest level of actuality, since it... (shrink)

In 1979, A. D. Woozley proposed an interpretation of Apology 29c–d which was intended to alleviate the well-known tension between the Apology and Crito on the citizen's obligation to obey the law. According to his interpretation, the court's hypothetical offer – to release Socrates on the condition that he will be put to death if he does not give up philosophy – is not an order, but a warning as to what would happen should he accept their acquittal and yet (...) continue to philosophize. Obviously, if this is the spirit of the words he puts into the court's mouth, Socrates would not be guilty of disobedience should he refuse the offer, or even if he should accept it and then disregard the warning. To support this interpretation, Woozley suggested a novel construal of the words in which Socrates makes his refusal – πεσομαι δ μγγον τ θε ἢ μν. (shrink)

In lieu of an abstract, here is a brief excerpt of the content:The "Thinking of Thinking" in MetaphysicsA.9 JOSEPH G. DE FILIPPO a+~6v &Qa voE[, e~eQ ~o~t ~6 XQ&~O~OV, xetl. I~o~tv ~1VdOloLgvo1]o~t0g v6"qotg. (A.9, 1o74b33-34) Therefore it thinks itself, if indeed it is most powerful, and its thinking is the thinking of thinking. Thus culminates Aristotle's treatment of God's activity in the twelfth book of the Metaphysics. The conclusion seems transparent. God is an intellect (vo~Sg); since he is also (...) the best thing, and it is absurd that he should think anything worse than himself, he therefore thinks himself. And since he is nothing other than a thinking (v6"qotg), what he thinks of is thinking (vo~loEe0g). "Thinking of thinking," however, is a notion that is as ambiguous in ancient Greek as it is in English. It is just not clear what sort of thing such a reflexively self-thinking mind would be, though it might seem natural to construe the activity of Aristotle's God as somehow an act of pure self-awareness, self-consciousness or self-knowledge. On such an understanding God would be a separately existing consciousness, with himself or his activity as the only object of that consciousness. In 196 9 R. Norman published a paper which disputed this sort of interpretation.' He argued that when one considers Aristotle's theory of, R. Norman, "Aristotle's Philosopher-God," Phronesis 14 (1969): 63-74 (reprinted in Articles on Aristotle, vol. 4, ed. J. Barnes, M. Schofield and R. Sorabji [London, 1979], 93-1o2). While I a-m indebted to Norman's basic approach, my treatment of God's self-thought differs substantially from his in two respects. First, although it isessential to recognize that in Aristotle human thought too thinks itself, this does not justify the conclusion that God's thinking is "of the same kind as our own normal abstract thought" (73). As 1shall argue below in section 4, the characterization of the Prime Mover as v6"qoLgvo~o~t0g is meant specifically to differentiate its activity from the selfthinking of human volSg;human thought could never be characterized as v6"qotgvofioEt0g.Secondly, Norman does not isolate the real source of A.9's argument, which, as l shall argue, is the potential conflict between Aristotle's own physical and psychological theories. M. Wedin, Mind and Imagination in Aristotle (New Haven, 1988), 229-45, also argues against Norman's thesis that human and divine thought are the same in kind. [543] 544 JOURNAL OF THE HISTORY OF PHILOSOPHY 33:4 OCTOBER 1995 volSg from De Anima III.4-8, it is clear that the conception of God as a selfthinking mind does not entail that he engages in self-contemplation in the normal sense of the term. On Norman's interpretation, God thinks himself because he thinks a definite content with which he is numerically identical, according to the well-known Aristotelian doctrine of the unity of vo~g with its object of thought (vo~lx6v). To describe this act simply as self-contemplation belies the fact that its object must have an intelligible content, which in the case of God is not adequately described as his "self," for this self would indeed be nothing more than an act of self-contemplation. While many commentators have followed Norman in recognizing that the description of God as "thinking himself" is drawn from the De Anima's characterization of self-thinking, there has persisted a tendency to interpret Aristotle 's God as engaged in an act of pure self-awareness or under some similar description.' The reason for this persistence is due, I believe, to a crucial difference between chapters 7 and 9 of book A. Whereas both chapters characterize the Prime Mover (PM) as a self-thinker, in chapter 9 he is also said to be ~1 v61]otg vo~loec0g V6tlOtg (the thinking of thinking), which understandably appears to be a much more specific delineation of his thought content. A. 7 tells us that the PM is a vofJg that thinks itself (to72b18-3o); according to De Anima III. 4 that simply is what voa)g does at the highest level of actuality, since it... (shrink)

In 1979, A. D. Woozley proposed an interpretation of Apology 29c–d which was intended to alleviate the well-known tension between the Apology and Crito on the citizen's obligation to obey the law. According to his interpretation, the court's hypothetical offer – to release Socrates on the condition that he will be put to death if he does not give up philosophy – is not an order, but a warning as to what would happen should he accept their acquittal and yet (...) continue to philosophize. Obviously, if this is the spirit of the words he puts into the court's mouth, Socrates would not be guilty of disobedience should he refuse the offer, or even if he should accept it and then disregard the warning. To support this interpretation, Woozley suggested a novel construal of the words in which Socrates makes his refusal – πεσομαι δ μγγον τ θε ἢ μν. (shrink)

This article asks whether a myth can be collapsed into history by describing the material signs of memory as "evidence." In a story dear to the Venetian hearts, in 1177 the Republic of Venice defeated the imperial fleet and brought peace back to Italy. For centuries, this narrative was recounted and its truth upheld against growing political and scholarly criticism, first through the ritual display of evidence, then through its discussion in a large literary production. In the seventeenth century, it (...) is argued, the notion of evidence was abandoned in favor of the sheer affirmation of power. (shrink)