Autonomous Weapons

Danaher (2016) has argued that increasing robotization can lead to retribution gaps: Situation in which the normative fact that nobody can be justly held responsible for a harmful outcome stands in conflict with our retributivist moral dispositions. In this paper, we report a cross-cultural empirical study based on Sparrow’s (2007) famous example of an autonomous weapon system committing a war crime, which was conducted with participants from the US, Japan and Germany. We find that (i) people manifest a considerable willingness (...) to hold autonomous systems morally responsible, (ii) partially exculpate human agents when interacting with such systems, and that more generally (iii) the possibility of normative responsibility gaps is indeed at odds with people’s pronounced retributivist inclinations. We discuss what these results mean for potential implications of the retribution gap and other positions in the responsibility gap literature. (shrink)

A legal obligation to conduct weapons reviews is a means by which the international community can ensure that States assess whether the use of new types of weapons in armed conflict would raise humanitarian concerns. The use of artificial intelligence in weapon systems greatly complicates the process of conducting reviews, particularly where a weapon system is capable of continuing to ‘learn’ on its own after being deployed on the battlefield. This paper surveys current understandings of the weapons review challenges presented (...) by in situ learning and outlines some concepts which may inform appropriate changes to existing weapons review processes. The main arguments made are that: a distinction should be drawn between learning capabilities that are limited to optimising existing behaviour and capabilities that extend to acquiring new behaviours; and the extent of a weapon system’s learning capabilities should be used as the basis for defining a schedule of weapons reviews. (shrink)

Advances in AI research have brought increasingly sophisticated capabilities to AI systems and heightened the societal consequences of their use. Researchers and industry professionals have responded by contemplating responsible principles and practices for AI system design. At the same time, defense institutions are contemplating ethical guidelines and requirements for the development and use of AI for warfare. However, varying ethical and procedural approaches to technological development, research emphasis on offensive uses of AI, and lack of appropriate venues for multistakeholder dialogue (...) have led to differing operationalization of responsible AI principles and practices among civilian and defense entities. We argue that the disconnect between civilian and defense responsible development and use practices leads to underutilization of responsible AI research and hinders the implementation of responsible AI principles in both communities. We propose a research roadmap and recommendations for dialogue to increase exchange of responsible AI development and use practices for AI systems between civilian and defense communities. We argue that generating more opportunities for exchange will stimulate global progress in the implementation of responsible AI principles. (shrink)

Value Sensitive Design (VSD) is a design methodology developed by Batya Friedman and Peter Kahn (2003) that brings in moral deliberations in an early stage of a design process. It assumes that neither technology itself is value neutral, nor shifts the value-ladennes to the sole usage of technology. This paper adds to emerging literature onVSD for autonomous weapons systems development and discusses extant literature on values in autonomous systems development in general and in autonomous weapons development in particular. I identify (...) opportunities, such as public debates, and threats, such as the classified nature of the design process, for VSD in autonomous weapons development. This paper contributes to academic debates about the non-neutrality of technology by pointing out that values have been and can be explicitly designed into autonomous systems. It is informative for policy makers and designers who are tasked with developing actual autonomous weapons or policies around such systems, as they learn about an established design methodology that is sensitive to societal concerns and industry needs and that can be applied to autonomous weapons systems. (shrink)

As artificial intelligence becomes more sophisticated and robots approach autonomous decision-making, debates about how to assign moral responsibility have gained importance, urgency, and sophistication. Answering Stenseke’s (2022a) call for scaffolds that can help us classify views and commitments, we think the current debate space can be represented hierarchically, as answers to key questions. We use the resulting taxonomy of five stances to differentiate—and defend—what is known as the “blank check” proposal. According to this proposal, a person activating a robot could (...) willingly make themselves answerable for whatever events ensue, even if those events stem from the robot’s autonomous decision(s). This blank check solution was originally proposed in the context of automated warfare (Champagne & Tonkens, 2015), but we extend it to cover all robots. We argue that, because moral answerability in the blank check is accepted voluntarily and before bad outcomes are known, it proves superior to alternative ways of assigning blame. We end by highlighting how, in addition to being just, this self-initiated and prospective moral answerability for robot harm provides deterrence that the four other stances cannot match. (shrink)

In “Accepting Moral Responsibility for the Actions of Autonomous Weapons Systems—a Moral Gambit” (2022), Mariarosaria Taddeo and Alexander Blanchard answer one of the most vexing issues in current ethics of technology: how to close the so-called “responsibility gap”? Their solution is to require that autonomous weapons systems (AWSs) may only be used if there is some human being who accepts the ex ante responsibility for those actions of the AWS that could not have been predicted or intended (in such cases, (...) the human being takes what the authors call the “moral gambit”). The authors then propose several institutional safeguards to implement in order to ensure that the moral gambit is taken in a fair and just way. This paper explores this suggestion in the context of the institutional settings within which AWSs are most likely to be deployed. It raises some concerns as to the feasibility of Taddeo and Blanchard’s proposal, in light of the recent empirical work on the incentive structures likely to exist within militaries. It then presents a potential problem that may arise in case the accountability mechanisms are successfully implemented. (shrink)

I here consider the reasonableness of punishing future autonomous military robots. I argue that it is an engineering desideratum that these devices be responsive to moral considerations as well as human criticism and blame. Additionally, I argue that someday it will be possible to build such machines. I use these claims to respond to the no subject of punishment objection to deploying autonomous military robots, the worry being that an “accountability gap” could result if the robot committed a war crime. (...) (Sparrow 2007). But these robots will be blameworthy and morally responsible for their wrongful conduct. I then show that we have reasons to punish them based both on good effects and the impact on the robot, itself. (shrink)

The introduction of Autonomous Military Systems (AMS) onto contemporary battlefields raises concerns that they will bring with them the possibility of a techno-responsibility gap, leaving insecurity about how to attribute responsibility in scenarios involving these systems. In this work I approach this problem in the domain of applied ethics with foundational conceptual work on autonomy and responsibility. I argue that concerns over the use of AMS can be assuaged by recognising the richly interrelated context in which these systems will most (...) likely be deployed. This will allow us to move beyond the solely individualist understandings of responsibility at work in most treatments of these cases, toward one that includes collective responsibility. This allows us to attribute collective responsibility to the collectives of which the AMS form a part, and to account for the distribution of burdens that follows from this attribution. I argue that this expansion of our responsibility practices will close at least some otherwise intractable techno-responsibility gaps. (shrink)

Anticiper les épidémies est le souhait le plus vif de toutes les institutions qui veillent sur la santé publique. Un raisonnement naïf permet de penser qu’il suffit de faire évoluer en laboratoire un organisme potentiellement pathogène pour savoir comment sa descendance pourra devenir plus virulente pour l’homme. Il est alors facile de faire croître cet organisme sur des cellules humaines, qu’il infecte mal pour commencer, puis de retenir ses descendants au fur et à mesure qu’ils deviennent plus infectieux. La vision (...) optimiste affirme alors que les caractériser permettra de construire des défenses appropriées. Bien des variations sur ce thème sont possibles, y compris avec l’aide de la synthèse artificielle de variants dont on saurait imaginer la pathogénicité pour l’homme. Depuis plus de deux décennies, des chercheurs développent des expériences de ce type, dite de « gain de fonction » (GdF). Leur interprétation repose sur l’illusion que nous savons prévoir l’évolution des êtres vivants. En réalité, ceux que nous observons aujourd’hui ont été sélectionnés au cours du temps précisément parce qu’ils conduisent toujours à des comportements imprévisibles. Une règle clé de l’évolution est que la vie, pour durer dans des circonstances qui ne peuvent être prévues, doit sans cesse faire émerger des formes innovantes. À ce caractère propre à la vie s’ajoute un imprévisible lié à l’impéritie humaine, cause de l’omniprésence des accidents de laboratoire. Or, en virologie spécialement, l’accident est la règle et non l’exception. Il faut ajouter que le caractère public de ces expériences les rend utilisables par toutes les structures malveillantes qui sont hélas répandues dans le monde. Il n’est pas difficile alors d’imaginer les conséquences catastrophiques des expériences de GdF. Nous proposons donc qu’elles soient purement et simplement interdites et que toute transgression soit sévèrement punie, en frappant d’abord au porte-monnaie, ce qui semble être la meilleure façon de faire dans nos sociétés devenues universellement vénales. (shrink)

Dans un monde dominé par les conflits, il arrive qu’il faille se défendre. L’invention des armes a évolué en parallèle avec le savoir technique, puis scientifique, et il est même arrivé que la recherche d’applications militaires ait joué un rôle moteur dans la découverte scientifique. Lorsque les armes ne servent pas à attaquer d’autres nations, leur fabrication et leur commerce se justifient morale- ment. Il faut cependant qu’elles ne puissent échapper à ceux qui les construisent et doivent en contrôler la (...) diffusion. Facilement mise en œuvre, la guerre utilisant des organismes vivants ou ce qu’ils pro- duisent comme armes est très ancienne. Or l’originalité des êtres vivants est qu’ils se multiplient, qu’un seul individu suffit à recouvrir la Terre de ses descendants. La biologie ne se prête jamais à un usage militaire moralement défendable parce que le confinement au champ de bataille est impossible. De ce fait, les applications de la biologie ont été vite écartées du domaine de la Défense permise par les traités internationaux. La Convention de 1972 sur l’Interdiction des Armes Biologiques et à Toxines (CIABT), entrée en vigueur en 1975, est aujourd’hui signée par 183 pays (ratifiée par 175) sur un total de 193, sans compter Taïwan qui l’a ratifiée mais n’est pas reconnu par l’ONU. Elle interdit la mise au point, la fabrication, l’acquisition, le transfert, la conservation, le stockage et l’utilisation des armes bio- logiques et à toxines. Elle constitue un élément essentiel de la lutte contre la prolifération des armes de destruction massive. (shrink)

Technological progress has brought about the emergence of machines that have the capacity to take human lives without human control. These represent an unprecedented threat to humankind. This paper starts from the example of chemical weapons, now banned worldwide by the Geneva protocol, to illustrate how technological development initially aimed at the benefit of humankind has, ultimately, produced what is now called the “Weaponization of Artificial Intelligence (AI)”. Autonomous Weapon Systems (AWS) fail the so-called discrimination principle, yet, the wider public (...) is largely unaware of this problem. Given that ongoing scientific research on AWS, performed in the military sector, is generally not made available to the public domain, many of the viewpoints on this subject, expressed across different media, invoke common sense rather than scientific evidence. Yet, the implications of a potential weaponization of our work as scientists, especially in the field of AI, are reaching further than some may think. The potential consequences of a deployment of AWS for citizen stakeholders are incommensurable, and it is time to raise awareness in the public domain of the kind of potential threats identified, and to encourage legal policies ensuring that these threats will not materialize. (shrink)

Militairy technology is developing at a rapid pace and we are seeing a growing number of weapons with increasing levels of autonomy being developed and deployed. This raises various legal, ethical, and security concerns. The absence of clear international rules setting limits and governing the use of autonomous weapons is extremely concerning. There is an urgent need for the international community to work together towards a treaty not only to safeguard ethical and legal norms, but also for our shared security. (...) This article explains why a treaty on autonomous weapons is needed and achievable. It goes into what a treaty could consist of to establish an international norm and set rules and limits on autonomy in weapon systems. (shrink)

Space warfare is warfare that takes place in outer space. It involves ground-to-space, space-to-ground, and space-to-space violence between nations or peoples. The violence can involve kinetic weapons, directed energy weapons, or electronic destruction. International law, specifically, the Outer Space Treaty and SALT I, currently bans weapons of mass destruction from being put into space, although one wonders if one country were to violate the ban whether others would follow suit. In this paper, I argue that that if there is a (...) non-consequentialist morality of space war, then one country can unjustly attack a second country’s space vehicles. This in turn depends on property rights. But there is no adequate theory of property rights with regard to space war. Hence, there is no non-consequentialist morality of a space war. (shrink)

In voicing commitments to the principle that the adoption of artificial-intelligence (AI) tools by armed forces should be done responsibly, a growing number of states have referred to a concept of “Responsible AI.” As part of an effort to help develop the substantive contours of that concept in meaningful ways, this position paper introduces a notion of “responsible reliance.” It is submitted that this notion could help the policy conversation expand from its current relatively narrow focus on interactions between an (...) AI system and its end-user to also encompass the wider set of interdependencies involved in fulfilling legal obligations concerning the use of AI in armed conflicts. The authors argue that to respect international humanitarian law and ensure accountability, states ought to devise and maintain a framework that ensures that natural persons involved in the use an AI tool in an armed conflict could responsibly rely at least on: (1) the tool’s technical aspects, (2) the conduct of other people involved in development and use of that AI tool; and (3) the policies and processes implemented at the state level. According to the authors, the “responsible reliance” notion could serve, among other examples, as a basis on which to articulate legal requirements, prohibitions, and permissions across diverse areas, from the design of AI tools to human-machine interactions to configuration of responsible-command frameworks. (shrink)

The article focuses on the inconsistency between the European Commission’s position on excluding military AI from its emerging AI policy, and at the same time EU policy initiatives targeted at supporting military and defence elements of AI on the EU level. It leads to the question, what, then, does the debate on military AI suggest to the EU’s actorness discussed in the light of Europe as a power debate with a particular focus on Normative Power Europe, Market Power Europe, and (...) Military Power Europe. By employing discourse analysis, the article examines the EU’s AI strategic discourse, consisting of selected AI-related policy documents from different EU institutions. As a result, the article proposes the Military Power Europe definition based on four categories proposed from the Europe as power debate: ways of action, self-definition, preferred international engagement, and the role of the military. It argues that alongside normative proposals for military AI governance, there are evident desires for militarization in the context of AI development, and a considerable role for the military in the future directions of the EU’s digital and security policies. Despite the inconsistency among EU institutions, military AI appears to be actively discussed within the selected discourse, and so is a part of the EU’s emerging AI policy. (shrink)

Reprint of a chapter first appearing in African Values, Ethics, and Technology: Questions, Issues, and Approaches (2021).

The ongoing debate on the ethics of using artificial intelligence (AI) in military contexts has been negatively impacted by the predominant focus on the use of lethal autonomous weapon systems (LAWS) in war. However, AI technologies have a considerably broader scope and present opportunities for decision support optimization across the entire spectrum of the military decision-making process (MDMP). These opportunities cannot be ignored. Instead of mainly focusing on the risks of the use of AI in target engagement, the debate about (...) responsible AI should (i) concern each step in the MDMP, and (ii) take ethical considerations and enhanced performance in military operations into account. A characterization of the debate on responsible AI in the military, considering both machine and human weaknesses and strengths, is provided in this paper. We present inroads into the improvement of the MDMP, and thus military operations, through the use of AI for decision support, taking each quadrant of this characterization into account. (shrink)

This paper addresses the ethical challenges raised by the use of lethal autonomous weapons systems. Using aspects of the philosophy of Martin Heidegger, the paper demonstrates that lethal autonomous weapons systems create ethical problems because of the lack of moral agency in an autonomous system, and the inauthentic nature of the deaths caused by such a system. The paper considers potential solutions for these issues before arguing that from a Heideggerian standpoint they cannot be overcome, and thus the development and (...) use of lethal autonomous weapons systems should be resisted and prohibited. (shrink)

In this paper we introduce a computational control framework that can keep AI-driven military autonomous devices operating within the boundaries set by applicable rules of International Humanitarian Law (IHL) related to targeting. We discuss the necessary legal tests and variables, and introduce the structure of a hypothetical IHL-compliant targeting system.

Technological developments in the sphere of artificial intelligence (AI) inspire debates about the implications of autonomous weapon systems (AWS), which can select and engage targets without human intervention. While increasingly more systems which could qualify as AWS, such as loitering munitions, are reportedly used in armed conflicts, the global discussion about a system of governance and international legal norms on AWS at the United Nations Convention on Certain Conventional Weapons (UN CCW) has stalled. In this article we argue for the (...) necessity to adopt legal norms on the use and development of AWS. Without a framework for global regulation, state practices in using weapon systems with AI-based and autonomous features will continue to shape the norms of warfare and affect the level and quality of human control in the use of force. By examining the practices of China, Russia, and the United States in their pursuit of AWS-related technologies and participation at the UN CCW debate, we acknowledge that their differing approaches make it challenging for states parties to reach an agreement on regulation, especially in a forum based on consensus. Nevertheless, we argue that global governance on AWS is not impossible. It will depend on the extent to which an actor or group of actors would be ready to take the lead on an alternative process outside of the CCW, inspired by the direction of travel given by previous arms control and weapons ban initiatives. (shrink)

A classic objection to autonomous weapon systems (AWS) is that these could create so-called responsibility gaps, where it is unclear who should be held responsible in the event that an AWS were to violate some portion of the law of armed conflict (LOAC). However, those who raise this objection generally do so presenting it as a problem for AWS as a whole class of weapons. Yet there exists a rather wide range of systems that can be counted as “autonomous weapon (...) systems”, and so the objection is too broad. In this article I present a taxonomic approach to the objection, examining a number of systems that would count as AWS under the prevalent definitions provided by the United States Department of Defense and the International Committee of the Red Cross, and I show that for virtually all such systems there is a clear locus of responsibility which presents itself as soon as one focuses on specific systems, rather than general notions of AWS. In developing these points, I also suggest a method for dealing with near-future types of AWS which may be thought to create situations where responsibility gaps can still arise. The main purpose of the arguments is, however, not to show that responsibility gaps do not exist or can be closed where they do exist. Rather, it is to highlight that any arguments surrounding AWS must be made with reference to specific weapon platforms imbued with specific abilities, subject to specific limitations, and deployed to specific times and places for specific purposes. More succinctly, the arguments show that we cannot and should not aim to treat AWS as if all of these shared all morally relevant features, but instead on a case-by-case basis. Thus, we must contend with the realities of weapons development and deployment, and tailor our arguments and conclusions to those realities, and with an eye to what facts obtain for particular systems fulfilling particular combat roles. (shrink)

The ethical Principle of Proportionality requires combatants not to cause collateral harm excessive in comparison to the anticipated military advantage of an attack. This principle is considered a major (and perhaps insurmountable) obstacle to ethical use of autonomous weapon systems (AWS). This article reviews three possible solutions to the problem of achieving Proportionality compliance in AWS. In doing so, I describe and discuss the three components Proportionality judgments, namely collateral damage estimation, assessment of anticipated military advantage, and judgment of “excessiveness”. (...) Some possible approaches to Proportionality compliance are then presented, such as restricting AWS operations to environments lacking civilian presence, using AWS in targeted strikes in which proportionality judgments are pre-made by human commanders, and a ‘price tag’ approach of pre-assigning acceptable collateral damage values to military hardware in conventional attritional warfare. The article argues that application of these three compliance methods would result in AWS’ achieving acceptable Proportionality compliance levels in many combat environments and scenarios, allowing AWS to perform most key tasks in conventional warfare. (shrink)

The use of AI in weapons systems raises numerous ethical issues. To date, work on weaponized AI has tended to be theoretical and normative in nature, consisting in critical policy analyses and ethical considerations, carried out by philosophers, legal scholars, and political scientists. However, adequately addressing the cultural and social dimensions of technology requires insights and methods from empirical moral and cultural psychology. To do so, this position piece describes the motivations for and sketches the nature of a normative, cultural (...) psychology of weaponized AI. The motivations for this project include the increasingly global, cross-cultural and international, nature of technologies, and counter-intuitive nature of normative thoughts and behaviors. The nature of this project consists in developing standardized measures of AI ethical reasoning and intuitions, coupled with questions exploring the development of norms, administered and validated across different cultural groups and disciplinary contexts. The goal of this piece is not to provide a comprehensive framework for understanding the cultural facets and psychological dimensions of weaponized AI but, rather, to outline in broad terms the contours of an emerging research agenda. (shrink)

This paper presents a new trilemma with respect to resolving the control and alignment problems in machine ethics. Section 1 outlines three possible types of artificial moral agents (AMAs): (1) 'Inhuman AMAs' programmed to learn or execute moral rules or principles without understanding them in anything like the way that we do; (2) 'Better-Human AMAs' programmed to learn, execute, and understand moral rules or principles somewhat like we do, but correcting for various sources of human moral error; and (3) 'Human-Like (...) AMAs' programmed to understand and apply moral values in broadly the same way that we do, with a human-like moral psychology. Sections 2–4 then argue that each type of AMA generates unique control and alignment problems that have not been fully appreciated. Section 2 argues that Inhuman AMAs are likely to behave in inhumane ways that pose serious existential risks. Section 3 then contends that Better-Human AMAs run a serious risk of magnifying some sources of human moral error by reducing or eliminating others. Section 4 then argues that Human-Like AMAs would not only likely reproduce human moral failures, but also plausibly be highly intelligent, conscious beings with interests and wills of their own who should therefore be entitled to similar moral rights and freedoms as us. This generates what I call the New Control Problem: ensuring that humans and Human-Like AMAs exert a morally appropriate amount of control over each other. Finally, Section 5 argues that resolving the New Control Problem would, at a minimum, plausibly require ensuring what Hume and Rawls term ‘circumstances of justice’ between humans and Human-Like AMAs. But, I argue, there are grounds for thinking this will be profoundly difficult to achieve. I thus conclude on a skeptical note. Different approaches to developing ‘safe, ethical AI’ generate subtly different control and alignment problems that we do not currently know how to adequately resolve, and which may or may not be ultimately surmountable. (shrink)

A 4ª Revolução Industrial é o culminar da era digital. Atualmente, tecnologias como robótica, nanotecnologia, genética e inteligência artificial prometem transformar nosso mundo e a maneira como vivemos. O campo da Segurança e da Ética da Inteligência Artificial são áreas de pesquisa emergentes que vêm ganhando popularidade nos últimos anos. Diversas organizações de cunho privado, público e não governamentais têm publicado diretrizes propondo princípios éticos para a regulamentação do uso e desenvolvimento de sistemas inteligentes autônomos. Meta-análises do campo de pesquisa (...) em Ética da IA apontam uma convergência sobre certos princípios éticos que, supostamente, governam a indústria da IA. Entretanto, pouco se sabe sobre a eficiência desta forma de “Ética”. Neste estudo, gostaríamos de realizar uma análise crítica do atual estado da Ética da IA, e sugerir que essa forma de governança baseada em diretrizes éticas principialista não é suficiente para normatizar a indústria da IA e seus desenvolvedores. Acreditamos que drásticas mudanças sejam necessárias, tanto nos processos de formação de profissionais das áreas ligadas ao desenvolvimento de software e sistemas inteligentes quanto no aumento da regulamentação desses profissionais e sua indústria. Para tanto, sugerimos que o Direito se beneficie das contribuições recentes da Bioética, de forma a explicitar em termos legais as contribuições da Ética da IA para a governança. (shrink)

Recent decades have witnessed tremendous progress in artificial intelligence and in the development of autonomous systems that rely on artificial intelligence. Critics, however, have pointed to the difficulty of allocating responsibility for the actions of an autonomous system, especially when the autonomous system causes harm or damage. The highly autonomous behavior of such systems, for which neither the programmer, the manufacturer, nor the operator seems to be responsible, has been suspected to generate responsibility gaps. This has been the cause of (...) much concern. In this article, I propose a more optimistic view on artificial intelligence, raising two challenges for responsibility gap pessimists. First, proponents of responsibility gaps must say more about when responsibility gaps occur. Once we accept a difficult-to-reject plausibility constraint on the emergence of such gaps, it becomes apparent that the situations in which responsibility gaps occur are unclear. Second, assuming that responsibility gaps occur, more must be said about why we should be concerned about such gaps in the first place. I proceed by defusing what I take to be the two most important concerns about responsibility gaps, one relating to the consequences of responsibility gaps and the other relating to violations of jus in bello. (shrink)

The application of modern technologies with artificial intelligence (AI) in all spheres of human life is growing exponentially alongside concern for its controllability. The lack of public, state, and international control over AI technologies creates large-scale risks of using such software and hardware that (un)intentionally harm humanity. The events of recent month and years, specifically regarding the Russian Federation’s war against its democratic neighbour Ukraine and other international conflicts of note, support the thesis that the uncontrolled use of AI, especially (...) in the military sphere, may lead to deliberate disregard for the moral standards of controlled AI or the spontaneous emergence of aggressive autonomous AI. The development of legal regulation for the use of technologies with AI is prolonged concerning the rapid development of these artefacts, which simultaneously cover all areas of public relations. Therefore, control over the creation and use of AI should be carried out not only by purely technical regulation (e.g., technical standards and conformance assessments, corporate and developer regulations, requirements enforced through industry-wide ethical codes); but also by comprehensive legislation and intergovernmental oversight bodies that codify and enforce specific changes in the rights and duties of legal persons. This article shall present the “Morality Problem” and “Intentionality Problem” of AI, and reflect upon various lacunae that arise when implementing AI for military purposes. (shrink)

Please contact me at [email protected] if you are interested in reading a particular chapter or being sent the entire manuscript for private use. -/- The thesis offers a comprehensive argument in favor of a regulationist approach to autonomous weapon systems (AWS). AWS, defined as all military robots capable of selecting or engaging targets without direct human involvement, are an emerging and potentially deeply transformative military technology subject to very substantial ethical controversy. AWS have both their enthusiasts and their detractors, prominently (...) advocating for a global preemptive ban on AWS development and use. Rejecting both positions, the author outlines a middle-of-the-road regulationist approach that is neither overly restrictive nor overly permissive. The disqualifying flaws of the rival prohibitionist approach are demonstrated in the process. After defining the core term of autonomy in weapon systems, the practical difficulties involved in applying an arms control regime to AWS are analyzed. The analysis shows that AWS are an extremely regulation-resistant technology. This feature when combined with their assumed high military utility makes a ban framework extremely costly to impose and enforce. As such it is ultimately very likely to fail at the benefit of the most unscrupulous international actors and at a very substantial risk to those abiding with international law. Consequently, to be ethically viable, a prohibitionist framework would need to offer substantial moral benefits impossible to attain through the rival regulationist approach. The remainder of the thesis undertakes to demonstrate that this is not the case. Comparing the considerations of military and strategic necessity to humanitarian concerns most commonly voiced by prohibitionists requires finding a common denominator for all values being referred to. Consequently, the thesis proceeds to show that both kinds of concerns are ultimately reducible to respect for basic human rights of all stakeholders, and so that the prohibitionist and regulationist approach may ultimately be compared in terms of consequences their adoption would generate for basic human rights realization. The author then evaluates both the potential humanitarian benefits, and the potential humanitarian hazards of AWS introduction. The benefits of leaving frontline combat to machines are outlined, with the unique kinds of suffering that would be abolished by such a development being described in detail. The arguments against AWS adoption are then divided into three classes: arguments related to alleged impossibility of compliance with The Laws of Armed Conflict, non-consequentialist and broad consequentialist arguments. This analysis, which comprises the greater part of the entire thesis, shows that the concerns behind compliance arguments are indeed substantial and have to be accommodated via a complex framework of best practices, regulations and localized restrictions on some kinds of AWS or AWS use in particular environments. They do not, however, justify a universal ban on using all the diverse forms of AWS in all environments. Non-consequentialist objections are found either reducible to other classes of arguments or thoroughly unconvincing, sometimes to the point of being actually vacuous. Broad consequentialist concerns are likewise found to be accommodable by regulation, empirically unfounded or causally disconnected from the actions of legitimate actors acquiring AWS, and therefore irrelevant to the moral permissibility of such actions. The author concludes that the proponents of prohibitionism are unable to point to moral benefits substantial enough to justify the costs and risks inherent in the approach. A global ban is, in fact, likely to have a worse humanitarian impact that well-regulated AWS adoption even if the strategic risks are disregarded. On the other hand, the analysis shows that there indeed exists an urgent need to regulate AWS through a variety of technological, procedural and legal solutions. These include, but are not limited to, a temporary moratorium on anti-personnel AWS use, development of internationally verified compliance software and eventual legal requirement of its employment, a temporary moratorium on AWS proliferation to state actors and a ban on their proliferation to non-state agents. (shrink)

The 2016-17 Iraqi offensive that recaptured the city of Mosul from the Islamic State have demonstrated the inability of contemporary armed forces to retake urban areas from a determined and ruthless enemy without either suffering debilitating casualties or causing thousands of civilian deaths and virtually destroying the city itself. The enemy’s willingness to refuse civilian evacuation via a humanitarian corridor and effectively take the inhabitants hostage is all it takes to impose this tragic dilemma on an attacking force. The civilian (...) death toll of the battle of Mosul alone is estimated to have reached between seven and forty thousand, with almost one million people internally displaced and large portions of the city levelled. Avoiding another such tragedy should be a priority for all practitioners and theorists of urban warfare. Indeed, confronting and defeating this challenge may be the most important task for ethics, theory and practice of urban combat in the coming decade. I argue that the only viable way of escaping the hostage siege dilemma consists of incorporating large numbers of unmanned and autonomous weapon systems into the blue force structure and using these to substitute for artillery and aerial fires presently being used to overpower defenders. While other measures may be used to marginally improve the outcome of hostage city assaults, only the automatization of the assault can dramatically change the calculus and in result either discourage rogue actors from creating hostage city dilemmas or defeat them decisively while suffering unequivocally proportional casualties. I proceed as follows: first, I provide an account of the 2016-17 Coalition assault on Mosul, giving special attention to the events and features relevant to the hostage city dilemma. I then use this case study to introduce the theoretical concept of the hostile city and the ethical dilemma it creates, discussing possible objections and making the case that the circumstances of the Battle of Mosul are bound to repeatedly transpire in the future. This generates a duty to find a way out of the hostage city dilemma. I argue that widespread use of unmanned and autonomous weapon platforms constitutes a viable way of assaulting hostage cities in a manner that assures respect for the ius in bello principles, proposing several ways in which such weapons may be designed and used to drastically limit both military and civilian casualties. Concluding that it may be not only permissible but morally required to employ autonomous weapons in the hostage city scenario, I briefly discuss the implications of this conclusion for a wider debate concerning the moral feasibility of autonomous weapons. (shrink)

Autonomous weapons systems, often referred to as ‘killer robots’, have been a hallmark of popular imagination for decades. However, with the inexorable advance of artificial intelligence systems (AI) and robotics, killer robots are quickly becoming a reality. These lethal technologies can learn, adapt, and potentially make life and death decisions on the battlefield with little-to-no human involvement. This naturally leads to not only legal but ethical concerns as to whether we can meaningful control such machines, and if so, then how. (...) Such concerns are made even more poignant by the ever-present fear that something may go wrong, and the machine may carry out some action(s) violating the ethics or laws of war. -/- Researchers, policymakers, and designers are caught in the quagmire of how to approach these highly controversial systems and to figure out what exactly it means to have meaningful human control over them, if at all. -/- In Designed for Death, Dr Steven Umbrello aims to not only produce a realistic but also an optimistic guide for how, with human values in mind, we can begin to design killer robots. Drawing on the value sensitive design (VSD) approach to technology innovation, Umbrello argues that context is king and that a middle path for designing killer robots is possible if we consider both ethics and design as fundamentally linked. Umbrello moves beyond the binary debates of whether or not to prohibit killer robots and instead offers a more nuanced perspective of which types of killer robots may be both legally and ethically acceptable, when they would be acceptable, and how to design for them. (shrink)

Efforts to ban Autonomous Weapon Systems were both unsuccessful and controversial. Simultaneously the need to address the detrimental aspects of AWS development and proliferation continues to grow in scope and urgency. The article presents several regulatory solutions capable of addressing the issue while simultaneously respecting the requirements of military necessity and so attracting a broad consensus. Two much stricter solutions – regional AWS bans and adoption of a no first use policy – are also presented as fallback strategies in case (...) achieving AWS’ compliance with the Laws of Armed Conflict proved elusive. Together, the solutions presented form an outline of a flexible regulatory strategy able to adjust to different technological outcomes and providing a sensible compromise to solve the current deadlock on the AWS issue. (shrink)

The exponential expansion and advancement of wartime technology has the potential to wipe out ‘war’ as a meaningful category. Assuming that the creation of new wartime technologies continues to accelerate, it could soon be the case that there will no longer be wars, but rather mass killings, slaughters, or genocides. This is because the concept of ‘war’ entails that opposing sides either will, or are able to, fight back against one another to some recognizable degree. In fact, this is one (...) of the differences between war and wholesale killing, slaughter, or genocide. With the asymmetric proliferation of killing and maiming wartime technologies, there may soon no longer be even the possibility of a fair, or somewhat fair, fight; there will only be scorched earth. (shrink)

This chapter considers how the adoption of autonomous weapons systems (AWS) may affect jus ad bellum principles of warfare. In particular, it focuses on the use of AWS in non-international armed conflicts (NIAC). Given the proliferation of NIAC, the development and use of AWS will most likely be attuned to this specific theater of war. As warfare waged by modernized liberal democracies (those most likely to develop and employ AWS at present) increasingly moves toward a model of individualized warfare, how, (...) if at all, will the principles by which we measure the justness of the commencement of such hostilities be affected by the introduction of AWS, and how will such hostilities stack up to current legal agreements surrounding their more traditional engagement? This chapter claims that such considerations give us reason to question the moral and legal necessity of ad bellum proper authority. (shrink)

Künstliche Intelligenz (KI) ist heute schon ein fester Bestandteil unseres Lebens, auch wenn sie oft im Verborgenen wirkt. Wo wird diese Entwicklung hinführen und was wird das für uns bedeuten? Jens Kipper erklärt, wie moderne KI funktioniert, was sie heute schon kann und welche Auswirkungen ihre Verwendung in Waffensystemen, in der Medizin und Wissenschaft, im Arbeitsleben und anderswo haben wird. Kipper argumentiert dafür, dass die Entwicklung von KI zu großen gesellschaftlichen Umwälzungen führen wird. Er erläutert zudem, wovon es abhängt, dass (...) diese Umwälzungen erfreulicher Natur sein werden und was wir tun können, um die durch den Einsatz von KI geprägten Veränderungen in die richtige Richtung zu lenken. (shrink)

Our reflections on LAWS issues are the result of the work of our research group on AI and ethics at the Informatics Center in partnership with the Information Science Department, both from the Federal University of Pernambuco, Brazil. In particular, our propositions and provocations are tied to Bianca Ximenes’s ongoing doctoral thesis, advised by Prof. Geber Ramalho, from the area of computer science, and co-advised by Prof. Diego Salcedo, from the humanities. Our research group is interested in answering two tricky (...) questions: What would an ethical AI be? And how can one guarantee that a given intelligent system will follow intercultural human ethical principles? (shrink)

Arguments from human dignity feature prominently in the Lethal Autonomous Weapons moral feasibility debate, even though their exists considerable controversy over their role and soundness and the notion of dignity remains under-defined. Drawing on the work of Dieter Birnbacher, I fix the sub-discourse as referring to the essential value of human persons in general, and to postulated moral rights of combatants not covered within the existing paradigm of the International Humanitarian Law in particular. I then review and critique dignity-based arguments (...) against LAWS: argument from faulty targeting process, argument from objectification, argument from underappreciation of the value of human life and the argument from the absence of mercy. I conclude that the argument from the absence of mercy is the only dignity-based argument that is both valid and irreducible to another class of arguments within the debate, and that it offers insufficient justification for a global ban on LAWS. (shrink)

Once the stuff of science fiction, recent progress in artificial intelligence, robotics, and machine learning means that these rapidly advancing technologies are finally coming into widespread use within everyday life. Such rapid development in these areas also brings with it a host of social, political and legal issues, as well as a rise in public concern and academic interest in the ethical challenges these new technologies pose. This volume is a collection of scholarly work from leading figures in the development (...) of both robot ethics and machine ethics; it includes essays of historical significance which have become foundational for research in these two new areas of study, as well as important recent articles. The research articles selected focus on the control and governance of computational systems; the exploration of ethical and moral theories using software and robots as laboratories or simulations; inquiry into the necessary requirements for moral agency and the basis and boundaries of rights; and questions of how best to design systems that are both useful and morally sound. Collectively the articles ask what the practical ethical and legal issues, arising from the development of robots, will be over the next twenty years and how best to address these future considerations. (shrink)

A review and commentary on Killing By Remote Control: the Ethics of an Unmanned Military, edited by Bradley Jay Strawser (forward by Jeff McMahan), (Oxford University Press). -/- .

Current precursors in the development of lethal, autonomous systems (LAS) point to the use of biometric devices for assessing, identifying, and verifying targets. The inclusion of biometric devices entails the use of a probabilistic matching program that requires the deliberate targeting of noncombatants as a statistically necessary function of the system. While the tactical employment of the LAS may be justified on the grounds that the deliberate killing of a smaller number of noncombatants is better than the accidental killing of (...) a larger number, it may nonetheless contravene a reemerging conception of right intention. Originally framed by Augustine of Hippo, this lesser-known formulation has served as the foundation for chivalric code, canon law, jus in bello criteria, and the law of armed conflict. Thus it serves as a viable measure to determine whether the use of lethal autonomy would accord with these other laws and principles. Specifically, examinations of the LAS through the lenses of collateral damage, the principle of double effect, and the principle of proportionality, reveal the need for more attention to be paid to the moral issues now, so that the promise of this emerging technology—that it will perform better than human beings—might actually come to pass. (shrink)

In the last decade we have entered the era of remote controlled military technology. The excitement about this new technology should not mask the ethical questions that it raises. A fundamental ethical question is who may be held responsible for civilian deaths. In this paper we will discuss the role of the human operator or so-called ‘cubicle warrior’, who remotely controls the military robots behind visual interfaces. We will argue that the socio-technical system conditions the cubicle warrior to dehumanize the (...) enemy. As a result the cubicle warrior is morally disengaged from his destructive and lethal actions. This challenges what he should know to make responsible decisions (the so-called knowledge condition). Nowadays and in the near future, three factors will influence and may increase the moral disengagement even further due to the decrease of locus of control orientation: (1) photo shopping the war; (2) the moralization of technology; (3) the speed of decision-making. As a result, cubicle warriors cannot be held reasonably responsible anymore for the decisions they make. (shrink)