"A runaway trolley is speeding down a track" So begins what is perhaps the most fecund thought experiment of the past several decades since its invention by Philippa Foot. Since then, moral philosophers have applied the "trolley problem" as a thought experiment to study many different ethical conflicts - and chief among them is the programming of autonomous vehicles. Nowadays, however, very few philosophers accept that the trolley problem is a perfect analogy for driverless cars or that the situations (...) autonomous vehicles face will resemble the forced choice of the unlucky bystander in the original thought experiment. This book represents a substantial and purposeful effort to move the academic discussion beyond the trolley problem to the broader ethical, legal, and social implications that autonomous vehicles present. There are still urgent questions waiting to be addressed, for example: how AVs might interact with human drivers in mixed or "hybrid" traffic environments; how AVs might reshape our urban landscapes; what unique security or privacy concerns are raised by AVs as connected devices in the "Internet of Things"; how the benefits and burdens of this new technology, including mobility, traffic congestion, and pollution, will be distributed throughout society; and more. An attempt to map the landscape of these next-generation questions and to suggest preliminary answers, this volume draws on the disciplines of philosophy, sociology, economics, urban planning and transportation engineering, business ethics and more, and represents a global range of perspectives. (shrink)

While autonomous vehicles (AVs) are not designed to harm people, harming people is an inevitable by-product of their operation. How are AVs to deal ethically with situations where harming people is inevitable? Rather than focus on the much-discussed question of what choices AVs should make, we can also ask the much less discussed question of who gets to decide what AVs should do in such cases. Here there are two key options: AVs with a personal ethics setting (PES) (...) or an “ethical knob” that end users can control or AVs with a mandatory ethics setting (MES) that end users cannot control. Which option, a PES or an MES, is best and why? This chapter argues, by drawing on the choice architecture literature, in favor of a hybrid view that requires mandated default choice settings while allowing for limited end user control. (shrink)

In this chapter, I argue that in addition to the generally accepted aim of reducing traffic-related injuries and deaths as much as possible, a principle of fairness in the distribution of risk should inform our thinking about how firms that produce autonomous vehicles ought to program them to respond in conflict situations involving human-driven vehicles. This principle, I claim, rules out programming autonomous vehicles to systematically prioritize the interests of their occupants over those of the occupants of other (...) vehicles, including human-driven vehicles. Because there is reason to think that most consumers would prefer to purchase autonomous vehicles that do systematically prioritize the interests of occupants over those of others, if my argument is correct it generates a substantial ethical restriction on firms’ efforts to gain market share in the initial stages of the introduction of autonomous vehicles onto the road. (shrink)

In this paper, we argue that if a set of plausible conditions obtain, then driving a standard vehicle rather than riding in an autonomous vehicle (AV) will become analogous to driving drunk rather than driving sober, and therefore impermissible. In addition, we argue that a ban on the production, sale, and purchase of new standard vehicles would also become justified. We make this case in part by highlighting that the central reasons typically offered in support of state-mandated (...) vaccination will also support mandating AV use. Finally, we discuss some of the implications of our argument for the obligations of vehicle-producing firms. (shrink)

Autonomous vehicles (AVs) are expected to improve road traffic safety and save human lives. It is also expected that some AVs will encounter so-called dilemmatic situations, like choosing between saving two passengers by sacrificing one pedestrian or choosing between saving three pedestrians by sacrificing one passenger. These expectations fuel the extensive debate over the ethics settings of AVs: the way AVs should be programmed to act in dilemmatic situations and who should decide about the nature of this programming (...) in the first place. In the article, the ethics settings problem is analyzed as a trilemma between AVs with personal ethics setting (PES), AVs with mandatory ethics setting (MES) and AVs with no ethics settings (NES). It is argued that both PES and MES, by being programmed to choose one human life over the other, are bound to cause serious moral damage resulting from the violation of several principles central to deontology and utilitarianism. NES is defended as the only plausible solution to this trilemma, that is, as the solution that sufficiently minimizes the number of traffic fatalities without causing any comparable moral damage. (shrink)

A number of companies including Google and BMW are currently working on the development of autonomous cars. But if fully autonomous cars are going to drive on our roads, it must be decided who is to be held responsible in case of accidents. This involves not only legal questions, but also moral ones. The first question discussed is whether we should try to design the tort liability for car manufacturers in a way that will help along the development (...) and improvement of autonomous vehicles. In particular, Patrick Lin’s concern that any security gain derived from the introduction of autonomous cars would constitute a trade-off in human lives will be addressed. The second question is whether it would be morally permissible to impose liability on the user based on a duty to pay attention to the road and traffic and to intervene when necessary to avoid accidents. Doubts about the moral legitimacy of such a scheme are based on the notion that it is a form of defamation if a person is held to blame for causing the death of another by his inattention if he never had a real chance to intervene. Therefore, the legitimacy of such an approach would depend on the user having an actual chance to do so. The last option discussed in this paper is a system in which a person using an autonomous vehicle has no duty of interfering, but is still held responsible for possible accidents. Two ways of doing so are discussed, but only one is judged morally feasible. (shrink)

Purpose Fully autonomous self-driving cars not only hold the potential for significant economic and environmental advantages but also introduce complex ethical dilemmas. One of the highly debated issues, known as the “trolley problems,” revolves around determining the appropriate actions for a self-driving car when faced with an unavoidable crash. Currently, the discourse on autonomous vehicle (AV) crash algorithms is primarily shaped by Western ethical traditions, resulting in a Eurocentric bias due to the dominant economic and political influence (...) of the West. However, considering that AV technology will be deployed across diverse cultural and religious contexts, this paper aims to contribute to the discourse by providing an Islamic perspective on programming the response of AVs in the event of an imminent crash. Design/methodology/approach This study proposes a novel methodology based on the Islamic concept of maṣlaḥa for the normative assessment of ethical decisions related to AV programming. Findings Drawing upon the works of classic Islamic jurists, this study highlights two distinct normative visions within Islamic traditions (akin to deontology and consequentialism) concerning the preservation of human lives in the context of AVs. This study explores the shared and divergent elements between Islamic and Western ethical approaches proposed for AVs. Originality/value This pioneering work examines AV crash algorithms from an Islamic perspective, filling a void in the global ethical discourse. This work will also serve an important role to bridge the gap between the theoretical Islamic ethical principles and their practical application in the realm of AVs. (shrink)

The ethics of autonomous vehicles has received a great amount of attention in recent years, specifically in regard to their decisional policies in accident situations in which human harm is a likely consequence. Starting from the assumption that human harm is unavoidable, many authors have developed differing accounts of what morality requires in these situations. In this article, a strategy for AV decision-making is proposed, the Ethical Valence Theory, which paints AV decision-making as a type of claim mitigation: (...) different road users hold different moral claims on the vehicle’s behavior, and the vehicle must mitigate these claims as it makes decisions about its environment. Using the context of autonomous vehicles, the harm produced by an action and the uncertainties connected to it are quantified and accounted for through deliberation, resulting in an ethical implementation coherent with reality. The goal of this approach is not to define how moral theory requires vehicles to behave, but rather to provide a computational approach that is flexible enough to accommodate a number of ‘moral positions’ concerning what morality demands and what road users may expect, offering an evaluation tool for the social acceptability of an autonomous vehicle’s ethical decision making. (shrink)

There is a growing body of scholarship on the ethics of autonomous vehicles. Yet the ethical discourse has mostly been focusing on the behavior of the vehicle in accident scenarios. This paper offers a different ethical prism: the implications of the autonomous vehicle for human well-being. As such, it contributes to the growing discourse on the wider societal and moral implications of the autonomous vehicle. The paper is premised on the neo-Aristotelian approach which (...) holds that as human beings, our well-being depends on developing and exercising our innate human capacities: to know, understand, love, be sociable, imagine, create and use our bodies and use our willpower. To develop and exercise these capacities, our environments need to provide a range of opportunities which will trigger the development and exercise of the capacities. The main argument advanced in the paper is that one plausible future of the autonomous vehicle—a future of single-rider autonomous vehicles—may effectively reduce the opportunities to develop and exercise our capacities to know, be sociable and use our willpower. It will therefore be bad for human well-being, and this provides us with a moral reason to resist this plausible future and search for alternative ones. (shrink)

Autonomous vehicles have the potential to save tens of thousands of lives, but legal and social barriers may delay or even deter manufacturers from offering fully automated vehicles and thereby cost lives that otherwise could be saved. Moral philosophers use “thought experiments” to teach us about what ethics might say about the ethical behavior of AVs. If a manufacturer designing an AV decided to make what it believes is an ethical choice to save a large group of lives (...) by steering away from the large group towards a single individual, could the manufacturer be liable to that single individual? I believe the answer is yes. This article discusses famous “trolley problem” thought experiments in moral philosophy, which news articles have recently applied to AVs. After establishing a hypothetical case, applying product liability law to AV behavior may show a way, in some jurisdictions, that will permit a manufacturer to implement an ethical choice without triggering liability, but liability risk remains. I then turn to possible traditional defenses to see if they prevent liability for a manufacturer making what it believes is an ethical design choice—necessity, defense of self, and the sudden emergency doctrine. None of them provides an effective defense. Under current law, maximizing collision avoidance appears to minimize legal risk. I conclude that the only way to provide effective protection to a manufacturer seeking to implement ethical choices about AV behavior is through special legislation or regulation. (shrink)

Through modern driver assistant systems, algorithmic decisions already have a significant impact on the behavior of vehicles in everyday traffic. This will become even more prominent in the near future considering the development of autonomous driving functionality. The need to consider ethical principles in the design of such systems is generally acknowledged. However, scope, principles and strategies for their implementations are not yet clear. Most of the current discussions concentrate on situations of unavoidable crashes in which the life of (...) human beings is existentially affected. In this paper, we argue that ethical considerations should be mandatory for any algorithmic decision of autonomous vehicles, instead of a limitation to hazard situations. Such an ethically aligned behavior is relevant because autonomous vehicles, like any other traffic participants, operate in a shared public space, where every behavioral decision impacts the operational possibilities of others. These possibilities concern the fulfillment of a road-user’s safety, utility and comfort needs. We propose that, to operate ethically in such space, an autonomous vehicle will have to take its behavior decisions according to a just distribution of operational possibilities among all traffic participants. Using an application on a partially-autonomous prototype vehicle, we describe how to apply and implement concepts of distributive justice to the driving environment and demonstrate the impact on its behavior in comparison to an advanced but egoistic decision maker. (shrink)

Trolley cases are widely considered central to the ethics of autonomous vehicles. We caution against this by identifying four problems. Trolley cases, given technical limitations, rest on assumptions that are in tension with one another. Furthermore, trolley cases illuminate only a limited range of ethical issues insofar as they cohere with a certain design framework. Furthermore, trolley cases seem to demand a moral answer when a political answer is called for. Finally, trolley cases might be epistemically problematic in (...) several ways. To put forward a positive proposal, we illustrate how ethical challenges arise from mundane driving situations. We argue that mundane situations are relevant because of the specificity they require and the scale they exhibit. We then illustrate some of the ethical challenges arising from optimizing for safety, balancing safety with other values such as mobility, and adjusting to incentives of legal frameworks. (shrink)

How should autonomous vehicles be programmed to behave in the event of an unavoidable accident in which the only choice open is one between causing different damages or losses to different objects or persons? This paper addresses this ethical question starting from the normative principles elaborated in the law to regulate difficult choices in other emergency scenarios. In particular, the paper offers a rational reconstruction of some major principles and norms embedded in the Anglo-American jurisprudence and case law on (...) the “doctrine of necessity”; and assesses which, if any, of these principles and norms can be utilized to find reasonable guidelines for solving the ethical issue of the regulation of the programming of autonomous vehicles in emergency situations. The paper covers the following topics: the distinction between “justification” and “excuse”, the legal prohibition of intentional killing outside self-defence, the incommensurability of goods, and the legal constrains to the use of lethal force set by normative positions: obligations, responsibility, rights, and authority. For each of these principles and constrains the possible application to the programming of autonomous vehicles is discussed. Based on the analysis, some practical suggestions are offered. (shrink)

Autonomous vehicles must be programmed with procedures for dealing with trolley-style dilemmas where actions result in harm to either pedestrians or passengers. This paper outlines a Rawlsian algorithm as an alternative to the Utilitarian solution. The algorithm will gather the vehicle’s estimation of probability of survival for each person in each action, then calculate which action a self-interested person would agree to if he or she were in an original bargaining position of fairness. I will employ Rawls’ assumption (...) that the Maximin procedure is what self-interested agents would use from an original position, and then show how the Maximin procedure can be operationalized to produce unique outputs over probabilities of survival. (shrink)

The philosophical–ethical literature and the public debate on autonomous vehicles have been obsessed with ethical issues related to crashing. In this article, these discussions, including more empirical investigations, will be critically assessed. It is argued that a related and more pressing issue is questions concerning safety. For example, what should we require from autonomous vehicles when it comes to safety? What do we mean by ‘safety’? How do we measure it? In response to these questions, the article will (...) present a foundation for a continued discussion on these issues and an argument for why discussions about safety should be prioritized over ethical concerns related to crashing. (shrink)

As vehicles become more autonomous, the task of designing guiding systems that make morally acceptable decisions is getting more urgent. It is sometimes assumed that one solution will be acceptable across various cultures. In this paper we argue for the importance of intercultural perspectives; in particular, we explore possible insights derived from Buddhist philosophy, taking avail of the virtue of compassion (karuṇā). We suggest that autonomous vehicles should first learn in supervised situations so that they reach a level (...) of decision-making ability that human experts within a given culture deem morally good. How will they be evaluated? In some debates it is assumed that there is a one-fit-all answer to be found based on an agreed-upon moral theory. Apart from problems of ethical pluralism within a given culture, there are also problems due to different moral preferences and driving styles across cultures. One way the problem might be addressed is to introduce a machine-learning training curriculum that would provide autonomous vehicles with a process comparable to a novice human driver learning to drive and passing her own driving license test. (shrink)

The ethical dilemma of whether autonomous vehicles should protect the passengers or pedestrians when harm is unavoidable has been widely researched and debated. Several behavioral scientists have sought public opinion on this issue, based on the premise that EDs are critical to resolve for AV adoption. However, many scholars and industry participants have downplayed the importance of these edge cases. Policy makers also advocate a focus on higher level ethical principles rather than on a specific solution to EDs. But (...) conspicuously absent from this debate is the view of the consumers or potential adopters, who will be instrumental to the success of AVs. The current research investigated this issue both from a theoretical standpoint and through empirical research. The literature on innovation adoption and risk perception suggests that EDs will be heavily weighted by potential adopters of AVs. Two studies conducted with a broad sample of consumers verified this assertion. The results from these studies showed that people associated EDs with the highest risk and considered EDs as the most important issue to address as compared to the other technical, legal and ethical issues facing AVs. As such, EDs need to be addressed to ensure robustness in the design of AVs and to assure consumers of the safety of this promising technology. Some preliminary evidence is provided about interventions to resolve the social dilemma in EDs and about the ethical preferences of prospective early adopters of AVs. (shrink)

The imminent deployment of autonomous vehicles requires algorithms capable of making moral decisions in relevant traffic situations. Some scholars in the ethics of autonomous vehicles hope to align such intelligent systems with human moral judgment. For this purpose, studies like the Moral Machine Experiment have collected data about human decision-making in trolley-like traffic dilemmas. This paper first argues that the trolley dilemma is an inadequate experimental paradigm for investigating traffic moral judgments because it does not include agents’ (...) character-based considerations and is incapable of facilitating the investigation of low-stakes mundane traffic scenarios. In light of the limitations of the trolley paradigm, this paper presents an alternative experimental framework that addresses these issues. The proposed solution combines the creation of mundane traffic moral scenarios using virtual reality and the Agent-Deed-Consequences (ADC) model of moral judgment as a moral-psychological framework. This paradigm shift potentially increases the ecological validity of future studies by providing more realism and incorporating character considerations into traffic actions. (shrink)

My starting point in this article is that investigating the ethics of autonomous vehicles through the lens of the trolley problem is not only limited but also unethical. I construct my case by aligning myself with Niklas Toivakainen, who argues against David Gunkel’s reading of Levinasian ethics as an answer to the “Machine Question”. I adumbrate Toivakainen’s critique that the attempt to give a Levinasian face to the machine is an example of a compensatory logic – a (...) way to avoid a deeper exploration of the moral dynamics from which our technologies emerge. I offer an extended reading of Levinas’s formulation of the face of the other to argue that while the machine cannot signify as a face, the machine can announce or anticipate the signification of the face which the self is infinitely responsible to. I reach this conclusion through focusing on the uncertainty and ambiguity of the signification of the face. I then argue that current approaches to the ethics of autonomous vehicles, based on variations of the trolley problem, fail to locate ethical responsibility to the other, precisely in its attempts to evade uncertainty. I demonstrate how an embrace of uncertainty can make the ethics of autonomous vehicles more ethical through a consideration of the concept of shared space and naked streets in the work of the late Dutch traffic engineer Hans Monderman. (shrink)

In 2017, the German ethics commission for automated and connected driving released 20 ethical guidelines for autonomous vehicles. It is now up to the research and industrial sectors to enhance the development of autonomous vehicles based on such guidelines. In the current state of the art, we find studies on how ethical theories can be integrated. To the best of the authors’ knowledge, no framework for motion planning has yet been published which allows for the true implementation (...) of any practical ethical policies. This paper makes four contributions: Firstly, we briefly present the state of the art based on recent works concerning unavoidable accidents of autonomous vehicles (AVs) and identify further need for research. While most of the research focuses on decision strategies in moral dilemmas or crash optimization, we aim to develop an ethical trajectory planning for all situations on public roads. Secondly, we discuss several ethical theories and argue for the adoption of the theory “ethics of risk.” Thirdly, we propose a new framework for trajectory planning, with uncertainties and an assessment of risks. In this framework, we transform ethical specifications into mathematical equations and thus create the basis for the programming of an ethical trajectory. We present a risk cost function for trajectory planning that considers minimization of the overall risk, priority for the worst-off and equal treatment of people. Finally, we build a connection between the widely discussed trolley problem and our proposed framework. (shrink)

This article reviews existing state laws related to autonomous vehicle (AV) safety, equity, and automobile insurance. Thirty states were identified with relevant legislation. Of these, most states had one or two relevant laws in place. Many of these laws were related to safety and insurance requirements. Data are needed to evaluate the effectiveness of these laws in order to guide further policy development.

In 2021, Germany passed the first law worldwide that regulates dilemma situations with autonomous cars. Against this background, this article investigates the permissibility of trade-offs between human lives in the context of self-driving cars. It does so by drawing on the debate about the traditional trolley problem. In contrast to most authors in the relevant literature, it argues that the debate about the trolley problem is both directly and indirectly relevant for the ethics of crashes with self-driving cars. (...) Drawing on its direct normative relevance, the article shows that trade-offs are permissible in situations with self-driving cars that are similar to paradigmatic trolley cases. In scenarios that are unlike paradigmatic trolley cases, the debate about the trolley problem can have indirect normative relevance because it provides reasons against the use of moral theories and principles that cannot account for the trolley problem. (shrink)

One of the primary, if not most critical, difficulties in the design and implementation of autonomous systems is the black-boxed nature of the decision-making structures and logical pathways. How human values are embodied and actualised in situ may ultimately prove to be harmful if not outright recalcitrant. For this reason, the values of stakeholders become of particular significance given the risks posed by opaque structures of intelligent agents (IAs). This paper explores how decision matrix algorithms, via the belief-desire-intention model (...) for autonomous vehicles, can be designed to minimize the risks of opaque architectures. Primarily through an explicit orientation towards designing for the values of explainability and verifiability. In doing so, this research adopts the Value Sensitive Design (VSD) approach as a principled framework for the incorporation of such values within design. VSD is recognized as a potential starting point that offers a systematic way for engineering teams to formally incorporate existing technical solutions within ethical design, while simultaneously remaining pliable to emerging issues and needs. It is concluded that the VSD methodology offers at least a strong enough foundation from which designers can begin to anticipate design needs and formulate salient design flows that can be adapted to the changing ethical landscapes required for utilisation in autonomous vehicles. (shrink)

Autonomous vehicles —and accidents they are involved in—attest to the urgent need to consider the ethics of artificial intelligence. The question dominating the discussion so far has been whether we want AVs to behave in a ‘selfish’ or utilitarian manner. Rather than considering modeling self-driving cars on a single moral system like utilitarianism, one possible way to approach programming for AI would be to reflect recent work in neuroethics. The agent–deed–consequence model :3–20, 2014a, Behav Brain Sci 37:487–488, 2014b) (...) provides a promising descriptive and normative account while also lending itself well to implementation in AI. The ADC model explains moral judgments by breaking them down into positive or negative intuitive evaluations of the agent, deed, and consequence in any given situation. These intuitive evaluations combine to produce a positive or negative judgment of moral acceptability. For example, the overall judgment of moral acceptability in a situation in which someone committed a deed that is judged as negative would be mitigated if the agent had good intentions and the action had a good consequence. This explains the considerable flexibility and stability of human moral judgment that has yet to be replicated in AI. This paper examines the advantages and disadvantages of implementing the ADC model and how the model could inform future work on ethics of AI in general. (shrink)

This article takes a speculative and design fiction approach to the critical analysis of the role of smart and autonomous vehicles (AVs) in the context of smart cities. The article explores arguments that these cars of the future will have decisive impacts on mobility, sustainability and road safety. The article examines the main parameters of smart city and smart car developments and then focuses on the visions of increasing AI-driven autonomy. The article demonstrates how these debates are linked to (...) speculative design as full autonomy does not currently exist but takes a speculative position as to what the critical issues are that face smart/autonomous city visions (enhanced surveillance and data mining) and considers the potentially hazardous ethical dilemmas that AVs may encounter once fully rolled out onto city roads. From a design fiction perspective, the article envisions the viability of AV prioritizing public rather than private transport as a means by which the ‘techno-utopic’ visions of smart city/AV integration can be realized and make positive impacts to enhance urban living in rendering future cities as more sustainable, efficiently mobile and safer urban spaces. (shrink)

The trolley problem is a classic thought experiment that evokes an ethical dilemma. Thomson’s “bystander” and “footbridge” versions of the trolley problem induce different intuitive judgments about what to choose in the ethical dilemma. However, we can question how robust these intuitive judgments are. We thus conducted an online survey experiment of Thomson’s versions of the trolley problem which showed that more respondents tended to choose not pulling the lever in the bystander version and pushing a person off the bridge (...) in the footbridge version when they believed they were not being watched. These results illustrate that people’s decisions deviate from the social norms in the absence of the public’s attention. To activate the practice of holding the manufacturers to a normative expectation based on the social norms, we suggest the law must be changed. (shrink)

This paper proposes that autonomous vehicles should be designed to reduce light pollution. In support of this specific proposal, a moral assessment of autonomous vehicles more comprehensive than the dilemmatic life-and-death questions of trolley problem-style situations is presented. The paper therefore consists of two interrelated arguments. The first is that autonomous vehicles are currently still a technology in development, and not one that has acquired its definitive shape, meaning the design of both the vehicles and the surrounding (...) infrastructure is open-ended. Design for values is utilized to articulate a path forward, by which engineering ethics should strive to incorporate values into a technology during its development phase. Second, it is argued that nighttime lighting—a critical supporting infrastructure—should be a prima facie consideration for autonomous vehicles during their development phase. It is shown that a reduction in light pollution, and more boldly a better balance of lighting and darkness, can be achieved via the design of future autonomous vehicles. Two case studies are examined through which autonomous vehicles may be designed for “driving in the dark.” Nighttime lighting issues are thus inserted into a broader ethics of autonomous vehicles, while simultaneously introducing questions of autonomous vehicles into debates about light pollution. (shrink)

With their prospect for causing both novel and known forms of damage, harm and injury, the issue of responsibility has been a recurring theme in the debate concerning autonomous vehicles. Yet, the discussion of responsibility has obscured the finer details both between the underlying concepts of responsibility, and their application to the interaction between human beings and artificial decision-making entities. By developing meaningful distinctions and examining their ramifications, this article contributes to this debate by refining the underlying concepts that (...) together inform the idea of responsibility. Two different approaches are offered to the question of responsibility and autonomous vehicles: targeting and risk distribution. The article then introduces a thought experiment which situates autonomous vehicles within the context of crash optimisation impulses and coordinated or networked decision-making. It argues that guiding ethical frameworks overlook compound or aggregated effects which may arise, and which can lead to subtle forms of structural discrimination. Insofar as such effects remain unrecognised by the legal systems relied upon to remedy them, the potential for societal inequalities is increased and entrenched, situations of injustice and impunity may be unwittingly maintained. This second set of concerns may represent a hitherto overlooked type of responsibility gap arising from inadequate accountability processes capable of challenging systemic risk displacement. (shrink)

The control principle implies that people should not feel guilt for outcomes beyond their control. Yet, the so-called ‘agent and observer puzzles’ in philosophy demonstrate that people waver in their commitment to the control principle when reflecting on accidental outcomes. In the context of car accidents involving conventional or autonomous vehicles, Study 1 established that judgments of responsibility are most strongly associated with expressions of guilt–over and above other negative emotions, such as sadness, remorse or anger. Studies 2 and (...) 3 then confirmed that, while people generally endorse the control principle, and deny that occupants in an AV should feel guilt when involved in an accident, they nevertheless ascribe guilt to those same occupants. Study 3 also uncovered novel implications of the observer puzzle in the legal context: Passengers in an AV were seen as more legally liable than either passengers in a conventional vehicle, or even their drivers–especially when participants were prompted to reflect on the passengers’ affective experience of guilt. Our findings document an important conflict–in the context of AV accidents–between people’s prescriptive reasoning about responsibility and guilt on one hand, and their counter-normative experience of guilt on the other, with apparent implications for liability decisions. (shrink)

Automated vehicles promise much in the way of both economic boons and increased personal safety. For better or worse, the effects of automating personal vehicles will not be felt for some time. In contrast, the effects of automated work vehicles, like semi-trucks, will be felt much sooner—within the next decade. The costs and benefits of automation will not be distributed evenly; while most of us will be positively affected by the lower prices overall, those losing their livelihoods to the automated (...) semi-trucks and other similar work vehicles will be much worse off. This sets up a classic distributive justice problem: how do we balance the harms and benefits of automation? In this paper, the authors recommend work alternatives: policies that the government would enact in order to ensure that each person could live a decent life even if one could not work, especially for reasons like having one’s job automated. After arguing for this position, we propose some policy guidelines. (shrink)

Autonomous vehicle moral dilemmas matter less for the particular outcomes of potential accidents than for their role in defining the values of the society we wish to live in. Different approaches have been suggested to determine the ethical settings that autonomous vehicles should be implemented in and identify the legitimate agents for making such decisions. Most of these, however, fail on theoretical grounds, facing severe issues related to moral justifications and compliance to the law, or on practical (...) grounds, being insufficiently universal, action-guiding, or technically viable to be implemented. The analogy with the “trolley problem” has been extensively discussed. However, researchers have rarely tried to adapt this framework to autonomous vehicle cases or investigate how it could be used to address these issues. In doing so, this paper aims to answer the two key problems of autonomous vehicle dilemmas. With regards to the decision-maker, it rejects autonomous vehicle users’ choice-based models, showing the absurdity of both switch of control and adaptative preferences and arguing for common legislator-determined ethical settings. With regards to the decisions themselves, it criticizes both utilitarian views and those based on individuals’ criteria to suggest a deontologist rights-based approach. This allows for the defence of a morally coherent, regulatory compliant, explainable, and easily implementable framework capable of addressing all autonomous vehicle moral dilemma scenarios present in the literature. (shrink)

In this article we summarise some previously described proposals for ethical governance of autonomous vehicles, critique them, and offer an alternative solution. Rather than programming cars to react to crash situations in the same way as humans, having humans program pre-set responses for a wide range of different potential scenarios, or applying particular ethical theories, we suggest that decisions should be made jointly between humans and cars. Given that humans lack the requisite processing capacity, and computers lack the necessary (...) ethical capacity, the medical paradigm of advance care planning can be retooled for this new context. Advance car-crash planning provides a way to combine humans’ ethical preferences with the advanced data processing capacities of computers to enable shared decision making in collision situations. (shrink)

Suppose a driverless car encounters a scenario where harm to at least one person is unavoidable and a choice about how to distribute harms between different persons is required. How should the driverless car be programmed to behave in this situation? I call this the moral design problem. Santoni de Sio defends a legal-philosophical approach to this problem, which aims to bring us to a consensus on the moral design problem despite our disagreements about which moral principles provide the correct (...) account of justified harm. He then articulates an answer to the moral design problem based on the legal doctrine of necessity. In this paper, I argue that Santoni de Sio’s answer to the moral design problem does not achieve the aim of the legal-philosophical approach. This is because his answer relies on moral principles which, at least, utilitarians have reason to reject. I then articulate an alternative reading of the doctrine of necessity, and construct a partial answer to the moral design problem based on this. I argue that utilitarians, contractualists and deontologists can agree on this partial answer, even if they disagree about which moral principles offer the correct account of justified harm. (shrink)

Self-driving vehicles will need to make decisions that carry ethical dimensions and manufacturers have (the responsibility) to pre-determine this underlying, deliberate decision-making process. With the rise of self-driving vehicles, scholars have simultaneously started investigating what ethical theories should guide machine behavior, but have not concluded as to which theory should be preferred and adopted. We aim to address this matter by providing a holistic and analytical review of the autonomous driving ethics literature. Based on this review, we summarize (...) the social, moral/legal, and functional advantages and disadvantages of applying particular (traditional) ethical theories to the decision-making of self-driving vehicles. Furthermore, we derive an integrated model that shows how each identified ethical theory could be technically implemented and guide the decision-making of self-driving vehicles. Overall, this article aims to set the groundwork for reflected, successful integration of ethical behavior within self-driving vehicles and deduce an updated research agenda. (shrink)

Accidents involving autonomous vehicles raise difficult ethical dilemmas and legal issues. It has been argued that self-driving cars should be programmed to kill, that is, they should be equipped with pre-programmed approaches to the choice of what lives to sacrifice when losses are inevitable. Here we shall explore a different approach, namely, giving the user/passenger the task of deciding what ethical approach should be taken by AVs in unavoidable accident scenarios. We thus assume that AVs are equipped with what (...) we call an “Ethical Knob”, a device enabling passengers to ethically customise their AVs, namely, to choose between different settings corresponding to different moral approaches or principles. Accordingly, AVs would be entrusted with implementing users’ ethical choices, while manufacturers/programmers would be tasked with enabling the user’s choice and ensuring implementation by the AV. (shrink)

Ethics settings allow for morally significant decisions made by humans to be programmed into autonomous machines, such as autonomous vehicles or autonomous weapons. Customizable ethics settings are a type of ethics setting in which the users of autonomous machines make such decisions. Here two arguments are provided in defence of customizable ethics settings. Firstly, by approaching ethics settings in the context of failure management, it is argued that customizable ethics settings (...) are instrumentally and inherently valuable for building resilience into the larger socio-technical systems in which autonomous machines operate. Secondly, after defining the preliminary condition of responsibility attribution and demonstrating how ethics settings enable humans to exert control over the outcomes of morally significant incidents, it is shown that ethics settings narrow the responsibility gap. (shrink)

The German Act on Autonomous Driving constitutes the first national framework on level four autonomous vehicles and has received attention from policy makers, AI ethics scholars and legal experts in autonomous driving. Owing to Germany’s role as a global hub for car manufacturing, the following paper sheds light on the act’s position within the ethical discourse and how it reconfigures the balance between legislation and ethical frameworks. Specifically, in this paper, we highlight areas that need to (...) be more worked out in the future either through ethical conventions, corporate measures or legal measures and examine how the law can be incorporated into the existing discourse on the regulation of technologies. Based on this examination, we derive implications for future discourse and elaborate on companies’ responsibilities in developing autonomous driving technologies in an ethical sense. (shrink)

Ethical thought experiments such as the trolley dilemma have been investigated extensively in the past, showing that humans act in utilitarian ways, trying to cause as little overall damage as possible. These trolley dilemmas have gained renewed attention over the past few years, especially due to the necessity of implementing moral decisions in autonomous driving vehicles. We conducted a set of experiments in which participants experienced modified trolley dilemmas as drivers in virtual reality environments. Participants had to make decisions (...) between driving in one of two lanes where different obstacles came into view. Eventually, the participants had to decide which of the objects they would crash into. Obstacles included a variety of human-like avatars of different ages and group sizes. Furthermore, the influence of sidewalks as potential safe harbors and a condition implicating self-sacrifice were tested. Results showed that participants, in general, decided in a utilitarian manner, sparing the highest number of avatars possible with a limited influence by the other variables. Derived from these findings, which are in line with the utilitarian approach in moral decision making, it will be argued for an obligatory ethics setting implemented in ADVs. (shrink)

ABSTRACTTwo categories of ethical questions surrounding military autonomous systems are discussed in this article. The first category concerns ethical issues regarding the use of military autonomous systems in the air and in the water. These issues are systematized with the Laws of Armed Conflict as a backdrop. The second category concerns whether autonomous systems may affect the ethical interpretation of LOAC. It is argued that some terms in LOAC are vague and can be interpreted differently depending on (...) which ethical normative theory is used, which may increase with autonomous systems. The impact of Unmanned Aerial Vehicles on the laws of war will be discussed and compared to Maritime Autonomous Systems. The conclusion is that there is need for revisions of LOAC regarding autonomous systems, and that the greatest ethically relevant difference between UAVs and MAS has to do with issues connected to jus ad bellum – particularly lowering the threshold for starting war – but also the sen... (shrink)

Recent advances in wireless technologies have led to the development of intelligent, in-vehicle safety applications designed to share information about the actions of nearby vehicles, potential road hazards, and ultimately predict dangerous scenarios or imminent collisions. These vehicle safety communication (VSC) technologies rely on the creation of autonomous, self-organizing, wireless communication networks connecting vehicles with roadside infrastructure and with each other. As the technical standards and communication protocols for VSC technologies are still being developed, certain ethical implications (...) of these new information technologies emerge: Coupled with the predicted safety benefits of VSC applications is a potential rise in the ability to surveil a driver engaging in her everyday activities on the public roads. This paper will explore how the introduction of VSC technologies might disrupt the “contextual integrity” of personal information flows in the context of highway travel and threaten one’s “privacy in public.” Since VSC technologies and their related protocols and standards are still in the developmental stage, the paper will conclude by revealing how close attention to the ethical implications of the remaining design decisions can inform and guide designers of VSC technologies to create innovate safety applications that increase public safety, but without compromising the value of one’s privacy in public. (shrink)

Arming uninhabited vehicles (UVs) is an increasing trend. Widespread deployment can bring dangers for arms-control agreements and international humanitarian law (IHL). Armed UVs can destabilise the situation between potential opponents. Smaller systems can be used for terrorism. Using a systematic definition existing international regulation of armed UVs in the fields of arms control, export control and transparency measures is reviewed; these partly include armed UVs, but leave large gaps. For preventive arms control a general prohibition of armed UVs would be (...) best. If that is unattainable, several measures should be taken. An explicit prohibition of autonomous attack, that is without a human decision, should be added to IHL. Concerning armed UVs remotely controlled by a human soldier, recommendations differ according to type or mission. New kinds of uninhabited nuclear-weapon carriers should be banned. Space weapons should be prohibited in general. UVs smaller than 0.2–0.5 m should be banned. Bigger remotely controlled armed UVs not equipped with weapons of mass destruction should be subject to numerical limitations in various categories. For these the Treaty on Conventional Armed Forces in Europe is an important precedent. (shrink)

This paper discusses the ethical implications of perverse incentives with regard to autonomous driving. We define perverse incentives as a feature of an action, technology, or social policy that invites behavior which negates the primary goal of the actors initiating the action, introducing a certain technology, or implementing a social policy. As a special form of means-end-irrationality, perverse incentives are to be avoided from a prudential standpoint, as they prove to be directly self-defeating: They are not just a form (...) of unintended side effect that must be balanced against the main goal or value to be realized by an action, technology, or policy. Instead, they directly cause the primary goals of the actors—i.e., the goals that they ultimately pursue with the action, technology, or policy—to be “worse achieved”. In this paper, we elaborate on this definition and distinguish three ideal-typical phases of adverse incentives, where only in the last one the threshold for a perverse incentive is crossed. In addition, we discuss different possible relevant actors and their goals in implementing autonomous vehicles. We conclude that even if some actors do not pursue traffic safety as their primary goal, as part of a responsibility network they incur the responsibility to act on the common primary goal of the network, which we argue to be traffic safety. (shrink)