The introduction of self-driving vehicles gives rise to a large number of ethical issues that go beyond the common, extremely narrow, focus on improbable dilemma-like scenarios. This article provides a broad overview of realistic ethical issues related to self-driving vehicles. Some of the major topics covered are as follows: Strong opinions for and against driverless cars may give rise to severe social and political conflicts. A low tolerance for accidents caused by driverless vehicles (...) may delay the introduction of driverless systems that would substantially reduce the risks. Trade-offs will arise between safety and other requirement on the road traffic system. Over-reliance on the swift collision-avoiding reactions of self-driving vehicles can induce people to take dangerous actions, such as stepping out in front of a car, relying on its fast braking. Children travelling alone can violate safety instructions such as the use of seatbelts. Digital information about routes and destinations can be used to convey commercial and political messages to car users. If fast passage can be bought, then socio-economic segregation of road traffic may result. Terrorists and other criminals can hack into a vehicle and make it crash. They can also use self-driving vehicles for instance to carry bombs to their designed places of detonation or to wreak havoc on a country’s road system. (shrink)

Self-driving vehicles offer great potential to improve efficiency on roads, reduce traffic accidents, increase productivity, and minimise our environmental impact in the process. However, they have also seen resistance from different groups claiming that they are unsafe, pose a risk of being hacked, will threaten jobs, and increase environmental pollution from increased driving as a result of their convenience. In order to reap the benefits of SDVs, while avoiding some of the many pitfalls, it is important (...) to effectively determine what challenges we will face in the future and what steps need to be taken now to avoid them. The approach taken in this paper is the construction of a likely future, through the process of a policy scenario methodology, if we continue certain trajectories over the coming years. The purpose of this is to articulate issues we currently face and the construction of a foresight analysis of how these may develop in the next 6 years. It will highlight many of the key facilitators and inhibitors behind this change and the societal impacts caused as a result. This paper will synthesise the wide range of ethical, legal, social and economic impacts that may result from SDV use and implementation by 2025, such as issues of autonomy, privacy, liability, security, data protection, and safety. It will conclude with providing steps that we need to take to avoid these pitfalls, while ensuring we reap the benefits that SDVs bring. (shrink)

The convergence of computing, sensing, and communication technology will soon permit large-scale deployment of self-driving vehicles. This will in turn permit a radical transformation of traffic control technology. This paper makes a case for the importance of addressing questions of social justice in this transformation, and sketches a preliminary framework for doing so. We explain how new forms of traffic control technology have potential implications for several dimensions of social justice, including safety, sustainability, privacy, efficiency, and equal (...) access. Our central focus is on efficiency and equal access as desiderata for traffic control design. We explain the limitations of conventional traffic control in meeting these desiderata, and sketch a preliminary vision for a next-generation traffic control tailored to address better the demands of social justice. One component of this vision is cooperative, hierarchically distributed self-organization among vehicles. Another component of this vision is a priority system enabling selection of priority levels by the user for each vehicle trip in the network, based on the supporting structure of non-monetary credits. (shrink)

The moral importance of liability to harm has so far been ignored in the lively debate about what self-driving vehicles should be programmed to do when an accident is inevitable. But liability matters a great deal to just distribution of risk of harm. While morality sometimes requires simply minimizing relevant harms, this is not so when one party is liable to harm in virtue of voluntarily engaging in activity that foreseeably creates a risky situation, while having reasonable (...) alternatives. On plausible assumptions, merely choosing to use a self-driving vehicle typically gives rise to a degree of liability, so that such vehicles should be programmed to shift the risk from bystanders to users, other things being equal. Insofar vehicles cannot be programmed to take all the factors affecting liability into account, there is a pro tanto moral reason not to introduce them, or restrict their use. (shrink)

Self-driving vehicles will need to make decisions that carry ethical dimensions and manufacturers have (the responsibility) to pre-determine this underlying, deliberate decision-making process. With the rise of self-driving vehicles, scholars have simultaneously started investigating what ethical theories should guide machine behavior, but have not concluded as to which theory should be preferred and adopted. We aim to address this matter by providing a holistic and analytical review of the autonomous driving ethics literature. Based (...) on this review, we summarize the social, moral/legal, and functional advantages and disadvantages of applying particular (traditional) ethical theories to the decision-making of self-driving vehicles. Furthermore, we derive an integrated model that shows how each identified ethical theory could be technically implemented and guide the decision-making of self-driving vehicles. Overall, this article aims to set the groundwork for reflected, successful integration of ethical behavior within self-driving vehicles and deduce an updated research agenda. (shrink)

The goal of this article is to deepen the concept of emerging urban mobility technology. Drawing on philosophical everyday and urban aesthetics, as well as the postphenomenological strand in the philosophy of technology, we explicate the relation between everyday aesthetic experience and urban mobility commoning. Thus, we shed light on the central role of aesthetics for providing depth to the important experiential and value-driven meaning of contemporary urban mobility. We use the example of self-driving vehicle (SDV), as potentially (...) mundane, public, dynamic, and social urban robots, for expanding the range of perspectives relevant for our relations to urban mobility technology. We present the range of existing SDV conceptualizations and contrast them with experiential and aesthetic understanding of urban mobility. In conclusion, we reflect on the potential undesired consequences from the depolitization of technological development, and potential new pathways for speculative thinking concerning urban mobility futures in responsible innovation processes. (shrink)

Recent simulations by Scanlon et al. showed seemingly spectacular results for the Waymo self-driving vehicle in simulations of real accident situations. In this paper, it is argued that the selection criteria for accident situations must be modified in accordance with the relevant policy alternatives. While Scanlon et al. compare Waymo with old human-driven vehicles, it is argued here that the relevant policy question is whether we ought to use self-driven vehicles or human-driven vehicles in (...) the future, which means that we need to consider whether other technological solutions, which are available but not broadly used in human-driven vehicles, could result in human-driven vehicles managing to avoid the same accidents. In this article, a proposal for a new standard of selection criteria is made. (shrink)

Questions of what a self-driving car ought to do if it encounters a situation analogous to the ‘trolley problem’ have dominated recent discussion of the ethics of self-driving cars. This paper argues that this interest is misplaced. If a trolley-style dilemma situation actually occurs, given the limits on what information will be available to the car, the dynamics of braking and tyre traction determine that, irrespective of outcome, it is always least risky for the car to (...) brake in a straight line rather than swerve. (shrink)

Almost all participants in the debate about the ethics of accidents with self-driving cars have so far assumed moral universalism. However, universalism may be philosophically more controversial than is commonly thought, and may lead to undesirable results in terms of non-moral consequences and feasibility. There thus seems to be a need to also start considering what I refer to as the “relativistic car” — a car that is programmed under the assumption that what is morally right, wrong, good, (...) bad, etc. is determined by the moral beliefs of one’s society or culture. My investigation of this idea involves six steps. First, I explain why and how the moral universalism/relativism debate is relevant to the issue of self-driving cars. Second, I argue that there are good reasons to consider accident algorithms that assume relativism. Third, I outline how a relativistic car would be programmed to behave. Fourth, I address what advantages such a car would have, both in terms of its non-moral consequences and feasibility. Fifth, I address the relativistic car’s disadvantages. Finally, I qualify and conclude my considerations. (shrink)

We motivate the desire for self-driving and explain its potential and limitations, and explore the need for—and potential implementation of—morals, ethics, and other value systems as complementary “capabilities” to the Deep Technologies behind self-driving. We consider how the incorporation of such systems may drive or slow adoption of high automation within vehicles. First, we explore the role for morals, ethics, and other value systems in self-driving through a representative hypothetical dilemma faced by a (...) self-driving car. Through the lens of engineering, we explain in simple terms common moral and ethical frameworks including utilitarianism, deontology, and virtue ethics before characterizing their relationship to the fundamental algorithms enabling self-driving. The concepts of behavior cloning, state-based modeling, and reinforcement learning are introduced, with some algorithms being more suitable for the implementation of value systems than others. We touch upon the contemporary cross-disciplinary landscape of morals and ethics in self-driving systems from a joint philosophical and technical perspective, and close with considerations for practitioners and the public, particularly as individuals may not appreciate the nuance and complexity of using imperfect information to navigate diverse scenarios and tough-to-quantify value systems, while “typical” software development reduces complex problems to black and white decision-making. (shrink)

The literature on self-driving cars and ethics continues to grow. Yet much of it focuses on ethical complexities emerging from an individual vehicle. That is an important but insufficient step towards determining how the technology will impact human lives and society more generally. What must complement ongoing discussions is a broader, system level of analysis that engages with the interactions and effects that these cars will have on one another and on the socio-technical systems in which they are (...) embedded. To bring the conversation of self-driving cars to the system level, we make use of two traffic scenarios which highlight some of the complexities that designers, policymakers, and others should consider related to the technology. We then describe three approaches that could be used to address such complexities and their associated shortcomings. We conclude by bringing attention to the “Moral Responsibility for Computing Artifacts: The Rules”, a framework that can provide insight into how to approach ethical issues related to self-driving cars. (shrink)

This article puts forward solutions to some of the ethical and legal dilemmas posed in the current discussion on how to program crash algorithms in autonomous or self-driving cars. The first part of the paper defines the scope of the problem in the criminal legal field, and the next section gives a critical analysis of the proposal to always prioritise the interest of the occupant of the vehicle in situations with conflict of interests. The principle of minimizing social (...) damage as a model for configuring self-driving cars is examined in the third section. Despite its apparent plausibility, within the framework of a liberal legal system that recognises humans as free agents who have rights and responsibilities, maximizing the function of social utility does not justify harmful interference into a person’s legal sphere. Therefore, in the fourth part, the author argues the need to program the crash algorithms of autonomous cars based on a deontological understanding of the system of justifications in criminal law. The solution to the dilemma lies in a prior analysis of the legal positions of all agents involved in the conflict, from a perspective of the principles of autonomy and solidarity as the core of the system of justifications. (shrink)

Every year, 1.35 million people are killed on roads worldwide and even more people are injured. Emerging self-driving car technology promises to cut this statistic down to a fraction of the current rate. On the face of it, this consideration alone constitutes a strong reason to legally require — once self-driving car technology is widely available and affordable — that all vehicles on public roads be self-driving. Here I critically investigate the question of (...) whether self-driving, or autonomous, vehicles should be legally mandated. I develop an argument — premised upon Mill’s Harm Principle — that any legislation mandating the use of self-driving vehicles on public roads is morally impermissible. The Harm Principle, under its most plausible interpretation, has it that the state is warranted in legislating against some activity only if that activity violates the rights of others. In brief, I argue that a human driver, who opts to drive herself on public roads rather than rely on self-driving technology, does not violate anyone’s rights when she so acts. Consequently, when granting the Harm Principle, it follows that the state is not warranted in mandating the use of self-driving vehicles on public roads. If I am correct, the proponent of a self-driving vehicle mandate must reject the Harm Principle. Given its intuitive plausibility and central place in liberal philosophical thought, this is a weighty cost. (shrink)

In this paper, I will argue that automated vehicles should not swerve to avoid a person or vehicle in its path, unless they can do so without imposing risks onto others. I will argue that this is the conclusion that we should reach even if we start by assuming that we should divert the trolley in the standard trolley case. In defence of this claim, I appeal to the distribution of moral and legal responsibilities, highlighting the importance of safe (...) spaces, and arguing in favour of constraints on what can be done to minimise casualties. My arguments draw on the methodology associated with the trolley problem. As such, this paper also defends this methodology, highlighting a number of ways in which authors misunderstand and misrepresent the trolley problem. For example, the ‘trolley problem’ is not the ‘name given by philosophers to classic examples of unavoidable crash scenarios, historically involving runaway trolleys’, as Millar suggests, and trolley cases should not be compared with ‘model building in the sciences’, as Gogoll and Müller suggest. Trolley cases have more in common with lab experiments than model building, and the problem referred to in the trolley problem is not the problem of deciding what to do in any one case. Rather, it refers to the problem of explaining what appear to be conflicting intuitions when we consider two cases together. The problem, for example, could be: how do we justify the claim that automated vehicles should not swerve even if we accept the claim that we should divert the trolley in an apparently similar trolley case? (shrink)

The potential development of self-driving cars (also known as autonomous vehicles or AVs – particularly Level 5 AVs) has called the attention of different interested parties. Yet, there are still only a few relevant international regulations on them, no emergency patterns accepted by communities and Original Equipment Manufacturers (OEMs), and no publicly accepted solutions to some of their pending ethical problems. Thus, this paper aims to provide some possible answers to these moral and practical dilemmas. In particular, (...) we focus on what AVs should do in no-win scenarios and on who should be held responsible for these types of decisions. A naturalistic perspective on ethics informs our proposal, which, we argue, could represent a pragmatic and realistic solution to the regulation of AVs. We discuss the proposals already set out in the current literature regarding both policy-making strategies and theoretical accounts. In fact, we consider and reject descriptive approaches to the problem as well as the option of using either a strict deontological view or a solely utilitarian one to set AVs’ ethical choices. Instead, to provide concrete answers to AVs’ ethical problems, we examine three hierarchical levels of decision-making processes: country-wide regulations, OEM policies, and buyers’ moral attitudes. By appropriately distributing ethical decisions and considering their practical implications, we maintain that our proposal based on ethical naturalism recognizes the importance of all stakeholders and allows the most able of them to take actions (the OEMs and buyers) to reflect on the moral leeway and weight of their options. (shrink)

Driverless automobility presents a “technological sublime” encompassing both promises and perils. The light side of the emerging transportation future lies, for instance, in the newly gained freedom from driving. The dark side of this sublime includes ethical challenges and potential harm resulting from the required socio-technical transformations of mobility. This article explores contemporary visions for the self-driving car future through the lens of the sublime and some of its theoretical variations, such as the natural, technological, electrical, and (...) digital sublime. Nissan’s IDS Concept preview clip and the Chevrolet FNR trailer serve as examples for this analysis, which aims to demythologize the visual rhetoric of the depicted awe-inspiring self-driving systems. The sublime’s inherent dialectic of inducing both pleasure and displeasure is removed in the corporate utopian visions in favor of an exalting partnership between human and machine. This strategy succeeds by setting the mobility future in the context of controlled parameters such as the trustworthy communicative vehicle, the vital and independent protagonists, and the harmless and unharmed environment. Recognizing such recurring strategies and identifying the controlled parameters which allow the sublime object to electrify, not terrify, is key for a sensible engagement with such imagined futures and their social, cultural, political, economic, environmental, and ethical implications. Such premediations of awe-inspiring technological formations and the underlying logics ask to be unpacked toward decision making that considers all potential facets of the sublime future. (shrink)

Ethical thought experiments such as the trolley dilemma have been investigated extensively in the past, showing that humans act in utilitarian ways, trying to cause as little overall damage as possible. These trolley dilemmas have gained renewed attention over the past few years, especially due to the necessity of implementing moral decisions in autonomous driving vehicles. We conducted a set of experiments in which participants experienced modified trolley dilemmas as drivers in virtual reality environments. Participants had to make (...) decisions between driving in one of two lanes where different obstacles came into view. Eventually, the participants had to decide which of the objects they would crash into. Obstacles included a variety of human-like avatars of different ages and group sizes. Furthermore, the influence of sidewalks as potential safe harbors and a condition implicating self-sacrifice were tested. Results showed that participants, in general, decided in a utilitarian manner, sparing the highest number of avatars possible with a limited influence by the other variables. Derived from these findings, which are in line with the utilitarian approach in moral decision making, it will be argued for an obligatory ethics setting implemented in ADVs. (shrink)

To operate in an unpredictable environment, a vehicle with advanced driving assistance systems, such as a robot or a drone, not only needs to register its surroundings but also to combine data from different sensors into a world model, for which it employs filter algorithms. Such world models, as this article argues with reference to the SLAM problem in robotics, consist of nothing other than probabilities about states and events arising in the environment. The model, thus, contains a virtuality (...) of possible worlds that are the basis for adaptive behavior. The article shows that the current development of these technologies requires new concepts because their complex adaptive behaviors cannot be explained by referring them to mere algorithmic processes. Instead, it proposes the heuristic instrument of microdecisions to designate the temporality of decisions between alternatives that are created by probabilistic procedures of world modeling. Microdecisions are more than the implementation of deterministic processes—they decide between possibilities and, thus, always open up the potential of their otherness. By describing autonomous adaptive technologies with this heuristic, the question of sovereignty inevitably arises. It forces us to re-think what autonomy means when decisions can be automated. (shrink)

How should driverless vehicles respond to situations of unavoidable personal harm? This paper takes up the case of self-driving cars as a prominent example of algorithmic moral decision-making, an emergent type of morality that is evolving at a high pace in a digitised business world. As its main contribution, it juxtaposes dilemma decision situations relating to ethical crash algorithms for autonomous cars to two edge cases: the case of manually driven cars facing real-life, mundane accidents, on the (...) one hand, and the dilemmatic situation in theoretically constructed trolley cases, on the other. The paper identifies analogies and disanalogies between the three cases with regard to decision makers, decision design, and decision outcomes. The findings are discussed from the angle of three perspectives: aspects where analogies could be found, those where the case of self-driving cars has turned out to lie in between both edge cases, and those where it entirely departs from either edge case. As a main result, the paper argues that manual driving as well as trolley cases are suitable points of reference for the issue of designing ethical crash algorithms only to a limited extent. Instead, a fundamental epistemic and conceptual divergence of dilemma decision situations in the context of self-driving cars and the used edge cases is substantiated. Finally, the areas of specific need for regulation on the road to introducing autonomous cars are pointed out and related thoughts are sketched through the lens of the humanistic paradigm. (shrink)

How should driverless vehicles respond to situations of unavoidable personal harm? This paper takes up the case of self-driving cars as a prominent example of algorithmic moral decision-making, an emergent type of morality that is evolving at a high pace in a digitised business world. As its main contribution, it juxtaposes dilemma decision situations relating to ethical crash algorithms for autonomous cars to two edge cases: the case of manually driven cars facing real-life, mundane accidents, on the (...) one hand, and the dilemmatic situation in theoretically constructed trolley cases, on the other. The paper identifies analogies and disanalogies between the three cases with regard to decision makers, decision design, and decision outcomes. The findings are discussed from the angle of three perspectives: aspects where analogies could be found, those where the case of self-driving cars has turned out to lie in between both edge cases, and those where it entirely departs from either edge case. As a main result, the paper argues that manual driving as well as trolley cases are suitable points of reference for the issue of designing ethical crash algorithms only to a limited extent. Instead, a fundamental epistemic and conceptual divergence of dilemma decision situations in the context of self-driving cars and the used edge cases is substantiated. Finally, the areas of specific need for regulation on the road to introducing autonomous cars are pointed out and related thoughts are sketched through the lens of the humanistic paradigm. (shrink)

Purpose Fully autonomous self-driving cars not only hold the potential for significant economic and environmental advantages but also introduce complex ethical dilemmas. One of the highly debated issues, known as the “trolley problems,” revolves around determining the appropriate actions for a self-driving car when faced with an unavoidable crash. Currently, the discourse on autonomous vehicle (AV) crash algorithms is primarily shaped by Western ethical traditions, resulting in a Eurocentric bias due to the dominant economic and political (...) influence of the West. However, considering that AV technology will be deployed across diverse cultural and religious contexts, this paper aims to contribute to the discourse by providing an Islamic perspective on programming the response of AVs in the event of an imminent crash. Design/methodology/approach This study proposes a novel methodology based on the Islamic concept of maṣlaḥa for the normative assessment of ethical decisions related to AV programming. Findings Drawing upon the works of classic Islamic jurists, this study highlights two distinct normative visions within Islamic traditions (akin to deontology and consequentialism) concerning the preservation of human lives in the context of AVs. This study explores the shared and divergent elements between Islamic and Western ethical approaches proposed for AVs. Originality/value This pioneering work examines AV crash algorithms from an Islamic perspective, filling a void in the global ethical discourse. This work will also serve an important role to bridge the gap between the theoretical Islamic ethical principles and their practical application in the realm of AVs. (shrink)

With the rapid development of sensor technology for automated driving applications, the fusion, analysis, and application of multimodal data have become the main focus of different scenarios, especially in the development of mobile edge computing technology that provides more efficient algorithms for realizing the various application scenarios. In the present paper, the vehicle status and operation data were acquired by vehicle-borne and roadside units of electronic registration identification of motor vehicles. In addition, a motion model and an identification (...) system for the single-vehicle lane-change process were established by mobile edge computing and self-organizing feature mapping. Two scenarios were modeled and tested: lane change with no vehicles in the target lane and lane change with vehicles in the target lane. It was found that the proposed method successfully identified the stochastic parameters in the process of driving trajectory simulation, and the standard deviation between simulation and the measured results obeyed a normal distribution. The proposed methods can provide significant practical information for improving the data processing efficiency in automated driving applications, for solving single-vehicle lane-change applications, and for promoting the formation of a closed loop from sensing to service. (shrink)

This book offers a systematic and thorough philosophical analysis of the ways in which driving automation crosses path with ethical values. Upon introducing the different forms of driving automation and examining their relation to human autonomy, it provides readers with in-depth reflections on safety, privacy, moral judgment, control, responsibility, sustainability, and other ethical issues. Driving is undoubtedly a moral activity as a human act. Transferring it to artificial agents such as connected and automated vehicles necessarily raises (...) many philosophical questions. When driving is automated, what happens to its ethical dimensions? Could artificial agents accomplish ethical objectives on our behalf, take moral decisions in our place, and drive us into a more ethical transportation future? In doing so, would they be “moral” as we are or in a way that is similar to, but also remarkably different from, our own? And what role is yet to be played by human responsibility and commitment? The book addresses these questions with the aim of stimulating an interdisciplinary dialogue between different stakeholders. They include automotive engineers, computer scientists, and moral philosophers, as well as industry representatives, policymakers, regulators, transportation experts, and the general public. Indeed, connected and automated vehicles will not take the high road for us . We must drive them there. (shrink)

Accidents involving autonomous vehicles raise difficult ethical dilemmas and legal issues. It has been argued that self-driving cars should be programmed to kill, that is, they should be equipped with pre-programmed approaches to the choice of what lives to sacrifice when losses are inevitable. Here we shall explore a different approach, namely, giving the user/passenger the task of deciding what ethical approach should be taken by AVs in unavoidable accident scenarios. We thus assume that AVs are equipped (...) with what we call an “Ethical Knob”, a device enabling passengers to ethically customise their AVs, namely, to choose between different settings corresponding to different moral approaches or principles. Accordingly, AVs would be entrusted with implementing users’ ethical choices, while manufacturers/programmers would be tasked with enabling the user’s choice and ensuring implementation by the AV. (shrink)

This paper discusses the ethical implications of perverse incentives with regard to autonomous driving. We define perverse incentives as a feature of an action, technology, or social policy that invites behavior which negates the primary goal of the actors initiating the action, introducing a certain technology, or implementing a social policy. As a special form of means-end-irrationality, perverse incentives are to be avoided from a prudential standpoint, as they prove to be directly self-defeating: They are not just a (...) form of unintended side effect that must be balanced against the main goal or value to be realized by an action, technology, or policy. Instead, they directly cause the primary goals of the actors—i.e., the goals that they ultimately pursue with the action, technology, or policy—to be “worse achieved”. In this paper, we elaborate on this definition and distinguish three ideal-typical phases of adverse incentives, where only in the last one the threshold for a perverse incentive is crossed. In addition, we discuss different possible relevant actors and their goals in implementing autonomous vehicles. We conclude that even if some actors do not pursue traffic safety as their primary goal, as part of a responsibility network they incur the responsibility to act on the common primary goal of the network, which we argue to be traffic safety. (shrink)

A number of companies including Google and BMW are currently working on the development of autonomous cars. But if fully autonomous cars are going to drive on our roads, it must be decided who is to be held responsible in case of accidents. This involves not only legal questions, but also moral ones. The first question discussed is whether we should try to design the tort liability for car manufacturers in a way that will help along the development and improvement (...) of autonomous vehicles. In particular, Patrick Lin’s concern that any security gain derived from the introduction of autonomous cars would constitute a trade-off in human lives will be addressed. The second question is whether it would be morally permissible to impose liability on the user based on a duty to pay attention to the road and traffic and to intervene when necessary to avoid accidents. Doubts about the moral legitimacy of such a scheme are based on the notion that it is a form of defamation if a person is held to blame for causing the death of another by his inattention if he never had a real chance to intervene. Therefore, the legitimacy of such an approach would depend on the user having an actual chance to do so. The last option discussed in this paper is a system in which a person using an autonomous vehicle has no duty of interfering, but is still held responsible for possible accidents. Two ways of doing so are discussed, but only one is judged morally feasible. (shrink)

Autonomous vehicles —and accidents they are involved in—attest to the urgent need to consider the ethics of artificial intelligence. The question dominating the discussion so far has been whether we want AVs to behave in a ‘selfish’ or utilitarian manner. Rather than considering modeling self-driving cars on a single moral system like utilitarianism, one possible way to approach programming for AI would be to reflect recent work in neuroethics. The agent–deed–consequence model :3–20, 2014a, Behav Brain Sci 37:487–488, (...) 2014b) provides a promising descriptive and normative account while also lending itself well to implementation in AI. The ADC model explains moral judgments by breaking them down into positive or negative intuitive evaluations of the agent, deed, and consequence in any given situation. These intuitive evaluations combine to produce a positive or negative judgment of moral acceptability. For example, the overall judgment of moral acceptability in a situation in which someone committed a deed that is judged as negative would be mitigated if the agent had good intentions and the action had a good consequence. This explains the considerable flexibility and stability of human moral judgment that has yet to be replicated in AI. This paper examines the advantages and disadvantages of implementing the ADC model and how the model could inform future work on ethics of AI in general. (shrink)

This paper discusses the ethical implications of perverse incentives with regard to autonomous driving. We define perverse incentives as a feature of an action, technology, or social policy that invites behavior which negates the primary goal of the actors initiating the action, introducing a certain technology, or implementing a social policy. As a special form of means-end-irrationality, perverse incentives are to be avoided from a prudential standpoint, as they prove to be directly self-defeating: They are not just a (...) form of unintended side effect that must be balanced against the main goal or value to be realized by an action, technology, or policy. Instead, they directly cause the primary goals of the actors—i.e., the goals that they ultimately pursue with the action, technology, or policy—to be “worse achieved”. In this paper, we elaborate on this definition and distinguish three ideal-typical phases of adverse incentives, where only in the last one the threshold for a perverse incentive is crossed. In addition, we discuss different possible relevant actors and their goals in implementing autonomous vehicles. We conclude that even if some actors do not pursue traffic safety as their primary goal, as part of a responsibility network they incur the responsibility to act on the common primary goal of the network, which we argue to be traffic safety. (shrink)

The ethics of autonomous cars and automated driving have been a subject of discussion in research for a number of years :28–58, 2016). As levels of automation progress, with partially automated driving already becoming standard in new cars from a number of manufacturers, the question of ethical and legal standards becomes virulent. For exam-ple, while automated and autonomous cars, being equipped with appropriate detection sensors, processors, and intelligent mapping material, have a chance of being much safer than human-driven (...) cars in many regards, situations will arise in which accidents cannot be completely avoided. Such situations will have to be dealt with when programming the software of these vehicles. In several instances, internationally, regulations have been passed, based on legal considerations of road safety, mostly. However, to date, there have been few, if any, cases of a broader ethics code for autonomous or automated driving preceding actual regulation and being based on a broadly composed ethics committee of independent experts. In July 2016, the German Federal Minister of Transport and Digital Infrastructure, Alexander Dobrindt, appointed a national ethics committee for automated and connected driving, which began its work in September 2016. In June 2017, this committee presented a code of ethics which was published in German and in English. It consists of 20 ethical guidelines. Having been a member of this committee, I will present the main ethical topics of these guidelines and the discussions that lay behind them. (shrink)

This paper investigates the importance of standardising and maintaining the transparency of advanced driver-assistance systems (ADAS) functions nomenclature, designs, and operations in all categories up until fully autonomous vehicles. The aim of this paper is to reveal the discrepancies in ADAS functions across automakers and discuss the underlying issues and potential solutions. In this pilot study, user manuals of various brands are reviewed systematically and critical analyses of common ADAS functions are conducted. The result shows that terminologies used to (...) describe ADAS functions vary widely across manufacturers and sometimes do not reflect their fundamental functions intuitively. Operational conditions and control procedures also vary across the selected models under this study. Due to this lack of consensus across the industry, drivers are not aware or well informed about ADAS functions in their vehicles, leading to a very low utilization rate and may lead to misuse of those functions. This paper provides insightful suggestions for the transport industry, Artificial Intelligence (AI) experts, and regulators to design frameworks and guidelines in governing the naming convention, operating conditions, control procedures, and information disclosure of ADAS. Such guidelines can be the foundations for regulating future AI-based self-driving functions. (shrink)

In 2021, Germany passed the first law worldwide that regulates dilemma situations with autonomous cars. Against this background, this article investigates the permissibility of trade-offs between human lives in the context of self-driving cars. It does so by drawing on the debate about the traditional trolley problem. In contrast to most authors in the relevant literature, it argues that the debate about the trolley problem is both directly and indirectly relevant for the ethics of crashes with self- (...) class='Hi'>driving cars. Drawing on its direct normative relevance, the article shows that trade-offs are permissible in situations with self-driving cars that are similar to paradigmatic trolley cases. In scenarios that are unlike paradigmatic trolley cases, the debate about the trolley problem can have indirect normative relevance because it provides reasons against the use of moral theories and principles that cannot account for the trolley problem. (shrink)

As driving functions become increasingly automated, motorists run the risk of becoming cognitively removed from the driving process. Psychophysiological measures may provide added value not captured through behavioral or self-report measures alone. This paper provides a selective review of the psychophysiological measures that can be utilized to assess cognitive states in real-world driving environments. First, the importance of psychophysiological measures within the context of traffic safety is discussed. Next, the most commonly used physiology-based indices of cognitive (...) states are considered as potential candidates relevant for driving research. These include: electroencephalography and event-related potentials, optical imaging, heart rate and heart rate variability, blood pressure, skin conductance, electromyography, thermal imaging, and pupillometry. For each of these measures, an overview is provided, followed by a discussion of the methods for measuring it in a driving context. Drawing from recent empirical driving and psychophysiology research, the relative strengths and limitations of each measure are discussed to highlight each measures’ unique value. Challenges and recommendations for valid and reliable quantification from lab to (less predictable) real-world driving settings are considered. Finally, we discuss measures that may be better candidates for a near real-time assessment of motorists’ cognitive states that can be utilized in applied settings outside the lab. This review synthesizes the literature on in-vehicle psychophysiological measures to advance the development of effective human-machine driving interfaces and driver support systems. (shrink)

AI and people do not compete on a level-playing field. Self-driving vehicles may be safer than human drivers, but laws often penalize such technology. People may provide superior customer service, but businesses are automating to reduce their taxes. AI may innovate more effectively, but an antiquated legal framework constrains inventive AI. In The Reasonable Robot, Ryan Abbott argues that the law should not discriminate between AI and human behavior and proposes a new legal principle that will ultimately (...) improve human well-being. This work should be read by anyone interested in the rapidly evolving relationship between AI and the law. (shrink)

: Autonomous driving systems promise important changes for future of transport, primarily through the reduction of road accidents. However, ethical concerns, in particular, two central issues, will be key to their successful development. First, situations of risk that involve inevitable harm to passengers and/or bystanders, in which some individuals must be sacrificed for the benefit of others. Secondly, and identification responsible parties and liabilities in the event of an accident. Our work addresses the first of these ethical problems. We (...) are interested in investigating how humans respond to critical situations and what reactions they consider to be morally right or at least preferable to others. Our experimental approach relies on the trolley dilemma and knowledge gained from previous research on this. More specifically, our main purpose was to test the difference between what human drivers actually decide to do in an emergency situations whilst driving a realistic simulator and the moral choices they make when they pause to consider what they would do in the same situation and to better understand why these choices may differs. Keywords: Self-driving Cars; Trolley Problem; Moral Choices; Moral Responsibility; Virtual Reality Dilemmi morali nelle automobili a guida autonoma Riassunto: I sistemi di guida autonomi promettono importanti cambiamenti per il futuro dei trasporti, principalmente attraverso la riduzione degli incidenti stradali. Tuttavia, vi sono preoccupazioni etiche, in particolare due questioni centrali, fondamentali per il loro sviluppo. In primo luogo, le situazioni di rischio che comportano inevitabili danni ai passeggeri e/o ai pedoni, ovvero situazioni in cui alcune persone devono essere sacrificate a beneficio di altri. In secondo luogo, l’identificazione delle parti responsabili in caso di incidente. Il nostro lavoro affronta il primo di questi problemi etici. Siamo interessati a studiare come gli umani rispondono a situazioni critiche e quali reazioni considerano moralmente giuste o almeno preferibili. Il nostro approccio sperimentale si basa sul trolley problem e sulle conoscenze acquisite da precedenti ricerche su questo ambito. Più specificamente, il nostro scopo principale è quello di testare la differenza tra ciò che i conducenti umani decidono effettivamente di fare in una situazione di emergenza, mentre guidano un simulatore realistico, e le scelte morali che compiono se posti nella stessa situazione e hanno la possibilità di decidere senza limiti di tempo. Lo scopo è inoltre comprendere come e perché queste scelte possono differire. Parole chiave: Automobili a guida autonoma; Trolley problem; Scelte morali; Responsabilità morale, Realtà virtuale. (shrink)

Crashes involving self-driving cars at least superficially resemble trolley dilemmas. This article discusses what lessons machine ethicists working on the ethics of self-driving cars can learn from trolleyology. The article proceeds by providing an account of the trolley problem as a paradox and by distinguishing two types of solutions to the trolley problem. According to an optimistic solution, our case intuitions about trolley dilemmas are responding to morally relevant differences. The pessimistic solution denies that this is (...) the case. An optimistic solution would yield first-order moral insights for the ethics of self-driving cars, but such a solution is difficult to come by. More plausible is the pessimistic solution, and it teaches us a methodological lesson. The lesson is that machine ethicists should discount case intuitions and instead rely on intuitions and judgments at a higher level of generality. (shrink)

Self-driving cars hold out the promise of being safer than manually driven cars. Yet they cannot be a 100 % safe. Collisions are sometimes unavoidable. So self-driving cars need to be programmed for how they should respond to scenarios where collisions are highly likely or unavoidable. The accident-scenarios self-driving cars might face have recently been likened to the key examples and dilemmas associated with the trolley problem. In this article, we critically examine this tempting (...) analogy. We identify three important ways in which the ethics of accident-algorithms for self-driving cars and the philosophy of the trolley problem differ from each other. These concern: the basic decision-making situation faced by those who decide how self-driving cars should be programmed to deal with accidents; moral and legal responsibility; and decision-making in the face of risks and uncertainty. In discussing these three areas of disanalogy, we isolate and identify a number of basic issues and complexities that arise within the ethics of the programming of self-driving cars. (shrink)

This paper sheds light on the role of digital platform labour in the development of today’s artificial intelligence, predicated on data-intensive machine learning algorithms. Focus is on the specific ways in which outsourcing of data tasks to myriad ‘micro-workers’, recruited and managed through specialized platforms, powers virtual assistants, self-driving vehicles and connected objects. Using qualitative data from multiple sources, we show that micro-work performs a variety of functions, between three poles that we label, respectively, ‘artificial intelligence preparation’, (...) ‘artificial intelligence verification’ and ‘artificial intelligence impersonation’. Because of the wide scope of application of micro-work, it is a structural component of contemporary artificial intelligence production processes – not an ephemeral form of support that may vanish once the technology reaches maturity stage. Through the lens of micro-work, we prefigure the policy implications of a future in which data technologies do not replace human workforce but imply its marginalization and precariousness. (shrink)

The prospective introduction of autonomous cars into public traffic raises the question of how such systems should behave when an accident is inevitable. Due to concerns with self-interest and liberal legitimacy that have become paramount in the emerging debate, a contractarian framework seems to provide a particularly attractive means of approaching this problem. We examine one such attempt, which derives a harm minimisation rule from the assumptions of rational self-interest and ignorance of one’s position in a future accident. (...) We contend, however, that both contractarian approaches and harm minimisation standards are flawed, due to a failure to account for the fundamental difference between those ‘involved’ and ‘uninvolved’ in an impending crash. Drawing from classical works on the trolley problem, we show how this notion can be substantiated by reference to either the distinction between negative and positive rights, or to differences in people’s claims. By supplementing harm minimisation with corresponding constraints, we can develop crash algorithms for autonomous cars which are both ethically adequate and promise to overcome certain significant practical barriers to implementation. (shrink)

Self‐driving cars hold out the promise of being much safer than regular cars. Yet they cannot be 100% safe. Accordingly, they need to be programmed for how to deal with crash scenarios. Should cars be programmed to always prioritize their owners, to minimize harm, or to respond to crashes on the basis of some other type of principle? The article first discusses whether everyone should have the same “ethics settings.” Next, the oft‐made analogy with the trolley problem is (...) examined. Then follows an assessment of recent empirical work on lay‐people's attitudes about crash algorithms relevant to the ethical issue of crash optimization. Finally, the article discusses what traditional ethical theories such as utilitarianism, Kantianism, virtue ethics, and contractualism imply about how cars should handle crash scenarios. The aim of the article is to provide an overview of the existing literature on these topics and to assess how far the discussion has gotten so far. (shrink)

The issue of how to identify the liability of subjects after a traffic accident takes place remains a puzzle regarding the SAE classification system. The SAE system is not good at dealing with the problem of responsibility evaluation; therefore, building a new classification system for self-driving cars from the perspective of the subject's liability is a possible way to solve this problem. This new system divides automated driving into three levels: i) assisted driving based on the (...) will of drivers, ii) automated driving based on the will of the manufacturers, and iii) fully automated driving based on social will. The corresponding responsible subjects for level one, level two, and level three are the drivers, the manufacturers, and society, respectively. (shrink)

Self‐driving cars hold out the promise of being much safer than regular cars. Yet they cannot be 100% safe. Accordingly, we need to think about who should be held responsible when self‐driving cars crash and people are injured or killed. We also need to examine what new ethical obligations might be created for car users by the safety potential of self‐driving cars. The article first considers what lessons might be learned from the growing legal (...) literature on responsibility for crashes with self‐driving cars. Next, worries about responsibility gaps and retribution gaps from the philosophical literature are introduced. This leads to a discussion of whether self‐driving cars are a form of agents that act independently of human agents. It is suggested that it is better to analyze their apparent agency in terms of human–robot collaborations, within which humans play the most important roles. The next topic is the idea that the safety potential of self‐driving cars might create a duty to either switch to self‐driving cars or seek means of making conventional cars safer. Lastly, there is a short discussion of ethical issues related to safe human–robot coordination within mixed traffic featuring both self‐driving cars and conventional cars. (shrink)

Self-driving cars currently face a lot of technological problems that need to be solved before the cars can be widely used. However, they also face ethical problems, among which the question of crash-optimization algorithms is most prominently discussed. Reviewing current debates about whether we should use the ethics of the Trolley Dilemma as a guide towards designing self-driving cars will provide us with insights about what exactly ethical research does. It will result in the view that (...) although we need the ethics of the Trolley Dilemma as important input for self-driving cars, the route towards simply implementing it into automated cars is blocked. (shrink)

The discourse around self-driving cars has been dominated by an emphasis on their potential to reduce the number of accidents. At the same time, proponents acknowledge that self-driving cars would inevitably be involved in fatal accidents where moral algorithms would decide the fate of those involved. This is a necessary trade-off, proponents suggest, in order to reap the benefits of this new technology. In this article, I engage this argument, demonstrating how an undue optimism and enthusiasm (...) about this technology is obscuring our ability to see what is at stake and explaining how moving beyond the dominant utilitarian framings around this technology opens up a space for both ethical inquiry and innovative design. I suggest that a genuine caring concern for the many lives lost in car accidents now and in the future—a concern that transcends false binary trade-offs and that recognizes the systemic biases and power structures that make certain groups more vulnerable than others—could serve as a starting point to rethink mobility, as it connects to the design of our cities, the well-being of our communities, and the future of our planet. (shrink)

This paper presents an emerging aspect of intentionality through recent Artificial Intelligence (AI) developments in art and design. Our main thesis is that, if we focus just on the outcome of the artistic process, the intentionality of the artist does not have any relevance. Intention is measured as a result of actions regardless of whether they are human-based or not as long as there is an esthetical value intersubjectively acknowledged. In other words, what matters is the ‘intentio’ embedded in the (...) work of art (the output) rather than the ‘intendere’ (the process). By considering the former aspect, intentionality becomes free of human intervention, therefore 'unintentional'. This thesis is supported through a range of examples related to AI in general and AI art in particular. However, in the case of a self-driving vehicle, the macro-goal of the system is fixed by humans, even though the AI system itself will take care of decomposing the macro-goal into a set of micro-goals and take the corresponding decisions. In the case of art, the system does not need to have any exterior goal to the extent that art is pure freedom of expression, as many philosophers have acknowledged. In this sense, the decisions of any AI system generating art or design are independent of humans and must be judged only on the ground of the final outcome. A work of art will be judged ‘artistic’ to the extent that humans will recognize an artistic intent in the work itself. In particular, the introduction of random processes makes an AI-generated artwork fully independent of human creativity. (shrink)

Self-driving cars promise solutions to some of the hazards of human driving but there are important questions about the safety of these new technologies. This paper takes a qualitative social science approach to the question ‘how safe is safe enough?’ Drawing on 50 interviews with people developing and researching self-driving cars, I describe two dominant narratives of safety. The first, safety-in-numbers, sees safety as a self-evident property of the technology and offers metrics in an (...) attempt to reassure the public. The second approach, safety-by-design, starts with the challenge of safety assurance and sees the technology as intrinsically problematic. The first approach is concerned only with performance—what a self-driving system does. The second is also concerned with why systems do what they do and how they should be tested. Using insights from workshops with members of the public, I introduce a further concern that will define trustworthy self-driving cars: the intended and perceived purposes of a system. Engineers’ safety assurances will have their credibility tested in public. ‘How safe is safe enough?’ prompts further questions: ‘safe enough for what?’ and ‘safe enough for whom?’. (shrink)

Self‐driving cars hold out the promise of being much safer than regular cars. Yet they cannot be 100% safe. Accordingly, they need to be programmed for how to deal with crash scenarios. Should cars be programmed to always prioritize their owners, to minimize harm, or to respond to crashes on the basis of some other type of principle? The article first discusses whether everyone should have the same “ethics settings.” Next, the oft‐made analogy with the trolley problem is (...) examined. Then follows an assessment of recent empirical work on lay‐people's attitudes about crash algorithms relevant to the ethical issue of crash optimization. Finally, the article discusses what traditional ethical theories such as utilitarianism, Kantianism, virtue ethics, and contractualism imply about how cars should handle crash scenarios. The aim of the article is to provide an overview of the existing literature on these topics and to assess how far the discussion has gotten so far. (shrink)

When explaining human actions, people usually focus on a small subset of potential causes. What leads us to prefer certain explanations for valenced actions over others? The present studies indicate that our moral attitudes often predict our explanatory preferences far better than our beliefs about how causally sensitive actions are to features of the actor's environment. Study 1 found that high-prejudice participants were much more likely to endorse non-agential explanations of an erotic same-sex encounter, such as that one of the (...) men endured a stressful event earlier that day. Study 2 manipulated participants' beliefs about how the agent's behavior depended on features of his environment, finding that such beliefs played no clear role in modeling participants' explanatory preferences. This result emerged both with low- and high-prejudice, US and Indian participants, suggesting that these findings probably reflect a species-typical feature of human psychology. Study 3 found that moral attitudes also predicted explanations for a woman's decision to abort her pregnancy (3a) and a person's decision to convert to Islam (3b). Study 4 found that luck in an action's etiology tends to undermine perceptions of blame more readily than perceptions of praise. Finally, Study 5 found that when explaining support for a rival ideology, both Liberals and Conservatives downplay agential causes while emphasizing environmental ones. Taken together, these studies indicate that our explanatory preferences often reflect a powerful tendency to represent agents as possessing virtuous true selves. Consequently, situation-focused explanations often appear salient because people resist attributing negatively valenced actions to the true self. There is a person/situation distinction, but it is normative. (shrink)