Since some AI algorithms with high predictive power have impacted human integrity, safety has become a crucial challenge in adopting and deploying AI. Although it is impossible to prevent an algorithm from failing in complex tasks, it is crucial to ensure that it fails safely, especially if it is a critical system. Moreover, due to AI’s unbridled development, it is imperative to minimize the methodological gaps in these systems’ engineering. This paper uses the well-known Box-Jenkins method for statistical modeling as (...) a framework to identify engineering pitfalls in the adjustment and validation of AI models. Step by step, we point out state-of-the-art strategies and good practices to tackle these engineering drawbacks. In the final step, we integrate an internal and external validation scheme that might support an iterative evaluation of the normative, perceived, substantive, social, and environmental safety of all AI systems. (shrink)

Virtual reality is a useful tool to study consumer behavior while they are immersed in a realistic scenario. Among several other factors, personality traits have been shown to have a substantial influence on purchasing behavior. The primary objective of this study was to classify consumers based on the Big Five personality domains using their behavior while performing different tasks in a virtual shop. The personality recognition was ascertained using behavioral measures received from VR hardware, including eye-tracking, navigation, posture and interaction. (...) Responses from 60 participants were collected while performing free and directed search tasks in a virtual hypermarket. A set of behavioral features was processed, and the personality domains were recognized using a statistical supervised machine learning classifier algorithm via a support vector machine. The results suggest that the open-mindedness personality type can be classified using eye gaze patterns, while extraversion is related to posture and interactions. However, a combination of signals must be exhibited to detect conscientiousness and negative emotionality. The combination of all measures and tasks provides better classification accuracy for all personality domains. The study indicates that a consumer’s personality can be recognized using the behavioral sensors included in commercial VR devices during a purchase in a virtual retail store. (shrink)

Artificial intelligence is a useful predictive tool for a wide variety of fields of knowledge. Despite this, the educational field is still an environment that lacks a variety of studies that use this type of predictive tools. In parallel, it is postulated that the levels of self-esteem in the university environment may be related to the strategies implemented to solve problems. For these reasons, the aim of this study was to analyze the levels of self-esteem presented by teaching staff and (...) students at university and to design an algorithm capable of predicting these levels on the basis of their coping strategies, resilience, and sociodemographic variables. For this purpose, the Rosenberg Self-Esteem Scale, the Perceived Stress Scale, and the Brief Resilience Scale were administered. The results showed a relevant role of resilience and stress perceived in predicting participants’ self-esteem levels. The findings highlight the usefulness of artificial neural networks for predicting psychological variables in education. (shrink)

This paper presents a method to compute a posteriori probability intervals when the initial conditional information is also given with probability intervals. The right way to make an exact computation is with the associated convex set of probabilities. Probability trees are used to represent these initial conditional convex sets because they greatly save the space required. This paper proposes a simulated annealing algorithm, which uses probability trees to represent the convex sets in order to compute the a posteriori intervals.

Ethics for Robots describes and defends a method for designing and evaluating ethics algorithms for autonomous machines, such as self-driving cars and search and rescue drones. Derek Leben argues that such algorithms should be evaluated by how effectively they accomplish the problem of cooperation among self-interested organisms, and therefore, rather than simulating the psychological systems that have evolved to solve this problem, engineers should be tackling the problem itself, taking relevant lessons from our moral psychology. Leben draws on the (...) moral theory of John Rawls, arguing that normative moral theories are attempts to develop optimal solutions to the problem of cooperation. He claims that Rawlsian Contractarianism leads to the 'Maximin' principle - the action that maximizes the minimum value - and that the Maximin principle is the most effective solution to the problem of cooperation. He contrasts the Maximin principle with other principles and shows how they can often produce non-cooperative results. Using real-world examples - such as an autonomous vehicle facing a situation where every action results in harm, home care machines, and autonomous weapons systems - Leben contrasts Rawlsian algorithms with alternatives derived from utilitarianism and natural rights libertarianism. Including chapter summaries and a glossary of technical terms, Ethics for Robots is essential reading for philosophers, engineers, computer scientists, and cognitive scientists working on the problem of ethics for autonomous systems. (shrink)

In philosophy of mind, zombies are imaginary creatures that are exact physical duplicates of conscious subjects but for whom there is no first-personal experience. Zombies are meant to show that physicalism—the theory that the universe is made up entirely out of physical components—is false. In this paper, I apply the zombie thought experiment to the realm of morality to assess whether moral agency is something independent from sentience. Algorithms, I argue, are a kind of functional moral zombie, such (...) that thinking about the latter can help us better understand and regulate the former. I contend that the main reason why algorithms can be neither autonomous nor accountable is that they lack sentience. Moral zombies and algorithms are incoherent as moral agents because they lack the necessary moral understanding to be morally responsible. To understand what it means to inflict pain on someone, it is necessary to have experiential knowledge of pain. At most, for an algorithm that feels nothing, ‘values’ will be items on a list, possibly prioritised in a certain way according to a number that represents weightiness. But entities that do not feel cannot value, and beings that do not value cannot act for moral reasons. (shrink)

One of the predominant debates in AI Ethics is the worry and necessity to create fair, transparent and accountable algorithms that do not perpetuate current social inequities. I offer a critical analysis of Reuben Binns’s argument in which he suggests using public reason to address the potential bias of the outcomes of machine learning algorithms. In contrast to him, I argue that ultimately what is needed is not public reason per se, but an audit of the implicit moral assumptions (...) of societies within which algorithms are built and applied. Public justification is appealing since it offers us the possibility to align the decision-making outcomes of the algorithm with the core moral values of stakeholders within a constitutional democratic society. My concern is that the common moral principles that form the foundation of public reason are not necessarily neutral, as they still express specific moral ideals and normative standards even if there is moral agreement by society as a whole, or among different stakeholders within society. Appealing to such normative standards may thus still lead to algorithmic outcomes being biased as common moral values may very well still be discriminatory even though they are formed from a consensus, and even if public reason is applied as a kind of filter for potential algorithmic outcomes. Hence, I argue that these implicit moral norms within society that we take as a given in public reasoning, need to be audited from generation to generation in order to effectively mitigate potential algorithmic bias. (shrink)

Recommender systems are artificial intelligence technologies, deployed by online platforms, that model our individual preferences and direct our attention to content we’re likely to engage with. As the digital world has become increasingly saturated with information, we’ve become ever more reliant on these tools to efficiently allocate our attention. And our reliance on algorithmic recommendation may, in turn, reshape us as moral agents. While recommender systems could in principle enhance our moral agency by enabling us to cut through (...) the information saturation of the internet and focus on things that matter, as they’re currently designed and implemented they’re apt to interfere with our ability to attend appropriately to morally relevant factors. In order to analyze the distinctive moral problems algorithmic recommendation poses, we develop a framework for the ethics of attention and an account of judicious attention allocation as a moral skill. We then discuss empirical evidence suggesting that attentional moral skill can be thwarted and undermined in various ways by algorithmic recommendation and related affordances of online platforms, as well as economic and technical considerations that support this concern. Finally, we consider how emerging technologies might overcome the problems we identify. (shrink)

Recently, a multinomial process tree model was developed to measure an agent’s consequence sensitivity, norm sensitivity, and generalized inaction/action preferences when making moral decisions (CNI model). However, the CNI model presupposed that an agent considersconsequences—norms—generalizedinaction/actionpreferences sequentially, which is untenable based on recent evidence. Besides, the CNI model generates parameters at the group level based on binary categorical data. Hence, theC/N/Iparameters cannot be used for correlation analyses or other conventional research designs. To solve these limitations, we developed the CAN algorithm (...) to computenormandconsequencesensitivities and overallaction/inactionpreferences algebraically in a parallel manner. We re-analyzed the raw data of the original CNI model to test the methodological predictions. Our results demonstrate that: (1) theCparameter is approximately equal between the CNI model and CAN algorithm; (2) theNparameter under the CNI model approximately equalsN/(1 −C) under the CAN algorithm; (3) theIparameter andAparameter are reversed around 0.5 – the larger theIparameter, the more the generalized inactionversusaction preference and the larger theAparameter, the more overall actionversusinaction preference; (4) tests of differences in parameters between groups with the CNI model and CAN algorithm led to almost the same statistical conclusion; (5) parameters from the CAN algorithm can be used for correlational analyses and multiple comparisons, and this is an advantage over the parameters from the CNI model. The theoretical and methodological implications of our study were also discussed. (shrink)

In this paper, we examine the qualitative moral impact of machine learning-based clinical decision support systems in the process of medical diagnosis. To date, discussions about machine learning in this context have focused on problems that can be measured and assessed quantitatively, such as by estimating the extent of potential harm or calculating incurred risks. We maintain that such discussions neglect the qualitative moral impact of these technologies. Drawing on the philosophical approaches of technomoral change and technological mediation (...) theory, which explore the interplay between technologies and morality, we present an analysis of concerns related to the adoption of machine learning-aided medical diagnosis. We analyze anticipated moral issues that machine learning systems pose for different stakeholders, such as bias and opacity in the way that models are trained to produce diagnoses, changes to how health care providers, patients, and developers understand their roles and professions, and challenges to existing forms of medical legislation. Albeit preliminary in nature, the insights offered by the technomoral change and the technological mediation approaches expand and enrich the current discussion about machine learning in diagnostic practices, bringing distinct and currently underexplored areas of concern to the forefront. These insights can contribute to a more encompassing and better informed decision-making process when adapting machine learning techniques to medical diagnosis, while acknowledging the interests of multiple stakeholders and the active role that technologies play in generating, perpetuating, and modifying ethical concerns in health care. (shrink)

Fairness is one of the most prominent values in the Ethics and Artificial Intelligence (AI) debate and, specifically, in the discussion on algorithmic decision-making (ADM). However, while the need for fairness in ADM is widely acknowledged, the very concept of fairness has not been sufficiently explored so far. Our paper aims to fill this gap and claims that an ethically informed re-definition of fairness is needed to adequately investigate fairness in ADM. To achieve our goal, after an introductory section aimed (...) at clarifying the aim and structure of the paper, in section “Fairness in algorithmic decision-making” we provide an overview of the state of the art of the discussion on fairness in ADM and show its shortcomings; in section “Fairness as an ethical value”, we pursue an ethical inquiry into the concept of fairness, drawing insights from accounts of fairness developed in moral philosophy, and define fairness as an ethical value. In particular, we argue that fairness is articulated in a distributive and socio-relational dimension; it comprises three main components: fair equality of opportunity, equal right to justification, and fair equality of relationship; these components are grounded in the need to respect persons both as persons and as particular individuals. In section “Fairness in algorithmic decision-making revised”, we analyze the implications of our redefinition of fairness as an ethical value on the discussion of fairness in ADM and show that each component of fairness has profound effects on the criteria that ADM ought to meet. Finally, in section “Concluding remarks”, we sketch some broader implications and conclude. (shrink)

We explore aversion to the use of algorithms in moral decision-making. So far, this aversion has been explained mainly by the fear of opaque decisions that are potentially biased. Using incentivized experiments, we study which role the desire for human discretion in moral decision-making plays. This seems justified in light of evidence suggesting that people might not doubt the quality of algorithmic decisions, but still reject them. In our first study, we found that people prefer humans with decision-making (...) discretion to algorithms that rigidly apply exogenously given human-created fairness principles to specific cases. In the second study, we found that people do not prefer humans to algorithms because they appreciate flesh-and-blood decision-makers per se, but because they appreciate humans’ freedom to transcend fairness principles at will. Our results contribute to a deeper understanding of algorithm aversion. They indicate that emphasizing the transparency of algorithms that clearly follow fairness principles might not be the only element for fostering societal algorithm acceptance and suggest reconsidering certain features of the decision-making process. (shrink)

Algorithmic systems and predictive analytics play an increasingly important role in various aspects of modern life. Scholarship on the moral ramifications of such systems is in its early stages, and much of it focuses on bias and harm. This paper argues that in understanding the moral salience of algorithmic systems it is essential to understand the relation between algorithms, autonomy, and agency. We draw on several recent cases in criminal sentencing and K–12 teacher evaluation to outline four key (...) ways in which issues of agency, autonomy, and respect for persons can conflict with algorithmic decision-making. Three of these involve failures to treat individual agents with the respect they deserve. The fourth involves distancing oneself from a morally suspect action by attributing one’s decision to take that action to an algorithm, thereby laundering one’s agency. (shrink)

Workers can be fired from jobs, citizens sent to jail, and adolescents more likely to experience depression, all because of algorithms. Algorithms have considerable impacts on our lives. To increase user satisfaction and trust, the most common proposal from academics and developers is to increase the transparency of algorithmic design. While there is a large body of literature on algorithmic transparency, the impact of unethical data collection practices is less well understood. Currently, there is limited research on the factors that (...) affect users’ trust in data collection practices and algorithmic transparency. In this research, we explore the relative impact of both factors as they relate to important outcome measures such as user’s trust, comfort level, and moral acceptability. We conducted two pilot studies to learn what real users consider to be ethical and unethical data collection practices, as well as high and low transparency for algorithms. We then used these findings in a 2 × 2 design to examine how transparency and the acceptability of data collection practices impact users’ acceptance, comfort, and trust in algorithms. Our results suggest that the singular emphasis on algorithmic transparency may be misplaced. Given the difference in their impact to increase acceptance, trust, and user satisfaction, a more effective strategy would be to also understand and abide by users’ views of ethical data collection practices. (shrink)

The selection and balancing of values for the Eurotransplant kidney allocation algorithm poses both practical and ethical challenges. The paper argues that any allocation algorithm can only be justified by reference to some substantive conception of a good life that reflects our value preferences regarding the allocation of scarce donor kidneys. It is concluded that the criterion of HLA compatibility maximizes overall rather than individual utility. The paper emphasizes that good pragmatic arguments for maintaining the primacy of HLA matching can (...) never replace a more systematic, independent ethical justification. As neither the selection nor the balancing of the different allocation criteria are based on an explicit ethical justification, the paper concludes that the choice of the Wujciak-algorithm was rather a product of pragmatic compromise than moral consensus. (shrink)

Machine learning algorithms are increasingly used to shape high-stake allocations, sparking research efforts to orient algorithm design towards ideals of justice and fairness. In this research on algorithmic fairness, normative theorizing has primarily focused on identification of “ideally fair” target states. In this paper, we argue that this preoccupation with target states in abstraction from the situated dynamics of deployment is misguided. We propose a framework that takes dynamic trajectories as direct objects of moral appraisal, highlighting three respects in (...) which such trajectories can be subject to evaluation in relation to their temporal dynamics, robustness, and representation. (shrink)

The moral and ethical challenges of living in community pertain not only to the intersection of human beings one with another, but also our interactions with our machine creations. This article explores the philosophical and theological framework for reasoning and decision-making through the lens of science fiction, religion, and artificial intelligence (both real and imagined). In comparing the programming of autonomous machines with human ethical deliberation, we discover that both depend on a concrete ordering of priorities derived from a (...) clearly defined value system. (shrink)

Machine intelligence already helps medical staff with a number of tasks. Ethical decision-making, however, has not been handed over to computers. In this proof-of-concept study, we show how an algorithm based on Beauchamp and Childress’ prima-facie principles could be employed to advise on a range of moral dilemma situations that occur in medical institutions. We explain why we chose fuzzy cognitive maps to set up the advisory system and how we utilized machine learning to train it. We report on (...) the difficult task of operationalizing the principles of beneficence, non-maleficence and patient autonomy, and describe how we selected suitable input parameters that we extracted from a training dataset of clinical cases. The first performance results are promising, but an algorithmic approach to ethics also comes with several weaknesses and limitations. Should one really entrust the sensitive domain of clinical ethics to machine intelligence? (shrink)

Algorithms influence every facet of modern life: criminal justice, education, housing, entertainment, elections, social media, news feeds, work… the list goes on. Delegating important decisions to machines, however, gives rise to deep moral concerns about responsibility, transparency, freedom, fairness, and democracy. Algorithms and Autonomy connects these concerns to the core human value of autonomy in the contexts of algorithmic teacher evaluation, risk assessment in criminal sentencing, predictive policing, background checks, news feeds, ride-sharing platforms, social media, and election interference. Using (...) these case studies, the authors provide a better understanding of machine fairness and algorithmic transparency. They explain why interventions in algorithmic systems are necessary to ensure that algorithms are not used to control citizens' participation in politics and undercut democracy. This title is also available as Open Access on Cambridge Core. (shrink)

While organizations today make extensive use of complex algorithms, the notion of algorithmic accountability remains an elusive ideal due to the opacity and fluidity of algorithms. In this article, we develop a framework for managing algorithmic accountability that highlights three interrelated dimensions: reputational concerns, engagement strategies, and discourse principles. The framework clarifies that accountability processes for algorithms are driven by reputational concerns about the epistemic setup, opacity, and outcomes of algorithms; that the way in which organizations practically engage with emergent (...) expectations about algorithms may be manipulative, adaptive, or moral; and that when accountability relationships are heavily burdened by the opacity and fluidity of complex algorithmic systems, the emphasis of engagement should shift to a rational communication process through which a continuous and tentative assessment of the development, workings, and consequences of algorithms can be achieved over time. The degree to which such engagement is, in fact, rational can be assessed based on four discourse-ethical principles of participation, comprehension, multivocality, and responsiveness. We conclude that the framework may help organizations and their environments to jointly work toward greater accountability for complex algorithms. It may further help organizations in reputational positioning surrounding accountability issues. The discourse-ethical principles introduced in this article are meant to elevate these positioning contests to extend beyond mere adaption or compliance and help guide organizations to find moral and forward-looking solutions to accountability issues. (shrink)

As algorithms have become ubiquitous in consequential domains, societal concerns about the potential for discriminatory outcomes have prompted urgent calls to address algorithmic bias. In response, a rich literature across computer science, law, and ethics is rapidly proliferating to advance approaches to designing fair algorithms. Yet computer scientists, legal scholars, and ethicists are often not speaking the same language when using the term ‘bias.’ Debates concerning whether society can or should tackle the problem of algorithmic bias are hampered by conflations (...) of various understandings of bias, ranging from neutral deviations from a standard to morally problematic instances of injustice due to prejudice, discrimination, and disparate treatment. This terminological confusion impedes efforts to address clear cases of discrimination. -/- In this paper, we examine the promises and challenges of different approaches to disambiguating bias and designing for justice. While both approaches aid in understanding and addressing clear algorithmic harms, we argue that they also risk being leveraged in ways that ultimately deflect accountability from those building and deploying these systems. Applying this analysis to recent examples of generative AI, our argument highlights unseen dangers in current methods of evaluating algorithmic bias and points to ways to redirect approaches to addressing bias in generative AI at its early stages in ways that can more robustly meet the demands of justice. (shrink)

Cities, such as industry or the universities, are immersed in a process of digital transformation generated by the possibility and technological convergence of the Internet of Things, Big Data and Artificial Intelligence and its consequences: hyperconnectivity, datafication and algorithmization. A process of transformation towards what has come to be called as Smart Cities. The aim of this paper is to show the impacts and consequences of digital connectivity, algorithmization and the datafication of urban digital society to outline possible ways of (...) resolving the underlying moral conflicts. (shrink)

As inductive decision-making procedures, the inferences made by machine learning programs are subject to underdetermination by evidence and bear inductive risk. One strategy for overcoming these challenges is guided by a presumption in philosophy of science that inductive inferences can and should be value-free. Applied to machine learning programs, the strategy assumes that the influence of values is restricted to data and decision outcomes, thereby omitting internal value-laden design choice points. In this paper, I apply arguments from feminist philosophy of (...) science to machine learning programs to make the case that the resources required to respond to these inductive challenges render critical aspects of their design constitutively value-laden. I demonstrate these points specifically in the case of recidivism algorithms, arguing that contemporary debates concerning fairness in criminal justice risk-assessment programs are best understood as iterations of traditional arguments from inductive risk and demarcation, and thereby establish the value-laden nature of automated decision-making programs. Finally, in light of these points, I address opportunities for relocating the value-free ideal in machine learning and the limitations that accompany them. (shrink)

In this paper we develop a general theory of algorithmic fairness. Drawing on Johnson King and Babic’s work on moral encroachment, on Gary Becker’s work on labor market discrimination, and on Strawson’s idea of resentment and indignation as responses to violations of the demand for goodwill toward oneself and others, we locate attitudes to fairness in an agent’s utility function. In particular, we first argue that fairness is a matter of a decision-maker’s relative concern for the plight of people (...) from different groups, rather than of the outcomes produced for different groups. We then show how an agent’s preferences, including in particular their attitudes to error, give rise to their decision thresholds. Tying these points together, we argue that the agent’s relative degrees of concern for different groups manifest in a difference in decision thresholds applied to these groups. (shrink)

This article examines the complaint that arbitrary algorithmic decisions wrong those whom they affect. It makes three contributions. First, it provides an analysis of what arbitrariness means in this context. Second, it argues that arbitrariness is not of moral concern except when special circumstances apply. However, when the same algorithm or different algorithms based on the same data are used in multiple contexts, a person may be arbitrarily excluded from a broad range of opportunities. The third contribution is to (...) explain why this systemic exclusion is of moral concern and to offer a solution to address it. (shrink)

The prospective introduction of autonomous cars into public traffic raises the question of how such systems should behave when an accident is inevitable. Due to concerns with self-interest and liberal legitimacy that have become paramount in the emerging debate, a contractarian framework seems to provide a particularly attractive means of approaching this problem. We examine one such attempt, which derives a harm minimisation rule from the assumptions of rational self-interest and ignorance of one’s position in a future accident. We contend, (...) however, that both contractarian approaches and harm minimisation standards are flawed, due to a failure to account for the fundamental difference between those ‘involved’ and ‘uninvolved’ in an impending crash. Drawing from classical works on the trolley problem, we show how this notion can be substantiated by reference to either the distinction between negative and positive rights, or to differences in people’s claims. By supplementing harm minimisation with corresponding constraints, we can develop crash algorithms for autonomous cars which are both ethically adequate and promise to overcome certain significant practical barriers to implementation. (shrink)

When Artificial Intelligence (AI) is applied in decision-making that affects people’s lives, it is now well established that the outcomes can be biased or discriminatory. The question of whether algorithms themselves can be among the sources of bias has been the subject of recent debate among Artificial Intelligence researchers, and scholars who study the social impact of technology. There has been a tendency to focus on examples, where the data set used to train the AI is biased, and denial on (...) the part of some researchers that algorithms can also be biased. Here we illustrate the point that algorithms themselves can be the source of bias with the example of collaborative filtering algorithms for recommendation and search. These algorithms are known to suffer from cold-start, popularity, and homogenizing biases, among others. While these are typically described as statistical biases rather than biases of moral import; in this paper we show that these statistical biases can lead directly to discriminatory outcomes. The intuitive idea is that data points on the margins of distributions of human data tend to correspond to marginalized people. The statistical biases described here have the effect of further marginalizing the already marginal. Biased algorithms for applications such as media recommendations can have significant impact on individuals’ and communities’ access to information and culturally-relevant resources. This source of bias warrants serious attention given the ubiquity of algorithmic decision-making. (shrink)

With increasing use of automated algorithmic decision-making, issues of algorithmic fairness have attracted much attention lately. In this growing literature, existing concepts from ethics and political philosophy are often applied to new contexts. The reverse—that novel insights from the algorithmic fairness literature are fed back into ethics and political philosophy—is far less established. However, this short commentary on Baumann and Loi (Philosophy & Technology, 36(3), 45 2023) aims to do precisely this. Baumann and Loi argue that among algorithmic group fairness (...) measures proposed, one—sufficiency (well-calibration) is morally defensible for insurers to use, whereas independence (statistical parity or demographic parity) and separation (equalized odds) are not normatively appropriate in the insurance context. Such a result may seem to be of relatively narrow interest to insurers and insurance scholars only. We argue, however, that arguments such as that offered by Baumann and Loi have an important but so far overlooked connection to the derivation of the minimal state offered by Nozick (1974) and thus to political philosophy at large. (shrink)

Based on a deliberative democracy, this book uses a hermeneutic-critical methodology to study bibliographical sources and practical issues in order to analyse the possibilities, limits and consequences of the digital transformation of democracy. Drawing on a two-way democracy, the aim of this book is intended as an aid for thinking through viable alternatives to the current state of democracy with regard to its ethical foundations and the moral knowledge implicit in or assumed by the way we perceive and understand (...) democracy. It is intended to stimulate reflection and discussion on the basis that, by addressing what we understand as democracy, we can inevitably influence the reality known as democracy. Democracy’s evident regression in today’s world makes this all too apparent: it has become a hostage to all kinds of autocracies and technopopulisms, which are supported to a greater or lesser extent by the current algorithmic revolution. (shrink)

Firms use algorithms to make important business decisions. To date, the algorithmic accountability literature has elided a fundamentally empirical question important to business ethics and management: Under what circumstances, if any, are algorithmic decision-making systems considered legitimate? The present study begins to answer this question. Using factorial vignette survey methodology, we explore the impact of decision importance, governance, outcomes, and data inputs on perceptions of the legitimacy of algorithmic decisions made by firms. We find that many of the procedural governance (...) mechanisms in practice today, such as notices and impact statements, do not lead to algorithmic decisions being perceived as more legitimate in general, and, consistent with legitimacy theory, that algorithmic decisions with good outcomes are perceived as more legitimate than bad outcomes. Yet, robust governance, such as offering an appeal process, can create a legitimacy dividend for decisions with bad outcomes. However, when arbitrary or morally dubious factors are used to make decisions, most legitimacy dividends are erased. In other words, companies cannot overcome the legitimacy penalty of using arbitrary or morally dubious factors, such as race or the day of the week, with a good outcome or an appeal process for individuals. These findings add new perspectives to both the literature on legitimacy and policy discussions on algorithmic decision-making in firms. (shrink)

The rapidly increasing role of automation throughout the economy, culture and our personal lives has generated a large literature on the risks of algorithmic decision-making, particularly in high-stakes legal settings. Algorithmic tools are charged with bias, shrouded in secrecy, and frequently difficult to interpret. However, these criticisms have tended to focus on particular implementations, specific predictive techniques, and the idiosyncrasies of the American legal-regulatory regime. They do not address the more fundamental unease about the prospect that we might one day (...) replace judges with algorithms, no matter how fair, transparent, and intelligible they become. The aim of this paper is to propose an account of the source of that unease, and to evaluate its plausibility. I trace foundational unease with algorithmic decision-making in the law to the powerful intuition that there is a basic moral and legal difference between showing that something is true of many people just like you and showing that it is true of you. Human judgment attends to the exception; automation insists on blindly applying the rule. I show how this intuitive thought is connected to both epistemological arguments about the value of statistical evidence, as well as to court-centered conceptions of the rule of law. Unease with algorithmic decision-making in the law thus draws on an intuitive principle that underpins a disparate range of views in legal philosophy. This suggests the principle is deeply ingrained. Nonetheless, I argue that the powerful intuition is not as decisive as it may seem, and indeed runs into significant epistemological and normative challenges. At an epistemological level, I show how concerns about statistical evidence's ability to track the truth can be resolved by adopting a probabilistic, rather than modal, conception of truth-tracking. At a normative level, commitment to highly individualized decision-making co-exists with equally ingrained and competing principles, such as consistent application of law. This suggests that the “rule of law” may not identify a discrete set of institutional arrangements, as proponents of a court-centric conception would have it, but rather a more loosely defined set of values that could potentially be operationalized in multiple ways, including through some level of algorithmic adjudication. Although the prospect of replacing judges with algorithms is indeed unsettling, it does not necessarily entail unreasonable verdicts or an attack on the rule of law. (shrink)

We argue that machine learning algorithms can inflict microaggressions on members of marginalized groups and that recognizing these harms as instances of microaggressions is key to effectively addressing the problem. The concept of microaggression is also illuminated by being studied in algorithmic contexts. We contribute to the microaggression literature by expanding the category of environmental microaggressions and highlighting the unique issues of moral responsibility that arise when we focus on this category. We theorize two kinds of algorithmic microaggression, stereotyping (...) and erasure microaggressions, and argue that corporations are responsible for the microaggressions their algorithms create. As a case study, we look at the problems faced by Google’s autocomplete prediction and at the insufficiency of their solutions. The case study of autocomplete demonstrates our core claim that microaggressions constitute a distinct form of algorithmic bias and that identifying them as such allows us to avoid seeming solutions that recreate the same kinds of harms. Google has a responsibility to make information freely available, without exposing users to degradation. To fulfill its duties to marginalized groups, Google must abandon the fiction of neutral prediction and instead embrace the liberatory power of suggestion. (shrink)

Recent studies have shown that recidivism scoring algorithms like COMPAS have significant racial bias: Black defendants are roughly twice as likely as white defendants to be mistakenly classified as medium- or high-risk. This has led some to call for abolishing COMPAS. But many others have argued that algorithms should instead be given access to a defendant's race, which, perhaps counterintuitively, is likely to improve outcomes. This approach can involve either establishing race-sensitive risk thresholds, or distinct racial ‘tracks’. Is there a (...) moral difference between these two approaches? We first consider Deborah Hellman's view that the use of distinct racial tracks (but not distinct thresholds) does not constitute disparate treatment since the effects on individuals are indirect and does not rely on a racial generalization. We argue that this is mistaken: the use of different racial tracks seems both to have direct effects on and to rely on a racial generalization. We then offer an alternative understanding of the distinction between these two approaches—namely, that the use of different cut points is to the counterfactual comparative disadvantage, ex ante, of all white defendants, while the use of different racial tracks can in principle be to the advantage of all groups, though some defendants in both groups will fare worse. Does this mean that the use of cut points is impermissible? Ultimately, we argue, while there are reasons to be skeptical of the use of distinct cut points, it is an open question whether these reasons suffice to make a difference to their moral permissibility. (shrink)

In recent years, a plethora of high-profile scientific publications has been reporting about machine learning algorithms outperforming clinicians in medical diagnosis or treatment recommendations. This has spiked interest in deploying relevant algorithms with the aim of enhancing decision-making in healthcare. In this paper, we argue that instead of straightforwardly enhancing the decision-making capabilities of clinicians and healthcare institutions, deploying machines learning algorithms entails trade-offs at the epistemic and the normative level. Whereas involving machine learning might improve the accuracy of medical (...) diagnosis, it comes at the expense of opacity when trying to assess the reliability of given diagnosis. Drawing on literature in social epistemology and moral responsibility, we argue that the uncertainty in question potentially undermines the epistemic authority of clinicians. Furthermore, we elucidate potential pitfalls of involving machine learning in healthcare with respect to paternalism, moral responsibility and fairness. At last, we discuss how the deployment of machine learning algorithms might shift the evidentiary norms of medical diagnosis. In this regard, we hope to lay the grounds for further ethical reflection of the opportunities and pitfalls of machine learning for enhancing decision-making in healthcare. (shrink)

Self-driving cars hold out the promise of being safer than manually driven cars. Yet they cannot be a 100 % safe. Collisions are sometimes unavoidable. So self-driving cars need to be programmed for how they should respond to scenarios where collisions are highly likely or unavoidable. The accident-scenarios self-driving cars might face have recently been likened to the key examples and dilemmas associated with the trolley problem. In this article, we critically examine this tempting analogy. We identify three important ways (...) in which the ethics of accident-algorithms for self-driving cars and the philosophy of the trolley problem differ from each other. These concern: the basic decision-making situation faced by those who decide how self-driving cars should be programmed to deal with accidents; moral and legal responsibility; and decision-making in the face of risks and uncertainty. In discussing these three areas of disanalogy, we isolate and identify a number of basic issues and complexities that arise within the ethics of the programming of self-driving cars. (shrink)

Over the past decade, scholars, institutions, and activists have voiced strong concerns about the potential of automated decision systems to indirectly discriminate against vulnerable groups. This article analyses the ethics of algorithmic indirect discrimination, and argues that we can explain what is morally bad about such discrimination by reference to the fact that it causes harm. The article first sketches certain elements of the technical and conceptual background, including definitions of direct and indirect algorithmic differential treatment. It next introduces three (...) prominent accounts of fairness as potential explanations if the badness of algorithmic indirect discrimination, but argues that all three are vulnerable to powerful levelling-down-style objections. Instead, the article demonstrates how proper attention to the way differences in decision-scenarios affect the distribution of harms can help us account for intuitions in prominent cases. Finally, the article considers a potential objection based on the fact that certain forms of algorithmic indirect discrimination appear to distribute rather than cause harm, and notes that we can explain how such distributions cause harm by attending to differences in individual and group vulnerability. (shrink)

It has long been argued that robotic systems decrease the error margin of civilian casualties that are often the consequence of human error. The focus on algorithmic decision-making and the gradual reduction of human control unveils moral and juristic issues that lead to the crucial question: can we take algorithms to court?

Algorithmic discrimination has rapidly become a topic of intense public and academic interest. This article explores three issues raised by algorithmic discrimination: 1) the distinction between direct and indirect discrimination, 2) the notion of disadvantageous treatment, and 3) the moral badness of discriminatory automated decision-making. It argues that some conventional distinctions between direct and indirect discrimination appear not to apply to algorithmic discrimination, that algorithmic discrimination may often be discrimination between groups, as opposed to against groups, and that it (...) is not necessarily the case that morally bad algorithmic discrimination gives us reason to not use automated decision-making. For each of the three issues, the article explores implications for algorithmic discrimination, suggests some alternative answers, and clarifies how we may want to think of discrimination more broadly in light of lessons drawn from the context of algorithmic discrimination. (shrink)

Criminal justice systems have traditionally relied heavily on human decision-making, but new technologies are increasingly supplementing the human role in this sector. This paper considers what general limits need to be placed on the use of algorithms in sentencing decisions. It argues that, even once we can build algorithms that equal human decision-making capacities, strict constraints need to be placed on how they are designed and developed. The act of condemnation is a valuable element of criminal sentencing, and using algorithms (...) in sentencing – even in an advisory role – threatens to undermine this value. The paper argues that a principle of “meaningful public control” should be met in all sentencing decisions if they are to retain their condemnatory status. This principle requires that agents who have standing to act on behalf of the wider political community retain moral responsibility for all sentencing decisions. While this principle does not rule out the use of algorithms, it does require limits on how they are constructed. (shrink)

This book focuses on how new technologies are raising and reshaping ethical questions and practices which aim to automate ethics into program outputs. With new powerful technologies come enhanced capacities to act, which in turn require new ethical concepts for guiding just and fair actions in the use of these new capabilities. The new algorithmic regimes, for their ethical articulation, build on prior ethics discourses in computer and information ethics, as well as the philosophical traditions of ethics generally. Especially as (...) our technologies become more autonomous, operating alongside us in the home, workplace or on the roads, ethics has the potential to limit negative effects and shape the new technical terrains in a more humanly recognizable way. The volume covers a critique of human-centered AI, the effects of AI and the internet of things in the domain of human resource management, how decentralized finance applications on the blockchain encode ethical norms into 'smart contracts,' and the personal surveillance risks of audio beacon technology operating invisibly in our cellphones. Scholars and students from many backgrounds, as well as policy makers, journalists and the general reading public will find a multidisciplinary approach to questions posed by research in algorithmic ethics from the fields of Management, Sociology, Social Policy, Public Service, Religion and Interactive Media. (shrink)

Machine learning algorithms bring out an under-appreciated puzzle of discrimination, namely figuring out when a decision made on the basis of a factor correlated with race is a decision made on the basis of race. I argue that prevailing approaches, which are based on identifying and then distinguishing among causal effects of race, in their metaphysical timidity, fail to get off the ground. I suggest, instead, that adopting a constructivist theory of race answers this puzzle in a principled manner. On (...) what I call a “thick constructivist” account of race, to be raced is to be socially positioned in the way indicated by a certain set of statistical regularities on the basis of particular phenotypical traits. A thick constructivist sees that acting on the basis of correlations that constitute race qua social position just is acting on the basis of race, because races just are social positions that subject their member individuals to a particular matrix of social relations that define the raced position. This conclusion has considerable ramifications for our understanding of discrimination, algorithms and beyond. (shrink)

Situations in which patients lack medical decision-making (MDM) capacity raise ethical challenges, especially when the patients decline care that their surrogate decision makers and/or clinicians agree is indicated. These patients are a vulnerable population and should receive treatment that is the standard of care, in line with their the values of their authentic self, just as any other patient would. But forcing treatment on patients who refuse it, even though they lack capacity, carries medical and psychological risks to the patients (...) and the hospital staff. It is also often impractical to force some treatments, especially in the long term. For example, independent of the ethical “should” question, how would one force hemodialysis for the rest of a patient’s life, or force a surgery that requires weeks of post-operative physical therapy? In this article we present a novel algorithm that can help clinicians with ethical and practical decision making, with the goals of achieving the best outcomes for patients and reducing moral distress for their caretakers and clinicians. (shrink)

This paper has two aims. The first is to explain a type of wrong that arises when agents obscure responsibility for their actions. Call it “agency laundering.” The second is to use the concept of agency laundering to understand the underlying moral issues in a number of recent cases involving algorithmic decision systems. From the Proceedings of the 14th International Conference, iConference 2019, Washington D.C., March 31-April 3, 2019.

Predictive policing, the practice of using of algorithmic systems to forecast crime, is heralded by police departments as the new frontier of crime analysis. At the same time, it is opposed by civil rights groups, academics, and media outlets for being ‘biased’ and therefore discriminatory against communities of color. This paper argues that the prevailing focus on racial bias has overshadowed two normative factors that are essential to a full assessment of the moral permissibility of predictive policing: fairness in (...) the social distribution of the benefits and burdens of policing as well as the distinctive role of consent in determining fair distribution. When these normative factors are given their due attention, several requirements emerge for the fair implementation of predictive policing. Among these requirements are that police departments inform and solicit buy-in from affected communities about strategic decision-making and that departments favor non-enforcement-oriented interventions. (shrink)

We argue that an essential element of understanding the moral salience of algorithmic systems requires an analysis of the relation between algorithms and agency. We outline six key ways in which issues of agency, autonomy, and respect for persons can conflict with algorithmic decision-making.

Machine learning algorithms may radically improve our ability to diagnose and treat disease. For moral, legal, and scientific reasons, it is essential that doctors and patients be able to understand and explain the predictions of these models. Scalable, customisable, and ethical solutions can be achieved by working together with relevant stakeholders, including patients, data scientists, and policy makers.

Algorithms silently structure our lives. Algorithms can determine whether someone is hired, promoted, offered a loan, or provided housing as well as determine which political ads and news articles consumers see. Yet, the responsibility for algorithms in these important decisions is not clear. This article identifies whether developers have a responsibility for their algorithms later in use, what those firms are responsible for, and the normative grounding for that responsibility. I conceptualize algorithms as value-laden, rather than neutral, in that algorithms (...) create moral consequences, reinforce or undercut ethical principles, and enable or diminish stakeholder rights and dignity. In addition, algorithms are an important actor in ethical decisions and influence the delegation of roles and responsibilities within these decisions. As such, firms should be responsible not only for the value-laden-ness of an algorithm but also for designing who-does-what within the algorithmic decision. As such, firms developing algorithms are accountable for designing how large a role individual will be permitted to take in the subsequent algorithmic decision. Counter to current arguments, I find that if an algorithm is designed to preclude individuals from taking responsibility within a decision, then the designer of the algorithm should be held accountable for the ethical implications of the algorithm in use. (shrink)

Firms increasingly deploy algorithmic pricing approaches to determine what to charge for their goods and services. Algorithmic pricing can discriminate prices both dynamically over time and personally depending on individual consumer information. Although legal, the ethicality of such approaches needs to be examined as often they trigger moral concerns and sometimes outrage. In this research paper, we provide an overview and discussion of the ethical challenges germane to algorithmic pricing. As a basis for our discussion, we perform a systematic (...) interpretative review of 315 related articles on dynamic and personalized pricing as well as pricing algorithms in general. We then use this review to define the term algorithmic pricing and map its key elements at the micro-, meso-, and macro levels from a business and marketing ethics perspective. Thus, we can identify morally ambivalent topics that call for deeper exploration by future research. (shrink)