Connecting corporate software work in the United States and Germany, this essay tracks the racialization of mostly male Indian software engineers through the casualization of their labor. In doing so, I show the connections between overt, anti-immigrant violence today and the ongoing use of race to sediment divisions of labor in the industry as a whole. To explain racialization in the tech industry, I develop the concept of race-as-algorithm as a device to unpack how race is made productive within digital (...) economies and to show the flexibility of race as it works to create orders of classification that are sensitive to context. Using evidence collected through observation in tech offices and through interviews with programmers over five years, I track race as an essential but continually disavowed variable within the construction of global tech economies. Historical racializations of casual labor in plantation economies illuminates how casualness marks laborers whose rights can be muted and allows corporations to deny their culpability in promoting discrimination within and outside of the tech industry. These denials occur across a political field that divides “good” from “bad” migrants. Using the ethnographic symptoms that Indian tech workers identify in their environments, this essay reads these signs as an antidote to these continued denials. (shrink)

In this APA blog, I appeal to two 2020 cases of algorithms gone wrong to motivate philosophical attention to algorithmic oppression. I offer a simple definition, then describe a few of the ways it is engendered. References and extends work by Safiya Noble, Cathy O'Neil, Ruha Benjamin, Virginia Eubanks, Sara Wachter-Boettcher, Michael Kearns & Aaron Roth.

This paper contributes to debates over algorithmic discrimination with particular attention to structural theories of racism and the problem of “proxy discrimination”—discriminatory effects that arise even when an algorithm has no information about socially sensitive characteristics such as race. Structural theories emphasize the ways that unequal power structures contribute to the subordination of marginalized groups: these theories thus understand racism in ways that go beyond individual choices and bad intentions. Our question is, how should a structural understanding (...) of racism and oppression inform our understanding of algorithmic discrimination and its associated norms? Some responses to the problem of proxy discrimination focus on fairness as a form of “parity,” aiming to equalize metrics between individuals or groups—looking, for example, for equal rates of accurate and inaccurate predictions between one group and another. We argue that from the perspective of structural theories, fairness-as-parity is inapt in the algorithmic context; instead, we should be considering social impact—whether a use of an algorithm perpetuates or mitigates existing social stratification. Our contribution thus offers a new understanding of what algorithmic racial discrimination is. (shrink)

The purpose of cultural competence education for medical professionals is to ensure respectful care and reduce health disparities. Yet as Berger and Miller (2021) show, the cultural competence framework is dated, confused, and self-defeating. They argue that the framework ignores the primary driver of health disparities—systemic racism—and is apt to exacerbate rather than mitigate bias and ethnocentrism. They propose replacing cultural competence with a framework that attends to two social aspects of structural inequality: health and social policy, and institutional-system (...) activity; and two psychological aspects of structural inequality: the clinical encounter, and the epistemic. -/- We agree with the structural approach. To that end, we think it would be fruitful to include attention to physical contributors to structural inequality, namely the material artifacts used in medicine. Devices, tools, and technologies can materialize biases, perpetuate oppression, and contribute to health disparities. Granted, not everything that interests philosophers can be squeezed into medical education. Nevertheless, there are compelling reasons for including the study of material artifacts in education designed to reduce health disparities. First, devices and tools often carry forward biases from the past, and keep biases hidden from plain sight. Second, by studying these artifacts, future clinicians can begin to see themselves as part of a larger sociotechnical system. Finally, as medicine becomes increasingly tech-laden, it’s important for clinicians to see how material artifacts (including algorithms) connect individuals to structures. This will help to undermine oversimplified narratives according to which objective tools and technologies can correct for the bias and subjectivity of flawed human beings. (shrink)

The power of algorithms has become a familiar topic in society, media, and the social sciences. It is increasingly common to argue that, for instance, algorithms automate inequality, that they are biased black boxes that reproduce racism, or that they control our money and information. Implicit in many of these discussions is that algorithms are permeated with normativities, and that these normativities shape society. The aim of this editorial is double: First, it contributes to a more nuanced discussion about (...) algorithms by discussing how we, as social scientists, think about algorithms in relation to five theoretical ideal types. For instance, what does it mean to go under the hood of the algorithm and what does it mean to stay above it? Second, it introduces the contributions to this special theme by situating them in relation to these five ideal types. By doing this, the editorial aims to contribute to an increased analytical awareness of how algorithms are theorized in society and culture. The articles in the special theme deal with algorithms in different settings, ranging from farming, schools, and self-tracking to AIDS, nuclear power plants, and surveillance. The contributions thus explore, both theoretically and empirically, different settings where algorithms are intertwined with normativities. (shrink)

A growing number of studies on fairness in artificial intelligence (AI) use the notion of intersectionality to measure AI fairness. Most of these studies take intersectional fairness to be a matter of statistical parity among intersectional subgroups: an AI algorithm is “intersectionally fair” if the probability of the outcome is roughly the same across all subgroups defined by different combinations of the protected attributes. This paper identifies and examines three fundamental problems with this dominant interpretation of intersectional fairness in AI. (...) First, the dominant approach is so preoccupied with the intersection of attributes/categories (e.g., race, gender) that it fails to address the intersection of oppression (e.g., racism, sexism), which is more central to intersectionality as a critical framework. Second, the dominant approach faces a dilemma between infinite regress and fairness gerrymandering: it either keeps splitting groups into smaller subgroups or arbitrarily selects protected groups. Lastly, the dominant view fails to capture what it really means for AI algorithms to be fair, in terms of both distributive and non-distributive fairness. I distinguish a strong sense of AI fairness from a weak sense that is prevalent in the literature, and conclude by envisioning paths towards strong intersectional fairness in AI. (shrink)

Diversity is often announced as a solution to ethical problems in artificial intelligence (AI), but what exactly is meant by diversity and how it can solve those problems is seldom spelled out. This lack of clarity is one hurdle to motivating diversity in AI. Another hurdle is that while the most common perceptions about what diversity is are too weak to do the work set out for them, stronger notions of diversity are often defended on normative grounds that fail to (...) connect to the values that are important to decision-makers in AI. However, there is a long history of research in feminist philosophy of science and a recent body of work in social epistemology that taken together provide the foundation for a notion of diversity that is both strong enough to do the work demanded of it, and can be defended on epistemic grounds that connect with the values that are important to decision-makers in AI. We clarify and defend that notion here by introducing emergent expertise as a network phenomenon wherein groups of workers with expertise of different types can gain knowledge not available to any individual alone, as long as they have ways of communicating across types of expertise. We illustrate the connected epistemic and ethical benefits of designing technology with diverse groups of workers using the examples of an infamous racist soap dispenser, and the millimeter wave scanners used in US airport security. (shrink)

Uncovering the world’s ethnic inequalities is hampered by a lack of ethnicity-annotated datasets. Name-ethnicity classifiers (NECs) can help, as they are able to infer people’s ethnicities from their names. However, since the latest generation of NECs rely on machine learning and artificial intelligence (AI), they may suffer from the same racist and sexist biases found in many AIs. Therefore, this paper offers an algorithmic fairness audit of three NECs. It finds that the UK-Census-trained EthnicityEstimator displays large accuracy biases with (...) regards to ethnicity, but relatively less among gender and age groups. In contrast, the Twitter-trained NamePrism and the Wikipedia-trained Ethnicolr are more balanced among ethnicity, but less among gender and age. We relate these biases to global power structures manifested in naming conventions and NECs’ input distribution of names. To improve on the uncovered biases, we program a novel NEC, N2E, using fairness-aware AI techniques. We make N2E freely available at www.name-to-ethnicity.com. (shrink)

Addressing biases in natural-language processing (NLP) systems presents an underappreciated ethical dilemma, which we think underlies recent debates about bias in NLP models. In brief, even if we could eliminate bias from language models or their outputs, we would thereby often withhold descriptively or ethically useful information, despite avoiding perpetuating or amplifying bias. Yet if we do not debias, we can perpetuate or amplify bias, even if we retain relevant descriptively or ethically useful information. Understanding this dilemma provides for a (...) useful way of rethinking the ethics of algorithmic bias in NLP. (shrink)

In this article, I present a critique of Robert Geraci's Apocalyptic artificial intelligence (AI) discourse, drawing attention to certain shortcomings which become apparent when the analytical lens shifts from religion to the race–religion nexus. Building on earlier work, I explore the phenomenon of existential risk associated with Apocalyptic AI in relation to “White Crisis,” a modern racial phenomenon with premodern religious origins. Adopting a critical race theoretical and decolonial perspective, I argue that all three phenomena are entangled and they should (...) be understood as a strategy, albeit perhaps merely rhetorical, for maintaining white hegemony under nonwhite contestation. I further suggest that this claim can be shown to be supported by the disclosure of continuity through change in the long‐durée entanglement of race and religion associated with the establishment, maintenance, expansion, and refinement of the modern/colonial world system if and when such phenomena are understood as iterative shifts in a programmatic trajectory of domination which might usefully be framed as “algorithmic racism.”. (shrink)

As digital labour becomes more widespread across the uneven geographies of race, gender, class and ability, and as histories of colonialism and inequality get drawn into these forms of labour, our imagination of what these worlds contain similarly needs to expand. Beyond the sensationalist images of the ‘brogrammer’ and the call-centre worker lie intersecting labour practices that bring together histories of bodies and materiality in new ways. In the recent past, these entanglements have yielded oppressive results. As scandals over predictive (...) policing, data mining and algorithmic racism unfold, digital labourers need both to be accounted for in analyses of algorithmic technologies and to be counted among the designers of these platforms. This article attempts to do both of these by highlighting particular cases in which digital labour frames embodied subjects, and to propose ways digital workers might train themselves to recognise ethical problems as they are emerging. I use the idea of attunements as a way to grasp what these forms of care might look like for the digital worker. (shrink)

In recent months, Covid‐19 has devastated African American communities across the nation, and a Minneapolis police officer murdered George Floyd. The agents of death may be novel, but the phenomena of long‐standing epidemics of premature black death and of police violence are not. This essay argues that racial health and health care disparities, rooted as they are in systemic injustice, ought to carry far more weight in clinical ethics than they generally do. In particular, this essay examines palliative and end‐of‐life (...) care for African Americans, highlighting the ways in which American medicine, like American society, has breached trust. In the experience of many African American patients struggling against terminal illness, health care providers have denied them a say in their own medical decision‐making. In the midst of the Covid‐19 pandemic, African Americans have once again been denied a say with regard to the rationing of scarce medical resources such as ventilators, in that dominant and ostensibly race‐neutral algorithms sacrifice black lives. Is there such thing as a “good” or “dignified” death when African Americans are dying not merely of Covid‐19 but of structural racism? (shrink)

How should we think about the ways search engines can go wrong? Following the publication of Safiya Noble’s Algorithms of Oppression (Noble 2018), a view has emerged that racist, sexist, and other problematic results should be thought of as indicative of algorithmic bias. In this paper, I offer an alternative angle on these results, building on Noble’s suggestion that search engines are complicit in a racial contract (Mills 1990). I argue that racist and sexist results should be thought of (...) as part of the workings of the social system of white ignorance. Along the way, I will argue that we should think about search engines not as sources of testimony, but as information-classification systems, and make a preliminary case for the importance of the social epistemology of technology. (shrink)

Artificial Intelligence (AI) is increasingly being used in disaster risk management applications to predict the effect of upcoming disasters, plan for mitigation strategies, and determine who needs how much aid after a disaster strikes. The media is filled with unintended ethical concerns of AI algorithms, such as image recognition algorithms not recognizing persons of color or racist algorithmic predictions of whether offenders will recidivate. We know such unintended ethical consequences must play a role in DRM as well, yet there (...) is surprisingly little research on exactly what the unintended consequences are and what we can do to mitigate them. The aim of this perspective is to call researchers working on fairness, accountability, and transparency to work with DRM and local experts—so we can ensure that disaster mitigation and relief is accountable, considers local values, and is not unintentionally biased. (shrink)

This article is a critical genealogy of Tay, an artificial-intelligence chatbot that Microsoft released on Twitter in 2016, which was quickly hijacked by internet trolls to reproduce racist, misogynist, and antisemitic language. Tay’s repetition and production of hate speech calls for an approach that draws on both media and cultural theory—the Frankfurt School’s dialectical analyses of language and ideology, in particular. Revisiting the Frankfurt School in the age of algorithmic reason shows that, contrary to views foundational to computing, a (...) neural-network chatbot like Tay does not sidestep meaning but rather carries and alters it, with unforeseen social and political consequences. A return to the work of Max Horkheimer and Theodor W. Adorno thus locates ideology in the digital world at the nexus of language’s ability to mean, language and meaning’s susceptibility to computation, and the design of a machine to compute both. Coming to critical terms with the antisemitism produced in Tay’s human-computer synthesis requires, as this article contends, addressing the uncanny embodiment and reflection of thought that is digital computation. (shrink)

Recently, there has been an upsurge of attention focused on bias and its impact on specialized artificial intelligence applications. Allegations of racism and sexism have permeated the conversation as stories surface about search engines delivering job postings for well-paying technical jobs to men and not women, or providing arrest mugshots when keywords such as “black teenagers” are entered. Learning algorithms are evolving; they are often created from parsing through large datasets of online information while having truth labels bestowed on (...) them by crowd-sourced masses. These specialized AI algorithms have been liberated from the minds of researchers and startups, and released onto the public. Yet intelligent though they may be, these algorithms maintain some of the same biases that permeate society. They find patterns within datasets that reflect implicit biases and, in so doing, emphasize and reinforce these biases as global truth. This paper describes specific examples of how bias has infused itself into current AI and robotic systems, and how it may affect the future design of such systems. More specifically, we draw attention to how bias may affect the functioning of a robot peacekeeper, a self-driving car, and a medical robot. We conclude with an overview of measures that could be taken to mitigate or halt bias from permeating robotic technology. (shrink)

If good ethics is the process of ongoing dialogical deliberation on basic normative questions for the purpose of instituting principles for action, then the COVID crisis, or any crisis, is not a good time for developing ethical precepts on the run. Given dominant ethical trends, such reactive ethics tends to lead to either individualized struggles over the right way to act or hasty sets of guidelines that leave out contextualizing questions concerning regimes of care. Good people will find themselves suggesting (...) strange things, from setting up lifeboat scenarios to supporting structural racism. This essay argues against both these paths—crisis-ridden agonism or algorithmic resource-allocation—and turns instead to a form of ethics of care which takes its departure from older forms of ethics, while recognizing that modern and postmodern challenges no longer allow their grounding in animated relations, natural rights, or cosmological truths. (shrink)

Racial capitalism depends on the reproduction of an existing racialized economic order. In this article, I argue that the disavowal of past injustice is a central way in which this reproduction is ensured and that market-based forms of knowledge production, such as for-profit predictive practices, play a crucial role in facilitating this disavowal. Recent debates about the fairness of algorithms, data justice, and predictive policing have intensified long-standing controversies, both popular and academic, about the way in which statistical and financial (...) modes of accounting and predicting articulate, represent and produce ascriptive categories of hierarchically ordered social difference, and reproduce unjust social hierarchies and inequalities. These debates have productively problematized the racial lives of seemingly apolitical predictive technologies and demanded the re-politicization of predictive practices. What has been missing from these debates so far, however, is a more explicit engagement with ways in which anti-racist movements and activists themselves have contested the entanglements of prediction and race making. I turn to a recent prominent example, namely the contestation over racial discrepancies in subprime lending to examine how fair lending activists have conceptualized and troubled the reproduction of a racialized economic order through for-profit predictive practices in the decade before the Great Financial Crisis. I situate this particular example in the broader historical and political context of politicizing prediction that first emerged with the ascendancy of a liberal, individualist-proprietary conception of risk, and the political problem space to which this has given rise. My analysis shows that actuarial conceptions of fairness continue to reverberate in anti-racist contestations of for-profit predictive practices, and that they tend to marginalize and undercut more radical strands of critique of the racialization of financial markets. Insofar as these modalities of contestation implicitly reproduce a liberal, proprietary-individualist conception of risk, I argue, they fail to effectively challenge the quasi-alchemical transformation of injustice into personal responsibility, and thus contribute to the disavowal of past injustice. (shrink)

Endre Begby’s Prejudice: A Study in Non-Ideal Epistemology engages a wide range of issues of enduring interest to epistemologists, applied ethicists, and anyone concerned with how knowledge and justice intersect. Topics include stereotypes and generics, evidence and epistemic justification, epistemic injustice, ethical-epistemic dilemmas, moral encroachment, and the relations between blame and accountability. Begby applies his views about these topics to an equally wide range of pressing social questions, such as conspiracy theories, misinformation, algorithmic bias, discrimination, and criminal justice. Through (...) it all, the book’s central thesis is that prejudices can be epistemically rational, a corrective against what Begby takes to be the received view that prejudices are always and everywhere bad. However, Begby’s arguments do not engage consistently with relevant empirical literatures, misrepresent the positions of his interlocutors, and rehearse ideas already well-established across a range of intellectual traditions. (shrink)

This essay diagnoses systemic interconnections between COVID-19 pandemics, anti-Black racism, and the intensification of digital capitalism. By drawing on Charles Mills’ rectificatory justice and Hannah Arendt’s reflections on understanding and action, it argues that the role of philosophy lies in safeguarding racial justice and understanding against the hegemony algorithmic governmentality.

A movement asking to take race out of medicine is growing in the US. While we agree with the necessity to get rid of flawed assumptions about biological race that pervade automatic race correction in medical algorithms, we urge caution about insisting on a blanket eliminativism about race in medicine. If we look at racism as a fundamental cause, in the sense that this notion has been introduced in epidemiological studies by Bruce Link and Jo Phelan, we must conclude (...) that race is indispensable to consider, investigate, and denounce the health effects of multilevel racism, and cannot be eliminated by addressing more specific risk factors in socially responsible epidemiology and clinical medicine. This does not mean that realism about human races is vindicated. While maintaining that there are no human races, we show how it is that a non-referring concept can nonetheless turn out indispensable for explaining real phenomena. (shrink)

This article presents Epistemic Spatialization as a new framework for investigating the interconnected patterns of biases when identifying objects with convolutional neural networks. It draws upon Foucault’s notion of spatialized knowledge to guide its method of enquiry. We argue that decisions involved in the creation of algorithms, alongside the labeling, ordering, presentation, and commercial prioritization of objects, together create a distorted “nomination of the visible”: they harden the visibility of some objects, make other objects excessively visible, and consign yet others (...) to permanent or haphazard invisibility. Our approach differs from those who focus on high-stakes misidentifications, such as errors tied to structural racism. Examining the far more dominant series of low-stakes mistakes shows the scope of errors, destabilizing the goal of image content identification with considerable societal impact. We explore these issues by closely examining the demonstration video of a popular convnet. This examination reveals an interlocking series of biases undermining the content identification process. The picture we paint is crucial for a better understanding of the errors that result as these convnets become further embedded in everyday products. The framework is valuable for critical work on computer vision, AI studies, and large-scale visual analysis. (shrink)

Should decision-making algorithms be held to higher standards of transparency than human beings? The way we answer this question directly impacts what we demand from explainable algorithms, how we govern them via regulatory proposals, and how explainable algorithms may help resolve the social problems associated with decision making supported by artificial intelligence. Some argue that algorithms and humans should be held to the same standards of transparency and that a double standard of transparency is hardly justified. We give two arguments (...) to the contrary and specify two kinds of situations for which higher standards of transparency are required from algorithmic decisions as compared to humans. Our arguments have direct implications on the demands from explainable algorithms in decision-making contexts such as automated transportation. (shrink)

Racism, sexism, and other forms of injustice are more than just bad attitudes; after all, such injustice involves unfair distributions of goods and resources. But attitudes play a role. How central is that role? Tommie Shelby, among others, argues that racism is an ideology and takes a cognitivist approach suggesting that ideologies consist in false beliefs that arise out of and serve pernicious social conditions. In this paper I argue that racism is better understood as a set (...) of practices, attitudes, social meanings, and material conditions, that systematically reinforce one another. Attitudes play a role, but even the cognitive/affective component of ideologies should include culturally shared habits of mind and action. These habits of mind distort, obscure, and occlude important facts about subordinated groups and result in a failure to recognize their interests. How do we disrupt such practices to achieve greater justice? I argue that this is sometimes, but not always, best achieved by argument or challenging false beliefs, so social movements legitimately seek other means. (shrink)

Algorithms wield increasing control over our lives—over the jobs we get, the loans we're granted, the information we see online. Algorithms can and often do wield their power in a biased way, and much work has been devoted to algorithmic bias. In contrast, algorithmic neutrality has been largely neglected. I investigate algorithmic neutrality, tackling three questions: What is algorithmic neutrality? Is it possible? And when we have it in mind, what can we learn about algorithmic (...) bias? (shrink)

In this paper, I develop an algorithmic impossible-worlds model of belief and knowledge that provides a middle ground between models that entail that everyone is logically omniscient and those that are compatible with even the most egregious kinds of logical incompetence. In outline, the model entails that an agent believes (knows) φ just in case she can easily (and correctly) compute that φ is true and thus has the capacity to make her actions depend on whether φ. The model (...) thereby captures the standard view that belief and knowledge ground are constitutively connected to dispositions to act. As I explain, the model improves upon standard algorithmic models developed by Parikh, Halpern, Moses, Vardi, and Duc, among other ways, by integrating them into an impossible-worlds framework. The model also avoids some important disadvantages of recent candidate middle-ground models based on dynamic epistemic logic or step logic, and it can subsume their most important advantages. (shrink)

In a recent article Duncan Purves, Ryan Jenkins, and B. J. Strawser argue that in order for a decision in war to be just, or indeed the decision to resort to war to be just, it must be the case that the decision is made for the right reasons. Furthermore, they argue that this requirement holds regardless of how much good is produced by said action. In this essay I argue that their argument is flawed, in that it mistakes what (...) makes an agent morally good for what makes an act morally good. I argue that the main thrust of Purves et al.’s argument in fact undermines the conclusion they wish to draw, and that the reasons for one’s action do not make an in principle difference to the morality of actions in war. I further argue that this position undermines the traditional ad bellum just war constraint of right intention, and that the morality of actions in war is, at core, only concerned with outcomes. I conclude by clarifying that one’s reasons for action do in fact matter when deciding to enter war or kill in war, but only because one’s reasons significantly impact the way in which one acts. The purpose of this paper is to clear the theoretical space by showing why intentions/reasons do not in principle matter when assessing the morality of war (or killing), but this should not be taken as an argument that we should ignore intentions/reasons altogether. (shrink)

It is well-established that algorithms can be instruments of injustice. It is less frequently discussed, however, how current modes of AI deployment often make the very discovery of injustice difficult, if not impossible. In this article, we focus on the effects of algorithmic profiling on epistemic agency. We show how algorithmic profiling can give rise to epistemic injustice through the depletion of epistemic resources that are needed to interpret and evaluate certain experiences. By doing so, we not only (...) demonstrate how the philosophical conceptual framework of epistemic injustice can help pinpoint potential, systematic harms from algorithmic profiling, but we also identify a novel source of hermeneutical injustice that to date has received little attention in the relevant literature: epistemic fragmentation. Epistemic fragmentation is a structural characteristic of algorithmically-mediated environments that isolate individuals, making it more difficult to develop, uptake and apply new epistemic resources. This, in turn, can impede the identification and conceptualisation of emerging harms in these environments. We trace the occurrence of hermeneutical injustice back to the fragmentation of the epistemic experiences of individuals, who are left more vulnerable by the inability to share, compare, and learn from shared experiences. (shrink)

Machine intelligence already helps medical staff with a number of tasks. Ethical decision-making, however, has not been handed over to computers. In this proof-of-concept study, we show how an algorithm based on Beauchamp and Childress’ prima-facie principles could be employed to advise on a range of moral dilemma situations that occur in medical institutions. We explain why we chose fuzzy cognitive maps to set up the advisory system and how we utilized machine learning to train it. We report on the (...) difficult task of operationalizing the principles of beneficence, non-maleficence and patient autonomy, and describe how we selected suitable input parameters that we extracted from a training dataset of clinical cases. The first performance results are promising, but an algorithmic approach to ethics also comes with several weaknesses and limitations. Should one really entrust the sensitive domain of clinical ethics to machine intelligence? (shrink)

The Fairness, Accountability, and Transparency (FAccT) literature tends to focus on bias as a problem that requires ex post solutions (e.g. fairness metrics), rather than addressing the underlying social and technical conditions that (re)produce it. In this article, we propose a complementary strategy that uses genealogy as a constructive, epistemic critique to explain algorithmic bias in terms of the conditions that enable it. We focus on XAI feature attributions (Shapley values) and counterfactual approaches as potential tools to gauge these (...) conditions and offer two main contributions. One is constructive: we develop a theoretical framework to classify these approaches according to their relevance for bias as evidence of social disparities. We draw on Pearl’s ladder of causation (Causality: models, reasoning, and inference. Cambridge University Press, Cambridge, 2000, Causality, 2nd edn. Cambridge University Press, Cambridge, 2009. doi:10.1017/CBO9780511803161) to order these XAI approaches concerning their ability to answer fairness-relevant questions and identify fairness-relevant solutions. The other contribution is critical: we evaluate these approaches in terms of their assumptions about the role of protected characteristics in discriminatory outcomes. We achieve this by building on Kohler-Hausmann’s (Northwest Univ Law Rev 113(5):1163–1227, 2019) constructivist theory of discrimination. We derive three recommendations for XAI practitioners to develop and AI policymakers to regulate tools that address algorithmic bias in its conditions and hence mitigate its future occurrence. (shrink)

Algorithms influence every facet of modern life: criminal justice, education, housing, entertainment, elections, social media, news feeds, work… the list goes on. Delegating important decisions to machines, however, gives rise to deep moral concerns about responsibility, transparency, freedom, fairness, and democracy. Algorithms and Autonomy connects these concerns to the core human value of autonomy in the contexts of algorithmic teacher evaluation, risk assessment in criminal sentencing, predictive policing, background checks, news feeds, ride-sharing platforms, social media, and election interference. Using (...) these case studies, the authors provide a better understanding of machine fairness and algorithmic transparency. They explain why interventions in algorithmic systems are necessary to ensure that algorithms are not used to control citizens' participation in politics and undercut democracy. This title is also available as Open Access on Cambridge Core. (shrink)

Machine learning algorithms are increasingly used to shape high-stake allocations, sparking research efforts to orient algorithm design towards ideals of justice and fairness. In this research on algorithmic fairness, normative theorizing has primarily focused on identification of “ideally fair” target states. In this paper, we argue that this preoccupation with target states in abstraction from the situated dynamics of deployment is misguided. We propose a framework that takes dynamic trajectories as direct objects of moral appraisal, highlighting three respects in (...) which such trajectories can be subject to evaluation in relation to their temporal dynamics, robustness, and representation. (shrink)

As algorithms have become ubiquitous in consequential domains, societal concerns about the potential for discriminatory outcomes have prompted urgent calls to address algorithmic bias. In response, a rich literature across computer science, law, and ethics is rapidly proliferating to advance approaches to designing fair algorithms. Yet computer scientists, legal scholars, and ethicists are often not speaking the same language when using the term ‘bias.’ Debates concerning whether society can or should tackle the problem of algorithmic bias are hampered (...) by conflations of various understandings of bias, ranging from neutral deviations from a standard to morally problematic instances of injustice due to prejudice, discrimination, and disparate treatment. This terminological confusion impedes efforts to address clear cases of discrimination. -/- In this paper, we examine the promises and challenges of different approaches to disambiguating bias and designing for justice. While both approaches aid in understanding and addressing clear algorithmic harms, we argue that they also risk being leveraged in ways that ultimately deflect accountability from those building and deploying these systems. Applying this analysis to recent examples of generative AI, our argument highlights unseen dangers in current methods of evaluating algorithmic bias and points to ways to redirect approaches to addressing bias in generative AI at its early stages in ways that can more robustly meet the demands of justice. (shrink)

This paper argues that racism should not only be conceived as a moral concept whose main aim is to condemn severe wrongs in the domain of race. The paper advances a complementary interpretation of racism as an explanatory concept--one that plays a key role in explaining race-based social problems afflicting members of subordinate racialized groups. As an explanatory concept, the term 'racism' is used to diagnose and highlight the causes of race-related social problems. The project of diagnosing (...) race-based social problems contributes to the pragmatic anti-racist end of developing better political and policy strategies for solving these social problems. The paper defends this interpretation of racism as an explanatory concept through a critical engagement with Urquidez's moral-philosophical account of racism. (shrink)

Precision medicine is an emerging approach to individualized care. It aims to help physicians better comprehend and predict the needs of their patients while effectively adopting in a timely manner the most suitable treatment by promoting the sharing of health data and the implementation of learning healthcare systems. Alongside its promises, PM also entails the risk of exacerbating healthcare inequalities, in particular between ethnoracial groups. One often-neglected underlying reason why this might happen is the impact of structural racism on (...) PM initiatives. Raising awareness as to how structural racism can influence PM initiatives is paramount to avoid that PM ends up reproducing the pre-existing health inequalities between different ethnoracial groups and contributing to the loss of trust in healthcare by minority groups. We analyse three nodes of a process flow where structural racism can affect PM’s implementation. These are: the collection of biased health data during the initial encounter of minority groups with the healthcare system and researchers, the integration of biased health data for minority groups in PM initiatives and the influence of structural racism on the deliverables of PM initiatives for minority groups. We underscore that underappreciation of structural racism by stakeholders involved in the PM ecosystem can be at odds with the ambition of ensuring social and racial justice. Potential specific actions related to the analysed nodes are then formulated to help ensure that PM truly adheres to the goal of leaving no one behind, as endorsed by member states of the United Nations for the 2030 Agenda for Sustainable Development. Structural racism has been entrenched in our societies for centuries and it would be naïve to believe that its impacts will not spill over in the era of PM. PM initiatives need to pay special attention to the discriminatory and harmful impacts that structural racism could have on minority groups involved in their respective projects. It is only by acknowledging and discussing the existence of implicit racial biases and trust issues in healthcare and research domains that proper interventions to remedy them can be implemented. (shrink)

An analysis of 'racism' in terms of disrespect. This article argues against the views that racism should be understood in reductive ways as, variously, an attitude of ill-will (Jorge Garcia), a cognitive object such as ideology (Tommie Shelby), a behavior (Michael Philips), or some disjunctive hybrid (Lawrence Blum). In fact, it argues that racism should be conceptually released from having any one location. The disrespect analysis favored here can accommodate a variety of important desiderata for a theory (...) of racism. (shrink)

Inspired by recent breakthroughs in predictive modeling, practitioners in both industry and government have turned to machine learning with hopes of operationalizing predictions to drive automated decisions. Unfortunately, many social desiderata concerning consequential decisions, such as justice or fairness, have no natural formulation within a purely predictive framework. In efforts to mitigate these problems, researchers have proposed a variety of metrics for quantifying deviations from various statistical parities that we might expect to observe in a fair world and offered a (...) variety of algorithms in attempts to satisfy subsets of these parities or to trade o the degree to which they are satised against utility. In this paper, we connect this approach to fair machine learning to the literature on ideal and non-ideal methodological approaches in political philosophy. The ideal approach requires positing the principles according to which a just world would operate. In the most straightforward application of ideal theory, one supports a proposed policy by arguing that it closes a discrepancy between the real and the perfectly just world. However, by failing to account for the mechanisms by which our non-ideal world arose, the responsibilities of various decision-makers, and the impacts of proposed policies, naive applications of ideal thinking can lead to misguided interventions. In this paper, we demonstrate a connection between the fair machine learning literature and the ideal approach in political philosophy, and argue that the increasingly apparent shortcomings of proposed fair machine learning algorithms reflect broader troubles faced by the ideal approach. We conclude with a critical discussion of the harms of misguided solutions, a reinterpretation of impossibility results, and directions for future research. (shrink)

Some people of colour feel shame in response to racist incidents. This phenomenon seems puzzling since, plausibly, they have nothing to feel shame about. This puzzle arises because we assume that targets of racism feel shame about their race. However, I propose that when an individual is racialised as non-White in a racist incident, shame is sometimes prompted, not by a negative self-assessment of her race, but by her inability to choose when her stigmatised race is made salient. I (...) argue that this can make sense of some shame responses to racism. My account also helps to highlight some of the emotional and cognitive costs of racism that have their root in shame as well as a new form of hermeneutical injustice and distinctive communicative harms, contributing to a fuller picture of what is objectionable about racism. (shrink)

Algorithms can now identify patterns and correlations in the (big) datasets, and predict outcomes based on those identified patterns and correlations with the use of machine learning techniques and big data, decisions can then be made by algorithms themselves in accordance with the predicted outcomes. Yet, algorithms can inherit questionable values from the datasets and acquire biases in the course of (machine) learning, and automated algorithmic decision-making makes it more difficult for people to see algorithms as biased. While researchers (...) have taken the problem of algorithmic bias seriously, but the current discussion on algorithmic fairness tends to conceptualize ‘fairness’ in algorithmic fairness primarily as a technical issue and attempts to implement pre-existing ideas of ‘fairness’ into algorithms. In this paper, I show that such a view of algorithmic fairness as technical issue is unsatisfactory for the type of problem algorithmic fairness presents. Since decisions on fairness measure and the related techniques for algorithms essentially involve choices between competing values, ‘fairness’ in algorithmic fairness should be conceptualized first and foremost as a political issue, and it should be (re)solved by democratic communication. The aim of this paper, therefore, is to explicitly reconceptualize algorithmic fairness as a political question and suggest the current discussion of algorithmic fairness can be strengthened by adopting the accountability for reasonableness framework. (shrink)

Data‐driven algorithms are widely used to make or assist decisions in sensitive domains, including healthcare, social services, education, hiring, and criminal justice. In various cases, such algorithms have preserved or even exacerbated biases against vulnerable communities, sparking a vibrant field of research focused on so‐called algorithmic biases. This research includes work on identification, diagnosis, and response to biases in algorithm‐based decision‐making. This paper aims to facilitate the application of philosophical analysis to these contested issues by providing an overview of (...) three key topics: What is algorithmic bias? Why and how can it occur? What can and should be done about it? Throughout, we highlight connections—both actual and potential—with philosophical ideas and concerns. (shrink)

In this paper, I engage with the debate between John Rawls and Charles Mills. In the first part, relevant works by Rawls and Mills are mainly examined. To this end, I first begin by examining Rawls’s ideal theory of justice and its relevance to the issue of racism. I then consider Mills’s non-ideal critique of Rawls and supplement it with the help of the notion of social norms. Whereas Rawls’s view can deal with racial injustice as discrimination, in my (...) view, I argue that Mills’s theory is mostly concerned with racist injustice as oppression. Between racial injustice as discrimination and racist injustice as oppression, however, there are varying degrees of race-related wrongs, which can be elucidated by harmful social norms. As a result, I conclude that Mills’s point can be shown without supposing that the Rawlsian basic structure is inherently racist. (shrink)

Algorithmic communications pose several challenges to democracy. The three phenomena of filtering, hypernudging, and microtargeting can have the effect of polarizing an electorate and thus undermine the deliberative potential of a democratic society. Algorithms can spread fake news throughout the society, undermining the epistemic potential that broad participation in democracy is meant to offer. They can pose a threat to political equality in that some people may have the means to make use of algorithmic communications and the sophistication (...) to be immune from attempts at manipulation, while other people are vulnerable to manipulation by those who use these means. My concern here is with the danger that algorithmic communications can pose to political equality, which arises because most citizens must make decisions about what and who to support in democratic politics with only a sparse budget of time, money, and energy. Algorithmic communications such as hypernudging and microtargeting can be a threat to democratic participation when persons are operating in environments that do not conduce to political sophistication. This constitutes a deepening of political inequality. The political sophistication necessary to counter this vulnerability is rooted for many in economic life and it can and ought to be enhanced by changing the terms of economic life. (shrink)

This is a short reply to Dan Demetriou's "Ashes of Our Fathers: Racist Monuments and the Tribal Right." Both are included in Oxford University Press's Ethics, Left and Right: The Moral Issues That Divide Us.

The words ‘racist’ and ‘racism’ have become so overused that they nowconstitute obstacles to understanding and interracial dialogue about racial matters. Insteadof the current practice of referring to virtually anything that goes wrong or amiss withrespect to race as ‘racism,’ we should recognize a much broader moral vocabulary forcharacterizing racial ills – racial insensitivity, racial ignorance, racial injustice, racialdiscomfort, racial exclusion. At the same time, we should fix on a definition of ‘racism’ thatis continuous with its historical (...) usage, and avoids conceptual inflation. I suggest two basic,and distinct, forms of racism that meet this condition – antipathy racism and inferiorizingracism. We should also recognize that not all racially objectionable actions are done froma racist motive, and that not all racial stereotypes are racist. (shrink)

Racist jokes are often funny. And part of this has to do with their racism. Many Polish jokes, for example, may easily be converted into moron jokes but are not at all funny when delivered as such. Consider two answers to ‘What has an I.Q. of 1007’: a nation of morons; or Poland. Similarly, jokes portraying Jews as cheap, Italians as cowards, and Greeks as dishonest may be told as jokes about how skinflints, cowards, or dishonest people get on (...) in the world. But they are less funny as such. As this suggests, racist humor is ‘put down’ humor. We laugh, in part, because we find put-downs funny, sometimes even if they are about us. In many contexts, this tendency is relatively harmless; indeed, within reason, it may be therapeutic to join others in a good laugh at oneself. Why, then, all the commotion about racist humor? (shrink)

If racism is a matter of possessing racist beliefs, then it would seem that its cure involves purging one’s mind of all racist beliefs. But the truth is more complicated, and does not permit such a straightforward strategy. Racist beliefs are resistant to subjective repudiation, and even those that are so repudiated are resistant to lasting expulsion from one’s belief system. Moreover, those that remain available for use in cognition can shape thought and behavior even in the event that (...) one has recognized their falsehood. Yet if one is intent upon combating the racism within one’s mind, one is not without effective cognitive countermeasures that can render one’s racist beliefs ineffectual. (shrink)

As government pressure on major technology companies builds, both firms and legislators are searching for technical solutions to difficult platform governance puzzles such as hate speech and misinformation. Automated hash-matching and predictive machine learning tools – what we define here as algorithmic moderation systems – are increasingly being deployed to conduct content moderation at scale by major platforms for user-generated content such as Facebook, YouTube and Twitter. This article provides an accessible technical primer on how algorithmic moderation works; (...) examines some of the existing automated tools used by major platforms to handle copyright infringement, terrorism and toxic speech; and identifies key political and ethical issues for these systems as the reliance on them grows. Recent events suggest that algorithmic moderation has become necessary to manage growing public expectations for increased platform responsibility, safety and security on the global stage; however, as we demonstrate, these systems remain opaque, unaccountable and poorly understood. Despite the potential promise of algorithms or ‘AI’, we show that even ‘well optimized’ moderation systems could exacerbate, rather than relieve, many existing problems with content policy as enacted by platforms for three main reasons: automated moderation threatens to further increase opacity, making a famously non-transparent set of practices even more difficult to understand or audit, further complicate outstanding issues of fairness and justice in large-scale sociotechnical systems and re-obscure the fundamentally political nature of speech decisions being executed at scale. (shrink)