The algorithmic accountability literature to date has primarily focused on procedural tools to govern automated decision-making systems. That prescriptive literature elides a fundamentally empirical question: whether and under what circumstances, if any, is the use of algorithmic systems to make public policy decisions perceived as legitimate? The present study begins to answer this question. Using factorial vignette survey methodology, we explore the relative importance of the type of decision, the procedural governance, the input data used, and outcome errors on perceptions (...) of the legitimacy of algorithmic public policy decisions as compared to similar human decisions. Among other findings, we find that the type of decision—low importance versus high importance—impacts the perceived legitimacy of automated decisions. We find that human governance of algorithmic systems increases perceptions of the legitimacy of algorithmic decision-making systems, even when those decisions are likely to result in significant errors. Notably, we also find the penalty to perceived legitimacy is greater when human decision-makers make mistakes than when algorithmic systems make the same errors. The positive impact on perceived legitimacy from governance—such as human-in-the-loop—is greatest for highly pivotal decisions such as parole, policing, and healthcare. After discussing the study’s limitations, we outline avenues for future research. (shrink)

Despite widespread recognition of an emergent politics of data in our midst, we strikingly lack a political theory of data. We readily acknowledge the presence of data across our political lives, but we often do not know how to conceptualize the politics of all those data points—the forms of power they constitute and the kinds of political subjects they implicate. Recent work in numerous academic disciplines is evidence of the first steps toward a political theory of data. This article maps (...) some limits of this emergent literature with an eye to enriching its theoretical range. The literature on data politics, both within political theory and elsewhere, has thus far focused almost exclusively on the algorithm. This article locates a further dimension of data politics in the work of formatting technology or, more simply, formats. Formats are simultaneously conceptual and technical in the ways they define what can even count as data, and by extension who can count as data and how they can count. A focus on formats is of theoretical value because it provides a bridge between work on the conceptual contours of categories and the technology-centric literature on algorithms that tends to ignore the more conceptual dimensions of data technology. The political insight enabled by format theory is shown in the context of an extended interrogation of the politics of racialized redlining. (shrink)

Data increasingly drive our lives. Often presented as a new trajectory, the deep immersion of our lives in data has a history that is well over a century old. By revisiting the work of early pioneers of what would today be called data science, we can bring into view both assumptions that fund our data-driven moment as well as alternative relations to data. I here excavate insights by contrasting a seemingly unlikely pair of early data technologists, Francis Galton and W.E.B. (...) Du Bois. Galton, well known for his contributions to eugenics, was first and foremost a tinkering technician of measure. There are numerous domains of science over which Galtonian conceptions retain considerable influence, presumably without his pride in racial inequality. A more viable, because more egalitarian, alternative for the present can be found in the early data work of Du Bois. (shrink)

In this essay I bring Gregory Bateson together with Deleuze and Guattari (primarily with the latter) to show their ecological compatibility, especially with Guattari’s ecosophy. I do this against the backdrop of the Anthropocene which presents us not only with a ‘climate’ of post-truth and political corruption, but also with the so-called climate crisis. In the context of these two broad examinations, I ask what can an artisan-artist-designer do given this problematic context? My reply is to call on ‘speculative design’ (...) by an ‘avant-garde without authority’ to expose the symptoms of the Anthropocene era. (shrink)

This paper examines racial discrimination and algorithmic bias in predictive policing algorithms (PPAs), an emerging technology designed to predict threats and suggest solutions in law enforcement. We first describe what discrimination is in a case study of Chicago’s PPA. We then explain their causes with Broadbent’s contrastive model of causation and causal diagrams. Based on the cognitive science literature, we also explain why fairness is not an objective truth discoverable in laboratories but has context-sensitive social meanings that need to be (...) negotiated through democratic processes. With the above analysis, we next predict why some recommendations given in the bias reduction literature are not as effective as expected. Unlike the cliché highlighting equal participation for all stakeholders in predictive policing, we emphasize power structures to avoid hermeneutical lacunae. Finally, we aim to control PPA discrimination by proposing a governance solution—a framework of a social safety net. (shrink)

Should you be targeted by police for a crime that AI predicts you will commit? In this paper, we analyse when, and to what extent, the person-based predictive policing (PP) — using AI technology to identify and handle individuals who are likely to breach the law — could be justifiably employed. We first examine PP’s epistemological limits, and then argue that these defects by no means refrain from its usage; they are worse in humans. Next, based on major AI ethics (...) guidelines (IEEE, EU, and RIKEN, etc.), we refine three basic moral principles specific to person-based PP. We also derive further requirements from case studies, including debates in Chicago, New Orleans, San Francisco, Tokyo, and cities in China. Instead of rejecting PP programs, we analyse what necessary conditions should be met for using the tool to achieve social good. While acknowledging its risks, we conclude that the person-based PP could be beneficial in community policing, especially when merging into a larger governance framework of the social safety net. (shrink)

Whereas using artificial intelligence (AI) to predict natural hazards is promising, applying a predictive policing algorithm (PPA) to predict human threats to others continues to be debated. Whereas PPAs were reported to be initially successful in Germany and Japan, the killing of Black Americans by police in the US has sparked a call to dismantle AI in law enforcement. However, although PPAs may statistically associate suspects with economically disadvantaged classes and ethnic minorities, the targeted groups they aim to protect are (...) often vulnerable populations as well (e.g., victims of human trafficking, kidnapping, domestic violence, or drug abuse). Thus, determining how to enhance the benefits of PPA while reducing bias through better management is important. In this paper, we propose a policy schema to address this issue. First, after clarifying relevant concepts, we examine major criticisms of PPAs and argue that some of them should be addressed. If banning AI or making it taboo is an unrealistic solution, we must learn from our errors to improve AI. We next identify additional challenges of PPAs and offer recommendations from a policy viewpoint. We conclude that the employment of PPAs should be merged into broader governance of the social safety net and audited publicly by parliament and civic society so that the unjust social structure that breeds bias can be revised. (shrink)

Existing discussions about automated decision-making focus primarily on its inputs and outputs, raising questions about data collection and privacy on one hand and accuracy and fairness on the other. Less attention has been devoted to critically examining the temporality of decision-making processes—the speed at which automated decisions are reached. In this paper, I identify four dimensions of algorithmic speed that merit closer analysis. Duration (how much time it takes to reach a judgment), timing (when automated systems intervene in the activity (...) being evaluated), frequency (how often evaluations are performed), and lived time (the human experience of algorithmic speed) are interrelated, but distinct, features of automated decision-making. Choices about the temporal structure of automated decision-making systems have normative implications, which I describe in terms of "disruption," "displacement," "re-calibration," and "temporal fairness," with values such as accuracy, fairness, accountability, and legitimacy hanging in the balance. As computational tools are increasingly tasked with making judgments about human activities and practices, the designers of decision-making systems will have to reckon, I argue, with when—and how fast—judgments ought to be rendered. Though computers are capable of reaching decisions at incredible speeds, failing to account for the temporality of automated decision-making risks misapprehending the costs and benefits automation promises. (shrink)