This article examines the complaint that arbitrary algorithmic decisions wrong those whom they affect. It makes three contributions. First, it provides an analysis of what arbitrariness means in this context. Second, it argues that arbitrariness is not of moral concern except when special circumstances apply. However, when the same algorithm or different algorithms based on the same data are used in multiple contexts, a person may be arbitrarily excluded from a broad range of opportunities. The third contribution is to explain (...) why this systemic exclusion is of moral concern and to offer a solution to address it. (shrink)

This paper explores the political potential of digital failure as a refusal to work in service of today’s dataveillance society. Moving beyond criticisms of flawed digital systems, this paper traces the moments of digital failure that seek to break, rather than fix, existing systems. Instead, digital failure is characterized by pesky data that sneaks through the cracks of digital capitalism and dissipates into the unproductive; it supports run-away data prone to misidentifications by digital marketers, coders, and content moderators; and it (...) celebrates data predisposed to “back-talk” with playful irreverence toward those that seek to bring order through normative categorization and moderation. I call these data entropic, fugitive, and queer and explore their mischievous practices through three case studies: the unaccountable data in identity resolution, public shaming of the ImageNet training data, and reading practices of sex worker and influencer, @Charlieshe. Together these case studies articulate the political potential of digital failure as a process of unbecoming the good data subject by pushing past the margins of legibility, knowability, and thinkability, to reveal what is made illegible, unknowable, and unthinkable to data’s seeing eye. As predictive analytics, automated decision-systems, and artificial intelligence take on increasingly central roles in public governance, digital failure reveals how data itself is a flawed concept prone to political abuse and social engineering to protect the interests of the powerful, while keeping those marginalized over-surveilled and underrepresented. (shrink)