Fueled by ever-growing amounts of (digital) data and advances in artificial intelligence, decision-making in contemporary societies is increasingly delegated to automated processes. Drawing from social science theories and from the emerging body of research about algorithmic appreciation and algorithmic perceptions, the current study explores the extent to which personal characteristics can be linked to perceptions of automated decision-making by AI, and the boundary conditions of these perceptions, namely the extent to which such perceptions differ across media, (public) health, and judicial (...) contexts. Data from a scenario-based survey experiment with a national sample (N = 958) show that people are by and large concerned about risks and have mixed opinions about fairness and usefulness of automated decision-making at a societal level, with general attitudes influenced by individual characteristics. Interestingly, decisions taken automatically by AI were often evaluated on par or even better than human experts for specific decisions. Theoretical and societal implications about these findings are discussed. (shrink)

The distinction between distributive and non-distributive profiles figures prominently in current evaluations of the ethical and epistemological risks that are associated with automated profiling practices. The diagnosis that non-distributive profiles may coincidentally situate an individual in the wrong category is often perceived as the central shortcoming of such profiles. According to this diagnosis, most risks can be retraced to the use of non-universal generalisations and various other statistical associations. This article develops a top-down analysis of non-distributive profiles in which this (...) fallibility of non-distributive profiles is no longer central. Instead, it focuses on how profiling creates various asymmetries between an individual data-subject and a profiler. The emergence of informational, interest, and perspectival asymmetries between data-subject and profiler explains how non-distributive profiles weaken the epistemic position of a profiled individual. This alternative analysis provides a more balanced assessment of the epistemic risks associated with non-distributive profiles. (shrink)

Digital technologies induce organised immaturity by generating toxic sociotechnical conditions that lead us to delegate autonomous, individual, and responsible thoughts and actions to external technological systems. Aiming to move beyond a diagnostic critical reading of the toxicity of digitalisation, we bring Bernard Stiegler’s pharmacological analysis of technology into dialogue with the ethics of care to speculatively explore how the socially engaged arts—a type of artistic practice emphasising audience co-production and processual collective responses to social challenges—play a care-giving role that helps (...) counter technology-induced organised immaturity. We outline and illustrate two modes by which the socially engaged arts play this role: 1) disorganising immaturity through artivism, most notably anti-surveillance art, that imparts savoir vivre, that is, shared knowledge and meaning to counter the toxic side of technologies while enabling the imagination of alternative worlds in which humans coexist harmoniously with digital technologies, and 2) organising maturity through arts-based hacking that imparts savoir faire, that is, hands-on knowledge for experimental creation and practical enactment of better technological worlds. (shrink)

Algorithms have developed into somewhat of a modern myth. On the one hand, they have been depicted as powerful entities that rule, sort, govern, shape, or otherwise control our lives. On the other hand, their alleged obscurity and inscrutability make it difficult to understand what exactly is at stake. What sustains their image as powerful yet inscrutable entities? And how to think about the politics and governance of something that is so difficult to grasp? This editorial essay provides a critical (...) backdrop for the special issue, treating algorithms not only as computational artifacts but also as sensitizing devices that can help us rethink some entrenched assumptions about agency, transparency, and normativity. (shrink)

Much bioethical literature and policy guidances for big data analytics in biomedical research emphasize the importance of trust. It is essential that potential participants trust so they will allow their data to be used to further research. However, comparatively, little guidance is offered as to what trustworthy oversight mechanisms are, or how policy should support them, as data are collected, shared, and used. Generally, “trust” is not characterized well enough, or meaningfully enough, for the term to be systematically applied in (...) policy development. Yet points made in the philosophical literature on trust can help. They allow us, not only to better distinguish the different ways the term “trust” may be interpreted, but also to better determine how different approaches to trust can align with policy and governance—in what ways they may relate to key bioethical concepts and related laws, and in what ways they can help to balance individual and group interests in data sharing. This article draws from the philosophical literature on trust to identify a relationship among consent, trust, and justice. Specifically, parallels are drawn between “character-trustworthiness” and “natural justice,” a set of widely held legal safeguards intended to ensure decision-makers follow a pattern of procedural fairness which protects the rights of the individual and thereby maintains public confidence in the decision-making process. Relevance to traditional bioethical principles, established laws, and consent procedures are addressed throughout. In conclusion, policy actions are suggested. (shrink)

Automated algorithms are silently making crucial decisions about our lives, but most of the time we have little understanding of how they work. To counter this hidden influence, there have been increasing calls for algorithmic transparency. Much ink has been spilled over the informational account of algorithmic transparency—about how much information should be revealed about the inner workings of an algorithm. But few studies question the power structure beneath the informational disclosure of the algorithm. As a result, the information disclosure (...) itself can be a means of manipulation used by a group of people to advance their own interests. Instead of concentrating on information disclosure, this paper examines algorithmic transparency from the perspective of power, explaining how algorithmic transparency under a disciplinary power structure can be a technique of normalizing people’s behavior. The informational disclosure of an algorithm can not only set up some de facto norms, but also build a scientific narrative of its algorithm to justify those norms. In doing so, people would be internally motivated to follow those norms with less critical analysis. This article suggests that we should not simply open the black box of an algorithm without challenging the existing power relations. (shrink)

In this paper we make the case for the emergence of novel kind of bias with the use of algorithmic decision-making systems. We argue that the distinctive generative process of feature creation, characteristic of machine learning (ML), contorts feature parameters in ways that can lead to emerging feature spaces that encode novel algorithmic bias involving already marginalized groups. We term this bias _assembled bias._ Moreover, assembled biases are distinct from the much-discussed algorithmic bias, both in source (training data versus feature (...) creation) and in content (mimics of extant societal bias versus reconfigured categories). As such, this problem is distinct from issues arising from bias-encoding training feature sets or proxy features. Assembled bias is not epistemically transparent in source or content. Hence, when these ML models are used as a basis for decision-making in social contexts, algorithmic fairness concerns are compounded. (shrink)

This paper investigates how the introduction of AI to decision making increases moral distance and recommends the ethics of care to augment the ethical examination of AI decision making. With AI decision making, face-to-face interactions are minimized, and decisions are part of a more opaque process that humans do not always understand. Within decision-making research, the concept of moral distance is used to explain why individuals behave unethically towards those who are not seen. Moral distance abstracts those who are impacted (...) by the decision and leads to less ethical decisions. The goal of this paper is to identify and analyze the moral distance created by AI through both proximity distance (in space, time, and culture) and bureaucratic distance (derived from hierarchy, complex processes, and principlism). We then propose the ethics of care as a moral framework to analyze the moral implications of AI. The ethics of care brings to the forefront circumstances and context, interdependence, and vulnerability in analyzing algorithmic decision making. (shrink)

Machine learning platforms have emerged as a new promissory technology that some argue will revolutionize work practices across a broad range of professions, including medical care. During the past few years, IBM has been testing its Watson for Oncology platform at several oncology departments around the world. Published reports, news stories, as well as our own empirical research show that in some cases, the levels of concordance over recommended treatment protocols between the platform and human oncologists have been quite low. (...) Other studies supported by IBM claim concordance rates as high as 96%. We use the Watson for Oncology case to examine the practice of using concordance levels between tumor boards and a machine learning decision-support system as a form of evidence. We address a challenge related to the epistemic authority between oncologists on tumor boards and the Watson Oncology platform by arguing that the use of concordance levels as a form of evidence of quality or trustworthiness is problematic. Although the platform provides links to the literature from which it draws its conclusion, it obfuscates the scoring criteria that it uses to value some studies over others. In other words, the platform “black boxes” the values that are coded into its scoring system. (shrink)

This article presents the results of methodological experimentation that utilises machine learning to investigate automated copyright enforcement on YouTube. Using a dataset of 76.7 million YouTube videos, we explore how digital and computational methods can be leveraged to better understand content moderation and copyright enforcement at a large scale.We used the BERT language model to train a machine learning classifier to identify videos in categories that reflect ongoing controversies in copyright takedowns. We use this to explore, in a granular way, (...) how copyright is enforced on YouTube, using both statistical methods and qualitative analysis of our categorised dataset. We provide a large-scale systematic analysis of removals rates from Content ID’s automated detection system and the largely automated, text search based, Digital Millennium Copyright Act notice and takedown system. These are complex systems that are often difficult to analyse, and YouTube only makes available data at high levels of abstraction. Our analysis provides a comparison of different types of automation in content moderation, and we show how these different systems play out across different categories of content. We hope that this work provides a methodological base for continued experimentation with the use of digital and computational methods to enable large-scale analysis of the operation of automated systems. (shrink)

Information security governance has become an elusive goal and a murky concept. This paper problematizes both information security governance and the broader concept of governance. What does it mean to govern information security, or for that matter, anything? Why have information technologies proven difficult to govern? And what assurances can governance provide for the billions of people who rely on information technologies every day? Drawing together several distinct bodies of literature—including multiple strands of governance theory, actor–network theory, and scholarship on (...) sociotechnical regimes—this paper conceptualizes networked action on a spectrum from uncertain governance to governing uncertainty. I advance a twofold argument. First, I argue that networks can better govern uncertainty as they become more able not only to enroll actors in a collective agenda, but also to cut ties with those who seek to undermine that agenda. And second, I argue that the dominant conception of information security governance, which emphasizes governing uncertainty through risk management, in practice devolves to uncertain governance. This is largely because information technologies have evolved toward greater connectedness—and with it, greater vulnerability—creating a regime of insecurity. This evolution is illustrated using the history of the US government’s efforts to govern information security. (shrink)

Due to the ongoing advancements in technology, socio-technical collaboration has become increasingly prevalent. This poses challenges in terms of governance and accountability, as well as issues in various other fields. Therefore, it is crucial to familiarize decision-makers and researchers with the core of human–machine collaboration. This study introduces a taxonomy that enables identification of the very nature of human–machine interaction. A literature review has revealed that automation and technical autonomy are main parameters for describing and understanding such interaction. Both aspects (...) must be carefully evaluated, as their increase has potentially far-reaching consequences. Hence, these two concepts comprise the taxonomy’s axes. Five levels of automation and five levels of technical autonomy are introduced below, based on the assumption that both automation and autonomy are gradual. The levels of automation were developed from existing approaches; those of autonomy were carefully derived from a review of the literature. The taxonomy’s use is also explained, as are its limitations and avenues for further research. (shrink)

Big data and Machine learning Techniques are reshaping the way in which food safety risk assessment is conducted. The ongoing ‘datafication’ of food safety risk assessment activities and the progressive deployment of probabilistic models in their practices requires a discussion on the advantages and disadvantages of these advances. In particular, the low level of trust in EU food safety risk assessment framework highlighted in 2019 by an EU-funded survey could be exacerbated by novel methods of analysis. The variety of processed (...) data raises unique questions regarding the interplay of multiple regulatory systems alongside food safety legislation. Provisions aiming to preserve the confidentiality of data and protect personal information are juxtaposed to norms prescribing the public disclosure of scientific information. This research is intended to provide guidance for data governance and data ownership issues that unfold from the ongoing transformation of the technical and legal domains of food safety risk assessment. Following the reconstruction of technological advances in data collection and analysis and the description of recent amendments to food safety legislation, emerging concerns are discussed in light of the individual, collective and social implications of the deployment of cutting-edge Big Data collection and analysis techniques. Then, a set of principle-based recommendations is proposed by adapting high-level principles enshrined in institutional documents about Artificial Intelligence to the realm of food safety risk assessment. The proposed set of recommendations adopts Safety, Accountability, Fairness, Explainability, Transparency as core principles (SAFETY), whereas Privacy and data protection are used as a meta-principle. (shrink)

The need for fair and just AI is often related to the possibility of understanding AI itself, in other words, of turning an opaque box into a glass box, as inspectable as possible. Transparency and explainability, however, pertain to the technical domain and to philosophy of science, thus leaving the ethics and epistemology of AI largely disconnected. To remedy this, we propose an integrated approach premised on the idea that a glass-box epistemology should explicitly consider how to incorporate values and (...) other normative considerations, such as intersectoral vulnerabilities, at critical stages of the whole process from design and implementation to use and assessment. To connect ethics and epistemology of AI, we perform a double shift of focus. First, we move from trusting the output of an AI system to trusting the process that leads to the outcome. Second, we move from expert assessment to more inclusive assessment strategies, aiming to facilitate expert and non-expert assessment. Together, these two moves yield a framework usable for experts and non-experts when they inquire into relevant epistemological and ethical aspects of AI systems. We dub our framework ‘epistemology-cum-ethics’ to signal the equal importance of both aspects. We develop it from the vantage point of the designers: how to create the conditions to internalize values into the whole process of design, implementation, use, and assessment of an AI system, in which values (epistemic and non-epistemic) are explicitly considered at each stage and inspectable by every salient actor involved at any moment. (shrink)

In an analysis of artificially intelligent systems for medical diagnostics and treatment planning we argue that patients should be able to exercise a right to withdraw from AI diagnostics and treatment planning for reasons related to (1) the physician’s role in the patients’ formation of and acting on personal preferences and values, (2) the bias and opacity problem of AI systems, and (3) rational concerns about the future societal effects of introducing AI systems in the health care sector.

In the age of ubiquitous computing and artificially intelligent applications, social machines serves as a powerful framework for understanding and interpreting interactions in socio-algorithmic ecosystems. Although researchers have largely used it to analyze the interactions of individuals and algorithms, limited attempts have been made to investigate the politics in social machines. In this study, I claim that social machines are per se political machines, and introduce a five-point framework for classifying influence processes in socio-algorithmic ecosystems. By drawing from scholars from (...) political theory, I use a notion of influence that functions as a meta-concept for connecting and comparing different conceptions of politics. In this way, I can associate multiple political aspects of social machines from a cybernetic perspective. I show that the framework efficiently categorizes dimensions of influence that shape interactions between individuals and algorithms. These categories are symbolic influence, political conduct, algorithmic influence, design, and regulatory influence. Using case studies, I describe how they interact with each other on online social networks and in algorithmic decision-making systems and illustrate how the framework is able to guide scientists in further research. (shrink)

Important decisions that impact humans lives, livelihoods, and the natural environment are increasingly being automated. Delegating tasks to so-called automated decision-making systems can improve efficiency and enable new solutions. However, these benefits are coupled with ethical challenges. For example, ADMS may produce discriminatory outcomes, violate individual privacy, and undermine human self-determination. New governance mechanisms are thus needed that help organisations design and deploy ADMS in ways that are ethical, while enabling society to reap the full economic and social benefits of (...) automation. In this article, we consider the feasibility and efficacy of ethics-based auditing as a governance mechanism that allows organisations to validate claims made about their ADMS. Building on previous work, we define EBA as a structured process whereby an entity’s present or past behaviour is assessed for consistency with relevant principles or norms. We then offer three contributions to the existing literature. First, we provide a theoretical explanation of how EBA can contribute to good governance by promoting procedural regularity and transparency. Second, we propose seven criteria for how to design and implement EBA procedures successfully. Third, we identify and discuss the conceptual, technical, social, economic, organisational, and institutional constraints associated with EBA. We conclude that EBA should be considered an integral component of multifaced approaches to managing the ethical risks posed by ADMS. (shrink)

Mature information societies are characterised by mass production of data that provide insight into human behaviour. Analytics has arisen as a practice to make sense of the data trails generated through interactions with networked devices, platforms and organisations. Persistent knowledge describing the behaviours and characteristics of people can be constructed over time, linking individuals into groups or classes of interest to the platform. Analytics allows for a new type of algorithmically assembled group to be formed that does not necessarily align (...) with classes or attributes already protected by privacy and anti-discrimination law or addressed in fairness- and discrimination-aware analytics. Individuals are linked according to offline identifiers and shared behavioural identity tokens, allowing for predictions and decisions to be taken at a group rather than individual level. This article examines the ethical significance of such ad hoc groups in analytics and argues that the privacy interests of algorithmically assembled groups in inviolate personality must be recognised alongside individual privacy rights. Algorithmically grouped individuals have a collective interest in the creation of information about the group, and actions taken on its behalf. Group privacy is proposed as a third interest to balance alongside individual privacy and social, commercial and epistemic benefits when assessing the ethical acceptability of analytics platforms. (shrink)

Many popular artificial intelligence (AI) ethics frameworks center the principle of autonomy as necessary in order to mitigate the harms that might result from the use of AI within society. These harms often disproportionately affect the most marginalized within society. In this paper, we argue that the principle of autonomy, as currently formalized in AI ethics, is itself flawed, as it expresses only a mainstream mainly liberal notion of autonomy as rational self-determination, derived from Western traditional philosophy. In particular, we (...) claim that the adherence to such principle, as currently formalized, does not only fail to address many ways in which people’s autonomy can be violated, but also to grasp a broader range of AI-empowered harms profoundly tied to the legacy of colonization, and which particularly affect the already marginalized and most vulnerable on a global scale. To counter such a phenomenon, we advocate for the need of a relational turn in AI ethics, starting from a relational rethinking of the AI ethics principle of autonomy that we propose by drawing on theories on relational autonomy developed both in moral philosophy and Ubuntu ethics. (shrink)

There is a long history in science and technology studies of tracking problematic objects, such as controversies, matters of concern, and issues, using various digital tools. But what happens when public problems do not play out in these familiar ways? In this paper, we will think through the methodological implications of studying “problems” in relation to recent events surrounding the sharing of patient data in the National Health Service in the United Kingdom. When a data sharing agreement called care.data was (...) announced in 2013, nearly 1.5 million citizens chose to opt out. Yet, in subsequent years, there has been little evidence of a robust public mobilising around data sharing. We will attempt to track this elusive ‘non problem’ using some digital tools developed in STS for the purpose of mapping issues and problem definitions within science. Although we find these digital tools are unable to capture the “problem,” the process of searching helps us map the terrain of the case and forces us to consider wider definitions. (shrink)

The increasing use of Artificial Intelligence for making decisions in public affairs has sparked a lively debate on the benefits and potential harms of self-learning technologies, ranging from the hopes of fully informed and objectively taken decisions to fear for the destruction of mankind. To prevent the negative outcomes and to achieve accountable systems, many have argued that we need to open up the “black box” of AI decision-making and make it more transparent. Whereas this debate has primarily focused on (...) how transparency can secure high-quality, fair, and reliable decisions, far less attention has been devoted to the role of transparency when it comes to how the general public come to perceive AI decision-making as legitimate and worthy of acceptance. Since relying on coercion is not only normatively problematic but also costly and highly inefficient, perceived legitimacy is fundamental to the democratic system. This paper discusses how transparency in and about AI decision-making can affect the public’s perception of the legitimacy of decisions and decision-makers and produce a framework for analyzing these questions. We argue that a limited form of transparency that focuses on providing justifications for decisions has the potential to provide sufficient ground for perceived legitimacy without producing the harms full transparency would bring. (shrink)

The combination of increased availability of large amounts of fine-grained human behavioral data and advances in machine learning is presiding over a growing reliance on algorithms to address complex societal problems. Algorithmic decision-making processes might lead to more objective and thus potentially fairer decisions than those made by humans who may be influenced by greed, prejudice, fatigue, or hunger. However, algorithmic decision-making has been criticized for its potential to enhance discrimination, information and power asymmetry, and opacity. In this paper, we (...) provide an overview of available technical solutions to enhance fairness, accountability, and transparency in algorithmic decision-making. We also highlight the criticality and urgency to engage multi-disciplinary teams of researchers, practitioners, policy-makers, and citizens to co-develop, deploy, and evaluate in the real-world algorithmic decision-making processes designed to maximize fairness and transparency. In doing so, we describe the Open Algortihms project as a step towards realizing the vision of a world where data and algorithms are used as lenses and levers in support of democracy and development. (shrink)

Organizations increasingly rely on algorithm-based HR decision-making to monitor their employees. This trend is reinforced by the technology industry claiming that its decision-making tools are efficient and objective, downplaying their potential biases. In our manuscript, we identify an important challenge arising from the efficiency-driven logic of algorithm-based HR decision-making, namely that it may shift the delicate balance between employees’ personal integrity and compliance more in the direction of compliance. We suggest that critical data literacy, ethical awareness, the use of participatory (...) design methods, and private regulatory regimes within civil society can help overcome these challenges. Our paper contributes to literature on workplace monitoring, critical data studies, personal integrity, and literature at the intersection between HR management and corporate responsibility. (shrink)

Algorithms are increasingly affecting us in our daily lives. They seem to be everywhere, yet they are seldom seen by the humans dealing with the consequences that result from them. Yet, in recent theorisations, there is a risk that the algorithm is being given too much prominence. This article addresses the interaction between algorithmic outputs and the humans engaging with them by drawing on studies of two distinct empirical fields – self-quantification and audit controls of taxpayers. We explore recalibration as (...) a way to understand the practices and processes involved when, on the one hand, decisions are made based on results from algorithmic calculations in counting and accounting software, and on the other hand, when decisions are made based on human experience/knowledge. In particular, we are concerned with moments when an algorithmic output differs from expectations of ‘normalcy’ and ‘normativity’ in any given situation. This could be a ‘normal’ relation between sales and VAT deductions for a business, or a ‘normal’ number of steps one takes in a day, or ‘normative’ as it is according to the book, following guidelines and recommendations from other sources. In these moments, we argue that a process of recalibration occurs – an effortful moment where, rather than treat the algorithmic output as given, individuals’ tacit knowledge, experiences and intuition are brought into play to address the deviation from the normal and normative. (shrink)

Purpose Artificial intelligence (AI) has sparked interest in various areas, including marketing. However, this exhilaration is being tempered by growing concerns about the moral and legal implications of using AI in marketing. Although previous research has revealed various ethical and legal issues, such as algorithmic discrimination and data privacy, there are no definitive answers. This paper aims to fill this gap by investigating AI’s ethical and legal concerns in marketing and suggesting feasible solutions. Design/methodology/approach The paper synthesises information from academic (...) articles, industry reports, case studies and legal documents through a thematic literature review. A qualitative analysis approach categorises and interprets ethical and legal challenges and proposes potential solutions. Findings The findings of this paper raise concerns about ethical and legal challenges related to AI in the marketing area. Ethical concerns related to discrimination, bias, manipulation, job displacement, absence of social interaction, cybersecurity, unintended consequences, environmental impact, privacy and legal issues such as consumer security, responsibility, liability, brand protection, competition law, agreements, data protection, consumer protection and intellectual property rights are discussed in the paper, and their potential solutions are discussed. Research limitations/implications Notwithstanding the interesting insights gathered from this investigation of the ethical and legal consequences of AI in marketing, it is important to recognise the limits of this research. Initially, the focus of this study is confined to a review of the most important ethical and legal issues pertaining to AI in marketing. Additional possible repercussions, such as those associated with intellectual property, contracts and licencing, should be investigated more deeply in future studies. Despite the fact that this study gives various answers and best practices for tackling the stated ethical and legal concerns, the viability and efficacy of these solutions may differ depending on the context and industry. Thus, more research and case studies are required to evaluate the applicability and efficacy of these solutions in other circumstances. This research is mostly based on a literature review and may not represent the experiences or opinions of all stakeholders engaged in AI-powered marketing. Further study might involve interviews or surveys with marketing professionals, customers and other key stakeholders to offer a full knowledge of the practical difficulties and solutions. Because of the rapid speed of technical progress, AI’s ethical and regulatory ramifications in marketing are continually increasing. Consequently, this work should be a springboard for more research and continuing conversations on this subject. Practical implications This study’s findings have several practical implications for marketing professionals. Emphasising openness and explainability: Marketing professionals should prioritise transparency in their use of AI, ensuring that customers are fully informed about data collection and utilisation for targeted advertising. By promoting openness and explainability, marketers can foster customer trust and avoid the negative consequences of a lack of transparency. Establishing ethical guidelines: Marketing professionals need to develop ethical rules for the creation and implementation of AI-powered marketing strategies. Adhering to ethical principles ensures compliance with legal norms and aligns with the organisation’s values and ideals. Investing in bias detection tools and privacy-enhancing technology: To mitigate risks associated with AI in marketing, marketers should allocate resources to develop and implement bias detection tools and privacy-enhancing technology. These tools can identify and address biases in AI algorithms, safeguard consumer privacy and extract valuable insights from consumer data. Social implications This study’s social implications emphasise the need for a comprehensive approach to address the ethical and legal challenges of AI in marketing. This includes adopting a responsible innovation framework, promoting ethical leadership, using ethical decision-making frameworks and conducting multidisciplinary research. By incorporating these approaches, marketers can navigate the complexities of AI in marketing responsibly, foster an ethical organisational culture, make informed ethical decisions and develop effective solutions. Such practices promote public trust, ensure equitable distribution of benefits and risk, and mitigate potential negative social consequences associated with AI in marketing. Originality/value To the best of the authors’ knowledge, this paper is among the first to explore potential solutions comprehensively. This paper provides a nuanced understanding of the challenges by using a multidisciplinary framework and synthesising various sources. It contributes valuable insights for academia and industry. (shrink)

A wave of recent scholarship has warned about the potential for discriminatory harms of algorithmic systems, spurring an interest in algorithmic accountability and regulation. Meanwhile, parallel concerns about surveillance practices have already led to multiple successful regulatory efforts of surveillance technologies—many of which have algorithmic components. Here, we examine municipal surveillance regulation as offering lessons for algorithmic oversight. Taking the 2017 Seattle Surveillance Ordinance as our primary case study and surveying efforts across five other cities, we describe the features of (...) existing surveillance regulation; including procedures for describing surveillance technologies in detail, requirements for public engagement, and processes for establishing acceptable uses. Although the Seattle Surveillance Ordinance was not intended to address algorithmic accountability, we find these considerations to be relevant to the law’s aim of surfacing disparate impacts of systems in use. We also find that in notable cases government employees did not identify regulated algorithmic surveillance technologies as reliant on algorithmic or machine learning systems, highlighting definitional gaps that could hinder future efforts toward algorithmic regulation. We argue that finer-grained distinctions between types of information systems in the language of law and policy, and risk assessment tools integrated into their implementation would strengthen future regulatory efforts by rendering underlying algorithmic components more legible and accountable to political and community stakeholders. (shrink)

In recent years, policymakers have begun to problematize how legislation stands in the way of the digitalization of the public sector. We are witnessing the emergence of a new phenomenon, digital-ready legislation, which implies that, whenever possible, new legislation should build on simple rules and unambiguous terminology to reduce the need for professional discretion and allow for the extended use of automated case processing in public-sector organizations. Digital-ready legislation has potentially wide-ranging consequences because it creates the conditions for how public (...) organizations are digitalized. The processes, practices, choices, and responsibilities for drafting digital-ready legislation, however, are not well-described or debated. Digital-ready legislation is a dormant issue. This paper develops the notion of the “double darkness” of digitalization to account for this. Based on a qualitative study, the paper investigates how digital-ready legislation as a sociotechnical arrangement is shaped by policy tools and by a complex, collaborative process where responsibility for legislation is fragmented. It argues that although the policy tools are aimed at making actors responsible for digitalization and creating clarity about the process, actors seem to be reluctant to take on the responsibility for making political decisions related to digitalization. (shrink)

Si la democracia consiste en posibilitar que todas las personas tengan iguales posibilidades de influir en las decisiones que les afectan, las sociedades digitales tienen que interrogarse por el modo de conseguir que los nuevos entornos hagan factible esa igualdad. Las primeras dificultades son conceptuales: entender cómo se configura la interacción entre los humanos y los algoritmos, en qué consiste el aprendizaje de estos dispositivos y cuál es la naturaleza de sus sesgos. Inmediatamente después nos topamos con la cuestión ineludible (...) de qué clase de igualdad estamos tratando de asegurar, teniendo en cuenta la diversidad de concepciones de la justicia que hay en nuestras sociedades. Si articular ese pluralismo no es un asunto que pueda resolverse con una técnica agregativa, sino que requiere compromisos políticos, entonces una concepción deliberativa de la democracia parece la más apta para conseguir esa igualdad a la que aspiran las sociedades democráticas. (shrink)

The growing presence of smart devices in our lives turns all of society into something largely unknown to us. The strategy of demanding transparency stems from the desire to reduce the ignorance to which this automated society seems to condemn us. An evaluation of this strategy first requires that we distinguish the different types of non-transparency. Once we reveal the limits of the transparency needed to confront these devices, the article examines the alternative strategy of explainable artificial intelligence and concludes (...) with the idea that these types of complex realities exceed individual capacities and are only comprehensible in a collective fashion. (shrink)

AI technology is capturing the African imaginations as a gateway to progress and prosperity. There is a growing interest in AI by different actors across the continent including scientists, researchers, humanitarian and aid organizations, academic institutions, tech start-ups, and media organizations. Several African states are looking to adopt AI technology to capture economic growth and development opportunities. On the other hand, African researchers highlight the gap in regulatory frameworks and policies that govern the development of AI in the continent. They (...) argue that this could lead to AI technology exacerbating problems of inequalities and injustice in the continent. However, most of the literature on AI ethics is biased toward Euro-American perspectives and lack the understanding of how AI development is apprehended in the Global South, and particularly Africa. Drawing on the case study of the first African Master’s in Machine Intelligence program, this paper argues for looking beyond the question of ethics in AI and examining AI governance issues through the analytical lens of the raciality of computing and the political economy of technoscience to understand AI development in Africa. By doing so, this paper seeks a different theorization for AI ethics from the South that is based on lived experiences of those in the margins and avoids the framings of technological futures that simplistically pathologize or celebrate Africa. (shrink)

PurposeThis paper aims to investigate algorithmic governmentality – as proposed by Antoinette Rouvroy – specifically in relation to law. It seeks to show how algorithmic profiling can be particularly attractive for those in legal practice, given restraints on time and resources. It deviates from Rouvroy in two ways. First, it argues that algorithmic governmentality does not contrast with neoliberal modes of government in that it allows indirect rule through economic calculations. Second, it argues that critique of such systems is possible, (...) especially if the creative nature of law can be harnessed effectively.Design/methodology/approachThis is a conceptual paper, with a theory-based approach, that is intended to explore relevant issues related to algorithmic governmentality as a basis for future empirical research. It builds on governmentality and socio-legal studies, as well as research on algorithmic practices and some documentary analysis of reports and public-facing marketing of relevant technologies.FindingsThis paper provides insights on how algorithmic knowledge is collected, constructed and applied in different situations. It provides examples of how algorithms are currently used and how trends are developing. It demonstrates how such uses can be informed by socio-political and economic rationalities.Research limitations/implicationsFurther empirical research is required to test the theoretical findings.Originality/valueThis paper takes up Rouvroy’s question of whether we are at the end of critique and seeks to identify where such critique can be made possible. It also highlights the importance of acknowledging the role of political rationalities in informing the activity of algorithmic assemblages. (shrink)

Algorithms, once obscure objects of technical art, have lately been subject to considerable popular and scholarly scrutiny. What does it mean to adopt the algorithm as an object of analytic attention? What is in view, and out of view, when we focus on the algorithm? Using Niklaus Wirth's 1975 formulation that “algorithms + data structures = programs” as a launching-off point, this paper examines how an algorithmic lens shapes the way in which we might inquire into contemporary digital culture.

It is widely believed that a conservative moral outlook is opposed to biomedical forms of human enhancement. In this paper, I argue that this widespread belief is incorrect. Using Cohen’s evaluative conservatism as my starting point, I argue that there are strong conservative reasons to prioritise the development of biomedical enhancements. In particular, I suggest that biomedical enhancement may be essential if we are to maintain our current evaluative equilibrium (i.e. the set of values that undergird and permeate our current (...) political, economic, and personal lives) against the threats to that equilibrium posed by external, non-biomedical forms of enhancement. I defend this view against modest conservatives who insist that biomedical enhancements pose a greater risk to our current evaluative equilibrium, and against those who see no principled distinction between the forms of human enhancement. (shrink)

While organizations today make extensive use of complex algorithms, the notion of algorithmic accountability remains an elusive ideal due to the opacity and fluidity of algorithms. In this article, we develop a framework for managing algorithmic accountability that highlights three interrelated dimensions: reputational concerns, engagement strategies, and discourse principles. The framework clarifies that accountability processes for algorithms are driven by reputational concerns about the epistemic setup, opacity, and outcomes of algorithms; that the way in which organizations practically engage with emergent (...) expectations about algorithms may be manipulative, adaptive, or moral; and that when accountability relationships are heavily burdened by the opacity and fluidity of complex algorithmic systems, the emphasis of engagement should shift to a rational communication process through which a continuous and tentative assessment of the development, workings, and consequences of algorithms can be achieved over time. The degree to which such engagement is, in fact, rational can be assessed based on four discourse-ethical principles of participation, comprehension, multivocality, and responsiveness. We conclude that the framework may help organizations and their environments to jointly work toward greater accountability for complex algorithms. It may further help organizations in reputational positioning surrounding accountability issues. The discourse-ethical principles introduced in this article are meant to elevate these positioning contests to extend beyond mere adaption or compliance and help guide organizations to find moral and forward-looking solutions to accountability issues. (shrink)

Responsible innovation in artificial intelligence calls for public deliberation: well-informed “deep democratic” debate that involves actors from the public, private, and civil society sectors in joint efforts to critically address the goals and means of AI. Adopting such an approach constitutes a challenge, however, due to the opacity of AI and strong knowledge boundaries between experts and citizens. This undermines trust in AI and undercuts key conditions for deliberation. We approach this challenge as a problem of situating the knowledge of (...) actors from the AI industry within a deliberative system. We develop a new framework of responsibilities for AI innovation as well as a deliberative governance approach for enacting these responsibilities. In elucidating this approach, we show how actors from the AI industry can most effectively engage with experts and nonexperts in different social venues to facilitate well-informed judgments on opaque AI systems and thus effectuate their democratic governance. (shrink)

This paper argues that data-driven medicine gives rise to a particular normative challenge. Against the backdrop of a distinction between the good and the right, harnessing personal health data towards the development and refinement of data-driven medicine is to be welcomed from the perspective of the good. Enacting solidarity drives progress in research and clinical practice. At the same time, such acts of sharing could—especially considering current developments in big data and artificial intelligence—compromise the right by leading to injustices and (...) affecting concrete modes of individual self-determination. In order to address this potential tension, two key elements for ethical reflection on data-driven medicine are proposed: the controllability of information flows, including technical infrastructures that are conducive towards controllability, and a paradigm shift towards output-orientation in governance and policy. (shrink)

The power of algorithms has become a familiar topic in society, media, and the social sciences. It is increasingly common to argue that, for instance, algorithms automate inequality, that they are biased black boxes that reproduce racism, or that they control our money and information. Implicit in many of these discussions is that algorithms are permeated with normativities, and that these normativities shape society. The aim of this editorial is double: First, it contributes to a more nuanced discussion about algorithms (...) by discussing how we, as social scientists, think about algorithms in relation to five theoretical ideal types. For instance, what does it mean to go under the hood of the algorithm and what does it mean to stay above it? Second, it introduces the contributions to this special theme by situating them in relation to these five ideal types. By doing this, the editorial aims to contribute to an increased analytical awareness of how algorithms are theorized in society and culture. The articles in the special theme deal with algorithms in different settings, ranging from farming, schools, and self-tracking to AIDS, nuclear power plants, and surveillance. The contributions thus explore, both theoretically and empirically, different settings where algorithms are intertwined with normativities. (shrink)

The ever-increasing application of algorithms to decision-making in a range of social contexts has prompted demands for algorithmic accountability. Accountable decision-makers must provide their decision-subjects with justifications for their automated system’s outputs, but what kinds of broader principles should we expect such justifications to appeal to? Drawing from political philosophy, I present an account of algorithmic accountability in terms of the democratic ideal of ‘public reason’. I argue that situating demands for algorithmic accountability within this justificatory framework enables us to (...) better articulate their purpose and assess the adequacy of efforts toward them. (shrink)

Purpose Algorithms are in the mainstream media news on an almost daily basis. Their context is invariably artificial intelligence and machine learning decision-making. In media articles, algorithms are described as powerful, autonomous actors that have a capability of producing actions that have consequences. Despite a tendency for deification, the prevailing critique of algorithms focuses on ethical concerns raised by decisions resulting from algorithmic processing. However, the purpose of this paper is to propose that the ethical concerns discussed are limited in (...) scope and suggest that it is not clear what concerns dominate the debate. Design/methodology/approach The paper uses a systematic mapping study approach to review articles appearing in leading UK national papers from the perspective of the ethical concerns over a period of one year. The articles are categorised using a widely cited framework detailing a taxonomy of ethical concerns. The UK context is important because of UK public policy initiatives around AI. Findings The research presented in this paper contributes the first systematic mapping study of articles appearing in leading UK national papers from the perspective of widely accepted ethical concerns such as inscrutable evidence, misguided evidence, unfair outcomes and transformative effects. Originality/value The research presented in this paper contributes the first systematic mapping study of articles appearing in leading UK national papers from the perspective of the ethical concerns. The UK context is important because of UK public policy initiatives around AI. To review the media content from the perspective of ethical concerns, this paper uses the synthesised conceptual map of ethical concerns developed by Mittelstad et al. Given the dominance of that framework, this paper’s contribution is also an important instantiation and experimental validation of using that conceptual map. (shrink)

Increasingly complex and autonomous systems require machine ethics to maximize the benefits and minimize the risks to society arising from the new technology. It is challenging to decide which type of ethical theory to employ and how to implement it effectively. This survey provides a threefold contribution. First, it introduces a trimorphic taxonomy to analyze machine ethics implementations with respect to their object (ethical theories), as well as their nontechnical and technical aspects. Second, an exhaustive selection and description of relevant (...) works is presented. Third, applying the new taxonomy to the selected works, dominant research patterns, and lessons for the field are identified, and future directions for research are suggested. (shrink)