There is increasing criticism on the use of big data and algorithms in public governance. Studies revealed that algorithms may reinforce existing biases and defy scrutiny by public officials using them and citizens subject to algorithmic decisions and services. In response, scholars have called for more algorithmic transparency and regulation. These are useful, but ex post solutions in which the development of algorithms remains a rather autonomous process. This paper argues that co-design of algorithms with relevant stakeholders from government and (...) society is another means to achieve responsible and accountable algorithms that is largely overlooked in the literature. We present a case study of the development of an algorithmic tool to estimate the populations of refugee camps to manage the delivery of emergency supplies. This case study demonstrates how in different stages of development of the tool—data selection and pre-processing, training of the algorithm and post-processing and adoption—inclusion of knowledge from the field led to changes to the algorithm. Co-design supported responsibility of the algorithm in the selection of big data sources and in preventing reinforcement of biases. It contributed to accountability of the algorithm by making the estimations transparent and explicable to its users. They were able to use the tool for fitting purposes and used their discretion in the interpretation of the results. It is yet unclear whether this eventually led to better servicing of refugee camps. (shrink)

Due to the ongoing advancements in technology, socio-technical collaboration has become increasingly prevalent. This poses challenges in terms of governance and accountability, as well as issues in various other fields. Therefore, it is crucial to familiarize decision-makers and researchers with the core of human–machine collaboration. This study introduces a taxonomy that enables identification of the very nature of human–machine interaction. A literature review has revealed that automation and technical autonomy are main parameters for describing and understanding such interaction. Both aspects (...) must be carefully evaluated, as their increase has potentially far-reaching consequences. Hence, these two concepts comprise the taxonomy’s axes. Five levels of automation and five levels of technical autonomy are introduced below, based on the assumption that both automation and autonomy are gradual. The levels of automation were developed from existing approaches; those of autonomy were carefully derived from a review of the literature. The taxonomy’s use is also explained, as are its limitations and avenues for further research. (shrink)

The development of numerical models to produce realistic prospective scenarios for the evolution of biological diversity is essential. Only integrative impact assessment models are able to take into account the diverse and complex interactions embedded in social-ecological systems. The knowledge used is objective, the procedure of their integration is rigorous and the data massive. Nevertheless, the technical choices made at each stage of the development of models and scenarios are mostly circumstantial, depending on both the skills of modellers on a (...) project and the means available to them. In the end, the scenarios selected and the way they are simulated limit the futures explored, and the options offered to decision makers and stakeholders to act. The ethical implications of these circumstantial choices are generally not documented, explained or even perceived by modellers. Applied ethics propose a coherent set of principles to guide a critical reflection on the social and environmental consequences of integrative modelling and simulation of biodiversity scenarios. Such reflection should be incorporated into the actual modelling process, in a broad participatory framework, and foster effective moral involvement of modellers, policy-makers and stakeholders, in preference to the application of fixed ethical rules. (shrink)

The algorithmic accountability literature to date has primarily focused on procedural tools to govern automated decision-making systems. That prescriptive literature elides a fundamentally empirical question: whether and under what circumstances, if any, is the use of algorithmic systems to make public policy decisions perceived as legitimate? The present study begins to answer this question. Using factorial vignette survey methodology, we explore the relative importance of the type of decision, the procedural governance, the input data used, and outcome errors on perceptions (...) of the legitimacy of algorithmic public policy decisions as compared to similar human decisions. Among other findings, we find that the type of decision—low importance versus high importance—impacts the perceived legitimacy of automated decisions. We find that human governance of algorithmic systems increases perceptions of the legitimacy of algorithmic decision-making systems, even when those decisions are likely to result in significant errors. Notably, we also find the penalty to perceived legitimacy is greater when human decision-makers make mistakes than when algorithmic systems make the same errors. The positive impact on perceived legitimacy from governance—such as human-in-the-loop—is greatest for highly pivotal decisions such as parole, policing, and healthcare. After discussing the study’s limitations, we outline avenues for future research. (shrink)

Firms use algorithms to make important business decisions. To date, the algorithmic accountability literature has elided a fundamentally empirical question important to business ethics and management: Under what circumstances, if any, are algorithmic decision-making systems considered legitimate? The present study begins to answer this question. Using factorial vignette survey methodology, we explore the impact of decision importance, governance, outcomes, and data inputs on perceptions of the legitimacy of algorithmic decisions made by firms. We find that many of the procedural governance (...) mechanisms in practice today, such as notices and impact statements, do not lead to algorithmic decisions being perceived as more legitimate in general, and, consistent with legitimacy theory, that algorithmic decisions with good outcomes are perceived as more legitimate than bad outcomes. Yet, robust governance, such as offering an appeal process, can create a legitimacy dividend for decisions with bad outcomes. However, when arbitrary or morally dubious factors are used to make decisions, most legitimacy dividends are erased. In other words, companies cannot overcome the legitimacy penalty of using arbitrary or morally dubious factors, such as race or the day of the week, with a good outcome or an appeal process for individuals. These findings add new perspectives to both the literature on legitimacy and policy discussions on algorithmic decision-making in firms. (shrink)

The concept of the digital phenotype has been used to refer to digital data prognostic or diagnostic of disease conditions. Medical conditions may be inferred from the time pattern in an insomniac’s tweets, the Facebook posts of a depressed individual, or the web searches of a hypochondriac. This paper conceptualizes digital data as an extended phenotype of humans, that is as digital information produced by humans and affecting human behavior and culture. It argues that there are ethical obligations to persons (...) affected by generalizable knowledge of a digital phenotype, not only those who are personally identifiable or involved in data generation. This claim is illustrated by considering the health-related digital phenotypes of precision medicine and digital epidemiology. (shrink)

Academic and public debates are increasingly concerned with the question whether and how algorithmic decision-making may reinforce social inequality. Most previous research on this topic originates from computer science. The social sciences, however, have huge potentials to contribute to research on social consequences of ADM. Based on a process model of ADM systems, we demonstrate how social sciences may advance the literature on the impacts of ADM on social inequality by uncovering and mitigating biases in training data, by understanding data (...) processing and analysis, as well as by studying social contexts of algorithms in practice. Furthermore, we show that fairness notions need to be evaluated with respect to specific outcomes of ADM systems and with respect to concrete social contexts. Social sciences may evaluate how individuals handle algorithmic decisions in practice and how single decisions aggregate to macro social outcomes. In this overview, we highlight how social sciences can apply their knowledge on social stratification and on substantive domains of ADM applications to advance the understanding of social impacts of ADM. (shrink)

Machine behavior that is based on learning algorithms can be significantly influenced by the exposure to data of different qualities. Up to now, those qualities are solely measured in technical terms, but not in ethical ones, despite the significant role of training and annotation data in supervised machine learning. This is the first study to fill this gap by describing new dimensions of data quality for supervised machine learning applications. Based on the rationale that different social and psychological backgrounds of (...) individuals correlate in practice with different modes of human–computer-interaction, the paper describes from an ethical perspective how varying qualities of behavioral data that individuals leave behind while using digital technologies have socially relevant ramification for the development of machine learning applications. The specific objective of this study is to describe how training data can be selected according to ethical assessments of the behavior it originates from, establishing an innovative filter regime to transition from the big data rationale n = all to a more selective way of processing data for training sets in machine learning. The overarching aim of this research is to promote methods for achieving beneficial machine learning applications that could be widely useful for industry as well as academia. (shrink)

Algorithmic systems are increasingly used by state agencies to inform decisions about humans. They produce scores on risks of recidivism in criminal justice, indicate the probability for a job seeker to find a job in the labor market, or calculate whether an applicant should get access to a certain university program. In this contribution, we take an interdisciplinary perspective, provide a bird’s eye view of the different key decisions that are to be taken when state actors decide to use an (...) algorithmic system, and illustrate these decisions with empirical examples from case studies. Building on these insights, we discuss the main pitfalls and promises of the use of algorithmic system by the state and focus on four levels: The most basic question whether an algorithmic system should be used at all, the regulation and governance of the system, issues of algorithm design, and, finally, questions related to the implementation of the system on the ground and the human–machine-interaction that comes with it. Based on our assessment of the advantages and challenges that arise at each of these levels, we propose a set of crucial questions to be asked when such intricate matters are addressed. (shrink)

Resumen: El presente artículo ofrece un estado del arte de cómo se ha venido a estudiar empíricamente la relación entre política y redes sociales en la última década, desde el punto de vista de la naturaleza del objeto de estudio, las nuevas técnicas de análisis y métodos sobre las que se han apoyado las ciencias sociales, las agendas de investigación a que ha dado lugar y algunos de los dilemas éticos que suscita. El artículo consta de tres partes. Primero, desarrollamos (...) una discusión de los principales métodos de análisis de la relación entre política y redes sociales, el big data y los algoritmos, y lo comparamos con los métodos tradicionales de encuestas de opinión. Segundo, ofrecemos una revisión de las principales preocupaciones que han guiado la investigación en los últimos diez años: la representación y campañas políticas, nuevas comunidades políticas y nuevas formas de acción colectiva. Finalmente, la tercera sección reflexiona sobre los dilemas éticos y debates que plantea el uso de las nuevas técnicas de análisis en las ciencias sociales.: The article offers a state of art about the empirical study of the relationship between politics and social media during the last decade, from the point of view of the nature of the subject, new research techniques and methods used in the social sciences, the research agendas it has fostered, and some of the ethical dilemmas this research raises. The article is divided in three main parts. Firstly, we develop a discussion of the main research methods to approach the relationship between politics and digital media, big data and algorithms, which we compare with traditional opinion polls. Secondly, we offer a review of the main issues that have guided this research agenda during the last ten years: political representation and campaigns, emerging political communities, and new forms of collective action. Finally, the last section reflects on the ethical dilemmas and debates that arise from the utilization of new research methods in the social sciences. (shrink)

As public administrations adopt artificial intelligence (AI), we see this transition has the potential to transform public service and public policies, by offering a rapid turnaround on decision making and service delivery. However, a recent series of criticisms have pointed to problematic aspects of mainstreaming AI systems in public administration, noting troubled outcomes in terms of justice and values. The argument supplied here is that any public administration adopting AI systems must consider and address ambiguities and uncertainties surrounding two key (...) dimensions: the algorithms’ results, and how public managers make decisions for and about AI systems’ design. The article is based on bibliographic research in institutional theory perspective that relates the design of AI systems and relevant literature on the decision-making process in public policy. The main finding is to explain why autonomous decision systems continue to reproduce ambiguities and uncertainties when applied in public administration and propose a reflection on AI governance in public administration. This article points to the need for design institutions that immerse themselves in understanding the nuances, details, and potential outcomes of AI governance for public administration. Such institutions would reconcile consequentialist logic with a logic of appropriateness to help navigate and mediate ambiguities and uncertainties. (shrink)

Although related studies have examined the impact of different images of artificial intelligence products on consumer evaluation, exploring the impact on consumer purchase decisions from the perspective of algorithmic decision autonomy remains under-explored. Based on the self-determination theory, this research discusses the influence of the agent decision-making role played by different AI algorithmic decision autonomy on consumer purchase decisions. The results of the 3 studies indicate that algorithmic decision autonomy has an inverted U-shaped effect on consumer’s purchase decisions, consumer’s self-efficacy (...) mediates the relationship between algorithmic decision autonomy and purchase decisions, and consumer’s power distance moderates the relationship between algorithmic decision autonomy, self-efficacy, and purchase decisions. The research results can provide references for marketers, retailers, algorithm designers, and other parties to formulate algorithm marketing strategies, make AI algorithm decisions better serve consumers, and achieve value co-creation with consumers. (shrink)

This paper argues that data-driven medicine gives rise to a particular normative challenge. Against the backdrop of a distinction between the good and the right, harnessing personal health data towards the development and refinement of data-driven medicine is to be welcomed from the perspective of the good. Enacting solidarity drives progress in research and clinical practice. At the same time, such acts of sharing could—especially considering current developments in big data and artificial intelligence—compromise the right by leading to injustices and (...) affecting concrete modes of individual self-determination. In order to address this potential tension, two key elements for ethical reflection on data-driven medicine are proposed: the controllability of information flows, including technical infrastructures that are conducive towards controllability, and a paradigm shift towards output-orientation in governance and policy. (shrink)

Critical algorithm scholarship has demonstrated the difficulties of attributing accountability for the actions and effects of algorithmic systems. In this commentary, we argue that we cannot stop at denouncing the lack of accountability for algorithms and their effects but must engage the broader systems and distributed agencies that algorithmic systems exist within; including standards, regulations, technologies, and social relations. To this end, we explore accountability in “the Generated Detective,” an algorithmically generated comic. Taking up the mantle of detectives ourselves, we (...) investigate accountability in relation to this piece of experimental fiction. We problematize efforts to effect accountability through transparency by undertaking a simple operation: asking for permission to re-publish a set of the algorithmically selected and modified words and images which make the frames of the comic. Recounting this process, we demonstrate slippage between the “complication” of the algorithm and the obscurity of the legal and institutional structures in which it exists. (shrink)

Human values seem to vary across time and space. What implications does this have for the future of human value? Will our human and (perhaps) post-human offspring have very different values from our own? Can we study the future of human values in an insightful and systematic way? This article makes three contributions to the debate about the future of human values. First, it argues that the systematic study of future values is both necessary in and of itself and an (...) important complement to other future-oriented inquiries. Second, it sets out a methodology and a set of methods for undertaking this study. Third, it gives a practical illustration of what this ‘axiological futurism’ might look like by developing a model of the axiological possibility space that humans are likely to navigate over the coming decades. (shrink)