Citizen science projects have started to utilize Machine Learning to sort through large datasets generated in fields like astronomy, ecology and biodiversity, biology, and neuroimaging. Human–machine systems have been created to take advantage of the complementary strengths of humans and machines and have been optimized for efficiency and speed. We conducted qualitative content analysis on meta-summaries of documents reporting the results of 12 citizen science projects that used machine learning to optimize classification tasks. We examined the distribution of tasks between (...) citizen scientists, experts, and algorithms, and how epistemic agency was enacted in terms of whose knowledge shapes the distribution of tasks, who decides what knowledge is relevant to the classification, and who validates it. In our descriptive results, we found that experts, who include professional scientists and algorithm developers, are involved in every aspect of a project, from annotating or labelling data to giving data to algorithms to train them to make decisions from predictions. Experts also test and validate models to improve their accuracy by scoring their outputs when algorithms fail to make correct decisions. Experts are mostly the humans involved in a loop, but when algorithms encounter problems, citizens are also involved at several stages. In this paper, we present three main examples of citizens-in-the-loop: when algorithms provide incorrect suggestions; when algorithms fail to know how to perform classification; and when algorithms pose queries. We consider the implications of the emphasis on optimization on the ideal of science and the role of citizen scientists from a perspective informed by Science and Technology Studies and Information Systems. Based on our findings, we conclude that ML in CS classification projects, far from being deterministic in its nature and effects, may be open to question. There is no guarantee that these technologies can replace citizen scientists, nor any guarantee that they can provide citizens with opportunities for more interesting tasks. (shrink)

Stories are an important indicator of our vision of the future. In the case of artificial intelligence, dominant stories are polarized between notions of threat and myopic solutionism. The central storytellers—big tech, popular media, and authors of science fiction—represent particular demographics and motivations. Many stories, and storytellers, are missing. This paper details the accounts of missing AI narratives by leading scholars from a range of disciplines interested in AI Futures. Participants focused on the gaps between dominant narratives and the untold (...) stories of the capabilities, issues, and everyday realities of the technology. One participant proposed a “story crisis” in which these narratives compete to shape the public discourse on AI. Our findings indicate that dominant narratives distract and mislead public understandings and conceptions of AI. This suggests a need to pay closer attention to missing AI narratives. It is not simply about telling new stories, it is about listening to existing stories and asking what is wanted from AI. We call for realistic, nuanced, and inclusive stories, working with and for diverse voices, which consider story-teller; genre, and communicative purpose. Such stories can then inspire the next generation of thinkers, technologists, and storytellers. (shrink)