The ongoing Post Office scandal in the UK and the 2021 Child Daycare Benefit Scandal in the Netherlands make up exemplary cases of how digital automation has changed and in fact severely harmed trust relations ranging from trust in oneself over trust in social roles, trust in institutions, trust in technology and general trust. By looking closer at how digital automation in these cases generated ruptures in the lives of ordinary citizens and (...) also affected the involved institutions and society at large I discuss how we may begin to discuss the impact of digitalization on trust in societies. I start with an introduction to the panoramic view of trust, followed by a short description of the two scandals. Based on the exploration of the two cases I take an ex post facto perspective on the dynamic social events as they unfolded in the scandals in order to analyze how the introduction of digital automation tools affected trust relations. I draw attention to some of the most noticeable changes in the four relations of trust, which is trust in institutions, interpersonal trust, trust in oneself and trust in technology. (shrink)

Engineers are traditionally regarded as trustworthy professionals who meet exacting standards. In this chapter I begin by explicating our trust relationship towards engineers, arguing that it is a linear but indirect relationship in which engineers “stand behind” the artifacts and technological systems that we rely on directly. The chapter goes on to explain how this relationship has become more complex as engineers have taken on two additional aims: the aim of social engineering to create and steer trust between (...) people, and the aim of creating automated systems that take over human tasks and are meant to invite the trust of those who rely on and interact with them. (shrink)

Fueled by ever-growing amounts of (digital) data and advances in artificial intelligence, decision-making in contemporary societies is increasingly delegated to automated processes. Drawing from social science theories and from the emerging body of research about algorithmic appreciation and algorithmic perceptions, the current study explores the extent to which personal characteristics can be linked to perceptions of automated decision-making by AI, and the boundary conditions of these perceptions, namely the extent to which such perceptions differ across media, (public) health, and judicial (...) contexts. Data from a scenario-based survey experiment with a national sample (N = 958) show that people are by and large concerned about risks and have mixed opinions about fairness and usefulness of automated decision-making at a societal level, with general attitudes influenced by individual characteristics. Interestingly, decisions taken automatically by AI were often evaluated on par or even better than human experts for specific decisions. Theoretical and societal implications about these findings are discussed. (shrink)

As software developers design artificial agents , they often have to wrestle with complex issues, issues that have philosophical and ethical importance. This paper addresses two key questions at the intersection of philosophy and technology: What is deception? And when is it permissible for the developer of a computer artifact to be deceptive in the artifact’s development? While exploring these questions from the perspective of a software developer, we examine the relationship of deception and trust. Are developers using deception (...) to gain our trust? Is trust generated through technological “enchantment” warranted? Next, we investigate more complex questions of how deception that involves AAs differs from deception that only involves humans. Finally, we analyze the role and responsibility of developers in trust situations that involve both humans and AAs. (shrink)

Following the studies of Araujo et al. (AI Soc 35:611–623, 2020) and Lee (Big Data Soc 5:1–16, 2018), this empirical study uses two scenario-based online experiments. The sample consists of 221 subjects from Germany, differing in both age and gender. The original studies are not replicated one-to-one. New scenarios are constructed as realistically as possible and focused on everyday work situations. They are based on the AI acceptance model of Scheuer (Grundlagen intelligenter KI-Assistenten und deren vertrauensvolle Nutzung. Springer, Wiesbaden, 2020) (...) and are extended by individual descriptive elements of AI systems in comparison to the original studies. The first online experiment examines decisions made by artificial intelligence with varying degrees of impact. In the high-impact scenario, applicants are automatically selected for a job and immediately received an employment contract. In the low-impact scenario, three applicants are automatically invited for another interview. In addition, the relationship between age and risk perception is investigated. The second online experiment tests subjects’ perceived trust in decisions made by artificial intelligence, either semi-automatically through the assistance of human experts or fully automatically in comparison. Two task types are distinguished. The task type that requires “human skills”—represented as a performance evaluation situation—and the task type that requires “mechanical skills”—represented as a work distribution situation. In addition, the extent of negative emotions in automated decisions is investigated. The results are related to the findings of Araujo et al. (AI Soc 35:611–623, 2020) and Lee (Big Data Soc 5:1–16, 2018). Implications for further research activities and practical relevance are discussed. (shrink)

This study examines how visibility of a content moderator and ambiguity of moderated content influence perception of the moderation system in a social media environment. In the course of a two-day pre-registered experiment conducted in a realistic social media simulation, participants encountered moderated comments that were either unequivocally harsh or ambiguously worded, and the source of moderation was either unidentified, or attributed to other users or an automated system (AI). The results show that when comments were moderated by an AI (...) versus other users, users perceived less accountability in the moderation system and had less trust in the moderation decision, especially for ambiguously worded harassments, as opposed to clear harassment cases. However, no differences emerged in the perceived moderation fairness, objectivity, and participants confidence in their understanding of the moderation process. Overall, our study demonstrates that users tend to question the moderation decision and system more when an AI moderator is visible, which highlights the complexity of effectively managing the visibility of automatic content moderation in the social media environment. (shrink)

With service robots becoming more ubiquitous in social life, interaction design needs to adapt to novice users and the associated uncertainty in the first encounter with this technology in new emerging environments. Trust in robots is an essential psychological prerequisite to achieve safe and convenient cooperation between users and robots. This research focuses on psychological processes in which user dispositions and states affect trust in robots, which in turn is expected to impact the behavior and reactions in the (...) interaction with robotic systems. In a laboratory experiment, the influence of propensity to trust in automation and negative attitudes toward robots on state anxiety, trust, and comfort distance toward a robot were explored. Participants were approached by a humanoid domestic robot two times and indicated their comfort distance and trust. The results favor the differentiation and interdependence of dispositional, initial, and dynamic learned trust layers. A mediation from the propensity to trust to initial learned trust by state anxiety provides an insight into the psychological processes through which personality traits might affect interindividual outcomes in human-robot interaction (HRI). The findings underline the meaningfulness of user characteristics as predictors for the initial approach to robots and the importance of considering users’ individual learning history regarding technology and robots in particular. (shrink)

Recent advances in machine learning methods have created opportunities to eliminate unfairness from algorithmic decision making. Multiple computational techniques (i.e., algorithmic fairness criteria) have arisen out of this work. Yet, urgent questions remain about the perceived fairness of these criteria and in which situations organizations should use them. In this paper, we seek to gain insight into these questions by exploring fairness perceptions of five algorithmic criteria. We focus on two key dimensions of fairness evaluations: distributive fairness and procedural fairness. (...) We shed light on variation in the potential for different algorithmic criteria to facilitate distributive fairness. Subsequently, we discuss procedural fairness and provide a framework for understanding how algorithmic criteria relate to essential aspects of this construct, which helps to identify when a specific criterion is suitable. From a practical standpoint, we encourage organizations to recognize that managing fairness in machine learning systems is complex, and that adopting a blind or one-size-fits-all mentality toward algorithmic criteria will surely damage people’s attitudes and trust in automated technology. Instead, firms should carefully consider the subtle yet significant differences between these technical solutions. (shrink)

Recent advances in automation technology have increased the opportunity for collaboration between humans and multiple autonomous systems such as robots and self-driving cars. In research on autonomous system collaboration, the trust users have in autonomous systems is an important topic. Previous research suggests that the trust built by observing a task can be transferred to other tasks. However, such research did not focus on trust in multiple different devices but in one device or several of the (...) same devices. Thus, we do not know how trust changes in an environment involving the operation of multiple different devices such as a construction site. We investigated whether trust can be transferred among multiple different devices, and investigated the effect of two factors: the similarity among multiple devices and the agency attributed to each device, on trust transfer among multiple devices. We found that the trust a user has in a device can be transferred to other devices and that attributing different agencies to each device can clarify the distinction among devices, preventing trust from transferring. (shrink)

With automation of routine decisions coupled with more intricate and complex information architecture operating this automation, concerns are increasing about the trustworthiness of these systems. These concerns are exacerbated by a class of artificial intelligence that uses deep learning, an algorithmic system of deep neural networks, which on the whole remain opaque or hidden from human comprehension. This situation is commonly referred to as the black box problem in AI. Without understanding how AI reaches its conclusions, it is (...) an open question to what extent we can trust these systems. The question of trust becomes more urgent as we delegate more and more decision-making to and increasingly rely on AI to safeguard significant human goods, such as security, healthcare, and safety. Models that “open the black box” by making the non-linear and complex decision process understandable by human observers are promising solutions to the black box problem in AI but are limited, at least in their current state, in their ability to make these processes less opaque to most observers. A philosophical analysis of trust will show why transparency is a necessary condition for trust and eventually for judging AI to be trustworthy. A more fruitful route for establishing trust in AI is to acknowledge that AI is situated within a socio-technical system that mediates trust, and by increasing the trustworthiness of these systems, we thereby increase trust in AI. (shrink)

Autonomous vehicles (AV) offer promising benefits to society in terms of safety, environmental impact and increased mobility. However, acute challenges persist with any novel technology, inlcuding the perceived risks and trust underlying public acceptance. While research examining the current state of AV public perceptions and future challenges related to both societal and individual barriers to trust and risk perceptions is emerging, it is highly fragmented across disciplines. To address this research gap, by using the Web of Science database, (...) our study undertakes a bibliometric and performance analysis to identify the conceptual and intellectual structures of trust and risk narratives within the AV research field by investigating engineering, social sciences, marketing, and business and infrastructure domains to offer an interdisciplinary approach. Our analysis provides an overview of the key research area across the search categories of ‘trust’ and ‘risk’. Our results show three main clusters with regard to trust and risk, namely, behavioural aspects of AV interaction; uptake and acceptance; and modelling human–automation interaction. The synthesis of the literature allows a better understanding of the public perception of AV and its historical conception and development. It further offers a robust model of public perception in AV, outlining the key themes found in the literature and, in turn, offers critical directions for future research. (shrink)

Algorithm Decision Making (ADM) systems designed to augment or automate human decision-making have the potential to produce better decisions while also freeing up human time and attention for other pursuits. For this potential to be realised, however, algorithmic decisions must be sufficiently aligned with human goals and interests. We take a Principal-Agent (P-A) approach to the questions of ADM alignment and trust. In a broad sense, ADM is beneficial if and only if human principals can trust algorithmic agents (...) to act faithfully on their behalf. This mirrors the challenge of facilitating P-A relationships among humans, but the peculiar nature of human-machine interaction also raises unique issues. The problem of asymmetric information is omnipresent but takes a different form in the context of ADM. Although the decision-making machinery of an algorithmic agent can in principle be laid bare for all to see, the sheer complexity of ADM systems based on deep learning models prevents straightforward monitoring. We draw on literature from economics and political science to argue that the problem of trust in ADM systems should be addressed at the level of institutions. Although the dyadic relationship between human principals and algorithmic agents is our ultimate concern, cooperation at this level must rest against an institutional environment which allows humans to effectively evaluate and choose among algorithmic alternatives. (shrink)

Could translation be fully automated? We must first acknowledge the complexity, ambiguity, and diversity of natural languages. These aspects of natural languages, when combined with a particular dilemma known as the computational dilemma, appear to imply that the machine translator faces certain obstacles that a human translator has already managed to overcome. At the same time, science has not yet solved the problem of how human brains process natural languages and how human beings come to acquire natural language understanding. We (...) will then distinguish between the task of translation and the responsibility of the translator. Thereafter, we will conduct a survey of the methods of machine translation (viz. RBMT, SMT, NMT, foundation models or large language models). These methods will then be critically evaluated both in general and relative to Bar-Hillel’s hypothesis about the impossibility of fully automatic, high-quality machine translation (F ahqmt ). Some concluding remarks will be made about the scope, prospects, and limits of machine translation. (shrink)

Transparency is now a fundamental principle for data processing under the General Data Protection Regulation. We explore what this requirement entails for artificial intelligence and automated decision-making systems. We address the topic of transparency in artificial intelligence by integrating legal, social, and ethical aspects. We first investigate the ratio legis of the transparency requirement in the General Data Protection Regulation and its ethical underpinnings, showing its focus on the provision of information and explanation. We then discuss the pitfalls with respect (...) to this requirement by focusing on the significance of contextual and performative factors in the implementation of transparency. We show that human–computer interaction and human-robot interaction literature do not provide clear results with respect to the benefits of transparency for users of artificial intelligence technologies due to the impact of a wide range of contextual factors, including performative aspects. We conclude by integrating the information- and explanation-based approach to transparency with the critical contextual approach, proposing that transparency as required by the General Data Protection Regulation in itself may be insufficient to achieve the positive goals associated with transparency. Instead, we propose to understand transparency relationally, where information provision is conceptualized as communication between technology providers and users, and where assessments of trustworthiness based on contextual factors mediate the value of transparency communications. This relational concept of transparency points to future research directions for the study of transparency in artificial intelligence systems and should be taken into account in policymaking. (shrink)

The study explores the future of AI-driven media and info-communication as envisioned by experts from all world regions, defining relevant terminology and expectations for 2050. Participants engaged in a 4-week series of surveys, questioning their definitions and projections about AI for the field of media and communication. Their expectations predict universal access to democratically available, automated, personalized and unbiased information determined by trusted narratives, recolonization of information technology and the demystification of the media process. These experts, as technology ambassadors, advocate (...) AI-to-AI solutions to mitigate technology-driven misuse and misinformation. The optimistic scenarios shift responsibility to future generations, relying on AI-driven solutions and finding inspiration in nature. Their present-based forecasts could be construed as being indicative of professional near-sightedness and cognitive dissonance. Visualizing our findings into a Glasses Model of AI Trust, the study contributes to key debates regarding AI policy, developmental trajectories, and academic research in media and info-communication fields. (shrink)

Future automated vehicles of different sizes will share the same space with other road users, e. g., pedestrians. For a safe interaction, successful communication needs to be ensured, in particular, with vulnerable road users, such as pedestrians. Two possible communication means exist for AVs: vehicle kinematics for implicit communication and external human-machine interfaces for explicit communication. However, the exact interplay is not sufficiently studied yet for pedestrians' interactions with AVs. Additionally, very few other studies focused on the interplay of vehicle (...) kinematics and eHMI for pedestrians' interaction with differently sized AVs, although the precise coordination is decisive to support the communication with pedestrians. Therefore, this study focused on how the interplay of vehicle kinematics and eHMI affects pedestrians' willingness to cross, trust and perceived safety for the interaction with two differently sized AVs. In this experimental online study, the participants interacted with the AVs in a shared space. Both AVs were equipped with a 360° LED light-band eHMI attached to the outer vehicle body. Three eHMI statuses were displayed. The vehicle kinematics were varied at two levels. Moreover, “non-matching” conditions were included for both AVs in which the dynamic eHMI falsely communicated a yielding intent although the vehicle did not yield. Overall, results showed that pedestrians' willingness to cross was significantly higher for the smaller AV compared to the larger AV. Regarding the interplay of vehicle kinematics and eHMI, results indicated that a dynamic eHMI increased pedestrians' perceived safety when the vehicle yielded. When the vehicle did not yield, pedestrians' perceived safety still increased for the dynamic eHMI compared to the static eHMI and no eHMI. The findings of this study demonstrated possible negative effects of eHMIs when they did not match the vehicle kinematics. Further implications for a holistic communication strategy for differently sized AVs will be discussed. (shrink)

Nonverbal signals color the meanings of interpersonal relationships. Humans rely on facial, head, postural, and vocal signals to express relational messages along continua. Three of relevance are dominance-submission, composure-nervousness and trust-distrust. Machine learning and new automated analysis tools are making possible a deeper understanding of the dynamics of relational communication. These are explored in the context of group interactions during a game entailing deception. The “messiness” of studying communication under naturalistic conditions creates many measurement and design obstacles that are (...) discussed here. Possibilities for their mitigation are considered. (shrink)

In this article we outline two different ways of ‘seeing’ autonomous driving (AD) cars. The first corresponds with the technological innovation narrative, published in online industry, policy, business and other news contexts, that pitches AD cars as the solution to societal problems, and urges users to trust and accept them so that such benefits can be accrued. The second is a narrative of everyday improvisation, which was visualized through our video ethnography and participant mapping exercises. Our research, undertaken in (...) Sweden, involved possible future everyday users of AD cars. We argue for a research and intervention agenda that examines how the visual narration of how AD cars might participate in human futures, could be shifted to create new modes of trust and reassurance for publics. (shrink)

While algorithmic decision-making (ADM) is projected to increase exponentially in the coming decades, the academic debate on whether people are ready to accept, trust, and use ADM as opposed to human decision-making is ongoing. The current research aims at reconciling conflicting findings on ‘algorithmic aversion’ in the literature. It does so by investigating algorithmic aversion while controlling for two important characteristics that are often associated with ADM: increased benefits (monetary and accuracy) and decreased user control. Across three high-powered (Ntotal (...) = 1192), preregistered 2 (agent: algorithm/human) × 2 (benefits: high/low) × 2 (control: user control/no control) between-subjects experiments, and two domains (finance and dating), the results were quite consistent: there is little evidence for a default aversion against algorithms and in favor of human decision makers. Instead, users accept or reject decisions and decisional agents based on their predicted benefits and the ability to exercise control over the decision. (shrink)

Giorgia Pozzi’s feature article1 on the risks of testimonial injustice when using automated prediction drug monitoring programmes (PDMPs) turns the spotlight on a pressing and well-known clinical problem: physicians’ challenges to predict patient behaviour, so that treatment decisions can be made based on this information, despite any fallibility. Currently, as one possible way to improve prognostic assessments of patient behaviour, Machine Learning-driven clinical decision support systems (ML-CDSS) are being developed and deployed. To make her point, Pozzi discusses ML-CDSSs that are (...) supposed to deliver physicians an accurate estimation of the likelihood of narcotic, sedative and stimulant opioid misuse by a given patient (like, e.g., ‘NarxCare’). Regarding cases of deviating assessment between human evaluators and automated systems, the medico-ethical discussion has so far mainly focused on disagreement between clinicians and Machine Learning (ML) algorithms,2 for example in ‘second opinions’ that have been reconstructed as disagreements between physician testimony and ML-CDSS testimony.3 Examples such as those of PDMPs, by contrast, draw attention to situations in which patient testimony potentially contradicts ML-CDSS testimony. Pozzi examines decision-making situations in which ML-CDSS testimony factually overrides the testimony of the person affected. Thus, in the presented case of patient Kathryn, the testimony of her own physical and mental condition was insufficient to overturn the unfavourable outcome for …. (shrink)

Despite pronouncements about the inevitable diffusion of artificial intelligence and autonomous technologies, in practice, it is human behavior, not technology in a vacuum, that dictates how technology seeps into—and changes—societies. To better understand how human preferences shape technological adoption and the spread of AI-enabled autonomous technologies, we look at representative adult samples of US public opinion in 2018 and 2020 on the use of four types of autonomous technologies: vehicles, surgery, weapons, and cyber defense. By focusing on these four diverse (...) uses of AI-enabled autonomy that span transportation, medicine, and national security, we exploit the inherent variation between these AI-enabled autonomous use cases. We find that those with familiarity and expertise with AI and similar technologies were more likely to support all of the autonomous applications we tested (except weapons) than those with a limited understanding of the technology. Individuals that had already delegated the act of driving using ride-share apps were also more positive about autonomous vehicles. However, familiarity cut both ways; individuals are also less likely to support AI-enabled technologies when applied directly to their life, especially if technology automates tasks they are already familiar with operating. Finally, we find that familiarity plays little role in support for AI-enabled military applications, for which opposition has slightly increased over time. (shrink)

Biometric technologies are becoming more pervasive in the workplace, augmenting managerial processes such as hiring, monitoring and terminating employees. Until recently, these devices consisted mainly of GPS tools that track location, software that scrutinizes browser activity and keyboard strokes, and heat/motion sensors that monitor workstation presence. Today, however, a new generation of biometric devices has emerged that can sense, read, monitor and evaluate the affective state of a worker. More popularly known by its commercial moniker, Emotional AI, the technology stems (...) from advancements in affective computing. But whereas previous generations of biometric monitoring targeted the exterior physical body of the worker, concurrent with the writings of Foucault and Hardt, we argue that emotion-recognition tools signal a far more invasive disciplinary gaze that exposes and makes vulnerable the inner regions of the worker-self. Our paper explores attitudes towards empathic surveillance by analyzing a survey of 1015 responses of future job-seekers from 48 countries with Bayesian statistics. Our findings reveal affect tools, left unregulated in the workplace, may lead to heightened stress and anxiety among disadvantaged ethnicities, gender and income class. We also discuss a stark cross-cultural discrepancy whereby East Asians, compared to Western subjects, are more likely to profess a trusting attitude toward EAI-enabled automated management. While this emerging technology is driven by neoliberal incentives to optimize the worksite and increase productivity, ultimately, empathic surveillance may create more problems in terms of algorithmic bias, opaque decisionism, and the erosion of employment relations. Thus, this paper nuances and extends emerging literature on emotion-sensing technologies in the workplace, particularly through its highly original cross-cultural study. (shrink)

What's wrong with targeted advertising in political campaigns? Are echo chambers a matter of genuine concern? How does data collection impact on trust in society? As decision-making becomes increasingly automated, how can decision-makers be held to account? This collection consider potential solutions to these challenges.

In their article ‘Who is afraid of black box algorithms? On the epistemological and ethical basis of trust in medical AI’, Durán and Jongsma discuss the epistemic and ethical challenges raised by black box algorithms in medical practice. The opacity of black box algorithms is an obstacle to the trustworthiness of their outcomes. Moreover, the use of opaque algorithms is not normatively justified in medical practice. The authors introduce a formalism, called computational reliabilism, which allows generating justified beliefs on (...) the algorithm reliability and trustworthy outcomes of artificial intelligence systems by means of epistemic warrants, called reliability indicators. However, they remark the need for reliability indicators specific to black box algorithms and that justified knowledge is not sufficient to justify normatively the actions of the physicians using medical AI systems. Therefore, Durán and Jongsma advocate for a more transparent design and implementation of black box algorithms, providing a series of recommendations to mitigate the epistemic and ethical challenges behind their use in medical practice. In this response, I argue that a peculiar form of black box algorithm transparency, called design publicity, may efficiently implement these recommendations. Design publicity encodes epistemic, that is, reliability indicators, and ethical recommendations for black box algorithms by means of four subtypes of transparency. These target the values and goals, their translation into design requirements, the performance and consistency of the algorithm altogether. I discuss design publicity applying it to a use case focused on the automated classification of skin lesions from medical images. (shrink)

The current discourse on fairness, accountability, and transparency in machine learning is driven by two competing narratives: sociotechnical dogmatism, which holds that society is full of inefficiencies and imperfections that can only be solved by better algorithms; and sociotechnical skepticism, which opposes many instances of automation on principle. Both perspectives, we argue, are reductive and unhelpful. In this chapter, we review a large, diverse body of literature in an attempt to move beyond this restrictive duality, toward a pragmatic synthesis (...) that emphasizes the central role of context and agency in evaluating new and emerging technologies. We show how epistemological and ethical considerations are inextricably intertwined in contemporary debates on algorithmic bias and explainability. We trace the dialectical interplay between dogmatic and skeptical narratives across disciplines, merging insights from social theory and philosophy. We review a number of theories of explanation, ultimately endorsing a sociotechnical pragmatism that combines elements of Floridi’s levelism and Mayo’s reliabilism to place a special emphasis on notions of agency and trust. We conclude that this hybrid does more to promote fairness, accountability, and transparency in machine learning than dogmatic or skeptical alternatives. (shrink)

In the article ‘Testimonial injustice in medical machine learning’, Pozzi argues that the prescription drug monitoring programme (PDMP) leads to testimonial injustice as physicians are more inclined to trust the PDMP’s risk scores over the patient’s own account of their medication history.1 Pozzi further develops this argument by discussing how credibility shifts from patients to machine learning (ML) systems that are supposedly neutral. As a result, a sense of distrust is now formed between patients and physicians. While there are (...) merits to Pozzi’s main argument of epistemic injustice caused by PDMPs, Pozzi mentions but ultimately glosses over the problem of automation bias. In this commentary, I will discuss automation bias and the affect it has on clinical decision making as well as a technical problem exacerbated by the usage of PDMPs that can potentially cause physical harms. It is reiterated in the article that the confidence physicians have in the PDMP’s risk scores over the patient’s testimony leads to misplaced trust in the ML systems. What Pozzi describes here is known as automation bias, which occurs when there is an over-reliance on ML systems. …. (shrink)

This paper explores the role and resolution of disagreements between physicians and their diagnostic AI-based decision support systems. With an ever-growing number of applications for these independently operating diagnostic tools, it becomes less and less clear what a physician ought to do in case their diagnosis is in faultless conflict with the results of the DSS. The consequences of such uncertainty can ultimately lead to effects detrimental to the intended purpose of such machines, e.g. by shifting the burden of proof (...) towards a physician. Thus, we require normative clarity for integrating these machines without affecting established, trusted, and relied upon workflows. In reconstructing different causes of conflicts between physicians and their AI-based tools—inspired by the approach of “meaningful human control” over autonomous systems and the challenges to resolve them—we will delineate normative conditions for “meaningful disagreements”. These incorporate the potential of DSS to take on more tasks and outline how the moral responsibility of a physician can be preserved in an increasingly automated clinical work environment. (shrink)

This paper critically assesses the possibility of moral enhancement with ambient intelligence technologies and artificial intelligence presented in Savulescu and Maslen (2015). The main problem with their proposal is that it is not robust enough to play a normative role in users’ behavior. A more promising approach, and the one presented in the paper, relies on an artifi-cial moral reasoning engine, which is designed to present its users with moral arguments grounded in first-order normative theories, such as Kantianism or utilitarianism, (...) that reason-responsive people can be persuaded by. This proposal can play a normative role and it is also a more promising avenue towards moral enhancement. It is more promising because such a system can be designed to take advantage of the sometimes undue trust that people put in automated technologies. We could therefore expect a well-designed moral reasoner system to be able to persuade people that may not be persuaded by similar arguments from other people. So, all things considered, there is hope in artificial intelli-gence for moral enhancement, but not in artificial intelligence that relies solely on ambient intelligence technologies. (shrink)

Sustainability is typically viewed as consisting of three forces, economic, social, and ecological, in tension with one another. In this paper, we address the dangers posed to societal sustainability. The concern being addressed is the very survival of societies where the rights of individuals, personal and collective freedoms, an independent judiciary and media, and democracy, despite its messiness, are highly valued. We argue that, as a result of various technological innovations, a range of dysfunctional impacts are threatening social and political (...) stability. For instance, robotics and automation are replacing human labor and decision-making in a range of industries; search engines, monetized through advertising, have access to, and track, our interests and preferences; social media, in connecting us to one another often know more about us than we ourselves do, enabling them to profit in ways which may not coincide with our well-being; online retailers have not only acquired the ability to track and predict our buying choices, but also they can squeeze vendors based on their outsize bargaining power; and, in general, virtual technologies have changed both the way we think and our sense of self. With the rising deployment of the Internet of Things, and developments in machine learning and artificial intelligence, the threats to individual freedoms and rights, societal cohesion and harmony, employment and economic well-being, and trust in democracy are being ratcheted up. This paper lauds the benefits and addresses the harm wrought by the high tech giants in Information and Communication Technologies. The search for rapidly growing revenues drives firms to accelerate product innovation without fully investigating the entire gamut of their impacts. As greater wealth accrues to the leaders of tech firms, inequalities within firms and societies are widening, creating social tensions and political ferment. We explore the ethical nature of the challenge employing a simple utilitarian calculus, complemented by approaches rooted in rights, justice, and the common good. Various options to address the challenges posed by ICTs are considered and evaluated. We argue that regulation may do little more than slow down the damage to society, particularly since societal values and political preferences vary internationally. Firms need to establish ethical standards, imbuing the upholders of these standards with sufficient authority, while creating a culture of morality. User involvement and activism, and shareholders’ concerns for the sustainability of societies on whose continued prosperity they depend, are imperative to humanity’s ability to decide the future direction of technology. (shrink)

Artificial intelligence (AI) has attracted an increasing amount of attention, both positive and negative. Its potential applications in healthcare are indeed manifold and revolutionary, and within the realm of medical imaging and radiology (which will be the focus of this paper), significant increases in accuracy and speed, as well as significant savings in cost, stand to be gained through the adoption of this technology. Because of its novelty, a norm of keeping humans “in the loop” wherever AI mechanisms are deployed (...) has become synonymous with good ethical practice in some circles. It has been argued that keeping humans “in the loop” is important for reasons of safety, accountability, and the maintenance of institutional trust. However, as the application of machine learning for the detection of lumbar spinal stenosis (LSS) in this paper’s case study reveals, there are some scenarios where an insistence on keeping humans in the loop (or in other words, the resistance to automation) seems unwarranted and could possibly lead us to miss out on very real and important opportunities in healthcare—particularly in low-resource settings. It is important to acknowledge these opportunity costs of resisting automation in such contexts, where better options may be unavailable. Using an AI model based on convolutional neural networks developed by a team of researchers at NUH/NUS medical school in Singapore for automated detection and classification of the lumbar spinal canal, lateral recess, and neural foraminal narrowing in an MRI scan of the spine to diagnose LSS, we will aim to demonstrate that where certain criteria hold (e.g., the AI is as accurate or better than human experts, risks are low in the event of an error, the gain in wellbeing is significant, and the task being automated is not essentially or importantly human), it is both morally permissible and even desirable to kick the humans out of the loop. (shrink)

Given a subject so imbued with contention and conflicting theoretical stances, it is remarkable that automated instruments ever came to replace the human eye as sensitive arbiters of color specification. Yet, dramatic shifts in assumptions and practice did occur in the first half of the twentieth century. How and why was confidence transferred from careful observers to mechanized devices when the property being measured – color – had become so closely identified with human physiology and psychology? A fertile perspective on (...) the problem is via the history of science and technology, paying particular attention to social groups and disciplinary identity to determine how those factors affected their communities’ cognitive territory. There were both common and discordant threads motivating the various technical groups that took on the problems of measuring light and color from the late nineteenth century onwards, and leading them towards the development of appropriate instruments for themselves. The transition from visual to photoelectric methods could be portrayed as a natural evolution, replacing the eye by an alternative roviding more sensitivity and convenience – indeed, this is the conventional positivist view propounded by technical histories. However, the adoption of new measurement technologies seldom is simple, and frequently has a significant cultural component. Beneath this slide towards automation lay a raft of implicit assumptions about objectivity, the nature of the observer, the role of instruments, and the trade-offs between standardization and descriptive power. While espousing rational arguments for a physical detector of color, its proponents weighted their views with tacit considerations. The reassignment of trust from the eye to automated instruments was influenced as much by the historical context as by intellectual factors. I will argue that several distinct aspects were involved, which include the reductive view of color provided by the trichromatic theory; the impetus provided by its association with photometry; the expanding mood for a quantitative and objective approach to scientific observation; and, the pressures for commercial standardization. As suggested by these factors, there was another shift of authority at play: from one technical specialism to another. The regularization of color involved appropriation of the subject by a particular set of social interests: communities of physicists and engineers espousing a ‘physicalist’ interpretation, rather than psychologists and physiologists for whom color was conceived as a more complex phenomenon. Moreover, the sources for automated color measurement, and instrumentation for measuring color, were primarily from the industrial sphere rather than from academic science. To understand these shifts, then, this chapter explores differing views of the importance of observers, machines and automation. (shrink)

Currently, we find ourselves in a paradigm in which we believe that accepting climate change data will lead to a kind of automatic action toward the preservation of our environment. I have argued elsewhere (Fried 2020) that this lack of civic action on climate data is significant when placed in the historical, military context of the technologies that collect this data––Earth remote sensing technologies. However, I have not yet discussed the phenomenological or moral implications of this context, which are deeply (...) interconnected. In this paper, I assert that Earth remote sensing technologies can, if we are not careful, lead us to a kind of environmental deskilling. This assertion comes in four parts. First, the military context of Earth remote sensing technologies––which collect important data on climate change––acts as a kind of stability, as defined by Don Ihde and others. Second, I invoke Sir Patrick Heelan to argue that the theoretical underpinnings of Earth systems science do not translate from military to environmental praxes as we imagine they do. Third, Hannah Arendt makes the case that a state’s trust in simplifying narratives like that of climate data, meant to create “islands” of certainty in an uncertain world, can be self-defeating. That is to say, they can silence public action. I extend these arguments through Vallor’s analysis of moral deskilling, in which she points out that an overemphasis on autonomous data collection––and trust in a kind of automated decision-making on that data––can deskill us from important questions relevant to our collective flourishing. In all of these examples, the lines between environmental and military research are blurry. (shrink)

Currently, we find ourselves in a paradigm in which we believe that accepting climate change data will lead to a kind of automatic action toward the preservation of our environment. I have argued elsewhere (Fried 2020) that this lack of civic action on climate data is significant when placed in the historical, military context of the technologies that collect this data––Earth remote sensing technologies. However, I have not yet discussed the phenomenological or moral implications of this context, which are deeply (...) interconnected. In this paper, I assert that Earth remote sensing technologies can, if we are not careful, lead us to a kind of environmental deskilling. This assertion comes in four parts. First, the military context of Earth remote sensing technologies––which collect important data on climate change––acts as a kind of stability, as defined by Don Ihde and others. Second, I invoke Sir Patrick Heelan to argue that the theoretical underpinnings of Earth systems science do not translate from military to environmental praxes as we imagine they do. Third, Hannah Arendt makes the case that a state’s trust in simplifying narratives like that of climate data, meant to create “islands” of certainty in an uncertain world, can be self-defeating. That is to say, they can silence public action. I extend these arguments through Vallor’s analysis of moral deskilling, in which she points out that an overemphasis on autonomous data collection––and trust in a kind of automated decision-making on that data––can deskill us from important questions relevant to our collective flourishing. In all of these examples, the lines between environmental and military research are blurry. (shrink)

Plagiarism detection services are a powerful tool to help encourage academic integrity. Adoption of these services has proven to be controversial due to ethical concerns about students’ rights. Central to these concerns is the fact that most such systems make permanent archives of student work to be re-used in plagiarism detection. This computerization and automation of plagiarism detection is changing the relationships of trust and responsibility between students, educators, educational institutions, and private corporations. Educators must respect student privacy (...) rights when implementing such systems. Student work is personal information, not the property of the educator or institution. The student has the right to be fully informed about how plagiarism detection works, and the fact that their work will be permanently archived as a result. Furthermore, plagiarism detection should not be used if the permanent archiving of a student’s work may expose him or her to future harm. (shrink)

What accounts for the prestige of quantitative methods? The usual answer is that quantification is desirable in social investigation as a result of its successes in science. Trust in Numbers questions whether such success in the study of stars, molecules, or cells should be an attractive model for research on human societies, and examines why the natural sciences are highly quantitative in the first place. Theodore Porter argues that a better understanding of the attractions of quantification in business, government, (...) and social research brings a fresh perspective to its role in psychology, physics, and medicine. Quantitative rigor is not inherent in science but arises from political and social pressures, and objectivity derives its impetus from cultural contexts. In a new preface, the author sheds light on the current infatuation with quantitative methods, particularly at the intersection of science and bureaucracy. (shrink)

Public trust in business, defined as the degree to which the public—meaning society at large—trusts business in general, is largely understudied. This article suggests four domains of existing trust research from which scholars of public trust in business can draw. The authors then propose four main hypotheses, which aim to predict the determinants of public trust, and test these hypotheses using a factorial vignette methodology. These results will provide scholars with more direction as this article is, (...) to the authors’ knowledge, one of the first empirical studies of public trust. Furthermore, the study will enable those companies interested in increasing public trust to understand better respective determinants of public trust. (shrink)

This paper sets out an account of trust in AI as a relationship between clinicians, AI applications, and AI practitioners in which AI is given discretionary authority over medical questions by clinicians. Compared to other accounts in recent literature, this account more adequately explains the normative commitments created by practitioners when inviting clinicians’ trust in AI. To avoid committing to an account of trust in AI applications themselves, I sketch a reductive view on which discretionary authority is (...) exercised by AI practitioners through the vehicle of an AI application. I conclude with four critical questions based on the discretionary account to determine if trust in particular AI applications is sound, and a brief discussion of the possibility that the main roles of the physician could be replaced by AI. (shrink)

In the context of the telegraphic distribution of Greenwich time, while the early experiments, the roles of successive Astronomers Royal in its expansion, and its impacts on the standardization of time in Victorian Britain have all been evaluated, the attempts of George Biddell Airy and his collaborators in constructing the Royal Observatory's time signals as the authoritative source of standard time have been underexplored within the existing historical literature. This paper focuses on the wide-ranging activities of Airy, his assistant astronomers, (...) telegraph engineers, clockmakers and others, which served to increase the reliability of the Royal Observatory's time service between the 1850s and 1870s. Airy and his collaborators aimed to mechanize and automate their telegraphic time distribution system in order to improve its accuracy and reliability. The accomplishment of such technological innovations was disseminated via public lectures, journal articles and correspondence with experts, secondary distributors of standard time and the general public. These communications were used to build public trust in the Greenwich time service. However, the unexplored archival material used in the present paper provides fresh insight into the unstable nature of the Greenwich time system, including its clear limits in terms of its scale of automation and degree of accuracy. (shrink)

The aim of this study was to report the results of a literature review of empirical studies on trust within the nurse–patient relationship. A search of electronic databases yielded 34 articles published between 1980 and 2011. Twenty-two studies used a qualitative design, and 12 studies used quantitative research methods. The context of most quantitative studies was nurse caring behaviours, whereas most qualitative studies focused on trust in the nurse–patient relationship. Most of the quantitative studies used a descriptive design, (...) while qualitative methods included the phenomenological approach, grounded theory, ethnography and interpretive interactionism. Data collection was mainly by questionnaires or interviews. Evidence from this review suggests that the development of trust is a relational phenomenon, and a process, during which trust could be broken and re-established. Nurses’ professional competencies and interpersonal caring attributes were important in developing trust; however, various factors may hinder the trusting relationship. (shrink)