The aim of this literature review was to compose a narrative review supported by a systematic approach to critically identify and examine concerns about accountability and the allocation of responsibility and legal liability as applied to the clinician and the technologist as applied the use of opaque AI-powered systems in clinical decision making. This review questions if it is permissible for a clinician to use an opaque AI system in clinical decision making and if a patient was harmed as a (...) result of using a clinician using an AIS’s suggestion, how would responsibility and legal liability be allocated? Literature was systematically searched, retrieved, and reviewed from nine databases, which also included items from three clinical professional regulators, as well as relevant grey literature from governmental and non-governmental organisations. This literature was subjected to inclusion/exclusion criteria; those items found relevant to this review underwent data extraction. This review found that there are multiple concerns about opacity, accountability, responsibility and liability when considering the stakeholders of technologists and clinicians in the creation and use of AIS in clinical decision making. Accountability is challenged when the AIS used is opaque, and allocation of responsibility is somewhat unclear. Legal analysis would help stakeholders to understand their obligations and prepare should an undesirable scenario of patient harm eventuate when AIS were used. (shrink)

BackgroundDuring the arrival of the COVID-19 pandemic, various professional ethical guidance was issued to (and for) health and social care professionals in England and Wales. Guidance can help to inform and support such professionals and their patients, clients and service users, but a plethora of guidance risked information overload, confusion, and inconsistency. MethodsDuring the early months of the pandemic, we undertook a rapid review, asking: what are the principles adopted by professional ethical guidance in England and Wales for dealing with (...) COVID-19? We undertook thematic content analysis of the 29 documents that met our inclusion criteria.ResultsThe 29 documents captured 13 overlapping principles: respect, fairness, minimising harm, reciprocity, proportionality, flexibility, working together, inclusiveness, communication, transparency, reasonableness, responsibility, and accountability.ConclusionsWe intend this attempt to collate and outline the prominent principles to be helpful, particularly, for healthcare practice during the COVID-19 pandemic and, hopefully, for future pandemic planning. We also offer some reflections on the guidance and the principles therein. After describing the principles, we reflect on some of the similarities and differences in the guidance, and the challenges associated not only with the specific guidance reviewed, but also with the nature and import of “professional ethical guidance”. (shrink)

Much has been written about the need for trustworthy artificial intelligence (AI), but the underlying meaning of trust and trustworthiness can vary or be used in confusing ways. It is not always clear whether individuals are speaking of a technology’s trustworthiness, a developer’s trustworthiness, or simply of gaining the trust of users by any means. In sociotechnical circles, trustworthiness is often used as a proxy for ‘the good’, illustrating the moral heights to which technologies and developers ought to aspire, at (...) times with a multitude of diverse requirements; or at other times, no specification at all. In philosophical circles, there is doubt that the concept of trust should be applied at all to technologies rather than their human creators. Nevertheless, people continue to intuitively reason about trust in technologies in their everyday language. This qualitative study employed an empirical ethics methodology to address how developers and users define and construct requirements for trust throughout development and use, through a series of interviews. We found that different accounts of trust (rational, affective, credentialist, norms based, relational) served as the basis for individual granting of trust in technologies and operators. Ultimately, the most significant requirement for user trust and assessment of trustworthiness was the accountability of AI developers for the outputs of AI systems, hinging on the identification of accountable moral agents and perceived value alignment between the user and developer’s interests. (shrink)

Artificially intelligent systems (AISs) are being created by software developing companies (SDCs) to influence clinical decision‐making. Historically, clinicians have led healthcare decision‐making, and the introduction of AISs makes SDCs novel actors in the clinical decision‐making space. Although these AISs are intended to influence a clinician's decision‐making, SDCs have been clear that clinicians are in fact the final decision‐makers in clinical care, and that AISs can only inform their decisions. As such, the default position is that clinicians should hold responsibility for (...) the outcomes of the use of AISs. This is not the case when an AIS has influenced a clinician's judgement and their subsequent decision. In this paper, we argue that this is an imbalanced and unjust position, and that careful thought needs to go into how personal moral responsibility for the use of AISs in clinical decision‐making should be attributed. This paper employs and examines the difference between prospective and retrospective responsibility and considers foreseeability as key in determining how personal moral responsibility can be justly attributed. This leads us to the view that moral responsibility for the outcomes of using AISs in healthcare ought to be shared by the clinical users and SDCs. (shrink)

With the introduction of artificial intelligence (AI) to healthcare, there is also a need for professional guidance to support its use. New (2022) reports from National Health Service AI Lab & Health Education England focus on healthcare workers’ understanding and confidence in AI clinical decision support systems (AI-CDDSs), and are concerned with developing trust in, and the trustworthiness of these systems. While they offer guidance to aid developers and purchasers of such systems, they offer little specific guidance for the clinical (...) users who will be required to use them in patient care.This paper argues that clinical, professional and reputational safety will be risked if this deficit of professional guidance for clinical users of AI-CDDSs is not redressed. We argue it is not enough to develop training for clinical users without first establishing professional guidance regarding the rights and expectations of clinical users.We conclude with a call to action for clinical regulators: to unite to draft guidance for users of AI-CDDS that helps manage clinical, professional and reputational risks. We further suggest that this exercise offers an opportunity to address fundamental issues in the use of AI-CDDSs; regarding, for example, the fair burden of responsibility for outcomes. (shrink)

Our aim was to employ a critical analytic lens to explicate the role of nursing research in supporting the notion of caring realities. To do this, we used case exemplars to illustrate the infusion of such discourses. The first exemplar examines the fundamental concept of caring: using Florence Nightingale's Notes on Nursing, the case study surfaces caring as originally grounded in ritualized practice and subsequently describes its transmutation, via competing discourses, to a more holistic concept. It is argued that in (...) the many and varied attempts to define the dynamic concept of care, caring has now become paradoxically, a more fragmented concept despite attempts to render it more holistic and inclusive. In the second exemplar, one of the authors draws on her personal experience of the gap between theory and practice, so pronounced that it pushed the author to revisit the concept of evidence‐based practice and nursing education. In our third and final exemplar, we refer to the absence of knowledge and practice generated through natural enquiry and curiosity, an absence which has led to production of corporate led rhetoric. Drawing together the central arguments of the three exemplars, we reflect on the influential role of nursing research in enabling the deconstruction of taken for granted assumptions such as caring, evidence‐based practice and empowerment; assumptions which have been generated by discourses riddled with confusion and alienation from the reality of practice and the natural spirit of professional enquiry. (shrink)