Patient data is used to drive an ecosystem of advanced digital tools in health care, like predictive models or artificial intelligence‐based decision support. Patients themselves, however, receive little information about these technologies or how they affect their care. This raises important questions about patient trust and continued engagement in a health care system that extracts their data but does not treat them as key stakeholders. This essay explores these tensions and provides steps forward for health systems as they design advanced (...) health information‐technology (IT) policies and practices. It centers patients, their concerns, and the ways they perceive trustworthiness to reframe advanced health IT in service of patient interests. (shrink)

While experience often affords important knowledge and insight that is difficult to garner through observation or testimony alone, it also has the potential to generate conflicts of interest and unrepresentative perspectives. We call this tension the paradox of experience. In this paper, we first outline appeals to experience made in debates about access to unproven medical products and disability bioethics, as examples of how experience claims arise in bioethics and some of the challenges raised by these claims. We then motivate (...) the idea that experience can be an asset by appealing to themes in feminist and moral epistemology, distinguishing between epistemic and justice-based appeals. Next, we explain the concern that experience may be a liability by appealing to empirical work on cognitive biases and theoretical work about the problem of partial representation. We conclude with preliminary recommendations for addressing the paradox and offer several questions for future discussion. (shrink)

McCradden et al. propose to close the “AI chasm” between algorithms and clinically meaningful application using the norms of evidence-based medicine and clinical research, with the rat...

Rapid advancements in artificial intelligence and machine learning (AI/ML) in healthcare raise pressing questions about how much users should trust AI/ML systems, particularly for high stakes clinical decision-making. Ensuring that user trust is properly calibrated to a tool’s computational capacities and limitations has both practical and ethical implications, given that overtrust or undertrust can influence over-reliance or under-reliance on algorithmic tools, with significant implications for patient safety and health outcomes. It is, thus, important to better understand how variability in trust (...) criteria across stakeholders, settings, tools and use cases may influence approaches to using AI/ML tools in real settings. As part of a 5-year, multi-institutional Agency for Health Care Research and Quality-funded study, we identify trust criteria for a survival prediction algorithm intended to support clinical decision-making for left ventricular assist device therapy, using semistructured interviews (n=40) with patients and physicians, analysed via thematic analysis. Findings suggest that physicians and patients share similar empirical considerations for trust, which were primarilyepistemicin nature, focused on accuracy and validity of AI/ML estimates. Trust evaluations considered the nature, integrity and relevance of training data rather than the computational nature of algorithms themselves, suggesting a need to distinguish ‘source’ from ‘functional’ explainability. To a lesser extent, trust criteria were also relational (endorsement from others) and sometimes based on personal beliefs and experience. We discuss implications for promoting appropriate and responsible trust calibration for clinical decision-making use AI/ML. (shrink)

Big data and artificial intelligence (“AI”) promise to transform virtually all aspects of biomedical research and health care (Matheny et al. 2019), through facilitation of drug development, diagno...

The application of machine learning in health care holds great promise for improving care. Indeed, our own team is collaborating with experts in machine learning and statistical modeling to bu...

Just last week (October 4, 2022), the U.S. White House released a blueprint for an A.I. Bill of Rights, consisting of “five principles and associated practices to help guide the design, use, and de...