Trustworthy medical AI systems need to know when they don’t know

Journal of Medical Ethics 47 (5):337-338 (2021)


There is much to learn from Durán and Jongsma’s paper.1 One particularly important insight concerns the relationship between epistemology and ethics in medical artificial intelligence. In clinical environments, the task of AI systems is to provide risk estimates or diagnostic decisions, which then need to be weighed by physicians. Hence, while the implementation of AI systems might give rise to ethical issues—for example, overtreatment, defensive medicine or paternalism2—the issue that lies at the heart is an epistemic problem: how can physicians know whether to trust decisions made by AI systems? In this manner, various studies examining the interaction of AI systems and physicians have shown that without being able to evaluate their trustworthiness, especially novice physicians become over-reliant on algorithmic support—and ultimately are led astray by incorrect decisions.3–5 This leads to a second insight from the paper, namely that even if some AI system happens to be opaque, it is still not built on the moon. To assess its trustworthiness, AI developers or physicians have different sorts of higher order evidence at hand. Most importantly, …

Download options


    Upload a copy of this work     Papers currently archived: 72,855

External links

Setup an account with your affiliations in order to access resources via your University's proxy server

Through your library


Added to PP

27 (#428,380)

6 months
7 (#101,828)

Historical graph of downloads
How can I increase my downloads?

Author's Profile

Citations of this work

No citations found.

Add more citations

Similar books and articles

The Importance of Being Trustworthy.Derek Sellman - 2006 - Nursing Ethics 13 (2):105-115.
Trustworthiness of Autonomous Systems.S. Kate Devitt - 2018 - In Hussein A. Abbass, Jason Scholz & Darryn Reid (eds.), Foundations of Trusted Autonomous Systems. Springer. pp. 161-184.
E-Trust and Reputation.Thomas W. Simpson - 2011 - Ethics and Information Technology 13 (1):29-38.
The Rise of Medical Systems and the Fall of Medical Tradition.David B. Schwartz - 1999 - Bulletin of Science, Technology and Society 19 (3):214-218.
Trustworthy Research—an Editorial Introduction.Caroline Whitbeck - 1995 - Science and Engineering Ethics 1 (4):322-328.
System and Disease: On the Fundamental Problem of Theoretical Pathology.Peter Hucklenbroich - 1984 - Theoretical Medicine: An International Journal for the Philosophy and Methodology of Medical Research and Practice 5:307-324.
Xin: Being Trustworthy.Winnie Sung - 2020 - International Philosophical Quarterly 60 (3):271-286.