#digitalethics2024, #noteType/litnote

"Who is afraid of black box algorithms? On the epistemological and ethical basis of trust in medical AI", by Juan Manuel Durán.

Commentary

This article is an examination of the tension between reliability and transparency in the context of AI usage in healthcare. In the article, the authors argue that block box algorithms may pose less of a threat than the literature makes them out to be. Transparency is not always needed in their view, arguing that reliability alone provides enough reason for trusting AI algorithms, coining the term 'computational reliabilism'. Ultimately, they argue that physicians collaborating with black box AI systems may contribute greatly to the improvement of medical care.

Excerpts & Key Quotes

The foundation of computational reliabilism

Page 332: "Computational reliabilism (CR) offers epistemic
justification for the belief that the algorithm is reliable and its
results are trustworthy. This, without necessitating to rely on external
algorithms or relinquishing black box algorithms altogether."

My Comment: In this passage, the authors outline the core concept of
computational reliabilism. They pose it as a solution to the epistemic
opacity that is presented by black box algorithms. Computational
reliabilism (CR) is presented as a framework used to justify the trust
we can put in algorithmic outputs based on their reliability, rather
than just their transparency. CR becomes a solution to epistemic opacity
in a unusual way, as it makes no actual attempts to solve it. What the
authors propose is a paradigm shift in how trust in medical AI can be
epistemically supported by admitting our own cognitive limitations,
circumventing epistemic concerns.

Ethical limits of reliability

Page 331: "Having justified knowledge from reliable indicators is,
therefore, necessary but not sufficient for normatively justifying
physicians to act."

My Comment: In this excerpt, the insufficiency of reliability for
ethical action is highlighted, showing that ethical medical practice
requires more than just algorithmic reliability. It also argues that we
need to have a comprehensive ethical evaluation, which considers the
implications of decisions made on the basis of reliable (yet opaque)
algorithmic recommendations. This questions the completeness od
computational reliabilism as a justification for medical actions on its
own.

Human oversight as an ethical imperative

Page 334: "Keeping humans in the loop of decision-making by algorithms is crucial to ensure that the integration of AI in medicine is both effective and ethical."

My Comment: This excerpt shows the importance of maintaining human
involvement in the decision-making processes when using AI systems,
especially in the context of healthcare, which is sensitive,
human-centered and complex. The authors argue for a model in which AI systems do not work in isolation. Rather, they should work in
collaboration with human medical professionals. This way, there is a
safeguard against risks associated with the epistemic opacity that is
faced when using black box algorithms.