Skip to main content

Research Repository

Advanced Search

The need to strengthen the evaluation of the impact of Artificial Intelligence-based decision support systems on healthcare provision

Cresswell, Kathrin; Rigby, Michael; Magrabi, Farah; Scott, Philip; Brender, Jytte; Craven, Catherine K.; Wong, Zoie Shui-Yee; Kukhareva, Polina; Ammenwerth, Elske; Georgiou, Andrew; Medlock, Stephanie; De Keizer, Nicolette F.; Nykänen, Pirkko; Prgomet, Mirela; Williams, Robin

Authors

Kathrin Cresswell

Michael Rigby

Farah Magrabi

Philip Scott

Jytte Brender

Catherine K. Craven

Zoie Shui-Yee Wong

Polina Kukhareva

Elske Ammenwerth

Andrew Georgiou

Stephanie Medlock

Nicolette F. De Keizer

Pirkko Nykänen

Mirela Prgomet

Robin Williams



Abstract

Despite the renewed interest in Artificial Intelligence-based clinical decision support systems (AI-CDS), there is still a lack of empirical evidence supporting their effectiveness. This underscores the need for rigorous and continuous evaluation and monitoring of processes and outcomes associated with the introduction of health information technology.

We illustrate how the emergence of AI-CDS has helped to bring to the fore the critical importance of evaluation principles and action regarding all health information technology applications, as these hitherto have received limited attention. Key aspects include assessment of design, implementation and adoption contexts; ensuring systems support and optimise human performance (which in turn requires understanding clinical and system logics); and ensuring that design of systems prioritises ethics, equity, effectiveness, and outcomes.

Going forward, information technology strategy, implementation and assessment need to actively incorporate these dimensions. International policy makers, regulators and strategic decision makers in implementing organisations therefore need to be cognisant of these aspects and incorporate them in decision-making and in prioritising investment. In particular, the emphasis needs to be on stronger and more evidence-based evaluation surrounding system limitations and risks as well as optimisation of outcomes, whilst ensuring learning and contextual review. Otherwise, there is a risk that applications will be sub-optimally embodied in health systems with unintended consequences and without yielding intended benefits.

Citation

Cresswell, K., Rigby, M., Magrabi, F., Scott, P., Brender, J., Craven, C. K., …Williams, R. (2023). The need to strengthen the evaluation of the impact of Artificial Intelligence-based decision support systems on healthcare provision. Health Policy, 136, Article 104889. https://doi.org/10.1016/j.healthpol.2023.104889

Journal Article Type Article
Acceptance Date Aug 4, 2023
Online Publication Date Aug 12, 2023
Publication Date 2023-10
Deposit Date Jan 24, 2024
Journal Health Policy
Print ISSN 0168-8510
Publisher Elsevier
Peer Reviewed Peer Reviewed
Volume 136
Article Number 104889
DOI https://doi.org/10.1016/j.healthpol.2023.104889
Keywords Health Policy
Publisher URL https://www.sciencedirect.com/science/article/pii/S0168851023001744?via%3Dihub
Additional Information This article is maintained by: Elsevier; Article Title: The need to strengthen the evaluation of the impact of Artificial Intelligence-based decision support systems on healthcare provision; Journal Title: Health Policy; CrossRef DOI link to publisher maintained version: https://doi.org/10.1016/j.healthpol.2023.104889; Content Type: article; Copyright: © 2023 The Author(s). Published by Elsevier B.V.