jmir-2020-2-e16866
jmir-2020-2-e16866
jmir-2020-2-e16866
Review
Justus Wolff1,2*, MSc; Josch Pauling1*, PhD; Andreas Keck2*, MD; Jan Baumbach1*, PhD
1
TUM School of Life Sciences Weihenstephan, Technical University of Munich, Freising, Germany
2
Strategy Institute for Digital Health, Hamburg, Germany
*
all authors contributed equally
Corresponding Author:
Justus Wolff, MSc
TUM School of Life Sciences Weihenstephan
Technical University of Munich
Maximus-von-Imhof-Forum 3
Freising, 85354
Germany
Phone: 49 40329012 0
Email: justus.wolff@syte-institute.com
Abstract
Background: Positive economic impact is a key decision factor in making the case for or against investing in an artificial
intelligence (AI) solution in the health care industry. It is most relevant for the care provider and insurer as well as for the
pharmaceutical and medical technology sectors. Although the broad economic impact of digital health solutions in general has
been assessed many times in literature and the benefit for patients and society has also been analyzed, the specific economic
impact of AI in health care has been addressed only sporadically.
Objective: This study aimed to systematically review and summarize the cost-effectiveness studies dedicated to AI in health
care and to assess whether they meet the established quality criteria.
Methods: In a first step, the quality criteria for economic impact studies were defined based on the established and adapted
criteria schemes for cost impact assessments. In a second step, a systematic literature review based on qualitative and quantitative
inclusion and exclusion criteria was conducted to identify relevant publications for an in-depth analysis of the economic impact
assessment. In a final step, the quality of the identified economic impact studies was evaluated based on the defined quality criteria
for cost-effectiveness studies.
Results: Very few publications have thoroughly addressed the economic impact assessment, and the economic assessment
quality of the reviewed publications on AI shows severe methodological deficits. Only 6 out of 66 publications could be included
in the second step of the analysis based on the inclusion criteria. Out of these 6 studies, none comprised a methodologically
complete cost impact analysis. There are two areas for improvement in future studies. First, the initial investment and operational
costs for the AI infrastructure and service need to be included. Second, alternatives to achieve similar impact must be evaluated
to provide a comprehensive comparison.
Conclusions: This systematic literature analysis proved that the existing impact assessments show methodological deficits and
that upcoming evaluations require more comprehensive economic analyses to enable economic decisions for or against implementing
AI technology in health care.
KEYWORDS
telemedicine; artificial intelligence; machine learning; cost-benefit analysis
In 2002, in a review of cost-effectiveness studies in the context As significant medical quality enhancements and cost-saving
of telemedicine interventions, Whitten et al [3] revealed that improvements through artificial intelligence (AI) as one of the
only 55 out of 612 identified articles presented actual key emerging technologies in digital health are expected, the
cost-benefit data, which were required to be included in a economic impact assessment of AI in health care has a crucial
detailed review. In addition, after analyzing these articles, the role for all stakeholders in health care and, thus, needs to be
authors concluded that the provided evidence was not sufficient analyzed in detail.
to assess whether telemedicine represents a cost-effective mean Objective
of delivering health care [3].
It was systematically investigated whether the existing
More than a decade later, in 2014, Elbert et al [4] described in cost-effectiveness evaluations meet the established quality
a review of systematic reviews and meta-analyses regarding criteria to enable comprehensive decision making regarding the
electronic health (eHealth) interventions in somatic diseases implementation of AI in health care. On the basis of these
that out of 31 reviews, 7 papers concluded that digital health is thorough economic assessments, the necessary information to
effective or cost-effective, 13 underlined that evidence is decide for or against the application of AI in hospitals, industry,
promising, and the other 11 found only limited or inconsistent and payer context will be provided.
proof. They also highlighted that the development and evaluation
of strategies to implement effective or cost-effective eHealth Methods
initiatives in daily practice needed to be significantly enhanced
[4]. A systematic literature review was performed as described in
the following sections.
In another systematic review study on the economic evaluations
of eHealth technologies from 2018, Sanyal et al [5] analyzed Search Strategies
multiple databases with publications between 2010 and 2016. A literature search was conducted utilizing the PubMed database
On the basis of 11 studies that fulfilled the inclusion criteria, and using the search terms provided in Table 1.
the authors found that most of the studies demonstrated efficacy
Table 1. Search terms (title and abstract) in the PubMed analysis (conducted on July 29, 2019).
Components Syntax Hits, n
Artificial intelligence OR machine learning AND cost (Artificial intelligence [title/abstract] OR machine learning [title/abstract]) AND 54
effectiveness cost effectiveness [title/abstract]
Artificial intelligence OR machine learning AND (Artificial intelligence [title/abstract] OR machine learning [title/abstract]) AND 9
economic impact economic impact [title/abstract]
Artificial intelligence OR machine learning AND cost (Artificial intelligence [title/abstract] OR machine learning [title/abstract]) AND 3
saving cost saving [title/abstract]
The search terms Artificial Intelligence and Machine Learning intelligence, support vector machines, and machine learning at
for the overall segment are not exhaustive as eg, Decision trees, least in the abstract. Finally, it was decided to use a Google
Support vector machines, or Deep neural networks could also Trends analysis comparing the most frequently used search
have been used as search terms for the database queries. terms regarding AI in health care over the last 12 months
Nonetheless, as strategic decisions based on economic impact globally [9]: The terms Artificial Intelligence and Machine
are mostly made on a strategic managerial and medical level Learning have been used the most by far, as illustrated in
without a specific technological background, the most frequently Multimedia Appendix 1.
used search terms regarding AI in health care have been used.
In addition, it is highly probable that papers about, for example,
deep neural networks would also include such terms as artificial
a
AI: artificial intelligence.
of the different search terms that include the most frequently Room for improvement (Figure 2) has been identified in two
searched phrases by far in this segment (Artificial Intelligence main areas:
and Machine Learning) in combination with the economic
• First, initial investment and operational costs for the AI
impact (Cost effectiveness, Economic impact, Cost saving),
infrastructure and service need to be included in the
there were only 66 PubMed hits. As AI strategies and
assessment. This is a core element for any strategic
consequent decision-making processes depend on solid data as
decision-making process, and the complete initial and
the basis for decision making, this is a significant challenge for
operational investment costs for an AI solution must be
both the management and medical staff, for example, when
compared with the expected economic benefits to provide
general pro and contra decisions and specific implementations
concrete decision-making support.
regarding AI are discussed.
• Second, further options to achieve similar impact must be
When accounting for the details given in the identified AI in evaluated to reach a sufficient basis for comprehensive and
health care publications, the economic assessment quality shows transparent decision-making, allowing comparisons among
several deficits that need to be overcome in the future. Only 6 different strategic and investment options (eg, a genetic
out of the 66 publications (9%) could be included in the detailed sequencing process or different medical expertise allocation
assessment. Out of these 6 studies, none comprised a complete for a diagnosis and treatment outcome improvement could
cost-benefit analysis; rather, they all focused on fragmented also be applied instead of an AI-driven patient screening).
cost or cost-saving aspects.
Figure 2. Result of the literature review and improvement areas for economic impact assessment of artificial intelligence (AI) in health care.
The conducted review has a rather narrow focus on economics society require further research. For instance, in rural areas
and business perspectives of AI in health care. However, the where the availability of primary care physicians is limited, AI
literature review revealed further significant success factors for can replace processes through focused test support, for example,
AI, for example, regarding the legal framework, such as for type 2 diabetes, thus addressing the challenges of
compliance with data security, protection, and privacy policies, demographic change [18]. The comparison between AI and
and also universally accepted technological requirements to physicians with regard to diagnosis performance demonstrated
enable comprehensive data collection and to analyze content that AI can deliver equal results, for example, in image
while complying with data privacy requirements. Despite the recognition–related fields [19]. This can, among others, also
benefits in assisting diagnostic and therapeutic decisions, so support a reallocation of medical capacities. In addition, AI can
far, no standards for these legal and technological issues have also enable a shift from a generalized to a more personalized
been defined, and these aspects should be analyzed in future treatment. AI-steered outcome prediction and clinical decision
research with a broader focus. support processes are already used today, for instance, for
patients in radiation therapy [20].
Furthermore, aside from the sole economic quantitative aspects,
the qualitative aspects of AI in health care for patients and the
Prior reviews in the digital health segment categorized the results (31/1657) in the study by Elbert et al [4], or 0.1% (11/1625) in
into groups, for example, computerized decision support system, the study by Sanyal et al [5]). This was because of two reasons:
Web-based physical activity intervention, internet-delivered (1) AI as a subsegment of digital health in business and industry
cognitive behavioral therapy, and telehealth. In addition, user’s is still not covered well in scientific publications and (2) the
age was differentiated (eg, children vs old patients), and high importance of quantitatively reported outcomes required
shortcomings such as a missing difference between short- and as inclusion criterion. Third, the evaluation of cost-effectiveness
long-term cost savings were highlighted [5]. They also covered studies has been conducted with a quality criteria catalog from
challenges that go beyond the cost-effectiveness aspect and a management perspective. As AI implementation is cost- and
mentioned, for instance, that the way to implement digital health labor-intensive and decisions are not exclusively driven by
in daily practice is still unclear [4] or that patient perspectives medical improvement rates, the business management decision
and collaborative approaches among a variety of stakeholders making basis has been chosen as crucial for positive
are needed [10]. implementation decisions and subsequent widescale
applications. The addition of the business management view
Note that the focus on AI in health care required considering
includes classical cost factors (onetime and running expenses)
novel factors and a refined search strategy as compared with
as well as decisions among different strategies to deliver cutting
typical reviews on digital health resulting in differential results.
edge health services.
First, in contrast to other reviews, Google Trends has proven
to be an effective tool to narrow the search space for a Conclusions
representative collection of results. On the basis of the Google Current research covers impact assessments of AI in health care
Trends analysis, the key phrases Artificial Intelligence and rather moderately and shows qualitative deficits in methodology.
Machine Learning could be identified as the most frequently Future cost-effectiveness analyses need to increase in number
used terms by far. Second, the review covered a higher and quality. They should include initial investment and running
percentage of included studies after applying the defined costs as well as the comparison with alternative technologies.
inclusion and exclusion criteria (9% of the analyzed papers were This way a comprehensive and clearly segmented cost-benefit
included), whereas prior reviews had much lower inclusion evaluation can be provided, which will serve as a sufficient
rates—8% (55/612) in the study by Whitten et al [3], 2% basis for decision making regarding AI implementations.
Acknowledgments
The authors would like to thank Julia Menacher for her editorial support in creating this publication. JB is grateful for financial
support from Villum Young Investigator grant number 13154. In addition, some of the work from JB was funded by H2020
project RepoTrial (number 777111). Contributions by JP are funded by the Bavarian State Ministry of Science and the Arts in
the framework of the Center Digitisation.Bavaria (Zentrum Digitalisierung.Bayern).
Conflicts of Interest
None declared.
Multimedia Appendix 1
Screenshot of a Google Trends analysis of search terms related to artificial intelligence in health care globally over the last 12
months (conducted on October 9, 2019).
[PNG File , 176 KB-Multimedia Appendix 1]
Multimedia Appendix 2
Analysis of the included economic impact studies.
[PNG File , 396 KB-Multimedia Appendix 2]
References
1. Phelps CE. Health Economics. Abingdon, Oxfordshire: Routledge; 2017.
2. Statistisches Bundesamt. Health Expenditure in 2017: +4.7% URL: https://www.destatis.de/DE/Presse/Pressemitteilungen/
2019/03/PD19_109_23611.html [accessed 2019-10-03]
3. Whitten PS, Mair FS, Haycox A, May CR, Williams TL, Hellmich S. Systematic review of cost effectiveness studies of
telemedicine interventions. Br Med J 2002 Jun 15;324(7351):1434-1437 [FREE Full text] [doi: 10.1136/bmj.324.7351.1434]
[Medline: 12065269]
4. Elbert N, van Os-Medendorp H, van Renselaar W, Ekeland A, Hakkaart-van Roijen L, Raat H, et al. Effectiveness and
cost-effectiveness of eHealth interventions in somatic diseases: a systematic review of systematic reviews and meta-analyses.
J Med Internet Res 2014 Apr 16;16(4):e110 [FREE Full text] [doi: 10.2196/jmir.2790] [Medline: 24739471]
5. Sanyal C, Stolee P, Juzwishin D, Husereau D. Economic evaluations of eHealth technologies: a systematic review. PLoS
One 2018;13(6):e0198112 [FREE Full text] [doi: 10.1371/journal.pone.0198112] [Medline: 29897921]
http://www.jmir.org/2020/2/e16866/ J Med Internet Res 2020 | vol. 22 | iss. 2 | e16866 | p. 6
(page number not for citation purposes)
XSL• FO
RenderX
JOURNAL OF MEDICAL INTERNET RESEARCH Wolff et al
6. Buvik A, Bergmo TS, Bugge E, Smaabrekke A, Wilsgaard T, Olsen JA. Cost-effectiveness of Telemedicine in remote
orthopedic consultations: randomized controlled trial. J Med Internet Res 2019 Feb 19;21(2):e11330 [FREE Full text] [doi:
10.2196/11330] [Medline: 30777845]
7. Nordyke RJ, Appelbaum K, Berman MA. Estimating the impact of novel digital therapeutics in Type 2 diabetes and
hypertension: health economic analysis. J Med Internet Res 2019 Oct 9;21(10):e15814 [FREE Full text] [doi: 10.2196/15814]
[Medline: 31599740]
8. Le LK, Sanci L, Chatterton ML, Kauer S, Buhagiar K, Mihalopoulos C. The cost-effectiveness of an internet intervention
to facilitate mental health help-seeking by young adults: randomized controlled trial. J Med Internet Res 2019 Jul
22;21(7):e13065 [FREE Full text] [doi: 10.2196/13065] [Medline: 31333199]
9. Google Trends. Compare URL: https://trends.google.com/trends/
explore?cat=45&q=Artificial%20Intelligence,Machine%20learning,Neural%20network,Linear%20regression,Support%20vector%20%20
machine [accessed 2019-10-09]
10. Ekeland AG, Bowes A, Flottorp S. Effectiveness of telemedicine: a systematic review of reviews. Int J Med Inform 2010
Nov;79(11):736-771. [doi: 10.1016/j.ijmedinf.2010.08.006] [Medline: 20884286]
11. Haycox A, Walley T. Pharmacoeconomics: evaluating the evaluators. Br J Clin Pharmacol 1997 May;43(5):451-456 [FREE
Full text] [doi: 10.1046/j.1365-2125.1997.00575.x] [Medline: 9159558]
12. Lee HK, Jin R, Feng Y, Bain PA, Goffinet J, Baker C, et al. An analytical framework for TJR readmission prediction and
cost-effective intervention. IEEE J Biomed Health Inform 2019 Jul;23(4):1760-1772. [doi: 10.1109/JBHI.2018.2859581]
[Medline: 30047916]
13. Gönel A. Clinical biochemistry test eliminator providing cost-effectiveness with five algorithms. Acta Clin Belg 2018 Dec
25:1-5. [doi: 10.1080/17843286.2018.1563324] [Medline: 30585134]
14. Bremer V, Becker D, Kolovos S, Funk B, van Breda W, Hoogendoorn M, et al. Predicting therapy success and costs for
personalized treatment recommendations using baseline characteristics: Data-driven analysis. J Med Internet Res 2018 Aug
21;20(8):e10275 [FREE Full text] [doi: 10.2196/10275] [Medline: 30131318]
15. Grover D, Bauhoff S, Friedman J. Using supervised learning to select audit targets in performance-based financing in
health: an example from Zambia. PLoS One 2019;14(1):e0211262 [FREE Full text] [doi: 10.1371/journal.pone.0211262]
[Medline: 30695057]
16. Lee I, Monahan S, Serban N, Griffin P, Tomar S. Estimating the cost savings of preventive dental services delivered to
Medicaid-enrolled children in six southeastern states. Health Serv Res 2018 Oct;53(5):3592-3616 [FREE Full text] [doi:
10.1111/1475-6773.12811] [Medline: 29194610]
17. Golas SB, Shibahara T, Agboola S, Otaki H, Sato J, Nakae T, et al. A machine learning model to predict the risk of 30-day
readmissions in patients with heart failure: a retrospective analysis of electronic medical records data. BMC Med Inform
Decis Mak 2018 Jun 22;18(1):44 [FREE Full text] [doi: 10.1186/s12911-018-0620-z] [Medline: 29929496]
18. Spänig S, Emberger-Klein A, Sowa J, Canbay A, Menrad K, Heider D. The virtual doctor: an interactive
clinical-decision-support system based on deep learning for non-invasive prediction of diabetes. Artif Intell Med 2019
Sep;100:101706. [doi: 10.1016/j.artmed.2019.101706] [Medline: 31607340]
19. Shen J, Zhang CJ, Jiang B, Chen J, Song J, Liu Z, et al. Artificial Intelligence versus clinicians in disease diagnosis:
systematic review. JMIR Med Inform 2019 Aug 16;7(3):e10010 [FREE Full text] [doi: 10.2196/10010] [Medline: 31420959]
20. Pillai M, Adapa K, Das SK, Mazur L, Dooley J, Marks LB, et al. Using Artificial Intelligence to improve the quality and
safety of radiation therapy. J Am Coll Radiol 2019 Sep;16(9 Pt B):1267-1272. [doi: 10.1016/j.jacr.2019.06.001] [Medline:
31492404]
Abbreviations
AI: artificial intelligence
eHealth: electronic health
GDP: gross domestic product
NPV: net present value
Edited by G Eysenbach; submitted 01.11.19; peer-reviewed by D Heider, N Przulj, E van der Velde; comments to author 07.12.19;
revised version received 18.12.19; accepted 19.12.19; published 20.02.20
Please cite as:
Wolff J, Pauling J, Keck A, Baumbach J
The Economic Impact of Artificial Intelligence in Health Care: Systematic Review
J Med Internet Res 2020;22(2):e16866
URL: http://www.jmir.org/2020/2/e16866/
doi: 10.2196/16866
PMID: 32130134
©Justus Wolff, Josch Pauling, Andreas Keck, Jan Baumbach. Originally published in the Journal of Medical Internet Research
(http://www.jmir.org), 20.02.2020. This is an open-access article distributed under the terms of the Creative Commons Attribution
License (https://creativecommons.org/licenses/by/4.0/), which permits unrestricted use, distribution, and reproduction in any
medium, provided the original work, first published in the Journal of Medical Internet Research, is properly cited. The complete
bibliographic information, a link to the original publication on http://www.jmir.org/, as well as this copyright and license information
must be included.