The Use of Artificial Intelligence in Pharmacovigilance: A Systematic Review of The Literature
The Use of Artificial Intelligence in Pharmacovigilance: A Systematic Review of The Literature
The Use of Artificial Intelligence in Pharmacovigilance: A Systematic Review of The Literature
https://doi.org/10.1007/s40290-022-00441-z
SYSTEMATIC REVIEW
Abstract
Introduction Artificial intelligence through machine learning uses algorithms and prior learnings to make predictions.
Recently, there has been interest to include more artificial intelligence in pharmacovigilance of products already in the market
and pharmaceuticals in development.
Objective The aim of this study was to identify and describe the uses of artificial intelligence in pharmacovigilance through
a systematic literature review.
Methods Embase and MEDLINE database searches were conducted for articles published from January 1, 2015 to July 9,
2021 using search terms such as ‘pharmacovigilance,’ ‘patient safety,’ ‘artificial intelligence,’ and ‘machine learning’ in the
title or abstract. Scientific articles that contained information on the use of artificial intelligence in all modalities of patient
safety or pharmacovigilance were reviewed and synthesized using a pre-specified data extraction template. Articles with
incomplete information and letters to editor, notes, and commentaries were excluded.
Results Sixty-six articles were identified for evaluation. Most relevant articles on artificial intelligence focused on machine
learning, and it was used in patient safety in the identification of adverse drug events (ADEs) and adverse drug reactions
(ADRs) (57.6%), processing safety reports (21.2%), extraction of drug–drug interactions (7.6%), identification of popula-
tions at high risk for drug toxicity or guidance for personalized care (7.6%), prediction of side effects (3.0%), simulation of
clinical trials (1.5%), and integration of prediction uncertainties into diagnostic classifiers to increase patient safety (1.5%).
Artificial intelligence has been used to identify safety signals through automated processes and training with machine learning
models; however, the findings may not be generalizable given that there were different types of data included in each source.
Conclusion Artificial intelligence allows for the processing and analysis of large amounts of data and can be applied to vari-
ous disease states. The automation and machine learning models can optimize pharmacovigilance processes and provide a
more efficient way to analyze information relevant to safety, although more research is needed to identify if this optimization
has an impact on the quality of safety analyses. It is expected that its use will increase in the near future, particularly with
its role in the prediction of side effects and ADRs.
* Priyanka Yalamanchili
pyalamanchi@dsi.com
Extended author information available on the last page of the article
Vol.:(0123456789)
M. Salas et al.
Fig. 1 Preferred Reporting
Items for Systematic Reviews Identification of studies via databases and registers
and Meta-Analyses (PRISMA)
flow diagram showing
documentation of the literature
Identification
found that there is a tradeoff between the amount of manual Another novel approach is using deep-learning neural net-
screening needed in lower levels of social media processing works or prediction models to model the ADR relationship
with its potential to miss adverse events when compared between a medication and symptoms [27, 29]. Specifically,
with higher levels of social media processing that use natural E-Synthesis is a Bayesian framework for safety assessments
language processing (NLP) [43]. that compiles data to provide the Bayesian probability of a
Basile et al. identified polypharmacy and patient diver- drug causing an ADR [30]. This association can be critical
sity as some of the opportunities to use machine learning to pharmacovigilance efforts and analyzing the safety profile
in detecting ADRs [11]. These opportunities can be pre- of medications.
sent in multiple phases of drug development ranging from
pre-marketing to post-marketing safety assessments [13]. 3.5 Using Artificial Intelligence to Process Safety
The automation present in machine learning techniques is Reports
becoming increasingly more useful as patients continue to
present with multiple disease states, medications, and ADRs. Another application of machine learning in pharmacovigi-
Some institutions, such as Connecticut Children’s Medi- lance is in assessing the skill of NLP to classify unstruc-
cal Center, have utilized machine learning to successfully tured free text within patient safety incident reports. Evans
streamline the use of adverse event reports by comparing et al. tested the ability of autonomously classifying free text
rule-based queries and semi-supervised machine learning within patient safety incident reports to determine severity
against a reference standard [15]. Aside from being used to of harm outcomes and found that NLP can act as a safety net
detect ADRs, machine learning can also be used specifically by identifying cases that lead to severe harm or death [36].
to classify ADRs. Chauvet et al. determined the seriousness However, it is not a perfect method and cannot yet replace
of patient cases through different algorithms based on their manual review altogether [36]. Additionally, the technical
precision, recall, and accuracy [19]. nature of medical text makes this process difficult to com-
Artificial intelligence can also play an important role in plete [36].
specific disease states, such as diabetes. HypoDetect, a NLP Many studies evaluated the use of machine learning in
system which allows users to see blood glucose measure- screening patient safety reports, such as within electronic
ments displayed in a graphical format and analyze the meas- health records. Marella et al. found that machine learning
urements for hypoglycemic events using an algorithm, has algorithms and text mining are useful methods for screening
been useful in detecting hypoglycemia incidents from secure and analyzing large, semi-structured, or unstructured data
data inputs early so that treatment can be promptly initiated sets of adverse event and near-miss reports collected through
[20]. In disease states like diabetes where early identifica- passive surveillance reporting systems [57]. Yang et al.
tion of symptoms is critical to patient safety, systems such took a more specific approach by developing a deep learn-
as HypoDetect can improve safety efforts and patient out- ing model that was evaluated on different data sets to iden-
comes [20]. On a similar note, the under-reporting of safety tify allergic reactions in the free-text narrative of hospital
events can compromise patient safety and has been an issue safety reports and evaluated their generalizability [72]. The
in recent years [58]. Ménard et al. used a curated data set study found that the model could be used to improve allergy
from 104 completed Roche/Genentech sponsored clinical care, potentially enabling real-time event surveillance for
studies which included patient demographics, vitals, and dis- medical errors and system improvement [72]. Ultimately,
ease areas to train a machine learning model to predict the machine learning has the potential to be used in many ways
number of adverse events [58]. The model has the potential for addressing pharmacovigilance needs in various settings
to be useful for initiating quality assurance measures early such as identifying keywords in patient safety reports that
on and promptly filing potential adverse events [58]. This may require attention to prevent harm at clinical sites and
can be crucial to the safety of patients as every ADR needs post-marketing surveillance of ADRs in the pharmaceutical
to be properly assessed within a set time frame. industry.
A common theme in many of the articles was the ability
for machine learning to analyze a large amount of data to 3.6 Using Artificial Intelligence to Extract Drug–
gather information about the side effects of therapies, which Drug Interactions
can subsequently be used to improve pharmacovigilance sys-
tems [23]. One innovative approach to this involves using Artificial intelligence can be used to extract drug–drug inter-
propensity scores to present a new automated signal detec- actions or predict the effect of a drug–drug interaction. Ben
tion strategy for pharmacovigilance systems [26]. Under- Abacha et al. incorporated machine learning techniques with
standably, one of the issues that arise from such techniques is both feature-based and kernel-based methods for successful
providing a reasonable number of signals for further analysis drug–drug interaction extraction [14]. Bouzillé et al. used
by experts with the fewest possible false associations [26]. a method to automatically detect drug–drug interactions to
M. Salas et al.
improve drug safety monitoring in a hospital setting [16]. 3.9 Using Artificial Intelligence to Simulate Clinical
They created an efficient machine learning model using lab- Trials
oratory tests and treatment data that could detect patients
that may have had an ADE that was linked to a drug–drug Chen et al. used machine learning in conjunction with real-
interaction [16]. Machine learning can be particularly useful world data to simulate colorectal cancer clinical trials and
in pharmacovigilance because these models can learn from evaluate serious adverse events [21]. The risk ratios of seri-
a small number of drug–drug interaction combinations to ous adverse events measured from simulations comparing
predict many potential drug–drug interactions [33]. two treatment arms were very close to the risk ratios cal-
culated from the trials, thus showing the potential utility of
3.7 Using Artificial Intelligence to Identify Patients machine learning and real-world data in simulating clinical
at High Risk for ADRs trials [21].
Machine learning can be used to identify populations at 3.10 Using Artificial Intelligence to Integrate
high risk for experiencing ADRs or to guide personalized Prediction Uncertainties
care. Chandak and Tatonetti developed a machine learning
algorithm called “AwareDX: Analysing Women At Risk Artificial intelligence can also be used to integrate predic-
for Experiencing Drug toXicity,” that predicts sex-specific tion uncertainties in patient safety. Laves et al. quantified the
risks of adverse drug effects with high precision by using uncertainty of deep learning-based computer-aided diagno-
a machine learning adaptation of propensity score match- sis for patient safety [50]. The basis for the work relies on
ing [18]. Machine learning techniques can also be used to the concept that models that are trained for the diagnosis
identify more targeted patients, such as those susceptible of cases often do not have the capability to indicate when a
to fluoropyrimidine toxicity due to DPD deficiency [25]. case is too ambiguous for an output [50]. The study found
Investigators used machine learning models to train patterns that modeling prediction uncertainty with deep learning can
of toxicity, which were later used to estimate the number produce more dependable results that can assist with safety
of patients with toxicity related to DPD and found that the efforts [50].
model has potential for future use but could have some over-
fitting [25]. While there is still some progress left to be made
in the application of machine learning in identifying patients 4 Discussion
at high risk of ADRs, these techniques are an excellent start-
ing point. 4.1 Overall Use of Artificial Intelligence
3.8 Using Artificial Intelligence to Predict Drug Side The main uses of artificial intelligence in pharmacovigi-
Effects lance are in the identification of ADEs and ADRs, the per-
formance of surveillance and signal detection, classification
Like its use in identifying patients at high risk for experienc- of free text within safety reports, extraction of drug–drug
ing ADRs, machine learning has also been used to predict interactions, identification of populations at high risk of
side effects from drugs. Mower et al. focused on post-mar- experiencing drug toxicity, prediction of drug side effects,
keting drug surveillance and demonstrated that knowledge and simulation of clinical trials. These can be applied in
extracted from literature can add to the performance of many different aspects of pharmacovigilance, ranging from
spontaneous reporting system methods using downstream Individual Case Safety Reports (ICSRs) to adverse event
machine learning [60]. This can be particularly useful in profiles. The use of machine learning can optimize phar-
predicting drug side effects because spontaneous reporting macovigilance processes by automating case processing of
systems often have bias and under-reporting which can limit ICSRs and provide a more efficient way to analyze safety
the availability of data [60]. Wang et al. predicted potential information. As a result of this optimization, there is time
side effects and ADRs using a tumor-biomarker knowledge freed up for humans to focus on the interpretation and action
graph and determined that this method is useful for potential required to respond to safety events. However, it is important
ADR identification based on biomarkers [71]. The model to note that there is variability in tool performance and there
can be valuable for future applications that may require are challenges associated with implementing artificial intel-
mechanism-based research of ADRs [71]. ligence in pharmacovigilance practices. Some challenges
The Use of Artificial Intelligence in Pharmacovigilance
are the need for people to have a baseline comprehension outcomes can be missed by medical professionals. Machine
of how the artificial intelligence technology they are using learning offers an opportunity to systematically capture
works and privacy concerns with using artificial intelligence potential safety events before they occur or within an appro-
to store healthcare data [75, 76]. There are also legal chal- priate time frame for managing a safety event. Some meth-
lenges in both Europe and the United States of America ods can even be completed in real-time which offers addi-
related to the liability for errors that may occur as a result of tional improvement over manual methods. Understandably,
artificial intelligence technology [77, 78]. More experiences these systems have not been perfected yet and may never be
and applications of artificial intelligence in patient safety 100% accurate. However, when it comes to pharmacovigi-
and pharmacovigilance are needed before these methods can lance efforts, even slight improvements have the potential to
be validated and promoted for widespread use. The current enhance safety efforts. These methods can be applied in the
industry perspective is that there is interest in using tech- institutional setting with electronic health record evaluation
nologies such as machine learning, NLP, and Natural Lan- and the pharmaceutical industry setting with post-marketing
guage Generation across ICSR process steps; however, the surveillance of drug safety events. However, more research
challenges include using quality training data for machine is needed to validate these methods.
learning models and regulatory guidance [79].
4.4 Using Artificial Intelligence to Extract Drug–
Drug Interactions
4.2 Using Artificial Intelligence to Detect ADRs
Machine learning has been used to identify drug–drug
One of the most common uses of artificial intelligence in
interactions and their potential effects. The extraction of
the literature was in the detection of ADRs, surveillance,
drug–drug interactions in some studies has been shown to
and signal detection. Most specifically, the classification of
be efficient and have manageable computation time [16].
ADRs in social media was evaluated in many of these stud-
These efforts are particularly useful because predictions can
ies. Since more and more patients use social media platforms
be used to enhance post-marketing surveillance systems and
to express their response to medication regimens, this cre-
detect drug–drug interaction effects sooner during the drug
ates an excellent platform for machine learning to be applied.
development process [33]. Deep learning methods can even
However, many challenges arise from the unstructured free
predict novel drug–drug interactions, which may not be as
text that occurs on platforms such as Twitter and Facebook,
feasible without machine learning [65]. Identifying potential
since people tend to use casual or slang terms. This cre-
drug–drug interactions is very important for patient safety
ates the need for ‘smarter’ machine learning models that
and pharmacovigilance efforts, since certain medications
are able to reduce the ‘noise’ reported in these posts. The
can be avoided or more closely managed in patients who
advantages of machine learning in detecting ADRs include
are at risk of experiencing a drug–drug interaction.
its potential ability to process and analyze large amounts of
data quickly and its application to various disease states.
4.5 Using Artificial Intelligence to Identify Patients
The success of machine learning in analyzing ADE reports
at High Risk for ADRs
has already been shown in certain institutions, such as Con-
necticut Children’s Medical Center [15]. Machine learning
Personalized care is becoming more prominent in medicine
can potentially be useful in certain subsets of the population
and machine learning has a place in identifying patients who
which can result in better patient outcomes and relief of the
may be at higher risk of experiencing certain ADRs. By
healthcare system. The association between a drug and its
using machine learning models to predict those who may
ADR was analyzed in many studies which demonstrated the
be at risk of experiencing an ADR, patient safety can poten-
potential effectiveness and potential capabilities with the use
tially be improved. For example, one study trained a model
of machine learning in pharmacovigilance.
to identify patients who may be more susceptible to fluoro-
pyrimidine toxicity due to DPD deficiency [25]. This study
4.3 Using Artificial Intelligence to Process Safety showed how machine learning models can assist in imputing
Reports the likely genotype of a patient from phenotypical mani-
festations to understand the influence of DPD deficiency
NLP has been used to classify unstructured free text within [25]. When applied to large pharmacovigilance databases,
ICSRs, patient safety event reports, and clinical narratives. machine learning can assist to respond to such questions
This can be critical to pharmacovigilance efforts because which may not be possible with traditional methods [25].
sometimes cases that lead to harm or negative patient
M. Salas et al.
4.6 Using Artificial Intelligence to Predict Drug Side augment pharmacovigilance professionals’ processing of
Effects spontaneous ICSRs quickly and accurately [9]. Big data
technology could also improve drug safety monitoring in
Artificial intelligence can be used to predict potential side clinical settings and could help pharmacovigilance profes-
effects from drugs. While many side effects can be predicted sionals make targeted hypotheses on ADEs due to drug–drug
from the mechanism of action and pharmacology of a drug, interactions [16]. Many of the studies showed that machine
there are some side effects that have yet to be established. learning can have some role in pharmacovigilance efforts,
Machine learning can complement data that has already with the degree of its impact depending on the type of data-
been made available from previous reports. Wang et al. base or source it is used on. For example, ADR detection
were able to use machine learning techniques to show how performance in social media is significantly improved by
a mechanistic approach to identifying ADRs was able to dis- using a contextually aware model and word embeddings
cover potential ADRs of antitumor drugs [71]. This shows formed from large, unlabeled datasets, which can be scal-
how machine learning can be used in addition to traditional able to large social media datasets [22]. Machine learning
methods to potentially enhance patient safety. models applied to large pharmacovigilance databases can
help answer certain research questions, which may be diffi-
4.7 Using Artificial Intelligence to Simulate Clinical cult to address with more traditional methods [25]. Artificial
Trials intelligence can be used in all stages of drug development
because of its wide applicability. With these machine learn-
Artificial intelligence can be used to simulate clinical tri- ing methods, there is a potential for better accuracy, auto-
als and compare adverse effect profiles. Models have been mation, and comprehensiveness in the evaluation of patient
used to compare risk ratios between trials and simulations, safety events. This can be useful in enhancing efforts that
which shows how machine learning can be used in conjunc- are already completed by members of safety teams and hos-
tion with real-world data [21]. Furthermore, when machine pital staff but may also be limited by the costs and time
learning is used to build external control arms, there is an associated with training the systems. As machine learning is
opportunity to simulate different scenarios (e.g., sensitivity evaluated more in various databases and healthcare settings,
analysis modifying some inputs in the model) and estimate especially the inpatient setting, its application and scalability
the impact on the frequency of adverse events. Deep learning may prove to be paramount in pharmacovigilance efforts.
methods and causal artificial intelligence methods can also The field is likely expanding to more healthcare settings as
assist with data bias issues that come with using real-world new artificial technology is used by pharmacovigilance sci-
data for clinical trial simulation [21]. entists in pharmaceutical companies and healthcare workers
in hospitals. An updated literature search was performed for
4.8 Using Artificial Intelligence to Integrate the time period of July 9, 2021 to June 9, 2022 and showed
Prediction Uncertainties uses of artificial intelligence that were related to the uses
identified in this review. Examples of more recent applica-
It is important for machine learning to be able to integrate tions included classifying patient safety reports using pre-
prediction uncertainties. Pharmacovigilance requires the dictive algorithms and using text mining to analyze patient
appropriate detection and prevention of adverse effects, safety narratives.
which means that inaccurate associations between a drug
and adverse event can deter patient safety efforts. There- 4.10 Limitations
fore, it is imperative that machine learning methods can
indicate when there may be uncertainty in classification. The limitations of this systematic review are that many
When machine learning methods can integrate prediction reviews did not include a clear definition of what they con-
uncertainties, the result is a more comprehensive and accu- sidered to be machine learning, some databases are more
rate contribution to pharmacovigilance. comprehensive than others so proper comparison of results
may be limited, and studies demonstrating a successful
4.9 Overall Findings of the Systematic Review application of machine learning are more likely to be pub-
lished than studies with unsuccessful attempts. The search
Using artificial intelligence in pharmacovigilance and strategy as shown in the tables in Online Resources 1–2 (see
patient safety can potentially have an impact on improving ESM) was also limited to articles published in English that
patient care and optimizing safety analyses as evidenced by mentioned the relevant terms in its title or abstract, so some
the success of some machine learning models in improv- articles with information related to this topic may not have
ing patient safety. The use of machine learning and NLP been included. Many of the results involved experimentation
techniques together can provide accurate outputs that may and were not used in real scenarios, so these findings are not
The Use of Artificial Intelligence in Pharmacovigilance
yet automatically operationally applicable. It is also difficult tor of the Institute of Pharmacovigilance. Omar Aimer is an employee
to pool these results into generalizable findings given that of Innovigilance. Dinesh Kasthuril is an employee of Labcorp Drug
Development. Sameer Dhingra is Associate Professor and Head of De-
there were different types of data included in each source. partment of Pharmacy Practice at National Institute of Pharmaceutical
Education and Research (NIPER), Hajipur. Toluwalope Junaid is an
4.11 Opportunities for Future Research employee of Syneos Health. Tina Bostic is an employee of PPD, part
of Thermo Fisher Scientific. The opinions and positions taken in this
article are personal to the authors and not their employer/affiliated in-
Since artificial intelligence is being increasingly used stitutions.
across various areas, not just limited to healthcare, there
are many opportunities for future research of its use in Ethics approval Not applicable.
pharmacovigilance. Future studies may use databases
Consent to participate Not applicable.
that have more complicated input text or more ‘noise’ to
test if the artificial intelligence technology can respond Consent for publication Not applicable.
accurately and efficiently. Regarding social media plat-
forms, most of the articles used Twitter as their database. Availability of data and material Not applicable.
Other social media platforms such as health care social Code availability Not applicable.
networks could be useful to evaluate with the increased
interest in the use of social media in pharmacovigilance Authors’ contributions MS: study design, study implementation (title/
in recent years [80]. Lastly, further studies evaluating how abstract screening, quality control), preparation of draft manuscript,
review of manuscript draft, and interpretation of results. JP: study
much cost savings in healthcare can result from automated design, study implementation (title/abstract screening), review of
machine learning methods might be useful. manuscript draft, and interpretation of results. PY: study design, study
implementation (full article screening, quality control), preparation
of draft manuscript, review of manuscript draft, and interpretation of
results. OA: study design, study implementation (title/abstract screen-
ing, full article screening), review of manuscript draft, and interpreta-
5 Conclusions tion of results. DK: study design, study implementation (full article
screening), review of manuscript draft, and interpretation of results.
Artificial intelligence is actively being used in pharma- SD: review of manuscript draft and interpretation of results. TJ:
study implementation (quality control), review of manuscript draft,
covigilance and patient safety to gather information on and interpretation of results. TB: study design, study implementation
ADRs and ADEs, to perform surveillance and signal (title/abstract screening, quality control), review of manuscript draft,
detection, to process ICSRs, to process patient safety interpretation of results, quality control of summary table. All authors
event reports and clinical narratives, to extract drug–drug have read and approved the final version of the manuscript and agree
to be accountable for the work presented.
interactions and predict the effects of drug–drug interac-
tions, to identify populations at high risk for experiencing
ADRs and guide personalized care, to predict drug side
effects, to simulate clinical trials, and to integrate predic-
tion uncertainties into diagnostic classifiers to increase References
patient safety. There is potential for artificial intelligence
to be used in pharmacovigilance and patient safety in more 1. Das S, dey A, Pal A, Roy N. Applications of artificial intelligence
in machine learning: review and prospect. Int J Comput Appl.
ways than were identified in this review in the coming 2015;115:31–41.
years as people gain more exposure to artificial intelli- 2. Hashimoto DA, Rosman G, Rus D, Meireles OR. Artifi-
gence methods. The growth of this field may be limited cial intelligence in surgery: promises and perils. Ann Surg.
by challenges related to the lack of validated, established 2018;268(1):70–6.
3. Esteva A, Robicquet A, Ramsundar B, Kuleshov V, DePristo M,
uses of artificial intelligence in real-life safety settings. Chou K, et al. A guide to deep learning in healthcare. Nat Med.
2019;25(1):24–9.
Supplementary Information The online version contains supplemen- 4. Macrae C. Governing the safety of artificial intelligence in health-
tary material available at https://doi.org/10.1007/s40290-022-00441-z. care. BMJ Qual Saf. 2019;28(6):495–8.
5. Grossman LV, Choi SW, Collins S, Dykes PC, O’Leary KJ, Rizer
Declarations M, et al. Implementation of acute care patient portals: recom-
mendations on utility and use from six early adopters. J Am Med
Inf Assoc. 2018;25(4):370–9.
Funding No funding was received. All authors are members of the 6. Bates DW, Levine D, Syrowatka A, Kuznetsova M, Craig KJT,
North America Chapter of the International Society of Pharmacovigi- Rui A, et al. The potential of artificial intelligence to improve
lance (NASoP), which is a non-profit scientific organization. patient safety: a scoping review. NPJ Digit Med. 2021;4(1):54.
7. Moher D, Liberati A, Tetzlaff J, Altman DG. Preferred report-
Conflicts of interest Maribel Salas and Priyanka Yalamanchili are em- ing items for systematic reviews and meta-analyses: the PRISMA
ployees of Daiichi Sankyo, Inc. Jan Petracek is the founder and direc- statement. PLoS Med. 2009;6(7):e1000097.
M. Salas et al.
8. Higgins JPT, Altman DG, Gøtzsche PC, Jüni P, Moher D, Oxman genotype from phenotypical manifestations of fluoropyrimidine
AD, et al. The Cochrane Collaboration’s tool for assessing risk of toxicity. Clin Pharmacol Ther. 2020;107(4):944–7.
bias in randomised trials. BMJ. 2011;343: d5928. 26. Courtois É, Pariente A, Salvo F, Volatier É, Tubert-Bitter P,
9. Abatemarco D, Perera S, Bao SH, Desai S, Assuncao B, Tetarenko Ahmed I. Propensity score-based approaches in high dimension
N, et al. Training augmented intelligent capabilities for pharma- for pharmacovigilance signal detection: an empirical comparison
covigilance: applying deep-learning approaches to individual case on the French spontaneous reporting database. Front Pharmacol.
safety report processing. Pharmaceut Med. 2018;32(6):391–401. 2018;9:1010.
10. Alvaro N, Conway M, Doan S, Lofi C, Overington J, Collier N. 27. Dandala B, Joopudi V, Devarakonda M. Adverse drug events
Crowdsourcing Twitter annotations to identify first-hand experi- detection in clinical notes by jointly modeling entities and rela-
ences of prescription drug use. J Biomed Inform. 2015;58:280–7. tions using neural networks. Drug Saf. 2019;42(1):135–46.
11. Basile AO, Yahi A, Tatonetti NP. Artificial intelligence for drug 28. Daniel C, Kalra D. Section editors for the IYSoCRI. Clinical
toxicity and safety. Trends Pharmacol Sci. 2019;40(9):624–35. research informatics. Yearb Med Inf. 2020;29(1):203–7.
12. Baumgartner M, Eggerth A, Ziegl A, Hayn D, Schreier G. Experi- 29. Davazdahemami B, Delen D. A chronological pharmacovigilance
menting with generative adversarial networks to expand sparse network analytics approach for predicting adverse drug events. J
physiological time-series data. Stud Health Technol Inform. Am Med Inform Assoc. 2018;25(10):1311–21.
2020;271:248–55. 30. De Pretis F, Landes J, Peden W. Artificial intelligence methods
13. Bean DM, Wu H, Iqbal E, Dzahini O, Ibrahim ZM, Broadbent for a Bayesian epistemology-powered evidence evaluation. J Eval
M, et al. Knowledge graph prediction of unknown adverse drug Clin Pract. 2021;27(3):504–12.
reactions and validation in electronic health records. Sci Rep. 31. Desai S, Chan K, Bannout K, Mingle E, Freeman J, Parikh U,
2017;7(1):16416. et al. A novel approach to standardizing data & detecting dupli-
14. Ben Abacha A, Chowdhury MFM, Karanasiou A, Mrabet Y, cates across adverse event data sources using machine learning.
Lavelli A, Zweigenbaum P. Text mining for pharmacovigilance: Drug Saf. 2018;41(11):1246–7.
using machine learning for drug name recognition and drug- 32. Desai SME, Egan B, Gulati R, Freeman J. A framework for lev-
drug interaction extraction and classification. J Biomed Inf. eraging emerging technologies in pharmacovigilance. Drug Saf.
2015;58:122–32. 2018;41(11):1223–4.
15. Benin AL, Fodeh SJ, Lee K, Koss M, Miller P, Brandt C. Elec- 33. Dewulf P, Stock M, De Baets B. Cold-start problems in data-
tronic approaches to making sense of the text in the adverse event driven prediction of drug-drug interaction effects. Pharmaceuti-
reporting system. J Healthc Risk Manag. 2016;36(2):10–20. cals. 2021;14(5):429.
16. Bouzillé G, Morival C, Westerlynck R, Lemordant P, Chazard 34. El-Allaly ED, Sarrouti M, En-Nahnahi N, Ouatik El Alaoui S. An
E, Lecorre P, et al. An Automated detection system of drug-drug adverse drug effect mentions extraction method based on weighted
interactions from electronic patient records using big data analyt- online recurrent extreme learning machine. Comput Methods Pro-
ics. Stud Health Technol Inf. 2019;264:45–9. grams Biomed. 2019;176:33–41.
17. Calix RA, Gupta R, Gupta M, Jiang K, editors. Deep gramula- 35. Eshleman R, Singh R. Leveraging graph topology and semantic
tor: improving precision in the classification of personal health- context for pharmacovigilance through twitter-streams. BMC Bio-
experience tweets with deep learning. In: 2017 IEEE Interna- inform. 2016;17(13):335.
tional Conference on Bioinformatics and Biomedicine (BIBM); 36. Evans HP, Anastasiou A, Edwards A, Hibbert P, Makeham M, Luz
2017;1154–1159. S, et al. Automated classification of primary care patient safety
18. Chandak P, Tatonetti NP. Using machine learning to identify incident report content and severity using supervised machine
adverse drug effects posing increased risk to women. Patterns (N learning (ML) approaches. Health Inf J. 2019;26(4):3123–39.
Y). 2020;1(7):100108. 37. Fan Y, He L, Zhang R. Evaluating automatic methods to extract
19. Chauvet R, Bousquet C, Lillo-Lelouet A, Zana I, Ben Kimoun I, patients' supplement use from clinical reports. Proceedings IEEE
Jaulent MC. Classification of the severity of adverse drugs reac- International Conference on Bioinformatics and Biomedicine.
tions. Stud Health Technol Inf. 2020;270:1227–8. 2017;2017:1258-61
20. Chen J, Lalor J, Liu W, Druhl E, Granillo E, Vimalananda V, 38. Fan Y, Zhang R. Using natural language processing methods to
et al. Detecting Hypoglycemia Incidents Reported in Patients’ classify use status of dietary supplements in clinical notes. BMC
Secure Messages: Using Cost-sensitive Learning and Oversam- Med Inf Decis Mak. 2018;18(2):51.
pling to Reduce Data Imbalance (Preprint). J Med Internet Res. 39. Fong A, Behzad S, Pruitt Z, Ratwani RM. A machine learning
2019;21(3):e11990. approach to reclassifying miscellaneous patient safety event
21. Chen Z, Zhang H, George T, Prosperi M, Guo Y, Braithwaite D, reports. J Patient Saf. 2021;17(8):e829–33.
et al. Abstract PO-071: simulation of colorectal cancer clinical 40. Fong A, Harriott N, Walters DM, Foley H, Morrissey R, Ratwani
trials using real-world data and machine learning. Clin Cancer RR. Integrating natural language processing expertise with patient
Res. 2021;27(5 Supplement):PO-071. safety event review committees to improve the analysis of medica-
22. Cocos A, Fiks AG, Masino AJ. Deep learning for pharma- tion events. Int J Med Inf. 2017;104:120–5.
covigilance: recurrent neural network architectures for labeling 41. Foufi V, Ing Lorenzini K, Goldman JP, Gaudet-Blavignac C, Lovis
adverse drug reactions in Twitter posts. J Am Med Inf Assoc. C, Samer C. Automatic classification of discharge letters to detect
2017;24(4):813–21. adverse drug reactions. Stud Health Technol Inf. 2020;270:48–52.
23. Colón-Ruiz C, Segura-Bedmar I. Comparing deep learning archi- 42. Gartland A, Bate A, Painter JL, Casperson TA, Powell GE. Devel-
tectures for sentiment analysis on drug reviews. J Biomed Inf. oping crowdsourced training data sets for pharmacovigilance
2020;110:103539. intelligent automation. Drug Saf. 2021;44(3):373–82.
24. Comfort S, Perera S, Hudson Z, Dorrell D, Meireis S, Nagarajan 43. Gavrielov-Yusim N, Kürzinger ML, Nishikawa C, Pan C, Pouget
M, et al. Sorting through the safety data haystack: using machine J, Epstein LB, et al. Comparison of text processing methods in
learning to identify individual case safety reports in social-digital social media-based signal detection. Pharmacoepidemiol Drug
media. Drug Saf. 2018;41(6):579–90. Saf. 2019;28(10):1309–17.
25. Correia Pinheiro L, Durand J, Dogné JM. An application of 44. Gupta J, Patrick J, Poon S. Clinical safety incident taxonomy per-
machine learning in pharmacovigilance: estimating likely patient formance on C4.5 decision tree and random forest. Stud Health
Technol Inform. 2019;266:83–8.
The Use of Artificial Intelligence in Pharmacovigilance
45. Gupta S, Pawar S, Ramrakhiyani N, Palshikar GK, Varma V. mentions using sequence labeling with word embedding cluster
Semi-supervised recurrent neural network for adverse drug reac- features. J Am Med Inf Assoc. 2015;22(3):671–81.
tion mention extraction. BMC Bioinform. 2018;19(8):212. 64. Noguchi Y, Tachi T, Teramachi H. Subset analysis for screening
46. Henriksson A, Zhao J, Dalianis H, Boström H. Ensembles of ran- drug-drug interaction signal using pharmacovigilance database.
domized trees using diverse distributed representations of clinical Pharmaceutics. 2020;12(8):762.
events. BMC Med Inf Decis Mak. 2016;16(2):69. 65. Qiu Y, Zhang Y, Deng Y, Liu S, Zhang W. A comprehensive
47. Jagannatha AN, Yu H. Bidirectional RNN for medical event detec- review of computational methods for drug-drug interaction detec-
tion in electronic health records. Proc Conf. 2016;2016:473–82. tion. IEEE/ACM Trans Comput Biol Bioinform. 2021;3:7487.
48. Kalaiselvan V, Sharma A, Gupta SK. “Feasibility test and appli- 66. Salathé M. Digital pharmacovigilance and disease surveillance:
cation of AI in healthcare”—with special emphasis in clinical, combining traditional and big-data systems for better public
pharmacovigilance, and regulatory practices. Health Technol. health. J Infect Dis. 2016;214(suppl_4):S399–403.
2021;11(1):1–15. 67. Sarker A, Gonzalez G. Portable automatic text classification
49. Kreimeyer K, Dang O, Spiker J, Muñoz MA, Rosner G, Ball R, for adverse drug reaction detection via multi-corpus training. J
et al. Feature engineering and machine learning for causality Biomed Inform. 2015;53:196–207.
assessment in pharmacovigilance: Lessons learned from applica- 68. Schmider J, Kumar K, LaForest C, Swankoski B, Naim K, Caubel
tion to the FDA Adverse Event Reporting System. Comput Biol PM. Innovation in pharmacovigilance: use of artificial intelli-
Med. 2021;135: 104517. gence in adverse event case processing. Clin Pharmacol Ther.
50. Laves M-H, Ihler S, Ortmaier T, Kahrs LA. Quantifying the uncer- 2019;105(4):954–61.
tainty of deep learning-based computer-aided diagnosis for patient 69. Simon ST, Mandair D, Tiwari P, Rosenberg MA. Prediction of
safety. Curr Dir Biomed Eng. 2019;5:223–6. drug-induced long QT syndrome using machine learning applied
51. Letinier LJJ, Miremont G, Bel-Letoile A, Salvo F, Rouby F, et al. to harmonized electronic health record data. J Cardiovasc Phar-
Machine learning and semantic information for unstructured macol Ther. 2021;26(4):335–40.
healthcare data: Comparison of methods through the automatic 70. Thompson P, Daikou S, Ueno K, Batista-Navarro R, Ji T, Ana-
analysis of adverse drug reaction reports. MAI TAI study. Fundam niadou S. Annotation and detection of drug effects in text for
Clin Pharmacol. 2021;35(SUPPL 1):20. pharmacovigilance. J Cheminform. 2018;10(1):37.
52. Li F, Liu W, Yu H. Extraction of information related to adverse 71. Wang M, Ma X, Si J, Tang H, Wang H, Li T, et al. Adverse drug
drug events from electronic health record notes: design of an end- reaction discovery using a tumor-biomarker knowledge graph.
to-end model based on deep learning. JMIR Med Inf. 2018;6(4): Front Genet. 2021;11:625659.
e12159. 72. Yang J, Wang L, Phadke NA, Wickner PG, Mancini CM, Blu-
53. Li H, Yang M, Chen Q, Tang B, Wang X, Yan J. Chemical- menthal KG, et al. Development and validation of a deep learning
induced disease extraction via recurrent piecewise convolutional model for detection of allergic reactions using safety event reports
neural networks. BMC Med Inf Decis Mak. 2018;18(2):60. across hospitals. JAMA. 2020;3(11):e2022836-e.
54. Liang C, Gong Y. Enhancing patient safety event report- 73. Yang X, Bian J, Gong Y, Hogan WR, Wu Y. MADEx: a system
ing by K-nearest neighbor classifier. Stud Health Technol Inf. for detecting medications, adverse drug events, and their relations
2015;218:93–9. from clinical notes. Drug Saf. 2019;42(1):123–33.
55. Lien F, Wang HY, Lu JJ, Wen YH, Chiueh TS. Predicting 2-day 74. Zhao J, Henriksson A, Asker L, Boström H. Predictive modeling
mortality of thrombocytopenic patients based on clinical labora- of structured electronic health records for adverse drug event
tory data using machine learning. Med Care. 2021;59(3):245–50. detection. BMC Med Inf Decis Mak. 2015;15 Suppl 4(Suppl
56. Liu F, Pradhan R, Druhl E, Freund E, Liu W, Sauer BC, 4):S1.
et al. Learning to detect and understand drug discontinua- 75. Sujan MBC, Salmon P, Pool R, Chozos N. Human factors and
tion events from clinical narratives. J Am Med Inf Assoc. ergonomics in healthcare AI. Chartered Institute of Ergonomics
2019;26(10):943–51. and Human Factors; 2021.
57. Marella WM, Sparnon E, Finley E. Screening electronic health 76. Lewis DJ, McCallum JF. Utilizing advanced technologies to aug-
record-related patient safety reports using machine learning. J ment pharmacovigilance systems: challenges and opportunities.
Patient Saf. 2017;13(1):31–6. Ther Innov Regul Sci. 2020;54(4):888–99.
58. Ménard T, Barmaz Y, Oettinghaus B, Bowling R, Popko L. Ena- 77. Artificial Intelligence in Health Care: Benefits and Challenges of
bling data-driven clinical quality assurance: predicting adverse Technologies to Augment Patient Care. US Government Account-
event reporting in clinical trials using machine learning. Drug ability Office. 2020. https://www.gao.gov/products/gao-21-7sp.
Saf. 2019;42(9):1045–53. Accessed 10 June 2022.
59. Mockute R, Desai S, Perera S, Assuncao B, Danysz K, Tetarenko 78. Laying down harmonised rules on artificial intelligence (Arti-
N, et al. Artificial intelligence within pharmacovigilance: a means ficial Intelligence Act) and amending certain union legislative
to identify cognitive services and the framework for their valida- acts. EUR-LEX. 2021. https://e ur-l ex.e uropa.e u/l egal-c onten t/E
N/
tion. Pharmaceut Med. 2019;33(2):109–20. TXT/?uri=CELEX%3A52021PC0206. Accessed 10 June 2022.
60. Mower J, Cohen T, Subramanian D. Complementing observational 79. Kassekert R, Grabowski N, Lorenz D, Schaffer C, Kempf D, Roy
signals with literature-derived distributed representations for post- P, et al. Industry perspective on artificial intelligence/machine
marketing drug surveillance. Drug Saf. 2020;43(1):67–77. learning in pharmacovigilance. Drug Saf. 2022;45(5):439–48.
61. Mower J, Subramanian D, Cohen T. Learning predictive models 80. Lee J-Y, Lee Y-S, Kim DH, Lee HS, Yang BR, Kim MG. The use
of drug side-effect relationships from distributed representations of social media in detecting drug safety-related new black box
of literature-derived semantic predications. J Am Med Inf Assoc. warnings, labeling changes, or withdrawals: scoping review. JMIR
2018;25(10):1339–50. Public Health Surveill. 2021;7(6): e30137.
62. Negi K, Pavuri A, Patel L, Jain C. A novel method for drug- 81. Page MJ, McKenzie JE, Bossuyt PM, Boutron I, Hoffmann TC,
adverse event extraction using machine learning. Inf Med Mulrow CD, et al. The PRISMA 2020 statement: an updated
Unlocked. 2019;17: 100190. guideline for reporting systematic reviews. BMJ. 2021;372: n71.
63. Nikfarjam A, Sarker A, O’Connor K, Ginn R, Gonzalez G. Phar-
macovigilance from social media: mining adverse drug reaction
M. Salas et al.