Chaneliere 2018
Chaneliere 2018
Chaneliere 2018
Abstract
Background: Patient safety incidents (PSIs) frequently occur in primary care and are often considered to be
preventable. Better knowledge of factors contributing to PSIs is required to build safer care. The aim of this work was
to describe the underlying factors, specifically the human factors, that are associated with PSIs in primary care using
CADYA (“CAtégorisation des DYsfonctionnements en Ambulatoire” or “Categorization of Errors in Primary Care”).
Methods: We followed a mixed method with content analysis and coding in CADYA of PSIs reported in the ESPRIT
study, a French cross-sectional survey of primary care. For each incident, a main contributing factor (MD) and, if
applicable, a secondary contributing factor (SD) were identified. Several descriptive keywords from an incremental
glossary have been suggested to describe each identified human factor (attitudes or behaviours). A descriptive
statistical analysis was then conducted.
Results: Among the 482 PSIs reported in the ESPRIT study, from 13,438 acts reported by 127 participating general
practitioners (GPs), we identified 590 contributing factors (482 MDs and 178 SDs). Overall, 35% were related to the
care process, 30% to human factors, 22% to the healthcare environment and 13% to technical factors. The contributing
factors, in decreasing order of frequency, were communication errors (13.7%), human factors related to healthcare
providers (12.9%) and human factors related to patients (12.9%). The human factors were mainly related to ‘lack of
attention’, ‘stress’, ‘anger’ and ‘fatigue’.
Conclusions: Our results tend to prove that human factors are often involved in PSIs in primary care, with GPs and
patients being equally responsible. Beyond the identification of communication errors, often found in other
international research, we have described the attitudes and behaviours contributing to unsafe care. Further
research exploring the links between working conditions and human factors is required.
Keywords: Patient safety, Primary care, Patient safety incident, Contributing factors, Human factor
* Correspondence: marc.chaneliere@univ-lyon1.fr
1
Family Medicine Department, Université Claude Bernard Lyon 1, 8 avenue
Rockefeller, 69008 LYON, France
2
Hospices Civils de Lyon, 3 quai des Célestins, 69002 Lyon, France
Full list of author information is available at the end of the article
© The Author(s). 2018 Open Access This article is distributed under the terms of the Creative Commons Attribution 4.0
International License (http://creativecommons.org/licenses/by/4.0/), which permits unrestricted use, distribution, and
reproduction in any medium, provided you give appropriate credit to the original author(s) and the source, provide a link to
the Creative Commons license, and indicate if changes were made. The Creative Commons Public Domain Dedication waiver
(http://creativecommons.org/publicdomain/zero/1.0/) applies to the data made available in this article, unless otherwise stated.
Chaneliere et al. BMC Family Practice (2018) 19:121 Page 2 of 13
Background results of the ESPRIT study [5], we only present the most
Patient safety incidents (PSIs) have been reported in pri- relevant methodological elements.
mary care under various names [1]. Fifteen years ago, the
report ‘To err is human’ led to an international awareness ESPRIT study’s methodological aspects
of the frequency and gravity of PSIs [2]. In 2007 [3], the The ESPRIT study [5] was a cross-sectional study. Its
frequency of PSIs in primary care was estimated to be from aim was to describe the incidence and nature of PSIs in
2 to 240 incidents per 1000 encounters, and 45–76% were primary care general practice settings. The incidence was
considered to be preventable. The wide range in PSI fre- defined as the ratio of the number of medical encounters
quencies can be explained by the type of study (prospective (or patient contacts) with one PSI and the total number of
or retrospective), the data collection methods, the multiple medical encounters included during the study period.
proposed definitions of PSIs and the trend of underreport- The target population was all of the patients seen or
ing [3, 4]. In France, two prospective studies [5, 6] have contacted by GPs working in France. The source population
estimated the frequency of PSIs in primary care to be in the ESPRIT study [5] was composed of all of the patients
between 0.5 and 1 event per day per general practitioner seen or contacted by a GP from the GROG (Groupes
(GP). PSIs in the primary care setting significantly differ régionaux d’observation de la grippe). The GROG was
from those in hospitals [7–9] regarding their contributing a nationwide network for influenza surveillance in
factors [10]. To classify PSIs, several taxonomies, both France that included more than 800 volunteer GPs. The
specific and not specific to primary care, exist [11–18]. GROG was considered sufficiently representative of the
The World Health Organization (WHO) classification [18] French GPs to support several epidemiological studies
is the most universal, but the consequent number of items in general practice in France and Europe. The sample
that compose this taxonomy limits its current use. The was composed of GPs that were randomly selected
first results of the French national survey on PSIs in from the GROG physicians. A minimum sample size of
primary care (ESPRIT) study [5] were based on the Threats 120 GPs, with a gender distribution consistent with the
to Australian Patient Safety (TAPS) version of the Inter- national distribution (70 men and 50 women), was
national Taxonomy of Medical Error in Primary Care [12] achieved. The GPs were invited to participate in the
(ITME-PC) and the Tempos classification [14]. The TAPS study by telephone. The GROG coordination team
Taxonomy [12] describes the nature of the incident ac- trained all participating GPs on the study protocol and
cording to 2 main types of errors: errors related to the care on the data collection method by phone and by a written
process with 5 sublevels and errors of knowledge or skills procedure sent by mail. A specific training on PSIs in
of the actors with 2 sublevels (diagnosis and patient primary care was also provided by phone or by web with a
management). The Tempos classification [14] uses time in video (available on the study website). The aim was to
classifying factors contributing to incidents according to ensure that all GPs had a same understanding of what
five categories: the tempo of illness and treatment, the constituted an incident to be reported in an effort to
tempo of the physician, the tempo of the office, the tempo reduce the selection bias and reporting bias.
of the patient and the tempo of the health system. CADYA
(Additional file 1: Appendix S1), which can be translated Data collection method
as Categorization of Errors in Primary Care, is a French Data collection for each GP occurred for one week,
taxonomy [17] that provides a complementary approach which was chosen by the GP within the six-week data
by describing more accurately the factors contributing collection period from May 2013–June 2013. The defin-
to PSIs, especially the human factors, as suggested by ition of an incident was proposed by focus groups and
research in aeronautics [19]. Four main levels are explored: approved by consensus among primary care experts and
technical factors, the environment (action background), GP representatives. An incident was defined as “an event
processes related to the decision or care process (skills), or circumstance that could have resulted, or did result,
and human factors (physical conditions, psycho-relational in harm to a patient, and which should not be repeated
elements, attitudes or behaviours). again” [20]. An “act” was defined as any patient contact,
The main objective of the present study was to including a consultation, visit, or intervention, in a nursing
describe the incidents of the ESPRIT study [5] using the home or by telephone. Participating GPs completed a daily
CADYA taxonomy to have better knowledge of their PSI reporting webform throughout the data collection
contributing factors. period. That form was developed by the ESPRIT study
expert group after a literature review. Three forms were
Methods used: a GP profile form, a register of acts (all acts during
A mixed method approach, combining qualitative analysis the collection week) and a PSI reporting form containing
(content analysis) and quantitative analysis (descriptive 25 questions capturing a description of each PSI along with
analysis), has been used. Because the work is based on the contributing factors and consequences. The data that were
Chaneliere et al. BMC Family Practice (2018) 19:121 Page 3 of 13
collected were compiled on a secure website; data quality the investigators were required to reach a consensus. At
was reviewed by an expert group (eight GPs competent in the end of data extraction, all coding was approved by the
the patient safety field and two epidemiologists). At the expert GP (MC). The review of each investigator, the
end of the data collection process, each reported event was consensus and the expert final review were compiled
reviewed during an expert group seminar to verify the into a single Excel® file. Finally, the investigators
situation as a PSI. The data quality control process was checked, for a second time, that there were no redundant
thus carried out in several stages. keywords in the thesaurus. If necessary, they gathered
redundant keywords into a single consensual keyword to
Ethical aspects obtain a reduced, but more precise, thesaurus.
No nominative, sensitive or personal health data concern-
ing patients were systematically collected. The ESPRIT Statistical analysis
study [5] was therefore not, strictly speaking, in the field of Descriptive data analysis was conducted according to the
biomedical research (fields of application of the French dimensions of CADYA. The frequencies of agreements
reference methodology MR 001), nor was it within the between the two coders and between coders and experts
framework of Chapter X of the French Data Protection were estimated. The inter-rater reliability was analysed with
Act. For this practice-oriented study, the only nominative a kappa coefficient [21], which was expressed as follows:
data collected were GP profile data that were available in a (% observed agreement - % agreement due to chance) /
database declared to the French CNIL (number 1684220). (1 -% agreement due to chance). The interpretation of
GPs provided their written informed consent to participate the calculated kappa coefficients followed the six classes
by email. The National Council of the French Medical by Landis and Koch [22].
Association was also informed due to the indemnification
of the participating GP. All ethical approvals of the study Results
were given by the Ile-de-France Paris III committee and by Main ESPRIT study results [5]
the CNIL (French Data Protection Authority). A total of 13,438 acts were recorded by the 127 partici-
pating GPs; 82% (n = 11,023) were office consultations,
Work methodological aspects 8% (n = 1090) were telephone contacts, 8% (n = 1022)
Recruiting coders were home visits and 2% (n = 303) were visits at nursing
Two GPs from the Family Medicine Department of the homes. The GPs in the sample were older and worked
Claude Bernard University Lyon 1 (DK and TM) volun- more often in groups and more often in rural areas than
teered to analyse the ESPRIT study PSIs. An expert GP the population of GPs in France (Table 1). The average
(MC) who was involved in patient safety in primary care number of patient encounters per day was 21. The aver-
trained the two recruiting coders to reach a shared under- age age of the patients was 48 years, and 4013 patients
standing of, and a similar level of expertise on, the root (30%) had a chronic or long-term condition.
cause analysis and the use of CADYA. A total of 694 events were reported by the participating
GPs, 674 (97%) of which were reported as being related to
Coding process a patient and 20 (3%) of which were reported as being not
All of the PSIs related to a patient were extracted from related to a particular patient. Additionally, of these 694
the ESPRIT study database. For each PSI, the investigators PSIs, 212 (31%) were not PSIs as defined. Thus, 482 (69%)
were required to independently search for the underlying PSIs were validated by experts, 475 (99%) of which were
factors contributing to their occurrence according to a related to one patient and 7 (1%) of which were non-pa-
qualitative approach (content analysis). Then, the investi- tient related (usually computer glitches affecting a whole
gators had to select a main contributing factor (MD), range of medical encounters). Of these 475 PSIs, 52% oc-
called “dysfunction” in French, and, if relevant, a second- curred in the office, 37% occurred in patients’ homes and
ary contributing factor (SD) in CADYA [17]. The choice 12% occurred in nursing homes. Additionally, 55% of the
between designating the factor as a MD or SD depended patients were women. The average age of the patients was
on the estimated impact of the contributing factor to the 56 years (standard deviation: 25 years), with a range from
PSI. In the case of a factor related to a human attitude or zero years (four patients) to 100 years (one patient).
behaviour, the investigators had to specify a keyword to Nearly half of the patients (40%) were retirees.
describe its nature (e.g., “anger”, “stress” or “tiredness”) There were no clinical consequences for 73% of PSIs,
according to an incremental glossary (a keyword that was and for 25%, there was a temporary disability. Only 2%
already identified, or, if necessary, a new keyword). The of the PSIs were serious in the ESPRIT study (n = 9),
investigators had to describe the human factor with as being associated with a life-threatening or permanent
much accuracy as possible. They regularly compared their disability (i.e., 0.07% of the 13,438 patient encounters
coding after every 50 reports. In cases of disagreement, observed in the study).
Chaneliere et al. BMC Family Practice (2018) 19:121 Page 4 of 13
Table 2 Distributions of the main (MD), secondary (SD) and total number of (TD) contributing factors
Dimensions Main contributing factor Secondary contributing factor Total contributing factor
n (% of MD) n (% of SD) n (% of TD)
ENVIRONMENTAL FACTORS 93 (22.8) 37 (20.3) 130 (22)
PATIENT’S SOCIAL CONTEXT 5 (1.2) 5 (2.7) 10 (1.7)
BACKGROUND OF CARE 38 (9.3) 20 (11) 58 (9.8)
Unplanned consultation 10 (2.4) 0 (0) 10 (1.7)
Place of care 6 (1.5) 8 (4.4) 14 (2.4)
Workload management 22 (5.4) 12 (6.6) 34 (5.7)
DISRUPTIVE ELEMENT 27 (6.6) 7 (3.9) 34 (5.7)
HEALTH SYSTEM 23 (5.7) 5 (2.7) 28 (4.7)
Healthcare service 20 (5) 2 (1) 22 (3.7)
Financial or administrative issue 3 (0.7) 3 (1.7) 6 (1)
HUMAN FACTORS 89 (21.8) 86 (47.3) 175 (29.7)
LINKED TO THE PATIENT 45 (11) 31 (17) 76 (12.9)
LINKED TO THE PROVIDER 35 (8.6) 41 (22.5) 76 (12.9)
LINKED TO OTHER PROVIDERS 4 (1) 7 (3.9) 11 (1.9)
LINKED TO A THIRD PARTY 5 (1.2) 7 (3.9) 12 (2)
TECHNICAL FACTORS 67 (16.4) 9 (4.9) 76 (12.9)
EQUIPMENT 21(5.2) 2 (1) 23 (3.9)
Failure, malfunction, unavailability 19 (4.7) 1 (0.5) 20 (3.4)
Incorrect use 2 (0.5) 1 (0.5) 3 (0.5)
INFORMATION SYSTEM 46 (11.3) 7 (3.9) 53 (8.9)
Incorrect or missing data 34 (8.3) 4 (2.2) 38 (6.4)
Failure of the communication system 12 (3) 3 (1.7) 15 (2.5)
PROCESS OF CARE 159 (39) 50 (27.5) 209 (35.4)
COGNITIVE DIMENSION 56 (13.8) 9 (5) 65 (11)
Lack in initial training 26 (6.4) 3 (1.7) 29 (4.9)
Incorrect recall (after training) 13 (3.2) 2 (1) 15 (2.5)
Incorrect synthesis 17 (4.2) 4 (2.2) 21 (3.6)
CARE PROCEDURE 34 (8.3) 14 (7.7) 48 (8.1)
Inappropriate or unachieved procedure 31 (7.6) 13 (7.1) 44 (7.5)
Lack of protocol 3 (0.7) 1 (0.6) 4 (0.6)
CARE COORDINATION 69 (16.9) 27 (14.8) 96 (16.3)
Communication failure 59 (14.5) 22 (12.1) 81 (13.7)
Lack of (or incorrect) monitoring 9 (2.2) 5 (2.7) 14 (2.4)
Lack of response after feedback 1 (0.2) 0 (0) 1 (0.2)
TOTAL 408 (100) 182 (100) 590 (100)
Process of care to Sammer et al. [26]. For that reason, we think that it may
Among the contributing factors related to the process of be useful to share some training programmes among all
care, miscommunication between healthcare professionals students across all health disciplines. The students could
constitutes a main source of errors, as suggested by several then better understand the mutual roles of all primary care
international studies [11, 23–25]. This result suggests the professionals. Other actions should be implemented and
need to develop communication training for health profes- evaluated, e.g., promoting learning via games and simula-
sionals (students or graduates). ‘Good communication tions during medical studies to develop relationship skills
skills’ is one of the seven pillars of a safety culture according in healthcare. To train primary care professionals to better
Chaneliere et al. BMC Family Practice (2018) 19:121 Page 6 of 13
structure their communication (especially oral), some Contributing factors related to cognitive dimensions
standardized tools such as ‘SBAR’ (‘Situation-Background– have been more frequently identified as being MDs rather
Assessment-Recommendation’) [27], which is already used than SDs. This difference seems to be related to the
in some hospitals, could be deployed. Patients should also importance of the cognitive processes themselves. Indeed,
play a leading role in their safety. They should have access even a small error at this level can produce a diagnostic or
to similar tools to better communicate with professionals therapeutic issue, sometimes serious [30]. This issue leads
[28]. If these standardized communication tools were to more frequent identification as a MD. In the ESPRIT
implemented in primary care, such tools could reduce the study [5], 17% of the PSIs were related to ‘knowledge and
number of PSIs [29]. skill errors’ according to the taxonomy of Makeham [12]
Chaneliere et al. BMC Family Practice (2018) 19:121 Page 7 of 13
Fig. 2 Distribution of all the identified contributing factors in CADYA (% of the total number). Legend: PC: Process of Care; HF: Human Factors; TF:
Technical Factors; EF: Environmental Factors
[5], unexpected emergencies were seldom identified communication technologies in the past twenty years,
because they were probably responsible for problems in many doubt that this type of error is experiencing a
the workload. This situation is consistent with the limited substantial decrease. The widespread use of secured
number of “true” emergencies in primary care [35] in software (including prescription-assistance modules)
modern health systems, where these acute events are and electronic patient records help to limit some errors
ideally supported by dedicated services. To our know- (e.g., illegible handwritten prescriptions), but this has also
ledge, no study has specifically quantified the impact of an contributed to other issues, e.g., common ‘click’ errors in
improvement in the environment of care on safety. How- medication lists.
ever, this seems pragmatic, as it is known that anything
that impairs communication within a team is likely to CADYA
generate PSIs [36]. CADYA is complementary to the taxonomy of Makeham
[12] and to the Tempos method [14], as it specifically
Technical factors: First, gaps in information systems explores processes related to medical decision-making
GPs perform a small number of technical acts, which and the unsafe attitudes and behaviours of actors. Regard-
is consistent with a rather low frequency of ‘purely ing the main dimensions, the strong coefficient kappa
technical’ errors. Moreover, with the development of between investigators argues for the robustness of CADYA.
Chaneliere et al. BMC Family Practice (2018) 19:121 Page 9 of 13
Fig. 3 Distribution of the main contributing factors in CADYA (% of the total number). Legend: PC: Process of Care; HF: Human Factors; TF:
Technical Factors; EF: Environmental Factors
The poor kappa between the investigators and the expert used as part of the ongoing PRisM national study,
seems paradoxical. This result is explained by the paradox which is dedicated to the assessment of a multifaceted
of Feinstein and Ciccheti [37]: with a similar level of programme on risk management in primary care. A
observed agreement, the estimated kappa is lower when full and validated English adaptation of CADYA is in
the symmetrical balance is perfect. Despite a very high level progress, following the recommendations of the
of agreement, we obtained an abnormally low kappa WHO [38].
coefficient.
CADYA is used in France by several morbidity and Strengths and limitations
mortality review groups [17] and was supported by the Some limitations are inherent to the ESPRIT study [5].
French High Health Authority. It was also used during The GPs were randomly selected within a network that
specific trainings on patient safety at Claude Bernard was potentially biased towards reporting. According to
University Lyon 1, especially in those on Family Practice. us, this strategy was preferable to improve the number
In its first version, CADYA did not provide a glossary of PSIs reported by GPs and to have a better data collec-
to describe human factors related to attitudes and tion quality. Yet, there is no indication that physicians
behaviours, and users could use any descriptive terms enrolled in GROG are more concerned by issues related
they wished. However, this flexibility created a limita- to patient safety. For that reason, we think that the
tion for a more standardized use. In addition, some findings - based on over 12,000 medical encounters -
users regretted the absence of a glossary that would were not necessarily affected by this choice and the
facilitate the identification of human factors. The Glossary generalizability still remains acceptable. More than 200
of human factors constitutes a useful upgrade in the most events of the 694 incidents reported by GPs were
recent version of CADYA. This version is currently being deemed to not be PSIs by the expert group. This finding
Chaneliere et al. BMC Family Practice (2018) 19:121 Page 10 of 13
Fig. 4 Distribution of the secondary contributing factors in CADYA (% of the total number). Legend: PC: Process of Care; HF: Human Factors; TF:
Technical Factors; EF: Environmental Factors
suggests that the level of understanding of the definition health statuses remains limited. However, the literature
of PSI by participant GPs was not high, despite the time on PSIs in primary care has already shown that factors
spent in GP training. This issue has been already de- such as an older patient age and an increased number
scribed in previous studies [3, 4]. The main limitation of comorbidities have been associated with an increased
of the ESPRIT study is that information on the patient risk for adverse events.
Fig. 5 Keywords used to describe the human factors linked with the patient, as ranked by the frequency of use (%)
Chaneliere et al. BMC Family Practice (2018) 19:121 Page 11 of 13
Fig. 6 Keywords used to describe the human factors linked with the GP, as ranked by the frequency of use (%)
Regarding our work, the coding of the PSIs was per- that reason, an expert validated the consistency of all
formed by only 2 investigators. However, they have the codes. The number of contributing factors identified
been previously trained to use the CADYA grid, and was consistent with the limited information reported by
they regularly compared their coding results. The lack the GPs in the ESPRIT study [5]. Thus, the reported
of information in some clinical descriptions, as well as events have often been reduced to one sentence or a
the absence of time constraints to code the PSIs, may few words. The investigators were instructed to only
have also constituted a subjective bias in coding. For consider information without interpreting it to limit the
risk of bias. Finally, the glossary consists of only 15 key- results; MC, PM and ID contributed to the writing and revising of the
words; our experience is that attitudes and behaviours manuscript. All the authors have approved the manuscript for submission.
14. Amalberti R, Brami J. 'Tempos’ management in primary care: a key factor for
classifying adverse events, and improving quality and safety. BMJ Qual Saf.
2012;21:729–36.
15. Kostopoulou O, Delaney B. Confidential reporting of patient safety events in
primary care: results from a multilevel classification of cognitive and system
factors. Qual Saf Health Care. 2007;16:95–100.
16. Rubin G, George A, Chinn DJ, Richardson C. Errors in general practice:
development of an error classification and pilot study of a method for
detecting errors. Qual Saf Health Care. 2003;12:443–7.
17. Chanelière M, Oriol J-M, Senez B, Keriel-Gascou M, Fanjat H, Blanc M-D, et al.
Un outil pour analyser des incidents en soins primaires. Risqual. 2014;12:34–40.
18. World Alliance For Patient Safety Drafting Group. Sherman H, Castro G,
fletcher M; world alliance for patient safety, Hatlie M, Hibbert P, Jakob R,
Koss R, Lewalle P, Loeb J, Perneger T, Runciman W, Thomson R, van Der
Schaaf T, Virtanen M. Towards an international classification for patient
safety: the conceptual framework. Int J Qual Health Care. 2009 Feb;21(1):2–8.
19. Wilf-Miron R, Lewenhoff I, Benyamini Z, Aviram A. From aviation to
medicine: applying concepts of aviation safety to risk management in
ambulatory care. Qual Saf Health Care. 2003;12:35–9.
20. Keriel-Gascou M, Brami J, Chanelière M, Haeringer-cholet A, Larrieu C,
Villebrun F, et al. Définition et Taxonomie pour une étude épidémiologique
Française sur les événements indésirables associés aux soins primaires. Rev
Epidemiol S Publ. 2014;62:41–52.
21. Cohen J. A coefficient of agreement for nominal scales. Educ Psychol Meas.
1960;20:37–46.
22. Landis JR, Koch GG. The measurement of observer agreement for
categorical data. Biometrics. 1977;33:159–74.
23. Kripalani S, LeFevre F, Phillips CO, Williams MV, Basaviah P, Baker DW.
Deficits in communication and information transfer between hospital-based
and primary care physicians: implications for patient safety and continuity
of care. JAMA. 2007;297:831–41. https://doi.org/10.1001/jama.297.8.831.
24. O’Beirne M, Sterling PD, Zwicker K, Hebert P, Norton PG. Safety incidents in
family medicine. BMJ Qual Saf. 2011;20:1005–10.
25. Rosser W, Dovey SM, Bordman R, White D, Crighton E, Drummond N.
Medical errors in primary care: results of an international study of family
practice. Can Fam Phys. 2005;51:386–7.
26. Sammer CE, Lykens K, Singh KP, Mains DA, Lackan NA. What is patient
safety culture? A review of the literature. J Nurs Scholarsh. 2010;42:156–65.
27. Pope BB, Rodzen L, Spross G. Raising the SBAR: how better communication
improves patient outcomes. Nursing. 2008;38:41–3.
28. Denham CR. SBAR for patients. J Patient Saf. 2008;4:38–48.
29. Raduma-Tomàs MA, Flin R, Yule S, Williams D. Doctors’ handovers in
hospitals: a literature review. BMJ Qual Saf. 2011;20:128–33.
30. Wachter RM. Why diagnostic errors don’t get any respect–and what can be
done about them. Health Aff (Millwood). 2010;29:1605–10. https://doi.org/
10.1377/hlthaff.2009.0513.
31. Butler C, Rollnick S, Stott N. The practitioner, the patient and resistance to
change: recent ideas on compliance. CMAJ. 1996;154:1357–62.
32. Mittman BS, Tonesk X, Jacobson PD. Implementing clinical practice
guidelines: social influence strategies and practitioner behavior change.
Qual Rev Bull. 1992;18:413–22.
33. Evidence-Based Medicine Working Group. A new approach to teaching the
practice of medicine. JAMA. 1992;268:2420–5. https://doi.org/10.1001/jama.
1992.03490170092032.
34. Vincent C, Taylor-Adams S, Chapman EJ, Hewett D, Prior S, Strange P, et al.
How to investigate and analyse clinical incidents: clinical risk unit and
association of litigation and risk management protocol. BMJ. 2000;320:777–81.
35. Pronovost PJ, Nolan T, Zeger S, Miller M, Rubin H. How can clinicians
measure safety and quality in acute care? Lancet. 2004;363:1061–7.
36. Leonard M, Graham S, Bonacum D. The human factor: the critical
importance of effective teamwork and communication in providing safe
care. Qual Saf Health Care. 2004;13(1):i85–90.
37. Feinstein AR, Cicchetti DV. High agreement but low kappa: I. The problems
of two paradoxes. J Clin Epidemiol. 1990;43:543–9.
38. Wild D, Grove A, Martin M, Eremenco S, McElroy S, Verjee-Lorenz A, et al.
Principles of good practice for the translation and cultural adaptation
process for patient-reported outcomes (PRO) measures: report of the ISPOR
task force for translation and cultural adaptation. Value Health. 2005;8:94–104.