Jamtvedt2003 SR PDF

Audit and feedback: effects on professional practice and
health care outcomes (Review)
Jamtvedt G, Young JM, Kristoffersen DT, Thomson O’Brien MA, Oxman AD
This is a reprint of a Cochrane review, prepared and maintained by The Cochrane Collaboration and published in The Cochrane Library
2003, Issue 3
http://www.thecochranelibrary.com
Audit and feedback: effects on professional practice and health care outcomes (Review)
Copyright © 2006 The Cochrane Collaboration. Published by John Wiley & Sons, Ltd.
TABLE OF CONTENTS
ABSTRACT . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 1
PLAIN LANGUAGE SUMMARY . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 1
BACKGROUND . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 2
OBJECTIVES . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 2
CRITERIA FOR CONSIDERING STUDIES FOR THIS REVIEW . . . . . . . . . . . . . . . . . . 3
SEARCH METHODS FOR IDENTIFICATION OF STUDIES . . . . . . . . . . . . . . . . . . . 3
METHODS OF THE REVIEW . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 4
DESCRIPTION OF STUDIES . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 6
METHODOLOGICAL QUALITY . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 7
RESULTS . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 7
DISCUSSION . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 11
AUTHORS’ CONCLUSIONS . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 12
POTENTIAL CONFLICT OF INTEREST . . . . . . . . . . . . . . . . . . . . . . . . . . . 13
ACKNOWLEDGEMENTS . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 13
SOURCES OF SUPPORT . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 13
REFERENCES . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 13
TABLES . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 22
Characteristics of included studies . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 22
Characteristics of excluded studies . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 62
ADDITIONAL TABLES . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 64
Table 01. Categories for continuous outcomes . . . . . . . . . . . . . . . . . . . . . . . . . 64
Table 02. Intensity of feedback . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 64
Table 03. Quality of included trials . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 72
ANALYSES . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 76
Comparison 01. Audit and feedback versus no intervention . . . . . . . . . . . . . . . . . . . . 76
Comparison 02. Short term effects of audit and feedback compared to longer-term effects after feedback stops . . . 77
Comparison 03. Audit and feedback + complementary interventions versus audit and feedback alone . . . . . . 77
Comparison 04. Audit and feedback versus other interventions . . . . . . . . . . . . . . . . . . . 77
Comparison 05. Comparisons of different ways of providing audit and feedback . . . . . . . . . . . . . 77
Comparison 06. Studies in which patients were randomised . . . . . . . . . . . . . . . . . . . . 77
Comparison 07. High quality studies . . . . . . . . . . . . . . . . . . . . . . . . . . . . 77
INDEX TERMS . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 77
COVER SHEET . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 77
GRAPHS AND OTHER TABLES . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 79
Figure 01. Adjusted relative risk versus baseline non-compliance . . . . . . . . . . . . . . . . . . . 79
Figure 02. Adjusted risk difference versus baseline non-compliance . . . . . . . . . . . . . . . . . . 80
Figure 03. Adjusted relative risk versus intervention type . . . . . . . . . . . . . . . . . . . . . 81
Analysis 01.01. Comparison 01 Audit and feedback versus no intervention, Outcome 01 Dichotomous outcomes . 82
Analysis 01.02. Comparison 01 Audit and feedback versus no intervention, Outcome 02 Continuous outcomes . . 86
Analysis 02.01. Comparison 02 Short term effects of audit and feedback compared to longer-term effects after feedback 89
stops, Outcome 01 All outcomes . . . . . . . . . . . . . . . . . . . . . . . . . . .
Analysis 03.01. Comparison 03 Audit and feedback + complementary interventions versus audit and feedback alone, 89
Outcome 01 All outcomes . . . . . . . . . . . . . . . . . . . . . . . . . . . . .
Analysis 04.01. Comparison 04 Audit and feedback versus other interventions, Outcome 01 All outcomes . . . . 91
Analysis 05.01. Comparison 05 Comparisons of different ways of providing audit and feedback, Outcome 01 All 91
outcomes . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .
Analysis 06.01. Comparison 06 Studies in which patients were randomised, Outcome 01 All outcomes . . . . . 92
Analysis 07.01. Comparison 07 High quality studies, Outcome 01 Dichotomous outcomes . . . . . . . . . 92
Audit and feedback: effects on professional practice and health care outcomes (Review) i
Copyright © 2006 The Cochrane Collaboration. Published by John Wiley & Sons, Ltd
Audit and feedback: effects on professional practice and
health care outcomes (Review)
Jamtvedt G, Young JM, Kristoffersen DT, Thomson O’Brien MA, Oxman AD
This record should be cited as:

Jamtvedt G, Young JM, Kristoffersen DT, Thomson O’Brien MA, Oxman AD. Audit and feedback: effects on professional
practice and health care outcomes. The Cochrane Database of Systematic Reviews 2003, Issue 3. Art. No.: CD000259. DOI:
10.1002/14651858.CD000259.
This version first published online: 21 July 2003 in Issue 3, 2003.

Date of most recent substantive amendment: 28 May 2003
ABSTRACT
Background
Audit and feedback continues to be widely used as a strategy to improve professional practice. It appears logical that healthcare
professionals would be prompted to modify their practice if given feedback that their clinical practice was inconsistent with that of
their peers or accepted guidelines. Yet, audit and feedback has not been found to be consistently effective.
Objectives
To assess the effects of audit and feedback on the practice of healthcare professionals and patient outcomes.
Search strategy
We searched the Cochrane Effective Practice and Organisation of Care Group’s register up to January 2001. This was supplemented
with searches of MEDLINE and reference lists, which did not yield additional relevant studies.
Selection criteria
Randomised trials of audit and feedback (defined as any summary of clinical performance over a specified period of time) that reported
objectively measured professional practice in a healthcare setting or healthcare outcomes.
Data collection and analysis
Two reviewers independently extracted data and assessed study quality. Quantitative (meta-regression), visual and qualitative analyses
were undertaken.
Main results
We included 85 studies, 48 of which have been added to the previous version of this review. There were 52 comparisons of dichotomous
outcomes from 47 trials with over 3500 health professionals that compared audit and feedback to no intervention. The adjusted Risk
Differences (RD’s) of non-compliance with desired practice varied from 0.09 (a 9% absolute increase in non-compliance) to 0.71 (a
71% decrease in non-compliance) (median = 0.07, inter-quartile range = 0.02 to 0.11). The one factor that appeared to predict the
effectiveness of audit and feedback across studies was baseline non-compliance with recommended practice.
Authors’ conclusions
Audit and feedback can be effective in improving professional practice. When it is effective, the effects are generally small to moderate.
The absolute effects of audit and feedback are more likely to be larger when baseline adherence to recommended practice is low.
PLAIN LANGUAGE SUMMARY

Providing healthcare professionals with data about their performance (audit and feedback) may help improve their practice.
Audit and feedback can improve professional practice, but the effects are variable. When it is effective, the effects are generally small to
moderate. The results of this review do not provide support for mandatory or unevaluated use of audit and feedback.
Audit and feedback: effects on professional practice and health care outcomes (Review) 1
BACKGROUND They located twelve eligible trials and concluded that profiling
had a statistically significant but minimally important effect.
This review updates and merges two previous Cochrane re-
In earlier versions of this review we found that the effects of audit
views of the effects of audit and feedback (Thomson OBrien
and feedback varied and that it was not possible to determine
1997a;Thomson OBrien 1997b). Audit and feedback continues
what features or contextual factors determine the effectiveness of
to be widely used as a strategy to improve professional practice. It
audit and feedback (Thomson OBrien 1997a;Thomson OBrien
appears logical that healthcare professionals would be prompted to
1997b).
modify their practice if given feedback that their clinical practice
was inconsistent with that of their peers or accepted guidelines. More recently, Stone and colleagues (Stone 2002) reviewed 108
Yet, audit and feedback has not been found to be consistently ef- studies to assess the relative effectiveness of various interventions,
fective (Grimshaw 2001). including audit and feedback, to improve adult immunisation and
cancer screening. Thirteen of the included studies involved provi-
Previous reviews have looked at factors associated with the effec- sion of feedback. Feedback was not found to improve immunisa-
tiveness of audit and feedback. Mugford and colleagues (Mugford tion or screening for cervical or colorectal cancer and only mod-
1991) identified 36 published studies of feedback of information. erately improved mammographic screening.
Information feedback was defined as the use of comparative infor-
mation from statistical systems. These authors distinguished pas- These reviews suggest that the provision of information alone
sive from active feedback where passive feedback was the provision results in little, if any change in practice. Kanouse and Jacoby
of unsolicited information and active feedback engaged the inter- (Kanouse 1988) suggest that, typically, the transfer of information
est of the clinician. They also assessed the impact of the recipient relies on a diffusion model that assumes that practitioners are ac-
of the information, the format of the information and the timing tive consumers of information and are willing to make changes in
of the feedback. Studies were included if their design used either the way they provide healthcare when they encounter information
a historical or a concurrent control group for comparison. The that suggests alternative practices. These authors propose that fac-
authors concluded that information feedback was most likely to tors such as the characteristics of the information provided, prac-
influence clinical practice if the information was presented close titioner motivation and characteristics of the clinical context need
to the time of decision-making and the clinicians had previously to be considered when a change in behaviour is desired. Similarly,
agreed to review their practice. Oxman and Flottorp (Oxman 2001) have outlined twelve cate-
gories of factors that should be considered when trying to improve
Axt-Adams and colleagues (Axt-Adam 1993) reviewed 67 pub- professional practice, including characteristics of the practice envi-
lished papers of interventions (26 studies of feedback) designed ronment, prevailing opinion, knowledge and attitudes. Both log-
to influence the ordering of diagnostic laboratory tests. They re- ical arguments and previous reviews have suggested that multi-
ported that a number of factors could be important such as the faceted interventions, particularly if they are targeted at different
message, the provider of the feedback, the addressee, the timeliness barriers to change, may be more effective than single interventions
and the vehicle. They concluded that there was considerable varia- (Grimshaw 2001). In this review, we examine some of the factors
tion among different studies. This variation could be explained in that could influence the effectiveness of the intervention such as
part by the extent, the timing, the frequency, and the availability the source of the feedback and whether audit and feedback is more
of comparative information related to peers. They also felt that effective when combined with other interventions.
the practice setting was an important factor.
Buntinx and colleagues (Buntinx 1993) conducted a systematic

review of 26 studies of feedback and reminders to improve di- OBJECTIVES
agnostic and preventive care practices in primary care. They cat-
egorised feedback that occurred after or during the target per- We address the following questions:
formance as feedback whereas feedback that occurred before the
• Is audit and feedback effective in improving professional prac-
target performance was called reminders. Ten of the 26 studies
tice and health care outcomes?
used randomised designs but the quality of the included trials was
not reported. The authors concluded that both feedback and re- • How does the effectiveness of audit and feedback compare with
minders might reduce the use of diagnostic tests and improve the that of other interventions, and can audit and feedback be made
delivery of preventive care services. However, they also reported more effective by modifying how it is done?
that it was not clear how feedback works, especially the use of peer
group comparisons. To answer the first question we considered the following five com-
parisons. These have been modified from our previous review to
Balas and colleagues (Balas 1996) reviewed the effectiveness of reflect subsequent evidence that interactive educational meetings
peer-comparison feedback profiles in changing practice patterns. are effective at changing professional practice (Thomson O’Brien
2001), whereas printed educational materials are not (Freemantle this update and we have considered the intensity of audit and
1997; Grimshaw 2001). feedback across studies in analysing the results, as described in the
methods section.
11. Any intervention in which audit and feedback is a component
compared to no intervention (+/- printed educational materials).
This was an overall comparison which also included the studies in CRITERIA FOR CONSIDERING
comparison 2, 3 and 4. STUDIES FOR THIS REVIEW
2. Audit and feedback (+/- printed educational materials) com-
pared to no intervention (+/- printed educational materials). Types of studies
3. Audit and feedback that includes educational meetings (+/- Randomised controlled trials (RCTs).
printed educational materials) compared to no intervention (+/-
printed educational materials). Types of participants
Healthcare professionals responsible for patient care. Studies that
4. Audit and feedback as part of a multifaceted intervention (i.e. included only students were excluded.
combined with reminders, opinion leaders, outreach visits, pa-
tient mediated interventions, local consensus processes or tailor- Types of intervention
ing strategies) compared to no intervention. Audit and feedback: defined as any summary of clinical perfor-
5. Short term effects of audit and feedback compared to longer- mance of health care over a specified period of time. The summary
term effects after feedback stops. may also include recommendations for clinical action. The infor-
mation may be given in a written, electronic or verbal format.
The following comparisons are considered in addressing the sec-
ond question. Types of outcome measures
Objectively measured provider performance in a health care set-
6. Audit and feedback combined with complementary interven-
ting or health care outcomes. Studies that measured knowledge or
tions (reminders, opinion leaders, educational outreach visits, pa-
performance in a test situation only were excluded.
tient mediated interventions, local consensus processes or tailor-
ing strategies) compared to audit and feedback alone.
7. Audit and feedback compared to other interventions (re- SEARCH METHODS FOR
minders, opinion leaders, educational outreach visits, patient me- IDENTIFICATION OF STUDIES
diated interventions, local consensus processes or tailoring strate-
gies) See: methods used in reviews.
8. All comparisons of different ways audit and feedback is done, The review has been updated primarily by using the EPOC
including: register and pending file. We identified all articles in the
Cochrane Effective Practice and Organisation of Care (EPOC)
• Audit and feedback that includes peer comparison versus indi-
register in January 2001 that had been coded as an RCT or
vidual feedback without peer comparison (change in content).
clinical controlled trial (CCT) and as ’audit and feedback’. The
• Audit and feedback from an influential source versus audit and EPOC pending file (studies selected from the EPOC search
feedback from any other source (change in source). (An influen- strategy results and awaiting assessment) was also searched in
tial source in this context is a person who is seen to be credible January 2001 using the terms ’audit’ or ’feedback’. In addition
and trustworthy by the professional). the previous MEDLINE strategy was used to search MEDLINE
from January 1997 to April 2000 and any articles already
• Individual audit and feedback versus group audit and feedback
identified by the EPOC strategy were excluded. This search did
(change in recipient).
not generate any relevant additional articles and therefore was not
• Audit and feedback in a verbal format versus audit and feedback repeated. The reference lists of new articles that were obtained
in a written format (change in format). were reviewed.
• Audit and feedback given once only versus audit and feedback Previous searches built upon earlier reviews (Thomson 1995;
given more frequent (change in frequency) Davis 1995; Oxman 1995; Davis 1992). We searched MEDLINE
from January 1966 to June 1997 without language restrictions.
• Audit and feedback given once only versus audit and feedback
These search terms were used: explode education, professional
over a period of time (change in duration)
(tw), explode quality of health care, chart review: or quality
In addition we have reported all direct comparisons of different assurance (tw), feedback (sh), audit (tw,sh) combined with these
ways of providing audit and feedback that we have identified in methodolological terms: clinical trial (pt), random allocation
(sh), randomised controlled trials (sh), double-blind method primary outcomes or protection against contamination. We
(sh), single-blind method (sh), placebos (sh), all random: (tw). assigned a rating of high protection against bias if the first three
The Research and Development Resource Base in Continuing criteria were scored as done, and there were no important concerns
Medical Education(RDRB/CME) (Davis 1991) was also related to the last three criteria, moderate if one or two criteria were
searched. The reference lists of related systematic reviews and all scored as not clear or not done, and low if more than two criteria
articles obtained were reviewed. were scored as not clear or not done. For cluster randomisation
trials, we rated protection against contamination as done. Further,
An updated search was done in November 2002. Potentially
for these study designs, we rated concealment of allocation as done
relevant studies found with the updated search are included
if all clusters were randomised at one time.
under References to studies awaiting assessment.
We also categorised the intensity of the audit and feedback,
the complexity of the targeted behaviour, and the level of
METHODS OF THE REVIEW baseline compliance. The intensity of the audit and feedback was
categorised based on the following characteristics listed in the
The following methods were used in updating this review: order that we hypothesised would be most important in explaining
differences in the effectiveness of the audit and feedback (with the
Two reviewers (GJ and JY) independently applied inclusion categories listed from ’more intensive’ to ’less intensive’ for each
criteria, assessed the quality of each study, and extracted data for characteristic):
newly identified studies using a revised data-collection form from
the EPOC Group. The same data were also collected from the • the recipient (individual versus group)
studies included in the original version of this review by these
• the format (verbal versus written, or both verbal and written)
two reviewers. The quality of all eligible studies was assessed using
criteria described in the EPOC module (see Group Details) and • the source (a supervisor or senior colleague, versus a
discrepancies were resolved by discussion. ’professionals standards review organisation’ or representative
of the employer or purchaser, versus the investigators)
In light of the results of a recent review of the effects of
continuing education meetings (Thomson O’Brien 2001), which • the frequency of the feedback, categorised as frequent (up to
suggests that interactive educational meetings frequently have weekly), moderate (up to monthly) and infrequent (less than
moderate effects on professional practice, in updating this review monthly)
we considered interactive, small group meetings separately from
• the duration of feedback, categorised as prolonged (one year or
written educational materials and didactic meetings, which have
more), moderate (between one month and one year) and brief
been found to have little or no effect on professional practice
(less than one month)
(Thomson O’Brien 2001;Freemantle 1997; Grimshaw 2001). A
revised definition for educational meetings was applied to all • the content of the feedback (patient information, such as blood
of the studies included in the review: participation of health pressure or test results, compliance with a standard or guideline,
care providers in meetings that included interaction among the or peer comparison; versus information about costs or numbers
participants, whether or not the meetings were outside of the of tests ordered or prescriptions)
participants practice settings.
We also categorised the overall intensity of the audit and feedback
We have used the term tailoring instead of marketing in the by combining the above
update, in keeping with another review (Cheater 1999) and we characteristics as “Intensive” (individual recipients) AND ((verbal
have defined multifaceted interventions as including two or more format) OR (a supervisor
interventions (+/- printed educational materials and educational or senior colleague as the source)) AND (moderate or prolonged
meetings). For multifaceted interventions that included audit and feedback); “Non-intensive” ((group feedback) NOT (from a
feedback two of us (GJ and JY) have independently categorised supervisor or senior colleague)) OR ((individual feedback) AND
the contribution of audit and feedback as a component of the (written format) AND (containing information about costs or
intervention in a subjective manner as a major, moderate or minor numbers of tests without personal incentives)); or “Moderately
component. intensive”(any other combination of characteristics than described
in Intensive or Non-intensive group).
For all of the studies included in the review an overall quality
rating (high, moderate, low protection against bias) was assigned The complexity of the targeted behaviour was categorised in
based on the following criteria: concealment of allocation, blinded a subjective manner independently by two of us (GJ and JY)
or objective assessment of primary outcome(s), and completeness as high, moderate or low. The categories depending upon the
of follow-up (mainly related to follow-up of professionals) and number of behaviours required, the extent to which complex
no important concerns in relation to baseline measures, reliable judgements or skills were necessary, and whether other factors
such as organisational change were required for the behaviour • complexity of the targeted behaviour
to be improved, and also depending on whether there was need
• baseline non-compliance
for change only by the individual/professional (one person) or
communication change or change in systems. Baseline compliance • study quality (high or moderate protection against bias)
with the targeted behaviours was treated as a continuous variable
ranging from zero to 100%, based on the pre-intervention We visually explored heterogeneity by preparing tables, bubble
level of compliance in the audit and feedback group before the plots and box plots (displaying medians, interquartile ranges, and
intervention. ranges) to explore the size of the observed effects in relationship to
each of these variables. The size of the bubble for each comparison
We used the following definitions for interventions other than corresponded to the number of healthcare professionals who
audit and feedback: participated. We also plotted the lines from the weighted regression
Educational materials: distribution of published or printed to aid the visual analysis of the bubble plots.
recommendations for clinical care, including clinical practice Each comparison was characterised relative to the other variables
guidelines, audio-visual materials and electronic publications. in the tables, looking at one potential explanatory variable
Educational meetings: participation of health care providers at a time. We looked for patterns in the distribution of
in conferences, lectures, workshops or traineeships outside the the comparisons, hypothesising that larger effects would be
providers’ practice settings. associated with multifaceted interventions, more intensive audit
and feedback, less complexity of the targeted behaviour, higher
Patient mediated interventions: any intervention aimed at baseline non-compliance, and lower study quality.
changing the performance of health care providers indirectly by
providing information, prompts, or support to the patient. The visual analyses were supplemented with multivariate statistical
analyses. We used weighted meta-regression to examine how the
Reminders: any intervention, manual or computerised, that size of the effect was related to the five potential explanatory
prompts the health care provider to perform a clinical action. variables listed above; weighted according to the number of
Tailoring: use of personal interviewing, group discussion (’focus health care professionals. These analyses were conducted using
groups’), or a survey of targeted providers to identify barriers to generalized linear modelling in SAS (Version 8.2. SAS Institute
change and subsequent design of an intervention that addresses Inc., Cary, NC, USA). Two main analyses were conducted for
identified barriers. comparison 1 (audit and feedback +/- educational meetings or
audit and feedback as part of a multifaceted intervention compared
Local consensus processes: inclusion of participating providers to no intervention): one using the adjusted relative risk as the
in discussion to ensure that they agreed that the chosen clinical measure of effect and one using the adjusted risk difference as the
problem was important and the approach to managing the measure of effect. We also conducted an analysis using the adjusted
problem was appropriate. odds ratios. However, the analyses using the adjusted OR were
Analysis heavily influenced by a single study (Thompson 2000). They did
not provide any additional insights into the relationship between
We only included studies of moderate or high quality in the the five potential explanatory variables that we investigated and
analyses. All outcomes were expressed as non-compliance with the size of the effect, and those results are therefore not reported
desired practice. here.
When several outcomes were reported in one trial we only
Because there were frequently important baseline differences in
extracted results from the primary outcome. If the primary
compliance between intervention and control groups in trials,
outcome was not specified, we calculated the mean effect size for
our primary analyses were based on adjusted estimates of effect,
the outcomes reported in the trial.
where we adjusted for baseline differences in non-compliance. For
Because of missing data and unit of analysis errors for continuous dichotomous outcomes we calculated the adjusted risk difference
outcomes, only dichotomous outcomes were included in the visual and relative risk as follows:
and statistical analyses for comparisons 1-4.
Adjusted risk difference (RD) = the difference in the risk of non-
We considered the following potential sources of heterogeneity to compliance after the intervention minus the difference before
explain variation in the results of the included studies: the intervention. A positive risk difference indicates that non-
compliance was reduced more in the audit and feedback group
• the type of intervention (audit and feedback alone, audit
than in the control group, e.g. adjusted risk difference of 0.09
and feedback with educational meetings, or multifaceted
indicates an absolute improvement in care (reduction in non-
interventions that included audit and feedback)
compliance) of 9%. [(Control non-compliance - Audit and
• the intensity of the audit and feedback feedback non-compliance) after the intervention] - [(Control non-
compliance - Audit and feedback non-compliance) before the care, hypertension, hand washing. For the most part, the com-
intervention]. plexity of the targeted behaviours was homogeneous and rated as
moderate (n= 68), for example ordering of laboratory tests, child
“Adjusted relative risk” (RR) = the ratio of the relative risk of
immunization, compliance with guidelines of various complexity
non-compliance after the intervention over the relative risk before
and screening. In ten studies the complexity of the targeted be-
the intervention. A relative risk less than one indicates that non-
haviour was assessed as low, for example inappropriate prescribing
compliance was reduced more in the audit and feedback group
of antibiotics and influenza vaccination. In seven studies the com-
than in the control group, e.g. adjusted relative risk of 0.8 indicates
plexity of the targeted behaviour was rated as high, for example
a relative improvement in care (reduction in non-compliance) of
provision of caesarean section deliveries and multiple behaviours
20%.
that were used to determine overall quality of care.
The results of studies that reported continuous outcomes were
Characteristics of interventions
analysed qualitatively, independently by two reviewers by looking
at the distribution of effects for continuous outcomes relative to Overall intensity of feedback
that of the dichotomous. The size of the effect for continuous
In 14 studies the overall intensity of feedback was rated as non-
outcomes were categorised as large negative, moderate negative,
intensive, in seven studies as intensive. In the remaining 64 studies
small negative, no difference, small positive, moderate positive
the intensity was moderate (Table 02). The interventions used
or large positive independently by two reviewers (GJ and JY)
were highly heterogeneous with respect to their content, format,
corresponding to the risk differences shown in Table 01.
timing and source.
Content
DESCRIPTION OF STUDIES
The feedback consisted of different kinds of information (Table
02). In 29 trials, the subjects received compliance scores with cri-
Eighty-five studies met the inclusion criteria. The unit of allocation
teria for care or guidelines, in 20 studies peer-comparison feed-
was the patient in three studies, health professional in 39, practice
back was given, in some studies together with compliance scores.
in 20, institution in six, and in 15 studies the unit of allocation
Feedback was also given as summaries of the numbers or costs of
was “other”, for example health units, departments or pharmacies.
diagnostic tests or prescriptions or as patient-related information
In one study the unit of allocation was not clear. Nine studies had
such as blood pressure recordings or test results.
four arms, 16 studies had three and the remaining 60 had two
arms. Source
Characteristics of setting and professionals For the majority of trials, the source of the feedback was not clear
and we assumed it was the investigators. In 12 studies a supervisor
Fifty-three trials were based in North America (46 in the USA,
or senior colleague was the source and in four trials the source was
seven in Canada), 16 in Europe (nine in United Kingdom, three in
specified as the Professional Standards Review Organisation.
The Netherlands, two in Denmark and one each in Finland, and
Belgium) eight in Australia, two in Thailand and one in Uganda. Recipient
Twenty-five studies took place in hospitals and outpatient clinics,
For most trials (n=60) , the individual was the recipient of the
two in pharmacies (De Almeida Neto 2000; Mayer 1998), one
feedback. In 22 trials feedback was provided to a group and in
in dental practices (Brown 1994) and the remaining studies were
three studies the recipient was not clear.
based in general practice, family medicine or community care. In
most trials the health professionals were physicians. One study Format
involved dentists (Brown 1994), in two studies the providers were
In most trials (n=46) the format was a printed computer report.
nurses (Jones 1996; Moongtui 2000), in two pharmacists (De
Less often (n=23) the feedback was given verbally. In 16 studies
Almeida Neto 2000; Mayer 1998) and five studies involved mixed
both written and verbal feedback were used.
providers (Palmer 1985; Meyer 1991; Smith 1995; Thompson
2000; Winickoff 1985). Timing (frequency and duration)
Targeted behaviours The frequency and duration of the feedback also varied. In 35
trials feedback was given only once. In eight it was given weekly or
There were 20 trials of preventive care, for example screening, vac-
more frequently. In the rest, feedback was repeated less frequently
cinations or prophylaxis for venous thrombosis; nine trials of test
than weekly. Nine studies had a duration of feedback for more
ordering, for example laboratory tests, x-ray; 18 of prescribing and
than one year (prolonged), 35 studies between one year and one
one of reduction in hospital length of stay. The remaining studies
month and the remaining less than a month (brief ) or once only.
were trials of general management of a variety of problems, for
example compliance with guidelines for different conditions, burn Audit and feedback as part of educational meetings
In 22 studies audit and feedback was provided as part of educa- Some studies were identified as duplicates. One study (Veninga
tional meetings. Some meetings were outreach visits. 1999) was a multi-centre study that included results from three
national studies, two of which were published separately.
Audit and feedback as part of a multifaceted intervention
There were 24 studies in which one or more groups received a Comparison 1. Any intervention in which audit and feedback
multifaceted intervention that included audit and feedback as one is a component compared to no intervention (+/- printed ed-
component. Two studies included feedback in two arms: with edu- ucational materials) (Other data tables 1.1 2.3)
cational meetings or as part of a multifaceted intervention (Kafuko
A total of 83 comparisons from 71 studies were included in this
1999; Vinicor 1987). In the 24 studies of multifaceted interven-
comparison. Fifty-two comparisons of dichotomous outcomes
tions the additional interventions included educational meetings,
from 47 trials with a total of over 3500 health professionals were
reminders, patient-mediated interventions, outreach visits, local
included in the primary analyses. Eight of these trials had a low
consensus processes and incentives or different types of support,
risk of bias. The rest had a moderate risk. There was important
such as office systems.
heterogeneity among the results and variation in both the RDs
Outcome measures and RRs across studies.
There was large variation in outcome measures, and some stud- The adjusted relative risk of non-compliance with desired practice
ies reported multiple outcomes, for example studies on compli- (n = 52) varied from 0.31 to 1.20 (median = 0.85, inter-quartile
ance with guidelines for preventive care. Most trials measured out- range = 0.74 to 0.96). None of the variables that we examined
comes on a professional practice level, for example number of pre- (type of intervention, baseline compliance, complexity of the tar-
scriptions or laboratory tests ordered per doctor or compliance geted behaviour, intensity of the audit and feedback, study qual-
with guidelines, or had a mixture of practice or patient outcome ity) helped to explain the variation in relative effects across stud-
measures. Only ten trials reported patient outcomes, for example ies in the statistical analysis (P values for the coefficients ranged
length of stay, control of hypertension or depression. There was a from 0.21 to 0.99), the visual analyses, or the qualitative analyses,
mixture of dichotomous (for example the proportion compliance which included studies with continuous outcomes. The adjusted
with guidelines, the proportion of tests done, and the proportion relative risk appeared to decrease (indicating larger effects) as base-
vaccinated) and continuous outcome measures (for example costs, line non-compliance increased (Coefficient = -0.24, P = 0.21), but
number of laboratory tests, number of prescriptions, length of this relationship was not strong (Figure 01).
stay).
The adjusted RDs of non-compliance with desired practice varied
from -0.09 (a 9% absolute increase in non-compliance) to 0.71 (a
METHODOLOGICAL QUALITY 71% decrease in non-compliance) (median = 0.07, inter-quartile
range = 0.02 to 0.11). The absolute effectiveness of multifaceted
See Table 03. Ten (high quality) studies had a low risk of bias. interventions increased as baseline non-compliance increased (co-
Fourteen (low quality) trials had a high risk of bias. The remaining efficient = 0.34, P = 0.03, suggesting that on average across studies
61 studies were of moderate quality. Randomisation was clearly the absolute reduction in non-compliance increased by 3% with
concealed or there was cluster randomisation in 43 trials, and in each 10% increase in baseline non-compliance (Figure 02). None
the rest of the studies the randomisation procedure was not clear. of the other variables that we examined (type of intervention, com-
There was adequate follow-up of health professionals in 50 trials, plexity of the targeted behaviour, intensity of the audit and feed-
inadequate follow-up in seven trials and the remaining 29 trials back, study quality) helped to explain the variation in absolute ef-
this was not clear. Outcomes were assessed blindly in 45 trials, not fects across studies in the statistical analysis, where the P values for
blindly in seven and was not clear in 43 studies. the other coefficients ranged from 0.14 to 0.37, the correspond-
ing visual analyses or the qualitative analyses that included studies
with continuous outcomes.
RESULTS
Three of the studies reported a large effect. One was a multifaceted
We identified 120 new studies that were potentially relevant and intervention aimed at increasing the provision of skin cancer pre-
retrieved these. We located studies mainly using the EPOC reg- ventive advice by pharmacists in the USA (Mayer 1998). The sec-
ister and pending file. Of the 37 studies included in the original ond was audit and feedback + educational meetings (outreach)
review, we excluded two (McPhee 1989; Putnam 1985) because aimed at reducing inappropriate prescriptions of tetracycline for
of insufficient data about the results. Fifty-two of the new stud- upper respiratory infections in the USA (McConnell 1982), and
ies that were retrieved were excluded for the reasons reported in the third one was audit and feedback alone aimed at improving
the excluded studies table. The updated search identified 17 ad- hand wash and glove use among nurses and patient care aids in
ditional studies that are awaiting assessment. Thailand. (Moongtui 2000).
The rest of the studies reported small negative to moderate positive Comparison 3. Audit and feedback with educational meetings
effects. For nine of 52 comparisons for which there were sufficient compared to no intervention (Other data tables 1.2 and 2.2)
data to calculate an adjusted risk difference the adjusted RD was
Twenty-four comparisons from 19 trials were included in this com-
larger than 10%. For 13 comparisons the adjusted RD was close
parison. Twelve comparisons from nine trials had dichotomous
to zero (-5% to 5%). For two comparisons from the same study
outcomes and 12 comparisons from 10 trials had continuous out-
(Mainous 2000) there was an absolute increase in non-compliance
comes. The 12 comparisons that reported dichotomous outcomes
of 9%, using either audit and feedback alone or a multifaceted in-
included over 1000 health professionals from nine studies. The
tervention aimed at reducing antibiotic prescribing rates for upper
studies had a variety of outcome measures. Two studies were of
respiratory infections.
high methodological quality (Hendryx 1998; Lomas 1991). The
The 31 comparisons from 24 studies that reported continuous out- intensity of the feedback was high in two studies (Brown 1994;
comes included over 2900 health professionals. One of these stud- Kim 1999), low in one study (Kafuko 1999) and moderate in the
ies was of high methodological quality (Hendryx 1998). The out- rest. The complexity of the targeted behaviours was low in one
come measures varied. The complexity of the targeted behaviours study (McConnell 1982), high in one (Hendryx 1998), and mod-
was moderate in all of the studies and the intensity of the audit erate in the rest. Baseline compliance ranged from 0% to 73%.
and feedback was moderate for all except for one study with low
The adjusted relative risk of non-compliance with desired practice
intensity (Kafuko 1999) and one study with high intensity (Mar-
ranged from 0.5 to 1.0 (median = 0.89, inter-quartile range =
tin 1980). The type of intervention was audit and feedback alone
0.62 to 0.94). The adjusted risk difference ranged from 0.5%
in 17 comparisons, audit and feedback with educational meetings
to 47% (median = 0.06, inter-quartile range = 0.03 to 0.24).
in 12 comparisons and a multifaceted intervention in two com-
One high quality study reported a large relative improvement in
parisons. The size of the effects was small or none in most of these
compliance with intensive care unit guidelines for processes of care
studies, suggesting small effects than for the dichotomous results.
(Hendryx 1998) (adjusted RR 0.63, adjusted RD 0.15). The other
Comparison 2. Audit and feedback alone compared to no in- high quality study found no effect (unadjusted RR = 1.02) on the
tervention (Other data tables 1.1 and 2.1) proportion of women with a previous caesarean section offered a
trial of labour (Lomas 1991).
Thirty-nine comparisons from 33 trials were included in this com-
parison. Twenty-two comparisons had dichotomous outcomes The 12 comparisons that reported continuous outcomes included
and 17 had continuous outcomes. The 22 comparisons that re- over 800 health professionals from ten studies. One of these stud-
ported dichotomous outcomes included over 1400 health pro- ies was of high methodological quality (Hendryx 1998). The out-
fessionals from 21 studies. The studies had a variety of outcome come measures varied. In one study the intensity of feedback was
measures. One study had a low risk of bias (Hillman 1999), the intensive (Martin 1980), in one low (Kafuko 1999) and in the rest
quality of the rest of the studies was moderate. The intensity of moderately intensive. The complexity of the targeted behaviours
the feedback was low in three comparisons (Hemminiki 1992; was moderate. The size of the effects was small or null in these stud-
McCartney 1997; Schectman 1995) and moderately intensive in ies. The one high quality study had a small positive effect (Hendryx
the rest. The complexity of the targeted behaviours was low in 1998). The multi-centre study of compliance with guidelines for
six studies, high in one (Balas 1998), and moderate in the rest. asthma in four countries (Veninga 1999) found small positive ef-
Baseline non-compliance ranged from 15% to 86%. fects in three countries and no effect in one.
Comparison 4. Audit and feedback as part of a multifaceted
The adjusted relative risk ranged from 0.6 to 1.1 (median = 0.84,
intervention compared to no intervention (Other data tables
inter-quartile range = 0.76 to 1.0). The adjusted risk difference
1.3 and 2.3)
ranged from 4% to 32% (median = 0.07, inter-quartile range =
0.02 to 0.10). The high quality study (Hillman 1999) found a This comparison included audit and feedback combined with ed-
small effect for compliance with guidelines for preventive care in ucational meetings, reminders, the use of opinion leaders, out-
children (adjusted relative risk 0.86, adjusted risk difference of reach visits, patient mediated interventions, local consensus pro-
6%). cesses or tailoring of interventions to address identified barriers to
change. Twenty comparisons from 19 trials were included in this
The 17 comparisons from 13 studies that reported continuous out-
comparison. Eighteen comparisons from 17 studies had dichoto-
comes included over 1900 health professionals. One study (Chas-
mous outcomes and two trials had continuous outcomes.
sin 1986) included 1483 health professionals. None of these stud-
ies were of high methodological quality. The outcome measures The 18 comparisons that reported dichotomous outcomes in-
varied. The complexity of targeted behaviours was moderate in all cluded over 1088 health professionals. The studies had a variety
studies. The size of the effects was small or none in most (n=11) of of outcome measures. Five studies were of high methodological
these studies, suggesting smaller effects than for the dichotomous quality. The intensity of the feedback was low in six studies and
outcomes. moderate in the rest. The complexity of the targeted behaviours
was low in one study (Buffington 1991) and moderate in the rest. 1990; Buffington 1991; Tierney 1986). These studies found no
Baseline compliance ranged from 0% to 57%. additive effect of adding reminders to audit and feedback.
The adjusted relative risk of non-compliance with desired practice One study with four arms compared audit and feedback alone to
ranged from 0.31 to 1.17 (median = 0.85, inter-quartile range = audit and feedback plus patient education materials to improve
0.79 to 0.92). The adjusted risk difference ranged from -9% to antibiotic prescribing for respiratory infections (Mainous 2000).
71% (median = 0.08, inter-quartile range = 0.05 to 0.12). The Adding patient education to audit and feedback had no influence
high quality studies had relative reductions in non-compliance on prescribing.
between 8% and 30%.
Two studies compared audit and feedback to audit and feedback
The two comparisons that reported continuous outcomes came plus incentives (Fairbrother 1999; Hillman 1999). Fairbrother,
from two studies (Kafuko 1999; van der Weijden 1999). Both a moderate quality study, had three arms comparing audit and
studies were of moderate quality. The main outcome measures feedback alone to audit and feedback plus an one-off financial
were the average number of drugs prescribed (Kafuko 1999) and bonus based on up-to-date coverage for four immunisations, and
compliance with guidelines for high cholesterol (van der Weij- audit and feedback plus “enhanced fee for service” (five dollars
den 1999). The intensity of feedback and the complexity of the for each vaccine administered within 30 days of its due date).
targeted behaviours was moderate for both studies. Both studies Rates of immunisation improved significantly from 29% to 54%
found no effect. coverage in the bonus group after eight months. However, the
Comparison 5. Short term effects of audit and feedback com- percentage of immunizations received outside the practice also
pared to longer term effects after feedback stops (Other data increased significantly in this group. The enhanced fee-for-service
table 2.1) and audit and feedback alone groups did not change. There were
only 15 physicians in each group and baseline differences, although
This comparison included 8 trials with 11 comparisons (Cohen this was controlled for in the analysis. The other, a high quality
1982; Buntinx 1993; Fairbrother 1999; Jones 1996; Norton 1985; study (Hillman 1999), showed no effect of adding incentives to
Ruangkanchanastr 93; Smith 1995; Zwar 1999). One study com- audit and feedback to implement guidelines for cancer screening.
prises three comparisons (Ruangkanchanastr 93). The follow-up
period after audit and feedback stopped varied from three weeks to Borgiel (Borgiel 1999) compared audit and feedback alone to audit
14 months. There were mixed results. In the trial by Cohen (Co- and feedback plus outreach visits and a continuing education plan
hen 1982), the control group demonstrated improvement during to improve quality of care. Adding the educational intervention
the three week follow-up period. The authors attributed these re- to audit and feedback did not result in better outcomes.
sults to a co-intervention (an interested team leader) in the control
Two studies compared audit and feedback alone to audit and feed-
group. In the trial by Fairbrother (Fairbrother 1999) both groups
back plus self-study (Dickinson 1981) or self audit (Brady 1988).
showed small improvements during follow-up. One study evalu-
In one of the studies there was no difference between the groups in
ated the effect of withdrawal of feedback on the quality of a hospi-
the proportion of patients with controlled blood pressure (Dick-
tal capillary blood glucose monitoring program (Jones 1996). This
inson 1981). The other study had mixed results with improved
study showed that the improvement in performance was main-
performance related to mammography screening in the self-audit
tained at six months, but deteriorated by 12 months. In the trial
group but no difference in influenza immunisations (Brady 1988).
by Norton (Norton 1985), the experimental group demonstrated
improvement in the management of cystitis but not in vaginitis Meyer (Meyer 1991) compared audit and feedback with peer re-
when assessed 14 months later. Buntinx (Buntinx 1993) and Zwar view and recommendations to audit and feedback alone to im-
(Zwar 1999) showed no improvement short term or at follow-up. prove prescribing. This study had a high risk of bias. It found no
In a study comparing audit and feedback plus educational meet- difference between the two methods.
ings to educational meetings alone to improve the presentation of
screening tests (Smith 1995), communication levels declined to In one high quality study, audit and feedback plus assistance to
baseline levels for both intervention groups at three months fol- develop an office system tailored to increase breast cancer screen-
low-up, but obstetricians and midwives continued to give more ing rates was compared to feedback alone (Kinsinger 1998). The
information to patients. intervention increased indicators of office systems, but had little
impact on breast cancer screening.
Comparison 6. Audit and feedback combined with comple-
mentary interventions compared to audit and feedback alone Comparison 7. Audit and feedback compared to other inter-
(Other data table 3.1) ventions (Other data table 4.1)
Thirteen trials were included in this comparison with various com- Five trials were included is this comparison. Audit and feedback
binations of interventions. Four trials compared audit and feed- was compared to reminders in two studies (Boekeloo 1990; Tier-
back to audit and feedback plus reminders (Baker 1997; Boekeloo ney 1986). In one study with a high risk of bias, the audit and
feedback group performed better than the reminder group in low- In one study that compared group audit and feedback with group
ering cholesterol (Boekeloo 1990). In another moderate quality plus individual feedback there was no difference in prophylaxis for
study the audit and feedback group performed slightly better in venous thromboembolism (Anderson 1994).
delivering preventive services (Tierney 1986).
In one study in which audit and feedback was compared to patient Comparison 9. Studies with patients as unit of allocation
education (Mainous 2000) there was no difference between groups (Other data table 6.1)
in antibiotic prescribing rates.
Smith (Smith 1995) compared audit and feedback plus educa- In three studies the unit of allocation was the patient and the
tional meetings to educational meetings alone to improve the pre- provider received feedback for some patients and not for others
sentation of screening tests by obstetricians and midwives. Knowl- (Belcher 1990; Meyer 1991; Simon 2000). In one study audit and
edge change scores and communication skills improved in both feedback alone was compared to audit and feedback plus care man-
groups, but the two groups did not differ in performance between agement to reduce costs and follow-up visits related to patients
baseline and post-intervention, or between baseline and follow- with depression (Simon 2000). Adding care management resulted
up. in higher costs and did not change follow-up visits. In a four arm
study (Belcher 1990) comparing audit an feedback plus meetings
Lomas (Lomas 1991) compared audit and feedback to local opin- and reminders to audit and feedback plus meetings, reminders and
ion leader education to implement guidelines for the management patient information to audit an feedback plus meetings, reminders
of women with a previous caesarean section in a high quality study. and inviting patients to a health promotion clinic to improve the
The opinion leader group reduced caesarean section rates and im- delivery of preventive services. Adding the invitation to the health
proved the quality of care. The audit and feedback group did not promotion clinic improved prevention rates. No improvements
differ from the control group. were observed in the other groups. Meyer (1991) compared a sin-
gle letter recommending that the number of medications received
Comparison 8. All comparisons of different ways audit and
by patients should be reduced to audit and feedback plus a com-
feedback are done (Other data table 5.1)
pliance index, peer review and recommendations; and to a control
Content group. At four months both intervention groups had significant
reductions in polypharmacy compared to the control group, but
One study compared audit and feedback with and without peer there was no difference between the two intervention groups.
comparison (Wones 1987). No difference was found in perfor-
mance between groups, but the study was underpowered with only
seven residents in each group. High quality studies (Other data table 7.1)
One study that compared feedback on medication with feedback

Overall ten studies were judged to be of high quality (with a low
on performance found no difference in control of blood pressure
risk of bias). One study (O’Connell 1999) that compared audit
(Gullion 1988).
and feedback to no intervention found that audit and feedback
Source had no impact on prescribing of general practitioners. Two stud-
ies included a comparison of audit and feedback plus educational
In one study mutual visits and feedback by peers was compared meetings to no intervention (Hendryx 1998; Lomas 1991). One
with visits and feedback by a non-physician observer to improve found a 37% relative reduction in non-compliance and the other
performance related to 208 indicators of practice management did not find a difference. Six studies included a comparison of
(van den Hombergh 99). Both programmes showed improvements a multifaceted intervention to no intervention (Hillman 1999;
after a year, but different aspects changed in each of the two pro- Kerse 1999; Leviton 1999; Manfredi 1998; Thompson 2000;
grammes. The improvement was more noticeable after mutual Wells 2000). They found relative reductions in non-compliance
practice visits than after a visit by a non-physician observer. of 8% to 30%. Two studies included a comparison of audit and
In another study with three arms audit and feedback alone was feedback plus complementary interventions to audit and feedback
compared to audit and feedback plus outreach by a physician, or alone (Hillman 1999; Kinsinger 1998). They did not find an ef-
audit and feedback plus outreach by a nurse (Ward 1996). This fect of adding incentives for paediatric preventive care (Hillman
study showed that both adding outreach by a peer and by a nurse 1999) or assistance to develop an office system tailored to increase
to audit and feedback improved diabetes management. There was breast cancer screening rates (Kinsinger 1998) to audit and feed-
no important difference in outcomes between outreach by a peer back. One study included a comparison of audit and feedback
or a nurse. to other interventions (Lomas 1991). It found that using local
opinion leaders reduced caesarean section rates whereas audit and
Recipient feedback plus educational meetings did not (Lomas 1991).
DISCUSSION a plausible association for any one of the criteria and the effects of
audit and feedback.
Audit and feedback can be a useful intervention, the adjusted RDs
There are a number of plausible explanations why some interven-
of non-compliance with desired practice varied from -0.09 (a 9%
tions were effective and others were not. Of the factors that we
absolute increase in non-compliance) to 0.71 (a 71% decrease in
specified, baseline compliance was the only factor that helped to
non-compliance) (median = 0.07, inter-quartile range = 0.02 to
explain variation in the absolute effectiveness across studies and
0.11) with or without educational meetings or other complemen-
possibly, to a lesser extent, in the relative effectiveness.
tary interventions. However, the effects of audit and feedback vary
from an apparent negative effect to a very large positive effect in In contrast to the conclusions of other systematic reviews
the trials included in this review. (Grimshaw 2001), but consistent with another more recent re-
view of interventions to implement clinical practice guidelines
In most of the included studies, the method of allocation was not
(Grimshaw 2002), we did not find evidence of a larger effect
clearly indicated in the published report. Although lack of alloca-
for multifaceted interventions compared with audit and feedback
tion concealment can result in overestimates of effect (Kunz 2002),
alone (Figure 03).
the importance of this criterion in trials where a group of health
professionals is randomised at one point in time has not been es- Seven of ten high quality studies included comparisons of multi-
tablished. In this review we have given cluster randomised trials faceted interventions with no intervention and two included com-
the benefit of the doubt and assumed that there was adequate con- parisons of audit and feedback + educational meetings with no in-
cealment of allocation for these studies. Nonetheless, we judged tervention. It is possible that an effect of methodological quality on
only ten of the 85 included studies to be of high methodological the observed effectiveness of audit and feedback was confounded
quality. with the type of intervention that was evaluated. Our assessments
of the intensity of audit and feedback may suffer from the same
In our primary analyses we chose to focus on comparisons with di-
problem as our assessments of methodological quality. Both are
chotomous outcomes where it was possible to calculate an adjusted
complex concepts for which there is no solid basis for deriving
relative risk and risk difference. The adjustments were based on
a summary assessment. Our assessments of the intensity of audit
pre-intervention measurements of the outcome in the audit and
and feedback were based on six components (the recipient, format,
feedback group. We excluded studies that we judged to be of low
source, frequency, duration and content). There are theoretical
quality from these comparisons. Because many studies included
and intuitive arguments for how we have categorised the overall
small numbers of health professionals, baseline differences were
intensity of audit and feedback, but no clear empirical basis. We
common and unadjusted estimates of effect often differed from the
considered the intensity of audit and feedback to be moderate in
adjusted estimates. Unit of analysis errors were common and be-
most (n=64) of the included studies. As with methodological qual-
cause of these it was frequently impossible to reliably estimate the
ity, we considered the risk of finding spurious associations greater
size of changes in comparisons with continuous outcomes. Qual-
than the likelihood of finding a plausible association for any one of
itative analyses that included comparisons that we considered to
the components of intensity and the effects of audit and feedback.
be less reliable did not suggest that those studies had dramatically
different results or that they contributed important information Five studies provided direct, randomised comparisons of different
not found in the results of the comparisons that we considered ways of providing audit and feedback. Based on these comparisons
more reliable. and indirect comparisons across studies it is not possible to deter-
mine what, if any features of audit and feedback have an impor-
Our overall assessments of study quality did not help to explain
tant impact on its effectiveness. Although there are hypothetical
the variation in results, largely because most of the studies (61/85)
reasons why some forms of audit and feedback might be more ef-
were judged to be of moderate quality. It has been recommended
fective than others, there is not an empirical basis for deciding how
that the use of quality scales or summary scores should not be
to provide audit and feedback. Decisions about how to provide
used in meta-regressions (Juni 1999; Juni 2001). In this review
audit and feedback must be guided by pragmatic factors and local
our global judgements about study quality can be considered as a
circumstances.
type of summary score. However, we chose not to investigate any
of the component criteria used to assess study quality as potential Twenty-five of the trials included in this review included peer-
variables that might help to explain the observed variation in re- comparison feedback (Table 02). The effects observed in these tri-
sults. With a single variable for study quality we already had five als are similar to the effects of audit and feedback generally. For the
explanatory variables and only 52 comparisons from 47 studies 15 comparisons with no intervention, the relative improvement
in the meta-regression. There is neither empirical evidence nor in compliance ranged from a 60% relative improvement to a 20%
strong logical arguments for selecting any of the component cri- relative reduction in compliance (median adjusted RR = 0.91, in-
teria as potential explanatory variables. We considered the risk of ter-quartile range = 0.81 to 1.00). No difference was found in the
finding spurious associations greater than the likelihood of finding one, underpowered direct comparison of peer-comparison feed-
back to feedback without peer comparison (Wones 1987). Thus, data to clarify when the effects of audit and feedback are most
there is at present no basis for concluding that peer-comparison likely to deteriorate after feedback stops.
feedback is either more or less effective than audit and feedback
Two of the studies reported a large effect of audit and feedback, one
generally. In contrast to the conflicting conclusions of Axt-Adams
a multifaceted intervention (Mayer 1998) and the other audit and
and colleagues (Axt-Adam 1993) and Balas and colleagues (Balas
feedback + outreach (McConnell 1982). Neither of these suggest
1996), these results suggest that audit and feedback can be a use-
that audit and feedback alone or in combination with educational
ful intervention, although the effects are generally small, with or
meetings is likely to have large effects. In the study by Mayer and
without peer-comparison.
colleagues, pharmacists, who provided very little, if any advice on
It was difficult to judge the complexity of the targeted behaviours skin cancer prevention prior to the intervention, were given an
in the included studies. We considered most (n=68) of the targeted intervention that included prompts, incentives and a video. In the
behaviours to be of moderate complexity. Consequently this did study by McConnell and colleagues, physicians in ambulatory care
not help to explain the observed variation in the effectiveness of who prescribed tetracycline inappropriately for upper respiratory
audit and feedback. infections received outreach visits.
A related concept that we were not able to assess is the motivation We found only five randomised comparisons of audit and feed-
of health professionals to change the targeted behaviour. The trial back with other interventions. The results of the two comparisons
by Palmer (Palmer 1985) was the only one where the investiga- of audit and feedback with reminders (Boekeloo 1990; Tierney
tors assessed the motivation of the providers to change practice. 1986) are consistent with the conclusions of Buntix and colleagues
They did this by asking providers to indicate the ’likelihood that (Buntinx 1993), that both can be effective, and do not provide
serious consequences for the patients’ would occur if performance strong support for either being clearly superior, although the audit
was poor. Contrary to what was expected, the results suggested and feedback group performed better than the reminder group
that more improvement occurred for tasks associated with mod- in both of these studies. To the extent that these results can be
erate to low motivation. The investigators attributed the lack of considered reliable, they would bring into question Mugford and
improvement in the high motivation tasks to problems with ad- colleagues conclusions that feedback close to the time of decision-
ministrative systems associated with these tasks. Another possible making is more likely to be more effective (Mugford 1991), since
explanation is that audit and feedback has marginal benefits for reminders by definition occur at the time of decision-making.
high motivation tasks because feedback is less needed or super-
Few trials reported the cost of the interventions. Small effects may
fluous if the provider is already motivated. This is similar to the
be worthwhile, if the costs of the intervention are small relative
findings of Sibley and colleagues who studied the effect of contin-
to the benefits gained. Intuitively this is more likely to be the
uing medical education packages (Sibley 1982), and also consis-
case when an audit can easily be conducted using computerised
tent with the findings of Foy et al (Foy 2002). They reported that
records, but the studies included in this review do not provide
quality of care improved only when topics were of low interest to
empirical data to support or refute this. Moreover, the usefulness
the providers. Theories of behaviour change suggest that motiva-
of computerised records for audit is dependent on the quality of
tion is an important component of the change process (Bandura
routinely collected data.
1986;Fox 1989;Green 1988;Prochaska 1992). It is possible that
differences in motivation could explain some of the observed vari-
ation in the effectiveness of audit and feedback across the included
studies, but we were unable to assess this. AUTHORS’ CONCLUSIONS
The results of this review do not support or refute the conclusions Implications for practice
of Mugford and colleagues (Mugford 1991) that feedback close
to the time of decision-making and prior agreement of clinicians Audit and feedback can be effective in improving professional
to review their practice are important factors in determining the practice. The effects are generally small to moderate. The absolute
effectiveness of audit and feedback. Nor do they support the con- effects of audit and feedback are more likely to be larger when
clusions of Axt-Adams and colleagues that the variation, extent, baseline adherence to recommended practice is low.
timing, frequency and availability of peer-comparisons explain the
The results of this review do not support the conclusions of pre-
observed variation in the effectiveness of audit and feedback (Axt-
vious reviews that multifaceted interventions are more likely to
Adam 1993). Eight trials with 11 comparisons included a follow-
be effective than single interventions (Grimshaw 2001). This is
up period after audit and feedback stopped. The length of follow-
consistent with the results of another recent review that has inves-
up, targeted behaviours, and the effect on performance varied in
tigated this relationship (Grimshaw 2002).
these trials. It is possible for performance to deteriorate, stay the
same, or improve after feedback stops. This may depend largely The evidence presented here does not support mandatory or un-
on the nature of the targeted behaviour, but there are insufficient evaluated use of audit and feedback. Audit and feedback should
be targeted where it is likely to effect change, particularly when POTENTIAL CONFLICT OF
adherence to recommended practice is low, and its effects should INTEREST
be evaluated. Decisions about whether audit and feedback is likely
to be effective for a particular problem and the design of inter- None known.
ventions using audit and feedback must rely on judgement and
practical considerations.
Implications for research ACKNOWLEDGEMENTS
The effects of audit and feedback are commonly small to mod-
erate, but may frequently be worthwhile. Studies should be large We would like to thank Dave Davis, Brian Haynes, Nick Free-
enough to detect small effects when these are considered impor- mantle and Emma Harvey for their contributions to an earlier ver-
tant. Sample size calculations should take account of clustering sion of this review, and Julian Higgins for statistical advise. We are
and appropriate analyses should be used to avoid unit of analy- grateful to Cynthia Fraser for conducting searches for this update
sis errors. Before and after measurements can be useful for mon- and for doing such a good job developing the EPOC specialised
itoring, to ensure that desired changes have occurred in practice, register that additional searches were found to be redundant.
but it is difficult to attribute causation based on before-after stud-
ies. They should not be used to evaluate the effects of audit and
feedback since they are likely to be misleading. Baseline measure-
ments should be undertaken both to determine the importance SOURCES OF SUPPORT
of intervening and to adjust for baseline differences when these
are found in randomised trials. Better reporting of study methods, External sources of support
targeted behaviours, characteristics of participants and interven-
tions is needed. Primary outcomes should be clearly specified and • No sources of support supplied
they should be clinically important.
Internal sources of support
In light of the results of a recent review (Grimshaw 2002), it is
possible that educational materials might have small effects on • Norwegian Directorate for Health and Social Welfare NOR-
professional practice. In future updates of this review we will ex- WAY
plore the extent to which audit and feedback combined with ed- • Surgical Outcomes Research Centre, Central Sydney Area
ucational materials might be more effective than audit and feed- Health Service AUSTRALIA
back alone, as well as exploring whether specific differences in how • Needs Assessment & Health Outcome Unit, Central sydney
audit and feedback is done and individual components of study Area Health Service AUSTRALIA
quality might help to explain differences in observed effects. • Hamilton Regional Cancer Centre CANADA
REFERENCES
References to studies included in this review Balas 1998 {published data only}
∗
Balas E, Boren SA, Hicks LL, Chonko AM, Stephenson K. Effect of
Anderson 1994 {published data only}
linking practice data to published evidence: A randomized controlled
Anderson FA Jr, Wheeler HB, Goldberg RJ, Hosmer DW, Forcier
trial of clinical direct reports. Med Care 1998;36:79–87.
A, Patwardhan NA. Changing clinical practice. Prospective study of
the impact of continuing medical education and quality assurance
Belcher 1990 {published data only}
programs on use of prophylaxis for venous thromboembolism. Arch ∗
Belcher DV. Implementing preventive services success and failure
Intern Med 1994;154:669–77.
in an outpatient trial. Arch Intern Med 1990;150:2533–2541.
Anderson 1996 {published data only}
Anderson JF, McEwan KL, Hrudey WP. Effectiveness of notification Berman 1998 {published data only}
and group education in modifying prescribing of regulated analgesics.
∗
Berman MF, Simon AE. The effect of a drug and supply cost feed-
CMAJ 1996;154:31–9. back system on the use of intraoperative resources by anesthesiolo-
gists. Anesth Analg 1998;86:510–515.
Baker 1997 {published data only}
∗
Baker R, Farooqui A, Tait C, Walsh S. Randomised controlled trial Boekeloo 1990 {published data only}
of reminders to enhance the impact of audit in general practice on Boekeloo BO, Becker DM, Levine DM, Belitsos PC, Pearson TA.
management of patients who use benzodiazepines. Quality in Health Strategies for increasing house staff management of cholesterol with
Care 1997;6:14–18. inpatients. Am J Prev Med 1990;6(suppl 2):51–9.
Bonevski 1999 {published data only} De Almeida Neto 2000 {published data only}
∗
Bonevski B, Sanson-Fisher RW, Campbell E, Carruthers A, Reid ∗
Neto ACDA, Benrimoj SI, Kavanagh DJ, Boakes RA. A pharmacy
ALA, Ireland M. Randomized controlled trial of a computer strategy based protocol and training program for non-prescription analgesics.
to increase general practitioner preventive care. Preventive Medicine Journal of Social and Administrative Pharmacy 2000;17(3):183–192.
1999;29:478– 486. Dickinson 1981 {published data only}
Borgiel 1999 {published data only} Dickinson JC, Warshaw GA, Gehlbach SH, Bobula JA, Muhlbaier
∗
Borgiel AEM, Williams JI, Davis DA, Dunn EV, Hobbs N, Hutchi- LH, Parkerson GR Jr. Improving hypertension control: impact of
son B, Wilson CR, Jensen J, ONeil JJS, Bass MJ. Evaluating the ef- computer feedback and physician education. Med Care 1981;19:
fectiveness of 2 educational interventions on family practice. Cana- 843–54.
dian Medical Association 1999;8:965–970. Everett 1983 {published data only}
Brady 1988 {published data only} Everett GD, deBlois CS, Chang PF, Holets T. Effect of cost education,
Brady WJ, Hissa DC, McConnell M, Wones RG. Should physicians cost audits, and faculty chart review on the use of laboratory services.
perform their own quality assurance audits?. J Gen Intern Med 1988; Arch Intern Med 1983;143:942–4.
3:560–5. Fairbrother 1999 {published data only}
Brown 1994 {published data only} ∗
Fairbrother G, Hanson KL, Friedman S, Butts GC. The impact
Brown LF, Keily PA, Spencer AJ. Evaluation of a continuing edu- of physician bonuses, enhanced fees, and feedback on childhood
cation intervention “Periodontics in General Practice”. Community immunization coverage rates. American Journal of Public Health 1999;
Dent Oral Epidemiol 1994;22:441–7. 89(2):171–175.
Buffington 1991 {published data only} Fairbrother 1999a {published data only}
Buffington J, Bell KM, LaForce FM. A target-based model for in- ∗
Fairbrother G, Hanson KL, Friedman S, Butts GC. The impact
creasing influenza immunizations in private practice. J Gen Intern of physician bonuses, enhanced fees, and feedback on childhood
Med 1991;6:204–9. immunization coverage rates. American Journal of Public Health 1999;
Buntinx 1993 {published data only} 89(2):171–175.
∗
Buntinx F, Knottnerus JA, Crebolder HF, Seegers T, Essed GG, Fairbrother 1999b {published data only}
Schouten H. Does feedback improve the quality of cervical smears? Fairbrother G, Hanson KL, Friedman S, Butts GC. The impact of
A randomized controlled trial. Br J Gen Pract 1993;43:194–8. physician bonuses, enhanced fees, and feedback on childhood im-
munization coverage rates. American Journal of Public Health 1999;
Buntinx F, Knottnerus JA, Crebolder HFJM, Esses GGM. Reactions
89(2):171–175.
of doctors to various forms of feedback designed to improve the
sampling quality of cervical smears. Quality Assurance in Health Care Feder 1995 {published data only}
1992;4(2):161–166.
∗
Feder G, Griffiths C, Highton C, Eldridge S, Spence M, South-
gate L. Do clinical guidelines intorduced with practice based educa-
Buntinx 1993a {published data only}
tion improve care of asthmatic and dibetic patients? A randomised
∗
Buntinx F, Knottnerus JA, Crebolder HF, Seegers T, Essed GG,
controlled trial in general practices in east London. BMJ 1995;311:
Schouten H. Does feedback improve the quality of cervical smears?
1473–8.
A randomized controlled trial. Br J Gen Pract 1993;43:194–8.
Gama 1991 {published data only}
Buntinx F, Knottnerus JA, Crebolder HFJM, Esses GGM. Reactions ∗
Gama R, Nightingale PG, Broughton PMG, Peters M, Bradby
of doctors to various forms of feedback designed to improve the GVH, Berg J, Ratcliffe JG. Feedback of laboratory usage and cost
sampling quality of cervical smears. Quality Assurance in Health Care data to clinicians: does it alter requesting behavior?. Ann Clin Biochem
1992;4(2):161–166. 1991;28:143–149.
Buntinx 1993b {published data only} Gehlbach 1984 {published data only}
Buntinx F, Knottnerus JA, Crebolder HF, Seegers T, Essed GG, Gehlbach SH, Wilkinson WE, Hammond WE, Clapp NE, Finn
Schouten H. Does feedback improve the quality of cervical smears? AL, Taylor WJ, et al. Improving drug prescribing in a primary care
A randomized controlled trial. Br J Gen Pract 1993;43:194–8. practice. Med Care 1984;22:193–201.
Buntinx F, Knottnerus JA, Crebolder HFJM, Esses GGM. Reactions Goldberg 1998 {published data only}
of doctors to various forms of feedback designed to improve the
∗
Goldberg HI, Wagner EH, Fihn SD, Martin DP, Horowitz CR,
sampling quality of cervical smears. Quality Assurance in Health Care Christensen DB, Cheadle AD, Diehr P, Simon G. A randomized
1992;4(2):161–166. controlled trial of QI teams and academic detailing: can they alter
compliance with guidelines?. Journal on Quality Improvement 1998;
Chassin 1986 {published data only}
24(3):130–142.
Chassin MR, McCue SM. A randomized trial of medical quality
assurance. Improving physicians’ use of pelvimetry. JAMA 1986;256: Grady 1997 {published data only}
1012–6.
∗
Grady KE, Lemkau JP, Lee NR, Caddell C. Enhancing mammogra-
Cohen 1982 {published data only} phy referral in primary care. Preventive Medicine 1997;26:791–800.
Cohen DI, Jones P, Littenberg B, Neuhauser D. Does cost informa- Gullion 1988 {published data only}
tion availability reduce physician test usage? A randomized clinical Gullion DS, Tschann JM, Adamson TE, Coates TJ. Management
trial with unexpected findings. Med Care 1982;20:286–92. of hypertension in private practice: a randomized controlled trial in
continuing medical education. The Journal of Continuing Education Jones 1996 {published data only}
in the Health Professions 1988;8:239–55. ∗
Jones HE, Cleave B, Zinman B, Szalai JP, Nichol HL, Hoffman
BR. Efficacy of feedback from quarterly laboratory comparison in
Gullion 1988a {published data only}
maintaining quality of a hospital capillary blood glucose monitoring
Gullion DS, Tschann JM, Adamson TE, Coates TJ. Management
program. Diabetes Care 1996;19(2):168–170.
of hypertension in private practice: a randomized controlled trial in
continuing medical education. The Journal of Continuing Education Kafuko 1999 {published data only}
in the Health Professions 1988;8:239–55. Kafuko JM, Zirabamuzaale, Bagena D. Rational drug use in rural
health units of Uganda:effect of national standard treatment guide-
Gullion 1988b {published data only}
lines on rational drug use. 1st International Conference on Improv-
Gullion DS, Tschann JM, Adamson TE, Coates TJ. Management ing Use og Medications. 1999.
of hypertension in private practice: a randomized controlled trial in
continuing medical education. The Journal of Continuing Education Kerry 2000 {published data only}
in the Health Professions 1988;8:239–55.
∗
Kerry S, Oakeshott P, Dundas D, Williams J. Influence of postal
distribution of the royal college of radiologists guidelines, together
Gullion 1988c {published data only} with feedback on radiological referral rates, on x-ray referrals from
Gullion DS, Tschann JM, Adamson TE, Coates TJ. Management general practice: a randomized controlled trial. Family Practice 2000;
of hypertension in private practice: a randomized controlled trial in 17(1):46–52.
continuing medical education. The Journal of Continuing Education
Kerse 1999 {published data only}
in the Health Professions 1988;8:239–55. ∗
Kerse NM, Flicker L, Jolley D, Arroll B, Young D. Improving the
Hemminiki 1992 {published data only} health behaviours of elderly people: randomised controlled trial of a
∗
Hemminiki E, Teperi J, Tuominen K. Need for and influence or general practice education programme. BMJ 1999;319:683–687.
feedback from the Finnish birth register to data providers. Quality Kim 1999 {published data only}
Assurance in Health Care 1992;4(2):133–139. ∗
Kim CS, Kristopaitis RJ, Stone E, Pelter M, Sandhu M, Weingarten
Henderson 1979 {published data only} SR. Physician education and report cards: Do they make the grade?
∗
Henderson D, D´ Alessandri R, Westfall B, Moore R, Smith R, Results from a randomized controlled trial. The American Journal of
Scobbo, Waldman R. Hospital cost containment: a little knowledge Medicine 1999;107:556–560.
helps. Clinical Research 1979;27:297A. Kinsinger 1998 {published data only}
∗
Kinsinger LS, Harris R, Qaqish B, Strecher V, Kaluzny A. Using an
Hendryx 1998 {published data only}
office system intervention to increase breast cancer screening. JGIM
∗
Hendryx MS, Fieselmann JF, Bock MJ, Wakefield DS, Helms CM,
1998;13:507–514.
Bentler SE. Outreach education to improve quality of rural icu care.
Am J Respir Crit Care Med 1998;158:418–423. Leviton 1999 {published data only}
∗
Leviton LC, Goldenberg RL, Baker CS, Schwartz RM, Freda MC,
Hershey 1986 {published data only} Fish LJ, Cliver SP, Rouse DJ, Chazotte C, Merkatz IR, Raczynski JM.
Hershey CO, Porter DK, Breslau D, Cohen DI. Influence of simple Methods to encourage the use of antenatal corticosteroid therapy for
computerized feedback on prescription charges in an ambulatory fetal maturation. JAMA 1999;281(1):46–52.
clinic. A randomized clinical trial. Med Care 1986;24:472–81.
Linn BS 1980 {published data only}
Hillman 1998 {published data only} Linn BS. Continuing medical education. Impact on emergency room
∗
Hillman AL, Ripley K, Goldfarb N, Nuamah I, Weiner J, Lusk E. burn care. JAMA 1980;244:565–70.
Physician financial incentives and feedback: Failure to increase can-
Linn BS, 1980 {published data only}
cer screening in medicaid managed care. American Journal of Public
Linn BS. Continuing medical education. Impact on emergency room
Health 1998;88(11):1698–1701.
burn care. JAMA 1980;244:565–70.
Hillman 1999 {published data only} Lobach 1996 {published data only}
∗
Hillman AL, Ripley K, Goldfarb N, Weiner J, Nuamah I, Lusk E. ∗
Lobach DF. Electronically distributed computer-generated feed-
The use of physician financial incentives and feedback to improve back enhances the use of acomputarized practice guidelines. Proceed-
pediatric preventive care in Medicaid care. Pediatrics 1999;104(4): ings/AMIA Annual Fall symposium. 1996:493–497.
931–935.
Lomas 1991 {published data only}
Holm 1990 {published data only} Lomas J. Making clinical policy explicit. International Journal of Tech-
∗
Holm M. Intervention against long-term use if hypnotics/sedatives nology Assessment in Health Care 1993;9:1:11–25.
in general practice. Scand J Prim Health Care 1990;8:113–117.
∗
Lomas J, Enkin M, Anderson GM, Hannah WJ, Vayda E, Singer
Howe 1996 {published data only} J. Opinion leaders vs audit and feedback to implement practice
Howe A. Detecting psychological distress: can general practitioners guidelines. Delivery after previous cesarean section. JAMA 1991;265:
improve their performance?. Br J Gen Pract 1996;46:407–10. 2202–7.
Hux 1999 {published data only} Mainous 2000 {published data only}
∗
Hux JE, Melady MP, DeBoer D. Confidential prescriber feedback ∗
Mainous AG, Hueston WJ, Love MM, Evans ME, Finger R. An
and education to improve antibiotic use in primary care: a controlled evaluation of statewide strategies to reduce antibiotic overuse. Family
trial. Canadian Medical Association 1999;161:388–392. Medicine 2000;32(1):22–29.
Manfredi 1998 {published data only} Norton 1985 {published data only}
∗
Manfredi C, Czaja R, Freels S, Trubitt M, Warnecke R, Lacey L. Im- Norton PG, Dempsey LJ. Self-audit: its effect on quality of care. J
proving cancer screening in physcians practices serving low-income Fam Pract 1985;21:289–91.
and minority populations. Arch Fam Med 1998;7:329–337. Norton 1985a {published data only}
Manheim 1990 {published data only} Norton PG, Dempsey LJ. Self-audit: its effect on quality of care. J
Manheim LM, Feinglass J, Hughes R, Martin GJ, Conrad K, Hughes Fam Pract 1985;21:289–91.
EF. Training house officers to be cost conscious. Effects of an edu- Norton 1985b {published data only}
cational intervention on charges and length of stay. Med Care 1990; Norton PG, Dempsey LJ. Self-audit: its effect on quality of care. J
28:29–42. Fam Pract 1985;21:289–91.
Martin 1980 {published data only}
O’Connell 1999 {published data only}
Martin AR, Wolf MA, Thibodeau LA, Dzau V, Braunwald E. A ∗
O´ Connell DL, Henry D, Tomlins R. Randomised controlled trial
trial of two strategies to modify the test-ordering behavior of medical
of effect of feedback on general practitioners prescribing in Australia.
residents. N Engl J Med 1980;303:1330–6.
BMJ 1999;318:507–511.
Marton 1985 {published data only}
Palmer 1985 {published data only}
Marton KI, Tul V, Sox HC Jr. Modifying test-ordering behavior in
Palmer RH, Louis TA, Hsu LN, Peterson HF, Rothrock JK, Strain
the outpatient medical clinic. A controlled trial of two educational
R, et al. A randomized controlled trial of quality assurance in sixteen
interventions. Arch Intern Med 1985;145:816–21.
ambulatory care practices. Med Care 1985;23:751–70.
Marton 1985a {published data only}
Marton KI, Tul V, Sox HC Jr. Modifying test-ordering behavior in Raasch 2000 {published data only}
the outpatient medical clinic. A controlled trial of two educational
∗
Raasch BA, Hays R, Buettner PG. An educational intervention to
interventions. Arch Intern Med 1985;145:816–21. improve diagnosis and management of suspicious skin lesions. The
Journal of Continuing Education in the Health Professions 2000;20:
Marton 1985b {published data only} 39–51.
Marton KI, Tul V, Sox HC Jr. Modifying test-ordering behavior in
the outpatient medical clinic. A controlled trial of two educational Reid 1977 {published data only}
interventions. Arch Intern Med 1985;145:816–21.
∗
Reid RA, Lantz KH. Physician profiles in training the graduate
internist. Journal of Medical Education 1977;52:300–305.
Mayefsky 1993 {published data only}
Mayefsky JH, Foye HR. Use of a chart audit: teaching well child care Roski 1998 {published data only}
to paediatric house officers. Med Educ 1993;27:170–4.
∗
Roski J. Changing practice patterns as a result of implementing the
Agency for Health Care Policy and Research guidelines in 20 primary
Mayer 1998 {published data only}
care clinics. Tob Control 1998;Suppl:S19-20:S25–5.
∗
Mayer JA, Eckhardt L, Stepanski BM, Sallis JF, Elder JP, Slymen
DJ, Creech L, Graf G, Palmer RC, Rosenberg C, Souvignier ST. Ruangkanchanastr 19 {published data only}
Promoting skin cancer prevention counseling. American Journal for ∗
Ruangkanchanastr S. Laboratory investigation utilization in pedi-
Public Health 1998;88(7):1096–1099. atric out-patient department ramathibodi hospital. J Med Assoc Thai
McAlister 1986 {published data only} 1993;76:194–199.
McAlister NH, Covvey HD, Tong C, Lee A, Wigle ED. Randomised Ruangkanchanastr a {published data only}
controlled trial of computer assisted management of hypertension in ∗
Ruangkanchanastr S. Laboratory investigation utilization in pedi-
primary care. BMJ 1986;293:670–4. atric out-patient department ramathibodi hospital. J Med Assoc Thai
1993;76:194–199.
McCartney 1997 {published data only}
∗
McCartney P, Macdowall W, Thorogood M. A randomised con- Ruangkanchanastr b {published data only}
trolled trial of feedback to general, practitioners of their prophylactic Ruangkanchanastr S. Laboratory investigation utilization in pediatric
aspirin prescribing. BMJ 1997;315:35–36. out-patient department ramathibodi hospital. J Med Assoc Thai 1993;
76:194–199.
McConnell 1982 {published data only}
McConnell TS, Cushing AH, Bankhurst AD, Healy JL, McIlvenna Ruangkanchanastr c {published data only}
PA, Skipper BJ. Physician behavior modification using claims data: Ruangkanchanastr S. Laboratory investigation utilization in pediatric
tetracycline for upper respiratory infection. The Western Journal of out-patient department ramathibodi hospital. J Med Assoc Thai 1993;
Medicine 1982;137(5):448–450. 76:194–199.
Meyer 1991 {published data only} Rust 1999 {published data only}
Meyer TJ, Van Kooten D, Marsh S, Prochazka AV. Reduction of
∗
Rust CT, Sisk FA, Kuo AR, Smith J, Miller R, Sullivan KM. Im-
polypharmacy by feedback to clinicians. J Gen Intern Med 1991;6: pact of resident feedback on immunization outcomes. ARCH Pediatr
133–6. Adolesc 1999;153:1165–1169.
Moongtui 2000 {published data only} Sanazaro 1978 {published data only}
∗
Moongtui W, Gauthier DK, Turner JG. Using peer feedback to im- Sanazaro PJ, Worth RM. Concurrent quality assurance in hospital
prove handwashing and glove usage among Thai health care workers. care. Report of a study by Private Initiative in PSRO. N Engl J Med
Am J Infect Control 2000;28:365–369. 1978;298:1171–7.
Sandbaek 1999 {published data only} Veninga 1999 {published data only}
∗
Sandbaek A, Kragstrup J. Randomized controlled trial of the effect Lagerløv P, Loeb M, Andrew M, Hjortdal P. Improving doctors pre-
of medical audit on aids prevention in general practice. Family Practice scribing behaviour through reflection on guidelines and prescribing
1999;16:510–514. feedback: a randomised controlled trial. Quality in Health Care 2000;
Schectman 1995 {published data only} 9:159–165.
Schectman JM, Kanwal NK, Schroth WS, Elinsky EG. The effect Lundborg CS, Wahlström, Oke T, Tomson G, Diwan V. Influencing
of an education and feedback intervention on group-model and net- prescribing for urinary tract infection and asthma in ed controlled
work-model health maintenance organization physician prescribing trial of an interactive educational intervention care in sweden: a ran-
behavior. Med Care 1995;33:139–44. domized controlled trial of an interactive eduactional intervention. J
Simon 2000 {published data only} Clin Epidemiology 1999;52(8):801–812.
∗
Simon GE, VonKorff M, Rutter C, Wagner E. Randomised trial Veninga CCM, Denig P, Zwaagstra R, Haaijer-Ruskamp FM. Im-
in monitoring, feedback, and management of care by telephone to proving drug treatment in general practice. Journal of Clinical Epi-
improve treatment of depression in primary care. BMJ 2000;320: demiology 2000;53:762–772.
550–554.
∗
Veninga CCM, Lagerløv P, Wahlstöm R, Muskova M, Denig P,
Sinclair 1982 {published data only}
Berkhof J, Kochen MM, Haaijer-Ruskamp FM, the Drug Education
∗
Sinclair C, Frankel M. The effect of quality assurance activities on
Project Group. Evaluating an educational intervention to improve
the quality of mental health services. QRB 1982;8(7):7–15.
the treatment of asthma in four European countries. Am J Respir Crit
Smith 1995 {published data only} Care Med 1999;160:1254–1262.
∗
Smith D, Christensen DB, Stergachis A, Holmes G. A randomized
controlled trial of a drug use review intervention for sedative hypnotic Veninga N. Improving prescribing in general practice. Rijksuiversiteit
medications. Prenatal Diagnosis 1998;15:1013–1021. Groningen, 2000.
Veninga 1999a {published data only}
Smith 1998 {published data only}
Lagerløv P, Loeb M, Andrew M, Hjortdal P. Improving doctors pre-
∗
Smith DK, Shaw RW, Slack J, Marteau TM. Training obstetricians
scribing behaviour through reflection on guidelines nd prescribing
and midwives to present screening tests evaluation of two brief inter-
feedback: a randomised controlled trial. Quality in Health Care 2000;
ventions. Prenatal Diagnosis 1995;15:317–324.
9:159–165.
Socolar 1998 {published data only}
∗
Socolar RRS, Raines B, Chen-Mok M, Runyan DK, Green C, Lundborg CS, Wahlström, Oke T, Tomson G, Diwan V. Influencing
Paterno S. Intervention to improve physician documentation and prescribing for urinary tract infection and asthma in ed controlled
knowledge of child sexual abuse: A randomized, controlled trial. Pe- trial of an interactive education intervention care in sweden: a ran-
diatrics 1998;101(5):817–824. domized controlled trial of an interactive eduactional intervention. J
Clin Epidemiology 1999;52(8):801–812.
Sommers 1984 {published data only}
Sommers LS, Sholtz R, Shepherd RM, Starkweather DB. Physician Veninga CCM, Denig P, Zwaagstra R, Haaijer-Ruskamp FM. Im-
involvement in quality assurance. Med Care 1984;22:1115–38. proving drug treatmet in general practice. Journal of Clinical Epi-
demiology 2000;53:762–772.
Thompson 2000 {published data only}
∗
Thompson RS, Rivara FP, Thompson DC, Barlow WE, Sugg NK,
∗
Veninga CCM, Lagerløv P, Wahlstöm R, Muskova M, Denig P,
Maiuro RD, Rubanowice DM. Identification and management of Berkhof J, Kochen MM, Haaijer-Ruskamp FM, the Drug Education
domestic violence a randomized trial. AM J Prev Med 2000;19(4): Project Group. Evaluating an Educational Intervention to Improve
253–263. the Treatment og Asthma in Four European Countries. Am j Respir
Tierney 1986 {published data only} Crit Care Med 1999;160:1254–1262.
Tierney WM, Hui SL, McDonald CJ. Delayed feedback of physician Veninga N. Improving prescribing in general practice. Thesis, Rijk-
performance versus immediate reminders to perform preventive care. suiversiteit Groningen 2000.
Effects on physician compliance. Med Care 1986;24(8):659–66. Veninga 1999b {published data only}
van den Hombergh 99 {published data only} Lagerløv P, Loeb M, Andrew M, Hjortdal P. Improving doctors pre-
∗
Hombergh Pvd, Grol R, Hoogen HJMvd, Bosch WJHMvd. Prac- scribing behaviour through reflection on guidelines nd prescribing
tice visits as a tool in quality improvement: mutual visits and feed- feedback: a randomised controlled trial. Quality in Health Care 2000;
back by peers compared with visits and feedback by non-physician 9:159–165.
observers. Quality in Health Care 1999;8:161–166.
Lundborg CS, Wahlström, Oke T, Tomson G, Diwan V. Influencing
van den Hombergh. Practice visits. Assessing and improving manage- prescribing for urinary tract infection and asthma in ed controlled
ment in general practice. University of Nijmegen, 1998. trial of an interactive education intervention care in sweden: a ran-
domized controlled trial of an interactive eduactional intervention. J
van der Weijden 1999 {published data only}
van der Weijden T, Grol RP, Knottinerus JA. Feasibility of a national
cholestrol guideline in daily practice. A randomized controlled trial Veninga CCM, Denig P, Zwaagstra R, Haaijer-Ruskamp FM. Im-
in 20 general practices. International Journal for Quality in Health proving drug treatmet in general practice. Journal of Clinical Epi-
Care 1999;11(2):131–137. demiology 2000;53:762–772.
Veninga CCM, Lagerløv P, Wahlstöm R, Muskova M, Denig P, Ward 1996 {published data only}
Berkhof J, Kochen MM, Haaijer-Ruskamp FM, the Drug Education ∗
Ward A, Kamien M, Mansfield F, Fatovich B. Educational feedback
Project Group. Evaluating an Educational Intervention to Improve in management of diabetes in general practice. Education for General
the Treatment og Asthma in Four European Countries. Am j Respir Practice 1996;7:142–150.
Crit Care Med 1999;160:1254–1262.
Wells 2000 {published data only}
Veninga N. Improving prescribing in general practice. Thesis, Rijk-
∗
Wells KB, Sherbourne C, Schoenbaum M, Duan N, Meredith L,
suiversiteit Groningen 2000. Unutzer J, Miranda J, Carney MF, Rubenstein LV. Impact of dissem-
inating quality improvement programs for depression in managed
Veninga 1999c {published data only}
primary care. JAMA 2000;283(2):212–220.
Lagerløv P, Loeb M, Andrew M, Hjortdal P. Improving doctors pre-
scribing behaviour through reflection on guidelines nd prescribing Winickoff 1984 {published data only}
feedback: a randomised controlled trial. Quality in Health Care 2000; Winickoff RN, Coltin KL, Morgan MM, Buxbaum RC, Barnett
9:159–165. GO. Improving physician performance through peer comparison
feedback. Med Care 1984;22:527–34.
prescribing for urinary tract infection and asthma in ed controlled Winickoff 1985 {published data only}
trial of an interactive education intervention care in sweden: a ran- Winickoff RN, Wilner S, Neisuler R, Barnett GO. Limitations of
domized controlled trial of an interactive eduactional intervention. J provider interventions in hypertension quality assurance. Am J Public
Clin Epidemiology 1999;52(8):801–812. Health 1985;75:43–6.
Veninga CCM, Denig P, Zwaagstra R, Haaijer-Ruskamp FM. Im- Winkens 1995 {published data only}
proving drug treatmet in general practice. Journal of Clinical Epi- Winkens RA, Pop P, Bugter-Maessen AM, Grol RP, Kester AD,
demiology 2000;53:762–772. Beusmands GH, et al. Randomised controlled trial of routine indi-
vidual feedback to improve rationality and reduce numbers of test
Veninga CCM, Lagerløv P, Wahlstöm R, Muskova M, Denig P, requests. Lancet 1995;345:498–502.
Berkhof J, Kochen MM, Haaijer-Ruskamp FM, the Drug Education
Project Group. Evaluating an Educational Intervention to Improve Wones 1987 {published data only}
the Treatment og Asthma in Four European Countries. Am j Respir Wones RG. Failure of low-cost audits with feedback to reduce labo-
Crit Care Med 1999;160:1254–1262. ratory test utilization. Med Care 1987;25:78–82.
Veninga N. Improving prescribing in general practice. Thesis, Rijk- Wones 1987a {published data only}
suiversiteit Groningen 2000. Wones RG. Failure of low-cost audits with feedback to reduce labo-
ratory test utilization. Med Care 1987;25:78–82.
Veninga 1999d {published data only}
Lagerløv P, Loeb M, Andrew M, Hjortdal P. Improving doctors pre- Wones 1987b {published data only}
scribing behaviour through reflection on guidelines nd prescribing Wones RG. Failure of low-cost audits with feedback to reduce labo-
feedback: a randomised controlled trial. Quality in Health Care 2000; ratory test utilization. Med Care 1987;25:78–82.
9:159–165. Zwar 1999 {published data only}
∗
Zwar N, Wolk J, Gordon J, Fisher RS, Kehoe L. Influencing antibi-
prescribing for urinary tract infection and asthma in ed controlled otic prescribing in general practice: a trial of prescriber feedback and
trial of an interactive education intervention care in sweden: a ran- management guidelines. Family Practice 1999;16(5):495–500.
domized controlled trial of an interactive eduactional intervention. J
References to studies excluded from this review
Anonymous I 1990
Veninga CCM, Denig P, Zwaagstra R, Haaijer-Ruskamp FM. Im- Anonymous. North of Englad study of standards and performence
proving drug treatmet in general practice. Journal of Clinical Epi- in general practice. University of Newcastle, report nr 40 and report
demiology 2000;53:762–772. nr 50.
Veninga CCM, Lagerløv P, Wahlstöm R, Muskova M, Denig P, Berwick 1986
Berkhof J, Kochen MM, Haaijer-Ruskamp FM, the Drug Education Berwick DM, Coltin KL. Feedback reduces test use in a health
Project Group. Evaluating an Educational Intervention to Improve maintenance organization. JAMA 1986;255(11):1450–4. [MedLine:
the Treatment og Asthma in Four European Countries. Am j Respir 86144192].
Crit Care Med 1999;160:1254–1262.
Billi 1987
Veninga N. Improving prescribing in general practice. Thesis, Rijk- Billi JE, Hejna GF, Wolf FM, Shapiri LR, Stross JK. The effects of a
suiversiteit Groningen 2000. cost-education program on hospital charges. J Gen Intern Med 1987;
Vinicor 1987 {published data only} 2:306–311.
Vinicor F, Cohen SJ, Mazzuca SA, Moorman N, Wheeler M, Kuebler Brown 1988
T, et al. DIABEDS: a randomized trial of the effects of physician Brown RL. Evaluation of a continuing medical education program for
and/or patient education on diabetes patient outcomes. J Chronic Dis primary care physcicians on the management of alcoholism. Journal
1987;40:345–56. of Medical Education 1988;63:482–484.
Buekens 1993 Hanlon 1996
Buekens P, Boutsen M, Kittel F, Vandenbussche P, Dramaix M. Does Hanlon JT, Weinberger M, Samsa GP, Kenneth E, Uttech KM, Lewis
awareness of rates of obstetric interventions change practice. BMJ IK, Cowper P, Landsman PB, Cohen JH, Feusser JR. A random-
1993;306:623. ized controlled trial of a clinical pharmacists intervention to improve
Carney 1992 inappropriate prescribing in elderly outpatients with polypharmacy.
Carney PA, Dietrich AJ, Keller A, Landgraf J, OConnor GT. Tools, Am J Med 1996;100(4):428–437.
teamwork and tenacity: An office system for cancer prevention. J Fam Hargraves 1996
Pract 1992;35:388–394. Hargraves JL, Palmer RH, Orav EJ, Wright EA. Practice characteris-
De Silva 1994 tics and performance of primary care practitioners. Med Care 1996;
De Silva M, Abrahanson G. Does medical audit change practice?. 34(9):67–76.
Transfucion Science 1994;15:277. Hershey 1988
Del Mar 1998 Hershey CO, Goldberg HI, Cohen DI. The effect of computer-
Del Mar CB, Lowe JB, Adkins P, Arnold E, Baade P. Improving ized feedback coupled with a newsletter upon outpatient prescribing
general practitioner clinical records with a quality assurance minimal charges. A randomized controlled trial. Med Care 1988;26(1):88–94.
intervention. British Journal of General Practice 1998;48:1307–1311. [MedLine: 88093018].
Dranitsaris 1995 Hetlevik 1998
Dranitsaris G, Warr D, Puodziunas A. A randomized trial of the
∗
Hetlevik I, Holmen J, Krüger Ø. Implementing clinical guidelines
effects of pharmacist intervention on the cost of antiemetic herapy in the treatment of hypertension in general practice. Scan J Prim
with ondansetron. Support Care Cancer 1995;3:183–189. Health Care 1999;17:35–40.
Everett Hetlevik I, Holmen J, Krüger Ø, Kristensen P, Iversen H. Imple-

Everett GD, deBlois CS, Chang PF, Holets T. Effect of cost education, menting clinical guidelines in the treatment of hypertension in gen-
cost audits, and faculty chart review on the use of laboratory services. eral practice. Blood Pressure 1998;7:270–276.
Arch Intern Med 1983;143:942–944. Horowitz 1996
Furniss 2000 Horowitz CR, Goldberg HI, Martin DP, Wagner EH, Fihn SD,
Furniss L, burns A, Craig SKL, Scobie S, Cooke J, Farafher B. Effects Christensen DB, Cheadle AD. Conducting a randomized controlled
of a pharmacists medication review in nursing homes. British Journal trial of CQI and academic detailing to implement clinical guidelines.
of Psychiatry 2000;176:563–567. Journal on Quality Improvement 1996;22(11):734–750.
Gask 1991 Johansen 1997
Gask L, Goldberg D, Boardman J, Craig T, Goddard C, Jones O, Johansen A. Using audit to improve senior officer training. Postgrad
et al. Training general practitioners to teach psychiatric interviewing Med J 1997;73:798–801.
skills: an evaluation of group training. Med Educ 1991;25(5):444– Johnson 1976
51. [MedLine: 92099912]. Johnson RE, Campbell WH, Azevedo DJ, Christensen DB. Studying
Gerbert 1988 the impact of patient drug profile in an HMO. Medical Care 1976;
Gerbert B, Maguire B, Badner V, Greenspan D, Greenspan J, Barnes 15:799–807.
D, Carlton R. Changing dentists knowledge, attitudes and be- Kroenke 1990
haviours relating to AIDS: a controlled educational intervention. Kroenke K, Pinholt EM. Reducing polypharmacy in the elderly. J
JADA 1988;116:851–854. Am Geriatr Soc 1990;38:31–36.
Goldberg 1980 Linn 1980
Goldberg DP, Steele JJ, Smith C, Spivey L. Training family doctors Linn LS, Yager J. The effect of screening, sensitization, and feedback
to recognise psychiatric illness with increased accuracy. The Lancet on notation of depression. Journal of Medical Education 1980;55:
1980;6:521–523. 942–949.
Grimshaw 1998 MacCosbe 1985
∗
Grimshaw J. Evaluation of four quality assurance initiatives to im- MacCosbe PE, Gartenberg G. Modifying empiric antibiotic prescrib-
prove out-patient referrals from general practice to hospital. Thesis, ing: Experience with one strategy in a medical residency program.
University of Aberdeen 1998. Hosp Formul 1985;20:986–999.
Guadagnoli 2000 Mandel 1985
Guadagnoli E, Soumerai SB, Gurwitz JH, Borbas C, Shapiro CL, Mandel I, Franks P, Dickinson J. Improving physician compliance
Weeks JC, Morris N. Improving discussion of surgical treatment with preventive medicine guidelines. J Fam Pract 1985;21(3):223–4.
options for patients with breast cancer: local medical opinion leaders [MedLine: 85291598].
versus audit and performance feedback. Breast Cancer Research and Mazzuca 1988
Treatment 2000;61:171–175. Mazzuca SA, Vinicor F, Cohen SJ, Norton JA, Fineberg NS, Fineberg
Hampshire 1999 SE. The Diabetes Education study: a controlled trial of the effects of
Hampshire A, Blair M, Crown N, Avery A, Williams I. Action re- intensive instructions of internal medicine residents on the manage-
search: a useful method of promoting change in primary care?. Fam- ment of diabetes mellitus. Journal of General Internal Medicine 1988;
ily Practice 1999;16(3):305–311. 3:1–8.
McDonel 1997 Rubenstein 1989
McDonel EC, Bond GR, Salyers M, Fekete D, Chen A, McGrew JH, Rubenstein LV, Calkin DR, Young RT, Cleary PD, Fink A, Kosecoff
Miller L. Implementing assertive community treatment programs in J, Jette A, Davies AR, Delbanco TL, Brook RH. Improving patient
rural settings. Administration and Policy in Mental Health 1997;25 function: a randomized trial of functional disability screening. Annals
(2):153–173. of Internal Medicine 1989;111(10):836–842.
McPhee 1989 Rubenstein 1999
∗
Fordham D, McPhee SJ, Bird JA, Rodnick JE, Detmer WM. The Rubenstein LV, Jackson-Tricke M, Unützer J, Miranda J, Minnium
cancer prevention reminder system. Clinical computing 1990;7(5): K, Pearson ML, Wells KB. Evidence-based care for depression in
289–295. managed primary care practices. Health affairs 1999;18(5):89–105.
McPhee SJ, Bird JA, Jenkins CNH, Fordham D. Promoting cancer Shaughnessy 1991
screening. A randomized controlled trial of three interventions. Arch Shaughnessy AF, DAmico F, Nickel RO. Improving prescription-
Intern med 1989;149:1868–1872. writing skills in a family practice residency. DICP, The Annals of
Munroe 1997 Pharmacotherapy 1991;25:17–21.
Munroe WP, Kunz K, Dalmandy-Israel C, Potter L, Schonfeld WH. Spector 1989
Economic evaluation of pharmacist involving in disease management Spector WD, Drugovich ML. Reforming nursing home quality reg-
in a community pharmacy setting. Clinical Therapeutics 1997;19(1): ulation. Impact on cited deficiencies and nursing home outcomes.
113–123. Med Care 1989;27(8):789–801. [MedLine: 89329844].
Nattinger 1989
Steele 1989
Nattinger AB, Panzer RJ, Janus J. Improving the utilization of screen- ∗
Steele MA, Bess DT, Franse VL, Graber SE. Cost effectiveness of
ing mammography in primary care practices. Arch Intern Med 1989;
two interventions for reducing outpatient prescribing costs. DICP,
149(9):2087–92. [MedLine: 89373341].
The Annals of Pharmacotherapy 1989;23:497–500.
North of England1992
North of England Study of Standards, Performance in General Prac- Szczepura 1994
tice. Medical audit in general practice. II: Effects on health of pa- Szczepura A, Wilmot J, Davies C, Fletcher J. Effectiveness and cost
tients with common childhood conditions. BMJ 1992;304(6840): of different strategies for information feedback in general practice.
1484–8. [MedLine: 92305734]. Br J Gen Pract 1994;44(378):19–24. [MedLine: 94145722].
Taylor 1997
North of England Study of Standards, Performance in General Prac-
Taylor EA, Thomas G, Cantrill JA. Changes in prescribing following
tice. Medical audit in general practice. I: Effects on doctors’ clin-
a pharmicists-led audit of ulcer-healing therapy in general practice.
ical behaviour for common childhood conditions. BMJ 1992;304
The pharmaceutical journal 1997;259:R6.
(6840):1480–4. [MedLine: 92305733].
Ogwal-Okeng 2001 The SUPPORT 1995
Ogwal-Okeng JW, Anokbonggo WW, Birungi H. Prescribing audit The SUPPORT Principal Investigators. A controlled trial to improve
with feedback intervention in six regional hospitals and Mulagi re- care for seriously ill hospitalized patients. JAMA 1995;274:1591–
ferral/teaching hospitals. International conference on Improving use 1598.
of Medicines. http:/www.who.int/dap-icium/posters/2C3. 2001. Weingarten 2000
Weingarten SR, Kim CS, Stone EG, Kristopaitis RJ, Pelter M,
Ottolini 1998
Sandhu M. Can peer-comparison feedback improve patient func-
Ottolini MC, Greenberg L. Development and evaluation of a CD-
tional status. Am J Manag Care 2000;6:35–39.
ROM computer program to teach residents telephone management.
Pediatrics 1998;101(3):E2. White 1995
Pearson 2001 White P, Atherton A, Hewett G, Howells K. Using information from
Pearson SD, Kleefield SF, Soukop JR, Cook EF, Lee TH. Critical asthma patients: a trial of information feedback in primary care. BMJ
pathways intervention to reduce length of hospital stay. Am J Med 1995;311:1065–1069.
2001;110:175–180. Wing 1987
Putnam 1985 Wing D, Duff HJ. Impact of a therapeutic drug monitoring program
Putnam RW, Curry L. Impact of patient care appraisal on physicians for digoxin. Arch Intern Med 1987;147:1405–1408.
behaviour in the office setting. CAN MED ASSOC J 1985;132:1025– Wing 1987 (II)
1029. Wing DS, Duff HJ. Evaluation of a therapeutic drug monitoring
Putnam W, Curry L. Patient care appraisal in the ambulatory setting: program frotheophylline in a teaching hospital. Drug Intelligence and
effectiveness as a continuing medical education tool. Annual Confer- Clinical Pharmacy 1987;21:702–705.
ence on Research in Medical Education 1980;19:207–12. Winkens 1997
Restuccia 1982 Winkens RAG, Knottnerus JA, Kester ADM, Grol RPTM, Pop P.
Restuccia JD. The effect of concurrent feedback in reducing inappro- Fitting a routine health-care ativity into a randomized trial: an ex-
priate hospital utilization. Med Care 1982;20(1):46–62. [MedLine: perimet possible without informed consent?. Clin Epidemiol 1997;
82194054]. 50(4):435–439.
References to studies awaiting assessment Olivarius 2001
Olivarius NF, Beck-Nielsen, Andreasen AH, Horder M, Pedersen
Baker 1996 PA. Randomised controlled trial of structured personal care og type
Baker R. The effectiveness of reminders. a randomized controlled 2 diabetes mellitus. BMJ 2001;323(7319):970–5.
trial of reminders with feedback compared to feedback alone for im-
plementing guidelines for the management of patients receiazepines Rantz 2001
in general practiceiving long term benzod. Eli Lilly National Clinical Rantz MJ, Popejoy L, Petroski GF, Madsen RW, Mehr DR, Zwygart-
Audit Centre, Leicester 1996. Stauffacher M, Hicks LL, Grando V, Wipke-Tevis DD, Bostick J,
Porter R, Conn VS, Maas M. Randomized clinical trial of quality
Eccles 2001 improvement intervention in nursing homes. Gerontologist 2001;41
Eccles M, Steen N, Grimshaw J, Thomas L, McNamee P, Soutter J, (4):525–538.
Wilsdon J, Matowe L, Needham G, Gilbert F, Bond S. Effect of au-
dit and feedback, and reminder messages on primary-care radiology Robling 2002
referrals: a randomised trial. Lancet 2001;357(9266):1406–9. Robling MR, Houston HL, Kinnersley P, Hourihan MD, Cohen
DR, Hale J, Hood K. General practitioners use of magnetic resonance
Fallowfield 2002 imaging: a open randomized controlled trial of different methods of
Fallowfield L, Jenkins V, Farewell V, Saul J, Duffy A, Eves R. Efficacy local guideliens dissemination. Clinical Radiology 2002;57(5):402–
of a cancer Research UK communication skills training model for 7.
oncologists: A randomised controlled trial. Lancet 2002;359(9307):
Sauaia 2000
650–656.
Sauaia A, Ralston D, Schluter WW, Marciniak TA, Havranek EP,
Finkelstein 2001 Dunn TR. Influencing care in acute myocardial infarction: a ran-
Finkelstein JA, Davis RL, Dowell SF, Metlay JP, Soumerai SB, Rifas- domized trial comparing 2 types of intervention. Am J Med Qual
Shiman SL, Higham M, Miller Z, Miroshnik I, Pedan A, Platt R. 2000;15:197–206.
Reducing antibiotic use in children: a randommized trial in 12 prac- Zwar 2002
tices. Pediatrics 2001;108(1):1–7. Zwar H, Henderson J, Britt H, McGeechan K, Yeo G. Influensing
Hayes 2001 antibiotic prescribing by prescriber feedback and management guide-
Hayes R, Bratzler D, Armour B, Moore l. Comparison of an en- lines: a 5 year follow-up. Family Practice 2002;19(1):12–7.
hanced versus written feedback model on the management of Medi-
care inpatients with venous thrombosis. Joint Commission Journal on Additional references
Quality Improvement 2001;27(3):155–68. Axt-Adam 1993
Axt-Adam P, van der Wouden JC, van der Does E. Influencing be-
Heller 2001
havior of physicians ordering laboratory tests: a literature study. Med
Heller RF, DEste C, Lim LL, OConnel RL, Powell H. Randomised
Care 1993;31:784–94.
controlled trial to change hospital management of unstable angina.
Medical Journal of Australia 2001;175(5):217–21. Balas 1996
Balas EA, Boren SA, Brown GD, Ewigman BG, Mitchell JA, Perkoff
Kiefe 2001 GT. Effect of physician profiling on utilization. Meta-analysis of ran-
Kiefe CI, Allison JJ, Williams OD, Person SD, Weaver MT, Weiss- domized clinical trials. J Gen Intern Med 1996;11:584–90.
man NW. Improving quality improvement using achievable bench-
marks for physician feedback: a randomized controlled trial. JAMA Bandura 1986
2001;285(22):2871–9. Bandura A. Social foundations of thought and action: a social cognitive
theory. Prentice-Hall, 1986.
McCartney 2001
MCCartney P, Macdowall W, Thorogood M. Feedback in primary Davis 1991
care can improve the prescribtion of hormon replacement therapy Davis D, Fox R. The Research and Development of Resource Base in
in women with a history of hyeterectomy. British Journal of Clinical CME. An Annotated Bibliography and Literature Searching Service.
Governance 2001;6(1):17–21. Annual Report. Hamilton: McMaster University, 1991.
Moher 2001 Davis 1992

Moher M, Yudkin P, Wright L, Turner R. Cluster randomised con- Davis DA, Thomson MA, Oxman AD, Haynes RB. Evidence for the
trolled trial to compare three methods of promoting secondary pre- effectiveness of CME . A review of 50 randomized controlled trials.
vention of coronary hearth disease in primary care. BMJ 2001;322 JAMA 1992;268:1111–7.
(7298):1338. Davis 1995
Davis DA, Thomson MA, Oxman AD, Haynes RB. Changing physi-
Nilsson 2001
cian performance. A systematic review of the effect of continuing
Nilsson G, Hjemdal P, Hassler A, Vitols S, Wallen NH, Krakau I.
medical education strategies. JAMA 1995;274:700–5.
Feedback on prescribing rate combined with problem-oriented phar-
macotherapy education as a model to improve prescribing among Fox 1989
general practitioners. European Journl of Clinical Pharmacology 2001; Fox R, Mazmanian P, Putnam RW. Changing and Learning in the
56(11):843–8. Lives of Physicians. New York: Praeger, 1989.
Foy 2002 Oxman 1995
Foy R, MacLennan G, Grimshaw J, Penney G, Campbell M, Grol Oxman AD, Thomson MA, Davis DA, Haynes RB. No magic bul-
R. Attributes of clinical recommendations that influence change in lets: a systematic review of 102 trials of interventions to improve pro-
practice following audit and feedback. Journal of Clinical Epidemiol- fessional practice. CMAJ 1995;153(10):1423–31.
ogy 2002;55:717–722. Oxman 2001
Freemantle 1997 Oxman AD, Flottorp S. An overview of strategies to promote imple-
Freemantle N, Harvey EL, Wolf F, Grimshaw JM, Grilli R, Bero LA. mentation of evidence based health care. In: Silagy C, Haines A, ed-
Printed educational materials: effects on professional practice and itor(s). Evidence Based Practice. 2 Edition. London: BMJ Publishers,
health care outcomes (Cochrane Review). In: Cochrane Library, 2, 2001:101–19.
2003.Oxford: Update Software.10.1002/14651858.CD004398 Prochaska 1992
Prochaska JO, DiClemente CC, Norcross JC. In search of how people
Green 1988 change. Applications to addictive behaviors. Am Psychol 1992;47:
Green LW, Eriksen MP, Schor EL. Preventive practices by physicians: 1102–14.
behavioural determinants and potential interventions. Am J Prev Med Sibley 1982
1988;4(suppl 4):101–7. Sibley JC, Sackett DL, Neufeld V, Gerrard B, Rudnick KV, Fraser W.
Grimshaw 2001 A randomized trial of continuing medical education. N Engl J Med
Grimshaw JM, Shirran L, Thomas R, Mowatt G, Fraser C, Bero L. 1982;306:511–5.
Changing provider behavior: An overview of systematic reviews of Stone 2002
interventions. Med Care 2001;39(Supplement 2):II-2 - II-45. Stone EG, Morton SC, Hulscher ME, et al. Interventions that in-
crease use of adult immunization and cancer screening services: a
Grimshaw 2002 meta-analysis. Annals of Internal Medicine 2002;136:641–651.
Grimshaw JM, Thomas RE, Maclennan G, Fraser C, Ramsay CR,
Thomson 1995
Vale L. Effectiveness and efficiency of guideline dissemination and
Thomson MA. A systematic review of three interventions to improve
implementation strategies. Final Report. Aberdeen: Health Service
health professional practice. Unpublished Master’s Thesis, McMaster
Research Unit, University oF Aberdeeen, 2002
University, Hamilton, Canada 1995.
Juni 1999 Thomson O’Brien 2001
Juni P, Witschi A, Bloch R, Egger M. The hazards og scoring the qual- Thomson O’Brien MA, Freemantle N, Oxman AD, Wolf F, Davis
ity of clinical trials for meta-analysis. JAMA 1999;282(15):1996–9. DA, Herrin J. Continuing education meetings and workshops: ef-
Juni 2001 fect on professional practice and health care outcomes (Cochrane Re-
juni P, Altman DG, Egger M. Assessing the quality of controlled view). In: The Cochrane Library, 4, 2001.Oxford: Update Software.
trials. BMJ 2002;323:42–46. Thomson OBrien 1997a
Thomson O’Brien MA, Oxman AD, Davis DA, Haynes RB, Free-
Kanouse 1988
mantle N, Harvey EL. Audit and feedback versus alternative strate-
Kanouse DE, Jacoby I. When does information change practitioners’
gies: effects on professional practice and health care outcomes
behavior?. Intl J Technol Assess Health Care 1988;4:27–33.
(Cochrane Review). In: The Cochrane Library, 1, 2003.Oxford: Up-
Kunz 2002 date Software.
Kunz R, Vist GE, Oxman AD. Randomisation to pro- Thomson OBrien 1997b
tect against selection bias in healthcare trials (Cochrane Re- Thomson O’Brien MA, Oxman AD, Davis DA, Haynes RB, Free-
view). In: The Cochrane Library, 1, 2003.Oxford: Update mantle N, Harvey EL. Audit and feedback: effects on professional
Software.10.1002/14651858.MR000012 practice and health care outcomes (Cochrane Review). In: The
Cochrane Library, 1, 2003.Oxford: Update Software.
Mugford 1991
Mugford M, Banfield P, O’Hanlon M. Effects of feedback of infor-
mation on clinical practice: a review. BMJ 1991;303:398–402. ∗
Indicates the major publication for the study
TABLES
Characteristics of included studies
Study Anderson 1994

Methods Overall quality; MODERATE
Characteristics of included studies (Continued )
Participants 646 physicians from 15 short-stay hospitals
Country: USA
Type of targeted behaviour: General management of a problem (prophylaxis for venous thromboembolism)
Complexity of targeted behaviour:
MODERATE
Interventions 1. A&F (non-intensive)
2. A&F (moderate)
3. Control
Outcomes Proportion or % of patients receiving prophylaxis for venous thromboembolism
Notes
Allocation concealment A
Study Anderson 1996

Methods Overall quality;
LOW
Participants 54 primary care physicians
Country: Canada
Type of targeted behaviour: General management of a problem (prescribing of analgesics)
MODERATE
Interventions 1. A&F (moderate)+ educational meeting
2. A&F (moderate)
3. Control
Outcomes Mean number of prescriptions per physician for analgesics
Notes
Allocation concealment C
Study Baker 1997

Participants 18 general practices
Country: UK
Type of targeted behaviour: General management of a problem (use of benzodiazepines)
MODERATE
2. A&F (non-intensive) + reminders
Contribution of A&F:
MODERATE
Outcomes % compliance with guidelines for use of benzodiazepines
Notes
Study Balas 1998
Participants 10 community based physicians from 5 dialysis centres
Country: USA
Type of targeted behaviour: General management of a problem (patients with end-stage renal disease)
HIGH
Interventions 1. A&F (moderate)
2. Control
Outcomes % patients on CAPD
Notes
Study Belcher 1990

Participants 1224 patients randomised to unclear number of physicians in primary care
Country: USA
Type of targeted behaviour: Preventive care
Complexity of targeted behaviour: MODERATE
Interventions 1. Multifaceted with A&F (A&F (non-intensive)+ educational meetings+ reminders )
Contribution of A&F;
MINOR
2. Multifaceted with A&F (A&F (non-intensive)+ educational meetings + reminders + patient mediated
prompts )
MINOR
3. Multifaceted with A&F
(A&F (non-intensive) + educational meeting + reminders+ prompts + patient invitation
MINOR
4. Control
Outcomes % patients on appropriate preventive services
Notes
Study Berman 1998

LOW
Participants 27 resident anaesthesiologists
Country: USA
Type of targeted behaviour: Prescribing for three procedures
HIGH
2. Control
Outcomes Costs of anaesthetics
Notes
Allocation concealment B
Study Boekeloo 1990

LOW
Participants 29 internal medicine interns from 1 hospital
Country: USA
Type of targeted behaviour: Prescribing (high blood cholesterol)
MODERATE
Interventions 1. Reminders
2. A&F (moderate)
3. Multifaceted with A&F (A&F (moderate) + reminders)
MODERATE
4. Didactic meeting
Outcomes (% meeting criteria)
Record of cholesterol history
Cholesterol level assessed
Ordering low fat diet
Lipid profile (tests)
Consultation with preventive
cardiology programme
Consultation with nutritionist
Notes
Study Bonevski 1999

Participants 19 general practitioners
Country: Australia
Type of targeted behaviour: General management of a problem
MODERATE
2. Written materials/control
Outcomes Accuracy of classification of patient risk status for preventive care
Notes
Study Borgiel 1999
Participants 56 family physicians
Country: Canada
Type of targeted behaviour: General management of a problem (four areas)
MODERATE
2. A&F (moderate) + educational meeting (outreach)
Outcomes Quality of care
Notes
Study Brady 1988

Participants 45 physicians (residents) from 1 outpatient clinic in 1 hospital
Country: USA
Type of targeted behaviour: Prescribing
(influenza vaccination or mammography screening)
MODERATE
Interventions 1. A&F (non-intensive) +educational materials + didactic meetings
2. A&F (non-intensive) + educational materials + didactic meetings + self-audit
3. A&F (non-intensive) + educational materials + conferences
Outcomes Proportion of eligible patients receiving flu vaccination and mammography
Notes
Study Brown 1994

Participants 24 private dental practices without hygienists
Country: Australia
Type of targeted behaviour: General management of a problem (periodontal care)
MODERATE
Interventions 1. A&F (intensive) + outreach visits (ed meeting)
2. Control
Outcomes Mean percentage of patients per practice receiving periodontic care
Notes * There were three study groups but only two (without hygenists) were randomly allocated into experimental
and control groups.
Study Buffington 1991
Participants 45 physicians from 13 practices
Country: USA
(influenza immunisations)
LOW
Interventions 1. Multifacted with A&F (A&F (moderate) + patient mediated interventions + conferences + other)
MODERATE
2. A&F (moderate)+ conferences + other (visits to office staff to aid data collection + telephone consultation
facility)
3. Control
Outcomes % patients influenza vaccinated
Notes Patient mediated=mailed postcard reminder
Study Buntinx 1993

Participants 179 physicians for unclear number of practices
Country: Belgium
Type of targeted behaviour: General management of a problem (quality of cervical smears)
MODERATE
Interventions 1. Multifacted with A&F (A&F (moderate)+ reminders)
Contribution of A&F: MODERATE
2. Multifacted with A&F (A&F (moderate)+ specific advice + reminders)
3. Reminders
4. Educational materials/control
Outcomes Quality of smears
Notes
Study Buntinx 1993a

Country: Belgium
Type of targeted behaviour: General management of a problem (quality of cervical smears)
MODERATE
Interventions 1. Multifaceted with A&F (A&F (moderate)+ reminders)
2. Multifaceted with A&F (A&F (moderate)+ specific advise + reminders)
3. Reminders
Notes
Study Buntinx 1993b

Interventions 1. Multifacted with A&F (A&F (moderate)+ reminders)
2. Multifacted with A&F (A&F (moderate)+ spesific advise + reminders)
3. Reminders
Notes
Allocation concealment D
Study Chassin 1986

Participants 1483 physicians from 120 hospitals
Country: USA
Type of targeted behaviour:
Prescribing (pelvimetry for pregnancy)
MODERATE
Interventions 1. A&F (moderate) + didactic meetings + written materials
2. Control
Outcomes Pelvimetry rate per 100 deliveries
Notes
Study Cohen 1982

Participants Physicians (residents & physicians) from 4 firms in 1 hospital
Country: USA
(lab tests and x-rays)
MODERATE
Interventions 1. A&F (moderate) on lab tests
2. A&F (moderate) on x-rays
Outcomes Number of tests ordered
Test charges
Notes
Study De Almeida Neto 2000

Participants 24 pharmacists 24 pharmacies
Country: Australia
Type of targeted behaviour: General management of a problem (identification of inappropriate over the
counter analgesics)
MODERATE
Interventions 1. A&F (moderate) + educational meetings
2. Control
Outcomes Identify misuse
discuss alternate medications
Notes
Study Dickinson 1981

Participants 40 physicians (residents & faculty) from 1 family medicine centre
Country: USA
(hypertension control)
MODERATE
2. Self-study
3. A&F + self study
4. Control
Outcomes % patients with controlled BT
Notes
Study Everett 1983

LOW
Participants 24 physicians (residents) from 5 ward teams in 1 hospital
Country: USA
Type of targeted behaviour: Prescribing (various clinical conditions)
MODERATE
Interventions 1. A&F (intensive) + written materials
2. Control
Outcomes Costs and use of lab tests
Notes
Study Fairbrother 1999

Participants 61 pediatricians and family physicians
Country: USA
Type of targeted behaviour: Preventive care (immunization coverage)
MODERATE
Interventions 1. A&F
2. Multifacted with A&F (A&F (moderate) + one-off bonus)
MODERATE
3. Multifacted with A&F
(A&F (moderate) + enhanced fee-for-service)
MODERATE
4. Control
Outcomes Immunisation status
Notes
Study Fairbrother 1999a

Country: USA
Type of targeted behaviour: Preventive care (immunizaton coverage)
MODERATE
MODERATE
MODERATE
4. Control
Notes
Study Fairbrother 1999b

Country: USA
Type of targeted behaviour: Preventive care (immunizaton coverage)
MODERATE
MODERATE
MODERATE
4. Control
Notes
Study Feder 1995

Participants 39 physicians from 24 general practices
Country: UK
Type of targeted behaviour: General management of a problem (asthma and diabetic care)
MODERATE
Interventions 1. Multifacted with A&F
(A&F (non-intensive) for asthma + written materials + educational meetings (outreach) + phys prompts)
MINOR
(A&F (non-intensive) for diabetes + written materials + educational meetings (outreach) + phys prompts)
MINOR
Outcomes Quality of care
Notes
Study Gama 1991

Participants 5 physicians in general medicine
Country: UK
Type of targeted behaviour: General management of a problem (laboratory use)
MODERATE
2. Control
Outcomes Laboratory use and costs
Notes
Study Gehlbach 1984

LOW
Participants 31 physicians (residents & faculty) ,
Country: USA
(drugs)
LOW
2. Control
Outcomes Proportion of generic prescriptions in targeted list
Notes
Study Goldberg 1998

Participants 95 physicians from 15 small group practices
Country: USA
Type of targeted behaviour: Compliance with guidelines
(hypertension and depression)
MODERATE
Interventions 1. A&F (moderate) + educational meetings (outreach)
2. A&F (moderate) + educational meetings (outreach) + CQI team facilitation
3. Control
Outcomes % compliance with guidelines;
increase hypertensives prescribed diuretics
increase hypertensives prescribed K sparing regimen
increase hypertensives prescribed beta blockers
not prescribed cc blockers
not prescribed ACE inhibitors
increase hypertensives controlled
not over controlled
increase eligible patients on antidepressants
not on 1st line antidepressants
increase patients on 2nd line antidepressants
increase patients on SSRIs
decrease SCL mean score
Notes
Study Grady 1997

Participants 95 primary care physicians from 65 practices
Country: USA
Type of targeted behaviour: Referrals
(mammography)
MODERATE
Interventions 1. Multifacted with A&F (A&F (moderate) + didactic meeting + phys prompts + incentives)
MODERATE
2. Didactic meeting + phys prompts
3. Didactic meeting
Outcomes Mammography referrals, completion and compliance rates
Notes
Study Gullion 1988

Participants 111 physicians in private practice
Country: USA
Type of targeted behaviour: General management of a problem (hypertensive care)
MODERATE
Interventions 1. A&F (moderate) on medication (medical records) + written materials + educational meeting (conference
call)
2. A&F (moderate) on performence (survey) + written materials + educational meeting (conference call)
3. Combined 1 + 2
4. Control
Outcomes Correct action to reduce diastolic blood pressure in uncontrolled patients
Total advice score
Diastolic blood pressure
Notes
Study Gullion 1988a

Country: USA
MODERATE
call)
3. Combined 1 + 2
4. Control
Total advice score
Notes
Study Gullion 1988b

Country: USA
MODERATE
call)
3. Combined 1 + 2
4. Control
Total advice score
Notes
Study Gullion 1988c

Country: USA
MODERATE
call)
3. Combined 1 + 2
4. Control
Total advice score
Notes
Study Hemminiki 1992

Participants 53 hospitals
Country: Finland
Type of targeted behaviour: General management of a problem (cesarean rates)
LOW
2. Control
Outcomes Mean section rates and cesarean rates
Notes
Study Henderson 1979

LOW
Participants Unclear number of hospital physicians (interns)
Country: USA
Type of targeted behaviour: General management of a problem (costs)
MODERATE
2 Control
Outcomes Costs
Notes
Study Hendryx 1998

HIGH
Participants 20 rural hospitals
Country: USA
Type of targeted behaviour: Compliance with guidelines (intensive care)
HIGH
Interventions 1. A&F (moderate) + educational meeting (outreach) + written materials + seminars + telephone consultation
service
2. Control
Outcomes Compliance with ICU guidelines
total processes of care
total lenght of stay (LOS) mean ICU LOS
mean ventilator days
Notes
Study Hershey 1986

MODERATE
Participants 48 physicians (residents) from 4 firms in 1 hospital
Country: USA
(drug)
HIGH
2. Control
Outcomes Cost per resident, prescription per resident
Mean charge per prescription
Mean charge per patient
Prescriptions per patient
Notes
Study Hillman 1998

Participants 52 primary care practices
Country: USA
Type of targeted behaviour: Preventive care (cancer screening)
MODERATE
(A&F (moderate)+ incentive)
MODERATE
2. Control
Outcomes Compliance scores for cancer screening
Notes
Study Hillman 1999

HIGH
Country: USA
Type of targeted behaviour: Preventive care (pediatric)
MODERATE
(A&F (moderate) + incentive)
MODERATE
2. A&F (moderate)
3. Control
Outcomes Compliance with pediatric preventive care guidelines
Notes
Study Holm 1990

Participants 365 physicians from general practice
Country: Denmark
Type of targeted behaviour: General management of a problem (long-term use of hypnotics/sedatives)
MODERATE
Interventions 1. A&F (moderate) + written materials
2. Meeting (didactic?) + written materials
3. Control
Outcomes Mean prescribed defined daily dose
Notes
Study Howe 1996

Country: UK
Type of targeted behaviour: General management of a problem (psychological distress)
MODERATE
Interventions 1. A&F (moderate) + written materials + self-assessment of video
2. Control
Outcomes Mean detection of psychological distress rate per physician
Notes
Study Hux 1999

Country: Canada
Type of targeted behaviour: Prescribing (antibiotic)
LOW
2. Control
Outcomes % first line antibiotics use, median prescribing costs
Notes
Study Jones 1996

Participants 124 nurses from one hospital
Country: USA
Type of targeted behaviour: General management of a problem (capillary blood glucose monitoring)
LOW
Interventions 1. A&F continued (moderate)
2. A&F withdrawn (moderate)
Outcomes Mean nurse accuracy in CBGM (blood glucose monitoring)
Notes
Study Kafuko 1999
Participants 127 health units from 6 districts in 4 regions
Country: Uganda
(rational drug use)
MODERATE
Interventions 1. A&F (intensive) + written materials + educational meetings
(A&F (intensive) + written materials + educational meetings + support)
MODERATE
3. Written materials
Outcomes Rational prescribing/average # of drug prescribed
%general cases treated according to guidelines
%malaria treated according to guidelines
Notes
Study Kerry 2000

Participants 175 physicians from 69 general practices
Country: UK
Type of targeted behaviour: Test ordering (x-ray referrals)
MODERATE
2. Control
Outcomes All referrals of x-ray
Notes
Study Kerse 1999

HIGH
Participants 42 physicians in general practice
Country: Australia
Type of targeted behaviour: Preventive care (health promotion for elderly people)
MODERATE
(A&F (non-intensive) + educational meetings (outreach) + phys prompts + didactic seminar or home study
+ written materials)
MINOR
2. Control
Outcomes % patients recall discussion about exercise
Notes
Study Kim 1999

Country: Scotland
Type of targeted behaviour: Preventive care (immunization and mammography)
MODERATE
Interventions 1. A&F (intensive) + educational meetings (outreach) + written materials
Outcomes Influenza vaccine
pneumococcal vaccine
tetanus vaccine
mammography
breast examination
exercise counseling
smoking cessation
Notes
Study Kinsinger 1998

HIGH
Participants 62 practices from family medicine and internal medicine
Country: USA
Type of targeted behaviour: Screening
(breast cancer)
MODERATE
2. A&F (non-intensive) + facilitation of office system
Outcomes % patients screened for breast cancer
Notes
Study Leviton 1999

HIGH
Participants Obstetricians in 27 hospitals
Country: USA
Type of targeted behaviour: General management of a problem (use of antenatal corticosteroids for fetal
maturation)
MODERATE
Interventions 1. Multifacted with A&F (A&F (moderate) + educational meetings + opinion leaders + phys prompts +
written materials)
MINOR
2. Control
Outcomes % use of antenatal corticosteroids
Notes
Study Linn BS 1980

Country: USA
Type of targeted behaviour: General management of a problem (burn care)
MODERATE
Interventions 1. A&F (moderate) + written materials + conferences (didactic?) + access to hotline
2. Control
Outcomes % deviations from treatment recommendations
Notes
Study Linn BS, 1980

Country: USA
Type of targeted behaviour: General management of a problem (burn care)
MODERATE
Interventions 1. A&F (moderate) + written materials + conferences (didactic?) + acess to hotline
2. Control
Outcomes % deviations from treatment recommendations
Notes
Study Lobach 1996
Country: USA
Type of targeted behaviour: General management of a problem (diabetes)
MODERATE
2. Control
Outcomes % compliance with diabetes guidelines
Notes
Study Lomas 1991

HIGH
Participants 76 physicians in 16 community hospitals
Country: Canada
Type of targeted behaviour: General management of a problem (cesarean rates)
2. Local opinion leaders + written materials + educational meetings
Outcomes % offered trial of labour, underwent trial of labour or vaginal birth
Notes
Study Mainous 2000

Country: USA
(antibiotic for respiratory infections)
2. Patient education materials
(A&F (moderate) + patient education)
MODERATE
4. Control
Outcomes Antibiotic prescribing rates
Notes
Study Manfredi 1998

HIGH
Participants 51 private physician practices
Country: USA
Type of targeted behaviour: Preventive care (cancer screening)
Interventions 1. Multifaceted with A&F (A&F (moderate) + educational meetings (outreach) + phys prompts + patients
prompts + written materials)
MINOR
Outcomes Number screened according to guideline
non HMO - CBE
non HMO - mammography
non HMO - pap
non HMO - FOBT
Notes
Study Manheim 1990

LOW
Participants 105 physicians (interns) from 2 hospitals
Country: USA
Type of targeted behaviour: Length of stay, costs
HIGH
2. Control
Outcomes Length of stay
Cost of episode
Notes
Study Martin 1980

Participants 24 physicians (residents) from 3 ward teams in 1 hospital
Country: USA
Type of targeted behaviour: Test ordering
(laboratory and radiologic)
Interventions 1. A&F (intensive) + seminar (didactic?) + written materials
2. Seminar (didactic?) + written materials + incentives
3. Seminar (didactic?) + written materials
Outcomes Mean number of laboratory tests per patient admission
Notes
Study Marton 1985

Participants 57 physicians (‘house staff ’) from 3 hospitals
Country: USA
(laboratory use)
MODERATE
3. A&F + written materials
4. Control
Outcomes Mean number tests per patient visit
Notes
Study Marton 1985a

Participants 57 physicians (‘housestaff ’) from 3 hospitals
Country: USA
(laboratory use)
MODERATE
4. Control
Notes
Study Marton 1985b
Participants 57 physicians (‘housestaff ’) from 3 hospitals
Country: USA
(laboratory use)
MODERATE
4. Control
Notes
Study Mayefsky 1993

Participants 28 physicians (pediatric house officers) from 2 outpatient clinics in 2 hospitals
Country: USA
Type of targeted behaviour: General management of a problem (child care)
MODERATE
2. Audit no Feedback
Outcomes % compliance with criteria for well child care
Notes
Study Mayer 1998

Participants 138 pharmacists from 54 pharmacies
Country: UK
Type of targeted behaviour: Preventive care (promoting skin cancer)
MODERATE
Interventions 1. Multifacted with A&F (A&F (moderate) + prompts + incentives + video)
MAJOR
2. Control
Outcomes Skin cancer prevention advice
Notes
Study McAlister 1986

Country: Canada
Type of targeted behaviour: Compliance with guidelines (hypertensive care)
(A&F (moderate) + patient reminders)
MODERATE
2. Control
Outcomes % patients with dbp < 90mmHg
Notes
Study McCartney 1997

Participants 28 general practices
Country: UK
Type of targeted behaviour: Preventive care? (aspirin prescribing)
MODERATE
2. Control
Outcomes % patients with CHD on aspirin
Notes
Study McConnell 1982

Participants 33 physicians in ambulatory care
Country: ?
Type of targeted behaviour: Prescribing (tetracycline for upper respiratory infections)
LOW
Interventions 1. A&F (moderate) + educational meetings (outreach) + written materials
2. Control
Outcomes % continuing to prescribe tetracycline inappropriate
Notes
Study Meyer 1991

LOW
Participants 141 physicians and nurses from 1 outpatient clinic in 1 hospital
Country: USA
Type of targeted behaviour: Compliance with guidelines (polypharmacy)
MODERATE
2. A&F (moderate) + peer review + recommendations
3. Control
Outcomes Mean number of prescriptions
Notes
Study Moongtui 2000

Participants 91 nurses and patient care aides
Country: Thailand
Complexity of targeted behaviour: LOW
2. Control
Outcomes Compliance rate for hand wash and glove use
Notes
Study Norton 1985

Participants 6 physicians in a teaching unit
Country: Canada
Type of targeted behaviour: Compliance with guidelines (vaginitis and cystitis)
MODERATE
Interventions 1. A&F (moderate) on cystitis
2. A&F (moderate) on vaginitis
Outcomes Compliance rate with standards for cystitis and vaginitis
Notes
Study Norton 1985a
Country: Canada
MODERATE
Interventions 1. A&F (moderate) on cystitis
2. Control (A&F (moderate) on vaginitis)
Outcomes Compliance rate with standards for cystitis
Notes
Study Norton 1985b

Country: Canada
MODERATE
Interventions 1. A&F (moderate) on vaginitis
2. Control (A&F (moderate) on vaginitis)
Outcomes Compliance rate with standards for vaginitis
Notes
Study O’Connell 1999

HIGH
Country: Australia
(five main drugs)
2. Control
Outcomes Prescribing rates
Notes
Study Palmer 1985

Participants 111 internists, 94 paediatricians, 343 residents and 163 non-physicians (mostly nurse practitioners), total =
711, in 16 primary care practices
Country: USA
(preventive services)
VARIOUS
Interventions 1. A&F (moderate) + educational meetings + written materials
2. Control
Outcomes Compliance with guidelines - various
Notes
Study Raasch 2000

Participants 46 family physicians
Country: Australia
Type of targeted behaviour: General management of a problem (diagnosis and management of suspicious
skin lesions)
MODERATE
2. Control
Outcomes % correct clinical diagnosis
Notes
Study Reid 1977

LOW
Participants 21 physicians (internal medicine)
Country: USA
MODERATE
2. Control
Outcomes Number of services, costs, consultation time
Notes
Study Roski 1998

Country: USA
MODERATE
Interventions 1. Multifaceted with A&F
(A&F (non-intensive) + educational meetings (outreach) + incentives + free NRT)
MINOR
2. Control
Outcomes Assessed smoking status
smoking advice
Notes
Study Ruangkanchanastr 19
Participants 18 physicians in pediatric out-patient hospital
Country: Thailand
Type of targeted behaviour: Tests
(laboratory)
MODERATE
Interventions 1. A&F (intensive) + seminar (didactic?)
2. Control
Outcomes Mean lab tests (paed) per patient
Notes
Study Ruangkanchanastr a
Country: Thailand
(laboratory)
MODERATE
2. Control
Outcomes Mean lab tests (paed) pr patient
Notes
Study Ruangkanchanastr b
Country: Thailand
(laboratory)
MODERATE
2. Control
Notes
Study Ruangkanchanastr c
Country: Thailand
(laboratory)
MODERATE
2. Control
Notes
Study Rust 1999

LOW
Participants 32 physicians in a hospital based primary care clinic
Country: USA
Type of targeted behaviour: General management of a problem (immunization)
MODERATE
2 Control
Outcomes Rates of immunisation
Notes
Study Sanazaro 1978
LOW
Participants Physicians from 50 hospitals
Country: USA
Type of targeted behaviour: Compliance with guidelines (7 conditions)
MODERATE
Interventions 1. A&F (moderate) + local consensus + written materials
2. Control
MAJOR
Outcomes Adherence to treatment criteria
Notes
Allocation concealment C
Study Sandbaek 1999

LOW
Country: Denmark
Type of targeted behaviour: Preventive care (AIDS)
LOW
Interventions 1. Multifacted with A&F (A&F (moderate) + educational meetings + written materials + reminders)
2. Control
Outcomes % advised about AIDS
Notes
Study Schectman 1995

Participants 63 internists and family physicians
Country: USA
(increase use of cimetidine over other histamine 2 receptor blockers)
Complexity of targeted behaviour: LOW
Interventions 1. A&F (non-intensive) + written materials
Outcomes Proportion of cimetidine prescriptions per physician
Notes
Study Simon 2000

Participants 613 patients
Country: USA
Type of targeted behaviour: General management of a problem (depression)
MODERATE
2. A&F + care management for patients
Outcomes Costs
Frequency of follow-up visits
Notes
Study Sinclair 1982

Participants 4 units from a child and family clinic
Country: Canada
Type of targeted behaviour: General management of a problem (child mental health)
MODERATE
2. Control
Outcomes Overall quality score
Notes
Study Smith 1995

MODERATE
Participants 9 obstetricians and 26 midwives
Country: UK
Type of targeted behaviour: Screening
MODERATE
Interventions 1. A&F (moderate) + educational meetings + written materials
2. Educational meetings + written materials
Outcomes Communications skills
Notes
Study Smith 1998
Methods Overall quality; HIGH
Participants 222 physicians
Country: USA
(drug use)
MODERATE
2. Control
Outcomes Information giving
Communication skills
Drug use
Tablets pr prescription
Nb of prescriptions
Notes
Study Socolar 1998

Participants 147 physicians
Country: USA
Type of targeted behaviour: General management of a problem (evaluation for child sexual abuse)
LOW
2. Control
n/a
Outcomes Change in documentation score
Change in documentation score Physical ex
Notes
Study Sommers 1984

LOW
Country: USA
(anaemia)
MODERATE
Interventions Phase 1
1. A&F (moderate) + local consensus process
2. A&F (moderate)
3. Control
Phase 2
all 3 groups received concurrent reminders for care (no control group)
MAJOR
Outcomes Compliance with criteria for anaemia
Notes
Study Thompson 2000

HIGH
Participants 179 members of adult care teams (physcians, nurses and other members) from 5 primary care clinics
Country: USA
Type of targeted behaviour: Compliance with guidelines (domestic violence)
MODERATE
(A&F (non-intensive) + educational meetings+ written materials + phys prompts + patient prompts + opinion
leaders)
MINOR
2. Control
Outcomes Asked about domestic violence
Case finding for domestic violence
Notes
Study Tierney 1986

Participants 135 physicians (residents) from 4 hospital clinics
Country: USA
Type of targeted behaviour: Preventive care
MODERATE
Interventions 2x2 design
1. A&F (moderate)
2. Reminders
MODERATE
Outcomes % patients who received prev care according to guidelines
Notes
Study Veninga 1999

MODERATE
Country: Netherlands, Sweden, Norway and SK
MODERATE
Interventions 1. A&F (moderate) on asthma + educational meetings
2. A&F (moderate) on UTI +educational meetings
Outcomes % of patients on inhaled corticosteroids (prescribing for asthma)
% of pt on continuous broncodilator monotherapy
% of pt on adequate level of inhaled corticosteroids
% of prescribtions for oral corticosteroids for exacerbations
Notes
Study Veninga 1999a

MODERATE
MODERATE
Interventions 1. A&F (moderate) on asthma + educational meetings
2. A&F (moderate) on UTI +educational meetings
Notes
Study Veninga 1999b

MODERATE
MODERATE
Interventions 565 physicians from general practice
MODERATE
Notes
Study Veninga 1999c

MODERATE
MODERATE
MODERATE
Notes
Study Veninga 1999d

MODERATE
MODERATE
MODERATE
Notes
Study Vinicor 1987

LOW
Participants 86 physicians (residents) from 1 clinic in 1 hospital
Country: USA
HIGH
Interventions 1. Physician education; Multifacted with A&F (A&F (intensive) + reminders + patient mediated intervention
+ consultation facility + educational meetings + written materials + hotline)
MODERATE
2. Patient education;
MINOR
3. Physician and patient education
4. Control
Outcomes Fasting plasma glucose
Glycosylated haemoglobin (Ahgb) 2 hour post prandial glucose
Weight
Systolic and diastolic blood pressure
Notes Patient mediated=diabetic patient education program, approximately 2 months long
Study Ward 1996

Country: Australia
MODERATE
2. A&F (moderate) + educational meeting (outreach) by peer
3. A&F (moderate) + educational meeting (outreach) by nurse
Outcomes Compliance with guidelines
Notes
Study Wells 2000

HIGH
Participants 181 physicians from 46 primary care practices
Country: USA
Type of targeted behaviour: General management of a problem (depression)
MODERATE
Interventions 1. Multifaceted with A&F (A&F (moderate) + opinion leaders + educational meetings (outreach) + written
materials) +phys prompts in medication
MODERATE
2. Multifaceted with A&F (A&F (moderate) + opinion leaders + educational meetings (outreach) + written
materials) + phys prompts in CBT
MODERATE
3. Control
Outcomes Overall appropriate care
% patients not depressed
Notes
Study Winickoff 1984

Participants 16 physicians from 1 practice
Country: USA
Type of targeted behaviour: Screening for colorectal cancer
MODERATE
2. Control
Outcomes %check-ups (test done) including BT
Notes
Study Winickoff 1985

Participants 32 physicians and nurses from 16 practices
Country: USA
Type of targeted behaviour: Compliance with guidelines (hypertension)
MODERATE
Interventions 1. Multifaceted with A&F
(A&F (moderate) + reminders )
2. Control
Outcomes Ordering all recommended tests
Follow up within 6 months
Notes
Study Winkens 1995

LOW
Participants 79 family physcians
Country: Netherlands
MODERATE
Interventions 1. A&F (intensive) for one set of tests
2. A&F (intensive) for a second set of tests
Outcomes Mean number of test requests per physician according to guideline
Notes
Study Wones 1987

Participants 21 physicians (residents) from unclear number of practices
Country: USA
Type of targeted behaviour: Lab tests
MODERATE
Interventions 1. A&F peer (moderate)
2. A&F without peer (moderate)
3. Control
Outcomes Tests per patient-day
Charges per patient-day
Tests per patient-day
Notes
Study Wones 1987a
Country: USA
MODERATE
3. Control
Notes
Study Wones 1987b

Country: USA
MODERATE
3. Control
Notes
Study Zwar 1999

Country: Australia
(antibiotic)
MODERATE
Interventions 1. A&F (moderate) + educational meetings + written materials on URT
2. A&F (moderate) + educational meetings + written materials on benzodiazepines
Outcomes Rate of antibiotic prescribing
Notes
Study van den Hombergh 99

HIGH
Interventions 1. A&F by peer (moderate)
2. A&F by non-physician observer (moderate)
Outcomes 208 indicators of practice management
Notes
Study van der Weijden 1999

LOW
Type of targeted behaviour: Compliance with guidelines (cholesterol)
MODERATE
Interventions 1. Multifacted with A&F (A&F (moderate) + educational meetings (outreach) + opinion leaders + written
materials)
MINOR
Outcomes Compliance with guidelines
Notes
Characteristics of excluded studies
Study Reason for exclusion

Anonymous I 1990 Not audit and feedback
Berwick 1986 Randomisation not specified
Billi 1987 Not audit and feedback
Brown 1988 Not RCT
Buekens 1993 Not RCT
Carney 1992 Not feedback on performence
De Silva 1994 Outcome was based on self-report
Del Mar 1998 Not audit and feedback
Dranitsaris 1995 Not feedback
Everett Insufficient data on results
Furniss 2000 Not feedback
Gask 1991 Outcome was teaching interviewing skills to medical students; feedback did not include audit
Gerbert 1988 Not RCT
Goldberg 1980 Not audit and feedback
Grimshaw 1998 Insufficient data on results
Guadagnoli 2000 No feedback on performance postintervention
Hampshire 1999 Insufficient data on results
Hanlon 1996 Not audit and feedback
Hargraves 1996 Not audit and feedback
Hershey 1988 No appropriate comparison
Hetlevik 1998 Not feedback
Horowitz 1996
Johansen 1997 Not audit and feedback
Johnson 1976 Not audit or summery of performence
Kroenke 1990 Not RCT
Linn 1980 Not audit and feedback
MacCosbe 1985 Not audit and feedback
Mandel 1985 Missing results
Mazzuca 1988 Not audit and feedback
McDonel 1997 Not feedback
McPhee 1989 Insufficient data on result
Munroe 1997 Not RCT
Nattinger 1989 Non-equivalent group design with pre-post measures
North of England1992 Missing results
Ogwal-Okeng 2001 Insufficient data on results
Ottolini 1998 Not audit and feedback
Pearson 2001 Not RCT, not feedback
Putnam 1985 Insufficient data on results
Restuccia 1982 Intervention did not include audit
Rubenstein 1989 Not feedback on performence
Rubenstein 1999 Not feedback
Shaughnessy 1991 Not audit, no summery of performence
Spector 1989 Intervention was a federal survey process
Steele 1989 Randomization broked in study
Szczepura 1994 Missing results
Taylor 1997 Not RCT
The SUPPORT 1995 No feedback on performence
Weingarten 2000
White 1995 Not feedback on performence
Characteristics of excluded studies (Continued )
Wing 1987 Not audit and feedback
Wing 1987 (II) Not audit and feedback
Winkens 1997 Insufficient data on results
ADDITIONAL TABLES
Table 01. Categories for continuous outcomes
Category Dichotomous Continuous - example

Large negative < -0.20 -
Moderate negative -0.10 to -0.20 -
Small negative -0.05 to -0.10 -
No difference 0 to 0.05 or -0.05 Socolar 1998, Change in documentation score
Small positive 0.05 to 0.10 Hendryx 1998, Quality of intensive care unit care
Moderate positive 0.10 to 0.20 Smith 1995, Information giving
Large positive > 0.20 Chassin 1986, Rate of pelvemetry
Table 02. Intensity of feedback
Overall
Study id Recipient Format Source Frequency Length Content intensity
Andersen Group Verbal Not clear Not clear Moderate Information Non-intensive
1994 -1 about costs,
numbers
of tests
ordered or
prescribtions
Andersen Individual Both Not clear Not clear Moderate Compliance Moderate
1994 -2 with guideline
Andersen Individual Written A professional Once only Once only Peer Moderate
1996 standard comparison
review
organisation or
representative
of the
employer or
purchaser
Baker 1997 Group Written Not clear Not clear ? Once only? Compliance Non-intensive
with
guideline, peer
comparison
Balas 1998 Individual Written Not clear Moderate Moderate Patient Moderate
information,
information
Table 02. Intensity of feedback (Continued )
Overall
about costs,
numbers
of tests
ordered or
prescribtions,
peer
comparison
Belcher 1990 Group Both Not clear Infrequent Prolonged Not clear Non-intensive
Berman 1998 Individual Written Not clear Frequent Moderate Peer Moderate
comparison,
information
about costs,
numbers
of tests
ordered or
prescribtions
Boekeloo Individual Written A supervisor Not clear Moderate Compliance Moderate
1990 or senior with guideline
colleauge
Bonewski Individual Written The Not clear Not clear Patient Moderate
1999 investigators information
Borgiel 1999 Individual Written, A supervisor Once only Brief Compliance Moder-
unclear or senior with guideline, ate,Non-
colleauge +2 information intensive
about costs,
numbers
of tests
ordered or
prescribtions
Brady 1988 Group Verbal Not clear Once only Once only Not clear Non-intensive
Brown 1994 Individual Both Not clear Once only Prolonged Peer Intensive
comparison
Buffington Group Written Not clear Frequent Moderate Information Moderate
1991 about costs,
numbers
of tests
ordered or
prescribtions
Buntinx 1995 Individual Written Not clear Moderate Moderate Peer Moderate
comparison
Chassin 1986 Group Written A supervisor Moderate Moderate Information Moderate
or senior about costs,
colleauge numbers
Overall
of tests
ordered or
prescribtions
Cohen 1982 Group Written A supervisor Moderate Moderate Information Moderate

colleauge numbers
of tests
ordered or
prescribtions
De almeida Individual Verbal The Not clear Not clear Patient Moderate
neto 2000 investigators information
Dickinson Individual Written Not clear Moderate Moderate Patient Moderate
1981 information
Everett 1983 Individual Both A supervisor Moderate Moderate Information Intensive
colleauge numbers
of tests
ordered or
prescribtions
Fairbrother Individual Written Not clear Infrequent Moderate Peer Moderate
1999 comparison
Feder 1995 Group Not clear The Once only Once only Compliance Non-intensive
investigators with guideline
Gana 1991 Individual Written Not clear Moderate Moderate Peer Moderate
comparison
Gehlbach Individual Written Not clear Moderate Moderate Information Moderate
1984 about costs,
numbers
of tests
ordered or
prescribtions
Goldberg Individual Both A professional Infrequent Moderate Compliance Moderate
1998 standard with
review guideline,Peer
organisation or comparison
representative
of the
employer or
purchaser
Grady 1997 Individual Written Not clear Infrequent Prolonged Peer Moderate
comparison
Guillon 1998 Individual Both A supervisor Once only Once only Patient Moderate
or senior information
Overall
colleauge
Hemminiki Group Written A professional Once only Once only Peer Non-intensive
1992 standard comparison
review
organisation or
representative
of the
employer or
purchaser
Henderson Individual Written Not clear Frequent Once only Information Moderate
about costs,
numbers
of tests
ordered or
prescribtions
Hendryx 1998 Group Both A supervisor Once only Once only Compliance Moderate
or senior with guideline
colleauge
Hersey 1986 Individual Written Not clear Moderate Moderate Information Moderate
about costs,
numbers
of tests
ordered or
prescribtions
Hillman 1999 Group Written Not clear Infrequent Prolonged Compliance Moderate
with guideline
Hillman 1998 Individual Written Not clear Infrequent Prolonged Compliance Moderate
with
guideline,Peer
comparison
Holm 1990 Individual Written Not clear Once only Once only Peer Moderate
comparison
Howe 1996 Individual Written The Once only Once only Compliance Moderate
Huw 1999 Individual Written Not clear Infrequent Moderate Peer Moderate
comparison
Jones 1996 Individual Not clear Not clear Infrequent Moderate Not clear Moderate
Kafuco 1 Group Both Not clear Once only Once only Peer Non-intensive
comparison
Kerry 2000 Individual Written Not clear Once only Once only Information Moderate
about costs,
numbers
of tests
Overall
ordered or
prescribtions
Kerse 1999 Not clear Verbal The Once only Once only Patient Non-intensive
investigators information
Kim 1999 Individual Both Not clear Infrequent Prolonged Compliance Intensive
with
guideline,Peer
comparison
Kinsinger Group Both The Once only Once only Information Non-intensive
1998 investigators about costs,
numbers
of tests
ordered or
prescribtions
Leviton 1999 Group Not clear A supervisor Infrequent Moderate Information Moderate
colleauge numbers
of tests
ordered or
prescribtions
Linn Individual Written Not clear Not clear Not clear Compliance Moderate
with guideline
Loback Individual Written Not clear Frequent Moderate Compliance Moderate
with guideline
Lomas 1991 Group Both The Infrequent Moderate Compliance Moderate
Mainous 2000 Individual Written Not clear Once only Once only Peer Moderate
comparison,
information
about costs,
numbers
of tests
ordered or
prescribtions
Manfredi Group Written Not clear Once only Once only Peer Moderate
1998 comparison
Manheim Individual Not clear Not clear Once only Once only Not clear Moderate
1990
Martin 1980 Individual Verbal The Frequent Moderate Compliance Intensive
Marton 1985 Individual Written Not clear Moderate Moderate Peer Moderate
comparison,
information
Overall
about costs,
numbers
of tests
ordered or
prescribtions
Mayefski 1993 Individual Written The Once only Once only Compliance Moderate
Mayer 1998 Group Written Not clear Frequent Moderate Information Moderate
about costs,
numbers
of tests
ordered or
prescribtions
Mc Alistor Individual Written Not clear Not clear Not clear Patient Moderate
1986 informa-
tion,Compliance
with
guideline,Peer
comparison
Mccartney Group Written Not clear Once only Once only Information Moderate
1997 about costs,
numbers
of tests
ordered or
prescribtions
Mcconnel Individual Both A supervisor Once only Once only Compliance Moderate
colleauge
Meyer 1991 Individual Written The Not clear, Not Not clear,Not Patient Moderate both
investigators clear clear information
both
Moongtui Group Written Not clear Frequent Moderate Compliance Moderate
1999 with guideline
Norton 1985 Individual Not clear Not clear Not clear Not clear Compliance Moderate
with
guidelines
Oonnell 1999 Individual Written A supervisor Infrequent Moderate Compliance Moderate
colleauge
Palmer 1985 Group Verbal Not clear Once only Once only Compliance Moderate
with guideline
Raash 2000 Individual Both The Once only Once only Patient infor- Moderate
investigators mation,Peer
Overall
comparison
Reid 1977 Individual Written Not clear Infrequent Moderate Patient infor- Moderate
mation,Peer
comparison
Roski Group Not clear Not clear Once only Once only Not clear Non-intensive
Ruangkan- Individual Not clear Not clear Frequent Moderate Information Intensive
cabactr about costs,
numbers
of tests
ordered or
prescribtions
Rust Individual Written Not clear Moderate Prolonged Peer Moderate
comparison
Sanzaro 1978 Individual Not clear Not clear Moderate Compliance Moderate
with guideline
Sandbaek Individual Written Not clear Once only Once only Peer Moderate
1999 comparison
Schectman Individual Written Once only Once only Information Non-intensive
1995 about costs,
numbers
of tests
ordered or
prescribtions
Simon 2000 Individual Written Not clear Not clear Moderate Patient Moderate
information
Sinclair 1982 Not clear Not clear A supervisor Once only Once only Compliance Moderate
colleauge
Smith 1998 Individual Written A professional Once only Once only Peer Moderate
standard comparison,
review information
organisation or about costs,
representative numbers
of the of tests
employer or ordered or
purchaser prescribtions
Smith 1995 Individual Both The Once only Once only Compliance Moderate
Socolar 1998 Individual Written The Once only Once only Compliance Moderate
Sommers Not clear Verbal Not clear Once only Once only Compliance Moderate
1984 with guideline
Thompson Group Verbal Not clear Infrequent Moderate Not clear Non-intensive
2000
Tierney 1986 Individual Written Not clear Moderate Moderate Patient Moderate
informa-
tion,Compliance
with guideline
van der Individual Both Not clear Once only Once only Peer Moderate
Homberg comparison
1999
van der Individual Verbal The Infrequent Moderate Compliance Moderate
Weijden 1999 investigators with guideline
Veninga 1999 Individual Both Not clear Once only Once only Compliance Moderate
with guideline
Vinicor 1987 Individual Both Not clear Once only Moderate Compliance Intensive
with guideline
Ward 1996 Individual Both Not clear Once only Once only Compliance Moderate
with guideline,
information
about costs,
numbers
of tests
ordered or
prescribtions
Wells 2000 Individual Verbal A supervisor Moderate Moderate Not clear Moderate
or senior
colleauge
Winicoff 1984 Individual Written Not clear Moderate Moderate Peer Moderate
comparison,
information
about costs,
numbers
of tests
ordered or
prescribtions
Winicoff 1985 Individual Written Not clear Infrequent Prolonged Compliance Moderate
with
guideline,Peer
comparison
Wienkens Individual Written A supervisor Infrequent Prolonged Compliance Intensive
colleauge
Wones 1987 Individual Written Not clear Moderate Moderate Peer Moderate
comparison,
information
about costs,
numbers
of tests
ordered or
prescribtions
Zwar 1999 Individual Written The Once only Once only Compliance Moderate
Overall
Table 03. Quality of included trials
Conceal- Baseline Reliable

ment of Follow-up Follow-up Blinded ass measure- prim Prot of
Study allo prof pat prim out ment outcom contamin Summary
Anderson DONE DONE NA NOT NOT DONE DONE MODER-
1994 CLEAR DONE ATE
Anderson NOT NOT NA DONE DONE NOT NOT LOW
1996 DONE DONE CLEAR CLEAR
Baker 1997 DONE DONE NA NOT DONE DONE DONE MODER-
DONE ATE
Balas 1998 DONE DONE NOT DONE NOT NOT NOT MODER-
CLEAR CLEAR CLEAR CLEAR ATE
Belcher DONE NOT NOT DONE NOT DONE NOT MODER-
1990 CLEAR DONE CLEAR CLEAR ATE
Berman NOT NOT NA NOT NOT NOT NOT LOW
1999 CLEAR CLEAR CLEAR DONE CLEAR CLEAR
Boekeloo DONE NOT NA NOT NOT NOT NOT LOW
1990 CLEAR CLEAR DONE CLEAR CLEAR
Bonevski NOT DONE NA DONE NOT NOT NOT MODER-
1999 CLEAR DONE CLEAR CLEAR ATE
Borgiel DONE DONE NA NOT DONE NOT NOT MODER-
1999 CLEAR CLEAR CLEAR ATE
Brady 1988 DONE DONE NA NOT NOT NOT NOT MODER-
CLEAR DONE CLEAR CLEAR ATE
Brown 1994 DONE DONE NA NOT NOT NOT DONE MODER-
CLEAR DONE CLEAR ATE
Buffington DONE DONE NA NOT NOT NOT NOT MODER-
1991 DONE DONE DONE CLEAR ATE
Buntinx NOT DONE NA DONE DONE NOT NOT MODER-
1993 CLEAR DONE CLEAR ATE
Chassin DONE NOT NA NOT NOT DONE DONE MODER-
Cohen DONE NOT DONE NOT DONE NOT DONE MODER-
De Almeida NOT DONE NA NOT DONE NOT DONE MODER-
Neto 2000 CLEAR CLEAR CLEAR ATE
Dickinson DONE NOT DONE NOT NOT NOT DONE MODER-

Table 03. Quality of included trials (Continued )

1981 CLEAR CLEAR CLEAR CLEAR ATE
Everett NOT DONE NA NOT NOT NOT NOT LOW

1983 CLEAR CLEAR CLEAR CLEAR CLEAR
Fairbrother NOT DONE NA DONE NOT NOT DONE MODER-
Feder 1995 NOT DONE NA DONE NOT NOT DONE MODER-
CLEAR CLEAR DONE ATE
Gama 1991 NOT DONE NA NOT NOT NOT NOT LOW
CLEAR CLEAR DONE CLEAR CLEAR
Gehlbach DONE NOT NA DONE DONE DONE NOT MODER-
1984 DONE CLEAR ATE
Goldberg DONE DONE NA NOT NOT NOT DONE MODER-
Grady 1997 DONE DONE DONE NOT DONE NOT DONE MODER-
CLEAR CLEAR ATE
Guillion NOT NOT NOT DONE DONE DONE NOT MODER-
Hemminiki DONE DONE NA DONE NOT NOT DONE MODER-
1992 DONE CLEAR ATE
Henderson NOT NOT NOT NOT NOT NOT NOT LOW
CLEAR CLEAR CLEAR CLEAR DONE CLEAR CLEAR
Hendryx DONE DONE NA DONE DONE DONE DONE HIGH
1998
Hershey NOT NOT NA DONE DONE DONE DONE MODER-
1986 CLEAR CLEAR ATE
Hillman NOT DONE NA NOT DONE NOT DONE MODER-
Hillman DONE DONE NA DONE DONE DONE DONE HIGH
1999
Holm 1990 NOT DONE NA NOT DONE NOT DONE MODER-
CLEAR CLEAR CLEAR ATE
Howe 1996 NOT DONE NA NOT DONE NOT NOT MODER-
Hux 1999 NOT NOT NA DONE DONE DONE DONE MODER-
CLEAR DONE ATE
Jones 1996 NOT DONE NA NOT DONE NOT DONE MODER-
Kafuko DONE DONE NA NOT DONE NOT DONE MODER-


CLEAR CLEAR ATE
Kerry 2000 DONE NOT NA DONE NOT DONE DONE MODER-

CLEAR CLEAR ATE
Kerse 1999 DONE DONE DONE DONE DONE NOT DONE HIGH
CLEAR
Kim 1999 NOT DONE NOT DONE DONE NOT NOT MODER-
Kinsinger DONE DONE NA DONE DONE NOT DONE HIGH
1998 CLEAR
Leviton DONE DONE NOT DONE DONE NOT DONE HIGH
1999 CLEAR CLEAR
Linn 1980 DONE DONE NA NOT NOT NOT DONE MODER-
CLEAR DONE CLEAR ATE
Lobach NOT DONE NA DONE DONE NOT DONE MODER-
Lomas 1991 DONE DONE NA NOT DONE DONE DONE HIGH
CLEAR
Mainous NOT DONE NOT DONE NOT NOT DONE MODER-
Manfredi DONE DONE NA DONE NOT NOT DONE HIGH
1998 CLEAR CLEAR
Manheim NOT NOT NA NOT NOT NOT NOT LOW
1990 CLEAR CLEAR CLEAR CLEAR CLEAR CLEAR
Martin NOT NOT NA NOT DONE NOT NOT MODER-
1980 CLEAR CLEAR CLEAR CLEAR CLEAR ATE
Marton NOT DONE NA NOT DONE NOT NOT MODER-
Mayefsky NOT DONE NA NOT DONE DONE NOT MODER-
Mayer 1998 DONE NOT NA DONE DONE NOT DONE MODER-
DONE CLEAR ATE
McAlister DONE DONE NOT 3 DONE NOT NOT MODER-
Mc Cartney DONE NOT NOT DONE DONE NOT DONE MODER-
McConnell DONE DONE NA DONE NOT NOT NOT MODER-
DONE CLEAR CLEAR ATE
Meyer 1991 DONE NA DONE DONE DONE DONE NOT MODER-


DONE ATE
Moongtui DONE NOT NA NOT NOT DONE DONE MODER-

Norton NOT DONE NA DONE NOT NOT NOT MODER-
OConnell DONE NOT NA DONE DONE DONE DONE HIGH
1999 CLEAR
Palmer DONE DONE NA NOT NOT DONE DONE MODER-
Raasch DONE DONE NA DONE NOT NOT NOT MODER-
2000 DONE DONE CLEAR ATE
Reid 1977 NOT NOT NA DONE NOT NOT NOT LOW
CLEAR CLEAR CLEAR CLEAR CLEAR
Roski DONE NOT NOT NOT NOT NOT DONE MODER-
CLEAR DONE CLEAR CLEAR CLEAR ATE
Runangkan- DONE NOT NA DONE DONE NOT NOT MODER-
chasnastr CLEAR CLEAR CLEAR ATE
1993
Rust 1999 NOT NOT NA NOT DONE NOT NOT LOW
CLEAR CLEAR DONE CLEAR CLEAR
Sanazaro NOT NOT NOT NOT NOT DONE DONE LOW
1978 DONE CLEAR CLEAR CLEAR CLEAR
Sandbaek NOT DONE NA NOT NOT NOT NOT LOW
1999 CLEAR DONE DONE CLEAR CLEAR
Scheetman NOT DONE NA DONE NOT DONE NOT MODER-
Simon 2000 DONE NA DONE DONE DONE NOT DONE MODER-
CLEAR ATE
Sinclair NOT DONE NA DONE NOT NOT DONE MODER-
Smith 1995 DONE NOT NOT DONE DONE NOT DONE MODER-
Smith 1998 NOT NOT NA NOT NOT NOT NOT MODER-
CLEAR DONE CLEAR DONE CLEAR CLEAR ATE
Socolar DONE NOT NA DONE DONE DONE DONE MODER-
1998 DONE ATE
Sommers NOT NOT NA NOT DONE NOT NOT LOW
1984 CLEAR DONE CLEAR CLEAR CLEAR

Thompson DONE DONE NA DONE NOT DONE DONE HIGH

2000 CLEAR
Tierney NOT NOT NA DONE NOT DONE NOT MODER-

van der NOT DONE NA NOT NOT NOT DONE MODER-

Homberg CLEAR CLEAR CLEAR CLEAR ATE
1999
van der DONE DONE NA DONE NOT NOT DONE MODER-

Weijden CLEAR CLEAR ATE
Veninga NOT NOT NOT DONE DONE NOT DONE MODER-

Vinicor NOT NOT NOT NOT NOT NOT NOT LOW

1987 CLEAR CLEAR DONE DONE CLEAR CLEAR CLEAR
Ward 1996 NOT NOT DONE DONE NOT NOT NOT MODER-
CLEAR CLEAR DONE CLEAR CLEAR ATE
Wells 2000 DONE DONE DONE NOT DONE NOT DONE HIGH
CLEAR CLEAR
Winicoff NOT DONE NA DONE DONE DONE NOT MODER-

1984 CLEAR DONE ATE
Winicoff NOT NOT NOT DONE DONE DONE NOT MODER-

Wienkens NOT DONE NA NOT NOT NOT NOT LOW

1995 CLEAR DONE CLEAR CLEAR CLEAR
Wones 1987 NOT DONE NA DONE NOT DONE NOT MODER-

Zwar 1999 NOT DONE NA NOT DONE NOT NOT MODER-

CLEAR DONE CLEAR CLEAR ATE
ANALYSES
Comparison 01. Audit and feedback versus no intervention
No. of No. of
Outcome title studies participants Statistical method Effect size
01 Dichotomous outcomes Other data No numeric data

02 Continuous outcomes Other data No numeric data
Comparison 02. Short term effects of audit and feedback compared to longer-term effects after feedback stops
No. of No. of
01 All outcomes Other data No numeric data
Comparison 03. Audit and feedback + complementary interventions versus audit and feedback alone
No. of No. of
Comparison 04. Audit and feedback versus other interventions
No. of No. of
Comparison 05. Comparisons of different ways of providing audit and feedback
No. of No. of
Comparison 06. Studies in which patients were randomised
No. of No. of
Comparison 07. High quality studies
No. of No. of
01 Dichotomous outcomes Other data No numeric data
INDEX TERMS
Medical Subject Headings (MeSH)
Education, Medical, Continuing; Feedback; Health Personnel [standards]; Health Services Research; Medical Audit; ∗ Outcome As-
sessment (Health Care); Physician’s Practice Patterns [∗ standards]; Professional Practice [∗ standards]
MeSH check words
Humans
COVER SHEET
Title Audit and feedback: effects on professional practice and health care outcomes
Authors Jamtvedt G, Young JM, Kristoffersen DT, Thomson O’Brien MA, Oxman AD
Contribution of author(s) GJ, JY and ADO prepared the protocol. GJ and JY applied the inclusion criteria, assessed
the quality and extracted the data for the included studies. DTK conducted the quantitative
analyses. GJ, JY, and ADO conducted the qualitative analyses. GJ drafted the manuscript
with input from JY and ADO. DTK and MAO provided comments on the manuscript.
Cynthia Fraser conducted searches for the literature. MAO and ADO prepared the protocol
for the first review and together with Nick Freemantle and Emma Harvey applied the
inclusion criteria, assessed the quality and extracted the data for the included studies for the
first version of this review.
Issue protocol first published 1996/3
Review first published 1998/1
Date of most recent amendment 27 August 2003
Date of most recent 28 May 2003
SUBSTANTIVE amendment
What’s New Two previous Cochrane reviews have been merged. The total number of included studies
has increased from 37 to 85. The main findings of this review are consistent with the
previous conclusion, that the effectiveness of audit and feedback varies. When it is effective,
it generally has small to moderate effects. The single factor that we identified that predicts
when audit and feedback is most likely to be effective is baseline adherence to recommended
practice.
Date new studies sought but Information not supplied by author
none found
Date new studies found but not 30 November 2002

yet included/excluded
Date new studies found and 28 May 2003

included/excluded
Date authors’ conclusions 28 May 2003

section amended
Contact address Gro Jamtvedt
Director
Departement for Research Dissemination and Support
Norwegian Directorate for Health and Social Welfare
Pb. 8054 Dep
Oslo
N-0031
NORWAY
E-mail: grj@shdir.no
Tel: +47 24 16 32 70
Fax: +47 24 16 30 03
DOI 10.1002/14651858.CD000259
Cochrane Library number CD000259
Editorial group Cochrane Effective Practice and Organisation of Care Group
Editorial group code HM-EPOC
GRAPHS AND OTHER TABLES
Figure 01. Adjusted relative risk versus baseline non-compliance
Figure 02. Adjusted risk difference versus baseline non-compliance
Figure 03. Adjusted relative risk versus intervention type
Analysis 01.01. Comparison 01 Audit and feedback versus no intervention, Outcome 01 Dichotomous
outcomes
Audit and feedback alone compared to no intervention

N health RD: RR: Baseline Study Intensity of
Study Outcome professiona Adj/Unadj Adj/Unadj compliance quality A&F Complexity
Balas 1998 patients on 10 - - - moderate moderate high

peritoneal 0.13 0.87
dialysis
Bonevski accuracy 19 0.07 0.75 0.71 moderate moderate moderate
1999 of classifi- 0.08 0.70
cation of
patient risk
status for
preventive
care
Buffington % 30 - - - moderate moderate low
1991 vaccinated 0.16 0.68
Buntinx quality of 88 0.00 1.00 0.85 moderate moderate moderate
1993a cervical 0.05 0.75
smears
group 1 vs
group 3
Buntinx quality of 88 -0.01 1.05 0.81 moderate moderate moderate
1993b cervical 0.00 1.00
smears
group 1 vs
group 4
Dickinson pts with 20 0.07 1.08 0.41 moderate moderate moderate
1981 controlled -0.10 1.53
BT
Fairbrother immu- 30 0.02 0.97 0.31 moderate moderate moderate
1999a nization -0.01 1.02
coverage
Gehlbach prescription 31 0.32 0.57 0.21 moderate moderate low
1984 rate for 0.35 0.55
generic
drug-
feedback
drug
Hemminiki cesarean 130 0.01 0.99 0.14 moderate low moderate
1992 section rate 0.01 0.99
Hillman compliance 28 0.11 0.75 0.49 moderate moderate moderate
1999 with 0.03 0.91
guidelines
for pediatric
preventive
care
Howe 1996 detection 19 0.10 0.82 0.43 moderate moderate low
rate for 0.08 0.86
depression
Hux 1999 first line 250 0.05 0.87 0.67 moderate moderate moderate
antibiotics 0.03 0.90
episodes
Lobach compliance 45 - 0.29 - moderate moderate moderate

1996 rate for - 0.69
diabetes
quidelines
Mainous antibiotic 111 - 0.09 1.20 0.29 moderate moderate moderate

2000 prescribing - 0.11 1.24
rates
Mayefsky compliance 28 0.08 0.75 0.71 moderate moderate moderate

1993 for criteria 0.07 0.77
for child
care
McCartney pat with 28 0.09 0.82 0.48 moderate low moderate

1997 aspirin 0.08 0.84
Moongtui compliance 91 0.20 0.59 0.49 moderate moderate low

2000 rate for 0.07 0.78
handwash
and glove
use
O’Connell prescrib- 2440 0.02 0.92 0.78 high moderate moderate

1999 tions 0.002 0.99
Raasch % correct 46 0.10 0.81 0.45 moderate moderate moderate

2000 clinical 0.04 0.93
diagnosis
Schectman % of H2 63 -0.04 1.05 0.23 moderate low low

1995 blockers 0.00 1.00
that are
cimeti-
dine/prescribed
Tierney % pts who 135 - - - moderate moderate moderate

1986 received 0.07 0.92
prev care
according to
guidelines
Winickoff %check-ups 16 0.05 0.82 0.66 moderate moderate moderate

1984 (test done) 0.03 0.86
including
BT
Audit and feedback + educational meetings versus no intervention
Brown 1994 recording of 24 0.06 0.91 0.23 moderate high moderate
periodontal 0.03 0.96
care
De Almeida misuse of 22 0.24 0.73 0.06 moderate moderate moderate
Neto 2000 medication 0.27 0.70
Goldberg compliance 41 0.01 0.99 0.47 moderate moderate moderate
1998 with 0.01 0.99
guidelines
for manage-
ment of hy-
pertension
Hendryx total unclear 0.15 0.63 0.59 high moderate high
1998 processes of 0.10 0.74
care
Kafuko cases treated 127 0.30 0.60 0.24 moderate low moderate
1999 according to 0.32 0.59
guidelines
Kim 1999 preventive 48 0.06 0.74 0.73 moderate high moderate
services 0.03 0.83
Lomas 1991 compliance 76 - - - high moderate moderate
with - 0.02 1.02
guidelines
for man-
agement of
women with
a previous
cesarean
section
McConnell did not 33 0.47 0.50 0.0 moderate moderate low
1982 prescribe 0.47 0.50
tetracycline
Veninga Compliance 181 0.06 0.93 0.48 moderate moderate moderate
1999a with 0.03 0.92
guidelines
for asthma -
Netherlands
1999b with 0.02 0.93
guidelines
for asthma -
Sweden
1999c with 0.02 0.97
guidelines
for asthma -
Audit and feedback + educational meetings versus no intervention (Continued )

Norway
1999d with 0.05 0.97
guidelines
for asthma -
Slovakia
Multifaceted interventions that include audit and feedback versus no intervention

Buffington influenza 32 - - - moderate moderate low
1991 vaccination 0.17 0.66
rates
Fairbrother immu- 30 0.11 0.83 0.29 moderate moderate moderate
1999a nization 0.06 0.90
coverage
group 2 vs
group 4
Fairbrother immu- 30 -0.02 1.04 0.46 moderate moderate moderate
1999b nization 0.09 0.85
coverage
group 3 vs
group 4
Feder 1995 preventive 39 0.10 0.82 0.34 moderate low moderate
services 0.12 0.80
Grady 1997 mammogra- 23 0.05 0.91 0.23 moderate moderate moderate
phy referrals 0.07 0.89
Hillman compliance 26 -0.001 0.96 0.27 moderate moderate moderate
1998 scores 0.06 0.88
Hillman compliance 34 0.06 0.86 0.54 high moderate moderate
1999 with 0.02 0.93
guidelines
Kafuko cases treated unclear 0.46 0.47 0.12 moderate low moderate
1999 according to 0.36 0.53
guideline
Kerse 1999 pt recall of 42 - - - moderate low moderate
discussion 0.13 0.84
about
exercise
Leviton use of 14 0.14 0.70 0.33 high moderate moderate
1999 antenatal 0.12 0.72
corticos-
teroids
Multifaceted interventions that include audit and feedback versus no intervention (Continued )

Mainous antibiotic 114 -0.09 1.18 0.35 moderate moderate moderate
2000 prescribing - 0.05 1.11
rates
Manfredi breast 170 0.07 0.91 0.30 high moderate moderate
1998 cancer 0.08 0.90
screening
Mayer 1998 skin cancer 138 0.71 0.31 0 moderate moderate moderate
prevention 0.64 0.34
advice
McAlister % pts with 25 - - 0.89 moderate moderate moderate
1986 dbp < 0.01 0.89
90mmHg
Roski 1998 smoking 10 0.12 0.79 0.57 moderate low moderate
advice 0.15 0.75
Thompson case finding 179 0.08 0.92 0.03 high low moderate
2000 for domestic 0.08 0.92
violence
Tierney % pts who 69 - - - moderate moderate moderate
1986 received 0.15 0.82
preventive
care
according to
guidelines
Wells 2000 overall 0.09 0.85 0.34 high moderate moderate
appropriate 0.10 0.84
care
Analysis 01.02. Comparison 01 Audit and feedback versus no intervention, Outcome 02 Continuous
outcomes
Audit and feedback alone compared to no intervention

N health Unit of Intensity of Complexity
Study Outcome professiona analysis err Size of effect Study quality A&F of target
Chassin 1986 rate of 1483 yes large positive moderate moderate moderate
pelvimetry
Cohen 1982 costs lab test unclear yes no difference moderate moderate moderate
and x-ray
Hershey 1986 charge per 48 yes no difference moderate moderate high
patient
Holm 1990 prescription 238 yes no difference moderate moderate moderate
of hypnotics
Hux 1999 antibiotic 250 small positive moderate moderate moderate

Audit and feedback alone compared to no intervention (Continued )
N health Unit of Intensity of Complexity

Study Outcome professiona analysis err Size of effect Study quality A&F of target
costs
Kerry 2000 referrals of x- 175 yes no difference moderate moderate moderate

rays
Marton number of 28 yes moderate moderate moderate moderate

1985a tests pr visit positive
group 1 vs
group 4
Marton pr visit 28 yes moderate moderate moderate moderate

1985b group 1 vs positive
group 3
Norton 1985a compliance 6 yes small positive moderate moderate moderate

for guidelines
with cystitis
Norton compliance 6 yes no difference moderate moderate moderate

1985b for guidelines
with vaginitis
Ruangkan- lab tests 10 yes no difference moderate moderate intensive

chanastr a residents 1
Ruangkan- lab tests 8 yes no difference moderate moderate intensive

chanastr b residents 2
Ruangkan- lab tests 18 yes positive moderate moderate intensive

chanastr c consultants
Smith 1998 drug use 188 no moderately moderate moderate moderate

for sedative positive
hypnotic
medications
Socolar 1998 change in 147 no no difference moderate moderate low

documenta-
tion score for
child sexual
abuse
Wones 1987a laboratory test 14 yes no difference moderate moderate moderate

group 1 vs
group 3
Wones 1987b laboratory test 14 yes no difference moderate moderate moderate

group 2 vs
group 3
Audit and feedback + educational meetings versus no intervention
N health Unit of Intensity of A Complexity
Study Outcome professiona analysis err Size of effect Study quality & F of target
Goldberg compliance 41 yes no difference moderate moderate moderate
1998 with
guidelines for
hypertension
and
depression
Gullion diastolic 54 yes no difference moderate moderate moderate
1988a blood pressure
group 1 vs 4
1988b blood pressure
group 2 vs 4
1988c blood pressure
group 3 vs 4
Hendryx quality of unclear yes small positive high moderate high
1998 intensive care
unit care
Kafuko 1999 drugs unclear yes small positive moderate low moderate
prescribed
Linn BS 1980 deviations 298 yes no difference moderate moderate moderate
from
algorithm in
burn care
Martin 1980 laboratory 16 yes small positive moderate high moderate
tests
Palmer 1985 various case 711 no small positive moderate moderate moderate
variant scores
Sinclair 1982 overall quality unclear yes moderate moderate moderate moderate
score positive
Smith 1995 information 24 no moderate moderate moderate moderate
giving positive
Zwar 1999 antibiotic 157 no no difference moderate moderate moderate
prescribing
Multifaceted interventions that include audit and feedback versus no intervention

Kafuko 1999 drug unclear yes small positive moderate low moderate
prescribed
van der compliance 20 no no difference moderate moderate moderate
Weijden 1999 with
guidelines for
Multifaceted interventions that include audit and feedback versus no intervention (Continued )

cholesterol
Analysis 02.01. Comparison 02 Short term effects of audit and feedback compared to longer-term effects
after feedback stops, Outcome 01 All outcomes
All outcomes
N of health Risk diff: RR: Size of Study Intensity Complexity
Study Outcome prof Adj/unadj Adj/unadj effect quality og A&F of behavi
Buntinx % 88 - 0.01 1.05 Moderate Moderate Moderate

1993a vaccinated 0.00 1.00
Buntinx quality of 88 -0.01 1.05 Moderate Moderate Moderate

1993b smears -0.01 1.00
Cohen mean # of unclear small Moderate Moderate Moderate

1982 lab tests positive
Fairbrother immunisa- 30 0.07 0.88 Moderate Moderate Moderate

1999a tion status 0.03 0.95
Jones 1996 capillary 124 no Moderate Moderate Low

blood difference
glucose
monitoring
Norton compliance 6 0.01 0.99 Moderate Moderate Moderate

1985a with 0.04 0.95
guidelines
for vaginitis
Norton compliance 6 0.32 0.48 Moderate Moderate Moderate

1985b with 0.29 0.53
guidelines
for cystitis
Ruangkan- mean lab 18 no Moderate High Moderate

chanastr a tests difference
Smith 1995 communi- 24 small Moderate Moderate Moderate

cation skills positive
Zwar 1999 antibiotic 157 no Moderate Moderate Moderate

prescribing difference
Analysis 03.01. Comparison 03 Audit and feedback + complementary interventions versus audit and
feedback alone, Outcome 01 All outcomes
All outcomes
N health RD; RR; Size of Study Intensity of Complexity
Study Outcome prof adj/unadj adj/unadj effect quality A&F of behavi
Baker 1997 compliance unclear -0.11 1.2 moderate low moderate
with -0.16 1.3
guidelines
for use of
benzodi-
azepines
Boekeloo manage- 29 0.09 0.89 moderate moderate moderate
1990 ment of 0.05 0.93
cholesterol
Borgiel quality of 56 0.01 0.99 moderate moderate moderate
1999 care 0.02 0.98 low
Buffington patients 45 - - moderate moderate low
1991 vaccinated - 0.01 1.03
Dickinson patients unclear 0.05 1.10 moderate moderate moderate
1981 with - 0.09 1.45
controlled
blood
pressure
Fairbrother immuniza- 61 -0.09 1.16 moderate moderate moderate
1999a tion status - 0.07 1.13
group 1 vs
group 2
Fairbrother immuniza- 61 0.04 0.94 moderate moderate moderate
1999b tion status - 0.10 1.20
group 1 vs
group 3
Hillman compliance 34 -0.05 1.13 high moderate moderate
1999 with -0.01 1.03
guidlines for
preventive
child care
Kinsinger breast 58 0.08 0.88 high low moderate
1998 cancer 0.06 0.91
screening
Mainous antibiotic 216 0.01 1.01 moderate moderate moderate
2000 prescribing - 0.06 1.12
Tierney compliance 64 - - moderate moderate moderate
1986 with -0.08 1.11
guidelines
for
preventive
care
Analysis 04.01. Comparison 04 Audit and feedback versus other interventions, Outcome 01 All outcomes
All outcomes
N health RD; RR; Size of Study Intensity
Study Outcome prof adj/unadj adj/unadj effect quality og A&F Complexity
Boekeloo manage- 29 -0.18 1.26 low moderate moderate
1990 ment of -0.13 1.19
cholesterol
Lomas 1991 compliance 76 - - high moderate moderate
with 0.16
guidelines
for man-
agement of
women with
a previous
cesarean
section
Mainous antibiotics 108 0.01 0.98 moderate moderate moderate
2000 prescribing 0.15 1.12
Smith 1995 perfor- 22 moderate moderate moderate
mence of no
presenting difference
screening
tests
Tierney compliance 65 - - moderate moderate moderate
1986 0.08 1.11
Analysis 05.01. Comparison 05 Comparisons of different ways of providing audit and feedback, Outcome 01
All outcomes
All outcomes
N of health RD; RR; Size of Study Intensity of
Study Outcome prof adj/unadj adj/unadj effect quality A&F Complexity
Gullion diastolic 55 no moderate moderate moderate
1988a blood difference
pressure, 1
vs 2
Kafuko drug unclear 0.28 0.35 moderate low moderate
1999 prescribed 0.06 0.87
Ward 1996 compliance 80 moderate moderate moderate
with small
guidelines positive
Wones laboratory 14 moderate moderate moderate
1987a tests no
difference
Wones laboratory 14 moderate moderate moderate
1987b tests no
All outcomes (Continued )
N of health RD; RR; Size of Study Intensity of

difference
van den indicators 90 no moderate moderate high
Hombergh of practice difference
99 manage-
ment
Analysis 06.01. Comparison 06 Studies in which patients were randomised, Outcome 01 All outcomes
All outcomes
N health RD; RR; Size of Study Intensity of
Belcher % patients 1224 small moderate low moderate
1990 on patients positive
preventive unclear # of
services health pro-
fessionals
Meyer 1991 number of no low moderate moderate
prescrib- difference
tions
Simon 2000 costs, 613 patients no moderate moderate moderate
frequence of difference
follow-up
visits
Analysis 07.01. Comparison 07 High quality studies, Outcome 01 Dichotomous outcomes
Dichotomous outcomes
N of health RD; RR; Baseline Study Intensity of
Study Outcome prof adj/unadj adj/unadj compliance quality A&F Complexity
Hendryx compliance unclear 0.15 0.61 0.59 high moderate high
1998 with 0.10 0.74
intensive
care unit
guidelines
(A&F + ed
meeting)
Hillman compliance unclear 0.06 0.81 0.54 high moderate moderate
1999 with 0.02 0.93
guidelines
for child
preventive
care
(MF)
Dichotomous outcomes (Continued )
N of health RD; RR; Baseline Study Intensity of

Study Outcome prof adj/unadj adj/unadj compliance quality A&F Complexity
Kerse 1999 pt recall of 42 - - - high low moderate
discussion 0.13 0.14
about
exercise
Kinsinger breast 58 0.08 0.88 0.28 high low moderate
1998 cancer 0.06 0.91
screening
Leviton use of unclear 0.14 0.61 0.70 high moderate moderate
1999 antenatal 0.12 0.72
corticos-
teroids
(MF)
Lomas 1991 compliance 76 - - - high moderate moderate
with -0.02 1.02
guidelines
for man-
agement of
women with
a previous
cesarean
section
Manfredi compliance 170 0.08 0.91 0.30 high moderate moderate
1998 with 0.07 0.90
guidelines
for cancer
screening
O’Connell prescribing 2440 0.02 1.06 0.30 high moderate moderate
1999 rate 0.001 0.90
Thompson case finding 179 0.08 0.92 0.03 high low moderate
2000 for domestic 0.08 0.92
violence
(MF)
Wells 2000 overall 181 0.09 0.85 0.34 high moderate moderate
appropriate 0.10 0.84
care for
depression
(MF)

Jamtvedt2003 SR PDF

Uploaded by

Copyright:

Available Formats

Jamtvedt2003 SR PDF

Uploaded by

Document Information

Original Title

Copyright

Available Formats

Share this document

Share or Embed Document

Sharing Options

Did you find this document useful?

Is this content inappropriate?

Copyright:

Available Formats

Jamtvedt2003 SR PDF

Uploaded by

Copyright:

Available Formats

Audit and feedback: effects on professional practice and

health care outcomes (Review)

Jamtvedt G, Young JM, Kristoffersen DT, Thomson O’Brien MA, Oxman AD

Jamtvedt G, Young JM, Kristoffersen DT, Thomson O’Brien MA, Oxman AD

This record should be cited as:

This version first published online: 21 July 2003 in Issue 3, 2003.

PLAIN LANGUAGE SUMMARY

Buntinx and colleagues (Buntinx 1993) conducted a systematic

One study that compared feedback on medication with feedback

Everett Hetlevik I, Holmen J, Krüger Ø, Kristensen P, Iversen H. Imple-

Moher 2001 Davis 1992

Characteristics of included studies

Study Anderson 1994

Study Anderson 1996

Study Baker 1997

Study Belcher 1990

Study Berman 1998

Study Boekeloo 1990

Study Bonevski 1999

Study Brady 1988

Study Brown 1994

Study Buntinx 1993

Study Buntinx 1993a

Study Buntinx 1993b

Study Chassin 1986

Study Cohen 1982

Study De Almeida Neto 2000

Study Dickinson 1981

Study Everett 1983

Study Fairbrother 1999

Study Fairbrother 1999a

Study Fairbrother 1999b

Study Feder 1995

Study Gama 1991

Study Gehlbach 1984

Study Goldberg 1998

Study Grady 1997

Study Gullion 1988

Study Gullion 1988a

Study Gullion 1988b

Study Gullion 1988c

Study Hemminiki 1992

Study Henderson 1979

Study Hendryx 1998

Study Hershey 1986

Study Hillman 1998

Study Hillman 1999

Study Holm 1990

Study Howe 1996

Study Hux 1999

Study Jones 1996

Study Kerry 2000

Study Kerse 1999

Study Kim 1999

Study Kinsinger 1998

Study Leviton 1999

Study Linn BS 1980

Study Linn BS, 1980

Study Lomas 1991