BMC Medical Research

Methodology BioMed Central

Research article Open Access

Development of AMSTAR: a measurement tool to assess the
methodological quality of systematic reviews
Beverley J Shea*1,5, Jeremy M Grimshaw†2, George A Wells3, Maarten Boers†4,
Neil Andersson5, Candyce Hamel†5, Ashley C Porter5, Peter Tugwell2,
David Moher6 and Lex M Bouter†1

Address: 1EMGO Institute, VU University Medical Center, Amsterdam, the Netherlands, 2Institute of Population Health, Ottawa, Ontario, Canada,
3Department of Epidemiology and Community Medicine, University of Ottawa, Ottawa, Ontario, Canada, 4Department of Clinical Epidemiology

and Biostatistics. VU University Medical Center, Amsterdam, the Netherlands, 5Community Information and Epidemiological Technologies
(CIETcanada), Ottawa, Ontario, Canada and 6University of Ottawa, Ottawa, Ontario, Canada
Email: Beverley J Shea* - bshea@ciet.org; Jeremy M Grimshaw - jgrimshaw@ohri.ca; George A Wells - gawells@ottawaheart.ca;
Maarten Boers - mboers@vumc.nl; Neil Andersson - neil@ciet.org; Candyce Hamel - chamel@uottawa.ca;
Ashley C Porter - ashleyclaireporter@hotmail.com; Peter Tugwell - ptugwell@uottawa.ca; David Moher - dmoher@uottawa.ca;
Lex M Bouter - lm.bouter@dienst.vu.nl
* Corresponding author †Equal contributors

Published: 15 February 2007 Received: 10 August 2006

Accepted: 15 February 2007
BMC Medical Research Methodology 2007, 7:10 doi:10.1186/1471-2288-7-10
This article is available from: http://www.biomedcentral.com/1471-2288/7/10
© 2007 Shea et al; licensee BioMed Central Ltd.
This is an Open Access article distributed under the terms of the Creative Commons Attribution License (http://creativecommons.org/licenses/by/2.0),
which permits unrestricted use, distribution, and reproduction in any medium, provided the original work is properly cited.

Abstract
Background: Our objective was to develop an instrument to assess the methodological quality of
systematic reviews, building upon previous tools, empirical evidence and expert consensus.
Methods: A 37-item assessment tool was formed by combining 1) the enhanced Overview Quality
Assessment Questionnaire (OQAQ), 2) a checklist created by Sacks, and 3) three additional items
recently judged to be of methodological importance. This tool was applied to 99 paper-based and
52 electronic systematic reviews. Exploratory factor analysis was used to identify underlying
components. The results were considered by methodological experts using a nominal group
technique aimed at item reduction and design of an assessment tool with face and content validity.
Results: The factor analysis identified 11 components. From each component, one item was
selected by the nominal group. The resulting instrument was judged to have face and content
validity.
Conclusion: A measurement tool for the 'assessment of multiple systematic reviews' (AMSTAR)
was developed. The tool consists of 11 items and has good face and content validity for measuring
the methodological quality of systematic reviews. Additional studies are needed with a focus on the
reproducibility and construct validity of AMSTAR, before strong recommendations can be made
on its use.

Background read an average of 17–20 original articles every day [1].

It has been estimated that healthcare professionals Increasingly, systematic reviews are being advocated as a
attempting to keep abreast of their field would need to way to keep up with current medical literature [2]. A well

Page 1 of 7
(page number not for citation purposes)
Luiza Maria Machado De Moura - luizamaria0011@gmail.com - CPF: 138.512.006-16
 BMC Medical Research Methodology 2007, 7:10 http://www.biomedcentral.com/1471-2288/7/10

conducted systematic review addresses a carefully formu- in systematic reviews remains controversial. Some studies
lated question by analyzing all available evidence. It have suggested that systematic reviews that include only
employs an objective search of the literature, applying English language publications tend to overestimate effect
predetermined inclusion and exclusion criteria to the lit- sizes [10], whereas other studies suggest that such lan-
erature, critically appraising what is found to be relevant. guage restrictions may not do so [11]. An item was added
It then extracts and synthesizes data from the available to determine whether a language restriction was applied
evidence base to formulate findings [3]. in selecting studies for the systematic review. 2) Publica-
tion bias: Publication bias refers to the tendency for
However, in spite of the care with which they are con- research with negative findings to get published less fre-
ducted, systematic reviews may differ in quality, and yield quently, less prominently, or more slowly, and the ten-
different answers to the same question [4]. As a result, dency for research with positive findings to get published
users of systematic reviews should be critical and look more than once. Publication bias has been identified as a
carefully at the methodological quality of the available major threat to the validity of systematic reviews. Empiri-
reviews [5]. cal research suggests that publication bias is widespread,
and that a variety of methods are now available to assess
A decade has passed since the initial development of tools publication bias [12-19]. An item was added to determine
to assess the quality of systematic reviews, such as those whether the authors assessed the likelihood of publica-
created by Oxman and Guyatt [6] and Sacks [7]. There are tion bias. 3) Publication status of studies suggests that pub-
now more than 24 instruments available to assess the lished trials are generally larger and may show an overall
quality of systematic reviews [8]. Nevertheless, the major- greater treatment effect than studies published in the 'grey'
ity of the available instruments are not widely used. Sev- literature [20]. The importance of including grey literature
eral are lengthy and include complicated instructions for in all systematic reviews has been discussed [21]. The
their use. Furthermore, since their development, consider- assessment of the inclusion of grey literature considers
able empirical research has accumulated about potential whether or not the authors reported searching for grey lit-
sources of bias in systematic reviews. For example, recent erature.
methodological research has highlighted the potential
importance of publication language and publication bias Objective 1
in systematic reviews [9-11]. The 37-item assessment tool was used to appraise 99
paper-based reviews identified from a database of reviews
Therefore, our goal was to develop a new instrument for and meta-analyses [22] and 52 Cochrane systematic
assessing the methodological quality of systematic reviews from the Cochrane Database of Systematic
reviews by building upon empirical data collected with Reviews [9]. After the list of selected systematic reviews
previously developed tools and utilizing expert opinion. was generated, full copies of these were retrieved, copied,
and masked to conceal author, institution, and journal.
This goal was pursued by two study objectives. Our first Reviews in languages other than English (i.e., French, Ger-
objective was to assess a large sample of systematic man, and Portuguese) were translated into English with
reviews using an item pool drawn from two available the assistance of colleagues before masking [23].
instruments used to assess methodological quality, sup-
plemented by additional items judged to be needed on For each included systematic review, two reviewers inde-
the basis of recent publications. We used exploratory fac- pendently assessed the methodological quality with the
tor analysis to identify the underlying component struc- 37 items (CH, BS).
ture. Our second objective was to build on the results of
this factor analysis, by using experts in a nominal group Statistical analyses and graphs displaying the results
technique (NGT) to reduce the items pool and to decide obtained were produced using SPSS version 13.0 for Win-
on a new assessment tool with face and content validity. dows. The 37 items were subjected to principal compo-
nents analysis, and Varimax rotations were used to rotate
Methods the components. Items with low factor loadings of < 0.50
We designed a 37-item assessment tool that we developed were removed.
by combining items from two available instruments: the
enhanced Overview Quality Assessment Questionnaire Objective 2
(OQAQ) [8] containing 10 items and a checklist created We convened an international panel of eleven experts in
by Sacks [7] containing 24 items. We supplemented this the fields of methodological quality assessment and sys-
with three additional items based upon methodological tematic reviews. The group was selected from three organ-
advances in the field since the development of the original izations involved both in the conduct of systematic
two instruments: Language restriction: Language restriction reviews and in the assessment of methodological quality.

Page 2 of 7
(page number not for citation purposes)
Luiza Maria Machado De Moura - luizamaria0011@gmail.com - CPF: 138.512.006-16
 BMC Medical Research Methodology 2007, 7:10 http://www.biomedcentral.com/1471-2288/7/10

The group was made up of clinicians, methodologists and the methodological quality of systematic reviews, by
epidemiologists, and reviewers who were new to the field. building upon empirical data on previously developed
Some individuals were previously involved in the tools, empirical evidence and utilizing expert opinion.
Cochrane Collaboration, while a number were not. By
examining the results of the factor analysis, they reflected Because we had already created a dataset of 151 systematic
critically on the components identified and decided on reviews assessed using 37 completed items for each
the items to be included in the new instrument. The nom- review, we were able to conduct a factor analysis as the
inal group process took place in San Francisco during a first step in the creation of the new tool. A more com-
one day session. monly used approach would have been to harvest appro-
priate items from existing questionnaires. This method
We conducted the following NGT in order to achieve has been used extensively in the development of instru-
agreement. After delivery of an overview of the project and ments for assessing the quality of both randomized and
the planned process for the day, the panel reviewed the non-randomized studies of health care interventions [24-
results of the factor analysis. The aim of the NGT was to 26]. The disadvantage of harvesting appropriate items
structure interaction within the group. Firstly, each partic- from existing questionnaires is that it relies heavily on the
ipant was asked to record his or her ideas independently validation of the source questionnaires [27]. Conducting
and privately. The ideas were then listed in a round-robin a factor analysis made it possible to determine whether
format. One idea was collected from each individual in the measured dimensions could in principle be assessed
turn and listed in front of the group by the facilitator, and using a smaller number of items.
the process was continued until all ideas had been listed.
Individuals then privately recorded their judgements. Traditionally, factor analysis is divided into two types of
Subsequent discussions took place. The individual judge- analyses: exploratory and confirmatory. As its name indi-
ments were aggregated statistically to derive the group cates, exploratory factor analysis aims to discover the
judgements. The nominal group was also asked to agree main constructs or dimensions of a concept by conduct-
on a final label for each of the 11 components. A descrip- ing a preliminary investigation of the correlations
tion was formulated for each of the items and a next-to- between all the identified variables. This process is also
final instrument was assembled. This was circulated elec- known as Principal Components Analysis (PCA). PCA has
tronically to the group for a final round of fine tuning. been recommended for use in test construction by Kline,
as a means of condensing the correlation matrix, rather
Results than as an aid to the interpretation of the factor-structure
Objective 1 of a questionnaire [28]. Items with low factor loadings
The items were subjected to factor analysis, and only those tend to be weakly correlated with other items, and there-
items that loaded highly on one component (>.50) were fore were removed. Various rotational strategies have also
retained. The described factor analysis made it possible to been proposed. The goal of all of them is to obtain a clear
reduce the 37-item instrument to a shorter (29-item) pattern of loadings, that is, factors that are somehow
instrument that measured 11 components (Table 1). clearly marked by high loadings for some variables and
low loadings for others [29,30]. We used this approach
Objective 2 because it is useful when a body of theory or principles
The nominal group discussed all 11 components (Table has been established, but has not yet been operationalised
1). The items most appropriate for the components (Table into an evaluative framework [31].
2), were included in the draft instrument [also see Addi-
tional file 1]. The instrument is an 11-item questionnaire The structured-discussion format employed in this project
that asks reviewers to answer yes, no, can't answer or not enabled all participants to contribute to the refining of the
applicable. A separate question on language was identi- assessment tool. The nominal technique followed
fied in the factor analysis as a significant issue, but the involved experts, discussion, and a consensus that was
nominal group felt that the contradictory evidence in the qualitative in nature. Consequently, it complemented the
literature warranted removing this item from the short- quantitative nature of factor analysis, and as a result the
ened item list and capturing it under the question on pub- final tool had face and content validity as judged by the
lication status. nominal consensus panel.

Discussion We recognize the need for further testing of AMSTAR.

Strengths and Weaknesses Additional studies are necessary with a focus on the repro-
Our purpose was to help users of systematic reviews to ducibility and construct validity of AMSTAR before strong
critically appraise systematic reviews. Therefore, we set out recommendations can be made on its use. The Canadian
with the goal of developing a new instrument for assessing Agency for Drugs and Technologies in Health (CADTH)

Page 3 of 7
(page number not for citation purposes)
Luiza Maria Machado De Moura - luizamaria0011@gmail.com - CPF: 138.512.006-16
 BMC Medical Research Methodology 2007, 7:10 http://www.biomedcentral.com/1471-2288/7/10

Table 1: Items identified through the factor analysis

Original instrument (item no) 1 2 3 4 5 6 7 8 9 10 11

1 Protocol Sacks .58

2 Literature Search Sacks .82
3 List of Trials Analyzed Sacks .75
4 Log of Rejected Trials Sacks .68
5 Treatment Assignment Sacks .80
6 Ranges of Patients Sacks
7 Range of Treatment Sacks .88
8 Range of Diagnosis Sacks .80
9 CombinabilityCriteria Sacks .88
10 Measurement Sacks .57
11 Selection Bias Sacks .85
12 Data abstraction Sacks .50
13 Inter-observer Agreement Sacks .65
14 Sources of Support Sacks .64
15 Statistical Methods Sacks .81
16. Statistical Errors Sacks
17 Confidence Intervals Sacks .73
18 Subgroup Analysis Sacks
19 Quality Assessment Sacks .77
20 Varying Methods Sacks .63
21 Publication Bias 1 Sacks .77
22 Caveats Sacks
23 Economic Impact Sacks .84
24 Language 1 Added to Sacks .79
25 Search Strategy OQAQ (1) .81
26 Was the search comprehensive OQAQ (2)
27 Criteria used for deciding which studies to OQAQ (3)
include
28 Was bias in the selection avoided OQAQ (4) .81
29 Were the criteria used for assessing the validity OQAQ (5) .75
reported?
30 Was the validity of all studies referred to in the OQAQ (6) .53 .60
text assessed using appropriate criteria
31 Were the methods used to combine the finding OQAQ (7)
of the relevant studies reported
32 Were the findings of the relevant studies OQAQ (8) .78
combined appropriately
33 Were the conclusions made by the author OQAQ (9) .68
supported by the data
34 Overall Summary OQAQ (10)
35. Publication Bias 2 Additional (1) .80
36 Publication Status Additional (2) .77
37 Language 2 Additional (3) .63

Page 4 of 7
(page number not for citation purposes)
Luiza Maria Machado De Moura - luizamaria0011@gmail.com - CPF: 138.512.006-16
 BMC Medical Research Methodology 2007, 7:10 http://www.biomedcentral.com/1471-2288/7/10

Table 2: AMSTAR is a measurement tool created to assess the methodological quality of systematic reviews.

1. Was an 'a priori' design provided? 䊐 Yes

The research question and inclusion criteria should be established before the conduct of the review. 䊐 No
䊐 Can't answer
䊐 Not applicable

2. Was there duplicate study selection and data extraction? 䊐 Yes

There should be at least two independent data extractors and a consensus procedure for disagreements should be in place. 䊐 No
䊐 Can't answer
䊐 Not applicable

3. Was a comprehensive literature search performed? 䊐 Yes

At least two electronic sources should be searched. The report must include years and databases used (e.g. Central, EMBASE, 䊐 No
and MEDLINE). Key words and/or MESH terms must be stated and where feasible the search strategy should be provided. All 䊐 Can't answer
searches should be supplemented by consulting current contents, reviews, textbooks, specialized registers, or experts in the 䊐 Not applicable
particular field of study, and by reviewing the references in the studies found.

4. Was the status of publication (i.e. grey literature) used as an inclusion criterion? 䊐 Yes
The authors should state that they searched for reports regardless of their publication type. The authors should state whether 䊐 No
or not they excluded any reports (from the systematic review), based on their publication status, language etc. 䊐 Can't answer
䊐 Not applicable

5. Was a list of studies (included and excluded) provided? 䊐 Yes

A list of included and excluded studies should be provided. 䊐 No
䊐 Can't answer
䊐 Not applicable

6. Were the characteristics of the included studies provided? 䊐 Yes

In an aggregated form such as a table, data from the original studies should be provided on the participants, interventions and 䊐 No
outcomes. The ranges of characteristics in all the studies analyzed e.g. age, race, sex, relevant socioeconomic data, disease 䊐 Can't answer
status, duration, severity, or other diseases should be reported. 䊐 Not applicable

7. Was the scientific quality of the included studies assessed and documented? 䊐 Yes
'A priori' methods of assessment should be provided (e.g., for effectiveness studies if the author(s) chose to include only 䊐 No
randomized, double-blind, placebo controlled studies, or allocation concealment as inclusion criteria); for other types of studies 䊐 Can't answer
alternative items will be relevant. 䊐 Not applicable

8. Was the scientific quality of the included studies used appropriately in formulating conclusions? 䊐 Yes
The results of the methodological rigor and scientific quality should be considered in the analysis and the conclusions of the 䊐 No
review, and explicitly stated in formulating recommendations. 䊐 Can't answer
䊐 Not applicable

9. Were the methods used to combine the findings of studies appropriate? 䊐 Yes
For the pooled results, a test should be done to ensure the studies were combinable, to assess their homogeneity (i.e. Chi- 䊐 No
squared test for homogeneity, I2). If heterogeneity exists a random effects model should be used and/or the clinical 䊐 Can't answer
appropriateness of combining should be taken into consideration (i.e. is it sensible to combine?). 䊐 Not applicable

10. Was the likelihood of publication bias assessed? 䊐 Yes

An assessment of publication bias should include a combination of graphical aids (e.g., funnel plot, other available tests) and/or 䊐 No
statistical tests (e.g., Egger regression test). 䊐 Can't answer
䊐 Not applicable

11. Was the conflict of interest stated? 䊐 Yes

Potential sources of support should be clearly acknowledged in both the systematic review and the included studies. 䊐 No
䊐 Can't answer
䊐 Not applicable

Page 5 of 7
(page number not for citation purposes)
Luiza Maria Machado De Moura - luizamaria0011@gmail.com - CPF: 138.512.006-16
 BMC Medical Research Methodology 2007, 7:10 http://www.biomedcentral.com/1471-2288/7/10

[32] undertook an independent assessment of available Additional material

quality assessment criteria for systematic reviews. Feed-
back from CADTH reviewers has been very positive. Fur-
ther preliminary experience suggests that AMSTAR has
Additional File 1
AMSTAR. AMSTAR is a measurement tool created to assess the method-
good reliability and convergent validity also suggesting ological quality of systematic reviews.
that appraisers can apply it in a consistent way. Click here for file
[http://www.biomedcentral.com/content/supplementary/1471-
AMSTAR, if used widely after external validation, could 2288-7-10-S1.doc]
also enable methodological research (i.e. meta-regression
of item of AMSTAR and effect size of reviews). Our instru-
ment is an attempt to achieve consensus amongst current
mainstream opinions. Inevitably, new evidence will mod- Acknowledgements
ify current thinking in some areas and at that point the The authors received no funding for this study. The authors would like to
thank the following members of the San Francisco Nominal Group for their
AMSTAR will be updated. This is indeed likely to be the
participation: Gail Kennedy, George Rutherford, George Wells, Peter Tug-
case with techniques to identify and quantify publication
well, Maarten Boers, Madhukar Pai, Kirby Lee, Maria Suarez-Almazor,
bias [33]. Although a number of alternative tests for pub- Kaveh Shojania, Tara Horvath, and Kathryn McDonald. Thank you to Vivian
lication bias exist, none has yet been validated [34]. Robinson, Adrienne Stevens, and David Henry for their helpful comments.

Publication bias remains an area of contention amongst The authors also thank Drs. Andy Oxman, Gordon Guyatt and Henry Sacks
those who assess the quality of systematic reviews. It for their methodological instruments and for providing additional informa-
remains a research priority because it is unclear what the tion.
impact of publication bias is on making decisions in
health care. We are aware of the 20 years of work that has References
1. Davidoff F, Haynes B, Sackett D, Smith R: Evidence-based medi-
gone in this area of research. This has given us some clear cine: a new journal to help doctors identify the information
answers as to the effect publication bias may have on the they need. BMJ 1995, 310:1085-6.
overall results of estimating the impact of interventions. 2. Lau J, Ioannidis JPA, Schmid CH: Summing up evidence: one
answer is not always enough. Lancet 1998, 351:123-127.
3. Systematic Review definition [http://www.nlm.nih.gov/nichsr/
AMSTAR will remain a living document and advances in hta101/ta101014.html]
4. Moher D, Soeken K, Sampson M, Campbell K, Ben Perot L, Berman
empirical methodological research will be reflected in fur- B: Assessing the quality of reports of systematic reviews in
ther improvements to the instrument. pediatric complementary and alternative medicine. BMC
Pediatr 2002, 2(2):.
5. Jadad A, Moher M, Browman G, Booker L, Sigouin C, Fuentes M: Sys-
Conclusion tematic reviews and meta-analyses on treatment of asthma:
A measurement tool for assessment of multiple systematic critical evaluation. BMJ 2000, 320:537-540.
6. Oxman AD: Checklists for review articles. BMJ 1994,
reviews (AMSTAR) was developed. The tool consists of 11 309:648-651.
items and has good face and content validity for measur- 7. Sacks H, Berrier J, Reitman D, Ancona-Berk VA, Chalmers TC: Meta-
ing the methodological quality of systematic reviews. analyses of randomized controlled trials. N Engl J Med 1987,
316(8):450-455.
Additional studies are needed with a focus on the repro- 8. Shea B, Dubé C, Moher D: Assessing the quality of reports of
ducibility and construct validity of AMSTAR, before strong systematic reviews: the QUOROM statement compared to
recommendations can be made on its use. other tools. In Systematic Reviews in Health Care: Meta-analysis in con-
text Edited by: Egger M, Smith GD, Altman DG. London: BMJ books;
2001:122-139.
Competing interests 9. The Cochrane Library. Volume 3. Chichester, UK: John Wiley &
Sons Ltd; 2004.
The author(s) declare that they have no competing inter- 10. Egger M, Zellweger-Zahner T, Schneider M, Junker C, Lengeler C,
ests. Antes G: Language bias in randomised controlled trials pub-
lished in English and German. Lancet 1997, 350:326-329.
11. Moher D, Pham B, Klassen T, Schulz K, Berlin J, Jadad A, Liberati A:
Authors' contributions What contributions do languages other than English make
BS designed and conducted the study and wrote the man- to the results of meta-analyses? Journal of Clinical Epidemiology
2000, 53:964-972.
uscript. MB, JG and LB participated in the design and 12. Pai M, McCulloch M, Colford J Jr, Bero LA: Assessment ofPublica-
coordination and assisted with writing the manuscript. tion Bias in Systematic Reviews on HIV/AIDS. [http://
CH and AP carried out the quality assessments and www.igh.org/Cochrane/pdfs/MSRI_workshop_talk_abstract.pdf].
13. Dickersin K: The existence of publication bias and risk factors
assisted with the writing. NA assisted in the conduct of the for its occurrence. JAMA 1990, 263:1385-1389.
study and commented on earlier drafts. GW assisted in the 14. Dickersin K: How important is publication bias? A synthesis of
design of the study and commented on the statistical anal- available data. AIDS Educ Prev 1997, 9:15-21.
15. Phillips C: Publication bias in situ. BMC Med Res Method 2004,
ysis. PT helped with the design of the study and assisted 4:20.
with the nominal group process. All authors read and 16. Pham B, Platt R, McAuley L, Sampson M, Klassen T, Moher D:
Detecting and minimizing publication bias. A systematic
approved the final manuscript.

Page 6 of 7
(page number not for citation purposes)
Luiza Maria Machado De Moura - luizamaria0011@gmail.com - CPF: 138.512.006-16
 BMC Medical Research Methodology 2007, 7:10 http://www.biomedcentral.com/1471-2288/7/10

review of methods. In Technical Report Thomas C. Chalmers Cen-

tre for Systematic Reviews, Ottawa, Canada; 2000.
17. Stern JM, Simes RJ: Publication bias: evidence of delayed publi-
cation in a cohort study of clinical research projects. BMJ
1997, 315:640-5.
18. Sterne JAC, Gavaghan D, Egger M: Publication and related bias in
meta-analysis – power of statistical tests and prevalence in
the literature. Journal of Clinical Epidemiology 2000, 53:1119-1129.
19. Sutton A, Duval S, Tweedie R, Abrams K, Jones D: Empirical
assessment of effect of publication bias on meta-analyses.
BMJ 2000, 320:1574-1577.
20. Hopewell S, McDonald S, Clarke M, Egger M: Grey literature in
meta-analyses of randomized trials of health care interven-
tions. In The Cochrane Library Volume 1. Chichester, UK: John Wiley
& Sons, Ltd; 2004.
21. McAuley L, Pham B, Tugwell P, Moher D: Does the inclusion of
grey literature influence the estimates of intervention effec-
tiveness reported in meta-analyses? Lancet 2000,
356:1228-1231.
22. Moher D, Cook DJ, Jadad AR, Tugwell P, Moher M, Jones A, Pham B,
Klassen TP: Assessing the quality of randomized controlledtri-
als: implications for the conduct of meta-analyses. Health
Technol Assess 1999, 3(12):i-iv. 1–98
23. Shea B: Assessing the quality of reporting meta-analyses of
randomized controlled trials. In MSc thesis University of Ottawa,
Department of Epidemiology and Community Medicine; 1999.
24. Downs S, Black N: The feasibility of creating a checklist for the
assessment of the methodological quality both of ran-
domised and non-randomised studies of health care inter-
ventions. J Epidemiol Community Health 1998, 52:377-384.
25. Jadad AR, Moore RA, Carroll D, Jenkinson C, Reynolds DJ, Gavaghan
DJ, McQuay HJ: Assessing the quality of randomized clinical
trials: is blinding necessary? Control Clin Trials 1996, 17:1-12.
26. Verhagen AP, de Vet HCW, de Bie RA, Kessels AGH, Boers M,
Bouter LM, Knipschild PG: The Delphi list: a consensus list for
quality assessment of randomized clinical trials for conduct-
ing systematic reviews developed by Delphi consensus. J Clin
Epid 1998, 51:1235-41.
27. Cluzeau FA: Development and application of an appraisal
instrument for assessing the methodological quality of clini-
cal practice guidelines in the United Kingdom. London, Eng-
land: St.George's Hospital Medical School, University of London;
2001.
28. Kline P: An Easy Guide to factor Analysis London: Routledge; 1994.
29. Norman GR, Streiner DL: Biostatistics: The Bare Essentials 2nd edition.
St. Louis: Mosby; 2000.
30. Streiner DL, Norman GR: Health Measurement Scales: A Practical Guide
to their Development and Use 3rd edition. UK: Oxford University Press;
2003.
31. McDowell I, Jenkinson C: Development standards for health
measures. Journal of health Services Research and Policy 1996,
1(4):238-246.
32. Canadian Agency for Drugs and Technologies in Health
[http://www.cadth.ca]
33. Lau J, Ioannidis JPA, Terrin N, Schmid CH, Olkin I: The case of the
misleading funnel plot. BMJ 2006, 333:597-600.
34. Rothstein HR, Sutton AJ, Borenstein M, eds: Publication bias in meta-
analysis: prevention, assessment and adjustments Sussex: John Wiley and
Sons; 2005.

Pre-publication history Publish with Bio Med Central and every

The pre-publication history for this paper can be accessed scientist can read your work free of charge
here: "BioMed Central will be the most significant development for
disseminating the results of biomedical researc h in our lifetime."

http://www.biomedcentral.com/1471-2288/7/10/prepub Sir Paul Nurse, Cancer Research UK

Your research papers will be:

available free of charge to the entire biomedical community
peer reviewed and published immediately upon acceptance
cited in PubMed and archived on PubMed Central
yours — you keep the copyright

Submit your manuscript here: BioMedcentral

http://www.biomedcentral.com/info/publishing_adv.asp

Page 7 of 7
(page number not for citation purposes)
Luiza Maria Machado De Moura - luizamaria0011@gmail.com - CPF: 138.512.006-16