Journal of Clinical Epidemiology - (2017) -

ORIGINAL ARTICLE

Further exploration of dissemination bias in qualitative research required

to facilitate assessment within qualitative evidence syntheses
Ingrid Toewsa, Andrew Boothb, Rigmor C. Bergc, Simon Lewinc,d, Claire Glentonc, Heather
M. Munthe-Kaasc, Jane Noyese, Sara Schroterf, Joerg J. Meerpohla,g,*
a
Cochrane Germany, Medical Center, Faculty of Medicine, University of Freiburg, Breisacher Str. 153, 79110 Freiburg, Germany
b
School of Health and Related Research, University of Sheffield, Regent Court, 30 Regent Street, Sheffield S1 4DA, UK
c
Norwegian Institute of Public Health, Pilestredet Park 7, N-0176 Oslo, Norway
d
Health Systems Research Unit, South African Medical Research Council, PO Box 19070, Tygerberg 7505, South Africa
e
School of Social Sciences, Bangor University, Bangor, Gwynedd ll57 2DG, UK
f
BMJ, Tavistock Square, London, WC1H 9JR
g

Cochrane France, Centre de Recherche Epid
emiologie et Statistique Sorbonne Paris Cit
e e U1153, Inserm/Universit
e Paris Descartes, H^opital H^otel-Dieu,
1 place du Parvis Notre Dame, 75181 Paris Cedex 04, France
Accepted 12 April 2017; Published online xxxx

Abstract
Objectives: To conceptualise and discuss dissemination bias in qualitative research.
Results: It is likely that the mechanisms leading to dissemination bias in quantitative research, including time lag, language, gray liter-
ature, and truncation bias also contribute to dissemination bias in qualitative research. These conceptual considerations have informed the
development of a research agenda.
Conclusion: Further exploration of dissemination bias in qualitative research is needed, including the extent of non-dissemination and
related dissemination bias, and how to assess dissemination bias within qualitative evidence syntheses. We also need to consider the mech-
anisms through which dissemination bias in qualitative research could occur to explore approaches for reducing it. Ó 2017 Elsevier Inc.
All rights reserved.

Keywords: Dissemination bias; Publication bias; Qualitative research; Qualitative evidence syntheses; Systematic review; Nondissemination

1. Qualitative research in health and social care: what feasibility of interventions, as well as implementation fidel-
is it used for? ity [1]. Qualitative research can also explore how and why
interventions, and different intervention components, might
Qualitative research aims to understand people’s experi-
lead to specific outcomes and contribute to theory develop-
ences and perspectives and can influence how health care
ment and the creation of explanatory hypotheses. Findings
and social interventions are conceptualized, developed,
from qualitative research can inform decisions on the use of
and implemented. Qualitative research is well suited to evidence-based health and social care interventions and
understanding factors that affect the acceptability and contribute to policy decisions in these fields. Decision
makers in health and social care are therefore increasingly
Conflict of interest: None. using qualitative evidence alongside other forms of evi-
Funding: This work was supported in part by funding from the Alliance dence to inform decisions [2e6].
for Health Policy and Systems Research (www.who.int/alliancehpsr/en/)
(Grant number: SPHQ15-APW-1510), WHO, Norad (Norwegian Agency
for Development Cooperation: www.norad.no), and the Research Council 1.1. Qualitative evidence synthesis
of Norway (www.forskningsradet.no) (Grant number: 237757/H10). S.L. is
supported by funding from the South African Medical Research Council Qualitative evidence is increasingly brought together in
(www.mrc.ac.za). The funders had no role in preparation of the manuscript qualitative evidence syntheses [7]. Qualitative evidence
or the decision to publish.
* Corresponding author. Tel.: þ49-761-2035715; fax: þ49-761- syntheses provide an overview of people’s views, perspec-
2039672850. tives, and experiences of a particular phenomenon. A quali-
E-mail address: meerpohl@cochrane.de (J.J. Meerpohl). tative evidence synthesis analyses and further interprets
http://dx.doi.org/10.1016/j.jclinepi.2017.04.010
0895-4356/Ó 2017 Elsevier Inc. All rights reserved.
2 I. Toews et al. / Journal of Clinical Epidemiology - (2017) -

for interventions that a guideline recommends (see Box

What is new? 1) [11].
Accordingly, systematic review organizations such as
Key findings Cochrane, NICE Public Health Guidelines, the EPPI Centre,
 Evidence on dissemination bias in qualitative Joanna Briggs, and UK funders such as the National Institute
research is scarce. for Health Research, increasingly value syntheses of qualita-
tive health and social care research [3]. A challenge to using
 Plausible biases that might affect the full dissemi-
evidence from qualitative research, however, has been
nation of qualitative studies include time-lag bias,
assessing and communicating how much confidence deci-
language bias, gray literature bias, and truncation
sion makers should have in the review findings.
bias.

What this study adds to what was known? 1.2. Assessing confidence in findings from qualitative
 Given the paucity of literature on dissemination evidence syntheses
bias in qualitative research, several subbiases are
proposed to help conceptualize dissemination bias The Grading of Recommendations Assessment, Develop-
in qualitative research. ment and Evaluation (GRADE) approach was originally
designed to assess how much confidence to place in
 Based on conceptual considerations, a research findings from reviews of quantitative studies of the
agenda has been developed. effectiveness of interventions. The GRADE Working Group
has since expanded its remit and now includes approaches
What is the implication and what should change for assessing confidence in a range of different types of
now? evidence. The GRADE-CERQual (Confidence in the
 More evidence on the extent of dissemination bias Evidence from Reviews of Qualitative research: www.
in qualitative research and its effects is needed; and cerqual.org) approach was specifically developed for find-
we need to further explore the underlying mecha- ings from syntheses of qualitative evidence [15]. According
nisms of dissemination bias in qualitative research. to the GRADE-CERQual approach, review authors and/or

evidence from individual qualitative research studies Box 1 Example of how findings from a qualitative
addressing similar research questions or phenomena of in- evidence synthesis can inform
terest. There are over 20 methods of qualitative evidence understanding of the factors affecting
syntheses to select from and new guidance has been implementation of a health care
published on selecting the most appropriate method for a intervention
specific context [8]. Qualitative evidence syntheses are The benefit of clinical safety checklists for patient
designed to create new understanding of phenomena of in- safety has been demonstrated in a large, prospective
terest, generate theoretical and conceptual models, identify study [12], but the uptake of checklists in clinical
research gaps, and provide evidence for the development, practice is slow [13]. To find out why clinical
implementation, and evaluation of interventions. These checklists are not regularly and successfully used in
syntheses can be used when developing fields of research, clinical settings, Bergs et al. [14] synthesized 18
for instance by contributing to empirical generalizations qualitative studies in a qualitative evidence
[9]. They can also be used to complement systematic re- synthesis aiming to identify the barriers and
views of quantitative evidence as part of clinical and health facilitators to implementing clinical checklists. The
system decision-making processes. For instance, qualitative evidence suggests that staff’s perceptions of
evidence syntheses are increasingly used in the develop- checklists play a major role, with some staff being
ment of clinical and health system guidelines [6,10]. Here, reluctant to use a checklist because they doubt its
they can help define the scope of the guideline, including evidence base. Staff’s perceptions of patient safety
detailing the populations, interventions, comparisons, and also influenced the use of checklists: for example,
outcomes on which each guideline question should focus nurses would not read out checklist items that
[11]. They can help assess the acceptability of the interven- might cause distress to patients. Finally, workflow
tion to key stakeholders as well as the intervention’s adjustments, such as changing the workflow of the
feasibility [11]. They can also ascertain how different involved staff, were identified as a barrier to
stakeholders and population groups value different out- implementing clinical checklists. The authors also
comes and help ensure that the voices of important and highlighted aspects which could improve the use
sometimes underrepresented groups of people are heard. and success of clinical safety checklists.
Finally, they can identify implementation considerations
 I. Toews et al. / Journal of Clinical Epidemiology - (2017) - 3

end users may have less confidence in a review finding if qualitative research. However, we are aware that some
there are concerns regarding: methodological limitations of journals are less likely to publish qualitative research than
the studies contributing to the review finding, relevance of quantitative research and that journals might not have
the included studies to the review question, coherence of specific publication guidelines and policies for qualitative
the review finding, or adequacy of data supporting the research. This in turn may contribute to nondissemination
review finding. In the development of this approach, or incomplete dissemination of qualitative studies [19].
however, there has also been much discussion on the degree Given that these mechanisms would affect any qualitative
to which dissemination bias might influence our confidence study irrespective of the nature of its findings, we do not
in a review finding. An assessment of dissemination bias is consider these mechanisms as contributing to dissemina-
not currently part of the GRADE-CERQual approach in tion bias within qualitative research itself as there are a
recognition of the very limited empirical evidence on its high number of journals that readily publish qualitative
extent in qualitative research and its impact on findings of research.
qualitative evidence syntheses. In addition, we have little
knowledge on ways of detecting such bias. Further research
2.1. Dissemination bias in quantitative research: causes
is needed to establish the extent of nondissemination and
related dissemination bias in qualitative research, to deter- Although the systematic nondissemination of research
mine how dissemination bias can be identified, and to assess has been commonly referred to as publication bias, the term
its impact on findings from qualitative evidence syntheses. dissemination bias is becoming more commonly used as
In this paper, we conceptualize and discuss the issue this allows us to acknowledge the underlying mechanisms
of dissemination bias in qualitative research. Although ev- more comprehensively [20]. Three issues are particularly
idence on dissemination bias in qualitative research is relevant when discussing the term dissemination bias. First,
scarce, the phenomenon has been investigated intensively although scientific evidence is usually made available in
in the quantitative research environment. Our discourse journal publications, other dissemination channels such as
about the causes and consequences of dissemination bias study registries or online data repositories are becoming
in qualitative research was therefore informed by reflecting increasingly important [17]. Second, the term dissemina-
on the available evidence from the quantitative research tion bias describes both the nondissemination of an entire
arena. We will highlight how mechanisms that cause study (nonpublication) as well as the selective nondissemi-
dissemination bias in quantitative research might also play nation of individual results (selective reporting). In addition
a role in qualitative research. to selective reporting of results, for example, from individ-
ual participants of the study, selective outcome reporting
describes the nonreporting of findings related to entire
2. What is dissemination bias?
outcomes. The most dominant mechanisms underlying the
Nondissemination and irretrievability of studies is first selective dissemination of quantitative studies and results,
and foremost unethical and a waste of resources [16]. In and resulting in dissemination bias, are described in
the case of systematic reviews of quantitative studies with Table 1 [17].
meta-analyses, such nondissemination might lead to inade- Third, dissemination bias also covers the practice of
quacy of data, which, in turn, might lead to imprecision of duplicate publication. Duplicate publication is an aspect
pooled effect estimates. Where nondissemination is system- of dissemination that describes the practice of producing
atic rather than randomdin other words, if disseminated multiple publications reporting the same findings from a
studies and findings differ systematically from nondissemi- single study [24]. Outcomes and results might, sometimes
nated studies and findingsdthis will distort review findings unreasonably, be split up into several reports. Duplicate
and cause dissemination bias. publications are not always clearly discernible and might
Dissemination bias therefore describes the systematic be confused for reports of different studies. As a conse-
error that occurs from nondissemination of studies and quence, the same study results might be included multiple
findings. The key underlying concern is the (non-) dissem- times in meta-analyses and thus bias the overall effect
ination of studies due to the nature of their content and estimate.
message [17]. In the context of this paper, we discuss
dissemination bias resulting from the nondissemination 2.2. Nondissemination and dissemination bias in
of studies and findings due to their content. We do not
quantitative research: prevalence and impact
use the term to describe the effects of the nondissemina-
tion of studies and findings due to other factors, such as Clear empirical evidence demonstrates that a large
the study’s design or the population under investigation, proportion of quantitative studies in clinical research
etc. For example, the extent to which journals decide to remain unpublished after completion [28e30]. For
publish qualitative research in general and the editorial example, in a systematic review of methodological research
policies that apply are not our primary area of interest projects including randomized controlled trials and other
[18] nor do we categorize this as dissemination bias in interventional and observational studies from general
4 I. Toews et al. / Journal of Clinical Epidemiology - (2017) -

Table 1. Biases and underlying mechanisms identified to play a role in quantitative research that might influence selective reporting of studies and
findings in qualitative research
Impact on systematic reviews Impact on evidence syntheses
Bias Description Causes of quantitative results of qualitative findings
Time-lag Striking findings are published Authors might pursue the Relevant and new results of Evidence syntheses might
bias sooner after completion of a publication of certain no, little, or even harmful be lacking up to date,
study than less noteworthy findings more vigorously so effects might not be relevant studies that
results [21] that more striking findings or available at a given point report a wider variety of
findings supporting a popular in time [22] findings
view are published sooner;
editors might prioritize the
publication of findings that
they consider more
newsworthy
Language Striking findings from a study Expressing small nuances of Studies in languages other Studies in languages other
bias might be more likely to be speech and language that than English are harder to than English are harder to
published in the English might add to the correct identify and retrieve [24] retrieve and identify, and
language in an international understanding of the therefore, some findings
journal. This, in turn, might phenomenon of interest is may be less represented
increase the retrievability more challenging for in evidence syntheses.
and accessibility of these researchers who are not It is prohibitively expensive
findings, compared to those writing in their first language to translate and back
that were seen as less [23]; less striking findings translate the study to
striking are probably more likely to ensure that conceptual
be published in journals meaning is not lost in
publishing in the native translation.
language and national
context of the researchers,
for which the reports are
more difficult to access
Gray literature Increased publication of less Limitations on article length Studies showing less striking Gray literature is not
bias noteworthy study findings can be overly restrictive for results are not indexed in indexed in major
[21] in outlets other than the full reporting of major scientific databases scientific databases and
peer-reviewed journals qualitative research; many and harder to be retrieved harder to retrieve for
researchers publish their and included in systematic evidence syntheses.
findings in reports, on reviews
websites and social media,
and in newsletters [23];
qualitative research is
frequently conducted
outside of an academic
context and published
routinely in organizational
gray literature reports. Small
effect sizes are more likely to
be published in gray
literature [25]
Truncation Studies that are published in The artificial word limit of Studies published in outlets Literature searches confined
bias outlets such as reports, scientific journals is often with strict manuscript word to journal articles may
books, theses, and too restrictive for the full limits, such as in scientific lead to ‘‘truncation’’ bias
dissertations might be more reporting of qualitative journals, might contain as the full details and
likely to report fuller findings research [23]; researchers incomplete reporting of findings of a qualitative
than those where an arbitrary often choose to use books findings study may not be
word limit is prescribed [26] and reports as a medium for published in a journal
communication as these article
allow longer articles and a
wider variety of formats than
journal articles [27]

medicine, different medical specialties, and epidemiology, for clinical, and political, decision-making [25]. Dissemina-
Schmucker et al. [31] found that only half of all studies tion bias was repeatedly found to lead to an overestimation
(46.2%) approved by research ethics committees were of the reported effects of health interventions because
published. The bias resulting from such nondissemination statistically significant and positive results had an increased
has been found to have consequences for the evidence base probability of getting published [16,31,32]. For example,
 I. Toews et al. / Journal of Clinical Epidemiology - (2017) - 5

the drug reboxetine was approved and consequently used as the same mechanisms lead to dissemination bias in qualita-
a safe and effective treatment for depression. A recent sys- tive research. Table 1 presents a description of how time
tematic review [33], however, revealed that the beneficial lag, language, gray literature, and truncation bias may
effect of reboxetine was based on selected patient data. occur in qualitative research and impact on qualitative
Data for 74% of the patients were not published in the evidence syntheses. Additional factors, observed in quanti-
primary studies. The review authors repeated the analyses tative research, which may also lead to dissemination bias
with published and unpublished primary data and found in qualitative research include findings that oppose current
that reboxetine was not more effective than placebo and beliefs, findings that may be viewed as unpopular by
caused more adverse events. This demonstrates that opinion leaders, findings that are discordant with the stance
dissemination bias is a threat to decisions in health and of those funding the research, and findings that have cost or
health care and consequently to the health and safety of other implications that are not seen as feasible [17].
individuals.
2.5. Nondissemination and dissemination bias in
2.3. Nondissemination and dissemination bias in qualitative research: empirical evidence
qualitative research
To date, very few studies on nondissemination and
We have previously defined dissemination bias in qualita- dissemination bias in qualitative research have been
tive research as a systematic distortion of the phenomenon of conducted, and more generally, meta-research on qualita-
interest due to selective dissemination of studies or findings tive research is rare. This scarcity of research on dissemina-
of studies [15]. tion bias may be a consequence of the relative novelty of
Although little empirical research is available on either qualitative evidence synthesis when compared to its
the extent of nondissemination of qualitative research or quantitative counterparts and highlights the need for more
on the extent of dissemination bias in this domain, it is very research to investigate the issue comprehensively. The
likely to be present. In clinical effectiveness research, the research priorities outlined below focus on nondissemina-
most common concern about dissemination effects is that tion of qualitative research as a first step in exploring the
the benefits or harms of a clinical intervention will be issue of dissemination bias. This research will also
overestimated or underestimated [25,31]. This distinction contribute to developing a broader research agenda on
between ‘‘positive’’ and ‘‘negative’’ findings is unhelpful dissemination bias in qualitative research.
in qualitative research which focuses on the varying views One of the few studies on this topic followed a cohort of
and experiences of participants regarding a health issue or 224 qualitative studies presented at a single medical sociol-
intervention and not on the direction of the overall effect. ogy conference to assess what proportion of these studies
Dissemination bias in qualitative research therefore cannot remained unpublished in the following 2 years [34]. The
be articulated within a discourse of outcome, but rather study searched for subsequent publication of the studies
needs to be viewed in relation to the complete and accurate in relevant databases and by contacting the study authors.
representation of the phenomenon of interest. Conse- They found that less than half (44%) of the studies had been
quently, it is challenging to explore whether particular types published up to 7 years after publication. Reporting quality
of content or types of findings or conclusions from qualita- in the abstracts was positively related to the subsequent
tive studies are more or less likely to be published. publication of the study. The authors concluded that the
In qualitative evidence syntheses, omission of data may extent of nonpublication of qualitative studies is similar
result in the loss of a particular perspective altogether or to that for quantitative studies.
may lead to a less nuanced interpretation of the phenome- A second study, an explorative cross-sectional survey
non. As a consequence, we may place more confidence in a of authors of qualitative studies, peer reviewers, and
finding than we should or a synthesis may be limited by the editors of scientific journals, demonstrated that nondisse-
omission of findings. Decision-making might therefore be mination in qualitative research is substantial and that
hampered by an incomplete evidence base or flawed assess- several stakeholder groups play an important role in the
ments of confidence in the evidence. However, because we ‘‘nondissemination’’ pathway [23]. Nondissemination,
are only now starting to explore dissemination bias in and the dissemination bias that may result, was not seen
qualitative research, we can only speculate about its by participants as merely a theoretical problem but was
consequences for the body of qualitative evidence and for seen as having important impacts on health and social
decision-making. care research, practice, and policy. Over half of
researchers reported that one or more of their qualitative
studies had not been published in a peer-reviewed journal
2.4. Dissemination bias in qualitative research: possible
(62%) or in another publicly accessible format (52%).
causes and consequences
Around one-third reported that important individual
Based on what is known about dissemination bias in findings were missing in one or more of their published
quantitative research, it may be reasonable to assume that reports.
6 I. Toews et al. / Journal of Clinical Epidemiology - (2017) -

3. Research priorities [3] Gulmezoglu AM, Chandler J, Shepperd S, Pantoja T. Reviews of

qualitative evidence: a new milestone for Cochrane. Cochrane Data-
The increasing use of qualitative research findings in base Syst Rev 2013;11:ED000073. ED000073.
clinical guidelines and health and social care decision- [4] Lewin S, Bosch-Capblanch X, Oliver S, Akl EA, Vist GE, Lavis JN,
et al. Guidance for evidence-informed policies about health systems:
making emphasizes the need to explore further the extent
assessing how much confidence to place in the research evidence.
and implications of nondissemination, and related dissem- PLoS Med 2012;9:e1001187.
ination bias, in qualitative research. As a starting point, [5] Kane GA, Wood VA, Barlow J. Parenting programmes: a systematic
we need to develop an evidence-informed taxonomy of review and synthesis of qualitative research. Child Care Health Dev
the different routes through which dissemination bias 2007;33:784e93.
[6] Glenton C, Lewin S, Gulmezoglu AM. Expanding the evidence base
may arise in the context of qualitative studies. A compre-
for global recommendations on health systems: strengths and chal-
hensive mapping review that can inform this taxonomy is lenges of the OptimizeMNH guidance process. Implement Sci
currently prepared. It describes the quantity and charac- 2016;11:98.
teristics of papers reporting nondissemination and dissem- [7] Hannes K, Macaitis K. A move to more systematic and transparent
ination bias in qualitative research and sets out to describe approaches in qualitative evidence synthesis: update on a review of
published papers. Qual Res 2012;12:402e42.
and categorize the mechanisms that contribute to dissem-
[8] Booth A, Noyes J, Flemming K, Gerhardus A, Wahlster P, van der
ination bias. Further studies are also needed across a range Wilt GJ, et al. Guidance on choosing qualitative evidence synthesis
of qualitative research domains, including different disci- methods for use in health technology assessments of complex inter-
plines (sociology, anthropology, etc.) and areas of ventions [Online] 2016. Available at http://wwwintegrate-htaeu/
research (health systems research, social welfare research, downloads/. Accessed October 18, 2016.
[9] Riese H, Carlsen B, Glenton C. Qualitative research synthesis: how
etc.), of the conversion rate of funded projects, abstracts,
the whole can be greater than the sum of parts. Antropology 2014;
or submissions into publicly accessible dissemination for- 21:22e30.
mats. In addition, we need to explore the causes of [10] Tan TP, Stokes T, Shaw EJ. Use of qualitative research as evidence
dissemination bias in qualitative researchdwhy some in the clinical guideline program of the National Institute for
qualitative studies or findings are not published or Health and Clinical Excellence. Int J Evid Based Healthc 2009;7:
169e72.
disseminateddand find feasible ways for decreasing and
[11] Glenton C, Lewin S, Norris S. Using evidence from qualitative
preventing dissemination bias. research to develop WHO guidelines (chapter 15). In: Norris S, edi-
Distortions in the results of reviews of quantitative tor. World health Organization Handbook for Guideline Development
effectiveness evidence can, under certain circumstances, (2nd edition). 2nd ed. Geneva, Switzerland: WHO: World Health Or-
be detected and adjusted for by statistical methods, so ganization; 2016.
[12] Haynes AB, Weiser TG, Berry WR, Lipsitz SR, Breizat AH,
increasing the validity of the overall estimate of effect
Dellinger EP, et al. A surgical safety checklist to reduce morbidity
[35]. Currently, no established methods or guidance exist and mortality in a global population. N Engl J Med 2009;360:
on how to assess whether, and to what extent, dissemination 491e9.
bias might be present in the findings of qualitative evidence [13] Korkiakangas T. Mobilising a team for the WHO Surgical Safety
syntheses. We also lack guidance on what precautions can Checklist: a qualitative video study. BMJ Qual Saf 2017;26:
177e88.
be taken when interpreting the findings of these syntheses.
[14] Bergs J, Lambrechts F, Simons P, Vlayen A, Marneffe W, Hellings J,
Further research on these questions is needed, as we discuss et al. Barriers and facilitators related to the implementation of surgi-
elsewhere [36]. People interested in this topic are cal safety checklists: a systematic review of the qualitative evidence.
encouraged to join the GRADE-CERQual DissQuS BMJ Qual Saf 2015;24:776e86.
(Dissemination bias in Qualitative Synthesis) subgroup [15] Lewin S, Glenton C, Munthe-Kaas H, Carlsen B, Colvin CJ,
G€ulmezoglu M, et al. Using qualitative evidence in decision making
and contribute to taking forward this area of research.
for health and social interventions: an approach to assess confidence
in findings from qualitative evidence syntheses (GRADE-CERQual).
PLoS Med 2015;12:e1001895.
Acknowledgments [16] Chan A-W, Song F, Vickers A, Jefferson T, Dickersin K,
Gøtzsche PC, et al. Increasing value and reducing waste: addressing
Our thanks for their feedback to those who participated inaccessible research. Lancet 2014;383:257e66.
in the GRADE-CERQual Project Group meetings in [17] Rothstein HR, Sutton AJ, Borenstein M. Publication bias in meta-
January 2014 and/or June 2015. analysis. Publication bias in meta-analysis. Chichester, UK: John Wi-
ley & Sons, Ltd; 2006:1e7.
[18] Loder E, Groves T, Schroter S, Merino JG, Weber W. Qualitative
References research and the BMJ. BMJ 2016;352:i641.
[19] Greenhalgh T, Annandale E, Ashcroft R, Barlow J, Black N,
[1] Noyes J, Popay J, Pearson A, Hannes K, Booth A. Qualitative Bleakley A, et al. An open letter to the BMJ editors on qualitative
research and Cochrane reviews. In: Higgins J, Green S, editors. research. BMJ 2016;352:i563.
Cochrane Handbook for Systematic Reviews of Interventions. Ox- [20] Meerpohl JJ, Schell LK, Bassler D, Gallus S, Kleijnen J,
ford, UK: The Cochrane Collaboration; 2011. Kulig M, et al. Evidence-informed recommendations to reduce
[2] Bohren MA, Hunter EC, Munthe-Kaas HM, Souza JP, Vogel JP, dissemination bias in clinical research: conclusions from the
Gulmezoglu AM. Facilitators and barriers to facility-based delivery OPEN (Overcome failure to Publish nEgative fiNdings) project
in low- and middle-income countries: a qualitative evidence synthe- based on an international consensus meeting. BMJ open 2015;
sis. Reprod Health 2014;11:71. 5:e006666.
 I. Toews et al. / Journal of Clinical Epidemiology - (2017) - 7

[21] Hopewell S, Clarke M, Stewart L, Tierney J. Time to publication for [29] Baudart M, Ravaud P, Baron G, Dechartres A, Haneef R, Boutron I.
results of clinical trials. Cochrane Database Syst Rev 2007; Public availability of results of observational studies evaluating an
MR000011. intervention registered at ClinicalTrials.go. BMC Med 2016;14:1e11.
[22] Tanner-Smith EE, Polanin JR. A retrospective analysis of dissemina- [30] Malicki M, Marusic A, OPEN Consortium. Is there a solution to
tion biases in the brief alcohol intervention literature. Psychol Addict publication bias? Researchers call for changes in dissemination of
Behav 2015;29:49e62. clinical research results. J Clin Epidemiol 2014;67:1103e10.
[23] Toews I, Glenton C, Lewin S, Berg RC, Noyes J, Booth A, et al. [31] Schmucker C, Schell LK, Portalupi S, Oeller P, Cabrera L, Bassler D,
Extent, awareness and perception of dissemination bias in qualitative et al. Extent of non-publication in cohorts of studies approved by
research: an explorative survey. PLoS One 2016;11:e0159290. research ethics committees or included in trial registries. PLoS One
[24] International Committee of Medical Journal Editors. Overlapping 2014;9:e114023.
Publications, 2016. Available at http://icmje.org/recommendations/ [32] Franco A, Malhotra N, Simonovits G. Publication bias in the social
browse/publishing-and-editorial-issues/overlapping-publications.html. sciences: unlocking the file drawer. Science 2014;345:1502e5.
Accessed October 18, 2016. [33] Eyding D, Lelgemann M, Grouven U, Harter M, Kromp M, Kaiser T,
[25] Song F, Parekh S, Hooper L, Loke YK, Ryder J, Sutton AJ, et al. et al. Reboxetine for acute treatment of major depression: systematic
Dissemination and publication of research findings: an updated re- review and meta-analysis of published and unpublished placebo and
view of related biases. Health Technol Assess 2010;14. iii, ix-xi, selective serotonin reuptake inhibitor controlled trials. BMJ 2010;
1e193. 341:c4737.
[26] Dwan K, Altman DG, Cresswell L, Blundell M, Gamble CL, [34] Petticrew M, Egan M, Thomson H, Hamilton V, Kunkler R,
Williamson PR. Comparison of protocols and registry entries to pub- Roberts H. Publication bias in qualitative research: what becomes
lished reports for randomised controlled trials. Cochrane Database of qualitative research presented at conferences? J Epidemiol Com-
Syst Rev 2011;MR000031. munity Health 2008;62:552e4.
[27] Campbell R, Pound P, Morgan M, Daker-White G, Britten N, [35] Sterne J, Egger M, Moher D. Chapter 10: Addressing reporting biases.
Pill R, et al. Evaluating meta-ethnography: systematic analysis Cochrane Handbook for Systematic Reviews of Intervention Version
and synthesis of qualitative research. Health Technol Assess 510. (updated March 2011). Oxford, UK: The Cochrane Collaboration;
2011;15:1e164. 2011.
[28] Blumle A, Meerpohl JJ, Schumacher M, von Elm E. Fate of clinical [36] Booth A, Lewin S, Glenton C, Munthe-Kaas H, Meerpohl JJ,
research studies after ethical approvaldfollow-up of study protocols Noyes J, et al. Applying the GRADE-CERQual approach (7): under-
until publication. PLoS One 2014;9:e87184. standing the potential impacts of dissemination bias 2016.