Determining what works in the treatment of PTSD

Clinical Psychology Review 30 (2010) 923–933 Contents lists available at ScienceDirect Clinical Psychology Review Determining what works in the treatment of PTSD Bruce E. Wampold a,⁎, Zac E. Imel b, Kevin M. Laska a, Steven Benish a, Scott D. Miller c, Christoph Flűckiger a, Aaron C. Del Re a, Timothy P. Baardseth a, Stephanie Budge a a b c University of Wisconsin, Madison, United States VA Puget Sound Healthcare System and University of Washington, United States International Center for Clinical Excellence, United States a r t i c l e i n f o Article history: Received 29 January 2010 Received in revised form 17 June 2010 Accepted 17 June 2010 Keywords: Post traumatic stress disorder Clinical trials Common factors Trauma focused a b s t r a c t Many researchers accept that trauma-focused treatments are superior to non-trauma focused treatments for Post-Traumatic Stress Disorder (PTSD). However, Benish, Imel, and Wampold (2008) recently published a meta-analysis of clinical trials directly comparing ‘bona fide’ PTSD treatments that failed to reject the null hypothesis that PTSD treatments are similarly effective. They concluded that the results of previous metaanalysis may have been influenced by several confounds, including the use of control treatments, to make conclusions about the relative efficacy of specific PTSD treatments. Ehlers et al. (2010) claim that the selection procedures of the Benish et al. meta-analysis were biased and cite results from individual studies and previous meta-analyses that suggest trauma-focused psychological treatments are superior to nontrauma focused treatments. We first offer a review and justification of the coding criteria and procedure used in Benish et al. In addition, we discuss the appropriateness of utilizing treatments designed to control for non-specifics or common factors such as ‘supportive therapy’ for determining the relative efficacy of specific PTSD treatments. Finally, we note several additional confounds, such as therapist effects, allegiance, and alteration of legitimate protocols, in PTSD research and describe conceptual problems involved in the classification scheme used to determine the “trauma focus” of interventions, which lead to inappropriate conclusions about what works in the treatment of PTSD. © 2010 Elsevier Ltd. All rights reserved. Contents 1. Bona fide treatments . . . . . . . . . . . . . . . . . 1.1. Conceptual validity of bona fide treatments . . . 1.2. Application of selection criteria to select studies . 2. Is supportive therapy a bona fide treatment for PTSD? . 3. Examining the alternative: trauma focused treatments are 4. Therapist effects and allegiance . . . . . . . . . . . . 5. Altering legitimate treatments . . . . . . . . . . . . . 6. Conclusions . . . . . . . . . . . . . . . . . . . . . . References . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . superior . . . . . . . . . . . . . . . . . . . . The randomized clinical trial (RCT) is a powerful methodology for determining the relative efficacy of PTSD treatments, but clinical trials are complex and subject to numerous threats to validity (Schnurr, 2007; Mohr et al., 2009). Benish, Wampold, and Imel (2008) sought to control several confounds in RCTs by conducting a meta-analysis of clinical trials that directly compared ‘bona fide’ treatments for PTSD. Their results ⁎ Corresponding author. E-mail address: wampold@education.wisc.edu (B.E. Wampold). 0272-7358/$ – see front matter © 2010 Elsevier Ltd. All rights reserved. doi:10.1016/j.cpr.2010.06.005 . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 924 924 925 926 927 930 930 931 932 provided little evidence for the differential effectiveness of PTSD treatments, suggesting that inclusion of non-bona fide treatments in previous reviews may have resulted in the spurious finding of differences among treatments. However, Ehlers et al. (2010), on the other hand, claimed that the results of Benish et al. were biased by an inappropriate exclusion of studies and arbitrary selection procedures. They cite previous meta-analyses that included control treatments and specific outcomes from individual studies to argue that the preponderance of evidence supports the superiority of trauma-focused psychological treatments over non-trauma focused treatments. The debate 924 B.E. Wampold et al. / Clinical Psychology Review 30 (2010) 923–933 regarding the relative efficacy of PTSD treatments raises important questions about the design and interpretation of clinical trials investigating PTSD treatments. Accordingly, the purpose of this paper is to examine the validity of Benish et al.'s and Ehlers et al.'s conclusions — a discussion that suggests that care must be taken in the design and interpretation of RCTs and the conclusions that are made from such trials. 1. Bona fide treatments Ehlers et al. (2010) critique centers on Benish et al.'s classification of therapies as bona fide. They argue that the classification scheme is conceptually flawed and inconsistently applied. Specifically, Ehlers et al. were surprised that some treatments, such as supportive therapies, were not coded as bona fide (i.e., not intended to be therapeutic) even when these treatments showed evidence of some positive effect on symptoms. They also wondered why some conditions were excluded from the meta-analysis while others were not. The criticisms raised by Ehlers et al.'s highlight important but familiar threats to validity in clinical trials and are important to address. To do so, we begin the discussion with a brief conceptual justification for why it is essential to distinguish between so-called bona fide and non-bona fide treatments in order to make valid conclusions about the relative efficacy of PTSD treatments. 1.1. Conceptual validity of bona fide treatments Studies designed to test relative efficacy have different implications than studies designed to test the specific ingredients of treatments. Relative efficacy is a term that refers to the comparative effectiveness of two established interventions that have been offered as a treatment for a particular problem, compliant, or disorder (e.g., Fluoxetine vs. Sertraline for depression; weight loss surgery vs. exercise and diet for obesity; interpersonal therapy vs. cognitive behavioral treatment for depression). On the other hand, establishing the specificity of a particular ingredient requires different designs, such as the comparison of bona fide treatment to a placebo or a dismantling strategy (see, e.g., Borkovec, 1990; Borkovec & Sibrava, 2005; Schnurr, 2007). Ehlers et al. acknowledge the complexity of using controls to test the mechanisms of psychotherapy, but did not consider the potential for such designs to create bias in tests of relative efficacy. For example, the superiority of an antidepressant over and above a pill placebo does not indicate that this antidepressant is any better than any other antidepressant or alternative nonpharmacological treatments. Moreover, the effectiveness of the pill placebo (i.e., in comparison to the natural course of the disorder) has no bearing on whether the pill placebo is a bona fide treatment for depression. However, inferences are often made about treatments based on comparisons with conditions that are designed by researchers as controls, but then later interpreted as treatments that are provided in routine care. The conflation of specificity with relative efficacy leads to inappropriate conclusions. In an attempt to make sense of the long history of equivocal qualitative and meta-analytic reviews regarding the relative efficacy of specific psychotherapies literature (Grissom, 1996; Luborsky, Singer, & Luborsky, 1975; D. A. Shapiro & Shapiro, 1982; Smith, Glass, & Miller, 1980), Lambert and Bergin (1994) made the following conclusion: There is a strong trend toward no difference between techniques in amount of change produced, which is counterbalanced by indications that, under some circumstances, cognitive and behavioral methods are superior even though they do not generally differ in efficacy between themselves. An examination of selected exemplary studies allows us to further explore this matter. Research carried out with the intent of contrasting two or more bona fide treatments show surprisingly small differences between the outcomes for patients who undergo a treatment that is fully intended to be therapeutic (p. 158; emphasis added). Lambert and Bergin were making a distinction between treatments that were bona fide and those that were not. The term bona fide – often used synonymously with the phrase ‘intended to be therapeutic’ – excludes treatments that were designed as controls or psychological placebos. These treatments, although containing some basic therapeutic elements such as active listening, typically were designed to control for basic commonalities of therapies, primarily a relationship with a therapist. These control treatments typically lack a treatment rationale based in psychological principles and do not contain therapeutic ingredients or actions. Often such treatments are designed to exclude certain aspects of the treatments to which they are compared, resulting in proscriptions that create conditions that have little resemblance to anything that therapists would offer to their patients. Indeed, many of these control conditions proscribe discussing the patient's presenting concerns or precipitating event (Borkovec, 1990; Borkovec & Sibrava, 2005; Mohr et al., 2009; Parloff, 1986; Wampold, 2001). Meta-analyses failing to make a distinction between treatments that are bona fide and those treatments that are not will likely lead to flawed conclusions (cf., Gloaguen, Cottraux, Cucherat, & Blackburn, 1998; and Wampold, Minami, Baskin, & Tierney, 2002). This issue led Wampold et al. (1997) to offer an initial test of Lambert and Bergin's supposition that all bona fide psychotherapies were equivalent. To do so, the authors expanded on the classic work of Frank and Frank (1991), operationalizing bona fide treatments1 as those that have ingredients common to all legitimate psychotherapies, including a cogent rationale for the disorder being treated, a treatment based on psychological principles, therapeutic actions consistent with the rationale, and active collaborative participation of both patient and therapist (Anderson, Lunnen, & Ogles, 2010; Frank & Frank, 1991; Imel & Wampold, 2008; Wampold, 2007). Such treatments do not proscribe the therapist from typical therapeutic actions, are flexible enough to accommodate individual patients, and are typically administered by therapists who are aligned with the treatment they are providing. This particular definition of a bona fide psychotherapy recognizes that typical common factor controls that exclude such components were not designed to test whether one treatment is superior to another but rather to establish that a treatment is superior to a placebo-type control (Westen, Novotny, & ThompsonBrenner, 2004). Four features of the initial Wampold et al. (1997) meta-analysis figure prominently in the different conclusions of Benish, Imel, and Wampold (2008) and Ehlers et al. (2010). First, the Wampold et al. meta-analysis was limited to trials that directly compared treatments, thus ruling out the multiple confounds created by examining effects produced by treatment compared to no-treatment controls (Shadish & Sweeney, 1991), such as participant characteristics, outcome measures employed, treatment standardization, measure reactivity, blinding procedures, treatment length, severity of disorder, and multiple unmeasured variables. Second, differences among treatments were examined without placing treatments into classes because determining features of a “class” of treatments is problematic in psychotherapy and, historically, classifications have been criticized by those who find the results disagreeable — an issue we will return to in the discussion of trauma-focused treatments (Wampold, 2001). Third, all outcome measures within studies included in the meta-analysis were aggregated 1 In the literature, the terms bona fide and intended to be therapeutic are used to differentiate legitimate treatments from minimally therapeutic control conditions. In this discussion we use the term bona fide because the latter is occasionally incorrectly taken to imply that the coding procedures are used to infer the intent of the researcher, which has never been the case. The language “intended to be therapeutic” derives from Lambert and Bergin (1994), which is quoted in the text. B.E. Wampold et al. / Clinical Psychology Review 30 (2010) 923–933 in order to account for the dependencies in the outcome variables. Often attention is paid to a statistically significant differences on one of many outcome measures, which ignores Type II error rate problems and creates an illusion of treatment superiority, a problem long noted in psychotherapy research (Dar, Serlin, & Omer, 1994). Fourth, and finally, only bona fide treatments – that is, trials that directly compared two or more legitimate treatments – were included in the analysis. 1.2. Application of selection criteria to select studies Ehlers et al. (2010) criticized Benish et al. (2008) meta-analysis for bias in selection of studies, suggesting that they included or excluded studies based on the efficacy of various treatments. However, the criteria used by Wampold et al. (1997) and Benish et al. (2008) were explicit, objective, and based on a cogent theory of the benefits of psychotherapy: First, the treatment must have involved a therapist with at least a master's degree and a meeting with a patient in which the therapist developed a relationship with the patient and tailored the treatment to the patient. Thus, any study that used solely tape-recorded instructions to patients or a protocol that was administered regardless of patient behavior (e.g., a progressive relaxation protocol that was not modified in any way for particular patients) was excluded. Second, the problem addressed by the treatment must have been one that would reasonably be treated by psychotherapy, although it was not required that the sample treated be classified as clinically dysfunctional. For example, treatments to increase time that a participant could keep a hand submerged in cold water would be excluded because cold-water stress would not reasonably be considered a problem for which one would present to a psychotherapist. However, any treatment for depression was included whether the participants met diagnostic criteria for any depressive disorder or scored below standard cutoffs on depression scales. Finally, the treatment had to satisfy two of the following four conditions: (a) A citation was made to an established approach to psychotherapy (e.g., a reference to Rogers' s, 1951, client-centered therapy), (b) a description of the therapy was contained in the article and the description contained a reference to psychological processes (e.g., operant conditioning), (c) a manual for the treatment existed and was used to guide the administration of the psychotherapy, and (d) the active ingredients of the treatment were identified and citations provided for those ingredients. Accordingly, any treatments designed to control for common or nonspecific factors, such as placebo control groups, alternative therapies, or nonspecific therapies, were excluded (Wampold et al., 1997, pp. 206–207). The criteria were designed to exclude control treatments that lacked the features of psychotherapies typically used in order to provide an omnibus test of the relative efficacy of bona fide psychotherapies. Importantly, in the Benish et al. meta-analysis, the a priori criteria were applied by independent raters blind to the results of the study (the coders only had access to the authors' description of the treatments and not the results), obviating any researcher effort to select studies based on the results of the studies or any other factors. Scientific method requires that research operations be stated clearly so that researchers can replicate the study; in this regard, the Benish et al., criteria, were explicit and replicable. To examine how the criteria were applied, we will examine three studies that Ehlers et al. (2010) cited as evidence that Benish et al. selected studies capriciously. Contrary to Ehlers et al.'s contention, an examination of these studies reveals that the criteria were applied consistently and validly. The first study cited by Ehlers et al. (2010), which was excluded by Benish et al. (2008), was the supportive counseling condition used in the Foa, Rothbaum, Riggs, and Murdock 925 (1991) as a control condition for prolonged exposure and stress inoculation training for women who had been sexually assaulted. Supportive counseling was described as follows: Supportive counseling followed the nine-session format, gathering information through the initial interview in the first session and presenting the rationale for treatment in the second session. During the remaining sessions, patients were taught a general problem-solving technique. Therapists played an indirect and unconditionally supportive role. Homework consisted of the patient's keeping a diary of daily problems and her attempts at problem solving. Patients were immediately redirected to focus on current daily problems if discussions of the assault occurred. No instructions for exposure or anxiety management were included. (pp. 717–718). This supportive counseling condition did not meet the criteria for a bona fide treatment because there was (a) no citation to an established psychological approach, (b) no psychological principles of change were referenced, (c) no manual of the treatment was available and used to guide treatment, and (d) no active ingredients of treatments were referenced. Indeed, Foa et al. (1991) indicated that the treatment was not designed to be therapeutic: “[Supportive Counseling] was included to control for nonspecific therapy effects” (E. B. Foa et al., 1991, p. 716). Moreover, therapists were proscribed from what, it is safe to say, all therapists, working from any approach would do: Allow patients to talk about the issue central to their dysfunction, a point with which Ehlers et al., agree when they state, “Therapists in the supportive counseling condition were instructed to steer clients away from talking about their specific traumatic events [and] we agree that this would not necessarily be representative of supportive therapy as it would be delivered by a practicing clinician, and may therefore underestimate the effect of counseling” (p. 271). In sum, the supportive counseling offered in this study was not bona fide and was therefore properly excluded. The second study cited by Ehlers et al. (2010) was Blanchard et al. (2003) comparison of cognitive behavioral therapy and supportive psychotherapy for survivors of motor vehicle accidents. The supportive psychotherapy was described by the researchers as follows: In this condition the first session was very similar to the first session in the CBT condition in that PTSD and its symptoms were described in detail and how the individual's symptoms fitted this picture. Again, there was an effort to “normalize” the experience. No relaxation was included. The next 3 sessions were devoted to a detailed review of the participant's life, from earliest childhood to the present, with particular attention to previous traumas and previous losses and how the participant had dealt with them. This was done in a supportive and caring fashion. The remaining 4–8 sessions were devoted to providing the patient with support on issues raised by the participant, including interpersonal or relationship issues, work issues, etc. Very little direct advice was given; instead, the participant was asked what he/she felt or thought. In most cases there were ongoing interpersonal issues that occupied the sessions. Care was taken not to encourage any driving. If the participant asked directly about a specific travel behavior, he or she was told to listen to his/her body and be guided by how he/she felt. Catastrophic thoughts about the MVA were not challenged. If a patient asked about relaxation or meditation, he/she was told to use his or her own best judgment. Thus, the effort was made not to encourage any of the specific elements of the CBT protocol, but instead to put those choices/ initiatives back on the patient. (p. 86). Again, independent blind raters applying explicit inclusion criteria to this condition excluded it because (a) there was no citation to an established psychological approach, (b) the treatment was not based on 926 B.E. Wampold et al. / Clinical Psychology Review 30 (2010) 923–933 any psychological principles, (c) no manual of the treatment was available and used to guide treatment, and (d) no active ingredients of treatments were referenced. Indeed, as can be seen, the description focuses more on what is proscribed than on the rationale for a legitimate psychotherapy. Presumably, had a patient in the supportive condition reported to the therapist, “My family is traveling to my mother-in-law's for the holiday and I really, really want to go, as I don't want to be home alone during this period, but something tells me that I shouldn't go,” the therapist would have responded, “Listen to your body and be guided by how you feel.” What legitimate ‘supportive’ psychotherapy would have therapists respond in this fashion? The last study Ehlers et al. (2010) asserted was inappropriately excluded by Benish et al. (2008) was a supportive counseling condition employed in a trial conducted by Bryant, Moulds, Guthrie, Dang, and Nixon (2003). As was true in the two prior studies, this treatment met none of the established criteria for being a bona fide treatment — there was no rationale, no psychological actions, no manual to direct therapy, and no specified therapeutic actions of any kind. Instead, supportive counseling “comprised education about trauma and general problem-solving skills and provided an unconditionally supportive role. Homework comprised diary keeping of current problems and mood states. [Supportive Counseling] specifically avoided IE [imaginal exposure] or CR [cognitive restructuring] techniques” (p. 707). Contrast the three treatments excluded by Benish et al. with Present Centered Therapy (PCT), as used by McDonagh et al. (2005) in a trial of CBT for women who had survived childhood sexual abuse. In this trail, PCT was as effective as CBT, with significantly fewer dropouts. Ehlers et al. (2010) suggested that PCT, as delivered in this study, was included in the Benish et al. (2008) meta-analysis because it was found to be as effective as CBT. However, the reason why PCT was included when the previous three conditions were excluded is apparent from the characteristics of the treatment: PCT was designed to describe an active therapeutic intervention that non-CBT clinicians might use in the treatment of PTSD–CSA. It is a collaborative therapeutic intervention in which the therapist's information and expertise are used to assist the client in addressing current life difficulties. It does so by helping the client to recognize the impact of her trauma history on her present coping style and by teaching her a systematic approach to problem solving to enhance coping (de Shazer et al., 1986; D'Zurilla & Goldfried, 1971; Nezu & Perri, 1989). The main elements of PCT are psychoeducation about the diagnosis of PTSD and the common aftereffects of childhood trauma, training in problem solving, and journal writing…. The first two sessions of PCT were spent establishing rapport, giving an overview of the treatment, presenting the psychoeducational material, and establishing a treatment plan based on the client's choices of problems to address. The framework used to assist in understanding the ways in which CSA trauma can impact the client's current life was that of traumagenic dynamics. Traumagenic dynamics organizes the consequences of the experience of CSA into four categories — a sense of betrayal, powerlessness, stigmatization, and traumatic sexualization (Finkelhor, 1987). Clients were guided in noticing these dynamics in current life difficulties and factoring them in as information in the problem-solving model. The problem-solving model was a modification of systematic problem solving, which has demonstrated efficacy in the treatment of depression (DeShazer et al., 1986; D'Zurilla & Goldfried, 1971; Nezu et al., 1989). Therapists were encouraged to provide empathy, unconditional regard, and genuineness in their sessions (Meador & Rogers, 1973). The foci of subsequent sessions were determined by the participants' choice of current issues to address with the problem-solving skills. Homework was designed to assist clients in consolidating the information conveyed in sessions, writing about their problem solving efforts and, tracking those efforts in a journal to be reviewed with the therapist. A full description of PCT is available in the therapy manual. (p. 518). Simply put, PCT was classified as bona fide and included in the Benish et al. (2008) meta-analysis because it met all the criteria for a bona fide treatment. In PCT, the trained therapists formed a relationship with the patient, the treatment was tailored to the patient, there were citations to a psychological approach to the problem, a description of therapy was provided with reference to psychological principles, a manual was used to guide the therapy, active ingredients of the therapy were described, along with citations, and the therapists were allowed to discuss the presenting concern. Clearly, even though PCT was designed for this study and is not a treatment that is provided in the “real world,” PCT was a treatment that therapists could faithfully deliver. Contrary to what Ehlers et al. imply, PCT was not included in the Benish et al. meta-analysis because it was as effective (again, the Benish et al. raters were blind to the results of the study) as CBT but rather because it met the inclusion criteria. The criteria, which were applied by blind raters, resulted in the rejection of three treatments that were without legitimate psychological bases and were in a sense designed to “fail,” as discussed by Westen et al. (2004). On the other hand, PCT in the McDonagh et al. study was designed to have components of all therapies that are intended to be therapeutic, regardless of theoretical orientation, and was given the chance “to succeed.” In the same way that behavioral activation accidently became an evidence-based treatment for depression (i.e., first as a condition in a dismantling study in Jacobson et al., 1996), it may well be that PCT, as developed by McDonagh et al., will end up an evidence-based treatment for PTSD. That PCT was a successful treatment may have much to do with the fact that it was well designed as a treatment. We have established that the conceptual basis of the Benish et al. (2008) criteria is found in common factors theory and an understanding of the logic of RCTs. The Benish et al. criteria were designed to exclude control treatments that did not contain common therapeutic elements (Frank & Frank, 1991), thus excluding trials that do not provide, and were not intended to provide, evidence for the relative efficacy of treatments of PTSD. The criteria functioned precisely as intended. That the control conditions, such as supportive counseling, produce some benefits for patients was not considered in the inclusion or exclusion of studies; the benefits of such treatments is certainly interesting, but are not relevant to the question of whether some bona fide treatments are superior to others. 2. Is supportive therapy a bona fide treatment for PTSD? Ehlers et al. (2010) did not question the meta-analytic conclusion that the treatments examined in the meta-analysis were equally effective. Rather, their criticism rests primarily on the selection of the studies included in the meta-analysis. Benish et al. were careful to state that their conclusion was restricted to bona fide therapies — that is, there is no evidence that there are differences in efficacy of bona fide treatments, as defined by Wampold et al. (1997). After arguing that control treatments are effective and should be included in metaanalyses assessing relative efficacy, Ehlers et al. pivot and claim that the supportive counseling offered in clinical trials aren't actually controls, but legitimate treatments because this type of treatment is offered to patients in the real world: “There is a good rationale for using supportive therapy to treat PTSD as social support has been shown to be one of the best predictors of recovery in PTSD” (Ehlers et al., 2010, p. 270). Surveys do indicate that therapists endorse providing an intervention called “supportive therapy.” For example, Ehlers et al. (2010) B.E. Wampold et al. / Clinical Psychology Review 30 (2010) 923–933 cited a survey that show supportive therapy “is the treatment most commonly offered to PTSD clients identified in primary care” in England and is “widely practiced in the United States” (p. 270). However, it clear that the intervention therapists offer in practice is not comparable to the supportive therapies designed and delivered in clinical trials. Indeed, it may be that the restrictions placed on therapists in clinical trials prevent therapists from being as supportive as they might be otherwise. One cannot make inferences about the nature of treatments based primarily on the term that it is applied to the treatment, as will become apparent as some scrutiny is applied to the claims about supportive therapy, as delivered in RCTs. Supportive therapy is a term often applied to a specific class of treatments that specify therapeutic actions that are very different from the supportive counseling conditions that are employed in clinical trials: According to Pinsker (2002), the term “supportive therapy” has acquired several distinct meanings. He indicated that supportive therapy was initially characterized as a treatment for the most impaired clients, namely “those who were not suitable for the intensive therapy that was almost universally accepted as the proper treatment for anyone who was intelligent enough and intact enough to participate” (p. 1). Subsequently, supportive psychotherapy has taken on a psychodynamic orientation deemed appropriate for a broad array of psychopathology. Although the definitions of what comprises supportive therapy may differ somewhat, there are several techniques that are common amongst the forms of supportive therapy. Novalis, Rojcewicz, and Peele (1993) contended that supportive therapy employs several different empirically based techniques, such as promoting supportive therapeutic relationships, encouraging clients to use their support systems and coping skills, fostering independence, and reducing their distress and behavioral dysfunctions. (Budge, Baardseth, Wampold, & Flűckiger, 2010, p. 25–26). When respondents to surveys indicate that they practice “supportive therapy,” it is not clear what meaning of supportive therapy they are referencing. They may likely be referring to psychodynamic interventions, which would then have little similarity to the supportive counseling interventions offered in clinical trials, or they could be referring to treatments they provide to patients who are not appropriate for or have refused intensive weekly psychotherapy — for example, patients who are excluded from PTSD clinical trials, remain symptomatic at the end of treatment, or drop-out of a more intensive psychotherapy prematurely (Bradley et al. 2005). However, it is unlikely therapists, when they indicate in surveys that they use supportive psychotherapy, are referring to a treatment that restricts therapeutic actions to being unconditionally supportive, prevents any therapist efforts to improve social relations, build social support, and does not permit the patient to talk about the issue that brought them to therapy! The surveys cited by Ehlers et al. also do not indicate that therapists treat PTSD with supportive therapy solely. Indeed, the survey in England (Ehlers, Gene-Cos, & Perrin, 2009) asked 159 general practice physicians what their mental health worker used to treat PTSD and most of the physicians indicated that the they thought the mental health workers used supportive therapy, which the physicians indicated “was often given in combination with other interventions” (emphasis added, p. 38), most often CBT. This is in contrast with clinical trials in which therapists are explicitly proscribed from using any behavioral or cognitive components (or for that matter, components of any school of therapy), obviating a test of relative efficacy. The second survey cited by Ehlers et al. (2010) provides even less support for the premise that therapists believe supportive therapy is the treatment of choice for the treatment of PTSD. Pingitore, Scheffler, Haley, Sentell, and Schwalm (2001) did find that 58% of therapists provided supportive therapy to their patients, but that they provided 927 supportive therapy to only 19% of their patients in their caseload (across various diagnoses), and moreover they offered supportive therapy most often in conjunction with other therapies. It is not known to what extent any of these therapists used only supportive therapy, why they used it, and what that supportive therapy contained. Even if it is true that a few therapists provide poor quality supportive therapy (i.e., in the manner of the various clinical trials) in routine care, one should not include such therapies in trials any more than one would deliver CBT as commonly practiced by some therapists in routine care in an RCT. As Clark, Fairburn, and Wessely (2007) noted, “In our experience, such misunderstandings [by practicing clinicians] of what CBT comprises are by no means unusual” (p. 631); the same could be said about those who claim that supportive therapies delivered in clinical trials are bona fide treatments or representative of what therapists use in routine care. In a similar manner, Schnurr et al. (2007) justified a form of “present centered-therapy” (PCT) because “a supportive, present-centered approach is clinically realistic because it is typically used by Department of Veterans Affairs (VA) clinicians” (p. 821), a conclusion based on a thorough survey of VA clinicians conducted by Rosen et al. (2004). However, in the Rosen et al. survey, present-centered procedures included coping skills training, psychoeducation, safety and trust issues, anger management, and sleep hygiene. Except for psychoeducation, the PCT delivered by Schnurr et al. did not include these ingredients and consequently had little resemblance to “present-centered procedures” that VA clinicians reported using. There is an interesting trial of EMDR that illustrates the illogicality of assigning legitimacy to a control condition. In 1989, F. Shapiro (1989), attempting to rule out exposure as the active ingredient in EMDR, compared EMDR to control group in which the subjects were exposed to their traumatic memory through being instructed to describe their memory. However, this “exposure” treatment did not resemble the systematic protocols of any known exposure treatment, contained no cogent explanation, and would not have met the criteria for a bona fide treatment, even though it had elements of flooding. It was control condition, pure and simple, just as Shapiro described it. In this trial, EMDR was clearly more efficacious that the exposure control condition. However, it would be illogical to cite this study as evidence that EMDR is more effective than exposure based treatments for PTSD or to claim that exposure is an inferior treatment because the exposure treatment in this study was not a legitimate treatment for PTSD, even if some therapists in practice might deliver exposure in the way that it was delivered in this study. This study has never been cited as evidence that EMDR is superior to exposure treatments, as far as we can tell. However, the attempt to include supportive counseling as a legitimate treatment for PTSD to conclude that trauma-focused treatments are superior rests on a logical basis that would have to then admit as evidence Shapiro's trial that shows that exposure is an inferior treatment for PTSD. We agree with Ehlers et al. (2010) that the supportive therapy as designed and implemented in many trials should not be used as a first line treatment for PTSD. However, the supportive therapies used in the clinical trials only support conclusions about the relative effects of various treatments vis-à-vis treatments defined primarily by what was proscribed, which yields treatments that do not resemble any recognized form of psychotherapy. 3. Examining the alternative: trauma focused treatments are superior A major objective of Ehlers et al. (2010) was to restate the evidence for the superiority of trauma focused treatments for PTSD vis-à-vis other treatments. Clearly, this is an alternative hypothesis that needs to be examined critically. As we will show, when arguing for the superiority of trauma focused therapies, Ehlers et al. (2010), rely on unblinded ratings derived from a classification system based on “clinical experience and categories used in the literature” (Jonathan I. Bisson et al., 2007b, p. 98) 928 B.E. Wampold et al. / Clinical Psychology Review 30 (2010) 923–933 as well as the National Institute of Clinical Excellence (NICE) Guidelines for classifying treatments as trauma focused. According to the NICE Guidelines, “The relevant consideration for the classification was whether or not the treatment mainly focused on the trauma memory and its meaning” (National Collaborating Centre for Mental Health, 2005, p. 54) (see Table 1 for a comparison of the coding protocols for determining whether or not a treatment was bona fide in contrast to whether it was trauma focused or not). In this section, we outline how this operational definition developed by NICE and subsequently used by Ehlers et al. not only presents significant threats to Ehlers et al.'s conclusions, but obscures our understanding of what works for PTSD treatment. To appropriately conclude that trauma focused treatments are superior, it is necessary to classify treatments based on whether or not they are trauma focused and then to compare the efficacy of treatments in these two classes (viz., trauma focused versus non-trauma focused). As we will see, there are formidable problems with the classification scheme — but more pernicious is the fact that there are no differences in treatment effectiveness between treatments classified as traumafocused and those that are not, when the comparisons are restricted to treatments that are bona fide. The trauma-focused distinction has been operationalized by Bisson and others by classifying treatments into five categories: (a) trauma focused CBT (TFCBT); (b) EMDR, (c) stress management, (d) CBT (not trauma focused), and (e) other treatment, of which the first two, TFCBT and EMDR are designated as trauma focused whereas the last three are designated as not trauma focused (J. I. Bisson & Andrew, 2009; Jonathan I. Bisson et al., 2007b), as summarized in Table 2 (which have also been segregated depending on whether they meet the Wampold et al. 1997 criteria for bona fide therapies). An inspection of the individual treatments included in these five categories reveals several inconsistencies in the classification strategy. According to the NICE guidelines (National Collaborating Centre for Mental Health, 2005), CBT treatments included those that “employ a range of therapeutic techniques that aim to change people's distressing emotions by changing their thoughts, beliefs and/or behavior” (p. 52) and include one or more of the components: exposure, cognitive therapy, and stress management. TFCBT are CBT that also “mainly focused on the trauma memory and its meaning.” Most of the TFCBT treatments in Table 2 contain some systematic exposure component, but some do not; for example, the cognitive restructuring condition in Tarrier et al. (1999) was designed specifically to exclude any exposure. Indeed, there is one TFCBT treatment that involved 30 min sessions of brainwave feedback (called neurofeedback by Bisson et al.) with no opportunity to talk about the trauma (or anything else) with their therapist (Peniston & Kulkosky, 1991). Many of the treatments include ingredients from other orientations, including, according to NICE, “elements of psychodynamic therapies” (National Collaborating Centre for Mental Health, 2005, p. 54). Indeed, TFCBT appears to be a category of exposure only, cognitive restructuring only, a combination of exposure and cognitive restricting, in combination with a variety of other ingredients, as well as a treatment that is not psychotherapy at all. The second category of trauma focused PTSD treatments was EMDR. Ehlers et al. (2010) characterize TBCBT and EMDR together as a super ordinate trauma focused treatment, which are purportedly superior to other treatments for PTSD. According to NICE, EMDR is “based on a theoretical model which posits that the dysfunctional intrusions, emotions and physical sensations experienced by trauma victims are due to the improper storage of the traumatic event in implicit memory. The EMDR procedures are based on stimulating the patient's own information processing in order to help integrate the targeted event as an adaptive contextualised memory” (National Collaborating Centre for Mental Health, 2005, p. 55). As is true for all treatments, the procedures of EMDR overlap with other treatments: For example, holding an image of the trauma in mind resembles imaginal exposure, although the exposure is much briefer and the patient does not verbalise the content of the image. Replacing negative cognitions associated with the trauma with positive cognitions overlaps with cognitive interventions. The associative techniques resemble those used in psychodynamic approaches. (National Collaborating Centre for Mental Health, 2005, p. 55). The classification of EMDR as trauma focused and therefore a privileged treatment for PTSD along with TFCBT presents an uncomfortable pairing. Some clinical scientists have labeled EMDR as pseudo-science (e.g., Herbert et al., 2000) and compared it to Mesmerism (McNally, 1999), due to the invalidity of the psychological bases of its ingredients, the lack of evidence for specificity of the active ingredients, the unjustified claims of efficacy and efficiency, and the manner in which it is publicized and disseminated (see also Davidson & Parker, 2001; G. M. Rosen, 1999). Herbert et al. (2000) claimed that “the promotion of EMDR provides a good illustration of pseudoscience in general and of how pseudoscience is marketed to mental health Table 1 Bases for designation as bona fide treatment and trauma focused treatments. Bona fide (Benish et al., 2008) Trauma focused (Bisson et al., 2007b) Theoretical bases Common factor theory (Frank & Frank, 1991; Wampold, 2007) Not mentioned Criteria Therapist with at least Master's degree Treatment Individually tailored to the patient Presenting concern for treatment clinically reasonable A citation was made to an established approach to psychotherapy (e.g., a reference to Rogers' s, 1951, client-centered therapy), A description of the therapy was contained in the article and the description contained a reference to psychological processes (e.g., operant conditioning) A manual for the treatment existed and was used to guide the administration of the psychotherapy The active ingredients of the treatment were identified and citations provided for those ingredients Rating procedures Raters blinded to results of study Independent non-author raters Rater agreement “Treatment mainly focused on trauma memory and its meaning” (National Collaborating Centre for Mental Health, 2005, p. 54) Clinical experience Raters not blinded to results of study Raters were authors Consensus 929 B.E. Wampold et al. / Clinical Psychology Review 30 (2010) 923–933 Table 2 Bisson et al. (2007a,b) classification of Treatments. Trauma focused Not trauma focused Bona fide Trauma focused CBT EMDR Stress management CBT Yes EMDR Stress inoculation training (without 3rd phase) Affect management (adjunct to individual) Psychodynamic Imagery rehearsal (not tested in RCT) Hypnotherapy Present centered group therapy (Classen et al., 2001) No Trauma desensitization Imaginal flooding Implosive flooding Prolonged exposure Image habituation training Gradual exposure and CR Imaginal exposure Cognitive therapy (no exposure) Brief eclectic therapy Trauma focused CBT Cognitive processing therapy Prolonged exposure Immediate cognitive therapy Narrative exposure therapy Neurofeedback training (not tailored to patient; Peniston & Kulkosky, 1991) Applied muscle relaxation Present centered therapy (Schnurr et al., 2007) Progressive muscle relaxation Biofeedback + assisted relaxation Relaxation clinicians, some of whom may be relatively unfamiliar with the published research on EMDR” (p. 955). Nevertheless, a meta-analysis concluded that EMDR and TFCBT are equally efficacious (Seidler & Wagner, 2006). Stress management is listed as a separate non-trauma focused category of treatments (see Table 2) and includes various relaxation techniques, biofeedback, and stress-inoculation training (SIT). The relaxation treatments in this category are standard protocols and are not treatments tailored to the patient (viz., Echeburúa, de Corral, Zubizarreta, & Sarasua, 1997; Marks, Lovell, Noshirvani, Livanou, & Thrasher, 1998; Vaughan, Armstrong, Gold, & O'Connor, 1994); they would not be classified as bona fide treatments using Benish et al.'s criteria (i.e., treatment not tailored to the patient) and would not be considered psychotherapy by many. Furthermore, it is not clear why neurofeedback, involving 30 min sessions with no contact with the clinician (Peniston, & Kulkosky, 1991) was classified as TFCBT when biofeedback plus relaxation was classified as stress management. The case of SIT is particularly mysterious because it is a bona fide treatment for PTSD (Meichenbaum, 1985;2007) that has many features in common with TFCBT. There are two trials that compared SIT to a TFCBT conducted by Foa et al. (1999;1991). For the most part, there were few, and inconsistent, differences between these two interventions. Thus, a bona-fide stress management treatment was found to be as effective as a TFCBT, a result confirmed by meta-analysis (Powers et al., 2010). It also is not clear why SIT was not classified as trauma focused. For example, Ehlers et al. (2010) classified the CBT used by Blanchard et al. (2003) as trauma focused, even though Blanchard's CBT is based to a large extent on Meichenbaum's SIT (viz., Meichenbaum, 1985). Indeed, a reading of Meichenbaum's work suggests that SIT meets Ehlers et al.'s criteria for being deemed “trauma focused” as SIT encourages patients to tell their stories about and explore the personal meaning of the trauma (Meichenbaum, 1985;2007). It is not clear on what basis it was determined that SIT was not mainly focused on the trauma memory and its meaning, given the large latitude given to other treatments determined to be TFCBT, as illustrated in Table 2. The fourth type of PTSD treatment in the Bisson et al. (2007b) taxonomy is CBT (not trauma focused) consisted of three treatments, affect management, imagery rehearsal, and present-centered therapy (PCT). Affect management was a group intervention used by Zlotnick et al. (1997) as an adjunct to individual therapy and was not intended to be a standalone treatment. Group imagery rehearsal was used by Krakow et al. (2001) to treat crime victims in an uncontrolled study that assessed efficacy with changes from baseline. The third CBT, non-trauma Other Supportive counseling Active listening focused, was PCT, which varied in content significantly across studies. The classification of PCT as a CBT is questionable because presentcentered therapy was designed explicitly to contain no components that would overlap with exposure or cognitive restructuring (McDonagh et al., 2005; Schnurr et al., 2007) and does not appear have any of the three ingredients the NICE Guidelines indicated were necessary to be classified as CBT (viz, exposure, cognitive therapy, and stress management; National Collaborating Centre for Mental Health, 2005). Incidentally, in the Bisson et al. (2007b) classification, there was one present-centered treatment that would have been classified as bona fide, and in this trial the trauma-focused group treatment was not superior to the PCT (Classen, Koopman, Nevill-Manning, & Spiegel, 2001). In this study PCT was actually described as traditional Yalomesque group psychotherapy that focused on processing here and now experience. Irvin Yalom, a dynamic-existential therapist, would certainly be surprised that the definition of CBT has expanded to include his treatment was well. The final category in Bisson's et al. (2007b) taxonomy is other treatments — including hypnotherapy and dynamically-oriented therapies. Ehlers et al. (2010) admit that this category was somewhat arbitrary and functions as a miscellaneous collection of treatments, as there are entire classes of treatments that have not been sufficiently tested (Stein et al., 2009). However, why hypnotherapy and dynamically-oriented therapies were not considered trauma-focused is unclear. After all, each of these treatments allows for, and frequently encourages, patients to discuss their traumas and the meaning attached to them. Hypnotherapy, as conceptualized by Brom et al. was a treatment in which discussion of trauma events was central to the process: The goal was to bring the patient in contact with the reality of the traumatic event and to bring about a decrease in the conditioned responses triggered by the event. Hypnosis was used, because it allows flexibility in the way the client deals, both cognitively and emotionally, with the perception of and adjustment to the trauma. (p. 607) The same is true of dynamic therapy, which was aimed at “solving of the intrapsychic conflicts resulting from the traumatic experience, with the therapist playing an active role, (Brom, Kleber, & Defares, 1989, p. 607). According to the NICE Guidelines (National Collaborating Centre for Mental Health, 2005) “the goal of [psychodynamic] treatment is to understand the meaning of the stressful event [italics added] in the context of the individual's personality, attitudes and 930 B.E. Wampold et al. / Clinical Psychology Review 30 (2010) 923–933 early experiences (Levy & Lemma, 2004) [and] the psychological meaning of the event is explored by a range of methods such as ‘sifting and sorting through wishes, fantasies, fears, and defenses stirred up by the event’ (Kudler et al, 2000)” (p. 56). This fits closely with “a focus on the patients' memories of their traumatic events and the personal meanings of the trauma” (Ehlers et al., p. 270). Both hypnotherapy and dynamic therapy appear to meet the description provided by Ehler et al. for trauma focused treatments. How is it that these treatments are not trauma focused when treatments that contain exposure only (no discussion of the meaning of the trauma), focus primarily on rhythmic bilateral stimulation (EMDR), or involve exclusively neurofeedback (no discussion of the trauma at all) are classified as trauma focused? In conclusion, there are two critical observations that can be made about the classification of treatments as trauma-focused and the conclusion that trauma-focused treatments are superior. First, the classification itself lacks validity. The definition of trauma-focused is ambiguous, the criteria for classification are unclear, and the ratings made by unblinded authors of the meta-analyses. Often the criteria were applied strictly such that treatments that would appear to be trauma focused were excluded and in other cases the criteria are applied quite liberally, including treatments that appear to go to great lengths to exclude a focus on the trauma. This classification strategy results in a notably heterogeneous collection of treatments. The second critical observation is that for the treatments displayed in Table 2, there are no clinical trials that have shown that the nontrauma focused bona fide treatments are less effective than bona fide trauma focused treatments to which they were compared (see Brom et al., 1989; Classen et al., 2001; Edna B. Foa et al., 1999). A recent meta-analysis of prolonged exposure (PE), a trauma-focused treatment, demonstrated that “there was no significant difference between PE and other active [i.e., bona fide] treatments” (p. x), including SIT (Powers et al., in press), a result that is consistent with Benish et al. (2008). It seems premature to conclude that trauma-focused treatments are superior to other treatments when no clinical trial has reliably shown that a trauma-focused treatment is more efficacious than another bona fide treatments and when meta-analyses (viz., Benish et al., 2008; Powers et al., 2010) that examine direct comparisons of bona fide treatments detect no differences as well. The problems discussed here should cast doubt on claims that trauma-focused treatments are superior to non-trauma focused treatments. In addition, a variety of new and promising treatments are coming on line, such as interpersonal therapy (Bleiberg & Markowitz, 2005), affective and interpersonal regulation (Cloitre, Koenen, Cohen, & Han, 2002), yoga breathing (Descilo et al., 2009), mindfulness training (Price, McBride, Hyerle, & Kivlighan, 2007), acupuncture (Hollifield, Sinclair-Lian, Warner, & Hammerschlag, 2007), and behavioral activation (Jakupcak et al., 2006). An attempt to characterize these treatments primarily on the dimension of focus on the trauma undermines the validity and utility of the current “trauma focused” versus “non-trauma focused” distinction. Are interpersonal therapy, behavioral activation, and mindfulness training trauma focused? The answer is not clear. It seems that behavioral activation could easily be conceptualized as in vivo exposure and mindfulness based approaches discourage avoidance and promote acceptance of internal experience. Until this classification system is revised, clinical guidelines based on its structure are suspect. some therapists consistently attain better outcomes than other therapists, which seems to be the case (P. Crits-Christoph et al., 1991; Huppert et al., 2001; Kim, Wampold, & Bolt, 2006; Wampold & Brown, 2005). Ignoring the dependence of observations within therapists has pernicious effects: Increased Type I error rates (i.e., the null hypothesis that treatments are equivalent is rejected more often than the nominal alpha) and inflated estimates of between treatment effects. The correct analysis involves treating therapists as a random factor; that is, therapists are considered to be sampled from a population of therapists with similar characteristics (Paul CritsChristoph & Mintz, 1991; Wampold, & Serlin, 2000). Even if therapists are ignored in the design and analysis, it is vital to keep in mind that the conclusions are generalizable to treatments delivered by therapists with characteristics similar to those in the study. This is particularly problematic if the therapists have an allegiance to one of the treatments or are trained and supervised by researchers who have an allegiance (Luborsky et al., 1999; Wampold, 2001). Therapist allegiance issues also are present in the PTSD literature. The first illustrative trial is one Ehlers et al. (2010) cited as evidence for the efficacy of trauma-focused treatments over non-trauma focused therapies. In this trial, Blanchard et al. (2003) compared CBT to supportive counseling for victims of motor vehicle accidents. This trial used only three therapists, who delivered both treatments, thus limiting the generalizability of the results, but more importantly the three therapists had a “general cognitive behavioral orientation” (p. 83) and were trained by the first two authors, who developed and promoted the CBT used in this study. In the second trial, Cottraux et al. (2008) compared CBT to Rogerian counseling in a multisite trial in France. The two sites were known for CBT treatments and the therapists (3 in one site and 5 in the other) who provided CBT and Rogerian counseling had diplomas in CBT, which involved three years of advanced training in CBT. Clearly, in these two trials the therapists had an allegiance to and experience with CBT and little allegiance or training in the alternative treatment. From these two trials, it might be said that TFCBT is superior (only slightly in the Cottraux trial) to minimal therapies when delivered by CBT therapists. The efficacy of these other treatments delivered by therapists who have an allegiance to them is not known. With the possible exception of McDonagh et al. (2005), none of the supportive therapies/present centered therapies were delivered by therapists who would ordinarily practice these treatments or had any allegiance to them, creating a bias. It is unlikely that an advocate of CBT would tolerate trials using therapists to deliver the CBT intervention who had an allegiance to humanistic therapies and were supervised by humanistic therapists. Fair comparative trials need to balance therapist allegiance across treatments and require therapists to be reasonably trained and competent to deliver the treatment (Hollon, 1999; Luborsky et al., 1999; Wampold, 2001). To use therapists who clearly have an allegiance to a particular treatment but also deliver the comparative treatment, or to have researchers who have an allegiance to a particular treatment train and supervise therapists of the other treatment, introduces threats to validity that cannot be easily dismissed. In the absence of blinding therapists, which as discussed earlier is not possible, fair trials must assiduously attend to allegiance issues. 4. Therapist effects and allegiance There are instances where two bona fide treatments are compared in clinical trials, but one of the treatments is altered, most frequently to eliminate ingredients that overlap with specific ingredients of TFCBTs. Typically, the treatment that is unaltered is one for which the researcher has allegiance vis-à-vis the attenuated treatment. An example in the PTSD literature involves the comparison of prolonged exposure (PE) with stress inoculation training (SIT) (E. B. Foa et al., 1991). In terms of symptomatology, at termination SIT was superior to PE; at follow up, the results were reversed in that PE appeared to be slightly superior to SIT. Two issues that create additional interpretation difficulties in clinical trials, but are typically overlooked, include therapist effects and researcher allegiance. The fact that patients are nested within therapists in RCTs is often ignored, thereby improperly specifying the model, a situation that can have dramatic deleterious effects on the validity of conclusions (Wampold & Serlin, 2000). The issue is this: Observations of patients within therapists are not independent if 5. Altering legitimate treatments B.E. Wampold et al. / Clinical Psychology Review 30 (2010) 923–933 On measures of psychopathology, there were no significant differences between the two treatments. The reversal from termination to follow up for SIT and PE was, according to Foa et al., based on SIT's focus on immediate relief due to the anxiety management nature of SIT. But the reversal may also be due to the fact that Foa et al.'s operationalization of SIT contained the skill acquisition phase of SIT but not the application and follow through phase (Meichenbaum, 1985, 2007). This final phase of SIT involves opportunities for patients to apply the variety of coping skills in a gradient of stressors and uses techniques such as imagery and behavioral rehearsal, modeling, role playing, and graded in vivo exposure, in order to prevent relapse. This phase of SIT was omitted by Foa et al., presumably because it contained exposure elements, which would overlap with PE. The omission of this phase of SIT could well explain the reversal. What is quite remarkable in the Foa et al. (1991) trial is that there were few differences in outcome between PE, a trauma focused therapy, and SIT, a non-trauma focused therapy, even when SIT was not properly implemented. Ironically, advocates of a particular treatment often claim that when their preferred treatment is found to be inferior to a comparison it was because their treatment was not faithfully operationalized (see e.g., Bhar & Beck, 2009; Clark et al., 2007 ; Jacobson, 1991). This occurs in the PTSD literature as well. Devilly and Foa (2001), taking exception to Tarrier et al.'s (1999) conclusion that “a significantly greater number of patients receiving IE [imaginal exposure] worsened over treatment” (p. 17) than in CT, claimed that Tarrier delivered IE inappropriately: For example, although Tamer et al. noted that the therapists guided the participants to speak in the present tense, was this integrated into the session effectively? Did the therapist note “hot spots” where appropriate and habituate the participants to these? (p. 115). Similarly, Taylor (2004), seeking to discredit Bryant et al. (2003) conclusion that cognitive restructuring (CR) added to IE was more efficacious than IE alone, noted, “In summary, this study does little to clarify the role of CR in the treatment of PTSD because of their [Bryant et al.'s ] atypical sample and unrepresentative exposure protocol” (p. 18). The finer points of this argument are critically important. These criticisms of treatment implementation suggest that the effectiveness of a treatment protocol depends on absolute adherence to a protocol. But then how can so many various treatments be effective? Even if one accepts Ehlers et al.'s argument that trauma-focused treatments are superior, this leaves 16 distinct protocols (see Table 2), each of which purportedly must be provided exactly as stipulated by the developers of the treatment. However, we know that therapists in clinical trials vary in the way they deliver treatment but that their adherence to the protocol does not predict outcome (Webb, DeRubeis, & Barber, 2010). How is it that a community of scientists can be so concerned about the precise manner in which two legitimate treatments for PTSD are implemented in comparative trials and yet be so unconcerned about conclusions based on the comparisons of a treatment to a protocol that has no rationale, proscribes the therapist from discussing the event that has led to presenting concerns, contains no viable therapeutic actions, and would rarely, if ever, be used by therapists — i.e., supportive counseling? And why, if concerns about absolute adherence to a protocol are critical, do protocols get modified, as was the case with Foa et al. (1991), so as to reduce the overlap with the other treatment (see for a vivid example Clark et al., 1994; see Wampold, Imel, & Miller, 2009 for a discussion of this trial)? 6. Conclusions The objective of both Benish et al. (2008) and Ehlers et al. (2010) was to better understand the nature of PTSD treatments and improve the quality of care. Benish et al. (2008) found that when non-bona fide treatments were excluded from a meta-analysis, treatments produce 931 approximately equal benefits. Ehlers et al. (2010) agreed that the exclusion of these treatments was responsible for the absence of differences between treatments, but believe control treatments were legitimate and thus persist in their belief that trauma focused therapies are superior to non-trauma focused therapies inclusive of control treatments. Putting aside the discrepancies between Benish et al. (2008) and Ehlers et al. (2010) and Bisson and colleagues (Bisson & Andrew, 2009; Bisson et al., 2007b), it is clear that a diverse array of psychotherapies for PTSD can be remarkably effective (viz, all the treatments in the TFCBT column and EMDR, as shown in Table 2). These include treatments that explicitly exclude discussion of the trauma memory and those that exclusively focus on the trauma memory, treatments with exposure and treatments that intentionally exclude exposure, treatments that some clinical scientists classify as pseudo-science, as well as treatments in which the patient does not discuss anything with the therapist. Moreover, there is no evidence to suggest that the non-trauma focused bona fide treatments in Table 2 are inferior to the trauma focused treatments. However, it is generally true that patients who receive bona fide interventions do better than patients in control treatments (Smits & Hofmann, 2009). Accordingly, it seems that treatments developed as psychotherapy controls that restrict therapists from performing logical interventions should not be offered as first line treatments for PTSD. However, it seems unlikely that the supportive counseling interventions used as controls in RCTs have ever been offered as a first line treatment outside of the context of an RCT. Meta-analysis continues to represent the best chance for the unbiased aggregation of research findings such that the quality of care for PTSD can be rigorously examined and improved. Ehlers et al. (2010) criticized meta-analyses as oversimplifying complex research questions and requiring arbitrary decisions about treatment categories. However, a method is only as good as its implementation. We demonstrated that the trauma focused classification, which has guided previous PTSD meta-analyses, is arbitrary and an oversimplification of a diverse array of treatments. More problematic than the potential limitations of meta-analysis is Ehlers et al.'s review of select outcomes from individual studies to discredit the results of particular studies that do not support an a priori conclusion. This strategy foregoes attention to the corpus of studies and the general aggregated effect in lieu of interpretations of individual studies and sometimes particular outcomes variables within studies, which historically has resulted in flawed conclusions (Cooper, Hedges, & Valentine, 2009; Hunt, 1997; Wampold, 2001). A perusal of effective treatments for PTSD reveals that such treatments contain a variety of specific ingredients and a set of Table 3 Possible factors important to successful treatments of PTSD. Cogent psychological rationale that is acceptable to patient Systematic set of treatment actions consistent with the rationale Development and monitoring of a safe, respectful, and trusting therapeutic relationship Collaborative agreement about tasks and goals of therapy Nurturing hope and creating a sense of self efficacy Psychoeducation about PTSD Opportunity to talk about trauma (i.e., tell stories) Ensuring the patient's safety, especially if the patient has been victimized as in the case of domestic violence, neighborhood violence, or abuse Helping patients learn how to avoid revictimization Identifying patient resources, strengths, survival skills and intra and interpersonal resources and building resilience Teaching coping skills Examination of behavioral chain of events Exposure (covert in session and in-vivo outside of session) Making sense of traumatic event and patient's reaction to event Patient attribution of change to his or her own efforts Encouragement to generate and use social supports Relapse prevention 932 B.E. Wampold et al. / Clinical Psychology Review 30 (2010) 923–933 common factors. An examination of the treatments, an understanding of the nature of PTSD, and grounding in models of psychotherapy, yields a rich array of possible therapeutic components of effective treatments, as shown in Table 3. Some treatments emphasize some of these components more than others and may label them in different ways. To identify a focus on trauma as the key feature, among a rich array of features, which characterizes effective treatments for PTSD is not scientifically justified at this point in time. Indeed, until such time that there is sufficient evidence to conclude that one particular treatment for PTSD is superior to others or that some well defined ingredient is crucial to successful treatments of PTSD, it is not consistent with the research evidence to privilege some treatments over others. It may turn out that treatments focused on trauma are more effective than other treatments, but there is little evidence to support that conclusion at this time. Indeed, the preponderance of the evidence is consistent with Benish et al.'s conclusion that all bona fide treatments for PTSD are equally effective. References Anderson, T., Lunnen, K. M., & Ogles, B. M. (2010). Putting models and techniques in context. In S. D. Miller, B. L. Duncan, M. A. Hubble, & B. E. Wampold (Eds.), The heart and soul of change (pp. 143−166)., 2nd ed. Washington DC: American Psychological Association. Benish, S., Imel, Z. E., & Wampold, B. E. (2008). The relative efficacy of bona fide psychotherapies of post-traumatic stress disorder: A meta-analysis of direct comparisons. Clinical Psychology Review, 28, 746−758. Bhar, S. S., & Beck, A. T. (2009). Treatment integrity of studies that compare short-term psychodynamic psychotherapy with cognitive-behavior therapy. Clinical Psychology-Science and Practice, 16, 370−378. Bisson, J. I., & Andrew, M. (2009). Psychological treatments of post-traumatic stress disorder (PTSD) (Review). The Cochrane Review. Bisson, J. I., Ehlers, A., Matthews, R., Pilling, S., Richards, D., & Turner, S. (2007a). Data supplement to Psychological treatments for chronic post-traumatic stress disorder: Systematic review and meta-analysis. British Journal of Psychiatry, 190, 97−104. Bisson, J. I., Ehlers, A., Matthews, R., Pilling, S., Richards, D., & Turner, S. (2007b). Psychological treatments for chronic post-traumatic stress disorder: Systematic review and meta-analysis. British Journal of Psychiatry, 190, 97−104. Blanchard, E. B., Hickling, E. J., Devineni, T., Veazey, C. H., Galovski, T. E., Mundy, E., et al. (2003). A controlled evaluation of cognitive behavioral therapy for posttraumatic stress in motor vehicle accident survivors. Behaviour Research and Therapy, 41, 79−96. Bleiberg, K. L., & Markowitz, J. C. (2005). A pilot study of interpersonal psychotherapy for posttraumatic stress disorder. The American Journal of Psychiatry, 162, 181−183. Borkovec, T. D. (1990). Control groups and comparison groups in psychotherapy outcome research. National Institute on Drug Abuse Research Monograph, 104, 50−65. Borkovec, T. D., & Sibrava, N. J. (2005). Problems with the use of placebo conditions in psychotherapy research, suggested alternatives, and some strategies (or the pursuit of the placebo phenomenon. Journal of Clinical Psychology, 61, 805−818. Bradley, R., Greene, J., Russ, E., Dutra, L., & Westen, D. (2005). A Multidimensional MetaAnalysis of Psychotherapy for PTSD. American Journal of Psychiatry, 162, 214−227. Brom, D., Kleber, R. J., & Defares, P. B. (1989). Brief psychotherapy for posttraumatic stress disorders. Journal of Consulting and Clinical Psychology, 57, 607−612. Bryant, R. A., Moulds, M. L., Guthrie, R. M., Dang, S. T., & Nixon, R. D. V. (2003). Imaginal exposure alone and imaginal exposure with cognitive restructuring in treatment of posttraumatic stress disorder. Journal of Consulting and Clinical Psychology, 71, 706−712. Budge, S., Baardseth, T. P., Wampold, B. E., & Flűckiger, C. (2010). Researcher allegiance and supportive therapy: Pernicious affects on results of randomized clinical trials. European Journal of Counselling and Psychotherapy, 12, 23−39. Clark, D. M., Fairburn, C. G., & Wessely, S. (2007). Psychological treatment outcomes in routin NHS services: A commentary on Stiles et al. (2007). Psychological Medicine, 38, 629−634. Clark, D. M., Salkovskis, P. M., Hackmann, A., Middleton, H., Anastasiades, P., & Gelder, M. (1994). A comparison of cognitive therapy, applied relaxation, and imipramine in the treatment of panic disorder. British Journal of Psychiatry, 164, 759−769. Classen, C., Koopman, C., Nevill-Manning, K., & Spiegel, D. (2001). A preliminary report comparing trauma-focused and present-focused group therapy against a waitlisted condition among childhood sexual abuse survivors with PTSD. Journal of Aggression, Maltreatment & Trauma, 4, 265−288. Cloitre, M., Koenen, K. C., Cohen, L. R., & Han, H. (2002). Skills training in affective and interpersonal regulation followed by exposure: A phase-based treatment for PTSD related to childhood abuse. Journal of Consulting and Clinical Psychology, 70, 1067−1074. Cooper, H., Hedges, L. V., & Valentine, J. C. (2009). The handbook of research synthesis and meta-analysis, 2nd ed. New York: Russel Sage Foundation. Cottraux, J., Note, I., Yao, S. N., de Mey-Guillard, C., Bonasse, F. O., Djamoussian, D., et al. (2008). Randomized controlled comparison of cognitive behavior therapy with Rogerian supportive therapy in chronic post-traumatic stress disorder: A 2-year follow-up. Psychotherapy and Psychosomatics, 77, 101−110. Crits-Christoph, P., Baranackie, K., Kurcias, J. S., Carroll, K., Luborsky, L., McLellan, T., et al. (1991). Meta-analysis of therapist effects in psychotherapy outcome studies. Psychotherapy Research, 1, 81−91. Crits-Christoph, P., & Mintz, J. (1991). Implications of therapist effects for the design and analysis of comparative studies of psychotherapies. Journal of Consulting and Clinical Psychology, 59, 20−26. Dar, R., Serlin, R. C., & Omer, H. (1994). Misuse of statistical tests in three decades of psychotherapy research. Journal of Consulting and Clinical Psychology, 62, 75−82. Davidson, P. R., & Parker, K. C. H. (2001). Eye movement desensitization and reprocessing (EMDR): A meta-analysis. Journal of Consulting and Clinical Psychology, 69, 305−316. Descilo, T., Vedamurtachar, A., Gerbarg, P. L., Nagaraja, D., Gangadhar, B. N., Damodaran, B., et al. (2009). Effects of a yoga breath intervention alone and in combination with an exposure therapy for post-traumatic stress disorder and depression in survivors of the 2004 South-East Asia tsunami. Acta Psychiatrica Scandinavica, 1−12. de Shazer, S., Berg, I. K., Lipchik, E., & Nunnally, E. (1986). Brief therapy: Focused solution development. Family Process, 25, 207−221. Devilly, G. J., & Foa, E. B. (2001). The investigation of exposure and cognitive therapy: Comment on Tarrier et al. (1999). Journal of Consulting and Clinical Psychology, 69, 114−116. D'Zurilla, T. J., & Goldfried, M. R. (1971). Problem solving and behavior modification. Journal of Abnormal Psychology, 78, 107−126. Echeburúa, E., de Corral, P., Zubizarreta, I., & Sarasua, B. (1997). Psychological treatment of chronic posttraumatic stress disorder in victims of sexual aggression. Behavior Modification, 21, 433−456. Ehlers, A., Bisson, J., Clark, D. M., Creamer, M., Pilling, S., Richards, A., et al. (2010). Do all psychological treatments really work the same in posttraumatic stress disorder. Clinical Psychology Review, 30, 269−276. Ehlers, A., Gene-Cos, N., & Perrin, S. (2009). Low recognition of posttraumatic stress disorder in primary care. London Journal of Primary Care, 2, 36−42. Finkelhor, D. (1987). The trauma of child sexual abuse: Two models. Journal of Interpersonal Violence, 2, 348−366. Foa, E. B., Dancu, C. V., Hembree, E. A., Jaycox, L. H., Meadows, E. A., & Street, G. P. (1999). A comparison of exposure therapy, stress inoculation training, and their combination for reducing posttraumatic stress disorder in female assault victims. Journal of Consulting and Clinical Psychology, 67, 194−200. Foa, E. B., Rothbaum, B. O., Riggs, D. S., & Murdock, T. B. (1991). Treatment of posttraumatic stress disorder in rape victims: A comparison between cognitivebehavioral procedures and counseling. Journal of Consulting and Clinical Psychology, 59, 715−723. Frank, J. D., & Frank, J. B. (1991). Persuasion and healing: A comparative study of psychotherapy, 3rd ed. Baltimore: Johns Hopkins University Press. Gloaguen, V., Cottraux, J., Cucherat, M., & Blackburn, I. (1998). A meta-analysis of the effects of cognitive therapy in depressed patients. Journal of Affective Disorders, 49, 59−72. Grissom, R. J. (1996). The magical number.7 ± .2: Meta-meta-analysis of the probability of superior outcome in comparisons involving therapy, placebo, and control. Journal of Consulting and Clinical Psychology, 64, 973−982. Herbert, J. D., Lilienfeld, S. O., Lohr, J. M., Montgomery, R. W., O'Donohue, W. T., Rosen, G. M., et al. (2000). Science and pseudoscience in the development of eye movement desensitization and reprocessing: Implications for clinical psychology. Clinical Psychology Review, 20, 945−971. Hollifield, M., Sinclair-Lian, N., Warner, T. D., & Hammerschlag, R. (2007). Acupuncture for posttraumatic stress disorder: A randomized controlled pilot trial. Journal of Nervous and Mental Disease, 195, 504−513. Hollon, S. D. (1999). Allegiance effects in treatment research: A commentary. Clinical Psychology: Science and Practice, 6, 107−112. Hunt, M. (1997). How science takes stock: The story of meta-analysis. New York: Russell Sage Foundation. Huppert, J. D., Bufka, L. F., Barlow, D. H., Gorman, J. M., Shear, M. K., & Woods, S. W. (2001). Therapists, therapists variables, and cognitive behavioral therapy outcomes in a multicenter trial for panic disorder. Journal of Consulting and Clinical Psychology, 69, 747−755. Imel, Z. E., & Wampold, B. E. (2008). The common factors of psychotherapy. In S. D. Brown, & R. W. Lent (Eds.), Handbook of counseling psychology (pp. 249−266)., 4th ed. New York: Wiley. Jacobson, N. S. (1991). Behavioral versus insight-oriented martial therapy: Labels can be misleading. Journal of Consulting and Clinical Psychology, 59, 142−145. Jacobson, N. S., Dobson, K. S., Truax, P. A., Addis, M. E., Koerner, K., Gollan, J. K., et al. (1996). A component analysis of cognitive-behavioral treatment for depression. Journal of Consulting and Clinical Psychology, 64, 295−304. Jakupcak, M., Roberts, L. J., Martell, C., Mulick, P., Michael, S., Reed, R., et al. (2006). A pilot study of behavioral activation for veterans with posttraumatic stress disorder. Journal of Traumatic Stress, 19, 387−391. Kim, D. M., Wampold, B. E., & Bolt, D. M. (2006). Therapist effects in psychotherapy: A random effects modeling of the NIMH TDCRP data. Psychotherapy Research, 16, 161−172. Krakow, B., Johnston, L., Melendrez, D., Hollifield, M., Warner, T. D., Chavez-Kennedy, D., et al. (2001). An open-label trial of evidence-based cognitive behavior therapy for nightmares and insomnia in crime victims with PTSD. The American Journal of Psychiatry, 158, 2043−2047. Kudler, H. S., Blank, A. S., Jr, Krupnick, J. L., Foa, E. B., Keane, T. M., & Friedman, M. J. (2000). Psychodynamic therapy. In Effective treatments for PTSD: Practice guidelines from the International Society for Traumatic Stress Studies (pp. 339−341). New York, NY US: Guilford Press. B.E. Wampold et al. / Clinical Psychology Review 30 (2010) 923–933 Lambert, M. J., & Bergin, A. E. (1994). The effectiveness of psychotherapy. In A. E. Bergin, & S. L. Garfield (Eds.), Handbook of psychotherapy and behavior change (pp. 143−189)., 4th ed. New York: Wiley. Levy, S., & Lemma, A. (2004). The perversion of loss: Psychoanalytic perspectives on trauma. Philadelphia, PA US: Whurr Publishers. Luborsky, L., Diguer, L., Seligman, D. A., Rosenthal, R., Krause, E. D., Johnson, S., et al. (1999). The researcher's own therapy allegiances: A “wild card” in comparisons of treatment efficacy. Clinical Psychology: Science and Practice, 6, 95−106. Luborsky, L., Singer, B., & Luborsky, L. (1975). Comparative studies of psychotherapies: Is it true that “everyone has won and all must have prizes?”. Archives of General Psychiatry, 32, 995−1008. Marks, I., Lovell, K., Noshirvani, H., Livanou, M., & Thrasher, S. (1998). Treatment of posttraumatic stress disorder by exposure and/or cognitive restructuring: A controlled study. Archives of General Psychiatry, 55, 317−325. McDonagh, A., Friedman, M., McHugo, G., Ford, J., Sengupta, A., Mueser, K., et al. (2005). Randomized trial of cognitive-behavioral therapy for chronic posttraumatic stress disorder in adult female survivors of childhood sexual abuse. Journal of Consulting and Clinical Psychology, 73, 515−524. McNally, R. J. (1999). EMDR and Mesmerism: A comparative historical analysis. Journal of Anxiety Disorders, 13, 225−236. Meador, B., & Rogers, C. (1973). Client-centered therapy. In R. Corsini (Ed.), Current psychotherapies. Itasca, IL: Peacock. Meichenbaum, D. (1985). Stress inoculation training. Elmsford, NY: Pergamon Press. Meichenbaum, D. (2007). Stress inoculation training: A preventative and treatment approach. In P. M. Lehrer, R. L. Woolfolk, & W. S. Sime (Eds.), Principles and practice of stress management (pp. 497−518). New York City: Guilford Press. Mohr, D. C., Spring, B., Freedland, K. E., Beckner, V., Arean, P., Hollon, S. D., et al. (2009). The selection and design of control conditions for randomized controlled trials of psychological interventions. Psychotherapy and Psychosomatics, 78, 275−284. National Collaborating Centre for Mental Health. (2005). Post-traumatic stress disorder: The management of PTSD in adults and children in primary and secondary care. National Clinical Practice Guideline, vol. 26, . London: Gaskell and the British Psychological Society. Nezu, A. M., & Perri, M. G. (1989). Social problem-solving therapy for unipolar depression: An inital dismantling investigation. Journal of Consulting and Clinical Psychology, 57, 408−413. Novalis, P. N., Rojcewicz, S. J., Jr, & Peele, R. (1993). Clinical manual of supportive psychotherapy. Washington: American Psychiatric Press. Parloff, M. B. (1986). Frank's ‘common elements’ in psychotherapy: Nonspecific factors and placebos. American Journal of Orthopsychiatry, 56, 521−530. Peniston, E. G., & Kulkosky, P. J. (1991). Alpha-theta brainwave neuro-feedback therapy for Vietnam veterans with combat-related post-traumatic stress disorder. Medical Psychotherapy: An International Journal, 4, 47−60. Pingitore, D., Scheffler, R., Haley, M., Sentell, T., & Schwalm, D. (2001). Professional psychology in a new era: Practice-based evidence from California. Professional Psychology: Research and Practice, 32, 585−596. Pinsker, H. (2002). A primer of supportive psychotherapy. Hillsdale, NJ: Analytic Press. Powers, M. B., Halpern, J. M., Ferenschak, M. P., Gillihan, S. J., & Foa, E. B. (2010). A metaanalytic review of prolonged exposure for posttraumatic stress disorder. Clinical Psychology Review, 30, 635−641. Price, C. J., McBride, B., Hyerle, L., & Kivlighan, D. R. (2007). Mindful awareness in bodyoriented therapy for female verterans with post-traumatic disorder taking a prescription for analgesics for chronic pain: A feasibility study. Alternative therapies, 13, 32−40. Rogers, C. R. (1951). Client-centered therapy. Boston: Houghton Mifflin. Rosen, C. S., Chow, H. C., Finney, J. F., Greenbaum, M. A., Moos, R. H., Sheikh, J. I., et al. (2004). VA practice patterns and practice guidelines for treating posttraumatic stress disorder. Journal of Traumatic Stress, 17, 213−222. Rosen, G. M. (1999). Treatment fidelity and research on Eye Movement Desensitization and Reprocessing (EMDR). Journal of Anxiety Disorders, 13, 173−184. Schnurr, P. P. (2007). The rocks and hard places in psychotherapy outcome research. Journal of Traumatic Stress, 20, 779−792. 933 Schnurr, P. P., Friedman, M. J., Engel, C. C., Foa, E. B., Shea, M. T., Chow, B. K., et al. (2007). Cognitive behavioral therapy for posttraumatic stress disorder in women: A randomized controlled trial. JAMA: Journal of the American Medical Association, 297, 820−830. Seidler, G. H., & Wagner, F. E. (2006). Comparing the efficacy of EMDR and traumafocused cognitive-behavioral therapy in the treatment of PTSD: A meta-analytic study. Psychological Medicine, 36, 1515−1522. Shadish, W. R., & Sweeney, R. B. (1991). Mediators and moderators in meta-analysis: There's a reason we don't let dodo birds tell us which psychotherapies should have prizes. Journal of Consulting and Clinical Psychology, 59, 883−893. Shapiro, D. A., & Shapiro, D. (1982). Meta-analysis of comparative therapy outcome studies: A replication and refinement. Psychological Bulletin, 92, 581−604. Shapiro, F. (1989). Efficacy of eye movement desensitization procedure in the treatment of traumatic memories. Journal of Traumatic Stress, 2, 199−203. Smith, M. L., Glass, G. V., & Miller, T. I. (1980). The benefits of psychotherapy. Baltimore: The Johns Hopkins University Press. Smits, J. A. J., & Hofmann, S. G. (2009). A meta-analytic review of the effects of psychotherapy control conditions for anxiety disorders. Psychological Medicine, 39, 229−239. Stein, D. J., Cloitre, M., Nemeroff, C. B., Nutt, D. J., Seedat, S., & Shaley, A. Y. (2009). Cape town consensus on posttraumatic stress disorder. CNS Spectrums, 14(Suppl 1), 52−58. Tarrier, N., Pilgrim, H., Sommerfield, C., Faragher, B., Reynolds, M., Graham, E., et al. (1999). A randomized trial of cognitive therapy and imaginal exposure in the treatment of chronic posttraumatic stress disorder. Journal of Consulting and Clinical Psychology, 67, 13−18. Taylor, S. (2004). Combined imaginal exposure and cognitive restructuring therapy is more effective than supportive counselling for treating post-traumatic stress disorder. Evidence-based Mental Health, 7, 18. Vaughan, K., Armstrong, M. S., Gold, R., & O'Connor, N. (1994). A trial of eye movement desensitization compared to image habituation training and applied muscle relaxation in post-traumatic stress disorder. Journal of Behavior Therapy and Experimental Psychiatry, 25, 283−291. Wampold, B. E. (2001). The great psychotherapy debate: Model, methods, and findings. Mahwah, NJ: Lawrence Erlbaum Associates. Wampold, B. E. (2007). Psychotherapy: The humanistic (and effective) treatment. American Psychologist, 62, 857−873. Wampold, B. E., & Brown, G. S. (2005). Estimating therapist variability: A naturalistic study of outcomes in managed care. Journal of Consulting and Clinical Psychology, 73, 914−923. Wampold, B. E., Imel, Z. E., & Miller, S. D. (2009). Barriers to the dissemination of empirically supported treatments: Matching messages to the evidence. The Behavior Therapist, 32, 144−155. Wampold, B. E., Minami, T., Baskin, T. W., & Tierney, S. C. (2002). A meta-(re)analysis of the effects of cognitive therapy versus “other therapies” for depression. Journal of Affective Disorders, 68, 159−165. Wampold, B. E., Mondin, G. W., Moody, M., Stich, F., Benson, K., & Ahn, H. (1997). A meta-analysis of outcome studies comparing bona fide psychotherapies: Empirically, “All must have prizes”. Psychological Bulletin, 122, 203−215. Wampold, B. E., & Serlin, R. C. (2000). The consequences of ignoring a nested factor on measures of effect size in analysis of variance. Psychological Methods, 5, 425−433. Webb, C. A., DeRubeis, R. J., & Barber, J. P. (2010). Therapist adherence/competence and treatment outcome: A meta-analytic review. Journal of Consulting and Clinical Psychology, 78, 200−211. Westen, D., Novotny, C. M., & Thompson-Brenner, H. (2004). The empirical status of empirically supported psychotherapies: assumptions, findings, and reporting in controlled clinical trials. Psychological Bulletin, 130, 631−663. Zlotnick, C., Shea, T. M., Rosen, K., Simpson, E., Mulrenin, K., Begin, A., et al. (1997). An affect-management group for women with posttraumatic stress disorder and histories of childhood sexual abuse. Journal of Traumatic Stress, 10, 425−436.