42648

The Counseling Psychologist

s) 2011

ermission:
TCP41110.1177/001100001244264

ournalsPermissions.nav

Major Contribution: Multicultural Supervision

The Counseling Psychologist

Effective and Ineffective 41(1) 28-47

© The Author(s) 2013
Reprints and permission:
Supervision sagepub.com/journalsPermissions.nav
DOI: 10.1177/0011000012442648
http://tcp.sagepub.com

Nicholas Ladany1,Yoko Mori2,

and Kristin E. Mehr3

Abstract
Although supervision is recognized as a significant tenant of professional
growth for counseling and psychotherapy students, the variability of the effec-
tiveness, or ineffectiveness, of supervision has come under scrutiny in recent
times. Our sample of 128 participants shed light on the most effective (e.g.,
encouraged autonomy, strengthened the supervisory relationship, and facili-
tated open discussion) and most ineffective (e.g., depreciated supervision, per-
formed ineffective client conceptualization and treatment, and weakened the
supervisory relationship) supervisor skills, techniques, and behaviors. Moreover,
effective and ineffective behaviors, along with best and worst supervisors, were
significantly differentiated based on the supervisory working alliance, supervi-
sor style, supervisor self-disclosure, supervisee nondisclosure, and supervisee
evaluation. Implications for supervision competencies and supervisor account-
ability are discussed.

Keywords
supervision, training, education, supervisor effectiveness

1
Loyola Marymount University, Los Angeles, California, USA
2
University of Illinois at Chicago, USA
3
Lehigh University, Bethlehem, Pennsylvania, USA

Corresponding Author:
Nicholas Ladany, PhD, Professor & Director, Counseling Program, Suite 1500, 1 LMU Dr.,
Loyola Marymount University, Los Angeles, CA 90094, USA.
Email: nladany@lmu.edu

Downloaded from tcp.sagepub.com by guest on July 25, 2016

Ladany et al. 29

Over the past decade, the empirical literature has pointed out that supervi-
sion, arguably with benevolent intentions, has also proven at times to be
problematic, counterproductive, harmful, and unethical (Ellis, 2001; Gray,
Ladany, Walker, & Ancis, 2001; Ladany, Lehrman-Waterman, Molilnaro, &
Wolgast, 1999; Nelson & Friedlander, 2001). The competency-based super-
vision movement has, in part, attempted to begin to prevent or mollify the
negative side of supervision (Falender, Burnes, & Ellis, 2013; Falender &
Shafranske, 2007). Some of the earliest empirical attempts to look at the
influence of supervision on supervisees examined the extent to which super-
vision consisted of good versus bad events (Worthen & McNeill, 1996), best
versus worst sessions (Martin, Goodyear, & Newton, 1987), best versus
worst experiences (Allen, Szollos, & Williams, 1986), or successful versus
unsuccessful (Tracey & Sherry, 1993). Methods typically employed were
case studies or solely qualitative, or solely quantitative, and all focused on
one supervision experience with supervisees in training. In general, these
studies offered partial insight into some of the behaviors that supervisors
exhibit that positively or negatively influence supervisees; however, few
were methodologically sound investigations and most were conducted over a
decade ago (Ellis & Ladany, 1997).
More recently, a few researchers have qualitatively examined primarily
effective supervisor interventions or supervisory events. Jacobsen and Tanggaard
(2009) investigated beginning Danish supervisees’ experiences of good and
bad supervisory events. His findings indicated that beginning supervisees
found supervisor guidance and support helpful; however, there were large
individual differences with respect to what supervisees found unhelpful. Bang
and Park (2009) in a qualitative study of Korean supervisors found that teach-
ing and exploration of personal issues were believed to be uniquely helpful to
supervisees. Finally, Ancis and Marshall (2010) were able to identify theoreti-
cally based interventions (e.g., supervisor-focused personal development of
the trainee) used by multiculturally competent supervisors.
The purpose of the present investigation was to extend the literature by
using a mixed-method design, qualitative and quantitative inquiry, across
multiple supervision experiences of the same participant supervisees. In addi-
tion, we were interested in supervisees who had experienced multiple types
of supervisors over the course of their professional development. Specifically,
we sought to identify supervisor skills, techniques, and behaviors that were
deemed effective in facilitating supervisee growth versus those skills, tech-
niques, and behaviors that were ineffective, or limited or hindered supervisee
growth. We believed this level of specificity, in particular, would extend the
literature to date, which tended to focus more on larger events in supervision.

Downloaded from tcp.sagepub.com by guest on July 25, 2016

30 The Counseling Psychologist 41(1)

A second primary purpose of this investigation was to examine the relation-

ship between effective and ineffective supervisor behaviors and supervi-
sion process and outcome, specifically the supervisory working alliance,
supervisor style, supervisor self-disclosure, supervisee nondisclosure, and
supervisee evaluation. A third and final purpose of our investigation was to
determine the differences between the best and worst supervisors in relation
to the supervisory working alliance, supervisor style, supervisor self-
disclosure, supervisee nondisclosure, and supervisee evaluation. It was
hypothesized that supervisors considered best/most effective, in comparison
to those deemed worst/least effective, would have been perceived as having
created a stronger working alliance, demonstrated more positive aspects of
supervisor style, would have disclosed more, would have facilitated less
supervisee nondisclosure, and would have developed a more favorable evalu-
ation process. In sum, we believed that the effective and ineffective skills,
techniques, and behaviors identified would form the basis for competency
benchmarks in supervision (Falender & Shafranske, 2007; Fouad et al.,
2009). We anticipated that our data would offer evidence both in support and
perhaps in contrast to the competency benchmarks.

Method
Participants

Trainees. One hundred and eighty individuals initiated participation in this

study. A total of 128 participants (100 female, 27 male, 1 unknown) averag-
ing 35.4 years in age fully completed the study; the data of 52 participants
were excluded because they did not complete the survey. The majority of
participants (109, 85.2%) identified as European American/White; eight par-
ticipants (6.3%) identified as Hispanic/Latino(a), five (3.9%) identified as
African American/Black, three (2.3%) identified as Asian American or
Pacific Islander, two (1.6%) identified as other race, and one did not specify
race. Participants reported currently pursuing or having already received
degrees from primarily (118) doctoral programs in clinical psychology
(57.8%), counseling psychology (28.9%), school psychology (4.7%), and
other related programs (8.6%). Participants described their experience as the
beginning level of practicum (2.3%), advanced level of practicum (17.2%),
internship (30.5%), postinternship (5.5%), or postdoctorate (42.2%), and
2.3% of participants did not describe their experience. For those participants
who were currently still in training, they had been in their program on aver-
age for 4.36 years (SD = 1.52).

Downloaded from tcp.sagepub.com by guest on July 25, 2016

Ladany et al. 31

Best supervisors. Each participant reported one “best supervisor.” Thus,

there were a total of 128 best supervisors described in the current study. In
terms of race, the supervisors were White (109, 85.2%), Hispanic/Latino (5,
3.9%), African American (5, 3.9%), Asian American (4, 3.1%), other race (2,
1.6%), and unknown (3). More than half (56.3%) of the supervisors were
female, while 41.4% were male. The majority of the supervisors (114, 87.6%)
had a doctoral degree, while the remainder had master’s degrees in counsel-
ing or social work. The degrees of three supervisors were not identified.
Within supervision, students were often evaluated based on a pass or fail
basis (76, 59.4%), letter grade (25, 19.5%), or other system (24, 18.8%).
Supervision was given in various sites: college counseling centers (44,
34.4%), community mental health centers (24, 18.8%), private hospitals (10,
7.8%), state hospitals (8, 6.3%), veterans hospitals (12, 9.4%), other (26,
20.3%), and unreported (4). These supervisors were employed at college
counseling centers (33, 25.8%), hospital settings (31, 24.2%), academic set-
tings (26, 20.3%), community mental health centers (17, 13.3%), and private
practice (14, 10.9%). The best supervisor provided an average 1.5 hours per
week for an average total of 48.9 sessions.
Worst supervisors. Each participant reported one “worst supervisor.” Thus,
there were a total of 128 worst supervisors described in the current study. In
terms of race, the supervisors were White (111, 86.7%), Hispanic/Latino (6,
4.7%), African American (3, 2.3%), Asian (3, 2.3%), and unknown (5). In
terms of gender, about half of the supervisors were female (64), slightly less
than half of the supervisors were male (60), and four supervisors’ genders
were not identified. The majority of supervisors were identified as holding a
doctoral degree (105, 82.0%), while the remainder had master’s degrees in
counseling and social work. The degrees of five supervisors were not identi-
fied. In supervision, the majority of trainees were evaluated on a pass or fail
basis (76, 59.4%), many received letter grades (30, 23.4%), some used
another evaluation system (17, 13.3%), and five participants did not respond
to the question. Supervision was given in college counseling centers (32,
25.0%), community mental health centers (32, 25.0%), private hospitals (13,
10.2%), state hospitals (10, 7.8%), veterans hospitals (10, 7.8%), and other
sites (26, 20.3%), and five participants did not respond to the question. These
supervisors were employed at hospital settings (32, 25.0%), academic set-
tings (29, 22.7%), college counseling centers (20, 15.6%), community mental
health centers (27, 21.1%), and private practice (10, 7.8%), and 10 partici-
pants did not respond to the question. The worst supervisor provided an aver-
age of 1.4 hours per week for an average total of 32.7 sessions.

Downloaded from tcp.sagepub.com by guest on July 25, 2016

32 The Counseling Psychologist 41(1)

Measures
Supervisee evaluation of supervisor form. A qualitative questionnaire was
developed for utilization in this study to inquire about helpful and hindering
behaviors of best/most effective and worst/least effective supervisors. Par-
ticipants were instructed to report at least three supervisor skills, techniques,
or behaviors that facilitated their growth as a supervisee. They were also
asked to list at least three supervisor skills, techniques, or behaviors that lim-
ited, or hindered, their growth as a supervisee. Participants responded to this
form, and thus reported helpful and hindering behaviors, for each
supervisor.
Working Alliance Inventory/Supervision–Short Form (WAI/S-Short). The WAI/
S-Short is a 12-item self-report questionnaire utilized to assess the three con-
structs of Bordin’s (1983) working alliance model: agreement on the goals of
supervision, agreement on the tasks of supervision, and an emotional bond
between the trainee and supervisor. The Working Alliance Inventory (WAI;
Horvath & Greenberg, 1989) and the Working Alliance Inventory–Short
(WAI-Short; Tracey & Kotovic, 1989) are widely utilized as measures of the
therapeutic alliance. The Working Alliance Inventory/Supervision (WAI/S;
Bahrick, 1989) and the WAI/S-Short are modified versions for supervision of
the WAI and WAI-Short, respectively. The supervisory working alliance has
been found to be one of the most robust supervision variables examined to
date (Ladany & Inman, in press). The internal consistency coefficients of the
total WAI/S-Short, Goal subscale, Task subscale, and Bond subscale for the
best supervisors were .85, .73, .64, and .78, respectively. The internal consis-
tency coefficients for the worst supervisors were .84, .76, .64, and .77,
respectively.
Supervisory Styles Inventory (SSI). The SSI (Friedlander & Ward, 1984) is a
25-item self-report questionnaire. The scale assesses three constructs: Attrac-
tive (e.g., friendliness, warmth, supportiveness), Interpersonally Sensitive
(commitment, therapeutic, perceptive), and Task Oriented (goal-oriented,
thorough, focused). The internal consistency coefficients in this study were
.84, .90, .86, and .78 for best supervisors. For worst supervisors, the internal
consistency coefficients were .85, .91, .86, and .84.
Supervisor Self-Disclosure Index (SSDI). The SSDI (Ladany & Lehrman-
Waterman, 1999) is a nine-item self-report questionnaire that measures
supervisors’ self-disclosure in the supervisory relationship. The items
describe various types of self-disclosing statements (e.g., “My supervisor
self-discloses information related to her or his present experiences”) made by

Downloaded from tcp.sagepub.com by guest on July 25, 2016

Ladany et al. 33

their supervisors. The internal consistency coefficient of the SSDI in the cur-
rent study was .83 for the best supervisors and .89 for the worst supervisors.
Trainee Disclosure Scale (TDS). The TDS (Ladany, Walker, Pate-Carolan, &
Gray Evans, 2008) is a 13-item self-report questionnaire that measures the
degree to which trainees withhold information in the supervisory relation-
ship. The scale was rationally and theoretically constructed based upon non-
disclosure types (e.g., negative reactions to supervisor, personal issues, and
clinical mistakes) described in the literature (Ladany, Hill, Corbett, & Nutt,
1996). The internal consistency of the SNI was adequate for both best super-
visor (α = .83) and worst supervisor (α = .80).
Evaluation Process Within Supervision Inventory (EPSI). The EPSI (Lehrman-
Waterman & Ladany, 2001) is a 21-item self-report scale that assesses the
evaluation process in supervision. The scale consists of two constructs: Goal
Setting (e.g., clarity, specificity) and Feedback (e.g., promptness objectivity).
In the current study, the internal consistency coefficients for the total scale,
Goal Setting subscale, and Feedback subscale for the best supervisor were
.87, .84, and .71. The internal consistency coefficients for the worst supervi-
sor were .81, .73, and .82, respectively.

Procedure
Data collection was solicited through multiple sources. Invitation to the
study with the link to an online survey was electronically sent to program
directors of American Psychological Association (APA)-accredited doctoral
programs in counseling psychology, clinical psychology, and school psy-
chology and master’s programs that are accredited by the Council for
Accreditation of Counseling and Related Educational Programs, as well as
Association of Psychology Postdoctoral and Internship Centers (APPIC)
internship training directors. Participants were instructed to reflect upon their
experiences with their best (most effective) supervisor and their worst (least
effective) supervisor. Participants were also invited to forward the survey to
anyone that might be interested in participating.

Results
Preliminary Analyses

To test for the potential confounding influence of the demographic variables on

the primary variables in this study, a series of multivariate analyses were con-
ducted. In each analysis, the demographic variable served as the independent

Downloaded from tcp.sagepub.com by guest on July 25, 2016

34 The Counseling Psychologist 41(1)

variable, while the primary variables served as the dependent variables. The
per comparison alpha coefficient was set to .001 to minimize Type I error and
yet maintain a conservative estimate of potential confounding effects.
Results indicate that none of the demographic variables (e.g., experience
level) were significantly related to any of the primary variables.

Descriptive Analyses
Utilizing qualitative information obtained from the Supervisee Evaluation of
Supervisor Form, discovery-oriented analysis was used to develop mutually
exclusive categories for effective supervisor behaviors and ineffective super-
visor behaviors. The effective supervisor behavior categories can be seen in
Table 1 and the ineffective supervisor behavior categories can be seen in
Table 2. In general, trainees reported an average of 5.1 effective behaviors
and 3.2 ineffective behaviors for “Best” supervisors. Trainees reported an
average of 2.6 effective behaviors and 3.5 ineffective behaviors for “Worst”
supervisors.”

Comparison of the “Best” Supervisors’

and “Worst” Supervisors’ Behaviors
The behaviors of the “best” supervisors were compared to the behaviors of
the “worst” supervisors. An overall chi-square analysis was conducted to
compare frequencies of the effective behaviors of the “best” supervisors to
those of the “worst” supervisors. The analysis indicated that the comparison
of the frequencies of effective behaviors of the “best” and “worst” supervi-
sors was not significant, χ2(9, N = 934) = 2.61, p = .98. An overall chi-square
analysis was conducted to compare frequencies of the ineffective behaviors
of the best supervisors to those of the worst supervisors. The analysis indi-
cated that the comparison of the frequencies of ineffective supervisor behav-
iors of the “best” and “worst” supervisors was significant, χ2 (9, N = 742) =
63.52, p < .001. Follow-up cell chi-square analysis further suggested that
putting emphasis on evaluation and limitations, χ2(1, N = 742) = 8.96, p <
.001, weakening the supervisory relationship, χ2(9, N = 742) =4.15, p < .05,
and other negative behaviors, χ2(9, N = 742) = 5.95, p < .05, made supervi-
sion with “best” supervisors less effective. Ineffective client conceptualiza-
tion and treatment, χ2(9, N = 742) = 4.44, p < .05, and emphasis on evaluation
and limitations, χ2(9, N = 742) = 5.67, p < .05, made supervision with
“worst” supervisors less effective.

Downloaded from tcp.sagepub.com by guest on July 25, 2016

 Table 1. Content of Effective Supervisor Skills, Techniques, or Behavior: Definitions and Examples

Content Definition Examples

Encouraging Autonomy Supervisor’s encouragement of supervisee’s “Expressed trust in my ability to work
6.5%B; 4.1%W self-directed decision making and autonomously.”
performance as well as self-reflection and “Gave me flexibility to provide the type of therapy
independent thinking. I wanted to.”
Strengthened Supervisory Supervisor’s support, encouragement, “Being non-judgmental regarding mistakes.”
Relationship acceptance, respect, trust, empathy, “Demonstrated trust and confidence in my
23.4%B; 7.1%W open-mindedness, and other behaviors abilities.”
that contributed to the development of a “Making me feel as though I am working with them
positive supervisory relationship. instead of for them.”
“Providing safe space in supervision.”
Open Discussion Supervisor’s provision of an open forum of “Brainstormed ideas with me.”
10.9%B; 6.2%W discussion in which supervisor listened “Ability to ask difficult or uncomfortable questions
to supervisees in order to promote their regarding my thoughts, feelings, and perceived
learning and growth. development in a non-provoking though not
sugar-coated manner.”
Positive Personal, Supervisor’s positive personal characteristics “Honesty”
Professional Qualities that facilitated supervision, as well as “Friendliness”

Downloaded from tcp.sagepub.com by guest on July 25, 2016

11.7%B; 9.8%W professional qualities that served as a “Modeling how to be/interact with clients in
model or positive influence for the trainee. egalitarian and supportive way.”
Demonstration of Clinical Supervisor’s demonstration of clinical “Very knowledgeable in field of expertise.”
Knowledge/Skills knowledge and skills, as well as providing “Strong analysis of transference/
23.1%B; 29.9%W specific guidance and promoting countertransference issues.”
professional development. “Provided concrete examples of potential
intervention strategies.”

35
(continued)
 36
Table 1. (continued)

Content Definition Examples

Providing Constructive Supervisor challenged supervisee to take “Challenged me to ‘think outside box.’”
Challenge risks, to experiment, and to go beyond “He encouraged me to work with clients I would
4.6%B; 3.0%W their comfort zone. have hesitated to work with.”
Offering Feedback and Supervisor’s ability to provide positive, “Provided effective feedback.”
Reinforcements negative, or both types of feedback in “My supervisor demonstrated the ability to deliver
7.1%B; 7.4%W reaction to supervisee’s performance, constructive criticism in the most positive
as well as ability to check supervisee manner coupled with a positive.”
progress.
Engagement and Valuing Supervisor’s positive attitude towards and “Listened to tapes and provided ideas.”
Supervision active involvement in supervision. “Continued availability if needed.”
8.6%B; 10.7%W “Treating the time as important, i.e., not taking
phone calls, minimizing interruptions.”
Other Supervisor’s skills, techniques, or behaviors “Made me more resourceful by seeking help from
4.2%B; 20.4%W that facilitated supervisee’s growth but did other supervisors and peers.”
not fit into other categories or reported “Direct communication with staff.”
learning experiences that were not “Checked in career goals.”“(I) learned to deal with
directly related to the supervisor’s skills, a difficult supervisor.”

Downloaded from tcp.sagepub.com by guest on July 25, 2016

techniques, and behaviors
None There were no supervisors’ skills,  
0%B; 1.5%W techniques, or behaviors that facilitated
supervisees’ growth.
Note. %B and %W refers to the percentage for best and worst supervisors, respectively.
 Table 2. Content of Ineffective Supervisor Skills, Techniques, or Behaviors: Definition and Examples
Content Definition Examples
Depreciating Supervisor’s lack of involvement and fostering of “She seemed very busy. She often cut
Supervision growth in supervision. supervision hour short.”
31.6%B; 22.5%W “Took phone calls during supervision”
Ineffective Client Supervisor was unable to understand client well or “Did not seem to care about patients, expected
Conceptualization understood him/her incorrectly, as well as neglected supervision to be quick and factual with no
and Treatment to promote effective interventions. discussions of patient dynamics or issues.”
1.9%B; 3.8%W
Weakened Supervisor’s condescension, humiliation, distrust, lack “Lack of boundaries, took us out to drink and
Supervisory of support and respect, uncomfortable nonverbal ‘pal’ around—confused.”
Relationship behaviors, inappropriate boundaries, lack of “Entered into the relationship in an attacking mode.”
12.9%B; 12.6%W understanding of supervisee’s feelings, needs, and “Making me feel as though I am working for
desires, and other behaviors that impaired the them instead of with them.”
supervisory relationship. “Assumed that his interpretations of my feelings/
perceptions were correct and, if I disagreed, I
was in denial.”
Insufficient Supervisor did not focus enough attention on “Not very knowledgeable about the population.”

Downloaded from tcp.sagepub.com by guest on July 25, 2016

Knowledge and Skill demonstration and teaching of clinical knowledge and “Was not aware of multicultural issues in
Development skills, provided inadequate guidance, direction, and counseling and shied away from the topic
9.7%B; 11.3%W suggestions, or demonstrated ineffective clinical skills. when I would bring it up.”
Insufficient Supervisor did not engage in adequate amount “Did not listen carefully to my session tapes.”
Observation and of direct observation (live, video, or audio) and “Extensive positive feedback with no detail or
Feedback neglected to provide an adequate amount of constructive criticism.”
9.2%B; 6.1%W feedback, provided primarily positive feedback, or “Insufficient positive encouragement and/or
provided feedback that was inconsistent. feedback about what I did well.”
“Vague and nonspecific feedback.”

37
(continued)
 38
Table 2. (continued)

Content Definition Examples

Emphasis on Supervisor was focused on evaluation and primarily “Criticized my performance in a demeaning and
Evaluation and provided negative or critical feedback, or was personal way (‘I can’t believe they send me
Limitations punitive in the provision of feedback. people like you who don’t know what they’re
1.0%B; 6.1%W doing!’).”
Negative Personal Supervisor’s negative personal characteristics and “Judgmental”
and Professional traits that hindered supervision, as well as expressing “Opinionated”
Qualities a dominating attitude toward supervision. “Injects own agenda into our work at times.”
17.0%B; 21.4%W “Micromanaging me every step of the way.”
Lack of and Supervisor lacked, or had limited, theoretical “Limited theoretical orientation.”
Misapplication of orientation and approach or did not appropriately “Pushed own theoretical orientation on me,
Theory use theoretical concepts and knowledge. despite knowledge that we had different
4.6%B; 4.5%W orientations.”
Other Supervisor’s skills, techniques, or behaviors that “Attempts to ‘make me think on my own’ that
7.0%B; 4.5%W hindered supervisee’s growth but do not belong in were confusing.”
other categories or reported negative experiences “I had to come to her for help.”
in supervision not directly related to the supervisor’s “She was strikingly attractive, and although she
skills, techniques, and behaviors. did not in any way flaunt this, I was somewhat

Downloaded from tcp.sagepub.com by guest on July 25, 2016

distracted as well as motivated not to reveal
my own doubts.”
None There were no supervisor skills, techniques, or “Honestly, I can’t think of any, perhaps, because
5.1%B; 2.7%W behaviors that hindered supervisee’s growth. the good so far outweighed any limitations/
faults.”
Ladany et al. 39

Differences in the Quantitative Variables

Between “Best” and “Worst” Supervisors
The overall analysis of group differences was significant, Pillai’s trace = .83,
F(10, 245) = 122.67, p < .001. Follow-up analyses revealed that there were
significant (p < .001) differences between the “best” and “worst” supervisors
on all of the variables. Specifically, supervisees rated having a stronger emo-
tional bond, F(1, 254) = 685.54, p < .001, and greater agreement on the tasks,
F(1, 254) = 903.09, p < .001, and goals, F(1, 254) = 737.13, p < .001, of
supervision with their best supervisors as compared to their worst supervi-
sors. Additionally, best supervisors were reported as having more attractive,
F(1, 254) = 344.12, p < .001, interpersonally sensitive, F(1, 254) = 607.09,
p < .001, and task-oriented, F(1, 254) = 43.78, p < .001, supervisor styles
than worst supervisors. Best supervisors were also reported to have disclosed
more, F(1, 254) = 30.41, p < .001, in supervision. Supervisees reported less
nondisclosure, F(1, 254) = 297.26, p < .001, in supervision with their best
supervisors as compared to their worst supervisors. Lastly, supervisees
reported more effective goal-setting, F(1, 254) = 255.57, p < .001, and feed-
back processes, F(1, 254) =275.38, p < .001, for best supervisors than worst
supervisors.

Effective Behaviors of the “Best” Supervisors

The proportion of the variance in the criterion variables accounted for by the
predictor variables was not significant, Pillai’s trace = .80, F(90, 1,053) =
1.15, p =.170. Thus, on a multivariate level, effective supervisor behaviors
of the best supervisors in general were not related to trainee’s perception of
the components of the working alliance, supervisor styles, supervisor’s self-
disclosure, supervisee disclosure, and evaluation processes in supervision.

Ineffective Behaviors of the “Best” Supervisors

The proportion of the variance in the criterion variables accounted for by the
predictor variables was significant, Pillai’s trace = 1.22, F(100, 1,170) =
1.63, p < .001. Follow-up analyses indicated that ineffective supervisor
behaviors of the best supervisors were related to supervisee disclosure (R2 =
.15, F[10, 117] = 2.00, p =.039) and the attractive supervisory style (R2 = .18,
F[10, 117] = 2.50, p = .009), as well as trainee perception of the bond (R2 =
.28, F[10, 117] = 4.45, p < .001), tasks (R2 = .16, F[10, 117] = 2.18, p = .024),
and goals (R2 = .19, F[10, 117] = 2.73, p = .005) components of the alliance.

Downloaded from tcp.sagepub.com by guest on July 25, 2016

40 The Counseling Psychologist 41(1)

Specifically, weakening the supervisory relationship was negatively related

to trainee perception of the bond (β = –.34, p < .001), tasks (β = –.20, p =
.036), and goals (β = –.42, p < .001) components of the alliance. Weakening
the supervisory relationship was also negatively related to perception of an
attractive supervisory approach (β = –.25, p = .009) and supervisee disclo-
sure (β = –.28, p = .005). Emphasizing evaluation and limitations was nega-
tively related to trainee perception of the bond (β = –.27, p ≤ .001) and tasks
(β = –.22, p = .013) components of the alliance.

Effective Behaviors of the “Worst” Supervisors

The proportion of the variance in the criterion variables accounted for by the
predictor variables was not significant, Pillai’s trace = .92, F(100, 1,170) =
1.19, p = .108. Thus, on a multivariate level, effective supervisor behaviors
of the worst supervisors in general were not related to trainee’s perception of
the components of the working alliance, supervisor styles, supervisor’s self-
disclosure, supervisee disclosure, and evaluation processes in supervision.

Ineffective Behaviors of the “Worst” Supervisors

The proportion of the variance in the criterion variables accounted for by the
predictor variables was significant, Pillai’s trace = .96, F(90, 1,053) = 1.39,
p = .011. Follow-up analyses revealed that ineffective supervisor behaviors
of the worst supervisors were related to trainee’s perceptions of the bond
component of the alliance (R2 = .15, F[9, 118] = 2.28, p = .021), the feedback
process of evaluation (R2 = .16, F[9, 118] = 2.55, p = .010), the attractive
supervisory style (R2 = .21, F[9, 118] = 3.51, p = .001), and the task-oriented
supervisory style (R2 = .25, F[9, 118] = 4.27, p < .001). Specifically, empha-
sizing evaluation and limitations was negatively associated with trainee’s
perceptions of the bond component of the alliance (ϐ = –.34, p < .001), the
feedback process of evaluation (ϐ = –.38, p < .001), and the attractive super-
visory style (ϐ = –.35, p < .001). Depreciating supervision was negatively
associated with perception of a task-oriented supervisory style (ϐ = –.41, p <
.001) and positively associated with perception of an attractive supervisory
style (ϐ = .19, p = .038).

Discussion
A mixed-methods investigation invariably generates large swaths of data that
can produce findings that are overwhelming and often unnecessarily complex.

Downloaded from tcp.sagepub.com by guest on July 25, 2016

Ladany et al. 41

The purpose of our discussion is to summarize the results along two lines of
findings or themes: (a) effective and ineffective behaviors and supervisor
competencies and (b) salient variables for understanding supervision process
and outcome. Along both themes, theoretical, empirical, and practical impli-
cations are offered.

Effective and Ineffective Behaviors

What did our participants tell us about effective and ineffective supervisor
skills, techniques, and behaviors? First, the identified effective supervisor
skills, techniques, and behaviors arguably can be seen as a primer for super-
visor competencies. Across both the best and worst supervisors, a series of
consistent behaviors were identified. Adding to the robust findings in the
literature to date (Ladany & Inman, in press) as well as current conceptual-
izations of supervision competencies (e.g., Falender & Shafranske, 2007),
the supervisory relationship seems to be a critical component and founda-
tional competency on which to attend (Rodolfa et al., 2005). Hence, it seems
that the relationship has an important influence on supervisee learning.
Bringing to bear counseling skills, such as empathy and encouragement, into
the supervision context arguably provides a foundation for the efficacy of
additional supervisory interventions. One unique finding that has only
received cursory attention in the theoretical and practical literature (Kaslow
& Bell, 2008; McWhirter & McWhirter, 2007) is the recognition that
empowering the supervisee via encouraging autonomy and facilitating open-
ness to the supervisee’s ideas is valued by supervisees. Of course, much of
these ideas can be traced back to Bordin (1983), who encouraged supervisors
to give supervisees the opportunity to self-direct in supervision. Ultimately,
it seems that the participant supervisees offered an important variable to
consider in understanding the interpersonal dynamics in supervision.
The supervisor as expert clinician also was identified as an effective
supervisory skill. Specifically, supervisors who were able to demonstrate
their clinical knowledge, that is, to self-disclose clinical information that was
relevant to the supervisees presenting concerns and was in the service of the
supervisee, seemed particularly helpful. This finding coincides with theoreti-
cal, empirical, and practical writings on supervisor self-disclosure (Ladany &
Walker, 2003). Finally, feedback that was positive and challenging seemed
to be uniquely beneficial to supervisees. Hence, these results suggest that
supervisors would do well not to offer unbridled optimism about supervisees
and recognize that supervisees can handle challenges, and in fact, may
welcome challenges, especially within a positive supervisory relationship.

Downloaded from tcp.sagepub.com by guest on July 25, 2016

42 The Counseling Psychologist 41(1)

One interesting aside is that the effective and ineffective supervisor skills,
techniques, and behaviors did not seem to differentiate supervisees who were
at different experience (i.e., developmental) levels. This lack of finding coin-
cides with questions about developmental hypotheses discussed in the theo-
retical literature that to date have limited empirical support (Ladany & Inman,
in press). Moreover, it calls into question a fundamental assumption about the
competency benchmarks, that is, the assumption that there are developmental
changes that should take place in the supervisor’s approach.
As the identified effective categories offer a primer for supervisor compe-
tencies, the identified ineffective categories perhaps offer a primer for supervi-
sor incompetence. Results from our study coincide with findings from
Magnuson, Wilcoxon, and Norem’s (2000) qualitative investigation of “lousy
supervisors.” One of our more striking findings was how supervisors, at times,
depreciated or devalued supervision. As has been found in the literature,
supervision can be a place where supervisees are harmed (Ellis, 2001). As can
be seen in the examples, presumably what is clearly not acceptable in a coun-
seling relationship (e.g., answering phone calls) was deemed acceptable in
supervision. A weakening of the supervisory relationship was understandably
problematic for supervisees. What is unclear, however, is if the weakening
was a rupture in the supervisory alliance that was irreparable (Ladany,
Friedlander, & Nelson, 2005). In supervision, it would be expected that, at
times, supervisors would not perform expertly. The question remains whether
the supervisory alliance was strong enough to withstand potentially problem-
atic supervisor behaviors. In sum, the results related to ineffective supervisor
skills, techniques, and behaviors speak to the challenges that many supervi-
sors seem to possess and have implications for training sites that may offer
little guidance or accountability of supervisor performance (Ladany, 2007).

Supervision Process and Outcome

A second overarching implication from the findings pertains to the impor-
tance relegated to the quantitative variables under study, specifically the
supervisory working alliance, supervisor style, supervisor self-disclosure,
supervisee nondisclosure, and evaluation. Best versus worst supervisors and,
to a large extent by extrapolation, effective and ineffective supervisor behav-
iors were linked with these aforementioned quantitative variables.
Hence, if one were to ask what an effective or highly competent supervisor
were to look like, the findings point to the following. First, the supervisor would
work toward developing a strong supervisory alliance by working toward mutu-
ally agreeing with the supervisee on the goals and tasks of supervision. The

Downloaded from tcp.sagepub.com by guest on July 25, 2016

Ladany et al. 43

supervisor would use basic counseling skills such as listening, reflection of

feelings, and empathy to facilitate the development of an emotional bond. An
effective supervisor would attend to and offer a balance of attractive or col-
legial interactions, interpersonal attentiveness, and task-oriented structure.
Moreover, self-disclosure would be used judiciously and in the service of the
supervisee. It would be anticipated that supervisees would offer less nondis-
closures, thereby facilitating a more meaningful supervision experience.
Finally, supervisors would attend specifically to the evaluation aspect of
supervision by facilitating the setting of supervisory goals and providing both
formative and summative feedback.
The significance found from these variables offers a foundation upon which
to conduct additional investigations. For example, along with and beyond the
quantitative variables found to be salient, future researchers would do well to
study additional variables such as supervisor ethical adherence, supervisor
training, supervisor theoretical approach to supervision, supervisor responsive-
ness, supervisor multicultural competence, and so forth, thereby adding to the
multifaceted skills required for the best supervisory experiences.

Limitations and Conclusions

The findings must be considered in light of the limitations to the experimen-
tal design. First, participant recall likely influenced the content and manner
in which the participants responded. It is possible that the salience or mean-
ingfulness of a supervision experience, be it positive or negative, played a
part in the types of recall that occurred. In a related fashion, the instructions
that asked participants to first consider their best, then their worst supervisors
could have subjected the results to anchoring effects (Kahneman, 2011).
Recommended future investigative work may or may not offer support to the
current findings if the variables examined were in the context of single ongo-
ing supervisory experiences. Second, only the supervisees’ perspectives were
considered. The extent to which supervisors actually engaged in these behav-
iors is unclear. Future work may want to consider assessing both members of
the supervisory dyad, as well as objective raters, to determine the congruence
with which they experience happenings in supervision. Finally, additional
variables not studied may prove useful to determine the adequacy of the
emerging model of variables based on the current investigation. For exam-
ple, it may prove useful to consider effective or ineffective supervisor behav-
iors in relation to variables such as supervisee openness to learning,
supervision setting, or supervisee anxiety. Moreover, particular kinds of
outcomes, such as supervisee multicultural competence or supervisor and

Downloaded from tcp.sagepub.com by guest on July 25, 2016

44 The Counseling Psychologist 41(1)

supervisee metacompetence (Falender & Shafranske, 2007), may be linked

to effective or ineffective behaviors yet to be identified.
In sum, this investigation offers insight into a variety of supervisor skills,
techniques, and behaviors that help, stagnate, or hinder supervisee growth.
Moreover, the results provide linkages to the relevance of the current trend of
defining supervision competency benchmarks (Fouad et al., 2009) both in
relation to supporting some of the proposed competency benchmarks (e.g.,
importance of the supervisory relationship, goal setting) as well as not sup-
porting other competency benchmarks (e.g., salience of developmental level).
Given the number of supervision experiences counseling students receive
and given the number of ineffective supervisory experiences reported, coun-
selor educators would do well to consider supervisor accountability in rela-
tion to an adequate training model. Specifically, supervisor accountability in
terms of demonstrating proficiency has received virtually no recognition in
the literature. At best, some state boards are requiring that supervisors receive
training, but the content and efficacy of this training has been largely unstud-
ied. To that end, we encourage supervisor trainers and researchers to seri-
ously attend to evaluating supervisor behaviors in the same fashion given to
counselor and psychotherapist evaluation to date.

Declaration of Conflicting Interests

The authors declared no potential conflicts of interest with respect to the research,
authorship, and/or publication of this article.

Funding
The authors received no financial support for the research, authorship, and/or
publication of this article.

References
Allen G. J., Szollos, S. J., & Williams, B. E. (1986). Doctoral students’ comparative
evaluations of best and worst psychotherapy supervision. Professional Psychol-
ogy: Research and Practice, 17, 91-99.
Ancis, J. R., & Marshall, D. S. (2010). Using a multicultural framework to assess
supervisees’ perceptions of culturally competent supervision. Journal of Counsel-
ing & Development, 88, 277-284.
Bahrick, A. S. (1989). Role induction for counselor trainees: Effect on the supervisory
working alliance. Unpublished doctoral dissertation, The Ohio State University,
Columbus, Ohio.
Bang, K., & Park, J. (2009). Korean supervisors’ experiences in clinical supervision.
The Counseling Psychologist, 37, 1042-1075.

Downloaded from tcp.sagepub.com by guest on July 25, 2016

Ladany et al. 45

Bordin, E. S. (1983). A working alliance based model of supervision. The Counseling

Psychologist, 11, 35-42.
Ellis, M. V. (2001). Harmful supervision, a cause for alarm: Comment on Gray et al.
(2001) and Nelson and Friedlander (2001). Journal of Counseling Psychology,
48, 401-406.
Ellis, M. V., & Ladany, N. (1997). Inferences concerning supervisees and clients in
clinical supervision: An integrative review. In C. E. Watkins (Ed.), Handbook of
psychotherapy supervision (pp. 447-507). Hoboken, NJ: John Wiley & Sons.
Falender, C. A., & Shafranske, E. P. (2007). Competence in competency-based super-
vision practice: Construct and application. Professional Psychology: Research
and Practice, 38(3), 232-240.
Fouad, N. A., Grus, C. L., Hatcher, R. L., Kaslow, N. J., Hutchings, P. S., Madson, M. B.,
. . . Crossman, R. E. (2009). Competency benchmarks: A model for understand-
ing and measuring competence in professional psychology across training levels.
Training and Education in Professional Psychology, 3(4 Suppl.), S5-S26.
Friedlander, M. L., & Ward, L. G. (1984). Development and validation of the supervi-
sory styles inventory. Journal of Counseling Psychology, 31, 542-558.
Gray, L. A., Ladany, N., Walker, J. A., & Ancis, J. R. (2001). Psychotherapy trainees’
experience of counterproductive events in supervision. Journal of Counseling
Psychology, 48, 371-383.
Horvath, A. O., & Greenberg, L. S. (1989). Development and validation of the work-
ing alliance inventory. Journal of Counseling Psychology, 36, 223-233.
Jacobsen, C. H., & Tanggaard, L. (2009). Beginning therapists experiences of what
constitutes good and bad psychotherapy supervision: With a special focus on indi-
vidual differences. Nordic Psychology, 61, 59-84.
Kahneman, D. (2011). Thinking, fast and slow. New York: Farrar, Straus, and Giroux.
Kaslow, N. J., & Bell, K. D. (2008). A competency-based approach to supervision.
In C. A. Falender & E. P. Sharanske (Eds.), Casebook for clinical supervision: A
competency-based approach (pp. 17-38). Washington, DC: American Psychologi-
cal Association.
Ladany, N. (2007). Does psychotherapy training matter? Maybe not. Psychotherapy:
Theory, Research, Practice, Training, 44, 392-396.
Ladany, N., & Walker, J. A. (2003). Supervisor self-disclosure: Balancing the uncon-
trollable narcissist with the indomitable altruist. In Session: Journal of Clinical
Psychology, 59, 611-621.
Ladany, N., Friedlander, M. L., & Nelson, M. L. (2005). Critical events in psycho-
therapy supervision: An interpersonal approach. Washington, DC: American
Psychological Association.
Ladany, N., Hill, C. E., Corbett, M. M., & Nutt, E. A. (1996). Nature, extent, and
importance of what psychotherapy trainees do not disclose to their supervisors.
Journal of Counseling Psychology, 43, 10-24.

Downloaded from tcp.sagepub.com by guest on July 25, 2016

46 The Counseling Psychologist 41(1)

Ladany, N., & Inman, A. G. (in press). Training and supervision. In E. M. Altmaier &
J. C. Hansen (Eds.), The Oxford handbook of counseling psychology. New York:
Oxford University Press.
Ladany, N., & Lehrman-Waterman, D. E. (1999). The content and frequency of super-
visor self-disclosures and their relationship to supervisor style and the supervisory
working alliance. Counselor Education and Supervision, 38, 143-160.
Ladany, N., Lehrman-Waterman, D. E., Molinaro, M., & Wolgast, B. (1999). Psy-
chotherapy supervisor ethical practices: Adherence to guidelines, the supervisory
working alliance, and supervisee satisfaction. The Counseling Psychologist, 27,
443-475.
Ladany, N., Walker, J. A., Pate-Carolan, L., & Gray Evans, L. (2008). Practicing
counseling and psychotherapy: Insights from trainees, clients, and supervisors.
New York: Routledge.
Lehrman-Waterman, D., & Ladany, N. (2001). Development and validation of the
evaluation process within supervision inventory. Journal of Counseling Psychol-
ogy, 48, 168-177.
Magnuson, S., Wilcoxon, S., & Norem, K. (2000). A profile of lousy supervision:
Experienced counselors’ perspectives. Counselor Education and Supervision, 39,
189-202.
Martin, J. S., Goodyear, R. K., & Newton, F. B. (1987). Clinical supervision: An inten-
sive case study. Professional Psychology: Research and Practice, 18, 225-235.
McWhirter, E. H., & McWhirter, B. T. (2007). Grounding clinical training and super-
vision in an empowerment model. In E. Aldarondo (Ed.), Advancing social justice
through clinical practice (pp. 417-442). Mahwah, NJ: Lawrence Erlbaum Associ-
ates Publishers.
Nelson, M. L., & Friedlander, M. L. (2001). A close look at conflictual supervisory
relationships: The trainee’s perspective. Journal of Counseling Psychology, 48,
384-395.
Rodolfa, E., Bent, R., Eisman, E., Nelson, P., Rehm, L., & Ritchie, P. (2005). A cube
model for competency development: Implications for psychology educators and
regulators. Professional Psychology: Research and Practice, 36, 347-354.
Tracey, T. J., & Kotovic, A. M. (1989). Factor structure of the working alliance inven-
tory. Psychological Assessment, 1, 207-210.
Tracey, T. J., & Sherry, P. (1993). Complementary interaction over time in successful
and less successful supervision. Professional Psychology: Research and Practice,
24, 304-344.
Worthen, V., & McNeill, B. W. (1996). A phenomenological investigation of “good”
supervision events. Journal of Counseling Psychology, 43, 25-34.

Downloaded from tcp.sagepub.com by guest on July 25, 2016

Ladany et al. 47

Bios

Nicholas Ladany, PhD, is Professor and Director of the Counseling Program at

Loyola Marymount University in Los Angeles, California. His scholarship attends to
the efficacy of counselor supervision and education, particularly the working alliance,
multicultural training, and social justice. He is the author of five books including:
Practicing Counseling and Psychotherapy; and Counselor Supervision (4th edition).

Yoko Mori, PhD, is the Interim Assistant Director and the Group Coordinator at the
Counseling Center at the University of Illinois at Chicago. Her area of interest is
supervision and training, international issues in counseling, multicultural counseling,
and group therapy. In addition, she has written articles and book chapters on supervi-
sion and training.

Kristin E. Mehr, PhD, is currently a postdoctoral fellow at the Center for Counseling
& Student Development at the University of Delaware in Newark, Delaware. She has
contributed to various conference presentations in the area of counseling and psycho-
therapy supervision. She is also the first author of an article focused on trainee non-
disclosure in supervision (Mehr, Ladany, & Caskie, 2010).

Downloaded from tcp.sagepub.com by guest on July 25, 2016