Shields 1997
Shields 1997
Shields 1997
To foster the study of emotion regulation beyond infancy and toddlerhood, a new criterion Q-sort
was constructed. In Study 1, Q-scales for emotion regulation and autonomy were developed, and
analyses supported their discriminant validity. Study 2 further explored the construct validity of the
Emotion Regulation Q-Scale within a sample of 143 maltreated and 80 impoverished children, aged
6 to 12 years. A multitrait—multimethod matrix and confirmatory factor analyses indicated impressive
This article is intended solely for the personal use of the individual user and is not to be disseminated broadly.
convergence among the Emotion Regulation Q-Scale and established measures of affect regulation.
This document is copyrighted by the American Psychological Association or one of its allied publishers.
This new scale also was discriminable from measures of related constructs, including Q-sort assess-
ments of ego resiliency. The use of this new measure was further supported by its ability to distinguish
between maltreated and comparison children and between groups of well-regulated versus dysregu-
lated children.
Early research and theory on emotional development have for measurement construction is that assessments must capture
highlighted the first years of life as fundamental to affect regula- both pathological and nonpathological processes. Furthermore,
tion (Cicchetti & Schneider-Rosen, 1986; Eisenberg & Fabes, measures ideally would be sensitive to heterotypic continuity
1992; Gianino & Tronick, 1988; Kopp, 1989). Recent research (Caspi & Moffitt, 1995; Kagan, 1971) and appropriate across
has expanded on this work by demonstrating that emotion regu- ages and research contexts. The Q-sort methodology effectively
lation continues to play a key role in adaptation during middle addresses these concerns by virtue of its capacity to span a wide
childhood. During these years, regulatory capacities become age range and to target a variety of adaptive and maladaptive
increasingly sophisticated and predict children's competence in characteristics within an individual's personality organization
a number of important domains, including behavioral regulation (Block, 1961; Block & Block, 1980a).
and social competence (Lewis & Haviland, 1993). To foster
the continued study of emotion regulation beyond infancy and Emotion Regulation in Middle Childhood
toddlerhood, a new criterion Q-scale for older children was Consistent with an organizational perspective, which concep-
constructed. tualizes development as the increasing differentiation and hierar-
This process was undertaken from a developmental psychopa- chical integration of biological and psychological systems (Cic-
thology perspective, which asserts that an understanding of de- chetti, 1993; Cicchetti & Sroufe, 1978), recent research has
velopment is best informed by studying pathways leading to shown that children's capacities for emotion regulation become
both adaptation and maladaptation (Cicchetti, 1990, 1993; more complex and integrated during the preschool and grade-
Sroufe, 1990). One important implication of this perspective school years (Fox, 1994; Saarni, 1990). These gains are fostered
by increased sophistication in representational thought and in-
formation processing abilities (Harris, 1989; Saarni & Harris,
1989), which promote emotional understanding and adaptive
Ann Shields and Dante Cicchetti, Mt. Hope Family Center and the
coping (Harter & Buddin, 1987; McCoy & Masters, 1985; Win-
Department of Clinical and Social Sciences in Psychology, University
tre & Vallence, 1994). Specifically, school-age children are more
of Rochester. Ann Shields is now at the Department of Psychiatry and
Human Behavior, Brown University School of Medicine. cognizant of display rules and of potential discrepancies be-
This work was supported by grants from the William T. Grant Founda- tween emotional expressions and internal experiences (Gnepp &
tion, the National Center on Child Abuse and Neglect, the Spunk Fund, Hess, 1986; Saarni, 1979). School-age children also are more
and a National Research Service Award granted to the Human Motivation adept at integrating complex cues regarding others' emotions
Program at the University of Rochester by the National Institute of and at managing their own affective expressions in accord with
Mental Health (MH18922). We are especially grateful to the criterion situational demands (Gnepp, 1989; Saarni, 1989; Underwood,
raters whose generous donations of time and expertise contributed so Coie, & Herbsman, 1992). Nevertheless, the study of children's
much to this project. We also wish to thank Michael Lynch, Fred Ro- emotional development still is in its formative stages, such
gosch, and Sheree Toth for their helpful comments on a draft of this that measurement development is an important step in fur-
article.
thering understanding of this complex developmental process
Correspondence concerning this article should be addressed to Ann
(Shields & Cicchetti, 1995).
Shields, Bradley Hospital, 1011 Veterans Memorial Parkway, East Provi-
dence, Rhode Island 02915, or Dante Cicchetti, Mt. Hope Family Center,
Q-Sort Methodology
University of Rochester, 187 Edinburgh Street, Rochester, New 'fork
14608. Electronic mail may be sent via Internet to AnnShields® A number of protocols for assessing emotion regulation have
Brown.edu. been developed, drawing upon physiological, observational, in-
906
CRITERION Q-SCALES 907
terview, and questionnaire methodologies. The Q-sort is distin- consistent with the method used by Block and Block (1980a, 1980b),
guished from these protocols in that it allows researchers to raters were asked to portray the personality characteristics of a hypotheti-
evaluate the role of a particular developmental process (such cal emotionally well-regulated or optimally autonomous child, without
as emotion regulation) within an individual's comprehensive regard for a specific target age. Raters were instructed to sort the 100
CCQ items into nine categories, ranging from extremely characteristic
personality profile (Block & Block, 1980a). Other advantages
to extremely uncharacteristic. These nine categories approximated a
include the Q-sort's ability to keep observers naive of target
normal distribution in that fewer items were placed in the extreme piles
constructs and to reduce response biases by requiring raters to than in the middle categories.
sort items into a fixed distribution (Waters, Noyes, Vaughn, & Composite Q-sort definitions then were created by averaging individ-
Ricks, 1985). Because the Q-sort is applicable across a wide ual raters' placements of all 100 descriptors (in Piles 1 through 9 ) .
age range, it is an effective tool for longitudinal research. Finally, Emotion Regulation and Autonomy Q-scales then were constructed from
a criterion Q-sort also would allow investigators to explore the 10 items (i.e., the 5 highest and 5 lowest items) rated as most salient
questions regarding children's emotional development in extant for each construct. As a first step toward examining discriminant validity,
data sets that include the California Child Q-Set (CCQ). Wilcoxon pairwise comparisons then were conducted across the auton-
omy and emotion scales. Significant differences between the mean place-
Q-sort scales also may be computed from criterion sorts. This
This article is intended solely for the personal use of the individual user and is not to be disseminated broadly.
distinct constructs.
of only those items identified as either most or least descriptive
of a particular construct (Eisenberg et al., 1996; John, Caspi,
Robins, Moffit, & Stouthamer-Loeber, 1994). This highly salient Results
subset, or Q-scale, may include those items falling in the ex-
Emotion Regulation Q-Sort
tremely characteristic and extremely uncharacteristic piles. By
disregarding less salient items lacking in discriminant validity, For this Q-sort, correlations between individual ratings and
the specificity and homogeneity of Q-sort assessments are en- the composite Q-sort ranged from .76 to .93. Internal consis-
hanced. Nevertheless, raters should continue to use the full 100- tency, as assessed through Cronbach's alpha, was .98. The Emo-
item CCQ deck when evaluating children to preserve the unique tion Regulation Q-Scale then was created with the five items
benefits of the Q-sort methodology. with the highest mean rating and the five items with the lowest
mean score across criterion raters. The Cronbach's alpha for
this Q-scale was .85.
Study 1
The items in this Q-scale are presented in Table 1, along with
Method pairwise Wilcoxon comparisons examining differences between
their mean placements (in Piles 1 through 9) in the emotion
Participants regulation versus autonomy sorts. Wilcoxon tests showed that
the ratings for all but 1 of the items comprising the emotion Q-
To compile the Emotion Regulation Q-Sort, from which the Emotion
scale were significantly different from those in the autonomy
Regulation Q-Scale was derived, 17 doctoral-level psychologists with
scale. However, 2 items reflecting a lack of resiliency in the
expertise in emotional development generated individual criterion Q-
sorts on a prototypically well-regulated child.1 For the purposes of this face of stress were placed in both scales: CCQ Item 46, '"fends
study, emotion regulation was denned in terms of lability, flexibility, to go to pieces under stress," and CCQ Item 39, "Tends to be
and situational responsivity and conceptualized as the capacity to mod- rigidly repetitive in stressful situations."
ulate one's emotional arousal such that an optimal level of engagement
with one's environment is fostered (Cicchetti, Ganiban, & Barnett,
Autonomy Q-Sort
1991; Thompson, 1994).
A criterion Autonomy Q-Sort also was compiled, then used in analyses For the Autonomy Q-Sort, correlations between individual
evaluating the discriminant validity of the new emotion Q-scale. Auton- criterion ratings and the composite sort ranged from .82 to .91.
omy was explored in conjunction with emotion regulation, as both are The Cronbach's alpha for the criterion sort was .98. An Auton-
early stage-salient issues that have been associated with adaptation in omy Q-Scale then was created, and the internal consistency of
middle childhood and adolescence (Ryan, 1993; Ryan, Deci. & Grolnick,
this scale, assessed through Cronbach's alpha, was .79. This Q-
1995). If the new emotion regulation sort were discriminable from the
Q-sort assessment of autonomy, then this would suggest that the emotion
scale assessed a specific construct rather than general competence associ- 1
We are indebted to the following raters for their generous contribu-
ated with adaptive early development. To develop this autonomy sort,
tions of time and expertise toward the construction of the Emotion
18 doctoral-level psychologists specializing in motivation and self-devel-
Regulation Q-Sort: Karen C. Barrett, Jude Cassidy, Jeffrey Cohn, Pamela
opment conducted ratings of an optimally autonomous child, defined
Cole, Patricia Crittenden, Kenneth A. Dodge, Byron Egeland, Nancy
along Deci and Ryan's (1985) criteria as one who displayed initiative,
Eisenberg, Judy Garber, John Gottman, Carroll Izard, Carol Malatesta-
agency, and self-determination.2
Magai, John Richters, Carolyn Saarni, Judith Solomon, Ross A. Thomp-
son, and Carolyn Zahn-Waxler,
2
Procedure We are grateful to the following Autonomy Q-Sort criterion raters,
whose generous contributions of time and expertise made this work
All 35 raters used the 100-item CCQ deck, following Block and possible: Jay Belsky, Edward L. Deci, Ann Easterbrooks, Wendy S.
Block's (1980a) methodology. Because the CCQ's emotion items cap- Grolnick, Daniel Hart, R. Rogers Kobak, Grazyna Kochanska, Richard
tured affective homeostasis, flexibility, situational responsivity, and em- Koestner, Michael Lewis, Alicia Lieberman, Sandra Pipp-Siegel, Robert
pathic attunement to others, these descriptors were judged to be broad Plant, Johnmarshall Reeve, Richard M. Ryan, Ellen Skinner, Susan
enough to conceptualize regulation across a wide age range. Accordingly, Spieker, Jaine Strauss, and Joan Vondra.
908 SHIELDS AND CICCHETTI
Table 1
Emotion Regulation Q-Scale: Mean-Item Comparisons With the Autonomy Q-Scale
Positively weighted
43 Can recover from stress 8.9 7.6*
81 Can admit negative feelings 8.4 7.3*
3 Is warm/responsive 8.1 7.3*
31 Is empathic 8.1 6.9*
9 Has genuine/close relationships 8.1 6.7*
Negatively weighted
91 Emotional reactions are inappropriate 1.2 3.2*
39 Is rigid/repetitive when stressed 1.4 2.3*
This article is intended solely for the personal use of the individual user and is not to be disseminated broadly.
Note. Reproduced by special permission of the distributor, MIND GARDEN, Inc., P.O. Box 60669, Palo
Alto, CA 94306 from the California Child Q-Sort items by Jack Block. Copyright 1980 by Consulting
Psychologists Press. All rights reserved. Further reproduction is prohibited without the distributor's written
consent. CCQ = California Child Q-Set.
*p< .05.
scale was correlated significantly with the emotion regulation issue across development. First, substantial interrater reliability
scale (/• = .56, p < .001). was found among criterion raters, suggesting consistency and
Table 2 lists the items that comprise this scale, along with clarity in the experts' use of Q-sort descriptors to define af-
pairwise Wilcoxon tests comparing these items' mean place- fective regulation. This high concordance is especially compel-
ments in the autonomy and emotion regulation sorts. The place- ling, given the raters' level of expertise.
ment of 8 of the 10 autonomy scale items differed significantly Item-by-item comparisons of scale items also suggested
from their placement in the emotion regulation sort. The 2 items substantial discriminant validity between the Emotion Regula-
on which no differences were found were CCQ Item 46, "Tends tion and Autonomy Q-scales. The rankings of 9 of the 10 items
to go to pieces under stress," and CCQ Item 23, "Is fearful in the Emotion Regulation Q-Scale were significantly different
and anxious." from their placement in the autonomy sort. Similarly, the rat-
ings for 8 of the 10 autonomy scale items were significantly
Discussion different from their placement in the emotion regulation sort.
Initial analyses support the promise of the new Emotion Regu- This discriminability is consistent with the suggestion that the
lation Q-Scale in furthering the study of this early stage-salient emotion Q-scale assesses a specific affective construct rather
Table 2
Autonomy Q-Scale: Mean-Item Comparisons With the Emotion Regulation Q-Scale
Positively weighted
88 Is self-reliant and confident 8.6 7.4*
40 Is curious and exploring 8.6 7.1*
36 Is resourceful/takes initiative 8.6 7.0*
83 Seeks to be autonomous 8.5 6.4*
82 Is self-assertive 8.2 6.4*
Negatively weighted
77 Feels unworthy/bad 1.8 2.8*
46 Goes to pieces under stress 1.9 1.5
53 Is indecisive/vacillating 2.2 3.9*
39 Is rigid/repetitive when stressed 2.3 1.4*
23 Is fearful and anxious 2.3 2.6
Note. Reproduced by special permission of the distributor, MIND GARDEN, Inc., P.O. Box 60669, Palo
Alto, CA 94306 from the California Child Q-Sort items by Jack Block. Copyright 1980 by Consulting
Psychologists Press. All rights reserved. Further reproduction is prohibited without the distributor's written
consent. CCQ = California Child Q-Set.
* p < .05.
CRITERION Q-SCALES 909
than general competence associated with adaptive early criminant validity is demonstrated whenever lower correlations
development. between the target measure and assessments of divergent con-
Furthermore, the descriptors comprising the Emotion Regula- structs are obtained.
tion Q-Scale reflected fundamental aspects of the construct Discriminant validity also would be supported if the emotion
definition presented to criterion raters. For example, 2 items Q-scale were a stronger predictor of criterion emotion measures
reflected emotional homeostasis versus lability: CCQ Item 54, than were related Q-sorts. One plausible hypothesis would be
"Has rapid shifts in mood, is emotionally labile," and CCQ that the emotion sort merely reflected a higher order construct,
Item 95, "Overreacts to minor frustrations, is easily irritated." such as ego resiliency, already captured by the Q-sort methodol-
Flexibility versus rigidity also was evident in 2 items—CCQ ogy. Indeed, ego resiliency has been suggested as an index of
Item 39, "Tends to become rigidly repetitive or immobilized optimal regulation (Eisenberg et al., 1996), as it is defined in
under stress," and CCQ Item 43, "Can recoup or recover after terms of key regulatory processes, including flexibility, adapt-
stressful experiences." Situational responsivity was captured by ability, and an ability to maintain homeostasis when faced with
Item 91 ("Emotional reactions are inappropriate."). changing situational demands (Block & Block, 1980b). This
Affective engagement with the social surround also was sug- overlap is further suggested by the fact that 3 of the 10 Emotion
This article is intended solely for the personal use of the individual user and is not to be disseminated broadly.
gested by CCQ Item 3, "Is warm and responsive," CCQ Item Regulation Q-Scale items also were rated as most salient in the
This document is copyrighted by the American Psychological Association or one of its allied publishers.
31, "Shows a recognition of others' feelings, is empathic," ego resiliency criterion sort: CCQ Item 39, "Tends to become
and CCQ Item 9, "Develops genuine and close relationships." rigidly repetitive or immobilized under stress," CCQ Item 46,
Although not specified in the criterion definition, emotional self- "Tends to go to pieces under stress," and CCQ Item 91, "Emo-
awareness was represented by CCQ Item 81, "Can acknowl- tional reactions are inappropriate."
edge unpleasant experiences and admit negative feelings." Items A confirmatory factor analysis (CFA) of the MTMM data
held in common between the emotion regulation and autonomy also was conducted (Marsh, 1989; Schmitt & Stults, 1986).
scales suggest that the criterion raters regarded both poorly This procedure is recommended because the MTMM design
regulated and non-autonomous children to be vulnerable to dis- lacks clear, quantifiable criteria for determining appropriate lev-
organization when stressed. els of convergence and divergence among variables (Jackson,
1969) and because it relies on observed rather than latent vari-
ables (Greenbaum, Dedrick, Prange, & Friedman, 1994). Al-
Study 2
though the complete CFA model, which contains separate latent
To evaluate the construct validity of the Emotion Regulation variables for trait and method effects, is the most widely used, it
Q-Scale, both convergent and discriminant validities were exam- tends to generate ill-defined solutions and unreliable estimations
ined in Study 2. First, it was necessary to demonstrate that (Greenbaum et al., 1994; Kenny & Kashy, 1992). Therefore, the
this Q-scale was related to established measures of emotion correlated uniqueness model, which does not include separate
regulation. Next, its discriminability from other assessments of method factors, was used in this study (Kenny & Kashy, 1992;
early developmental competence (e.g., autonomy) was ex- Marsh & Bailey, 1991).
plored. Finally, the extent of overlap between the Q-scale and Finally, the capacity of the Emotion Regulation Q-Scale to
other Q-sort assessments of putatively related constructs also discriminate between well-regulated versus dysregulated chil-
was evaluated. dren was examined. An ability to differentiate between mal-
Three statistical procedures were used: First, a multitrait- treated and nonmaltreated children also would support construct
multimethod matrix (MTMM) was constructed to examine rela- validity, for abused children evidence chronic patterns of af-
tionships among assessments of emotion regulation, autonomy, fective negativity and lability from toddlerhood through middle
and ego resiliency; second, confirmatory factor analyses were childhood (Alessandri, 1991; Erickson, Egeland, & Pianta,
conducted to further evaluate this matrix; and, finally, t tests 1989; Shields & Cicchetti, 1995). Specifically, maltreated chil-
were conducted to determine whether the Q-scale could distin- dren evidence heightened sensitivity to anger (Cummings, Hen-
guish between well-regulated and dysregulated groups. To en- nessy, Rabideau, & Cicchetti, 1994; Hennessy, Rabideau, Cic-
sure the broadest applicability of this new measure, populations chetti, & Cummings, 1994; Pollak, Cicchetti, Klorman, & Bru-
likely to evidence a wide range of functioning were targeted. maghim, in press), rigid and situationally inappropriate
Participants were maltreated children, who are at enhanced risk affective displays (Rogosch, Cicchetti, & Aber, 1995; Shields
for deficits in self-regulation (Cicchetti et al., 1991), and non- et al,, 1994), attenuated empathy (Howes & Eldredge, 1985;
maltreated comparison children. Because comparison partici- Main & George, 1985), and a vulnerability to dysphoria and
pants were matched on a number of important demographic internalizing symptoms (Hart, Gunnar, & Cicchetti, 1996; Kauf-
indexes, including low socioeconomic status, the entire sample man, 1991; Kazdin, Moser, Colbus, & Bell, 1985; Toth,
experienced variable degrees of risk for negative developmental Manly, & Cicchetti, 1992).
outcomes.
MTMMs allow researchers to evaluate construct validity by Method
examining a measure's convergent and discriminant validity
among related and divergent methods and constructs (Camp- Participants
bell & Fiske, 1959). This matrix is comprised of two or more The participants in the study were 143 maltreated children (52 girls
traits assessed by two or more methods. Convergent validity is and 91 boys) and 80 nonmaltreated comparison children (29 girls and
supported when strong correlations are found between a new 51 boys), aged 6 to 12 years (Af = 9 years 11 months). Groups were
measure and established assessments of the same construct. Dis- equivalent on age, sex, and minority status: Sixty-nine percent of the
910 SHIELDS AND CICCHETTI
maltreated children and 75% of the nonmaltreated children were His- Measures, Multitrait—Multimethod Analysis
panic, African American, or biracial; the remainder of each group was
Caucasian. The two groups also were matched on socioeconomic status California Child Q-Set. The CCQ (Block & Block, 1980a) method-
in that both maltreating and nonmaltreating families were impoverished: ology requires raters to sort 100 descriptors of children's common be-
Total yearly income, which included public assistance payments, aver- haviors and personality characteristics into ipsative personality profiles
aged $18,200.00 for maltreating families, whereas nonmaltreating fami- (Block & Block, 1980a). At least two counselors completed these sorts
lies' incomes averaged $ 15,700.00. The per capita annual incomes for the on each child after camp sessions, and means were computed across
maltreating and nonmaltreating families were $3,866.00 and $3,956.00, reliable raters. From these Q-sorts, individual scores on the Emotion
respectively (see Table 3). Regulation and Autonomy Q-scales were generated. In addition, an Ego
Although efforts were taken to ensure that the groups were compara- Resiliency Q-Scale was computed for inclusion in the MTMM analysis,
ble, some differences were evident in family size and configuration. along the lines of the Q-scale methodology reported by John et al.
More adults tended to be present in maltreating families (t = —2.75, (1994) and Eisenberg et al. (1996). As few as 7 and as many as 24 items
p < .001), and maltreating families tended to include more children have been used to compile these Q-scales, all of which are constructed to
(M = 3.4) than nonmaltreating families (M = 2.6), ?(220) = -4.05, produce purer assessments of target constructs. For this study, the Ego
p < .001. However, no associations were found between these factors Resiliency Q-Scale was comprised of the 10 most extreme items to be
This article is intended solely for the personal use of the individual user and is not to be disseminated broadly.
and the outcome variables in Study 2. in keeping with the broader literature on Q-sort assessments (specifying
This document is copyrighted by the American Psychological Association or one of its allied publishers.
The maltreating families were identified by the Monroe County De- the two most extreme piles as most salient in defining a target construct)
partment of Social Services (DSS) as having documented reports of and to be consistent with the construction of the Emotion Regulation
child abuse or maltreatment in accordance with New %rk state laws. and Autonomy Q-scales.
After parental consent to do so was obtained, these families' Child Emotion Regulation Checklist. The Emotion Regulation Checklist
Protective Services records were examined and coded according to the (ERC; Shields & Cicchetti, 1995), a 24-item other-report measure of
Barnett, Manly, and Cicchetti (1993) maltreatment classification system. children's self-regulation, also was completed by counselors.3 This
The presence or absence of each subtype of maltreatment was ascer- checklist was selected as a criterion emotion measure, as its conceptual-
tained for the families, and the children were classified in the following ization of regulation is consistent with the construct definition presented
ways: sexually abused (13.3%, n = 19); physically abused (41.3%, n = to expert raters in Study 1. That is, this measure targets such processes
59); physically neglected (37.8%, n = 54), and emotionally maltreated as affective lability, intensity, valence, flexibility, and situational appro-
(7.6%, n — 11), It should be noted that, consistent with the literature, priateness. The checklist includes both positively and negatively
the majority of the children (i.e., 53.8%) experienced multiple subtypes weighted items to be rated on a 4-point Likert scale.
of maltreatment (Cicchetti & Rizley, 1981). Because the focus of this To evaluate the factor structure of the ERC, a principal-components
article is on the development and validation of a new criterion Q-sort factor analysis with a varimax rotation was conducted. This analysis
scale, research questions and statistical analyses center on maltreated yielded two factors representing Lability/Negativity and Emotion Regu-
versus nonmaltreated comparisons, and analyses by subtype were not lation. The Lability/Negativity subscale is comprised of items represent-
conducted. ing a lack of iexibility, mood lability, and dysregulated negative affect;
The nonmaltreated comparison families were recruited through post- sample items include "Exhibits wide mood swings," and "Is prone
ers in welfare offices, housing projects, and neighborhoods similar to to angry outbursts." The Emotion Regulation subscale includes items
those in which the maltreating families resided. An initial telephone describing situationally appropriate affective displays, empathy, and
screening ensured that the families met the criteria for the study. Nonmal- emotional self-awareness; sample items include "Is empathic toward
treated status in the comparison group was determined by interviews others," and "Can say when s/he is feeling sad, angry or mad, fearful
conducted with the parents by trained research assistants and, on receipt or afraid." Internal consistencies, assessed through Cronbach's alpha,
of parental consent to examine DSS records, by a careful check of CPS were .96 for Lability/Negativity and .83 for Emotion Regulation. In
records to verify the absence of maltreatment reports. addition, the two subscales were significantly correlated (r = -.50,
p < .001). A composite ERC score also was generated, as a single
emotion regulation criterion measure was needed for confirmatory factor
Procedure analyses, and it was important that this variable capture processes related
to both regulation and dysregulation. The internal consistency of this
Data were collected during the eighth annual summer day camp for composite ERC score, which was computed by reverse scoring all nega-
at-risk inner-city children sponsored by Mt. Hope Family Center at the tively weighted items and then averaging across all 24 items, was .89.
University of Rochester. Maltreated and comparison children attended Emotion regulation observations. Emotion regulation observations
camp for week-long sessions, participating in both recreational activities (Shields et al., 1994) took place on 2 days during each camp session,
and research (see Cicchetti & Manly, 1990, for detailed information when independent observers, not camp counselors, coded children's
about the camp setting). Before camp, parents gave informed consent emotion expressions during semistructured play periods. These periods
for their children's participation in research activities during the week. were chosen because they provided an opportunity to observe partici-
Children then were given a choice about participating in research inter- pants' naturalistic interactions in a social context that was both familiar
views, for which they earned points to be exchanged for small prizes. and important to grade-school children. During these play periods, chil-
Six 1-week sessions were held during the summer. Bach week, chil- dren participated in recreational activities with both peers and authority
dren were assigned to groups of 8 to 10 children of the same sex figures; often, counselors gave children a choice of several group activi-
and age; groups also included a balance of maltreated and comparison ties or games. No effort was made by experimenters to direct these
children. Each group was led by three counselors, who were unaware interactions or to present specific social and emotional challenges to
of the experimental hypotheses and of the children's maltreatment status.
Counselors' intensive contact with the children (approximately 35 hours
across the week) enabled them to assess children's functioning effec-
3
tively across a number of domains. Independent observers, similarly Copies of the Emotion Regulation Checklist, along with instructions
unaware of hypotheses and participant status, also conducted observa- for administration and scoring, may be obtained from Dante Cicchetti,
tions and rated children's functioning on 2 days during each camp Mt. Hope Family Center, University of Rochester, 187 Edinburgh Street,
Rochester, New York 14608.
CRITERION Q-SCALES 911
Table 3
Contrasts of Maltreated and Nonmaltreated Groups on Demographic Variables
Maltreated Nonmaltreated
Demographic variable Af SD M SD t
children, and the observers made every effort to be as unobtrusive as included as an approximation of low levels of agency and autonomy,
possible. for higher scores reflect an extreme reliance on others, whereas lower
Three 10-min observations of each child's affective expressions were scores suggest that the child demonstrates a high degree of self-direction.
collected, during which emotions were identified by facial expressions Correlations and standard deviations among counselors' ratings for
(Ekman & Friesen, 1975), voice tone, and body posture (see Izard, individual children were computed. If these analyses indicated that a
1982; Scherer, 1979) and evaluated for valence, intensity, flexibility, and rater was unreliable across measures, then the data provided by that
situational responsivity. Emotion expressions were judged to be adaptive counselor were dropped from analyses. Means were computed across
or appropriate if they were situationally responsive to the context in reliable raters to yield one score each for the Agency, Dependency,
which the child was interacting. For example, expressions of anger to- Positive Mood, and Negative Mood scales.
ward an intrusive or aggressive peer were coded as appropriate, for Student-Teacher Relationship Scale. The Student-Teacher Rela-
these served important self-protective goals. However, if this anger were tionship Scale (STRS; Pianta, 1992; Pianta & Steinberg, 1992) is a 5-
directed toward an inappropriate target (e.g., a nonaggressive child), point Likert scale, completed by adults familiar with a child, that targets
then this expression would be coded as maladaptive/inappropriate. Sim- the quality of children's relationships with important adults. Five rela-
ilarly, if this anger were so intense as to undermine an adaptive response, tional characteristics are assessed by the STRS: dependent, conflicted,
or so persistent as to outlive its usefulness, then it would be coded as warm, open, and troubled. The Dependency subscale again was selected
maladaptive/inappropriate as well. as another approximation of a child's lack of relative agency and auton-
Coders noted the presence versus the absence of seven emotion catego- omy. The internal consistency of this subscale, assessed through Cron-
ries during each 10-min observation period: appropriate positive with bach's alpha, was .81.
peers or adults, inappropriate positive with peers or adults, appropriate
negative with peers, and inappropriate negative with peers or adults.
Measures, Confirmatory Factor Analysis
(Appropriate negative with adults was not coded because of a failure
to achieve reliability on this variable, perhaps because of its low base Big Five scales of the California Child Q-Set. The Big Five scales
rate.) Interobserver agreement on the emotion scales ranged from a of the California Child Q-Set (John et al., 1994) reflect dimensions of the
kappa of .54 for inappropriate positive with peers to a kappa of .74 for Five-Factor Model of personality (McCrae & John, 1992): extroversion,
inappropriate negative with peers. conscientiousness, openness/intellect, agreeableness, and neuroticism.
A composite emotion regulation observation score then was computed Two of these scales, Neuroticism and Agreeableness, were included in
as follows: First, mean scores for each of the seven emotion expression confirmatory factor analyses, as overlaps between these two constructs
categories were computed across the three observation periods; then, and emotion regulation were thought to be likely. The Big Five scales
mean scores of all inappropriate affective displays were summed, as were derived by expert raters who selected Q-sort items most salient
were the scores of all emotion expressions judged to be appropriate. The for each target construct. Items lacking in discriminant validity then
summary maladaptive/inappropriate score then was subtracted from the were dropped, resulting in a 10-item scale for Neuroticism and a 13-
summary adaptive/appropriate score to create the emotion regulation item scale for Agreeableness.
observation composite. Higher scores reflected more adaptive self- That neuroticism has a significant affective component is suggested
regulation. by the fact that individuals high on this personality characteristic are
Minnesota Behavior Ratings. The Minnesota Behavior Ratings seen as anxious, tense, and emotionally labile. Similarly, agreeable indi-
(Sroufe, 1983), an other-report measure, asks adults familiar with indi- viduals are described as warm and empathic. Furthermore, these two
vidual children to generate molar ratings of adaptive functioning in a scales were the only ones of the Big Five to hold items in common with
number of key domains. Subscales assessing dispositional tendencies the Emotion Regulation Q-Scale. For neuroticism, three shared items
toward Positive Mood and Negative Mood were included as approxima- were found: CCQ Item 39, "Tends to become rigidly repetitive or immo-
tions of children's affective modulation. Higher scores on Positive Mood bilized under stress," CCQ Item 43, "Can recoup after stressful experi-
reflect a tendency toward warmth and equanimity, whereas lower scores ences," and CCQ Item 46, "Tends to go to pieces under stress." For
reflect characteristically flat or negative expressions. Similarly, higher Agreeableness, two items were held in common: CCQ Item 3, "Is
scores on Negative Mood reflect chronic anger and hostility, whereas warm and responsive," and CCQ Item 9, "Develops genuine and close
lower scores reflect a tendency toward positive emotion expressions. relationships." Therefore, testing whether the new Emotion Regulation
Also included as a criterion autonomy measure was the Agency sub- scale could be distinguished statistically from the Neuroticism and
scale, on which higher scores reflect a tendency toward self-assertion, Agreeableness scales was seen as a rigorous test of discriminant validity.
mastery, and intrinsic motivation, whereas lower scores suggest a relative Child Behavior Checklist—Teacher's Report Form. The Child Be-
passivity and a lack of initiative. The Dependency subscale also was havior Checklist—Teacher's Report Form (CBCL-TRF; Achenbach,
912 SHIELDS AND CICCHETTI
1991), one of the most widely used and well-validated other-report correlations were found between this Q-scale and the criterion
measures of children's adjustment, is composed of 118 items designed measures of emotion regulation. These correlation coefficients
to assess children's emotional and behavioral difficulties. This measure ranged from r = .44, p < .001, for the emotion regulation
yields two broad-band factors representing internalizing and externaliz-
observations to r = - . 7 9 , p < .001, for Lability/Negativity.
ing difficulties as well as nine subscales assessing more specific symp-
tom patterns (Achenbach, 1991). Of these nine scales, the Anxiety/ In general, these correlations were of greater magnitude than
Depression subscale was paired with Neuroticism in confirmatory factor were those between the Emotion Regulation Q-Scale and the
analyses, for both target symptoms of anxiety and dysphoria. criterion measures of autonomy. That is, correlations between
Counselors completed the CBCL-TRF on all children in their groups, the emotion scale and the criterion autonomy measures ranged
and correlations and standard deviations among ratings then were com- from r = - . 1 3 , p < .05, for the STRS rating of dependency to
puted. If these analyses indicated that a rater was unreliable across r= -.24, p< .001, for the Minnesota dependency rating. These
measures for a child, then these data were dropped from analyses. Means findings suggest that the Emotion Regulation Q-Sort captures
were computed across reliable raters to yield one score for Anxiety/ regulation, in particular, rather than developmental competence
Depression.
associated with adaptive early development, which also presum-
Pupil Evaluation Inventory. The Pupil Evaluation Inventory (PEI;
Pekarik, Prinz, Liebert, Weintraub, & Neale, 1976) is an assessment
ably would be captured by autonomy measures.
This article is intended solely for the personal use of the individual user and is not to be disseminated broadly.
Table 4
Multitrait-Multimethod Matrix: Convergent and Discriminant Validity of the Emotion Regulation Q-Scale
Subscale 1 2 3 4 5 6 7 8 9 10
1. E Reg Q-Scale
2. Lability/Negativity -.79*** —
3. Emotion Regulation .68*** -.50*** —
4. Emotion Observations .44*** _49*** .23***
5. Positive Mood 72*** -.58*** .77*** .32***
6. Negative Mood _.74*** .74*** -.55*** _ 40*** -.74*** —
7. Autonomy Q-Scale .57*** -.30*** .55*** .13* .50*** -.38*** —
8. Agency .20** .07 .43*** .01 .35*** -.04 .69*** —
9. Dependency 1 -.24*** .23*** -.08 -.18** -.17* .26*** -.45*** -.22**
10. Dependency 2 -.13* .13 .02 .06 -.03 .07 -.35*** -.16** .63*** —
11. Ego Resiliency .69*** -.37*** .67*** .18** .64*** -.49*** .90*** .62*** -.37*** -.26***
seen, the ego resiliency scale was as highly correlated with Factor 3, Agreeableness, is comprised of Agreeableness and Likability;
This document is copyrighted by the American Psychological Association or one of its allied publishers.
Group Differences: Maltreatment Status Scale was much less sensitive to indexes of negative mood,
emotional lability, and inflexible and maladaptive affective dis-
In addition, t tests evidenced significant differences between
plays. This finding supports the specificity of the Emotion Regu-
maltreated and nonmaltreated children. As predicted, maltreated
lation Q-Scale and suggests that it is more sensitive to a broader
children (M = 48.77) were more likely than were their nonmal-
range of regulatory processes, including indicators of maladap-
treated peers (M = 52.44) to evidence emotion dysregulation
tation and dysregulation. Therefore, the Emotion Regulation Q-
(t = 2.51, p < .05).
Scale makes a substantial contribution to existing Q-sort assess-
ments by providing a means by which to assess processes lead-
Developmental Differences in Emotion Regulation ing to both regulation and dysregulation in children.
The next set of analyses evaluated whether the new emotion Confirmatory factor analyses also suggested that this new
Q-scale would tap developmental differences in children's emo- measure captured a distinct construct: Regulation as assessed by
tion regulation. First, the correlation between children's age and the Emotion Regulation Q-Scale was statistically discriminable
their Emotion Regulation Q-Scale scores was examined and from other key variables assessed through the Q-sort methodol-
found to be not significant (r = .01, ns). This finding was ogy. That is, emotion regulation emerged as separable from the
corroborated by median-split analyses in that no differences in potentially related (and confounded) constructs of autonomy,
children's Q-scale scores were found for older versus younger neuroticism, and agreeableness. It was not possible to further
children. In addition, no age differences in children's ego resil- evaluate ego resiliency in this analysis, as no secondary assess-
iency scores were found. ment of resiliency was available for pairing in the confirmatory
factor analysis. However, because autonomy overlapped sub-
stantially with ego resiliency, this finding supports by proxy
Discussion the assertion that this new assessment of emotion regulation is
In Study 2, three sets of analyses were conducted to evaluate discriminable from similar measures of ego resiliency.
the Emotion Regulation Q-Scale, and the pattern of results As predicted, the Emotion Regulation Q-Scale also differenti-
strongly supported the construct validity of this new measure. ated between well-regulated and dysregulated groups as well as
First, convergent validity was demonstrated in MTMM analyses, between maltreated and nonmaltreated children. In addition, the
evidencing substantial correlations between the new scale and Q-scale appears to be equally informative with younger and
established measures of emotion regulation. That is, the Q-scale older children in the 6- to 12-year age range. Although these
was a strong predictor of emotional lability and negativity, sad findings suggest that the Q-scale may be an effective research
and negative mood states, and a tendency toward positive mood tool, analyses exploring its validity were limited to school-age
and adaptive regulation, as assessed by independent observers' children who were at enhanced risk for negative developmental
and counselors' reports. On the whole, lesser correlations were outcomes. Therefore, exploring the validity and utility of this
obtained between the emotion Q-scale and assessments of auton- measure among more normative school-age populations, as well
omy, suggesting that the Emotion Regulation Q-Scale captures as among preschoolers, would be an important step. In addition,
affective regulation and dysregulation rather than general adap- continuing to evaluate the use of this measure among other
tation related to optimal early development. groups of children at heightened risk for dysregulation also is
The specificity of this new measure was further underscored critical.
by MTMM comparisons supporting its discriminant validity in Nevertheless, the Q-scale promises to be an important compo-
relation to ego resiliency. The possibility that this new scale nent of assessment batteries targeting emotion regulation among
would overlap with or be subsumed by the higher order construct school-age children. In making a different type of methodology
of ego resiliency was suggested by commonalities across con- available to emotion researchers, this new assessment increases
struct definitions and shared items between the two Q-scales. the diversity of extant measures. Strengths of the Q-sort method-
Ego resiliency is conceptualized as the aspect of children's func- ology include its minimization of response bias and its use
tioning that promotes a tolerable level of anxiety, an appropriate across a wide age range. Furthermore, in generating comprehen-
degree of impulse control, and flexible adaptation in response sive profiles of individual personality functioning, this methodol-
CRTTERION Q-SCALES 915
ogy facilitates evaluations of the target construct relative to other Brody & I. Sigel (Eds.), Family research: volume 2. Families at risk
salient personality processes. (Thus, although Q-scales enhance (pp. 87-133). Hillsdale, NJ: Erlbaum.
the specificity of criterion sorts, raters should use the full 100- Cicchetti, D., & Rizley, R. (1981). Developmental perspectives on the
item CCQ deck to preserve the unique benefits of the Q-sort etiology, intergenerational transmission, and sequelae of child mal-
methodology.) Finally, this new Q-scale should enable research- treatment. In R. Rizley & D. Cicchetti (Eds.), Developmental perspec-
tives on child maltreatment (pp. 32-59). San Francisco: Jossey-Bass.
ers to explore questions regarding emotional development in
Cicchetti, D., & Schneider-Rosen, K. (1986). An organizational ap-
extant data sets that contain the CCQ. Therefore, the Emotion
proach to childhood depression. In M. Rutter, C. Izard, & P. Read
Regulation Q-Scale promises to make a unique and important (Eds.), Depression in young people, clinical and developmental per-
contribution to investigations of regulation beyond the early spectives, (pp. 71-134). New "&rk: Guilford Press.
years of life. Cicchetti, D., & Sroufe, L. A. (1978). An organizational view of affect:
Illustration from the study of Down's syndrome infants. In M.
Lewis & L. Rosenblum (Eds.), The development of affect (pp. 309-
References 350). New York: Plenum Press.
Cohen, J., & Cohen, P. (1983). Applied multiple regression/correlation
This article is intended solely for the personal use of the individual user and is not to be disseminated broadly.
Achenbach, T. M. (1991). Manual for the Teacher's Report Form and analysis for the behavioral sciences. Hillsdale, NJ: Erlbaum.
This document is copyrighted by the American Psychological Association or one of its allied publishers.
1991 Profile. Burlington: University of Vermont, Department of Cummings, E. M., Hennessy, K. D., Rabideau, G. J., & Cicchetti, D.
Psychiatry. (1994). Responses of physically abused boys to interadult anger in-
Alessandri, S. M. (1991). Play and social behavior in maltreated pre- volving their mothers. Development and Psychopathalogy, 6, 31—41.
schoolers. Development and Psychopathology, 3, 191-205. Deci, E. L., &Ryan, R. M. (1985). Intrinsic motivation and self-deter-
Anderson, J. C , & Gerbing, D. W. (1988). Structural equation modeling mination in human behavior. New %rk: Plenum Press.
in practice: A review and recommended two-step approach. Psycho- Eisenberg, R , & Fabes, R. A. (1992). Emotion, regulation, and the
logical Bulletin, 103, 411-423.
development of social competence. In M. S. Clark (Ed.), Review of
Barnett, D., Manly, J. T, & Cicchetti, D. (1993). Denning child mal- personality and social psychology: Vol. 14. Emotion and social behav-
treatment: The interface between policy and research. In D. Cic-
ior (pp. 119-150). Newbury Park, CA: Sage.
chetti & S. L. Toth (Eds.), Advances in applied developmental psy-
Eisenberg, N., Fabes, R. A., Guthrie, I. K., Murphy, B. C , Maszk, P.,
chology: Vol. 8. Child abuse, child development and social policy
Holmgren, R., & Suh, K. (1996). The relations of regulation and
(pp. 7 - 7 3 ) . Norwood, NJ: Ablex.
emotionality to problem behavior in elementary school children. De-
Bentler, P. M. (1990). Comparative fit indices in structural models. Psy-
velopment and Psychopathology, 8, 141 — 162.
chological Bulletin, 107, 238-246.
Ekman, P., & Friesen, W. V. (1975). Unmasking the face: A guide to
Bentler, P. M. (1992). On the fit of models to covariances and methodol-
recognizing emotions from facial cues. Palo Alto, CA: Consulting
ogy to the Bulletin. Psychological Bulletin, 112, 400-404.
Psychologists Press.
Block, J. (1961). The Q-sort method in personality assessment and
Erickson, M. E, Egeland, B., & Pianta, R. (1989). The effects of mal-
psychiatric research. Palo Alto, CA: Consulting Psychologists Press.
treatment on the development of young children. In D. Cicchetti &
Block, J., & Block, J. H. (1980a). The California Child Q-set. Palo
V. Carlson (Eds.), Child maltreatment: Theory and research on the
Alto, CA: Consulting Psychologists Press. (Original work published
causes and consequences of child abuse and neglect (pp. 647-684).
1969)
New York: Cambridge University Press.
Block, J., & Block, J. H. (1980b). The role of ego-control and ego-
Fox, N. A. (Ed.). (1994). The development of emotion regulation: Bio-
resiliency in the organization of behavior. In W. A. Collins (Ed.),
logical and behavioral considerations. Monographs of the Society for
Development of cognition, affect, and social relations. The Minnesota
Symposia on Child Psychology (Vol. 13, pp. 39-101). Hillsdale, NJ: Research in Child Development, 5 9 ( 2 - 3 , Serial No. 240).
Erlbaum. Gianino, A., & Tronick, E. Z. (1988). The mutual regulation model:
Campbell, D. T, & Fiske, D. W. (1959). Convergent and discriminant The infant's self and interactive regulation and coping and defensive
validation by the multitrait-multimethod matrix. Psychological Bulle- capacities. In T. M. Field, P. M. McCabe, & N. Schneiderman (Eds.),
tin, 56, 81-105. Stress and coping across development (pp. 4 7 - 6 8 ) . Hillsdale, NJ:
Caspi, A., & Moffitt, T. (1995). The continuity of maladaptive behavior: Erlbaum.
From description to understanding in the study of antisocial behavior. Gnepp, J. (1989). Children's use of personal information to understand
In D. Cicchetti & D. J. Cohen (Eds.), Developmental psychopathol- other people's feelings. In C. Saarni & P. L. Harris (Eds.), Children's
ogy: Volume 2. Risk, disorder, and adaptation (pp. 472-511). New understanding of emotion (pp. 151-177). New Ifork: Cambridge Uni-
York: Wiley. versity Press.
Cicchetti, D. (1990). The organization and coherence of socioemotional, Gnepp, J., & Hess, D. L. R. (1986). Children's understanding of verbal
cognitive and representational development: Illustrations through a and facial display rules. Developmental Psychology, 22, 103-108.
developmental psychopathology perspective on Down syndrome and Greenbaum, P. E., Dedrick, R. R, Prange, M. E., & Friedman, R. M.
child maltreatment. In R. Thompson (Ed.), Nebraska Symposium on (1994). Parent, teacher, and child ratings of problem behaviors of
Motivation: Vol. 36. Socioemotional development (pp. 259—366). youngsters with serious emotional disturbances. Psychological Assess-
Lincoln: University of Nebraska Press. ment, 6, 141-148.
Cicchetti, D. (1993). Developmental psychopathology: Reactions, re- Harris, P. L. (1989). Children and emotion: The development of psycho-
flections, projections. Developmental Review, 13, 471—502. logical understanding. Oxford, England: Blackwell.
Cicchetti, D., Ganiban, J., & Barnett, D. (1991). Contributions from Hart, J., Gunnar, M., & Cicchetti, D. (1996). Altered neuroendocrine
the study of high risk populations to understanding the development activity in maltreated children related to symptoms of depression.
of emotion regulation. In J. Garber & K. Dodge (Eds.), The develop- Development and Psychopathology, 8, 201-214.
ment of emotion regulation and dysregulation (pp. 15—49). New Harter, S., & Buddin, B. J. (1987). Children's understanding of the
\brk: Cambridge University Press. simultaneity of two emotions: A five-stage developmental acquisition
Cicchetti, D., & Manly, J. T. (1990). A personal perspective on conduct- sequence. Developmental Psychology, 23, 288-299.
ing research with maltreating families: Problems and solutions. In E. Hennessy, K. D., Rabideau, G. J,, Cicchetti, D., & Cummings, E. M.
916 SHIELDS AND CICCHETTI
(1994). Responses of physically abused and nonabused children to tives on motivation (Vol. 40, pp. 1-56). Lincoln: University of Ne-
different forms of interadult anger. Child Development, 65, 815-828. braska Press.
Howes, C , & Eldredge, R. (1985). Responses of abused, neglected, Ryan, R. M., Deci, E. L., & Grolnick, W. S. (1995). Autonomy, relat-
and nonmaltreated children to the behaviors of their peers. Journal edness and the self: Their relation to development and psychopathol-
of Applied Developmental Psychology, 6, 261-270. ogy. In D. Cicchetti & D. J. Cohen (Eds.), Developmentalpsychopa-
Izard, C. (Ed.). (1982). Measuring emotions in infants and children. thology: Vol. 1. Theory and methods (pp. 618-655). N e w a r k : Wiley.
New \brk: Cambridge University Press. Saarni, C. (1979). Children's understanding of display rules for expres-
Jackson, D. N. (1969). Multimethod factor analysis in the evaluation sive behavior. Developmental Psychology, 15, 424-429.
of convergent and discriminant validity. Psychological Bulletin, 72, Saarni, C. (1989). Children's understanding of strategic control of emo-
30-49. tional expression in social transactions. In C. Saarni & P. L. Harris
John, O. P., Caspi, A., Robins, R. W., Moffitt, T. E., & Stouthamer- (Eds.), Children's understanding of emotion (pp. 181-208). New
Loeber, M. (1994). The "Little Five": Exploring the nomological \fork: Cambridge University Press.
network of the Five-Factor Model of personality in adolescent boys. Saarni, C. (1990). Emotional competence: How emotions and relation-
Child Development, 65, 160-178. ships become integrated. In R. A. Thompson (Ed.), Nebraska Sympo-
Kagan, J. (1971). Change and continuity in infancy. New York: Wiley. sium on Motivation: Vol. 36. Socioemotional development (pp. 115-
This article is intended solely for the personal use of the individual user and is not to be disseminated broadly.