See discussions, stats, and author profiles for this publication at: https://www.researchgate.

net/publication/256072952

Child Behavior Checklist—Mania Scale (CBCL-MS): Development and

Evaluation of a Population-Based Screening Scale for Bipolar Disorder

Article  in  PLoS ONE · August 2013

DOI: 10.1371/journal.pone.0069459 · Source: PubMed

CITATIONS READS

27 462

7 authors, including:

Efstathios (Steven) Papachristou Johan Ormel

University College London University of Groningen
73 PUBLICATIONS   651 CITATIONS    865 PUBLICATIONS   64,002 CITATIONS   

SEE PROFILE SEE PROFILE

Albertine J Oldehinkel Marinos Kyriakopoulos

University of Groningen King's College London
584 PUBLICATIONS   24,695 CITATIONS    96 PUBLICATIONS   1,672 CITATIONS   

SEE PROFILE SEE PROFILE

Some of the authors of this publication are also working on these related projects:

Psychiatrie View project

A systematic review of associations between functional MRI activity and polygenic risk for schizophrenia and bipolar disorde View project

All content following this page was uploaded by Johan Ormel on 20 May 2014.

The user has requested enhancement of the downloaded file.

Child Behavior Checklist—Mania Scale (CBCL-MS):
Development and Evaluation of a Population-Based
Screening Scale for Bipolar Disorder
Efstathios Papachristou1, Johan Ormel4, Albertine J. Oldehinkel4, Marinos Kyriakopoulos1,2,
Marı́a Reinares1, Abraham Reichenberg3, Sophia Frangou3*
1 Child Psychiatry Department, Institute of Psychiatry, King’s College London, London, United Kingdom, 2 Child and Adolescent Mental Health Services, Maudsley
Hospital, London, United Kingdom, 3 Ichan School of Medicine at Mount Sinai, New York City, New York, United States of America, 4 Interdisciplinary Center of
Psychpathology and Emotion Regulation, University Medical Center Groningen, University of Groningen, Groningen, The Netherlands

Abstract
Context: Early identification of Bipolar Disorder (BD) remains poor despite the high levels of disability associated with the
disorder.

Objective: We developed and evaluated a new DSM orientated scale for the identification of young people at risk for BD
based on the Child Behavior Checklist (CBCL) and compared its performance against the CBCL-Pediatric Bipolar Disorder
(CBCL-PBD) and the CBCL-Externalizing Scale, the two most widely used scales.

Methods: The new scale, CBCL-Mania Scale (CBCL-MS), comprises 19 CBCL items that directly correspond to operational
criteria for mania. We tested the reliability, longitudinal stability and diagnostic accuracy of the CBCL-MS on data from the
TRacking Adolescents’ Individual Lives Survey (TRAILS), a prospective epidemiological cohort study of 2230 Dutch youths
assessed with the CBCL at ages 11, 13 and 16. At age 19 lifetime psychiatric diagnoses were ascertained with the Composite
International Diagnostic Interview. We compared the predictive ability of the CBCL-MS against the CBCL-Externalising Scale
and the CBCL-PBD in the TRAILS sample.

Results: The CBCL-MS had high internal consistency and satisfactory accuracy (area under the curve = 0.64) in this general
population sample. Principal Component Analyses, followed by parallel analyses and confirmatory factor analyses, identified
four factors corresponding to distractibility/disinhibition, psychosis, increased libido and disrupted sleep. This factor
structure remained stable across all assessment ages. Logistic regression analyses showed that the CBCL-MS had
significantly higher predictive ability than both the other scales.

Conclusions: Our data demonstrate that the CBCL-MS is a promising screening instrument for BD. The factor structure of
the CBCL-MS showed remarkable temporal stability between late childhood and early adulthood suggesting that it maps on
to meaningful developmental dimensions of liability to BD.

Citation: Papachristou E, Ormel J, Oldehinkel AJ, Kyriakopoulos M, Reinares M, et al. (2013) Child Behavior Checklist—Mania Scale (CBCL-MS): Development and
Evaluation of a Population-Based Screening Scale for Bipolar Disorder. PLoS ONE 8(8): e69459. doi:10.1371/journal.pone.0069459
Editor: Andreas Reif, University of Wuerzburg, Germany
Received December 15, 2012; Accepted June 10, 2013; Published August 14, 2013
Copyright: ß 2013 Papachristou et al. This is an open-access article distributed under the terms of the Creative Commons Attribution License, which permits
unrestricted use, distribution, and reproduction in any medium, provided the original author and source are credited.
Funding: TRAILS has been supported by grants from the Netherlands Organization for Scientific Research NWO (Medical Research Council program grant GB-MW
940-38-011; ZonMW Brainpower grant 100-001-004; ZonMw Risk Behavior and Dependence grants 60-60600-97-118; ZonMw Culture and Health grant 261-98-
710; Social Sciences Council medium-sized investment grants GB-MaGW 480-01-006 and GB-MaGW 480-07-001; Social Sciences Council project grants GB-MaGW
452-04-314 and GB-MaGW 452-06-004; NWO large-sized investment grant 175.010.2003.005; NWO Longitudinal Survey and Panel Funding 481-08-013), the Dutch
Ministry of Justice (WODC), the European Science Foundation (EuroSTRESS project FP-006), Biobanking and Biomolecular Resources Research Infrastructure
BBMRI-NL (CP 32), and the participating universities. The funders had no role in study design, data collection and analysis, decision to publish, or preparation of
the manuscript.
Competing Interests: The authors have declared that no competing interests exist.
* E-mail: sophia.frangou@mssm.edu

Introduction disorder. The typical delay between onset and diagnosis is 5–10
years [4–6] and is associated with greater clinical severity,
Bipolar Disorder (BD) is a complex mental disorder affecting increased psychosocial morbidity and higher treatment costs [7–
between 0.1% and 4.4% of the general population [1]. BD is the 9]. Although mania is the diagnostic hallmark of BD [10,11] the
sixth leading cause of disability worldwide particularly amongst differential diagnosis from Major Depressive Disorder (MDD) is
adolescents and young adults [2]. This is partly due to the typically often difficult as BD is commonly dominated by depressive
early onset of BD with the majority of patients presenting between symptoms [12,13]. Furthermore, BD is also associated with high
19–25 years of age [1,3]. More important however is the failure in rates (between 60–80%) of psychotic symptoms during mood
recognizing and treating BD particularly in the early stages of the episodes [14,15]. High rates of psychotic symptoms have also been

PLOS ONE | www.plosone.org 1 August 2013 | Volume 8 | Issue 8 | e69459

 Development of the CBCL-MS

reported in young patients and confirm their role as a key scale following the methodology defined for DSM-oriented
symptom dimension of BD in adolescence [16,17]. Additional subscale development by Achenbach et al. (2003) [42]. Content
diagnostic challenges arise from the symptomatic overlap between validity of the new scale was evaluated by an expert panel of child
BD and Attention Deficit Hyperactivity Disorder (ADHD), which and adolescent psychiatrists who selected 19 CBCL items that
also presents with poor attentional and emotional regulation [18]. relate directly to the diagnostic criteria for mania as currently
In response to the need for the early identification of individuals operationalized in the DSM5 (details in File S1). The new scale
at high risk for BD there have been several attempts to develop called CBCL-Mania Scale (CBCL-MS) was tested for its
and validate screening instruments. In adults, one of the most psychometric properties, sensitivity and specificity on data from
widely studied screening instruments is the Mood Disorder the TRacking Adolescents’ Individual Lives Survey (TRAILS)
Questionnaire (MDQ) [19], a self-report questionnaire based on [http://www.trails.nl/en/] [43]. TRAILS is a prospective study of
the Diagnostic and Statistical Manual of Mental Disorders (DSM) an epidemiologically representative cohort of 2230 Dutch
criteria for mania. A positive MDQ screen is based on participants adolescents who were assessed with the full CBCL at age 11,13
endorsing 7 or more lifetime manic symptoms, several co- and 16. Clinical outcomes were evaluated at age 19 using the
occurring, resulting to moderate or serious functional impairment. Composite International Diagnostic Interview (CIDI) [44]. We
In outpatient psychiatric settings the MDQ was reported to also compared the performance of the CBCL-MS against the
achieve sensitivity and specificity rates of 67%–83% and 86%, CBCL-Externalising Scale and the CBCL-PBD to test whether it
respectively [5]. Although specificity and sensitivity are theoreti- presents an improvement in terms of accuracy and predictive
cally independent of prevalence, in practice they are influenced by ability.
the clinical composition of the sample (e.g. proportion of severe to
mild cases) and interviewers’ assumptions about the frequency of a Methods
disorder [20]. Typically, in general population samples sensitivity
is lower and specificity is higher than that reported in clinical Participants
populations. For example, the sensitivity and specificity of the The sample consisted of participants of 2230 Dutch youth
MDQ in the general population are respectively 23–25% and 97– participating in the TRacking Adolescents’ Individual Lives
99% [21,22]. Additionally, many individuals with MDD, anxiety Survey (TRAILS). The sampling procedure and cohort details
disorders or ADHD screen positive on the MDQ [22,23]. for TRAILS have been previously described in detail [43] and can
A significant number of screening instruments for juvenile BD be found at the study website [http://www.trails.nl/en/]. Briefly,
have been developed and have been used mostly in clinical the cohort includes children born between 1 October 1989 and 30
populations. These include the Parent version of the Young Mania September 1991 in a well-defined geographic area in the north
Rating Scale (P-YMRS) [24], the Parent General Behavior Netherlands (information about the representativeness of the
Inventory (P-GBI) [25], the Adolescent General Behavior Inven- sample in S2). Permission to use anonymised data from the
tory (A-GBI) [26], the Youth Self Report (YSR) [27], the Teacher TRAILS was granted by the study management committee and
Report Form (TRF) [28], the Child Mania Rating Scale (CMRS) ethical approval was granted by the Dutch Central Committee on
[29], the Child Behaviour Checklist (CBCL) [30], and the Mood Research Involving Human Subjects (CCMO). All data were
Disorder Questionnaire Adolescent Version (MDQ-A) [31]. The anonymised by a research company TNS NIPO [http://www.tns-
CBCL [30] is the instrument most commonly used to generate nipo.com/].
profiles relevant to BD in youth. The CBCL is a parent report
checklist of 118 items mapping onto multiple aspects of Assessments
psychopathology over a 6-month period [30,32]. The CBCL When TRAILS cohort members were 11, 13 and 16 years old,
items are grouped in eight behavioural domains: aggressive their parents or parent surrogates completed the CBCL 6–18.
behaviour, anxiety/depression, attention problems, rule-breaking Each CBCL item was scored on a three point scale (0 = not true,
behavior, withdrawal/depression, somatic complaints, social 1 = somewhat or sometimes true, 2 = very true or often true) on the
problems and thought problems [30]. Different scales have been basis of the preceding 6 months. At age 19 years the diagnostic
generated based on varied combinations of these behavioural status of the TRAILS participants was ascertained using the
domains. Of relevance to BD, are the Externalizing Scale Computer Assisted Personal Interview version 20 (CAPI) of the
(comprising item scores from the rule-breaking and aggressive CIDI [http://www.hcp.med.harvard.edu/wmhcidi/]. The CIDI
behavior domains) and the CBCL-Pediatric Bipolar Disorder scale is a comprehensive, structured interview which was used by
(CBCL-PBD) (comprising item scores from the aggressive behav- trained lay interviewers to assess mental disorders according to
ior, anxiety/depression and attention problems domains) [33]. current diagnostic systems. It has high test-retest reliability for the
The CBCL-PBD is also referred to as the Dysregulation Profile as diagnosis of BD type I (BD-I) [45] as well as excellent concordance
it has been associated with disorder involving extensive behav- rates with the Structured Clinical Interview for DSM-IV (SCID)
ioural and emotional dysregulation [34] including BD [35]. for lifetime bipolar spectrum disorders [46]. Diagnostic assess-
However all available instruments have limited specificity for BD ments were conducted blind to participants’ CBCL scores.
as they have been associated with MDD, ADHD and anxiety
disorders [36–40]. Child Behavior Check List - Mania Scale (CBCL-MS)
Therefore there is still a need for screening instruments for BD An expert panel of child and adult psychiatrists, based at the
particularly for use in non-clinical populations of young individ- Institute of Psychiatry and the South London and Maudsley NHS
uals. In an attempt to address this need we developed and Foundation Trust, independently screened all CBCL items to
evaluated a new screening scale for BD in children and adolescents select those that correspond to the DSM operational criteria for
based on the CBCL 6–18 [30]. Despite the limited success of mania. As the diagnostic criteria for mania in DSM5 and ICD-10
previous CBCL-based screening instruments for juvenile BD we are identical [www.who.int/classificatios/icd/en/GRNBOOK.
decided to use it as the base of the new scale because of its cross- pdf] this selection is applicable to both diagnostic systems. In
cultural generalizability [41]. However, instead of using summary addition, the panel considered CBCL items relating to psychotic-
scores of the existing behavioural domains we constructed this new like experiences as childhood and adolescent psychosis and high

PLOS ONE | www.plosone.org 2 August 2013 | Volume 8 | Issue 8 | e69459

 Development of the CBCL-MS

CBCL total scores are frequently associated with later develop- oblique or varimax rotation (as appropriate). For each assessment
ment of mania [37–40]. Following consensus meetings, 19 items age, the model fit of the final solutions was established using
were selected for inclusion in the new CBCL- Mania Scale Confirmatory Factor Analysis (CFA) and assessed using two fit
(CBCL-MS) (Table 1). Detailed information on the item selection indices, the Root Mean Square Error of Approximation (RMSEA)
procedure is included in File S1. The scoring of the CBCL-MS at (cut-off values less than 0.06 indicate good fit and values as high as
each assessment age was based on summing the scores of each of 0.08 represent reasonable errors of approximation in the
the 19 individual items. Scores were then standardized (T scores) population) and the Confirmatory Fit index (CFI) (values above
following the scoring procedure recommended by Achenbach and 0.90–0.95 indicate good fit) [49].
Rescorla (2001) [32] using the TRAILS data as the standardiza- Sensitivity and Specificity of the CBCL-MS. Omnibus
tion sample. Standardization of the CBCL scores for the CBCL- tests using the standardized T scores of the CBCL-MS, the CBCL-
MS, as well as for other CBCL-based syndrome scales, was Externalizing Scale and the CBCL-PBD were performed to
performed separately at each assessment age. The CBCL-MS and compare the scores of participants with CIDI diagnoses of BD type
its scoring are available in File S5. I (BD-I) to those of healthy participants and participants with other
CIDI diagnoses that are considered relevant to BD as they involve
Statistical Analysis mood abnormalities (anxiety or depression) or inattention and
Analyses were performed using IBM SPSS Statistics, Version 19 behavioral disruption. We present data on Major Depressive
(www.spss.com) and MPlus 6.0 (www.statmodel.com). Disorder (MDD), General Anxiety Disorder (GAD), and ADHD
Reliability and validity of the CBCL-MS. As the CBCL- as the most pertinent exemplars. Finally, Receiver Operating
MS is a new scale the consistency of its items at each assessment Characteristics (ROC) curves [50] were used to calculate the
wave was evaluated using Cronbach’s alpha. In order to determine diagnostic efficiency of the CBCL-MS, CBCL-PBD and CBCL-
the number of factors that best describe the latent factor structure Externalizing Scale. A ROC curve illustrates the sensitivity (true
of the CBCL-MS at ages 11, 13 and 16 the following criteria were positive rate) of different cut-offs on the y axis and the 1-specificity
considered: the shape of the scree plot, parallel analysis using a (false positive rate) of the corresponding cut-offs on the x axis. In
permutated data approach (number of data sets: 5000; confidence the ROC analysis, the area under the curve (AUC) statistic
interval 95%) [47,48], the Kaiser criterion as an upper bound for provides a summary of test performance. AUC values range from
the number of factors to be retained, and the interpretability of the 0 to 1 with higher values denoting greater discriminative power
obtained factor structure. In order to conduct the parallel analysis, and diagnostic efficiency. The focus of the analysis was on BD-I as
principal components analysis (PCA) was performed first, with the usefulness of a test with poor discriminative ability for core

Table 1. Child Behavior Checklist-Mania Scale items and corresponding core and extended DSM-IV criteria for Mania.

CBCL Items DSM-IV criteria for Mania

Core Symptoms
37. Gets in many fights A distinct period of abnormally and persistently elevated, expansive or irritable mood
87. Sudden changes in mood or feelings
96. Thinks about sex too much
74. Showing off or clowning Inflated self-esteem or grandiosity
94. Teases a lot
76. Sleeps less than most kids Decreased need for sleep (e.g., feels rested after only 3 hours of sleep)
100. Trouble sleeping
93. Talks too much More talkative than usual or pressure to keep talking
104. Unusually loud Flight of ideas or subjective experience that thoughts are racing
78. Inattentive or easily distracted Distractibility (i.e., attention too easily drawn to unimportant or irrelevant external
stimuli)
10. Can’t sit still, restless or hyperactive Increase in goal-directed activity (at work, at school, or sexually) or psychomotor
agitation
60. Plays with own sex parts too much
41. Impulsive or acts without thinking Excessive involvement in pleasurable activities that have a high potential for painful
consequences (e.g., engaging in unrestrained buying sprees, sexual indiscretions, or
foolish business investments)
59. Plays with own sex parts in public
Extended Symptoms
34. Feels others are out to get him/her Delusions
85. Strange ideas
89.Suspicious
40. Hears sound or voices that aren’t there Hallucinations
70. Sees things that aren’t there

doi:10.1371/journal.pone.0069459.t001

PLOS ONE | www.plosone.org 3 August 2013 | Volume 8 | Issue 8 | e69459

 Development of the CBCL-MS

syndromal BD would be questionable. However, we also distinguishing BD-I cases from healthy TRAILS participants
performed ROC analysis using a more expanded definition of (AUC = 61%, p = 0.002).
caseness that also included BD type II (BD-II) and hypomania Comparison to the Externalizing scale of the CBCL: External-
with no major depressive episode. izing scale mean scores were significantly higher for TRAILS
participants with BD-I (58.49, SD = 18.27) in comparison to
Results healthy participants (48.44, SD = 8.25) and participants with
MDD (52.00, SD = 9.80) or GAD (50.52, SD = 7.80), but not
TRAILS participants with BD compared to participants with ADHD (p = 0.23). ROC analysis
At age 19, 56 of the TRAILS participants were diagnosed with showed that the Externalizing scale had AUC = 63%, p = 0.003
BD-I. Thirty-four cases had attracted other psychiatric diagnoses when discriminating between TRAILS cases with BD-I and
prior to being diagnosed with BD; seventeen had a single previous healthy participants.
diagnosis either for Oppositional Defiant Disorder (ODD) (n = 5) A forward stepwise logistic regression model showed that the
or Conduct Disorder (CD) (n = 5) or ADHD (n = 4) or GAD CBCL-MS had significantly increased ability to predict BD-I
(n = 3). Of the remaining seventeen BD cases, eleven had two prior compared to the CBCL-PBD (Wald x2 = 12.69, p,0.001) and the
diagnoses (ADHD/ODD = 2, ADHD/CD = 1, ADHD/GAD = 1, CBCL-Externalizing Scale (Wald x2 = 3.47, p = 0.05).
ADHD/MDD = 1, ODD/CD = 3, ODD/GAD = 2, ODD/
MDD = 1), five had three prior diagnoses (ODD/CD/GAD = 3, Discussion
ODD/CD/ADHD = 2) and one had four (ODD/CD/ADHD/
GAD). We present data on the psychometric properties and discrim-
inative ability of the CBCL-MS, a new DSM based screening scale
for BD-I based on the CBCL. We demonstrate that the new scale
Internal consistency and factor structure of the CBCL-MS has excellent psychometric properties; its discriminative ability and
Reliability analysis demonstrated high internal consistency for accuracy in a general population sample of young people represent
the 19 items of the CBCL-MS at all assessment ages (Cronbach’s an improvement over other commonly used scales particularly
alpha$80; total item correlation .0.37). A PCA of the CBCL-MS CBCL-PBD and the CBCL-Externalizing Scale.
data assessed at age 16 extracted four factors corresponding to: (1)
distractibility/disinhibition (2) psychotic symptoms (3) increased
Prevalence and Characteristics of TRAILS participants
libido (4) disrupted sleep (Figure 1). These factors were plausible
and interpretable as items’ loading segregated among the four
with BD
The lifetime prevalence of BD-I in the TRAILS sample was
factors as shown in File S3 and Table S2. The factor structure
2.5% which is identical to that reported in a recent epidemiolog-
represents the orthogonal solution of the PCA, as the factors
ical study of US adolescents [51]. Also consistent with previous
extracted using oblique rotation were only weakly correlated
literature, nearly 61% of BD-I cases in the TRAILS sample had
(r,0.3). Both the Parallel Analysis and Kaiser’s criterion supported
prior diagnoses associated with disruptive behaviour most
the retention of four factors. The scree plot of the extracted
commonly ADHD and ODD [18,50–54].
eigenvalues from the parallel analysis is given in Figure S1,
available on line. Analyses of the CBCL-MS data at ages 11 and
13 years resulted in an almost identical factor structure indicating Factor Structure of the CBCL-MS reveals developmentally
longitudinal stability of this solution (File S3 and Tables S2, S3 and meaningful dimensions of liability to BD
S4). Confirmatory factor analyses further supported the goodness The structural model of the CBCL-MS consisted of four factors.
of fit of 4-factor structure (RMSEA#0.05 and CFI$0.92). These factors correspond to dimensions of distractibility/disinhi-
bition, psychosis, increased libido and disrupted sleep. The factor
structure of the CBCL-MS showed remarkable temporal stability
Discriminative ability and performance of the CBCL-MS
between the ages of 11 to 16 which strongly supports the notion
Table 2 presents the mean total and CBCL-MS factor scores for
that it defines developmentally meaningful dimensions of liability
TRAILS participants who were diagnosed with BD-I and for those
to BD. This report is the first to describe developmental
who did not have any lifetime psychiatric diagnosis. Participants
dimensions of liability to BD. All other studies have focused on
with BD had significantly higher mean total CBCL-MS scores symptom dimensions during acute manic episodes in patients with
compared with participants with MDD (n = 178; p = 0.002) and established BD [54–58]. Nevertheless, there are significant
GAD (N = 20; p = 0.004) but not ADHD (N = 26; p.0.05) similarities. Cassidy and colleagues identified 5 factors in acute
(Figure 2). mania of which the ‘‘psychomotor pressure’’, ‘‘psychosis’’ and
The ROC curve analysis on the CBCL-MS data at age 16 is ‘‘increased hedonic’’ factors correspond to the distractibility/
illustrated in Figure 3. The AUC was 0.64 (p,0.01) which disinhibition, psychosis and increased libido factors in this study
represents a satisfactory performance for a general population [55]. Picardi et al [56] defined a four factor structure of acute
sample with low prior probability of true positives. The AUC mania based on the Brief Psychiatric Rating Scale. The factors
remained unchanged when caseness was expanded to include BD- they named ‘‘mania’’ and ‘‘disorganisation’’ include items similar
II and hypomania without major depressive episode. Moreover, to the distractibility/disinhibition factor identified here. In
the total CBCL-MS score performed better than the scores of each addition their ‘‘positive symptoms’’ factor overlaps with the
individual factors used independently or sequentially (details in psychosis factor in this study. Cassano et al [57] identified 5
supporting information S4 and Tables S5 and S6). factors in acute mania of which ‘‘psychomotor agitation’’ and
Comparison to CBCL-PBD: TRAILS participants with BD-I ‘‘psychoticism’’ correspond to the factors of distractibility/disinhi-
had significantly higher CBCL-PBD mean scores (56.51, bition and psychosis in the TRAILS cohort. All three studies also
SD = 15.91) in comparison to healthy participants (49.79, defined factors relating to dysphoric/euphoric mood and aggres-
SD = 9.69) but not compared to participants with MDD sion that seem to be present only during acute mania and may not
(p = 0.54), GAD (p = 0.51) or ADHD (p = 0.37). ROC curve represent an independent dimension of developmental liability to
analysis showed a moderate ability of the CBCL-PBD in BD.

PLOS ONE | www.plosone.org 4 August 2013 | Volume 8 | Issue 8 | e69459

 Development of the CBCL-MS

Figure 1. Factors and Factor Loadings of the Child Behavior Checklist-Mania Scale.
doi:10.1371/journal.pone.0069459.g001

Discriminative ability of the CBCL-MS individuals to have BD (true positives). A CBCL-MS score of 70 or
The overall accuracy was 0.64 for the CBCL-MS and CBCL- above will correctly identify 8,775 individuals (90% of this sample)
Externalising Scale and 0.61 for the CBCL-PBD. The results of as not having BD (true negatives). At the same threshold, 1,050
the logistic regression comparing the three scales showed that the individuals will be classified as possible cases. This sample will
CBCL-MS was statistically better in predicting BD outcome. include 75 true cases of BD (true positives) and 975 individuals
As seen by the ROC, different cut-off scores will influence the without BD (false positives). At first glance, one might be
sensitivity and specificity of the CBCL-MS. In general population concerned about the number of false positive cases. However,
screening the emphasis is usually on specificity thus selecting those scoring above 70 in the CBCL-MS were at a six-fold
individuals at highest risk for detailed follow-up assessments. To increased risk for BD (Positive Predictive Value: 16.57%; Negative
illustrate this point, in a hypothetical community sample of 10000 Predictive Value: 98.01%) compared to the rest of the sample and
youth with a 2.5% prevalence of BD we would expect 250 therefore they represent a high risk group. The field of early

PLOS ONE | www.plosone.org 5 August 2013 | Volume 8 | Issue 8 | e69459

 Development of the CBCL-MS

Table 2. Child Behavior Checklist-Mania Scale total and Factor Scores in TRAILS participants with Bipolar Disorder (BD) and healthy
participants.

BD participants (N = 56) Healthy Participants (N = 1201)

Mean Score (Standard Deviation) p

Total CBCL-MS 57.28 (16.08) 49.33 (8.95) ,0.001

Distractibility/Disinhibition 56.01 (14.71) 49.18 (8.98) ,0.001
Psychotic Symptoms 56.19 (20.20) 49.44 (8.84) ,0.001
Increased Libido 51.99 (12.28) 49.48 (8.88) 0.088
Disrupted Sleep 54.42 (12.28) 49.63 (9.33) 0.002

doi:10.1371/journal.pone.0069459.t002

intervention in BD is currently in its infancy [59] but as effective correlates [18,63,64], and frequent comorbidity [18]. Additionally,
therapies become available [60] scales such as the CBCL-MS may there is significant overlap in the symptoms of the two disorders
contribute to the identification of those at high risk. particularly with regards to increased activity, talkativeness and
None of the scales differentiated participants with BD from mood dysregulation [10,11]. Two main features distinguishing BD
those with ADHD in terms of mean scores. The relationship from ADHD have been proposed. Geller and colleagues
between these two disorders is complex. Available evidence emphasized the importance of either elevated mood or grandiosity
suggests at least partially overlapping aetiology and pathophysiol- for a diagnosis of mania [65]. However, in our study these
ogy for BD and ADHD because of familial co-segregation of the symptoms clustered with others in one factor and did not differ
two disorders [61,62], commonalities in their neurobiological across the two diagnostic categories. Others have suggested that

Figure 2. Child Behavior Checklist-Mania Scale Scores in TRAILS Participants.

doi:10.1371/journal.pone.0069459.g002

PLOS ONE | www.plosone.org 6 August 2013 | Volume 8 | Issue 8 | e69459

 Development of the CBCL-MS

Figure 3. Receiver Operating Characteristics curve of the Child Behavior Checklist-Mania Scale for Bipolar Disorder vs. healthy
TRAILS participants.
doi:10.1371/journal.pone.0069459.g003

episodicity is more indicative of BD than ADHD [66] but this Supporting Information
distinction seems less clear in children and adolescents [51]. It is
therefore possible that scales based on observed behaviour lack Figure S1 Scree Plot.
assay sensitivity in distinguishing between BD and ADHD. (TIF)

File S1 Selection of Items for the CBCL-MS.

Methodological Issues and Future Directions (DOC)
The present study has a number of strengths and limitations.
The TRAILS sample is representative of the population of young File S2 Representativeness of the TRAILS sample.
people in the Netherlands (Table S6). The prevalence of BD in the (DOC)
TRAILS is nearly identical to that of general population samples
elsewhere [1,51] which supports the generalizability of findings. File S3 Psychometric properties of the CBCL-MS across
Case ascertainment in the TRAILS does not depend on help- ages.
seeking behaviour or concern about impairment or severity and (DOC)
thus the sample is free from referral bias present in clinical
populations. However, the CIDI although widely used is designed File S4 Discriminative ability of the CBCL-MS.
for lay interviewers who rely on its structured format and may not (DOC)
probe or interpret participant responses further. Case ascertain-
ment was conducted at age 19. Since participants have not yet File S5 Appendix: CBCL-MS (scale and scoring sheet).
passed the entire period of risk for BD it is possible that further (PDF)
cases of BD may present in the future.
The CBCL-MS performed well within the context of a general Acknowledgments
population sample, it represents an improvement on available
Participating centers of TRAILS in the Netherlands, including the
scales and could contribute to future public health initiatives for
University Medical Center and University of Groningen, the Erasmus
the identification of youth at high risk for BD. Its accuracy is University Medical Center Rotterdam, the University of Utrecht, the
moderate and in the same broad range of the other CBCL-based Radboud Medical Center Nijmegen, and the Parnassia Bavo group.
screening instruments. Although it could be argued that this is
reflects limitations in the CBCL we would suggest that behavioural Author Contributions
ratings alone are unlikely to provide us with high levels of accuracy
in case identification for BD or any mental disorder. However a Conceived and designed the experiments: JO AO SF AR. Conceived and
designed the experiments: JO AO SF AR. Performed the experiments: JO
great strength of this study was the availability of CBCL
AO. Performed the experiments: JO AO. Analyzed the data: EP SF AR.
assessments at multiple time points from late childhood to early Analyzed the data: EP SF AR. Contributed reagents/materials/analysis
adulthood. This allowed us to test the temporal stability of the tools: EP SF AR MR MK. Contributed reagents/materials/analysis tools:
CBCL-MS factors which supports the validity of these dimensions EP SF AR MR MK. Wrote the paper: EP JO AO MK MR AR SF. Wrote
as developmentally meaningful premorbid indicators of BD. the paper: EP JO AO MK MR AR SF.

PLOS ONE | www.plosone.org 7 August 2013 | Volume 8 | Issue 8 | e69459

 Development of the CBCL-MS

References
1. Merikangas KR, Jin R, He JP, Kessler RC, Lee S, et al (2011) Prevalence and 28. Achenbach TM (1991) Manual for the Teacher Report Form and 1991 profile.
correlates of bipolar spectrum disorder in the world mental health survey Deaprtment of Psychiatry, Burlington, VT: University of Vermont.
initiative. Arch Gen Psychiatry 68: 241–251. 29. Pavuluri MN, Henry DB, Devineni B, Carbray JA, Birmaher B (2006) Child
2. Murray CJ, Lopez AD, Jamison DT (1994) The global burden of disease in mania rating scale: development, reliability, and validity. J Am Acad Child
1990: summary results, sensitivity analysis and future directions. Bull World Adolesc Psychiatry 45:550–560.
Health Organ 72: 495–509. 30. Achenbach TM (1991) Manual for the Child Behavior Checklist/4-18 and 1991
3. Perlis RH, Miyahara S, Marangell LB, Wisniwski SR, Ostacher M (2004) Long- Profile. Department of Psychiatry. Burlington, VT: University of Vermont.
term implications of early onset in bipolar disorder: data from the first 1000 31. Wagner KD, Hirschfeld RM, Emslie GJ, Findling RL, Gracious BL, et al (2006)
participants in the systematic treatment enhancement program for bipolar Validation of the Mood Disorder Questionnaire for bipolar disorders in
disorder (STEP-BD). Biol Psychiatry 55: 875–881. adolescents. J Clin Psychiatry 67:827–830.
4. Lish JD, Dime-Meenan S, Whybrow PC, Price RA, Hirschfeld RM (1994) The 32. Achenbach TM, Rescorla LA (2001) Manual for the ASEBA School-Age Forms
National Depressive and Manic-depressive Association (DMDA) survey of & Profiles. Research Center for Children, Youth and Families. Burlington, VT:
bipolar members. J Affect Disord 31: 281–294. University of Vermont.
5. Hirschfeld RM, Lewis L, Vornik LA (2003) Perceptions and impact of bipolar 33. Biederman J, Wozniak J, Kiely K, Ablon S, Faraone S, et al (1995) CBCL
disorder: how far have we really come? Results of the national depressive and clinical scales discriminate prepubertal children with structured interview-
manic-depressive association 2000 survey of individuals with bipolar disorder. derived diagnosis of mania from those with ADHD. J Am Acad Child Adolesc
J Clin Psychiatry 64: 161–174. Psychiatry 34: 464–471.
6. Berk M, Dodd S, Callaly P, Berk L, Fitzgerald P, et al. (2007) History of illness 34. Kim J, Carlson GA, Meyer SE, Bufferd SJ, Dougherty LR, et al (2012)
prior to a diagnosis of bipolar disorder or schizoaffective disorder. J Affect Correlates of the CBCL-dysregulation profile in preschool-aged children. J Child
Disord 103: 181–186. Psychol Psychiatry 53: 918–926.
7. Keck PE Jr, Kessler RC, Ross R (2008) Clinical and economic effects of 35. Faraone SV, Althoff RR, Hudziak JJ, Monuteaux M, Biederman J (2005) The
unrecognized or inadequately treated bipolar disorder. J Psychiatr Pract 14 CBCL predicts DSM bipolar disorder in children: a receiver operating
Suppl 2: 31–38. characteristic curve analysis. Bipolar Disord 7: 518–524.
8. Post RM, Leverich GS, Kupka RW, Keck PE Jr, McElroy SL, et al (2010) Early- 36. Mick E, Biederman J, Pandina G, Faraone SV (2003) A preliminary meta-
onset bipolar disorder and treatment delay are risk factors for poor outcome in analysis of the child behavior checklist in pediatric bipolar disorder. Biol
adulthood. J Clin Psychiatry 71: 864–872. Psychiatry 53:1021–1027
9. Ketter TA (2011) Strategies for the early recognition of bipolar disorder. J Clin 37. Youngstrom EA, Findling RL, Calabrese JR, Gracious BL, Demeter C, et al
Psychiatry 72: e22. (2004) Comparing the diagnostic accuracy of six potential screening instruments
10. American Psychiatric Association (2013) Diagnostic and Statistical Manual for for bipolar disorder in youths aged 5 to 17 years. J Am Acad Child Adolesc
Mental Disorders, 5th Edition (DSM5). Washington, D.C.: American Psychiatric Psychiatry 43:847–858.
Association. 38. Volk HE, Todd RD (2007) Does the Child Behavior Checklist Juvenile Bipolar
11. World Health Organisation (1992) ICD-10 Classifications of Mental and Disorder Phenotype Identify Bipolar Disorder? Biol Psychiatry 62: 115–120.
Behavioural Disorder: Clinical Descriptions and Diagnostic Guidelines. Geneva: 39. Diler RS, Birmaher B, Axelson D, Goldstein B, Gill M, et al (2009) The Child
World Health Organisation. Behavior Checklist (CBCL) and the CBCL-bipolar phenotype are not useful in
12. Perugi G, Micheli C, Akiskal HS, Madaro D, Socci C, et al (2000) Polarity of the diagnosing pediatric bipolar disorder. J Child Adolesc Psychopharmacol 19: 23–
first episode, clinical characteristics, and course of manic depressive illness: a 30.
systematic retrospective investigation of 320 bipolar I patients. Compr 40. Althoff RR, Verhulst FC, Rettew DC, Hudziak JJ, van der Ende J (2010) Adult
Psychiatry 41: 13–18. outcomes of childhood dysregulation: a 14-year follow-up study. J Am Acad
13. Judd LL, Akiskal HS, Schettler PJ, Endicott J, Maser J, et al (2002) The long- Child Adolesc Psychiatry 49: 1105–1116.
term natural history of the weekly symptomatic status of bipolar I disorder. Arch 41. Ivanova MY, Dobrean A, Dopfner M, Erol N, Fombonne E, et al (2007) Testing
Gen Psychiatry 59: 530–537. the 8-syndrome structure of the child behavior checklist in 30 societies. Holland
14. Keck PE Jr, McElroy SL, Havens JR, Altshuler LL, Nolen WA (2003) Psychosis to Australia. J Clin Child Adolesc Psychol 36: 405–417.
in bipolar disorder: Phenomenology and impact on morbidity and course of 42. Achenbach TM, Dumenci L, Rescorla LA (2003) DSM-oriented and empirically
illness. Compr Psychiatry 44: 263–269. based approaches to constructing scales from the same item pools. J Clin Child
15. Coryell W, Leon AC, Turvey C, Akiskal HS, Mueller T, et al (2001) The Adolesc Psychol 32: 328–340.
significance of psychotic features in manic episodes: A report from the NIMH 43. Huisman M, Oldehinkel AJ, de Winter A, Minderaa RB, de Bildt A, et al (2008)
collaborative study. J Affect Disord 67: 79–88. Cohort Profile: The Dutch ‘TRacking Adolescents’ Individual Lives ‘Survey’;
16. Johns LC, Nazroo JY, Bebbington P, Kuipers E (2002) Occurrence of TRAILS. Int J Epidemiol 37: 1227–1235.
hallucinatory experiences in a community sample and ethnic variations. 44. World Health Organisation (1990) Composite International Diagnostic
Br J Psychiatry 180:174–178. Interview (CIDI). Geneva:World Health Organisation.
17. Topor DR, Swenson L, Hunt JI, Birmaher B, Strober M, et al (2013) Manic 45. Wittchen HU (1994) Reliability and validity studies of the WHO–Composite
symptoms in youth with bipolar disorder: Factor analysis by age of symptom International Diagnostic Interview (CIDI): a critical review. J Psychiatr Res
onset and current age. J Affect Disord 145: 409–412. 28:57–84.
18. Skirrow C, Hosang GM, Farmer AE, Asherson P (2012) An update on the 46. Kessler RC, Akiskal HS, Angst J, Guyer M, Hirschfeld RM, et al (2006)Validity
debated association between ADHD and bipolar disorder across the lifespan. of the assessment of bipolar spectrum disorders in the WHO CIDI 3.0. J Affect
J Affect Disord 141:143–159. Disord 96:259–269.
19. Hirschfeld RM, Williams JB, Spitzer RL, Calabrese JR, Flynn L (2000) 47. Horn JL (1965) A rationale and test for the number of factors in factor
Development and validation of a screening instrument for bipolar spectrum analysis.Psychometrika 30:179–185.
disorder: the Mood Disorder Questionnaire. Am J Psychiatry 157: 1873–1875. 48. Zwick WR, Velicer WF (1986) Comparison of Five Rules for Determining the
20. Robins LN (1985) Epidemiology: reflections on testing the validity of psychiatric Number of Components to Retain. Psychol Bull 99: 432–442.
interviews. Arch Gen Psychiatry 42: 918–924. 49. Hu Lt, Bentler PM (1999) Cutoff criteria for fit indexes in covariance structure
21. Hirschfeld RM, Holzer C, Calabrese JR, Weissman M, Reed M (2003) Validity analysis: Conventional criteria versus new alternatives. Struct Equ Modeling 6:
of the mood disorder questionnaire. Am J Psychiatry 160:178–180. 1–55.
22. Dodd S, Williams LJ, Jacka FN, Pasco JA, Bjerkeset O, et al (2009) Reliability of 50. Grzybowski M, Younger JG (1997) Statistical methodology: III. Receiver
the Mood Disorder Questionnaire: comparison with the Structured Clinical operating characteristic (ROC) curves. Acad Emerg Med 4: 818–826.
Interview for the DSM-IV-TR in a population sample. Aust N Z J Psychiatry 43: 51. Merikangas KR, Cui L, Kattan G, Carlson GA, Youngstrom EA, et al (2012)
526–530. Mania with and without depression in a community sample of US adolescents.
23. Zimmerman M, Galione JN, Ruggero CJ, Chelminski I, Dalrymple K, et al Arch Gen Psychiatry 69:943–951.
(2011) Are screening scales for bipolar disorder good enough to be used in 52. Biederman J, Faraone SV, Wozniak J, Mick E, Kwon A, et al (2004) Further
clinical practice? Compr Psychiatry 52:600–606. evidence of unique developmental phenotypic correlates of pediatric bipolar
24. Gracious BL, Youngstrom EA, Findling RL, Calabrese JR (2002) Discriminative disorder: findings from a large sample of clinically referred preadolescent
validity of a parent version of the Young Mania Rating Scale. J Am Acad Child children assessed over the last 7 years. J Affect Disord 82 Suppl 1: S45–58.
Adolesc Psychiatry 41:1350–1359. 53. Duffy A, Alda M, Crawford L, Milin R, Grof P (2007) The early manifestations
25. Youngstrom EA, Findling RL, Danielson CK, Calabrese JR (2001) Discrimi- of bipolar disorder: a longitudinal prospective study of the offspring of bipolar
native validity of parent report of hypomanic and depressive symptoms on the parents. Bipolar Disord 9:828–838.
General Behavior Inventory.Psychol Assess 13:267–276. 54. Biederman J, Petty CR, Monuteaux MC, Mick E, Parcell T, et al (2008) The
26. Danielson CK, Youngstrom EA, Findling RL, Calabrese JR (2003) Discrimi- longitudinal course of comorbid oppositional defiant disorder in girls with
native validity of the general behavior inventory using youth report. J Abnorm attention-deficit/hyperactivity disorder: findings from a controlled 5-year
Child Psychol 31:29–39. prospective longitudinal follow-up study. J Dev Behav Pediatr 29: 501–507.
27. Achenbach TM (1991) Integrative guide for the 1991 CBCL/4-18, YSR and 55. Cassidy F, Forest K, Murry E, Carroll BJ (1998) A Factor analysis of the signs
TRF profiles. Deaprtment of Psychiatry, Burlington, VT: University of Vermont and symptoms of mania. Arch Gen Psychiatry 55: 27–32.

PLOS ONE | www.plosone.org 8 August 2013 | Volume 8 | Issue 8 | e69459

 Development of the CBCL-MS

56. Picardi A, Battisti F, de Girolamo G, Morosini P, Norcio B, et al (2008) 61. Neslihan Inal-Eiroglu F, Ozerdem A, Miklowitz D, Baykara A, Akay A (2008)
Symptom structure of acute mania: a factor study of the 24-item Brief Mood and disruptive behavior disorders and symptoms in the offspring of
Psychiatric Rating Scale in a national sample of patients hospitalized for a manic patients with bipolar I disorder. World Psychiatry 7:110–112.
episode. J Affect Disord 108:183–189. 62. Faraone SV, Biederman J, Mennin D, Wozniak J, Spencer T (1997) Attention-
57. Cassano GB, Mula M, Rucci P, Miniati M, Frank E, et al (2009) The structure deficit hyperactivity disorder with bipolar disorder: a familial subtype? J Am
of lifetime manic-hypomanic spectrum. J Affect Disord 112: 59–70. Acad Child Adolesc Psychiatry 36: 1378–1387.
58. Agrawal A, Nurnberger JI Jr, Lynskey MT (2010) Item response modeling of 63. Passarotti AM, Sweeney JA, Pavuluri MN (2010) Neural correlates of response
DSM-IV mania symptoms in two representative US epidemiological samples. inhibition in pediatric bipolar disorder and attention deficit hyperactivity
Psychol Med 40:1549–1558. disorder. Psychiatry Res 181:36–43.
59. Berk M, Hallam K, Malhi GS, Henry L, Hasty M, et al (2010) Evidence and 64. Passarotti AM, Sweeney JA, Pavuluri MN (2010) Emotion processing influences
implications for early intervention in bipolar disorder. J Ment Health 19:113– working memory circuits in pediatric bipolar disorder and attention-deficit/
126. hyperactivity disorder. J Am Acad Child Adolesc Psychiatry 49:1064–1080.
60. Miklowitz DJ, Schneck CD, Singh MK, Taylor DO, George EL, et al (2013) 65. Geller B, Williams M, Zimerman B, Frazier J, Beringer L, et al (1998)
Early intervention for symptomatic youth at risk for bipolar disorder: a Prepubertal and and early adolescent bipolarity differentiate from ADHD by
manic symptoms, grandiose delusions, ultra-rapid or ultradian cycling. J Affect
randomized trial of family-focused therapy. J Am Acad Child Adolesc Psychiatry
Disord 51:81–91.
52:121–131.
66. Leibenluft E, Rich BA (2008) Pediatric bipolar disorder. Annu Rev Clin Psychol
4: 163–187.

PLOS ONE | www.plosone.org 9 August 2013 | Volume 8 | Issue 8 | e69459

View publication stats