Diagnostic Accuracy in Adolescents of Several Depression
Diagnostic Accuracy in Adolescents of Several Depression
Bek~viw Ckecklist (CBCL) and Youth Self-report IYSR) in a clinicahy referred sample of adolescents
1’41s
= 6671 sh owed that t?aeir performance in discriminating between depressed afid not depressed
patients was comparable to other specifically designed depression rating scales (area under the ROC
curves bemeen 0.75 and 0.82). Performance was better for boys th2.n for girls. These results suggest that
specific depression selfreport rating scales may be unnecessary in adolescents if CBCL or YSR data arz
available, and that self-reports are not more accurate than parent-reports in the identification of
dqxession in this age group.
Kq worrls= Depression rating scales; Child Behaviour Checklist; Adolescents; Receiver operating charac-
teristic analysis
scale and a Behavior Problem scale. The Behav- Gort WSR) (Achenbach and Edelbrock, 1987) is
ior Problem scale consists of 1118items (e.g. ‘Acts essentially identical to the CBCL with the excep-
too young for his/her age’) which are endorsed tions that it is a self-rating scale for children or
according to three response options: not true adolescents aged 11 to 18 years, it is worded in
(rating = O), somewhat/ sometimes true (rating = the first person (e.g. ‘I act too young for my age’)
1) and very true (rating = 2). The Youth Self-Re- and it contains only 102 items of the 118 in the
TABLE 1
Item content of the various depression scales. I: CBCL-NUR; 2: YSR-CDT; 3: YSR-DEPB; 4: YSR-DEPG; 5: Anxious/depressed
(all items from parent and child reports except items 18 and 91 from YSR on!;). Composite scale equals CBCL-NUR plus
YSR-CDL divided by two.
51 Feels dizzy +
54 Overtired + -t +
56B Headaches +
62 Poorly coordinated +
69 Secretive +
71 Self-conscious + + +
75 Shy or timid + + +
77 Sleeps more +
tsn Stares blankly +
86 Stubborn +
x7 Sudden changes in mood + -k -t
x9 Suspicious -t -t -t
111 Withdrawn +
Worrying “I- + + + +
112
10
diagnosis in this group was separation anxiety of these scales (Costello and Angold, 1988; Hsiao
(9.3%). et al., 1989; Mossman and Somoza, 1989). ROC
analysis was carried out using Dorfman and Alfs
Statistical analysis maximum likelihood estimation program
Analysis of tests, scales or diagnostic systems RSCORE-2 (Dor.?nan and Aff, 1969; Swets itnd
in terms of the receiver operating characteristic Pickett, 1982). QnIy data about the areas under
(RQC) of signal detection theory provides a mea- the ROC curves (AUC) are presented, other
sure of diagnostic accuracy not influenced by statistics of the ROC curves are available on
decision biases and prior probabilities, and it request.
places the performances of different systems on a Cases were not included in analyses if more
common, easy to interpret scale (Swets, 19881. than 10% of item responses in the scale being
This technique has been recommended as an studied were missing; this may result in slightly
appropriate means of testing the relative accuracy different number of cases in the various groups.
TABLE 2
Comparison of depression scores according to six scales between diagnostic grocps.
YSR-CDI
I. Major depression 24 19.5 1.13 7.0
2. No major depression 635 11.4 - 0.04 6.8 (l-2) 5.76 0.000
3. Dysthymia 60 16.8 0.74 7,2 (l-3) 1.58 0.118
4. Separation anxiety 61 10.9 -0.11 6.3 (l-415.52 0.000
Y SR-DEPB
I. Major depression 14 24.51 1,25 9.4
2. No major depression 368 14.5 - 0.05 7.7 (l-2) 4.89 0.000
3. Dyst hymia 27 18.5 0.46 7.7 (1-3) 2.31 0.026
4. Separation anxiety 30 14.7 - 0.02 6.2 if -4) 4.25 0.000
YSR-DEPG
1. Major depression 10 37.3 0.79 11.4
2. No major depression Xl 26.4 - 0.03 13.2 (l-2) 257 0.011
3. Dyst hymia 33 36.2 0.71 12.6 (l-3) 0.24 0.814
4. Separation anxiety 30 23.5 - 0.25 14.2 (l-4) 2.79 0.008
Composite
1. Major depression 23 21.0 1.24 6.1
2. No major depression 627 12.9 - 0.05 6.3 (l-2) 6.25 0.000
3. Dysthymia 62 17.4 0.66 6.0 (l-3) 2.45 0.016
4. Separation anxiety 57 13.0 - 0.03 6.1) (I-4) 5.36 0.000
Anxious/depressed
1. Major depression 24 35.0 1.10 10.0
2. No major dspiessicul 623 22.3 - 0.04 10.9 (l-2) 5.59 0.000
3. Dysthymia 60 29.a 0.64 11.2 (l-3) 1.9s 0.055
4. Separation anxiety 59 21.6 -0.11 10.5 (l-4) 5.35 n.000
” two-tailed I-test.
12
TABLE 3
Areas under the ROC curves for six depression scales when discriminating between patients with major depression and those with
other diagnoses (95% confidence intervals between parentheses)
SlXl-G
according to DSM-III criteria is only moderate,
SE SP SE SP SE SP
but comparable to that obtained in other reliabil-
Ih 0.X3 O.bX 0.7’1 0.71 (1.90 0.49 ity studies in children and adolescents. For exam-
1X 0.78 0.76 0.79 0.80 0.W MI
ple, Rey et al. (1989) reported an average reliabil-
20 O.hS 0.x3 0.7 1 O.Xh 0.X1 I Oh8
33
__ OX 0.89 0.71 II.91 Wio (I.76 ity across published studies of K = 0.56 for major
-- depression, k = 0.36 for dysthymia, and k = 0.74
for separation anxiety. A detailed study of the
Diagnostic Interview Schedule for Children-Re-
vised (DISC-R) reported poor agreement be-
so. At the contrary, depression scores for girls tween diagnoses obtained from the DISC-R and
with dysthymia were significantly higher than for those obtained from clinicians using a semi-stcuc-
boys 0 = XL_, T3* df = 56, 1’ < 0.(102) and the size of tured interview covering the same information
the difference was substantial IN74SD). That is, base as the DISC-R and employing a similar
boys with major depression had scores as high or (computerized) procedure to assign diagnoses.
higher than girls, while dysthymic boys had dc- Agreement for major depression was K = 0.36 for
pression scores significantly lower than dysthymic parent interview and K = 0.39 for child interview.
girls. Boys with other disorders also had signifi- The parallel results for separation anxiety were
cantly lower scores (0.5SD) than girls (I = 6.78, 0.30 and 0.27 (Schaffer et al., 1988). The overall
dC = 573. I’ = O.U~NN diagnostic performance of the various scales
Perform;ince of the different depression scales within this clinical sample is only moderate. Be-
according to age (younger = 12, 13 years: older = cause reliability of clinical diagnosis sets a ceiling,
14- 56 years) could not be estimated because there the performance of the scales would probably
were not enough cases of major depression (N = have been higher if diagnoses were more reliable.
3) in the younger age groups. However, scores on The results reported here might underestimate
the Composite scale were significantly higher (t the ability of these scales to discriminate between
= 4.29, df = 595, P < O.(?OObin the older than in depressed and not depressed patients.
the younger age group, but the size of the differ- Mood disorders in this clinical sample were
ence was not large (0.35SD). Similar results were not diagnosed often (13.9%) and major depres-
obtained with the other scales. sion, in particular, was diagnosed in only 3.6% of
Table 4 shows senstivity GE) and specificity cases. This may be reflection of the patterns of
(SP) at various cut-off points for the Composite referral to the unit, predominantly disruptive dis-
scale, when comparing diagnostic accuracy be- orders, and that clinicians were conservative when
tween major depression and other disorders. SEs assigning major depression diagnosis.
and SPs are modest. For example, a cut-off of 20 Confidence intervals of the AlJCs for the five
identified 65% of patients with major depression scales overlap (this may be due largely to the
in this referred sample: a false negative rate of small number of subjects in the major depression
15% and a rate of 17% for false positives. It also
-_ group). Therefore, we can only conclude that
shows that SE and SP vary according to the sex of there are no significant differences in accuracy
the patient. between them. Yet, some scales appear to Per-
form better than others, but this can only be
Discussion considered suggestive at this stage. Overall, there
is no support for the assumption that self-reports
These results need to be interpreted on the are more accurate in identifying depressed ado-
assumption that major depression, dysthymia and lescents than parent reports Mngold, 1988).
1-l
It is of interest that scores in subjects with depression may be easier to identif) in adolescent
majL>r depression are significantly different from boys than in girls. Dysthymic girls had depression
those without major depression (and those with scores almost one SD higher than dysthymic boys.
separation anxiety disorder) on all five SC&S. whiic that was not the case for those with major
The size of the difference I z-SCOITS, Table 1) is depression. Girls with other disorders also had
largest for YSR-DEPB Il.30SDI followed by the depression scores significantly higher than boys
Composite scale (1.29SDL Y SR-CD] t ].17SD), (0.5 SD). Thi s may reflect the higher preva!;;:nce
Anxious/ depressed ! 1.1LcSDJ, CBCLNU R ( 1 .W of depressive symptoms in girls and this, in turn,
SD) and YSR-DEPG (0.82 DI. Similar analyses may make it more difficult to discriminate be-
with other scales have produced conflicting rc- tween major depression and other disorders, par-
suits (Costello and Angold, 1988). For example ticularly dysthymia, in girls than in boys. Sex
Nelson et al. (1987) reported no difference in differences in the prevalence and pattern of de-
CDI scores between depressed and conduct dis- pressive symptoms in adolescents needs further
ordered children and adolescent inpatients, while study.
Rotundo and Hcnsley ( 198% found significant The AUCs for the comparison between ado-
diffcrcnccs bctwecn dcprcsscd and non-dc- lescents with and without major depression (Ta-
pressed subjects. hle 3) art: comparable tu the AUC found by
The depression factor found in the YSR of Mossman and Somoza (1989) for the composite
girls (YSR-DEPG) is substantially less accurate of seven studies using the Dexamethasone Sup-
than the one found in boys (YSR-DEPB). The pression Test (DST; AUC = 0.79). This suggests
difference is particularly marked when comparing that the scales considcrcd here arc as accurate as
major depression with dysthymia. However, this is the DST in discriminating patients with major
largely accounted for by the sex of the patient depression from those with other disorders. How-
rather than the scale itself. These results provide cvcr, pcrformancc of depression scales may have
some evtdcnce supporting the validity of these been enhanced by the fact that scales and diag-
YSR scales. The anxious/deprcsscd factor re- nosticians use the same sources of information
ccntly extracted by Achenbach et al. (19896 that (reports by parents and children), and diagnosti-
uses parent and child reports, discriminates be- cians were not blind to actual item responses in
tween major depression and separation anxiety as the questionnaires, while the DST is a diagnostic
accurately as bctwccn major depression and other test based on information independent from that
disorders. This suggests that this factor taps used to make a clinical diagnosis. This may have
mainly a depression construct, despite the fact cnhanccd the performance of these scales.
that, at face value, item content corresponds to a SE and SP of these scales in this referred
mixture of anxiety and depressive symptoms. sample are only moderate, but comparable to
In a recent report, Roberts et al. (19911 com- those reported for other depression scales in clin-
pared the performance of two depression scales, ical samples. For example, Asarrsow and Carlson
the Center for Epidemiological Studies Depres- (198% found a SE = 88% and SP = 88% in a
sion Scaie KES-D) and the Beck Depression group of 82 inpatients assessed with the K-SADS-
Inventory (BDI) in a community sample (N = E for a cut-off score of 13 on the Depression
1,704) diagnosed with the K-SADS administered Self-Rating Scale. Administering the same scale,
by trained interviewers. They reported an AUC 3irlcson et al. (1987) reported an SE = 07% and
for males of 0.93 (BDI) and 0.87 (CES-D). For SP = 77% in a group of 155 outpatients with
females the AUCs were 0.83 for both scales. clinical ICD-9 diagnosis, when a cutpoint of 15
Results for girls are comparabIe to those found was used. Kashani et al. (1990) described a SE =
with the Composite scale in our study, while 48% and SP = 87% for a cutpoint of 16 on the
those for boys are higher. The pattern of higher Beck Depression Inventory administer-cd to a
accuracy for boys than girls is similar to that group of 100 adolescent inpatients diagnosed with
found in this study. This suggests that major the DICA.
AL’hcnhack. T.M. and Edelbro~k. C.S. (1987) Manual fr,r the
Yuuth Self-Report rtnd f’rtrfile. University of Vermont,
It can be concluded that scales extracted from kpartmenr uf Psychiatry. Burlington. VT.
Achcnhtich, T.M.. McConaughy. S.H. and Howei]. C.T. f 19x7)
the CBCL and YSR can indeed be used to idcn-
Chibd/ad~~escenF behavioral and emotional problems: im-
ti@ depressed subjects. In clinical samples, they plicntionr of cross-informant f3melations for situational
seem to be as accurate discriminating Individuals specificiQ.PsychoI. BuEl. IfIl, 213-232.
with major depression as other specifically de- .kkIhCh. -t-hf.. ~OIICWFS, C.K., Quay, H.C., Verhulst, F.C.
signed instruments and laboratory tests such as and HOWL% C.T. t 19x9) Replication nf empiricatty derived
syndromes as a brtsis fur taxonomy of child/adolescent
the DST. If both CBCL and YSR arc available,
psychopaFhalogy. .I Abnorm. Child Psychicrl. 17. 29%32.3.
the composite of CBCL-NUR and YSR-CDI may A~dd, A. t I9XKI Childhood 2nd adolescent depression, II:
be the best measure to utilize. If only the YSR is research in clinical populatirrns. 6i. J. Ps&iatry 15, 476
available. any of the three scales described here 492.
may be used, although YSR-DEPG appears to Asarntrw. J.R. and Carlcon. G.A. (IYXh) Depression self-rat-
ing scale: utility with child psychiatric inpatients. J. Con-
perform wf)rst, particularly when discriminating
sult. C‘lin. Psychology 53. 491-499.
hetwcern major depression and dysthymia. There Birlcson. P.. I ludwn. I.., Buchanan, D.G. and Wolf. S. i 19X7)
may be some advltntagc in using different cut off Clinical cvsluation of self-rating scale for depressive disor-
scores for buys and girls hecausc of the differ- der in childhood. J. Child Psychol. Psychiatry 2% Xi-1%
C’ostello. E.J. and Artgold. A. t 15)X8) Scaies to assess child and
ences in :i+.:uracy found in this study and by
adolcscenl depression: Checklists. screens and nets. J. Am.
Roberts et al. WI91 I. This, and the pattern of
Acad. Child Adtrlesc. Psychiatry 27, 72f1-737.
depressive symptoms according to age requires Dorfmart. D.D. and Alf, E. (1YhY) Maximum likelihood esti-
further study. In cases in which only parent rc- mation ot parameters of signal detection theory and deter-
ports are available, CBCL-NUR may be used. mination of confidence intervals: rating method data. J.
l-bwever, more rcscarcR w the efficiency of multi Mathem& Psychology h, 4X7-496.
Endicott, J., Cohen, .I.. Ncc, J., Fteiss. J. and Sarantakos. S.
vs. single purpose instrunlents needs to be carried ( IWI ) Hrtmilton Dcprcssinn Rating Scale extracted from
out. The findings reported hcrc will allow the regular and change,7 =A t:ersions of the Schcdulc f-a- Affective
a/cry large body of normal and clinical CBCL and Disorders and Schiznphrcnia. Arch. Gen. Psychiatry 3K.
YSR data already available to he used in the 9% I(i.7.
Guildford. J.P. I l%4) Psychometric Methods. New York. Mc-
study of adolescent depression. This study also
Graw Hill.
illustrates the usefulness of Receiver Operating licppcrlin. C.M., Stewart. G.W. and Rey, J.M. (IYYO) Extrac-
Characteristic (ROC) analysis in this type of rc- tion of depression scores in ndolesccnts from a general
search. purpose behaviour checklist. J. Affect. Dis. 18 105-I 12.
tlsiao, J.K.. Qrtko. J.J. ;:~.d Potter. W.Z. (‘;;83i i;iagnosing
Acknowledgments diagnoses:Receiver Operating Characteristic methods and
psychiatry. Arch. Gen. Psy<hi;,try 46, 664-667.
Joshi. P.T.. Capozolli, J.A. and Coyle. J.. . (ic390) The 2&ns
TM. Achenbach. V.R. Hensley, J.M. Plapp
Iiopkins Depression S&t!: normative: data and validation
and G.W. Stewart made comments to earlier in child psychiatry patients. J. Am. Acad. Child Adoksc.
drafts. Some of the dater included here were Psychiatry 29, 2XLXX.
presented at the 12th Congress of the Kashani. J.H.. Sherman. D.D., Parker, D.R. and Reid. J.C.
(IW!) Utility of the Beck Depression inventory with clinic
I.A.C.A.P.A.P., Kyoto (Japan), July lh-2CI, 1990.
rt:fcrred adolcsccnts. J. Am. Acad. Child Adolesc. Psychia-
This work was carried out with support from the
try 29. 27X-282.
National Health and Medical Research Council Kazdin, A.E. ( IWO) Childhabd depression. 3. Child PsychoI.
of Australia. Psychiatry 31. 121-160.
K(lvacs, M. ( 1981) Rating scales to assess depression in school
aged children. AcFa Paedopsychia;%r; 46, 305-315.
Mossman, D. and Scmoza, E. (1989) Mayimizing diagnostic
Achenhach, T.M.. Edelhrock and C.S. (19X3) Manual for thr information from the: dexamttthasone suppression tat.
Child Behavior Checklist and Revised Child Behavior Pro- Arch. Gen. Psychiatry. 36, 653-660.
file, University of Vermont, Department of Psychiatry. Nelson. W.M.. Politzno, P.M.. Finch. A.J.. Wendell. N. and
Burlington, VT. Mayh;dl, C. (1987) Children’s Depression Inventory: nor-
mative data and utility with emotionally disturbed chil- pression scales. J. Am. Acad. Child Adolesc. Psychiatry 30,
dren. J. Am. Acad. Child Adolesc. Psychiatry 26, 43-48. 5%hh.
Nurcombe, B., Seifer, R., Scioli. A., Tramontana, M.G.. Rotundo, N. and Hensley, V.R. (19851 The Children’s De-
Crapentine. W.G. and Beauchesne. 14-C. (1989) Is major pression Scale. A study of its validity. J. Child Psychol.
depressive disorder in adolescence : distinct diagnostic Psychiatry 26. 917-927.
entity?. J. Am. Acad. Child Adolesc. Psychiatry 28, 333- Shaffer, D.. Schwab-Stene, M., Fisher, P., Davies, M., Piacen-
342. tini, J. and Gioia, P. (198,) A revelsed version of the
Rey, J.M., Plapp, J-M. and Stewart, G.W. (1989) Reliability of Diagnostic Interview Schedule for children (DISC-R)
psychiatric diagnosis in referred adolescents. J. Child Psy- Washington, DC: National Institute of Mental Health.
chol. Psychiatry 30, 879-888. Swets, J.E. (19%) Measucin~ the accuracy of diagnostic sys-
Rey, J.M. and Morris-Yates, A. (1991) Ad~llescent depression tems. Science 240, 1285-1293.
and the Child Behavior Chechlist (CBCL). J. Am. Acad. Swets. J.E. and Picket& R.M. 11982) Evaluation of diagnostic
Child Adolesc. Psychiatry 30, -123-427. systems: Methods from signal detection theory, New York,
Roberts, R-E.. Lewinsohn. P.M. and ipeltey, J.R. (1991) Academic Press,
Screening for adolescent depression: .r comparison of de-