Journal of Clinical Child & Adolescent Psychology, 42(6), 749761, 2013

Copyright # Taylor & Francis Group, LLC

ISSN: 1537-4416 print=1537-4424 online
DOI: 10.1080/15374416.2013.773516

ADHD

Clinical Usefulness of Observational Assessment

in the Diagnosis of DBD and ADHD in Preschoolers
Tessa L. Bunte and Sarah Laschen
Department of Psychiatry and Rudolf Magnus
Institute of Neuroscience, University Medical Center Utrecht

Kim Schoemaker
Department of Psychiatry and Rudolf Magnus
Institute of Neuroscience, University Medical Center Utrecht and
Department of Child and Adolescent Studies, Utrecht University

David J. Hessen and Peter G. M. van der Heijden

Department of Methodology and Statistics, Utrecht University

Walter Matthys
Department of Psychiatry and Rudolf Magnus
Institute of Neuroscience, University Medical Center Utrecht and
Department of Child and Adolescent Studies, Utrecht University

The aim of the present study was to investigate the clinical usefulness of an observa-
tional toolthe Disruptive Behavior Diagnostic Observation Schedule (DB-DOS)in
the diagnosis of disruptive behavior disorders (DBD) and attention decit=hyperactivity
disorder (ADHD) in preschoolers. We hypothesized that the DB-DOS may help
support the presumption of a diagnosis generated by the information from parents
and teachers (or other caregivers). Participants were referred preschool children with
externalizing behavioral problems (N 193; 83% male) and typically developing chil-
dren (N 58; 71% male). In view of the clinical validity study each child was given a
diagnosis of either DBD (N 40), or ADHD (N 54) or comorbid (DBD ADHD;
N 66) based on best-estimate diagnosis. The DB-DOS demonstrated good interrater
and testretest reliability for DBD and ADHD symptom scores. Conrmatory factor
analysis demonstrated an excellent t of the DB-DOS multidomain model of DBD
symptom scores and a satisfactory t of ADHD symptom scores. The DB-DOS demon-
strated good convergent validity, moderate divergent validity, and good clinical validity
on a diagnostic group level for DBD and ADHD symptom scores. The Receiver
Operating Characteristic curve analyses revealed that for DBD the sensitivity and speci-
city are moderate and for ADHD good to excellent. The presumption of a diagnosis
based on information from parents, teachers, and cognitive assessment was supported

We are grateful to the parents and children who participated in this study. We especially thank Lauren Wakschlag, at the Department of Medical
Social Sciences, Feinberg School of Medicine, Northwestern University, Chicago, Illinois, for her cooperation in providing assessment procedures used
in this study and in training the Utrecht research group in administering and scoring the DB-DOS. We also thank Eva van der Kleij, Justa Kamstra, and
Susanne van Reedt Dortland, at the Department of Psychiatry, University Medical Center Utrecht, for their assistance with the data collection.
Correspondence should be addressed to Tessa L. Bunte, Department of Psychiatry, University Medical Center Utrecht, P.O. Box 85500, 3508
GA, The Netherlands. E-mail: t.bunte-rosingh@umcutrecht.nl
750 BUNTE ET AL.

by the DB-DOS in 60% for DBD and 75% for ADHD. The DB-DOS can be used to
help support a presumption of a DBD and=or ADHD diagnosis in preschool children.

There is an increasing number of young children referred viors on a daily basis parent, observations may be
to mental health clinics (Howell & Teich, 2008). Among biased due to a number of factors such as personality
these children, attention decit and disruptive behavior characteristics (Collishaw, Goodman, Ford, Rabe-
problems are the most common reasons for referral Hesketh, & Pickels, 2009). Therefore, direct observation
(Wilens et al., 2002). Epidemiological studies have demon- that is not ltered through the perceptions of the parent
strated that the prevalence rates of oppositional deant may provide a different window on the childs function-
disorder (ODD), conduct disorder (CD), and attention ing (Le Couteur & Gardner, 2008). Our clinical experi-
decit=hyperactivity disorder (ADHD; American Psychi- ence suggests that many parents desire the clinician to
atric Association, 2000) in preschool children are similar observe the child in order to arrive at a diagnosis, as par-
to those later in childhood (Egger & Angold, 2006). ents who consult a clinician would not accept that the
Furthermore, there is evidence of the predictive validity decision on the presence or absence of a disorder would
of ODD and CD in clinically referred preschool children be based only on parent and teacher reports; parents
(Keenan et al., 2011). Likewise, the validity of the Diag- want clinicians to look at their child themselves.
nostic and Statistical Manual of Mental Disorders (4th For clinicians who need to make a decision whether a
ed. [DSMIVTR]; American Psychiatric Association, child is diagnosed with a disorder, a crucial issue in the
2000) diagnosis of ADHD in younger children has been use of direct observation, however, is how to weigh
supported by a study demonstrating that the symptoms information from observation against information from
and associated impairment are likely to persist well into parents and teachers. Clearly, a limitation of obser-
elementary school (Lahey et al., 2004). vation is its brevity and contextual dependency; in other
Yet clinicians have long been reluctant to diagnose words, the behaviors observed need not be representa-
young children with these disorders. The question has tive of all problem behaviors (Le Couteur & Gardner,
indeed been raised as to how one differentiates clinical 2008). Even though clinicians do not expect to observe
from normative transient disruptive behaviors within the each symptom, they may want to see at least the tip
preschool age, as most preschoolers exhibit at least some of the iceberg. Thus, although the presumption of the
of the behaviors that fall under the rubric of disruptive presence of ODD or ADHD may be generated based
behavior. This makes clinical assessment of preschool on information from parents and teachers, direct obser-
behavioral problems particularly subtle and complex vation may be used to support this presumption or not.
(Wakschlag et al., 2005). In connection with this, a point Various observational tools have been developed
of concern is the possibility of overidentication of pre- such as The Dyadic ParentChild Interaction Coding
school children as having ODD, CD, or ADHD, as the System (DPICS; Robinson & Eyberg, 1981). This is a
manifestations of most of the symptoms of these disorders short (15 min) observational assessment of parentchild
are not atypical per se when observed in a preschooler, for interaction that is sensitive to treatment change and
example, losing his temper and physical aggression therefore has often been used to evaluate behavioral
(Keenan et al., 2007). Clearly, diagnosing a young child parent training for young children with conduct prob-
with, for example, ODD comorbid with ADHD, and start- lems (e.g., Posthumus et al., 2012). The DPICS is an
ing treatment (e.g., pharmacotherapy with methylpheni- observational assessment focusing not only on child
date) in the absence of these disorders is inappropriate. behavior but also on parent behavior; the DPICS is
Diagnosing young children with ODD, CD, and parent based, not examiner based. Examiner-based
ADHD is a challenge. Recently, however, assessment assessments are designed to be clinically sensitive by
procedures have been developed, including (semi)struc- standardizing adult responses in a manner that presses
tured interviews with the parents such as the Kiddie for a range of clinically salient behaviors in the child;
Disruptive Behavior Disorder Schedule (K-DBDS; however, they lack the ecological validity of parent
Keenan et al., 2007) and the Preschool Age Psychiatric child assessments. Therefore, combining examiner- and
Assessment (Egger et al., 2006). Although these newly parent-based behavioral observation paradigms pro-
developed parent-report diagnostic interviews have vides complementary methods for incorporating the
advanced identication of preschool psychopathology, interactive nature of social behavior into the assessment
the limitations of sole reliance on parents as informants of clinical signicance (Wakschlag, Hill, et al., 2008).
are amplied during the preschool years because young Wakschlag and colleagues (Wakschlag, Briggs-Gowan,
children cannot serve as informants about their own et al., 2008; Wakschlag, Hill, et al., 2008) developed
behavior (Wakschlag, Hill, et al., 2008). Furthermore, the more extended (60 min) Disruptive Behavior
although parent reports are based on childrens beha- Diagnostic Observation Schedule (DB-DOS), that is, a
 PRESCHOOLERS WITH DBD AND ADHD 751

standardized observational assessment of the preschool accounted for 6%, secondary for 25%, intermediate
child for diagnosing ODD and CD (also called disruptive vocational for 34%, higher vocational for 16%, and univer-
behavior disorder [DBD]). The DB-DOS is specically sity for 19%. Children with behavior problems (N 193)
designed to examine multiple, qualitative, and quanti- were referred by general practitioners, pediatricians, and
tative aspects of preschool childrens behavior and well-baby clinics for clinical assessment at the Outpatient
socioemotional functioning hypothesized to distinguish Clinic for Preschool Children with Behavioral Problems,
normative and disruptive behavior. Child behavior is Department of Psychiatry, Utrecht, the Netherlands.
assessed in three interactional contexts: one parent Inclusion criteria for the referred patient group were a
context and two examiner contexts (Wakschlag, score above the 90th percentile either on the Aggressive
Briggs-Gowan, et al., 2008; Wakschlag, Hill, et al., 2008). Behavior scale or on the Attention Problems scale of
The DB-DOS has been shown to be a reliable and the Child Behavior Checklist completed by parents
valid tool to distinguish normative maladaptive beha- (CBCL=1.55) and the Child Teacher Report Form com-
viors from DBD symptoms (Wakschlag, Briggs-Gowan, pleted by teachers or daycare caregivers (C-TRF=1.55;
et al., 2008; Wakschlag, Hill, et al., 2008). However, Achenbach & Rescorla, 2000). The TD group (N 58)
for clinical use, that is, to identify individual clinical was recruited from regular elementary schools and daycare
cases, it requires further research. Indeed, Wakschlag, centers. Children with a score in the clinical range either on
Briggs-Gowan, et al. (2008) investigated the clinical the Attention Problems scale or on the Aggressive Beha-
validity of the DB-DOS on a group level and not on vior scale of the CBCL=1.5-5 or C-TRF=1.5-5 were
an individual level, which is required to identify individ- excluded. All the children (referred and TD) with an IQ
ual clinical cases. Furthermore, in research on the below 70, estimated with the average score on the Raven
DB-DOS no attention has been paid to ADHD symp- Coloured Progressive Matrices (Raven, Court, & Raven,
toms, whereas the correlation between DBD symptoms 1998) and the Peabody Picture Vocabulary TestIIINL
and hyperactivity-impulsivity in preschool children is (Dunn & Dunn, 2005; Schlichting, 2005), were excluded.
0.79 (Sterba, Egger, & Angold, 2007). Therefore, while None of the participants were on medication.
assessing DBD symptoms in referred young children To study the reliability and convergent=divergent
with behavior problems, it seems evident to also include validity children referred with behavioral problems (N
the assessment of ADHD symptoms in the DB-DOS. 193) and TD children (N 58) were included. To examine
The main aim of the present study was to examine the testretest reliability, 24 children of the original group
clinical usefulness of the DB-DOS in the diagnosis of were reassessed after 8 weeks (M 58 days, SD 7.6,
DBD and ADHD in preschool children. Our main range 4072 days), where 20 of these preschoolers were
hypothesis was that the DB-DOS may help support the referred for behavior problems and four were TD children.
presumption of a diagnosis generated by the information For the purpose of the clinical validity study, each child
from parents and teachers (or other caregivers). First, we was given a diagnosis of DBD, ADHD, or comorbid DBD
examined the reliability and validity of DBD and ADHD and ADHD. As there is no gold standard by which to diag-
symptom scores in a European sample of referred pre- nose a psychiatric disorder, the procedure of best-estimate
school children with behavioral problems; a typically diagnosis was used, that is, an independent child psy-
developing (TD) control group was also included in the chiatrist and clinical child psychologist consider the infor-
study. Second, we examined clinical validity of DB-DOS mation about the child, discuss all the ndings, and reach
symptom scores on a group level by comparing the TD consensus (Lord et al., 2006). Before the consensus meet-
group with DBD and ADHD groups. Third, we exam- ing, each clinician had made a decision on the diagnosis of
ined clinical validity on an individual level by determin- the particular child (DBD, ADHD, both or no diagnosis=
ing cutoff points of DBD and ADHD symptom scores, another diagnosis). Only when there was no agreement on
which divide clinical from TD cases. Finally, we exam- the diagnosis was a discussion needed. In 75% of the chil-
ined how often a diagnosis of DBD and=or ADHD dren there was agreement on the diagnosis, in 15% there
based on information from parents, teachers, and cogni- was agreement on one of the two diagnoses, and in 10%
tive assessment was conrmed by the DB-DOS. there was agreement on neither of both diagnoses.
Both clinicians were unfamiliar with the child. The
information about the child consisted of the following:
METHOD
1. An interview with the parents about the reasons
Participants
for referral and the development of the child.
Participants were 251 children aged 3.5 to 5.5 years 2. The symptoms coded on the basis of the K-DBDS
(M 55 months, SD 7.8); 80% were male, 86% White, (Keenan et al., 2007), a semistructured DSMIV
12% Turkish=Moroccans, 2% African American, and TR-based parent interview for the assessment of
0.5% Asian. In terms of parent education, primary ODD, CD, and ADHD in preschool children.
752 BUNTE ET AL.

3. The scores of the Attention Problem scale and the Procedure

Aggressive Behavior Scale of the CBCL=1.5-5 and
Children were assessed during one morning session.
the C-TRF=1.5-5.
First, general intellectual functioning was assessed fol-
4. The scores on the Child Global Assessment
lowed by the executive functions tasks. After a break
Schedule (CGAS; Schaffer et al., 1983), that is, a
the DB-DOS and K-DBDS were administered. The
measure of the impairment of the functioning of
tasks were administered in a quiet and nondistractible
the childs behavior, lled out by the parents as
room with a one-way mirror. Parents received a small
well as the teacher=caregiver.
nancial compensation for participating, and children
5. The scores on the Raven Coloured Progressive
received two small gifts. Written informed consent from
Matrices (Raven et al., 1998) and the Peabody
the parents was gathered before participating. The
Picture Vocabulary Test IIINL (Dunn & Dunn
Medical Ethical Review Committee of the University
2005; Schlichting, 2005).
Medical Center Utrecht, approved this study.
6. The results of the executive function tasks: three
inhibition tasks (GoNoGo task, Snack Delay,
and Shape School) and two working memory Measures
tasks (Delayed Alternation and Nine boxes; DB-DOS. The DB-DOS is a method for observa-
Schoemaker et al., 2012). tional assessment of disruptive behavior in pre-school
7. The videotaped observations of the childs beha- children (Wakschlag, Briggs-Gowan, et al., 2008;
vior during the psychological tasks. Wakschlag, Hill, et al., 2008). It is divided into three
interactional contexts: two examiner contexts and one
To reach a clinical consensus diagnosis the DSMIV parent context. In the rst examiner context, the exam-
TR criteria of one of the DBDs (ODD, CD, or DBD- iner is ostensibly responsive to child behavior, the
NOS, i.e., dened as either three ODD symptoms and so-called Examiner Engaged context. Then, within the
at least one CD symptom or at least one ODD symptom context of minimal support, the child is observed while
and two CD symptoms) and ADHD were strictly working independently, with the examiner being busy
applied (American Psychiatric Association, 2000). On doing his or her own work (Examiner Busy context).
the basis of the consensus best-estimate diagnosis, chil- In addition, the examiner also briey leaves the room
dren were positioned in the DBD, ADHD, or comorbid to probe potential covert rule-breaking behaviors. Pro-
(DBD and ADHD) group. In this way the patients were cedures are explained to the parent before starting the
divided into four groups: TD (N 58), DBD (N 40), tasks, and parents are provided with simply worded
ADHD (N 54) and comorbid (N 66). The characteris- instructions on ip cards. The primary caregiver was
tics of the four groups are displayed in Table 1. In the involved in the Parent context. The different tasks, last-
clinical validity study there were group differences in esti- ing approximately 5 min, are parallel across the mod-
mated IQ (p < .001), with the TD group signicantly out- ules, including frustration, internalization of rules,
performing the three clinical groups, who performed compliance, and social play tasks. The DB-DOS coding
similarly to each other. The four groups did not differ system for behavior problems consists of problems in
in age or gender. the domains of Behavior Regulation (15 items), and

TABLE 1
Means (and Standard Deviation) for the Demographics and Control Variables in the Four Groups

DBDa ADHDb DBD ADHDc

M (SD) M (SD) M (SD) M (SD)

Age (Months) 56.0 (7.3) 53.1 (8.4) 54.7 (8.8) 53.4 (7.1)
% Boys 71 85 76 86
IQa 112.4 (10.4) 101.8 (9.8)


101.1 (11.3)


101.2 (12.7)



CBCL Aggression 50.5 (1.3) 75.0 (10.5)


63.3 (11.0)


75.6 (11.1)



CBCL Attention 50.8 (2.1) 63.4 (8.1)


68.6 (6.7)


69.7 (6.4)



TRF Aggression 52.0 (3.1) 61.5 (9.7)


62.4 (9.3)


67.0 (11.1)



TRF Attention 52.0 (3.8) 58.5 (6.6)


70.8 (10.9)


69.4 (11.7)

Note. IQa is an estimate based on mean Raven and Peabody. DBD disruptive behavior disorder; ADHD attention
decit=hyperactivity disorder; CBCL Child Behavior Checklist; TRF Teacher Report Form.
a
N 40.
b
N 54.
c
N 66.




p < .001 (compared with the typically developing group).
 PRESCHOOLERS WITH DBD AND ADHD 753

Anger Modulation (six items). Although there is a CGAS. To provide a global assessment of function-
Competence scale, in this study we concentrate on the ing, parents and teachers=caregivers completed the
behavior problems domains. nonclinician version of the CGAS (Schaffer et al.,
Furthermore, extending the DB-DOS, we developed 1983). The CGAS is scored from 0 to 100, with lower
10 ADHD items organized in terms of problem behaviors scores indicating greater impairment.
related to Inattention (four items), Hyperactivity (four
items), and Impulsivity (two items). The ADHD domain Impact on family scale. The Impact on Family
consists of these 10 items. The items were developed on Scale (Sheeber & Johnson, 1992) completed by the
those DSMIVTR criteria for ADHD, which could be mother assesses the social, nancial, and personal bur-
observed as behaviors and scored during the obser- den resulting from child behavior problems. In the
vation (see the appendix). present study Cronbach alpha was 0.95.
DB-DOS codes are ratings of child behavior ranging
from 0 to 3 and comprising two categories: typical (code
Social Skills Rating Scale. Parents reports on the
0 normative behavior and 1 normative misbehavior)
summary score of the Social Skills Rating Scale (SSRS;
and clinically concerning behavior (code 2 of concern
Gresham & Elliot, 1990) were used to evaluate childrens
and 3 atypical). Each item is rated separately for each
social skills. This scale was completed during the parent
context. DB-DOS uses the quantitative and the qualitat-
module of the DB-DOS. In the present study Cronbach
ive dimensions of disruptive behavior for the ratings, as
alpha was 0.91.
in preschool children it is not only the frequency but
also the intensity of such behaviors that distinguishes
clinical problems from normative behavior. Early Child Inventory. Parents and teachers=
caregivers reported on child behavior symptoms with
the Early Child Inventory (ECI; Gadow & Sprafkin,
DB-DOS administration and coding. The study 1996), a DSM-based checklist. The Pervasive Develop-
group of Wakschlag trained our study group for admin- mental scale of the ECI was used in view of examining
istration and coding of the DB-DOS. Training for admin- divergent validity of DBD and ADHD scores. In the
istration included a review of the DB-DOS manual, live present study Cronbach alpha was 0.77 (parent) and
and videotaped observations, and practice administra- 0.82 (teacher).
tions. For coding, reliability was established via 80%
exact item-level agreement with the coders of the study
group of Wakschlag. For the present study two criterion Statistical Analyses
coders trained an independent team of research students For all scales used the percentage of missing data was at
for reliable coding. Coders were independent and blind to most 4.1%, with the exception of the ECI teacher ques-
the childs clinical status. Following Wakschlag, approxi- tionnaire (4.8%) and the CGAS teacher questionnaire
mately 15% of the videotaped sessions were randomly (6.8%). For the reliability analyses the internal consist-
selected for double coding to monitor ongoing interrater ency was examined with Cronbach alpha (Cronbach,
reliability (Wakschlag, Hill, et al., 2008). 1951). Interrater agreement and testretest reliability at
the domain level and by context were assessed using intra-
K-DBDS. Parents were administrated the K-DBDS class correlation coefcients (ICCs). We also examined
(Keenan et al., 2007), a semistructured clinical interview mean level differences in domain scores and by context
assessing DSMIVTR (American Psychiatric Associ- at test and retest. Conrmatory factor analyses were used
ation, 2000) DBD and ADHD symptoms in preschool to test the t of the multidomain model of disruptive
children. For the scoring we used the developmentally behavior as proposed by Wakschlag et al. (Wakschlag,
modied approach; it is designed to adhere as closely as Hill, et al., 2008), and an extension of this model includ-
possible to DSMIVTR. In the present study we found ing the newly proposed ADHD domain. Convergent and
the following Cronbach alphas (Cronbach, 1951): divergent validity were examined by computing the corre-
ODD 0.75, DBD 0.79, CD 0.64, ADHD 0.83. lations of DB-DOS domain scores and multimethod
assessment of child functioning as evaluated by parent
(K-DBDS, CGAS, CBCL=1.5-5, SSRS, ECI) and
CBCL=C-TRF. Disruptive behavior symptoms teacher=caregivers reports (CGAS, C-TRF=1.5-5, ECI).
and attention problems were assessed using the scores To assess clinical validity, examined on a diagnostic
of the Attention Problem scale and the Aggressive group level, analyses of (co)variance were conducted to
Behavior Scale of the CBCL=1.5-5 (Achenbach & compare the DBD and ADHD groups (with and without
Rescorla, 2000) and the C-TRF=1.5-5 (Achenbach & comorbid children) and the comorbid group separately
Rescorla, 2000). with the TD group, while controlling for age and gender
754 BUNTE ET AL.

for the DBD total group. Group membership was based statistically (Lord et al., 1989). Five items (all in the
on consensus best-estimate diagnosis. The analyses were Behavioral Regulation domain) were rated as present
carried out with and without controlling for IQ, using fewer than 10 times in the clinical concerning range:
Bonferroni corrections. Statistical signicance was based verbal aggression (two items), directed aggression,
on two-sided tests. Patients groups were not compared spiteful behavior, and sneaky behavior. Because estima-
with each other, as this was not an aim of the study. tes of reliability can be unduly biased by differences
To use the DB-DOS as a clinical assessment tool on an between a single pair of raters for items with a very low
individual level, an appropriate cutoff point of the occurrence, these items were excluded from domain
DB-DOS score is required to divide clinical from nonclini- level analyses.
cal cases for DBD and=or ADHD. For this, Receiver
Operating Characteristic (ROC) curve analysis was used. Internal Consistency
The optimal cutoffs were determined by nding the values Overall the three domains (Behavioral Regulation,
that led to a balance between sensitivity (percentage true Anger Modulation, and ADHD) exhibited good inter-
positives) and specicity (percentage true negatives). All nal consistency (see Table 2), both in terms of total
cutoffs, of course, result in some degree of misclassica- domain scores (Cronbach a M .82, range .77.91)
tion. For clinical use a high sensitivity and an acceptable and by context (Cronbach a M .82, range .69.94).
specicity (>70%) is required (Kim & Lord, 2012). On
the other hand, diagnosing a child with a psychiatric
disorder while there is no diagnosis clearly is inappropri- Interrater Reliability
ate; from this perspective a high specicity is essential. It showed excellent interrater reliability as Table 2
We note that in clinical practice diagnosis will not be based illustrates, both in terms of total domain scores
exclusively on the basis of the direct observation of the (ICC M .92, range .88.95) and by context (ICC
childs symptoms in the outpatient clinic. Information on M .92, range .86.97).
the childs everyday behavior from parents and teachers=
caregivers should also be taken into account. TestRetest Reliability of Domain Scores
To examine how the identication of the presence
Testretest analyses indicated moderate reliability
or absence of DBD and=or ADHD, according to the
across domains and contexts both (see Table 2) in terms
DB-DOS, may be used in supporting or rejecting a
of total domain scores (ICC M .64, range .59.71)
possible diagnosis of DBD and=or ADHD, based on
and by context (ICC M .64, range .52.80). Across
the information from parents, teachers (or other care-
the contexts, domain scores tended to decrease slightly
givers), and cognitive assessment, using best-estimate
in the second testing.
diagnosis (see participants), we computed the percentage
agreement between the best-estimate diagnosis DBD
and=or ADHD (or no diagnosis) and the DB-DOS Domain Intercorrelations Across Contexts
diagnosis DBD and=or ADHD (or no diagnosis). For Pearson correlations among the DB-DOS domains
these analyses we used data from the total sample, across context indicated modest to substantial consist-
including children who are not diagnosed, and checked ency in child behavior across the different DB-DOS con-
if a child diagnosed only with ADHD (DBD), based texts. Cross-context associations within a domain were
on best-estimate diagnosis, will get an ADHD (DBD) signicantly higher across the two examiner contexts
diagnosis based on the DB-DOS but will not receive a (Pearson correlation ranged.44.58) relative to associa-
DBD (ADHD) diagnosis on the DB-DOS. We assumed tions between each of the examiner contexts and the
the TD children have no diagnosis based on their score Parent context (Pearson correlation ranged.18.40).
beneath the 90th percentile on the Attention Problems
scale and on the Aggressive Behavior scale of the
Confirmatory Factor Analyses
CBCL=1.5-5 and C-TRF=1.5-5. We used the BR=DBD
total score cutoff points for the DBD diagnosis and the Conrmatory factor analyses were conducted to test
ADHD total score cutoff point for the ADHD diagnosis. the t of two models. Model 1 was the multidomain
model of disruptive behavior as proposed by Wakschlag,
Hill, et al. (2008). Model 2 was an extended model that
includes the ADHD domain. In both models, three of
RESULTS
the observed domain scores were regressed on the latent
Anger Modulation construct, three other observed domain
Domain Level Reliability
scores were regressed on the latent Behavioral Regulation
Diagnostic observation measures may include clinically construct, and the latent Anger Modulation and Beha-
salient items with an occurrence too low to be evaluated vioral Regulation constructs were regressed on the latent
 PRESCHOOLERS WITH DBD AND ADHD 755

TABLE 2
Domain Score Reliability

Domain Cronbachs aa Interrater ICCb TestRetest ICCc Test M (SD) Retest M (SD)

Behavioral Regulation M .79 M .95 M .63

Examiner Engaged .77 .95 .52 3.8 (2.6) 3.0 (2.7)
Examiner Busy .75 .96 .62 5.6 (4.5) 3.1 (3.4)
Parent .84 .92 .74 11.1 (5.6) 6.5 (4.1)
Anger Modulation M .91 M .88 M .59
Examiner Engaged .86 .86 .53 1.6 (1.8) 2.2 (2.3)
Examiner Busy .92 .91 .68 3.8 (3.3) 2.2 (2.3)
Parent .94 .88 .57 7.7 (5.2) 3.5 (4.0)
ADHD M .77 M .92 M 0.71
Examiner Engaged .82 .94 .68 11.0 (4.0) 11.5 (4.2)
Examiner Busy .81 .94 .80 12.3 (4.8) 10.3 (4.0)
Parent .69 .87 .64 14.8 (6.0) 12.7 (5.1)

Note. ICC intraclass correlation coefcient; ADHD attention decit=hyperactivity disorder.

a
N 251.
b
Examiner Engaged (N 39); Examiner Busy (N 47); and Parent Context (N 28).
c
N 24.

Disruptive Behavior construct. Following Wakschlag, allowed to correlate with the errors of other observed
Hill, et al. (2008), the factor loadings of Anger Modulation domain scores from the same context. The latent ADHD
and Behavioral Regulation onto Disruptive Behavior were construct was allowed to correlate with the latent
constrained to be equal. The errors of observed domain Disruptive Behavior construct. The parameter estima-
scores from the same context were allowed to correlate. tion results of the two models are shown in Figure 1.
Model 2 extends Model 1; here, three new observed domain As in the study of Wakschlag, Hill, et al. (2008), the t
scores were regressed on the latent Inattention construct, of Model 1 was excellent, v2(5) 8.1, p .151, compara-
three other new observed domain scores were regressed tive t index .992, root mean square error of
on the latent Hyperactivity=Impulsivity construct, and approximation .051. The t of Model 2 could be
the latent Inattention and Hyperactivity=Impulsivity considered satisfactory, v2(33) 74.41, p .000, com-
constructs were regressed on the latent ADHD con- parative t index .946, root mean square error of
struct. The errors of these domain scores were again approximation .073.

FIGURE 1 Conrmatory factor analysis. Note. DBD disruptive behavior disorder; ADHD attention decit=hyperactivity disorder; AM
Anger Modulation; BR Behavior Regulation; IN Inattention; HI Hyperactivity=Impulsivity; EA Examiner Engaged Context;
EB Examiner Busy context; PC Parent context.
756 BUNTE ET AL.

Validity differed from the TD group on all DB-DOS scores, with

the exception of the DBD group with respect to the
Convergent validity. As Table 3 illustrates, signi-
Behavior Regulation Examiner Engaged context and
cant correlations were established between DB-DOS
the ADHD group with respect to the Anger Modulation
domain scores and parent and teacher=caregiver
Examiner Engaged=Examiner Busy context.
questionnaires of symptoms and impairment (K-DBDS,
To examine clinical validity on an individual level, an
CGAS, CBCL=1.5-5, C-TRF=1.5-5, Impact on Family
appropriate cutoff point of the DB-DOS score is
Scale). Correlations between the Behavioral Regulation
required. We used the total DBD group (DBD and
domain and the parent and teacher=caregiver question-
DBD ADHD both consensus best-estimate diagnosis)
naires, with the exception of the CBCL=1.5-5 Aggressive
and the total ADHD group (ADHD and DBD ADHD
Behavior scale and Behavioral Regulation domain in
consensus best-estimate diagnosis), with the TD group for
the Examiner Engaged context, were signicant and
the ROC analysis. Overall, the estimated area under
as expected. Correlations between the Anger Modu-
curve (AUC) values and their 95% condence intervals
lation domain and the questionnaires were slightly
(CI) were satisfactory. First we determined a cutoff
lower but signicant as well. Correlations between
point of the total (Examiner Engaged, Examiner Busy,
the ADHD scales and the questionnaires were signi-
and the Parent context together) DBD score (Behavior
cant, and the absolute values were higher than the
Regulation and Anger Modulation). The ROC analysis
correlations between the Behavioral Regulation=Anger
yielded an estimated AUC of .81, 95% CI [.74, .88].
Modulation domain and the parent and teacher=
Using a total DBD score of 24 yielded a sensitivity of
caregiver questionnaires.
77% and a specicity of 69% (see Table 5). On the basis
of this DBD cutoff score, sensitivity of 77% means that
Divergent validity. Small but signicant minor cor- three out of four children, who according to clinical con-
relations were found between the majority of DB-DOS sensus were diagnosed with DBD, will receive a DBD
scores and parent and teacher=caregiver questionnaires diagnosis. Specicity of 69% means that 69% of the
(SSRS, ECI-PDD; Table 3). TD children will not be diagnosed with DBD, according
to consensus best-estimate diagnosis, and 31% will be
diagnosed with DBD while the diagnosis is not given.
Clinical validity. To investigate clinical validity on a Second, as this low-sensitivity=specicity was unsatisfac-
group level, analyses of (co)variance were conducted to tory, we explored whether a better cutoff point of the
examine group differences in DB-DOS scores between total (Examiner Engaged, Examiner Busy, and the Par-
the TD group and the clinical groups without control- ent context together) Behavior Regulation score could
ling for IQ (see Table 4). The results were similar for be determined (i.e., part of the total DBD score). The
the analyses when controlling for IQ. All clinical groups ROC analysis yielded an AUC of .85, 95% CI [.78, .91].

TABLE 3
Association (Pearson r Correlation) of the Disruptive Behavior Diagnostic Observation Schedule (DB-DOS);
Scores by Context and Multi-Informant Assessments of Child Disruptive Behavior and Functioning

Behavioral Regulation Anger Modulation ADHD

EE EB PC Total EE EB PC Total EE EB PC Total

Parent Reported
K-DBDS .19

. 20

.27

.28

.24

.15
.17

.24

.30

.32

.30

.39


CGAS .28

.30

.35

.41

.21

.14
.21

.25

.30

.35

.42

.46


CBCL .09 .17

.23

.22

.14
.14
.15
.18

.29

.34

.41

.44


IFS (M) .22

.26

.25

.32

.19

.14
.14
.20

.28

.33

.34

.31


SSRS .11 .17

.37

.30

.07 .11 .24

.21

.12 .16
.32

.27


ECI (PDD) .10 .09 .12 .13
.18

.15
.06 .15
.13 .08 .10 .11

Teacher=Caregiver Reported
CGAS .40

.38

.25

.45

.23

.24

.17

.29

.34

.34

.28

.41


TRF .34

.41

.26

.44

.25

.29

.24

.36

.30

.37

.32

.42


ECI-PDD .26

.17

.22

.28

.18

.17

.24

.29

.19

.18

.16

.23

Note. ADHD attention decit=hyperactivity disorder; EE Examiner Engaged context; EB Examiner Busy context. PC Parent context;
K-DBDS Kiddie Disruptive Behavior Disorder Schedule; CGAS Clinical Global Assessment Scale. CBCL Child Behavior Checklist;
IFS Impact on Family Scale; SSRS Social Skills Rating Scale; IFS (M) Impact on. Family Scale (Mother); ECI (PDD) Early Child Inventory
(Pervasive Developmental Disorder).


p < .05.

p < .01.
 PRESCHOOLERS WITH DBD AND ADHD 757

TABLE 4
Means (and Standard Deviations) of the DB-DOS Domains in the Six Groups

DBD only and ADHD only and

TDa DBDb ADHDc DBD ADHDd DBD ADHDe DBD ADHDf
M (SD) M (SD) M (SD) M (SD) M (SD) M (SD)

BR EE 2.5 (2.1) 3.4 (2.4) 4.7 (2.9) 5.4 (3.4) 4.7 (3.2) 5.1 (3.2) (bcde)



BR EB 2.4 (2.5) 5.3 (3.8) 6.4 (4.5) 6.9 (5.2) 6.3 (4.8) 6.7 (4.9) (abcde)



BR PC 5.4 (3.9) 9.9 (4.2) 9.2 (4.1) 10.8 (4.5) 10.5 (4.4) 10.1 (4.4) (abcde)



BR tot 10.4 (6.0) 18.7 (8.1) 20.3 (8.7) 23.1 (9.8) 21.4 (9.4) 21.8 (9.4) (abcde)



AM EE 1.9 (1.7) 3.1 (3.0) 2.2 (1.9) 3.1 (3.1) 3.1 (3.0) 2.7 (2.7) (ae)
(cd)


AM EB 2.7 (3.4) 4.2 (4.2) 3.6 (2.9) 4.4 (4.3) 4.3 (4.2) 4.0 (3.7) (acde)

AM PC 4.4 (4.4) 6.9 (5.0) 6.3 (4.7) 7.4 (4.8) 7.2 (4.9) 7.0 (4.8) (ab)
(de)

(c)



AM tot 9.0 (6.1) 14.1 (9.6) 11.9 (5.7) 15.0 (9.3) 14.6 (9.4) 13.8 (8.0) (b)
(a)

(cde)



ADHD EE 8.8 (3.1) 10.9 (4.7) 14.1 (4.2) 13.7 (5.0) 12.6 (5.0) 13.9 (4.6) (a)
(bcde)



ADHD EB 9.4 (3.8) 11.4 (3.8) 15.4 (4.1) 14.5 (5.0) 13.3 (4.8) 14.9 (4.6) (a)
(bcde)



ADHD PC 9.4 (4.2) 13.7 (4.3) 15.7 (4.5) 16.9 (5.6) 15.7 (5.4) 16.4 (5.2) (abcde)



ADHD tot 27.6 (7.2) 35.7 (8.8) 45.3 (9.4) 42.2 (11.1) 41.6 (11.2) 45.3 (10.4) (abcde)

Note. TD typically developing; DBD disruptive behavior disorder; ADHD attention decit=hyperactivity disorder; DBD total DBD
only and DBD ADHD; ADHD Total ADHD only and ADHD DBD; BR Behavior Regulation; AM Anger Modulation;
EE Examiner Engaged context; EB Examiner Busy context; PC Parent context; tot total; a DBD vs. TD; b ADHD vs. TD; c DBD A
A DHD vs. TD; d DBD total vs. TD; e ADHD total vs. TD; e.g., (ae)
means p value for a and e lower than .05.
a
N 58.
b
N 40.
c
N 54.
d
N 66.
e
N 106.
f
N 120.


p < .05.

p < .01.


p < .001.

TABLE 5 Using a total Behavior Regulation score of 13 yielded a

Receiver Operating Characteristic Curve
sensitivity of 83% and a specicity of 72%. This cutoff
Positive Negative point for Behavior Regulation was slightly better than
DB-DOS Predictive Predictive the one for total DBD score. We have explored the role
DBD Totala Score Sensitivity Specicity Value Value of different contexts (Examiner Engaged, Examiner
Total DBD 20 85 59 67 80 Busy, and the Parent context) for the total DBD, the
Score 24 77 69 71 75 Behavior Regulation, and the Anger Modulation scores,
28 67 79 76 71 but the results of the sensitivity=specicity balance for
31 56 83 77 65 the different contexts were unsatisfactory. We explored
Total BR 12 87 60 69 82 also whether the total (Examiner Engaged, Examiner
Score 13 83 72 75 81 Busy, and the Parent context together) Anger Modu-
16 68 81 78 72
lation score would be better; a total Anger Modulation
18 63 86 94 70
score of 11 yielded a sensitivity of 62% and a specicity
Total BR 13 90 72 76 88
of 64%, using a total Anger Modulation score of 13
Score 3.54.5 14 87 76 78 85
Years 15 81 79 79 81 yielded a sensitivity of 55% and a specicity of 74%.
16 77 81 80 78 The ROC analysis yielded an AUC of 0.69, 95% CI
ADHD Totalb 32 91 69 76 88 [.60, .77]. These cutoff points for Anger Modulation
Total ADHD 33 89 72 76 87 were unsatisfactory. Fourth we examined whether the
Score 34 87 79 81 86 Behavior Regulation cutoff point for younger and older
36 83 83 83 83 children differed in specicity and sensitivity. Using a
38 79 95 94 82
total Behavioral Regulation score for the 3.5- to
Note. Bold numbers indicate maximized sensitivities and specici- 4.5-year-old children of 14 yielded a sensitivity of 87%
ties depending on criteria used in selecting cut-off scores. DBD and a specicity of 76% (see Table 5), whereas for 4.5
disruptive behavior disorder; DBD total DBD only and to 5.5 years of age, sensitivity was 67% and specicity
DBD ADHD; ADHD attention decit=hyperactivity disorder; was 76%. The ROC analysis yielded an AUC of .89,
ADHD Total ADHD only and ADHD DBD; BR Behavior
Regulation. 95% CI [.83, .95]. Fifth we examined whether the cutoff
a
N 106. point for referred patients with clinical scores on the
b
N 120. TRF 1.55 (T score above 70) differed in specicity
758 BUNTE ET AL.

and sensitivity for the different modules. The referred With respect to the clinical validity of the DB-DOS on
children with DBD with clinical scores on the TRF a group level, all clinical groups differed from the TD
1.55 (T score above 70) showed a better sensitivity group on DB-DOS DBD and ADHD symptom scores,
and specicity balance compared to the total DBD with the exception of the DBD group with respect to
group. Categorization of the total DBD group in sex, the Behavior Regulation Examiner Engaged context
IQ (IQ above or below 104), CGAS parents or and the ADHD group with respect to the Anger Modu-
teachers=caregivers (above or below 60), and CBCL lation Examiner Engaged=Examiner Busy context; the
(above or below 70) did not show better results com- exception of the DBD group can be explained by the
pared to the total DBD group. For the ADHD domain, highly structured context with an unknown examiner,
a cutoff point of 34 was suitable (see Table 5); using a the exception of the ADHD group is in line with expecta-
total (Examiner Engaged, Examiner Busy, and the Par- tions. Besides, children with ADHD showed signicantly
ent context together) ADHD score of 34 yielded a sensi- higher DB-DOS symptom scores in all the three Behavior
tivity of 87% and a specicity of 79% (see Table 5). The Regulation contexts and the Anger Modulation Parent
ROC analysis yielded an AUC of .92, 95% CI [.88, .96]. context compared with the TD group. This can be
Thus, on the basis of the ADHD score, sensitivity of explained by the high correlation of DBD and ADHD
87% means almost nine out of 10 patients who accord- symptoms in preschool children (Sterba et al., 2007).
ing to clinical consensus were diagnosed with ADHD The main aim of the present study was to examine
will receive an ADHD diagnosis. Specicity of clinical validity on an individual level using clinical
79% means that 79% of the TD children will not be cutoff points for DBD and ADHD symptom scores.
diagnosed with ADHD, according to consensus While exploring the clinical cutoff point for DBD,
best-estimate diagnosis, and 21% will be diagnosed with the Behavior Regulation total score appeared to have
ADHD while the diagnosis is not given. slightly better sensitivity=specicity than the total
Finally, the percentage agreement was computed DBD score. Thus, using the Behavioral Regulation
between the best-estimate diagnosis DBD and=or ADHD total score, four out of ve children, who according
(or no diagnosis) and the DB-DOS diagnosis DBD and= to clinical consensus were diagnosed with DBD, will
or ADHD (or no diagnosis). The percentage agreement, receive a DBD diagnosis. By contrast the Anger Modu-
in this study group (referred and typically developing lation total score was unsatisfactory. Furthermore,
children), between the best-estimate DBD diagnosis (or sensitivity of the clinical cutoff point for Behavior
not) and the DB-DOS DBD diagnosis (or not) based Regulation appeared to be somewhat better in the
on the cutoff point of the BR total score of 13 (see Table younger age group (3.54.5 years) than in the older
5) is 59% and the percentage agreement based on the cut- age group (4.55.5 years), maybe because older pre-
off point of the DBD total score of 24 (see Table 5) is schoolers can better inhibit their own behavior than
60%. For the best-estimate diagnosis ADHD (or not) younger preschoolers. The better results for referred
and the DB-DOS ADHD diagnosis (or not) based on patients with clinical scores on the TRF 1.55 (T score
the cutoff point of 34 (see Table 5), the percentage agree- above 70) are in line with evidence that youth with
ment is 75%. symptoms according to both parent and teacher had
more mental health concerns than youth with symp-
toms according to either parent or teacher (Gadow &
DISCUSSION Drabick, 2012).
The clinical cutoff point for ADHD symptom scores
The results of reliability (internal consistency, interrater appeared to be good. Indeed, almost nine out of 10
reliability, testretest reliability) and validity (conver- children, who according to clinical consensus were diag-
gent and divergent) of DBD symptom scores in the nosed with ADHD, will receive an ADHD diagnosis.
current study in a European sample are similar to the Based on ROC curve analyses the sensitivity and speci-
ndings of the studies by Wakschlag and colleagues city for the ADHD diagnosis when compared with
(Wakschlag, Briggs-Gowan, et al., 2008; Wakschlag, DBD diagnoses appeared to be better for ADHD than
Hill, et al., 2008) in a U.S. sample. Likewise, the results for DBD. This may be due to the cross-situational
of reliability and validity of the ADHD symptom scores character of ADHD symptoms (Gadow & Drabick,
are satisfactory. Thus, the current study not only repli- 2012). In contrast, ODD symptoms may be present in
cated but also extended the U.S. study by adding a just one context (at home or at school). Thus, it may
reliable and valid ADHD domain. This is important, be more difcult to elicit DBD symptoms than ADHD
as in the clinical assessment of preschool children with symptoms in an observational procedure at an out-
externalizing behavior problems, comorbidity with patient clinic. Another explanation could be that young
ADHD should be investigated. The results of conrma- children with ADHD are typically unable to control
tory factor analyses for ADHD domain are satisfactory. their impulsive behavior in a novel context, whereas a
 PRESCHOOLERS WITH DBD AND ADHD 759

child with ODD or CD may be less likely to display their to preschool disruptive behavior has been developed
symptoms in an unfamiliar context. (Wakschlag et al., 2012).
When investigating the clinical validity of the The study also has limitations. First, the testretest
DB-DOS on an individual level we should keep in reliability was only moderate for DBD, in line with
mind that in everyday clinical practice the decision of the Wakschlag, Hill, et al. (2008) study, and for ADHD.
diagnosing a child with ODD, CD, or ADHD is not In our study we organized the retest after 8 weeks. Fam-
based on the results of a single measure but on the iliarity with the tasks and parents promising presents for
combination of results from multiple measures such their childs good behavior may explain the lower score
as a standardized parent and teacher=caregiver rating on the retest. However, a longer period for the retest
scales and a (semi)structured DSM-oriented interview (e.g., 6 months) is not an appropriate alternative
with the parents (Matthys & Lochman, 2010). Our because preschool children show fast developmental
main hypothesis was that the DB-DOS may help sup- changes. Second, in the course of studying divergent
port the presumption of a diagnosis generated by the clinical validity, in line with the Wakschlag, Briggs-
information from parents, teachers (or other care- Gowan, et al. (2008) study, small but signicant minor
givers), and cognitive assessment. In this study group correlations were found between the majority of DB-
the DB-DOS BR=DBD total score supported approxi- DOS scores and parent reports of social skills on the
mately 60% the DBD diagnosis (or not), generated SSRS; one may question whether the SSRS is an appro-
by the information from parents, teachers (or other priate measure to examine divergent clinical validity, as
caregivers), and cognitive assessment using the social skills are less well developed in children with DBD
best-estimate diagnosis. This means that for approxi- and ADHD (Ronk, Hund, & Landau, 2011). Therefore
mately six out of 10 children diagnosed with DBD we added the ECI-PDD part in view of studying diver-
(or not), generated by the information from the par- gent validity. As expected, the correlation between the
ents, teachers (or other caregivers), and cognitive ECI-PDD part and the Behavioral Regulation=Anger
assessment, the DBD diagnosis has been conrmed Modulation=ADHD symptom scores was weak. Children
(or not) by the DB-DOS. The DB-DOS ADHD total with an IQ below 70 were excluded; results of the study
score supported nearly 75% of the ADHD diagnosis thus are not generalizable to children with IQs below 70.
(or not), generated by the information from parents, Third, the very small number of girls constrains the ability
teachers (or other caregivers), and cognitive assessment to discuss sex differences. Another limitation of this study is
using the best-estimate diagnosis, which means that for that in this data set no subscales of ADHD (i.e., Inatten-
nearly three of four children the diagnosis ADHD (or tion and Hyperactivity=Impulsivity) are used in the clinical
not), generated by the information from the parents validity study because of the low (N 6) ADHD inatten-
and teachers (or other caregivers), has been conrmed tion subtype diagnosis, based on consensus best-estimate
(or not) by the DB-DOS. To our knowledge this is the diagnosis.
rst study that gives a quantitative indication of what
may be expected from a standardized direct obser-
Implications for Practice and Research
vation of the child within the context of a broad clini-
cal assessment for preschool children suspected for First, the DB-DOS may be used as an observational tool
DBD and ADHD. to help support (or not) a presumption of a DBD and=
The replication of the study by Wakschlag and or ADHD, generated by the information from parent,
colleagues (Wakschlag, Briggs-Gowan, et al., 2008; teachers (or other caregivers), and cognitive assessment.
Wakschlag, Hill, et al., 2008) in a European sample The complexity of diagnosing DBD and ADHD in
of referred children; the newly developed ADHD preschool children necessitates a combination of infor-
domain; the determination of cutoff points to dis- mation about the childs behavior not only from a par-
tinguish clinical from nonclinical cases; and the ent interview=questionnaire and a teacher=caregiver
determination of the importance of the DB-DOS to questionnaire but also from an observation of the child
support (or not) a presumption of a DBD and=or (Le Couteur & Gardner, 2008, Wakschlag, Briggs-
ADHD or no diagnosis, generated by the infor- Gowan, et al., 2008; Wakschlag, Hill, et al., 2008).
mation from parent, teachers (or other caregivers), Second, accurate identication of ADHD in preschoo-
and cognitive assessment are strengths of the present lers is crucial as treatment of these children may include
study. In the present study the DB-DOS is examined pharmacotherapy and concerns have been raised about
in the context of DSM-IVTR-based diagnoses, overuse of psychostimulants to decrease symptoms of
whereas the use of DBD diagnoses in young children ADHD in the absence of a clear diagnosis (Rey &
has been questioned; as an alternative, based on a Sawyer, 2003). Third, further research is required to
dimensional conceptualization of psychopathology, study the predictive validity of the DB-DOS. Fourth,
a multidimensional, developmentally based approach it should be considered whether a more practical and
760 BUNTE ET AL.

less time-consuming way of coding may be developed in Lord, C., Risi, S., DiLavore, S., Shulman, C., Thurm, A., & Pickles, A.
view of using the DB-DOS in clinical practice. Fifth, the (2006). Autism from 2 to 9 years of age. Archives of General
Psychiatry, 63, 694701.
incremental knowledge gained from the DB-DOS Lord, C., Rutter, M., Goode, S., Heemsbergen, J., Jordan, H.,
beyond parent interviews and parent=teacher question- Mawhood, L., & Schopler, E. (1989). Autism diagnostic observation
naires is a future research direction as well. schedule: A standardized observation of communicative and social
behavior. Journal of Autism and Developmental Disorders, 19,
185212.
REFERENCES Matthys, W., & Lochman, J. E. (2010). Oppositional deant
disorder and conduct disorder in childhood. Chichester, UK: Wiley-
Achenbach, T. M., & Rescorla, L. A. (2000). Manual for the ASEBA Blackwell.
preschool forms and proles. Burlington: Research Center for Posthumus, J. A., Raaijmakers, M. A., Maassen, G. H., van Engeland,
Children, Youth & Families, University of Vermont. H., & Matthys, W. (2012). Sustained effects of incredible years as a
American Psychiatric Association. (2000). Diagnostic and statistical preventive intervention in preschool children with conduct
manual of mental disorders (4th ed., text rev.). Washington, DC: problems. Journal of Abnormal Child Psychology, 40, 487500.
Author. Raven, J. C., Court, J. H., & Raven, J. (1998). Raven Coloured
Collishaw, S., Goodman, R., Ford, S., Rabe-Hesketh, S., & Pickels, A. Progressive Matrices. Oxford, UK: Oxford Psychologist Press.
(2009). How far are associations between child, family and com- Rey, J. M., & Sawyer, M. G. (2003). Are psychostimulant drugs being
munity factors and child psychopathology informant-specic and used appropriately to treat child and adolescent disorders? The
informant-general? Journal of Child Psychology and Psychiatry, 50, British Journal of Psychiatry, 182, 284286.
571580. Robinson, E. A., & Eyberg, S. M. (1981). The dyadic parentchild
Cronbach, L. (1951). Coefcient alpha and the internal structure of interaction coding system: Standardization and validation. Journal
tests. Psychometrika, 16, 297334. of Consulting and Clinical Psychology, 49, 245250.
Dunn, L. M., & Dunn, L. M. (2005). Peabody Picture Vocabulary Test Ronk, M. J., Hund, A. M., & Landau, S. (2011). Assessment of social
(IIINL) [Dutch version by L. Schlichting]. Amsterdam, the competence of boys with attention decit=hyperactivity disorder:
Netherlands: Harcourt Assessment. Problematic peer entry, host responses, and evaluations. Journal of
Egger, H. L., & Angold, A. (2006). Common emotional and behavioral Abnormal Child Psychology, 39, 829840.
disorders in preschool children: Presentation, nosology, and Schaffer, D., Goud, M. S., Brasic, J., Ambrosini, P., Fisher, P., Bird,
epidemiology. Journal of Child Psychology and Psychiatry, 47, 313337. H., & Aluwahlia, S. (1983). A childrens global assessment scale
Egger, H. L., Erkanli, A., Keeler, G., Potts, E., Walter, B. K., & (CGAS). Archives of General Psychiatry, 40, 12281231.
Angold, A. (2006). Testretest reliability of the Preschool Age Schlichting, L. (2005). Peabody Picture Vocabulary Test IIINL.
Psychiatric Assessment (PAPA). Journal of the American Academy Amsterdam, the Netherlands: Hartcourt Assessment.
of Child and Adolescence Psychiatry, 45, 538549. Schoemaker, K., Bunte, T., Wiebe, S. A., Espy, K. A., Dekovic, M., &
Gadow, K., & Drabick, D. (2012). Anger and irritability symptoms Matthys, W. (2012). Executive function decits in preschool children
among youth with ODD: Cross-informant versus source-exclusive with ADHD and DBD. Journal of Child Psychology and Psychiatry,
syndromes. Journal of Abnormal Child Psychology, 40, 10731085. 53, 111119.
Gadow, K., & Sprafkin, J. (1996). Early childhood inventories manual. Sheeber, L., & Johnson, J. (1992). Applicability of the Impact on Fam-
Stony Brook, NY: Checkmate Plus. ily Scale for assessing families with behaviorally difcult children.
Gresham, F., & Elliot, S. (1990). Social Skills Rating System. Circle Journal of Child Psychology and Psychiatry, 17, 155159.
Pines, MN: American Guidance Service. Sterba, S., Egger, H. L., & Angold, A. (2007). Diagnostic specicity
Howell, E. M., & Teich, J. (2008). Variations in Medicaid mental and nonspecicity in the dimensions of preschool psychopath-
health service use and cost for children. Administration and Policy ology. Journal of Child Psychology and Psychiatry, 48,
in Mental Health, 35, 220228. 10051013.
Keenan, K., Boeldt, D., Chen, D., Coyne, C., Donald, R., Duax, J., . . . Wakschlag, L. S., Briggs-Gowan, M. J., Hill, C., Danis, B., Leventhal,
Humphries, M. (2011). Predictive validity of DSMIV oppositional B. D., Keenan, K., . . . Carter, A. S. (2008). Observational Assess-
deant and conduct disorders in clinically referred preschoolers. ment of Preschool Disruptive Behavior, Part II: Validity of the
Journal of Child Psychology and Psychiatry, 52, 4755. Disruptive Behavior Diagnostic Observation Schedule (DBDOS).
Keenan, K., Wakschlag, L. S., Danis, B., Hill, C., Humphries, M., Duax, J., Journal of the American Academy of Child and Adolescent
& Donald, R. (2007). Further evidence of the reliability and validity of Psychiatry, 47, 632640.
DSMIV ODD and CD in preschool children. Journal of the American Wakschlag, L. S., Henry, D. B., Tolan, P. H., Carter, A. S., Burns, J. L.,
Academy of Child and Adolescent Psychiatry, 26, 457468. & Briggs-Gowan, M. J. (2012). Putting theory to the test: Modeling a
Kim, S. H., & Lord, C. (2012). New Autism Diagnostic Interview multidimensional, developmentally-based approach to preschool
Revised algorithms for toddlers and young pre-schoolers from 12 disruptive behavior. Journal of the American Academy of Child and
to 47 months of age. Journal of Autism and Developmental Disorders, Adolescent Psychiatry, 51, 593604.
42, 8293. doi: 10.1007=s10803-011-1213-1 Wakschlag, L. S., Hill, C., Carter, A. S., Danis, B., Egger, H. L.,
Lahey, B., Pelham, W. E., Loney, D., Kipp, H., Ehrhardt, A., Lee, Keenan, K., . . . Briggs-Gowan, M. J. (2008). Observational assess-
S, . . . Massetti, G. (2004). Three-year predictive validity of ment of preschool disruptive behavior, Part I: Reliability of the
DSMIV attention decit hyperactivity disorder in children diag- Disruptive Behavior Diagnostic Observation Schedule (DBDOS).
nosed at 46 years of age. American Journal of Psychiatry, 161, Journal of the American Academy of Child and Adolescent
20142020. Psychiatry, 47, 622631.
Le Couteur, A., & Gardner, F. (2008). Use of structured interviews Wakschlag, L. S., Leventhal, B. L., Briggs-Gowan, M. J., Danis,
and observational methods in clinical settings. In M. Rutter, D. B., Keenan, K., Hill, C., . . . Carter, A. S. (2005). Dening the
Bishop, D. Pine, S. Scott, J. Stevenson, E. Taylor, & A. Thapar disruptive in preschool behavior: What diagnostic observation
(Eds.), Rutters child and adolescent psychiatry (5th ed., pp. can teach us. Clinical Child and Family Psychology Review, 8,
271288). Oxford, UK: Blackwell. 183201.
 PRESCHOOLERS WITH DBD AND ADHD 761

Wilens, T. E., Biederman, J., Brown, S., Monuteaux, M., Prince, J., & 3. Easily distracted
Spencer, T. J. (2002). Patterns of psychopathology and dysfunction 4. Easily bored
in clinically referred preschoolers. Journal of Developmental &
Behavioral Pediatrics, 23, S31S36.
Hyperactivity
5. Fidgets, squirms (minor movements)
APPENDIX 6. Difculty remaining seated and vigorous motor
activity (major movements=hyperactivity)
7. Difculty playing quietly
DBDOS ADHD items 8. Talks excessively
Inattention Impulsivity
1. Careless=inattention to details 9. Difculty waiting (non verbal)
2. Difculty sustaining attention 10. Interrupts; difculty waiting (verbal)
Copyright of Journal of Clinical Child & Adolescent Psychology is the property of Taylor &
Francis Ltd and its content may not be copied or emailed to multiple sites or posted to a
listserv without the copyright holder's express written permission. However, users may print,
download, or email articles for individual use.