490916

research-article2013
JPA32110.1177/0734282913490916Journal of Psychoeducational Assessment X(X)Test Review

Test Review
Journal of Psychoeducational Assessment
2014, Vol 32(1) 88­–92
Test Review © 2014 SAGE Publications
Reprints and permissions:
sagepub.com/journalsPermissions.nav
jpa.sagepub.com

Lord, C., Luyster, R. J., Gotham, K., & Guthrie, W. (2012). Autism Diagnostic Observation Schedule, Second
Edition (ADOS-2) Manual (Part II): Toddler Module. Torrance, CA: Western Psychological Services, 2012.
Lord, C., Rutter, M., DiLavore, P. C., Risi, S., Gotham, K., & Bishop, S. Autism Diagnostic Observation
Schedule, Second Edition. Torrance, CA: Western Psychological Services, 2012.

Reviewed by: Adam McCrimmon, University of Calgary, Calgary, AB, Canada; Kristin Rostad, Alberta
Children’s Hospital, Calgary, AB, Canada
DOI: 10.1177/0734282913490916

Test Description
The Autism Diagnostic Observation Schedule, Second Edition (ADOS-2; Lord, Luyster, Gotham,
& Guthrie, 2012; Lord, Rutter et al., 2012) is a newly updated, semistructured, standardized mea-
sure of communication, social interaction, play/imagination, and restricted and/or repetitive behav-
iors published by Western Psychological Services. Used in clinical and research settings, the
ADOS-2 is often referred to as the “gold standard” measure of observational assessment for autism
spectrum disorder (ASD; Kanne, Randolph, & Farmer, 2008). It is a Level C measure that can be
administered and interpreted by appropriately credentialed professionals from medicine, psychol-
ogy, or a related discipline in approximately 40 to 60 min, depending on the module selected and
the specific behavior demonstrated by the examinee. Extensive prior experience with individuals
with ASD is strongly recommended, and administrators must have completed clinical training pro-
vided by an independent trainer or through the test publisher.
The ADOS-2 kit is a large container consisting of most of the required toy items and materials
(some materials must be supplied and replenished by the examiner), and 10 protocols for each of
the 5 modules. The manual is expansive and well laid out, beginning with an overview of the
measure, guidelines for module selection, administration and coding procedures, instructions for
each module, and case examples to assist with interpretation.

Measure Development
The ADOS-2 was designed to improve the accuracy and effectiveness of the diagnostic algorithms
of the original ADOS (Lord, Rutter, DiLavoire, & Risi, 1999). Specific changes include revision
and expansion of the diagnostic algorithms used in Modules 1 to 3 and the addition of a Toddler
Module. A new comparison score in Modules 1 through 3 was added to allow for comparability
over time. Minor modifications to the administration instructions for some tasks were also made;
however, experienced ADOS administrators will notice that the majority of the administrative
procedures remain identical to the previous version of the measure. The protocols have also been
updated to include an enhanced description of administration and the behavior(s) being observed.

Administration and Scoring

General Description and Administration
The ADOS-2 is designed to be individually administered by one examiner. It consists of five
modules, the selection of which is based primarily on the examinee’s expressive language level
Test Review 89

as determined by formal language assessment and/or observation by the examiner, and second-
arily on their chronological age and/or relevance of tasks to the examinee’s interests and abilities.
Each module consists of specified tasks designed to elicit communicative, social, and behavioral
characteristics consistent with ASD. The format of the ADOS-2 differs from the majority of other
standardized assessment measures in that the assessment is dynamic and examiners should not
constantly refer to the manual during administration so that social interaction with the examinee
is natural and maintained.
The specific activities and subsequent behavioral coding within each module differ; however,
there is some overlap and the general administration and scoring principles remain consistent.
Administration necessitates delivery of specific tasks in a semistructured manner whereby exam-
iners deliberately modify their behavior according to a hierarchy of structured and unstructured
social presses and observe the examinee’s behaviors in response. Administration of tasks is flex-
ible and does not have to follow a set sequence. Behavioral and language observations should be
noted throughout the administration, particularly regarding social and communicative behaviors
related to ASD diagnostic criteria. The tasks are standardized in that examiners are provided
instructions regarding task administration and the hierarchy of behaviors to be employed. The
tasks are designed to provide structure to the interactions between the examiner and examinee
and should not be robotically administered without regard for the nature of the interactions.
Coding should occur directly following the administration using the behavioral notes taken by
the examiner. A parent/caregiver who is familiar with the child must be present in the room dur-
ing administration of the Toddler Module, Module 1, and Module 2.

Toddler module. The Toddler Module is designed for children 12 to 30 months of age who do
not consistently use phrase speech (i.e., flexible nonechoed, three-word utterances that include
a verb). It consists of 11 primary activities and 4 secondary tasks that inform 41 coded items
pertaining to Social Affect (SA) and Restricted and Repetitive Behavior (RRB). Two scoring
algorithms were created: All Younger/Older with Few to No Words, and Older with Some Words.

Module 1. Module 1 is designed for children 31 months of age or older who do not consis-
tently use phrase speech. It consists of 10 activities that inform 34 coded items pertaining to SA
and RRB. There are now two algorithms: Few to No Words, and Some Words.

Module 2. Module 2 is designed for children of any age who use phrase speech but who have
not developed verbally fluent speech (i.e., expressive language skills of a typical 4-year-old). It
consists of 14 activities that inform 29 coded items pertaining to SA and RRB. Two algorithms
now exist: Younger than 5 years, and Aged 5 Years or Older.

Module 3. Module 3 is suited for verbally fluent children and adolescents for whom action-
figure toy play is age appropriate (approximately 16 years or younger). It consists of 14 activities
that inform 29 coded items pertaining to SA and RRB.

Module 4. Module 4 is suited for verbally fluent older adolescents and adults. It consists of 10
to 15 activities that inform 32 coded items. As Module 4 was not revised with the ADOS-2, its
coded items are structured for separate communication, reciprocal social interaction, and RRB
domains.

Scoring
Behavioral and language indicators noted during the assessment inform specific coded items for
each module pertaining to (a) Language and communication, (b) reciprocal social interaction,
90 Journal of Psychoeducational Assessment 32(1)

(c) play and imagination, (d) stereotyped behaviors and restricted interests, and (e) other behav-
iors. Coding is a primary activity of the ADOS-2 and is the basis for clinical decision making
when the codes are transferred to the algorithm and summed. As such, familiarity with coding
concepts and conventions are integral to effective and appropriate ADOS-2 administration.
Specifically, a code of 0 indicates no evidence of abnormal behavior as specified by the code, 1
indicates mildly abnormal or slightly unusual behavior, 2 indicates definite abnormality of
behavior, and 3 indicates markedly abnormal behavior. Other codes (4, 7, 8, and 9) are used to
identify specific language ability, atypical behaviors that are present but not of the specified type,
or an item that cannot be scored for a number of possible reasons. Additional information pertain-
ing to coding can be found in the manual and through specialized training programs offered by
the test publisher or test developers.

Technical Adequacy
Standardization
Given the equivalence of ADOS and ADOS-2 items, the authors provide psychometric data on
the ADOS validation sample (98 children; 29 Module 1s, 23 Module 2s, 26 Module 3s, and 20
Module 4s) as well as an ADOS-2 extended validation sample (1,574 children; 847 Module 1s,
329 Module 2s, and 398 Module 3s) and an ADOS-2 replication sample (1,282 children; 620
Module 1s, 205 Module 2s, and 457 Module 3s). Across all of these samples, the majority of
participants received a diagnosis of Autism (45%-76%), followed by nonautism ASD (7%-29%),
and nonspectrum (17%-26%). Participants were predominantly male (57%-86%) and Caucasian
(71%-91%). For the Toddler Module validation sample, 182 children were seen for a total of 360
assessments (i.e., most children had two assessments). The majority of the participants were male
(76%) and Caucasian (80%). Of the 182 children, 25% had an ASD diagnosis, 20% had a non-
spectrum disorder, and 55% were typically developing. The majority of participants in the ADOS,
ADOS-2 extended validation, and Toddler Module validation samples were obtained through
clinics in Illinois and Michigan. The ADOS-2 replication sample data were obtained from 10
sites across the United States.

Item Selection and Algorithm Development

Items (codes) were selected based on their ability to differentiate between clinical diagnoses
(Lord, Rutter et al., 2012). Theoretical importance was also considered, such as the inclusion of
items and behaviors considered to be clinically meaningful (e.g., self-injurious behaviors). The
ADOS-2 standardization sample was divided into five developmental cells that correspond to the
five new ADOS-2 algorithms for Modules 1 through 3 to determine which codes to include.
Codes were considered based on their ability to distinguish spectrum from nonspectrum diagno-
ses (e.g., no more than 20% of autism cases could have an assigned rating of 0 on a code) as well
as for their comparability across modules. An exploratory multifactor item-response analysis was
used to organize the codes into new domains: SA and RRB. A confirmatory factor analysis veri-
fied the goodness of fit, and a logistic regression supported the predictive value of scores from
each of the two new factors. Revised algorithms were created based on verbal skills for Module
1 and chronological age for Module 2.

Reliability
Internal consistency. Cronbach’s α values for Modules 1 through 3 were high for the SA domain
(.87-.92) and moderate for the RRB domain (.51-.66). The same pattern was true for the Toddler
Test Review 91

Module, with the highest alphas in the SA domain (.88-.90) as compared to the RRB domain
(.50). For Module 4, Cronbach’s αs exceeded .75 for the communication domain, .85 for the
social interaction domain, and .47 for the RRB domain.

Test–retest reliability. Test–retest reliability was calculated for Modules 1 through 3 using a
sample of 75 participants who were administered the same module twice within an average of
10 months. Analyses indicated that SA, RRB, and overall total scores had correlations ranging
from .68 to .92. Test–retest reliability for the Toddler Module was calculated for 39 children who
received two administrations within a 2-month span. The SA, RRB, and overall total correlations
ranged from .64 to .88.

Interrater reliability. For the ADOS validation, ADOS-2 extended validation, and Toddler
Module validation samples, examiners had to achieve research reliability (i.e., 80% or greater
exact agreement with other reliable raters). Across the five modules, interrater reliability for item
coding was 71% or higher with the majority of mean weighted kappas exceeding .60. Correla-
tions were calculated for SA, RRB, and overall total and ranged from .79 to .98 across the five
modules. Agreement in diagnostic classification ranged from 92% to 98% in Modules 1 through
3 and from 87% to 97% in the Toddler Module (Module 4 not calculated).

Validity
Content and construct validity. Codes were selected for inclusion based on their unique con-
tribution to the original ADOS algorithm (i.e., no interitem correlations greater than .70). A
two-factor solution was identified based on exploratory and confirmatory factor analysis: SA,
consisting of the formerly known reciprocal social interaction and communication items, and
RRB. For Modules 1 through 3, logistic regressions indicated that both the SA and RRB domains
made significant independent contributions to the prediction of diagnosis. However, the overall
total score produced the highest predictive value, supporting the use of this score in diagnostic
decisions. Codes for the Toddler Module were selected based on their differential distributions
across diagnostic groups as well as their relatively low correlations with other items, chronologi-
cal age, and measured intelligence. Exploratory factor analysis revealed that once the codes were
split into All Younger/Older with Few to No Words and Older with Some Words, chronological
and verbal mental age no longer significantly contributed to any factor, supporting the develop-
ment of these two separate algorithms.
According to item–total correlations, almost all of the items across the five algorithms used
for Modules 1 through 3 and in the Toddler Module correlated more strongly with their assigned
domain than with each other (and those that did not typically differed by a correlational value of
.05 or less). Item and domain correlations for Module 4 ranged from .50 to .88.

Predictive validity. Receiver operating characteristic curves evaluated the predictive validity of
the ADOS-2 to identify individuals on the autism spectrum accurately. When comparing the ADOS
to the ADOS-2, sensitivity and specificity values were largely comparable or improved with the
new algorithms, particularly in Modules 1 and 2 where dual algorithms were introduced based
on expressive language and chronological age (sensitivity ranging from 60%-95% and specificity
ranging from 75%-100%). Sensitivity and specificity for the Toddler Module were at 86% or above.

Commentary and Recommendations

The ADOS-2 has several features that support its continued recognition as a “gold standard”
assessment measure for ASD. The manual is more expansive and detailed than the previous
92 Journal of Psychoeducational Assessment 32(1)

measure. The new protocols allow for additional space for behavioral notes and include additional
information on hierarchies of examiner prompts and important behaviors to be observed during
specific tasks. The behavioral descriptions attached to codes have been clarified. A new compari-
son score has been included. The revised algorithms, including the splitting of algorithms based
on language ability or age in Modules 1-3 and the re-conceptualization from three domains into
two (SA and RRB), allow for enhanced diagnostic clarity as coding is less affected by language
impairment and cognitive functioning and the factor structure is in alignment with the forthcom-
ing DSM-5 framework. As a result, the sensitivity and specificity of the measure has been
improved. Finally, the inclusion of the Toddler Module allows for administration to a wider client
population, thus enhancing the measure’s effectiveness as a diagnostic tool.
Despite the positive attributes, the ADOS-2 also contains some limitations. Module 4 was not
updated. Although the authors indicate that a lack of sample size was responsible, it remains
problematic for researchers and clinicians who may use that module. The training videos offered
by the test publisher have not been updated, with the exception of the Toddler Module. As subtle
nuances and administrative procedures have changed from the ADOS, updated training tools
would be beneficial. The standardization sample was primarily male, Caucasian, and American,
which may limit the generalizability of the results to a broader population due to such factors as
gender or culturally appropriate social behaviors (e.g., eye contact). The RRB domain had lower
internal consistency and test–retest values; however, the authors explained that this is likely due
to the limited opportunity to observe these behaviors during administration, as they are lower
base rate behaviors. Finally, there is no appropriate module for older individuals who are lower
functioning.
Overall, the ADOS-2 is an effective clinical and research instrument for the ASD population.
Although the measure does present with some limitations, it remains a primary diagnostic tool
for this population and will enhance clinical and research utility.

Declaration of Conflicting Interests

The author declared no potential conflicts of interest with respect to the research, authorship, and/or publi-
cation of this article.

Funding
The author received no financial support for the research, authorship, and/or publication of this article.

References
Kanne, S. M., Randolph, J. K., & Farmer, J. E. (2008). Diagnostic and assessment findings: A bridge to aca-
demic planning for children with autism spectrum disorders. Neuropsychology Review, 18(4), 367-384.
Lord, C., Luyster, R. J., Gotham, K., & Guthrie, W. (2012). Autism diagnostic observation schedule, second
edition (ADOS-2) manual (Part II): Toddler module. Torrance, CA: Western Psychological Services.
Lord, C., Rutter, M., DiLavore, P. C., & Risi, S. (1999). Autism diagnostic observation schedule: Manual.
Los Angeles, CA: Western Psychological Services.
Lord, C., Rutter, M., DiLavore, P. C., Risi, S., Gotham, K., & Bishop, S. (2012). Autism diagnostic observa-
tion schedule, second edition. Torrance, CA: Western Psychological Services.