NIH Public Access Author Manuscript J Autism Dev Disord. Author manuscript; available in PMC 2010 May 28. NIH-PA Author Manuscript Published in final edited form as: J Autism Dev Disord. 2009 June ; 39(6): 856–864. doi:10.1007/s10803-009-0694-7. Multi-informant Ratings of Psychiatric Symptom Severity in Children with Autism Spectrum Disorders: the Importance of Environmental Context Stephen M. Kanne, Department of Health Psychology and Thompson Center for Autism and Neurodevelopmental Disorders, University of Missouri Anna M. Abbacchi, and Department of Psychiatry and Pediatrics, Washington University School of Medicine, St. Louis, Missouri NIH-PA Author Manuscript John N. Constantino Department of Psychiatry and Pediatrics, Washington University School of Medicine, St. Louis, Missouri Abstract The present study examines co-occurring psychiatric syndromes in a well-characterized sample of youths with autism spectrum disorders (ASD; n = 177) and their siblings (n = 148), reported independently by parents and teachers. In ASD, parents reported substantial comorbidity with affective (26%), anxiety (25%), attentional (25%), conduct (16%), oppositional (15%), and somatic problems (6%). Teachers reported a much lower prevalence. Autistic severity scores for children with ASD exhibited moderate correlations with general psychopathology within- but not acrossinformants, whereas sibling correlations were significant both within- and across-informants. Results support the role of environmental context in psychiatric symptom expression in children affected by autism and suggest that informant discrepancies may more provide critical cues for these children via specific environmental modifications. NIH-PA Author Manuscript Keywords Autism; Pervasive Developmental Disorder; Prevalence; CBCL; Parent; Teacher The core features of an Autism Spectrum Disorder (ASD) relate to deficits in communication, social skills, and repetitive/stereotyped movements. However, individuals diagnosed with an ASD often have concomitant emotional and behavioral difficulties that have a significant impact on their functioning. Knowing prevalence rates of psychiatric symptoms in individuals with ASD and understanding of how these psychiatric symptoms are reported across settings directly impacts diagnostic decisions and treatment planning. Interpretive challenges exist when this information is discrepant or contradictory. Despite the weight clinicians give to reports of psychiatric symptoms from various settings (e.g., parents and teachers), few Correspondence concerning this article should be addressed to: Stephen M. Kanne, Thompson Center for Autism and Neurodevelopmental Disorders, University of Missouri – Columbia, 300, Portland Street, Columbia, MO, 65211, USA, kannest@health.missouri.edu. Thompson Center for Autism and Neurodevelopmental Disorders, 300 Portland Street, University of Missouri, Columbia, Missouri, 65211, USA. Anna Abbacchi, Department of Psychiatry and Pediatrics, 24 South Kingshighway, Washington University School of Medicine, St. Louis, Missouri, 63108, USA. John N. Constantino, Department of Psychiatry and Pediatrics, 24 South Kingshighway, Washington University School of Medicine, 660 S. Euclid Ave. Campus Box 8134, St. Louis, Missouri, 63110. Kanne et al. Page 2 researchers have examined the nature of cross-informant reporting differences in individuals with ASD. NIH-PA Author Manuscript Clinicians who work with children and adolescents diagnosed with ASD are well aware of the difficulties that these individuals experience beyond the core features typically associated with the disorder. These clinical observations have prompted researchers to examine the prevalence and quality of the co-occurring psychiatric difficulties, with the most common found to be symptoms of anxiety and depression (Howlin, 2000; Kim, Szatmari, Bryson, Streiner, & Wilson, 2000; Lainhart, 1999; Simonoff, Pickles, Charman, Chandler, & Baird, 2008; Tantam, 2000), but also including behavioral difficulties (Lecavalier, 2006), attentional dysregulation (Gadow, DeVincent, & Pomeroy, 2006; Leyfer et al., 2006; Simonoff et al., 2008), and obsessive tendencies (Leyfer et al., 2006). There are several obstacles to quantifying psychiatric co-morbidity in individuals with ASD. These include problems of a) symptom overlap; b) symptom masking; c) interpretation of emotional states; d) misinterpretation of the meaning of symptoms; and e) inherent difficulties with self-report. Furthermore, cognitive deficits and impaired insight may compromise a child’s ability to report or describe symptoms. Studies have used a variety of measures, standardized or otherwise, to assess for the presence and degree of psychiatric symptoms, making it difficult to compare across studies or to arrive at a common consensus. NIH-PA Author Manuscript NIH-PA Author Manuscript Despite these obstacles, most studies have found higher rates of psychiatric symptoms in individuals with ASD. For example, Leyfer and colleagues (2006) found that 72% of the children in their study met criteria for at least one DSM-IV Axis I disorder. More specifically, 44% of the children with autism in their sample met the criteria for a specific phobia, 37% met criteria for Obsessive-Compulsive Disorder (OCD), 31% met criteria for ADHD, and 10% met criteria for Major Depression. In another study, Lecavalier (2006) found a high endorsement rate of behavior and emotional difficulties in a non-clinically referred sample of individuals with ASD of an average of 9 years of age. Gadow and colleagues have found over half of the children ages 6–12 met criteria for ADHD (Gadow et al., 2006; Gadow, DeVincent, Pomeroy, & Azizian, 2004, 2005). Morgan et al. (2003) found that adults with both a learning disability and autism had high rates of depression (20%) and Bipolar Affective Disorder (11%) (Morgan, Roy, & Chance, 2003). Ghaziuddin and colleagues have noted high prevalence rates of depression in their work with individuals with ASD (Ghaziuddin, Ghaziuddin, & Greden, 2002; Ghaziuddin, Weidmer-Mikhail, & Ghaziuddin, 1998) and Howlin noted in her review of adult outcomes in Asperger Syndrome and “able” individuals with autism, that over a third of the reported psychiatric diagnoses were depression often associated with severe anxiety (Howlin, 2000). In a more recent study, de Bruin and colleagues found that 80.9% of the children with Pervasive Developmental Disorder – Not Otherwise Specified had at least one co-morbid psychiatric disorder (de Bruin, Ferdinand, Meester, de Nijs, & Verheij, 2007). In sum, all of these studies note a higher rate of prevalence in the ASD populations compared to a typical population, where prevalence estimates of psychiatric difficulties typically range from 8% to 18% (e.g., Roberts, Attkisson, & Rosenblatt, 1998). In addition to direct observation and assessment, clinicians often gather information regarding a child’s emotional and behavioral functioning through the use of behavioral report forms. These forms, filled out by parents, caregivers, and teachers, can be a rich source of information on how the child is functioning across settings and over time. An important consideration when interpreting these behavioral report forms involves informant source; previous research has demonstrated that discrepancies often exist when different informants rate a child’s behaviors using the same measure (De Los Reyes & Kazdin, 2005; Szatmari, Archer, Fisman, & Steiner, 1994). J Autism Dev Disord. Author manuscript; available in PMC 2010 May 28. Kanne et al. Page 3 NIH-PA Author Manuscript Ratings can differ for a variety of reasons. The instrument may fail to precisely characterize a specific trait across environmental contexts, such as between school and home, due to differential environmental contingencies. Ratings can also differ due to rater bias wherein discrepant ratings of an individual occur due to differing interpretations of the measurement scale or unique perceptions of the individual being rated (Hoyt, 2000). A possible exacerbating factor may be inherent in how the measure is designed; that is, many behavioral report forms are constructed to assess state characteristics, rather than trait characteristics. State characteristics are more vulnerable to situational variables and thus more prone to discrepant findings across informants and settings. Examples of measures that have discrepant findings amongst informants include the Behavior Assessment System for Children – 2nd Edition (BASC-2; Reynolds & Kamphaus, 2000), which reports a minimal relationship between teachers and parents on composite scores for Internalizing Problems, Externalizing Problems, and the Behavioral Symptoms Index. The widely used Child Behavior Checklist (CBCL) and its comparison instrument for teachers, the Caregiver/Teacher Report Form (C-TRF), report inter-rater agreement between teachers and parents ranging from r = .12 to r = .44 (Achenbach & Rescorla, 2001b).Similarly, the Behavior Rating Inventory of Executive Function (BRIEF; Gioia, Isquith, Guy, & Kenworthy, 2000) reports such a low level of agreement on the Global Executive Composite between teachers and parents that it is common for different conclusions to emerge. NIH-PA Author Manuscript In contrast, inter-rater agreement (e.g., between parent and teacher) tends to be greater when considering autistic symptomatology. These rating scales are likely tapping into more stable, trait characteristics that are less prone to discrepancies amongst informers from different settings. For example, in a study involving 577 children of which 406 had a pervasive developmental disorder, Constantino and colleagues demonstrated that teachers exhibited a high level of inter-rater agreement with parents (r = 0.72) when assessing autistic severity using the Social Responsiveness Scale (SRS; Constantino et al., 2007). Given the need to gather information across multiple sources and settings when assessing individuals with an ASD, it is important to use measures that are validated across clinical and educational settings. NIH-PA Author Manuscript This study had several objectives. First, using the parent-report CBCL and its associated instrument for teachers, the C-TRF, we examined a consecutively ascertained population of children and adolescents diagnosed with ASD to estimate the prevalence rates of comorbid psychiatric syndromes and sought to determine whether the various symptoms were either present, absent, or manifested in a range of severity in this clinical sample. Second, in order to determine the reliability of measurements of comorbid syndromes, we examined the extent to which parents and teachers agreed on quantitative characterization of these symptom domains in youth with ASD and their siblings. Using siblings allows for the comparison of psychiatric difficulties in a group of individuals who experienced a similar environment and were rated by the same informant (i.e., the parents). Finally, we attempted to elucidate domains of symptoms that might be better appreciated in home/family environments versus educational settings. Methods Participants Study participants were enrolled in a longitudinal research study of social development at Washington University School of Medicine. Children were recruited to participate in the research study either through the Washington University Child and Adolescent Psychiatry Clinic or through referrals, recruitment flyers, and letters from physicians and mental health professionals to the families of their patients in the Saint Louis Metropolitan area. To meet eligibility for the longitudinal study, the designated ASD case was required to have at least one J Autism Dev Disord. Author manuscript; available in PMC 2010 May 28. Kanne et al. Page 4 male sibling, either affected or unaffected with ASD, though a small number were recruited as singletons. NIH-PA Author Manuscript The ASD group (n=177) included male and female designated probands diagnosed with an Autism Spectrum Disorder (73% Asperger’s/PDD-NOS; 27% Autistic disorder). ASD participants were between the ages of 3 and 18 (mean age of 7.3 years); 85% of these participants were male and 15% were female (see Table 1 below). The sibling group included the closest male sibling of each child with ASD (n = 148) and were between the ages of 1 and 16 (mean age of 6.5 years). The remaining 29 probands had been recruited as singletons or had siblings outside the correct age range. Of the siblings, 9% were diagnosed with Autistic Disorder and 13% were diagnosed with Asperger’s/PDD-NOS. This study protocol was approved by the Washington University Institutional Review Board. NIH-PA Author Manuscript The study sample was phenotypically characterized using the CBCL and C-TRF in 100% of the study participants including both children with ASD and siblings. The Autism Diagnostic Interview – Revised (ADI-R; Rutter, Le Couteur, & Lord, 2003) had been completed to confirm a ASD diagnosis in 92% of the ASD participants, the Autism Diagnostic Observation Schedule (ADOS; Lord, Rutter, DiLavore, & Risi, 2002) was administered to 87% of the ASD participants, and the SRS (Constantino et al., 2003) was administered to 100% (parent-report) and 98% (teacher-report) of the ASD participants. Eighty-nine percent of the ASD participants met criteria for an ASD on the ADI-R. Of the remaining 19, all met criteria for an ASD on the ADOS or by expert clinician diagnosis. The ASD diagnosis was confirmed in 85% percent of the siblings using the ADI-R and the ADOS. Of the remaining 5, all met criteria for an ASD based on expert clinician diagnosis. Measures Achenbach System of Empirically Based Assessment (ASEBA; Achenbach & Rescorla, 2001a)—The Achenbach System of Empirically Based Assessment, which includes the Child Behavior Checklist and the Teacher Report Form, is widely used by clinicians to screen and assess for the reported presence of psychiatric symptoms (Achenbach, 1991a, 1991b, 1991c; Achenbach & Edelbrock, 1983; Achenbach & Rescorla, 2001a). NIH-PA Author Manuscript Child Behavior Checklist (CBCL)—Parents completed the 99 item, Preschool Form of the CBCL (1 ½ to 5 years of age) or the 112 item, School Age Form (6 to 18 years of age), which requires parents to rate their child’s specific behaviors, emotions, and emotional problems within the past six months on a scale ranging from “0” (not true) to “2” (very true or often true). Mean test-retest reliabilities of r = .85 and r = .88 across an eight day period have been reported for the Preschool and School-Age forms, respectively (Achenbach & Rescorla, 2001a, 2001b). Caregiver/Teacher Report Form (C-TRF)—Teachers or non-parent caregivers completed the C-TRF which involves ratings of children on the same 0 to 2 scale as the CBCL. Like the CBCL Preschool Form, the C-TRF is intended for ratings of children ages 1 ½ to 5 and consists of 99 behavioral/emotional items. For children over 5, teachers completed the TRF/6–18 consisting of 112 items. Mean test-retest reliabilities of r = .81 and r = .85 have been reported across an eight day period for the Preschool and School-Age forms, respectively (Achenbach & Rescorla, 2001a, 2001b). With regard to the ASD participants, 144 (81%) had the TRF completed concurrent with the CBCL. The remaining 33 completed the TRF an average of 1.7 years after the CBCL was completed. Social Responsiveness Scale (SRS; Constantino et al., 2003)—The SRS is a quantitative measure of autistic social impairment completed by parents, teachers or caregivers J Autism Dev Disord. Author manuscript; available in PMC 2010 May 28. Kanne et al. Page 5 NIH-PA Author Manuscript who routinely observe children in their naturalistic social contexts for a period of at least two months. The questionnaire requires 15 minutes to complete and generates five symptom domain scales (social awareness, social cognition, social communication, social motivation, and autistic mannerisms) and a singular total score for autistic social impairment. Psychometric studies of the SRS indicate that scores are continuously distributed across the general population, and that the SRS shows strong test-retest reliability (Constantino et al., In press; Constantino et al., 2003; Constantino, Przybeck, Friesen, & Todd, 2000; Constantino & Todd, 2000, 2003, 2008), inter-rater reliability (Pine, Luby, Abbacchi, & Constantino, 2006), discriminant validity (Constantino et al., 2007; Constantino et al., 2000; Constantino & Todd, 2000, 2003), and concurrent validity (Constantino et al., 2003). Data Analysis NIH-PA Author Manuscript To estimate prevalence rates, frequency distributions were calculated for the CBCL/TRF DSM oriented subscales using T-scores. As suggested by the CBCL and TRF manuals when using these DSM subscales, T-score values equal to or above 70 were considered to be clinically significant. Frequency distributions were examined to determine the range of severity of symptom manifestation reported. Measurement reliability across informant sources was examined by using a correlational analysis of the CBCL, TRF, and parent and teacher SRS raw scores to determine the extent to which parents and teachers agreed on quantitative characterization of the symptom domains. Bonferroni corrections were made to determine level of significance. Results Frequency Distributions of Teacher and Parent Behavioral Reports NIH-PA Author Manuscript Frequency distributions for the children with ASD were examined with respect to the DSM oriented subscale T-Score values from the CBCL. As shown in Table 2 below, parents rated 26% of their children with ASD in the clinically significant range on the Affective problems scale, 25.4% on the Anxiety problems subscale, 24.9% on the Attention Deficit Hyperactivity problems, and 15.3% on the Oppositional Defiant problems subscale. With regard to the subscales specific to the older children (CBCL/6–18, n = 101), parents rated 15.8% in the clinically significant range on the Conduct problems subscale and 5.9% on the Somatic problems subscale. For the subscales specific to the younger children (CBCL/1 ½ –5, n = 76), parents rated 68.4% in the clinically significant range on the Pervasive Developmental problems subscale. In order to examine the distribution of responses, frequency distributions are shown in Figure 1 that present the raw data for males using the CBCL/6–18 report form (for purposes of homogeneity). Distributions of scores for each of these syndromes was continuous (not bimodal) in the sample. Thirty-three percent of the individuals were below the clinically significant range on all subscales, 32% were in the clinically significant range on one subscale, 17% on two subscales, 10% on three subscales, and the remaining 8% were in the clinically significant range on four or five subscales. Teachers rated 6.2% of the children with ASD in the clinically significant range on the Affective problems subscale, 15.3% on the Anxiety problems subscale, 11.9% on the Attention Deficit Hyperactivity problems subscale, and 8.5% on the Oppositional Defiant problems subscale. With regard to the subscales specific to the older children (TRF/6–18, n = 119), teachers rated 1.7% in the clinically significant range on the Conduct Problems subscale and 2.5% on the Somatic Problems subscale. For the subscales specific to the younger children (CTRF, n = 58), teachers rated 44.8% in the clinically significant range on the Pervasive Developmental problems subscale. J Autism Dev Disord. Author manuscript; available in PMC 2010 May 28. Kanne et al. Page 6 NIH-PA Author Manuscript Sibling prevalence rates are also presented in Table 2 for purposes of comparison. Though lower than the ratings for the children with ASD (ranging from 1% to 25%), parent report of problem behaviors was higher than teacher report for the siblings on all subscales but one. Correlations between Parent and Teacher Report Ratings of Psychiatric Symptoms in Children with ASD The correlations between parent and teachers’ ratings of psychiatric difficulties in children with ASD were low for internalizing psychiatric symptoms (e.g., affective, anxiety) and moderate for all other CBCL domains, as shown in Table 3. This degree of correspondence for internalizing symptoms is generally less than what is observed for typically developing children (e.g., r = 0.23 for Affective Problems and Anxiety Problems in the overall normative sample for the CBCL; Achenbach & Rescorla, 2001b), and less than was observed for siblings in our sample. Correlations between Autistic Severity and Psychiatric Symptoms NIH-PA Author Manuscript In general, the correlations between autistic severity and psychiatric symptoms in ASDaffected children were substantial within-informant but very low across-informant, as shown in Table 4. Review of the data (i.e., scatter plots) indicated that this association withininformant was not driven by children rated in an extreme manner; that is, this association was not determined by outliers in the data. To see whether across-informant reports were accurate overall, we examined sibling correlations and found them to be highly congruent whether calculated across- or within-informant. Discussion Psychiatric Symptom Severity in Children with ASD NIH-PA Author Manuscript The results of this study are consistent with past research that has indicated a large proportion of individuals with ASD also evidence significant emotional and behavioral difficulties. Approximately one-fourth of the ASD affected children/adolescents in the current study were reported by their parents as having significant mood difficulties, specifically with respect to symptoms of depression and anxiety. Also, one-fourth of the children/adolescents in this sample were reported to have significant symptoms of attentional and behavioral dysregulation often associated with Attention Deficit Hyperactivity Disorder (ADHD). Fewer were reported to have significant symptoms associated with Oppositional Defiant Disorder (15%) and Conduct Disorder (16%). These proportions are generally consistent with those reported in other studies; for example, Simonoff and colleagues (2008) found 29% of their ASD cohorts met DSM-IV criteria for social anxiety disorder, 28% for ADHD, and 28% for Oppositional Defiant Disorder (Simonoff et al., 2008). In contrast, teachers reported a far lesser degree of symptomatology in all domains. Such differences could arise if parents and teachers observe the same syndromes but rate them at different levels of severity. We examined this possibility by computing bivariate correlations for each of the CBCL symptom domains and found low correlations especially for internalizing symptoms (e.g., affective and anxiety problems), suggesting that this discrepancy was not simply a function of parents and teachers rating the same problems at different levels of severity. To examine further the factors driving these rating discrepancies, we examined the associations between psychiatric symptoms in siblings, who are presumably less sensitive to variation in the social environment. A greater degree of inter-rater agreement of psychiatric symptoms was found for siblings, which would not be predicted if rater bias was the primary factor in the discrepancies. Though it remains possible that parent’s report of overall higher prevalence rates J Autism Dev Disord. Author manuscript; available in PMC 2010 May 28. Kanne et al. Page 7 NIH-PA Author Manuscript compared to teachers in both ASD-affected individuals and siblings reflects rater bias (e.g., caretaker burden) to some degree, these results provide preliminary evidence that state characteristics (i.e., psychiatric symptoms) are manifested differentially across environmental contexts for children with ASD. Autism Symptom Severity To consider this further, we examined how ratings of ASD traits associated across settings. In the children affected by ASD, there was a significant association between parent and teacher report of autistic trait severity (r = .24, p < .001) even in the narrow range of the distribution represented by an exclusively autistic sample. In the siblings, this association was more robust (r = .69, p < .001). Some of the diminution in correlation between autistic severity and psychiatric problems across-informant may be a function of a more limited extent of variation in SRS scores within the selected ASD-affected group (i.e., truncated range). Nevertheless, the degree of variation was adequate to identify strong across trait correlations within-informants. Thus, consistent with past research, inter-rater agreement between teachers and parents ratings of autism severity was higher compared to other psychiatric problems, supporting the notion that these ratings are assessing more stable trait characteristics, and that these trait characteristics are reported more consistently across informants. Psychiatric Symptom Severity and Autism Symptom Severity NIH-PA Author Manuscript We next attempted to gain a better understanding of how ratings of psychiatric symptoms associated with ratings of autistic traits across- and within-informants. For example, if a parent rated a child as having more severe ASD traits, did they and teachers also report more problem behaviors? For the siblings, CBCL/TRF problem behaviors correlated with SRS scores withinin-formant and across-informant. That is, when a teacher or parent rated a sibling as having more ASD traits, both parents and teachers saw more problem behaviors across settings. Remarkably, for the children with ASD, CBCL/TRF problem behaviors correlated with SRS scores within-informant, but not across-informant. More specifically, when the parent and teacher differed in ratings of autistic symptoms, there was a greater chance of discordance in their reporting of specific psychiatric symptoms as well. For example, if a parent rated their child as having more autistic symptoms compared to a teacher’s ratings, it was more likely that the parent also reported more psychiatric symptoms (such as anxiety) compared to the teacher. The correlations between the parent/teacher SRS difference scores and the DSM oriented subscale difference scores ranged from r = .21 to r = .73. Thus, in contrast to the siblings, when an ASD-affected child manifests more severe ASD symptoms in a particular setting, they also manifest more psychiatric symptoms. NIH-PA Author Manuscript A recent study also suggests that the association between psychiatric problems and ASD traits are closely linked. Pine and colleagues (2008) examined reports from the parents of 352 youths with a variety of mood and anxiety disorders and assessed for the presence of ASD symptoms using the Children’s Communication Checklist (Bishop, 1998), the Social Communication Questionnaire (Rutter, Bailey, Berument, Le Couteur, & Lord, 2003), and SRS. Those youths with mood disorders reportedly significantly more ASD symptoms (Pine, Guyer, Goldwin, Towbin, & Leibenluft, 2008). Role of Environmental Context To summarize this pattern of results, informants across settings were discrepant in their ratings of psychiatric symptoms, especially internalizing symptoms, and more so in children with ASD compared to siblings. These discrepancies likely represent variations in state characteristics manifested in different contexts. Although rater bias may account for these results, there was considerable agreement across informants as represented by sibling correlations and for J Autism Dev Disord. Author manuscript; available in PMC 2010 May 28. Kanne et al. Page 8 inherited trait correlations. This pattern of results supports the role of environmental context differentially affecting individuals with ASD. NIH-PA Author Manuscript These results have implications for the interpretation of CBCL data in children with ASD. The lack of concordance between parent and teacher reports of affective and anxiety-based symptoms emphasizes the importance of gathering information from multiple sources and settings, including direct observation and assessment. Each source provides information that can influence the clinician’s decisions whether to treat co-existing internalizing symptoms. In fact, the current results suggest caution when making conclusions or interpretations regarding the presence of co-morbid psychiatric problems based on a single informant source or environmental context. Reliance on one type of ascertainment methodology (e.g., parent report, teacher report, clinical ratings) may not be sufficient in arriving at a complete understanding of the behaviors and symptoms present, and can lead to false or misleading diagnostic conclusions, especially in children diagnosed with an ASD (Filipek et al., 2000; Lord & Corsello, 2005). The substantial variation in psychiatric difficulties in children with ASD that may be directly related to environmental factors suggests opportunities for differential intervention and measuring outcome. NIH-PA Author Manuscript The current study also highlights important aspects of interpreting more general information regarding the core features of autism reported by parents and teachers; more specifically, the PDD subscale. The name of this scale implies diagnostic utility in assessing children for ASD symptoms; however, using the current scoring algorithm, the CBCL and C-TRF were able to identify Pervasive Developmental Disorders (PDD) in only a moderate number of medically diagnosed ASD cases in the current sample, with 32% of the children in the current ASD sample not reaching clinical significance on the PDD subscale. Thus, the sensitivity of this assessment system for autistic psychopathology is not as high as for other domains it has traditionally measured. When using the CBCL, others have found the use of an Autistic/Bizarre factor based on specific CBCL items (i.e., Confused, Repeats Acts, Strange Behavior, Strange Ideas, and Withdrawn) and the use of the Thought Problems scale to be more effective in identifying children with ASD (Duarte, Bordin, de Oliveira, & Bird, 2003). It is important for clinicians to know the limitations of the CBCL PDD scales when interpreting and using the measure to aid diagnostic differentiation and to be aware of these alternatives. NIH-PA Author Manuscript This study has several limitations. The sample size and the nature of how this sample was referred suggest that some caution should be used in generalizing the results. Further research in this area with larger samples and a non-referred population that could replicate and extend these findings would be of benefit. Also, a small portion of the sample received the teachers’ report an average of 1.7 years after the parents’ report was finished, which could obviously introduce variability in the perceptions given possibility of changes in behaviors across that period. However, eliminating this small cohort had little impact on the prevalence rates or strength of associations, and, given the pervasive nature of the disorder, it was felt appropriate that this cohort be included. Another limitation involves using siblings of ASD-affected individuals as a comparison group. This group was chosen due to their experience of similar environments and having the same informant, reducing several confounds in interpreting the differences between the groups. However, 22% of these siblings were diagnosed with an ASD, and the others are expected to demonstrate a greater degree of ASD traits, and thus may be expected to have a greater degree of psychiatric co-morbidity. The reporting of psychiatric syndromes has several limitations as well. Though the current study uses a well-validated measure to assess for psychiatric difficulties, this measure is based on informant report rather than direct observation or a more comprehensive diagnostic interview. A more complete interview and history could help in understanding whether the psychiatric difficulties are separate co-occurring disorders and may help in elucidating the role J Autism Dev Disord. Author manuscript; available in PMC 2010 May 28. Kanne et al. Page 9 NIH-PA Author Manuscript of family history. Finally, this study uses a relatively conservative interpretation of clinical significance with regard to the CBCL and C-TRF T-Scores (i.e., T > = 70), which is suggested when using the DSM oriented subscales. Using a lower T-Score to indicate clinical significance, such as T >= 65, as is used for the Syndrome Scales, could increase the prevalence estimates. Acknowledgments This research was supported by the National Institute of Child Health and Human Development (R01-HD42541-01) to Dr. John Constantino. The authors wish to thank the families who participated in this study for their ongoing contributions to scientific research. References NIH-PA Author Manuscript NIH-PA Author Manuscript Achenbach, TM. Manual for Teacher's Report Form and 1991 Profile. Burlington, VT: University of Vermont, Department of Psychiatry; 1991a. Achenbach, TM. Manual for the Child Behavioral Checklist/4–18 and 1991 Profile. Burlington, VT: University of Vermont, Department of Psychiatry; 1991b. Achenbach, TM. Manual for the Youth Self-Report and 1991 Profile. Burlington, VT: University of Vermont, Department of Psychiatry; 1991c. Achenbach, TM.; Edelbrock, C. Manual for the Child Behavioral Checklist and Revised Child Behavioral Profile. Burlington, VT: University of Vermont, Department of Psychiatry; 1983. Achenbach, TM.; Rescorla, LA. Manual for the ASEBA Preschool Forms & Profiles. Burlington, VT: University of Vermont, Research Center for Children, Youth, & Families; 2001a. Achenbach, TM.; Rescorla, LA. Manual for the ASEBA School-Age Forms & Profiles. Burlington, VT: University of Vermont, Research Center for Children, Youth, & Families; 2001b. Bishop DVM. Development of the Children's Communication Checklist (CCC): A method for assessing qualitative aspects of communicative impairment in children. Journal of Child Psychology and Psychiatry 1998;39(6):879–891. [PubMed: 9758196] Constantino JN, Abbacchi A, LaVesser PD, Reed H, Givens L, Chiang L, et al. Developmental Course of Autistic Social Impairment In Males. Development and Psychopathology. (In press). Constantino JN, Davis SA, Todd RD, Schindler MK, Gross MM, Brophy SL, et al. Validation of a Brief Quantitative Measure of Autistic Traits: Comparison of the Social Responsiveness Scale with the Autism Diagnostic Interview-Revised. Journal of Autism and Developmental Disorders 2003;33(4): 427–433. [PubMed: 12959421] Constantino JN, LaVesser PD, Zhang Y, Abbacchi AM, Gray T, Todd RD. Rapid quantitative assessment of autistic social impairment by classroom teachers. Journal of the American Academy of Child & Adolescent Psychiatry 2007;46(12):1668–1676. [PubMed: 18030089] Constantino JN, Przybeck T, Friesen D, Todd RD. Reciprocal social behavior in children with and without pervasive developmental disorders. Journal of Developmental & Behavioral Pediatrics 2000;21(1): 2–11. [PubMed: 10706343] Constantino JN, Todd RD. Genetic structure of reciprocal social behavior. American Journal of Psychiatry 2000;157(12):2043–2044. [PubMed: 11097975] Constantino JN, Todd RD. Autistic traits in the general population: A twin study. Archives of General Psychiatry 2003;60(5):524–530. [PubMed: 12742874] Constantino JN, Todd RD. Hudziak JJ. Genetic epidemiology of pervasive developmental disorders. Developmental psychopathology and wellness: Genetic and environmental influences 2008:209– 224. de Bruin EI, Ferdinand RF, Meester S, de Nijs PFA, Verheij F. High rates of psychiatric co-morbidity in PDD-NOS. Journal of Autism and Developmental Disorders 2007;37(5):877–886. [PubMed: 17031447] De Los Reyes A, Kazdin AE. Informant Discrepancies in the Assessment of Childhood Psychopathology: A Critical Review, Theoretical Framework, and Recommendations for Further Study. Psychological Bulletin 2005;131(4):483–509. [PubMed: 16060799] J Autism Dev Disord. Author manuscript; available in PMC 2010 May 28. Kanne et al. Page 10 NIH-PA Author Manuscript NIH-PA Author Manuscript NIH-PA Author Manuscript Duarte CS, Bordin IAS, de Oliveira A, Bird H. The CBCL and the identification of children with autism and related conditions in Brazil: Pilot findings. Journal of Autism and Developmental Disorders 2003;33(6):703–707. [PubMed: 14714937] Filipek PA, Accardo PJ, Ashwal S, Baranek GT, Cook EH Jr. Dawson G, et al. Practice parameter: Screening and diagnosis of autism: Report of the Quality Standards Subcommittee of the American Academy of Neurology and the Child Neurology Society. Neurology 2000;55(4):468–479. [PubMed: 10953176] Gadow KD, DeVincent CJ, Pomeroy J. ADHD Symptom Subtypes in Children with Pervasive Developmental Disorder. Journal of Autism and Developmental Disorders 2006;36(2):271–283. [PubMed: 16477513] Gadow KD, DeVincent CJ, Pomeroy J, Azizian A. Psychiatric Symptoms in Preschool Children with PDD and Clinic and Comparison Samples. Journal of Autism and Developmental Disorders 2004;34 (4):379–393. [PubMed: 15449514] Gadow KD, Devincent CJ, Pomeroy J, Azizian A. Comparison of DSM-IV symptoms in elementary school-age children with PDD versus clinic and community samples. Autism 2005;9(4):392–415. [PubMed: 16155056] Ghaziuddin M, Ghaziuddin N, Greden J. Depression in persons with autism: Implications for research and clinical care. Journal of Autism and Developmental Disorders 2002;32(4):299–306. [PubMed: 12199134] Ghaziuddin M, Weidmer-Mikhail E, Ghaziuddin N. Comorbidity of Asperger syndrome: A preliminary report. Journal of Intellectual Disability Research 1998;42(4):279–283. [PubMed: 9786442] Gioia GA, Isquith PK, Guy SC, Kenworthy L. Behavior Rating Inventory of Executive Function. Child Neuropsychology 2000;6(3):235–238. [PubMed: 11419452] Howlin P. Outcome in adult life for more able individuals with autism or Asperger syndrome. Autism 2000;4(1):63–83. Hoyt WT. Rater bias in psychological research: When is it a problem and what can we do about it? Psychological Methods 2000;5(1):64–86. [PubMed: 10937323] Kim JA, Szatmari P, Bryson SE, Streiner DL, Wilson FJ. The prevalence of anxiety and mood problems among children with autism and Asperger syndrome. Autism 2000;4(2):117–132. Lainhart JE. Psychiatric problems in individuals with autism, their parents and siblings. International Review of Psychiatry 1999;11(4):278–298. Lecavalier L. Behavioral and emotional problems in young people with pervasive developmental disorders: Relative prevalence, effects of subject characteristics, and empirical classification. Journal of Autism and Developmental Disorders 2006;36(8):1101–1114. [PubMed: 16897387] Leyfer OT, Folstein SE, Bacalman S, Davis NO, Dinh E, Morgan J, et al. Comorbid Psychiatric Disorders in Children with Autism: Interview Development and Rates of Disorders. Journal of Autism and Developmental Disorders 2006;36(7):849–861. [PubMed: 16845581] Lord, C.; Corsello, C. Diagnostic Instruments in Autistic Spectrum Disorders. In: Volkmar, FR.; Paul, R.; Klin, A.; Cohen, D., editors. Handbook of autism and pervasive developmental disorders, Vol. 2: Assessment, interventions, and policy. 3rd ed.. Hoboken, NJ: John Wiley & Sons Inc; 2005. p. 730-771. Lord, C.; Rutter, M.; DiLavore, PC.; Risi, S. Autism Diagnostic Observation Schedule. Los Angeles, CA: Western Psychological Services; 2002. Morgan CN, Roy M, Chance P. Psychiatric comorbidity and medication use in autism: A community survey. Year of Publication 2003. Psychiatric Bulletin 2003;27(10):378–381. Pine DS, Guyer AE, Goldwin M, Towbin KA, Leibenluft E. Autism Spectrum Disorder Scale scores in pediatric mood and anxiety disorders. Journal of the American Academy of Child & Adolescent Psychiatry 2008;47(6):652–661. [PubMed: 18434923] Pine E, Luby J, Abbacchi A, Constantino JN. Quantitative assessment of autistic symptomatology in preschoolers. Autism 2006;10(4):344–352. [PubMed: 16908478] Reynolds, CR.; Kamphaus, RW. Behavior Assessment System for Children - 2nd Edition. Circle Pines, MN: AGS Publishing; 2000. Roberts RE, Attkisson CC, Rosenblatt A. Prevalence of psychopathology among children and adolescents. American Journal of Psychiatry 1998;155(6):715–725. [PubMed: 9619142] J Autism Dev Disord. Author manuscript; available in PMC 2010 May 28. Kanne et al. Page 11 NIH-PA Author Manuscript Rutter, M.; Bailey, A.; Berument, SK.; Le Couteur, A.; Lord, C. Social Communication Questionnaire (SCQ) Manual. Los Angeles: Western Psychological Services; 2003. Rutter, M.; Le Couteur, A.; Lord, C. Autism Diagnostic Interview - Revised. Los Angeles, CA: Western Psychological Services; 2003. Simonoff E, Pickles A, Charman T, Chandler TL, Baird G. Psychiatric disorders in children with autism spectrum disorders: prevalence, comorbidity, and associated factors in a population-derived sample. Journal of the American Academy of Child & Adolescent Psychiatry 2008;47(8):921–929. [PubMed: 18645422] Szatmari P, Archer L, Fisman S, Steiner DL. Parent and teacher agreement in the assessment of pervasive developmental disorders. Journal of Autism and Developmental Disorders 1994;24(6):703–717. [PubMed: 7844095] Tantam D. Psychological disorder in adolescents and adults with Asperger syndrome. Autism 2000;4(1): 47–62. NIH-PA Author Manuscript NIH-PA Author Manuscript J Autism Dev Disord. Author manuscript; available in PMC 2010 May 28. Kanne et al. Page 12 NIH-PA Author Manuscript NIH-PA Author Manuscript NIH-PA Author Manuscript Figure 1. Frequency distributions of the DSM oriented subscales raw scores of the CBCL/6–18 for males. J Autism Dev Disord. Author manuscript; available in PMC 2010 May 28. Kanne et al. Page 13 Table 1 Sample characteristics for children with ASD (n = 177) and siblings (n = 148) NIH-PA Author Manuscript Male Mean Age (sd) Age Range Children with ASD Siblings 85% 100% 7.3 (3.9) 6.5 (4.1) 3 – 18 1 – 16 Autism 27% 9% Asperger’s/PDD-NOS 73% 13% 102.1 (27.0) 45.5 (39.1) 27 – 175 1 – 159 Parent SRS Total Score - Mean (sd) Parent SRS Total Score - Range NIH-PA Author Manuscript NIH-PA Author Manuscript J Autism Dev Disord. Author manuscript; available in PMC 2010 May 28. Kanne et al. Page 14 Table 2 NIH-PA Author Manuscript Proportion of emotional/behavioral problems in clinically significant range per CBCL/TRF DSM oriented subscales (T-score >= 70) for children with ASD and siblings Children with ASD CBCL TRF Siblings CBCL TRF Affective Problems 26% 6% 10% 2% Anxiety Problems 25% 15% 10% 9% Somatic Problems 6% 3% 4% 1% ADH Problems 25% 12% 10% 5% OD Problems 15% 9% 5% 2% Conduct Problems 16% 2% 1% 0% PD Problems 68% 45% 25% 27% NIH-PA Author Manuscript NIH-PA Author Manuscript J Autism Dev Disord. Author manuscript; available in PMC 2010 May 28. Kanne et al. Page 15 Table 3 NIH-PA Author Manuscript Correlations between Teacher (TRF) and Parent (CBCL) DSM oriented subscales (raw score) for children with ASD and siblings Children with ASD Siblings Affective 0.08 0.43** Anxiety 0.14 0.37** Somatic 0.49** 0.08 ADH 0.30** 0.52** OD 0.26* 0.46** Conduct 0.35** 0.64** 0.32 0.73** PD ** indicates p < .001; * indicates p < .0025; ψ indicates p < .01 NIH-PA Author Manuscript NIH-PA Author Manuscript J Autism Dev Disord. Author manuscript; available in PMC 2010 May 28. NIH-PA Author Manuscript NIH-PA Author Manuscript NIH-PA Author Manuscript Table 4 Children with ASD Siblings Parent SRS Teacher SRS Parent SRS Teacher SRS Affective Problems −0.03 0.35** 0.53** 0.68** Anxiety Problems 0.02 0.24* 0.28* 0.56** Somatic Problems 0.07 .043 0.14 .12 ADH Problems 0.09 0.48** 0.60** 0.72** OD Problems 0.08 0.24* 0.52** 0.51* Conduct Problems 0.09 0.16 0.37** 0.42** PD Problems 0.33 0.69** 0.74** 0.92** Affective Problems 0.44** −0.00 0.63** 0.44** Anxiety Problems 0.27** −0.09 0.54** 0.42** Somatic Problems 0.17 −0.20 0.13 0.01 ADH Problems 0.43** 0.01 0.59** 0.44** OD Problems 0.30** −0.16 0.46** 0.30* Conduct Problems 0.29ψ −0.19 0.46ψ 0.23 PD Problems 0.68** 0.20 0.85** 0.76** Kanne et al. CBCL/TRF DSM oriented subscale (raw score) correlations with parent and teacher SRS total score for children with ASD and siblings TRF J Autism Dev Disord. Author manuscript; available in PMC 2010 May 28. CBCL ** indicates p < .001; * indicates p < .0025; ψ indicates p < .01 Page 16