Holistic student assessments

Assessments are an important component when providing program services to support student growth and success. This article outlines a holistic approach to student assessments. 7 Holistic student assessments Cindy H. Liu, Tina Malti, Gil G. Noam TOO OFTEN STUDENTS ’ problems as well as their strengths and resiliency remain undetected in schools. This problem has been recognized, and schools are increasingly employing mental health screenings to supplement their academic assessments. In this article, we describe the rationale for assessing youth in RALLY, a holistic perspective that covers a developmental, relational, and contextual frame. We designed assessments that aim to understand the overall well-being and social and emotional functioning of the youth, while increasing ease of implementation and utilization. Assessment can be thought of as a critical aspect of a youth programming. An assessment should help to set goals for all staff and youth involved in the program by carefully observing and tracking development, academic work, and resiliency to determine the program focus for each individual. For instance, promoting youth development, academic success, and resilience requires not only knowledge of the youth’s level of behavioral or academic performance, but an understanding of the extent to which behavioral or academic change occurred over time. In doing so, it should grasp the nature of the youth’s relationships with teachers, peers, parents, NEW DIRECTIONS FOR YOUTH DEVELOPMENT, NO. 120, WINTER 2008 © GIL G. NOAM Published online in Wiley InterScience (www.interscience.wiley.com) • DOI: 10.1002/yd.289 139 140 YOUTH DEVELOPMENT MEETS MENTAL HEALTH and the community and pinpoint areas of both developmental risk and resiliency. A comprehensive understanding helps to generate ideas in directing the program course that best addresses the need and potential of adolescents and those who work with them. On a broader level, assessment should help to inform policy by indicating where and how policy change should take place.1 Assessments in schools Given that school and after-school programs could assess an almost never-ending list of factors deemed significant, it is important to target the most relevant factors in the promotion of youth development and academic success and assess them as comprehensively as possible given the logistic constraints. The reauthorization of the Individuals with Disabilities Education Act in 1997 prompted increased attention toward school-based assessment. Since then, the Functional Behavioral Assessment (FBA) has been instituted in schools nationwide to understand individual problem behaviors.2 It was an important first step in school assessments for behavior problems and proposes a comprehensive assessment with the inclusion of questionnaires, teacher interviews, and observations, both natural and experimental. Although it is widely used, it also has limitations, including the focus on problem behavior rather than on resilience, which results in little consideration for the developmental and contextual factors that are crucial to understanding problem behavior.3 Furthermore, there is often a lack of integration between the efforts of those who conduct such an assessment (teachers, outside consultants) with school staff (coordinators) who link the student and parents with outside services using the FBA.4 Finally, the FBA is often used with the most at-risk children in the form of an Individualized Education Program, thus singling out the children who need the highest level of support and services and not providing sufficient screening for all children. What may be less stigmatizing yet efficient are screens that are administered to all students in the school, such as the TeenScreen, which is used to determine suicidality in high new directions for youth development • DOI: 10.1002/yd HOLISTIC STUDENT ASSESSMENTS 141 school students.5 To our knowledge, however, the existing schoolbased assessments provide little information regarding the overall well-being and social and emotional functioning of the child, including developmental or contextual factors.6 RALLY takes a theoretically based holistic perspective that aims to understand adolescent functioning from a developmental and relational perspective in an integrative way (see the second article). The holistic perspective is not a set of measures, but rather a purposeful way of thinking about how to obtain relevant information about youth development, education, and resiliency. A holistic assessment is linked with the services provided; all assessments and many services are housed within the same system where all students take part. The main factors RALLY considers especially important to include are developmental level, resiliencies, risks, and social relationships, all of which play a role in an adolescent’s social and emotional functioning. In programs, we thus include measures for development (such as developmental level of social cognition; see the eighth article), resiliencies (such as empathy and emotion regulation skills), risks (such as behavioral and emotional problems), and social relationships (such as perceived relationship quality in parent-child and teacher-peer relationships). We elaborate more closely on the measures in the next chapter. Considering these factors helps to generate a holistic picture of the individual adolescent by identifying emerging or existing risks to address while also considering strengths to promote. Rather than focusing on the techniques of administering an assessment, we are encouraging practitioners or youth workers to begin thinking in terms of these factors as they work with their students. The goals for assessment in a prevention program are admittedly lofty. Striking a balance with all the theoretical and practical considerations to increase the potential for an assessment is crucial for producing positive social and emotional change and academic gains in students. This article describes the various considerations adopted in the development of the holistic assessment used in RALLY, the aspects of adolescents’ experience that we felt were important to assess, the measures used that we felt would best fit new directions for youth development • DOI: 10.1002/yd 142 YOUTH DEVELOPMENT MEETS MENTAL HEALTH the assessment goals, and the way we chose to incorporate it in the overall model. Practical considerations for implementing a holistic assessment Concerns pertaining to the collaboration and pragmatics of the different systems (for example, school or community) should be addressed with the design and implementation of the assessment.7 First and foremost, the assessment needed to address the concerns of the neighborhood community that it serves and to elevate the values that they espouse. For instance, if it is important for the community that teachers are supported in their classroom teaching and behavioral management and that student participation is promoted in after-school programming, the assessment is needed to provide student information to these youth workers so they can determine the best way the community believes that individual students can benefit from the services. Second, one of the major problems in conducting assessment in the schools is the protracted time in evaluating and referring a child for services.8 As teachers get to know their students, they may realize that an assessment would be useful in pinpointing student needs. The typical process in a school system often consists of teachers submitting a request for their student to obtain an assessment a couple of months after the beginning of the school year. For various reasons—the comprehensive nature of standardized psychological and educational assessments, a lack of resources, and contacting potential programs to refer—administering and interpreting the assessment can take months, with a referral for services occurring in the spring. Without major intervention between fall and spring term, however, student problem behaviors escalate as academic performance declines. It is unfortunate, but not uncommon, that students are not able to participate during the school year in the services for which they were referred. This can be very discouraging for teachers and parents who were hopeful in obtaining support new directions for youth development • DOI: 10.1002/yd HOLISTIC STUDENT ASSESSMENTS 143 for the student to achieve positive change. The length of the assessment session is a noted hurdle in prevention programs in general.9 Furthermore, the lengthy process itself may be especially discouraging for communities in which risks for behavioral and emotional problems are common, where it seems as if all students should receive some sort of intervention. In order to maximize students’ participation in and benefit from the program, earlier detection of risks is essential. Shorter assessments that are simpler to administer allow greater student participation. The ease of administering the assessment would ideally allow everyone in a classroom or school to be screened. Early detection is a core principle of RALLY and is underscored in the first and second articles in this volume. It is no surprise, then, that an assessment in a prevention program needs to be simple so that it can be administered quickly, easily, and universally (given parental permission). A challenge for RALLY was to build an assessment that could be completed within one hour during the school day in a group setting, like a classroom. Furthermore, school psychologists, who typically administer the assessment, are scarce, primarily because a typical school is unable to hire enough. For a school with many students who show emerging risks for behavioral and emotional problems, there simply are not enough psychologists, social workers, or psychiatrists to go around. Thus, another aim was designing an assessment that could be easily administered by less trained professionals (the practitioners). This would free up the time for the school psychologists to address the many student issues that require more extensive and in-depth assessments and to be involved in interpretation rather than data collection. Finally, assessments for all students are useful only when they are easily analyzed and interpretable. This is especially salient when referrals are needed as soon as possible so that the student can take part in the services available. Another consideration in the design to expedite the assessment process is uniting assessment results to appropriate services by developing a set of recommendations based on the interpretation of the outcomes and to deliver the reports to the school staff (for example, the teachers), the school prevention team (the practitioners), and students. An assessment that produced new directions for youth development • DOI: 10.1002/yd 144 YOUTH DEVELOPMENT MEETS MENTAL HEALTH results that are easily understood is crucial for collaborative and joint decision making in the referrals. A simpler assessment can help meet these expectations for ease of referrals at a screening level, with built in further more detailed evaluations when necessary (e.g., neuropsychological testing, etc.). Theoretical considerations in holistic assessment While prevention programs are often theoretically driven, the assessments themselves may not always be as rigorously informed by theory. With the aim of understanding the overall well-being of the child, we became convinced that a developmental approach needed to guide the design of the assessments. Our developmental approach to youth and preventive practice (elaborated in the second article) is based in constructivist social-cognitive theory and clinical-developmental research. It assumes that all students are active meaning makers able to (re)invent their adolescent worlds. This contributes to the development of a student because it opens up the opportunity for new meanings and conceptualizations of the social world. This approach is called social-cognitive because the development of thinking that children, adolescents, and adults go through makes them understand people, relationships, and social situations in ever more complex and mature ways. We describe further why we believe the developmental approach is the underlying component to other factors in the operating model. This operating model, which includes these factors, is illustrated in Figure 7.1. RALLY draws on the social-cognitive development given the research evidence for links between social-cognitive development with risks, resiliency, and social relationships. For instance, socialcognitive skills as measured by self-understanding or moral reasoning are negatively associated with externalizing behavioral problems in childhood, adolescence, and early adulthood.10 Interestingly, research also indicates that more complex development was related to more symptoms of depression and suicidality.11 It is notable that social-cognitive development is related to resiliency as new directions for youth development • DOI: 10.1002/yd HOLISTIC STUDENT ASSESSMENTS 145 well. It has been found that individuals with more complex social cognitions hold greater empathic attitudes. Resilience is considered a normative process,12 and its development seems to conceptually co-occur with social-cognitive development.13 With good reason, researchers have called for harnessing children’s resilience to ameliorate risks for behavioral and emotional problems.14 Relationships are an important factor to assess in a holistic approach as well. While common sense suggests that the way children think about themselves and their way of relating to others has an impact on their behaviors and academic performance, there is also strong research support that draws links between social context with behavioral and academic outcomes. For instance, adolescents may show greater aggressive tendencies, which may be linked to nonresponsive parenting or low social cohesion in the community,15 whereas supportive relationships enhance adolescents’ development.16 This research is in line with the ecological perspective and shows that relationships need to be understood within the various contexts that they reside in. A holistic assessment that identifies the context and its role in the relationship quality provides great clarity Figure 7.1. Four factors in a holistic assessment approach new directions for youth development • DOI: 10.1002/yd 146 YOUTH DEVELOPMENT MEETS MENTAL HEALTH to our understanding of the individual student’s experiences within relationships. Measurement rationale It is important to note that the measures chosen here are not fixed since this is our first step toward instituting a set of ideal measures that best captures the overall well-being and social and academic functioning of the individual youth. In fact, the measures that we present here should be considered as an example set that reflects an overarching holistic framework—that is, one that measures the various factors important in capturing the whole youth: development, resiliency, risks, and relationships. Those who develop a prevention program are encouraged to consider measures that cover these components while also addressing the issues salient to their specific program. In being thoughtful about what should be included in this pilot study, we made a number of decisions to create a holistic assessment that would work best for the RALLY program in particular. Our choice for assessments included a combination of widely used and validated measures with measures we have designed specifically for RALLY. The inclusion of the validated measures ensured that our measurements were as accurate, useful, and reliable as could be given that they were rigorously developed with empirical research. However, it was also important for us to collect elaborated reports of RALLY student experiences to understand their particular opinions and insights. The next article describes each of the measurements we used in a school that implemented a RALLY program in greater detail. Conclusion We developed an assessment model that represents our conceptual approach and measures. We feel that a holistic assessment strikes the right balance in maintaining the theoretically driven but logisnew directions for youth development • DOI: 10.1002/yd HOLISTIC STUDENT ASSESSMENTS 147 tically sound approach that RALLY uses. The holistic assessment best captures the main factors that help us to understand adolescent problems and strengths so that a proper referral can be made, while also ensuring that the process itself is as efficient as possible. The approach is not rigid; as illustrated with the assessment for RALLY, measures may need to be developed to address the particular needs of the students. In summary: • The length of an assessment procedure during the school year slows the process of helping the student obtain appropriate services (early detection as a mean to serve all students’ needs). School and districts are typically backed up for months. • A holistic assessment must make sense based on the clinicaldevelopmental research data (for example, relationships and resiliency are associated). In that way, standardized norms can be developed for the set used in a program. • An assessment for youth should be theoretically driven by a developmental theory that best explains the resiliencies and risks of central concern to the prevention and intervention program. • In general, a holistic assessment considers the overall well-being of the adolescent and seeks to understand the adolescent’s social and emotional development and academic functioning in relation to social relationships. • The measures used in an assessment should match up with the aspects of the adolescent that one is looking to change in a program. • A holistic assessment, also called a screen, can be easily administered by practitioners supervised by a psychologist. • The assessment findings should point out what type of services are needed. This link should be understandable to everyone working with the student. • Those who administer the assessment should show and translate the assessment findings to the referral in order to explain why the service is needed. Further in-depth assessments should be available with the system when the screen uncovers significant or atypical problems. new directions for youth development • DOI: 10.1002/yd 148 YOUTH DEVELOPMENT MEETS MENTAL HEALTH Notes 1. Browne, G., Gafni, A., Roberts, J., Byrne, C., & Majumdar, B. (2003). Effective/efficient mental health programs for school-age children: A synthesis of reviews. Social Science and Medicine, 58, 1367–1384; Ferrer-Wreder, L., Stattin, H., Lorente, C. C., Tubman, J. G., & Adamson, L. (2004). Successful prevention and youth development programs: Across borders. New York: Kluwer. 2. Crone, D. A., Hawken, L. S., & Bergstrom, M. K. (2007). A demonstration of training, implementing, and using Functional Behavioral Assessment in 10 elementary and middle school settings. Journal of Positive Behavior Interventions, 9, 15–29. 3. Weber, K. P., Killu, K., Derby, K. M., & Barretto, A. (2005). The status of Functional Behavioral Assessment (FBA): Adherence to standard practice in FBA methodology. Psychology in the Schools, 42, 737–744. 4. Crone, D. A., & Horner, R. H. (2003). Building positive behavior support systems in schools: Functional behavioral assessment. New York: Guilford Press. 5. Shaffer, D., Scott, M., Wilcox, H., Maslow, C., Hicks, R., Lucas, C. P., et al. (2004). The Columbia SuicideScreen: Validity and reliability of a screen for youth suicide and depression. Journal of the American Academy of Child and Adolescent Psychiatry, 43, 71–79. 6. Achenbach, T. M., & Rescorla, L. A. (2006). Developmental issues in assessment, taxonomy, and diagnosis of psychopathology: Life span and multicultural perspectives. In D. Cicchetti & D. J. Cohen (Eds.), Developmental psychopathology, Vol. 1: Theory and method (pp. 13–180). Hoboken, NJ: Wiley. 7. Malti, T., Liu, C. H.-J., & Noam, G. G. (in press). Holistic assessment in school-based, developmental prevention. Journal of Prevention and Intervention in the Community; Okamoto, S. K., LeCroy, C. W., Tann, S. S., Rayle, A. D., Kulis, S., Dustman, P., & Berceli, D. (2006). The implications of ecologically based assessment for primary prevention with indigenous youth populations. Journal of Primary Prevention, 27, 155–170. 8. Noam, G. G., Pucci, K., & Foster, E. (1999). Development, resilience, and school success in youth: The prevention-practitioner and the Harvard-RALLY program. In D. Cicchetti & S. Toth (Eds.), Developmental psychopathology: Developmental approaches to prevention and intervention (pp. 424–464). Hoboken, NJ: Wiley. 9. Maike, M. M., & Nixon, A. (2007). Tacoma public schools. Responsive advocacy for life and learning in youth: Rally. Unpublished report, Tacoma public schools; Ferrer-Wreder et al. (2004); Kern, L., Hilt, A. M., & Gresham, F. (2004). An evaluation of the Functional Behavioral Assessment process used with students with or at risk for emotional and behavioral disorders. Education and Treatment of Children, 27, 440–452. 10. Krettenauer, T., Ullrich, M., Hofmann, V., & Edelstein, W. (2003). Behavioral problems in childhood and adolescence as predictors of ego-level attainment in early adulthood. Merrill-Palmer Quarterly, 49, 125–153; Lochman, J. E., & Wells, K. C. (2002). Contextual social–cognitive mediators and child outcome: A test of the theoretical model in the Coping Power program. Development and Psychopathology, 14, 945–967. new directions for youth development • DOI: 10.1002/yd HOLISTIC STUDENT ASSESSMENTS 149 11. Noam, G. G., Paget, K., Valiant, G., Borst, S., & Bartok, J. (1994). Conduct and affective disorders in developmental perspective: A systematic study of adolescent psychopathology. Development and Psychopathology, 6, 519–532. 12. Masten, A. S. (2001). Ordinary magic: Resilience processes in development. American Psychologist, 56, 227–238. 13. Cicchetti, D., Rappaport, J., Sandler, I., & Weissberg, R. P. (Eds.). (2000). The promotion of wellness in children and adolescents. Washington, DC: CWLA Press. Masten, A. S. (2007). Resilience in developing systems: Progress and promise as the fourth wave rises. Development and Psychopathology, 19, 921–930. 14. Daniel, B., & Wassell, S. (2002). Adolescence: Assessing and promoting resilience in vulnerable children. London: Jessica Kingsley Publishers; Masten, A. S., Burt, K. B., & Coatsworth, J. D. (2006). Competence and psychopathology in development. In D. Cicchetti & D. J. Cohen (Eds.), Developmental psychopathology (pp. 696–738). Hoboken, NJ: Wiley; Masten, A. S., & Coatsworth, J. D. (1998). The development of competence in favorable and unfavorable environments: Lessons from research on successful children. American Psychologist, 53, 205–220; Noam, G. G. (1992). Development as the aim of clinical intervention. Development and Psychopathology, 4, 679–696; Weissberg, R. P., Kumpfer, K. L., & Seligman, M.E.P. (2003). Prevention that works for children and youth: An introduction. American Psychologist, 58, 425–432. 15. Lochman, J. E. (2006). Translation of research into interventions. International Journal of Behavioral Development, 30, 31–38. 16. Noam, G. G., & Hermann, C. (2002). Where education and mental health meet: Developmental prevention and early intervention in schools. Development and Psychopathology, 14, 861–875. CINDY H. LIU is a postdoctoral fellow of clinical psychology at Harvard Medical School–Children’s Hospital of Boston. TINA MALTI is a visiting research scientist at McLean Hospital and Harvard University. GIL G. NOAM is the founding director of the RALLY Program and associate professor of developmental psychopathology at McLean Hospital and Harvard University. new directions for youth development • DOI: 10.1002/yd