Evaluating effectiveness in adolescent mental health inpatient units: A systematic review

Title Page Manuscript Category: Review Article ii. A short informative title that contains the major key words. The title should not Author Manuscript i. contain abbreviations. Evaluating Effectiveness in Adolescent Mental Health Inpatient Units: A Systematic Review. iii. The full names of the authors: Claire Hayes, Magenta Simmons, Christine Simons and Malcolm Hopwood. iv. The author’s institutional affiliations at which the work was carried out: Claire Hayes: The Department of Psychiatry, The University of Melbourne, Melbourne, Australia. The Albert Road Clinic, Melbourne, Australia. Magenta Simmons: The Centre for Youth Mental Health, The University of Melbourne, Melbourne, Australia. Orygen Youth Health Research Centre, Parkville, Melbourne, Australia. Christine Simons: The Department of Psychiatry, The University of Melbourne, Melbourne, Australia. The Albert Road Clinic, Melbourne, Australia. Malcolm Hopwood: The Department of Psychiatry, The University of Melbourne, Melbourne, Australia. The Albert Road Clinic, Melbourne, Australia. v. An authorship declaration: in keeping with the latest guidelines of the International Committee of Medical Journal Editors, each author’s contribution to the paper is to be quantified. Claire Hayes, Magenta Simmons, Christine Simons and Malcolm Hopwood designed the protocol for the systematic review. Claire Hayes undertook the systematic review and drafted the article. Magenta Simmons undertook double screening. Claire Hayes, Magenta Simmons, Christine Simons and Malcolm Hopwood supervised the design, reviewed the results, and revised the article. All authors read drafts regularly. All authors read and approved the final version of the manuscript. vi. The full postal and email address, plus telephone number, of the author to whom correspondence about the manuscript should be sent. Claire Hayes, Suite A, Albert Road Clinic, 31 Albert Road, Melbourne, 3004, Victoria, Australia. (03) 9256 8331. This is the author manuscript accepted for publication and has undergone full peer review but has not been through the copyediting, typesetting, pagination and proofreading process, which may lead to differences between this version and the Version of Record. Please cite this article as doi: 10.1111/inm.12418 This article is protected by copyright. All rights reserved vii. Authorship statement. This must acknowledge i) that all authors listed meet the authorship criteria according to the latest guidelines of the International Committee of Medical Journal Editors, and ii) that all authors are in agreement with the manuscript. Author Manuscript All of the authors meet the authorship criteria according to the latest guidelines of the International Committee of Medical Journal Editors. We all agree to this manuscript. viii. Acknowledgements. The source of financial grants and other funding should be acknowledged, including a frank declaration of the author’s industrial links and affiliations. The contribution of colleagues or institutions should also be acknowledged. Thanks to anonymous reviewers are not allowed. This research is supported by the Ramsay Healthcare Ella Lowe Scholarship. ix. Disclosure statement. Authors must declare any financial support or relationships that may pose conflict of interest. This includes any financial arrangements authors have with a company whose product figures prominently in the submitted manuscript or with a company making a competing product. We, the authors declare that there is no conflict of interest. x. Word count, including abstract and acknowledgements, but not table or figure legends and references. 4472 This article is protected by copyright. All rights reserved : Review Article Author Manuscript Article type Corresponding author mail id : chayes2@student.unimelb.edu.au Abstract Adolescent mental health research is a developing area. Inpatient units are the most widely used acute element of adolescent mental health services internationally. Little is known about inpatient units, particularly when it comes to measuring improvement for adolescents. Clinical outcome measurement in the broad context has gathered momentum in recent years, driven by the need to assess services. The measurement of outcomes for adolescents who access inpatient care is critical, as they are particularly vulnerable and are often considered the most difficult to treat. The aim of this review was to assess if adolescent inpatient units are effective and understand how outcomes are measured. CINAHL, MEDLINE with Full Text, ERIC, PsychINFO and Cochrane databases were systematically searched. Studies were included if the inpatient units were generic and adolescents were between the mean age of 12-25 years. Furthermore, studies published in English within the last ten years were included. Exclusions were outpatient and disorder specific inpatient settings. A total of 16 studies were identified. Each study demonstrated effectiveness on at least one outcome measure in terms of symptom stabilisation. However, several outcome measures were used and therefore inpatient units lack consistency in how they measure improvement. Inpatient units are effective for the majority of young people as they result in symptom stabilisation. Whilst symptom stabilisation can be achieved, future research examining the mechanism of change is needed. Introduction Globally, mental illness among adolescents is a critical health concern. The majority of mental health problems have their peak period of incidence during adolescence (Merikangas et al., 2009; Rickwood et al., 2015). Approximately one in five adolescents experience serious mental health problems, accounting for an estimated 13% of the total This article is protected by copyright. All rights reserved burden of disease (World Health Organization, 2013). Suicide is the second leading cause of mortality among young people between the ages of fifteen to twenty-nine (World Health Organization, 2014). Furthermore, adolescents experiencing mental Author Manuscript health problems are more likely to have suicidal thoughts or engage in suicidal ideation (Allen and McKenzie, 2015; Vander Stoep et al., 2009). Given the vulnerability of this population, it is no surprise that admissions to Inpatient Units (IPUs) are often required. IPUs are used for symptom stabilisation in the event of psychiatric crisis (Goldman et al., 1998; Tharayil et al., 2012). IPU admissions often occur when adolescents and their caregivers are unable to manage the young person’s mental health symptoms at home. Consequently, more intense professional support is required (Sadock et al., 2007; Smith et al., 2015). According to Kronstrom et al. (2016), adolescents being treated in IPUs are considered to be the most distressed in society. As well as having severe mental health disorders, they often have psychosocial risk factors and a history of traumatic life events (Case et al., 2007). Adolescent IPUs introduce intensive interventions for complex disorders over a defined period (Green and Worrall-Davies, 2009). These IPU interventions are often associated with high costs, particularly when clients are admitted for several weeks or months. Consequently, it is important to establish the effectiveness of adolescent IPUs, not only in terms of cost effectiveness, but more importantly, to establish the outcomes for young people and their families. As mental health demands increase, many inpatient services have been encouraged to reduce costs with a parallel pressure to measure outcomes and effectiveness (Hall et al., 2014). In terms of this review, positive outcomes are related to sustained health improvement of young people. The demand for evidence-based practice (EBP) underpins the need to examine valid and reliable outcome measures, which capture changes in symptoms, functioning and determine effectiveness within the context of short-term interventions (Tharayil et al., 2012; Koch et al., 1998). Measuring this change and assessing outcomes provides an evaluation of the service as a whole, as well as highlighting areas for improvement (Yuan, 2015). Whilst this is undoubtedly important, it’s perceived difficult to implement and measure outcomes. It is difficult to measure outcomes in IPU settings, particularly when admitting different types of This article is protected by copyright. All rights reserved adolescents with various clinical problems and at various time-points. In addition, every IPU is different in terms of typical length of stay (LOS), admission policies, organisational cultures and types of interventions offered. There is a significant time Author Manuscript constraint on what can be offered, particularly for short stays of 1-7 days. Each IPU will have a unique model of care, whether it is designed to provide safety and containment within 1-7 days, or perhaps more long term therapy provided within 6-8 weeks. Either way, an inpatient admission is a critical point in a person’s life and it is crucial that clinicians and researchers understand how effective they are. One key paper by Pfeiffer and Strzelecki (1990) reviewed outcome studies for adolescents in IPU treatment. The review examined the literature from 1975- 1990 and included the results of 34 studies. These papers included children (n=6), adolescents (n=17) and combined children and adolescents (n=11). Studies consisted of both qualitative and quantitative data. 23 studies were designed as post-discharge (followup), and the remaining 11 examined patient status at the time of discharge. This review combined follow-up and outcome. Results found that inpatient admissions were often beneficial, particularly if a specialised treatment program and aftercare were available. This review found only four studies, which examined various aspects of inpatient interventions, such as therapeutic alliance, planned discharge, completion of treatment program and the efficacy of a cognitive-based problem-solving skills training package (Gossett et al., 1977; John F. Clarkin et al., 1987; Kazdin et al., 1987; White et al., 1979). In all four studies, these interventions yielded positive outcomes and predicted a favourable post-discharge status. In terms of aftercare, only four studies were found reporting the relationship of aftercare and the post-discharge environment to outcome (Cohen-Sandler et al., 1982; Gossett et al., 1977; Koret, 1980; Stewart and Leone, 1978). All four found a strong positive association. Furthermore, the study found that clients with less symptom severity had more favourable outcomes. Age and sex were not associated with favourable outcomes, and IQ and LOS yielded only a modest relationship to outcome. Similarly to the previous key paper, Blanz and Schmidt (2000) cautiously convey that inpatient admissions can be beneficial, especially when special aspects of treatment This article is protected by copyright. All rights reserved interventions are fulfilled. These include those outlined previously, which include positive therapeutic alliance, planned discharge, completion of a cognitive-based problem-solving skills training package and aftercare. Healthier clients have more Author Manuscript desirable outcomes, particularly those with adequate intelligence, later onset of symptoms and non-psychotic diagnoses. Poor family functioning was considered to be an important factor in the outcome of hospitalised adolescents. However, the review found methodological limitations for several studies, making interpretation of results challenging. Lack of research relating to LOS and outcome information was outlined, as well as factors influencing decisions to admit adolescents to IPUs. Finally, descriptions of models of care in IPU settings remains complex and somewhat mysterious. In the absence of this information, researchers and clinicians alike are unable to distinguish the relationships between a particular model of care and certain outcome variables. Consequently, future studies need to characterise the key components of successful IPU treatment and identify those who respond favourably. In order for this to occur, there needs to be more clarity in relation to how outcomes are measured for young people. Research in this area is accumulating, however, to the researcher’s knowledge, no systematic reviews have been conducted in the last sixteen years. Given that adolescent IPUs would have changed during this time, there is limited up-to-date research to draw any firm conclusions on the effectiveness of IPUs. To address this research gap and update current practice, the main aim of this review was to assess if generic adolescent IPUs are effective. A secondary aim was to understand ways in which outcomes are measured and what domains are measured. Methods This review utilised the PICOT (population, intervention, control/comparison, outcomes and time) format to frame the research question. A search was undertaken to identify relevant literature pertaining to adolescents, IPUs, mental health and treatment outcomes. Searches of CINAHL (Cumulative Index to Nursing and Allied Health), MEDLINE (Medical Journals), ERIC (Education Resources Information Center), This article is protected by copyright. All rights reserved PsychINFO and Cochrane were undertaken. The literature search was limited to studies from the January 2006 to June 2017. The years were limited to ensure the review was based on contemporary practice in adolescent IPUs. Search terms consisted of five Author Manuscript concepts, which included adolescent, inpatient, mental health setting, treatment outcome and change (see Table 1). Searches were based on article titles, abstracts, subjects and further studies were selected through hand searching the references of relevant studies and reviews. Articles considered for inclusion were randomly double screened by a coauthor. The search methodology and reported results adhere to the relevant sections of the Preferred Reporting Items for Systematic Reviews and Meta-Analyses (PRISMA) statement. The review was retrospectively registered with the International Prospective Register of Systematic Reviews (PROSPERO). Insert Table 1 Here Table 1 Search Strategy Inclusion Criteria      The setting is a generic adolescent inpatient unit. Mean age of participants between 12 and 25. Written in English with full text available. Published between 2006 and 2017. Qualitative and Quantitative Studies. Exclusion Criteria  Disorder specific settings (Eg. Eating Disorder units, substance abuse units, Bipolar Disorder units, psychosis units, Post-Traumatic Stress Disorder units).   Community, outpatient and/or forensic settings. Intellectual disability population. This article is protected by copyright. All rights reserved Data Extraction Following the initial search, duplicates were removed. Titles and abstracts were then screened for excluded studies and consequently removed. Once potential relevant Author Manuscript articles were identified, the first author inspected these against eligibility criteria for inclusion. Figure 1 illustrates the PRISMA flow diagram for inclusion. Data extraction was undertaken from included articles and the following variables were entered into a spreadsheet: age, gender, ethnicity, socioeconomic status, country, diagnoses, sample size, research design, setting, treatment outcomes and follow-up time. Insert Figure 1 here Fig. 1 PRISMA flow diagram of systematic search procedures, yielding 16 quantitative studies Results Search Results Despite including qualitative studies in the search strategy, 16 quantitative studies were identified relating to adolescent mental health treatment outcome measurement in generic IPUs (see Figure 1). The studies examined mental health treatment outcomes following episodes of inpatient care. The studies predominantly utilised empirical, prospective cohort research designs. Three studies adopted retrospective designs, whilst two were national outcome studies and a quality improvement project. Outcomes were rated from a range of perspectives including clients, caregivers and clinicians. Whilst some studies investigated one single adolescent IPU, others depicted results from several units. One study examined treatment outcomes for persons with and without intellectual disabilities. For the purpose of this review and in line with our exclusion criteria, results are reported only on those without intellectual disabilities. None of the identified studies used control groups. Description of Studies and Settings The details of the included studies are presented in Table 2. The majority of studies were from the US (n=4) and the UK (n=4). Two studies were from Australia and one This article is protected by copyright. All rights reserved from New Zealand, whilst the remaining were from Canada, Japan, Turkey, Switzerland and Norway. Twelve studies presented single IPUs, whilst the remaining included multiple units. Of those who reported, the number of inpatient beds at each IPU ranged Author Manuscript from 6 beds to 110, with the lowest from a single IPU in the UK and the latter from a single IPU in the US. Of ten studies which reported on attached services, five had attached outpatient services, whilst five had attached schools. One of these studies had both outpatient services and a school. The inpatient admission inclusion criteria was outlined in five studies. These included the following: a diagnosis of at least one DSMIV1 Axis 1 disorder, acute suicidal thoughts, psychosis, severe PTSD and anxiety. Another study described severe hyperactivity, violence to self or others, whilst another was imminent threat to harm self or others. Two studies outlined exclusion criteria for inpatient admissions. Those who were excluded were clients with a conduct disorder alone, intellectual disability, Eating Disorders or substance abuse. In terms of staff, nine studies described that the unit/s were staffed by the multi-disciplinary team (MDT). The type of treatment intervention/s provided at each setting were not described in seven studies. Of those, which did report, treatment interventions consisted of the following: Milieu Therapy, Individual and Group Psychotherapy, Psychoeducation, Family Therapy, Music Therapy, Recreational and Sporting Activities. Other interventions included illness and self-awareness education, anger and stress management, problem solving, medication as prescribed and artistic activities. In terms of treatment provided, school related activities were described in a number of studies (n=4). Insert Table 2 Here Table 2 List of Included Studies and Key Characteristics Details of Adolescent Participants’ Characteristics The majority of participants were females (87.5%) and mood disorders (62.5%) were the most common primary diagnoses of adolescents. LOS varied considerably between each IPU ranging from lowest mean of 4 days to the highest mean of 335.4 days. The voluntary versus involuntary admission status of adolescents were not mentioned in 1 Diagnostic and Statistical Manual of Mental Disorders. 4th Edition. This article is protected by copyright. All rights reserved thirteen studies. For those studies that did report on status, all three reported that the Author Manuscript majority in each IPU were admitted voluntarily. Details of Outcome Measures Each study utilised a variety of assessment tools to measure different outcomes (see Table 3). The most frequently cited measures included the Health of the Nation Outcomes Scale for Children and Adolescents (HoNOSCA) (n=7), Children’s Global Assessment Scale (CGAS) (n=4), Youth Self Report (YSR) (n=3), Children’s Depression Inventory (CDI) (n=3) and the Global Assessment of Functioning (GAF) (n=2). The domains which were measured mainly consisted of the following: various areas of functioning, relationships, symptom severity related to depression, anxiety, psychosis and mania, as well as risk factors. Other domains included internalising and externalising problems as well as academic abilities. All of the studies utilised clinician reports, nine included client reports in addition to clinicians, whilst three included caregivers, as well as clients and clinicians. In terms of timing, thirteen studies measured outcomes on admission and discharge. Only one study measured outcomes at the time of agreeing to be admitted, admission, discharge and one year post discharge. Three studies examined some outcomes at admission, during treatment and discharge. During treatment included 4 weeks after admission (Green et al., 2007), 14 days after admission (Greenham and Bisnaire, 2008) and the remaining study did not indicate the time-points (Herdzik, 2009). Insert Table 3 Here Table 3 Descriptions of Included Outcome Measures, Domains Measured, Reporters, Follow-Up Times and Improvement Values Discussion We have systematically identified and appraised the studies with the aim to understand if generic adolescent IPUs are effective. A secondary outcome was to assess how adolescent treatment outcomes are measured and what domains are measured. This article is protected by copyright. All rights reserved In terms of how adolescent treatment outcomes are measured, each IPU utilized various outcome measures, at various times and with various reporters. Therefore, IPUs are inconsistent with measuring outcomes, thus limiting reliable comparisons to be made Author Manuscript with other adolescent populations and services. The absence of set routine outcome measures in IPU settings indicate the diversity between each setting, their clientele and what domains are a priority. One needs to consider whether outcome measures for each IPU are selected based on available resources rather than measures designed specifically for IPU settings. For instance, HoNOSCA was commonly used in several studies. One item of this scale measures school attendance, yet most adolescents will not be attending school during an inpatient admission. Therefore, scores on discharge could be considered irrelevant, as the client was admitted during the rating period and thus could not attend school. Ideally, IPUs should measure a multitude of domains, as the needs of clients are varied. Measures need to be chosen based on the unique needs of the population utilising the IPU. Furthermore, measures should be adopted, which specifically measure the impact of adolescent IPU treatment. Mental health care, unlike physical care, does not have consistent measures available to reliably monitor client health and alert clinicians to negative responses to treatment (Newnham et al., 2010). This creates significant concerns for those who are engaged in treatment. Although there is no fixed pattern of change for all clients, monitoring progress provides clinicians with the opportunity to improve mental health outcomes in real time for the benefit of each particular young person (Lambert et al., 2005; Lutz, 2003; Lutz et al., 2009). Discharging from an IPU can create a variety of emotions for adolescents. Whilst some may be keen to leave, others may experience anxiety at the thought of leaving a containing environment. Others might experience anger as a result of feeling abandoned or rejected. With this in mind, these emotions are likely to influence self-reports. A number of client-rated measures indicated smaller effect sizes than clinicians. One could argue which report is more accurate, as clinicians could be subject to performance bias. Only one of the included studies examined outcomes one year post discharge. This study found that positive outcomes were sustained for the majority of adolescents oneyear post discharge. The ‘cooling off’ period between discharge and one year post- This article is protected by copyright. All rights reserved discharge could remove the raw emotions surrounding discharge, thus producing more reliable findings. Author Manuscript In terms of effectiveness, most of the studies found clinically and statistically significant benefits for the majority of clients across various domains. However, each IPU would have had various client cohorts depending on countries and regions, admission policies, organisational cultures and varying LOS. For instance, Setoya et al. (2011) had a LOS of 11 months. This is a considerable length of time for a hospital admission and one would expect some improvement across this time. However, regardless of LOS, changes in symptom severity were found from admission to the follow-up time point for a number of studies. For instance, despite a short LOS of 4 or 6 days, clients improved (Barnes, 2009; Greenham and Bisnaire, 2008). Guvenir et al. (2009) suggest a dosage effect where exposure to admission is an active ingredient in change. Few studies explore this further and in the absence of Randomized Controlled Trials (RCTs) or follow-up data post discharge, it is difficult to determine. In addition, one needs to question whether each outcome measurement assessment in mental health is an intervention in itself. Quality of Evidence In terms of sample size, there were 6 studies with a sample of less than 100, which could be considered small. Only 50% (n=8) of studies described the selection criteria for the study population. In 14 studies, not all eligible participants that met the prespecified criteria were enrolled. The response rates varied across studies, however of those who reported (n=13), several presented a response rate less than 50% at discharge (n=6). On the contrary, the one study with one year follow-up found a response rate of 78% (Green et al., 2007). In 7 studies, there was insufficient details describing the interventions provided and whether these were delivered consistently across the study population. None of the individuals assessing the outcomes were blind to the interventions provided. In addition, none of the studies carried out multiple outcome assessments prior to the intervention. For all studies, blinding of participants was difficult due the This article is protected by copyright. All rights reserved nature of the adolescent inpatient settings and interventions. In addition, a number of outcome measures were based on subjective self-reports, thus performance bias are likely to have been present. Clinicians could be considered biased, when it comes to Author Manuscript rating their own work. In table 4, the identified evidence is provided (National Institutes of Health, 2014). Several limitations in the quality of evidence are immediately evident. A risk of bias assessment was conducted for each study based on the National Institutes of Health (2014) assessment tool. This tool is designed to assess the quality of before-after (prepost) studies with no control group. Based on this assessment tool, the risk of bias was poor to fair. Limitations This review is not without limitations. The eligibility criteria excluded articles not written in English, thus inpatient outcome studies for other cultures were excluded. Article authors were not contacted for further information. The studies included in the review all have methodological limitations. Firstly, none of the studies were RCTs. Several studies utilised a variety of clinically reliable measures, which are widely used in youth mental health care. However, the measures lacked consistency across studies, making comparisons difficult. This reflects the lack of a universal adolescent measure for IPU treatment. In addition, considering clinicians rated the majority of measures in each study, performance and rater-bias might have prevailed. A number of studies (n=6) lacked adequate sample sizes for statistical power, with sample sizes of less than 100 (Burmeister and Aitken, 2012). In addition, studies have limited systematic information on diagnoses, pharmacotherapy, symptomatology and demographics. Therefore, the ability to make firm recommendations based on the available evidence is limited. Insert Table 4 Here Table 4 Risk of Bias Assessment This article is protected by copyright. All rights reserved Conclusion The studies described support that IPU admissions are indeed helpful for adolescents. Author Manuscript Despite the obvious demand for EBP in adolescent IPUs, the evidence base is vague and characterised by studies with small sample sizes and heterogeneous research designs. In particular, there is a paucity of well-conducted studies such as those of a longitudinal design with large sample sizes, RCTs and multiple measure informants, particularly clients. However, this is not particularly surprising as research in adolescent inpatient settings is limited. This is often limited as a result of the challenges which present when studying young people. For instance, access to services given the vulnerability of the population. Measuring the efficacy of adolescent IPUs is difficult. These challenges include the use of comparison and control groups in such studies. Who would be an appropriate control group? How would the IPU treatment be measured? Given these difficulties, it is important to consider alternate methods of evaluation, when measuring outcomes. One way would be to standardize outcome measures and domains for adolescent IPU settings. Heterogeneity of the problems and treatments required, in addition to difficulties in conducting blind ratings provide further challenges. IPUs can serve for symptom stabilisation within a short stay. The inpatient admission itself can offer containment, as part of overall effectiveness, through a combination of removal from stressful environments, intensive 24-hour care and support and/or positive effects of group milieu (Green et al., 2007). However, there is uncertainty in relation to the effective components of interventions, LOS, suitability and whether positive effects gained are maintained post-discharge (Green et al., 2007). Furthermore, whilst this review demonstrates that mental health stabilisation can be achieved following an episode of inpatient care, we are unsure as to ‘how’ and ‘why’ this is achieved. There are inconsistencies in how each IPU measures outcomes and who completes the ratings. This review presents the combined data from over 6,500 adolescent inpatients, demonstrating symptomatic improvement following IPU treatment across measures and raters. This suggests evidence of symptomatic improvement as a result of IPU This article is protected by copyright. All rights reserved treatment, however cannot evaluate how IPU treatment would compare to treatments provided in other settings with similar populations. The complex needs of adolescents in crisis makes research into the mechanisms of change and specific IPU treatments Author Manuscript challenging, but a worthwhile endeavour yet to be performed sufficiently. Relevance for Clinical Practice One of the aims of this review was to assess if adolescent IPUs are effective. The findings confirm that adolescent IPUs are effective for the majority of young people. This informs mental health nursing practice by assuring nurses that the work, which they pursue contributes to improvement. This is an extremely positive and powerful affirmation for mental health nurses, who constitute the bulk of the IPU workforce. Such information is important for mental health nurses and managers alike to be aware of, and can enhance morale and positively influence clinical practice. This review also sought to establish how outcomes were measured in adolescent IPU settings. Whilst it is unclear which clinicians completed each of the outcome measures on IPUs, we can assume that mental health nurses completed the majority. This is a key contribution, which mental health nurses play in clinical practice on a daily basis. Given that outcome measures are often used to determine the efficacy of a particular service, there should be more value placed on this role for mental health nurses. This article is protected by copyright. All rights reserved References Author Manuscript Allen KA and McKenzie VL. (2015) Adolescent Mental Health in an Australian Context and Future Interventions. International Journal of Mental Health 44: 80-93. Barnes LE. (2009) Relationship of age group, gender, and race to basis-24 subscales for adolescent psychiatric inpatients: An exploratory study. Dissertation Abstracts International: Section B: The Sciences and Engineering 69: 7128. Blanz B and Schmidt MH. (2000) Practitioner Review: Preconditions and Outcome of Inpatient Treatment in Child and Adolescent Psychiatry. Journal of Child Psychology and Psychiatry 41: 703-712. Bobier C, Dowell J and Swadi H. (2009) An examination of frequent nursing interventions and outcomes in an adolescent psychiatric inpatient unit. International Journal of Mental Health Nursing 18: 301-309. Burgess P, Pirkis J and Coombs T. (2009) Modelling candidate effectiveness indicators for mental health services. The Australian And New Zealand Journal Of Psychiatry 43: 531-538. Burmeister E and Aitken LM. (2012) Sample size: how many is enough? Australian Critical Care: Official Journal Of The Confederation Of Australian Critical Care Nurses 25: 271-274. Case BG, Olfson M, Marcus SC, et al. (2007) Trends in the inpatient mental health treatment of children and adolescents in US community hospitals between 1990 and 2000. Archives of General Psychiatry: 89. Chaplin R, Roach S, Johnson H, et al. (2015) Inpatient Children and Adolescent Mental Health Services (CAMHS): outcomes of young people with and without intellectual disability. Journal Of Intellectual Disability Research: JIDR 59: 995-998. Cohen-Sandler R, Berman AL and King RA. (1982) A Follow-up Study of Hospitalized Suicidal Children. Journal of the American Academy of Child & Adolescent Psychiatry 21: 398-403. This article is protected by copyright. All rights reserved Duddu V, Rhouma A, Qureshi M, et al. (2016) An acute in-patient psychiatric service for 16- to 17-year-old adolescents in the UK: a descriptive evaluation. Bjpsych Bulletin 40: 261-265. Author Manuscript Goldman W, McCulloch J and Sturm R. (1998) Costs and use of mental health services before and after managed care. Health Affairs 17: 40-52 13p. Gossett JT, Barnhart D, Lewis JM, et al. (1977) Follow-up of adolescents treated in a psychiatric hospital: Predictors of outcome. Archives of General Psychiatry 34: 1037-1042. Green J, Jacobs B, Beecham J, et al. (2007) Inpatient treatment in child and adolescent psychiatry--a prospective study of health gain and costs. Journal Of Child Psychology And Psychiatry, And Allied Disciplines 48: 1259-1267. Green J and Worrall-Davies A. (2009) Provision of Intensive Treatment: In-Patient Units, Day Units and Intensive Outreach. Rutter's Child and Adolescent Psychiatry. Blackwell Publishing Ltd., 1126-1142. Greenham SL and Bisnaire L. (2008) An Outcome Evaluation of an Inpatient Crisis Stabilization and Assessment Program for Youth. Residential Treatment for Children & Youth 25: 123-143. Guvenir T, Varoltas F and Ozbek A. (2009) Child and adolescent mental health inpatient services in Turkey: is there a need and are they effective?/Turkiye'de cocuk ve ergen ruh sagligi yatakli servisleri, gerekli mi ve ise yariyor mu? Archives of Neuropsychiatry: 143. Haggerty G, Kahoud D, Walsh E, et al. (2013) A brief inpatient measure of global psychiatric symptom severity: initial validation of the brief symptom measure25 for an adolescent inpatient sample. Journal of Nervous & Mental Disease 201: 971-976. Hall CL, Moldavsky M, Taylor J, et al. (2014) Implementation of routine outcome measurement in child and adolescent mental health services in the United Kingdom: a critical perspective. European Child & Adolescent Psychiatry: 239. Hanssen-Bauer K, Heyerdahl S, Hatling T, et al. (2011) Admissions to acute adolescent psychiatric units: a prospective study of clinical severity and outcome. International Journal Of Mental Health Systems 5: 1-1. This article is protected by copyright. All rights reserved Herdzik KM. (2009) Brief inpatient psychiatric treatment of adolescents: Outcomes and effectiveness. Dissertation Abstracts International: Section B: The Sciences and Engineering 69: 5028. Author Manuscript John F. Clarkin, Stephen W. Hurt, and and Janet L. Crilly. (1987) Therapeutic Alliance and Hospital Treatment Outcome. Psychiatric Services 38: 871-875. Kazdin AE, Esveldt-Dawson K, French NH, et al. (1987) Problem-solving skills training and relationship therapy in the treatment of antisocial child behavior. J Consult Clin Psychol 55: 76-85. Koch JR, Lewis A and McCall D. (1998) A multistakeholder-driven model for developing an outcome management system. Journal of Behavioral Health Services & Research 25: 151. Koret S. (1980) Follow-up study on residential treatment of children, ages six through twelve. J Natl Assoc Priv Psychiatr Hosp 11: 43-47. Kronstrom K, Ellila H, Kuosmanen L, et al. (2016) Changes in the clinical features of child and adolescent psychiatric inpatients: A nationwide time-trend study from Finland. Nordic Journal of Psychiatry 70: 436-441. Lambert MJ, Harmon C, Slade K, et al. (2005) Providing feedback to psychotherapists on their patients' progress: Clinical results and practice suggestions. Journal of Clinical Psychology 61: 165-174. Lutz W. (2003) Efficacy, Effectiveness, and Expected Treatment Response in Psychotherapy. Journal of Clinical Psychology 59: 745-750. Lutz W, Stulz N and Köck K. (2009) Research report: Patterns of early change and their relationship to outcome and follow-up among patients with major depressive disorders. Journal of Affective Disorders 118: 60-68. Mathai J and Bourne A. (2009) Patients who do well and who do less well in an inpatient adolescent unit. Australasian Psychiatry 17: 283-286. Merikangas KR, Nakamura EF and Kessler RC. (2009) Epidemiology of mental disorders in children and adolescents. Dialogues in Clinical Neuroscience 11: 720. National Institutes of Health. (2014) Quality Assessment Tool for Before-After (PrePost) Studies With No Control Group, Study Qual. Assess. Tools. Available at: This article is protected by copyright. All rights reserved https://www.nhlbi.nih.gov/health-pro/guidelines/in-develop/cardiovascular-riskreduction/tools/before-after. Newnham EA, Hooke GR and Page AC. (2010) Research report: Progress monitoring Author Manuscript and feedback in psychiatric care reduces depressive symptoms. Journal of Affective Disorders 127: 139-146. Pfeiffer SI and Strzelecki SC. (1990) Inpatient psychiatric treatment of children and adolescents: a review of outcome studies. Journal of the American Academy of Child and Adolescent Psychiatry: 847. Rickwood DJ, Mazzer KR, Telford NR, et al. (2015) Changes in psychological distress and psychological functioning in young people visiting headspace centres for mental health problems,. Medical Journal of Australia, 202: 537-542. Sadock BJ, Kaplan HI and Sadock VA. (2007) Kaplan & Sadock's Synopsis of Psychiatry: Behavioral Sciences/clinical Psychiatry: Wolter Kluwer/Lippincott Williams & Wilkins. Setoya Y, Saito K, Kasahara M, et al. (2011) Evaluating outcomes of the child and adolescent psychiatric unit: A prospective study. International Journal Of Mental Health Systems 5: 7-14. Smith L, Strümpher J and Morton DG. (2015) Perceptions of mental healthcare professionals regarding inpatient therapy programmes for adolescents in the Eastern Cape, South Africa. Journal of Child & Adolescent Mental Health 27: 59-73. Sperbeck DJ and Mayo M. (2016) Measuring child and adolescent psychiatric treatment outcomes: The development and validation of the Symptom Acuity Rating Scale. Journal of Projective Psychology & Mental Health 23: 82-87. Stewart MA and Leone L. (1978) A family study of unsocialized aggressive boys. Biol Psychiatry 13: 107-117. Tharayil PR, James S, Morgan R, et al. (2012) Examining outcomes of acute psychiatric hospitalization among children. Social Work in Mental Health 10: 205-232. Urben S, Pihet S, Graap C, et al. (2015) Clinical Utility of the 2 New Scales of the Health of the Nation Outcome Scales for Children and Adolescents (HoNOSCA): A Naturalistic, Prospective Study in a Psychiatric Unit for Adolescents. Journal of Psychiatric Practice 21: 232-240. This article is protected by copyright. All rights reserved Vander Stoep A, McCauley E, Flynn C, et al. (2009) Thoughts of Death and Suicide in Early Adolescence. Suicide and Life-Threatening Behavior 39: 599-613. White TA, Benn R, Gross D, et al. (1979) Assessing the need for follow-up: The Author Manuscript relationship of prognosis to posthospitalization adjustment. Child Psychiatry and Human Development 10: 91-102. World Health Organization. (2013) Mental health action plan 2013–2020,. Geneva. World Health Organization. (2014) Global Health Estimates 2013: Deaths by Cause, Age and Sex, Estimates for 2000-2012. In: Organization WH (ed). Geneva: World Health Organization. Yuan. (2015) HoNOSCA in an adolescent psychiatric inpatient unit: An exploration of outcome measures. Psychiatria Danubina 27: S357-S363. This article is protected by copyright. All rights reserved Tables Author Manuscript Table 1 Search Strategy Search Strategy adolescent* or “young person*” or “youth*” or “young adult*” or teen* or teenager* inpatient* or "in-patient*" or client or clients or clientele or patient* or "service user*” “mental health setting*” or "inpatient unit*" or in-patient unit*" or hospitalisation or hospitalization or hospitalised or hospitalized or "acute care" or admission or admissions “treatment outcome*” or “routine outcome measur*” or “routine outcome*” or measur* or assess* or rate* or tool* or checklist* or screen* or scale* or efficac* or effect* or evalu* change* or improve* or progress* or deterior* *Is a wildcard character that may be used in place of any number of characters in a search word. This article is protected by copyright. All rights reserved Author Manuscript Table 2 Included Studies and Key Characteristics Author Research (s) Design Country Setting Treatment (year) (Barnes Empirical, , 2009) Quantitativ e Study Sample Diagnosis Mean Age Gender Admission Length of Size % of sample (M) (%) Status Stay (LOS) Standard Females Voluntary Days Deviation (F) (V) Mean & (SD) Males Involunta Standard (M) ry (I) Deviation N= (SD) US Single IPU Not Specified Bed Number: Not 422 Mood M= 15.58 Disorders SD= 1.07 Specified. 60.7% Attached Intermittent Services: Explosive Outpatient Disorder Admission 15.2% Criteria: Mood Diagnosed with at Disorder with least one DSM-IV Psychosis Axis I disorder. Staff: MDT This article is protected by copyright. All rights reserved 14% F= 60% Not M=6.12 Specified SD= 6.6 Empirical, et al., Quantitativ 2009) e Study Author Manuscript (Bobier (Burge National ss et Outcome al., 2009) NZ Single IPU Psychoeducation Bed Number: 8 Nursing. Attached Illness Education Services: Anger & Stress Outpatient and Management. day facilities. 46 Mood Mood Mood Not Disorders M=6.18 F=64% Specified 48% Mixed Mixed Mixed M=16.53 F=87% Disorders Psychosis Psychosi Relaxation. 33% M=16.78 s F=44% Admission Medication Psychosis Criteria: Does not Education. 19% accept conduct Problem Solving. disorder or Self-Awareness substance abuse Education. disorder as the Individual sole diagnosis. Support. Staff: MDT Sporting. M=24.01 Artistic activities. Australia Multiple Units Not Specified 1421 Not Specified Description of Data Not Not Not Not Specified Specified Specified Specified Neurotic/Emot M=14.4 F=60% People People ional SD= 2.51 without ID without ID (V=84%) M=78.2- Setting: Not Collection Specified. (Chapli Quality UK Multiple Units: 14 n et al., Improvem Description of 2015) ent Project Setting: Not Not Specified Specified. This article is protected by copyright. All rights reserved 151 Disorders (28%) Mood 117.0 Author Manuscript Disorders (Duddu Retrospecti et al., ve Review (23%) Schizophrenia (14%) UK Empirical, et al., Quantitativ 2007) e Study Not Specified 97 Adjustment, 59.8% F= anxiety (n=58) were 54.6% Attached disorders, 17. Services: PTSD, social Outpatient phobia Admission (32.6%) Bed Number: 6 2016) (Green Single IPU Criteria: Emerging Eating disorders Personality & intellectual Traits or disabilities were Disorders excluded, unless (15.8%) the main focus V=90.7% 1st Year (M= 30) 2nd Year (M=23.2) Schizophrenia was a mental (14.7%) health related crisis Staff: MDT UK Multiple Units Structured Milieu 4 Child Individualised 4 Adolescent Intervention This article is protected by copyright. All rights reserved 150 Mood M= 15.4 Disorders SD= .96 (43%) F= 46% Not M=16.6 Specified weeks SD=12.5 Author Manuscript (Green Empirical, ham Quantitativ and e Study Bed Number: Not Strategies Oppositional Specified Psychological Defiant Attached Therapy Disorder Services: Not Medication, (ODD) Specified. Psychosocial (27%) Admission Family-Oriented Criteria: Not & Educational Specified. Interventions. Staff: Not Specified. Canada Single IPU Not Specified Bed Number: 18 Attached 211 Mood Crisis Crisis Crisis Crisis Disorders M=14.8 F= 64% V=69% M=4 46% SD =2.0 Assessm Assessmen Assessment Adjustment Assessment ent t services Bisnair Services: Not e, Specified. Disorders M=14.9 F=70% V=77% M=13 Admission 9% SD 1.8 Transitio Transition Youth Psychosis Transition n V= 100% referred for 9% M=15.3 F= 80% 2008) Criteria: Acute suicidal thoughts, psychosis, nonresponse or severe side effects to medication, severe PTSD or severe anxiety or This article is protected by copyright. All rights reserved SD 1.2 inpatient transitional care M=19 Author Manuscript mood (Guven Empirical, ir et al., Retrospecti 2009) Individual. Bed Number: 10 Parent and Family ve, Attached Therapy. Quantitativ Services: Group, e Study Outpatient Psychological and 24.3% Admission Physical Therapy. Anxiety Criteria: NS Treatment linked Disorders Staff: MDT with therapeutic 11% Empirical, rty et Quantitativ 2013) Staff: MDT Single IPU (Hagge al., disturbances. e Study Turkey 90 Mood M=15.3 Disorders F= Not M= 77.3 67.8% Specified SD 25.5days M= 52% Not M= 10.81 Specified days. 37.7% Psychosis milieu. US Single IPU Individual Bed Number: NS Mood M=15.7 Psychotherapy. Disorders SD 1.19 Attached Group Therapy. 64% Services: NS Medication as Conduct Admission Prescribed. Disorders Criteria: NS Anger Staff: NS Management. Psychosis Substance Use 3% Psychoeducation. Academic This article is protected by copyright. All rights reserved 75 30% SD=5.23 Author Manuscript Programming. (Hanss Empirical, en- Quantitativ Recreation therapy. Mood M= 15.7 Therapy. Disorders SD= 1.4 Attached Individual 28% days), 25th et al., Services: Psychotherapy. Externalising percentile=3 2011) Outpatient and Family Therapy. 26% days, 75th School. Medication. Neurotic percentile=2 Admission School. 18% 9 days Bauer e Study (Herdzi Empirical, k, Quantitativ 2009) Norway e Study Multiple Units: 4 Ward Milieu Bed Number: 31 192 F= 70% Not M= 8.5 days Specified (range 1-351 Criteria: Psychosis, severe hyperactivity, violence causing risk to self or others, anxiety, depression and delirium. Staff: Not Specified. US Single IPU Psychiatric Bed Number: 16 Evaluation. Disorders Attached Individual 40% This article is protected by copyright. All rights reserved 60 Mood M= 15 F= Not 58.3% Specified M= 14 Author Manuscript (Matha Empirical, i and Quantitativ Bourne et al., Quantitativ 2011) e Study Adjustment Admission Staff. Criteria: Psychoeducation 18.3% Imminent threat Group Sessions. Externalizing of harm to self or Occupational others. Therapy. Staff: Not Recreational Specified. Therapy. Disorders Disorders 11.7% School. Australia , 2009) Empirical, Meetings with Family meetings. e Study (Setoya Services: School Single IPU Activity Groups. Bed Number: 12 Educational & Attached Vocational Services: School Sessions. Admission Group Therapy. Criteria: Not Individual Specified. Counselling. Staff: MDT Medication as 157 Not Specified M= 15.12 F= Not M= 10.67 74.5% Specified days SD= 19.34 Prescribed. Japan Single IPU Milieu Therapy. Bed Number: 41 Nursing Attached Interventions. Services: School Individual This article is protected by copyright. All rights reserved 126 Obsessive- M= 12.8 F= Not M=335.4 Compulsive SD= 1.9 60.3% Specified SD=336.2 Disorder 16.7% Author Manuscript (Sperbe Empirical, ck and Quantitativ Mayo, e Study Empirical, 2015) Retrospecti Psychotherapy. Eating Criteria: Not Family Therapy. Disorders Specified. Occupational Staff: MDT Therapy Pervasive Developmenta Excursions. l Disorders School. 12.7% Family Groups. US Single IPU Not Specified 3,150 Bed Number: 110 Attached Mood M= 13.2 Disorders SD=3.0 F= 52% Not M= 31.8 Specified SD=2.5 Not M= 89 days 42% Services: Not Disruptive Specified. Behaviour Admission Disorders Criteria: Not 22% Specified. Anxiety Staff: Not Disorders 20% Specified. UK Single IPU Not Specified Bed Number: Not ve, Specified. Quantitativ Attached e Study 14.3% Group Therapy. 2016) (Yuan, Admission Services: School This article is protected by copyright. All rights reserved 32 Not Specified M= 16 F= 71% Specified Author Manuscript Admission (Urben Criteria: Emotional and behavioural disorders including selfharm and suicide risk. Staff: MDT Naturalisti Switzerla et al., c nd 2015) Prospectiv e Study Single IPU Family Therapy. Bed Number: 10 Mood M=15.8 SD= Story-Telling Disorders 1.4 Attached Workshops. 37% Services: Not Music Therapy. Conduct Specified. Media Review, Disorders Admission Art. Criteria: Not Emotional centred Specified. workshop. Staff: MDT Educational & cultural focus. School This article is protected by copyright. All rights reserved 260 15% Anxiety Disorders 12% F= 57% Not M= 24.9 Specified SD=21.8 Author Manuscript Table 3 Descriptions of Included Outcome Measures, Domains Measured, Reporters, Follow-Up Times and Improvement Values Author(s) Outcome (year) Measures & Reported By Follow- Up Measure Admission Discharge One Year Improvement Follow-Up Values, P Values2 Domains Barnes (2009) Bobier et al. (P), Effect Size (ES)3 BASIS-24 4 BASIS-24: Clients 5 GAF HoNOSCA6 BASIS-24 and GAF: Clinicians Clinicians M: 1.56 M: 1.22 SD:.51 SD: .43 ES: 0.66 M: 26.15 M:40.58 P < .05* SD: 6.14 SD:8.9 ES= 2.35** Admission HoNOSCA M: 24.01 Not and Total SD: 16.90 Specified discharge Problems Symptom M: 15.26 Problems SD:5.52 * Indicates a statistically significant difference (p<0.001). ** Large effect size (d ≥ 0.8) 4 Depression. Functioning. Relationships. Self-Harm. Emotional Lability. Psychosis. Substance Abuse. 3 . 6 N/A P < .001* discharge GAF (2009) 2 Admission Behaviour, Impairment, Symptoms and Social. This article is protected by copyright. All rights reserved N/A P= 0.027* HoNOS7, al. (2009) HoNOS65+, and HoNOSCA discharge. Author Manuscript Burgess et Chaplin et Clinicians HoNOSCA8 Clinicians (2007) Not Specified Specified CGI Clinicians Admission HoNOSCA Admission CGI and SEM= 57.1% “Significant M: 20.1 M: 10.5 N/A P < 0.001* SD: 9.2 SD: 6.18 Not Not Specified Specified M: 44.0 M:56.0 M: 58.3 Admission- SD: 1.1 SD: 1.0 SD: 1.5 Discharge ES= 1.04** N/A “Much Improved” 53.7% discharge 10 CGAS CSRI 11 CGAS: Clinicians CGAS: CGAS SDQ: Clients and Admission, 12 Caregivers discharge P < .001* 13 TRF: Teachers and one year ES= 10.9** S.NASA: Client, Caregiver follow-up. Admission-Follow FEQ 14 SDQ 15 SNASA and Clincians. Behaviour, Impairment, Symptoms and Social. Behaviour, Impairment, Symptoms and Social. 9 Severity of Illness, Clinical Progress & Therapeutic Efficacy. 10 Functioning. 11 Background Client Information. 12 Functioning. 13 Family Engagement. 14 Emotional Problems. Conduct. Hyperactivity/Inattention. Peer Relationships. Prosocial Problems. 8 N/A discharge 9 FAD 7 Not Improvement” and (2016) Green et al. HoNOSCA ES=0.5 al. (2015) Duddu et al. Admission This article is protected by copyright. All rights reserved Up 15 16 Author Manuscript TRF16 FAD: Clients. P <.001* CSRI: Caregivers. ES= 13** FEQ: Clinican SDQ: SDQ- M: 22.9 M:20.6 M: 19.4 Admission- Admission, Caregiver SD: .75 SD: .82 SD: .95 Discharge discharge P < .001* and one year ES= 3.0** follow-up. Admission-Follow Up P < .01 TRF: Pre admission, after 4 ES= 4.6** TRF M: 42.1 M: 41.6 SD: 3.1 SD: 2.8 N/A AdmissionDischarge weeks of P <.001* admission ES= 0.16 and at discharge by the unit school, and at one year follow-up. Functioning including social, psychiatric, educational and life skills. Demographics Functioning. This article is protected by copyright. All rights reserved Author Manuscript Greenham & S.NASA: admission, M: 5.6 M: 4.0 Admission- SD: 2.6 SD: .30 SD: .32 Discharge P <.001* one year ES: 1.1** follow-up. Admission-Follow Up FAD: P <.001* Baseline/Ad ES= 1.73** mission CSRI: Prior admission and at follow-up. FEQ: (after 1 month of admission) CAPI17 18 CAPI: Clinician CAPI: CBCL CBCL: Caregivers Admission (2008) CDI19 CDI: Client and 18 M: 8.5 discharge & Bisnaire 17 S.NASA Risk Factors. Symptoms. Functioning. Systems Support. Emotional & Behavioural Functioning. This article is protected by copyright. All rights reserved CAPI Not Crisis Not Improved=Reliable Specified M: 11.0 Specified Change SD: 8.8 (RCI) Index CSPI 20 discharge. MASC MASC: Client CBCL: Assessmen Crisis YSR22 YSR: Clients Admission t 88% Improved CDI: M:12.2 Admission SD: 9.5 Author Manuscript 19 CSPI: Clinicians 21 & 14 days Assessment after Transition admission. M: 11.0 CSPI: SD: 11.4 82% Improved Admission? Transition MASC: 80% Improved Admission & 14 days after admission. YSR: Admission & 14 days after admission. Depressive Symptoms. Symptoms. Risk Factors. Functioning. Comorbidity. Systems Factors. 21 Anxiety. 22 Internalising & Externalising Problems. 20 > 1.96 This article is protected by copyright. All rights reserved Guvenir et CGAS23 CGAS: Clinicians Author Manuscript al. (2009) CDI24 CDI: Clinicians Y-BOCS: Clinicians Admission M: 41.3 M: 65.2 P= 0.000* and SD: 10.0 SD: 10.5 ES= 2.39** CDI CDI CDI CDI: M: 28.1 M: 7.0 P= 0.012* Admission SD: 14.1 SD: 5.2 ES=1.49** Y-BOCS Y-BOCS Y-BOCS discharge. M: 33.5 M:20.7 P= 0.002* Y-BOCS: SD: 11.9 SD: 10.7 ES=1.07** YMRS YMRS YMRS and M: 30.5 M: 8.5 P= 0.002* discharge. SD: 7.6 SD:9.1 ES= 2.89** HDRS HDRS HDRS Admission M: 22.6 M:12.0 P= 0.000* and SD: 6.0 SD:7.7 ES= 1.76** PANSS PANSS PANSS HDRS: M: 83.0 M: 50.7 P= 0.000* Admission SD: 33.3 SD: 24.2 ES= 0.96** Admission HDRS: Clinicians YMSR26 PANSS: Clinicians 27 HDRS GRA: Clinicians 28 YMSR: discharge. PANSS TSP: Clinicians and Functioning Depressive Symptoms. 25 Obsessive & Compulsive Symptoms Scale. 26 Mania. 27 Depression. 28 Positive & Negative Syndrome Scale. 24 N/A CGAS and YMSR: Clinicians CGAS CGAS discharge. Y-BOCS25 23 CGAS: This article is protected by copyright. All rights reserved CDI Y-BOCS YMSR HDRS PANSS CGAS 29 Author Manuscript GRA al. (2013) PANSS: TSP and discharge. GRA: Admission and discharge. TSP: 30 Admission and discharge. BSM-2531 BSM-25: Clients 33 34 IIP-32 35 YSR Admission M: 51.59 M: 34.28 P= 0.50* SOS-10: and SD:29.9 SD:28.5 ES= 0.57 Clients discharge. SOS-10 SOS-10 SOS-10 M: 36.15 M: 44.17 P= 0.49* SOS-10 SOS-10: Individual Difficulties. Family Relationships. Academic or vocational motivation & social life. Psychiatric Symptoms. Impulsivity. Functioning. 31 Psychiatric Symptoms. 32 Self- & Interpersonal Relational Experience. 33 Social, Occupational & Psychological Functioning. 34 Interpersonal Behaviours. 30 N/A BSM-25 SOS-10 BSM-25: BSM-25 BSM-25 32 GAF 29 GRA Admission TSP Haggerty et discharge. This article is protected by copyright. All rights reserved BSM-25 Author Manuscript GAF: Clinicians Hanssen- IIP-32: Clients YSR: Clients ES= 0.55 GAF GAF GAF discharge. M: 41.57 M:51.23 P= 0.53* GAF: SD:6.12 SD:5.88 ES= 1.57** M: 18.7 M: 13.6 SD: 6.3 SD: 7.1 Admission IIP-32: Admission YSR: Admission 36 HoNOSCA Clinicians Admission HoNOSCA and N/A P < 0.001* ES: 0.80** discharge 37 BASC-2 38 CDI 39 40 I-TAS 41 PES BASC-2: Client BASC-2: BASC-2- BASC-2- BASC-2 CDI: Client Admission Emotional Emotional Emotional Emotional FACES-IV: Clinicians and Symptoms Symptoms Symptoms Symptoms PES: Client discharge M: 81.8 M: 56.0 P < .05 PPVT-III: Clients CDI: SD:15.9 SD:12.9 ES= 1.62** Internalising & Externalising Problems. Behaviour, Impairment, Symptoms and Social. 37 Behavioural & Emotional Strengths & Challenges 38 Depressive Symptoms. 39 Adaptability & Cohesion in Family Interactions 40 Alliance to Inpatient Care. 36 GAF discharge. FACES-IV 35 SD: 12.14 and (2011) (2009) SD: 14.35 and Bauer et al. Herdzik Admission This article is protected by copyright. All rights reserved N/A BASC-2 PPVT-III42 43 Author Manuscript SOS SRP-A44 45 WRAT-III 41 SOS: Client Admission SRP-A: Client WRAT-III: Client CDI CDI CDI CDI and M: 14.5 M: 9.6 P < .05 discharge SD: 10.4 SD: 9.2 ES= 0.47 SOS SOS SOS During M: 34.7 M: 41 P < .05 treatment. SD: 15.5 SD: 15.4 ES= 0.40 FACES-IV: SOS I-TAS: Discharge PES: Discharge PPVT-III: During treatment SOS: Admission and discharge. SRP-A: Admission and Perceived Effectiveness Scale. Vocabulary Test. 43 Self- & Interpersonal Relational Experience. 44 Relations with Parents, Interpersonal Relations, Self-Esteem & Self-Reliance. 45 Reading Comprehension. 42 This article is protected by copyright. All rights reserved Author Manuscript discharge. Mathai & WRAT-III: During treatment 46 HoNOSCA Clinicians Bourne Admission HoNOSCA and (2009) M: 17.13 M: 9.98 SD: 5.88 SD: 4.71 CGAS CGAS N/A P < 0.0001* ES= 1.21** discharge 47 Setoya et al. CBCL CBCL: Caregivers CBCL: (2011) CGAS48 CGAS: Clinicians Admission M: 38.1 M: 57.9 P = 0.00* YSR: Client and SD: 13.9 SD: 14.6 ES= 1.42** CBCL CBCL CBCL CGAS: Full Score Full Score Full Score Admission M: 49.9 M: 38.7 P= 0.00* and SD: 30.5 SD: 26.6 ES= 0.36 YSR YSR YSR YSR: Full Score Full Score Full Score Admission M: 63.1 M: 53.7 P = 0.02* and SD: 26.4 SD: 28.9 ES= 0.35 49 YSR 46 discharge discharge discharge Behaviour, Impairment, Symptoms and Social. Emotional & Behavioural Functioning. 48 Functioning. 49 Internalising & Externalising Problems. 47 This article is protected by copyright. All rights reserved CGAS CBCL YSR N/A CGAS Sperbeck 51 CIS Author Manuscript (2016) BPRS-C50 DASS52 53 SARS Yuan (2015) BPRS-C: CIS: Clients Admission DASS: Clients and SARS: Clinicians discharge SARS NS NS N/A P= 0.000* Admission and discharge DASS: Admission and discharge SARS: Admission and discharge 54 CGAS 55 CGAS: Clinicians. CGAS: HoNOSCA: Clients and Clinicians HoNOSCA N/A HoNOSCA HoNOSC Admission (Clinician) A (Clinician) and M: 18.0 (Clinician) P< 0.001* Behaviour Problems. Depression. Psychomotor Excitation. Anxiety. Organicity. Functioning. 52 Anxiety & Depression 53 Behaviours. Affect. Social & Family Functioning. Self-Harm. Academic Problems. Cognitive Functioning. Previous Treatment. 54 Functioning. 55 Behaviour, Impairment, Symptoms and Social. 51 SARS CIS: HoNOSCA 50 BPRS-C: Clients This article is protected by copyright. All rights reserved HoNOSCA Author Manuscript Urben et al. SD: 7.2 M: 9.7 ES= 1.1** HoNOSCA: HoNOSCA SD: 2.9 HoNOSCA Admission (Client) HoNOSC (Client) and M: 25.2 A (Client) P< 0.001* discharge SD: 8.4 M: 10.3 ES= 1.7** SD: 12.4 56 HoNOSCA Clients? (2015) 56 discharge. Admission HoNOSCA M: 20.5 M: 13.3 and Total Score SD: 6.7 SD: 6.6 discharge Behaviour, Impairment, Symptoms and Social. This article is protected by copyright. All rights reserved N/A P< 0.001* ES= 1.07** Author Manuscript Table 4 Risk of Bias Assessment Criteria/ Yes (Y), No (N) 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 1.Was the study question or objective clearly stated? Y Y Y Y Y Y Y Y Y Y Y Y Y Y Y Y 2.Were eligibility/selection criteria for the study population pre-specified Y N N Y N Y N N N Y Y N Y Y N Y Y Y N N Y Y Y Y Y Y Y Y Y Y Y Y N N N N Y N N N N N N Y N N N N Y N Y N N Y Y N Y Y N Y Y Y N Y N Y N N Y N Y Y N N Y Y Y N Y Y Y Y N Y N Y Y Y Y Y Y Y Y Y Y Y N N N N N N N N N N N N N N N N Y Y N Y Y Y N N N N Y N Y N N Y Y Y Y Y Y Y Y Y Y Y Y Y Y Y Y Y and clearly described? 3. Were the participants in the study representative of those who would be eligible for the test/service/intervention in the general or clinical population of interest? 4. Were all eligible participants that met the pre-specified entry criteria enrolled? 5. Was the sample size sufficiently large to provide confidence in the findings? 6. Was the test/service/intervention clearly described and delivered consistently across the study population? 7. Were the outcome measures pre-specified, clearly defined, valid, reliable, and assessed consistently across all study participants? 8. Were the people assessing the outcomes blinded to the participants' exposures/interventions? 9. Was the loss to follow-up after baseline 20% or less? Were those lost to follow-up accounted for in the analysis? 10. Did the statistical methods examine changes in outcome measures This article is protected by copyright. All rights reserved from before to after the intervention? Were statistical tests done that Author Manuscript provided p values for the pre-to-post changes? 11. Were outcome measures of interest taken multiple times before the N N N N N N N N N N N N N N N N Y Y N Y Y Y Y Y N Y Y Y Y N N Y F P P P P F P P P P F F F P P F intervention and multiple times after the intervention (i.e., did they use an interrupted time-series design)? 12. If the intervention was conducted at a group level (e.g., a whole hospital, a community, etc.) did the statistical analysis take into account the use of individual-level data to determine effects at the group level? Quality Rating Good (G), Fair (F), Poor (P) Good= (0-2 No) Fair= (2-4 No) Poor= (4+ No) This article is protected by copyright. All rights reserved Author Manuscript This article is protected by copyright. All rights reserved Author Manuscript inm_12418_f1.docx This article is protected by copyright. All rights reserved Minerva Access is the Institutional Repository of The University of Melbourne Author/s: Hayes, C;Simmons, M;Simons, C;Hopwood, M Title: Evaluating effectiveness in adolescent mental health inpatient units: A systematic review Date: 2018-04 Citation: Hayes, C., Simmons, M., Simons, C. & Hopwood, M. (2018). Evaluating effectiveness in adolescent mental health inpatient units: A systematic review. INTERNATIONAL JOURNAL OF MENTAL HEALTH NURSING, 27 (2), pp.498-513. https://doi.org/10.1111/inm.12418. Persistent Link: http://hdl.handle.net/11343/293963