Parenting Early Intervention Programme Evaluation

Research Report DFE-RR121(a) Parenting Early Intervention Programme Evaluation Geoff Lindsay, Steve Strand, Mairi Ann Cullen, Stephen Cullen, Sue Band, Hilton Davis, Gavan Conlon, Jane Barlow and Ray Evans Acknowledgements We would like to thank all the parents who completed our questionnaires and those who gave their time to be interviewed: the local authority lead officers, facilitators and other colleagues for their time and assistance to make the evaluation successful; and the members of the steering group; and other LA colleagues who provided advice on the final reports. Finally, we would like to thank the CEDAR research secretaries who processed the substantial amounts of data necessary for this project. AUTHORS Geoff Lindsay1, Steve Strand1, Mairi Ann Cullen1, Stephen Cullen1, Sue Band1, Hilton Davis2, Gavan Conlon3, Jane Barlow4 and Ray Evans1 1 CEDAR, University of Warwick 2 Kings College, London 3 London Economics 4 Warwick Medical School This research was commissioned before the new UK Government took office on 11 May 2010. As a result the content may not reflect current Government policy and may make reference to the Department for Children, Schools and Families (DCSF) which has now been replaced by the Department for Education (DFE). The views expressed in this report are the authors’ and do not necessarily reflect those of the Department for Education The views expressed in this report are the authors’ and do not necessarily reflect those of the Department for Education ACKNOWLEDGEMENTS 2 CONTENTS 3 EXECUTIVE SUMMARY 6 1. Introduction 13 1.1 The rationale for the PEIP 13 1.2 The efficacy of parenting programmes 14 1.3 The Parenting Early Intervention Programme (PEIP) 15 1.4 Aims and Methodology 16 1.5 The structure of the report 21 2. Impact of the PEIP 23 2.1 Introduction 23 2.2 Local authority activity and parent engagement 23 2.3 Parent characteristics 25 2.3.1 Parent demographics 25 2.3.2 Parent mental well-being and parenting skills 29 2.4 2.5 Child characteristics 31 2.4.1 Demographics 31 2.4.2 Child Behaviour (Strengths and Difficulties questionnaire) 33 Measuring Pre-course to Post-course change 35 2.5.1 Post-course booklet return rate 36 2.5.2 Do those responding to the post-course differ from Non- 37 responders? 2.5.3 Are there particular characteristics of those who dropped 38 out of the programme? 2.5.4 2.6 2.7 2.8 Return rate by programme 39 Did PEIP improve parent and child outcomes? 39 2.6.1 40 Overall effect averaged across programmes Effects of parent background and programmes 44 2.7.1 Is PEIP effective for the full range of parents and children? 45 2.7.2 Difference in improvement between programmes 46 Post course evaluation – How was your group? 3 50 2.9 PEIP Follow Up 53 2.9.1 Introduction 53 2.9.2 Description of sample 53 2.9.3 Were those followed up representative of those completing 54 Parenting programmes? 2.9.4 2.10 54 Findings Qualitative evidence of impact 58 2.10.1 Outcomes for parents 59 2.10.2 Outcomes for children/young people 62 2.10.3 Outcomes for family relationships 66 2.10.4 Outcomes for parents’ relationships with schools 67 2.10.5 Outcomes for community relationships 69 2.10.6 Outcomes for practitioners as facilitators 70 2.10.7 Outcomes for professionals referring parents to 71 programmes 71 2.10.8 Conclusions 3. 4. The Facilitators 73 3.1 Introduction 73 3.2 Facilitators’ demographics: Gender, age and ethnic group 73 3.3 Facilitators’ educational qualifications 74 3.4 Facilitators’ prior experience 76 3.5 Comparing facilitators’ qualifications and prior experience 78 3.6 Lead facilitator characteristics and parent outcomes 79 3.6.1 The matching process and achieved sample 79 3.6.2 Lead facilitator qualifications and experience in relation to parent and child outcomes 80 Process factors underpinning effective outcomes 82 4.1 Programme factors 82 4.1.1 The programmes 82 4.1.2 The initial training 86 4.1.3 Further training and accreditation 88 4.1.4 Supervision requirements 89 4.2 Local infrastructure factors affecting effectiveness and 90 implementation 4.2.1 Efficiency and effectiveness 4 90 4.2.2 Leadership and coordination across a local authority 91 4.2.3 Meshing PEIP into the local context 94 4.2.4 LA-wide delivery models 97 4.2.5 Recruitment, retention, support and supervision of 101 facilitators 4.2.6 5. 6. Recruitment, engagement and retention of parents 104 Cost effectiveness of the PEIP 109 51 Introduction 109 5.2 Cost effectiveness analysis 109 5.3 Findings 111 5.4 Disaggregation of expenditure 112 5.5 Costs of programme outputs 114 5.6 Costs of parenting outputs 117 5.7 Costs of child outputs 118 5.8 Conclusions 118 Discussion, Conclusions and Recommendations 120 6.1 Introduction 120 6.2 Was parenting training effective in the new sites? 120 6.3 Are the positive effects sustained after the post-intervention 122 period? 6.4 What were the characteristics of successful implementation of the 122 PEIP? 6.5 Conclusions 124 6.6 Recommendations 125 References 127 Appendix 1 Methodology 129 Appendix 2 The five PEIP programmes 136 References for Appendix 2 160 Appendix 3 Statistical analysis for the four main PEIP programmes 163 Appendix 4 Summary reports for all eight PEIP programmes 165 5 EXECUTIVE SUMMARY Behaviour problems during early and middle childhood are associated with antisocial behaviour during adolescence and increase the risk of negative outcomes in adulthood. Successful parenting is a key element in preventing children developing behavioural difficulties. However, parents differ in the internal and external resources on which they can draw. Internal resources include their own mental well-being and resilience. External resources include poverty, social disadvantage and the absence of a support network. The recent Allen Report (2011) stressed the need to use effective methods of early interventions, including parenting programmes. There is now strong evidence from rigorous efficacy trials that parenting programmes can improve parenting skills and, as a result, reduce children’s behavioural difficulties. Evidence is also necessary to show their effectiveness when programmes are implemented on a large scale, in community settings. This report presents the evidence for the effectiveness of the national roll out of parenting programmes in England. The Parenting Early Intervention Programme (PEIP, 2008-11) provided government funding to all 150 local authorities (LAs) in England to deliver selected parenting programmes that already had evidence of their efficacy in improving parent outcomes and associated reductions in children’s behavioural difficulties 1 . This report examines the effectiveness in everyday use in community settings across England of five parenting programmes initially selected by the government for use in the PEIP; these were Families and Schools Together (FAST), Positive Parenting Program (Triple P), Strengthening Families Programme 10-14 (SFP 10-14), Strengthening Families, Strengthening Communities (SFSC), and The Incredible Years. Key Findings • The national roll-out of PEIP was successful in increasing the support available for parents concerned about their child's behaviour. • Outcomes were equally positive for the parents of older children (8-13 years, the target age group for PEIP) as they were for parents of younger children. Parenting 1 http://webarchive.nationalarchives.gov.uk/20100202100514/dcsf.gov.uk/everychildmatters/strategy/parents/id91 askclient/localauthority/fundingforparents/ From June 2010, LAs were able to fund other programmes through PEIP. 6 programmes in the PEIP can therefore be effective interventions for a wide range of age groups. • All four main parenting programme used by the PEIP (Triple P, Incredible Years, Strengthening Families Programme 10-14 and Strengthening Families Strengthening. Communities) were effective in improving outcomes for parents and children, and these outcomes were maintained one year on from the end of the programme. • These programmes had a positive effect on parents’ mental well-being and style of parenting, as well as on their children’s behaviour; these are all key protective factors for achieving positive long term child outcomes. • The cost to local authorities of funding the delivery of parenting programmes should be lower in future as infrastructure set up costs, especially the training of facilitators, have been met through PEIP. Detailed findings The parents and children: • Local authorities used PEIP to target parents appropriately. Overall, the parents were more disadvantaged than the general population. o 44% were living in single parent households. o 63% lived in rented accommodation. o 69% had sought help from one or more professionals in the previous six months o 54% had educational qualifications below the level of 5 GCSE A* -C or equivalent o 75% scored below the national median for mental well-being. • Compared with the national population the child about whom parents were most concerned displayed the following characteristics: o Four times more likely to have a statement of special educational need (11.8% v 2.7%). o Three times more likely to be entitled to a free school meal (49% of children v 16%). o Six times more likely to be classified as having serious behavioural difficulties. In addition: o Nearly a third (31%) received additional support at school. o Just over a half (54%) were in the PEIP target age range of 8-13 years (mean age 8.6 years: standard deviation 3.9 years). 7 Short term outcomes • Outcomes for parents on all four programmes were significantly improved after programme completion: o Overall, 79% of parents showed improvements in their mental well-being. o The average level of parental mental well-being increased from that of the bottom 25th percent of the population to the national average. • A considerable proportion of parents changed their parenting behaviour over the course of the programme: o 74% of parents reported reductions in their parenting laxness. o 77% of parents reported reductions in their over-reactivity. o The percentage of parents who reported that their child had serious conduct problems reduced by a third, from 59% to 40%. • Parents were highly positive about their experiences of the parenting groups they attended: o 98% reported that they had found the group helpful. o 95% reported that the programme had helped them deal with their problems. o 95% reported that the programme had helped them to deal with their children’s behaviour. o 86% reported that they experienced fewer problems after completing the programme. o Over 98% reported that the group leader, showed positive characteristics, including making them feel respected and working in partnership. • There were differences in the effects on outcomes between the individual programmes but these were relatively small compared to the overall improvements reported by parents. One year follow up • Improvements in parent laxness and over-reactivity were maintained. • Improvements in their children’s behaviour were also maintained. • There was a small reduction in parents’ reported mental well-being but this remained significantly higher than when they started their parenting programme. Implementation effectiveness • The number of parents supported through the programme varied substantially between LAs, ranging from over 750 parents in one LA to 30 or fewer or others. • There was substantial variation between LAs in the cost-effectiveness of the PEIP. 8 • The proportion spent on management reduced over time as the PEIP became established. • The average cost of funding a parent who started a PEIP parenting programme was approximately £1244; this increased to £1658 if the 75% completion rate found in the LAs providing data is taken into account. • The lowest cost per parent, in one LA that had been operating for the full three years of the programme, was only £534. • Several models of local authority organisation for delivering the PEIP were effective, including a core team, a multi-agency team, commissioning outside the authority, and hybrid models: the key was to match the model to local circumstances. • Successful implementation was related to: o Effective leadership and coordination of the PEIP. o How well the PEIP was integrated into the LA context, including its parenting or Think Family strategy. o Effective recruitment, retention, support and supervision of group facilitators. o Effective recruitment, engagement and retention of parents. Conclusions • Evidence-based parenting programmes can be effective when implemented under variable local conditions. • All four PEIP programmes were effective in improving parenting skills, parent mental well-being and in reducing children’s behaviour difficulties for parents and children across the full range of demographic backgrounds, including children with SEN. • Outcomes were maintained one year on from the end of the programme. • Differences in outcomes between programmes were small. • Positive outcomes in children’s behaviour and wellbeing would be expected to impact positively on educational attainment. • The cost of delivering parenting programmes reduced with time, as set up costs e.g. infrastructure and training facilitators, are front loaded. Future costs should therefore be lower on average than those reported here. 9 Recommendations • Local authorities should make parenting programmes available as part of their prevention and intervention strategies to prevent the development or reduce the impact of behavioural difficulties in children. • Provision of parenting programmes should be directed mainly at those in greatest need; however, there are also benefits in recruiting a broader spectrum of parents in order to optimise group dynamics and achieve better outcomes. • Differences in outcomes between programmes were small, therefore the choice of evidence-based programmes for local use should be made in alignment with: local needs and priorities, how efficiently they use existing trained workforce, experience of delivery, and development of the local offer to parents. • Effective implementation by a local authority requires strong leadership, effective day to day management and organisation, as well as a clear parenting policy. • Several organisational and delivery models work well; the key is to match the model to local circumstances. • A diverse workforce, including parents and non-graduates, can deliver parenting programmes effectively when provided with appropriate training, support and supervision. • Effective selection of facilitators should be based on their capacity to deliver programmes, and the skills and personal qualities that enable them to engage with parents. • Local authorities should ensure that the programmes are quality assured and maintain fidelity to their evidence-based models of implementation as set out in the guidance 2 . The evaluation Aims and objectives The aim of the study was to: • Evaluate the impact and benefits of the national roll out of the Parenting Early Intervention Programme. 2 http://webarchive.nationalarchives.gov.uk/20100202100514/dcsf.gov.uk/everychildmatters/strategy/p arents/id91askclient/localauthority/fundingforparents/ 10 The objectives were to: • Evaluate whether the positive impacts of parenting programmes demonstrated in research are replicated when these programmes are rolled out on a larger scale under variable conditions in a range of community settings. • Assess whether the positive effects of parenting programmes are sustained one year after the short term evidence of the immediate impact of group training. • Consider how the PEIP fits with other local authority parenting provision (e.g. parenting experts, Parent Support Advisors) and how far this provision provides a preventative approach to working with children and families at risk of negative outcomes. Methods PEIP enabled LAs to fund one or more of five evidence-based parenting programmes as approved by the National Academy of Parenting Practitioners (NAPP). These were: • Families and Schools Together (FAST) • Incredible Years • Strengthening Families Strengthening Communities (SFSC) • Strengthening Families Programme 10-14 (SFP10-14) • Triple P All five parenting programmes have an evidence base for improving parent and child outcomes when tested in small scale, controlled trials. Our evaluation examined whether these outcomes could be maintained and replicated when the programmes were rolled out nationally and implemented in all LAs in England. To do so we collected information on: • Reported changes in parenting styles, child behaviour and parent mental well-being following the attendance of one of four of the NAPP approved parenting programmes (measure of impact) 3 • The organisational factors that support effective implementation of PEIP and its roll out To assess the short term impact of the parenting programmes on parent and child outcomes, we provided parents with questionnaires to complete at the beginning and end of their 3 As there was insufficient data from parents attending FAST, impact could only be examined for the other four programmes. However, implementation effectiveness of all five programmes was examined. For summary outcome data for 44 parents completing FAST see Appendix 4. 11 course and these scores were compared in order to measure change. We collected precourse data from 6143 parents in 43 sample LAs and also obtained post-course data from 3325 of them, who were representative of the total sample (overall response rate 54%) 4 . To assess the sustainability of positive effects follow up questionnaires were obtained from 212 parents, who were representative of those who had completed the programmes, one year after they had finished their programme (response rate 30% providing data). The measures of outcome used in the questionnaires completed by parents have been substantiated by and used in prior research, including the PEIP Pathfinder report. These measures assess parental mental well-being, parental laxness and over-reactivity in dealing with their child’s behaviour, and the parent’s view of the child’s behaviour, all of which are risk factors for child outcomes that are expected to be positively influenced by the parenting programmes. As administrative data were not available, in order to assess cost effectiveness we provided LAs with a questionnaire to complete towards the end of the. Indicative findings are derived from 15 of the 43 LAs that also provided parent data. To examine the organisational factors that support effective rollout of PEIP, 429 interviews were conducted with a sample of parents and professionals. 4 The non-response rate comprises both those parents that did not complete the programme and those for whom questionnaires were not returned to CEDAR despite programme completion (see section 2.5.1). 12 1. INTRODUCTION 1.1 The rationale for the PEIP Many children and young people exhibit behavioural difficulties. A national study by Green et al. (2004) found prevalence rates of between 10-20 per cent within the UK. Furthermore, prevalence rates have increased since the mid 1970s (Maughan et al., 2008). Such difficulties have serious implications for children, their families and society as a whole. For example, at school age, 12% of children aged 4-10 years and 18% aged 11-15 years who have statements of special educational needs (SEN) have behavioural, emotional and social difficulties (BESD) (Department for Education, 2010). When pupils with statements and those at School Action Plus are combined, 30 per cent of pupils with SEN have BESD. Pupils with SEN at School Action Plus are 20 times more likely to receive a permanent exclusion than those with no SEN. Pupils with BESD were the most likely to be permanently excluded. We also know that conduct problems during early and middle childhood are associated with antisocial behaviour during adolescence and put the young person at risk of criminal behaviour. These enhanced risks persist: early behavioural difficulties are associated with adult mental health problems, crime, relationship and parenthood difficulties and substance dependence (Ferguson et al., 2009). There are economic considerations as well as personal and social impacts of children exhibiting behaviour problems. The SEN statistics above indicate the financial resources that are necessary to provide support for pupils with BESD if they are at School Action Plus, receiving support from a professional from outside the school or, even more so, if the pupil has a statement. Parents are fundamental to their children’s development and so successful parenting is a key element in preventing children developing behavioural difficulties (Pugh, De’Arth & Smith, 1994). However, parents vary in their capabilities. All parents find parenting a challenge at times but parents differ in the internal and external resources on which they can call. Internal resources include their own mental well-being and personal resilience. External factors include poverty, social disadvantage and the absence of supportive families and friends. These factors interact and multiple adverse factors may lead to a parent not 13 developing or not implementing effective parenting skills, in order to provide an appropriate environment for the development of their children. Supporting parents to develop effective parenting skills has therefore been recognised as an important prevention and intervention strategy. As group methods offer the potential for greater cost-effectiveness, a variety of group training programmes have been developed and implemented, including the five main programmes examined in this evaluation (see below). 1.2 Efficacy and effectiveness There is now good evidence for a number of parenting programmes, derived from carefully devised and implemented efficacy trials. The ‘gold standard’ approach is to run these as randomized controlled trials. These provide the best evidence of real effects by randomly allocating parents to either the parenting programme or a control group that does not receive it. These trials require care to implement the programme according to its specified guidance (typically set out in a manual), using well trained facilitators and appropriate measures of outcome. Ideally such trials should be replicated and preferably by independent researchers. There is now extensive evidence that parenting programmes can have positive effects on both parent outcomes (e.g. improving parenting skills and parents well-being) and that these are associated with changes in their children (e.g. reduced behavioural problems). Overviews of evidence are available from the United Nations Office on Drugs and Crime http://www.unodc.org/docs/youthnet/Compilation/10-50018_Ebook.pdf and from systematic reviews of studies (e.g. Barlow & Coren, 2000; Barlow & Stewart-Brown, 2000; National Institute for Health and Clinical Excellence, 2005). The research report from the Allen Inquiry into early intervention (Allen, 2011) also reviews parenting programmes and other methods of early intervention, providing useful indications of the quality of the evidence available for each programme. Efficacy trials provide the basic evidence and are essential pre-requisites. However, it is also important to examine the implementation of evidence-based programmes in ‘real world’ settings as argued cogently by Weisz (e.g. 2004). Efficacy trials are typically very well resourced and carefully conducted in order to obtain results under optimal conditions. Such conditions do not match the circumstances found in everyday practice. Here the task is much more challenging. Practitioners, local authorities and health trusts are expected to provide services to meet public need not the rigours of research. It is important, therefore to examine both the efficacy of parenting programmes under ideal conditions and their effectiveness in real life, community settings. 14 1.3 The Parenting Early Intervention Pathfinder (PEIP) The previous government prioritised parenting support as a means of reducing antisocial behaviour among young people and preventing crime, thereby enhancing communities. The Respect Action Plan, a Home Office initiative, had a budget which included £52 million over two years to provide a number of parent support initiatives (Respect Task Force, 2006). The Parenting Early Intervention Pathfinder (2006-08) was funded in 18 LAs for £7.6 million. Its focus was on parents of children aged 8-13 exhibiting or at risk of behavioural problems as this age group was judged not to have the level of support available to younger and older children. On the basis of a review of evidence by Moran, Ghate and van der Merwe (2004) three programmes were selected: Triple P, Incredible Years and Strengthening Families, Strengthening Communities. Eighteen LAs that were judged by the Department for Education and Skills 5 to have experience in parenting support were selected. The LAs were each allocated funding to implement one of the three programmes as determined by the DfES (6 per programme). The results of our evaluation (Lindsay, et al, 2008) were sufficiently positive to encourage the Department for Children, Schools and Families to try to implement parenting support more widely. The Parenting Early Intervention Programme (PEIP) was an initiative begun in 2008 to fund all 150 local authorities (LAs) in England to deliver evidence-based parenting programmes. Findings from our evaluation of the Pathfinder showed that there were substantial improvements in parents’ mental well-being, parenting styles (reductions in over-reactivity and laxness) and improvements in their children’s behaviour as a result of attendance at parenting programmes. This evidence was then used to inform the Guidance 6 that was issued to all LAs to help them set up and deliver PEIP, which was rolled out to a further 23 LAs (Wave 2) from 2008, and nationally from 2009 (Wave 3). 5 The Department for Education and Skills (DfES) was later replaced by Department for Children, Schools and Families (DCSF). This in turn was replaced by the Department for Education (DfE) by the Coalition Government in May 2010. We use the name of the department at the appropriate time throughout this report 6 http://webarchive.nationalarchives.gov.uk/20100202100514/dcsf.gov.uk/everychildmatters/strategy/p arents/id91askclient/localauthority/fundingforparents/ (Accessed 21.4.11) From June 2010, LAs were able to fund other parenting programmes through PEIP. 15 The PEIP operated within a new policy framework, Think Family 7 . This brought together several parenting support initiatives including the PEIP. Importantly, the policy allowed LAs greater freedom to vire Think Family grants across interventions, allowing them greater freedom of decision making. Local authorities were allowed to use funding through PEIP to implement any of five programmes approved by the DCSF on the advice of the newly created National Academy of Parenting Practitioners (NAPP); those used in the Pathfinder together with Strengthening Families Programme 10-14 and Families and Schools Together (FAST) 8 . From June 2010, LAs were able to fund other parenting programmes through PEIP. During the early part of the PEIP, LAs were required to submit regular reports to the DCSF as our Pathfinder evaluation had identified a vast range in LA implementation and hence cost-effectiveness. This requirement was later rescinded. As a result of these changes there were no administrative data available. Consequently it was not possible to know how many groups were run, parents supported or the income and costs of implementation for each LA. This does not affect our study of the programmes’ impact or implementation of PEIP; however we cannot assess overall effectiveness of the PEIP in terms of parent and group numbers. Furthermore, in the absence of systematic DfE data on funding and expenditure for each LA, we have used LAs’ own (unaudited) data collected by means of a survey. 1.4 Aims and Methodology The evaluation adopted a combined methods approach, using both quantitative and qualitative methods to address the research questions. A summary of the methodology is presented here. We specify the aims and objectives of the study and describe the parenting programmes. We then set out details of the study: our samples of parents and the measures we used to evaluation impact and implementation of the PEIP. 7 http://education.gov.uk/publications/standard/publicationDetail/Page1/DCSF-00264-2010 (Accessed 8.3.2011) 8 Our final report on the Pathfinder recommended that LAs should be allowed to use funding to implement other programmes that had a good evidence base. 16 The aim of the study was to: • Evaluate the impact and benefits of the national roll out of the Parenting Early Intervention Programme. The objectives were to: • Evaluate whether the positive impacts of parenting programmes demonstrated in research are replicated when these programmes are rolled out on a larger scale under new conditions. o Measured by parents’ reports of their mental well-being, their laxness and overreactivity, and their reports of their children’s behaviour. • Assess whether the positive effects of parenting programmes are sustained beyond the short term evidence of the immediate impact of group training. o Examined by a one year follow up of parents who had completed parenting programmes • Consider how the PEIP fits with other local authority parenting provision (e.g. parenting experts, Parent Support Advisors) and how far this provision provides a preventative approach to working with children and families at risk of negative outcomes. o Examined by interviews with parents and a range of LA staff. These translated into three research questions: • Is parenting training effective in the new sites? • Are the effects sustained after the post-intervention period? • How does PEIP fit with other parenting provision and to what extent is it preventative? The study therefore had objectives that required examination of impact (benefits to parents and children) and the methods of implementation (processes). The programmes Five programmes were initially approved by the DCSF as eligible for funding through the PEIP on the advice of the National Academy of Parenting Practitioners (NAPP) – see Figure 1.1: • Families and Schools Together (FAST) • Incredible Years • Strengthening Families Strengthening Communities (SFSC) 17 • Strengthening Families Programme 10-14 (SFP10-14) • Triple P All are designed to address parenting skills and children’s behaviour but there are differences in the theoretical bases, aims and structures. For example, length of programmes varies and SFP 10-14 and FAST also involve children in the programme. During the PEIP, guidance to LAs changed allowing use of the grant funding for other programmes. Consequently, our evaluation primarily focuses on these five but reference is also made to three others that LAs used. Details of the five main PEIP programmes and reference to the research that provides their evidence base is presented in Appendix 2. Summary outcome data for all eight programmes used in PEIP are given in Appendix 4. Samples The Pathfinder (2006-08) was retrospectively named Wave 1. These 18 LAs were not part of the study, except for interviews with LA lead officers on two occasions (see Appendix 1 Table A.1 for a full list of interviews). All 23 Wave 2 LAs and a sample of 24 Wave 3 were selected for study. The Wave 3 LA selection took into account the need for a sample that reflected geographic spread, urban/rural, and levels of socioeconomic disadvantage. Overall, we collected pre-course data on 6143 parents that attended a parenting group and also obtained post-course data from a representative sample of 3325 of them. Follow-up questionnaires were also obtained from a representative sample of 212 one parents year after completing their parenting programme 9 . We also collected data from 429 interviews with parents and professionals (see Interviews below). 9 As the majority of parents attending parenting courses did so during the last year of PEIP, follow up was not possible during the period of the evaluation. 18 Figure 1.1 The five PEIP programmes (For full details, see Appendix 2) FAST (Families and Schools Together) – originated in USA 3 -18 years Age range Core programme 8 weekly 2.5 hour evening sessions, school-based. Followed by two years of parent-led, school-supported, monthly booster sessions. Session structure Family tables including meal; peer activity (parent group; child group); parentchild activity, closing tradition. Families with children in a year group in a school serving a multiple risk Participants neighbourhood. 10 families per hub, with school running 4 to 6 hubs at a time. Five per hub - professionals from multiple agencies and parents of children in an Facilitators older year group in the school; young people also in secondary schools. To increase protective factors for child [more aims in Appendix] Example aim Incredible Years – originated in USA 8-13 years (for the PEIP programme) Age range Core programme Combines elements of the School Age BASIC program (12-16 sessions) with the ADVANCE parent Program (9 sessions) making 18-22 sessions of 2-2.5 hours. Session structure Set out in manual – includes group discussion, video and live modelling, role play or small group rehearsal. Refreshments provided. Group of 10-14 parents Participants Two group leaders – ideally drawn from professionals with postgraduate Facilitators qualifications in fields such as psychology, psychiatry, social work, nursing Treatment and prevention of child behaviour problems [more aims in Appendix] Example aim Strengthening Families Programme 10-14 (SFP 10-14) – originated in USA 10-14 years Age range Core programme Seven weekly 2 hour sessions. Followed by four optional booster sessions beginning 6-12 months afterwards. Session structure Set out in manual – parallel groups for parents and young people, family activities. Includes refreshments and may include a meal. Up to 12 families Participants At least three facilitators (one for parents, two for young people) – drawn from all Facilitators professional groups and parents who have previously attended the course. To decrease alcohol and drug use during adolescence [more aims in Appendix] Example aim Strengthening Families Strengthening Communities (SFSC) – originated in USA 3-18 years Age range Core programme Thirteen weekly 3 hour sessions. Session structure Set out in manual – includes facilitator modelling, role play, lectures, discussion. Group of 8-15 parents Participants Co-facilitation model – practitioners from any occupation, ideally with Level 3 Facilitators qualifications and experience and expertise in working with parents To promote protective factors for child [more aims in Appendix] Example aim Triple P (Positive Parenting Program) – originated in Australia 0-16 years Age range Core programme For PEIP, typically Level 4 Group or Group Teen – eight sessions: five as 2-hour group sessions, three as 30 minute telephone calls. Session structure Set out in manual – includes presentations, video demonstrations, discussion 10-12 parents Participants One facilitator required – basic professional training required - typically drawn Facilitators from psychologists, social workers, teachers, family counsellors, nurses To enhance parents’ knowledge, skills, confidence [more aims in Appendix] Example aim 19 Measures Parent questionnaires Three instruments were administered by the group facilitator and completed by parents as they started their programme (pre-course), at the last session (post-course). Parents completed the same questionnaires sent by post one year after the programme finished. These are all well established self report measures and had been used for the Pathfinder evaluation. The measures assessed: • Parent mental well-being The Warwick-Edinburgh Mental Well-being Scale (WEMWBS) 10 examines how the parent feels, e.g. ‘I’ve been feeling useful’, and ‘I’ve been feeling good about myself’. Two important dimensions of parenting style were measured by the Parenting Scale • Parental laxness This scale examines whether parents are too lax when dealing with their child: for example, whether a parent backs down and gives in if their child becomes upset after being told ‘no’. • Parental over-reactivity This scale examines parents’ over-reactions: for example, whether a parent raises their voice or yells when their child misbehaves as opposed to speaking to the child calmly. Children’s behaviour was measured by the Strengths and Difficulties Questionnaire (SDQ) • Children’s behaviour Parents rated the behaviour of their ‘target child’, i.e. the child about whom they had most concern, on the SDQ. We report three measures: children’s conduct problems; an aggregate measure (total difficulties) of conduct problems, hyperactivity, peer problems and emotional symptoms; and the impact of the children’s behaviour problems. In addition, parents provided demographic information about themselves and their target child in the family at pre-course. They also completed a questionnaire: ‘How was your group?’ at the end of their parenting programme in order to provide information on their 10 Full details of the measures are given in Appendix 1. 20 group experience, with particular reference to the effectiveness of the group facilitator’s style and the helpfulness of the programme. Facilitator questionnaire Facilitators (N = 1277) completed a form giving details of their qualifications and previous parenting programme experience. This enabled an analysis of the relationship of those factors with improvements shown on the parent-completed measures. Cost effectiveness questionnaire Fifteen of the 43 local authorities surveyed completed a questionnaire specifically designed for the PEIP and piloted in five LAs (one per main programme). This captured costs and number of parents supported and was used to examine cost-effectiveness of implementing the PEIP. Interviews A total of 429 interviews were held with LA strategic leads, operational leads, group facilitators, school representatives and parents across our total sample of 47 Waves 2 and 3 LAs and 18 Wave 1 LAs (Appendix 1, Table A.1). These provided the opportunity to explore factors that supported or inhibited implementation. Semi-structured interviews administered face-to-face or by phone were used at appropriate stages of the study: strategic lead (73 interviews), operational leads (92), combined role lead (13), facilitator (77), other parenting support professionals (83) school representative (16), parent (75). 1.5 The structure of the report The main focus of the report is the evaluation of the PEIP 2008-11, but this built upon the Pathfinder (Wave 1) evaluation 2006-08. Two interim reports were produced. The first (Lindsay et al, 2009) examined the first year of the PEIP (to summer 2009), reporting on its implementation and the training of facilitators. Data were presented on the first 714 parents that had started parenting programmes in Wave 2. We showed that the PEIP had been slow to get underway, especially considering the success of the Pathfinder. A key factor was the training of facilitators as this process went on for some considerable time, so delaying start up. There were also LA organisational factors. The 2nd Interim Report (Lindsay et al., 2010) examined the second year (to summer 2010). It included data on 3131 parents which suggested that the PEIP was addressing the needs of appropriate parents and children and was having a positive impact. 21 The present report focuses on evidence collected throughout the project including outcome data on all parents. It is structured as follows: Section 2 presents the evidence on impact. This draws primarily on the data from the parents attending the groups. A separate subsection reports the one year follow-up data which examines whether gains made as a result of attending the groups are maintained. Interview data about the impact of PEIP programmes are also presented. Section 3 presents data on the facilitators. The interest here concerns both the characteristics of facilitators and the extent to which facilitator characteristics are an important factor in programme impact, which is examined in the next section. Section 4 examines the implementation of the PEIP. Drawing mainly on qualitative data, this explores programme and local authority organisation factors associated with successful implementation. Also included is a consideration of the impact of the facilitators. Section 5 presents an analysis of the costs of implementing parenting programmes through PEIP and an analysis of the cost effectiveness of the parenting programmes delivered through PEIP. Section 6 comprises a discussion of the findings, conclusions and recommendations. Appendices provide additional information on the methodology, the five main parenting programmes and additional statistical analysis. A full technical report of the statistical analyses is available on the CEDAR website (http://www2.warwick.ac.uk/fac/soc/cedar/peip/). 22 2. IMPACT OF THE PEIP Main Findings • The roll-out of evidence-based parenting programmes through the PEIP has been successful on a national scale and has significantly increased support for a large number of parents. • The parenting programmes have had positive effects on the parents’ mental wellbeing and their style of parenting, as well as their children’s behaviour. o The improvements in children’s behaviour are comparable to those found in recent UK small scale studies. • 2.1 Positive effects were maintained at follow up a year after the programme ended. Introduction This section focuses on the impact of the PEIP based on data from questionnaires completed by parents attending parenting groups and interviews with a sample of parents. We present the following findings: engagement by the local authorities (LAs) and the use of the different programmes; characteristics of the parents who attended parenting programmes; characteristics of their target child, about whom they were most concerned; the changes in parents and their children associated with attending their parenting group; the parents’ evaluations of their groups; the evidence from a follow up of parents one year after they finished their parenting programme; and interview data about impact. The main questions we address are: 2.2 • How effective was the PEIP overall in improving parent and child outcomes? • Was the PEIP effective across the full range of parents and children? • Were there differences in effectiveness between the parenting programmes? • Were improvements maintained one year later? Local authority activity and parent engagement • The sample comprised 6143 parents from 43 local authorities. • Local authorities varied greatly in their responses, from 754 to just 2 parents. Parent questionnaires were returned by 43 Local Authorities (LAs): 22 of the 23 Wave 2 and 21 of the 24 Wave 3 LAs in our sample. Data were available on 6,143 parents (4,223 Wave 2 and 1920 Wave 3). The single largest proportion of parents was from Greater London 23 (17%) and the South East regions (17%), but parents were drawn from across all regions of England. The highest numbers of parents were not surprisingly from Wave 2 LAs (4223) with relatively fewer returns (1920) from Wave 3 LAs as they started a year later. Some LAs returned substantially more questionnaires than others. The largest return was from an LA with 754 parents representing 12% of the total sample and the smallest was just 2 parents, Appendix 1 Table A.2). Overall 10 LAs accounted for over half (52%) of all the parents. However 26 LAs provided reasonably large samples, returning more than 100 parents. A total of eight programmes were being used in the PEIP with Triple P the most dominant (attended by 52% of all parents). However only four programmes (Triple P, Incredible Years, SFP 1014 and SFSC) were taken by significant numbers of parents. As shown in Table 2.1 together these four programmes account for 94% of all parents in the sample. Table 2.1 Number of parents by programme Programme Wave 2 Wave 3 Total parents % all parents FAST SFP 1014 Incredible Years SFSC Triple P Parent Power STOP Parents Plus 71 590 522 565 2390 85 0 0 33 379 260 303 781 19 102 43 104 969 782 868 3171 104 102 43 1.7% 15.8% 12.7% 14.1% 51.6% 1.7% 1.7% 0.7% Total 4223 1920 6143 100.0% Twenty LAs were running a single programme. The remainder were running multiple programmes, with 11 LAs running two programmes, 10 running three programmes and 2 running four or more. However where multiple programmes were running it was usual for one programme to predominate (Appendix 1, Table A.2). 24 2.3 Parent characteristics • Parents tended to be more socially disadvantaged than the general population. • Most parents attending a parenting programme were female (85%). • As they started their group, parents generally had significantly lower mental wellbeing and higher levels of parenting laxness and parenting over-reactivity than the general population. These are risk factors for negative child outcomes. 2.3.1 Parent demographics Relationship to child The vast majority of those involved in the programme (91%) were the child’s biological parents (Table 2.2). The ‘other’ group (3%) was in almost all cases a grandparent. Table 2.2 Parent relationship to child Relationship to child Biological parent Step parent Parent's partner (living together) Adoptive parent Foster parent Other relationship Total Frequency % 5413 147 141 59 36 181 5977 90.6 2.5 2.4 1.0 0.6 3.0 Note: 166 missing cases Parent gender The vast majority (85%) of parents involved in the programme were female (Table 2.3). Table 2.3 Parent gender Parent gender Male Female Total Frequency % 888 5207 6095 14.6 85.4 100.0 Note: 48 missing cases. Family structure A high proportion of the parents (44%) were living in single parent households (Table 2.4). This is almost twice the national average of around 24% (Strand, 2010). 25 Table 2.4 Family structure Family structure Frequency % 2580 3299 5879 43.9 56.1 100.0 Single parent Living with partner or other adult Total Note: 264 missing cases. Housing A high proportion (63%) of parents was living in rented accommodation as opposed to owning their own property (Table 2.5). This is 2.5 times the national average of around 27% (Strand, 2010). Table 2.5 Housing statistics Housing status Own property Rented property Other Total Frequency % 1873 3653 312 5838 32.1 62.6 5.3 100.0 Note: 305 missing cases. Parental health Parents were asked if in the last six months whether they had sought help from a number of professionals. More than two-thirds of the parents (69%) had sought help from one or more professionals (Table 2.6). The most frequently cited professional was the family doctor (49%) followed by social worker (21%), counsellor (15%) and psychiatrist (9%). Table 2.6 Parental Health Role frequency % 3010 49.0 Psychiatrist 525 8.5 Counsellor 910 14.8 Social worker 1312 21.4 Other professional 1456 23.7 Any of the above 4253 69.2 Family doctor 26 The category ‘Other professionals’ included a wide and varied range, including teachers, health visitors, psychologists, community workers, community or school nurse, family support workers, occupational therapists, police and behaviour support teams. Parents’ highest level of education Just under one-quarter of parents (24%) reported they had no educational qualifications, and around one-third (30%) reported their highest educational qualifications were fewer than five GCSEs (Table 2.7). Relatively few parents reported their highest educational qualification as five or more GCSE at A*-C, or A/AS levels. The sample is therefore skewed to parents with low educational qualifications. However nearly one-third of parents (31%) reported being educated to higher education levels, including 11% who reported their highest educational qualification as a degree. There is clearly strong heterogeneity in the educational levels of the parents involved in the programmes. Table 2.7 Parents’ educational qualification Educational qualifications Freq. % No qualifications Fewer than five GCSEs Five or more GCSE at A*-C A/AS level HE below degree Degree Total 1327 1707 546 312 1106 638 5636 23.5 30.3 9.7 5.5 19.6 11.3 Note: 507 missing cases Parent ethnicity The majority of parents (81%) were of White British ethnicity (Table 2.8). The largest minority groups were Asian groups (Indian, Pakistani, Bangladeshi, Other Asian) who constituted 8.2% of the sample and Black groups who constituted 4.3% of the sample. 27 Table 2.8 Parents’ ethnicity Ethnic group Frequency White % UK Census 2001 4992 83.3 92.1 4840 80.7 na 35 .6 na Traveller-Irish Heritage 1 .0 na Gypsy/Roma 9 .2 na 107 1.8 na 161 2.7 1.2 Mixed White and Black Caribbean 74 1.2 na Mixed White and Black African 22 .4 na Mixed White and Asian 30 .5 na Any other mixed background 35 .6 na 491 8.2 4.0 Indian 138 2.3 1.8 Pakistani 182 3.0 1.3 Bangladeshi 131 2.2 0.5 40 .7 0.4 256 4.3 2.0 Black Caribbean 100 1.7 1.0 Black African 148 2.5 0.8 8 .1 0.2 7 .1 0.4 88 1.5 0.4 5995 100.0 100.0 White British White Irish Any other white group Mixed heritage Asian Any other Asian group Black Any other Black group Chinese Any other ethnic group Total Note: 148 missing cases. na = not applicable Relative to the UK 2001 census, minority ethnic groups are over-represented, constituting 19.3% of the sample but only 7.9% of the national population. However this is likely to reflect (i) the higher minority ethnic population among the younger generations, particular for those with young families, and (ii) the high proportion of the sample drawn from greater London where minority ethnic groups are particularly concentrated. For example it is notable that in the LA with the largest number of returns, minority ethnic parents constituted 84% of the sample. 28 2.3.2 Parent mental well-being and parenting skills Parent mental well-being Parental mental well-being is a protective factor for child outcomes. The Warwick-Edinburgh Mental Well-Being Scale (WEMWBS) provides an indication of the mental well-being of the parents as they started their parenting group. It comprises 14 items scored from 1 (none of the time) through to 5 (all of the time). Examples are: ‘I’ve been feeling good about myself’; ‘I’ve been feeling useful’. A WEMWBS score was calculated where at least 12 of the 14 items were completed. The measure was highly reliable with a Cronbach’s alpha of 0.93 11 . High scores represent greater mental well-being. Normative data are available from Tennant et al. (2007) based on 1,749 respondents in the September 2006 wave of the Scottish Health Education Population Survey (HEPS) and the 2006 “Well? What do you think?” Scottish survey on attitudes to mental health, mental well being and mental health problems. The PEIP sample scored significantly below the national population norms as they started their parenting programme (Figure 2.1). For the PEIP sample the median score was 43 (inter-quartile range 36 to 51) while for the national standardisation the median score was 51 (inter-quartile range 45-56) 12 . Thus approximately 75% of the PEIP parents score below the national median on mental well-being, a substantial difference. 11 Cronbach’s alpha is a measure of the internal consistency of the scale. . The mean WEMWBS score for the PEIP sample was 43.1, SD=10.9, n=5916. The median and inter-quartile range are reported in the text because this is how the national norms are reported. 12 29 Figure 2.1: Comparison of median and inter-quartile range for scores on the Warwick-Edinburgh Mental Well-Being Scale as parents started their programme Parenting laxness and parental over-reactivity Parenting style is a predictor of child outcomes. The Parenting Scale (Irvine, Biglan, Smolkowski & Ary, 1999) was used to identify parenting style. Two dimensions of parenting, parental laxness and parental over-reactivity, are identified with each measured by six items on a 7-point scale, range 6-42. For example, a parent responds to this laxness item ‘If my child gets upset when I say “No”’, by choosing on a 7 point scale from ‘I back down and give into my child’ to ‘I stick to what I said’. The thirteenth item on monitoring the child’s activities does not contribute to these two dimensions but is included in the total score range 13-91). Scale scores were created where there were responses to at least five of the six items. Both measures were of good reliability with a Cronbach’s alpha of 0.78 for the Parenting Laxness scale and 0.74 for the Parental over-reactivity scale. Consistent with the original scale, items are scored so that high scores indicate less effective disciplinary practices. The correlation between the parenting laxness score and the parenting over-reactivity scores was significant but of a relatively small magnitude (r = .33, n = 5770, p < .0005). The two scales are therefore measuring different aspects of parenting style. The pre-course mean scores on the parenting measures are given in Table 2.9. 30 Table 2.9 PEIP pre-course parenting scores Mean SD 13 N Parenting Laxness score 21.6 7.2 5,869 Parenting Over-reactivity score 22.4 6.9 5,856 Parenting total score 46.9 12.1 5,818 Parenting Measure 2.4 Child characteristics • The children about whom their parent had most concern were: o mainly boys (61%: 39%), of mean age 8.6 years. o over four times more likely than national average to have a statement of special educational needs o six times more likely to have significant behavioural difficulties than the national population. 2.4.1 Demographics Child gender Parents were asked the gender of the child about whom they were most concerned. A majority (3539 or 61.3%) were boys compared to 2234 (38.7%) girls. Child age Child’s age was missing for quite a large proportion (8.8%) of the sample. For those parents who did report the child’s age, the mean age was 8.6 years (SD 3.9 years). However only just over half (54%) were in the PEIP target 8-13 age range. Relatively few were aged 14 or above (9%) but over one-third (37%) were aged 7 or below (Table 2.10). This indicates that LAs prioritised the PEIP target age range but also included younger children. Table 2.10 Child age Age range Frequency % 0-7 years 8-13 years 14+ years Total 2083 3013 509 5605 37.2 53.8 9.1 100.0 Missing cases 538. 13 SD: standard deviation, a measure of distribution of the scores 31 Figure 2.2: Age distribution of target child Special Educational Needs Over 1 in 10 parents (11.5%) reported that their child had a statement of special educational needs SEN (Table 2.11). This compares to a national figure of 2.7% (DfE, 2010). Table 2.11 Prevalence of special educational needs SEN Frequency % No 5435 88.5 Yes 708 11.5 Total 6143 100.0 Educational support Nearly one-third (32%) of parents reported that their child was receiving extra support (Table 2.12). Table 2.12 Prevalence of additional educational support Educational support No Yes Total Frequency % 4198 1945 6143 68.3 31.7 100.0 In answer to an open-ended question to specify the extra support, a wide range of support was mentioned: teaching assistants, one-to-one support in the classroom/school, anger 32 management, booster classes, counselling, dyslexia / dyscalculia, home tutoring, learning mentors/support and nurture classes / groups. Entitlement to a Free School Meal (FSM) Just under half (49%) of the children were entitled to a free school meal, more than three times the national average of 16% (Strand, 2010). Table 2.13 Entitlement to free school meals (FSM) FSM status Not entitled FSM Entitled FSM Total 2.4.2 Frequency % 3141 3002 6143 51.1 48.9 100.0 Child Behaviour (Strengths and Difficulties Questionnaire) The parents rated the behaviour of their target child on the Strengths and Difficulties Questionnaire (SDQ). The SDQ provides four scales representing separate problems: Emotional symptoms; Conduct Problems; Hyperactivity and Peer Problems. These four scales are summed to provide a Total Difficulties score. In addition there is a five item Prosocial scale that measures positive behaviours and an Impact scale which measures the extent of the impact of the child’s difficulties on the family, school and wider community. A mean score for each SDQ scale was calculated where at least four of the five items had been completed for a case. The SDQ total difficulties score was created by summing the four problem scales where four valid scores existed. SDQ scores could be calculated from the ratings of 5702 of the 6143 parents. Of the parents where scores could not be calculated, around one-third had not completed any of the SDQ questions and the rest had completed fewer than 20 of the 25 SDQ items. Reliability estimates were calculated using Cronbach’s alpha. Peer problems had lower reliability (.62) but all other scales were at least 0.70 or above. These indicate high reliability given they are short scales with only five items per scale. The Total Difficulties scale has the greatest reliability (0.83) and this measure should be preferred in any modelling of change or improvement, where measurement reliability is an important concern. The SDQ Total Difficulties score was also normally distributed, making it particularly robust for parametric statistical analysis. 33 Comparison to national norms Mean SDQ problem scale scores for the PEIP sample were significantly higher than the national average, and the prosocial score was significantly below the national average (Table 2.14). Not only were these differences statistically significant they were also extremely large. The last column expresses the difference in mean score between the PEIP sample and the national average in terms of the standard deviation of the national sample. The PEIP sample differed by well over a whole standard deviation from the national average in all cases and substantially more for conduct problems (1.7 SD) and the Impact of the child’s difficulties (2.4 SD). Table 2.14 scores Comparison of PEIP sample against the national average: SDQ mean National average PEIP SDQ scale Emotional Symptoms Conduct problems Hyperactivity Peer problems Prosocial scale Total Difficulties Impact score N Mean SD 5,871 5,856 5,840 5,851 5,891 5,702 5,778 3.8 4.5 6.3 3.3 6.4 17.9 3.0 2.6 2.5 2.7 2.3 2.3 7.3 2.9 Mean 1.9 1.6 3.5 1.5 8.6 8.4 0.4 SD 2.0 1.7 2.6 1.7 1.6 5.8 1.1 Difference (SD Units) 1.0 1.7 1.1 1.1 -1.4 1.6 2.4 Notes: National norms are drawn from interviews with the parents of a nationally representative sample of 10,298 pupils aged 5-15 (see Meltzer et al., 2000) SDQ classifications Another way of comparing the PEIP sample to national averages is by classifying the SDQ continuous scores as normal, borderline and abnormal, using the published cut scores (available from http://www.sdqinfo.com/b1.html). The results for the PEIP sample are given in Table 2.15 alongside the national averages and in Figure 2.3 for the three main child measures: SDQ Conduct problems, Total Difficulties and Impact. Approximately 57% of PEIP children had substantial behaviour problems (classified as ‘abnormal’ on the SDQ Total Difficulties scale), compared with the national average of just under 10%. A similar pattern of over-representation was observed for all four problem scales. 34 Table 2.15 Comparison of PEIP sample against the national average: SDQ classifications SDQ measure Emotional symptoms % Normal National PEIP 80.8 48.8 % Borderline National PEIP 7.8 12.7 % Abnormal National PEIP 11.4 38.5 Conduct problems 76.4 23.0 10.9 14.8 12.7 62.2 Hyperactivity 77.9 39.0 7.4 12.6 14.7 48.4 Peer problems 78.0 40.0 10.2 15.7 11.7 44.3 SDQ Total Difficulties 82.1 28.8 8.2 14.5 9.8 56.7 Prosocial scale 95.0 65.1 2.7 15.5 2.3 19.4 SDQ Impact 83.4 27.6 7.8 11.2 8.8 61.2 Note: For sample size see Table 2.14. Figure 2.3 Comparison of PEIP sample against the national average: SDQ ‘abnormal’ scores 70 60 50 40 National 30 PEIP 20 10 0 Conduct problems Total Difficulties 2.5 Impact Measuring pre-course to post-course change • 53.5% of parents completed post-course questionnaires • Non-responders did not differ substantially from those that did respond in terms of pre-course scores • Parents who did not complete their programme were more likely to be single parents, to have lower mental well-being and higher parenting laxness, as they started their programme. 35 In order to measure the impact of the programmes we need to measure changes on our measures between pre- and post-course. This is complicated by parents dropping out of their programme. The question arises – are these parents different from those who completed? If there are significant differences there is a danger of over- (or under-) estimating the effects of the programmes. As with the Pathfinder, we know that we did not receive all post-course questionnaires, for several reasons: some parents left the programme but in other cases there were administrative problems, e.g. the parents completed the programme but were not given the questionnaire or we did not receive their responses. In this section we examine the overall response rate and that for each programme. We then go on to explore reasons for non-responses. We examine whether the non-responders (for whatever reason) differ from the responders and also compare those parents we know dropped out from those that responded. 2.5.1 Post-course booklet return rate Of the 6143 parents who completed pre-course booklets, 3325 (53.5%) returned post-course booklets (Table 2.16). Facilitators were asked to report on the reason for parent noncompletion of post-course booklets and for 16% of parents the facilitator provided a reason. These included (a) the parent did not complete the programme or only completed a limited number of sessions (12%) (b) the parent completed the programme but declined to complete the booklets (1%) or (c) the parent did not complete the booklet for some other reason (2%) (e.g. transferred to another group, moved away from the area, parent ill health, sick child or got job). For a large proportion of parents (31%) there was no facilitator’s report from the group, so reasons for non-completion are not known. However it seems likely that this reflects issues of administration at the group level, rather than individual parent non-response. We can get an indication of this by analysing the group level data. Pre-course data were drawn from a total of 860 PEIP groups (average group size was 7.2, SD = 3.3, range 1-22). Post-course booklets were received for all parents in 132 groups and from one or more parents in 547 groups. However there were 181 groups where no post-course booklets at all and no facilitator form were returned. This suggests a substantial proportion of the non-response (19%) is due to administrative problems at the group level rather than parent drop-out (Table 2.16). It is important to note, 36 therefore, that ‘drop out’ cannot be calculated simply as those who did not respond, i.e. 100% - 53.5% which gives 46.5%. We know 12% dropped out but there are likely to be drop outs from the groups where facilitators did not supply a reason or there was no response at all from a group (Table 2.16). The data from LAs reported in Section 5 indicate a reasonable estimate is a drop out of about a quarter, as the 15 LAs in the cost effectiveness study report an average of 73% completion rate. Table 2.16 Pattern of response at post-course Post-course response Responded Did not complete programme Declined booklet Other reason Facilitator did not specify reason Whole group non-response Total Frequency % 3325 722 84 126 748 1138 6143 53.5 11.8 1.4 2.1 12.2 18.5 100.0 In terms of valid post-course data, 3319 parents (52% of the pre-course sample) provided useable data for at least one or more of the scales. The actual number of valid cases varied across the measures, from 3199 (SDQ Total Difficulties) to 3265 (Parent Mental Well-Being). 2.5.2 Do those responding to the post-course differ from non-responders? 14 In this section we compare the responders against all the 46.5% of parents who were nonresponders at post-course. In the next section we compare responders with those parents known to have dropped out. There were some significant differences between post-course non-responders and responders in demographic characteristics. Those who did not respond at post-course were more likely to be socio-economically disadvantaged than responders: specifically they were more likely to have no educational qualifications (28% vs . 20%, p < .001) less likely to own their own property (28% vs. 35%, p < .001), more likely to have a child entitled to FSM (52% vs. 47%, p < .001), and more likely to be from single parent families (47% vs 42%, p < .001). There were no significant differences in terms of parent gender or ethnicity, or child age, gender or SEN. 14 We use the term ‘non-responders’ to describe parents for whom we received pre-course questionnaires but no post-course questionnaires. As explained in the previous section, this group comprises those who actually dropped out and those where there was an administrative error. 37 However it is differences in pre-course scores that are most salient to the comparison. There were no significant differences between responders and non-responders on any child behaviour measure, parent over-reactivity or parenting total score. There were only two significant differences (p < .01) indicating those who did not respond at post-course were more likely to have lower mental well-being and higher parenting laxness scores at the start of the programme than those who did respond. However these differences, although statistically significant because of the large sample size, were very small, for example the mental well-being score difference was less than one score point on a measure with a SD of over 10 points. In sum there is no evidence that non-responders at post-course differed substantially from those that did respond in terms of pre-course scores. 2.5.3 Are there particular characteristics of those who dropped out of the programme? In this section we focus on the sub-group of non-responders (n = 722) that did not complete the programme – the ‘drop outs’ (see Table 2.16). This was the only group of nonresponders that differed significantly on any measure from those known to have completed the programme 15 . This group of parents had significantly lower mental well-being scores at pre-test (mean difference = 2.1 points, p < .001) and higher parenting laxness (mean difference = 1.4, p < .001) than those who responded. Parents who were reported to have dropped out of the programme were not significantly more socio-economically disadvantaged or less educationally qualified than the other nonresponse groups. They were however particularly likely to be single parents (51%, the highest of any group and compared to the whole group average of 44%). The most distinctive factor about parents who were identified as dropping out was their lower average mental well-being at pre-course, their higher average parenting laxness and their single parent status. In conclusion, there is some evidence to suggest that the particular group identified as not completing the programme (n = 722, 11.8% of the total sample) may differ in particular ways from those who responded at post-course. However overall parents without a post-course response (n = 1138) do not differ significantly in their pre-course scores from those who did respond, and can be considered to be approximately missing at random. As a result subsequent analyses will focus on the change in scores for those parents with both precourse and post-course data. It is important to note though that because the minority of 15 A one-way ANOVA with post-hoc multiple comparisons of means using Bonferroni tests. Full details of analyses are presented in the Technical Report http://www2.warwick.ac.uk/fac/soc/cedar/peip/ 38 parents who dropped out of the programme were not a random sample this may introduce an element of bias, although it is not possible to estimate this with any precision. Our conclusions will apply only to those parents who undertake and complete a PEIP programme. 2.5.4 Return rate by programme Table 2.17 presents the post-course response rates by programme. Table 2.17 Post-course response rates by programmes Programme Pre- course Post- course % with post-course 104 782 104 43 969 868 102 3171 6143 44 444 50 34 565 470 56 1656 3319 42.3% 56.8% 48.1% 79.1% 58.3% 54.1% 54.9% 52.2% 54.0% FAST Incredible Years Parent Power Parents Plus SFP 1014 SFSC STOP Triple P Total Appendix 4 includes a one-page summary of the number of responses, mean and SD of the pre-course and post-course scores, the improvement in scores and the effect size for each of the above eight programmes for each outcome measure. However in subsequent analyses of programme effectiveness, only the four programmes with significant numbers of parents responding to the post-course questionnaire can support detailed analysis. These programmes are: • Incredible Years (IY) (n=444) • Strengthening Families 10-14 (SF10-14) (n=565) • Strengthening Families Strengthening Children (SFSC) (n=470). • Triple P (n=1656) Response rates for these four programmes were broadly similar, ranging from 52.2% for Triple P to 58.3% for Incredible Years. 2.6 Did PEIP improve parent and child outcomes? • There were significant improvements on all parent and child outcomes following the parenting programmes, for all four programmes examined. 39 2.6.1 Overall effect averaged across programmes The first set of analyses aims simply to establish whether there is significant change in the parent and child outcome measures for the PEIP intervention overall, combining all programmes together. Change scores (post-test score minus pre-course score) were calculated for all scales. The results are presented in Table 2.18. The mean and SD of the pre-test and post-test scores are given with the change expressed as an effect size (ES) 16 . Table 2.18 Improvements in parent and child outcomes with effect size (All PEIP programmes) Per Protocol Analysis Measure Occasion Mean N SD Effect Size Parent measures Parent mental well-being Parental laxness Parental over-reactivity Parenting total score pre-course post-course 43.5 51.6 3160 3160 10.8 9.5 0.79 pre-course post-course 21.3 16.5 3125 3125 7.2 6.5 -0.72 pre-course post-course 22.5 16.9 3140 3140 6.9 6.4 -0.85 pre-course post-course 46.6 35.7 3093 3093 12.1 11.9 -0.91 pre-course post-course 3.8 2.9 3154 3154 2.6 2.5 -0.37 pre-course post-course 4.4 3.4 3144 3144 2.5 2.3 -0.45 pre-course post-course 6.2 5.3 3138 3138 2.7 2.7 -0.34 pre-course post-course 3.3 3.0 3131 3131 2.3 2.2 -0.15 pre-course post-course 17.8 14.5 3028 3028 7.2 7.4 -0.45 pre-course post-course 6.4 6.9 3165 3165 2.3 2.2 0.22 pre-course post-course 3.0 1.7 3046 3046 2.8 2.4 -0.52 Child measures Emotional symptoms Conduct problems Hyperactivity Peer problems SDQ total difficulties Prosocial scale SDQ Impact score Notes: Table includes only parents with valid pre-course and post-course scores on the relevant measure. All changes are statistically highly significant (p < .001). 16 Effect size measure is Cohen’s d. An effect size of .02 is small, .5 is medium and .8 is large. Negative values indicate that improvement was indicated by a lower post-course score, e.g. reduced parenting laxness or over-reactivity. 40 Parent outcomes There are substantial changes particularly for the parenting measures with an effect size of .79 for improvement in mental well-being and effect sizes of -.72 and -.85 for reductions in parenting laxness and over-reactivity respectively. Effect sizes for the child measures are somewhat lower, but still very substantial particularly for Conduct Problems (ES = -.45), SDQ total difficulties (ES = -.45) and for SDQ Impact score (ES = -.52). Figure 2.4 graphs the effect sizes showing the improvements on the parent and child measures. All are statistically highly significant. Improvements are particularly large over the period of the parenting programme for the three parent measures, shown by effect sizes of about 0.8 17 . Figure 2.4: Improvements in parenting and child outcomes from pre- to post-course Notes: Effect size: 0.2 = small, 0.5 = medium, 0.8 = large improvement To keep all data on a common scale the absolute effect size (regardless of sign) is shown. Effect sizes are an established and standardised method of reporting the degree of change resulting from an intervention. Other metrics are possible. For example, overall 79% of parents showed an increase in mental well-being, 74% showed a reduction in laxness and 17 In Section 6.2 we compare these results with recent UK randomized control trials that have SDQ data for child outcomes. In summary, the pre-post course gains in mean scores are comparable. 41 77% showed a reduction in over-reactivity. Additionally, the average mental well-being score increased from the national 25th percentile at pre-course to the national 50th percentile at post-course, a substantial improvement and indicating that, by the end of the programme, the average mental-well being score for the parents did not differ from the national average. Child outcomes Figure 2.4 also shows improvements in the children’s behaviour as rated by their parents, represented by three measures from the Strengths and Difficulties Questionnaire (SDQ): Conduct problems, the Total Difficulties score, and Impact. Again, all improvements are statistically highly significant. The scale of the child behaviour improvements associated with the parents completing the programmes is in the borderline small to medium range (about 0.45 to 0.5). Table 2.19 and Figure 2.5 show the pre- and post-course percentages of children rated as having significant behavioural difficulties (‘abnormal’) with respect to our three main measures: (i) children’s conduct problems; (ii) an aggregate measure (SDQ total difficulties) of conduct problems, hyperactivity, peer problems and emotional symptoms; and (iii) the SDQ impact of the child’s behaviour problems. • The percentage of children with significant behaviour problems overall (SDQ total difficulties) fell from 56% to 38%, a reduction of about a third. • There was a similar reduction for the specific area of conduct problems (59% to 40%) and SDQ impact (62% to 36%). All these represent very substantial improvements, which are comparable to results from recent UK small scale trials (Section 6.2). 42 Table 2.19 Proportion of children rated as ‘normal’, ‘borderline’ and ‘abnormal’ on the SDQ at pre-course and post course. N % Normal % Borderline % Abnormal Pre-course Post-course 3154 3154 49 64 13 11 38 25 Conduct problems Pre-course Post-course 3144 3144 23 41 15 18 61 41 Hyperactivity Pre-course Post-course 3138 3138 40 55 13 12 47 33 Peer problems Pre-course Post-course 3131 3131 41 46 15 17 43 37 SDQ total difficulties Pre-course Post-course 3028 3028 29 48 14 14 57 38 Prosocial Pre-course Post-course 3165 3165 66 73 15 15 19 13 SDQ Impact Pre-course Post-course 3046 3046 27 52 11 11 62 36 Measure Occasion Emotional symptoms Figure 2.5 Percentage of children rated by their parents on the SDQ at pre- and post-parenting course stage as having significant behavioural difficulties. 43 2.7 Effects of parent background and programmes • PEIP programmes were broadly effective for parents and children across the full range of background variables. • All four programmes showed significant improvements on all measures • There were some significant differences between programmes but these were relatively small in scale o Parent outcomes: For parent mental well-being and parenting laxness, SFP 1014 was less effective than Triple P, but no other differences were statistically significant. For parenting over-reactivity both SFP 10-14 and Incredible Years were less effective than Triple P, but no other differences were statistically significant. o Child outcomes: There were no differences between programmes for child SDQ total difficulties or for SDQ impact. However Triple P was significantly better than the other three programmes at reducing reported child conduct problems. In this section we explore whether the positive effects of the parenting programmes varied with respect to different parent or child characteristics, for example the parents’ educational level 18 , or whether the programmes had different effects. The specific questions asked through the statistical analysis were: • Are there any significant relationships between parent or child variables and improvement? i.e. is PEIP more effective for some types of parents / children than others? • Are there any differences in the degree of improvement between different PEIP programmes? While the PEIP original four recommended programmes all have evidence of efficacy from trials, were they equally effective in the context of the national roll out of PEIP? 19 18 Analysis of variance (ANOVA) of change from baseline, defined as post-test minus pre-test, was employed to assess the effect of various factors on improvement. ANOVA of change scores, rather than analysis of covariance (ANCOVA) with pre-test score as a covariate was used, as a review of the literature indicates that, while ANCOVA may have greater power in randomised designs, ANOVA of change scores is less biased in non-randomised studies of pre-existing groups, such as those defined here (Van Breuken, 2006). This is because the assumption in ANCOVA of equal pre-test means is violated in the present study where there were significant differences between programmes in precourse scores. 19 At the start of the PEIP, only five programmes were approved, as all had evidence for efficacy: Triple P, Incredible Years, Strengthening Families, Strengthening Communities, FAST and Strengthening Families Programme 10-14. Subsequently, LAs were allowed to fund other programmes. Our analysis examines the four for which we have an appropriate amount of data. 44 2.7.1 Is PEIP effective for the full range of parents and children? All parent and pupil background factors had significant associations with one or more initial scores. For example compared to parents with no educational qualifications, parents with degrees on average had higher mental well-being, less parenting laxness and over reactivity and lower reported child total and behavioural difficulties. A similar profile was evident for those living in rented as opposed to owner occupied housing. Single parents were more at risk for mental well-being and greater parenting laxness than those living with a partner, and so on. Similarly for the target children: parent rating of child total difficulties, conduct problems, and the impact of the difficulties were higher for boys than girls, for pupils on FSM and for those with SEN. Compared to children in the target age range (8-13), younger children had lower ratings for total difficulties and conduct problems and parents were less over reactive, while older children (14+) had significantly higher ratings for behaviour problems and their parents had lower mental well-being and greater laxness. Full details are given in Appendix 3. The question then arises as to whether PEIP was equally effective for these different parent and child groupings. Interestingly while many demographic variables had statistically significant associations with initial scores they had markedly lower associations with change in scores, or improvement as a result of PEIP. Detailed results are given in Appendix 3. Parents’ relationship to the child and the child’s gender had no association with improvement. There were also no parent or child background effects at all on change in child total difficulties. Some differences were apparent. Parents with no educational qualifications made greater improvements in mental well-being and greater reductions in laxness than parents with degrees, fathers tended to have smaller improvements in mental well-being and smaller reductions in laxness and over reactivity than mothers, and Black Caribbean parents were less likely to reduce in laxness and overreactivity than white British parents. Interestingly the age of the child had an effect on improvements in four of the six outcomes. Compared to the 8-13 age group, where the target child was aged 0-7 there tended to be less of a reduction in the impact of the child’s difficulties, while where the target child was older (14+) there tended to be a greater improvement in parent mental well-being and greater reduction in over-reactivity, conduct disorders and in the impact of the child’s difficulties compared to the 8-13 age range. 45 However, overall those associations with parent and child variables that did remain were substantially attenuated. Figure 2.6 shows the percentage of variance in pre-course scores (blue bars) and the percentage of variance in improvement or change score (red bars) accounted for by parent and pupil background variables. There is a separate graph for each measure. What is apparent is that even where an association is present with change score it is substantially reduced. While individual contrasts (like having no educational qualification vs. having a degree, see Appendix 3 for full details) were in some cases statistically significant, they explained little of the improvement in the measures, accounting on average for only around 3% of the variance in outcomes. Figure 2.6: The percentage of variance in pre-course scores (blue bars) and the percentage of variance in improvement or change score (red bars) accounted for by parent and pupil background variables. In summary, most demographic variables were unrelated to change, or where there were significant relationships the patterns were inconsistent across outcomes and explained only a small proportion of the variance in improvement. We therefore conclude the PEIP programmes were broadly effective for parents and children across the full range of background variables. 2.7.2 Differences in improvement between programmes In this section we examine whether there were any statistically significant differences between the programmes with respect to improvements on the parent and child measures. The analyses of programme effectiveness included only the four programmes with substantial numbers of parents which together accounted for 93% of all parents. These four 46 programmes were: Incredible Years (IY); Strengthening Families 10-14 (SF10-14); Strengthening Families Strengthening Children (SFSC); and Triple P. As the largest programme, Triple P was set as the base programme against which the other programmes were compared. There were small differences between programmes in the demographic profile of their parents and children in terms of the parent and child background variables. The analyses controlled for these differences by estimating programme effects after accounting for variation associated with all significant parent and child background variables. Unlike Appendix 3 these models included only those parent and child background variables that were significantly associated with each outcome. The models also included fixed effects for each of the 43 LAs to control for variation between LAs in outcomes. When interpreting these results it is important to remember that all four programmes examined were effective in improving parent and child outcomes, and that the differences found between programmes, while statistically significant, were relatively small. The data are presented graphically in Figure 2.7 (for the parent measures) and Figure 2.8 (for the child measures). For each outcome, these reference the mean improvement for each programme against the mean improvement for Triple P (which is represented by 0). The bars therefore tell us how much better, or worse, the other programmes have done compared to Triple P. To indicate whether these differences are statistically significant, the red line represents the 95% confidence interval for the mean of each of the other programmes. Where these do not cross the zero line we can be 95% sure that the score for that programme differed significantly from Triple P. The main findings were: o Parent outcomes: For parent mental well-being and parenting laxness, SFP 1014 was less effective than Triple P, but no other differences were statistically significant. For parenting over-reactivity both SFP 10-14 and Incredible Years were less effective than Triple P, but no other differences were statistically significant. o Child outcomes: There were no differences between programmes for child total difficulties or for impact. However Triple P was significantly better than the other three programmes at reducing reported child conduct problems. 47 Figure 2.7: Parenting outcomes: Mean improvement (relative to Triple P) with 95% confidence intervals Notes: The mean improvement in outcome for Triple P is represented by zero. The bars then show how well each of the other programmes has done relative to Triple P. To show whether these differences are statistically significant, if the red line indicating the 95% confidence interval does not overlap the zero line then that programme is significantly different from Triple P. - 48 - Figure 2.8: Child outcomes: Mean improvement (relative to Triple P) with 95% confidence intervals Notes: See footnote to Figure2.7 for how to interpret these graphs. 49 It is notable however that even after including parent demographics, programme type and LA the R2 for each outcome was never higher than 6.4% 20 . Therefore while these effects are statistically significant they explain only a small proportion of the change in scores. The majority of variance in change relates to individual differences between parents that is not captured in any of the broad parent or child demographics or structural features such as programme type or LA. Some parents were more responsive than others to PEIP, but this reflects unique features of the parents as individuals. We evaluate the extent to which particular characteristics of the group facilitator (particularly their qualifications and previous experience) may impact on outcomes in section 3.7. 2.8 Post-course evaluation - How was your group? • Parents were highly positive about the group experience, for all four programmes. • Over 98% were positive about most aspects of group leader style. • Over 95% were positive about most aspects of programme helpfulness. Parents were invited to complete a post-course questionnaire: How was your group? Generally parents were extremely positive in their evaluation of the group, as reported in Table 2.20. Factor analysis revealed that the items formed two groups: Group leader style and Programme helpfulness (Table 2.20). With respect to group leader style, over 98 per cent of parents gave positive ratings to six items, the exception being ‘I felt I had control over what happened in the group’. Even so, 93 per cent rated this positively also. About 95 per cent or more parents gave positive ratings for programme usefulness except for the item, ‘I had fewer problems than before coming to the group’ where 80 per cent rated this positively. This was also the item with the lowest proportion of parents giving a strong positive rating (37%). 20 Alternative models such as ANCOVA with the pre-course score entered as a covariate give a higher R2 (e.g. 25.8% for parent laxness). However this is because in such models the R2 represents the variation in the outcome score (or state) that can be explained, rather than the variation in the change or improvement score. With ANOVA the R2 is a direct measure of the amount of variation in change that can be accounted for. 50 Table 2.20: Responses to the ‘How was your group’ questions (%) Question N % Strongly disagree % Disagree % Agree % Strongly agree Group leader style The group leader(s) made me feel respected 3261 0.9 0.4 22.4 76.3 The group leader(s) understood me and my situation 3257 0.9 0.7 28.7 69.7 The group leader(s) worked in partnership with me 3247 0.8 0.7 33.0 65.5 I felt I had control over what happened in the group 3232 0.9 6.1 53.7 39.2 I felt I could be honest about my family 3267 0.8 1.1 32.5 65.6 The group leader(s) made me feel good about myself 3243 0.8 1.0 37.0 61.2 The group leader(s) were interested in what I had to say 3260 0.9 0.6 27.7 70.8 The parenting group has been helpful to me 3249 1.1 1.3 32.5 65.1 The programme helped me personally to cope with the problems I had 3224 1.2 3.8 43.8 51.2 The programme has helped me deal with my child’s behaviour 3216 1.1 3.6 46.9 48.4 I had fewer problems than before coming to the group 3189 2.3 11.7 49.6 36.5 Programme helpfulness All four programmes were rated positively and differences between programmes were relatively small. However, there were significant differences between programmes in parents’ ratings of both group leader style and programme helpfulness. On both dimensions, parents’ ratings’ for Strengthening Families Programme 10-14 were significantly lower than the parent ratings for the other three programmes. The contrast between Strengthening Families Programme 10-14 and Incredible Years for group leaders style was only just significant (p < .05) but the contrast between Strengthening Families Programme 10-14 and the other programmes for style, and all three contrasts for group leaders’ helpfulness were all highly significant (p < .001). (See Figures 2.9 and 2.10.) 51 Figure 2.9: Programme differences in ratings of group leaders’ style Figure 2.10: Programme differences in ratings of programme helpfulness These programme differences remained significant in an ANOVA with the five parent and child demographic variables (housing, education, ethnic group, child age and SEN) also included as controls. 52 2.9 PEIP Follow Up • The sample of parents followed up one year later was representative of parents completing a parenting group. • The improvements in parenting laxness, over-reactivity and child behaviour were maintained at follow up one year later. • Parental mental well-being reduced at follow up compared with post-course but remained significantly higher than when the parents started their programme. 2.9.1 Introduction The research has demonstrated that there are substantial improvements in a range of parent and child outcomes immediately after completing a PEIP group. An important question is whether these improvements are maintained in the longer term. This section reports the findings from 209 parents who returned usable follow up questionnaires one year after finishing their programme. The follow up sample though small was representative of the much larger sample of parents who completed post-course questionnaires (see section 2.9.3). The response rate in terms of total returns was 39% of the 705 sent out, including just under one-third (30%) completed booklets and 9% where the booklet was returned but indicating the addressee had moved away. High mobility was a key issue for the PEIP parents, with two-thirds living in rented property compared to around one-quarter nationally. Mobility was also particularly high in two Wave 2 LAs. The level of response achieved is good given the one-year gap, the absence of any financial incentive to return the booklet and the mobile nature of the sample population. 2.9.2 Description of sample Of the 212 parents who returned completed follow-up booklets, 209 parents could be matched to their pre-course and post-course responses. Parents were drawn from 20 LAs, and included parents who had undertaken one of the five PEIP recognised programmes (Table 2.21). 53 Table 2.21: Follow-up response by programme Frequency % 5 2.4 2 SFP 10-14 46 22.0 3 Incredible Years 42 20.1 4 SFSC 42 20.1 5 Triple P 74 35.4 209 100.0 1 FAST Total 2.9.3 Were those followed up representative of those completing parenting programmes? To address this question we compared the follow up sample with the total sample of those for whom we had usable post-course data (n = 3319). There were no significant differences between the follow-up sample and all post-course respondents in terms of single parent families, parent educational qualifications, parent gender or child age, gender, entitlement to a FSM or SEN. Neither were there any significant differences on any of the three parenting or three child behaviour measures. There were only two significant differences. A higher proportion of the follow-up sample was owner occupiers than among all post-course responders (51% vs 35%). This is not surprising since those living in rented accommodation tend to be less likely to be still living at the same address one-year than owner occupiers and so were inevitably less likely to be contactable. The follow-up sample also contained a higher proportion of ethnic minority parents (30% vs 18%). This simply reflects the fact that, because the PEIP programme had to have been completed one year prior to the follow-up, the follow-up parents were more likely to be drawn from Wave 2 LAs which on average were more likely to be urban LAs with a high proportion of minority ethnic parents (24% minority ethnic parents in Wave 2 compared to 9% in Wave 3). We conclude that the follow up sample (n = 209) was representative of those completing a post-course booklet (n = 3319). 2.9.4 Findings We compared the parent and child measures at three time points: pre-course, post-course and one year later (follow up). Our analysis 21 showed whether the improvements between pre- and post-course found from the total sample were maintained one year later. We present the data as graphs (descriptive statistics are available in the Technical Report 22 ). 21 22 One-way repeated measures ANOVA followed by Bonferroni post hoc comparison of mean scores http://www2.warwick.ac.uk/fac/soc/cedar/peip/ 54 Parental mental well-being As shown in Figure 2.11, parent mental well-being at post-course was significantly higher than at the pre-course (p < .001). The mean Mental well-being at follow-up had declined relative to immediate post-course (p < .001) but was still significantly higher than at precourse (p < .001). Figure 2.11: Mean parent Mental well-being scores at pre-course, post-course and follow-up with 95% confidence intervals. Parenting behaviours: Laxness and over-reactivity As shown in Table 2.22, there were significant reductions in the negative parenting behaviours of laxness and over-reactivity at post-test (p < .001) which were completely maintained at follow up with no significant differences between post-course and follow up scores. Figure 2.12 shows this for the Total score; the other two scales had very similar patterns. 55 Table 2.22: follow-up. Parenting style scores (mean and SD) at pre-course, post-course and Measure Pre-course Post-course Follow-up Parenting laxness Mean SD 19.8 7.2 15.8 6.0 16.5 6.3 Parenting over-reactivity Mean SD 21.5 6.7 16.6 6.6 17.3 6.6 Parenting total score Mean SD 43.9 12.0 34.8 11.6 35.9 12.0 Note: Based on 194, 192 and 192 parents for laxness, over-reactivity and parenting total score respectively with valid scores at all three time points. Mauchly’s test supported the assumption of sphericity. Figure 2.12: Mean Parenting style total score at pre-course, post-course and followup with 95% confidence intervals. Mean score with 95% confidence interval 48 46 44 42 40 38 36 34 32 30 Pre-course Post-course Follow-up Child problems: Strengths and Difficulties Questionnaire (SDQ) The data for child total difficulties are presented in the Table 2.23 and Figure 2.13. There were highly significant reductions in child difficulties at post test (p < .001) on all these measures which were completely maintained at follow up with no significant differences between post-course and follow up scores. Figure 2.13 shows the SDQ Total difficulties score – the other two scales had very similar patterns. 56 Table 2.23: Child Total difficulties score (mean and SD) at pre-course, post-course and follow-up. SDQ Total difficulties: SDQ Total difficulties: SDQ Total difficulties: Pre-course Post-course Follow-up Mean SD 17.2 14.3 14.1 8.0 7.8 8.6 Note: based on 180 parents with valid scores on all three occasions. Figure 2.13: Child Total difficulties score at pre-course, post-course and follow-up with 95% confidence intervals. A similar pattern was found then the percentages of children with substantial behavioural difficulties (the ‘abnormal’ category on the SDQ measure) - Figure 2.14. 57 Figure 2.14: Percentage of target children rated ‘abnormal’ for SDQ conduct problems and Total difficulties at pre-course, post-course and follow-up. Percentage of target children .60 .50 .40 .30 .20 .10 .00 Pre‐course Post‐course Follow‐up Pre‐course Post‐course Follow‐up Conduct problems 2.10 • SDQ total difficulties Qualitative evidence of impact Both parents and facilitators provided many detailed personal examples of the positive impact of the parenting programme, both immediately and later. In this section we report evidence of the impact of PEIP derived from interviews with parents, facilitators and LA operational and strategic lead officers. We explore the outcomes for parents, for their children and young people, for family relationships, and also for the professionals who referred parents. The kinds of evidence facilitators used in assessing impact on parents and children include: session evaluations, programme evaluations, parental and young people’s feedback during sessions and their own observation of family interaction or of parents’ and young people’s behaviour. Some facilitators also referred to their continuing knowledge of the family, and comments from staff at the schools attended by the young people who have attended programme sessions. Our report of the Pathfinder in 2006-08 (Lindsay et al., 2008) provides evidence on the three programmes studied at the time: Triple P, Incredible Years and Strengthening Families, Strengthening Communities. In this section we focus primarily on the two additional 58 programmes approved for the PEIP when it started in 2008: FAST and the Strengthening Families Programme 10-14. 2.10.1 Outcomes for parents Outcomes for individual parents varied in accordance with the reasons for their referral, or self-referral, and the complexity of their problems. The majority of facilitators and the parents themselves described positive outcomes, for example one facilitator said ‘…Parents…feel there is a big, positive difference in the way they see themselves, and in the way they parent their children as well.’, and from a parent: ‘It’s been a brilliant turnaround – hats off to them [facilitators]. They did what they said they’d do.’ Box 2.1 Improving life for a single parent and sons A father found himself in a desperate financial situation, unemployed and having moved to a new location, with the arrival of his three sons for a holiday who then decided they would prefer to live with him rather than with their mother. He came close to suffering a nervous breakdown and through a child care centre was introduced to a school nurse and a council carer. They referred him to a FAST programme where he was unable to communicate with anyone for the first few weeks of the programme. However the support of staff and parents ‘brought me out of my shell. It helped me 100 percent and I am back to normal and can mix now.' Both older boys, just entering their teens, went along from the second week onwards, and one who had been severely introverted began to go out more with his brother and brother’s friends. They were now both gaining additional confidence from joining a youth club and Outward Bound course, as suggested by the FAST team. The boys’ behaviour had shown significant improvement at home and in school. This father had much praise for the staff at his sons’ school, particularly for the FAST team who ‘are all trying to help the family group’. The family’s fortunes had also been improved by a move from the father's two bedroom flat into accommodation suitable for four. Facilitators working with parents/children in the course of their day role, as in the above example, described ways in which they were able to draw on the PEIP programme's strategies/principles in supporting the family in the home and/or school context. Parents of children with special educational needs and disabilities (SEND) (and indeed the children themselves) were among those who benefited in this way. The following example is taken from a parent interview: 59 Box 2.2 Support for a parent with SEN Both John and Sally attended a local special school, whose teachers were all described as ‘very supportive’. The father heard about the parenting programme through the schoolbased PSA (parent support adviser): ‘always available and there as almost like a counsellor, and easy to talk to for parents’. The PSA also facilitated the (Triple P) parenting programme and adjusted it to suit this family’s needs, incorporating symbols used in school to aid the boys’ learning. John used a communication aid and the facilitator was able to adapt symbols for use with that too. She also visited the children’s home and loaded a programme used at school onto the home computer. Facilitators pointed out that evidence for the huge benefit that parents derive from the programmes comes not only from self reporting but from observed behaviour, e.g. ‘Parents’ body language and interactions with others and managing their own families’ situation evidences their greater confidence through attending.’ (FAST facilitator). While the majority of parents and young people were reported to have ‘enjoyed’ their programme, there is plenty of evidence from quantitative evaluation, observation and from parents’ and young people’s feedback of the positive outcomes experienced by both groups. In addition, local evaluation post-course measures provided a clear indication of the positive changes parents had made in reaching for their goals, complementing parents’ self reported positive progress during reviews of each session. For example, in one LA an internal qualitative evaluation of longer term outcomes 6-12 months after programme completion was reported to be showing excellent results from a sample of 40 parents. A facilitator in another LA, in touch with a parent a few months after completing a Triple P group commented: ‘She said, you know, she keeps going back to her book. She says it has helped improve her relationship with her children and her children’s behaviour in general.’ Positive outcomes Participating parents were feeling more positive and confident about their parenting. They were making changes in their behaviour towards addressing the issues and problems pertinent to their individual family by: • Putting in place strategies learnt on the programme, e.g. routines and rewards, choices and consequences • Putting boundaries in place, which they had previously found difficult to do • Learning to say ‘no’ and sticking to this • Learning to stay calm • Using more praise • Discussing issues and difficulties with other parents 60 • Linking programme strategies with facilitators engaged with them in their day role (e.g. PSAs) In addition to improved parenting skills, increasing confidence engendered self development for some parents in the form of, for example: • Taking a role as programme co-facilitator (Strengthening Families Programme 10-14, Strengthening Families, Strengthening Communities), or graduating to be a partner with the FAST team. • Using skills gained on the programme to underpin a new career • Becoming involved with the school community • Taking a lead in organising social events with group members The opportunity for self-development is built into programmes that encourage suitable parents to co-facilitate groups or graduate to share in delivery of two years of post programme FASTWORKS sessions, and several LAs cited examples of successful development for parents in this way. In one LA offering FAST, parents trained as facilitators had progressed to form a consortium providing parenting programmes for the LA. In another LA the strategic lead offered case study evidence of parents progressing to a whole range of programmes linked to developing skills and confidence in preparation for further training and employment, all woven into the LA’s parenting offer. One mother, now volunteering at the Children’s Centre and supporting other parents had been an alcoholic. The operational lead reported that ‘The [Incredible Years] parenting course had been a key turning point…it was what kept her going, motivated her, and she’s just completely turned her life around.’ In another LA a parent learned to make cakes to serve to the group during her programme and has now been approached to make cakes commercially. Facilitators made the point that while changes may sometimes be viewed objectively as quite small, they may be of considerable significance to the individual. As regards rising self confidence, one mother now felt able to suggest a group outing to fellow (FAST) participants; with regard to improved experience of parenting children, one parent undertaking an SFP1014 group with his son said: ‘He’s come in every night on time and put the bins out when I asked him.’ 61 Negative outcomes There were a few comments reporting negative or less successful outcomes for parents. In these cases facilitators suggested that parents were unable to benefit from the programme because of: • Parents’ own mental health difficulties or anger about issues in their own lives • Parent’s unwillingness to attend the programme • Parents seeing no need to change their behaviour • Young person’s unwillingness to participate (applies to SFP10-14 and to FAST) One facilitator commented: ‘Parents who have not benefitted are generally those who have their own mental health difficulties, struggling with managing their own lives, so to be able to make changes in their life to affect their child is very difficult for them to do. Or where parents are negative to begin with, or complacent to begin with, it’s very difficult for them to implement the changes.’ The issue of parents' readiness to attend a parenting programme is addressed in Section 4.2.6 of this report; sometimes preparatory work or an alternative to a parenting programme may be needed. Moreover facilitators and operational leads have highlighted choice of programme and group composition as important issues in the referral process. Several facilitators had noticed reluctance on the part of a young person to attend a FAST or SFP10-14 programme, prompting the parent to withdraw them both from the programme. For example one mother with ten participating children who all behaved badly during the first two sessions had subsequently withdrawn, deciding that the programme was ‘not for me.’ This issue highlights the need for facilitators to consider, where applicable, both parent and young person when deciding upon programme suitability and allocation to a particular group. 2.10.2 Outcomes for children/young people Positive direct outcomes As might be expected (and hoped for) the positive direct outcomes most frequently mentioned by interviewees were: • Improvement in the child’s behaviour at home and/or in school • Increased confidence 62 One facilitator had found that two children described as previously unconfident ‘loners’ in school were able now post- FAST to join other children from the programme for play. The facilitator rated their proactivity in doing so as a remarkably successful outcome from the programme. In one LA it was reported that a pupil whose behaviour in school had improved was now much calmer after attending a (SFP10-14) programme. A parent described a significant change in her target son’s behaviour following an SFP10-14 programme: ‘He’s gone from a very naughty child: disobeying, stealing, he’s not as bad; he gets his ups and downs, being ADHD, but he knows not to bow to peer pressure no more. He’ll go off, had an argument with me, then he’ll come back and apologise and oh, my God, I didn’t think I’d ever have him doing that. And if it weren’t for that group I don’t think I would have got as far as I have with him now.’ (Parent) While improvements were attributed at least in part to parents’ participation (and sometimes also the child’s participation) on a programme, other changes at school or in the home may well have been a contributing factor. For example a boy who was unhappy at school during the time of the programme, had subsequently progressed, his relationship with his parents reportedly much happier, and with no adverse reports on his behaviour from the new school. Positive indirect outcomes In addition to the direct benefits for children/young people who attended the programmes, there were reports of indirect positive outcomes. In one LA children whose parents had attended a Triple P programme had found life a lot calmer at home. The facilitator said: ‘Children have commented that they have found differences in their parents, whether they’re aware of the parenting course or not.’ There were examples from different LAs of children remaining at home rather than being taken into care, or being able to return home following their parents’ completion of a programme. In one LA delivering a ‘tailored’ Triple P programme for parents of children in care, a 24 per cent reduction in looked-after children was reported, the interviewee claiming specific knowledge of four individual children who had benefited in this way. The operational lead in another LA also reported individual cases of families coming off the Child Protection register, following parents’ participation in Incredible Years. This was particularly salient in the context of an overall rise in numbers of children being taken into care. 63 Mixed outcomes A number of parents interviewed for follow up feedback during this final phase of the evaluation conceded that improvements that seemed secure at the end of the programme were not maintained, or were only partially maintained. For example, a mother reported that her 14 year old who had been attending school more frequently was absenting himself once more. Nevertheless, this parent asserted that she and her husband were taking more time to discuss problems with their son and that he had become more helpful at home and school, a positive change that she attributed to the (SFP10-14) programme. The complex difficulties faced by many families may mean that outcomes from attending a programme are only positive in part, or outcomes may be positive for a parent but have little impact on the child, or vice versa. Box 2.3 Dealing with a complex situation A parent reported that since the time of the first interview her 14 year old autistic son was now with foster parents. Before taking the (Triple P) programme, she had felt scared of him and found it difficult to take control because he was frequently ‘hurting me’. The programme had helped her to put boundaries in place. Following his placement the mother/son relationship had improved. She was able to draw on strategies learnt on the programme to work with the foster parents towards a common approach to behavioural problems, though acknowledging that the process was led by them: ‘I have to go with what they do and say’. She commented: ‘He is accepting routine a lot more and that the grown-ups are in control…I am working with carers and social workers and hope he may come home some day, but this will take a while.’ One facilitator suggested that the (SFP10-14) programme had not had positive outcomes for a boy who behaved very badly during the sessions he attended. He did not join in with any of the activities but wandered off to sit alone, and after several sessions was reluctant to attend again. His mother continued with the programme, however, and had learnt new strategies that she was beginning to find helpful in dealing with her son’s behaviour at home, though it was too early to say whether this would have a long term significant impact. Conversely, a mother who disliked using the SFP10-14 DVD continued to attend the group because she felt her daughter was benefitting. The daughter had rated the group 5/5 for enjoyment every week and her behaviour was reportedly much improved. Negative outcomes Reasons given by facilitators and parents for young people not benefitting from, or discontinuing attendance at sessions were as follows: 64 • Feeling at odds with others in the group because of age difference or other incompatibility (SFP10-14) • Unwillingness to attend with parent (FAST/SFP10-14) • Entrenched behaviour resistant to change Facilitators identified the importance of aiming for group compatibility. In the context of the youth groups the possibility that children aged 10 might be included in the same group with 14 year olds has clear potential for giving rise to difficulties (for the facilitators no less than for the participants themselves) or conversely, to benefits. Younger children might emulate the less desirable ways of older and more streetwise young people or young teenagers might simply be bored in the company of less mature children. One facilitator observed that while younger children enjoyed the ‘running around’ activities of the SFP10-14 programme, the older ones did not want to take part in them. These concerns run counter to the possibility that the older (perhaps) more responsible child could prove a positive role model or support the younger participant. For example on one SFP10-14 programme a boy who could read had helped an illiterate child in a ‘scavenger hunt’ game by sending him off to find items rather than making an issue of his illiteracy. As with the adult groups, facilitators’ judgment is crucial in assessing, as far as possible, whether participants’ individual characteristics make for a broadly compatible group. Facilitators and parents spoke of some young people's unwillingness to attend a group with a parent, for example: ‘A girl of about 14 refused to come to the (SFP10-14) programme any more – she just didn’t want to be there or be seen out with her parent. She is now in foster care. The relationship had truly broken down.’ This case is an example of intervention at a very late stage of the family’s difficulties, seemingly too late to have a positive effect on family relationships. One facilitator expressed this as follows: ‘If the relationship between parent and child has been very bad, behaviour is entrenched, it makes changes very much harder – the programme is just the beginning of a very long journey.’ Facilitators across programmes asserted, as did some parents, that undertaking a parenting programme is often not a 'quick fix' solution, nor necessarily the only intervention required. 65 2.10.3. Outcomes for family relationships There were many comments from facilitators and parents attributing improvements in parent/child relationships, relationships between partners, and between siblings to programme participation. The opportunity to discuss family relationships and other issues with other parents was very welcome across programmes (whether or not family sessions were an element of the programme). For example a facilitator reported that the parents of a child with ADHD who attended together: ‘found their Triple P programme really helpful. It made such an improvement to their life and in terms of understanding where their little boy was coming from, understanding more about his behaviour. This in turn helped him to manage his angry behaviour and find alternatives.’ (Facilitator) A parent from the SFP10-14 programme commented: ‘…at the end of being separated we would get together and we’d do something together, which was lacking in a lot of the families that went there, where the children didn’t do anything with the parents, you know. That helped a great deal, because you realised what you were missing out on.’ (Parent) Facilitators and parents found the ‘Special Time’ element of their SFSC programme important for helping parents to build up relationships with their children. For facilitators who worked with the SFP10-14 and FAST programmes the inclusion of children/young people was a critical element of the programmes’ success. For example, one facilitator noted that: ’Feedback from parents indicates that family sessions were particularly beneficial – a relaxed hour to spend together for families where parents do not normally spend a lot of time with their children. Many now have a family evening once a week.’ (Facilitator) The comment suggests that the time spent together as a family on programme activities gives the opportunity to work towards improving family relationships; at the same time modelling a way for families to adjust their home life, making space for enjoying time together. Comments from facilitators and parents confirmed that some families had started to do this, perhaps sitting down to a communal family meal for the first time. The benefits most frequently mentioned by facilitators and parents from the shared family sessions were: • Time together to share enjoyable activities • Time together to discuss issues/problem solve 66 • Parents’ increased understanding of their children, linked with improved family dynamics • Children’s increased understanding of their parents, linked with improved family dynamics • Changes in behaviour at home or in school (children) mirroring behaviour modelled during the sessions A facilitator for SFP10-14 commented that ‘Family activities after the meal are the major factor in effectiveness: parents and young people co-operating, and the facilitators modelling how to speak to the young people. I think parents get a lot out of that.’ Interviewees across the programmes made clear that the steps towards improvements in family relationships were often small, but significant for those families with complex needs, as expressed by one facilitator: ‘Some parents did not achieve the goals set for them at the beginning of the programme, but they did make some improvements in terms of communication with their children, feeling closer to them, and knowing the toolbox to use for misbehaviour.’ For some parents improved communications with a child represented the change that enabled the family to pull back from the brink of complete family breakdown, as expressed by one mother in Box 2.4. Box 2.4 Improving a mother-daughter relationship ‘Without that (SFP10-14) programme my daughter wouldn’t be here, she’d be somewhere else. I’d got to that stage where I am thinking: ‘No, I can’t do this anymore.’ You know, and I was willing to open that door and say: ‘Goodbye. Go to your dad’s.’ But no, it was definitely down to the programme. ‘Cos I mean to say when we had finished we was more…when she came back from school we would sit down and we would talk about the programme. And we’d talk about what we went through that day and things like that. And I found that my daughter would come and talk to me afterwards and she still comes and talks to me, which is nice because she would never talk before. She would bottle everything up or go and talk to her friends and what have you, but she would never say anything to me. She does now.’ 2.10.4 Outcomes for parents’ relationships with schools Many of the positive comments concerning outcomes for parents’ relationships with the schools their children attended came from facilitators and parents working with FAST, SFP10-14 and SFSC programmes. Improved parent/school relationships were attributed essentially to parents becoming more confident through their participation in the programme. There was evidence of increased parental involvement with the school at a number of levels: • monitoring their children’s education 67 • addressing with school staff issues of concern at an early stage • supporting the school’s social activities • participating in the life of the school as volunteers More parents were approaching staff by telephone or in person at the beginning or end of the school day to talk about troubling matters before they became major concerns or asking questions of teachers regarding their children’s education. One facilitator observed: ‘You see parents approaching staff…getting into the playground earlier, saying: ‘can I have a quick word with you?’ rather than ‘Oh, no, I haven’t got time now, I’m rushing off’. That improved.’ A parent commented: ‘A lot of the people who worked the FAST programme, I didn’t know before, and now when we go to drop the children off at school or pick them up everybody always says ‘hi’ and asks how you are whereas before when we didn’t know each other everybody used to walk straight past each other.’ Another parent commented very favourably upon the integrated support the family was receiving at the school attended by his sons, facilitated by attending the FAST programme: ‘The teachers are very helpful and understand our family circumstances. The head too is very helpful. We have had two meetings with the school nurse and one with the housing people. People from FAST were there, all trying to help the family group.’ There were examples too of increased numbers of parents supporting the school’s extracurricular activities. One head teacher in a FAST school reported a fourfold increase in parents attending the nativity play and much more active involvement in the Christmas Fair. At another level, facilitators commented that the SFSC programme had been successful in building up parents’ willingness to contribute to the running of the school, for example joining a parents’ focus group or becoming a school governor. Where parents were trained to deliver a programme, it was reported that parent to parent support benefited the school with regard to improved behaviour management, time keeping and discipline boundaries. Improved parent/school relationships were starting to have a beneficial effect generally upon pupils in terms of classroom behaviour and upon the behaviour of specific children; for example a deputy head teacher had said: ‘I can’t believe 68 how different this [primary school] child is’, following his parent’s participation in a Triple P programme. 2.10.5 Outcomes for community relationships Participation on a parenting programme had encouraged some parents to become involved in the wider community. One strategic lead commented: ‘It’s [SFSC] had a huge impact on community cohesion and networks beyond school’. A strategic lead explained that where children have come to the attention of the attendance service or police, families are being helped [through SFSC] ‘to think about themselves not just isolated within their families but within the wider community, which potentially effects a bigger change’. The community aspect of SFSC ‘does help people to think a little bit more about themselves in their context, within the community.’ Greater parental involvement in their community was evidenced by parents putting themselves forward to work in voluntary organisations or supporting/mentoring other parents as volunteers. The following factors were identified as giving rise to this change: • Increased confidence through participating in a parenting programme; • Participating on programmes with compatible parents with whom parents could gel; • Inclusion of a community focused element in the programme (SFSC); • Group participants continuing to meet after the programme. Post-programme meetings might be organised or informal, the latter perhaps taking the form of setting up a Facebook group or going out for a meal together. In one school Incredible Years programme finishers were hoping to set up a support group in the school with the help of the facilitator. In one LA, families were continuing to meet two years after completing FAST and the operational lead commented: ‘that builds community spirit and helps them support each other in the community. And what I’ve noticed as well is, … you can hear them making arrangements, saying: ‘well, I’m going there at the weekend, I'll give you a lift.’ FAST programme organisers envisaged that delivery would be organised through several hubs of parents. In practice few LAs found it possible to recruit parents in sufficient numbers to enable this to happen. In one LA where this was successful, however, links were built between the communities of Somalis, Congolese, Turks and Poles comprising each hub: the strategic/operational lead commented that this ‘brought together communities and they’d be communicating with each other. Now they would never have done that before’. In another, 69 FAST parents were planning FASTWORKS with the support of a community planner, building in special time with the children too. 2.10.6. Outcomes for practitioners as facilitators Over and above receiving their initial training and subsequently gaining satisfaction from seeing a favourable impact upon participating parents and young people, facilitators reported benefits from delivering the programmes at an individual level, and from using learning from their role as facilitators in their day to day work as follows: • Reflecting upon and evaluating their practice; • Becoming increasingly confident in their day role, including their one to one work with parents; • Increasing their understanding of more specialist areas of work (e.g. drug and alcohol issues); • Making links between the parenting programme(s) and their individual case work with children for consistency and continuity; • Co-facilitating with individuals from other services and agencies, giving insights into alternative approaches that might be used with parents and families. At a service level facilitators reported the following benefits in the context of increased multiagency working: • LA level support for working in an integrated way with other agencies • Increased understanding of the job functions and challenges of working in other agencies • Personal acceptance of facilitating as an element of day to day service rather than as an extra (although this view was not always reflected at management level). The following comment from a strategic lead encompasses the views outlined above: ‘The facilitators have now got increased knowledge about working with parents and families as well as practical skills of facilitation and group-work and also of linking in that multi-agency way; that all helps integrated working. Understanding one another, appreciating the difficulties that each agency is working under and understanding those roles and responsibilities.’ 70 2.10.7 Outcomes for professionals referring parents to programmes For those agencies referring parents to a parenting programme, PEIP had brought into focus the important role of parenting programmes as one of a range of approaches in helping parents on their caseloads, with a consequent rise in the number of referrals. At practitioner level, individual comments referred to: • Greater understanding of parents’ difficulties and the challenging circumstances in which some are living • Recognising parents’ need for parenting support • Knowing where to go to access parenting support for a parent in need. One facilitator had delivered a training session with her manager to the whole Youth Offending Team on how Parenting Orders might fit in relation to Triple P. Staff feedback indicated that they now realised their mistake in focusing solely on the child without considering pressures on the parent, and the potential consequences of these pressures for the child. At service level, and in the context of the development of multi-agency team working, professionals in referring agencies commented upon: • Increased ability to work effectively with other professionals • Increased sharing of knowledge and cross-fertilisation of ideas • Using the PEIP as the standard for specialist parenting work with other agencies and other issues, e.g. drug and alcohol problems. 2.10.8 Conclusions • Comments from Wave 3 interviewees reflected those from earlier Waves, and across programmes, in emphasising positive outcomes overall for participating parents, for children, for the schools, for family relationships and for community cohesion. • Parents and facilitators reported changes in their approach to parenting using strategies learned on their programme. • There were many examples of improved relationships between parents and children and family sessions were seen as particularly successful in this respect. • Self-development was a frequently mentioned outcome for parents, involving them in increased participation in the school’s governance and social activities, or more widely in the community. 71 • Outcomes for parents/children, particularly in the case of complex needs, were sometimes small but significant, sometimes mixed, but often incremental: a first step in a long ‘journey’. • A positive attitude towards the need for change (and young people’s willingness to attend) were important indicators of successful outcomes, without which participants showed little benefit, frequently withdrawing at an early stage. • The issue of ‘readiness’ to participate in a parenting programme emerged as an important issue affecting outcomes, some parents needing pre-programme support or signposting to some other intervention more appropriate to their needs. • Continuing support from a facilitator or other agency during the programme and for some time afterwards may impact upon the programmes’ sustained outcomes. • Group compatibility emerged as an important issue for parents and (where applicable) for young people. • On a professional level, facilitating practitioners gained a greater understanding of appropriate levels of parenting support and were transferring this understanding into their day to day roles, bringing additional benefits to parents and families. • In the context of multi-agency team working facilitators (working with co-facilitators) as well as professionals referring parents to programmes benefited from increased sharing of relevant knowledge across Services. • As reported at earlier stages of this evaluation, facilitators rated parenting programmes highly as one of a range of alternative or supplementary interventions. 72 3. THE FACILITATORS Main finding • There was no difference on most parent and child outcomes between lead facilitators with different levels of training or experience gained prior to PEIP, but non-graduate lead facilitators: o had greater impact on parents’ mental well-being than graduates o had slightly higher parent ratings of group leader’s style and group effectiveness 3.1 Introduction Facilitators are fundamental to the success of parenting programmes. The programmes vary in their requirements for those judged to be suitable for acceptance for training and in particular the prior qualifications and experience deemed necessary. This section reports on the characteristics of the 1227 facilitators whose details were received by 1 February 2011 in terms of demographics (gender, age and ethnicity), qualifications and experience delivering parenting programmes as a facilitator prior to the PEIP. In Section 3.7. we report on the relationship between facilitators’ characteristics and parent outcomes. 3.2 Facilitators’ demographics: Gender, age and ethnic group Gender and age Of the 1277 PEIP facilitators the vast majority 1142 (89.4%) were female and only 135 (10.6%) were male (Table 3.1). The largest percentage (37%) was aged 40-49, although a large proportion was in their 30’s (24%) and 50’s (22%). A minority (15%) were aged 29 or under and 2% were 60 and above. The male facilitators were on average significantly more likely to be younger than female facilitators (p < .05); 23% of males were aged 29 or less compared to 14% of females. Ethnic group Fifteen per cent of the facilitators were from minority ethnic groups, close to the 19% of parents from minority ethnic groups attending PEIP programmes (Table 3.1). This indicates a generally appropriate match of facilitators and parents in terms of ethnicity. Both figures are higher than the 8% ethnic minority UK population (UK Census, 2001) as a result of area and demographic factors, as explained in Section 2.3. 73 Table 3.1: Facilitators by ethnic group Ethnic group Frequency White British White Irish Any other White group Mixed White and Black Caribbean Mixed White and Asian Any other mixed background Indian Pakistani Bangladeshi Any other Asian group Black Caribbean Black African Any other Black group Chinese Other Total 3.3 1079 25 24 7 6 6 20 15 7 3 43 17 5 2 10 1269 Percent 85.0 2.0 1.9 .6 .5 .5 1.6 1.2 .6 .2 3.4 1.3 .4 .2 .8 100.0 Facilitators’ educational qualifications Qualifications levels were explored as the National Academy of Parenting Practitioners had specified qualification criteria. Highest educational qualifications Forty two per cent of facilitators held a Bachelor degree or above. An additional 38% had a foundation degree or Higher Education (HE) qualifications but below degree level. A minority (9%) had A/AS level as their highest qualification or five or more GCSE at A*-C (5%). A small proportion (6%) had only some GCSEs and just 1.3% had no educational qualifications (Table 3.2). 74 Table 3.2: Facilitators by gender and highest educational qualification Gender Highest educational qualification 1 2 3 4 5 6 7 8 9 No qualifications Some GCSEs Five or more GCSE at A*-C A/AS level HE below degree Foundation degree Bachelor degree Masters degree Doctorate or equivalent % Female % Male 1.4 5.8 4.3 8.9 30.0 9.0 32.3 7.9 .4 .8 6.1 7.6 7.6 17.4 11.4 32.6 15.2 1.5 Total 1.3 5.8 4.6 8.7 28.7 9.3 32.4 8.6 .6 N = 1277 Male facilitators were more likely to be highly qualified than females, with 17% having a Masters degree or above compared to 8% of females (p < .01). Those in the 25-29 age group were the highest qualified (n = 150, mean = 6.4) while those in the 60+ age group had the lowest qualifications (n = 24, mean = 4.8). No other contrasts were statistically significant. There were no statistically significant differences between ethnic groups in educational qualifications. Professional qualifications Nearly two-thirds (64%) reported having a professional qualification. Facilitators were asked to name their professional qualification/s. In coding the responses it was clear that ‘professional’ qualifications had been interpreted as anything related to a job, as opposed to an older, more traditional view of ‘professional’. As a result the data were coded in terms of the main domain of qualification (e.g. ‘Education’ covers teaching assistant to qualified teacher and every sector from early years to adult education). Where people gave more than one qualification, the first or highest qualification was coded. The main domains varied greatly but the most frequent were education (30%), National Nurseries Examination Board (NNEB) or health & social care (11%), social work (11%), health (10%), youth work (6%) and counselling or therapy (6%). Only a small proportion (3%) reported a qualification as a psychologist with just 11 (1.4) reporting that they had qualifications as a practitioner psychologist. ‘Other’ included a wide range of roles – ordained minister, lawyer, qualified gardener. 75 NAPP qualifications criteria We also summarise facilitators’ qualifications against the criteria of the National Academy for Parenting Practitioners (NAPP) for eligibility for training in evidence-based programmes. We have based this on the highest education qualification and professional qualifications. The NAPP criterion related to ‘experienced practitioners’ was unclear as to whether it meant experienced in their current job or in delivering parenting programmes. We have interpreted ‘graduate’ to mean Bachelors degree and excluded foundation degrees. So we have categorised according to the following NAPP groupings: • Qualified at graduate level or above AND in a helping profession (qualified social worker, health visitor, nurse, occupational therapist, clinical or educational psychologist, systemic or family psychotherapist, child psychotherapist, speech and language therapist); • Qualified at graduate level but not in a helping profession (as defined above); • QCF Level 3 specifically involving Working with Parents (i.e. not just any L3 qualification); • Any other than the above; • Parent coaches. The results are presented in the Table 3.3. Clearly levels 3 and 5 contain very few cases, so levels 3-5 will be combined in subsequent analyses. Table 3.3: NAPP qualification criteria NAPP qualifications criteria 1. 2. 3. 4. 5. 6. 3.4 Frequency Percent Helping profession graduate Other graduate Level 3 in work with parents Other non-graduate (not 1,2 or 3) Parent coach Not enough information to code 115 408 6 730 7 11 9.0 32.0 .5 57.3 .5 .7 Total 1275 100 Facilitators’ prior experience Prior training in programme facilitation One-third of facilitators (34%) reported no training in a parenting programme prior to PEIP. However, 39% had received prior training in a single programme, while 27% had previously 76 been trained in two or more programmes. Those who had not received training prior to PEIP were more likely to have their highest educational qualification below A/AS level compared to all facilitators (16% vs. 11%) and were slightly more likely to be male than among all facilitators (14% vs 11%). Both comparisons were statistically significant (p < .01) but the differences were not large. Overall, of the 842 facilitators reporting they had had parenting programme training prior to PEIP, a total of 1303 references to programmes was specified. Nearly one-third (32%) of facilitators had received prior training in Triple P, closely followed by Incredible Years (28%), Strengthening Families Programme10-14 (22%) and Strengthening Families Strengthening Communities (14%). The number that had received training in FAST prior to PEIP was low (3%). In addition around one-third of facilitators had received training in another evidencedbased programme (33%) and one quarter (24%) had training in a non-evidenced based programme 23 prior to PEIP. A very wide range of other evidence-based programmes were mentioned. Table 3.4 lists those programmes mentioned by 10 or more facilitators. Table 3.4: Prior training in ‘other evidenced-based’ programmes reported by PEIP facilitators Other evidence-based programmes N faciitators Family Links ESCAPE Family Nurture Peers Early Education Partnership (PEEP) Solihull STOP (Supporting Together Offenders Parents) Family Caring Trust Positive Parenting Mellow Parenting 28 18 15 15 15 15 14 13 11 Prior experience of delivering parenting programmes Facilitators were asked if, prior to PEIP training, they had delivered (facilitated or cofacilitated) any other parenting programmes, and if so how often they had done so. Levels of prior experience in terms of delivery were relatively low. Although 65% had received prior training in one or more programmes (as described above), only one-third (32%) had 23 As designated by the facilitators themselves 77 previously delivered (facilitated or co-facilitated) any of the original five recognised PEIP programmes. Incredible Years and Triple P were the individual programmes that had most frequently been delivered (13% and 12% of facilitators respectively) see Table 3.5. If other evidenced-based programmes are included the proportion who had previously delivered at least one parenting programme rose to 40% and to 46% if this included any parenting programme (including those not formally evidenced and not published). Table 3.5 Facilitators’ prior experience of delivering (facilitating or co-facilitating) parenting programmes Programme Groups delivered 0 1+ Delivered any PEIP recognised programmea Delivered any evidenced based programmeb Delivered any programme % of facilitators delivered 1+ group 865 412 32.3% 765 512 40.1% 693 584 45.7% Note: a Any of the five programmes specified at the start of the PEIP b As designated by the respondent A summary measure of parenting programme experience prior to PEIP From the above data on prior training/delivery of evidenced-based parenting programmes, facilitators were grouped into one of three categories. The data are presented in the Table 3.6. This provides a summary measure of experience, based on both prior experience of training in programmes and prior experience of delivering programmes. Table 3.6: Facilitators’ prior training in/delivery of parenting programmes before undertaking PEIP training Level 1. No prior training in, or delivery of, any evidenced-based parenting programme 2. Prior training in at least one evidenced-based parenting programme but no prior experience of delivery 3. Prior training in at least one evidenced-based parenting programme and prior experience of delivery to at least one group Total 78 N % 435 34.1 354 27.7 488 38.2 1277 100.0 3.5 Comparing facilitators’ qualifications and prior experience A higher proportion of graduates in the helping professions had prior experience of training and delivery (55%) compared to the other two qualification categories (35% and 38% respectively). However the Spearman’s correlation between qualifications and experience is not significant (r = -.03). These therefore represent only weakly related dimensions. An interesting question is therefore whether either of these factors relates to parent outcomes, and if they do the relative importance of each. The facilitator form did not identify the PEIP programme that facilitators received training on as part of PEIP, because facilitators could have been trained on more than one programme. However for those 266 lead facilitators who were matched to the parent questionnaire (see details in next section) it was possible to crosstabulate the programmes they actually delivered by their qualification and prior experience. Chi-square tests indicated no significant differences between programmes in the qualifications of facilitators delivering the programmes. However, there was a significant difference in the prior experience of lead facilitators delivering different programme (p < .05). Facilitators delivering Strengthening Families Strengthening Communities were least likely to have had experience of training and delivery prior to their PEIP training (41%) compared particularly to SFP10-14 (60%) and Incredible Years (65%). 3.6 Lead facilitator characteristics and parent outcomes Having examined the characteristics, qualifications and prior experience of facilitators we now turn to the main issue: do any of these factors relate to parent outcomes? 3.6.1 The matching process and achieved sample We were able to match 253 lead facilitators running 470 groups with 3476 parents completing pre-course questionnaires and 1755 completing post-course questionnaires. Qualifications and prior experience were confounded as nearly all lead facilitators who were helping profession graduates had previously run groups. We therefore categorised lead facilitators into seven groups based on the combination of qualifications and prior experience (dropping the 10 cases where the helping profession graduate had no prior training/delivery experience). Aggregation can still be used to evaluate overall effects of qualification level or 79 prior experience if trends are apparent across the seven categories (e.g. to compare helping profession graduates, other graduates and non-graduates). Table: 3.12 Number of parents with pre- and post-course responses in groups with different levels of lead facilitator qualifications and experience Qualifications and prior experience Frequency % 1 Helping professions graduate with training & delivery 112 6.4 2 Other graduate with training & delivery 272 15.5 3 Other graduate with training only 101 5.8 4 Other graduate with neither 172 9.8 5 Non graduate with training & delivery 643 36.6 6 Non graduate with training only 176 10.0 279 1755 15.9 100.0 7 Non graduate with neither Total 3.6.2 Lead facilitator qualifications and experience in relation to parent and child outcomes Parent outcomes 24 There was only one significant difference in relation to lead facilitators which was for improvement in parents’ mental well-being. There was no significant difference between the seven facilitator categories, but the three non-graduates groups all had the greatest improvement in parents’ mental well-being. When lead facilitators were aggregated by qualification the improvement in parent mental well-being for facilitators who were nongraduates was equal to those of helping profession graduates, and significantly higher than other graduates (p < .05). There was no separate effect when groups were aggregated by prior experience in training/delivery. Child outcomes There were no significant differences between the seven facilitator categories, neither were there any aggregate effects for lead facilitator qualifications or for prior experience. Parent evaluation of the groups After controlling for programme there was a significant difference between the seven facilitator categories (p < .05) for the parent ratings of group leader style and group 24 Three sets of multivariate ANOVAs were run including the seven facilitator groups along with a fixed effect to control for differences between programmes, which we saw earlier were substantial. 80 effectiveness (see Section 2.8). Perhaps surprisingly, groups run by non-graduates with no prior training/delivery experience had significantly higher ratings for group leader style than any other facilitator grouping. There were no aggregated effects for qualifications overall or for prior experience overall. Summary Programme effects were much larger than facilitator effects for parenting and child behaviour improvements, frequently by a large margin. Certainly the programme a facilitator was trained in was the overriding factor, and there were no significant effects related to facilitators’ prior experience of training or delivery before PEIP training. However after programme effects were controlled there were facilitator effects in two specific areas: improvements in parents’ mental well-being and parents’ ratings of group leader style. For parent mental-wellbeing, perhaps surprisingly, non-graduates were associated with greater improvement than graduates, and in the case of group leader style one of the non-graduate groups had even higher ratings than helping professional graduates. In these particular domains we may hypothesise that since the majority of parents were non-graduates there may be benefits to a match between facilitator and parents on this factor. However caution is necessary in interpreting the finding since even though these effects were statistically significant they were small and for the majority of outcomes there were no differential effects related to facilitator qualification. 81 4. PROCESS FACTORS UNDERPINNING EFFECTIVE OUTCOMES Main findings • Initial training provided by the programmes was highly regarded by facilitators. • Local authorities generally found rolling out parenting programmes on a large scale very challenging and only a minority managed the balance of higher numbers of parents completing a programme and above average outcomes. • Successful LAs had strong leadership, effective day to day management and organisation, and a clear policy. • Several organisational and delivery models worked well; the key was to match the model with local circumstances. • PEIP was viewed as having reduced the stigma previously associated with parenting support; contributing to workforce development; and giving referring professionals a better understanding of what parenting programmes could offer families. 4.1 Programme factors In this section we report comments made about the programmes by our interviewees who comprised facilitators and LA strategic and operational lead officers. Overall they were very positive. The comments reported here must be read bearing this in mind. All of the programmes have been shown to be effective in efficacy trials. Nevertheless, these comments, positive and negative, reflect experiences in the national roll out and so provide evidence on the factors that enhance (or impede) the effectiveness of the delivery of parenting programmes in community settings. 4.1.1 The programmes Overall, the opportunity to work with, or, for some LAs, to extend their use of evidence-based parenting programmes, was strongly welcomed. Interviewees were universally positive about the PEIP. In terms of the PEIP programmes, comments were made in relation to particular aspects of the various programmes. Both FAST and SFP10-14 were welcomed as programmes that engaged children as well as their parents – this aspect of both programmes was valued particularly because families were seen to be learning by undertaking tasks together, and that the joint experience of participating in the programmes created new bonds within families (Box 4.1). 82 Box 4.1 Parents and children, SFP10-14 and FAST ‘I see FAST as being of great value because the parents are not separated out from the family, you are dealing with the family as a unit’, (Facilitator). ‘It [SFP10-14] offers families time away from home together, with an expectation that they will work together […] the family time is valuable, and also the young [people] being with other children and working towards something is quite a nice way to spend time’, (Facilitator). ‘It is that in [SFP10-14] the adults are being listened to, the children are being listened to, and [they] can have fun together because [they] want to be together […] people feel valued and they feel they are in a safe place, to have fun’, (Facilitator). In addition, in the LAs which had engaged with FAST, there was positive commentary both about the strength of the programme in building links between parents and schools, and about the community building capacity of the programme. PEIP programmes and working with parents and children with SEND The majority of LAs used the PEIP programmes with parents and children with Special Educational Needs and Disabilities (SEND). In most cases, this was with individual parents and/or children as part of bigger parenting groups but, in a small number of cases, groups were specifically run for parents and/or children with SEND. Interviewees noted that there was a need to adapt content and/or delivery for programme delivery in this situation. For example, in relation to Triple P, one LA ran, through a third sector agency, groups specifically for parents of children with autism. To do this successfully, the programme had to be adapted to take account of the specific needs of these families with, for example, changes being made to parenting strategies. Similarly, one LA ran Triple P groups for parents of children with disabilities, and for parents with learning difficulties. The strategic lead noted that for parents with SEND it was the case that ‘the whole delivery was different. It [Triple P] was much shorter, more visual, and it was in very much smaller groups […] the entire delivery was programmed entirely to their needs’. Other examples of programme adaptation included support and adaptation for children involved with FAST and SFP10-14. (Box 4.2). As indicated in Appendix 2, SFSC provides versions in Braille, in large print and in easy words and pictures for parents with learning difficulties or disabilities. These were found to be helpful: ‘We now have an Easy Words and Pictures manual [for SFSC], and have a large print manual that is helpful, linking to the facilitators’ output [and we] give more practical activities that illustrate the points well,’ (Operational Lead). 83 Triple P also offers a version (Stepping Stones) designed for parents of children with a developmental disability. Box 4.2 Adapting FAST and SFP10-14 ‘We have delivered FAST in a special school with disabled children. So we had to adapt it slightly, obviously, because the disabled children couldn’t get up and feed their parents […] so we changed it and that went down very well,’ (Operational Lead). ‘At least 25 per cent of the children had SEN on the [FAST] course […] and we were able to call upon additional staff from Early Years to support them, particularly during the play sessions. Play session itself is not adaptable, but [you] can adapt the play you do within that play session, within the parameters of the training,’ (Facilitator). ‘We had a child with difficulty writing [SFP10-14], but it didn’t deter him. One to one support was adequate to help him, reading everything out […] The training did address this issue and we were able to draw on individual experience and prior training to address where there are SEN issues, while keeping to the aims of the programme,’ (Facilitator). General issues with programmes In addition to the many positive comments, a small number of issues concerning the use of Triple P and SFP10-14 with parents and families emerged from the interviews. A minority of interviewees raised issues about what was seen to be outdated materials in the Triple P programme. In particular, the language and DVDs were seen by three interviewees to be in need of updating. There were also concerns surrounding the effectiveness of the telephone contact element of Triple P, with two interviewees noting that they were a less effective form of contact than face-to-face contact with parents. Finally, two interviewees questioned the effectiveness of the Triple P group and Triple P teen programmes with older children. In a similar fashion, a number of criticisms were raised in relation to the SFP10-14. For example, two facilitators noted that there was not enough time available for parent discussion and reflection. In addition, three facilitators questioned the effectiveness of aspects of the programme for older children; for example, the ‘Toot Beep Beep’ game was seen to be effective only with the youngest children. Facilitators from two LAs also criticised the SFP1014 DVDs, saying that they needed to be updated, while two facilitators from another LA questioned the value of the booster sessions, which they thought were too repetitive. Of the four PEIP programmes, the SFP10-14 produced the greater number of concerns. Interviewees raised issues concerning the importance of group makeup, and the degree of skill and experience needed to deliver a SFP10-14 programme successfully. In particular, the inclusion of teenagers in the programme delivery led to some problems, with one facilitator noting that there had to be a balance between targeted families and self-referred 84 families because ‘if all the children are targeted, all you’re doing is fighting fires the whole time. You’re just managing behaviour’. This type of issue linked, however, with the question of the skills and experience required of facilitators. It was common for SFP10-14 facilitators to note that they became more confident in their delivery skills as time went on, and from course to course. In addition, it was noted that each course required significant levels of planning and preparation, and if facilitators came from different agencies, careful liaison between facilitators. Fidelity versus flexibility We have found over our evaluations of the PEIP, and the earlier Pathfinder, that delivery based on programme manuals raises concerns with facilitators and LA lead officers: how flexible, if at all, can one be in delivering a programme? For example, can the language be Anglicised if judged ‘too American’? Fidelity is important as deviation from the manual runs the risk of the programme in practice no longer being that which has been demonstrated to be efficacious. However, some degree of flexibility is inevitable, and is accepted by programme designers such as Carolyn Webster-Stratton of Incredible Years who stressed this in her training course for PEIP Incredible Years practitioners in Manchester (May, 2010). The task, therefore, is to strike a balance. A key issue in relation to the delivery of evidence-based parenting programme is fidelity to programme content and delivery by facilitators. Many interviewees recognized that for effective delivery there had to be some degree of flexibility possible. A number of interviewees explained that programmes had been delivered flexibly, although still remaining close to the core of the programmes. With the larger number of PEIP courses delivered being Triple P, comments relating to the issue of fidelity and flexibility most frequently referred to this programme (Box 4.3). Box 4.3 Fidelity and flexibility in programme delivery (Triple P) In one LA, Triple P has been delivered to 13 year old mums to be – ‘So, what I’ve not been able to deliver is true Triple P. I’ve tried to do it, but after the first two sessions realised that actually this isn’t going to work with these girls […] so what I did was taught them Triple P strategies in a way that was accessible to them,’ (Operational Lead). In another LA, Triple P has been delivered to hard to reach families, and the programme is seen as ‘too much, too long [so] the programme was adapted into bite-sized chunks,’ (Strategic Lead). Interviewees also reported flexible delivery of SFP10-14, but less so in relation to IY. The need to deliver flexibly most typically reflected facilitator responses to parent learning 85 difficulties, cultural concerns, and with Strengthening Families Programme 10-14, problematic responses to activities for young people that were seen as unsuitable and ineffective (Box 4.4). Box 4.4 Fidelity and flexibility in programme delivery (SFP10-14) ‘We changed some of the activities that weren’t to do with the course material so much, just the gap activities. We changed them to suit the children that we had because you don’t want them to feel that you’re insulting them. We did wiggle it around as much as we could but still keep the content in […] it’s quite hard to predict which games will go down well and which won’t, for example, the fishing game worked well but other games didn’t work because this particular group was only four children and some games needed a larger group to work,’ (Facilitator). ‘During [my] initial training [I] got the message that “you can’t be flexible. You have to follow it exactly as it is”, but from experience [I] know it worked better if you were a bit flexible. One [big] change made was to run it as a nine week programme not seven weeks – so [we] had added in an initial session for parents to talk and get things off their minds and had inserted booster session 1 in before session 7. This gave parents the additional time needed to feel a bit more confidence, a bit more “I can do this,”’ (Facilitator). 4.1.2 The initial training Sustainability, access to training and training the trainers In Appendix 2, the approaches of the five PEIP programmes to initial training for facilitators are summarised, as well as what is involved in becoming a programme trainer. A minority of interviewees raised questions concerning the cost of training facilitators, and the capacity of parenting programme providers to undertake training. There was a particular issue connected with the need to access trainers from outside the UK, this often being the case for Triple P, Strengthening Families, Strengthening Communities, and Incredible Years. For example, an operational lead noted that it would take one of her LA’s staff ‘several years’ to become a trainer for Incredible Years, while another operational lead noted that it would require £25000 in funding for her LA to have a Triple P trainer on their staff. A strategic lead for a Wave 3 PEIP LA criticised Triple P and Incredible Years in particular for being expensive, and for delays in getting facilitators trained (Box 4.5). 86 Box 4.5 Training facilitators – costs and access to training ‘We’ve had a hold-up because of not being able to get people trained. I think one of the lessons learned is that, if you can grow your own training it’s so much better. If you can have accredited trainers within local authorities, then you build in sustainability and, with courses where you have to have accredited trainers that belong to the organisation, it does mean that it becomes expensive but it is also difficult to get people trained because there has been such a high demand for these places, whether or not they’re offered through the CWDC 25 as free places or whether we’re trying to buy them in,’ (Strategic Lead). In some cases, delays in accessing training for facilitators had notable impacts on the ability to deliver parenting programmes. This had been an issue in early waves of the PEIP, and was still a problem for a few LAs in Wave 3. For example, in one LA, Incredible Years training for 20 people in September 2010 was cancelled by Incredible Years, and rescheduled (for 25 people) to March 2011; the result being that this LA’s PEIP was run with only six instead of 20 trained facilitators. Quality of initial training With a small number of exceptions, the training to become a group facilitator provided by all the PEIP parenting providers was highly regarded by the interviewees. Issues that were raised largely concerned matters that were subsequently addressed by the training providers in consultation with the LAs; although there were a minority of interviewees who continued to have concerns. Typically, interviewees explained that they viewed the training provided very positively. For example, Triple P training was described, by one operational lead, as ‘excellent [...] the person who trained the facilitators was very good, very knowledgeable’. A facilitator described Strengthening Families Programme 10-14 as being ‘brilliant. One of the best training courses we’ve had. We were very well supported’; Strengthening Families Strengthening Communities training was described as being ‘very good training [it] dealt with the programme in depth [the trainer] was good at explaining why the programme works and the underlying theory’ (an operational lead); while Incredible Years ‘initial training is excellent. I haven’t met anybody who hasn’t enjoyed it’ (an operational lead); and FAST, ‘I really enjoyed it; it was interesting to see the research behind it’ (a facilitator). Two issues were raised in relation to the training – individual trainers and the demands of training on trainees. A very small number of leads raised issues in relation to individual programme trainers. For example, in relation to a Triple P trainer, an operational lead noted 25 Children’s Workforce Development Council 87 that there had been problems which led ‘to disillusionment with [some] potential facilitators’, while another operational lead identified problems with Triple P trainers that were, nonetheless, resolved. In a similar vein, a combined operational and strategic lead noted that ‘there’s been a marked difference in the abilities and skills of the [Triple P] trainers’. Similar criticisms, concerning the trainers, were raised in relation to the Strengthening Families Programme10-14. Several interviewees had concerns about the experience of their Strengthening Families Programme 10-14 trainers, as explained by this operational lead who stated that, ‘Strengthening Families Programme10-14 allowed people to do the training [of] facilitators if they had delivered one programme, and were planning to deliver a second […] some of them hadn’t delivered the parents group, some of them hadn’t delivered the young people’s group, and that was really apparent’. Issues raised in relation to the various training courses, by a small number of interviewees, included the pace and intensity of the training sessions, the exposition of the theoretical underpinnings of the programmes, and the need for the provision of group facilitation skills which were not provided by the parenting programmes. Some interviewees saw Triple P and Strengthening Families Strengthening Communities training as being particularly intensive. 4.1.3 Further training and accreditation The five PEIP programmes vary in the accreditation and further training available to those who have trained as facilitators (Appendix 2). Interviewees expressed a range of views concerning accreditation in the different programmes. For example, equal numbers of interviewees noted both positive and negative points concerning Triple P accreditation, with three saying that accreditation was, for example, ‘good in terms of the coaching session and preparation for delivery’ (Operational Lead), and three interviewees noting that accreditation was heavily theoretical rather than delivery-focussed. In two LAs, Triple P accreditation of facilitators was viewed as too divorced from programme delivery. For example, in one LA the Triple P accreditation certificates were not given out to facilitators until that person codelivered with the operational lead and had been through local quality assurance processes with a member of the parenting team. The issue was that the Triple P training was viewed as ‘quite highbrow’ and ‘very research-based’: ‘Some practitioners come away without a real understanding of it. It’s not until you break it down with them and put it into practice [that] they understand and it slots into place. which is fine but, the one thing I don’t like about Triple P is that I fear it has moved far too much towards a business model where they want to get everyone through [the accreditation] […] The money side of it, to get money in through the 88 books, and having trained practitioners is more important [to them] than the service that the parents are getting. […] I have passed that on to Triple P’. (Operational Lead). The Incredible Years accreditation process was widely regarded as being rigorous, but, at the same time, lengthy. 4.1.4 Supervision requirements The five main PEIP programmes each have rather different requirements as to the level of programme supervision of facilitators delivering the programme (see Appendix 2 for details). Incredible Years programme requires the greatest level of facilitator supervision including, for example, video recordings of session delivery (focused on the facilitators, not the parents) which are reviewed in peer support meetings using Peer and Self-evaluation checklists. FAST includes programme supervision of first delivery in its training package involving telephone consultation during delivery, three visits to the site to monitor programme integrity and advise on local adaptations. Strengthening Families Programme 10-14 provides programme supervision for teams of facilitators seeking Level 2 training. With Strengthening Families Strengthening Communities, programme supervision is optional but can be provided individually or in groups. However, full accreditation as a SFSC facilitator involves observation of delivery to ensure fidelity and quality assurance. Triple P has an accreditation process for trained facilitators but does not require programme supervision of delivery. Where programme supervision was required, this was accepted by interviewees, although the cost of this was commented on by a minority. For example, an operational lead commented that FAST’s programme supervision was very good ‘because you get three visits from a trainer, and more supervision, and more on site training as you go along, it definitely makes schools and deliverers more … you know, there’s a lot more fidelity to the programme, and a lot less drift’. However, this interviewee went on to note that there were significant additional cost considerations involved, and for some schools it can be seen to be too expensive. Where programme supervision was not required, for example for SFP 10-14 facilitators not going for Level 2 training, or Triple P facilitators, a minority of interviewees argued that this should be tightened up so that delivery was quality assured by the programme. (In section 4.2.5 LA supervision of facilitators is discussed). 89 4.2 Local infrastructure factors affecting effectiveness of implementation This section focuses on aspects of the local infrastructure supporting efficient and effective delivery of the PEIP parenting programmes. It begins by describing our analysis of LA efficiency (numbers of parents completing a PEIP group) and effectiveness (outcomes for parents and young people). On the basis of all the analyses, we are confident that the main infrastructure factors that impacted most on delivering PEIP parenting programmes efficiently and effectively were: • leadership and coordination of PEIP • how well PEIP was meshed in to the LA context • implementation models and their relationship to sustainability of PEIP • recruitment, retention, support and supervision of facilitators • recruitment, engagement and retention of parents. These are considered in turn. 4.2.1 Efficiency and effectiveness The findings reported are based on three stages of analysis exploring efficiency and effectiveness of the roll-out of PEIP in our sample of 47 LAs across England. First, all the qualitative interviews conducted during the evaluation were analysed in relation to interviewees’ views about which factors were important in ensuring that the PEIP project was managed and delivered in such a way that as many target parents as possible completed a PEIP programme successfully. All the factors that emerged from this analysis had been addressed in DCSF PEIP Guidance to Local Authorities 2008 and 2009 versions, guidance that was itself based on findings from the Parenting Early Intervention Pathfinder. Second, using the number of pre- and post-group parent booklets that were returned to CEDAR as a measure of efficient roll-out in relation to outputs (parents starting; parents completing), interviews conducted in a sample of the highest and lowest output LAs per Wave were analysed in relation to adherence to the DCSF PEIP Guidance to Local Authorities 2008 and 2009 versions. We acknowledge that the CEDAR booklets are a crude measurement of output as we know that no LA returned a pre- and post- group booklet for every parent starting and finishing a PEIP programme. However, it is the best measure available after the DfE rescinded the requirement that LAs report these data as part of regular monitoring. The measure reflects both organisational efficiency and facilitator acceptance of the importance of evaluating outcomes for parents. This qualitative analysis 90 confirmed that efficiency in terms of outputs was related to how well the local infrastructure supporting the PEIP had been developed. The third stage of analysis, to explore whether and how outputs were related to outcomes, was done separately using only the quantitative data. LAs were ranked by their outputs in terms of post-group booklets (parents completing a course). In addition, LAs were ranked on their mean score on each of three measures of outcomes for parents and children (mental wellbeing, parenting total score, and child SDQ total score). These three outcome rankings were then compared to the outputs rankings to explore the relationship between effectiveness (outcomes) and efficiency (outputs). These analyses suggest that there was a cost, in terms of outcomes for parents, of achieving higher than average numbers of parents completing a course. Among Wave 2 LAs, nine achieved 100 or more parents completing a post-course booklet. Of these, only two also achieved top twenty rankings across all three outcome measures. Among Wave 3 LAs, only one achieved 100 or more parents completing a post-course booklet: this LA was also in the top 20 LAs for two of the outcome measures and just below (21st) for the third. Eight other Wave 3 LAs achieved between 50 and 99 parents completing a post-course booklet. Of these, two also achieved top 20 rankings across all three outcome measures with one being the only LA in our sample to achieve top ten rankings in each of these. In summary, these analyses suggest that rolling out a PEIP parenting programme in a LA on the scale expected by the PEIP was a difficult task. Only a minority of LAs in our sample (referred to as the top five LAs) managed the balance of achieving both high numbers of parents completing a course (over 50 for Wave 3; over 100 for Wave 2) and above average high quality outcomes consistently across parent mental wellbeing, parenting total score (reducing laxness and over-reactivity) and improved child total difficulties. These analyses form the basis for considering key infrastructure factors affecting the efficiency and effectiveness of PEIP. 4.2.2 Leadership and coordination across a local authority Strategic leadership and operational coordination supported roll-out of the PEIP parenting programmes in LAs. Where these were not in place, the PEIP was less efficient in organising groups and reaching target parents. Strategic leadership helped to establish PEIP and to mesh it in to local priorities and the existing infrastructure of parenting support history and of staffing capacity (Box 4.6). Although most often this strategic leadership role was given to 91 one person, there were also examples where aspects of the role were shared and of an existing group of senior service managers taking joint strategic leadership. Box 4.6 Examples of the difference strategic leadership made to PEIP In one Wave 3 LA, PEIP began with only operational level staff who reported that it made ‘lots of difference’ when they got a strategic lead appointed – essentially, because PEIP was then organised and planned strategically, it made it possible and easier to gain referrals from different services and to work in a multi-agency way; it enabled monitoring to take place and it ensured that PEIP took targeted families, while schools did more universal level work. All of this meant that the LA reached 170 parents, exceeded their target of 150. In a Wave 1 PEIP, the operational lead reported that, when the strategic lead moved to a different LA and was not replaced, the PEIP ‘lost momentum’. As a result, the multidisciplinary pool of trained facilitators fell back in to ‘silo’ delivery rather than multidisciplinary delivery, exacerbated by turnover in service manager staff. Operational coordination helped PEIP to be delivered across a local authority (Box 4.7). How coordination worked varied to suit each LA context (size, staffing capacity, existing coordination levels such as localities, areas, local networks). Some LAs had one central coordinator and others divided the coordination role on a geographic basis. However, some central coordination enabled LA-wide oversight of processes and outcomes. The PEIP operational lead from the local authority returning by far the most pre- and post-group parent booklets to CEDAR adopted this model of area coordinators taking responsibility for their area but also linking back to the strategic centre: ‘The way that [PEIP] was set up was that there would be one [lead parenting] practitioner in each of the four quadrants [of the LA], so that that particular person took on a central role in liaising with the practitioners that have trained in their area, providing peer support, and linking them into the strategic support that we were offering, and also to manage the waiting list.’ 92 Box 4.7 Examples of benefits of operational coordination Operational lead appointed and role valued The strategic lead in a large county, viewed the appointment of an operational lead coordinator as key to the project’s organisational success –‘You need someone to keep it all together, otherwise it can be very fragmented and lead to inconsistency’. Operational lead vacancy not filled and role missed The strategic lead in a city unitary authority, where the operational lead role had not been replaced when the original post-holder left, acknowledged that, ‘You need somebody who is there and is responsible for co-ordinating the programme, coordinating the facilitators, and we didn’t have that in place’. The LAs from all three Waves of PEIP that took part in the final round of interviews in late autumn 2010 had adopted different leadership and coordination models to manage the PEIP project (Figure 4.1). Eight of the top ten LAs in terms of numbers of pre-course parent booklets returned (i.e. parents starting a PEIP programme) and nine of the top ten in terms of post-course group booklets returned (parents completing) adopted the Type 1 model of separate individuals taking on the strategic leadership and operational coordination of the PEIP. Figure 4.1 PEIP project management models Model Description Type 1 Separate individuals took on strategic and operational roles Type 2 One person combined the strategic and operational functions of leadership and coordination – this had three sub-types: Type 2.1 – worked well Type 2.2 – lack of strategic effectiveness Type 2.3 – lacked operational effectiveness Type 3 Strategic leadership was shared amongst more than one person -operational coordination was led by one person Type 4 Operational coordination was shared amongst more than one person – strategic leadership was one person Type 5 No strategic leadership – operational coordination shared amongst more than one person In a minority of cases the project management model was adapted over time: for example, in, a large mixed urban/rural Wave 1 LA, the original Type 1 project management model evolved over time into one lacking strategic leadership but coordinated operationally by 93 leads in different areas of the LA linked to third sector providers (Type 5). In part, such changes reflected adaptation to circumstances (for example, staff turnover) and in part they reflected perceptions that, as the PEIP project became more embedded in these LAs, it required less management overall. In a minority of cases, too, the project management model adopted was not from choice but because local agreement was not granted to fund separate strategic and operational roles. For example, in a Wave 2 and Wave 3 LA, the Type 2 combined model was perceived as ineffective operationally because no funding was available locally for an operational coordinator role. On the other hand, in another Wave 3 LA, the joint strategic/operational role worked well because PEIP was delivered through schools by working across agencies in each locality and the role-holder had strong links both with schools and with relevant service heads. 4.2.3 Meshing PEIP into the local context A key task for the PEIP lead/s in each LA was to work out how best to add the PEIP project to what was already in the authority. This meant building on the existing parenting support strategy and on experience of parenting group delivery; or developing parenting support in the LA; or using PEIP to do both (see Box 4.8). In LAs that already had a strategic plan encompassing parenting support (sometimes called a parenting strategy, sometimes called a family or Think Family strategy), PEIP leads were able to place the PEIP project in that context from the start. Where there was also strong strategic leadership of the PEIP, this combination was viewed as working well, creating a solid foundation for the PEIP roll-out. Where there was no pre-existing relevant strategic plan, either this remained a problem for the PEIP roll-out or the PEIP project was used as the catalyst to develop such a strategy. 94 Box 4.8 Placing PEIP in varied local contexts around parenting support Examples where PEIP fitted the existing strategic plan and also developed further In one LA Wave 2 parenting support was a strategic priority prior to PEIP. The PEIP was used to develop a core team of increasingly skilled and experienced parenting practitioners co-delivering groups to many more parents than prior to the PEIP. The success of this gave parenting support an even higher priority in the LA. A long history of successful partnership working in a Wave 3 LA meant that support was given to PEIP from every key agency, which had ‘blown the road blocks away’. Parenting programmes had been run and co-ordinated LA-wide for several years prePEIP but PEIP, ‘has allowed us to expand the delivery of programmes so that most, if not all, services working with parents and families know that this is something that is out there, and that it’s a really effective intervention…It really, really gave a turbo charge on what we were able to do.’ (quotes from operational lead) Example where PEIP fitted the existing strategic plan A Wave 3 LA found that, ‘PEIP fitted within the Parenting Strategy. We had a working group from the Strategy that oversaw the finance and how it was managed.’ (quote from joint strategic/operational lead) Examples where PEIP was the catalyst for developing a strategic plan In a Wave 1LA PEIP ‘kick-started’ parenting support as a priority; enabled delivery county-wide that was accessible to all parents; and raised the profile of supporting parents for ‘workers right across the board’. (quotes from operational lead) Difficulties highlighted in the final round of interviews around how well or not PEIP fitted the local parenting support context included situations where: • differing organisational cultures across education, health and social care affected efficiency of multi-agency delivery • local priorities around parenting and family support did not mesh well with the PEIP aims (e.g. the age group 8-13 not a priority) • there was limited capacity to deliver additional groups (e.g. small LAs) • there was some resistance to delivering PEIP programmes instead or, or as well as, those that had a history of delivery in the LA. All of these would have had some impact on the efficiency of PEIP delivery. Choice of PEIP programme The reasons for LAs choosing particular programmes for the PEIP were discussed in an earlier report (Lindsay et al., 2010a). In the main, the choice of which of the five PEIP programmes to use was made in relation to the local context. Hence, LAs with experience of any of the five programmes approved for PEIP wanted to build on that and perhaps add in 95 one or more additional and complementary PEIP programmes to the menu on offer to parents. These LAs were at an advantage in terms of achieving PEIP outputs (numbers of parents starting and completing a group) because they already had trained facilitators in one or more PEIP programmes. A minority of LAs had no experience of prior delivery of any of the five PEIP programmes. Clearly, they could be expected to take longer to reach the point where PEIP groups were running as they first had to train facilitators. This would have affected efficiency but not necessarily effectiveness. Three LAs adopted as a PEIP-funded programme one that was not included in the list of five approved programmes – using STOP, Parent Power and Parents Plus respectively. These decisions were made because of a local history of successful delivery, or a strong local need that the chosen programme was regarded as meeting, and a local refusal to allow national PEIP guidance to force choice of programme locally. (The effectiveness of these programmes is summarised in Appendix 4.) Two of these three LAs achieved relatively high numbers of parents completing a PEIP-funded course but none achieved high numbers and top 20 rankings across the three outcome measures. However, neither were they noticeably worse than other LAs using only approved PEIP programmes. Of the top five LAs in terms of balancing high outputs and high outcomes, four offered Triple P as their main PEIP programme; the fifth ran both Incredible Years and SFP 10-14. In four of these cases, the LAs were building on a prior history of delivering evidence-based programmes but Triple P was added in to their menu during PEIP in two cases. In the fifth case, there was no prior history of delivering evidence-based programmes. This indicates that a prior history may be helpful in achieving efficiency and effectiveness but is not necessary. Reflections on the wider benefits of the PEIP project in LAs In the final set of interviews, we asked people to reflect on what PEIP as a project had achieved in their local authority. Although people had a range of views about government policies, guidance and funding rules around PEIP, the overall message was one of very positive developments associated with PEIP that were expected to have a lasting legacy in LAs (so long as expected public sector cuts in staffing did not decimate the workforce). The majority view was that PEIP had changed the local culture for parenting by reducing or removing the stigma previously associated with parenting support and had contributed to workforce development, not only for those trained as facilitators but also for referring professionals who now had a much better understanding of what parenting programmes could offer families. The PEIP had left most authorities with many more people having the 96 knowledge, skills and understanding to support families better than ever before. The value of using evidence-based programmes with fidelity and of giving facilitators appropriate training and support, including supervision, was much more clearly established than was the case prior to PEIP. Box 4.9 summarises some of the main benefits mentioned across LAs to provide a flavour of the wider impact of PEIP, over and above the benefits for participating families. Box 4.9 Summary examples of the wider benefits of PEIP from across LAs A more holistic way of looking at family problems has developed – realisation that compartmentalisation into antisocial behaviour, domestic violence, drug abuse and so on is not appropriate – the parents who are being helped tend to have a combination of issues. Workforce development – for example, development of multi-agency working increased sharing of knowledge, ability to work together, cross-fertilisation of ideas; encouraged partnership working from different community backgrounds, promoted more social inclusion and community cohesion; skilled up the workforce in a common understanding of parents’ and young people’s needs. Raised the status of parenting support because of the evidence of effectiveness the evidence-based PEIP programmes gave practitioners the opportunity to reevaluate their own working practises – many now working along these principles as opposed to what they did previously – given them a new structured way of working where they can assess whether the intervention has made a difference or not; PEIP programmes now highly valued within agencies; importance of supporting parents recognised at strategic level. Expanded capacity to support families - allowed many more families to be reached and to be offered consistent, high quality support because of numbers of staff trained; enabled coordinated delivery across a LA; enabled higher need families to be included because of funding to remove barriers to access such as lack of transport or childcare or need for one-to-one support prior to parenting group; greater recognition that some families need more than a parenting programme and therefore a range of complementary support put in place. 4.2.4 LA-wide delivery models Among the LAs participating in evaluation interviews, there were three main models of PEIP delivery, plus a fourth ‘hybrid’ model (Box 4.10). During the final round of interviews, we asked strategic and/or operational leads to highlight the strengths and weaknesses of their delivery model for PEIP. No single model emerged as the ‘best’; rather, it was a case of ‘best fit’ in any particular LA context. For example, among the top five LAs, three used a core team (M1) of parenting practitioners working with others to deliver PEIP, one used an areabased multi-agency delivery model (M2.2) and one used a hybrid model (M4). 97 Figure 4.2 PEIP delivery models Model M1 Description Core team delivery - co-facilitated with wider group of others M1.1 – PEIP core team M1.2 – parenting core team M2 Multi-agency delivery M2.1 - centrally coordinated M2.2 – not centrally co-ordinated M3 Delivery commissioned out by LA M3.1 – to one provider M3.2 – to more than one provider M4 Hybrid delivery (e.g. partly M2.1 and partly M3.1) The reported strengths of the core team (M1) delivery model were that it helped ensure delivery per se and that it created a team of skilled, experienced parenting professionals able to work with a wide range of co-facilitators whilst ensuring quality of practice. It helped to embed systematic evaluation and supported a centralised referral and referral screening system. The perceived weaknesses were in potential vulnerability to budget cuts post-PEIP and where there was reluctance by other services to release staff to co-deliver with the core team, thus reducing numbers of groups that could be offered. This issue particularly affected the LA that achieved the top scores for outcomes across all three measures but was only able to deliver to a small number of parents. In this case, the quality of outcomes was high but the efficiency of outputs was low. Multi-agency delivery (M2) was valued as potentially the most sustainable model post-PEIP although even it was viewed as vulnerable post-PEIP if public sector budget cuts affect the numbers of trained facilitators remaining across services and agencies. The multi-agency delivery model was also valued for the range of skills, perspectives and support for parents brought by the diverse group of facilitators; the ability to involve voluntary sector agencies as well as statutory; and the partnerships and developments it generated over and above the PEIP work. On the other hand, interviewees reported that multi-agency delivery was complex to manage and required effective communications to work well. When centrally coordinated, some of these issues could be managed more easily, such as a central referral system, and coordination of the timetabling and location of groups, central collection of data and ordering of resources, and central monitoring of quality assurance. One risk noted with the centrally coordinated multiagency model was that it could become limited in the agencies 98 and services involved. The devolved multi-agency model (M2.2) was valued because of the flexibility it offered to deliver according to local need but, unless managed locally by, for example, locality parenting networks or locality teams, this model risked disorganisation of recruitment, referral and quality control of delivery. The main negative point raised about the multi-agency delivery model was its reliance on facilitators who were not necessarily freed from other workloads and so might view delivery as an added burden rather than as an alternative, and potentially better, way of achieving desired aims for the parents/families on their caseload. A minority of LAs in Waves 1 and 2, and in our sample of Wave 3 LAs commissioned out the delivery of PEIP to one or more provider/s. No clear messages about strengths and weaknesses of this approach emerged: these seemed idiosyncratic within specific LA contexts. Similarly, a small minority of Wave 2 LAs and among our Wave 3 sample adopted hybrid models of delivery, such as partly core team (M1) plus some commissioned out (M3), or using a core team for key organisational and coordination tasks but not for delivery. Again, the strengths and weaknesses of these seemed to be idiosyncratic. Sustainability of PEIP delivery A key factor about the PEIP delivery model chosen in each LA was how sustainable it would prove to be once the specific project funding ended in March 2011. Interviewees from eighteen of the 47 Wave 2 and 3 LAs in our sample spoke about the value of conducting local systematic evaluation, in addition to, or derived from, local data contributing to the national evaluation. This was viewed as essential for building the business case for local effectiveness of programme delivery. Our last set of interviews took place before final local authority budgets had been set for the 2011-12 financial year. This discussion of sustainability needs to be read in that context of uncertainty. However, at the time of interview, the majority of interviewees were hopeful (though not certain) that the PEIP parenting programmes would continue to be offered, albeit on a smaller scale, and would continue to be delivered along the PEIP principles of running them with fidelity, with trained and supported facilitators. The message that this was what ensured quality outcomes for parents had been heard. Interviewees from about half the LAs were sure that the PEIP would continue, a high proportion given the climate of public sector cuts and the overall reduction in LA budgets. The main reasons for the level of confidence that existed in many LA for the future of PEIP are summarised in Box 4.10. 99 Box 4.10 Main reasons for hopes about sustainability of PEIP programme delivery • • • • • • • • the knowledge that the evidence of effectiveness had gained strategic level support for PEIP parenting programmes as part of early intervention strategies and that these would continue to be commissioned to meet local need a good fit between the local infrastructure and the PEIP which meant that carrying on with PEIP was readily feasible a long history of commitment to high quality parenting support in the LA which PEIP had reinforced having ‘trained the trainers’ during PEIP gave the LA capacity to train further facilitators as needed (one LA, had piloted a profit-making trading training service that would include selling training for IY, SFSC and SFP 10-14) expected cuts would still allow delivery although of a smaller number of courses plan for post-March 2011 involved retaining successful delivery model – especially multi-agency delivery (M2) but also small core teams and/or central coordinating roles viewed as efficient in ensuring delivery and monitoring maintenance of quality and effectiveness plan for post-March 2011 involved changing delivery model – for example, cutting core team and focusing on multi-agency delivery; shifting from a centralised multiagency model to a devolved multi-agency model to integrate more easily into locality team structures; changing to a model where schools buy in parenting support (although there was some uncertainty about whether or not schools would, in fact become commissioners of parenting support in this way) the multi-agency delivery model would ensure that, given continued support from the different agency and service managers, delivery would continue There were some concerns that expected cuts in funding would mean that, as well as forcing a reduction in the amount of groups run, there would also be a reduction in the quality of what was offered because cuts would affect coordinator posts and/or core team posts (i.e. the most experienced facilitators) and the ability to attract parents by using pleasant venues and offering healthy food, childcare and transport. A small minority of LA representatives interviewed (4 LAs) were not optimistic about the future. In one LA, for example, the lead feared that a lack of strategic level interest and support would mean that parenting support would revert to its pre-PEIP home of limited provision through the local CAMHS team. In another LA, it was the ‘costly’ nature of the PEIP programmes (requiring food, crèche; length of delivery) that caused the operational lead to query whether schools, perceived as the main commissioners for the future, would buy them. (Both these LAs achieved higher than average outcomes for parents but had relatively low numbers of parents going through, at least as measured by booklets returned to CEDAR.) In the two other cases, the interviewees were facilitators who were not optimistic about their own service’s continued involvement post-March 2011 because of expected cuts in staffing. 100 4.2.5 Recruitment, retention, support and supervision of facilitators In this sub-section, the results of our study of PEIP facilitators and their impact on outcomes for parents and young people is presented first, followed by interviewees’ views about what worked well in recruiting, retaining and supporting facilitators. PEIP facilitators and their impact on outcomes for parents and young people A key infrastructure variable across LAs was the pool of facilitators delivering PEIP. As reported earlier (Lindsay et al., 2010a), during the first round of interviews with PEIP strategic and/or operational leads in autumn 2008, there was evidence of tension in perceptions about guidance coming from the National Academy for Parenting Practitioners (NAPP) that graduate helping professionals should be trained first to lead the delivery of new PEIP programmes 26 versus LA views that what mattered was training facilitators with qualities that would engage parents. The PEIP evaluation included exploring what, if any, impact there was on outcomes for parents of facilitators’ education, professional qualifications and prior experience of delivering parenting programmes. (Full details of the method, analysis and findings are in Section 3, a summary of findings is presented here.) The facilitators used in PEIP were mainly women (89%) but otherwise a very diverse group in age (20s to over 60) and ethnicity (15% from minority ethnic groups). They were also diverse in terms of their highest educational qualification (ranging from 42% with a Bachelors’ degree to 1.3% with no educational qualifications) and in their employment backgrounds (including, for example, education, health and social care, social work, health, youth work, counselling and therapy). Among this diverse group of people, prior to PEIP, only a minority had been trained in one or more of the five approved PEIP programmes (32% Triple P, 28% Incredible Years, 22% SFP 10-14, 14% SFSC, and 3% FAST), 33% in another evidenced based programme and 26% in a non-evidenced-based programme. Levels of prior experience in terms of delivery were relatively low - only one-third (32%) had previously delivered (facilitated or co-facilitated) any of the recognised PEIP programmes. The second key finding was that this wide range of people was successful in delivering the PEIP programmes effectively. There were no significant effects related to facilitators’ prior experience of training or delivery before PEIP. There were facilitator effects in two specific 26 NAPP application and guidance form sent to local authorities: ‘Phase 4. Parenting Commissioner and Service Manager Application for Practitioner Training in an Evidenced Based Programme (EBP)’ p.6. 101 areas (improvements in parents’ mental well-being and parents’ ratings of group leader style) that related to educational qualifications. For parent mental-wellbeing, non-graduates were associated with greater improvement than graduates and, in the case of group leader style, non-graduates with no prior training/delivery experience had significantly higher ratings for group leader style than any other facilitator grouping. These two facilitator effects were statistically significant but small. The main finding is that for the majority of the outcomes we measured (parent and child), there were no differential effects related to facilitator qualification. Our findings therefore support the view that a diverse workforce may be used effectively to deliver PEIP parenting programmes. This finding would probably not have been foreseen prior to PEIP. Local authority views From the final set of interviews with PEIP leads, there was a sense of confidence about using a diverse pool of facilitators drawn from a range of services and agencies. The main concern this had raised was ensuring those who were trained had the capacity to deliver, and the support of their manager to release them to do so. These concerns continued throughout both the Pathfinder (Lindsay et al., 2008) and PEIP. Nevertheless, most LAs reported being pleased with the ratio of facilitators trained to those who delivered. Examples of LA reports of these included: 60 trained: 59 delivered; 125 trained: 120 delivered; 60% of those trained delivered at least one programme. Each of these examples represents a higher delivery to training ratio than for the facilitators trained prior to PEIP (see above). Reported numbers of facilitators trained during PEIP varied widely (in part because of varying numbers previously trained). Examples include: more than 200 accredited Triple P practitioners delivering in one of the top five LAs, 150 trained in another compared to 16 and nine in two others. In addition to capacity to deliver, and a diverse range of employment backgrounds, local authorities also sought key skills, personality characteristics and qualities, and experience as opposed to levels of qualification, according to most PEIP lead interviewees. This approach was corroborated by facilitators’ views of what was required to be an effective facilitator (summarised in Figure 4.3). A minority of PEIP leads did look for qualifications – for example, a strategic lead in and an operational lead in two different LAs judged Parent Support Advisers (PSAs) not to have sufficient qualifications and experience to be facilitators whereas, in many other LAs, PSAs were viewed as potentially ideal, having qualities and skills to engage parents. 102 Figure 4.3 Composite summary of perceived requirements of effective facilitators Skills - able to: build trusting relationships manage groups, skilfully responding to issues whilst maintaining fidelity, keeping on task and to time and giving everyone a voice foster learning encourage and guide though programme plan and prepare for sessions lead and pass on skills of delivery to co-facilitators and helpers manage challenging behaviour teach strategies and new knowledge work within delivery team (including parents, where relevant) with clarity about roles and responsibilities For FAST specifically offer one to one coaching Experience of: Knowledge of: a responsible post working with adolescent development parents and/or children group dynamics and how to use and/or families these positively group work or delivery to groups parenting when and how to follow-up issues For parents on FAST initial cycle linking with local school and raised by parents and/or completing FAST training young people levels of need for which the group For parent partners on later is appropriate cycles completing previous FAST cycle the underpinning theory on which the principles of the For parent helpers on other programme are built programmes completing the programme the research showing the volunteering on the programme effectiveness of the programme For parent co-facilitators as above, plus completing the programme training Source: interviews with facilitators. For programme expectations of facilitators, see Appendix 2. Qualities committed to delivering programme respectful good listener down to earth/approachable quick thinker outgoing professional judgement but accepting and nonjudgemental in approach flexible good humoured patient responsible aware responsive supportive empathic enthusiastic Supervision of facilitators Interviewees reported that the PEIP has been instrumental in raising awareness in local authorities of the value and relevance of supervision for parenting programme facilitators. Supervision was valued by leads and facilitators alike as providing workforce development, quality assurance and fidelity assurance (Box 4.11) 103 Box 4.11 Example of value of supervision for facilitators The Facilitator received weekly supervision from a manager; ‘I think the supervision is an absolutely essential part of the planning and the evaluation of the course’ because, during delivery of SFP 10-14, the facilitators are running parallel groups so supervision allows them to be together and to decide how best to ‘tweak’ delivery by, for example, agreeing that one facilitator from the parent and youth group meet up for a minute before the break and family session to update each other on any issues arising; having supervision gave her confidence that she ‘had back-up’ and that, if she didn’t know the answer for her cofacilitators, that it didn’t matter because she could take it to supervision: ‘It was done in such a way that it was a very supportive process. Everybody had an opportunity to give their views and we were able to unpick certain situations that arose.’ She believed that this enabled this new team of facilitators to work well together. Across the sample LAs, what was described as ‘supervision’ took many forms, only some of which would be classed as supervision in the helping professions where this practice originated. For example, a composite list of practices reported would be very long but would include: peer supervision, post-programme peer discussions, weekly supervision with a qualified Incredible Years’ mentor plus group supervision, weekly meetings with team leader, plus monthly supervision from SFP 10-14 trainer; half-termly refresher workshop for SFSC facilitators; parenting co-ordinator meeting with facilitators before, half-way through and at the end of the programme, plus networks of facilitators meeting three times a year. Some concerns were raised about supervision in some LAs – for example, where it was not being provided by a helping professional, such as a psychologist, where managers were reluctant to release staff for supervision as well as for preparation and delivery of the parenting programme, or where facilitators did not attend. This is an important cost-benefit decision that LAs need to make, balancing the developmental and quality assurance aspects of supervision against the additional costs and time away from day jobs. Arguments based around effective outcomes for parents would suggest that supervision is a necessary cost 4.2.6 Recruitment, engagement and retention of parents Recruitment and engagement Having facilitators from a range of different services and agencies and from a range of backgrounds was one central theme about what worked well in recruiting and engaging parents to start a PEIP programme. Facilitators were able to engage parents known to them from their own agencies and geographic areas, based on an existing trusting relationship. Using a diverse workforce as facilitators was thus also of benefit to the efficiency of PEIP, as 104 well as to its effectiveness (as reported above). Box 4.12 provides a summary of the main other approaches to recruiting and engaging parents successfully mentioned by interviewees. Box 4.12 • • • • • • • • • • • • Other reported ways of successfully recruiting and engaging parents personal contact maintained prior to the start of the group important in itself but also to ensure that the parenting group is appropriate for that family at that particular time advertising as a universal programme for any parent (reported strength of Triple P) if targeting high need families, offer parenting group alongside whatever other family support is being accessed (and ensure that appropriate other support is being offered); have clear targeting criteria; offer one to one work to parents who are not ready to cope with a group as a bridge to accessing the parenting programme in time ‘light touch suggestion’ and ‘coaxing’ of parents, rather than heavy-handed referral especially effective with parents with higher levels of need self-referral – supported by word of mouth recommendations, by parenting information packs, by DVDs showing views and experiences of parents who have previously completed the programme recruitment by professionals known to the family, including outreach workers who can encourage and support parents to engage – supported by ensuring that local professionals, including statutory services, understand what the parenting programme/s offer (e.g. offer taster days or provide written information for professionals to find out about the programme/s) advertising and marketing, plus one to one conversations through the Common Assessment Framework (CAF) process through schools, especially through pastoral, family and parent support workers in schools to recruit and engage fathers in particular: having dedicated workers for this purpose; delivering fathers only groups; offering the parenting programme at local father contact centres (for divorced or separated fathers); running evening and weekend groups; having male facilitators; marketing widely enough so that solicitors working with fathers seeking access to their children know about the programme/s offer transport and childcare where lack of these would be a barrier to attending offer parenting groups in local areas and in accessible venues One barrier around successful recruitment reported was not (yet) having established the PEIP as a trustworthy project in the minds of local professionals who would be expected to refer and engage parents. This was reported in particular by two LAs that were especially unsuccessful in recruiting parents – in one, a Wave 2, this was attributed to lack of an operational lead and in the other, a Wave 3 LA affected by delays in accessing training for facilitators, to the subsequent pressure to deliver PEIP groups quickly resulting in not taking the time needed to build trusting relationships with potential referrers. Other reported barriers, or issues raised as needing further reflection in particular LAs around recruitment and engagement, were: 105 • language issues amongst diverse immigrant populations where English was not spoken or understood fluently (translators were used but language remained a barrier to recruitment) • parents being unwilling to attend the programmes, especially if they had been ‘directed’ to attend through, for example, Parenting Orders or Parenting Assessments or CAFs, or had felt particularly ‘targeted’ • how to engage fathers • how best to manage and prioritise waiting lists • how best to recruit most needy families The attention and time given to the recruitment/referral process, and crucially to the precourse engagement of parents (and young people), was a key variable in the LA infrastructure supporting PEIP roll-out. Where this was not done successfully, LAs reported multiple groups starting and having to close prematurely due to disengagement (drop-out) or groups being planned but not starting because parents did not turn up as expected. Clearly this affected both efficiency and effectiveness. Some LAs had processes in place to follow up parents who did not turn up to the first session as expected. For example, parents were routinely offered the programme a set number of times, or were referred back to the referring professional, or, where capacity allowed, facilitators offered one to one support instead of the group. In one LA, facilitators routinely did a ‘disengagement visit’ to find out reasons why the parent had not turned up. In other LAs, though, there was no process in place to follow up these parents. Acknowledging that a parenting group was not always an appropriate intervention Interviewees, especially operational leads and facilitators, discussed the fact that in some cases, a PEIP parenting group was not an appropriate intervention. Sometimes, this was because the parent and/or the young person was not convinced that it was right for them. More often it was because the professionals judged that a group experience of the intensity of a PEIP parenting programme was too much for the mental state of the parent or young person at that point in time or that the parent was not ready to access a programme that expected them to change (as opposed to expecting only their child’s behaviour to change). In such cases, a number of LAs had put in place alternative support prior to reoffering a PEIP parenting group at a later date. These included, for example, one-to-one work on topics covered in the PEIP programme (e.g. Triple P one to one or personal coaching based on another PEIP programme), or a parent only parenting group instead of a parent and child 106 parenting group, or a less intense parenting group, such as Family Nurturing or the Solihull Approach. Retention Retaining parents who started a programme is an indicator of the quality of the experience and the appropriateness of the processes around recruitment and engagement. Interviewees’ views about what worked well in retaining parents (and young people) who began a PEIP programme included: parents already knowing at least one facilitator (i.e. prior relationship); having confident facilitators delivering to a high standard (i.e. quality of the experience in the group); having facilitators who know the area well and so can relate the programme to participants’ lives; the warmth and encouragement of the facilitators and the acceptance that change takes practice and time; seeking weekly feedback from parents (and young people); facilitator skill in managing group dynamics (and creating groups with a useful rather than unhelpful mix of families); other appropriate support being in place where needed as well as the parenting programme (e.g. other agencies involved in a CAF ‘doing their bit’). Retention was perceived as difficult when crises happened in families’ lives (domestic violence, child protection issues, illness, bereavement) and when referrers had not shared, or had not known, the full picture of a family’s needs. Some degree of non-completion was expected and accepted, given the contingencies of life. However, many LAs ensured that facilitators followed up those who left prior to completing to find out why this had happened and, where appropriate, to offer another opportunity to do the programme or to offer alternative support. In most cases, where it was relevant to do so, such cases were referred back to the referrer. Follow-up after completing a parenting group As raised in the 2nd Interim report (Lindsay et al., 2010b), interviewees reported that a minority of parents with particular issues would benefit from further support following the PEIP programme. This need took two forms: a) a need for further support to help embed implementation of the new strategies learned on the programme or b) a need for different support for other issues over and above parenting support, such as debt management. In the final round of interviews, further information was gained about the kinds of support parents were offered. Those who needed support for non-parenting issues were signposted, referred or introduced to the relevant service or agency that could offer support. In most LAs, but not all, those who completed a PEIP parenting programme were offered some form of continued contact with facilitators and/or other parents who had also completed a 107 programme. Across the LAs, this took many forms from a simple giving out of facilitator telephone numbers that parents could use to contact them again if they needed support to programme specific follow-on sessions (booster sessions for SFP 10-14, refresher workshops for SFSC, FASTWORKS for FAST) to systematic follow-up phone calls from facilitators 3 and/or 6 months post-course. The local evaluation of PEIP in a minority of LAs included follow-up interviews with a sample of parents Box 4.13 gives some LA specific examples of follow-up. Box 4.13 Examples of continued contact with parents post-course Local evaluation follow up by questionnaire; active encouragement for peer support; signposting to other services available for support (SFSC) Parents contacted before group finishes to see if any want to train as a parent volunteer. Follow up six times a year on a particular subject, in the form of a workshop, on a subject that’s cropped up needing further exploration. Many parents continue to meet up as part of a Parents’ Forum or through fund raising for a local school or community organisation. Parents offered follow up sessions, but facilitator keeps in touch with parents in the normal course of her work. Programme specific follow-up. Opportunity to progress to FASTWORKS- good uptake. (Triple P) 3 month follow up phone calls (TP) Added a Speakeasy workshop to many programmes and worked closely with local colleges and Family Learning Service so many families have progressed to other projects; SFP 10-14 booster sessions. Also RARPA (Recognising and Recording Progress and Achievement), an award certificate for a non-accredited programme, is popular. Easiest for children’s centres who have natural ongoing contact; facilitators encouraged to have a 3 and 6 month call back system; if referred through schools or CAF parents tend to move to another form, another intervention. Follow up has not been universal, or uniform. There was recognition that offering parents some form of continued contact with facilitators and/or fellow ‘graduates’ of the parenting programme was a useful way of helping parents to maintain the progress they had made around changes in their parenting and in their relationship with their children. 108 5. COST EFFECTIVENESS OF THE PEIP Main findings • There was substantial variation in the cost effectiveness of the PEIP between LAs, e.g. the proportions of expenditure on management varied greatly. • The proportion spent on management reduced over time as the PEIP became established. • The proportion of expenditure on management compared with parenting group delivery varied greatly between the four programmes. • The average cost of funding a parent who started a PEIP parenting programme was approximately £1,250; this increased to approximately £1,700 per parent completing if the 75% completion rate found in the LAs providing data is taken into account. • The lowest cost per parent in one LA that had been operating for the full three years of the programme was only £534 5.1 Introduction In this section we report on the costs and cost effectiveness of the PEIP. It was not possible to use DfE grant allocation data as LAs were able to vire monies across different programmes and an initial DCSF monitoring system, which recorded the number of groups run and parents supported, was not used throughout the PEIP. Consequently, it was necessary to approach LAs direct for their own data. The findings reported here are derived from 15 of the 43 LAs that supplied end of PEIP information on costs (response rate: 35%). This information is unaudited and the sample size is relatively small given the variation in number of parents supported by the 43 LAs for whom we have data from parenting measures (Section 2). Consequently, because of these limitations with the data we were unable to carry out a comparative analysis of the cost effectiveness of different programmes. 5.2 Cost effectiveness analysis The costs for each LA were derived from the questionnaire designed for this study. These were compared with the outputs, namely the improvement in parent mental well-being and their reduction in laxness and over-reactivity, and improvements in child behaviour. Ultimately it is the latter that is of primary interest. However, we include parent factors as these are the factors most directly addressed by the programmes overall. 109 The PEIP provided specific funding to each LA to spend on implementing the parenting programmes and on infrastructure to provide leadership, organisation and management. How LAs spent their grant could vary as they were free to vire monies across different strands of the Think Family total grant. Also, as this grant was allocated, rather than the service being purchased, high numbers of parents being supported would indicate lower costs per parent. Parents not completing a programme, however, would increase costs per parent overall. Relating these factors with the parent and child outcome measures would indicate cost effectiveness in terms of the cost of different levels of improvement on our measures. A rigorous cost effectiveness study was not possible as DCSF decided not to monitor and audit LA use of the grant. Our parent level data concerning impact were satisfactory as we selected recognised measures and administered the data collection – see Section 2. We have data on over 6000 parents but we have no way of knowing how many other parents attended groups but did not return pre-course and post-course questionnaires. We know from interviews with LA lead officers and from the facilitator forms that this occurred but the scale is unclear. We requested cost data from 43 LAs that had parent- completed questionnaires (Section 2) by means of a questionnaire designed for the purpose, following a pilot in five LAs. We collected the following data: • total costs of the programme per year • aggregate data as the pilot suggested that very disaggregated (activity) cost data is problematic to collect; • whether there was additional revenue raised from sources outside the core parenting programme allocation; • confirmatory data about the number of parenting groups and families that had participated in the programme We received responses from 15 LAs for parenting intervention costs between 2008/09 and 2010/11. This corresponds to a response rate of 35% (which was high given the detail of the information being requested). The response rate may be systematically higher or lower for certain types of LA and this could introduce some bias into our estimates of average costs and prevents a valid comparison of the cost effectiveness of the different programmes. 110 5.3 Findings In Figure 5.1, we provide information from the survey respondents on the total revenues received and costs, aggregated over the three years, associated with the provision of parenting programmes in each of the LAs (in 2010/11 prices). In any particular year, the total resource allocation was approximately £136,000, of which 86% was received from central government and remaining 14% received from other sources (such as other Local Authority funding, Standards funding or specific projects (e.g. Working Neighbourhoods)). However, of the budgets allocated to PEIP, the survey responses indicate that approximately £12,000 per LA per annum was spent on activities other than those undertaken to ensure delivery with fidelity of any of the five main PEIP programmes mainly to parents of children aged 8-13. Figure 5.1 Costs associated with parenting intervention programmes Total Net Income £1,000,000 Total Costs £900,000 £800,000 £482,738 £600,000 £500,000 £428,052 £700,000 £400,000 £300,000 £200,000 £100,000 £0 Note: All monetary values expressed in 2010/11 constant prices. Source: Analysis of CEDAR primary LA data There is significant variation around the mean, with some LAs receiving more than £300,000 in any one financial year and others receiving less than £80,000. We have also presented the average reported costs of provision of the programme in individual years by LA, which demonstrates the fact that the sum of the direct and indirect (e.g. backfill costs /opportunity 111 costs) is approximately £13,000 less than the total resource costs allocated to the programme. In aggregate over the entire life of the intervention, the average total resources allocated to support the parenting programmes stands at approximately £483,000 per LA though there was significant variation around the mean. Total costs vary depending on size of PEIP grant, and whether the LA was in Wave 2 (started 2008) or Wave 3 (started 2009). The largest allocation of resources from central government was received by LA J (more than £850,000); however a number of LAs were able to supplement the central government funding received with substantial resources from elsewhere. In particular, four LAs received more than 23% of their total income from non-central government sources. 5.4 Disaggregation of expenditure As part of the survey instrument, we attempted to ascertain the actual expenditure patterns of the LAs. We asked for information on the set up costs associated with the parenting programmes, as well as the project management costs, training costs and the costs associated with the delivery of parenting groups. To assess the total economic cost associated with various parenting interventions, we also asked for information on the opportunity costs associated with the programmes during the initial phase of activity (for instance backfill costs covering replacement staff incurred during the provision of training to facilitators). Expenditure information was asked on a year-by-year basis, and as a result, there is a significant variation in the nature of the expenditure incurred by year (training dominating the first year(s) of activity and the delivery of parenting groups dominating in subsequent years). Average costs for the PEIP as a whole across all Local Authorities were: • Project management: £192,830 • Training: £113,386 • Parenting group delivery: £108,651 • Backfill: £8,780 112 Figure 5.2 Nature of expenditure associated with parenting intervention programmes Parenting Group Delivery, £108,651, 26% Project Management Costs, £192,830, 45% Backfill Costs, £8,780, 2% Training Costs, £113,386, 27% Note: All monetary values expressed in 2010/11 constant prices. Source: Analysis of CEDAR primary LA data Despite the possible inconsistency in the treatment of the various costs at LA level, the analysis in Figure 5.2 suggests that a significant proportion of the total costs associated with the programme was subsumed by project management costs (approximately 45% of the total costs including indirect (opportunity) costs). The classification of project management costs included the value of the Strategic Lead’s time on PEIP, the Operational lead(s)/PEIP co-ordinator(s) and any administrative assistance. Therefore, project management costs were approaching twice those of the actual delivery of the parenting programme (45% v 26%). It may be the case that a number of LA respondents allocated expenditure to this category as a default; however, it does appear that a relatively large proportion of total resource allocation was not spent on either training facilitators or delivering the parenting intervention directly. There was significant variation around the mean. In one LA, the stated expenditure on project management costs reached 87% of total stated expenditure, while in three LAs, the proportion of total expenditure committed to project management activities was less than 20%. The potential expenditure on different activities may well be a function of the Wave of respondent. The average proportion of total cost spent on project management and parenting group delivery was influenced by Wave, whereas the proportion of total cost spent on training and backfill was essentially independent of Wave (just 2 percentage points higher in Wave 2 compared to Wave 3). In particular, for those Local Authorities in Wave 2, project 113 management costs as a proportion of total costs stood at approximately 40% (compared to approximately 58% for Wave 3 respondents), while parenting group delivery costs as a proportion of total costs stood at 30% for Wave 2 respondents compared to 14% for Wave 3 Local Authorities. Apart from project management costs, the remainder of the expenditure undertaken by LAs was relatively evenly split between training and parenting group delivery. Specifically, 27% of total expenditure was committed to the training of facilitators, while 26% of expenditure on average was related to parenting group delivery. There was again substantial variation between programmes but caveats regarding these differences should be recognized for the reasons stated above. 5.5 Costs of programme outputs Turning from the costs of delivery, we consider below the outcomes associated with the parenting interventions. Given the time lag between the initial implementation of the programme and the actual delivery of PEIP, we have aggregated all information at LA level (and by programme type) to illustrate a number of outcomes. The analysis illustrates that, on average, approximately 73% of parents complete the parenting intervention, although again there are differences across LAs. In two LAs, parental completion rates stand at between 50% and 60%, while in five LAs, completion rates are in excess of 80%. However, there is no obvious statistical relationship between the parental completion rate and the proportion of PEIP funds that are allocated to front line activities (training and parenting group delivery cost). For the individual programmes, the information collected from LAs indicates that the average completion rate amongst Triple P interventions was considerably lower in this sample than the other three interventions (Figure 5.3). In particular, Triple P parenting group completion rates stood at approximately 64%, compared to completion rates of between 77% and 78% for the other three interventions; however, care should be taken when considering this information disaggregated by particular programme since the choice of programme may be related to the severity of parental need in a particular area and Triple P may have been adopted by LAs with more challenging circumstances. It is also possible that the sample of respondents is not representative of the wider population of Triple P parenting interventions – certainly the 64% non-completion rate in this sample is substantially lower than for the PEIP sample as a whole. 114 Figure 5.3 Parental completion rates 95% 90% 83% 85% 75% 67% 89% 80% 66% 70% 65% 82% 79% 78% 77% 66% 66% 77% 78% 78% 73% 64% 59% 50% 55% 45% 35% 25% Note: All monetary values expressed in 2010/11 constant prices. Source: Analysis of CEDAR primary LA data Costs of parenting groups. In terms of delivery, we have compared the costs of provision with the actual output associated with the parenting interventions. We have estimated the total cost of the programme in each of the LAs and compared this to the total number of parents completing the parenting groups over the period of intervention. As PEIP had not been fully completed at the time of data collection, we asked respondents to provide an estimate of the expected completion rate for those parents still involved in the programme up until the end of the PEIP (in March 2011). We are also aware that the parenting programmes did provide other forms of intervention in addition to the parenting groups. In particular, we also collected information on the number of parents receiving one-to-one support, the number of parents attending a Triple P seminar that was funded through PEIP, and if there were any other incidences of support being provided through PEIP. The analysis in Figure 5.4 illustrates that the average cost per parental intervention occurring within a group session (i.e. the reported costs divided by the number of parents starting a parenting group across all LAs across all financial years of operations) stood at approximately £1,244 (blue bars); however, once the completion rate of 73% is incorporated into the calculations, the average cost per completed (or ‘successful’) parenting intervention (i.e. the reported costs divided by the number of parents completing the parenting group) stood at approximately £1,658 (red bars). There is again some variation around the mean with two LAs indicating that the average cost per parenting group intervention was approximately £2,200 (with an associated cost per completed parenting group intervention of approximately £3,000). However, it is impossible to indicate whether this variation 115 demonstrates genuine differences in the actual costs of delivery across these LAs or differences in the quality of the reported cost information. In addition to the average cost per parent completing a programme, we have also estimated the average cost per parenting intervention, where we compare the total direct and indirect costs associated with the delivery of the programme to all interventions with parents (i.e. completed parenting group attendance, one-to one assistance, attendance at a PEIP funded Triple P seminar or ‘other’ assistance). The analysis indicates that the average cost per intervention across all LAs stood at just under £1,000 per intervention. Figure 5.4 Costs per intervention £4,500 £4,000 £3,500 Cost per group session 'completion' (number of parents completing the parenting intervention within a group session environment) Cost per group session 'intervention' (number of parents starting the parenting intervention within a group session environment) Cost per wider intervention Cost per training place £3,000 £1,658 £1,244 £997 £989 £2,500 £2,000 £1,500 £1,000 £500 £0 Note: All monetary values expressed in 2010/11 constant prices. Source: Analysis of CEDAR primary Local Authority data Costs of training facilitators. The analysis presented in Figure 5.4 also illustrates the average cost per training position across the LAs. We compared the total stated cost associated with training activities and the total number of NAPP/CWCD training places taken up for PEIP programmes, the number of facilitators attending group facilitation skills training and the number of facilitators attending ‘train the trainer’/becoming a mentor (Incredible Years). In aggregate, the responses indicate that approximately 72% of training activity related to the provision of NAPP/CWDC places taken up for facilitators to be trained to implement PEIP programmes, with 26% of training activity relating to group facilitation skills (and approximately 2% of activity relating to facilitators attending ‘train the trainer’ or becoming a mentor). 116 On average the cost per training intervention stood at just under £1,000 across all LAs in all years; however, this estimate may be influenced by outliers in the data (i.e. some Local Authorities reporting high average costs per training intervention). Another measure that could be considered is the median value, which is the ‘middle’ value of the distribution of the cost per training intervention. When each Local Authority’s year by year financial information is assessed, the median cost per individual trained years stands at £644, while when each Local Authority’s aggregate financial information is assessed, the median cost per individual trained years stands at £715). This implies that the sample responses may be subject to some bias resulting from the inclusion of outlier data in the sample. 5.6 Costs of parenting outputs The final part of this element of the analysis involves the assessment of the costs associated with achieving the improvements in parental outcomes suggested previously in the evaluation report. In particular, the analysis illustrated 27 that amongst those parents completing the parenting programmes, the estimated gain on the mental well being metric stood at 0.79 of an effect size, the gain on the parenting laxness metric stood at 0.72 of an effect size, while the estimated gain on parenting over-reactivity stood at 0.85 of an effect size. We have estimated the average cost per standard deviation improvement (1.0 of an effect size) 28 where the lower bound represents the total stated direct and indirect cost associated with the provision of parenting interventions across all LAs in all the three financial years, while the upper bound represents the total net income associated with the provision of the parenting intervention (i.e. central government allocation of resources plus resources from other sources minus resources spent on PEIP activities). The analysis indicates that the indicative cost of achieving a mental well being effect size stands at between £2,100 and £2,400; the cost of achieving a one ef’/fect size reduction in parenting laxness stands at between £2300 and £2600, while the estimated cost associated with achieving a 1 effect size reduction in parenting over-reactivity stood at between £2,000 and £2,200. 27 See Section 2.5 This method allows comparison of costs for a unit of improvement. The sums presented are indicative only as a linear relationship between effect size and cost cannot be assumed. See Scott, Sylva et al. (2010) for another example of this approach. 28 117 Table 5.1: Estimated costs per parental effect size Parenting over- Outcome Mental well being Parenting Laxness Effect Size 0.79 0.72 0.85 Cost Range £2,112- £2,381 £2,317-£2,613 £1,963-£2,213 reactivity Note: All monetary values expressed in 2010/11 constant prices. Source: Analysis of CEDAR primary LA data 5.7 Costs of child outputs The analysis also provided some indication of the impact of the parenting interventions on the fundamental focus of the policy - child outcomes. As reported in Section 2.5 the effect sizes for the child measures are somewhat lower than those achieved for parents, but still very substantial: SDQ Conduct problems (Effect size = -.45), SDQ Total difficulties (Effect size = -.45) and for SDQ Impact score (Effect size = .-52). These lower levels are unsurprising given the fact that the main interaction throughout the intervention occurs between facilitators and parents rather than with the children directly. The analysis indicates that the indicative cost of achieving a one effect size reduction in conduct problems stands at between £3,800 and £4,300; the cost of achieving a one effect size reduction in SDQ total difficulties stands at between £3,800 and £4,300, while the estimated cost associated with achieving a 1 effect size reduction in the SDQ impact score stood at between £3,300 and £3,700. This is presented in Table 5.2. Table 5.2: Estimated costs per child effect size Outcome Conduct Problems SDQ total difficulties SDQ Impact Score Effect Size 0.45 0.45 0.52 Cost Range £3,795- £4,280 £3,795- £4,280 £3,285-£3,704 Note: All monetary values expressed in 2010/11 constant prices. Source: Analysis of CEDAR primary LA data 5.8 Conclusions This section presented an analysis of the variation in costs and an indication of the cost effectiveness of implementing PEIP. These results must be interpreted with caution as we had data from only one third of LAs. Furthermore, the accuracy of LA data could not be checked as there was no DfE system of monitoring how grant was spent. 118 There was significant variation in LA spend over the period of the PEIP that reflected front loading of expenditure on training followed by the expenditure on running parenting groups. Management costs varied greatly between LAs and also for different programmes. There was an interaction effect as Incredible Years comprises the most sessions and has the greatest proportion of expenditure on parenting group delivery; Triple P, by contrast, is the shortest and had the highest proportion of spend (one third) allocated to training. Cost effectiveness was a function of both expenditure and completion rates. The former was a function of recruitment success, which also varied greatly between LAs. Over the PEIP as a whole the cost of a parent training intervention was £1244. When the overall completion rate of 73 per cent is taken in account this rises to £1658. It is also interesting to consider that the lowest cost per parent in one LA that had been operating for the full three years of the programme was only £534 These results may be compared with recent studies where costs per parent have been reported as £2380 per child (Scott, O’Connor et al., 2010), and £1343 per child (Scott, Sylva et al., 2010) and £1289 (Edwards et al., 2007). The estimate of costs in the Pathfinder (Lindsay et al., 2008) was £2135 per parent starting a group and £2955 per parent completing (to take account of a completion rate of 73%, the same as the current study). Overall, therefore, the PEIP compares favourably with the costs of other studies, especially as the trials are typically costed in terms of delivery of groups, excluding infrastructure costs which are included in the analysis of PEIP (project management, training). 119 6. DISCUSSION AND CONCLUSIONS AND RECOMMENDATIONS 6.1 Introduction The evaluation of the Parenting Early Intervention Programme (PEIP) took place 2008-11 and examined the national roll out of parenting programmes across all 150 English local authorities (LAs). It built upon the 2006-08 evaluation of a Pathfinder (Wave 1) in 18 LAs. In addition to the evidence examined in this final report, earlier reports provide further evidence of this five year development (Lindsay et al., 2007a, 2007b, 2008, 2009, 2010). The main focus of this final report is to present the end of PEIP evidence in relation to the aim of evaluating the impact and benefits of the national roll out of the PEIP. In this section we review the evidence presented in this report with reference to the three research objectives. 6.2 Was parenting training effective in the new sites? Previous research had demonstrated that parenting programmes can make a positive contribution to improving parenting skills of the parents who attend the programmes and reducing the behavioural difficulties of their children. The five programmes originally approved by DCSF had evidence from carefully derived and implemented trials of their efficacy. Our study of the 2006-08 Pathfinder showed that the three programmes used then could be effective when rolled out across the community settings in 18 LAs. This distinction (efficacy v effectiveness) is important. Efficacy trials show that a parenting programme can work under particular conditions. Ideally they comprise randomised control trials and allow comparison of changes in important measures shown by those parents who undertake the programme compared with those who do not. For policy development, however, it is necessary to investigate further whether this will hold up when implemented under normal conditions of ‘real world’ service delivery. Firstly, the scale is much greater: trials typically include around 100 or fewer parents receiving the programme. Secondly, the facilitators are drawn from community settings implementing the programme as part of their work: they are not selected for a trial. Thirdly, the programme developer is not directly engaged in the study. Fourthly, staff are not associated with the programme and so have no investment in its success. Fifthly, the staff must operate under normal working conditions, meeting day to day pressures. The present study has extended this evidence base for effectiveness from 18 LAs in the Pathfinder to 43 LAs, with data on over 6000 parents, who experienced one of four evidence-based parenting programmes: Triple P, Incredible Years, Strengthening Families, 120 Strengthening Communities, and the Strengthening Families Programme 10-14 (SFP 1014) 29 . Essentially the PEIP evaluation has demonstrated that all four programmes were effective in increasing parents’ mental well-being, reducing their parenting laxness and overreactivity, and also reducing their children’s behavioural difficulties, all key protective factors for long term child outcomes. Triple P produced greater improvements in children’s conduct problems than the other three programmes. It was also more effective for improving parenting laxness than SFP 10-14, and for improving parenting over-reactivity than both SFP 10-14 and Incredible Years. There were no significant differences between Triple P and Strengthening Families, Strengthening Communities. However, it is important to stress that all four programmes were effective on all measures and the differences between them, although statistically significant, were relatively small. All four programmes were also rated very positively by parents with respect to group leader style and the helpfulness of the programme although SFP 10-14 was rated less positively than the other programmes. These results compare well with recent studies in the UK. Hutchings et al (2007) implemented Incredible Years in North Wales. This was a trial with 104 children’s parents receiving the parenting programme and 49 controls that did not. Using the same measures of child behaviour (the Strengths and Difficulties Questionnaire: SDQ) the intervention group’s improved mean scores were similar to those reported in our large scale study across all four programmes. An earlier study of the Incredible Years by Scott et al, (2001), with a sample of 63 children, reported less improvement and a more recent study (N = 89) in England that used the Incredible Years and a reading programme reported no significant improvements in conduct problems (Scott et al, 2010b). Improvement in conduct problems for the PEIP was less than Hutchings et al., but greater than the Scott et al. studies; hyperactivity was comparable to Hutchings et al; SDQ Total difficulties improvement was greater than Hutchings et al. This comparison suggests that the PEIP achieved comparable improvements in child behaviour (as reported by parents). The overall cost of implementing the programmes through the PEIP was £1244 per parent starting a programme although this rises to £1658 when allowance is made for the programme completion rate of 73%. We do not report comparisons of costs between 29 Insufficient data were available for an examination of the fifth programme (FAST) – summary outcome data on 44 FAST parents is in Appendix 4. 121 programmes as the data do not support such an analysis. However, these costs can be compared with recent studies. Scott et al (2001) report a cost of £571 per parent. The Scott, O’Connor et al. (2010) study already mentioned reports costs per parent of £1343, although this is reduced by additional parents over and above those in the trial that were included, increasing the 89 to 131 parents. In an earlier study, Scott, Sylva et al. (2010), also implementing Incredible Years and a reading programme, report costs per parent at £2,380 per child. Furthermore, this latter study also reports cost effectiveness as £4,500 per standard deviation (SD) improvement in children’s antisocial behaviour. This compares with our estimate for conduct problems of between £3,795 and £4,280 per SD improvement for PEIP as a whole. In summary, our evaluation has provided support for the use of these four parenting programmes over the short term when implemented in community settings. 6.3 Are the positive effects sustained after the post-intervention period? A number of efficacy trials have demonstrated that improvements from effective programmes are maintained 6-12 months later, although there may be a relatively small reduction of the effect. Our study also found this positive result for the four PEIP programmes. At one year follow up there were no significant differences from the post-course scores for parenting laxness, parenting over-reactivity or for the two child measures: conduct problems and SDQ total difficulties. Parental mental well-being was the only measure to show a significant reduction at follow up but, even so, the level remained significantly higher than when the parent started the parenting programme. In summary, our findings indicate that the improvements made over the period of the parenting programme were essentially maintained one year later. This, also, is a very positive finding as it indicates that the resources allocated to parenting programmes will have a positive impact, on both improving parenting and reducing children’s behavioural difficulties, over time. 6.4 What were the characteristics of successful implementation of the PEIP? We have rephrased this third research question to reflect a broader issue. As discussed above, the focus of the study was on effectiveness of the programme when rolled out on a large scale across 150 LAs. Our study of 47 LAs (supported by the evidence from the 18 LAs in the earlier Pathfinder) provides evidence of those factors that support successful 122 implementation. This is an important issue to explore as we found that LAs varied greatly in the number of groups run and parents supported. Our parent level data do not represent all parents supported; we know from interviews with LAs that various administrative errors occurred such that data from some groups were not submitted. Nevertheless, a variation from a total of 754 parents in one LA to just 2, 16, 27, 28 and 29 in the five LAs with lowest returns suggests a major variation in efficiency, and hence cost effectiveness 30 . This is supported by our analysis of costs variation both between LAs and between different areas of expenditure, including project management compared with delivery of training. We explored a range of factors, informed by our Pathfinder study, to try to identify those that were important in optimising delivery of the PEIP. Some were specific to a particular programme. For example, the length of programmes varied; SFP 10-14 and FAST were school-based and therefore required engagement and cooperation from schools together with the need to organise for children as well as parents. In the first year of PEIP a major factor was the system for training facilitators, (Lindsay, et al., 2009). Wave 2 LAs often had to wait for their facilitators to be trained, leading to a general delay across LAs before groups began. In Section 4.2 we explored the LA infrastructure features that appeared to be particularly important by the end of the PEIP by which time LAs had built up substantial experience. We identify five main factors. Firstly, strategic leadership and operational coordination were fundamental to an efficient implementation (Section 4.2.2). Successful practice could take different forms suggesting that there are important elements such as strong leadership, effective management skills and good communication, but variations could be made successfully to meet local needs. Secondly, effectiveness was characterised by the meshing of the PEIP with other parent support initiatives in the local context (Section 4.2.3). The LA’s parenting or Think Family strategy was an important organising framework. It was also essential for inter-departmental and inter-agency cooperation to work well. Thirdly, there were several different models for delivering PEIP that were successful (Section 4.2.4). These included core team, multi-agency and commissioning out of LA, as well as hybrid models. Each had its own strengths. For example, the successful core team model 30 This is partly, but not wholly, related to whether the LA was Wave 2 or 3. 123 had a team of skilled, experienced professionals with efficient operating systems. The multiagency network model brought the benefits of a range of professionals. Fourthly, recruitment, retention, support and supervision of facilitators were fundamental to delivery of the parenting programmes themselves. An early tension existed between guidance from the National Academy of Parenting Practitioners, who stressed the need for graduate helping professionals to be trained first to lead the delivery of programmes, and LAs that stressed personal qualities that would engage parents. Our study of 1277 facilitators has shown that they were a very diverse group. Of particular interest is the – perhaps surprising – finding that effectiveness in programme outcome was not related to the qualifications of the facilitators. This suggests that a diverse workforce may be used to deliver these programmes. This supports the very positive views held by LAs about their facilitators. However, supervision and support are also important, as shown particularly in our Pathfinder report (Lindsay et al., 2008). Fifthly, the ways in which recruitment, engagement and retention of parents was carried out was associated with successful implementation (Section 4.2.6). A wide range of recruitment strategies was used (see Box 4.13) to meet local needs, for example where there were minority ethnic communities. Time spent in initial engagement was important, which also benefitted from effective collaboration between the facilitators and referring professionals where this occurred. Retention was often related to competing demands on parents and a dropout rate of about a quarter is comparable to rates found in other studies. Keeping in touch between sessions and other strategies were helpful. It is also important not to see the parenting group necessarily as the end point: some parents need continued support, so follow up could be very useful (Box 4.13). In summary, there were many factors important in optimising the PEIP implementation, associated with both the individual programme and LA organisation. An important finding is the diversity: there was no single way that worked best but, rather, a number of beneficial approaches that could be adopted and modified to meet local needs. 6.5 Conclusions The Parenting Early Intervention Programme has demonstrated that evidence-based parenting programmes can be successfully rolled out on a national scale. Parents’ selfreports of their mental well-being and of their parenting laxness and over-reactivity showed statistically highly significant improvements. There were also highly significant improvements 124 in their children’s behaviour. These results were found for all four of the programmes for which we had sufficient data: Triple P, Incredible Years, Strengthening Families Strengthening Communities, and the Strengthening Families Programme 10-14. These results are comparable to recent small scale studies in terms of impact and cost effectiveness. Previous research had established the efficacy of parenting programmes. The National Institute of Health and Clinical Excellence review supported their use as a cost effectiveness approach to reduce children’s behavioural difficulties, with compared benefits to society. These results support the potential benefits of investment in evidence-based parenting programmes on a large scale. They also indicate that the means by which they are implemented by local authorities can have an important impact on efficiency and costeffectiveness. A number of models for implementation can be effective which indicates the scope for local authorities to choose one which fits local circumstances. However, it is important also to monitor the implementation and to evaluate outcomes for improvements in parenting skills and child behaviour and the costs of implementation, in order to ensure optimal benefits for the community and value for money. 6.6 Recommendations • Local authorities should make parenting programmes available as part of their prevention and intervention strategies to prevent the development or reduce the impact of behavioural difficulties in children. • Provision of parenting programmes should be directed mainly at those in greatest need; however, there are also benefits in recruiting a broader spectrum of parents in order to optimise group dynamics and achieve better outcomes. • Differences in outcomes between programmes were small, therefore the choice of evidence-based programmes for local use should be made in alignment with: local needs and priorities, how efficiently they use existing trained workforce, experience of delivery, and development of the local offer to parents. • Effective implementation by a local authority requires strong leadership, effective day to day management and organisation, as well as a clear parenting policy. • Several organisational and delivery models work well; the key is to match the model to local circumstances. 125 • A diverse workforce, including parents and non-graduates, can deliver parenting programmes effectively when provided with appropriate training, support and supervision. • Effective selection of facilitators should be based on their capacity to deliver programmes, and the skills and personal qualities that enable them to engage with parents. • Local authorities should ensure that the programmes are quality assured and maintain fidelity to their evidence-based models of implementation as set out in the guidance 31 . • The cost of delivering parenting programmes reduced with time, as set up costs e.g. infrastructure and training facilitators, are front loaded. Future costs should therefore be lower on average than those reported here. 31 http://webarchive.nationalarchives.gov.uk/20100202100514/dcsf.gov.uk/everychildmatters/strategy/ parents/id91askclient/localauthority/fundingforparents/ 126 REFERENCES Allen, G. (2011). Early intervention: The next steps, London: Cabinet Office Department for Education (2010). Children with special education needs 2010: An analysis Barlow, J., Coren, E., Stewart-Brown, S. (2002). ‘Meta-analysis of the effectiveness of parenting programmes in improving maternal psychosocial health’. British Journal of General Practice, 52, 223-233. Barlow, J., & Stewart-Brown, S., (2000). ‘Behaviour problems and group based parenting education programmes’. Developmental and Behavioural Paediatrics, 21, 356-370. Edwards, R.T., Ó Céilleachair, A., Bywater, T., Hughes, D.A., Hutchings, J. (2007). ‘Parenting programme for parents of children at risk of developing conduct disorder: cost effectiveness analysis’. BMJ Research, doi: 10.1136/bmj.39126.699421.55. Goodman, R. (1997). The Strengths and Difficulties Questionnaire: a research note. Journal of child Psychology and Psychologists, 38, 581-586.See also the web access http://www.sdqinfo.com/b1.html Hutchings, J., Bywater, T., Daley, D., Gardner, F., Whitaker. C., Jones, K., Eames, C., Edwards, R.T. (2007). Parenting Intervention in Sure Start services for children at risk of developing conduct disorder: pragmatic randomised controlled trial. BMJ, doi:10.1136/bmj.39126.620799.55 *published 9 March 2007) Irvine, A.B., Biglan, A., Smolkowski, K., & Ary, D.V. (1999). The value of the Parenting Scale for measuring the discipline practices of parents of middle school children. Behavioural Research and Therapy, 37, 127-142 Lindsay, G., Band, S., Cullen, M.A., & Cullen, S. (2007a). Evaluation of the parenting early intervention pathfinder: Additional study of the involvement of extended schools DCSF-RW035. (50pp). London: DCSF. http://www.dfes.gov.uk/research/data/uploadfiles/DCSF-RW036.pdf Lindsay, G., Band, S., Cullen, M.A., & Cullen, S. (2007b). Evaluation of the parenting early intervention pathfinder 2nd Interim report. DCSF-RW035. London: DCSF. http://www.dfes.gov.uk/research/data/uploadfiles/DCSF-RW035.pdf Lindsay, G. Cullen, S., Band, S., Cullen, M.A., & Strand. S. (2010a). Evaluation of the Parenting Early Intervention Programme: 1st Interim Report. London: DCSF. http://www.dcsf.gov.uk/research/data/uploadfiles/RR193.pdf Lindsay, G. Strand, S., Cullen, M.A.., Band, S., & Cullen, S. (2010b). Parenting Early Intervention Programme 2nd Interim Report. DFE-RR047. London: DfE. (35pp) http://publications.education.gov.uk/eOrderingDownload/DFE-RR047.pdf 127 Lindsay, G., Davis, H., Strand, S., Band, S., Cullen, M.A., Cullen, S., Hasluck, C., Evans, R., & Stewart-Brown, S. (2008). Parenting early intervention pathfinder evaluation DCSF-RW054. http://www.dfes.gov.uk/research/data/uploadfiles/DCSF-RW054.pdf Meltzer, H., Gatward, R., Goodman R. & Ford, T. (2000). Mental Health of Children and Adolescents in Great Britain. London: The Stationery Office. Moran, P., Ghate, D., & van der Werwe, A. (2004). What works I parenting support? A review of the international literature. Research report 574. London: Department for Education and Skills. National Institute for Health and Clinical Excellence (NICE) (2005). Parent-training/education programmes in the management of children with conduct disorders. London, NICE. Scott, S., O’Connor, T.G., Futh, A., Matias, C., Price, J., & Doolan, M. (2010a). ‘Impact of a parenting program in a high-risk, multi-ethnic community: the PALS trial’. Journal of Child Psychology and Psychiatry, 51 (12), 1331-1341. Scott, S., Spender, Q., Doolan, M., Jacobs, B., & Aspland, H. (2001). Multicentre controlled trial of parenting groups for childhood antisocial behaviour in clinical practice. British Medical Journal, 323, 28 July, 1-7. Scott, S., Sylva, K., Doolan, M., Price, J., Jacobs, B., Crook, C, & Landau (2010b). ‘Randomised controlled trial of parent groups for child antisocial behaviour targeting multiple risk factors: the SPOKES project’. Journal of Child Psychology and Psychiatry, 51 (1), 48-57. Tennant, R., Hiller, L., Fishwick, R., Platt, S., Joseph, S., Weich, S., Parkinson, J., Secker, J. & Stewart-Brown, S. (2007). The WEMWBS: Development and UK validation. Health and Quality of life Outcomes, 5, (63). Tennant, R., Fishwick, R., Platt, S., Joseph, S., & Stewart-Brown, S. (2006). Monitoring positive mental health in Scotland: validating the Affectometer 2 scale and developing the Warwick Edinburgh Mental Well-being Scale for the UK. Edinburgh: NHS Health Scotland United Nations Office on Drugs and Crime (2010). Compilation of evidence-based family skills training programmes. http://www.unodc.org/docs/youthnet/Compilation/1050018_Ebook.pdf . New York: United Nations Office on Drugs and Crime. Van Breuken G, J. P. (2006). ANCOVA versus change from baseline had more power in randomised studies and more bias in nonrandomised studies. Journal of Clinical Epidemiology, 59, 920-925. Weisz, J. (2004). Psychotherapy for children and adolescents: evidence-based treatments and case examples. Cambridge: Cambridge University Press. 128 APPENDIX 1 METHODOLOGY 1. Design The aim of the evaluation was to examine the impact of the PEIP, its costs and cost effectiveness, and the factors that enhanced (or inhibited) its successful implementation. This was a large-scale evaluation of a national roll out of parenting programmes to examine their effectiveness in ‘real life’ community settings. It was not an efficacy trial. There were no control groups as would be expected in a trial, nor, consequently, randomization of parents to the programme or control groups. Rather, the design included a comparison between the programmes. Choice of programme(s) was made locally by each LA, as was recruitment and acceptance onto a programme. A cost effectiveness study examined the relationship between the costs of implementing each programme with impact on the parent and child behaviour outcomes. The evaluation comprised a combined methods design, with quantitative and qualitative methods to address the different research issues. 2. Samples 2.1 Local authorities The LA sample comprised all 23 Wave 2 and 24 Wave 3 LAs, the latter selected to ensure coverage of the five programmes initially specified by the DCSF, geographic location, urban/rural and the main demographic variable of social disadvantage and ethnicity. 2.2 Parents There were two parent samples: Parenting groups: All parents from whom we received the questionnaires. Interviewees: A sample of parents for interview were identified by 24 LAs (12 Wave 2, 12 Wave 3) selected to focus mainly on FAST and Strengthening Families Programme 10-14. We asked the facilitators interviewed to invite two parents who had completed their group to participate in a telephone or face to face interview, as the parents preferred. Parents’ willingness was the only criterion. A total of 75 interviews were held, including eight with fathers. 2.3 Local authority staff Questionnaire data were collected from 1277 parenting group facilitators. The following interviews were held with LA staff: 129 • 73 Strategic leads – a senior LA officer with responsibility that included the PEIP and the parenting strategy. • 92 Operational leads – staff with day to day management responsibilities for the PEIP. Most LAs had one but some had two or more. Titles varied, including ‘coordinators’ and ‘project manager’ • 13 Combined role lead interviews • 77 Facilitators • 83 other parenting support professionals – for example, Parenting Experts, Parent Support Advisor leads. • 16 School representatives – for example, head teachers/senior staff Interviews were held throughout all three years of the study (Table A.1) 2.4 Cost effectiveness Fifteen of the 43 LAs that had provided data from parents who completed pre- and postparent groups provided data for a cost effectiveness study. 3. The Programmes Initially (2008) five programmes were selected by the DCSF on the advice of the National Academy of Parenting Practitioners that they meet the criteria for an evidence base: Triple P, Incredible Years, Strengthening Families, Strengthening Communities, FAST and the Strengthening Families Programme 10-14. Information about each of the programmes is presented in Appendix 2. Each programme provided training for group facilitators, according to their normal specification. The importance of fidelity is stressed by all the programmes as a means to ensure consistency of implementation according to the materials and methods for which there is an evidence base, and to quality assure the intervention experienced by participants. 4. The Measures 4.1 Parent Questionnaires Pre-course and post-course booklets were prepared comprising three measures: the Warwick-Edinburgh Mental Well-being Scale, the Parenting Scale, and the Strengths and Difficulties Questionnaire. The Parenting Scale (Irvine et al., 1999) This is a 13 item 7-point scale which examines two dimensions of parenting, Laxness and Over-reactivity, each comprising six items (range for each scale 6-42). For example, a parent responds to this laxness item ‘If my child gets 130 upset when I say “No”’ by choosing on a 7 point scale from ‘I back down and give in to my child’ (1) to ‘I stick to what I said’ (7). The 13th item on monitoring their child’s activities contributes only to the total score Warwick-Edinburgh Mental Well-Being Scale (WEMWBS) (Tennant et al, 2006, Tennant et al, 2007). This is a 14 item 5-point scale which assesses mental well-being (range 14-70). It includes items such as ‘I’ve been feeling useful’, ‘I’ve been feeling good about myself’. This scale was selected as it is worded positively: its focus is positive (well-being) rather than illness-oriented (e.g. depression). Strengths and Difficulties Questionnaire (SDQ) (Goodman, 1997). This is a 25 item measure of the parent’s views of the behaviour of the target child. Each item has a 3-point scale (Not true, Somewhat true and Certainly true, scored 0-2). It comprises four scales, each of five items that assess levels of problems: Emotional symptoms, Conduct problems, Hyperactivity, and Peer problems (range for each scale 0-10). These can be summed to produce a Total difficulties score (range 0-40). An example of the Conduct problems scale is ‘often fights with other children or bullies them’. In addition, the SDQ Impact scale comprises five items concerning the impact of the child’s behaviour. Demographic data were collected from parents at pre-course and a questionnaire ‘How was your group? 32 comprising 11 items, each scored on a 4-point scale, was completed at postgroup (range 11-44). The one-year follow up booklet comprised the three main questionnaires only. 4.2 Programme costs A questionnaire was designed to capture the programme costs and provide the basis for an analysis of cost effectiveness. Fifteen of the 43 LAs for which we have parent data responded. 32 Developed from the Pathfinder (Wave 1) study 131 4.3 Facilitators A facilitator questionnaire was designed to collect key data on demographic details on the age, gender, ethnic group, educational qualifications and prior parenting programme experience of all facilitators. The forms were sent to LAs who were asked to have them completed by facilitators either following their training or when they first delivered a PEIP recognised programme. Facilitators were only required to complete the form once. 4.4 Interviews Semi-structured interview proformas were developed for the different participants at different stages of the evaluation. Each comprised a short series of main questions with probes. This allowed a balance between a) a conversational, more open ended format to encourage free flowing responses that drew upon each interviewee’s own interests and knowledge; and b) the need to gain comparable information to allow comparisons between participants in each sample. 5. Procedure 5.1 Parent questionnaires The pre-course booklets of questionnaires were distributed to each LA, pre-designated with each group identifier. Guidance to facilitators was provided to enable them to administer the questionnaires. Booklets were collected and returned to CEDAR. Post-course booklets of questionnaires pre-populated with each parent’s name were then dispatched. Once completed, these were returned to CEDAR together with a monitoring form completed by the facilitator to specify reasons for non-completion of any forms. One-year follow-up questionnaires were sent by post, with a reply paid envelope and letter of invitation to all parents who had provided post-course booklet of questionnaires. 5.2 Interviews The majority of interviews were by telephone and all were recorded with permission. The majority were transcribed and all were analysed thematically. Interviews were coded and collated against pre determined themes (deduction analysis) and emergent themes (inductive analysis). The development of the coding frame was an iterative process agreed among the researchers. A full list of interviews is presented in Table A.1. 5.3 Facilitators The facilitator form was sent with the pre-course booklets. Each facilitator was asked to complete only one form and return direct to CEDAR. 132 Table A.1 Semi-structured interviews conducted during the evaluation Wave Date Details Number Wave 1 LAs (N = 18) Spring 2009 Operational leads 19 Winter 2010- 11 Strategic leads 7 Operational leads 13 Combined role leads 1 Strategic leads 23 Operational leads 12 Other professionals 8 NAPP* representative 1 Operational leads 21 Parenting Experts 17 Parent Support Advisor leads 17 NAPP* representative 1 Facilitators 23 January Parents** 39 2010 School representatives 9 Strategic leads 12 Operational leads 16 Combined role leads 3 Facilitators 17 Winter 2009 Strategic leads 24 Summer 2010 Operational leads 6 Parenting Experts 20 Parent Support Advisor leads 19 Facilitators 22 January Parents** 36 2011 School representatives 7 Strategic leads 7 Operational leads 5 Combined role leads 9 Facilitators 15 Wave 2 LAs (N = 23) Winter 2008 Summer 2009 In 12 Wave 2 case study LAs Summer 2009 to only Winter 2010-11 Wave 3 Sample LAs (N = 24) In 12 Wave 3 case study LAs Summer 2010 to only Winter 2010-11 Note: * NAPP = National Academy of Parenting Practitioners; ** In both Wave 2 and 3, four fathers were interviewed (8 in total). 133 5.4 Costs The questionnaire for LAs to record the costs of implementing PEIP was piloted in five LAs, each of which focused on one programme. Each was visited by the project director (GL) and. where possible, the field researcher for that LA (SB, MAC, or SC). These meetings, typically about an hour long, were held with one or more LA staff in order to clarify and refine the questionnaire. Initially it was considered that the five pilots might be used as exemplars as it was apparent that accounting methods and practices for using PEIP funding varied between LAs, rendering a questionnaire problematic. However, the pilot enabled a questionnaire to be devised that could allow different LAs to report. All 47 LAs in our sample were sent the questionnaire: 15 completed forms were returned, each of which could be analysed. 6. Follow Up A follow-up booklet comprised three of the original scales; parent mental well-being, parenting laxness and over-reactivity and the child Strength and Difficulties (SDQ) questionnaire. This was to keep the booklet brief (just four sides of A4) so as not to discourage responding. In the twelfth month after each group ended, all parents who had completed post-course questionnaires were sent the follow-up booklet with a pre-paid return envelope. If no response was received from the parent within a month, then a second reminder letter was sent. We also contacted the LAs concerned to request their co-operation in raising the response rate. For example one LA arranged coffee mornings with the intention supporting parents, particularly those with English as an Additional Language, in completing the follow-up booklet. In others the original facilitators called the parents for a chat and encouraged them to fill-in the booklets. 134 Table A.2 Number of parents per programme returning a pre-course questionnaire booklet within each LA. Parents Plus STOP Parent Power Triple P SFSC Incredible Years Total SFP 10-14 FAST LA LA1 LA2 LA3 LA4 LA5 LA6 LA7 LA8 LA9 LA10 LA11 LA12 LA13 LA14 LA15 LA16 LA17 LA18 LA19 LA20 LA21 LA22 LA23 LA24 LA25 LA26 LA27 LA28 LA29 LA30 LA31 LA32 LA33 LA34 LA35 LA36 LA37 LA38 LA39 LA40 LA41 LA42 LA43 754 513 358 8 49 9 47 16 174 282 189 202 209 4 97 5 97 72 123 73 37 63 6 11 178 161 19 94 85 71 148 143 10 10 39 3 111 23 76 4 59 3 9 113 54 102 102 101 101 100 4 27 26 69 67 7 9 47 36 11 62 51 3 31 39 16 65 26 42 17 16 7 30 29 5 27 23 16 16 2 104 969 782 868 3171 135 85 102 62 Total 754 513 358 290 238 227 221 209 194 189 172 164 164 157 156 148 143 131 128 124 120 113 102 101 101 100 98 89 77 69 67 66 65 65 42 33 30 29 28 27 23 16 2 6143 APPENDIX 2: The five PEIP programmes FAST Incredible Years Strengthening Families Programme 10-14 Strengthening Families, Strengthening Communities Triple P 136 FAST (Families and Schools Together) Originator: Lynn McDonald (Professor of Social Work Research, Middlesex University, London) Country of Origin: USA; Madison, Wisconsin. First published: 1988. Implemented: Across the USA, in Canada, Australia, Germany, Netherlands, Russia, Philippines, Kazakhstan, Tajikistan, Kyrgyzstan and UK (2500 sites in 14 countries). Child Age Range: 3-18 years General Aims/Goals: • • • • • • • • To increase protective factors for the child and especially the parent-child relationship and parent involvement at school. To improve parents’ ability to listen and communicate with their children, to give clear commands and follow-through, and to use support from others. To increase children’s well-being across domains at home, at school and in the community. To increase the family social network of support, including develop social capital with other local parents, increasing parent involvement with school staff, increasing parent leadership over time in the community, and knowing community agencies for specialist services and appropriate referrals. To increase academic achievement of children, increase attention span and social skills, and reduce stress. To reduce bullying and aggression in school, and prevent child mental health problems, juvenile delinquency, violence, addiction and school dropout. To enhance family functioning and reduce family conflict. To maintain gains of 8 weekly sessions. Theoretical Background: • • • • • • • • Family systems theory. Family stress theory. Social ecological theory of child development. Attachment theory. Social Learning theory. Small group theory. Social capital. Community organizing and adult education. 137 Programme in action: • • • • • • • Families involved: universal recruitment of all children at a grade level within a school serving a targeted neighbourhood with multiple risks such as high poverty or high mobility or socially marginalised populations. Known for engaging low income parents. Recruitment: includes repeated home visits in non-traditional hours preferably by Fast parent graduates to assure participation of socially excluded families. Successful outreach enhanced by: o Training which includes discussion of social inclusion, for example, and the skills of listening respectfully; o Providing transport, infant care, meals and a fixed lottery. Core programme: o 8 week programme, school based, evening groups for 10 families per hub with schools running 4-6 hubs at a time in order to include whole year groups. o Followed by 2 years of monthly booster sessions led by parent graduates with school support to maintain gains over time of networks of parents. Sessions are 2.5 hours long: o Family tables (60 minutes) where parents are supported by the trained coaches as they lead a non-conflicted family meal, do family introductions, participate in rhythm and music, and play family communication games (a drawing and talking game; feelings identification game for 3-9 year olds; a family board game for 8-13 year olds) o Peer activity with generations separated into rooms for groups for parents and groups children (60 minutes); o Parent-child activity (15 minutes): one-to-one activity where team coach parents to provide “responsive” play (for ages 3-9) or to talk one on one about a topic selected by the youth group (for ages 10-15) o Closing tradition (15 minutes) to gather in a large circle to celebrate family which won lottery, thank family which prepared the meal, birthdays and achievements recognized, personal announcements shared, general school and community announcements and silence. Retention rates (of 80% on average-only 20% drop out with predominantly lowincome, socially marginalized parents) are achieved by: o FAST teams being: • Culturally representative of the families being served; • Including service users/carer/parents on teams; • Multi-agency teams with professionals from health, education and social care; o Outreach including home visits to engage socially marginalized parents; o Universal recruitment and access: whole families are invited; no targeting children; high need communities are a priority; o The fixed lottery (each family wins once, but random reinforcement schedule is used); universal reciprocal behaviour helps each family who wins to contribute and give back the next week by getting money from the programme to use to shop and prepare a meal for the whole group. Process of the multi-family group intervention: • Based on experiential learning through structured interactive processes with team members coaching parents and supporting positive parenting; imbedded behavioural rehearsals of positive parenting within fun parent led family activities. Emphasis on building relationships across multiple levels. 138 • • • • Prevention and early intervention through building protective factors largely involving positive, non-conflicted time within multiple relationships. These include: o Parent-Child bonds: Through 15 minutes per day of uninterrupted, responsive parenting in play with one child, which is coached every week in FAST sessions and then implemented each day at home, with emphasis upon responsive listening, and parents being non-directive and nonjudgemental in child initiated free play. o Parenting: Through 60 minutes weekly family table activities, based upon specific activities (e.g. parents rehearse giving imbedded compliance requests to child to serve food, to co-construct a family flag, to draw and explain pictures, to take turns, to play act feelings) where parents are encouraged to be firmly in charge yet loving. o Parental relationship: each week 15 minutes in which partners listen to each other’s concerns without given advice; (daily hassle review) o Peer social network: 45 minutes in which the parents meet in a group to reduce isolation and help each other to help their children succeed at home and school; no didactic presentation; parents determine agenda. o Empowering parents: firstly to take charge of children with all activities structured to enable parents to practice controlling children without coercion. Small groups of interdependent parents with children of the same age at the same school become close; 86% make a new friend they see for years at FAST. Local parents are on each FAST team in partnership with professionals. Over time, parent graduates become partners with the FAST team, and co-facilitate monthly meetings. Over time, the FAST teams at a school leading multi-family groups can be 75% parent graduates welcoming new school years into the school community; parents begin to work with school, volunteer and advocate as part of child’s education and begin to lead within the community with other parents. o Involvement in school/community of children and family: repeated low level and personal interactions with all school personnel (e.g. teachers and counsellors) during FAST sessions strengthens relationships leading to more appropriate use of resources. Multi-family groups for universal populations include probably 15% who might need more and self-refer or be referred for specialist services. Protective factors assumed to reduce risk factors (e.g. behaviour problems) correlated with delinquency, substance abuse, violence and school failure. Offers structured involvement in repeated, relationship-building interactions with primary caretaking parent, other family members, other families, peers and school representatives and community representatives. To provide a “social safety net” of long-term relationships as protective factors in troubled times; social support and social capital are systematically built up within communities. Social capital is correlated with improved academic success, reduced crime, drugs, increased health and longevity. Training the team to run the multi-family group programme: • • Fast Team Replication Training is team based and not individual. Teams are made up of professionals and parents from the schools, whose children are in a later school year. The proportion of parents (and youth for secondary school FAST) varies from 25% to 75%. The professionals should be from multiple agencies (schools, health social services, and community agencies) to facilitate referrals to specialist services. 139 • • • • • • • • • • Training varied professionals together in groups of 15 with 5 local school parents enables the building of 4 teams which can serve four multi-family groups in one extended school programme with 40-50 families participating over 8 weeks. A certified FAST trainer can prepare 4 teams to deliver the whole package. Training involves 4 full days over 4 months. This includes: 2 initial days; telephone consultation during implementation; 3 site observational visits of the 8 weekly groups to monitor program integrity and advise on local adaptations; and a final training day to review evaluative data reports, hear from service user/parents about their experiences, and plan the monthly sessions. Value-based on 10 beliefs (e.g. every parent loves his/her child). Training includes discussing recruitment of families, three hours of planning time, review of the research and values, walking through each program activity, and role plays largely concerned to enable workers to respect, empower and support parents and not undermine them. 5 scenarios are role played twice (wrongly and rightly) with discussion of each. These include: orientation to children; asking for support; celebrating children’s success; helping when child is hurt; disciplining the child. Training and supervision also involves extensive discussion to adapt programme to local needs, situation and challenges. Training involves reading through a summary of the research underpinning each activity with a CD provided. Middlesex University, Department of Mental Health, Social Work and Interdisciplinary Studies is offering credits for participating in this FAST Training project (requires an assessed 3000 work paper) Expects teams implementing the programme to be ethnically and culturally similar to those children and families being served; representation on the team increases retention rates of socially excluded groups, including fathers. Evaluation of Parenting Programme: Site certification includes: • • • • • • • Graduating 5 families from a hub. Full participation of the team at every level: training, doing, reviewing. Demonstrating acceptable levels of program integrity in implementation. Public interviews with graduated parents to give feedback on experiences. Plus submitting before and after outcome data on 6 standardised instruments for analyses and review. These include: o SDQ Strengths and Difficulties Questionnaire by Parent report on child behaviour at home; o Strengths and Difficulties Questionnaire by Teacher report on child behaviour in the classroom; o Moos Family Environment Scale FES o Family Social Isolation (from Abidin’s Parenting Stress Index); o Epstein’s parent involvement in school scale; o Consumer feedback and satisfaction. Each new FAST site must be evaluated with McDonald Outcome FAST Evaluation Package; as well as FAST trainers making 3 onsite visits per multifamily group programme to coach the quality of implementation. 140 Costs: • • • Team training for 20 participants (including 10-15 professionals from health education and social care and 5-10 parents from the local school) to serve 40 families at the school; supervision (including 3 site visits to monitor programme integrity and support local adaptations by the team); on-going technical assistance; training materials; evaluation with parent panel and pre-post questionnaires; data analyses and evaluation report costs £18,000 (plus travel and accommodation). Meals and programme supplies for 40 families at 8 weekly sessions for programme implementation and 22 parent-led monthly groups of 100 costs the school £30005000. Becoming a FAST trainer costs £2500 (plus travel and accommodation). Training of Certified FAST Trainers: • Takes as little as 12 months to become certified as a FAST trainer and it involves two separate primary schools. This increases local sustainability of FAST and it includes: o Participating in training, supervised implementation and evaluation as a FAST team member over 4 months in one school. o Attending a one week training of trainer to become a trainer intern. o Training a school based team under supervision of a certified FAST supervisor, including 3 site visits during implementation to monitor program integrity and encourage local adaptation and to evaluate results over 4 months in a second school. o Local trainers require one supervised training of a new team. o National trainers require two supervised trainings of two new teams. Research: Considerable research has been conducted. For example: • • • • • • Billingham (1993) experimental study showed FAST children improved significantly more than controls. In Wisconsin and California state-wide implementations (Billingham 1993; McDonald, 1993) there were significant changes on Quay-Peterson Revised Behaviour Problem Checklist for teachers and parents and other measures in evaluation package. Sayger and McDonald (1998) found significant pre-post changes in child behaviour and the family environment maintained at 6 months. Follow-up with comparisons showed improved child functioning 2-4 years later, improved third grade reading scores and family cohesiveness and increased social involvement. Unanticipated long-term parent outcomes: 85% increased parent involvement in schools; 44% went for further education; 36% found employment; 18% self referred for counselling; 10% FAST parent leaders. Abt Associates (2001) conducted a randomised trial of 400 children with intention to treat analysis. At one year there was a significant improvement in parent rated child social skills and lower externalising problems on CBCL (effect size = 0.26). Parents also volunteered more and were more often involved as parent leaders. McDonald et al (2006) compared FAST to FAME, a parent programme based upon educational booklets and behaviour modification with active follow up. The study was randomised with intention to treat. At 2 years FAST children were rated as more competent academically and Latino FAST children as more socially skilled and less aggressive, although for total group there were no significant differences. Kratochwill et al (2004) randomised 100 Native American children to FAST or control. At 1 year, teachers rated FAST children as more competent academically and parents rated them as less withdrawn. 141 • • Kratochwill et al (2008) conducted a randomised trial which resulted in relatively few differences. 67 matched pairs of families were randomly assigned to control and FAST. FAST families improved more in adaptability and FAST children had more attention problems after 8 week programme. At 9-12 months again FAST families more adaptable (Es =0.47), and children showed less externalising behaviours (Es = 0.42). In terms of referral for special education, fewer FAST children were referred at a saving of $160,000. One of 23 programmes recommended by the United Nations Office on Drugs and Crime (www.unodc.org/docs/youthnet/compilations/10-50018 Ebook.pdf). 142 Incredible Years Programs Originator: Carolyn Webster-Stratton Country of Origin: USA; Seattle, Washington. First published: 1984. Implemented: Widely within the USA, UK, Canada, Europe, Ireland, Norway, Denmark, Sweden, New Zealand and Australia. Currently there are 9 accredited trainers and 50 mentors in Canada, Denmark, New Zealand, Norway, UK and US. Child Age Range: 8-13 years (for the PEIP programme). General Aims/Goals: • • • Treatment of child behaviour problems and prevention of conduct problems, delinquency, violence and drug abuse. Enabling child learning generally, including academic, social and emotional adaptation; Through improved parent-child interaction, building positive parent-child relationships, improved parental functioning, less harsh and more nurturing parenting, and increased social support. Theoretical Background: • Cognitive social learning theory and particularly Patterson’s coercion hypothesis of negative reinforcement developing and maintaining deviant behaviour and Bandura’s notions of modelling and self-efficacy. Programme in action: • • • • • • • The Incredible Years is a series of three interlocking training programmes for parents, children and teachers and spans the age range of 0-13 years. The parent training was specifically adapted for PEIP by combining elements of the School Age BASIC Program (12-16 sessions) and the ADVANCE Parent Program (9 sessions) making an 18-22 session programme in total. Each session is 2 to 2.5 hours long and conducted at weekly intervals. This gives a total time involvement of somewhere between 36 and 55 hours. The course involves 2 group leaders and 10-14 participants. Typically food, child care and transportation are provided for each session. Training set in a problem solving format with identified goals, assessments of barriers, and potential solutions. Methods include: o Group discussions, sharing and problem solving; o Goal setting (long term and short term); 143 Values exercises regarding benefits of and barriers to strategies; Principles training; Buddy Calls (building support networks); Behavioural, Cognitive and Emotional Skills training; Group brainstorming to identify social learning principles and child behaviour management strategies; o Discussion of DVD vignettes of parents and their children; o Probing questions by group leaders to promote self-reflection; o Role play or behavioural rehearsal in small groups to practice behavioural and cognitive strategies; o Home Activities assignments & experiential learning; o Rewards for goals met; o Weekly group leader phone calls to parents Materials include: o The Incredible Years book or CD for each parent; o 14 DVDs to show vignettes during the course; o Handouts and home activity assignments; o Group buzz exercises; o Refrigerator notes. Content includes: o Respect for and understanding of children and their developmental milestones; modelling social skills; following child’s lead; balancing power; descriptive commenting; academic, social, emotion and persistence coaching; differential attention; ignoring; modelling principle; and having fun. o Having developmentally appropriate expectations for child depending on child’s temperament and abilities. o Positive parenting, controlling own emotions and improving relationships, effective communication skill, family problem solving, enhancing children’s learning, anger management, and managing conflict. o Establishing rules, predictable routines and children’s responsibilities, as well as ongoing monitoring and supporting children’s academic achievement through coaching children’s homework and partnering with teachers. o o o o o • • Process: • • • • The training which is based on principles of video and live modelling, rehearsal, selfmanagement, and cognitive self-control. Assumes a self-learning model involving self-monitoring and weekly goal setting. Also explicitly assumes a collaborative relationship between group leaders, parents and teachers. Reduction of risk factors and increased protective factors through increasing parent, teacher and child competencies. Training to run the programme: • • • • Group leaders ideally assumed to be drawn from professionals with masters, higher degrees or diplomas in fields such as psychology, psychiatry, social work, nursing, or counselling, with knowledge of child development and social learning theory and experience of working with parents. 4 or 5 days in workshops of up to 25 participants. Group leader materials include: manual; other reading; DVD-vignettes to show to parents; sample group session DVDs for self-study; handouts; books or CD for parents. Training workshop methods for group leaders involve: 144 Participants act as parents and group leader methods are modelled, followed by discussion, analysis and reflection; o Role plays/practices as parents and group leaders; o Collaborative processes modelled implicitly with extensive discussion and practice in workshops plus consultation based on collaborative model; o DVDS of actual program sessions delivered by accredited group leaders; o Brainstorming; o Homework. Supervision includes: o Peer support meetings to review session tapes using Peer and Selfevaluation checklists; o Monthly telephone consultations with accredited coaches and mentors; o Feedback on a mid-course DVD of group sessions delivered; o Consultation workshops sharing DVDs of sessions. Accredited by: o Attending accredited group leader workshop; o Conducting a minimum of two complete group programmes; o Attend consultation workshops from an accredited mentor/trainer presenting DVDs of group sessions for peer and mentor feedback. o Receiving satisfactory mentor/trainer review of a session DVD, all required session protocols, and evaluations of the programmes by participants; o • • Evaluation: • • • • • Following session protocols; Weekly session and final course parent evaluations; Group leader checklist review of all sessions; Group leader self, peer and/or trainer evaluation forms of a DVD; Pre and post questionnaires such as: o Demographics; o Beck Depression; o Eyberg or parent and teacher SDQ; o Parenting Scale; o O’Leary-Porter Overt Hostility Scale. Costs: • • • The cost to train a group leader averages $400 (USD) (approx. £248) but depends upon whether there are existing group leaders in the LA and whether the training is done by a mentor or trainer. The initial costs of materials for running a parent course are $1805 (USD) (approx. £1121) for the BASIC programme and $2205 (USD) (approx. £1370) for the School Age BASIC plus ADVANCE programme. Other programmes are available and the prices vary. Additionally the group leaders provide books for parents (available in the UK for £16). However, the DVDs once purchased can be used repeatedly on any number of further courses. Costs to enable group leader accreditation average $627 (approx. £390) per person including supervision/consultation time. Trainers: • The programme has designated accredited mentors and trainers for specific IY programmes (baby, preschool, school age and advance). 145 • • • Mentors are permitted to train within their own agency or defined geographic district whereas, accredited IY trainers may conduct training more widely. Mentor accreditation includes: o Accreditation as group leader; o Nomination by an existing mentor or trainer; o Successful completion of at least 8 BASIC programmes with submission of feedback documentation and DVDs; o Participation in consultation led by a certified trainer; o Positively evaluated group video and review by a trainer post group leader accreditation; o Training in supervision process and workshop delivery protocol; o Positively evaluated workshop video and supervision session by trainer; o Completed competent DVD reviews for group leader certification and submitted supervisory reports; o Positive evaluations from 6 group leaders mentored; o Completed numerous positively evaluated mentor workshops; o Co-trained with other Incredible Years trainers. Trainer accreditation includes for example: o Accreditation as a mentor; o Research training, experience and knowledge-based related to evidencebased interventions and evaluation methods; o Understanding and experience regarding dissemination and implementation strategies to promote fidelity of program delivery by practitioners and organizations; o Masters or preferably doctoral level education; o Ongoing relationship with Incredible Years organization in Seattle and with developer. Research: • • • • See Webster-Stratton and Reid (2010) for the most recent review of treatment and prevention studies. Multiple randomized trials of the parenting programmes by Webster-Stratton and colleagues including several independent replications with both treatment and prevention populations have shown: o Increased parental positive affect (e.g. praise and reduced criticism and negative commands). o Increased effective parental limit-setting, replacing spanking and harsh discipline with non-violent discipline techniques and increased monitoring of children. o Reductions in parental depression and increases in parental self-confidence. o Increased positive family communication and problem-solving. o Reduced conduct problems in children’s interactions with parents and increases in positive affect and compliance. A ten-year follow-up (Webster-Stratton, Rinaldi & Reid, 2011) indicated that young people whose families had participated in an Incredible Years programme had maintained immediate post-treatment levels of adjustment and were well adjusted with rates of risky behaviour within normal limits for adolescents. Randomised trials have also been conducted in the UK (Patterson et al, 2002; Scott et al, 2001; Scott et al (2010); Hutchings et al, 2007; Gardner et al, 2006) indicating: o Benefits for externalising behaviour of children from 2-9; o Maintained up to 18 months after the programme; o Benefits for parenting and parent mental health; 146 Using health visitors, child mental health specialists, social workers, nursery nurses, teachers and people from adult education, and the child and family voluntary sector. Recruitment rates ranged from 65-78%; Attrition ranging from 11-43%. A recent pre-post evaluation of the first study with the PEIP program for children ages 9-12 years indicated positive effects (Hutchings et al, in press; Lindsay et al, 2008). One of 23 programmes recommended by the United Nations Office on Drugs and Crime (www.unodc.org/docs/youthnet/compilations/10-50018 Ebook.pdf). An evaluation by Scott et al (2010) suggested that in high-risk, multi-ethnic communities child outcomes may be affected by poor engagement and low attendance rates and selection of children with low rates of problem behaviour. o • • • • • 147 Strengthening Families Programme 10-14 (SFP10-14) Originator: Karol Kumpfer developed original Strengthening Families Programme, subsequently modified by Virginia Molgaard with Karol Kumpfer as SFP10-14. Country of Origin: USA: original programme developed at University of Utah and then SFP10-14 developed at Iowa State University. First published: Original in 1983 and SFP10-14 in about 1992. Implemented: Across USA, and in Central and South America, Sweden, UK, Spain and Poland. First introduced in the UK by Barnsley Primary Care Trust in 2001 and has been adapted for UK and Europe by Prof David Foxcroft and team at Oxford Brookes. Child Age Range: 10-14 years General Aims/Goals: • • • • To decrease alcohol and drug use during adolescence. To reduce behaviour problems. Through improved parental skills in nurturing and child management. And through improved personal and interpersonal skills in young people. Theoretical Background: • • • Biopsychosocial vulnerability model; family coping and resources buffer family stressors. Resiliency model where build up of protective skills (i.e. emotional management skills, interpersonal social skills, reflective skills, academic and job skills, ability to restore self-esteem, planning skills and problem solving ability) assumed to foster resilience. Family process model linking economic stress and adolescent adjustment. Programme in action: • • • • Can be implemented as universal or targeted. Involves seven 2 hour sessions, plus 4 optional booster sessions 6-12 months after initial programme. These are particularly recommended for targeted families. Involves up to 12 families and at least 3 facilitators (one for parents and two for the young people). Group leaders facilitate the programme and it is suggested that they each take responsibility for a third of the families on the course. 148 • • • • • • • • • • • Sessions held in schools, churches, community centres or agency facilities. Each session includes an hour of skills building groups for parents and young people separately and then an hour together in supervised family activities. Youth and parent sessions parallel each other in content and family sessions provide reinforcement and practice. Extensive use is made of DVD material with parents in that all parental sessions are based on DVDs showing scenarios of ineffective parenting and alternative approaches. These are discussed and the skills involved are practised. DVDs are included as a small part of weeks 5 and 6 of the young people’s sessions and in two of the family sessions. Groups may provide incentives of food vouchers, and may include meals during the sessions. The provision of child care, transport and a meal or snack is seen as important, particularly in relation to retention. Youth sessions: o Involve small and large group discussions, game-like learning activities, skills building practice and social bonding activities. o They are intended to improve: goal setting; appreciating parents; dealing with stress and strong emotion; communication skills; increasing responsible, prosocial behaviour; handling peer pressure. Parent sessions: o Are each based upon a videotape for the whole hour. The tape includes didactic presentations, viewing positive and negative videotaped interactions with youths, role play, group discussions, skill building and group support. o Intended to cover: nurturing youth; setting rules; monitoring compliance; appropriate discipline; encouraging good behaviour; using consequences; building bridges; protecting against substance misuse. Family sessions: o Involve parent-youth discussion, communication exercises, poster making projects, games, skills building and viewing an instructional videotape. o Intended to improve listening and communicating with respect, to empathise with each other, to identify family strengths and values, use family meetings to teach responsibility and problem solving and to plan enjoyable family activities. o 66% of session spent in family units and 33% in large multifamily group. Various methods are used to enhance the maintenance of changes, including posters put on home walls, certificates with photographs and letters parents and children write to each other in the last session. Boosters sessions involve: o For young people: making good friends, handling conflict and reinforcing skills from basic course. o For parents: handling stress, dealing with partner disagreement, reinforcing what previously learnt. Process: • • To reduce a range of risk factors (e.g. poor communication, harsh and inappropriate discipline, family conflict, poor school performance, poor social skills, peer conflict, etc.) and increase protective factors (e.g. supportive family relationships, appropriate parental expectations, increased empathy, improved communication and social skills, etc). Facilitators assist families and model skills involved. 149 Training to run the programme: • • • • • • 415 page instructor manual details the initial 7 sessions. 215 page manual details the booster sessions. All videotapes are provided. Group leaders require good presentation and facilitation skills, experience working with families/youth, flexibility, responsibility and organisational skills. Drawn from all professional groups and parents who have attended the course. There are four levels of training. o Level 1: Participants attend a three day training in groups of 21, covering the content and implementation of the programme, plus 3 hours of reading time. o Level 2: Following level one training participants can apply for level two when they have delivered a minimum of two SFP 10-14 programmes. ƒ This training and supervision supports the development of quality planning and delivery and is available for a team of up to 7 trained facilitators. It is provided for areas who wish to build on their quality and increase their capacity to deliver SFP10-14. ƒ It aims to: • Support efficiency through improved planning and preparation of programme delivery; • Build on the existing strengths of facilitators; • Improve confidence and skills; • Ensure fidelity to evidence based outcomes for families; • Aid professional development. ƒ Facilitators who meet the entry requirements receive mentoring and support, within the context of a 7 week programme, during their initial planning meeting and final debrief session. There is also observation (up to 3 sessions) carried out by a Trainer of Trainers to allow for a full assessment of practice, skills and knowledge base prior to certification. ƒ The learner is observed allocating roles and responsibilities, describing sessions to other facilitators/helpers and delivering the programme to families with fidelity and feedback is provided to the trainees ƒ Level 2 is an essential requirement for the application of Trainer of Facilitator Training (TOFT). o Level 3: Trainer of Facilitators Training qualifies participants to train SFP Group Facilitators in their locality. ƒ It aims to: • Support planning and preparation for training delivery; • Improve confidence in delivery of programme materials in a training environment; • Encourage reflective practice; • Enable practitioners to provide and receive constructive feedback from other trainers and facilitators; • Provide a platform to hold discussions about best practice. ƒ Two to four certified Group Facilitators who have met the entry requirements and have been successful at interview attend a TOF Training. ƒ This takes place within the context of a 3 day Group Facilitator training event and entails an additional intensive 8 hours of instruction. ƒ Participants observe training techniques during the regular 18 hour training event and are expected to deliver elements of the Group Facilitator training under the supervision and assessment of the TOF 150 o trainers. This allows for a full assessment of their practice, skills and knowledge base prior to certification. ƒ At the conclusion of the training, participants normally become provisionally certified SFP10-14(UK) Trainers of Facilitators. ƒ Full certification follows an observation of successful training practice during their first Facilitator Training. ƒ The Oxford Brookes University team continue to provide telephone and e-mail consultation following successful completion. ƒ As part of the TOF certification, practitioners are required to attend an annual mentoring/supervision session and face-to-face evaluation of trainings facilitated by Oxford Brookes University. Level 4 (Academic Award): This is for the academic development of practitioners who would like to use the SFP10-14(UK) training and their professional work towards an academic award. It is open to all practitioners and can be at undergraduate or post graduate level by distance learning. Evaluation of Parenting Programme: • • • • In house questionnaire included in the manual to be completed by parents and young people after the programme (i.e. SFP10-14 Parent/Caregiver Survey questionnaire (PCSQ) and the SFP10-14 Young Persons’ Survey questionnaire (YPSQ)). It is also suggested that facilitators supplement these with other measures (e.g. SDQ). Brief weekly evaluations by each parent/carer and young person are suggested to check how they found the session to aid in the discussion and monitoring of each person’s progress. Oxford Brookes offer a free data analysis service. Costs: • • • • • Level 1: £743 per person trained to run the course. Level 2: £3,460 to train up to seven people. Level 3: £2,210 per person. The costs for all three levels include trainers and their travel/accommodation and materials (manuals, posters, DVDs). Level 4: according to the university modules completed (at present £540). Trainers: • • • There are 6 certified trainers in the UK, all trained by Virginia Molgaard and with over 8 years experience of the programme. Only these trainers have an agreement with Iowa State University to train across the UK. All trainers are members of the SFP10-14(UK) steering group and meet at least twice a year. Research: • • Original Kumpfer programme has been evaluated positively in a series of studies including randomised controlled trials. The initial, shorter version modified by Molgaard (i.e. the Iowa Strengthening Families Programme) was tested through Project Family and published in a variety of papers (e.g. Redmond, Spoth, Shin & Lepper, 1999; Spoth, Redmond & Lepper, 151 • • • • • 1999; Spoth, Redmond & Shin, 1998) in which 446 randomly assigned families were followed from 6th to 12th grade (with post-test, and follow up at 18 months, 30 months and four years). Results indicated: o Improved parenting and relationship with children at most assessment points; o Reduced substance use, conduct problems, school related problems and affiliation with antisocial peers. o Intervention was associated with significant delays in the onset of substance use including alcohol, tobacco and cannabis and their frequency of use, with differences in uptake and frequency between intervention and controls becoming greater with time over more than 3.5 years. In a further paper (Spoth et al, 2008) using structural equation analyses, the parenting programme increases in parenting competencies and reduced substance related risks in the 6th grade were found to be associated with positive effects on academic performance in the 12th grade by way of positive school engagement in the 8th grade. Cochrane Collaborative Systematic Review endorsed SFP10-14 as a promising prevention of substance abuse. One of 23 programmes recommended by the United Nations Office on Drugs and Crime (www.unodc.org/docs/youthnet/compilations/10-50018 Ebook.pdf). Coombes, Allen, Marsh, & Foxcroft (2009) in a study of 50 families in Barnsley found significant predicted changes on PCSQ and YPSQ (e.g. communication, alcohol/drug use, emotional management) and on SDQ Total Difficulties score (N=16) after 7 sessions. European controlled trial ongoing and 4 year RCT in progress across Wales. 152 Strengthening Families Strengthening Communities (SCSF) Originator: Marilyn Steele with colleagues, Jerry Tello, Ronald Johnson and Marilyn Marigna. Country of Origin: USA: Los Angeles, California. In the early 1990s and now entitled Strengthening First published: Multi-ethnic Families and Communities Program. Adapted for use in the UK by the Race Equality Foundation in 1999 and entitled Strengthening Families, Strengthening Communities (SFSC). Implemented: Throughout the USA and the UK, as well as in Ireland and Australia. Child Age Range: 3-18 years. General Aims/Goals: • To promote the protective factors associated with better outcomes for children (e.g. strong and warm relationships). • To help parents to explore and develop strategies for dealing with the factors in parenting associated with poor outcomes for children (e.g. harsh and/or inconsistent discipline). To support parents to develop a better understanding of child development, use positive discipline techniques and promote children’s social skills and self-discipline. To increase parental confidence and help parents achieve positive change in family relationships. To prevent and reduce violence against self, family and community; To develop an understanding of community and increase family involvement in community • • • • Theoretical Background: • • • • • • • Physical, cognitive, social and emotional milestones of development. Attachment theory. Behavioural theories. Social learning theory. Social ecological theory of development. Social psychology. Experiential learning. Programme in action: 153 • • • • • • • Materials for parents in more than 20 community languages and formats (including Braille, large print and easy words and pictures for parents with learning disabilities) 13 x 3 hour weekly sessions (total 39 hours). Co-facilitation model of delivery Optimum group size 8-15 participants. Delivery methods include: o Facilitator modelling; o Role play; o Lectures; o Discussion; o Parent homework activities. The curriculum is based around five components: o Cultural/spiritual/ethnic and family roots; o Relationships. o Positive Discipline. o Rites of Passage. o Community involvement. Includes information on: o Child development; o Developing positive relationships; o Managing anger, conflict and stress; o Communicating effectively; o Solution building; o Enhancing children’s learning; o Family and community violence associated with substance abuse, depression and suicide. Process: • Assumed to develop parental strengths and resources, to reduce risk factors associated with poor outcomes for children and to increase protective factors. Training to run the programme: • • • • • • • • The programme can be facilitated by a range of parenting practitioners, ideally with a minimum of QCF level three or equivalent and a demonstrated level of expertise and experience in working with parents. However there are no prescribed occupations for taking up the facilitator training. Instead emphasis is placed on essential skills and knowledge including empathy, commitment to working with parents and the ability to cope with the professional and personal challenges of delivering SFSC effectively. In practice SFSC facilitators come from a range of occupations including for example: community development workers; social workers; youth workers; family support staff; teachers; and clinical psychologists. In addition, parents who have completed the programme as participants are also trained as facilitators and linked with more experienced facilitators in order to deliver the programme and become certified. All practitioners are expected to be able to demonstrate an understanding of how to support families from a diverse range of ethnic backgrounds and family circumstances. Facilitator training runs for five consecutive days of approximately seven hours and occurs in groups of up to 24 people. The training: o Outlines the concepts underlying the programme; 154 Takes an experiential approach to allow attendees to explore the impact of culture and values on their own upbringing and child-rearing attitudes; o Provides facilitators with all materials and support needed to deliver the SFSC programme; o Provides a step-by-step guide to the curriculum. Facilitators receive a step by step manual for programme delivery, a CD containing all delivery materials and a copy of the parent manual in English. Supervision for facilitators can be provided individually or in groups. Having completed the five day core programme full accreditation involves delivery of a full SFSC thirteen week programme with a co-facilitator and completing the SFSC quality assurance processes, which includes being observed in delivering the programme to ensure fidelity and completing and returning all programme paperwork. There are a range of advanced skills programmes for trained facilitators to help them further develop confidence and competence. Advanced skills courses include: o Effective recruitment and retention of parents (1 day); o Effective co-facilitation (1 day); o Implementing group work effectively in parenting programmes (1 day); o Working with challenging parents (1 day); o using signposting to promote the impact of SFSC (1 day); o SFSC as a violence prevention tool (1 day). o • • • • • Evaluation of Parenting Programme: • • Data are carefully collected from each programme that is run and this has been the basis for a number of evaluation studies. The data include: o Demographic information on all parents; o Pre- and post-test questionnaires responses in relation to: ƒ Parents’ involvement in community activities; ƒ Intergenerational patterns of family drug use, domestic violence and child abuse; ƒ Parents’ sense of competence in 4 areas; ƒ Frequency of use of a variety of discipline measures and communication strategies; ƒ Participation in family activities; ƒ Perception of child’s competence in 4 areas; ƒ Parents’ ratings of the value and effects of the course. o Facilitator reports reviewing programmes. Costs: • • Facilitator training is £975 per person. The advanced skills packages are each £150 per person. Trainers: • Facilitator training is provided by US and UK based trainers. 155 Research: • • • • • • No current randomised controlled trials available. Studies (e.g. Wilding & Barton, 2007 and 2009) of a large number of families participating in the programme using data gathered from pre- and post-test questionnaires completed by parents have reported statistically significant change in: o Parents’ self-esteem; o Parents’ confidence in their parenting; o Family relationships; o Relationships with children; o Child’s self-esteem and self-control. Overall these studies indicated a 15% attrition rate. Farber and Maharaj (2005) report ‘statistically significant and practically meaningful trends’ on the use of SFSC programme with 39 parents of high-risk families of children with disabilities. Other published studies have used scientifically validated tests to report on these and other outcomes. Matthew et al (2005) used the Family Environment Scale to explore family attachment, cohesion, resilience and conflict and drew data from 1080 parents (699 of whom had completed SFSC) and concluded this programme was most successful (of the four model programmes) in promoting family resilience and dealing with family conflict. Lindsay et al. (2008) using a range of validated tests, reported improvements in parents’ mental well-being and parenting efficacy as part of a comparative study of three model programmes delivered in the UK. This study also indicated benefits of equal magnitude for all three programmes in parenting skills, parent mental health and child behaviour. 156 Triple P: Positive Parenting Programme Originator: Matt Sanders and colleagues Country of Origin: Australia: The University of Queensland, Brisbane. Early 1980s. First published: Australasia (Australia, New Zealand), Asia (Japan, Hong Kong, Implemented: Singapore), North America (USA, Canada), UK (England, Scotland, Wales), Republic of Ireland, Europe (Austria, Belgium, Germany, Luxembourg, Netherlands, Romania, Sweden, Switzerland), Middle East (Iran), Curacao and the BES Islands. Child Age Range: 0-16 years. General Aims/Goals: • • • To enhance the knowledge, skills, confidence and resourcefulness of parents. To promote a nurturing, safe, engaging, non-violent and low conflict environment for children. To promote all aspects of children’s development and prevent behavioural, emotional and developmental problems. Theoretical Background: • • • • • • Social learning models of parent-child interaction. Child and family behaviour therapy research. Developmental research on parenting in everyday contexts and social competence. Social information processing models. Developmental psychopathology research. Public/population health framework. Programme in action: • Triple P is a unique, multi-level model of family intervention for the prevention and treatment of behaviour. The system of intervention is grouped into five levels reflecting change in intensity and content of intervention with increasing need. o Level 1 Universal: a universal parenting information strategy; o Level 2 Selected: brief (one or two session) primary health care intervention for mild behavioural difficulties; or a three session large group seminar series on positive parenting o Level 3 Primary Care: 4 session intervention for mild to moderate difficulties including parent skills training; or a one-off brief 2 hour discussion group (multiple topics available). 157 Level 4 Standard: intensive 8-10 session individual, group or self directed (with telephone support) programme for more severe problems; o Level 5 Enhanced: intensive behavioural family intervention where parenting problems occur in the context of other family difficulties (e.g. conflict, depression). Within PEIP it is possible for LAs to include any aspects of the Triple P system, but the most common in wave 1 was Level 4 Group Triple P and Level 4 Group Teen Triple P. This involves 8 sessions: 4 in a group, 3 by telephone, plus a final group session. Group sessions 2 hours long and telephone sessions up to 30 minutes. Maximum contact about 11.5 hours. Typically conducted weekly. Requires one facilitator working with 10-12 parents. Programme methods include: o Presentations; o Skills training; o Observation; o Discussion; o Practice; o Feedback; o Video demonstrations; o Homework tasks. Content largely involves parental management of child behaviour and reduction of parental stress, with emphasis on enabling a safe, interesting positive learning environment; assertive discipline; realistic expectations; and taking care of oneself as a parent. o • • • • • • • • Process: • Central processes are provision of expertise for parents via a variety of educative processes, while attempting to foster self-sufficiency and self-efficacy. Training to run the programme: • • • • • • Group leaders require basic professional training; typically drawn from psychologists, family counsellors, social workers, parent educators, teachers, nurses, and other allied health professionals. Require knowledge of child/adolescent development & child/teen problems, and skills in social learning theory, group facilitation, and interpersonal communication. For example, 3 days for Level 4 Group Triple P Provider training Accreditation involves attending facilitator training course, running a group or practice with peers, and attend an accreditation day for each level of training, where knowledge & competencies tested by quiz & role play. Skills-based training approach with: o Didactic presentation; o Clinical problem solving exercises; o Small group exercises to practise consultation skills; o Video and live demonstration of core consultation skills; o Feedback and peer tutoring; o Homework; o Emphasis is placed on practitioner self-regulation skills. Courses run for up to 20 people. 158 • Facilitator’s kit for Group Triple P includes: Facilitator’s Manual; CD Rom with slide presentations; Every Family Group Workbook; Every Parent’s Survival Guide (DVD); and participant Notes for Group Triple P Provider Training Course. Evaluation of Parenting Programme: • • Parent questionnaires including: o Demographics; o Eyberg/SDQ; o Parenting Scale; o Being a parent; o Parent Problem Checklist; o Relationship Quality Index; o Depression Anxiety Stress Scale; o Parent Daily Report Checklist; o Client Satisfaction Questionnaire. Record of client attendance and session summary checklist completed by facilitator to monitor and promote fidelity. Costs: • £821.50 for a practitioner to train to facilitate Group Triple P or £1253.60 to train as a group facilitator on a combined Group and Group Teen Triple P provider course. Trainers: • Clinical and educational psychologists accredited by the University of Queensland. Research: • • • • Triple P has been evaluated in multiple studies, including randomized controlled trials, conducted by a network of international researchers. Independent replication studies have been completed in several Asian, European, Middle Eastern, and North American countries (see Sanders 2008). For independent meta-analyses see Norwak & Heinrichs (2008). The results indicate significant and clinically meaningful benefits for children and families maintained over time at the different intervention levels and for a range of families, including those with difficulties (e.g. parental mental health and relationship problems) that put children at increased risk for problems. 159 References for Appendix 2 Abt Associates. (2001). National evaluation of family support programs: Volume B. Research studies: Final report. Cambridge, MA. (see http://www.abtassoc.com/reports/NEFSP-VolB.pdf ) Billingham, S. (1993). Evaluation research design for Families and Schools Together program (FAST). Unpublished doctoral dissertation. Chicago, IL: Department of Psychology, DePaul University. Coombes, L., Allen, D., Marsh, M. & Foxcroft, D. (2009). Implementation of the Strengthening Families Program (SFP) 10-14 in Barnsley: the perspectives of facilitators and families. Child Abuse Review, 18, 41-59. Farber, M. & Maharaj, R. (2005). Empowering high-risk families of children with disabilities. Research on Social Work Practice, 15, 501-515. Gardner, F, Burton J, & Klimes I. (2006). Randomised controlled trial of a parenting intervention in the voluntary sector for reducing child conduct problems: outcomes and mechanisms of change. Journal of Child Psychology and Psychiatry, 47, 11231132. Hutchings, J., Bywater, T., Daley, D., Gardner, F., Whitaker, C., Jones, K., Eames, C., & Edwards, R. (2007). Parenting intervention in Sure Start services for children at risk of developing conduct disorder: pragmatic randomised controlled trial. British Medical Journal, 334, 678. Hutchings, J., Bywater, T., Williams, M., Whitaker, C. & Lane, E. (In press). Evidence for the extended School Aged Incredible Years parent programme with parents of high-risk 8 to 13 year olds. Child and Adolescent Mental Health. Kratochwill, T., McDonald, L., Levin, J., Scalia, P. & Coover, G. (2008). Families and schools together: an experimental study of multi-family support groups for children at risk. Unpublished manuscript. Kratochwill, T. R., McDonald, L., Levin, J. R., Young Bear-Tibbetts, H., & Demaray, M. K. (2004). Families and Schools Together: An experimental analysis of a parentmediated multi-family group program for American Indian children. Journal of School Psychology, 42, 359–383. Lindsay, G., Davis, H., Band, S., Cullen, M., Cullen, S., Strand, S., Hasluck, C., Evans, R., and Stewart-Brown, S. (2008). Parenting Early Intervention Pathfinder Evaluation; Final Research Report. London: Department for Children, Schools and Families. Matthew. R., Wang, M., Bellamy, N. & Copeland, E. (2005). Test of efficacy of model family strengthening programs. American Journal of Health Studies, 20, 164-170. 160 McDonald, L. (1993). Families and Schools Together (FAST). Final Report. Washington, DC: U.S. Department of Health and Human Services, Office of Human Development Services, Administration for Children and Families. McDonald, L., Moberg, D. P., Brown, R., Rodriguez-Espiricueta, I., Flores, N., Burke, M. P., et al. (2006). After-school multifamily groups: A randomized controlled trial involving low-income, urban, Latino children. Children and Schools, 18, 25–34. NICE (2006), Costing Template and Costing Report: Parent-training/education in the management of children with conduct disorders, National Institute for Health and Clinical Excellence, September. http://www.nice.org.uk/guidance/TA102/costtemplate/xls/English/download.dspx Nowak, C. & Heinrichs, N. (2008). A comprehensive meta-analysis of Triple P-Positive Parenting Program using hierarchical linear modeling: effectiveness and moderating variables. Clinical Child and Family Psychology Review, 11, 114-144. Patterson, J., Barlow, J., Mockford, C., Klimes, I., Pyper, C. & Stewart-Brown, S. (2002a). Improving mental health through parenting programmes: block randomised trial. Archives of Disease in Childhood, 87, 472-477. Redmond, C., Spoth, R., Shin, C. & Lepper, H. (1999). Modelling long-term parent outcomes of two universal family-focused preventive interventions: one year followup results. Journal of Consulting and Clinical Psychology, 67, 975-984. Sanders, M. (2008). The Triple P-Positive Parenting Program as a public health approach to strengthening parenting. Journal of Family Psychology, 22, 506-517. Sayger, T. & McDonald, L. (1998). Evaluation Report for Australian FAST Schools. Madison WI: FAST International. Scott, S., Spender, Q., Doolan, M., Jacobs, B., & Aspland, H. (2001). Multicentre controlled trial of parenting groups for childhood antisocial behaviour in clinical practice. British Medical Journal, 323, 194-198. Scott, S., O’Connor, T, Futh, A., Matias, C., Price, J. & Doolan, M. (2010). Impact of a parenting program in a high-risk, multi-ethnic community: the PALS trial. Journal of Child Psychology and Psychiatry, 51, 1331-1341. Spoth, R., Randall, K. & Shin, C. (2008). Increasing school success through partnershipbased family competency training: experimental study of long-term outcomes. School Psychology Quarterly, 23, 70-89. Spoth, R., Redmond, C. & Lepper, H. (1999). Alcohol initiation outcomes of universal familyfocused preventive interventions: one- and two-year follow-ups of a controlled study. Journal of Studies on Alcohol (supplement 13) 103-111. Spoth, R., Redmond, C. & Shin, C. (1998). Direct and indirect latent variable parenting outcomes of two universal family-focused preventive interventions: extending a 161 public health-oriented research base. Journal of Consulting and Clinical Psychology, 66, 385-399. Webster-Stratton, C., Rinaldi, J. & Reid, J. (2011). Long-term outcomes of Incredible Years parenting programme: predictors of adolescent adjustment. Child and Adolescent Mental Health, 16, 38-46. Webster-Stratton, C. & Reid, J. (2010). The Incredible Years parents, teachers and children training series: A multifaceted treatment approach for young children with conduct problems. In J. Weisz & A. Kazdin (Eds.). Evidence-based Psychotherapies for Children and Adolescents. New York: Guilford Publications. Wilding, J. & Barton, M. (2007). Evaluation of the Strengthening Families, Strengthening Communities Programme 2004/5. London: Race Equality Foundation. Wilding, J. & Barton, M. (2009). Evaluation of the Strengthening Families, Strengthening Communities Programme 2005/6 and 2006/7. London: Race Equality Foundation. Summary Report 162 APPENDIX 3: Statistical analyses of the four programmes (Triple P, IY, SFSC and SFP 10-14) Table A.3: Pre-course scores by parent and child variables and programme Variable Value Intercept Parent gender Missing Male Female (base) Family Composition Missing Living with partner Single parent (base) Housing Mental well-being Coeff Sig SE 40.2 0.56 Laxness Coeff Sig SE 23.5 .38 Overreactivity Coeff Sig SE 24.3 0.37 Child SDQ total Coeff Sig SE 17.2 .37 Child conduct Coeff Sig SE 4.5 .13 Child Impact Coeff Sig SE 2.9 .15 -1.47 2.13 *** 0.00 1.58 0.39 .21 1.09 -1.39 *** .26 0.12 -1.12 *** 1.08 0.26 .44 -.01 1.09 .26 .07 -.02 .36 .09 .27 -.30 ** .40 .10 1.33 1.82 *** 0.00 0.78 0.31 .85 -1.07 *** .51 .21 -0.52 -0.13 0.50 0.21 -.25 .30 .52 .21 .02 .08 .17 .07 -.09 -.02 .20 .08 Other/Missing Rented Owner occupier (base) -0.60 -1.29 *** 0.00 0.55 0.35 1.67 *** 1.34 *** .37 .23 -0.06 -0.03 0.36 0.23 .60 1.08 *** .37 .23 .09 .29 *** .12 .08 .05 .03 .14 .09 Education Missing Some GCSEs 5+ A*-C A/AS HE below degree Degree No qualifications (base) 0.92 0.51 2.45 *** 2.18 ** 2.03 *** 3.81 *** 0.00 0.61 0.39 0.54 0.67 0.44 0.55 -1.27 -1.04 -2.40 -2.70 -2.83 -3.52 ** *** *** *** *** *** .40 .26 .37 .45 .30 .37 0.06 -0.06 -1.08 ** -0.25 -0.39 -1.15 ** 0.39 0.26 0.36 0.44 0.29 0.37 .03 .04 -.64 -.54 -.72 * -1.68 *** .40 .26 .36 .45 .30 .37 -.01 -.08 -.31 * -.26 -.35 *** -.53 *** .13 .09 .12 .15 .10 .13 .22 .14 .16 .58 *** .29 * .13 .16 .10 .14 .17 .12 .14 Ethnic group White Other Mixed Heritage Indian Pakistani Bangladeshi Black Caribbean Black African Any other/Refused White British (base) 3.62 1.23 5.28 5.02 5.02 4.11 8.33 3.04 0.00 *** 0.87 0.85 0.93 0.81 0.95 1.07 0.90 0.70 -.59 -.35 2.26 2.43 2.80 -2.53 -.40 -.44 -1.37 -0.65 -0.33 -0.93 -0.94 -1.37 -1.15 -0.39 * 0.58 0.56 0.62 0.54 0.62 0.70 0.61 0.45 -2.88 -1.02 -3.36 -2.11 -3.37 -2.94 -4.23 -.86 *** *** *** *** *** .59 .57 .64 .55 .63 .71 .62 .46 .59 .58 .63 .56 .65 .72 .63 .47 *** * *** *** *** ** *** *** .20 .20 .21 .18 .22 .24 .21 .16 -1.02 -.73 -1.42 -1.09 -1.66 -.94 -1.45 -.41 *** *** *** *** *** *** *** * .23 .22 .25 .21 .25 .28 .24 .18 0.66 0.33 0.51 .21 -.18 .68 * .45 .22 .34 -1.58 *** -1.85 *** 0.20 0.44 0.21 0.33 -2.05 *** -1.87 *** .09 .45 .22 .34 -.61 *** -.62 *** .49 *** .15 .07 .11 -.91 *** -1.02 *** .58 *** .17 .08 .13 0.03 0.14 0.00 0.78 0.29 -.73 -.50 ** .53 .19 -0.54 -0.42 0.52 0.19 .53 1.40 *** .54 .19 .22 .43 *** .18 .06 .18 .49 *** .21 .07 ** .22 .23 ** .07 .17 Child age Missing 0-7 14+ 8-13 (base) *** *** *** *** *** *** 0.90 0.56 -1.79 *** 0.00 *** *** *** *** *** -.89 -.46 -1.45 -1.45 -1.66 -.78 -1.41 -.73 Child sex Missing Boy Girl (base) FSM Entitled Not entitled (base) -0.34 0.00 0.32 -.04 .22 0.15 0.21 SEN Yes No (base) -0.47 0.00 0.43 .29 .29 -0.30 0.28 3.80 *** .29 .66 *** .10 1.70 *** .11 Programme Other programme Incredible Years SF 10-14 SFSC Triple P (base) 1.84 0.99 -1.18 1.13 0.00 .40 .29 .28 .28 -0.77 -0.29 0.57 * -1.05 *** 0.40 0.28 0.27 0.27 -2.16 *** -.42 .94 *** -.90 ** .39 .29 .27 .28 -.85 *** -.26 ** .23 * -.47 *** .13 .10 .09 .09 -.48 ** -.17 .50 *** -.47 *** .15 .11 .11 .11 R2 10.0 ** * ** ** 0.60 0.43 0.42 0.41 -.80 * -.44 .16 -.94 *** 8.0 * 4.3 - 163 - .69 14.0 12.7 15.3 * .08 Table A.4: Change (improvement) in scores by parent and child factors and programmes Variable Value Intercept Parent gender Missing Male Female (base) 2.17 -1.45 ** 0.00 2.13 0.52 Family Composition Missing Living with partner Single parent (base) -0.86 -0.63 0.00 1.03 0.42 Housing Other/Missing Rented Owner occupier (base) Education Missing Some GCSEs 5+ A*-C A/AS HE below degree Degree No qualifications (base) White Other Mixed Heritage Indian Pakistani Bangladeshi Black Caribbean Black African Any other/Refused White British (base) Ethnic group Child age Missing 0-7 14+ 8-13 (base) 0.49 0.91 0.00 1.08 1.49 1.92 *** .36 -1.02 .86 ** Overreactivity Coeff Sig SE -7.0 0.54 Child SDQ total Coeff Sig SE -3.6 .44 Child conduct Coeff Sig SE -1.3 .16 Child Impact Coeff Sig SE -1.6 .19 1.08 1.26 *** 1.56 0.36 .48 .33 1.24 .30 .68 .05 .44 .11 .26 .04 .50 .13 .70 .29 0.41 0.00 0.72 0.30 .76 .20 .59 .25 .01 -.03 .21 .09 -.39 -.07 .25 .10 .51 .31 0.16 0.21 0.52 0.32 .10 -.27 .43 .27 -.02 -.12 .15 .09 .28 .09 .19 .11 .56 .38 .51 .60 .42 .51 0.14 -0.32 0.06 -0.65 -0.65 0.64 0.57 0.39 0.52 0.61 0.43 0.52 .49 -.36 -.20 .42 -.59 -.28 .48 .32 .43 .51 .36 .43 .28 .05 .10 .25 -.01 .09 .17 .11 .15 .18 .13 .15 .49 .06 .44 .26 -.02 .20 0.80 0.86 0.90 0.78 0.93 0.97 0.93 0.64 -.99 .85 -.04 .57 1.14 .75 .71 .47 .66 .72 .73 .65 .79 .80 .80 .55 .09 .34 .12 .36 .85 .50 .23 .40 * .23 .25 .26 .23 .28 .29 .28 .19 .42 .61 .77 .58 .78 .43 .51 .03 0.63 0.31 0.48 .42 .01 -.14 .53 .25 .40 .25 -.03 -.40 ** .18 .09 .14 .45 * .26 * -.62 *** .22 .11 .17 .26 .10 -.01 -.17 -1.79 * -0.30 -1.68 * -2.20 * -1.70 ** -2.48 *** 0.00 0.82 0.55 0.74 0.87 0.61 0.74 1.16 -.07 -.09 -.44 .15 1.23 -1.65 0.43 -0.51 -0.08 -1.43 -0.32 0.35 -1.02 0.00 1.13 1.22 1.27 1.09 1.31 1.40 1.25 0.93 -.56 1.13 -.41 -.82 1.29 3.39 *** .51 1.04 .77 .84 .88 .75 .90 .94 .91 .63 0.48 0.22 0.42 0.52 1.52 1.91 2.01 0.23 0.88 0.44 0.69 -.21 -.52 -.59 .62 .30 .48 1.57 0.35 -1.17 -0.54 -0.33 0.00 1.05 0.39 .69 .33 .74 .27 -0.69 0.08 0.75 0.27 -.14 .30 .64 .22 -.28 .14 .22 .08 -.09 -.06 0.37 0.00 0.43 .00 .30 -0.09 0.31 -.41 .25 -.11 .09 -.21 .30 .40 0.16 0.41 -.31 .34 .00 .12 -.54 *** .14 .56 .39 .38 .38 1.04 1.38 *** 2.14 *** 2.04 *** 0.58 0.40 0.38 0.39 .89 .63 .89 .95 .47 .33 .31 .32 .36 * .29 * .31 ** .55 *** .17 .12 .11 .11 .30 .36 ** .06 .54 *** .20 .14 .13 .14 0.12 0.24 2.09 ** 0.00 Missing Boy Girl (base) FSM Entitled Not entitled (base) SEN Yes No (base) -0.22 0.00 0.58 Programme Other programme Incredible Years SF 10-14 SFSC Triple P (base) -0.79 -0.49 -2.17 *** -1.18 * 0.00 0.82 0.56 0.55 0.55 2 Laxness Coeff Sig SE -6.7 .52 0.75 0.46 * Child sex R 2 R with LA - 164 - Mental well-being Coeff Sig SE 9.8 0.76 3.0 4.3 * * 1.06 .95 * 1.86 *** 2.18 *** 4.3 6.4 2.8 4.5 * * * * 1.6 2.4 ** ** 2.3 3.4 ** 4.1 6.0 * * * * * * * .20 .14 .18 .21 .15 .18 .28 .30 .31 .27 .34 .34 .32 .24 .11 APPENDIX 4: SUMMARY REPORTS FOR ALL EIGHT PEIP PROGRAMMES. 1. FAST 2. SFP 10-14 3. Incredible Years 4. SFSC 5. Triple P 6. Parent Power 7. STOP 8. Parents Plus. Note: Programmes 1-5 were the PEIP programmes originally specified by DCSF. Programme 6 (Parent Power) is a local programme used in one LA. Programmes (STOP) and 8 (Parents Plus) are manualised programmes with some previous evidence of efficacy. - 165 - - 166 - - 167 - - 168 - - 169 - - 170 - - 171 - - 172 - - 173 - Ref: DFE- Ref: DFE-121(a) ISBN: 978-1-84775-921-4 © Department for Education May 2011 174