Evaluation of Pause

Evaluation of Pause Research report July 2017 Katie McCracken, Sarah Priest, Ana FitzSimons, Kelly Bracewell, Kasturi Torchia, and Will Parry - Opcit Research with Nicky Stanley - University of Central Lancaster Children’s Social Care Innovation Programme Evaluation Report 49 Contents List of figures 3 List of tables 4 Executive summary 6 Summary of literature review 11 How Pause defines itself 13 Recruitment of the Pause cohort 14 The evaluation 15 Characteristics of the Pause cohort 19 Findings from baseline Client Monitoring Forms 19 Findings from baseline interviews 26 Is Pause effective in meeting its aims? 32 Impact on pregnancies 32 Impact on Pause women 33 How does Pause work? 42 Influencing partner agencies 42 Engaging women during a pregnancy-free period 45 Resource requirements 52 A cost benefit analysis of Pause 55 Implications and recommendations 58 Answers to the central evaluation questions 58 Recommendations 60 Appendix A: Evaluation limitations 63 Appendix B: Pause’s own baseline data 65 Appendix C: Analysis of impact on pregnancies 66 Appendix D: Summary of the Pause theory of change 73 Appendix E: Case studies 74 Appendix F: Additional cost benefit analysis 79 2 List of figures Figure 1: Pregnancies prior to intervention 67 Figure 2: Pregnancy rate per woman per year over the 5 years prior to intervention 68 3 List of tables Table 1: Summary of data collection 16 Table 2: Age range of Pause cohort 19 Table 3: Child removals 20 Table 4: Child contact 20 Table 5: Psychological wellbeing 21 Table 6: Domestic violence 22 Table 7: Higher risk alcohol use 23 Table 8: Problematic drug use 23 Table 9: Criminal justice involvement 24 Table 10: Housing 25 Table 11: Engagement with services 25 Table 12: Contraception use 26 Table 13: Impact on pregnancies 33 Table 14: Changes in incidents of domestic violence 36 Table 15: Changes in higher risk drinking 37 Table 16: Changes in problematic drug use 38 Table 17: Changes in housing 40 Table 18: Changes in contraception use 41 Table 19: Cost savings from removals avoided (Model 1) 56 Table 20: Cost savings from removals avoided (Model 2) 56 Table 21: Pregnancies in follow-up period, by group 68 Table 22: Model 1 (Pause vs. comparison) 69 Table 23: Pregnancies in pseudo follow-up period 71 4 Table 24: Model 2 (Pause vs. Pause) 71 Table 25: Cost savings from reductions in domestic violence 80 Table 26: Cost savings from reductions in higher risk alcohol use 81 Table 27: Cost savings from reductions in problematic drug use 82 Table 28: Cost savings figures 82 5 Executive summary Project Pause is a voluntary programme for women who have experienced, or are at risk of, repeat removals of children from their care. It aims to reduce the number of children being removed into care by working with women who have had children removed to improve their wellbeing, resilience, and stability. Pause offers women an 18-month, individually-tailored, intensive package of support, delivered by a dedicated Practitioner, which is intended to address a broad range of emotional, psychological, practical, and behavioural needs. As a condition of beginning this voluntary programme, women agree to use an effective form of reversible contraceptive for the 18-month duration of the programme. This is intended to allow women the opportunity to reflect and focus on their own needs. In addition to providing support, Pause works in collaboration with partner agencies (such as health and domestic violence services), at operational and strategic levels, to improve the broader service response to Pause women. Pause began in Hackney in 2013, and received funding from the Department for Education Innovation Programme in 2015 to expand the pilot within Hackney, and to Doncaster, Greenwich, Hull, Islington, Newham, and Southwark. While most pilot Practices worked only with women who had had at least 2 children removed from their care, one Practice worked only with women who had had one child removed, and another – the first Pause Practice – worked with multiple (‘one child removed’ and ‘2 or more children removed’) cohorts. Evaluation This evaluation report assesses the impact of programme delivery, as well as the processes through which impact was achieved, for 125 women engaging with Pause at these 7 pilot Practices. It also offers an assessment of the fiscal costs and benefits of delivering the programme. The evaluation period ran from March 2015 to September 2016. It should be noted that, due to the timing of the evaluation period, at the first Pause Practice only the ‘1 child removed’ cohort took part in the evaluation. In total, the evaluation cohort included 95 women who had had 2 or more children removed at 5 Practices, and 30 women who had had one child removed at 2 Practices. The central evaluation questions were: • to what extent is the Pause model effective in reducing the numbers of children removed from women’s care? 6 • • • • to what extent does engagement with the Pause programme have a positive impact on women’s wellbeing, resilience and stability? how cost-effective is the Pause programme? how do individual elements of the Pause programme relate to its impact? what factors enable, or hinder, the achievement of Pause’s aims? To address these questions, the evaluation team collected and analysed an extensive range of quantitative and qualitative data. Two statistical models were developed to estimate the impact of Pause on women’s pregnancy rates during their intervention, using data on pregnancy histories collected from the 125 women who were engaged with Pause, and 134 women in a comparison group. The team also developed Client Monitoring Forms (CMFs) to measure a range of outcomes at quarterly intervals throughout women’s engagement with Pause. Statistical analyses were conducted of data from 326 Client Monitoring Forms, completed by 115 women, at up to 5 time-points. To understand delivery processes, women’s experiences of engagement, mechanisms of change, and key enablers of, and barriers to, the achievement of the programme’s objectives, in-depth, semi-structured, one-to-one interviews were carried out with a broad range of respondents. These included 105 women, 25 Pause Practitioners, 8 Practice Leads, 6 Coordinators, and 34 professionals from partner agencies, including 10 local authority senior managers. The majority of respondents were interviewed at 3 timepoints. In-depth case studies of 14 women were also completed, and 4 focus groups were conducted with a total of 33 women across 4 pilot Practices. Two group activity sessions and 5 Pause Board meetings were also observed. Key findings The main findings from this evaluation are summarised below. Findings from qualitative and quantitative data suggest that Pause generally had a positive and significant impact on the women engaging with the programme, many of whom had complex, multiple, and mutually-reinforcing needs. Counterfactual impact analysis suggests that Pause was extremely effective in reducing the number of pregnancies experienced by women during their 18-month interventions. While 2 women became pregnant during their time with Pause, it is estimated that between 21 and 36 pregnancies would have occurred, had the cohort of 125 women not been engaged in the programme. Given the women’s histories, these pregnancies would have been likely to have resulted in removals. The cost benefit analysis indicates that the full costs of delivering Pause to the cohort of 125 women are likely to be offset by savings to local authorities within 2 to 3 years, with 7 estimated net cost savings of between £1.2 million and £2.1 million per year after the 18month intervention period. Qualitative data and data from CMFs indicate that women’s access to, and engagement with, services, including GP, housing, and substance misuse services, generally increased over time, and was associated with improved outcomes for some women. CMF data indicate that, by the end of the evaluation period, 25.6% of women who began Pause living in insecure housing had moved to secure housing; 30.8% of those who had been drinking alcohol at high risk levels had reduced their consumption to safer levels; and 27.3% of those who had been experiencing problematic Class A substance misuse were no longer using Class A substances. Further, almost half (46.4%) of women who disclosed that they had experienced an incident of domestic violence during their intervention reported that no further incidents had taken place during the final months of the evaluation. Qualitative data show that significant improvements to levels of confidence and self-worth were experienced by women engaged in the programme. Women also reported the benefit of learning new skills, behavioural responses, and coping mechanisms, which had helped them address past traumas and ongoing, day-to-day challenges more effectively. Qualitative data show that, while many women began their interventions with limited aspirations for the future, by the end, many had formulated new goals, and were taking steps toward their achievement. This included entering employment, education, or volunteering. Analysis of qualitative data on the processes through which these outcomes were achieved indicates the key mechanisms of change: • • • the provision of an intensive, bespoke programme of support addressing women’s emotional, psychological, practical and behavioural needs, delivered on a one-toone basis by a dedicated Practitioner during an 18-month pregnancy-free period direct advocacy to influence professional practice within partner agencies work at the strategic level to increase Pause women’s access to, and engagement with, partner agencies by adjusting systemic protocols That each of these mechanisms operated simultaneously was often fundamental to women’s progress, enabling problems to be tackled holistically. Recommendations Based on these findings, the evaluation team offer a number of recommendations. 8 Commissioning Given the positive impact of Pause on women, and the very high likelihood of investment in the programme resulting in very significant cost savings within a relatively short time period, there is good reason to continue and expand provision of the service, provided other key recommendations are met. Programme delivery The provision of support and advocacy to women by highly skilled, dedicated Practitioners is key to effecting change. The flexibility of the programme, which enables Practitioners to use their professional judgement and skill in tailoring their approach to meet the unique needs of individual women, should be maintained. Limits to Practitioners’ caseloads should remain at 6 to 8 women. This is necessary to allow for the intensity of work that is required to establish trusting relationships and support women to make sustainable changes. Practitioners should continue to be equipped with a budget to spend on each individual woman, to facilitate the delivery of key elements of the support package, and enable Practitioners to buy in additional services where appropriate. Management and strategic planning To ensure continuous professional development, maintain wellbeing, and avoid burn-out, Practitioners should receive effective and ongoing training, managerial support, and supervision, including clinical supervision. Highly skilled Practice Leads should be in place at all times, to provide Practitioners with appropriate opportunities for effective support, supervision, and professional development, and to ensure Practitioners are safeguarded in their work. Inter-agency collaboration at the strategic level is necessary to ensure services make the adjustments required to meet women’s fundamental needs, during and after their interventions. Pause Boards should continue to foster active participation from key decision-makers within partner agencies at every Practice. Efforts should focus on implementing adjustments to improve access to health, housing, and alcohol and substance misuse services. Pause should maintain its independence from social care services, and its status as a non-statutory, voluntary programme. The evidence suggests that women would be less likely to both begin and sustain meaningful engagement with Pause, if Practices were perceived as being part of social care services. 9 Pause materials A comprehensive induction package for Practitioners would facilitate the replication of Pause in other areas. Pause should continue to develop and trial materials and tools for the purpose of guiding reflective activities with women, and monitoring women’s progress. Evaluation Further longitudinal evaluation, over a longer period of time, should be conducted to identify the medium- and long-term impact of Pause on women, and on the number of children removed from their care. In particular, longitudinal tracking of individual women is required to ascertain whether changes made during the period of intervention are sustained. 10 Summary of literature review A review of relevant research literature was conducted to help inform the design of the evaluation of the Pause pilot, including the development of research tools. An important finding from the review is that there is a lack of evidence regarding what works in reducing multiple short-interval pregnancies that result in care proceedings. Other key findings from the review, which relate to repeat care proceedings, contraception, birth spacing, and the needs of women at risk of child removals, are summarised below to set the context for this report. Broadhurst and colleagues (2015) found that, in England,15.5% of mothers involved in care proceedings were linked to 29% of all care applications between 2007 and 2013. 1 A greater risk of unintended pregnancies has been identified for young women aged between 18 and 24 years, and women with lower levels of education or income. 2 Long-acting, reversible contraceptives are an effective method for reducing unintended pregnancies. 3 They are, however, under-used by the general population. The World Health Organisation (WHO), and the National Institute for Health and Care Excellence (NICE), suggest there is a need for broader education about their use and efficacy, to increase uptake. Crowne and colleagues (2012) suggest there is a relationship between close birth spacing and maltreatment of older children after the birth of a younger child. Their research found that close birth spacing is associated with significant negative outcomes for older children’s behaviour and development. 4 To engage with women who experience, or are at risk of, unintended and repeat pregnancies that result in child removals, professionals should work to build trusting and secure relationships with individual women, offer support in a flexible and open manner, 1 Broadhurst, K., Shaw, M., Kershaw, K., Harwin, J., Alrouh, B., Mason, C., and Pilling, M. (2015) ‘Vulnerable birth mothers and repeat losses of infants to public care: is targeted reproductive health care ethically defensible?’, in Journal of Social Welfare & Family Law, Vol. 37, No. 1, Pp. 84–98. 2 Brown, S., and Eisenberg, L. (Eds.) (1995) The best intentions: unintended pregnancy and the well-being of children and families, National Academies Press; Finer, L., and Henshaw, S. (2006) ‘Disparities in rates of unintended pregnancy in the United States, 1994 and 2001’, in Perspectives on sexual and reproductive health, Vol. 38, No. 2, Pp. 90-96; Gillespie, D., Ahmed, S., Tsui, A., and Radloff, S. (2007) ‘Unwanted fertility among the poor: an inequity?’, in Bulletin of the World Health Organization, Vol. 85, No. 2, Pp. 100107. 3 Blumenthal, P., Voedisch, A., and Gemzell-Danielsson, K. (2011) ‘Strategies to prevent unintended pregnancy: increasing use of long-acting reversible contraception’, in Human reproduction update, Vol. 17, No. 1, Pp. 121-137. 4 Crowne, S., Gonsalves, K., Burrell, L., McFarlane, E., and Duggan, A. (2012) ‘Relationship between birth spacing, child maltreatment, and child behavior and development outcomes among at-risk families’, in Maternal and child health journal, Vol. 16, No. 7, Pp. 1413-1420. 11 and recognise the emotional and time investment that is required for effective engagement. 5 5 Barlow, J., Kirkpatrick, S., Stewart-Brown, S., and Davis, H. (2005) ‘Hard-to-reach or out-of-reach? Reasons why women refuse to take part in early interventions’, in Children and Society, Vol. 19, No. 3, Pp. 199-210. 12 How Pause defines itself The following summary of Pause’s mission is based on Pause’s own management and promotional literature: ‘Pause aims to prevent the damaging consequences of thousands more children being taken into care each year, by working with women who have experienced, or are at risk of, repeated pregnancies that result in children needing to be removed from their care. It seeks to give women the chance to pause and take control over their lives, breaking a destructive cycle that causes both them and their children deep trauma, as well as costing the taxpayer hundreds of millions of pounds. It offers an intense programme of emotional, psychological, practical and behavioural support. Highly skilled Practitioners work with small caseloads of 6-8 women, to promote and sustain change. Each woman has an individual programme designed around their needs looking at the various elements of their system. Pause works with women in a radically different way, addressing everybody in their lives, including the fathers of their children, family members, partners and friends, as well as professionals within social services, housing, the NHS and the justice system. Pause also works at a strategic level to foster inter-agency collaboration, which is necessary to ensure that services make the adjustments required to meet women’s fundamental needs both during and after their interventions. Pause never gives up on the women, many of whom have been in care themselves. Nor does Pause label them according to their problems, such as drug or alcohol addiction, criminal conviction, domestic violence, or mental health. Instead, it is pioneering a new approach, offering women the support they need to gain better control of their lives, tackle destructive patterns, develop new skills, and avoid further trauma. This helps them set in place strong foundations on which they can build a more positive future for themselves. As a requirement of beginning this voluntary programme, women agree to take an effective, long-acting, reversible form of contraception for their time on Pause, under the careful monitioring of sexual health services. This allows them the opportunity to reflect and focus on their own needs, often for the first time in their lives. Originating from Hackney, Pause secured funds from the Department for Education’s Innovation Fund to extend the pilot to a further 6 local authorities: Doncaster, Greenwich, Hull, Islington, Newham, and Southwark.’ 6 6 For further information, see, for example, http://www.pause.org.uk/aboutpause. 13 Recruitment of the Pause cohort Pause Practices used varying criteria to determine which women they invited to engage in the programme. Notably, while most pilot Practices worked only with women who had had at least 2 children removed from their care, one Practice worked only with women who had had one child removed, and another – the first Pause Practice – worked with multiple (‘one child removed’ and ‘2 or more children removed’) cohorts. It should be noted that, due to the timing of the evaluation period, at the first Pause Practice only the ‘one child removed’ cohort took part in the evaluation. In total, the evaluation cohort included 95 women at 5 Practices who had had 2 or more children removed, and 30 women at 2 Practices who had had one child removed. Pause Practices also employ exclusion criteria. Although each Practice considers a range of different factors before deciding whether to invite a woman to engage in the programme, Pause generally will not work with women who are currently pregnant (at least where it is uncertain that the unborn will be removed from their care), or women who currently have children in their care. Pause women are encouraged to prioritise their own needs, and, as such, the programme is not appropriate for those living with dependents. Pause also seeks to work with those most at risk of removals, and so will not generally work with women who are not of child-bearing age, or women who have not experienced any child removals for several years. While Pause receives referrals from partner agencies, most of the research cohort was identified through Pause’s own scoping exercises. During this process, social care databases are searched for women who meet the basic criteria listed above. That data is triangulated with data sought from partner agencies, and staff then exercise their professional judgement to identify those women with the most complex, multiple, and mutually-reinforcing needs. A key aspect of the recruitment and engagement process is ‘assertive outreach’: staff are persistent in locating and contacting the women shortlisted for invitation to the programme, even where, for example, contact details held by partner agencies are out of date. Quarterly reports from each Practice indicate that, over the evaluation period, a total of 17 women ended their engagement with Pause before the end of their 18-month intervention period. 14 The evaluation Research questions The evaluation team conducted evaluations of both the impact and process of the Pause programme. Pause aims to reduce the numbers of children removed from women’s care, by supporting women to make lasting positive changes in their lives. The evaluation sought to provide evidence of whether, and to what extent, it is successful in achieving this aim. The central questions of the impact element of the evaluation were: • • • to what extent is the Pause model effective in reducing the numbers of children removed from women’s care? to what extent does engagement with the Pause programme have a positive impact on women’s wellbeing, resilience and stability? how cost-effective is the Pause programme? An assessment of processes was also undertaken to determine how the impact of Pause is connected to specific mechanisms of programme delivery. The central questions of the process element of the evaluation were: • • how do individual elements of the Pause programme relate to its impact? what factors enable, or hinder, the achievement of Pause’s aims? Research methods To address these questions, the evaluation team engaged in a very broad range of quantitative and qualitative research methods. The evaluation was commissioned to run over an 18-month period between March 2015 and September 2016: all data was therefore collected and analysed during this evaluation period. Two statistical models were developed to estimate the impact of Pause on women’s pregnancy rates during their intervention. The first model uses a comparison group of women who met the Pause programme selection criteria, but who lived in 2 local authorities delivering no similar intervention, while the second model uses extrapolation from Pause women’s own pregnancy histories. These models support a counterfactual analysis identifying how many pregnancies Pause women would have been likely to have experienced during the 18-month intervention period, had they not been engaging with Pause. 15 Table 1: Summary of data collection Data collection Numbers of participants Data on Pause women’s child removals 125 Data on comparison group’s child removals 134 Client management data from Practitioners, quarterly reports 125 Client Monitoring Forms (CMFs): Time 1, Time 2, Time 3, Time 4, and Time 5 115, 84, 68, 40, 19 Interviews with Pause Practitioners: Time 1, Time 2, and Time 3 25, 19, 15 Interviews with Pause Leads: Time 1, Time 2, and Time 3 8, 7, 5 Interviews with Pause Coordinators: Time 1, Time 2, and Time 3 6, 4, 6 Interviews with women: Time 1, Time 2, and Time 3 105, 80, 60 Focus groups with women 33 Case study interviews with women 14 Case study interviews with Practitioners 14 Case study interviews with partner agency professionals 9 Case study interviews with women’s partners, friends, and family 15 Interviews with local authority senior managers 10 Interviews with partner agency professionals 24 Group activity sessions observed (number of sessions) 2 Pause Boards observed (number of Boards) 5 At the start of their intervention, and at quarterly intervals thereafter, women worked with their Practitioners to complete a Client Monitoring Form (CMF). The CMF was designed by the evaluation team with input from Pause professionals, and assessed a range of issues, including women’s psychological wellbeing; experiences of domestic violence; drug and alcohol consumption; criminal justice involvement; housing; and engagement 16 with services. Statistical analysis of the data captured from 115 women’s CMFs was conducted. CMFs completed at the start of women’s interventions provided baseline data on women’s situations. To analyse the impact of engagement with Pause, this baseline data was then compared to data provided by women in their CMFs at quarterly intervals throughout their intervention. Analyses of the costs and benefits of the programme, and of the potential return on investment, were conducted, to identify the economic implications of the Pause model. These analyses used the impact measure of reductions in pregnancies as a proxy measure for reductions in care proceedings resulting in child removals, and calculated the cost savings to local authorities associated with these reductions. In addition, cost savings associated with reductions in women’s experiences of domestic violence, higher risk alcohol use, and problematic Class A drug use, which were identified through the evaluation’s impact analysis, were also calculated. To understand women’s experiences of engagement, mechanisms of change, and key enablers of, and barriers to, the achievement of the programme’s objectives, in-depth, semi-structured, one-to-one interviews were carried out with a broad range of respondents. These included 105 women and 39 Pause professionals, most of whom were interviewed at 3 time-points. Multiple interviews with women, at different time-points throughout their interventions (including baseline interviews held early on during their engagement), also enabled the evaluation team to identify women’s own views of the impact of their engagement with Pause, as well as changes in how they described their own lives. Focus groups were conducted with Pause women at 4 Practices. Two group activity sessions and 5 Pause Boards were also observed. In-depth case studies of 14 women were conducted, in order to provide a richer, individualised picture of the situations of some of the women involved in the programme, and the changes they made throughout their engagement. In most cases, developing these case studies involved conducting case study interviews with women, their Practitioners, some of their friends, partners, or family members, and professionals with whom they had been working closely (such as specialist learning disability social workers, personal advisors, and supported housing key workers). Interviews were also conducted with 2 men working with the men’s Practitioner at one Pause Practice. To understand how Pause affected the ways in which women engaged with other services, and the ways in which those services engaged Pause women, professionals from a broad range of partner agencies were interviewed about their experiences working with Pause women and Practitioners. These partner agencies included health, housing, probation, substance and alcohol misuse services, and services working with women who exchange sex. 17 Pause Practitioners completed regular records of their work, including their phone calls with women, one-to-one direct work, and advocacy for women within services. These records were analysed to identify the type and frequency of work conducted. For a discussion of methodological limitations to this evaluation, please see Appendix A. 18 Characteristics of the Pause cohort Below, we present findings on women’s circumstances and characteristics at the start of their engagement with Pause. These findings are based on analysis of baseline data gathered though Client Monitoring Forms (CMFs), and interviews conducted with women, their Practitioners, and other professionals from Pause and partner agencies. Findings from baseline Client Monitoring Forms During the early stages of their engagement with Pause, 115 women (92% of the whole Pause cohort of 125 women) filled in a CMF. Analysis of Time 1 CMF data demonstrates that Pause women, as a cohort, have experienced high numbers of removals of children from their care, and high rates of domestic violence and abuse, ‘higher risk’ drinking, Class A drug use, and involvement with the criminal justice system. In terms of psychological wellbeing, most women reported very high levels of grief associated with the loss of their children, and a significant majority reported that they had a mental health diagnosis. The data also suggest low levels of engagement with services, given the high level needs identified within the cohort. Pause’s own baseline data, drawn from social care case files during scoping exercises at each Practice, are outlined in Appendix B. Age The age ranges of the 115 women who filled in a CMF at Time 1 are given in the table below. Table 2: Age range of Pause cohort Age 20-24 25-29 30-34 35-39 40-44 45-50 Number of women 22 26 31 25 10 1 As % of women who answered this question (n=115) 19.1% 22.6% 27.0% 21.7% 8.7% 0.9% Children Women were asked about the child removals they had experienced. A total of 368 children were removed from the 108 women who filled in this section of the CMF. Of these children, 328 (89.1%) were subject to care proceedings. Over half of children 19 (56.8%) were placed in local authority care, while 40.8% were placed with extended family. Table 3: Child removals Child removals Number of children removed Number of children subject to care proceedings Number of children placed in extended family Number of children placed in local authority care Total 368 328 150 209 As % of children removed (n=368) 100% 89.1% 40.8% 56.8% Women were also asked about child contact arrangements. The majority of women (91 out of 107 who filled in this section of the CMF, or 85.0%) had some form of contact with their children. Just under half of these women (41) had supervised face-to-face contact with their children, while 28 women had contact with adopted children through a letterbox service. Other, less common, forms of contact included unsupervised face-to-face contact, and telephone contact. Table 4: Child contact Child contact Number of women in contact with children Number of women not in contact with children Total 91 16 As % of women who answered this question (n=107) 85.0% 15.0% Psychological wellbeing The CMF encouraged women to discuss their mental health, and disclose any current mental health diagnoses. A significant majority of women reported that they had a mental health diagnosis. Only 6 women declined to answer this question, and 35 reported that they had no diagnosis. The remaining 74 (or 67.9% of the women who answered this question) reported that they did have a diagnosis. The most frequently reported 20 diagnoses were for depression (42 women), anxiety (11 women), bipolar (10 women), and personality disorder (10 women). The CMF also asked women to answer a series of questions about various aspects of their self esteem, based on Rosenberg’s (1965) Self Esteem Scale. 7 Their answers were then analysed to produce self esteem scores on a scale ranging from 0 to 30. Scores between 15 and 25 are within the normal range, while scores below 15 suggest low self esteem. The mean average score for the Pause cohort was 14.5, indicating low self esteem. Women were then asked about their feelings of loss and grief. Questions were in the form of standardized measures drawn from Machin’s (2001) Adult Attitude to Grief Scale. 8 These questions were answered using a 5 point scale, from ‘strongly agree’ to ‘strongly disagree'. Two outcomes were then calculated from the responses: feelings of overwhelm, and feelings of resilience. Answers were scored from 0 to 4. The higher the number, the greater the feelings of overwhelm or resilience. At Time 1, 86 women answered this section of the CMF. The mean average score for overwhelm was 3 (and the median, 4), out of a maximum score of 4, suggesting high levels of overwhelm. The mean average score for resilience was 2 (and the median, 2), out of a maximum score of 4, suggesting mid-low levels of resilience. Table 5: Psychological wellbeing Psychological wellbeing Number of women with a mental health dignosis Women’s average self esteem score Women’s average overwhelm score Women’s average resilience score Totals 74 14.5 3 2 Domestic violence As the table below demonstrates, a very high percentage of women reported having experienced domestic violence at some point in their lives: 77 out of 92 women who responded to this question (or 83.7%). A significant proportion of women also reported that they had experienced some form of domestic violence during the previous 3 months: 19 out of 98 women who responded to this question (or 19.4%). Findings from interviews with women and their Practitioners (discussed in greater depth later in this report) reveal 7 8 Rosenberg (1965) http://www.yorku.ca/rokada/psyctest/rosenbrg.pdf Machin (2001) https://www.keele.ac.uk/mappinggrief/adultattitudetogriefscale/ 21 that Pause women often faced a range of obstacles to exiting abusive relationships. These included financial and housing insecurity, inadequate support networks, and low levels of self esteem, which some women directly attributed to their experiences of abuse as children. Table 6: Domestic violence Domestic violence Number of women who reported having ever experienced domestic violence Number of women who reported having experienced recent domestic violence Total 77 19 As % of women who answered this question (n=98) 83.7% 19.4% Higher risk alcohol use CMFs asked women about their level of alcohol consumption. Collection of data on alcohol consumption was based on 3 questions from the Alcohol Use Disorders Identification Tool (AUDIT). 9 Scores were provided for each response and a total score was calculated. A total score of 5 or more suggested women were in a ‘higher risk’ or ‘potentially dependent’ drinking category. 10 Out of 115 women who filled in a CMF, 13 (11.3%) declined to answer questions about their alcohol consumption at baseline. Of the 102 women who did respond, 28 (27.5%) reported higher risk drinking. Most of these women reported in interviews that they used drinking – and, in some cases, had used it for many years – as a coping mechanism to deal with the grief and trauma associated with the removal of their children, and with the constant challenges of day-to-day life. 9 AUDIT-C (2017) https://www.alcohollearningcentre.org.uk/Topics/Latest/AUDIT-C/ For the sake of brevity, we refer to ‘higher risk’ drinking hereafter. 10 22 Table 7: Higher risk alcohol use Alcohol Number of women who reported higher risk drinking Number of women who reported no higher risk drinking Total 28 74 As % of women who answered this question (n=102) 27.5% 72.5% Problematic drug use Out of 115 women who filled in a CMF, 24 (20.9%) declined to answer questions about illegal drug use at the start of their programme. Responses from the 91 women who did respond were analysed according to Singleton and colleagues’ methodology, and a binary outcome of ‘problematic drug use’ or ‘non-problematic drug use’ was developed for each woman. 11 Of the 91 women who responded to these questions, 24 (26.4%) reported drug use falling into the ‘problematic’ category. Interviews with these women often revealed a tension between their desire to overcome addiction, and their (in some cases, long-standing) reliance on drugs as a coping mechanism. Some women also discussed a paradoxical connection between the removal of their children and their addiction: while the goal of being a better mother drove their motivation to stop taking drugs, the need for temporary relief or distraction from intense feelings of guilt and shame about their perceived failings as mothers, or of anger toward the services that implemented removals, propelled their choices to misuse. Table 8: Problematic drug use Class A drugs Number of women who reported using Class A drugs Number of women who reported not using Class A drugs Total 24 67 As % of women who answered this question (n=91) 26.4% 73.6% 11 Singleton et al. (2006) http://webarchive.nationalarchives.gov.uk/20110218135832/rds.homeoffice .gov.uk/rds/pdfs06/rdsolr1606.pdf 23 Criminal justice involvement Women were asked about their involvement with the criminal justice system. Of the 115 women who filled in a CMF, 8 (7.0%) reported having been in prison within the previous 3 months, and a further 6 (5.2%) declined to answer this question. Five women (4.3%) reported having been arrested within the previous 3 months, and a further 21 women (18.2%) declined to answer this question. Ten women (8.7%) reported that they were currently on probation, and 15 (13.0%) declined to answer this question. Table 9: Criminal justice involvement Criminal justice Number of women who had been in prison in the last 3 months Number of women who had been arrested in the last 3 months Number of women who were on probation Total 8 5 10 As % of women who answered this question (n=109, 94, 100) 7.3% 5.3% 10% Housing The CMFs asked women about their current housing situation. A total of 110 women answered this question, and over half of these (52.7%) reported having a legal tenancy. A further 8 women were living in supported housing. The housing situations of the remaining 44 women (40%) were less secure, with 14 women reporting that they were living with family, 10 on other people’s floors or sofas, 9 with their partners, 7 in a hostel or temporary accommodation, and one at a refuge. One woman reported that she was living rough. Findings from interviews with women and their Practitioners suggest very strongly that obtaining and maintaining appropriate, secure housing was an important priority for many Pause women, who had often, for various reasons, been excluded from, or placed at the bottom of, housing waiting lists. 24 Table 10: Housing Legal tenancy With family On others’ floors/sofas With partner Supported housing Hostel or temp accomodation Refuge Living rough Other Housing Total 58 14 10 9 8 7 1 1 2 As % of women who answered this question (n=110) 52.7% 12.7% 9.1% 8.2% 7.3% 6.4% 0.9% 0.9% 1.8% Engagment with services Women were asked about the services with which they were currently in contact. The service with which women reported being in contact most frequently was social care (31 women), followed by mental health services (23 women), drug and alcohol services (19 women), housing (14 women), employment (8 women), probation (7 women), GPs (6 women), education (2 women), and Independent Domestic Violence Advisor (IDVA) services (2 women). Given the high rates of mental health diagnoses, domestic violence, alcohol and drug misuse, and insecure housing identified above, baseline rates of engagement with the services set up to address these needs appear low. Table 11: Engagement with services Social services Mental health Drugs and alcohol Housing Employment Probation GP Education IDVA Services Total 31 23 19 14 8 7 6 2 2 As % of women who answered this question (n=81) 38.3% 28.4% 23.5% 17.3% 9.9% 8.6% 7.4% 2.5% 2.5% Contraception The CMFs asked women about their current contraception use. As the table below indicates, a high percentage of the women reported using a form of long-acting, 25 reversible contraception at Time 1 (83 out of 115 women who filled in this section, or 72.2%). It is important to note that, at the time of the first assessment, many women had already had some involvement with their Pause Practitioners, who aim, during the early stages of engagement, to encourage uptake of contraception. The high percentage of contraception uptake at Time 1 is therefore expected. The most commonly-used form of long-acting, reversible contraception at Time 1 was the implant (51 women), followed by the intrauterine device (IUD) (15 clients), and injections (14 clients). 32 women used other forms of contraception, which were sometimes combined with long-acting, reversible contraception. The most commonly reported of these other forms of contraception were condoms (10 women) and birth control pills (9 women). Table 12: Contraception use Contraception Number of women using LARC Number of women using the implant Number of women using an IUD Number of women using injections Number of women using other forms of contraception Total 83 51 15 14 32 As % of women who answered this question (n=115) 72.2% 44.3% 13.0% 12.2% 27.8% Findings from baseline interviews During interviews held at an early stage of their interventions, women were asked about their current circumstances, and aspirations for the future. In particular, women were asked about what they wanted to achieve from working with Pause; important relationships in their lives; their engagement with services; their attitudes toward contraception; and various aspects of their psychological wellbeing. A key finding from these interviews was that women’s thoughts and feelings about each of these issues were interconnected with their thoughts and feelings about parenthood, which were, in turn, profoundly affected by the trauma of having children removed from their care. Aspirations and goals When asked by the evaluation team at the start of their interventions what they wanted to achieve from their work with Pause, different women gave very different answers. However, the majority of women related at least part of their answers directly to 26 parenting. Nearly one-quarter of women (24.7%) stated that they wanted their children to be returned to them. A similar proportion (23.1%) reported that they wanted to improve their contact arrangements with their children (for example, by increasing frequency of contact, or by having unsupervised face to face contact). A smaller proportion of women (9.7%) stated that they definitely wanted to have, and maintain care of, a child in future, and 3.2% said that, if they had another child in future, they wanted to be able to maintain care. These women reported that they hoped Pause would help them to pursue these goals. Some women felt that they needed to make improvements in their own lives before the desired arrangements could be made (for example, by becoming more emotionally or financially stable, overcoming addiction, or improving their housing), and that Pause would help them to do this. Others hoped Pause would play a role in enabling children’s services and the courts to understand them better, and to recognise that such arrangements should be allowed. These findings demonstrate a degree of dissonance, at this early stage of engagement, between some women’s understanding of the function of Pause and Pause’s actual objectives. However, as helping women to assess their views and expectations of parenthood is an integral part of the programme, these differences might be expected. In addition to these parenting-related goals, Pause women often expressed the view that working with Pause was an opportunity to get the focused support they needed to improve their own current or future circumstances, and increase their own level of wellbeing. Examples of issues for which women wanted support included housing, benefits, debt, health, employment, volunteering, education, or access to particular services. Specific issues included the need to gather the required records and other paperwork to apply for a passport, open a bank account, or enroll in a college education course. A common theme to emerge from interviews with women at the early stages of their interventions is a restricted sense of agency, often due to a range of complex and intersecting reasons. Almost all women described instances or periods in their lives when they had been unable to prioritise their own needs, or evaluate the consequences of the different options available to them. Frequently, this was linked to a sense of the constant difficulties of daily life, including the ongoing trauma of losing their children, abusive or unhealthy relationships, insecure housing, financial insecurity, drug and alcohol misuse, and dealing with professionals toward whom they felt considerable hostility. These difficulties were often mutually reinforcing, and their cumulative effect, too overwhelming to overcome without support. They were also compounded by an absence of positive, fun activities, and, Practioners reported, women limiting their sphere of activity to the immediate local area. 27 Almost all women interviewed by the evaluation team explained that they had never had anything like Pause in their lives. They had never worked with any service they could trust to be on their side and to persevere in supporting them in all aspects of their lives. Relationships A pervading theme in many women’s lives prior to Pause was the absence of positive role models and supportive relationships, and the presence of unhealthy or abusive relationships. Women who had themselves experienced being in care as children commonly reported feeling a lack of reliable and loving parent figures. Women who reported experiencing abuse, often perpetrated by family members, as children, , or growing up witnessing domestic violence and abuse between parents or other family members, also reflected on the impact this abuse had had on their lives. For example, relating her current circumstances directly to her childhood experiences, one woman explained, ‘it’s because of my past and that. My mum sent me out prostituting when I was 14. That’s how it all came to us going into care’ (Client 33, Time 1). Practitioners also observed that many women had not had the benefit of security, stability and support during their development through childhood and adolescence. As indicated by data from women’s CMFs, over five-sixths (83.7%) of women reported that they had been, or were currently, involved in violent intimate relationships. Forms of domestic abuse reported by women in interviews included physical, sexual, emotional, psychological, and financial abuse. Practitioners also reported cases in which they felt their clients were in abusive relationships but had not yet recognised or acknowledged the abuse. Women’s lack of supportive relationships, or anyone to talk to about the issues they were facing, was a recurrent theme in interviews. For example, one woman explained her need to be listened to: ‘I’ve had depression, because I’ve been through domestic violence with my ex and that. He used to beat me up and everything. […] I just want someone to sit there, and talk to me, and just listen to what I’ve been through’ (Client 27, Time 1). Another important finding is that, where women’s children were placed with members of their families or the families of the fathers, this could engender, or compound, women’s vulnerability to coercion or manipulation by their children’s carers. This frequently left women feeling disempowered by both their children’s carers and social care professionals, each of whom were often perceived as hostile and unsympathetic. 28 Psychological wellbeing The loss of their children into care was reported by all women as profoundly painful, and a cause of substantial ongoing emotional trauma. The psychological repercussions of child removals were described by the overwhelming majority of women in terms of marked grief and depression. One woman’s account provides a succinct summary of many others’: ‘when your children are taken, it feels like they have died, because that’s it. It is sort of a grieving process. They are gone’ (Client 61, T1). In addition, acute feelings of guilt and shame following the removal of their children were frequently reported. For some, this was exacerbated by rejection from family and friends following removals, which left them feeling stigmatised and isolated. Removals were often, therefore, reported by women and their Practitioners as having been a central cause or catalyst of a downward spiral in mental health and self-care more generally. Nonetheless, most women reported that, before engaging with Pause, they had not received any support to deal with the psychological impact of losing their children. In the words of one woman: ‘When I lost [my child], they said, ‘we'll get you counselling’ and stuff like that. They never did. They took my bairn and left me. […] I got depression, and then when I had [another child] I got postnatal depression, and that made the depression even worse. Over the years I have just got worse’ (Client 40, T1). Women were asked in interviews about how they attempted to manage the pain of loss. Some tried not to think about it, while others engaged in various forms of self-harming behaviour. One woman reported: ‘I used to overdose and self-harm. But, instead of doing that now, because it marks my body, I decided to starve myself instead. It’s a way of dealing with my grief’ (Client 88, T1). Several women also described feeling conflicted about whether having more children would help them to manage their sense of loss. As one explained: ‘Because I’m grieving for my kids, I just feel like I want to get pregnant, because it’s going to stop that feeling. And then I think about getting pregnant and having other kids, but I don’t want that to happen’ (Client 70, T1). Engagement with services A negative view of, and hostility toward, social services was almost universal amongst women at the early stages of their interventions. This was often related to a belief that children should not have been removed, and that social services had not listened or 29 provided adequate support during the processes that led to removals. Another common theme was the feeling that women had simply been dropped by services following child removals, rather than supported to access the help they needed. A perceived distinction between social services and Pause therefore emerges as an important mechanism for building trust between women and Practitioners. As one woman stated: ‘The social worker’s not my social worker. She’s the children's social worker. So it's all about the children, not me. Whereas [my Pause Practitioner] is for me. [She] supports me as a person’ (Client 18, T1). A number of services were reported by women as being difficult for them to engage with. These included housing, the criminal justice system, health, education (including colleges), social services, mental health services, services for people with experience of substance and alcohol misuse, and services for people who exchange sex. Difficulties in accessing services were due, in part, to a range of complex and intersecting factors in women’s circumstances, but were often exacerbated where services did not follow a model requiring professionals to be flexible in making patient, persistent efforts to engage their clients. Professionals from both Pause and partner agencies reported, for example, that when women consistently missed appointments, they could sometimes be de-prioritised either by individual professionals or, indeed, by inflexible systemic protocols that put them to the back of the queue or removed them from waiting lists altogether. Factors in women’s individual circumstances that constituted barriers to accessing services included residual frustration with particular services, and anger and hostility toward particular professionals. Women with learning disabilities and difficulties reported that it was often challenging to navigate systems and follow processes without appropriate support. Mental or emotional health issues, including anxiety, depression, and some forms of psychosis, often made it difficult for women to leave their homes, or face interacting with professionals. These issues could also make it difficult for women to spend long periods of time among strangers in waiting rooms. Similarly, substance or alcohol use could create difficulties in attending or engaging in appointments. Women who engaged in exchanging sex reported that, given the hours during which they did so, they faced specific difficulties in attending morning appointments. Attitudes to contraception Findings from interviews with Pause women and their Practitioners indicated a range of responses to the requirement to take contraception for the duration of the programme. In discussions of this requirement, the large majority of women (90.2%) reported feeling completely comfortable with this facet of Pause, viewing it as a way to ensure a welcome break from pregnancy. As one explained, ‘it's important, because I've had 9 children and 30 I don't want another baby just yet’ (Client 45, T2). Of these women, nearly two-fifths (38.2%) were already using a long-acting, reversible form of contraception before they signed up with Pause, and the same proportion indicated that they did not want any more children in future. One woman noted: ‘I’ve been on it for a long while. […] Because my 2 are going through what they are going through, I can’t risk it [becoming pregnant]. So I went for the implant and I have had it ever since’ (Client 63, T1). Three women reported to the evaluation team that they were seeking sterilisation before signing up to the Pause programme. One of these women explained: ‘I've had 6 kids, 16 miscarriages. […] The last 3 of my babies really […] affected my disability. Each one of them posed a real risk of me actually dying. So I've done my fair share of having babies. I didn't want anymore. I was supposed to have been sterilised about 2 years ago. I had a date and everything, and fell pregnant’ (Client 84, T1). A minority of women (9.8%) reported a greater degree of ambivalence about contraception, but ultimately made the decision to accept the requirement, despite initial reservations. One woman explained her decision in terms of making a mutual commitment: ‘I was a bit shocked by the contraception bit. But once it was explained what the idea and the thinking was: ‘we want you to make a commitment to us, like we’re making a commitment to you,’ that's how it kind of felt to me. […] I was absolutely fine with it’ (Client 79, T1). Most women’s reservations centred on the side effects of contraception, including irregular bleeding. Women reported that having the opportunity to discuss their contraceptive options at a sexual health service had reassured them of the benefits of taking a reliable, long-acting, reversible form of contraception, and helped them to reach an informed decision to engage with the programme. It should be noted that, as taking contraception is a requirement of voluntary engagement with Pause, women who, on balance, would not make this choice are not eligible for the programme. 31 Is Pause effective in meeting its aims? Impact on pregnancies One of the key mechanisms through which Pause seeks to achieve positive and sustainable change for women is the establishment of a pregnancy-free period in women’s lives, during which they are supported to focus on meeting their own needs. A central element of the evaluation is, therefore, an assessment of the extent to which Pause achieved this goal. A counterfactual analysis, using 2 statistical models, was developed to estimate the impact of Pause on women’s pregnancy rates during their intervention 12. Both models demonstrate a very significant reduction in pregnancies experienced by Pause women, with a very high level of confidence that the reduction is directly attributable to women’s engagement with the programme. Two women became pregnant during their time with Pause, and were transitioned out of the programme. Nonetheless, the counterfactual analysis demonstrates that, had the cohort not been engaged with Pause, a far higher number of pregnancies would have occurred. Model 1 estimates how many pregnancies Pause women avoided, based on a comparison group of women who met the Pause programme selection criteria at 2 local authorities, where no similar interventions to Pause were being delivered. A search of social care records at these local authorities was conducted to identify all women who had (recorded) child removals during 2 index years. Exclusions were then made where the social care files of the women, their children, and other family members indicated the women would not be engaged by Pause because they were pregnant, had children in their care, were not of child-bearing age, or had not experienced recent removals. 13 Model 2 estimates how many pregnancies Pause women avoided based on extrapolation from their own pregnancy histories. Both models conditioned on pregnancy histories and age at intervention. Further information on the statistical methods used in this analysis is provided at Appendix C. The results of the statistical analysis of the impact of Pause on women’s pregnancy rates during their intervention are summarised in the table below. Model 1 resulted in an estimate of 21.1 pregnancies avoided during the 18-month intervention period (with an 12 An additional comparison was made to another project that appeared to have similar aims to Pause, based in another city. However, analysis revealed there were too many differences between the population groups for the comparison to be meaningful. Reporting of this analysis was not possible, due to the need to maintain the project’s anonymity, which could not be guaranteed. 13 It should be noted that the Pause selection process involves triangulation of social care data with partner agencies, and staff exercise their professional judgement to identify those women with the most complex, multiple, and mutually-reinforcing needs. 32 aggregate rate per woman per year of 0.113). Model 2 resulted in an estimate of 36.8 pregnancies avoided (with an aggregate rate per woman per year of 0.196). Table 13: Impact on pregnancies Pregnancies Model 1 Model 2 Total number of Pause women 125 125 Total pregnancies avoided in 18 months 21.1 36.8 Aggregate rate per woman per year* 0.113 0.196 Mean predicted rate per women per year 0.080 0.181 Upper confidence interval for mean 0.093 0.194 Lower confidence interval for mean 0.069 0.169 *Aggregate rate (a simple average) is affected by positive skew in outcome (count variable) The 125 women in the Pause evaluation sample included 30 women who had had only one child removed. The remaining 95 women had had two or more children removed. A check was carried out on model 2, without the inclusion of the group of women who had had one child removed, to see if this changed the results of the analysis. This revealed that differences in the impact on pregnancies during the 18-month intervention period were not significant when women with a history of only one child removed were excluded from the analysis. Impact on Pause women While establishing a pregnancy-free period in women’s lives is an important part of the Pause model, Pause staff indicated in interviews that the purpose of this period is, in part, to enable women to identify and address the full range of their needs. These staff expressed that their objective was not only to work therapeutically with women to develop resilience, and increase wellbeing and stability, but also to ensure they have access to the resources and services they need to make positive, sustainable changes. Therefore, 33 an assessment of the extent to which Pause women achieved these changes during their engagement was undertaken. Longitudinal analyses were conducted of data on a range of measures from women’s CMFs, which they completed with Practitioners at the start of their interventions, and then at quarterly intervals. A total of 115 women completed Time 1 CMFs near the start of their engagement with Pause. Of these, 84 women completed a CMF at Time 2, 68 at Time 3, 40 at Time 4, and 19 at Time 5. As women began their engagement with Pause at different stages of the evaluation period, it was not possible to gather CMF data in the middle or final stages of some women’s interventions, as these went beyond the evaluation period. The findings from these analyses indicate that the requirement to take an effective, longacting, reversible form of contraception was met by the overwhelming majority of Pause women within the first few months of their engagement of the programme. The findings also suggest an increase over time in many women’s access to, and engagement with, the services they need to meet their basic needs, including GP and housing services. While a significant proportion of women began their engagement with Pause living in insecure housing, over one-quarter of these women were able to move into secure housing during their engagement with the programme. In terms of domestic violence, the findings suggest that Pause women, as a cohort, have experienced relatively high levels of domestic violence, compared to women in the general population. Analysis of the frequency of incidents during the programme produces a mixed picture: while some women reported fewer incidents as they progressed through their intervention, others were still experiencing incidents of violence by the end of the evaluation period. Analyses of alcohol and drug consumption indicate that most Pause women did not fall into established ‘higher risk’ or ‘problematic’ categories. However, the consumption levels of a significant minority either fluctuated into, or remained stable at, a high level. Nonetheless, just under one-third of women who started their engagement with Pause with high levels of consumption of alcohol or drugs were found to have considerably reduced their intake. In some cases, these reductions occurred while women were engaging with alcohol and substance misuse services. The central mechanisms through which changes were pursued are examined in full in the next section of this report. 34 Psychological wellbeing Women were asked a series of questions about various aspects of their self-esteem, and changes in reported levels of self-esteem over time were then analysed according to Rosenberg’s (1965) methodology. 14 It should be noted that measures of self-esteem (and of loss and grief, which are examined below) can inform only an imperfect assessment of changes in psychological wellbeing. To illustrate, recorded changes may result from developments in women’s ability to reflect on issues or their willingness to discuss issues with Practitioners, as well as from changes in how they relate to, or understand, their experiences. The analysis of changes to self-esteem provides a mixed picture: while some women reported experiencing improvements over time, others reported no change, and some reported reductions in their self-esteem. Of the 80 women who provided answered questions about their self esteem at Time 1, 45 provided follow-up data at Time 3. Of these 45 women, 20 (44%) experienced an improvement in self-esteem, 18 (40%) experienced a decrease in self-esteem, and 7 (15%) experienced no change. Scores were based on a scale of 0 to 30. Where improvements between Time 1 and Time 3 were experienced, the average score increase was 3.8 points. Where decreases in self-esteem scores were noted, the average reduction was also 3.8 points. At Time 4, 20 women provided follow up data on their self-esteem. Of these 20 women, 8 (40%) experienced an increase in self-esteem, compared to Time 1. Ten women (50%) experienced a decrease, and 2 (10%) experienced no change. Where an improvement was experienced, the average score increase was 4.8 points. Where a decrease was experienced, the average score reduction was 4 points. Women were also asked to answer a series of questions on loss and grief. Although analysis of these data also presents a mixed picture, there generally appears to be a decrease in women’s feelings of overwhelm and an increase in feelings of resilience. Of the 86 women who had answered this question at Time 1, 21 (24.4%) provided data at Time 4, allowing an analysis of changes to these outcomes over time. Of these 21 women, 7 (33%) had reportedly experienced a reduction in overwhelm feelings, 10 (48%) had experienced no change, and 4 (19%) had experienced an increase. Where changes were reported, these were no greater than plus or minus one point in either direction. With regard to resilience, 13 of the 21 women (62%) who provided data at Time 4 reported an increase in resilience. Four women (19%) reported having experienced no 14 Rosenberg (1965) http://www.yorku.ca/rokada/psyctest/rosenbrg.pdf 35 change in resilience, and the remaining 4 women (19%) reported a reduction in resilience. A higher proportion of women provided follow up data at Time 3, compared to Time 4. At Time 3, 45 of the 86 women (52.3%) who had answered this question at Time 1 provided follow-up answers. While women had been engaging with Pause for a shorter length of time at this time-point, all would have been at least 6 months into their intervention. At Time 3, 18 women (40%) reported a decrease in feelings of overwhelm since Time 1, 14 (31%) reported no change, and 13 (29%) women appeared to have experienced an increase in feelings of overwhelm. In terms of feelings of resilience, 21 women (47%) had reportedly experienced an increase, 8 women (18%) had experienced no change, and 16 women (35%) had experienced a reduction in resilience. Domestic violence Using CMF data recording whether women had experienced any recent incident of domestic violence (that is, any incident in the last quarter), a longitudinal analysis was conducted of changes to experiences of incidents of domestic violence. The table below indicates the total numbers of women who disclosed recent DVA at each time-point. Table 14: Changes in incidents of domestic violence Domestic Violence Time 1 Time 2 Time 3 Time 4 Time 5 Number of women who disclosed recent DVA 19 14 8 7 5 As % of women who answered this question (n=98, 78, 65, 27, 13) 19.4% 17.9% 12.3% 25.9% 38.5% Over the entire period of data collection, 28 women disclosed at least once that they had experienced an incident of domestic violence during the preceding quarter. Of these, 13 women (46.4%) reported no incidents at the latest time-point at which data were collected, following at least one previous report of an incident. However, 3 women reported domestic violence victimisation at each time-point. A further 4 reported victimisation at the first time-point, but did not complete a CMF at any later point, and longitudinal analysis of changes in these women’s experiences of domestic violence was therefore not possible. An increase in domestic violence incidents was recorded for 8 women: these women had disclosed no incidents at the first time-point, but went on to report victimisation at the latest time-point. However, it should be noted that 36 qualitative findings demonstrate that the establishment of trust and openness in relationships between women and Practitioners tended to occur over the first few months of engagement. This should bear on the interpretation of these increases in disclosures of domestic violence incidents: it is possible that such increases indicate greater willingness to disclose, rather than a rise in the frequency of incidents. Higher risk alcohol use The section of the CMF asking women to record their alcohol consumption over the previous quarter was completed by 108 women at least once. Seven women who filled in at least one CMF declined to answer this section in every CMF they completed. Using 3 questions from the Alcohol Use Disorders Identification Tool (AUDIT), women’s responses at each time-point were analysed to produce an outcome indicating whether or not women were in a ‘higher risk’ or ‘potentially dependent’ drinking category. 15 The table below indicates the total numbers of women who disclosed higher risk drinking at each time-point. Table 15: Changes in higher risk drinking Higher risk drinking Time 1 Time 2 Time 3 Time 4 Time 5 Number of women who disclosed higher risk drinking 28 15 18 11 2 As % of women who answered this question (n=102, 74, 65, 32, 14) 27.5% 20.3% 27.7% 34.4% 14.3% Longitudinal analysis of data on alcohol consumption across the entire period of data collection indicates that no higher risk drinking was identified for 47 women, for whom longitudinal data were available, and a further 22, who provided this data at only one time-point. The analysis suggests that the remaining 39 experienced higher risk drinking at one or more time-points. A significant proportion of these women, 12 (30.8%), had reduced their alcohol consumption to a lower level of risk by the end of the recorded period. Of these, 6 recorded that they were in treatment at some point during their intervention. 15 AUDIT-C (2017) https://www.alcohollearningcentre.org.uk/Topics/Latest/AUDIT-C/ 37 However, 10 women reported higher risk drinking at the first and last recorded timepoints, while a further 6 reported higher risk drinking at one time-point, but did not provide longitudinal data on this question. Eleven women who were not in the higher risk category at the start of their intervention had, by the latest recorded time-point, moved into the higher risk category. It is not certain, however, whether these changes were the result of increased willingness to report, rather than heightened risk. Problematic drug use The section of the CMF asking women about their illegal drug use was completed at least once by 99 women. Sixteen women who filled in at least one CMF declined to answer this section in every CMF they completed. Following Singleton and colleagues’ methodology, a binary outcome of ‘problematic drug use’ or ‘non-problematic drug use’ was developed for each woman at each time-point. 16 The table below indicates the total numbers of women who disclosed problematic drug use at each time-point. Table 16: Changes in problematic drug use Problematic drug use Time 1 Time 2 Time 3 Time 4 Time 5 Number of women who disclosed problematic drug use 24 13 11 8 1 As % of women who answered this question (n=91, 69, 56, 30, 14) 26.4% 18.8% 19.6% 26.7% 7.1% No problematic drug use was identified for 48 women for whom longitudinal data were available, or for a further 18, who provided this data at only one time-point. The remaining 33 women experienced problematic drug use at one or more time-points. A significant proportion of these women – 9 (27.3%) – had moved out of the problematic drug use category at the latest recorded time-point during their Pause intervention, after having been in that category at any previous time-point. Of these women, 2 reported having been in treatment at some point during their intervention. However, 6 women reported problematic drug use at the first and last recorded timepoints. A further 11 reported problematic drug use at one time-point, but did not provide 16 Singleton et al. (2006) http://webarchive.nationalarchives.gov.uk/20110218135832/rds.homeoffice .gov.uk/rds/pdfs06/rdsolr1606.pdf 38 longitudinal data on this question. Seven women had reportedly moved into the problematic drug use category at the latest recorded time-point, having not been in that category at the start of their engagement. 17 Criminal justice involvement Longitudinal analysis of Pause women’s involvement with the criminal justice system did not identify any trends toward either a reduction or an increase in involvement. For example, a longitudinal analysis of self-reported recent arrests (those taking place within the last quarter) found that 5 women had been recently arrested at Time 1, 3 further women at Time 2, and one woman at both Time 2 and Time 3. That no significant trends emerged from this data may be due to the relatively low numbers of women reporting involvement with the criminal justice system. Housing A longitudinal analysis was conducted to determine changes to women’s housing situations. For the purpose of this analysis, secure housing was treated as comprising legal tenancies and supported housing, and all other housing (living rough; sleeping on others’ floors/sofas; living in a refuge; living in a hostel or temporary housing; living with a partner; and living with family) was treated as insecure. The table below indicates the total numbers of women living in secure and insecure housing at each recorded timepoint. The longitudinal analysis of women’s housing situations yielded mixed results. However, most changes in housing status represented improvements. There were 111 Pause women who reported on their housing status at least once. Of these women, 68 (61.3%) were living in secure housing at each recorded time-point. The remaining 43 women (38.7%) were living in insecure housing at least at one time-point. Thirty women were living in insecure housing at each recorded time-point, and 2 women moved from secure to insecure housing during their engagement with Pause. However, 11 women recorded a change from insecure to secure housing. 17 As with disclosures of domestic violence incidents and higher risk drinking, interpretation of this data should bear in mind that increases in disclosures may indicate increasing trust in and openness with Practitioners. 39 Table 17: Changes in housing Housing Time 1 Time 2 Time 3 Time 4 Time 5 Number of women living in secure housing 66 49 45 17 11 As % of women who answered this question (n=110, 81, 67, 32, 14) 60.0% 60.5% 67.2% 53.1% 78.6% Number of women living in insecure housing 42 29 21 15 3 As % of women who answered this question (n=110, 81, 67, 32, 14) 38.2% 35.8% 31.3% 46.9% 21.4% Number of women living in ‘other’ (unspecified) housing 2 3 1 0 0 Engagement with services Longitudinal analysis of engagement with services demonstrated a general trend of increasing engagement, particularly with GP, housing, social care, drug and alcohol, mental health and education services, at least up until Time 3. (There was some degree of fluctuation in rates of engagement at Times 4 and 5, but this is likely to be due to variations in sample sizes.) Importantly, 6 women reported at Time 1 that they were not registered at any GP practice. While 3 of these women did not complete a further CMF, the remaining 3 had all registered with a GP by Time 3. Contraception Analysis of women’s use of contraception shows a marked increase between Time 1 and Time 2 in the proportion of women using an effective, long-acting, reversible form of contraception. Thereafter, rates of contraception use remain fairly stable at very high levels. As the table below illustrates, there was also a marked trend over time in favour of the implant, and a decrease in the proportion of women opting for an IUD or injections. 40 Table 18: Changes in contraception use Contraception Time 1 Time 2 Time 3 Time 4 Time 5 Number of women using long-acting contraception 83 82 64 31 14 As % of women who answered this question (n=115, 84, 67, 33, 14) 72.1% 97.6% 95.5% 93.9% 100% Number of women using the implant 51 52 45 25 13 As % of women using long-acting contraception (n=83, 82, 64, 31, 14) 61.4% 63.4% 70.3% 80.6% 92.9% Number of women using an IUD 15 16 10 5 1 As % of women using long-acting contraception (n=83, 82, 64, 31, 14) 18.1% 19.5% 15.6% 16.% 7.1% Number of women using injections 14 13 9 1 0 As % of women using long-acting contraception (n=83, 82, 64, 31, 14) 16.9% 15.9% 14.1% 3.2% 0.0% 41 How does Pause work? Pause seeks to reduce the numbers of children removed into care by ensuring that, by the end of their interventions, women are in a position from which they can build a positive future for themselves. The key strategic focus is therefore placed on enabling women, during an 18-month pregnancy-free period, to reach sufficient levels of wellbeing, resilience, and stability. Pause seeks to achieve this not only by engaging directly with women, but also by ensuring partner agencies offer sustainable, effective support to address women’s ongoing needs. Two central elements of the Pause model are, then, the delivery of intense emotional, psychological, practical, and behavioural support directly to women during an 18-month pregnancy-free period, and collaboration with partner agencies to improve accessibility and, ultimately, Pause women’s outcomes. In what follows, we set out the theory of change underpinning the Pause model, drawing on findings from both the impact and the process elements of the evaluation. We identify the key mechanisms through which change was achieved, both in women’s lives and within the services that support them, as well as the resource inputs and outcomes of these mechanisms. The theory of change is summarised in brief in Appendix D. An individual view of how women were supported to achieve change is given in Appendix E, which sets out case studies of 4 women. Influencing partner agencies Interviews with Pause and professionals from partner agencies during the early stages of the evaluation indicated that respondents hoped relevant services would work with Pause to adapt their practice and systemic protocols in order to improve Pause clients’ access to those services. Because many Pause women have complex and mutually-reinforcing needs, problems in one area can impact negatively on each of the rest, blocking women’s progress toward better outcomes. However, as discussed earlier in this report, women often faced considerable challenges attending appointments or engaging in other ways with vital services. Consequently, to achieve positive outcomes for Pause women, it was seen as crucial that services collaborate and innovate, at both operational and strategic levels, to meet women’s fundamental needs. As one housing professional noted: ‘Some form of housing, whether it’s local authority or housing association, really needs to be on board, because if you’re going to move forward with someone and they’re homeless then that’s a really difficult situation to be in. […] It’s all very well setting up counselling and getting all their benefits in place, but if they’ve actually got nowhere to live at the end of it then it’s no good’ (Agency 18, T1). 42 Direct advocacy Where service systems remained largely as they were pre-Pause, Practitioners reported that whether or not their clients got a positive outcome was often largely dependent on the particular professional dealing with the case. Several emphasised the difference that sympathetic and pro-active individuals on the frontline can make, but reported that professionals can sometimes (and for a range of reasons) prioritise quick wins in their work, rather than persevering with providing support to clients they perceive as difficult. A significant aspect of the Practitioner role was, therefore, the provision of direct advocacy for individual women within partner agencies. Practitioners reported that they spent a great deal of time explaining Pause women’s needs, and their own roles, to professionals they encountered on an ad hoc basis, and building up positive working relationships with key professionals dealing with their clients’ cases. In some cases, Practitioners who had previously worked in other services were able to draw on their existing professional networks, working with ex-colleagues or contacts to achieve good outcomes for their clients. Practitioners generally took a tenacious approach to direct advocacy, and this had a significant positive impact on many women’s access to services. One woman explained, for example, that, ‘[my Practitioner] got me a counsellor within 2 weeks, when I've been on the waiting list for 3 years’ (Client 60, T1). Improving the level of contact they had with their children was a high priority for many women. In interviews held toward the end of their interventions, a minority described how, once they had made certain changes, social care had agreed to increase the frequency or improve the quality of their contact with children: in the majority of these cases, women attributed the very welcome improvements, in large part, to the role their Practitioners had played in advocating for better contact on their behalf to social care professionals. Systemic change Nonetheless, it was also recognised that, within most services, professionals have a duty to adhere to the systemic protocols that regulate their practice. There were, therefore, limits to what advocacy at the operational level could achieve, particularly where established protocols recommended particular clients’ exclusion from, or de-prioritisation within, services. To illustrate, a range of professional respondents reported that a common systemic barrier to access is the requirement within several services that professionals close cases if clients miss multiple appointments. To address the need for systemic change, regular Pause Board meetings were held with the intention of facilitating buy-in from professionals working at a strategic level in relevant services. It was expected that Board members would spearhead sustainable system-level change and improvements in frontline practice. Board members included 43 managerial and, in some cases, operational professionals from a range of relevant services and agencies; namely, those from which flexible support is required if women’s fundamental needs are to be met. At most Practices, Boards included children’s and adult’s social care, health (including sexual and mental health), housing, criminal justice (including the courts, police and probation), drug and alcohol services, and third sector women’s organisations. Interviews with Board members indicated universal support for the model and broad aims of the Pause programme. However, attendance at, and engagement with, Pause Boards varied. Findings from interviews with Pause and allied professionals indicated that, at each Practice, a number of services were slow to enact the system-level change felt necessary. These services were mental health at most Practices, housing (and particularly local authority housing) at some Practices, and probation at one Practice. At some Practices, however, housing and other services made significant adaptations as a result of collaboration with Pause, thereby improving their ability to deliver better outcomes for women. For example, one Practice worked with the local Homelessness and Housing Options Manager, who agreed to personally consider the housing needs of Pause women. As a direct consequence of this pro-active, flexible approach to working together in innovative ways to meet women’s needs, some women were re-assigned to the top priority band. Similar changes were reported at another Practice: ‘We made special arrangements. We won’t be able to do it for everyone, but [we did it] to give them a chance and see how it goes. […] We changed the procedure around giving people housing priority, which would never have happened [without Pause]’ (Agency 18, T1). Findings from interviews with professionals and clients suggested that, across all Practices, third sector women’s organisations tended to be both understanding and supportive of Pause’s mission. Professionals within these agencies exhibited an in-depth understanding of how difficult women’s circumstances can be and a strong commitment to making adjustments to address them. As one explained, ‘we’re patient with the women of Pause, because it takes a lot of time to turn the ship around’ (Agency 16, T1). Pause professionals and clients across all Practices reported overwhelmingly positive experiences of working with sexual health services. Significant adaptations were made within these services to increase accessibility for Pause women. These included allowing women to drop in rather than requiring them to make appointments; fast-tracking clients once they arrived at clinics, to avoid long waits in waiting rooms; and even, at one Practice, making home visits to clients who wanted to access sexual health services but did not feel able to attend a clinic. As one sexual health consultant explained, ‘we try and facilitate whenever is convenient for the patient, so we don’t make the patient come in at our set times. […] We try and be very flexible’ (Agency 19, T1). 44 Following these system-level changes within services, driven by Board members, and more targeted advocacy, delivered by Practitioners, many women’s access to a range of services, and their associated outcomes, improved during their Pause intervention. Engaging women during a pregnancy-free period In addition to working with services to ensure women’s needs are met, Pause staff described aiming to encourage women to make their own changes during an 18-month pregnancy-free period. A key component of the Pause model, as described in Pause literature and in interviews with Leads and Practitioners, is that women are required to take effective contraception for the duration of their intervention. In response to a perception that women have chaotic lives, in which managing a contraception regime is difficult, Pause women are required to use a long-acting, reversible form of contraception: implants, IUDS, or injections. Pause staff stated that the purpose of this is to provide women with a space in which they do not have to think about falling pregnant, or struggle to maintain their contraception. Unless there are medical grounds for exemption, all Pause women are required to take up contraception early on in their engagement. Practitioners reported in interviews that the first few weeks of interventions are often very much focused on supporting women to make an informed choice to use contraception. At most Practices, Leads also collaborated with sexual health service leads to implement bespoke access arrangements for Pause women. The figures on contraceptive use suggest these aspects of the Pause model were effective in encouraging uptake and maintenance of contraception. As outlined in the section of this report on findings from baseline interviews with women, a large majority of women were confortable with this requirement. A minority had already been taking a long-acting contraceptive. Each woman was allocated to a single Practitioner for the duration of her 18-month intervention, excepting changes in staff. Throughout that time, Practitioners provided a programme of support that varied according to individual women’s needs. Practitioners’ caseloads were limited (usually to between 6 and 8 women), reportedly to enable them to invest the significant time required to deliver effective support to each woman. There was no fixed schedule of development to which all women had to adhere, nor any strict specification of how support should be delivered. This was reported to be deliberate, in order to allow Practitioners flexibility and creativity in their approaches. Although the outcomes that women worked toward varied, there were some core objectives that Pause clearly aimed to achieve for the entire cohort. In the most general terms, these are that women feel in control of their lives, are engaging in positive and healthy relationships, are able to manage emotional and psychological pressures, are exercising good self-care and independent living skills, are able to access support from services, have their physical and mental health needs addressed, are living in 45 appropriate and secure housing, and have low risk levels of alcohol and drug consumption. The forms of support Practitioners provided to women to achieve these goals can generally be categorised into 3 types: emotional and psychological, practical, and behavioural. In what follows, we consider the types of support in turn, examining how each is connected to changes women made throughout their engagement with the programme. Emotional and psychological support An essential element of the Pause programme is the provision of emotional and psychological support to women. As indicated earlier in this report, many Pause women lacked a support network of close friends and family with whom they could talk about the grief and trauma of losing children from their care, and find solace from the struggles of their day-to-day lives. A key struggle for many women was dealing with their identity as mothers and the trauma of child removals, both of which were often bound to profound feelings of shame. One woman described the effect on her psychological wellbeing: ‘I kept thinking to myself, ‘I don't have a right to be happy. I let my kids down. I don't have a right to smile’’ (Client 31, T3). In addition to this trauma, Pause women often faced ongoing daily struggles with a range of issues, including abusive or unhealthy relationships, insecure housing, and financial insecurity. Many described being overwhelmed by the feeling of constantly needing to firefight against a combination of pressures, resulting in significant stress, upset, anger, or despair. In order to cope with these pressures, many women had developed unhealthy coping mechanisms, including resorting to substance or alcohol misuse, or self-harm. For most women, then, the offer of focused and intensive psychological support from a Practitioner in whom they could confide, and who was focused on their wellbeing, was strongly welcomed. For a therapeutic relationship to be developed, however, trust had to be established. Women and Practitioners observed that, for some women, this happened very quickly. Others discussed in interviews how they overcame an initial sense of distrust, often following a long history of feeling badly let down by professionals or other important figures in their lives. The establishment of trust was enabled by certain key characteristics of Practitioners’ approaches to assertive outreach: it was evident that Practitioners showed tenacity in their commitment to the women; were emotionally available; had belief in, and empathy for, their clients; and were consistent in honouring their commitments. As one woman reflected: ‘[My Practitioner] will pester me if I don’t answer the phone. So, like, that’s really good. That’s someone that really wants you to do well in life. […] It’s good to trust again’ (Client 48, T3). As trust was established, Practitioners aimed to provide effective emotional and psychological support, often through discussion. A majority of women emphasised that 46 having someone supportive with whom they could talk about the trauma of losing their children had helped them to identify, and deal with, the emotional and psychological impact. One woman expressed her need for this kind of therapeutic support as follows: ‘I was up for it. Anything to help me. I need to deal with what I've been through. I just wanted help, and I always wanted a support worker. I just wanted someone to sit there and talk to me’ (Client 27, T1). Women and Practitioners also reported that discussion was often aimed at bolstering women’s confidence and self esteem, and identifying what women wanted for themselves in the future. One woman described the impact of engagement on her hopes for the future: ‘At the beginning, I was really low. And then Pause came along, and I saw a bigger and brighter future, where I can maybe get a job, have a better life. I've seen that there's going to be a future, whereas in the beginning there wasn’t a future for me. […] I don’t care what anyone says or thinks. It’s what I think, and I think I can get a job and be good at it, and that's thanks to [my Practitioner]’ (Client 23, T3). At times, it appears that Practitioners held difficult conversations with their clients, offering challenging support to enable them to see things from a new perspective. Evidently, a key issue for many women was the locus of responsibility for the removal of their children. A significant minority of women reported that talking with their Practitioner had enabled them to understand why their children had been removed, and to accept their share of responsibility for that outcome. One woman explained: ‘I’ve mentally come to terms with knowing that [the social worker] had a job to do. I was in a bad place, but she prioritised my kids’ needs, and that’s the best thing that anybody could’ve done’ (Client 6, T3). Practitioners reported that they sought to ensure these conversations maintained trust and open communication, rather than making women feel judged, or discouraging them from open discussion in future. As one Practitioner explained: ‘It’s about forming meaningful relationships that have an impact and that create space for them to make some changes or to think about things differently’ (Practitioner 7, T2). Where appropriate, discussions centred around how women might develop healthier relationships, and avoid abuse or exploitation. With the support of their Practitioners, some women were able to exit violent relationships. One woman explained: 47 ‘If it wasn't for [my Practitioner], I don't know where I'd be. Probably back living in the same domestic violence that I was in for 5 years, basically. I just needed that bit of encouragement’ (Client 26, T1). Notably, a small number of women reported that the support of their Practitioners had enabled them to feel strong enough, for the first time, to report current or historical abuse to the police. One woman explained, ‘me and [my Practitioner] went to the police station and we got [my mum’s ex-partner] arrested, for the abuse that he done to me as a child’ (Client 73, T3). This woman emphasised how her trust in her Practitioner had enabled her to take this step: ‘I didn’t want to tell no one, because I didn’t think no one would believe me. So, I told [my Practitioner], and she took it seriously, and the police arrested him’ (Client 73, T3). Another woman, who had been attacked by her ex-partner, took the case to trial, which resulted in the perpetrator’s conviction. Following this, the woman was supported to realise her goal of using her experience to help young people. She explained, ‘[my Practitioner] got me linked with a lady who works for [the local authority]. So I’ve been going into schools, and sharing my story’ (Client 77, T2). Practitioners reported that women sometimes experienced periods of crisis, during which the provision of emotional and other support often intensified, as Practitioners attempted to help their clients to stabilise and be safe. Importantly, it appears that emotional and psychological support was intended to help women develop sustainable, healthy coping strategies for long-standing trauma, ongoing daily pressures, and periods of crisis. As one Practitioner explained, ‘they’ll call me when they’re feeling low, rather than selfharming, or bottling it up, or having fisticuffs with their partner, or using [drugs]’ (Practitioner 13, T2). The Pause model also recognises the therapeutic benefit to women’s emotional and psychological wellbeing of positive experiences. A key element of the offer is therefore the provision of opportunities to try new, fun activities in a safe environment. A discretionary budget is provided to Practitioners to spend on individual women, which enables them to go on outings together, and experience an enjoyable day. Women have chosen, for example, to go with their Practitioners to the zoo, or for a manicure, or to a café. Women emphasised the benefit of being given the opportunity to choose something fun to do ‘just for themselves’: some described it in terms of introducing a sense of normality to their day, for the first time in a long time; others described the feeling of being reminded that it was possible to make such choices. At some Practices, Pause women were given the opportunity to engage in group activities with others in their cohort, including playing bingo, cooking, baking and cake decorating, jewellery making, arts and crafts, bowling, and go-karting. Not all women chose to participate, but, of those who did, several reported therapeutic benefits, including having fun, feeling a sense of achievement from learning new skills, and also gaining peer support from women who had shared some of the same experiences. In 48 some cases, Pause women appeared to develop new friendships with each other, often emphasising the importance of having a friend by whom they did not feel judged or condemned. It should be noted that Practitioners and Leads affirmed the need to exercise careful judgement in deciding whether, and which, group activities were appropriate for their cohorts, and highly developed skill to manage potentially volatile group dynamics. Interviews with women in the later stages of their interventions indicated that the emotional and psychological support they had received had made a significant difference to their psychological wellbeing. As one commented, ‘with the support from [my Practitioner] that I've had, it’s helped me get through it’ (Client 23, T3). Most women also spoke about a change in their own ability to manage emotionally and psychologically. One woman described the progress she had made: ‘I didn’t want to sort myself out. I was just happy to let myself die basically. But I’m realising, with [my Practitioner’s] help, and a lot of my family’s help that, yes, I’ve lost my children, and yes, it’s a big deal, but it’s not the end of the world. I still need to be a role model for them. I need to be me (Client 38, T2). Another reported: ‘I'm so happy, where I am now. I'm not where I want to be, but I'm not where I used to be, and I definitely don't want to go back there. I was just existing. It wasn't a life’ (Client 47, T2). Practical support In addition to emotional and psychological support, Practitioners evidently offered women significant help to resolve more practical issues. The focus here was often on setting women up, in an attempt to ensure they could end their engagement with a degree of stability in their lives. A key aim for many women was to develop good self-care and independent living skills, including budgeting, paying bills, and shopping for food and household products. To illustrate: in several cases, Practitioners supported women to go through all of their financial income and outgoings, to work out how much was available each month. With the introduction of monthly payments of benefits under Universal Credit, several women who had been used to managing their budgets for shorter periods reported that this had been beneficial. We also recorded instances of Practitioners directly providing women with the items they needed, spending part of their discretionary budgets on essential food, clothes, or household goods. In some cases, we noted that women were encouraged to take up opportunities for volunteering, education, or employment. Practitioners provided support to identify 49 suitable opportunities, complete and submit application paperwork, and prepare for interviews. Engaging in these opportunities was identified by these women, and their Practitioners, as enabling not only the development of skills, but also a sense of purpose and achievement, and hope for the future. Importantly, Practitioners also provided women with practical support in accessing services. For example, access to several services requires a passport or bank account, which some Pause women did not have at the start of their engagement. Practitioners aimed to ensure that all Pause women had these vital resources and so, where necessary, supported them through the administrative process of gaining them. For those women who found attending appointments difficult (either in general, or with particular services), practical support also regularly involved going with women to their appointments. A number of Practitioners reported that supporting women to get a correct diagnosis for previously undiagnosed mental health conditions, and cognitive or physical disabilities, was a high priority, not least because it entitled them to certain benefits. Undiagnosed learning difficulties also made some women vulnerable to various forms of exploitation and abuse (including sexual and financial abuse), as they had little, or no, formal support in place prior to Pause. As a result of practical support to access health services (and also, in some cases, direct advocacy within those services), several women received appropriate diagnoses that entitled them to improved support from health, social care, and housing services. More broadly, Practitioners provided women with crucial advice and guidance on how to navigate complex benefits and service systems; for example, supporting applications for housing, or helping women make representations to children’s social care for increased contact with their children. One woman explained, ‘because I’m dyslexic, I can’t read and write very well, and I messed up all my benefits’ (Client 84, T2). This woman was asked to make repayments that, she said, ‘added up to over £5000. And [my Practitioner] helped me write a letter, and she did a supporting letter, […] and I got a phone call saying that I don’t owe anything’ (Client 84, T2). Practitioners also often assisted women, including those with low levels of literacy, in writing letters for their children. We observed that this had a positive impact on some women’s ability to manage their loss. For example, one woman who had yearly contact with her child through a letterbox service noted: ‘[My Practitioner] was there with me, helping me, and that was a big, massive help. This year I felt a bit more confident in what to write. [My Practitioner] was the biggest help’ (Client 15, T3). Behavioural support Practitioners emphasised that, in much of their work with women, they aimed to model positive social interaction. To illustrate, outings with women were intended not only to 50 provide a good experience, but also to model ways in which women could have fun in safe environments. We also observed cases in which Practitioners of women with abusive partners modelled keeping safe, by, for example, explaining why the partner’s history of violence meant the Practitioner could not visit their home alone. Practitioners frequently reported that, at times, women could target their anger or frustration at their Practitioners. Practitioners reported that responding to this with calmness, empathy, and pragmatism not only helped the development of trust, but also modelled how to respond in a positive way to difficult situations. Much of Practitioners’ behavioural support reportedly focused on modelling positive engagement with professionals, and social care staff in particular. Many of the Pause cohort started their engagement feeling very angry toward social care. Some of these women reported that they found it difficult not to let that anger express itself in their interactions with social care staff, even when they recognised that would be selfdefeating. Practitioners encouraged women to work through exactly what they wanted to achieve from such interactions, and helped them identify what approach might enable the achievement of those aims. A significant minority of women described how their approach to engagement with professionals had changed over time, due in part to receiving this kind of support from their Practitioners. As one woman explained: ‘Just sitting down with [my Practitioner], and writing out what I want to say to the social worker, and having her there, made me calm down a lot and realise shouting at the social worker is not going to get you anywhere’ (Client 63, T2). Another woman reflected on the developments in her behaviour at meetings with her children’s social worker: ‘I almost kicked off so bad. I would have done some stupid things, but I managed to keep my calm, and say, ‘look my blood is getting really hot, so let me just say that I have to go.’ So there’s a big difference in me. [… Before Pause] I would have sat there and I would have kicked off until they just told me to leave’ (Client 70, T2). By the later stages of their engagement, most women explained that the cumulative effect of the intense, bespoke support they had received had been to enable them to consider their options more clearly, and make better choices to promote their wellbeing. In some cases, supporting women effectively required working to provide some support to their partners. Practitioners noted the importance of acknowledging that, when women’s partners have their own high level needs, supporting them to address those needs can benefit both parties. One of the partners we interviewed reported that his engagement with Pause had had a significant positive impact on him, and his 51 relationship: he had started counselling, and made contact with his children through a letterbox service. Resource requirements Effective direct advocacy, systemic change, and support to women requires the input of certain key resources. These include Practitioners, Coordinators, Leads, Board members drawn from partner agencies, and budgets for individual women, group activities, professional development and support, and office space. Findings from interviews with Pause professionals suggest a number of lessons for the deployment of these resources, which are outlined below. Highly skilled Practitioners are crucial to the success of the model. Effective Practitioners must be able to demonstrate belief in, and empathy for, their clients; take a nonjudgemental approach; demonstrate tenacity in their commitment to supporting their clients; be consistent with and honour commitments to clients; have extremely advanced interpersonal, communication, and relationship-building skills; be creative and solutionfocused; have professional expertise in relevant areas; and have the high levels of personal resilience needed to thrive in the job. Practitioners emphasised the importance of ongoing training and formal opportunities for practice development. While some expected a more thorough, formal induction package at the start of the pilot, and greater opportunities for ongoing training, many reported benefits to their practice from the training they had received. Practitioners’ Learning Forums, usually held quarterly and attended by staff from multiple Pause Practices, were also generally well-received, and viewed as an opportunity to build confidence and develop professionally by sharing learning around good practice. Practitioners repeatedly highlighted that their effectiveness and resilience were very much affected by the quality of supervision provided to them. Clinical supervision to address Practitioners’ own mental health needs was noted as particularly crucial to the avoidance of burn-out. As one Practitioner noted, ‘it gives us a chance to sound off’ (Practitioner 4, T2). Practitioners also emphasised that peer group supervision enabled them to draw on others’ different and complementary skills and experiences, while oneto-one supervision with managers provided guidance or reassurance about practice. Pause staff also reported the importance of having a stable, fully staffed, and supportive team. Across Practices, it was emphasised that the entire team took a degree of collective responsibility for each woman. When Practitioners took annual leave, for example, others (including Coordinators and Leads) continued to provide a point of contact and support for their clients. Where Practices had no Practice Leads for a time 52 due to temporary staffing shortages, Practitioners generally reported a negative effect on team functioning and morale, resulting from the absence of managerial support. Some Practitioners reported that it would have been helpful to have had a more extensive range of Pause-specific materials and tools, both to guide reflective activities with women, and to capture forms of progress that are relatively difficult to measure numerically. These forms of progress include, for example, the development of social skills, such as making eye contact, or talking to strangers. Some Practices had, however, either begun developing, or developed and begun trialling, new tools for these purposes. Several staff discussed the need to develop these further in light of learning from the pilot. With regard to the length of interventions, all Pause staff judged that 18 months provides sufficient time for some women to make significant, sustainable positive changes. However, several Practitioners reported that some women needed longer term intensive support. While, as indicated above, some women were optimistic about their futures, some reported feeling anxious or fearful about their interventions coming to an end. As one Coordinator explained: ‘We get some women saying, ‘you can’t just leave us.’ And there are other women who are really accepting of it: ‘when you’ve gone, I’m going to be able to do this.’’ (Co-ordinator 2, T2). Of those who were more anxious, one woman stated: ‘I think they’re doing a great job. I don’t want them to leave me in September. I don’t know if I’m ready. Just the bond I’ve got with [my Practitioner]. It’s a bond I’ve never had’ (Client 36, T2). Some talked in more explicitly fearful, desperate, despairing terms of the oncoming end of their interventions: ‘if it wasn't for them helping me in a lot of ways… Pause is finished with me next week, and where does that leave me?’ (Client 18, T2). Locating Pause teams in appropriate buildings was a significant challenge at several Practices. Some teams were based within local authority buildings shared with social care. At one Practice, this reportedly brought some benefit, including facilitating closer relationships between Pause and social care staff. As the Practice Lead reported: ‘It has benefitted us because we can just literally stand up and walk across the corridor and find a social worker or find the adoption team. So we’ve been able to integrate ourselves better into the service’ (Practice Lead 7, T3). Overall, however, staff generally felt that the costs outweighed this benefit. As many Pause women had very negative associations with social care buildings, and did not feel 53 they were safe spaces, they would not enter them by choice. Lacking a welcoming space for women to drop in to, Practitioners at these Practices generally met women out in the community. While going to cafés and other public places with their Practitioners could be a very positive experience for some Pause women, the lack of client-friendly Pause premises meant that some very difficult, personal conversations took place in public settings. It also had a financial cost. Conversely, the benefits at one Practice of locating the team within the premises of a local third sector women’s organisation were repeatedly emphasised by all members of that team. As the Practice Lead highlighted: ‘The women are happy to go there. They’re being treated with respect when they arrive. […] It’s been nice having premises that are non-stigmatising, away from social care offices.’ (Practice Lead 1, T2). Pause professionals at other Practices also recognised the benefits of this arrangement: ‘It would make a difference to have a situation similar to theirs. Somewhere where the women can drop in, have privacy, and feel comfortable to open up’ (Practitioner 12, T1). 54 A cost benefit analysis of Pause A fiscal cost benefit analysis was conducted, focussing on savings attached to reductions in the number of child removals during the Pause intervention period. The analysis includes all 125 women in the Pause evaluation cohort, including women who have had one child removed from their care, and those who have had two or more children removed. The analysis indicates that the full costs of delivering Pause to the cohort of 125 women are likely to be offset by savings to local authorities within 2 to 3 years, with estimated net cost savings of between £1.2 million and £2.1 million per year after the 18month intervention period. Further potential cost savings from reductions in levels of domestic violence, harmful alcohol use, and Class A drug use are identified in Appendix F. These savings are estimated to total between £628,207 and £732,005 per year. The total cost of delivering Pause for a period of 18 months was £2,525,230, which equates to £1,683,487 for a 12 month period. This includes only steady state costs, such as staff running costs, office costs, and individual budgets that were available for each woman. Set-up costs, strategic management costs, and in-kind costs have been excluded from the analysis. Given that 125 women were supported, the cost per woman supported was £20,202 over 18 months, equivalent to £13,468 for a 12 month period. The analysis uses the impact measure of reductions in pregnancies as a proxy measure for reductions in care proceedings resulting in child removals. The yearly cost savings attached to each child removal avoided are estimated at £57,102. This is comprised of £52,676, which is the mean yearly cost of a child in care across a range of placement types (excluding ongoing wider costs to social care associated with looked after children), plus other associated costs. 18 The mean is used because it is unknown what the outcome of care proceedings would have been for any avoided proceedings (for example, some may have resulted in adoption, and others in foster care, or special guardianship orders). Associated costs comprise, initially, a one-off cost of £1,151 for a child protection core assessment. 19 A conservative figure of £3,275 for the legal cost per care proceeding has also been included in initial one-off costs, which excludes legal aid costs for parents and families. It is based on the mean average figure for legal costs from two sources. 20 As it is unknown what degree of involvement from social care would have been required in each case following removal, yearly associated costs are held stable to account for the subsequent costs of (differing degrees of) continuing involvement from social care after care proceedings have occurred. 18 New Economy (2015) http://neweconomymanchester.com/our-work/research-evaluation-cost-benefitanalysis/cost-benefit-analysis/unit-cost-database 19 Ibid. 20 Plowden (2009) http://dera.ioe.ac.uk/1044/1/court-fees-child-care-proceedings.pdf and HM Treasury (2014) https://www.gov.uk/government/publications/supporting-public-service-transformation-cost-benefitanalysis-guidance-for-local-partnerships 55 The tables below provide the results of the cost benefit analysis. This draws on both of the modelled estimates of the impact of Pause on pregnancies to produce a range of possible overall costs and savings over a 5 year timeframe. Table 19: Cost savings from removals avoided (Model 1) Cost savings (Model 1) Year 1 Year 2 Year 3 Year 4 Year 5 Yearly cost of Pause £ 1,683,487 841,743 0 0 0 Estimated cumulative number of removals avoided (proxy measure) 14.066 21.1 21.1 21.1 21.1 Estimated yearly cost savings due to removals avoided £ 803,197 1,204,852 1,204,852 1,204,852 1,204,852 Estimated yearly net cost savings -880,290 363,109 1,204,852 1,204,852 1,204,852 Estimated discounted* yearly net cost savings £ -880,290 350,830 1,124,742 1,086,708 1,049,959 Estimated yearly cumulative cost savings (undiscounted) £ -880,290 -517,181 687,671 1,892,523 3,097,376 Net present value of programme £ 2,731,949 * In accordance with HM Treasury guidelines, the annual discount rate applied is 3.5%. 21 21 HM Treasury (2013) https://www.gov.uk/government/publications/green-book-supplementary-guidancediscounting 56 Table 20: Cost savings from removals avoided (Model 2) Cost savings (Model 2) Year 1 Year 2 Year 3 Year 4 Year 5 Yearly cost of Pause £ 1,683,487 841,743 0 0 0 Estimated cumulative number of removals avoided (proxy measure) 24.53 36.8 36.8 36.8 36.8 Estimated yearly cost savings due to removals avoided £ 1,400,712 2,101,354 2,101,354 2,101,354 2,101,354 Estimated yearly net cost savings -282,775 1,259,611 2,101,354 2,101,354 2,101,354 Estimated discounted* yearly net cost savings £ -282,775 1,217,015 1,961,636 1,895,301 1,831,208 Estimated yearly cumulative cost savings (undiscounted) £ -282,775 976,836 3,078,189 5,179,543 7,280,896 Net present value of programme £ 6,622,385 * In accordance with HM Treasury guidelines, the annual discount rate applied is 3.5%. 22 22 HM Treasury (2013) https://www.gov.uk/government/publications/green-book-supplementary-guidancediscounting 57 Implications and recommendations The evaluation found that Pause women, as a cohort, had significant, multiple, and often mutually-reinforcing needs. Women began their engagement with the programme experiencing high levels of grief, depression, anxiety, domestic violence and abuse, substance and alcohol misuse, and insecure housing. While many of the problems they faced had been directly linked to removal of their children, women overwhelmingly reported that the support provided to them by services once their children were removed had been wholly inadequate to address their needs. Problems therefore not only remained unaddressed, but were compounded by the profound psychological trauma of having their children removed from their care. Answers to the central evaluation questions To what extent is the Pause model effective in reducing the numbers of children removed from women’s care? Counterfactual impact analysis demonstrates that Pause was extremely effective in achieving its stated aim of reducing the number of pregnancies experienced by women during their 18-month interventions. While 2 women became pregnant during their time with Pause, it is estimated that between 21 and 36 pregnancies would have occurred, had the cohort of 125 women not been engaged in the programme. Given the women’s histories, these pregnancies would have been likely to have resulted in removals. How cost-effective is the Pause programme? Our analysis indicates that the full costs of delivering Pause to the cohort of 125 women are likely to be offset by savings to local authorities within 2 to 3 years, with estimated net cost savings of between £1.2 million and £2.1 million per year after the 18-month intervention period. To what extent does engagement with the Pause programme have a positive impact on women’s wellbeing, resilience and stability? Findings from quantitative and qualitative analyses demonstrate that Pause had a significant, positive impact on many women’s outcomes. The cohort’s access to, and engagement with, services, including GP, housing, and substance misuse services, generally increased over time, and was associated with improved outcomes for some women. 58 While, toward the end of their interventions, some Pause women were found to still be experiencing insecure housing, alcohol and substance misuse, and domestic violence, a significant proportion had moved to secure housing, reduced use to lower risk levels, and safeguarded themselves against violence and abuse. Interviews with women revealed significant improvements to self-reported confidence and self-worth. Women also reported the benefit of learning new skills, behavioural reponses, and coping mechanisms, which had helped them address both past traumas and ongoing, day-to-day challenges more effectively. While many started their interventions with limited aspirations for the future, by the end, many Pause women had formulated new goals, and were taking steps toward their achievement. These goals included entering employment, education or volunteering. How do individual elements of the Pause programme relate to its impact? Analysis of the processes through which changes were achieved indicates a number of key mechanisms of change. These were the provision of an intensive, bespoke programme of support addressing women’s emotional, psychological, practical and behavioural needs; direct advocacy to influence professional practice within partner agencies; and work at the strategic level to increase Pause women’s access to partner agencies by adjusting systemic protocols. Pause provides this within a pregnancy-free 18-month period. Enabling a pregnancy-free period is a core aspect of the programme’s theory of change, and evidence from this evaluation suggests that a pause in pregnancies was an important enabler of observed changes. That each mechanism of change operated simultaneously was often fundamental to women’s progress, enabling problems to be tackled holistically. To illustrate, reductions in women’s substance misuse were seen as Pause Practitioners provided emotional and psychological support to address the traumas and challenges that motivate misuse and promote healthy coping mechanisms; practical support to attend appointments at substance misuse services; behavioural support to improve the quality of engagement with professionals at those services; and direct advocacy to discourage professionals from de-prioritising or closing women’s cases; while Board members collaborated to improve service accessibility. What factors enable or hinder the achievement of Pause’s aims? The central factors enabling the achievement of Pause’s aims include having in place highly skilled, committed and resilient Practitioners working intensively with low 59 caseloads of up to 8 women; a full complement of Practice staff, including Coordinators and Leads; flexibility in how the intervention is delivered to each woman; effective and ongoing training, management and supervision for Practitioners; active Pause Boards with participation from professionals in partner agencies; and independence from social care services, as a non-statutory, voluntary programme. Recommendations On the basis of findings from this evaluation of the impact of Pause, and of the processes through which impact was achieved, we make a number of key recommendations. Commissioning Given our findings that Pause has had a significant, positive impact on the lives of a large proportion of women who engaged with programme, and is highly likely to result in substantial cost savings over time, there is good reason to continue, and expand, provision of the service, provided other key recommendations are met. Programme delivery The provision of support and advocacy to women by highly skilled, dedicated Practitioners is key to effecting change. Meaningful engagement with women requires Practitioners to be empathetic, consistent, tenacious, resilient, creative, and solutionfocussed, to have extremely advanced interpersonal, communication, and relationshipbuilding skills, and to have highly developed professional expertise in relevant areas. The flexibility of the programme, which enables Practitioners to use their professional judgement and skill in tailoring their approach to meet the unique needs of individual women, should be maintained. Limits to Practitioners’ caseloads should remain at 6 to 8 women. This is necessary to allow for the intensity of work that is required to establish trusting relationships and support women to make sustainable changes. Practitioners should continue to be equipped with a budget to spend on each individual woman. This facilitates the delivery of key elements of the support package, including therapeutic activities, and essential items, such as furniture or passports. Management and strategic planning To ensure continuous professional development, maintain wellbeing, and avoid burn-out, Practitioners should receive effective, ongoing training, managerial support, and supervision, including clinical supervision. 60 Highly skilled Practice Leads should be in place at all times, to provide Practitioners with appropriate opportunities for effective advice, support, supervision, and professional development, and also to ensure Practitioners are safeguarded in their work. Inter-agency collaboration at the strategic level, to improve access protocols within partner agencies, is necessary to ensure services make the adjustments required to meet women’s fundamental needs, during and after their interventions. Pause Boards should continue to foster active participation from key decision-makers within partner agencies at every Practice. Efforts should focus, in particular, on implementing adjustments to improve access to health (including GP and mental health) services, housing services, and alcohol and substance misuse services. Pause should maintain its independence from social care services, and its status as a non-statutory, voluntary programme. The evidence suggests that women would be less likely to both begin and sustain meaningful engagement with Pause, if Practices were located within social care teams. For similar reasons, Pause Practices should avoid locating their offices within local authority buildings. Pause materials A comprehensive induction package for Practitioners would facilitate the replication of Pause in other areas. Pause should continue to develop and trial materials and tools for the purpose of guiding reflective activities with women, and monitoring women’s progress. Tools for monitoring progress should be designed to capture qualitative, as well as quantitative, outcomes. Evaluation The evaluation identified several areas that merit further evaluation. Additional longitudinal evaluation, over a longer period of time, is required to identify the mediumand long-term impact of Pause on women and on the number of children removed into care. In particular, longitudinal tracking of individual women is required to ascertain whether changes are sustained beyond the 18-month intervention period. Also of importance is whether 18 months represents the optimum length for Pause interventions, or whether greater flexibility might enable more women to sustain positive changes. Further evaluation should also be undertaken to assess the relative benefits of intervening at earlier stages in women’s lives, before multiple removals have occurred. There are significant opportunities to learn more about the kinds of support vulnerable care leavers might need to avoid entering a cycle of child removals. Also of interest is whether fathers who have experienced child removals might benefit from a man-centred equivalent to the woman-centred Pause model. Finally, we note the breadth of Pause 61 women’s previously unmet needs, from undiagnosed learning disabilities, to insecure and unsafe housing. Further research should be undertaken to identify how gaps in service provision to women with complex, and often high-level, needs can best be addressed. 62 Appendix A: Evaluation limitations The scope of this evaluation was expansive. A total of 202 participants were interviewed individually, most at multiple time-points, and 105 of these were women who are sometimes described as ‘hard to reach’ or ‘hard to engage’. 23 Certainly, many on the Pause programme face difficulties in attending appointments. To ensure high rates of participation in the research, the evaluation team worked closely with Practitioners to ensure that women were fully informed about the evaluation and supported to take part. This often involved Practitioners identifying suitable places to conduct interviews (as some women had negative experiences of some services’ buildings), and accompanying women to interview. In a very small number of cases, women stated that they would be uncomfortable being interviewed alone and, in these cases, Practitioners stayed with them throughout the interview. Many women in the research cohort were vulnerable for a range of reasons, and the research team therefore took a highly sensitive, womancentred approach to interviewing practice. It was also emphasised to women that participation in the research was voluntary, and an opportunity for them to have their say on services. Generally, women reported very positive experiences of their involvement in the research. Several emphasised the emotional value of taking part in the research, stating that they wanted to help other women by sharing their experiences. Certain limitations apply to each of the 2 models used to estimate the impact of Pause on pregnancies. Both produce estimates conditioning on pregnancy histories and age at intervention, but it was not possible to condition on further variables, as sufficiently complete data were unavailable. The first model uses data on comparison women collected from social care records, which did not always provide full and accurate data about women’s circumstances. Therefore, the model assumes that the Pause women are similar in relevant respects to the comparison women, after conditioning on pregnancy history and age at intervention. However, the comparison women were based in different localities, and may also have differed in a variety of other characteristics. It was also assumed that their hypothetical intervention took place several years earlier. That dates of births were missing for a small proportion of children, due to the locking of case files after adoptions, may have biased the final estimate downwards in this analysis, by reducing the observed number of pregnancies for the comparison group. The second model assumes that the pattern of fertility among Pause women remains static over time, conditioning on pregnancy history and age at intervention. As pregnancy 23 Boag-Munroe, G. and Evangelou, M. (2010) ‘From Hard to Reach to How to Reach: A Systematic Review of the Literature on Hard to Reach Families’, in Research Papers in Education, Vol. 27, No. 2, Pp. 209-239. 63 rates decreased in this group over time, this may have biased the final estimate upwards in this analysis. As with the first model, there are also likely to have been other aspects of the women’s experiences which were different during the historical period. Further, the model assumes that Pause would have intervened with these women, based on an earlier perspective of their pregnancy histories. At 2 pilot Practices, which work with women with one child removed, this may not have been the case. Further limitations apply to the analyses of data from women’s CMFs. First, as CMF completion rates declined toward the end of women’s interventions, some changes may not have been captured in the analysis. Second, it is well documented that levels of domestic violence, harmful alcohol use, Class A drug use, and criminal justice involvement tend to be under-estimated when based on self-reported measures. Further, in some cases, a longitudinal increase in disclosures may indicate the development of an increasingly open relationship between women and Practitioners, rather than a genuine increase in the frequency of incidents. Finally, it cannot be assumed that Pause women would not go on to experience further domestic violence, higher risk drinking, Class A drug use, and criminal justice involvement after the period recorded. Estimates of changes in these outcomes should therefore be treated with caution. For the same reasons, estimates of potential cost savings resulting from reductions in domestic violence, higher risk drinking, and Class A drug use should also be treated with caution. Moreover, and importantly, owing to the lack of a counterfactual group for this analysis, estimated cost savings are not proven to be the result of Pause. Where Pause staff were not interviewed at Time 2 or Time 3, this was usually due to staff leaving their posts. Further, women began their engagement with Pause at different stages of the evaluation period. It was therefore not possible to conduct interviews or CMFs in the middle or final stages of some women’s interventions, as these went beyond the evaluation period. 64 Appendix B: Pause’s own baseline data Pause gathered its own baseline data on women’s presenting issues from social care case files, during early scoping exercises. These data suggest that 35%, 36%, 36%, 55%, 63%, 69%, and 75% of women at each of the 7 Practices, respectively, had experienced issues with their mental health. Three Practices also recorded baseline data on diagnoses of personality disorders, which suggest that 5%, 10%, and 20% of women at those Practices had a diagnosis. At each Practice, 29%, 45%, 60%, 65%, 80%, 81%, and 85% of women, respectively, were recorded as having experienced some form of domestic violence and abuse at some point in their lives. At 6 Practices, 10%, 23%, 34%, 45%, 43%, and 57% of women, respectively, were recorded as misusing alcohol at the start of their engagement with Pause. At the remaining Practice, 34% of women were recorded as having issues with alcohol or drug misuse. At 5 Practices, 10%, 15%, 26%, 59%, and 62% of women, respectively, were recorded as having issues with Class A drug use. At a further Practice, 57% of women were recorded as having issues with substance misuse, and, at the remaining Practice, 34% of women were recorded as having issues with drug or alcohol misuse. At each of the pilot sites, 10%, 15%, 25%, 26%, 33%, 35%, and 56% of women, respectively, were recorded as having had involvement with the criminal justice system. Finally, data gathered at 3 Practices suggests that 15%, 23%, and 34% of women, respectively, had experienced previous, or current, homelessness. At one Practice, 27% of women were recorded as having had experience of transient lifestyles. 65 Appendix C: Analysis of impact on pregnancies Statistical analyses were conducted to identify the effect of the Pause intervention on pregnancies, using statistical models to control for potential bias using regression. Two statistical analyses were conducted. The first compared the Pause women to a similar, comparison group of women, who were not receiving any similar intervention. The data were collected from the social care database systems of 2 local authorities. The women were identified based on their similarity to the women that Pause aims to intervene with. The total number of women in the comparison group was 134. As these women did not experience an intervention, a hypothesised, dummy intervention start date was used as a temporal marker. Pregnancies occurring in the 18 months following this marker were assumed to have occurred during the dummy intervention follow-up period. This provided a way of estimating the number of pregnancies avoided by Pause women during their 18month intervention follow-up period. The second analysis compared Pause women to themselves at an earlier time. Essentially, pregnancies at an earlier time were used to model the number of pregnancies avoided at a later time due women’s engagement with Pause. The total number of women in the Pause group was 125. The data on these women were collected directly from the Pause sites. Data Both analyses modelled the number of pregnancies avoided due to women’s engagement with Pause. The available data included variables on location; the age of the mother at intervention (based on their date of birth); the assumed dates at which they became pregnant (based on dates of birth of their children); and the date at which they began intervention (or dummy intervention, in the case of the comparison group). The children’s dates of birth were backdated by 9 months to create an assumed pregnancy/conception date. This approach does not take account of differences in term due to premature or overdue births. Likewise, any aborted or miscarried pregnancies were not included. Nevertheless, it provides a reasonable estimate of pregnancies relevant to potential future care proceedings for these women. In the Pause group, women engaged with intervention across dates ranging from April 2015 to December 2016. The dummy intervention date for the comparison group was set at 1st June 2011. This was to ensure sufficient time had passed for relevant court proceedings to have concluded and data to have been incorporated into social care databases. For all groups, a follow-up period of 18 months was used. Except where evidence of a pregnancy was available, the Pause women were assumed to have not become pregnant during follow-up (even where the follow-up period had not yet concluded). Where comparison group women were already pregnant at the dummy intervention date, the intervention date was shifted to occur immediately after the birth of 66 the relevant child. This was to reflect the Pause enrolment criterion requiring that women are not pregnant. In some cases, there was incomplete information on the dates of birth of children (but complete information on total number of children born). There were 6 women in the Pause group and 27 women in the comparison group for which this was true. The main reason for this was the locking of a social care database file once a child had been adopted. These records were nonetheless retained in the analyses, as in most cases this applied to a single child for women who had experienced multiple removals. Descriptive analysis Descriptive comparisons were made between the groups based on age at intervention, the number of pregnancies prior to intervention, and pregnancy rates prior to intervention. Figure 1: Pregnancies prior to intervention Pause Frequency Comparison Pregnancies prior to intervention The age profiles were broadly similar in each group. The figure below shows the total number of pregnancies over the whole period prior to intervention for each group. As with the age at intervention, the profiles were broadly similar. The average total number of pregnancies prior to intervention was 3.02 in the Pause group and 2.29 in the comparison group, but there was a high degree of positive skew to the distributions, with more women having fewer pregnancies than this. In the Pause group, 56 women had one or 2 pregnancies, 45 had 3 or 4 pregnancies, and 24 had 5 or more pregnancies prior to intervention. For the comparison group, 82 had one or 2 pregnancies, 32 had 3 or 67 4 pregnancies, and 13 had 5 or more pregnancies prior to the dummy intervention start date. There were also 7 women who had 0 pregnancies before intervention in the comparison group. The figure below shows the pregnancy rate per woman per year, limited to the 5 year period prior to intervention for both groups. The rates vary between 0.11 and 0.28, and appear to be decreasing near the point of intervention. The error bars of the estimates overlap considerably, indicating that there is no statistically significant difference between the two groups. Figure 2: Pregnancy rate per woman per year over the 5 years prior to intervention, by group ____ Pause ____ Comparison Model 1: Pause vs. comparison group The table below shows the number of pregnancies in the follow-up period for each group. In the Pause group, 2 women had a pregnancy in the follow-up period. In the comparison group, 22 women became pregnant, one of these women twice. Table 21: Pregnancies in follow-up period, by group Pregnancies Pause Comparison 0 123 112 1 2 21 2 0 1 68 These figures suggest a pregnancy rate of 0.11 per woman per year in the comparison group during follow-up. Hypothetically, if the Pause women experienced similar rates of pregnancy, 21 pregnancies would have occurred during the follow-up period. Thus, these can be regarded as naïve estimates (due to varying age profiles, histories, localities, etc.) of the possible number of (carried to term) pregnancies avoided due to Pause over the 18-month follow-up period. A Poisson count regression model was estimated, to control for potential confounding by the various characteristics of the women in each group. The resulting estimates are shown in the table below. The model was used to predict the numbers of pregnancies that would have occurred if the Pause women had been in the comparison group. The model returned a prediction of 21.1 pregnancies in the 18-month follow-up period. This equates to an aggregate rate of 0.113 pregnancies per woman per year during follow-up. However, calculation of the mean using individual-level estimates on the log-scale produced an average rate of 0.080, with a confidence interval of 0.069 to 0.093. Table 22: Model 1 (Pause vs. comparison) Estimates from regression model of pregnancies in 18-month follow-up period Coef. Std. Err. P-value Lower CI Upper CI Pregs. 1yr prior to intervention 3.1201 0.8745 0.0000 1.4062 4.8341 Pregs. 2-3yrs prior to intervention 1.2916 0.5793 0.0260 0.1562 2.4270 Pregs 1yr * pregs 2-3yrs prior -15.0629 0.9132 0.0000 -16.8527 -13.2731 Group (Pause early intervention) -8.9524 2.6355 0.0010 -14.1179 -3.7868 Group (comparison) 22.4480 7.8960 0.0040 6.9722 37.9238 Group (Pause early int) * pregs 1yr prior -2.7062 1.1778 0.0220 -5.0146 -0.3979 Group (comparison) * pregs 1yr prior -4.0567 0.8770 0.0000 -5.7755 -2.3378 Group (Pause early int) * pregs 2-3yrs prior -1.1764 0.7114 0.0980 -2.5708 0.2179 Group (comparison) * pregs 2-3yrs prior -0.6675 0.7506 0.3740 -2.1386 0.8037 Group (comparison) * pregs 1yr * pregs 2-3yrs prior 15.0955 1.5783 0.0000 12.0022 18.1889 69 Coef. Std. Err. P-value Lower CI Upper CI Total pregnancies prior to intervention 10.7521 7.3715 0.1450 -3.6958 25.2000 Age at intervention 1.1777 0.3391 0.0010 0.5129 1.8424 Age at intervention * total pregs prior to int -0.4403 0.3007 0.1430 -1.0297 0.1492 Group (comparison) * total pregs prior to int -13.0426 7.3656 0.0770 -27.4790 1.3937 Group (Pause early int) * age at intervention -0.1618 0.0723 0.0250 -0.3034 -0.0201 Group (comparison) * age at intervention -0.7570 0.3254 0.0200 -1.3948 -0.1191 Group (comparison) * total pregs prior * age at int 0.5032 0.3006 0.0940 -0.0859 1.0922 Age at intervention squared -0.0104 0.0022 0.0000 -0.0146 -0.0062 Constant -26.8389 7.9511 0.0010 -42.4227 -11.2551 Note: Poisson count regression model estimates; reference category for group is Pause without early intervention (early intervention Pause sites were Hackney and Greenwich); standard errors clustered by locality (9 clusters); pseudo R-square=0.3049; N=259; all asterisks indicate model interaction terms. Model 2: Pause vs. Pause An additional analysis was conducted for the women in the Pause group. Rather than compare the Pause group to another group of women, they were instead compared to themselves at a previous time. A model was estimated based on a pseudo follow-up period of 18 months prior to the women being enrolled on Pause (taking into account that they could also not be pregnant at this time). Thus, the intervention date was set as 18 months (plus an additional 9 months) prior to their actual intervention date. This enabled the model to be used to estimate the association between the control variables and the number of births during this pseudo follow-up period. 70 The table below shows the number of pregnancies in the pseudo follow-up period. 38 women became pregnant, one of these women twice. This equates to a pregnancy rate of 0.20 per woman per year during follow-up. Table 23: Pregnancies in pseudo follow-up period Pregnancies Pause 0 87 1 37 2 1 Pseudo versions of age at intervention and pregnancies prior to intervention were calculated and incorporated into the Poisson count model estimating pseudo follow-up pregnancies. The resulting estimates are shown in the table below. The model was used to predict the number of pregnancies that would have occurred if the real-time Pause women had behaved similarly to themselves during the pseudo follow-up period. The model returned a prediction of 36.8 pregnancies in the 18-month follow-up period. This equates to an aggregate rate of 0.196 pregnancies per woman per year during follow-up. However, calculation of the mean, using individual-level estimates on the log-scale, produced an average rate of 0.181, with a confidence interval of 0.169 to 0.194. Table 24: Model 2 (Pause vs. Pause) Estimates from regression model of pregnancies in 18-month follow-up period Coef. Std. Err. P-value Lower CI Upper CI Pregs. 1yr prior to intervention -0.9141 0.5223 0.0800 -1.9378 0.1096 Pregs. 2-3yrs prior to intervention -0.3180 0.3117 0.3080 -0.9290 0.2929 Pregs 1yr * pregs 2-3yrs prior 1.7088 0.4358 0.0000 0.8546 2.5629 Total pregnancies prior to intervention 0.3419 0.4054 0.3990 -0.4527 1.1365 Age at intervention -0.3285 0.2071 0.1130 -0.7344 0.0773 71 Age at intervention * total pregs prior to int -0.0149 0.0121 0.2200 -0.0386 0.0089 Age at intervention squared 0.0062 0.0038 0.1050 -0.0013 0.0137 Constant 3.3240 2.4398 0.1730 -1.4579 8.1059 Note: Poisson count regression model estimates; standard errors clustered by locality (7 clusters); pseudo R-square=0.0325; N=125; all asterisks indicate model interaction terms. 72 Appendix D: Summary of the Pause theory of change Target outcome Women are able to build a positive future for themselves and avoid further child removals. Intermediate outcomes Women have wellbeing and resilience • • • • • • • • • • Services address needs Women feel in control of their lives. Women are engaging in positive, healthy relationships. Women are able to manage emotional and psychological pressures. Women are exercising good self-care and independent living skills. Women are able to access support from services. Women’s physical health needs are addressed. Women’s housing is stabilised. Women’s alcohol and drug consumption is at low risk levels. Women are able to maintain reproductive agency. Women are able to access appropriate employment, education, and volunteering opportunities. • • Services implement protocols that facilitate Pause women’s access. Key professionals understand and respond to the needs of individual Pause women. Outputs Engaging women during a pregnancy-free period Influencing services Practitioners provide women with: Emotional and psychological support • Talking issues through to promote wellbeing and self-worth, deal with trauma, and develop healthy coping mechanisms. • Providing opportunities for positive experiences, including outings, classes, and group activities. Practical support • Accessing a wide range of services and opportunities. • Developing practical self-care and life skills. Behavioural support • Modelling positive engagement with professionals. • Modelling positive social interaction, and healthy relationships with friends, family, and partners. Board members • Work at the strategic level to enact changes to service protocols. Practitioners • Work at the operational level to provide direct advocacy for individual women within services. • • • Inputs Funded roles: Practitioners, Coordinators, and Managers. Non-funded roles: Board members drawn from partner agencies. Funding: budgets for office space, professional development, women, and group activities. 73 Appendix E: Case studies Please note that all case studies use pseudonyms. Jade Jade began her engagement with Pause in early summer 2015, while in her early thirties. She had experienced 4 children being removed from her care. Two were adopted, while 2 were in the care of a paternal grandmother. Case study participants described Jade as self-conscious, negative, lacking in confidence and always expecting the worst. Jade had suffered sexual abuse as a child from a family member who lived locally. She had also experienced domestic violence in childhood and adulthood. Although she presented as confident, Jade explained that she had low self esteem and was very insecure. She reported that she was struggling to manage the emotional impact of the loss of her children, was ‘constantly crying’, felt depressed, had no motivation, and was also affected by flashbacks related to previous experiences of abuse. Jade was facing issues with heroin and alcohol, and was using methadone, but not accessing any other support. She described using substances as a coping mechanism. She also reported feeling very distrustful toward professionals and services. She explained that she had very poor family relationships, particularly with her mother. By her final interview for the evaluation, Pause had helped Jade to secure new, permanent housing, through a dedicated pathway arranged by Pause Board members. Jade stated this was the most important factor in helping her to achieve change, find stability, and escape drugs. Jade’s Practitioner had helped her access treatment services for her substance misuse. Jade had also started counselling, enrolled in college on catering and maths courses, and was doing ad-hoc voluntary work. Jade’s Practitioner had also helped her to successfully engage in group activities with other Pause women, taken Jade on outings to the hairdresser and beautician, and provided practical support with buying household items, debt, and budgeting. Jade had also significantly reduced her methadone use. When asked to reflect on what she had gained from engagement with Pause, Jade described herself as ‘more stable’ and ‘more positive’. She had been refused face-to-face contact with her children, but was accepting of this, and wanted to focus on continuing to better herself for them in the hope that this might change. Jade’s partner and sister both described seeing a ‘big difference’ in Jade since she started Pause. Jade’s partner reported that Jade had ‘improved with herself and her motivation’. This included going out more, attending appointments, being more organised, and ‘getting her selfconfidence back’. Both described her as more confident, happier, and in more control of her life. They reported that their relationships had improved. Jade’s sister also felt that Jade was more honest with her, and more willing to listen than she had been previously. 74 Jade confirmed that she was able to communicate more effectively with family, friends, and professionals. Scarlett Scarlett was in her early thirties when she began her engagement with Pause. She was referred to Pause shortly after care proceedings had resulted in 3 of her children being put on an adoption plan, and the remaining 2 being placed in long term foster care. As a child, Scarlett had witnessed DVA between her parents and experienced sexual abuse as a child. She first became pregnant at the age of 14, but miscarried due to domestic violence from her boyfriend. She reported that she had only one source of support in her life: her aunty. Scarlett reported feeling suicidal before engaging with Pause, and was using cocaine as a way of coping. Scarlett’s five children had been removed following allegations they made of sexual abuse by her partner. She was pregnant with his child at the time. Several violent incidents had been recorded, and she agreed with children’s social care to leave her partner, but continued to see him. Scarlett was perceived by children’s social care to be unable to protect her children, and as prioritising her own relationship above their safety, and she started to disengage from the service. Her partner was sentenced to a term in prison for sexual offences towards the children. Scarlett was described as sad and regretful by the children’s social worker, and also as highly vulnerable and isolated: ‘she’s not a bad person but, unfortunately, she’s so vulnerable that she’s misled by her relationships’. In terms of support, the children’s social worker recognised that Scarlett needed ‘somebody for herself’, as children’s social care focused on the children, and Scarlett was not accessing any other support services. Pause assisted Scarlett with her physical health, which she had been neglecting. Following a routine smear test, Scarlett was diagnosed with cervical cancer, and underwent a hysterectomy. Initially, the social worker was concerned that Pause might withdraw support. However, Pause continued to support Scarlett, to help her to come to terms with her feelings regarding her physical health and her inability to have further children. Pause provided support to Scarlett at meetings with social care regarding her children, and the children’s social worker considered this to have been particularly important in helping to maintain a relationship, and move forward in the perceived best interests of the children. During her engagement with Pause, a reduction in Scarlett’s cocaine consumption was observed, and changes to her physical appearance were noted. The social worker linked this to improvements in her self esteem, and physical activities provided by Pause, such as swimming. The social worker remained concerned about the impact that ending engagement with Pause would have on Scarlett, noting, ‘I do think she saw [her Practitioner] as her rock at a time when she needed somebody’. 75 Scarlett’s Pause Practitioner reflected that it had taken time to build up an open, therapeutic relationship with her. However, Scarlett and her Practitioner reported that she had become committed to working with Pause, and had changed her ‘mind set’ about her whole life. By her final interview, Scarlett was accessing counselling, and reported better levels of confidence, happiness and self esteem. She also reported an improvement in coping mechanisms. While, prior to Pause, Scarlet would use cocaine, she felt that she could now talk about how she was feeling. Scarlett also reported a decrease in her anxiety and panic attacks, and was working toward enrolling in training courses. She also reported improvements in her relationships with other agencies. Describing the children’s social worker, Scarlet explained, ‘I’ve mentally come to terms with knowing that she had a job to do. I was in a bad place, but she prioritised my kids’ needs, and that’s the best thing that anybody could’ve done. I have no hard feelings against any authoritative person now. I work with them’. Her Practitioner also considered her benefits and housing to be stable. When describing the help she received from Pause she said, ‘they’ve made you feel different. It’s not just me that’s done it. She’s helped me. And if it wasn’t for her, then I could guarantee that I probably wouldn’t be here’. Skye Skye began her engagement with Pause in her early thirties. She had had 3 children removed from her care. Skye had experienced domestic violence and abuse in multiple relationships, and had a history of substance misuse and a previous criminal record. She was described by her parents as having had problems when she was a child: she was described as very easily led, and reportedly had never had a ‘true friend’. Her parents reported that she had been involved in abusive relationships from an early age, and attributed this to her fear of being ‘on her own’. Skye had initially been reluctant to engage with Pause, and stated that she had repeatedly ‘put them off’. Skye reported that she experienced high levels of anxiety, and did not trust people, and explained that she had been anxious about starting something new without knowing what it would be like. However, she stated that, once she realised that ‘they are not against you, they are just there to help you, you just go with the flow’. Her Pause Practitioner reported that, having initially faced difficulties in encouraging Skye to engage, she had sought assistance from another Pause Practitioner, and also Skye’s father. In an interview with Skye’s parents, they revealed that their relationships with Skye prior to Pause had gradually deteriorated, due to Skye’s abusive partners, the removal of her children, and her drug abuse. They reported that they had felt anguished over what had happened for several years, but had received no support for themselves. They said that, at one time, they would have felt relieved if Skye had jumped off a bridge, but now felt guilty for having felt that way. They reported that Skye still did not open up to them very much, but were very grateful for the support Pause provided to her. Skye’s parents observed that Pause was helping Skye with everyday tasks, enrolling at college, 76 accessing better housing, and buying toys for the child they had contact with every other weekend. She had also passed her driving test and had a car. They reported that she was more confident and better able to maintain eye contact during conversation, and that they had seen a difference in Skye’s physical presentation, including her clothes and hair. These changes were confirmed by Skye’s Pause Practitioner. Her mother reported, ‘it's like having her back, knocking twenty years off her’, while her father described her as ‘a completely different person’ since engaging with Pause. By her second interview, Skye was horse riding and helping out at the local stables, and thinking about attending college. She had been going to the gym with her Pause Practitioner, which, she reported, had bolstered her confidence and self esteem, particularly with regard to her feelings about her weight. She had also started to attend 2 domestic violence programmes, to understand the effects of domestic violence and abuse on children. By her final interview, Skye had started at college and had purchased the equipment needed for her course, with the help of Pause and her parents. She was also engaging with mental health services. She reported that her parents had become more supportive, as they had seen her make progress. She described their relationship as improving, and was seeing her son every other weekend at their house. She wanted more support and advice from them, but recognised they were keen for her to be independent. Skye felt that the biggest turning point had been Pause helping her to get into college: ‘I never, ever, ever, thought I’d save my life at college’. Ruby Ruby started Pause in Autumn 2015, while she was in her late twenties. She had had three children removed from her care, who were living with a paternal grandmother. At the start of her engagement with Pause, Ruby was carrying a great deal of grief following the removal of her children, as well as trauma linked to childhood experiences of domestic abuse, and further experiences of extreme domestic and sexual abuse as an adult. She was experiencing domestic abuse in her current intimate relationship, but was not receiving any support from services for this, or for her grief and trauma. Ruby was also experiencing significant financial hardship, including debts, and was entitled to limited benefits. She described herself as ‘very emotional’, anxious, and self-conscious, and also reported increasing memories of the DVA she had experienced as a child. She was described as having ‘significant’ anger issues. During her first interview, Ruby reported that her flat had recently been trashed by her boyfriend, leaving ‘windows and doors missing’. Pause provided practical support to Ruby, helping her to re-decorate her flat, and supporting her to develop her budgeting skills, and to pay for phone credit and energy bills. Her Practitioner also supported her to address her physical health, as she was having heavy periods, pain, and other issues. To try to improve her self esteem, and 77 reduce feelings of anxiety about going out, her Practitioner took her on an outing to the hairdresser. Further emotional and psychological support was provided one to one sessions with her Pause practitioner, and she reflected that this had been effective in helping to increase her confidence, and enabling her to attend some group activities, including baking. Her Practitioner was also supporting her to reduce her cannabis use. Ruby was referred to counselling, but this was not considered sufficient to address her trauma-related needs. However, the Practitioner reported, toward the end of Ruby’s engagement, that her efforts to advocate within mental health services, including to the Head of Service, for Ruby’s access to more intensive psychological support had not been successful. In the spring of 2016, Ruby had ended her relationship and obtained a non-molestation order against her ex-partner, following two recent assaults. Her Practitioner reported feeling dismayed by the standard MARAC process in the area: the perpetrator was released on bail with no conditions, and this was reported to be reflective of the standard response to cases of DVA within the area, indicating a significant systemic problem. The perpetrator breached the order 3 times within the first month, and received a fine. Ruby was referred to a local DVA agency, and a mutual relationship between Pause and the organisation was developed. The DVA practitioner described the benefits of working with Pause: ‘she’s having that regular contact with the Pause worker, and obviously we’re liaising with the Pause worker as well, and I think there’s that encouragement from the Pause worker to link in with us and keep us updated on the situation’. Although Ruby had engaged with this service previously, the DVA practitioner felt it had been difficult to support her effectively in the past, due to the level of control and manipulation by the perpetrator. The Pause Practitioner also gave some support to Ruby’s mum, who was fearful that Ruby’s ex-boyfriend was going to kill her, and supported Ruby through the process of gaining an emergency housing move, away from where the perpetrator knew she was living. When interviewed, Ruby’s mother felt that, since being involved with Pause, family relationships had improved, and Ruby was better able to communicate about how she was feeling. By the end of the evaluation, Ruby appeared to be more positive about herself, and her self-confidence had improved. She had enrolled in Maths, English, and Photography at college. Her Pause Practitioner, her mother, and her DVA practitioner all hoped that Ruby would remain away from her ex-partner, continue to build her confidence, and be safe and happy. 78 Appendix F: Additional cost benefit analysis In addition to savings resulting from reductions in pregnancies, further potential savings relating to reductions in domestic violence, harmful alcohol use and class A drug use experienced by Pause women were estimated through a cost savings model. The model excludes simple transfers (for example, increased take up of welfare payments and changes in housing status), as these are not true savings. Other outcomes that are likely to have a fiscal impact, such as increased take up of GP services and decreased demand for emergency services, are also excluded, as an analysis of the associated costs and benefits would require separate evaluation, and was outside the scope of the present evaluation. The model also excludes consideration of increases in self-reported rates of domestic violence, harmful alcohol use and class A drug use, as these are likely to be accounted for, to a significant degree, by increased willingness to disclose to Practitioners, rather than suggesting genuine increases, and also because real increases may not be attributable to engagement with Pause. Importantly, owing to the lack of a counterfactual group for this analysis, reductions and related cost savings are estimates only and are not proven to be the result of Pause. Further reasons to treat these estimates with caution are set out in Appendix A. Domestic violence Using CMF data on incidents of domestic violence, an estimate of the number of reduced victimisations over a 12 month period was produced, in order to support an estimate of potential cost savings. This was modelled using actual records of Pause women’s recorded victimisations and existing estimates of annual repeat incidents per victim. The number of domestic violence incidents that correspond to a ‘reduced victimisation’ was estimated using two methods. Method 1 establishes the annual number of incidents per victim, based on estimates of annual repeat domestic violence victimisation rates from the 2015 Crime Survey for England and Wales (70% experienced one victimisation, 15% 2 victimisations, and 16% 3 victimisations). 24 Method 2 follows Walby’s methodology of multiplying each report of domestic abuse by 5 to achieve an estimate of the number of annual incidents each report of victimisation represents. 25 24 25 ONS (2015) http://www.crimesurvey.co.uk/ Walby (2009) Cost of Domestic Violence 79 Based on estimates from this analysis, and savings estimates provided below, potential cost savings from reductions in incidents of domestic violence are calculated for a 12 month period. Table 25: Cost savings from reductions in domestic violence Cost savings: domestic violence Number of women experiencing a reduction in DV victimisation over 15 months 13 Equivalent for a 12 month period 10.4 Estimated number of incidents corresponding to reduced victimisation (method 1) 15.4 Estimated number of incidents corresponding to reduced victimisation (method 2) 52 Estimated total cost per incident £2836 Estimated total annual cost saving from reductions in incidents for a 12 month period (range) £43,674 to £147,472 Higher risk alcohol use Based on data collected in CMFs on higher risk alcohol use, and savings estimates provided in the table below, potential cost savings from reductions in higher risk alcohol use are calculated for a 12 month period, accounting for mean cost of treatment, where applicable. 80 Table 26: Cost savings from reductions in higher risk alcohol use Cost savings: harmful alcohol use Numbers of women exiting ‘higher risk drinking’ category 12 Of those: women in treatment 6 Estimated annual cost saving per in-treatment woman, including cost of treatment £975 Estimated total annual cost saving for in-treatment women £5850 Women exiting ‘higher risk drinking’ category not in treatment 6 Estimated annual cost saving per woman £2015 Estimated total annual cost saving for women not in treatment £12,090 Estimated total annual cost saving £17,940 Problematic drug use Based on data collected in CMFs on problematic drug use, and savings estimates provided in the table below, potential cost benefits from reductions in problematic drug use are calculated for a 12 month period, accounting for mean cost of treatment where applicable. 81 Table 27: Cost savings from reductions in problematic drug use Cost savings: Class A drug use Women exiting ‘problematic drug use’ category 9 Of those: women in treatment 2 Estimated annual cost saving per in-treatment woman, including cost of treatment £59,993 Estimated total annual cost saving for in-treatment women £119,986 Women exiting ‘problematic drug use’ category not in treatment 7 Estimated annual cost saving per woman £63,801 Estimated total annual cost saving for women not in treatment £446,607 Estimated total annual cost saving £566,593 Further explanation of the cost savings figures used in the preceding analysis is given in the table below. Table 28: Cost savings figures Domestic violence Unit definition Fiscal cost per incident (regardless of whether reported to police) Fiscal saving used in analysis £2836 Explanation This figure is the fiscal cost only and is based on proportions of fiscal, economic and social costs reported by Sylvia Walby (2009). This includes costs to services as a result of domestic violence incidents, 82 including the criminal justice system, health, social services, housing, and civil and legal services Which agencies are responsible or will accrue saving? Source Criminal justice agencies (33% of outcome) Health (44% of outcome) Social services (7.3% of outcome) Housing and refuges (5% of outcome) Civil legal services (10% of outcome) Walby S. (2009) in New Economy Cost Database (2015) http://neweconomymanchester.com/our-work/research-evaluation-costbenefit-analysis/cost-benefit-analysis/unit-cost-database Higher risk alcohol use Unit definition Alcohol misuse estimated cost to NHS of dependency, per year, per dependent drinker. Fiscal saving used in analysis £2015 Explanation This figure is the fiscal cost only. Which agencies are responsible or will accrue saving? Additional (non-Pause) cost of treatment Clinical Commissioning Groups (100% of outcome). Saving used (deducting costs of treatment) Source £40 per session with a specialist counsellor. This figure is based on the cost of an alcohol counsellor. We have used a figure of 1 session every 2 weeks, equating to £20 per week (PSSRU, 2010). £975 per dependent drinker per year NICE (2011) https://www.nice.org.uk/guidance/cg115/resources/alcohol-dependenceand-harmful-alcohol-use-costing-report-136379341 PSSRU Health and social care costs (2010) http://www.pssru.ac.uk/archive/pdf/uc/uc2010/uc2010.pdf Problematic drug use Unit definition Fiscal cost/saving used in analysis Fiscal cost per class A drug user per year, minus costs of treatment if applicable. This includes drug related crime, health care costs, social care costs, and drug related deaths. This figure excludes costs of specific drug rehabilitation treatment. £63,801 Explanation This is based on 2003/4 prices and uprated annually by Bank of England inflation rate 3.1% to 2016, minus the costs of treatment if applicable. Which agencies are responsible or will accrue Criminal justice agencies (90% of outcome) Health (3% of outcome) 83 saving? Social care (0.4% of outcome) Additional (non-Pause) cost of treatment In-patient treatment at £994 per patient per week for 4 weeks: £3976 (PSSRU, 2010). Specialist prescribing: £50 per patient per week (PSSRU, 2010). £40 per session with a specialist counsellor. This figure is based on the cost of an alcohol counsellor. We have used a figure of £20 per week, which represents one session every 2 weeks (PSSRU, 2010). £59,825 per drug user per year (fiscal saving minus cost of in-patient treatment only) to £60,161 per drug user per year (fiscal saving minus cost of out-patient specialist prescribing and counselling only). Mean saving: £59993 per drug user per year. PSSRU Health and social care costs (2010) http://www.pssru.ac.uk/archive/pdf/uc/uc2010/uc2010.pdf Range of savings used (deducting costs of treatment) Source The findings of this analysis of the costs and potential benefits of the programme suggest that Pause is highly likely to produce significant cost savings over time. 84 © Department for Education Reference: DFE-RR595 ISBN: 978-1-78105-665-3 The views expressed in this report are the authors’ and do not necessarily reflect those of the Department for Education. Any enquiries regarding this publication should be sent to us at: richard.white@education.gov.uk or www.education.gov.uk/contactus This document is available for download at www.gov.uk/government/publications 85