Location via proxy:   [ UP ]  
[Report a bug]   [Manage cookies]                
SlideShare a Scribd company logo

ICCM_Sara Riese_10.16.13

•Download as PPTX, PDF•
•
CORE Group
CORE Group

This summary provides an overview of 3 implementation research studies on integrated community case management (iCCM) conducted by the University Research Co., LLC. The first study analyzed iCCM policies in 6 countries to understand how policy context, actors, and processes influence iCCM implementation. It found that policies often did not explicitly mention iCCM and were developed with technical staff but lacked engagement from key stakeholders. External funding was critical for policy development. The second study developed an iCCM costing and financing tool to help countries estimate costs and plan long-term financing. It was tested in Malawi and Senegal. The third study examined an iCCM monitoring improvement project in an unnamed country. Overall, the studies provide insights into real-

1 of 56
iCCM in action Results from 3 implementation research studies Sara Riese, Research Advisor October 16, 2013 UNIVERSITY RESEARCH CO., LLC 1a
Translating Research into Action (TRAction) Project Overview TRAction is a USAID-funded project that awards grants to research organizations to develop, test, and compare approaches to more effectively deliver, increase, and scale-up evidence-based interventions that address priority public health challenges. Through implementation research, TRAction addresses “know-do” gaps, or delays, in the discovery of effective interventions and their wide-scale application. TRAction emphasizes local ownership and partnerships in order to scale up equitable and sustainable efforts.
What is implementation research? Public Health Knowledge (what we know) Implemen tation Research Public Health Practice (what we do) • IR seeks to determine the best ways to implement evidence-based interventions in real world settings • IR seeks to optimize programs that are currently not achieving adequate coverage or quality 3
Unique features of implementation research • Done in real world rather than controlled settings – Real world settings with existing staff, facilities, and budgets – Real world populations rather than select sub-groups • Focus on context – Political, social, economic, cultural – Existing health care system and network of stakeholders • Central role of stakeholders and end-users – Health managers, policy makers – Involved closely in setting agenda – Goal of efforts is to help them, not publish papers
Intervention Spectrum What is the goal of the intervention? Real-life effectiveness Equitabili ty Program integratio n Replicati on Sustainability Common practice at scale Consider where the intervention falls on the intervention “spectrum”
Real-life Effectiveness 1.) Under what conditions does the program work? Replication Sustainability Common practice at scale 2.) Is the tool, intervention, or strategy worth it? Is it costeffective? 1.) Why don‟t tested programs work when transferred to new settings or work in some new settings and not others? 3.) Does the program achieve the intended public health impact? 2.) How can implementation be improved to assure reliability?
Why iCCM? • There are many gaps in our understanding of the optimal approaches to the implementation, scale-up, and sustainability of iCCM Programs. (Hamer, et al, Am J Trop Med Hyg, 2012) 7
Real-life Effectiveness Equitability 1.) Cost and Finance : Is iCCM cost-effective? Program Integration Replication 1.) Policy: How is iCCM policy developed in different settings? 2.) Policy: How does iCCM policy impact and does it improve implementation? 3.) Embedded: How can iCCM monitoring be improved to Sustainability Common Practice at Scale 1.) Cost and Finance : What are the financial considerations for a country to sustain their iCCM program?
iCCM Policy Analysis PI: Sara Bennett Research team: Asha George, Daniela C. Rodríguez, Jessica Shearer 9
Study Aims • To document and analyze the specific expressions of policy (policy, plans, directives, guidelines etc.) that support or inhibit implementation of iCCM of childhood and neonatal conditions. • To identify facilitators or barriers to policy and program change in the area of iCCM, related to policy context, policy content, the process of policy change, and the actors involved in policy-making. • To assess the role of ideas and evidence (comprising research evidence, country data, learning from other countries, international guidelines and tacit knowledge) in supporting policy and program development. • To identify policy elements that enable the eventual implementation and/or improvement of integrated community case management programs.
Study Design & Methods • Document review • Semi-structured interviews: – Stakeholders in iCCM policy: Government officials, development partners and multilateral organizations, bilateral donors, civil society organizations, research institutions, etc. – Start with respondents identified in document review, and snowball until saturation • Fieldwork undertaken April – September 2012 • Thematic analysis using NVivo software by country and JHSPH researchers • Synthesis of country studies
KEY FINDINGS
Content of iCCM Policy • In none of the study countries was iCCM a stand-alone policy – iCCM was most likely to be viewed as the community component of IMCI with treatment services targeted at hard to reach areas. • iCCM policies for malaria and diarrhea were supportive across country case studies with less support shown for pneumonia or newborn CCM. • Integration across iCCM conditions varied with more difficulties faced in countries with well-funded, parallel malaria programs.
Content of iCCM Policy • Higher-level policy documents varied in their mention of CHWs or components of iCCM but program documents and training guidelines were more consistent. – Policies were not developed in a linear fashion • CHW profiles varied substantially. – In some places, iCCM is part of an upgrading of community services (Niger, Mali, Mozambique, Malawi) while in others it aims to build on a foundation of volunteers (Burkina Faso, Kenya)
Policy Context, Actors and Process • The history of primary health care and community health worker programs in each country had a substantial, albeit nuanced, impact upon the development of iCCM policies. • Policy change was typically led by technocrats within the MoH. – High level policy champions were rare, and a number of actors who could have been supportive of the policy were not really engaged in policy formulation. • Despite the centrality of funding issues, Ministries of Finance were not brought into policy discussions.
Policy Context, Actors and Process • Availability of funding to support both the scale up of a paid CHW cadre and necessary training and drugs for iCCM was critical to successful policy formulation across all countries. – In all cases, countries were highly dependent upon the promise of external funding for iCCM. • One of the key factors that focused attention on iCCM was political commitment to the MDGs. This was combined with recognition that existing strategies to achieve MDG 4 (notably IMCI) were not working as effectively as they should.
Policy Process High Level Focus on MDGs FUNDING AVAILABLE Offer of funding from Development partner or Funding windfall Countries not on track to achieve MDGs iCCM Policy window IMCI is not working as well as expected Children are dying at home New Countryled Community strategies POLICY SOLUTION Emerging evidence of Effectiveness of iCCM
The Role of Evidence and Ideas • Both scientific and experiential evidence cited during policy development, with much of it coming from outside the country. – Local evidence was highly valued and the lack of local evidence on key issues had the potential to slow the policy process. • Country-based evidence was used most to identify the key issues at stake while research evidence, both international and local, was used to identify and prioritize interventions. Evidence from other countries were seen as “proof of concept” and learning opportunities. • Evidence was identified and promoted by “elite actors,” such as UNICEF, WHO, NGOs and a few key government officials.
Implementation of iCCM Policy • Implementation of iCCM is taking place in all study countries to some extent, with the exception of Kenya. • Ministries of Health are involved in implementation but partner organizations are playing a significant role in carrying out activities as well as funding them. • Key questions remain about the long-term financing and sustainability of iCCM policy and its implementation. • There are a series of cross-cutting barriers to implementation that arise from existing challenges within health systems and related to CHWs. • There are further country-specific challenges related to the policy content and/or the policy development process for iCCM that also need to be addressed.
Acknowledgements Funding from • We would like to thank all country teams for their hard work. • Sincere appreciation to study participants for sharing their time with us.
iCCM Costing and Financing Tool Presented by Colin Gilmartin PI: David Collins Research Team: Zina Jarrah, Kate Wright Management Sciences for Health UNIVERSITY RESEARCH CO., LLC 21a
Presentation Outline iCCM Costing and Financing Tool Overview Results from Malawi and Senegal Application of iCCM Costing Tool 22
Project Overview • Project Scope – Major Objective – To develop and disseminate a simple cost and financing model for integrated Community Case Management (iCCM) • Project Approach & Strategies – Develop a prototype iCCM costing tool – Test in three research countries (Rwanda, Malawi and Senegal) – Refine tool after each country study – Field testing of the tool conducted by another organization (WorldVision in Indonesia) – Finalize tool and disseminate results – Support early adopters of tool 23
iCCM Costing and Financing Tool Description • Excel-based, open source, user-friendly • User guide and implementation manual • Baseline Year Costs and Financing + 5 Projection Years • Automatic conversion between local currency and USD • Automatic generation of output graphs • Combination of standard and actual costs • Uses financial costs only • Does not calculate cost-effectiveness but can be used as cost inputs for CEA 24
iCCM Costing Methodology • Data collected from two main sources: 1. Program implementing agency (i.e., MOH, NGO) • Program assumptions, population coverage • Standard treatment protocols • Numbers of CHWs and cases treated in year of analysis • Costs of medicines and equipment • Management, supervision and training costs 2. Interviews with CHWs and their supervisors • “Reality check” to compare with data collection from partners/central level • Time spent by CHWs on iCCM activities 25 • Time spent by CHW supervisors
Data Requirements for iCCM Costing • National Population, total population covered by iCCM Program • Breakdown of population by age (newborns, infants, children <5, pregnant) • Interventions in iCCM package and standard treatment guidelines • Source of financing and amounts of funding secured for iCCM program • Number of CHWs current providing iCCM services • Number of iCCM cases treated • Incidence rate for each iCCM service (i.e. pneumonia, diarrhea, malaria) 26
Data Requirements for iCCM Costing • • • • • • • • List of standard equipment provided to CHW and cost List of essential medicines provided to CHW and cost Standard CHW working hours and salary, if applicable Management costs: list of all staff providing iCCM program management, salaries, and time spent on iCCM Supervision costs: list of all staff providing supervision to CHWs, salaries, and time spent on iCCM Meeting costs: list of all meetings required for CHWs and supervisors, costs Training costs: list of all trainings required for CHWs (initial/pre-service training and refresher training), costs Any additional iCCM Program Recurrent or Startup costs 27
RESULTS FROM MALAWI AND SENEGAL 28
Malawi, Rwanda and Senegal: iCCM Programs at a Glance 29
Utilization per Capita, Cost per Capita, Actual and Target Utilization (USD) 30
Utilization per capita by disease area, Actual and Target utilization 31
Cost per Service, Actual and Target Utilization (USD) 32
Total program costs, Actual and Target Utilization (USD) 33
USES AND APPLICATION OF ICCM COSTING TOOL 34
iCCM Implementation and Potential Uses • • • • • Clear linkage between iCCM policy development and costing Developing national policies and budgets should be a simultaneous, iterative process Identify areas where policy decisions may be driven by costs, and create scenarios (i.e. usage of RDTs, payment of CHWs) Develop concrete national targets for iCCM coverage that can be costed Cost implications of these policy decisions can be calculated using the iCCM tool 35
Applications • Determine cost efficiency and effectiveness of iCCM programs • Provide detailed analysis of start-up and implementing costs • Develop multi-year projections of program costs • Develop line item budgets • Determine the cost to achieve impact (cost per DALY, cost per life saved, etc.) • Conduct „what-if‟ scenarios – adding new services to the package, changing treatment protocols, etc. • Empower national stakeholders to advocate for iCCM program implementation and mobilize the necessary 36 resources
Resources • For more information: www.msh.org, Health Care Financing subsection http://www.msh.org/our-work/projects/integrated-communitycase-management-cost-financing-analysis-model-development • Download the tool: http://www.msh.org/resources/integratedcommunity-case-management-costing-financing-tool • Please direct all questions and feedback to Zina Jarrah (zjarrah@msh.org) 37
ICCM Embedded Research: Improved data for improved programs (CCM-IDIP) PI: Jennifer Bryce Research team: Elizabeth Hazel, Tanya Guenther (StC) UNIVERSITY RESEARCH CO., LLC
Embedded within ICCM program activities to improve existing MOH M&E systems – Four-country analysis of ICCM M&E systems – Malawi as “in-depth” case study to test “innovative approaches” for improving M&E Objectives 1) Provide an in-depth analysis of the ICCM M&E systems of 4 countries a) ICCM Benchmark indicators as the framework b) Data quality assessment snapshot 2) Describe successes/challenges of “innovative approaches” in Malawi and how they could be adapted to other settings. 3) Conduct a cross-country analysis of the ICCM M&E systems with recommendations for improvements.
Malawi UNIVERSITY RESEARCH CO., LLC
Activity timeline Activity Date National level consultations Dec 2011 & March 2012 1. Desk review March – April 2012 2. Baseline Data quality assessment May – June 2012 3. District level consultations Oct 2012 Implementation of Innovative approaches: 4. Cell phone interview validation study Oct-Nov 2012 5. Data quality and use improvement package. Dec 2012 – April 2013 6. Endline Data quality assessment June 2013 7. Dissemination July – August 2013
Desk review • ICCM program is well-established and coordinated under the IMCI/MOH • Country focus on core implementation strength indictors – Developed consensus indictors and adapted routine reporting tools to collect • Prioritized two areas for further implementation research – Data management and use at district level for improved decision‐making. – Routine sources for quality of care
HSA Form 1A Routine monitoring structure Partner support Health center Form 1B Partner support District Form 1C National Partner support • Monthly reporting forms • Significant partner support • DHIS-2 being scaled-up
Data quality assessment • Objectives – To describe data collection availability, completeness, quality and reporting at all levels – To identify any gaps in ICCM monitoring • Methods – 2 districts – Interviewed HSA, supervisors, district health staff – Results verification: comparing reporting forms with HSA registers
Results verification ratio: HSA forms Dowa Kasungu Av. (range) Av. (range) Fever cases 0.89 (0.14-1.0) 0.77 (0.22-1.20) Diarrhea cases 0.88 (0.25-1.18) 0.99 (0.57-5.00) Fast breathing cases 0.95 (0.32-1.0) 0.59 (0.06-1.29) Stocks dispensed: LA 6x1 0.93 (0.23-1.01) 0.84 (0.17-1.89) LA 6x2 0.96 (0.14-3.2) 1.36 (0.71-2.73) Cotrim 0.88 (0.13-1.04) 0.90 (0.39-1.27) ORS 0.88 (0.33-1.12) 0.85 (0.04-1.43) RVR = Count from HSA Register/Reported by HSA in MRF
Major findings: strengths • Well-defined structure for reporting with clear deadlines & expectations • High reported ease-of-use with reporting forms • System of quality checks in place • Good levels of reporting and completeness • Reasonable levels of consistency with a few exceptions. • HSAs meet regularly with health center staff/community leaders
Major findings: weaknesses • ICCM data not kept at health center or HSA level • Concerns with data quality reported by participants – drug stock & caseload • Very limited training on data use, processing and interpretation. • Data use in decision making is low, mostly top-down approach.
“Innovative” approach – cell phone validation study • Collect IS indicators via mobile telephone interviews with HSAs • Main objective: to determine whether telephone interviews with HSAs is a reliable approach for measuring ICCM implementation strength
Methods • Interviewed ICCM-trained HSAs on national consensus implementation strength indicators: – Training, supervision/mentoring, utilization, current/minimum drug stocks, previous stock-outs • Validated responses by comparing responses to health center reporting forms and direct observation at the HSA village clinics • Interviewed HSAs to determine the reason for any discrepancies • Sampled 241 ICCM-trained HSAs in 2 districts
Preliminary Data: Supervision and current drug stocks Implementation strength indicator Validation Method Reported % (n/N) Observe d % (n/N) Sensitivity/ specificity No. HSAs supervised in CCM in the last 3 months Supervisor records & routine forms 26.5% (53/200) 26.5% (53/200) 81.1%/93.2% No. HSAs mentored in CCM in the last 3 months Supervisor records & routine forms 26.0% (52/200) 17.5% (35/200) 82.9%/86.1% No. of HSAs with all key CCM drugs at the time of assessment Observation at village clinics 56.6% (112/198) 55.6% (110/198) 100%/97.7%
Conclusions • This method is feasible; HSAs reachable by cell phone. 18% not available by cell phone but available for face-toface interviews. • Produces accurate results – Less accurate for drug stockouts and supervision/mentorship – However, given DQ issues with routine forms, interviewing HSAs may be the most reliable method for collecting this information short-term. • Good method for scale-up
“Innovative” approach – Data use and quality improvement package • Objectives – To improve the MOH ICCM monthly reporting data quality – To increase data use at the district, health center and HSA levels. – To document feasibility and costs of package implementation
Specific package components • General training on data management, use and interpretation; • Refresher training for HSAs and senior HSAs on completing the monthly reporting forms; • Simple templates for displaying the monthly ICCM data; • Calculators for HSAs and senior HSAs to assist with completing monitoring forms; • Training for senior HSAs to identify poor-reporting HSAs for targeted supervision; • Working with district IMCI coordinator to identify reporting “benchmarks” and „action thresholds‟
Update • Implementing in 2 districts (Dowa and Kasungu) with 46 health facilities and 320 HSAs • Evaluate with endline DQA • Interim supervisory visits show that HSAs/supervisors are using the graphs and feedback is positive. • Partners are considering incorporating the data use templates/training in the ICCM package
Questions? 55
IR seeks to determine the best ways to implement evidence-based interventions in real world settings Based on what you’ve seen today: - What are the lessons learned? - What are the key messages for uptake into policy and practice? 56

More Related Content

ICCM_Sara Riese_10.16.13

  • 1. iCCM in action Results from 3 implementation research studies Sara Riese, Research Advisor October 16, 2013 UNIVERSITY RESEARCH CO., LLC 1a
  • 2. Translating Research into Action (TRAction) Project Overview TRAction is a USAID-funded project that awards grants to research organizations to develop, test, and compare approaches to more effectively deliver, increase, and scale-up evidence-based interventions that address priority public health challenges. Through implementation research, TRAction addresses “know-do” gaps, or delays, in the discovery of effective interventions and their wide-scale application. TRAction emphasizes local ownership and partnerships in order to scale up equitable and sustainable efforts.
  • 3. What is implementation research? Public Health Knowledge (what we know) Implemen tation Research Public Health Practice (what we do) • IR seeks to determine the best ways to implement evidence-based interventions in real world settings • IR seeks to optimize programs that are currently not achieving adequate coverage or quality 3
  • 4. Unique features of implementation research • Done in real world rather than controlled settings – Real world settings with existing staff, facilities, and budgets – Real world populations rather than select sub-groups • Focus on context – Political, social, economic, cultural – Existing health care system and network of stakeholders • Central role of stakeholders and end-users – Health managers, policy makers – Involved closely in setting agenda – Goal of efforts is to help them, not publish papers
  • 5. Intervention Spectrum What is the goal of the intervention? Real-life effectiveness Equitabili ty Program integratio n Replicati on Sustainability Common practice at scale Consider where the intervention falls on the intervention “spectrum”
  • 6. Real-life Effectiveness 1.) Under what conditions does the program work? Replication Sustainability Common practice at scale 2.) Is the tool, intervention, or strategy worth it? Is it costeffective? 1.) Why don‟t tested programs work when transferred to new settings or work in some new settings and not others? 3.) Does the program achieve the intended public health impact? 2.) How can implementation be improved to assure reliability?
  • 7. Why iCCM? • There are many gaps in our understanding of the optimal approaches to the implementation, scale-up, and sustainability of iCCM Programs. (Hamer, et al, Am J Trop Med Hyg, 2012) 7
  • 8. Real-life Effectiveness Equitability 1.) Cost and Finance : Is iCCM cost-effective? Program Integration Replication 1.) Policy: How is iCCM policy developed in different settings? 2.) Policy: How does iCCM policy impact and does it improve implementation? 3.) Embedded: How can iCCM monitoring be improved to Sustainability Common Practice at Scale 1.) Cost and Finance : What are the financial considerations for a country to sustain their iCCM program?
  • 9. iCCM Policy Analysis PI: Sara Bennett Research team: Asha George, Daniela C. Rodríguez, Jessica Shearer 9
  • 10. Study Aims • To document and analyze the specific expressions of policy (policy, plans, directives, guidelines etc.) that support or inhibit implementation of iCCM of childhood and neonatal conditions. • To identify facilitators or barriers to policy and program change in the area of iCCM, related to policy context, policy content, the process of policy change, and the actors involved in policy-making. • To assess the role of ideas and evidence (comprising research evidence, country data, learning from other countries, international guidelines and tacit knowledge) in supporting policy and program development. • To identify policy elements that enable the eventual implementation and/or improvement of integrated community case management programs.
  • 11. Study Design & Methods • Document review • Semi-structured interviews: – Stakeholders in iCCM policy: Government officials, development partners and multilateral organizations, bilateral donors, civil society organizations, research institutions, etc. – Start with respondents identified in document review, and snowball until saturation • Fieldwork undertaken April – September 2012 • Thematic analysis using NVivo software by country and JHSPH researchers • Synthesis of country studies
  • 13. Content of iCCM Policy • In none of the study countries was iCCM a stand-alone policy – iCCM was most likely to be viewed as the community component of IMCI with treatment services targeted at hard to reach areas. • iCCM policies for malaria and diarrhea were supportive across country case studies with less support shown for pneumonia or newborn CCM. • Integration across iCCM conditions varied with more difficulties faced in countries with well-funded, parallel malaria programs.
  • 14. Content of iCCM Policy • Higher-level policy documents varied in their mention of CHWs or components of iCCM but program documents and training guidelines were more consistent. – Policies were not developed in a linear fashion • CHW profiles varied substantially. – In some places, iCCM is part of an upgrading of community services (Niger, Mali, Mozambique, Malawi) while in others it aims to build on a foundation of volunteers (Burkina Faso, Kenya)
  • 15. Policy Context, Actors and Process • The history of primary health care and community health worker programs in each country had a substantial, albeit nuanced, impact upon the development of iCCM policies. • Policy change was typically led by technocrats within the MoH. – High level policy champions were rare, and a number of actors who could have been supportive of the policy were not really engaged in policy formulation. • Despite the centrality of funding issues, Ministries of Finance were not brought into policy discussions.
  • 16. Policy Context, Actors and Process • Availability of funding to support both the scale up of a paid CHW cadre and necessary training and drugs for iCCM was critical to successful policy formulation across all countries. – In all cases, countries were highly dependent upon the promise of external funding for iCCM. • One of the key factors that focused attention on iCCM was political commitment to the MDGs. This was combined with recognition that existing strategies to achieve MDG 4 (notably IMCI) were not working as effectively as they should.
  • 17. Policy Process High Level Focus on MDGs FUNDING AVAILABLE Offer of funding from Development partner or Funding windfall Countries not on track to achieve MDGs iCCM Policy window IMCI is not working as well as expected Children are dying at home New Countryled Community strategies POLICY SOLUTION Emerging evidence of Effectiveness of iCCM
  • 18. The Role of Evidence and Ideas • Both scientific and experiential evidence cited during policy development, with much of it coming from outside the country. – Local evidence was highly valued and the lack of local evidence on key issues had the potential to slow the policy process. • Country-based evidence was used most to identify the key issues at stake while research evidence, both international and local, was used to identify and prioritize interventions. Evidence from other countries were seen as “proof of concept” and learning opportunities. • Evidence was identified and promoted by “elite actors,” such as UNICEF, WHO, NGOs and a few key government officials.
  • 19. Implementation of iCCM Policy • Implementation of iCCM is taking place in all study countries to some extent, with the exception of Kenya. • Ministries of Health are involved in implementation but partner organizations are playing a significant role in carrying out activities as well as funding them. • Key questions remain about the long-term financing and sustainability of iCCM policy and its implementation. • There are a series of cross-cutting barriers to implementation that arise from existing challenges within health systems and related to CHWs. • There are further country-specific challenges related to the policy content and/or the policy development process for iCCM that also need to be addressed.
  • 20. Acknowledgements Funding from • We would like to thank all country teams for their hard work. • Sincere appreciation to study participants for sharing their time with us.
  • 21. iCCM Costing and Financing Tool Presented by Colin Gilmartin PI: David Collins Research Team: Zina Jarrah, Kate Wright Management Sciences for Health UNIVERSITY RESEARCH CO., LLC 21a
  • 22. Presentation Outline iCCM Costing and Financing Tool Overview Results from Malawi and Senegal Application of iCCM Costing Tool 22
  • 23. Project Overview • Project Scope – Major Objective – To develop and disseminate a simple cost and financing model for integrated Community Case Management (iCCM) • Project Approach & Strategies – Develop a prototype iCCM costing tool – Test in three research countries (Rwanda, Malawi and Senegal) – Refine tool after each country study – Field testing of the tool conducted by another organization (WorldVision in Indonesia) – Finalize tool and disseminate results – Support early adopters of tool 23
  • 24. iCCM Costing and Financing Tool Description • Excel-based, open source, user-friendly • User guide and implementation manual • Baseline Year Costs and Financing + 5 Projection Years • Automatic conversion between local currency and USD • Automatic generation of output graphs • Combination of standard and actual costs • Uses financial costs only • Does not calculate cost-effectiveness but can be used as cost inputs for CEA 24
  • 25. iCCM Costing Methodology • Data collected from two main sources: 1. Program implementing agency (i.e., MOH, NGO) • Program assumptions, population coverage • Standard treatment protocols • Numbers of CHWs and cases treated in year of analysis • Costs of medicines and equipment • Management, supervision and training costs 2. Interviews with CHWs and their supervisors • “Reality check” to compare with data collection from partners/central level • Time spent by CHWs on iCCM activities 25 • Time spent by CHW supervisors
  • 26. Data Requirements for iCCM Costing • National Population, total population covered by iCCM Program • Breakdown of population by age (newborns, infants, children <5, pregnant) • Interventions in iCCM package and standard treatment guidelines • Source of financing and amounts of funding secured for iCCM program • Number of CHWs current providing iCCM services • Number of iCCM cases treated • Incidence rate for each iCCM service (i.e. pneumonia, diarrhea, malaria) 26
  • 27. Data Requirements for iCCM Costing • • • • • • • • List of standard equipment provided to CHW and cost List of essential medicines provided to CHW and cost Standard CHW working hours and salary, if applicable Management costs: list of all staff providing iCCM program management, salaries, and time spent on iCCM Supervision costs: list of all staff providing supervision to CHWs, salaries, and time spent on iCCM Meeting costs: list of all meetings required for CHWs and supervisors, costs Training costs: list of all trainings required for CHWs (initial/pre-service training and refresher training), costs Any additional iCCM Program Recurrent or Startup costs 27
  • 28. RESULTS FROM MALAWI AND SENEGAL 28
  • 29. Malawi, Rwanda and Senegal: iCCM Programs at a Glance 29
  • 30. Utilization per Capita, Cost per Capita, Actual and Target Utilization (USD) 30
  • 31. Utilization per capita by disease area, Actual and Target utilization 31
  • 32. Cost per Service, Actual and Target Utilization (USD) 32
  • 33. Total program costs, Actual and Target Utilization (USD) 33
  • 34. USES AND APPLICATION OF ICCM COSTING TOOL 34
  • 35. iCCM Implementation and Potential Uses • • • • • Clear linkage between iCCM policy development and costing Developing national policies and budgets should be a simultaneous, iterative process Identify areas where policy decisions may be driven by costs, and create scenarios (i.e. usage of RDTs, payment of CHWs) Develop concrete national targets for iCCM coverage that can be costed Cost implications of these policy decisions can be calculated using the iCCM tool 35
  • 36. Applications • Determine cost efficiency and effectiveness of iCCM programs • Provide detailed analysis of start-up and implementing costs • Develop multi-year projections of program costs • Develop line item budgets • Determine the cost to achieve impact (cost per DALY, cost per life saved, etc.) • Conduct „what-if‟ scenarios – adding new services to the package, changing treatment protocols, etc. • Empower national stakeholders to advocate for iCCM program implementation and mobilize the necessary 36 resources
  • 37. Resources • For more information: www.msh.org, Health Care Financing subsection http://www.msh.org/our-work/projects/integrated-communitycase-management-cost-financing-analysis-model-development • Download the tool: http://www.msh.org/resources/integratedcommunity-case-management-costing-financing-tool • Please direct all questions and feedback to Zina Jarrah (zjarrah@msh.org) 37
  • 38. ICCM Embedded Research: Improved data for improved programs (CCM-IDIP) PI: Jennifer Bryce Research team: Elizabeth Hazel, Tanya Guenther (StC) UNIVERSITY RESEARCH CO., LLC
  • 39. Embedded within ICCM program activities to improve existing MOH M&E systems – Four-country analysis of ICCM M&E systems – Malawi as “in-depth” case study to test “innovative approaches” for improving M&E Objectives 1) Provide an in-depth analysis of the ICCM M&E systems of 4 countries a) ICCM Benchmark indicators as the framework b) Data quality assessment snapshot 2) Describe successes/challenges of “innovative approaches” in Malawi and how they could be adapted to other settings. 3) Conduct a cross-country analysis of the ICCM M&E systems with recommendations for improvements.
  • 41. Activity timeline Activity Date National level consultations Dec 2011 & March 2012 1. Desk review March – April 2012 2. Baseline Data quality assessment May – June 2012 3. District level consultations Oct 2012 Implementation of Innovative approaches: 4. Cell phone interview validation study Oct-Nov 2012 5. Data quality and use improvement package. Dec 2012 – April 2013 6. Endline Data quality assessment June 2013 7. Dissemination July – August 2013
  • 42. Desk review • ICCM program is well-established and coordinated under the IMCI/MOH • Country focus on core implementation strength indictors – Developed consensus indictors and adapted routine reporting tools to collect • Prioritized two areas for further implementation research – Data management and use at district level for improved decision‐making. – Routine sources for quality of care
  • 43. HSA Form 1A Routine monitoring structure Partner support Health center Form 1B Partner support District Form 1C National Partner support • Monthly reporting forms • Significant partner support • DHIS-2 being scaled-up
  • 44. Data quality assessment • Objectives – To describe data collection availability, completeness, quality and reporting at all levels – To identify any gaps in ICCM monitoring • Methods – 2 districts – Interviewed HSA, supervisors, district health staff – Results verification: comparing reporting forms with HSA registers
  • 45. Results verification ratio: HSA forms Dowa Kasungu Av. (range) Av. (range) Fever cases 0.89 (0.14-1.0) 0.77 (0.22-1.20) Diarrhea cases 0.88 (0.25-1.18) 0.99 (0.57-5.00) Fast breathing cases 0.95 (0.32-1.0) 0.59 (0.06-1.29) Stocks dispensed: LA 6x1 0.93 (0.23-1.01) 0.84 (0.17-1.89) LA 6x2 0.96 (0.14-3.2) 1.36 (0.71-2.73) Cotrim 0.88 (0.13-1.04) 0.90 (0.39-1.27) ORS 0.88 (0.33-1.12) 0.85 (0.04-1.43) RVR = Count from HSA Register/Reported by HSA in MRF
  • 46. Major findings: strengths • Well-defined structure for reporting with clear deadlines & expectations • High reported ease-of-use with reporting forms • System of quality checks in place • Good levels of reporting and completeness • Reasonable levels of consistency with a few exceptions. • HSAs meet regularly with health center staff/community leaders
  • 47. Major findings: weaknesses • ICCM data not kept at health center or HSA level • Concerns with data quality reported by participants – drug stock & caseload • Very limited training on data use, processing and interpretation. • Data use in decision making is low, mostly top-down approach.
  • 48. “Innovative” approach – cell phone validation study • Collect IS indicators via mobile telephone interviews with HSAs • Main objective: to determine whether telephone interviews with HSAs is a reliable approach for measuring ICCM implementation strength
  • 49. Methods • Interviewed ICCM-trained HSAs on national consensus implementation strength indicators: – Training, supervision/mentoring, utilization, current/minimum drug stocks, previous stock-outs • Validated responses by comparing responses to health center reporting forms and direct observation at the HSA village clinics • Interviewed HSAs to determine the reason for any discrepancies • Sampled 241 ICCM-trained HSAs in 2 districts
  • 50. Preliminary Data: Supervision and current drug stocks Implementation strength indicator Validation Method Reported % (n/N) Observe d % (n/N) Sensitivity/ specificity No. HSAs supervised in CCM in the last 3 months Supervisor records & routine forms 26.5% (53/200) 26.5% (53/200) 81.1%/93.2% No. HSAs mentored in CCM in the last 3 months Supervisor records & routine forms 26.0% (52/200) 17.5% (35/200) 82.9%/86.1% No. of HSAs with all key CCM drugs at the time of assessment Observation at village clinics 56.6% (112/198) 55.6% (110/198) 100%/97.7%
  • 51. Conclusions • This method is feasible; HSAs reachable by cell phone. 18% not available by cell phone but available for face-toface interviews. • Produces accurate results – Less accurate for drug stockouts and supervision/mentorship – However, given DQ issues with routine forms, interviewing HSAs may be the most reliable method for collecting this information short-term. • Good method for scale-up
  • 52. “Innovative” approach – Data use and quality improvement package • Objectives – To improve the MOH ICCM monthly reporting data quality – To increase data use at the district, health center and HSA levels. – To document feasibility and costs of package implementation
  • 53. Specific package components • General training on data management, use and interpretation; • Refresher training for HSAs and senior HSAs on completing the monthly reporting forms; • Simple templates for displaying the monthly ICCM data; • Calculators for HSAs and senior HSAs to assist with completing monitoring forms; • Training for senior HSAs to identify poor-reporting HSAs for targeted supervision; • Working with district IMCI coordinator to identify reporting “benchmarks” and „action thresholds‟
  • 54. Update • Implementing in 2 districts (Dowa and Kasungu) with 46 health facilities and 320 HSAs • Evaluate with endline DQA • Interim supervisory visits show that HSAs/supervisors are using the graphs and feedback is positive. • Partners are considering incorporating the data use templates/training in the ICCM package
  • 56. IR seeks to determine the best ways to implement evidence-based interventions in real world settings Based on what you’ve seen today: - What are the lessons learned? - What are the key messages for uptake into policy and practice? 56

Editor's Notes

  1. In spite of iCCM successes and potential, some low income countries have not implemented iCCM or have delayed starting or expanding programs.
  2. The integrated delivery for children under 5 of: Treatment for pneumonia with antibiotics,Treatment for diarrhea with zinc and low osmolarity ORS, Treatment for malaria with artemisinin combination therapy (ACTs), and Home visits of newborns with treatment of neonatal sepsis with antibiotics…by community or lay health workers at household and/or community levels.
  3. Policies were not developed in a linear fashion whereby higher level policy frameworks preceded lower level implementation guidelines.
  4. In-country pilots and programs provided local implementation evidenceTo convince policy makers (pneumonia pilots in Kenya and Burkina)To demonstrate feasibility (APEs in Mozambique, newborn CCM in Malawi)As project roll-out (Niger)
  5. Barriers to implementation:Health system barriers: supervision of CHWs, data management, poor coordination, drug dispensing guidelines, shortages, weak community ownershipCHW related barriers: recruitment, motivation and retention, weak ties with the communityFacilitators to implementation:Relationships and support from donorsCommunity support, where presentExisting CHW cadre and health system infrastructure
  6. NOTE: RWANDA RESULTS CURRENTLY UNDER REVIEW OF MOH
  7. Note – RDTs in Senegal and Rwanda, presumptive malaria treatment in Malawi at the time of this analysis
  8. Over-reporting