Location via proxy:   [ UP ]  
[Report a bug]   [Manage cookies]                
SlideShare a Scribd company logo

Evaluating Advocacy: Challenges, Methodologies and Solutions

•
•

This document discusses challenges, methodologies, and solutions for evaluating advocacy efforts. It begins by defining advocacy and distinguishing it from other types of campaigns. Key challenges include focusing on activities rather than outcomes and proving impact. The document recommends understanding the desired changes, monitoring progress, selecting appropriate evaluation methods, estimating influence on changes, and sharing lessons learned. A variety of evaluation methods are described, from stakeholder interviews to contribution analysis. The goal is to integrate evaluation into advocacy strategies to continually improve efforts and demonstrate successes.

1 of 20
Download to read offline
1 Evaluating Advocacy: Challenges, Methodologies and Solutions Dr Glenn O’Neil, Owl RE 4 March 2021
Topics 2 A.Challenges faced in advocacy evaluation B.Methods and solutions C.Recommended actions
What is advocacy? "Championing or supporting a cause or policy goal" Gardner and Brindis (2017), Advocacy and Policy Change Evaluation “An intervention intended to catalyse, stimulate or otherwise seed some form of change” ODI (2014) -Indirect and direct -Private and public -Formal and informal 3
What advocacy is not? 4 Advocacy is not campaigning to change the knowledge, attitudes, behavior of individuals – where there is no link to building public support, creating a supportive environment or policy change Advocacy is not having people represent or advocate on behalf of a brand or a company for greater awareness, e.g. “brand advocates”
A. Challenges for advocacy evaluation 5 Tendency to measure activities rather than outcomes Matching advocacy ambitions to reality of change possible The long-term non-linear nature of change No “magic” advocacy evaluation tool exists The challenge to “prove” where you’ve made an impact The increased pressure for advocacy and public affairs to show results
6 Output Impact Outcome Level of your influence Ambitions
B. Methods and solutions 7 1. Understand what change is desired and how 2. Have in place ways to monitor change 3. Select appropriate methods to evaluate change 4. Estimate advocacy’s influence on change seen 5. Share and learn from the findings
1. Understanding what change is desired and how What is advocacy aiming to achieve? 8 Advocacy activities Outputs Interim outcomes Outcomes Impact - Research - Coalition- building - Meetings - Research produced - Decision makers reached Policy issue prioritized by decision makers New policy adopted Positive changes for citizens Example:
1. Understanding what change is desired and how What is advocacy aiming to achieve? 9 Examples: Interim outcomes Media coverage Collaboration/alignment Growth of supporter base Recognition of issue(s) Priority of issue(s) Recognition of organisation Access for organisation to advocate Outcomes Policy development Policy adoption Policy revision Policy blocking Policy implementation Budget change Practice change
STIMULUS Theory of Change – NGO and UN advocacy on water, sanitation and hygiene (WASH) in Health Care Facilities (HCF) - Tanzania INPUTS ACTIVITES OUTPUTS INTERIM OUTCOMES IMPACT HCF research with SoapBox in Zanzibar Look outside of household WASH Development of approach / thinking Input into Healthy Start priorities by TZ Engagement with MoH Zanzibar Dissemination of Zanzibar HCF research findings Unicef review of Zanzibar assessment tools Unicef research on HCF in mainland (7 districts) WAT financial support to policy process Align and input from Healthy Start global advocacy priority validation workshop in different zones organised Tech. WG group set up for guidelines Engagement of the consultant for leading the process TWG meetings for revising guidelines Development of concept note on HCF guidelines with MoH Creation of roadmap & mapping of stakeholders Approval and Launch of guidelines on WASH in HCF by MoH Stakeholder reached through events Promotional events with government and parliament held Budgets for WASH in 100 HCFs (FY 17-18) WHA 2019 resolution on WASH in HCF WASH in HC integrated in WSDP II Guidelines implementation starts in HCF Every health care facility in Tanzania has WASH to provide quality care with attention to the needs of women, girls and children OUTCOMES WASH in HCF on WHA 2019 agenda Awareness raised on guidelines Guidelines on WASH in HCF produced 1. Understanding what change is desired and how Coalition building for WHA
2. Have in place ways to monitor change 11
3. Select appropriate methods to evaluate change 12 Stakeholder interviews Stakeholder surveys Outcome mapping Document/data review Stakeholder focus groups Case studies Policy analysis Policy tracking Media tracking Media analysis Bellwether interviews Contribution analysis Theory of change Social network analysis Methods used often in all types of evaluations Methods particular to advocacy evaluation Methods seen in all types of evaluation but often in advocacy evaluation
4. Estimate advocacy’s influence on change seen 13 Has change to policies and practices occurred? What is the significance of this change? What are the possible influences on this change? What is the estimated influence of your advocacy on the change seen based on the evidence collected?
14 Has change to policies and practices occurred? What is the significance of this change? What are the possible influences on this change? What is the estimated influence of your advocacy on the change seen based on the evidence collected? Example – influencing policy change (fictional) - Regional government issues a concept paper featuring ABC - ABC mentioned in new regional policy - Key national governments have issued statements of support for ABC - High significance as first time ABC has a profile in the policy development phase - Evidence showing critical situation for ABC - Interest of national governments - Coalition of actors involved - Policy process facilitated by our organisation -Advocacy of our organisation was a key influence on the policy changes seen along with the above interests
4. Estimate advocacy’s influence on change seen 15 Activities achieved? Outcomes Change seen? Significance? Your contribution? That issue 345 is on the agenda of XYZ commission Yes -research paper produced; input in consultative process Medium - Issue 345 is mentioned in scoping policy paper; good possibility it will be on agenda High - First time that issue 345 is recognised by XYZ commission Medium - Our organisation was main source of information on issue 345 along with like- minded alliance Etc.
5. Share and learn from the findings 16 Track progress to constantly improve advocacy Celebrate and communicate advocacy successes with partners / members / sponsors Feed findings into future advocacy initiatives Learn from the challenges and successes Communicate findings using different formats and channels
C. Recommended actions for integrating evaluation and measurement in advocacy and public affairs 17 1. Be clear and pragmatic about what you are trying to achieve 2. Monitoring and learning must be integrated into strategy and implementation i.e. as the advocacy is happening… 3. “Measure what Matters” 4. Advocacy done well can “work”!
Available resources • Theory of change • Advocacy monitoring and evaluation methods/approaches • Guidelines • Communicating monitoring and evaluation results • “Think pieces” on advocacy evaluation • Examples of advocacy evaluation reports See: full list of resources: https://intelligentmeasurement.net/ 18
Q & A 19
Contact 20 oneil@owlre.com glennoneil www.owlre.com @glenn_oneil Contacts: www.intelligentmeasurement.net

More Related Content

Evaluating Advocacy: Challenges, Methodologies and Solutions

  • 1. 1 Evaluating Advocacy: Challenges, Methodologies and Solutions Dr Glenn O’Neil, Owl RE 4 March 2021
  • 3. What is advocacy? "Championing or supporting a cause or policy goal" Gardner and Brindis (2017), Advocacy and Policy Change Evaluation “An intervention intended to catalyse, stimulate or otherwise seed some form of change” ODI (2014) -Indirect and direct -Private and public -Formal and informal 3
  • 4. What advocacy is not? 4 Advocacy is not campaigning to change the knowledge, attitudes, behavior of individuals – where there is no link to building public support, creating a supportive environment or policy change Advocacy is not having people represent or advocate on behalf of a brand or a company for greater awareness, e.g. “brand advocates”
  • 5. A. Challenges for advocacy evaluation 5 Tendency to measure activities rather than outcomes Matching advocacy ambitions to reality of change possible The long-term non-linear nature of change No “magic” advocacy evaluation tool exists The challenge to “prove” where you’ve made an impact The increased pressure for advocacy and public affairs to show results
  • 6. 6 Output Impact Outcome Level of your influence Ambitions
  • 7. B. Methods and solutions 7 1. Understand what change is desired and how 2. Have in place ways to monitor change 3. Select appropriate methods to evaluate change 4. Estimate advocacy’s influence on change seen 5. Share and learn from the findings
  • 8. 1. Understanding what change is desired and how What is advocacy aiming to achieve? 8 Advocacy activities Outputs Interim outcomes Outcomes Impact - Research - Coalition- building - Meetings - Research produced - Decision makers reached Policy issue prioritized by decision makers New policy adopted Positive changes for citizens Example:
  • 9. 1. Understanding what change is desired and how What is advocacy aiming to achieve? 9 Examples: Interim outcomes Media coverage Collaboration/alignment Growth of supporter base Recognition of issue(s) Priority of issue(s) Recognition of organisation Access for organisation to advocate Outcomes Policy development Policy adoption Policy revision Policy blocking Policy implementation Budget change Practice change
  • 10. STIMULUS Theory of Change – NGO and UN advocacy on water, sanitation and hygiene (WASH) in Health Care Facilities (HCF) - Tanzania INPUTS ACTIVITES OUTPUTS INTERIM OUTCOMES IMPACT HCF research with SoapBox in Zanzibar Look outside of household WASH Development of approach / thinking Input into Healthy Start priorities by TZ Engagement with MoH Zanzibar Dissemination of Zanzibar HCF research findings Unicef review of Zanzibar assessment tools Unicef research on HCF in mainland (7 districts) WAT financial support to policy process Align and input from Healthy Start global advocacy priority validation workshop in different zones organised Tech. WG group set up for guidelines Engagement of the consultant for leading the process TWG meetings for revising guidelines Development of concept note on HCF guidelines with MoH Creation of roadmap & mapping of stakeholders Approval and Launch of guidelines on WASH in HCF by MoH Stakeholder reached through events Promotional events with government and parliament held Budgets for WASH in 100 HCFs (FY 17-18) WHA 2019 resolution on WASH in HCF WASH in HC integrated in WSDP II Guidelines implementation starts in HCF Every health care facility in Tanzania has WASH to provide quality care with attention to the needs of women, girls and children OUTCOMES WASH in HCF on WHA 2019 agenda Awareness raised on guidelines Guidelines on WASH in HCF produced 1. Understanding what change is desired and how Coalition building for WHA
  • 11. 2. Have in place ways to monitor change 11
  • 12. 3. Select appropriate methods to evaluate change 12 Stakeholder interviews Stakeholder surveys Outcome mapping Document/data review Stakeholder focus groups Case studies Policy analysis Policy tracking Media tracking Media analysis Bellwether interviews Contribution analysis Theory of change Social network analysis Methods used often in all types of evaluations Methods particular to advocacy evaluation Methods seen in all types of evaluation but often in advocacy evaluation
  • 13. 4. Estimate advocacy’s influence on change seen 13 Has change to policies and practices occurred? What is the significance of this change? What are the possible influences on this change? What is the estimated influence of your advocacy on the change seen based on the evidence collected?
  • 14. 14 Has change to policies and practices occurred? What is the significance of this change? What are the possible influences on this change? What is the estimated influence of your advocacy on the change seen based on the evidence collected? Example – influencing policy change (fictional) - Regional government issues a concept paper featuring ABC - ABC mentioned in new regional policy - Key national governments have issued statements of support for ABC - High significance as first time ABC has a profile in the policy development phase - Evidence showing critical situation for ABC - Interest of national governments - Coalition of actors involved - Policy process facilitated by our organisation -Advocacy of our organisation was a key influence on the policy changes seen along with the above interests
  • 15. 4. Estimate advocacy’s influence on change seen 15 Activities achieved? Outcomes Change seen? Significance? Your contribution? That issue 345 is on the agenda of XYZ commission Yes -research paper produced; input in consultative process Medium - Issue 345 is mentioned in scoping policy paper; good possibility it will be on agenda High - First time that issue 345 is recognised by XYZ commission Medium - Our organisation was main source of information on issue 345 along with like- minded alliance Etc.
  • 16. 5. Share and learn from the findings 16 Track progress to constantly improve advocacy Celebrate and communicate advocacy successes with partners / members / sponsors Feed findings into future advocacy initiatives Learn from the challenges and successes Communicate findings using different formats and channels
  • 17. C. Recommended actions for integrating evaluation and measurement in advocacy and public affairs 17 1. Be clear and pragmatic about what you are trying to achieve 2. Monitoring and learning must be integrated into strategy and implementation i.e. as the advocacy is happening… 3. “Measure what Matters” 4. Advocacy done well can “work”!
  • 18. Available resources • Theory of change • Advocacy monitoring and evaluation methods/approaches • Guidelines • Communicating monitoring and evaluation results • “Think pieces” on advocacy evaluation • Examples of advocacy evaluation reports See: full list of resources: https://intelligentmeasurement.net/ 18