Location via proxy:   [ UP ]  
[Report a bug]   [Manage cookies]                
SlideShare a Scribd company logo
Workshop
Introduction to project evaluations
Inka Píbilová
inka@evaluace.com
10 February 2014

1
Climbing Kilimanjaro (5,895 m) in 1 year

Source: Suzanne Field

2
EC Project cycle management

Evaluation

Implementation
and monitoring

Financing

Programming

Identification

Formulation

http://www.europa.eu.int/comm/europeaid/qsm/index_en.htm

3
Review logical framework

http://www.europa.eu.int/comm/europeaid/qsm/index_en.htm

4

Recommended for you

Project Management Plan - Group Presentation
Project Management Plan - Group PresentationProject Management Plan - Group Presentation
Project Management Plan - Group Presentation

This document is a project management plan for a presentation on "Developing the project team". It includes details such as the project scope, stakeholders, work breakdown structure (WBS), schedule, costs, quality management plan, risk register, and change control. The project will involve research, design, testing, and delivery of the presentation over 40 hours with a $5,000 budget. The team members are Agatha Maia, Anusha Nandagopalandagopal, Eduardo Ikezaki, Mikhail Sopin, and Olga Dubovyk.

Progress Reporting - notes
Progress Reporting - notesProgress Reporting - notes
Progress Reporting - notes

Handout for module 11 of a course on project management, Project Supporting Plans. Progress reporting

Ten rules of project risk management
Ten rules of project risk managementTen rules of project risk management
Ten rules of project risk management

The document outlines 10 rules for effective risk management in projects. The rules include making risk management a part of the project, identifying risks early, communicating about risks, considering both threats and opportunities, clarifying ownership of risks, prioritizing risks, analyzing risks, planning and implementing risk responses, registering risks in a log, and tracking risks and associated tasks over time. Following these 10 rules can help project managers implement risk management successfully and deliver projects on time, on budget, and with the expected quality.

Use both quantitative and qualitative indicators
Compare using trends (increase), thresholds (min. 30%), targets (strategy by 12/Y1)
Quantitative - SMARTER
Specific / Simple (to understand, collect)
Measurable
Attainable/Available at cceptable costs
Relevant to project / stakeholders
Time-bound
Evaluate/Engaging
Reevaluate/Recordable

Qualitative - SPICED
Subjective
Participatory
Interpreted and communicable
Cross-checked and compared
Empowering
Diverse / disaggregated (by gender)

Min. 30 % of participants initiate a project
aiming to address a local issue.

Reasons why participants have (not)
implemented a project to address a local
issue

http://www.smarttoolkit.net/?q=node/391
http://www.europa.eu.int/comm/europeaid/qsm/index_en.htm

5
Monitoring x evaluation x audit
Evaluation
• Assessment of project efficiency, effectiveness, impact, relevance and
sustainability for the purpose of learning and accountability to stakeholders
Monitoring
• Ongoing analysis of project progress towards achieving planned results with
the purpose of improving management decision making
Audit
• Assessment of (i) the legality and regularity of project expenditure and
income i.e. compliance with laws and regulations and with applicable
contractual rules and criteria; (ii) whether project funds have been used
efficiently and economically i.e. in accordance with sound financial
management;; and (iii) whether project funds have been used effectively i.e.
for purposes intended.
• Primarily a financial and financial management focus, with the focus of
effectiveness being on project results.
http://www.europa.eu.int/comm/europeaid/qsm/index_en.htm

6
Evaluation principles for a HORRIBLE evaluation

Source: Participants of the workshop

7
Evaluation principles for GREAT evaluations

Source: Participants of the workshop

8

Recommended for you

Proposal for NGO Capacity
Proposal for NGO CapacityProposal for NGO Capacity
Proposal for NGO Capacity

This document is a project proposal by Gerson Bergeth to build the capacity of small and medium NGOs. It notes that many NGOs lack administrative, technical, and advocacy skills which limits their access to funding and their ability to partner with other organizations. The proposed project would develop training modules and a handbook to provide NGOs with skills in areas like administration, technical capacity, monitoring and evaluation, proposal writing, and advocacy in order to empower communities and strengthen civil society. The training materials would be developed over 11 months and include user testing, an e-learning version, and monitoring and evaluation of the program.

Proposal writing sample 2
Proposal writing sample 2Proposal writing sample 2
Proposal writing sample 2

This proposal outlines a case study of conflict resolution initiatives in Jangalmahal, West Bengal. Maoist violence has affected over 90 districts in India, including Jangalmahal. Recently, authorities and police have successfully brought the situation in Jangalmahal under control. The proposal aims to investigate the reasons for this success through qualitative research methods like interviews and observations. The case study findings will provide lessons for handling similar conflicts elsewhere. It will examine the specific actions, unresolved issues, and systemic features in Jangalmahal that could be replicated. The estimated cost is Rs. 5,00,000 and will be conducted over 5 weeks by a chief investigator, 2 research assistants, and utilize existing organization

writerfreelance writerfreelance copywriter
Planning Effective Linkages
Planning Effective LinkagesPlanning Effective Linkages
Planning Effective Linkages

Linkages between donor and partner organizations are important for effective information sharing, but establishing and maintaining them requires substantial time, effort and resources. The document outlines a four step planning cycle to develop effective linkage strategies: 1) define linkage strategies and objectives with partners, 2) identify potential mechanisms, 3) develop action plans, and 4) periodically monitor and assess progress. Key factors for successful linkage planning include awareness, consensus, commitment, adequate funding, and involvement of stakeholders at all levels.

partner linkages
Evaluation standards and principles
• EC http://ec.europa.eu/europeaid/
– Impartiality and independence of the evaluation process from the
programming and implementation functions;
– Credibility of the evaluation, through use of appropriately skilled and
independent experts and the transparency of the evaluation process,
including wide dissemination of results;
– Participation of stakeholders in the evaluation process, to ensure
different perspectives and views are taken into account; and
– Usefulness of the evaluation findings and recommendations, through
timely presentation of relevant, clear and concise information to
decision makers.
• OECD DAC key norms, standards, criteria
http://www.oecd.org/development/evaluation/dcdndep/41612905.pdf
• UNDP Evaluation policy http://web.undp.org/evaluation/policy.htm
• IPDET Handbook on Evaluation Ethics, Politics, Standards, Principles
http://dmeforpeace.org/sites/default/files/M14_NA.pdf
• Patton, Michael Quinn.(2008) Utilization-Focused Evaluation: 4th edition

9
Source: Participants of the workshop

10
Evaluation Terms of Reference - structure
•
•
•
•
•
•
•
•
•

Background (project, context, stakeholders)
Purpose
Use of outputs
Scope and focus
Evaluation criteria and questions
Methodology
Timetable
Budget
Human resources – responsibilities, expertise required

Guidelines:
• UNDP Handbook (p. 194 - 200) ttp://web.undp.org/evaluation/handbook/
• EC/EuropeAid Project Cycle Management Guidelines (p. 126 - 136)
http://ec.europa.eu/europeaid/multimedia/publications/documents/tools/
europeaid_adm_pcm_guidelines_2004_en.pdf
• New Zealand Guide
11
http://dmeforpeace.org/sites/default/files/NZAID_ToR%20Guideline.pdf
Problem Tree
Example: River Pollution

http://www.europa.eu.int/comm/europeaid/qsm/index_en.htm

12

Recommended for you

10 Project Proposal Writing
10 Project Proposal Writing10 Project Proposal Writing
10 Project Proposal Writing

A series of modules on project cycle, planning and the logical framework, aimed at team leaders of international NGOs in developing countries. New improved version of Writing Project Proposals in February 2014.

fundingprojectdonor
3 proj plan notes
3 proj plan notes3 proj plan notes
3 proj plan notes

This document provides an overview of development project planning and management. It discusses various planning tools like stakeholder analysis, logical framework analysis, and monitoring and evaluation. It also outlines the typical stages of a project cycle including identification, appraisal, implementation, monitoring, and evaluation. Key aspects at each stage like needs assessment, baseline studies, work plans, budgets, and lessons learned are reviewed. Limitations of the rational planning approach are also acknowledged.

3 proj plan handouts
3 proj plan handouts3 proj plan handouts
3 proj plan handouts

This document discusses development project planning and management. It covers key concepts like the project cycle, logical framework analysis, stakeholder analysis, monitoring and evaluation. The project cycle involves identification, appraisal, implementation, monitoring and evaluation. Logical framework analysis uses a matrix to define objectives, indicators, assumptions and risks. Stakeholder analysis identifies those impacted and how to engage them. Monitoring tracks inputs, outputs and outcomes against indicators. Evaluation assesses efficiency, effectiveness, impact and lessons learned to improve future projects. Thorough planning is important but too much can limit flexibility and involvement of intended beneficiaries.

Stakeholder Analysis Matrix
Example: River Pollution

http://www.europa.eu.int/comm/europeaid/qsm/index_en.htm

13
Influence
/Power

Stakeholder Analysis – Power / Interest / Attitude

KEEP SATISFIED

INFLUENCE & ENGAGE

Attitude

Policy makers

CSOs
MONITOR

Activists

KEEP INFORMED

Interest
14
Evaluation Terms of Reference - example
Background (project, context)
- project name, identification
- history of the project, objectives, results, key activities, progress over
time (add logical framework if you wish – you get more tailored proposals)
- organisational, social and political context in which the evaluation occurs
- main stakeholders involved in the project incl. target groups, beneficiaries,
partners, donors
- focus and scope of the evaluation – which project components,
geographical area, time period, target groups etc.
Our project:

15
Evaluation Terms of Reference - example
The rationale and purpose of the evaluation
- Why the evaluation is being undertaken (what do you want to get out of it)
including for accountability, learning, improvement
- Why now
Use of outputs
- How it will benefit the different stakeholders
- To whom, when and how the findings will be reported (debriefing,
presentation, report, videos, posters – printed or on-line…)
Our project:

16

Recommended for you

Components of Project Proposal
Components of Project ProposalComponents of Project Proposal
Components of Project Proposal

This document outlines the key components that should be included in a project proposal. It discusses including a face sheet with basic organization and project details, an executive summary stating the problem, objectives, activities and budget. It also recommends providing background on the organization and need for the project, describing the target area, problems and people. The proposal should include the goal, objectives, target group, timeline, activities and personnel. It should also cover the budget, sustainability, monitoring and evaluation plan, and reporting requirements.

project proposalproposal writing
Logical framework and project proposal
Logical framework and project proposalLogical framework and project proposal
Logical framework and project proposal

This document provides guidance on writing project proposals using the logical framework approach. It begins with general remarks on ensuring thoughtful planning and addressing a genuine problem. It then covers identifying the project idea, conducting a problem analysis including a problem tree, and analyzing objectives to transform problems into measurable objectives. The document explains the logical framework matrix and how it provides a summary of the project's objectives, strategy, assumptions and means of verification. It emphasizes defining objectively verifiable indicators and considering risks and assumptions. The document also provides guidance on activity planning, resource planning, and writing the different sections of a project proposal, including the executive summary, presentation of the organization, methodology, budget, and sustainability.

Project Proposal Hints
Project  Proposal  HintsProject  Proposal  Hints
Project Proposal Hints

This document provides hints for writing different sections of a project proposal, including the project title, overview, details, resources, budget, and evaluation. For the title, it recommends including collaborators' names, using a clear descriptive sentence, and avoiding unnecessary words. The overview should concisely paint a picture and highlight collaborators. The details section should specify date, objectives, concept, target participants, staffing roles. The resources section lists available and needed resources like sponsors, equipment, supplies and budgets. The evaluation plan should reference objectives and consider sustainability.

Methodology - sources of evaluation questions
• OECD/DAC evaluation criteria (similar to the EC/EuropeAid)
– using project logical framework
• Questions, concerns and values of stakeholders
• Previous research / evaluations
• Guidelines / Evaluation Tools such as Kirkpatrick Model
• Experts

17

Source: Road to Results
How to measure learning outcomes?

http://leanlearning.wikispaces.com/learning_analytics

18
OECD/DAC Evaluation criteria

http://www.europa.eu.int/comm/europeaid/qsm/index_en.htm

19
OECD/DAC Evaluation Criteria
Relevance – are we doing the right things?
The appropriateness of project objectives to the problems that it was
supposed to address, and to the physical and policy environment within
which it operated. The extent to which the project is suited to the priorities
and policies of the target group, recipient and donor.
It should include and including an assessment of the quality of project
preparation and design – i.e. the logic and completeness of the project
planning process, and the internal logic and coherence of the project design.
Potential evaluation questions:
• To what extent are the objectives of the programme still valid?
• Are the activities and outputs of the programme consistent with the
overall goal and the attainment of its objectives?
• Are the activities and outputs of the programme consistent with the
intended impacts and effects?
20

Source: OCEED/DAC, http://www.europa.eu.int/comm/europeaid/qsm/index_en.htm

Recommended for you

Persuasive Project Proposal Writing for Effective Fundraising
Persuasive Project Proposal Writing for Effective FundraisingPersuasive Project Proposal Writing for Effective Fundraising
Persuasive Project Proposal Writing for Effective Fundraising

The purpose of this course is to train participants in the art of writing a persuasive project proposal that will win funding to enable the implementation of projects and the overall continue viability of their respective organizations and institutions. The course is designed to be taught through Power Point presentation and slides with interactive visual of photos, clip arts and icons depicting meaningful connotation for each topic.

emmanuel t. barrowmbabsc
Project Proposal
Project ProposalProject Proposal
Project Proposal

This document contains a template for creating a project proposal. It outlines the key sections that should be included: project title, justification of the project which describes the problem and objectives, activities and implementation timeline, methodology, human, material and financial resources required, monitoring and evaluation plans, and approval. The proposal provides a detailed description of a series of activities aimed at solving a clearly defined problem.

project proposaleducationstudent
Proposal Writing Workshop
Proposal Writing WorkshopProposal Writing Workshop
Proposal Writing Workshop

This document provides guidance on writing a funding proposal in 10 steps. It discusses developing goals and objectives, identifying project activities and developing a work plan, budget, and evaluation plan. It also covers writing a project description, describing your organization, and assembling the full proposal package, including a cover letter and letters of support. Participants work through examples and practice exercises to develop components of a sample funding proposal.

proposalswriting
OECD/DAC Evaluation Criteria
Effectiveness – are we doing things right?
An assessment of the contribution made by results to achievement of the
Project Purpose, and how Assumptions have affected project achievements.
This should include specific assessment of the benefits accruing to target
groups, including women and men and identified vulnerable groups such as
children, the elderly and disabled.
Potential evaluation questions:
• To what extent were the objectives achieved / are likely to be achieved?
• What were the major factors influencing the achievement or nonachievement of the objectives?

21

Source: OCEED/DAC, http://www.europa.eu.int/comm/europeaid/qsm/index_en.htm
OECD/DAC Evaluation Criteria
Efficiency – is the project worthwhile?
The fact that the project results have been achieved at reasonable cost, i.e.
how wellinputs/means have been converted into activities, in terms of
quality, quantity and time, and the quality of the results achieved.
The project shou use the least costly resources possible in order to achieve
the desired results.
This generally requires comparing alternative approaches to achieving the
same outputs, to see whether the most efficient process has been adopted.
Potential evaluation questions:
• Were activities cost-efficient?
• Were objectives achieved on time?
• Was the programme or project implemented in the most efficient way
compared to alternatives?
22

Source: OCEED/DAC, http://www.europa.eu.int/comm/europeaid/qsm/index_en.htm
OECD/DAC Evaluation Criteria
Impact – what changes has the project achieved / contributed to?
The effect of the project on its wider environment, and its contribution to the
wider policy or sector objectives (as summarised in the project’s Overall
Objective).
The positive and negative changes produced by a project, directly or
indirectly, intended or unintended, positive and negative.
This involves the main impacts and effects resulting from the project on the
local social, economic, environmental and other development indicators. It
must also include the positive and negative impact of external factors.
Potential evaluation questions:
• What has happened as a result of the programme or project?
• What real difference has the activity made to the beneficiaries?
• How many people have been affected?

Source: OCEED/DAC, http://www.europa.eu.int/comm/europeaid/qsm/index_en.htm

23
OECD/DAC Evaluation Criteria
Sustainability – wil the changes (nebefits for target group) last?
An assessment of the likelihood of benefits produced by the project to
continue to flow after external funding has ended, and with particular
reference to factors of ownership by beneficiaries, policy support, economic
and financial factors, socio-cultural aspects, gender equality, appropriate
technology, environmental aspects, and institutional and management
capacity.
Sustainability is concerned with measuring whether the benefits of a project
are likely to continue after donor funding has been withdrawn. Projects need
to be environmentally as well as financially (and socially) sustainable.
Potential evaluation questions:
• To what extent did the benefits of a programme or project continue after
donor funding ceased?
• What were the major factors which influenced the achievement or nonachievement of sustainability of the programme or project?
24

Source: OCEED/DAC, http://www.europa.eu.int/comm/europeaid/qsm/index_en.htm

Recommended for you

Basic project planning & proposal making
Basic project planning & proposal makingBasic project planning & proposal making
Basic project planning & proposal making

This document outlines the basic steps in project planning, including identifying problems, conducting stakeholder and root cause analyses, setting goals and SMART objectives, developing action plans, and evaluating success. It discusses evaluating a project based on achieving objectives, efficiency, impact, and sustainability. The document proposes that a mini-project proposal should include sections on the problem statement, stakeholder analysis, proposed solutions, goals and objectives, work plan, evaluation plan, and sustainability plan.

project on construction of house report.
project on construction of house report.project on construction of house report.
project on construction of house report.

The document provides details of a project to construct a house including the project charter, scope, schedule, and resources. The project has defined activities to construct the house over a 1 year period within a budget of Rs. 14,365,047. Key stakeholders include the customer Mr. Ali Hamza and supplier vendors. The project manager developed a work breakdown structure and activity list to plan and track the house construction.

project management
Feasibility report -basic concepts with example
Feasibility report -basic concepts with exampleFeasibility report -basic concepts with example
Feasibility report -basic concepts with example

This feasibility report analyzes a proposed waste water system project. It recommends the project proceed based on identified needs in the community and project viability. Key points include: the existing system is deficient; a new system is needed to serve current and projected population; and the estimated capital costs and financing plan make the project economically feasible. The report provides background on the area's needs, outlines the proposed system components, and recommends next steps for further investigation and implementation.

Evaluation Questions - Descriptive
•
•
•
•

What „is“
Describe project (inputs, activities, outputs) or process
Simple: Who, what, where, when, how, how many …
Often used to gather opinions from target groups.

Examples:
• What is the project objective from the perspectives of different
stakeholders?
• What were the reasons for joining the program?
• How many persons were reached?
• How was the project implemented?
Evaluation design to answer them:
One-shot, before-and-after, time series, (long-term) panel, case studies…
Source: Road to Results

25
Evaluation Questions - Normative
• Compare what „is “ with what „should be“ (target)
Examples:
• Do the project activities address the needs of stakeholders?
• To what extent has the project achieved the result/objective indicators?
• Have min. 5.000 persons been reached through the campaign?
• Has the number of schools involved in Global Learning increased?
(baseline!)
Evaluation design to answer them:
One-shot, before-and-after, time series, (long-term) panel, case studies…

Source: Road to Results

26
Evaluation Questions – Cause-and-Effect
•
•
•
•

Determine what difference the project makes - what change has it brought
Often refer to outcome, impact
Compare indicators before and after, with and without the project (graph)
Careful about attribution x contribution!
– Can you say that the project achieved this or has contributed to this?
– Are there any alternative explanations (external factors) for achievements?

Examples:
• As a result of the training, have teachers incorporated Global Learning in
their lesson plans?
Evaluation design to answer them:
Experimental (control group), quasi
experimental (compare group),
nonexperimental (causual tracing, case
study, story harvesting, outcome
mapping…)

Goal

with
without

27

Source: Road to Results

T=0

T=1

Time
Evaluation Terms of Reference – Evaluation Matrix
• Evaluation criteria and questions

Design, Data
Our project – Question Baseline data Indicators Sources Collection Methods

28

Recommended for you

Project proposal
Project proposalProject proposal
Project proposal

The document proposes constructing a one-storey residential building to address the client's housing needs. It will include three bedrooms, two kitchens, and a porch. The building will cost approximately 1.15 million pesos to build and will solve the client's current problems of high rental costs and transportation expenses, while providing a comfortable living environment. Facility requirements like appliances, electricity, and water access are also addressed. The construction is proposed to alleviate the client's housing issues for many years.

technical writingproposal
Project proposal writing civil engg
Project  proposal  writing civil enggProject  proposal  writing civil engg
Project proposal writing civil engg

This document provides guidelines for writing a project proposal, including an outline of the typical sections and content. The key sections are: 1. Executive Summary: A short summary of the project including what, where, why, for whom, with whom, and duration. 2. Presentation of Organization: Establishes credibility by describing the organization's mission, history, expertise, and major supporters. 3. Project Background: Explains the context and need for the project, how it fits with existing efforts, and any previous related work. 4. Problem Statement: Clearly defines the nature and scope of the problem being addressed, its causes and effects, and why the problem requires intervention. 5.

Final Year Project proposal (Degree of Agrotech)
Final Year Project proposal (Degree of Agrotech)Final Year Project proposal (Degree of Agrotech)
Final Year Project proposal (Degree of Agrotech)

this is my slide about the final year project proposal with title " effect of water temperature and dipping time in crisping process of ipomoea reptans"

asmasams
Tips for evaluation questions
• Avoid questions on multiple issues – separate these
– Has the methodology been developed and disseminated to min. 5.000
teachers?
• You can develop subquestions for a particular question on an issue
– What concerns have teachers while introducing Global Learning at school ?
(descriptive)
– Has the project addressed these concerns of the teachers? (normative)
– Has the number of teachers using Global Learning increased as a result of
the project? (cause-and-effect)
• Set a realistic number of questions!

Source: Road to Results

29
Select a few, most important questions…how?
Would the evaluation question…

Q1 Q2 Q3..

•
•
•
•
•
•
•
•

be in line with the evaluation purpose?
be of interest to stakeholders?
reduce present uncertainty?
yield important information?
be of continuing (not fl eeting) interest?
be critical to the evaluation´s scope and comprehensiveness? (or nice to have)
have an impact on the course of events?
be answerable given the fi nancial and human resources, time, methods, and
technology available?
• Be reasonable to ask given the project cycle?
(Questions about impact, for example, are best answered after the project has
been fully operational for a few years)
Source: Road to Results, Kusters et al.: Making evaluations Matter

30
Data Collection Methods - Quantitative x Qualitative x Mixed
Sampling random x non-random (quota, typical case, snow-ball…), sample size!
Participatory – community meetings, mapping, transect walks..

Conversation
with
Concerned
Individuals

Expert (panel)
Key Informant judgement
Interviews Delhi technique

Participant
Observations:
Community
Diaries,
Interviews
Reviews journals, selfreported
of
checklists
Official
FieldRecords
Visits

Panel
Surveys

Focus
Group
Interviews

One-Time
Surveys

Direct
Observations

Informal/Less Structured Methods

Source: Road to Results / IPDET © 2009, adopted

Censuses

Field
Experiments
Surveys

Formal/More Structured Methods
31
Evaluator
Selection

Interviews
Surveys
Focus groups
Case studies

Final
debriefing
of all partners
Draft evaluation
report
commented by
all partners

Final evaluation report

Desk study

Preliminary findings & conclusions

Initial briefing and inception

Terms of Reference – Objectives,
scope, stakeholders, questions,
budget, schedule, outputs, use.

Timeline and resources

Communication with the Project Partners

Inception
phase

Field
research

Reporting
phase

1-3 months

1-3 months

1-2 months

32

Recommended for you

51432284 project-finance
51432284 project-finance51432284 project-finance
51432284 project-finance

Project finance involves investing in large industrial or infrastructure projects through a legally independent project company financed primarily with debt. It creates value through its organizational, contractual, and governance structures. The organizational structure addresses agency costs and risk contamination by separating the project from sponsors' balance sheets. Contractual structures allocate risk to parties best able to manage it, lowering overall risk costs. Governance relies on debt covenants to monitor management.

Psychology Internship Report
Psychology Internship ReportPsychology Internship Report
Psychology Internship Report

This document provides an overview of psychological therapies used at a mental health institution. It discusses the history and profile of the institution's director, which has been providing mental health services for 43 years. It then describes various psychological therapies used there, including behavioral therapies like relaxation therapy, systematic desensitization, exposure therapy, response prevention, and modeling. Other therapies covered are thought stopping, biofeedback, and aversion therapy. The document provides details on how each therapy works and which psychiatric conditions it can be used for.

Building construction-report (1)
Building construction-report (1)Building construction-report (1)
Building construction-report (1)

The document reports on the construction of three sites - a fire station in Kuala Lumpur, an eco-friendly mosque in Cyberjaya, and terrace houses in Klang. It introduces the projects, provides an overview of each site, and discusses site safety precautions, machinery and equipment used, site preparation processes, and various construction stages. Photos are included to illustrate the different elements discussed in the report.

Internal evaluators

or

₊ May have a better understanding of
the projetct, context, policies
₊ Develop organisational capacities
₊ Higher ownership of
recommendations by the organisation
₊ Usually cheaper
₋ May not be able to see alternative
perspectives, solutions
₋ Influenced more by the implementing
organisation (want to keep their jobs)
₋ May be less credible to stakeholders
₋ May be time-consuming

external evaluators

₊ May bring a new perspective or
special (technical, evaluation)
expertise
₊ More independent from the
implementer – may facilitate better
between stakeholders (across
hierarchies, in case of mistrust)
₊ Usually perceived as more credible
₋ May not be able to comprehend fully
the project due to time/other
contrains
₋ Usually more expensive

or mixed - participatory
! External evaluation is not necessarily independent! (Who pays the evaluator?
Who checks the quality?)
! Even external evaluation consumes time of the project team!
33
! It depends WHO the evaluator is 
Evaluation Terms of Reference - example
• Timetable
• Budget
• Human resources – responsibilities, expertise required
Our project:

34
Are we ready?
Evaluability
- Are clear why we do the evaluation?
- Do we have an (updated) logical framework?
- Do we have sufficient (baseline, monitoring) data available?
- Do we have accessible reliable information sources?
- Do we have sufficient funds for an internal/external evaluation? Will the
evaluation be cost-effective, will it bring reasonable benefits vs. costs?
- Is it likely that it will be used to improve actions in future? Can stakeholders
influence the evaluation decisions? Will they accept and use the findings? Is
there a strong leadership to put the recommendations in practice?
- Are there no major factors hindering the evaluation? Are staff members or
other stakeholders overloaded due to other priorities? Are there any
tendencies that would affect impartiality?

Source: UNDP Handbook, Road to Results, Making evaluations matter

35
Qualitative Data Analysis – frequency and relations
Evaluation question: Were parents satisfied with the training?
Topics

Quotes

Findings

Parents decide on topics

I think the process of
deciding would be valuable.

There was a strong feeling
that parents should be more
involved in the choice of
topics

Sometimes we just got into
a topic and then it was time
to leave or move to
something else. We need
more time to discuss.

Many participants (38 or 52
interviewed) thought there
should be more time for
discussion.

//// //// //// //// //// ////
//// ////
Cover a couple of topics per
session
//// //// //// ///

Not enough time spent on
each topic
//// //// //// ///

Develop categories, check reliability, analyse, interpret, share draft, report
Source: IPDET © 2009

36

Recommended for you

Chapter 1 overview of financial management
Chapter 1   overview of financial managementChapter 1   overview of financial management
Chapter 1 overview of financial management

Financial management concerns the acquisition, financing, and management of assets to achieve overall goals. It involves three main decision functions: investment decisions about what assets to acquire and manage; financing decisions about obtaining funds and setting dividend policy; and asset management decisions about efficiently operating existing assets. The goal of financial management is to maximize shareholder wealth by increasing the market value of the firm's common stock, which reflects the firm's investment, financing, and asset management decisions.

Acknowledgement
AcknowledgementAcknowledgement
Acknowledgement

The document acknowledges those who helped and supported the author during their internship and project work at Britannia Industries LTD. The author thanks their manager for providing the opportunity, two other officials for guidance and encouragement, and staff members at Britannia for their help during the project. The author also thanks the director of their institute for allowing them to undertake the project.

Proposal format
Proposal formatProposal format
Proposal format

The document outlines guidelines for formatting a final year project proposal. It includes sections for the project title, student names and roll numbers, main text formatting, headings formatting, figures and tables, and references. Guidelines are provided for font type, size, indentation, spacing, capitalization, and other formatting rules to maintain a consistent structure and appearance.

Quantitative Data Analysis
• Clean data
• Use mode (most frequent), median (middle value), mean (average), range
(lowest to highest value), statistical methods (correlations…)
• Produce tables, graphs etc

Source: IPDET © 2009, docs. Google.com

37
Data Analysis and Interpreting

• Needs to be clear before data collection
• Methodology incl. reliability and limits is a part of report

• Categorize and triangulate findings
• Use quotes, examples, graphs
• Distinguiish between findings (evidence) and interpreting (conclusions)

• Do not generalize findings from 3 respondents to the whole sector!
• Have a short summary for those who cannot read the whole report

Source: Road to Results, evaluace.com

38
Reporting: as per the expected use by each stakeholder

Case
study
39
Reporting – example of quantitative data

the Project A
Project A
Project A
Project A
Project A

Source: Inka Pibilova

40

Recommended for you

My research proposal.ppt
My research proposal.pptMy research proposal.ppt
My research proposal.ppt

This document outlines a research proposal that investigates teachers' perceptions of assessing students' oral reading skills in rural primary schools. The study aims to understand teachers' views on reading aloud assessments, and whether their perceptions differ based on years of teaching experience or education level. A mixed methods approach is proposed, using questionnaires to collect quantitative data on 80-120 teachers, and interviews of 12 teachers to obtain qualitative views. The research questions focus on teachers' perceptions of reading assessments, any differences related to experience or qualifications, and factors influencing perceptions. The significance, limitations, and methodology are described in the proposal.

Sample Methodology
Sample MethodologySample Methodology
Sample Methodology

The document discusses methodology sections in research papers. It provides examples of methodology sections and discusses what they should include. It lists things like when and where the research was conducted, the data collection procedures, criteria for including subjects, a description of surveys used to collect data, and how results will be presented. It also includes multiple links to methodology sections from published research papers that could be used as examples.

Monitoring and evaluation of human rights projects
Monitoring and evaluation of human rights projectsMonitoring and evaluation of human rights projects
Monitoring and evaluation of human rights projects

This document provides information about monitoring and evaluating human rights projects. It discusses the differences between evaluation, monitoring, and auditing. Evaluation assesses project effectiveness, impact, and sustainability for learning and accountability. Monitoring analyzes ongoing project progress toward planned results to improve management. The document also outlines why monitoring and evaluation are important, the project cycle, stakeholder analysis, problem trees, objectives, indicators, risks analysis, and tools for monitoring and evaluating projects.

evaluaceprojectevaluation
Reporting – example of quantitative data
No. of women reached
Participationosvětových akcích
Účast na in awareness raising

1033

Checked-up inv mobilních…
Vyšetřeny mobile clinics

3244

Odeslané do onkocenter
Sent to oncocentres

476

Checked-up in oncocentre
Vyšetřeny v onkocentrech

303

Rakovina
Diagnosed with cancer

31

Diagnosed Prekanceróza
with pre-cancer

48

Treated with cancer
Léčba rakoviny

Léčba with pre-cancer
Treated prekancerózy
Vyléčeno
Cured

20
2

?

Source: Naviga4: Prevention and early detection of women with cancer, Georgia, MFA CR

41
Reporting – example of qualitative data

Project A

Finding: Most volunteers expressed doubts about
the Programme, which partially impacted external
communication and advocacy. They believed the
donor should better clarify the program objectives
– if the programme is to primarily serve the
communities in the South or the young
professionals from Europe.
Source: Inka Pibilova

“I believe the development sector
needs well trained and well managed
professionals, not volunteers sent
with a weak/unclear mandate to 'do
something'. This may end up doing
more harm than good." Volunteer
42
Reporting –– example of qualitative data
Reporting example of qualitative data

Too expensive
treatment...
Project A

...incorrect
Radiation
inaccessible
Chemistry teacher and mother, 26 years old
Cervical precancerosis
Incorrect treatment and relapse
Source: Inka Pibilova

43
Where to learn more?
• Examples of evaluation reports (see bottom of evaluation section) and
other tools at http://www.evaluace.com/ - or contact inka@evaluace.com
• Road to Results
https://openknowledge.worldbank.org/handle/10986/2699
• EPDET development evaluation training 31 August – 6 September 2014 in
Slovakia – check http://www.dww.cz/index.php?page=epdet
• www.Betterevaluation.org
• EC evaluation guidelines
http://ec.europa.eu/europeaid/evaluation/methodology/tools/too_en.ht
m
• UNDP Handbook (p. 194 - 200) ttp://web.undp.org/evaluation/handbook/
• OECD DAC key norms, standards, criteria
http://www.oecd.org/development/evaluation/dcdndep/41612905.pdf
• RISK, UK: How do we know it is working?
http://www.developmenteducationreview.com/issue11-review1
• The Most Significant Change Guide:
http://www.mande.co.uk/docs/MSCGuide.pdf
44

Recommended for you

Masterclass on project building
Masterclass on project buildingMasterclass on project building
Masterclass on project building

Materials from a training for NGOs about how to develop high quality project proposals for EU funding.

PCM - Project Cycle Management, Training on Evaluation
PCM - Project Cycle Management, Training on EvaluationPCM - Project Cycle Management, Training on Evaluation
PCM - Project Cycle Management, Training on Evaluation

The document discusses key concepts in project evaluation including definitions of evaluation, the project cycle, when evaluations should occur, purposes of evaluations, and criteria for evaluating development assistance including relevance, efficiency, effectiveness, impact, and sustainability. It provides examples of questions to consider under each of the five evaluation criteria. The summary focuses on defining the five evaluation criteria - relevance, efficiency, effectiveness, impact, and sustainability.

M&E.ppt
M&E.pptM&E.ppt
M&E.ppt

Monitoring involves continuous assessment of project implementation to provide feedback and identify successes and problems. It focuses on schedules, inputs, and services. Evaluation assesses outcomes, impacts, effectiveness, and sustainability. The document discusses the importance of monitoring and evaluation for improving decision-making, achieving outcomes, and organizational learning. It provides definitions and comparisons of monitoring and evaluation. Participatory approaches are emphasized to empower stakeholders. Clear objectives and indicators are needed to measure progress.

More Related Content

What's hot

Walking Through Your Project Proposal
Walking Through Your Project ProposalWalking Through Your Project Proposal
Walking Through Your Project Proposal
The Gifted School of Basra
 
Writing and Presenting a Project Proposal by Andile Andries Ndlovu
Writing and Presenting  a Project Proposal by Andile Andries NdlovuWriting and Presenting  a Project Proposal by Andile Andries Ndlovu
Writing and Presenting a Project Proposal by Andile Andries Ndlovu
Andile Andries Ndlovu
 
Sample proposal tips
Sample proposal tipsSample proposal tips
Sample proposal tips
ethelmer
 
Project Management Plan - Group Presentation
Project Management Plan - Group PresentationProject Management Plan - Group Presentation
Project Management Plan - Group Presentation
Agatha Maia Duarte de Assis
 
Progress Reporting - notes
Progress Reporting - notesProgress Reporting - notes
Progress Reporting - notes
Tony
 
Ten rules of project risk management
Ten rules of project risk managementTen rules of project risk management
Ten rules of project risk management
Tony
 
Proposal for NGO Capacity
Proposal for NGO CapacityProposal for NGO Capacity
Proposal for NGO Capacity
Consultant
 
Proposal writing sample 2
Proposal writing sample 2Proposal writing sample 2
Proposal writing sample 2
Momy Saikia
 
Planning Effective Linkages
Planning Effective LinkagesPlanning Effective Linkages
Planning Effective Linkages
Ghulam Qadir
 
10 Project Proposal Writing
10 Project Proposal Writing10 Project Proposal Writing
10 Project Proposal Writing
Tony
 
3 proj plan notes
3 proj plan notes3 proj plan notes
3 proj plan notes
Tony
 
3 proj plan handouts
3 proj plan handouts3 proj plan handouts
3 proj plan handouts
Tony
 
Components of Project Proposal
Components of Project ProposalComponents of Project Proposal
Components of Project Proposal
Zera Bai Rajan
 
Logical framework and project proposal
Logical framework and project proposalLogical framework and project proposal
Logical framework and project proposal
Sowad Ahmed
 
Project Proposal Hints
Project  Proposal  HintsProject  Proposal  Hints
Project Proposal Hints
guest6b1035
 
Persuasive Project Proposal Writing for Effective Fundraising
Persuasive Project Proposal Writing for Effective FundraisingPersuasive Project Proposal Writing for Effective Fundraising
Persuasive Project Proposal Writing for Effective Fundraising
Emmanuel Tokpah-Barrow
 
Project Proposal
Project ProposalProject Proposal
Proposal Writing Workshop
Proposal Writing WorkshopProposal Writing Workshop
Proposal Writing Workshop
Lisa Campbell
 
Basic project planning & proposal making
Basic project planning & proposal makingBasic project planning & proposal making
Basic project planning & proposal making
kirkngai
 

What's hot (19)

Walking Through Your Project Proposal
Walking Through Your Project ProposalWalking Through Your Project Proposal
Walking Through Your Project Proposal
 
Writing and Presenting a Project Proposal by Andile Andries Ndlovu
Writing and Presenting  a Project Proposal by Andile Andries NdlovuWriting and Presenting  a Project Proposal by Andile Andries Ndlovu
Writing and Presenting a Project Proposal by Andile Andries Ndlovu
 
Sample proposal tips
Sample proposal tipsSample proposal tips
Sample proposal tips
 
Project Management Plan - Group Presentation
Project Management Plan - Group PresentationProject Management Plan - Group Presentation
Project Management Plan - Group Presentation
 
Progress Reporting - notes
Progress Reporting - notesProgress Reporting - notes
Progress Reporting - notes
 
Ten rules of project risk management
Ten rules of project risk managementTen rules of project risk management
Ten rules of project risk management
 
Proposal for NGO Capacity
Proposal for NGO CapacityProposal for NGO Capacity
Proposal for NGO Capacity
 
Proposal writing sample 2
Proposal writing sample 2Proposal writing sample 2
Proposal writing sample 2
 
Planning Effective Linkages
Planning Effective LinkagesPlanning Effective Linkages
Planning Effective Linkages
 
10 Project Proposal Writing
10 Project Proposal Writing10 Project Proposal Writing
10 Project Proposal Writing
 
3 proj plan notes
3 proj plan notes3 proj plan notes
3 proj plan notes
 
3 proj plan handouts
3 proj plan handouts3 proj plan handouts
3 proj plan handouts
 
Components of Project Proposal
Components of Project ProposalComponents of Project Proposal
Components of Project Proposal
 
Logical framework and project proposal
Logical framework and project proposalLogical framework and project proposal
Logical framework and project proposal
 
Project Proposal Hints
Project  Proposal  HintsProject  Proposal  Hints
Project Proposal Hints
 
Persuasive Project Proposal Writing for Effective Fundraising
Persuasive Project Proposal Writing for Effective FundraisingPersuasive Project Proposal Writing for Effective Fundraising
Persuasive Project Proposal Writing for Effective Fundraising
 
Project Proposal
Project ProposalProject Proposal
Project Proposal
 
Proposal Writing Workshop
Proposal Writing WorkshopProposal Writing Workshop
Proposal Writing Workshop
 
Basic project planning & proposal making
Basic project planning & proposal makingBasic project planning & proposal making
Basic project planning & proposal making
 

Viewers also liked

project on construction of house report.
project on construction of house report.project on construction of house report.
project on construction of house report.
Hagi Sahib
 
Feasibility report -basic concepts with example
Feasibility report -basic concepts with exampleFeasibility report -basic concepts with example
Feasibility report -basic concepts with example
Abhijeet Bhosale
 
Project proposal
Project proposalProject proposal
Project proposal
Jayson Patalinghug
 
Project proposal writing civil engg
Project  proposal  writing civil enggProject  proposal  writing civil engg
Project proposal writing civil engg
बुद्धिसागर बास्तोला
 
Final Year Project proposal (Degree of Agrotech)
Final Year Project proposal (Degree of Agrotech)Final Year Project proposal (Degree of Agrotech)
Final Year Project proposal (Degree of Agrotech)
Asma Sams
 
51432284 project-finance
51432284 project-finance51432284 project-finance
51432284 project-finance
amritakamarath
 
Psychology Internship Report
Psychology Internship ReportPsychology Internship Report
Psychology Internship Report
Saravanan Psyd
 
Building construction-report (1)
Building construction-report (1)Building construction-report (1)
Building construction-report (1)
Soh Shing
 
Chapter 1 overview of financial management
Chapter 1   overview of financial managementChapter 1   overview of financial management
Chapter 1 overview of financial management
Sudipta Saha
 
Acknowledgement
AcknowledgementAcknowledgement
Acknowledgement
Surendra Sonawane
 
Proposal format
Proposal formatProposal format
Proposal format
Mr SMAK
 
My research proposal.ppt
My research proposal.pptMy research proposal.ppt
My research proposal.ppt
nanimamat
 
Sample Methodology
Sample MethodologySample Methodology
Sample Methodology
Aiden Yeh
 

Viewers also liked (13)

project on construction of house report.
project on construction of house report.project on construction of house report.
project on construction of house report.
 
Feasibility report -basic concepts with example
Feasibility report -basic concepts with exampleFeasibility report -basic concepts with example
Feasibility report -basic concepts with example
 
Project proposal
Project proposalProject proposal
Project proposal
 
Project proposal writing civil engg
Project  proposal  writing civil enggProject  proposal  writing civil engg
Project proposal writing civil engg
 
Final Year Project proposal (Degree of Agrotech)
Final Year Project proposal (Degree of Agrotech)Final Year Project proposal (Degree of Agrotech)
Final Year Project proposal (Degree of Agrotech)
 
51432284 project-finance
51432284 project-finance51432284 project-finance
51432284 project-finance
 
Psychology Internship Report
Psychology Internship ReportPsychology Internship Report
Psychology Internship Report
 
Building construction-report (1)
Building construction-report (1)Building construction-report (1)
Building construction-report (1)
 
Chapter 1 overview of financial management
Chapter 1   overview of financial managementChapter 1   overview of financial management
Chapter 1 overview of financial management
 
Acknowledgement
AcknowledgementAcknowledgement
Acknowledgement
 
Proposal format
Proposal formatProposal format
Proposal format
 
My research proposal.ppt
My research proposal.pptMy research proposal.ppt
My research proposal.ppt
 
Sample Methodology
Sample MethodologySample Methodology
Sample Methodology
 

Similar to Introduction to project evaluations for SLOGA / Trialog

Monitoring and evaluation of human rights projects
Monitoring and evaluation of human rights projectsMonitoring and evaluation of human rights projects
Monitoring and evaluation of human rights projects
Inka Pibilova
 
Masterclass on project building
Masterclass on project buildingMasterclass on project building
Masterclass on project building
tamsin.rose
 
PCM - Project Cycle Management, Training on Evaluation
PCM - Project Cycle Management, Training on EvaluationPCM - Project Cycle Management, Training on Evaluation
PCM - Project Cycle Management, Training on Evaluation
rexcris
 
M&E.ppt
M&E.pptM&E.ppt
M&E.ppt
selam49
 
Ngo’s project management
Ngo’s project managementNgo’s project management
Ngo’s project management
Mohamed Ahmed Said
 
Project Design and System Life Cycle Methodologies
Project Design and System Life Cycle MethodologiesProject Design and System Life Cycle Methodologies
Project Design and System Life Cycle Methodologies
Shivi Kashyap
 
Session 1 PMP 4th edition
Session 1 PMP 4th editionSession 1 PMP 4th edition
Session 1 PMP 4th edition
محمد عصمت عبد الرازق
 
13 Project Monitoring And Evaluation
13 Project  Monitoring And  Evaluation13 Project  Monitoring And  Evaluation
13 Project Monitoring And Evaluation
rajkpandey2000
 
13 project monitoring and evaluation
13 project monitoring and evaluation13 project monitoring and evaluation
13 project monitoring and evaluation
rajkpandey2000
 
Daidalos Evaluation ECIME 2007
Daidalos Evaluation  ECIME 2007Daidalos Evaluation  ECIME 2007
Evaluability Assessments and Choice of Evaluation Methods
Evaluability Assessments and Choice of Evaluation MethodsEvaluability Assessments and Choice of Evaluation Methods
Evaluability Assessments and Choice of Evaluation Methods
Debbie_at_IDS
 
Part III. Project evaluation
Part III. Project evaluationPart III. Project evaluation
Part III. Project evaluation
Priscila Montesdeoca
 
M & E Presentation DSK.ppt
M & E Presentation DSK.pptM & E Presentation DSK.ppt
M & E Presentation DSK.ppt
ShafayetHossain30
 
SOEDS, 11th April 2022 How to Evaluate CSR Projects and Programmes.pptx
SOEDS, 11th April 2022 How to Evaluate CSR Projects and Programmes.pptxSOEDS, 11th April 2022 How to Evaluate CSR Projects and Programmes.pptx
SOEDS, 11th April 2022 How to Evaluate CSR Projects and Programmes.pptx
RAKESHNANDAN7
 
Project management
Project managementProject management
Project management
Abhishek Singh
 
How to prepare a good proposal H2020
How to prepare a good proposal H2020How to prepare a good proposal H2020
How to prepare a good proposal H2020
The European GNSS Agency (GSA)
 
Project Sustainability workshop session; Anna Lindh Foundation
Project Sustainability workshop session; Anna Lindh FoundationProject Sustainability workshop session; Anna Lindh Foundation
Project Sustainability workshop session; Anna Lindh Foundation
Karl Donert
 
Logical Framework And Project Proposal
Logical Framework And Project ProposalLogical Framework And Project Proposal
Logical Framework And Project Proposal
rexcris
 
Almm monitoring and evaluation tools draft[1]acm sir revised
Almm monitoring and evaluation tools draft[1]acm sir revisedAlmm monitoring and evaluation tools draft[1]acm sir revised
Almm monitoring and evaluation tools draft[1]acm sir revised
Alberto Mico
 
Project cycle
Project cycleProject cycle
Project cycle
Dhwani Shah
 

Similar to Introduction to project evaluations for SLOGA / Trialog (20)

Monitoring and evaluation of human rights projects
Monitoring and evaluation of human rights projectsMonitoring and evaluation of human rights projects
Monitoring and evaluation of human rights projects
 
Masterclass on project building
Masterclass on project buildingMasterclass on project building
Masterclass on project building
 
PCM - Project Cycle Management, Training on Evaluation
PCM - Project Cycle Management, Training on EvaluationPCM - Project Cycle Management, Training on Evaluation
PCM - Project Cycle Management, Training on Evaluation
 
M&E.ppt
M&E.pptM&E.ppt
M&E.ppt
 
Ngo’s project management
Ngo’s project managementNgo’s project management
Ngo’s project management
 
Project Design and System Life Cycle Methodologies
Project Design and System Life Cycle MethodologiesProject Design and System Life Cycle Methodologies
Project Design and System Life Cycle Methodologies
 
Session 1 PMP 4th edition
Session 1 PMP 4th editionSession 1 PMP 4th edition
Session 1 PMP 4th edition
 
13 Project Monitoring And Evaluation
13 Project  Monitoring And  Evaluation13 Project  Monitoring And  Evaluation
13 Project Monitoring And Evaluation
 
13 project monitoring and evaluation
13 project monitoring and evaluation13 project monitoring and evaluation
13 project monitoring and evaluation
 
Daidalos Evaluation ECIME 2007
Daidalos Evaluation  ECIME 2007Daidalos Evaluation  ECIME 2007
Daidalos Evaluation ECIME 2007
 
Evaluability Assessments and Choice of Evaluation Methods
Evaluability Assessments and Choice of Evaluation MethodsEvaluability Assessments and Choice of Evaluation Methods
Evaluability Assessments and Choice of Evaluation Methods
 
Part III. Project evaluation
Part III. Project evaluationPart III. Project evaluation
Part III. Project evaluation
 
M & E Presentation DSK.ppt
M & E Presentation DSK.pptM & E Presentation DSK.ppt
M & E Presentation DSK.ppt
 
SOEDS, 11th April 2022 How to Evaluate CSR Projects and Programmes.pptx
SOEDS, 11th April 2022 How to Evaluate CSR Projects and Programmes.pptxSOEDS, 11th April 2022 How to Evaluate CSR Projects and Programmes.pptx
SOEDS, 11th April 2022 How to Evaluate CSR Projects and Programmes.pptx
 
Project management
Project managementProject management
Project management
 
How to prepare a good proposal H2020
How to prepare a good proposal H2020How to prepare a good proposal H2020
How to prepare a good proposal H2020
 
Project Sustainability workshop session; Anna Lindh Foundation
Project Sustainability workshop session; Anna Lindh FoundationProject Sustainability workshop session; Anna Lindh Foundation
Project Sustainability workshop session; Anna Lindh Foundation
 
Logical Framework And Project Proposal
Logical Framework And Project ProposalLogical Framework And Project Proposal
Logical Framework And Project Proposal
 
Almm monitoring and evaluation tools draft[1]acm sir revised
Almm monitoring and evaluation tools draft[1]acm sir revisedAlmm monitoring and evaluation tools draft[1]acm sir revised
Almm monitoring and evaluation tools draft[1]acm sir revised
 
Project cycle
Project cycleProject cycle
Project cycle
 

More from Inka Pibilova

Impacts of Czech cancer-fighting projects in Georgia and Serbia: lessons for ...
Impacts of Czech cancer-fighting projects in Georgia and Serbia: lessons for ...Impacts of Czech cancer-fighting projects in Georgia and Serbia: lessons for ...
Impacts of Czech cancer-fighting projects in Georgia and Serbia: lessons for ...
Inka Pibilova
 
Evaluace českého onkologickéo projektu v Srbsku 2015
Evaluace českého onkologickéo projektu v Srbsku 2015 Evaluace českého onkologickéo projektu v Srbsku 2015
Evaluace českého onkologickéo projektu v Srbsku 2015
Inka Pibilova
 
Participativní plánování, monitoring a evaluace
Participativní plánování, monitoring a evaluaceParticipativní plánování, monitoring a evaluace
Participativní plánování, monitoring a evaluace
Inka Pibilova
 
Monitoring zamereny na vysledky a evaluace
Monitoring zamereny na vysledky a evaluace Monitoring zamereny na vysledky a evaluace
Monitoring zamereny na vysledky a evaluace
Inka Pibilova
 
How European Civil Society Organisations Strive for their Development Effecti...
How European Civil Society Organisations Strive for their Development Effecti...How European Civil Society Organisations Strive for their Development Effecti...
How European Civil Society Organisations Strive for their Development Effecti...
Inka Pibilova
 
How European CSOs strive for their development effectiveness
How European CSOs strive for their development effectivenessHow European CSOs strive for their development effectiveness
How European CSOs strive for their development effectiveness
Inka Pibilova
 
Prevence přenosu HIV z matek na děti v Keni
Prevence přenosu HIV z matek na děti v KeniPrevence přenosu HIV z matek na děti v Keni
Prevence přenosu HIV z matek na děti v Keni
Inka Pibilova
 
Using Istanbul CSO Development Effectiveness Principles to evaluate a health ...
Using Istanbul CSO Development Effectiveness Principles to evaluate a health ...Using Istanbul CSO Development Effectiveness Principles to evaluate a health ...
Using Istanbul CSO Development Effectiveness Principles to evaluate a health ...
Inka Pibilova
 
How to plan EC-funded projects
How to plan EC-funded projectsHow to plan EC-funded projects
How to plan EC-funded projects
Inka Pibilova
 
The most significant change as a part of a project evaluation in india
The most significant change as a part of a project evaluation in indiaThe most significant change as a part of a project evaluation in india
The most significant change as a part of a project evaluation in india
Inka Pibilova
 
Pribehy nejvetsich zmen v ramci evaluace projektu v indii
Pribehy nejvetsich zmen v ramci evaluace projektu v indiiPribehy nejvetsich zmen v ramci evaluace projektu v indii
Pribehy nejvetsich zmen v ramci evaluace projektu v indii
Inka Pibilova
 
Gender Analysis of Child Trafficking Prevention and Rehabilitation in Ethiopia
Gender Analysis of Child Trafficking Prevention and Rehabilitation in EthiopiaGender Analysis of Child Trafficking Prevention and Rehabilitation in Ethiopia
Gender Analysis of Child Trafficking Prevention and Rehabilitation in Ethiopia
Inka Pibilova
 
Mutual Peer Reviews between Organisations
Mutual Peer Reviews between Organisations Mutual Peer Reviews between Organisations
Mutual Peer Reviews between Organisations
Inka Pibilova
 
Lessons learnt from evaluations of Development Education / Awareness Raising ...
Lessons learnt from evaluations of Development Education / Awareness Raising ...Lessons learnt from evaluations of Development Education / Awareness Raising ...
Lessons learnt from evaluations of Development Education / Awareness Raising ...
Inka Pibilova
 
Evaluační výzkum v zahraniční rozvojové spolupráci - příležitosti pro student...
Evaluační výzkum v zahraniční rozvojové spolupráci - příležitosti pro student...Evaluační výzkum v zahraniční rozvojové spolupráci - příležitosti pro student...
Evaluační výzkum v zahraniční rozvojové spolupráci - příležitosti pro student...
Inka Pibilova
 
Poučení z evaluací vzdělávacích projektů
Poučení z evaluací vzdělávacích projektůPoučení z evaluací vzdělávacích projektů
Poučení z evaluací vzdělávacích projektů
Inka Pibilova
 
FoRS Code on Effectiveness, Peer Reviews and Development Coffee
FoRS Code on Effectiveness, Peer Reviews and Development CoffeeFoRS Code on Effectiveness, Peer Reviews and Development Coffee
FoRS Code on Effectiveness, Peer Reviews and Development Coffee
Inka Pibilova
 
Evaluace projektu prevence a léčby HIV v Keni
Evaluace projektu prevence a léčby HIV v KeniEvaluace projektu prevence a léčby HIV v Keni
Evaluace projektu prevence a léčby HIV v Keni
Inka Pibilova
 
Evaluation and capacity building tools for Czech CSOs
Evaluation and capacity building tools for Czech CSOsEvaluation and capacity building tools for Czech CSOs
Evaluation and capacity building tools for Czech CSOs
Inka Pibilova
 

More from Inka Pibilova (19)

Impacts of Czech cancer-fighting projects in Georgia and Serbia: lessons for ...
Impacts of Czech cancer-fighting projects in Georgia and Serbia: lessons for ...Impacts of Czech cancer-fighting projects in Georgia and Serbia: lessons for ...
Impacts of Czech cancer-fighting projects in Georgia and Serbia: lessons for ...
 
Evaluace českého onkologickéo projektu v Srbsku 2015
Evaluace českého onkologickéo projektu v Srbsku 2015 Evaluace českého onkologickéo projektu v Srbsku 2015
Evaluace českého onkologickéo projektu v Srbsku 2015
 
Participativní plánování, monitoring a evaluace
Participativní plánování, monitoring a evaluaceParticipativní plánování, monitoring a evaluace
Participativní plánování, monitoring a evaluace
 
Monitoring zamereny na vysledky a evaluace
Monitoring zamereny na vysledky a evaluace Monitoring zamereny na vysledky a evaluace
Monitoring zamereny na vysledky a evaluace
 
How European Civil Society Organisations Strive for their Development Effecti...
How European Civil Society Organisations Strive for their Development Effecti...How European Civil Society Organisations Strive for their Development Effecti...
How European Civil Society Organisations Strive for their Development Effecti...
 
How European CSOs strive for their development effectiveness
How European CSOs strive for their development effectivenessHow European CSOs strive for their development effectiveness
How European CSOs strive for their development effectiveness
 
Prevence přenosu HIV z matek na děti v Keni
Prevence přenosu HIV z matek na děti v KeniPrevence přenosu HIV z matek na děti v Keni
Prevence přenosu HIV z matek na děti v Keni
 
Using Istanbul CSO Development Effectiveness Principles to evaluate a health ...
Using Istanbul CSO Development Effectiveness Principles to evaluate a health ...Using Istanbul CSO Development Effectiveness Principles to evaluate a health ...
Using Istanbul CSO Development Effectiveness Principles to evaluate a health ...
 
How to plan EC-funded projects
How to plan EC-funded projectsHow to plan EC-funded projects
How to plan EC-funded projects
 
The most significant change as a part of a project evaluation in india
The most significant change as a part of a project evaluation in indiaThe most significant change as a part of a project evaluation in india
The most significant change as a part of a project evaluation in india
 
Pribehy nejvetsich zmen v ramci evaluace projektu v indii
Pribehy nejvetsich zmen v ramci evaluace projektu v indiiPribehy nejvetsich zmen v ramci evaluace projektu v indii
Pribehy nejvetsich zmen v ramci evaluace projektu v indii
 
Gender Analysis of Child Trafficking Prevention and Rehabilitation in Ethiopia
Gender Analysis of Child Trafficking Prevention and Rehabilitation in EthiopiaGender Analysis of Child Trafficking Prevention and Rehabilitation in Ethiopia
Gender Analysis of Child Trafficking Prevention and Rehabilitation in Ethiopia
 
Mutual Peer Reviews between Organisations
Mutual Peer Reviews between Organisations Mutual Peer Reviews between Organisations
Mutual Peer Reviews between Organisations
 
Lessons learnt from evaluations of Development Education / Awareness Raising ...
Lessons learnt from evaluations of Development Education / Awareness Raising ...Lessons learnt from evaluations of Development Education / Awareness Raising ...
Lessons learnt from evaluations of Development Education / Awareness Raising ...
 
Evaluační výzkum v zahraniční rozvojové spolupráci - příležitosti pro student...
Evaluační výzkum v zahraniční rozvojové spolupráci - příležitosti pro student...Evaluační výzkum v zahraniční rozvojové spolupráci - příležitosti pro student...
Evaluační výzkum v zahraniční rozvojové spolupráci - příležitosti pro student...
 
Poučení z evaluací vzdělávacích projektů
Poučení z evaluací vzdělávacích projektůPoučení z evaluací vzdělávacích projektů
Poučení z evaluací vzdělávacích projektů
 
FoRS Code on Effectiveness, Peer Reviews and Development Coffee
FoRS Code on Effectiveness, Peer Reviews and Development CoffeeFoRS Code on Effectiveness, Peer Reviews and Development Coffee
FoRS Code on Effectiveness, Peer Reviews and Development Coffee
 
Evaluace projektu prevence a léčby HIV v Keni
Evaluace projektu prevence a léčby HIV v KeniEvaluace projektu prevence a léčby HIV v Keni
Evaluace projektu prevence a léčby HIV v Keni
 
Evaluation and capacity building tools for Czech CSOs
Evaluation and capacity building tools for Czech CSOsEvaluation and capacity building tools for Czech CSOs
Evaluation and capacity building tools for Czech CSOs
 

Recently uploaded

Split Shifts From Gantt View in the Odoo 17
Split Shifts From Gantt View in the  Odoo 17Split Shifts From Gantt View in the  Odoo 17
Split Shifts From Gantt View in the Odoo 17
Celine George
 
AI_in_HR_Presentation Part 1 2024 0703.pdf
AI_in_HR_Presentation Part 1 2024 0703.pdfAI_in_HR_Presentation Part 1 2024 0703.pdf
AI_in_HR_Presentation Part 1 2024 0703.pdf
SrimanigandanMadurai
 
BRIGADA ESKWELA OPENING PROGRAM KICK OFF.pptx
BRIGADA ESKWELA OPENING PROGRAM KICK OFF.pptxBRIGADA ESKWELA OPENING PROGRAM KICK OFF.pptx
BRIGADA ESKWELA OPENING PROGRAM KICK OFF.pptx
kambal1234567890
 
L1 L2- NLC PPT for Grade 10 intervention
L1 L2- NLC PPT for Grade 10 interventionL1 L2- NLC PPT for Grade 10 intervention
L1 L2- NLC PPT for Grade 10 intervention
RHODAJANEAURESTILA
 
Understanding and Interpreting Teachers’ TPACK for Teaching Multimodalities i...
Understanding and Interpreting Teachers’ TPACK for Teaching Multimodalities i...Understanding and Interpreting Teachers’ TPACK for Teaching Multimodalities i...
Understanding and Interpreting Teachers’ TPACK for Teaching Multimodalities i...
Neny Isharyanti
 
How to Install Theme in the Odoo 17 ERP
How to  Install Theme in the Odoo 17 ERPHow to  Install Theme in the Odoo 17 ERP
How to Install Theme in the Odoo 17 ERP
Celine George
 
NLC English 7 Consolidation Lesson plan for teacher
NLC English 7 Consolidation Lesson plan for teacherNLC English 7 Consolidation Lesson plan for teacher
NLC English 7 Consolidation Lesson plan for teacher
AngelicaLubrica
 
Views in Odoo - Advanced Views - Pivot View in Odoo 17
Views in Odoo - Advanced Views - Pivot View in Odoo 17Views in Odoo - Advanced Views - Pivot View in Odoo 17
Views in Odoo - Advanced Views - Pivot View in Odoo 17
Celine George
 
Conducting exciting academic research in Computer Science
Conducting exciting academic research in Computer ScienceConducting exciting academic research in Computer Science
Conducting exciting academic research in Computer Science
Abhik Roychoudhury
 
Capitol Doctoral Presentation -June 2024v2.pptx
Capitol Doctoral Presentation -June 2024v2.pptxCapitol Doctoral Presentation -June 2024v2.pptx
Capitol Doctoral Presentation -June 2024v2.pptx
CapitolTechU
 
Beyond the Advance Presentation for By the Book 9
Beyond the Advance Presentation for By the Book 9Beyond the Advance Presentation for By the Book 9
Beyond the Advance Presentation for By the Book 9
John Rodzvilla
 
AI Risk Management: ISO/IEC 42001, the EU AI Act, and ISO/IEC 23894
AI Risk Management: ISO/IEC 42001, the EU AI Act, and ISO/IEC 23894AI Risk Management: ISO/IEC 42001, the EU AI Act, and ISO/IEC 23894
AI Risk Management: ISO/IEC 42001, the EU AI Act, and ISO/IEC 23894
PECB
 
Is Email Marketing Really Effective In 2024?
Is Email Marketing Really Effective In 2024?Is Email Marketing Really Effective In 2024?
Is Email Marketing Really Effective In 2024?
Rakesh Jalan
 
ENGLISH-7-CURRICULUM MAP- MATATAG CURRICULUM
ENGLISH-7-CURRICULUM MAP- MATATAG CURRICULUMENGLISH-7-CURRICULUM MAP- MATATAG CURRICULUM
ENGLISH-7-CURRICULUM MAP- MATATAG CURRICULUM
HappieMontevirgenCas
 
Final ebook Keeping the Memory @live.pdf
Final ebook Keeping the Memory @live.pdfFinal ebook Keeping the Memory @live.pdf
Final ebook Keeping the Memory @live.pdf
Zuzana Mészárosová
 
(T.L.E.) Agriculture: Essentials of Gardening
(T.L.E.) Agriculture: Essentials of Gardening(T.L.E.) Agriculture: Essentials of Gardening
(T.L.E.) Agriculture: Essentials of Gardening
MJDuyan
 
Credit limit improvement system in odoo 17
Credit limit improvement system in odoo 17Credit limit improvement system in odoo 17
Credit limit improvement system in odoo 17
Celine George
 
Front Desk Management in the Odoo 17 ERP
Front Desk  Management in the Odoo 17 ERPFront Desk  Management in the Odoo 17 ERP
Front Desk Management in the Odoo 17 ERP
Celine George
 
Howe Writing Center - Orientation Summer 2024
Howe Writing Center - Orientation Summer 2024Howe Writing Center - Orientation Summer 2024
Howe Writing Center - Orientation Summer 2024
Elizabeth Walsh
 
"DANH SÁCH THÍ SINH XÉT TUYỂN SỚM ĐỦ ĐIỀU KIỆN TRÚNG TUYỂN ĐẠI HỌC CHÍNH QUY ...
"DANH SÁCH THÍ SINH XÉT TUYỂN SỚM ĐỦ ĐIỀU KIỆN TRÚNG TUYỂN ĐẠI HỌC CHÍNH QUY ..."DANH SÁCH THÍ SINH XÉT TUYỂN SỚM ĐỦ ĐIỀU KIỆN TRÚNG TUYỂN ĐẠI HỌC CHÍNH QUY ...
"DANH SÁCH THÍ SINH XÉT TUYỂN SỚM ĐỦ ĐIỀU KIỆN TRÚNG TUYỂN ĐẠI HỌC CHÍNH QUY ...
thanhluan21
 

Recently uploaded (20)

Split Shifts From Gantt View in the Odoo 17
Split Shifts From Gantt View in the  Odoo 17Split Shifts From Gantt View in the  Odoo 17
Split Shifts From Gantt View in the Odoo 17
 
AI_in_HR_Presentation Part 1 2024 0703.pdf
AI_in_HR_Presentation Part 1 2024 0703.pdfAI_in_HR_Presentation Part 1 2024 0703.pdf
AI_in_HR_Presentation Part 1 2024 0703.pdf
 
BRIGADA ESKWELA OPENING PROGRAM KICK OFF.pptx
BRIGADA ESKWELA OPENING PROGRAM KICK OFF.pptxBRIGADA ESKWELA OPENING PROGRAM KICK OFF.pptx
BRIGADA ESKWELA OPENING PROGRAM KICK OFF.pptx
 
L1 L2- NLC PPT for Grade 10 intervention
L1 L2- NLC PPT for Grade 10 interventionL1 L2- NLC PPT for Grade 10 intervention
L1 L2- NLC PPT for Grade 10 intervention
 
Understanding and Interpreting Teachers’ TPACK for Teaching Multimodalities i...
Understanding and Interpreting Teachers’ TPACK for Teaching Multimodalities i...Understanding and Interpreting Teachers’ TPACK for Teaching Multimodalities i...
Understanding and Interpreting Teachers’ TPACK for Teaching Multimodalities i...
 
How to Install Theme in the Odoo 17 ERP
How to  Install Theme in the Odoo 17 ERPHow to  Install Theme in the Odoo 17 ERP
How to Install Theme in the Odoo 17 ERP
 
NLC English 7 Consolidation Lesson plan for teacher
NLC English 7 Consolidation Lesson plan for teacherNLC English 7 Consolidation Lesson plan for teacher
NLC English 7 Consolidation Lesson plan for teacher
 
Views in Odoo - Advanced Views - Pivot View in Odoo 17
Views in Odoo - Advanced Views - Pivot View in Odoo 17Views in Odoo - Advanced Views - Pivot View in Odoo 17
Views in Odoo - Advanced Views - Pivot View in Odoo 17
 
Conducting exciting academic research in Computer Science
Conducting exciting academic research in Computer ScienceConducting exciting academic research in Computer Science
Conducting exciting academic research in Computer Science
 
Capitol Doctoral Presentation -June 2024v2.pptx
Capitol Doctoral Presentation -June 2024v2.pptxCapitol Doctoral Presentation -June 2024v2.pptx
Capitol Doctoral Presentation -June 2024v2.pptx
 
Beyond the Advance Presentation for By the Book 9
Beyond the Advance Presentation for By the Book 9Beyond the Advance Presentation for By the Book 9
Beyond the Advance Presentation for By the Book 9
 
AI Risk Management: ISO/IEC 42001, the EU AI Act, and ISO/IEC 23894
AI Risk Management: ISO/IEC 42001, the EU AI Act, and ISO/IEC 23894AI Risk Management: ISO/IEC 42001, the EU AI Act, and ISO/IEC 23894
AI Risk Management: ISO/IEC 42001, the EU AI Act, and ISO/IEC 23894
 
Is Email Marketing Really Effective In 2024?
Is Email Marketing Really Effective In 2024?Is Email Marketing Really Effective In 2024?
Is Email Marketing Really Effective In 2024?
 
ENGLISH-7-CURRICULUM MAP- MATATAG CURRICULUM
ENGLISH-7-CURRICULUM MAP- MATATAG CURRICULUMENGLISH-7-CURRICULUM MAP- MATATAG CURRICULUM
ENGLISH-7-CURRICULUM MAP- MATATAG CURRICULUM
 
Final ebook Keeping the Memory @live.pdf
Final ebook Keeping the Memory @live.pdfFinal ebook Keeping the Memory @live.pdf
Final ebook Keeping the Memory @live.pdf
 
(T.L.E.) Agriculture: Essentials of Gardening
(T.L.E.) Agriculture: Essentials of Gardening(T.L.E.) Agriculture: Essentials of Gardening
(T.L.E.) Agriculture: Essentials of Gardening
 
Credit limit improvement system in odoo 17
Credit limit improvement system in odoo 17Credit limit improvement system in odoo 17
Credit limit improvement system in odoo 17
 
Front Desk Management in the Odoo 17 ERP
Front Desk  Management in the Odoo 17 ERPFront Desk  Management in the Odoo 17 ERP
Front Desk Management in the Odoo 17 ERP
 
Howe Writing Center - Orientation Summer 2024
Howe Writing Center - Orientation Summer 2024Howe Writing Center - Orientation Summer 2024
Howe Writing Center - Orientation Summer 2024
 
"DANH SÁCH THÍ SINH XÉT TUYỂN SỚM ĐỦ ĐIỀU KIỆN TRÚNG TUYỂN ĐẠI HỌC CHÍNH QUY ...
"DANH SÁCH THÍ SINH XÉT TUYỂN SỚM ĐỦ ĐIỀU KIỆN TRÚNG TUYỂN ĐẠI HỌC CHÍNH QUY ..."DANH SÁCH THÍ SINH XÉT TUYỂN SỚM ĐỦ ĐIỀU KIỆN TRÚNG TUYỂN ĐẠI HỌC CHÍNH QUY ...
"DANH SÁCH THÍ SINH XÉT TUYỂN SỚM ĐỦ ĐIỀU KIỆN TRÚNG TUYỂN ĐẠI HỌC CHÍNH QUY ...
 

Introduction to project evaluations for SLOGA / Trialog

  • 1. Workshop Introduction to project evaluations Inka Píbilová inka@evaluace.com 10 February 2014 1
  • 2. Climbing Kilimanjaro (5,895 m) in 1 year Source: Suzanne Field 2
  • 3. EC Project cycle management Evaluation Implementation and monitoring Financing Programming Identification Formulation http://www.europa.eu.int/comm/europeaid/qsm/index_en.htm 3
  • 5. Use both quantitative and qualitative indicators Compare using trends (increase), thresholds (min. 30%), targets (strategy by 12/Y1) Quantitative - SMARTER Specific / Simple (to understand, collect) Measurable Attainable/Available at cceptable costs Relevant to project / stakeholders Time-bound Evaluate/Engaging Reevaluate/Recordable Qualitative - SPICED Subjective Participatory Interpreted and communicable Cross-checked and compared Empowering Diverse / disaggregated (by gender) Min. 30 % of participants initiate a project aiming to address a local issue. Reasons why participants have (not) implemented a project to address a local issue http://www.smarttoolkit.net/?q=node/391 http://www.europa.eu.int/comm/europeaid/qsm/index_en.htm 5
  • 6. Monitoring x evaluation x audit Evaluation • Assessment of project efficiency, effectiveness, impact, relevance and sustainability for the purpose of learning and accountability to stakeholders Monitoring • Ongoing analysis of project progress towards achieving planned results with the purpose of improving management decision making Audit • Assessment of (i) the legality and regularity of project expenditure and income i.e. compliance with laws and regulations and with applicable contractual rules and criteria; (ii) whether project funds have been used efficiently and economically i.e. in accordance with sound financial management;; and (iii) whether project funds have been used effectively i.e. for purposes intended. • Primarily a financial and financial management focus, with the focus of effectiveness being on project results. http://www.europa.eu.int/comm/europeaid/qsm/index_en.htm 6
  • 7. Evaluation principles for a HORRIBLE evaluation Source: Participants of the workshop 7
  • 8. Evaluation principles for GREAT evaluations Source: Participants of the workshop 8
  • 9. Evaluation standards and principles • EC http://ec.europa.eu/europeaid/ – Impartiality and independence of the evaluation process from the programming and implementation functions; – Credibility of the evaluation, through use of appropriately skilled and independent experts and the transparency of the evaluation process, including wide dissemination of results; – Participation of stakeholders in the evaluation process, to ensure different perspectives and views are taken into account; and – Usefulness of the evaluation findings and recommendations, through timely presentation of relevant, clear and concise information to decision makers. • OECD DAC key norms, standards, criteria http://www.oecd.org/development/evaluation/dcdndep/41612905.pdf • UNDP Evaluation policy http://web.undp.org/evaluation/policy.htm • IPDET Handbook on Evaluation Ethics, Politics, Standards, Principles http://dmeforpeace.org/sites/default/files/M14_NA.pdf • Patton, Michael Quinn.(2008) Utilization-Focused Evaluation: 4th edition 9
  • 10. Source: Participants of the workshop 10
  • 11. Evaluation Terms of Reference - structure • • • • • • • • • Background (project, context, stakeholders) Purpose Use of outputs Scope and focus Evaluation criteria and questions Methodology Timetable Budget Human resources – responsibilities, expertise required Guidelines: • UNDP Handbook (p. 194 - 200) ttp://web.undp.org/evaluation/handbook/ • EC/EuropeAid Project Cycle Management Guidelines (p. 126 - 136) http://ec.europa.eu/europeaid/multimedia/publications/documents/tools/ europeaid_adm_pcm_guidelines_2004_en.pdf • New Zealand Guide 11 http://dmeforpeace.org/sites/default/files/NZAID_ToR%20Guideline.pdf
  • 12. Problem Tree Example: River Pollution http://www.europa.eu.int/comm/europeaid/qsm/index_en.htm 12
  • 13. Stakeholder Analysis Matrix Example: River Pollution http://www.europa.eu.int/comm/europeaid/qsm/index_en.htm 13
  • 14. Influence /Power Stakeholder Analysis – Power / Interest / Attitude KEEP SATISFIED INFLUENCE & ENGAGE Attitude Policy makers CSOs MONITOR Activists KEEP INFORMED Interest 14
  • 15. Evaluation Terms of Reference - example Background (project, context) - project name, identification - history of the project, objectives, results, key activities, progress over time (add logical framework if you wish – you get more tailored proposals) - organisational, social and political context in which the evaluation occurs - main stakeholders involved in the project incl. target groups, beneficiaries, partners, donors - focus and scope of the evaluation – which project components, geographical area, time period, target groups etc. Our project: 15
  • 16. Evaluation Terms of Reference - example The rationale and purpose of the evaluation - Why the evaluation is being undertaken (what do you want to get out of it) including for accountability, learning, improvement - Why now Use of outputs - How it will benefit the different stakeholders - To whom, when and how the findings will be reported (debriefing, presentation, report, videos, posters – printed or on-line…) Our project: 16
  • 17. Methodology - sources of evaluation questions • OECD/DAC evaluation criteria (similar to the EC/EuropeAid) – using project logical framework • Questions, concerns and values of stakeholders • Previous research / evaluations • Guidelines / Evaluation Tools such as Kirkpatrick Model • Experts 17 Source: Road to Results
  • 18. How to measure learning outcomes? http://leanlearning.wikispaces.com/learning_analytics 18
  • 20. OECD/DAC Evaluation Criteria Relevance – are we doing the right things? The appropriateness of project objectives to the problems that it was supposed to address, and to the physical and policy environment within which it operated. The extent to which the project is suited to the priorities and policies of the target group, recipient and donor. It should include and including an assessment of the quality of project preparation and design – i.e. the logic and completeness of the project planning process, and the internal logic and coherence of the project design. Potential evaluation questions: • To what extent are the objectives of the programme still valid? • Are the activities and outputs of the programme consistent with the overall goal and the attainment of its objectives? • Are the activities and outputs of the programme consistent with the intended impacts and effects? 20 Source: OCEED/DAC, http://www.europa.eu.int/comm/europeaid/qsm/index_en.htm
  • 21. OECD/DAC Evaluation Criteria Effectiveness – are we doing things right? An assessment of the contribution made by results to achievement of the Project Purpose, and how Assumptions have affected project achievements. This should include specific assessment of the benefits accruing to target groups, including women and men and identified vulnerable groups such as children, the elderly and disabled. Potential evaluation questions: • To what extent were the objectives achieved / are likely to be achieved? • What were the major factors influencing the achievement or nonachievement of the objectives? 21 Source: OCEED/DAC, http://www.europa.eu.int/comm/europeaid/qsm/index_en.htm
  • 22. OECD/DAC Evaluation Criteria Efficiency – is the project worthwhile? The fact that the project results have been achieved at reasonable cost, i.e. how wellinputs/means have been converted into activities, in terms of quality, quantity and time, and the quality of the results achieved. The project shou use the least costly resources possible in order to achieve the desired results. This generally requires comparing alternative approaches to achieving the same outputs, to see whether the most efficient process has been adopted. Potential evaluation questions: • Were activities cost-efficient? • Were objectives achieved on time? • Was the programme or project implemented in the most efficient way compared to alternatives? 22 Source: OCEED/DAC, http://www.europa.eu.int/comm/europeaid/qsm/index_en.htm
  • 23. OECD/DAC Evaluation Criteria Impact – what changes has the project achieved / contributed to? The effect of the project on its wider environment, and its contribution to the wider policy or sector objectives (as summarised in the project’s Overall Objective). The positive and negative changes produced by a project, directly or indirectly, intended or unintended, positive and negative. This involves the main impacts and effects resulting from the project on the local social, economic, environmental and other development indicators. It must also include the positive and negative impact of external factors. Potential evaluation questions: • What has happened as a result of the programme or project? • What real difference has the activity made to the beneficiaries? • How many people have been affected? Source: OCEED/DAC, http://www.europa.eu.int/comm/europeaid/qsm/index_en.htm 23
  • 24. OECD/DAC Evaluation Criteria Sustainability – wil the changes (nebefits for target group) last? An assessment of the likelihood of benefits produced by the project to continue to flow after external funding has ended, and with particular reference to factors of ownership by beneficiaries, policy support, economic and financial factors, socio-cultural aspects, gender equality, appropriate technology, environmental aspects, and institutional and management capacity. Sustainability is concerned with measuring whether the benefits of a project are likely to continue after donor funding has been withdrawn. Projects need to be environmentally as well as financially (and socially) sustainable. Potential evaluation questions: • To what extent did the benefits of a programme or project continue after donor funding ceased? • What were the major factors which influenced the achievement or nonachievement of sustainability of the programme or project? 24 Source: OCEED/DAC, http://www.europa.eu.int/comm/europeaid/qsm/index_en.htm
  • 25. Evaluation Questions - Descriptive • • • • What „is“ Describe project (inputs, activities, outputs) or process Simple: Who, what, where, when, how, how many … Often used to gather opinions from target groups. Examples: • What is the project objective from the perspectives of different stakeholders? • What were the reasons for joining the program? • How many persons were reached? • How was the project implemented? Evaluation design to answer them: One-shot, before-and-after, time series, (long-term) panel, case studies… Source: Road to Results 25
  • 26. Evaluation Questions - Normative • Compare what „is “ with what „should be“ (target) Examples: • Do the project activities address the needs of stakeholders? • To what extent has the project achieved the result/objective indicators? • Have min. 5.000 persons been reached through the campaign? • Has the number of schools involved in Global Learning increased? (baseline!) Evaluation design to answer them: One-shot, before-and-after, time series, (long-term) panel, case studies… Source: Road to Results 26
  • 27. Evaluation Questions – Cause-and-Effect • • • • Determine what difference the project makes - what change has it brought Often refer to outcome, impact Compare indicators before and after, with and without the project (graph) Careful about attribution x contribution! – Can you say that the project achieved this or has contributed to this? – Are there any alternative explanations (external factors) for achievements? Examples: • As a result of the training, have teachers incorporated Global Learning in their lesson plans? Evaluation design to answer them: Experimental (control group), quasi experimental (compare group), nonexperimental (causual tracing, case study, story harvesting, outcome mapping…) Goal with without 27 Source: Road to Results T=0 T=1 Time
  • 28. Evaluation Terms of Reference – Evaluation Matrix • Evaluation criteria and questions Design, Data Our project – Question Baseline data Indicators Sources Collection Methods 28
  • 29. Tips for evaluation questions • Avoid questions on multiple issues – separate these – Has the methodology been developed and disseminated to min. 5.000 teachers? • You can develop subquestions for a particular question on an issue – What concerns have teachers while introducing Global Learning at school ? (descriptive) – Has the project addressed these concerns of the teachers? (normative) – Has the number of teachers using Global Learning increased as a result of the project? (cause-and-effect) • Set a realistic number of questions! Source: Road to Results 29
  • 30. Select a few, most important questions…how? Would the evaluation question… Q1 Q2 Q3.. • • • • • • • • be in line with the evaluation purpose? be of interest to stakeholders? reduce present uncertainty? yield important information? be of continuing (not fl eeting) interest? be critical to the evaluation´s scope and comprehensiveness? (or nice to have) have an impact on the course of events? be answerable given the fi nancial and human resources, time, methods, and technology available? • Be reasonable to ask given the project cycle? (Questions about impact, for example, are best answered after the project has been fully operational for a few years) Source: Road to Results, Kusters et al.: Making evaluations Matter 30
  • 31. Data Collection Methods - Quantitative x Qualitative x Mixed Sampling random x non-random (quota, typical case, snow-ball…), sample size! Participatory – community meetings, mapping, transect walks.. Conversation with Concerned Individuals Expert (panel) Key Informant judgement Interviews Delhi technique Participant Observations: Community Diaries, Interviews Reviews journals, selfreported of checklists Official FieldRecords Visits Panel Surveys Focus Group Interviews One-Time Surveys Direct Observations Informal/Less Structured Methods Source: Road to Results / IPDET © 2009, adopted Censuses Field Experiments Surveys Formal/More Structured Methods 31
  • 32. Evaluator Selection Interviews Surveys Focus groups Case studies Final debriefing of all partners Draft evaluation report commented by all partners Final evaluation report Desk study Preliminary findings & conclusions Initial briefing and inception Terms of Reference – Objectives, scope, stakeholders, questions, budget, schedule, outputs, use. Timeline and resources Communication with the Project Partners Inception phase Field research Reporting phase 1-3 months 1-3 months 1-2 months 32
  • 33. Internal evaluators or ₊ May have a better understanding of the projetct, context, policies ₊ Develop organisational capacities ₊ Higher ownership of recommendations by the organisation ₊ Usually cheaper ₋ May not be able to see alternative perspectives, solutions ₋ Influenced more by the implementing organisation (want to keep their jobs) ₋ May be less credible to stakeholders ₋ May be time-consuming external evaluators ₊ May bring a new perspective or special (technical, evaluation) expertise ₊ More independent from the implementer – may facilitate better between stakeholders (across hierarchies, in case of mistrust) ₊ Usually perceived as more credible ₋ May not be able to comprehend fully the project due to time/other contrains ₋ Usually more expensive or mixed - participatory ! External evaluation is not necessarily independent! (Who pays the evaluator? Who checks the quality?) ! Even external evaluation consumes time of the project team! 33 ! It depends WHO the evaluator is 
  • 34. Evaluation Terms of Reference - example • Timetable • Budget • Human resources – responsibilities, expertise required Our project: 34
  • 35. Are we ready? Evaluability - Are clear why we do the evaluation? - Do we have an (updated) logical framework? - Do we have sufficient (baseline, monitoring) data available? - Do we have accessible reliable information sources? - Do we have sufficient funds for an internal/external evaluation? Will the evaluation be cost-effective, will it bring reasonable benefits vs. costs? - Is it likely that it will be used to improve actions in future? Can stakeholders influence the evaluation decisions? Will they accept and use the findings? Is there a strong leadership to put the recommendations in practice? - Are there no major factors hindering the evaluation? Are staff members or other stakeholders overloaded due to other priorities? Are there any tendencies that would affect impartiality? Source: UNDP Handbook, Road to Results, Making evaluations matter 35
  • 36. Qualitative Data Analysis – frequency and relations Evaluation question: Were parents satisfied with the training? Topics Quotes Findings Parents decide on topics I think the process of deciding would be valuable. There was a strong feeling that parents should be more involved in the choice of topics Sometimes we just got into a topic and then it was time to leave or move to something else. We need more time to discuss. Many participants (38 or 52 interviewed) thought there should be more time for discussion. //// //// //// //// //// //// //// //// Cover a couple of topics per session //// //// //// /// Not enough time spent on each topic //// //// //// /// Develop categories, check reliability, analyse, interpret, share draft, report Source: IPDET © 2009 36
  • 37. Quantitative Data Analysis • Clean data • Use mode (most frequent), median (middle value), mean (average), range (lowest to highest value), statistical methods (correlations…) • Produce tables, graphs etc Source: IPDET © 2009, docs. Google.com 37
  • 38. Data Analysis and Interpreting • Needs to be clear before data collection • Methodology incl. reliability and limits is a part of report • Categorize and triangulate findings • Use quotes, examples, graphs • Distinguiish between findings (evidence) and interpreting (conclusions) • Do not generalize findings from 3 respondents to the whole sector! • Have a short summary for those who cannot read the whole report Source: Road to Results, evaluace.com 38
  • 39. Reporting: as per the expected use by each stakeholder Case study 39
  • 40. Reporting – example of quantitative data the Project A Project A Project A Project A Project A Source: Inka Pibilova 40
  • 41. Reporting – example of quantitative data No. of women reached Participationosvětových akcích Účast na in awareness raising 1033 Checked-up inv mobilních… Vyšetřeny mobile clinics 3244 Odeslané do onkocenter Sent to oncocentres 476 Checked-up in oncocentre Vyšetřeny v onkocentrech 303 Rakovina Diagnosed with cancer 31 Diagnosed Prekanceróza with pre-cancer 48 Treated with cancer Léčba rakoviny Léčba with pre-cancer Treated prekancerózy Vyléčeno Cured 20 2 ? Source: Naviga4: Prevention and early detection of women with cancer, Georgia, MFA CR 41
  • 42. Reporting – example of qualitative data Project A Finding: Most volunteers expressed doubts about the Programme, which partially impacted external communication and advocacy. They believed the donor should better clarify the program objectives – if the programme is to primarily serve the communities in the South or the young professionals from Europe. Source: Inka Pibilova “I believe the development sector needs well trained and well managed professionals, not volunteers sent with a weak/unclear mandate to 'do something'. This may end up doing more harm than good." Volunteer 42
  • 43. Reporting –– example of qualitative data Reporting example of qualitative data Too expensive treatment... Project A ...incorrect Radiation inaccessible Chemistry teacher and mother, 26 years old Cervical precancerosis Incorrect treatment and relapse Source: Inka Pibilova 43
  • 44. Where to learn more? • Examples of evaluation reports (see bottom of evaluation section) and other tools at http://www.evaluace.com/ - or contact inka@evaluace.com • Road to Results https://openknowledge.worldbank.org/handle/10986/2699 • EPDET development evaluation training 31 August – 6 September 2014 in Slovakia – check http://www.dww.cz/index.php?page=epdet • www.Betterevaluation.org • EC evaluation guidelines http://ec.europa.eu/europeaid/evaluation/methodology/tools/too_en.ht m • UNDP Handbook (p. 194 - 200) ttp://web.undp.org/evaluation/handbook/ • OECD DAC key norms, standards, criteria http://www.oecd.org/development/evaluation/dcdndep/41612905.pdf • RISK, UK: How do we know it is working? http://www.developmenteducationreview.com/issue11-review1 • The Most Significant Change Guide: http://www.mande.co.uk/docs/MSCGuide.pdf 44