Project Evaluation and Auditing

(Chapter 10: 6 hours)

Prof. Dr. Hari K. Shrestha

Nepal Engineering College
Changunarayan Municipality-4, Bhaktapur
hari@nec.edu.np; hari.k.shrestha@gmail.com
Update: November 21, 2018
 Project Evaluation
• Project Evaluation is a step-by-step process of collecting, recording
and organizing information about project results, including short-
term outputs (immediate results of activities, or project
deliverables), and immediate and longer-term project outcomes
(changes in behavior, practice or policy resulting from the project).
• Definition depends on type/objective of organization/project.
• Evaluation involves the systematic collection of information about
the activities, characteristic and outcomes of an activity or action, in
order to determine its worth or merit (Dart et al, 1998). It is a major
part of learning, and can provide a wealth of useful information on
the outcomes of a project or action, and the dynamics of those who
undertook the work. Through the identification of the highlights and
lowlights of the project, evaluation draws conclusions which can
inform future decision making, and assist to define future projects
and policies (Patton, 1997).
 Why Project Evaluation?
Evaluation of projects are undertaken for a number of reasons:

• to contribute information, for example for the review of a management strategy

• to be accountable to those who are funding the project
• as a debrief to the group on the completion of a project
• to reflect on how you’re developing as a group or organisation, and whether you need to
improve the way things are being done
• to review and adjust the management methods and techniques used in an area
• to develop the skills and understanding of people involved in a project by enabling them to
reflect on, and learn from, their experiences
• to provide information for planning future projects
• to determine the worth of the project
• to justify and promote a management action to the wider community
• to create a historical record of management success over time
• to review management methods and techniques, and how effective they were
• to see if the work is contributing to the management objectives for an area
• to determine whether you have met project objectives.
• It is important to be aware of the reasons for undertaking an evaluation before commencing, as
this will define how the evaluation process should be designed, and what outcomes are expected
as a result.
http://www.planning.wa.gov.au/dop_pub_pdf/5_project_evaluation.pdf
 Project Types
Successful :
• The project is completed on time and on budget, with all features
and functions originally specified

Challenged :
• The project is completed and operational, but over-budget, over
the time estimate, and with fewer features and functions than
initially specified.

Failed :
• The project is cancelled before completion or never implemented."
 Types of Evaluation
Based on Timing
• Ex-ante evaluation (before major investment and construction activities initiated)
• Mid-term evaluation
• Ex-Post evaluation (after major investment and construction activities completed)
Based on position of evaluator
• External evaluation
• Internal evaluation or self-assessment
Based on Purpose
• Formative (during the project implementation)
• Summative (after the project implementation)
 Project Evaluation Criteria (Example)
Link to Strategic Imperatives
Active Sponsor Engagement https://www.isixsigma.com/implementatio
Team Actively Engaged n/project-selection-tracking/10-criteria-
Broad Organizational Awareness of the Project use-evaluating-six-sigma-projects/
Project Delivered the Anticipated Results
• Low – The deliverables of the project do not meet the expectations.
• Medium – Project has made substantial progress towards the initial goals , but overall the deliverables fall short of
expectations.
• High – The project delivered the promised results.
Project Completed on Time
• Low – While the project was eventually completed, the overall duration exceeded the initial schedule by
far.
• Medium – Project has been completed within a reasonable amount of time, but the team has been
struggling to complete specific phases.
• High – The team completed the project within the allotted time and the project leader has managed the
project effectively.
Successful Transition of Ownership to Process Owner
Improvement Sustained Over Time
Replication of Results
• Low –  The team has not conducted a thorough analysis of whether and how the results of this project could be
replicated.
• Medium – The team has identified opportunities for replicating the results of the original projects but does not
have a plan on how to make this happen.
• High – The team has developed a thorough plan that not only shows how the improvements could be replicated
but also who will be involved.
Example of a project evaluation
http://unicef.org.np/uploads/files/46671265533
7210343-aligning-for-action.pdf
 Methods of Financial Evaluations
(methods based on cash flow forecasts)

The Five Methods/criteria more often used for financial

evaluation of a project are:
1. Simple Rate of return (SRR)
2. Payback Period (PBP) / Discounted Payback Period
3. Benefit Cost Ratio (BCR)/ Profitability Index (PI)
4. Net Present Value (NPV) or Net Present Worth(NPW)
5. Adjusted Present Value
6. Internal Rate of Return (IRR).
a) Financial IRR
b) Economic IRR
https://
www.businessmanagementideas.com/investment/proposals-investment/evaluation-of-investme
nt-proposals-7-methods-financial-management/16523
 Challenges in Project Evaluation
(in developing countries)
• Lack of regular evaluation culture
• Evaluation viewed as unnecessary wastage of resources
• Evaluation reports not used and hence low priority (only for formality)
• No or low budget allocation
• Evaluation schedule, method, criteria… not in the project design
• Time to conduct evaluation not allocated in project design
• Low level of data collection and documentation
• Lack of standard format/ inconsistent format/ non-portability
• Lack of baseline data
• Evaluation criteria not pre-determined
• Evaluation method not pre-determined
• Lack of trained evaluation staff; what to ask, how to frame questions, response
consistency, specific evaluation method, software use
• Unwillingness of the project managers to cooperate
• Unwillingness of the project staff to voluntarily share data/information
• No or low political will; intervention to “manage” report content
Audit: systematic, independent and
documented process for obtaining audit
evidence and evaluating it objectively to
determine the extent to which audit criteria are
fulfilled.
Types of Audit
• Management Audit
• Internal Audit
• External Audit
• Financial Audit
• Technical Audit
• Compliance Audit
• Environmental Audit
• Social Audit
• Energy Audit
• Performance Audit
 Technical Audit
A systematic, independent process for obtaining evidence and
evaluating this objectively to determine the extent to which needs or
expectations are fulfilled.

Technical audit is the process of maintaining the sustainability in the

quality of any Public development activities through the assessment of
cost effectiveness.

Technical audit is done to monitor the activities, which is done

according to the rules and regulation of Nepal Government, Norms of
development and contract document. The main purpose of audits is to
examine how well the planned or design criteria are met by the project
within the allocated time frame and budget. Technical Audit depends
upon the nature and volume of the public development activities.
 Social Audit
The process of evaluating a firm's (or project’s) various
operating procedures, code of conduct, and other factors
to determine its effect on a society.

The goal is to identify what, if any, actions of the firm (or

project) have impacted the society in some way. A social
audit may be initiated by a firm that is seeking to
improve its image within the society. If the results are
positive, they may be released to the public.

If a factory (or project) is believed to have a negative

impact, the company may have a social audit conducted
to identify actions that actually benefit the society.
 Social Audit
A formal review of a company’s (or project’s) endeavors in social responsibility.
Most of the big multinational funded projects requires social audit.

Parameters of a social audit:

• charitable giving,
• volunteer activity,
• energy use,
• transparency,
• work environment,
• worker pay and benefits,
• gender balance,
• social inclusion
• economic impact to society

Social Audit helps company (or project) to portray positive public image, which
can play a part in project success/failure.
 Environmental audit
Environmental audit (EA) means various types of
evaluations intended to identify environmental
compliance and management system implementation
gaps, along with related corrective actions.

EA is an emerging area of concern in Nepal and helps to

understand environmental performance of the proponent
and stakeholders involved in implementing the
development project. EA provides useful information
about the state of the environment on pre-project and
post-construction stage in a given area. The auditing
information also provides a basis to select environmental
protection measures to address environmental problems
of similar nature in similar ecosystem and may contribute
to the improvement of environmental conditions.

The EPA 1996 and EP Rules, 1997 urge to conduct EA

after two years of service provided by the project. The
Ministry of Population and Environment has undertaken
necessary steps to expedite EA in the recent days.
 Depth of the Audit
• Time and money are two of the most common limits on depth of
investigation and level of detail presented in the audit report.
– General audit - normally most constrained by time and resources and is
usually a brief review of the project touching lightly on the six parts
(phases) of an audit

– Detailed audit - usually conducted when a follow-up to the general audit

is required. When a general audit exposes a serious problem in the project
or in the methods by which the project was conducted, a detailed audit
may be required to study such problems in detail and recommend
solutions.

• Accumulation, storage, and maintenance of auditable data are

important cost elements.
• Two often overlooked issues are the self protective activity of team
members during an audit, and the potential for project morale to
suffer as a result of a negative audit.
 Benefits of a Well-Done Project Audit
• Identify problems earlier
• Clarify performance/cost/schedule relationships
• Improve project performance
• Identify future opportunities
• Evaluate performance of project team
• Reduce costs
• Inform client of project status/prospects
• Reconfirm feasibility of/commitment to project
 Major Tasks of a Project Audit
1. Evaluate if the project delivered the expected
benefits to all intended stakeholders.
• Was the project managed well?
• Was the customer/targeted beneficiary satisfied?
2. Assess what was done wrong and what
contributed to successes.
3. Identify changes to improve the delivery of
future projects.
 Types of Project Audits
(time wise)

In-Process Project Audits

• Allow for corrective changes if conditions have
changed and for concentration on project
progress and performance.
Post-Project Audits
• Take a broader and longer-term view of the
project’s role in the organization and
emphasize improving the management
 The Project Audit Life Cycle
Six basic phases of a Project Audit Life Cycle:

1. Project audit initiation

• Focus and scope of audit; assess methodologies, team members required
2. Baseline Definition
• Determine the standards against which performance will be measured
3. Establishment of Audit Database
• Gathering/organizing pertinent data; Focus on what is necessary
4. Data Analysis
• The judgment phase: Comparison of actual to standard
5. Audit Report Preparation
• Present findings to PM first: provides a chance to respond, and clarify factual
errors
• Then, prepare final report
6. Audit Termination
• Review of audit process
• Disbanding of team
 Responsibilities of Project Auditor/Evaluator

• First and foremost the auditor should “tell the truth”

• The auditor must approach the audit in an objective and
ethical manner
• Should seek help in technical issues
• Must assume responsibility for what is included and excluded
from consideration in the report
• The auditor/evaluator must maintain political and technical
independence during the audit and treat all materials as
confidential
• Should provide opportunity to explain before finalizing the
report.
 Challenges faced by professional project
auditors in developing countries
• Project culture: very weak documentation, treating every information as
confidential unless marked otherwise, considering audit as unnecessary hassle in
project development
• Institutional set up: frequent changes in personnel, budget disbursement at the
last minute, changes in departments responsible for project completion, non-
coordination among institutions
• Legal: murky legal requirements
• Socio-economic: expectation of “favorable” evaluation, fee and duration too low for
in-depth auditing, suggestions for improvements interpreted negatively
• Political: indirect pressure to “play with words”
• Inconsistencies and weaknesses in Standard Operating Procedure
• Audit-phobia: not volunteering information, data, documents
• Cooked Data: Fake data
• Non payment of audit fee(?)
Add/edit based on your study/experience and expert discussions.
 Difference between evaluation
and audit
An Audit is the evaluation of a project,
primarily to verify adherence to a set
of pre-defined processes.
An evaluation is the determination of
merit, worth and/or significance using
a set of standards.
सामान्यतयाः परीक्षण वा अडिटले प्रकृ या र उत्तरदायित्वमा
वढी ध्यान दिन्छ भने मुल्याङ्कनले महत्व सफलता र उद्देष्य
प्राप्तिमा वढी जोड दिन्छ।

http://www.aquaknow.net/en/water-toolkit-trainings/monitoring-evaluation-and-audit-water-
development-projects/monitoring-evaluation-and-audit-water-dev/15689
Test Yourself.
1. Discuss the basic criteria often used in project evaluation, along with their
advantages and disadvantages.
2. Discuss the different common methods used in project evaluations. Which is
the best method to evaluation the performance a project?
3. Discuss the different phases of a project audit. Which phase is most crucial in
the project audit?
4. Propose a set of parameters to be assessed during the audit of an irrigation
project to determine the degree of success/failure of the project.
5. What are the challenges in proper/timely project evaluation in a developing
country?
6. What are the practical methods an auditor can use to obtain real data and
honest opinions of project staff during an audit process?
7. Parameters and evaluation methods need to be pre-established before
initiating project evaluation. List the evaluation parameters for a pico-hydro
project (25 kW) in a remote location, and suggest evaluation method for each
parameter.
8. Discuss, with examples, the basic parameters of (a) social audit and (b) financial
audit of an infrastructure development type construction project.
9. Even highly qualified and trained auditors in developing countries face a series
of challenges in project auditing. List the challenges (policy, institutional,
regulatory, cultural, and financial) and suggest practical ways to tackle them.
 Additional References
• http://www.bia.ca/systems-audit.htm
• http://wiki.answers.com/Q/What_do_you_ mean_by_project_audit
• http://www.ateec.org/services/evaluation/index.htm
• https://www.nsf-proj-eval-guide.org/html/home.htm
• http://www.projectsmart.co.uk
• http://www.syntropy.co.uk
• https://en.wikipedia.org/w/index.php?title=Evaluation&oldid=341930989
• https://en.wikipedia.org/w/index.php?title=Audit&oldid=341668393
• http://www.diffen.com/difference/Audit_vs_Evaluation
• http://www.differencebetween.net/business/difference-between-audit-and-evaluation/
• http://www.differencebetween.info/difference-between-audit-and-evaluation
• http://www.aquaknow.net/en/water-toolkit-trainings/monitoring-evaluation-and-audit-
water-development-projects/monitoring-evaluation-and-audit-water-dev/15689