MONITORING AND EVALUATION OF HUMAN SETTLEMENT PROJECTS (HME321)

CHAPTER 1: INTRODUCTION TO MONITORING AND EVALUATION (M&E) IN THE

SOUTH AFRICAN PUBLIC SECTOR - PART II

6. PUBLIC POLICY MONITORING AND EVALUATION

6.1. Purpose of monitoring public policy

• Policy monitoring is performed to ensure that activities and programmes are on

track to achieve the desired goals.
• It ensures that the activities outlined in the implementation plan take place as
envisaged.
• Monitoring as a managerial control function, seeks to ensure that actual
performance is in line with expected (planned) performance and any gasps
identified are addressed.
• For policy monitoring to be effective, the monitoring system must be simple and
flexible.
• Monitoring can be done as a stand-alone activity without necessarily evaluating
the policy.
• However, no evaluation can take place without monitoring.
• Evaluation should be undertaken after continuous monitoring.
• In effect, monitoring entails managing three key variables, namely: cost, time and
quality.

6.2. Key variables in monitoring public policy

These variables can be used as major measuring yardsticks of the policy’s effectiveness.
6.2.1. Cost

• Financial resources are always required in the implementation of any policy.

• Poor policy monitoring during the implementation could easily lead to wasteful
expenditure.
• Management must therefore ensure that costs are carefully monitored and linked
to tangible outputs.
6.2.2. Time
• Time is another important element for monitoring.
• Managers must continually seek to improve their time-management skills.
• Time management includes the ability to judge the amount of time needed for an
activity, the order in which the activities need to be tackled, and the ability to
complete tasks on schedule.
• With good time-management skills, managers will:
✓ Be more productive, proactive, and effective in their monitoring function.
✓ Be more organised.
✓ Support their subordinates better.
✓ Plan more effectively
✓ Monitor plans effectively and ensure long-term focus areas are kept in view.

1
Prepared by Dr Z.G. Gotyi 07 August 2024
 MONITORING AND EVALUATION OF HUMAN SETTLEMENT PROJECTS (HME321)

✓ Eliminate time and resource wastage.

6.2.3. Quality

• It is important that high quality must be attained in every policy implementation.

• Quality work starts with each phase of policy implementation and continues
throughout.
• Managers must ensure that service exceeds the expectations, and the service is
worth the resources put in.
• Managers must ensure a culture of promoting quality is adopted by every
employee in the organisation, with quality indicators developed and agreed to for
the effective and efficient implementation of the policy.
• Therefore, effective monitoring with quality-assurance mechanisms must be part
of the policy process.
• This will require the use of the following tools:

6.2.3.1. Quality-improvement monitoring techniques

6.2.3.1.1. Benchmarking
• This is a very useful tool in policy-monitoring activities.
• It provides a perspective on the policy, programme or projects by comparing them
to similar activities elsewhere.
• A good benchmark sets an above-average target of excellence to measure against,
and to improve upon.
• Benchmarking is the best practice or good management practice that seeks to be
comparable to the best.
• Benchmarking should not be limited to the final product but should cover every
aspect of policy implementation.
• It could be time-consuming and expensive as people may visit the organisations to
undertake comparative analyses, but it is worth considering as it improves results.

6.2.3.1.2. Peer monitoring

• It is also another useful tool for policy-monitoring.
• Policy monitoring function is not only applicable within a department or sphere of
government but may also be used by government to ensure that policy imperatives
cascades to national. provincial and local government activities.
• By cascading down monitoring systems and processes, the government ensures
that policies are effectively implemented and have the most impact at the local
level, which is closest to the people.
• In South Africa, there are peer-monitoring guidelines that are in place to ensure
provincial monitoring of municipalities.
• Section 105 of the Municipal Systems Act empowers the provincial Member of the
Executive Committee (MEC) for local government to establish mechanisms,
processes, and procedures in terms of section 155 (6) of the constitution to:
✓ Monitor municipalities in managing their own affairs, exercising their
powers, and performing their functions.

2
Prepared by Dr Z.G. Gotyi 07 August 2024
 MONITORING AND EVALUATION OF HUMAN SETTLEMENT PROJECTS (HME321)

✓ Monitor the development of local government capacity.

✓ Assess the support needed by municipalities to strengthen their capacity to
manage their own affairs, exercise their powers and perform their
functions.

6.2.3.1.3. Reviews and action plans

• Reviews and actions plans are other instruments for policy monitoring.
• These tools can be very effective if the plans are focused, well developed and
specific.
• A good start in improving service delivery in South Africa is to review the status of
the implementation of the Batho Pele (People First) policy across the government.
• The policy was meant to get public servants to be service orientated, to strive for
excellence in service delivery and to commit to continuous service delivery
improvement.
• It is a simple and transparent mechanism, which allows citizens to hold public
servants accountable for the level of services they deliver (Batho Pele Handbook-
A Service Delivery Improvement Guide).
• After a review, action plans must be developed to support the improvement
initiatives.
• The adherence to Batho Pele principles would ensure that vital quality service is
delivered to the citizens of the country.
• Government departments or agencies must ensure that they attain these policy
outputs through the effective implementation of government’s Batho Pele policy.
• In other words, the Batho Pele principles should be used as part of the quality
criteria in meeting the country’s service delivery needs.
• The enhancement of the implementation of Batho Pele through monitoring
reviews and the development of action plans would enhance quality service
delivery.
• The monitoring plans should always consist of objectives that meet the SMART
test. They must be:
✓ Specific enough to avoid ambiguity.
✓ Measurable enough to be able to determine if they have been met or not.
✓ Achievable, it senseless to have targets that are unattainable.
✓ Relevant, they must be meaningful to the community.
✓ Timely
• Policy monitoring should be integrated into the implementation plan and should
interface with the information systems to allow progress to be tracked and
considered objectively in decision-making.
• Policy monitoring plans should be time-bound, such as: short-term monitoring
plans (less than a year); medium-term monitoring plans (1 to 5-year plans), and
long-term plans (which exceed 5-year period).
• Planning for monitoring is important because it:
✓ Promotes cooperation between the various departments and the public.
✓ It facilitates control.

3
Prepared by Dr Z.G. Gotyi 07 August 2024
 MONITORING AND EVALUATION OF HUMAN SETTLEMENT PROJECTS (HME321)

✓ It creates higher levels of predictability as it gives direction to management.

✓ This contributes to the effective handling of rapid change in local
communities.
✓ Compels managers to look forwards to the future.
✓ Eliminates crisis-management by making managers anticipate changes
from the environment.
✓ Necessitates well-formulated goals and development of various aligned
plans.
✓ Provides a unifying framework against which decisions can be assessed.
• Managing quality through continuous monitoring is an ongoing process, with
agreed indicator mechanisms built into the delivery process.

7. POLICY EVALUATION
• Evaluation can be viewed as a judging process to compare explicit and implicit
policy objectives with the real or projected outcomes or results.
• Administrative techniques such as programme evaluation and review techniques
can enhance the success of policy implementation.
7.1. Necessity for public policy evaluation

• It promotes political and administrative accountability.

• It promotes transparency and openness as there is a thorough analysis of the
decisions made, the nature of the processes utilized, and the implementation of
the policy. What successes could be replicated elsewhere, which ones should be
avoided in future?
• It enables the government to compare inputs against the results of the policy. It
therefore promotes financial prudence and reduces wasteful expenditure.
• It provides the opportunity to engage stakeholders and get their perspectives on
the appropriateness of the policy.
• It could improve management as there will be better conceived programmes and
projects, and improved use of management functions and tools.
• It could lead to an improved implementation of programmes and projects due to
enhanced technical feasibility and thoroughness in the way the activities are
conceived and delivered.
7.2. Practical steps in public-policy evaluation
Step 1: Make a decision to evaluate and follow through. This decision must be carefully
considered as it can be complex task that need substantial resources.
Step 2: Develop an overarching philosophy for the evaluation task. It is important to have
a philosophy that will guide the entire evaluation exercise.
Step 3: Clarify exactly what is being evaluated. (Processes, impact, cost benefits, etc).
Step 4: Clarify the strategy and design for executing evaluation. This step seeks to provide
answers to these questions:
• Is the evaluation comprehensive?
4
Prepared by Dr Z.G. Gotyi 07 August 2024
 MONITORING AND EVALUATION OF HUMAN SETTLEMENT PROJECTS (HME321)

• Is it a fragmented evaluation?
• Will it involve concurrent evaluation?
• Will it extend to or include internal or external role players?
• Will it be a participatory evaluation?

Step 5: Planning in detail for the evaluation activity. There should be detailed planning of:
What, How, Who, Where, and When. These questions enable the evaluators to scope the
magnitude of work to be done. Other issues to be considered in the evaluation plan
include:
• Design of the evaluation tools and instruments, data collection issues, intervals for
data collection.
• Evaluation resources- infrastructure, finance, and staff.
• Gathering baseline information (environmental and political analysis).
• Literature review (Identification of sources and conducting preliminary literature
review).
• Identifying participants or the target audience
• Development of an evaluation framework including targets and indicators. These
will provide standards against which measurements and assessments may be
done. The indicators developed should be simple, specific, reliable, valid, and
meaningful.
Step 6: Implement the evaluation plan.
Step 7: Prepare evaluation reports with clear policy recommendations.
7.3. Developing an assessment framework in policy evaluation: a systems approach

• From a systems perspective, a comprehensive evaluation framework consists of

the following important elements: inputs, process, outputs, and feedback.
• The development of the evaluation framework or the evaluation itself should
never be seen as an isolated process.
• The policy process consists of interdependent and inter-related activities that are
arranged to follow each other consecutively, and the decisions taken at a certain
stage affect the next stage of the process.
• Monitoring and evaluation could be better understood and improved from a
systems perspective.
• M&E should be viewed as an activity within a larger system consisting of various
subsystems.

5
Prepared by Dr Z.G. Gotyi 07 August 2024
 MONITORING AND EVALUATION OF HUMAN SETTLEMENT PROJECTS (HME321)

OUTPUT

FEEDBACK
PROCESS

INPUT

Figure: The systems theory

• The systems theory departs from understanding that the policy process takes
inputs from society (agitation, demands, protests demonstrations, etc).
• Once the inputs are made into the system, various mechanisms are applied to
process them.
• The output of the processing could be policy development, implementation, and
its management. The delivery of outputs often leads to the feedback from the
community through policy related activities, including monitoring and evaluation.

7.4. Forms of public-policy evaluation

The decision to evaluate can be executed in various forms.
7.4.1. Pre-implementation evaluation
• This is an evaluation process that is undertaken before the policy is implemented.
• This process is proactive in nature and is very useful as it limits full-scale blunders
that may arise because of new policy.
• This evaluation focusses on the feasibility, workability, sustainability, acceptance
of a policy within the political, economic, social, and technological environments.
• There are two variants of evaluations under this category:

7.4.1.1. Primary pre-implementation evaluation

• The primary pre-implementation state commences with the development of
policy options.
• This is used to describe the process of analysing and evaluating the various policy
options that may be available to determine the best course of action to follow.
• An effort should be spent identifying the appropriate criteria that the policy will
be tested or evaluated against.
• Once possible policy options have been developed, the next step is to identify
criteria that these options will be tested against to make an assessment and select
the most viable option.

6
Prepared by Dr Z.G. Gotyi 07 August 2024
 MONITORING AND EVALUATION OF HUMAN SETTLEMENT PROJECTS (HME321)

• The following criteria have been identified as critical elements that will enable a
policy decision to be made:
✓ Cost to be incurred to implement the policy.
✓ Physical resources needed.
✓ Appropriateness of the policy.
✓ Time frame.
• Criteria for assessing the policy option should be identified, and finally measured
through the allocation of quantifiable values, for example in percentages.
7.4.1.2. Secondary pre-implementation evaluation
• This is used after a policy has been selected.
• This evaluation is not to compare policy options but to specifically look and
forecast possible effects and consequences of the policy option.
• The exercise assists in the understanding of the selected policy and may lead to
modification of the policy direction before the it is implemented.
• This can assist in avoiding administrative inefficiencies.
• This is like the feasibility study.
• This evaluation seeks to determine whether the policy decision would be
embraced by the community and would work.
• The following are range of methods that may be used under this type of evaluation:
✓ Trend-projection techniques
✓ Modelling
✓ Scenario-building
✓ Cost-benefit analyses (financial, time, resource, and personnel costs).
✓ Pilot study
7.4.2. Implementation evaluation
• This is the monitoring and evaluation of the policy while being implemented.
• This is an ongoing process.
• This is to check whether policy objectives are being progressively attained while
making sure public resources are effectively and efficiently utilised.
7.4.3. Post-implementation evaluation
• All public policies are aimed at improving some aspect of the quality of life of
citizens.
• This evaluation is aimed at assessing the extent to which policy implementation
has been effective.
• It is when the policy has been implemented that the ideals and intentions of policy
makers can be verified.
• Policy implementation is putting into effect the ideals, intentions, and course of
action of policy makers.
• This evaluation seeks to determine whether they have been any progress.
• It determines whether the desired state has been achieved and has this been in full
or partially. Are there any successes or challenges identified, lessons noted for the
future.
• Essentially, post-implementation evaluation shed light on the issues of:
✓ Equity.
✓ Appropriateness of the policy.

7
Prepared by Dr Z.G. Gotyi 07 August 2024
 MONITORING AND EVALUATION OF HUMAN SETTLEMENT PROJECTS (HME321)

✓ Responsiveness to demands from the environment.

✓ Adequacy levels of the services being rendered.
7.5. Key considerations in public-policy evaluation
These are essential elements that need to be carefully considered in an evaluation
exercise as they can affect the quality of the evaluation.
7.5.1. Development of clear objectives
• It is very important to develop clear objectives to keep evaluation focussed on
specific issues or subjects.
7.5.2. Data collection mechanisms
• Tools to be used for data collection must be properly considered.
• Why are they chosen and how the limitations going to be managed?
• Will the evaluation be quantitative or qualitative or both?
• Different data collection tools generate data of various nature.
• The sampling procedure to be used must be developed and data analysis
techniques must be specified.
7.5.3. Time frame for the evaluation
• The evaluation time frame must be indicated. The depth and the scope of
evaluation has a direct relationship on the time available for the evaluation.
7.5.4. Selection of evaluation focus
An evaluation focus or foci must be selected from a range of the following policy
evaluation categories:
• Linear or single focus policy evaluation (which elements would the evaluation
focussed on).
• Multi- foci / Comprehensive policy evaluation
• Self/internal evaluation and external evaluation
• Formative or summative (Formative seeks to generate insight, improve delivery
while summative evaluation is conducted after delivery to determine the direction
of the policy or project).
• Thematic evaluation (evaluation of a range of developmental interventions that
address a specific priority issue.
• Sectorial or integrated evaluation (does it target one sectoral policy or several
sectorial policies at the same time).
• Joint evaluation (combine donor agencies or partners).
• Multi or single-phase evaluation
• End-users service evaluation
• Independent evaluation
7.6. Analytical framework for policy evaluation

• The analytical framework for policy evaluation exercise depends on the key
provisions of the policy evaluated.

8
Prepared by Dr Z.G. Gotyi 07 August 2024
 MONITORING AND EVALUATION OF HUMAN SETTLEMENT PROJECTS (HME321)

• The development of the framework for policy evaluation often requires

consideration of inputs from stakeholders and the emerging issues from the
preliminary literature review.
• The process starts with the determination of the extent by which the environment
is ready for evaluation.
• It is necessary that the culture of evaluation is cultivated in the organisation by
ensuring that managers embrace the need for policy evaluation.
• Internal and external processes must be established whether they support policy
evaluation.
• To what extent do the internal processes support an evaluative activity and ensure
that it is productive exercise that is value-adding? This includes the consideration
of range of issues such as: leadership processes, resource allocation and
availability, time, quality management systems, e.g. records management, reporting
systems, etc.
• Equally, the support of the external processes must be established. This include
determining the level of engagement of clients, coalitions, and the existence of
supportive partnerships.
• The framework must also consider the availability and the relevance of after-
service support outlets. This means an evaluation of the avenues available to
clients after they have been serviced or not. Are the channels opened for after-
service- related communication?
• Another aspect to be considered, is the extent to which mediation opportunities
exist or have been provided to manage any issues that may not have been resolved
at the after-sales-service point.
• The final element examines the extent to which the policy has impacted on the
lives of the beneficiaries or intended beneficiaries. This impact analysis should be
detailed and include social and economic impacts. All this information could feed
back into the policy process again, to strengthen or adjust the policy.

8. FACTORS IMPACTING ON MONITORING AND EVALUATION OF PUBLIC POLICY

Several factors may affect the quality of the M&E exercise.
8.1. Political maturity and culture
Embracive or hostile political culture may affect policy evaluation.
8.2. Technicality
The M&E may be negatively affected if:
• The terms of reference are not specific enough or are loosely conceived.
• There is a lack of quality baseline information.
• There are questions about the validity, accuracy and reliability of baseline and
other information.
• Due processes are not followed.
• Goals and objectives are unclear and perhaps do not meet the SMART criteria.
• If poor recording-keeping practices and poor tracking processes result in
incomplete unavailable information.

9
Prepared by Dr Z.G. Gotyi 07 August 2024
 MONITORING AND EVALUATION OF HUMAN SETTLEMENT PROJECTS (HME321)

• Indicators are inappropriate or irrelevant.

8.3. Resource limitations.
• Shortage of resources (time, finance, physical, personnel, informational).
9. POSSIBLE OUTCOMES OF A POLICY-MONITORING AND EVALUATION EXERCISE
• Reconfirmation and recommitment to policy
• Policy abandonment
• Policy review
• Strengthening of administrative processes
• Restructuring of delivery mechanisms
• Resource allocation (better resourcing or funding of the project/programme)
• Replication of programmes to other sites or reduction to fewer or single site.
• Identification and determination of impacts:
✓ Social impact (crime or poverty)
✓ Demographic impact (youth, gender, disabled, groups)
✓ Economic impact (income, economic activity, employment, accessibility, etc)
✓ Environmental impact (health, pollution, preservation)
✓ Political impact (participation, inclusivity, representation, equity).

10. CONCLUSION

Public policies seek to promote the public good and welfare of society. Public policy
decisions have the potential to affect the entire country favourably or unfavourably.
Policies generally arise because of the need to solve a problem or address an issue, meet
certain needs or improve upon service delivery or management. This chapter explored
the fundamentals of public policy monitoring and evaluation. The key monitoring
variables and tools were identified and discussed. The purpose and necessity of
monitoring and evaluation in public policy context were presented. Factors affecting
monitoring and evaluation as well as the possible outcomes of the phenomenon were
identified.

10
Prepared by Dr Z.G. Gotyi 07 August 2024