Impact Evaluation of policies,

programs and projects

(Econ 713)
By: Tesfaye Etensa (Ass.Prof.)
Department of Economics, LIC
Email: [email protected]
 1. Introduction

By: Tesfaye Etensa (Ass.Prof.)

Department of Economics, LIC
Email: [email protected]
1.1 Monitoring and Evaluation
 1.1. What is Monitoring?

Several approaches can be used to evaluate programs.

Monitoring is a continuous process that tracks what is
happening within a program and uses the data collected to
inform program implementation and day-to-day
management and decisions.
Using mostly administrative data, monitoring tracks
program performance against expected results, makes
comparisons across programs, and analyzes trends over
time.
Usually, monitoring tracks inputs, activities, and outputs,
though occasionally it can include outcomes, such as
progress toward national development goals.
 1.1. What is Monitoring?
▪ The ongoing process by which stakeholders obtain regular
feedback on the progress being made towards achieving their
goals and objectives.
▪ The process of routinely gathering information on the
process of project implementation.
▪ Day-to-day follow up of activities during implementation to
measure progress and identify deviations.
▪ Routine follow up to ensure activities are proceeding as
planned and are on schedule.
▪ Routine assessment of activities and results.
▪ Answers the question, “what are we doing?”
5
 1.1. What is Monitoring?

Monitoring tracks key indicators of progress over the

course of a program as a basis on which to evaluate
outcomes of the intervention.
The challenges in monitoring progress of an intervention
are to:-
■ Identify the goals that the program or strategy is designed
to achieve, such as reducing poverty or improving schooling
enrollment of girls.
* For example, the Millennium Development Goals initiative
sets eight broad goals across themes such as hunger, gender
inequalities, schooling, and poverty to monitor the
performance of countries and donors in achieving outcomes
 1.1. What is Monitoring?
■ Identify key indicators that can be used to monitor progress against
these goals.
 In the context of poverty, for example, an indicator could be the
proportion of individuals consuming fewer than 2,100 calories per
day or the proportion of households living on less than a dollar a
day.
■ Set targets, which quantify the level of the indicators that are to be
achieved by a given date. For instance, a target might be to halve the
number of households living on less than a dollar a day by 2015.
■ Establish a monitoring system to track progress toward achieving
specific targets and to inform policy makers.
• Such a system will encourage better management of and
accountability for projects and programs.
 1.2. What is Evaluation?

➢ Operational evaluation examines how effectively programs were

implemented and whether there are gaps between planned and realized
outcomes.
 Impact evaluation studies whether the changes in well-being are indeed
due to the program intervention and not to other factors.
 Evaluation is a rigorous and independent assessment of either completed
or ongoing activities to determine the extent to which they are achieving
stated objectives and contributing to decision making.

● An assessment of the extent to which a project is achieving or has

achieved its stated outcome goals.

● A process which attempts to determine as systematically and objectively

as possible the relevance, effectiveness, efficiency, sustainability and
impact of activities with regard to specified objectives.
8
 1.2. What is Evaluation?
Evaluations are periodic, objective assessments of a
planned, ongoing, or completed project, program, or
policy.
Evaluations are used to answer specific questions related
to design, implementation, and results.
In contrast to continuous monitoring, they are carried out
at discrete points in time and often seek an outside
perspective from technical experts.
Their design, method, and cost vary substantially
depending on the type of question the evaluation is trying
to answer.
 1.2. What is Evaluation? (Contd.)

 Relevance: The validity of the overall goal and

project purpose to the problems of the target
community/users
 Effectiveness: The degree to which the project
purpose has been achieved by the project outputs
 Efficiency:
 The productivity in project implementation
 The degree to which inputs have been converted
into outputs
10
 1.2. What is Evaluation? (Contd.)
 Sustainability: The durability of the benefits and development
effects produced by the project after its completion
 Impact: Positive and negative changes produced, directly or
indirectly as a result of the Implementation of the project
▪ Assessment of overall achievement and impacts
▪ Systematic way of learning from experience to IMPROVE
current activities and promote better planning for future
action
▪ Designed specifically with intention to attribute changes to
intervention itself
▪ Answers the question, “what have we achieved and what
impact have we made”
11
 1.3.Relationship between M & E

Item Monitoring Evaluation

Frequency ▪ Periodic, routine, regular ▪ Episodic
(quarterly)
Main action ▪ Keeping track / oversight of ▪ Deeper review, assessment,
ongoing projects verifying the achievement of
the objectives and goals
Purpose ▪ Improve implementation ▪ Improve effectiveness,
pace efficiency, impact, relevance
▪ Adjust work plan and sustainability
Focus ▪ Inputs, outputs, process, ▪ Process, Outcome and impact
▪ work plans
Information ▪ Field visit, observation, ▪ Same, plus surveys, studies –
sources progress reports, rapid analytical
assessments
Undertaken ▪ Internal staff ▪ Internal staff
by ▪ Users ▪ Stakeholders
▪ Supervisors ▪ Funders/donors
▪ Funders/donors ▪ External evaluators
▪ Users 12
 1.4 Evaluation Approaches and Tools
1.4.1 Ways of doing Evaluation
A. Internal/Self-evaluation:
▪ Involves an organization or project holding
up a mirror to itself and assessing how it is
doing, as a way of learning and improving
practice
B. External evaluation:
▪ An evaluation done by a carefully chosen
outsider team, usually consultants.
C. Interactive/joint evaluation:
▪ Involves a very active interaction between an
outside evaluator or evaluation team and the
organization or project being evaluated.

13
 1.4.2. Types of Evaluation

I. Formative Evaluation
II. Process Evaluation
III. Summative (outcome/impact) Evaluation

I. Formative evaluation:
 is the critical examination of a project before or
immediately after the start of implementation of
activities.
 It is looking at the identification report, which selects
and ranks the various solutions from the standpoints
of relevance, technical, financial and institutional
feasibility as well as socio-economic sustainability
14
 1.4.2. Types of Evaluation (Contd.)

II. Process Evaluation

Could be done mid-term or towards the end of
the project
This is an evaluation carried out during activity
implementation of a certain project/program
To verify whether the project is on the
right track and the relevance of a certain
project objectives and goal.
To understand whether the project
implementation have the potential to
bring about the desired change.
15
 1.4.2. Types of Evaluation (Contd.)

III. Summative (outcome/impact) evaluation

Done after intervention completed.
To determine the effectiveness and efficiency of
implementation
The focus is the outcome & possibly impact of
the intervention.
Check sustainability of results/ benefits
Could serve the decision makers as to what do
with the project - to continue or scale up or
discontinue, etc.
16
Thank You For Your
Attention!

17
 Thank You for
Reading and attending the
discussion!
Stay Safe!