Monitoring and Evaluation

Monitoring
• Monitoring is a continuous function to inform the programme or
project managers and stakeholders of progress achieved against
planned results (outputs, outcome and objectives).
• Data on specific, predetermined indicators is systematically collected
and analysed to track actual programme or project performance for
management decision making(IAEA-TC Glossary).
• Monitoring generally involves collecting and analysing data on
implementation processes, strategies and results.
• “Monitoring is a continuing function that uses systematic collection of
data on specified indicators to provide management and the main
stakeholders of an on-going development intervention with
indications of the extent of progress and achievement of objectives
and progress in the use of allocated funds.” (Organisation for
Economic Co-operation and Development - Development Assistance
Committee (OECD-DAC) expert group, 2002-2008)
• “Monitoring can be defined as the on-going process by which stakeholders
obtain regular feedback on the progress being made towards achieving their
goals and objectives.” (UNDP Handbook on Planning, Monitoring and
Evaluation for Development Results, 2009).
• “Monitoring is defined as the systematic and continuous collecting, analysing
and using of information for the purpose of management and decision-making.
• The purpose of monitoring is to achieve efficient and effective performance of
an operation.
• Monitoring provides an ‘early warning system’, which allows for timely and
appropriate intervention if a project is not adhering to the plan.” (European
Commission, 2008)
Evaluation
• Evaluation is an objective, independent and systematic examination of
the extent to which a programme or project has achieved (or is
achieving) over time its stated objective and, therefore, is meeting the
needs and priorities of Member States.
• Evaluation assesses the efficiency, effectiveness, relevance, impact,
and sustainability of a programme or project (IAEA-TC Glossary).
• “Evaluation is the systematic and objective assessment of an on-going or
completed project, programme or policy, its design, implementation and
results.” (OECD–DAC expert group)
• “An evaluation is an assessment, as systematic and impartial as possible, of an
activity, project, programme, strategy, policy, topic, theme, sector, operational
area, institutional performance, etc.
• It focuses on expected and achieved accomplishments, examining the results
chain, processes, contextual factors and causality, in order to understand
achievements or the lack thereof.
• It aims at determining the relevance, impact, effectiveness, efficiency and
sustainability of the interventions and contributions of the organizations of the
UN system”. (UN Norms for Evaluation, 2005)
• An independent evaluation uses rigorous standards and must be
conducted by persons or entities independent of those who designed
and implemented the programme or project.
• An evaluation can be formative (e.g. midterm evaluation) or
summative (e.g. final evaluation and impact evaluation).
• Evaluation seeks to provide information that is credible and useful,
enabling the incorporation of lessons to learn into the decision-
making process of the organizations of the UN system and those of
Member States.
Monitoring and Evaluation Criteria
 Rationale for Monitoring and Evaluation
• Accountability
• Projects are implemented by staff on behalf of different stakeholders, e.g. donors,
government, hosting organization and beneficiaries.
• Therefore, it is important that project implementers are held responsible for their
actions, i.e. they are accountable to all stakeholders.
• Improving performance
• Monitoring identifies the extent to which a project is making progress in producing
expected outputs and achieving a desired outcome.
• Corrective measures can be taken in time to improve project performance.
• Ongoing monitoring also allows the assessment of whether inputs and resources
are being used efficiently.
• Learning
• Monitoring and evaluation can provide valuable lessons for other projects within the same country or
organization.
• These lessons may be used for the ongoing cycle or can be applied during the next programming cycle
to repeat successes or to avoid failures.
• Communication
• M&E activities improve the communication between different stakeholders, thus enabling a better
understanding of implementation issues and supporting better achievement reporting.
• In order to make communication effective, a favourable environment for exchange and discussion is
essential.
• Clear and transparent communication mechanisms such as regular meetings, workshops, reporting,
and information sharing via internet or printed media should also be established.
• Empowerment of stakeholders
• M&E creates opportunities for beneficiaries to provide useful feedback to the implementers.
• Furthermore, the involvement of different stakeholders in the process can increase their motivation
and skills for planning and implementing future projects.
• M&E can thus strengthen the participation of primary and end beneficiaries in decisions about project
performance, and therefore increase stakeholder ownership.
 Considerations for M&E
(for technical cooperation projects)
• The following objectives are expected to be achieved through M&E
implementation
a. To increase the performance and accountability of TC projects;
b. To improve communication and participation of TC stakeholders;
c. To enhance learning and continuous improvement within the
organization;
• The TC programming cycle is divided into three main phases: Planning & Design;
Implementation; and End Review.
• Each stage implies different M&E functions.
• The first phase of the TC programming cycle consists of project identification,
planning and design.
• This includes upstream work and the preparation of national, regional and
interregional programme notes.
• These notes give an overview of the national, regional or interregional programme
and include project concepts.
• This is followed by the preparation and design of project documents using the
logical framework approach (LFA), and the elaboration of a detailed workplan and
budget.
• During this first phase, it is essential to consider M&E tasks that will be undertaken
later on.
• This requires the development of an M&E matrix (or plan).
• It is also necessary at this stage to ensure that TC quality criteria are embedded in the
project design.
• During the implementation phase, intended project results are delivered, resources
are managed, and progress is monitored and reported for decision making purposes.
• At this point, the M&E tasks (defined at the design stage) are carried out as planned.
• These include information gathering and analysis, preparation of reports and
dissemination of information.
• The need to ensure the application of quality criteria during implementation must
also be considered here.
• The end of the programming cycle corresponds to the review phase.
• At this stage, the M&E function includes conducting self-evaluations or
self-assessments, and the documentation and dissemination of lessons
to be learned.
• The findings will be used for continual improvement of the TC
programme, but also will serve as inputs for independent, external
evaluations conducted.
Planning for Monitoring & Evaluation
• STARTING POINT: THE LOGICAL FRAMEWORK APPROACH (LFA)
• The LFA is the methodology adopted and used for the design of TC projects.
• This is aligned with the results-based management approach applied within
the Agency.
• A capacity building package and related material has been developed
specifically for TC projects.
 Logical framework approach in summary
• “The LFA helps stakeholders to think through and analyse the “logic‟ of a
project in a systematic and structured way, first by conducting a detailed
analysis of a number of elements, and secondly by relating the results of
these analyses to each other and to the overall project objective.
• This ensures a sound project proposal and a high-quality project.
• The LFA provides a project structure in which major components are
explicitly and clearly interrelated, and interrelationships clarified.
• The LFA plays a particularly critical role in project planning and design,
but it can also be used throughout the project cycle, including during
monitoring and evaluation.
• The LFA is essentially a sequence of analytical steps, comprising a situation
analysis that reviews project context and relevance, a stakeholder analysis
that covers counterpart mandate and vision, end-users and any other
organizations or group or institution having an interest or being affected by
the project, a problem analysis that examines the problem in detail from the
perspective of different stakeholders, and finally an objectives analysis where
the project team decides on the scope of the project.
• On the basis of these analyses, the project team constructs a Logical
Framework Matrix (LFM) that summarizes the project and shows the logical
linkages between the project elements.
• This is an iterative process of testing, review and validation that then
continues with the preparation of a suitable work plan”.
 Results Chain/Hierarchy of Results
• The primary purpose of M&E is to track the performance of a project, i.e. the degree to
which it is implemented as planned and how successfully it achieves its intended results.
• The relationship between each result level is described in the LFM and is also known as
the Result Chain (or Hierarchy of Results).
• When designing the project, it is effective to start by defining the overall objective to
which the project will contribute and defining the expected outcome (the project
specific objective).
• Then the outputs needed to achieve the outcome are defined, and thereafter the
activities and inputs.
• When it comes to project implementation, one starts by putting inputs and resources in
place, and these are converted into activities: activities implemented will produce
outputs, and the outputs achieved will, in turn, result in the desired outcome.
• This logic is known as a Result Chain or the Hierarchy of Results .
• It is necessary that the logic of the project design is clear and
straightforward.
• A result is a change that can be observed, described and measured in some
way, and for which the cause can be identified.
• Results may a) appear within a short time or take years to be fully realized;
b) be planned or unforeseen; c) be either positive or negative; d) be
reflected at the level of individuals, groups of individuals, institutions, or
society. Different levels of results exist.
• At the operational level there are immediate results or ‘Outputs’; at the
developmental level, there are medium term results or ‘Outcomes’ (project
specific objective) and long-term result or ‘Impact’ (overall objective).