European Commission – EuropeAid Project Cycle Management Handbook

2.7.4.1. What is Monitoring?

Project monitoring is an integral part of day-to-day management. It provides informa-
tion by which management can identify and solve implementation problems, and as-
sess progress. The Logical Framework, the implementation Schedule and the Activ-
ity and Resource Schedules provide the basis. The following basic issues need to
be regularly monitored:
Which Activities are underway and what progress has been made (e.g. at
weekly intervals)?
What to At what rate are means being used and cost incurred in relation to progress in
monitor? implementation (e.g. monthly)?
Are the desired Results being achieved (e.g. quarterly update)? (efficiency)
To what extent are these Results furthering the Project Purpose (e.g. half-yearly
analysis)? (effectiveness)
What changes in the project environment occur? Do the Assumptions hold true?
Project management checks how the objectives are met, and analyses the changes
in the project environment including key stakeholder groups, local strategies and
policies. If progress falls short, corrective action has to be taken. Details of any ac-
tion have to be included in the next progress report.

2.7.4.2. Reporting on Progress

During the inception period of a project, mechanisms for communication have to be
established to ensure that the necessary information is generated and utilized in a
timely and effective manner. In this context:
Progress review meetings are useful to review progress against the plan. This
may be also an opportunity for written reports to be presented and discussed, or
simply for a rapid oral assessment of current issues and problems.
Project progress reports provide periodic summaries of project progress
incorporating key information from the physical and financial indicators included
in the logframe, Activity Schedule and Resource Schedule.
Progress reports are to be written in a standard format allowing for comparison be-
tween reports over time. The purpose of progress reports is to provide updates on
How to re- achievements against indicators and milestones, using the following framework:
port?
Data about intended achievements, is compared with
Data on actual achievements, to identify...
significant deviations from plan, as a basis for...
identification of problems and opportunities, to identify...
corrective action and alternatives.
Chapter 5.3.4 provides details about reporting formats and types.

25
European Commission – EuropeAid Project Cycle Management Handbook

2.8. Evaluation

2.8.1. Introduction
Evaluation is an “assessment, as systematic and objective as possible, of an ongo-
ing or completed project, programme or policy, its design, implementation and Re-
sults. The aim is to determine the relevance and fulfilment of objectives, develop-
mental efficiency, effectiveness, impact and sustainability. An evaluation should pro-
vide information that is credible and useful, enabling the incorporation of lessons
learned into the decision-making process of both recipients and donors”15.
An evaluation can be done during implementation (“mid-term”), at its end (“final
evaluation”) or afterwards (“ex post evaluation”), either to help steer the project or to
draw lessons for future projects and programming. “Ex ante” evaluation16 refers to
studies during the preparatory phases of the project cycle (pre-feasibility or feasibil-
ity studies). These kinds of studies are treated in section 2.5.
A typical evaluation mission would last several weeks in the partner country, fol-
lowed by a shorter period in the European Union. The major principles governing
evaluation can be summarised as follows:
Figure 9: Major Principles of Evaluation

Major Principles of Evaluation

Impartiality & independence of the evaluation process in its function from the
process concerned with policy making, the delivery and management of
assistance (= separation of evaluation and responsibility for the project/
programme/policy)
Credibility depending on expertise and independence of the evaluators &
transparency to be sought through an open process, wide availability of
results, distinction between findings and recommendations
Usefulness: relevant, presented in a clear and concise way, reflecting the
interests and needs of the parties involved, easily accessible, timely and at
the right moment improved decision-making
Participation of stakeholders (donor, recipient...); if possible: views and
expertise of groups affected should form integral part of the evaluation;
involving all parties capacity building
DAC 1991

2.8.2. Overall Responsibilities for Evaluation

As regards responsibilities, two types of evaluation can be distinguished within EC’s
external co-operation programmes:
1. Evaluation of individual projects: EuropeAid is responsible for the evaluation of
individual projects and for sending a copy of each completed evaluation report to
the service maintaining the database of all evaluations. Evaluation studies are
financed under project/programme funds.
2. Evaluation of the results of country/regional and sectoral policies and pro-
grammes, of programming performance and of the policies mix: This type of
evaluation is managed by EuropeAid’s Evaluation Unit (H/6) under the direct au-

15
OECD / DAC, 1998: Review of the DAC Principles for Evaluation of Development As-
sistance.
16
See http://europa.eu.int/comm/budget/evaluation/pdf/ex_ante_guide_en.pdf for a
guide.

27
European Commission – EuropeAid Project Cycle Management Handbook

thority of the Board of EuropeAid. The unit feeds the results back into the policy-
making and programming process.
The Evaluation Unit has a separate budget and is completely independent of the
operational services. It has also a key role as an advisory service:
At the level of project evaluations it provides guidance, on request, to the ser-
vices concerned on such aspects as terms of reference, choice of consultants,
and quality of draft reports.
The Unit participates in the activities of the Inter-Service Quality Support Group
(i-QSG) which focuses on the programming level, while “Office Quality Support
Groups” have the primary responsibility for looking at the quality of the design of
the funded operations.

2.8.3. Types of Evaluations

Evaluations can take place:
1. when the project is still under way: such interim evaluation are usually under-
taken at mid-term (mid-term evaluation), to review progress and propose al-
terations to project design during the remaining period of implementation;
2. at the end of a project (final or end-of-project evaluation), to document the re-
sources used, results and progress towards objectives. The objective is to
generate lessons about the project which can be used to improve future de-
signs;
3. a number of years after completion (ex post evaluation), often focusing on
impact.

2.8.4. Evaluation Criteria

Evaluations under EC funds follow the evaluation criteria of the DAC that are closely
linked to the logframe.
Table 7: Evaluation Criteria Used by the European Commission
Relevance The appropriateness of project objectives to the problems that it was sup-
posed to address, and to the physical and policy environment within which
it operated, and including an assessment of the quality of project prepara-
tion and design – i.e. the logic and completeness of the project planning
process, and the internal logic and coherence of the project design.
Efficiency The fact that the Results have been achieved at reasonable cost, i.e. how
well inputs/means have been converted into Results, in terms of quality,
quantity and time, and the quality of the Results achieved. This generally
requires comparing alternative approaches to achieving the same outputs,
to see whether the most efficient process has been adopted.
Effectiveness An assessment of the contribution made by Results to achievement of the
Project Purpose, and how Assumptions have affected project achieve-
ments.
Impact The effect of the project on its wider environment, and its contribution to
the wider sectoral objectives summarised in the project’s Overall Objec-
tives, and on the achievement of the overarching policy objectives of the
EC.
Sustainability An assessment of the likelihood of benefits produced by the project to con-
tinue to flow after external funding has ended, and with particular refer-
ence to factors of ownership by beneficiaries, policy support, economic
and financial factors, socio-cultural aspects, gender equality, appropriate
technology, environmental aspects, and institutional and management ca-
pacity.

28
European Commission – EuropeAid Project Cycle Management Handbook

Figure 10: Linking Evaluation Criteria to the Logframe

Linking Evaluation Criteria to the Logframe

Sustain- Have and will products and benefits
ability
be maintained?
Overall
Objectives Which benefits on society and
Impact
change sector?
Project Purpose
+ Assumptions
Effective- How well did the Results contribute
utilisation ness to the achievement of the Project
Results
+ Assumptions Purpose?
action How were inputs and activities
Efficiency
Activities converted into Results?
+ Assumptions
allocation
Quality of planning and adaptation,
Means including relevance of problems to
+ Pre-conditions correct beneficiaries, OVIs, means,
Relevance
cost, assumptions, risks

2.8.5. What is the Difference Between Monitoring, Evaluation and Audit?

Frequently, there is confusion about what monitoring, evaluation and audit are,
where and how they differ and how they can be delimited from each other. The fol-
lowing figures define and compare the three terms as they are in use in the EC ex-
ternal co-operation context.
Figure 11: Comparing Monitoring, Evaluation and Audit

Evaluation, Monitoring and Audit (1) Evaluation, Monitoring and Audit (2)
Evaluation: Monitoring:
mainly analysis of the efficiency, mainly analysis of efficiency and
What? effectiveness, impact, relevance and effectiveness (i.e. measuring actual against
sustainability of aid policies and actions What? planned deliverables); is a systematic
How? in-depth analysis management activity
rapid and continuous analysis, immediately
external evaluators specialised in the useful to improve on-going actions; of key
Who? subjects evaluated How?
importance to improving performance
once or twice, essentially at the end or 'ex
post' drawing lessons from the past in order to Who? internal and external (staff, monitors.....)
When?
orient future policies and actions but also
during implementation: mid-term evaluation to When? regularly, several times per year
(re-) orient implementation

Evaluation, Monitoring and Audit (3)

Audit:
traditionally checks whether financial
operations and statements are in compliance
.
with the legal and contractual obligations. More
concerned with compliance, but better financial
What? management can also contribute to improving
current and future actions. More recently:
Performance audit is strongly concerned with
questions of efficiency and good management
How? verification of financial records (financial audit)

Who? external, professional auditors

When? during or after implementation

29