The Final Evaluation Report

For the Sustainable Livelihood Project in Longchuan County

（2012-2016）

Peter Liu Yueping Xiang

December 2016
 Table of Content
Executive Summary ...................................................................................................................... - 3 -
A. Introduction ............................................................................................................................... - 4 -
B. Evaluation Purpose and Questions to Be Answered ........................................................... - 5 -
C. Evluation Methodology and Process ..................................................................................... - 8 -
1. Evaluation plan and guideline for interview ................................................................................. - 8 -
2. Target group interviewed in the evaluation ............................................................................... - 8 -
3. Evaluation methodology and process ......................................................................................... - 8 -
4. Evaluation strategy........................................................................................................................ - 9 -
D. Evaluation Findings ............................................................................................................... - 11 -
1. Promotion of more natural agricultural practices: ..................................................................... - 11 -
2. Development of other agricultural income channels: ............................................................ - 12 -
3. Development of non-agricultural income channels: .............................................................. - 14 -
4. Awareness and knowledge promotion of sustainability: ....................................................... - 15 -
5. Project planning and design .......................................................................................................... - 16 -
6. Limitations of the evaluation ..................................................................................................... - 17 -
E. Evaluation Conclusion ........................................................................................................... - 18 -
1. Project relevance ............................................................................................................................. - 18 -
2. Project effect................................................................................................................................. - 19 -
3. Project Impact .............................................................................................................................. - 23 -
4. Project Effectiveness and Efficiency ......................................................................................... - 23 -
5. Project Management ....................................................................................................................... - 24 -
6. Project implementation .............................................................................................................. - 25 -
7. Project sustainability ...................................................................................................................... - 26 -
F. Follow up Suggestions and Comments ................................................................................ - 27 -
1.Suggestions and comments on project planning and design ......................................................... - 27 -
2.Suggestions and comments on the seed fund .................................................................................. - 28 -
3. Suggestions and comments on experience promotion .................................................................. - 29 -
4. Suggestions and comments on the women’s group ....................................................................... - 29 -
5. Suggestions on the handover ......................................................................................................... - 29 -
G. Annex (only for Chinese version) ......................................................................................... - 30 -
Executive Summary

The Longchuan Sustainable livelihood project (2012-2016) is made up of four components,

namely promotion of more natural agricultural practices, development of other agricultural
income channels, development of non-agricultural income channels (in some project
documents also named as “women’s development”), and awareness and knowledge
promotion of sustainability. By the time of this evaluation, some of the expected outcomes
have been realized, some have not though they are moving positively towards the
destination. For those unrealized outcome targets such as changing of the production and
income structures, the evaluation team think that though it has something to do with
project activity design (some activities not directly or effectively contribute to the outcome),
they are too ambitious for a 4 year project of this scale, especially under the context that the
sugarcane planting is the leading industry in the communities. At the same time, the project
also has the practice of developing annual plan and targets. From this angle, most of the
expected outputs for these annual plans and targets have been achieved. Specifically:

1. The Expected outcomes for sustainability awareness raising, cross planting, skill
training, and green fertilizer utilization have been achieved.

2. The Expected outcomes for improving the fertilizer structure, establishing

diversified income structure, women’s development and income generation etc.
have not been achieved, thought they are moving positively towards the targets.

3. Project activities were not specified in the project designing document, though
annual targets were set in the “strategic plan” part after the logic framework. At the
same time, baseline data were not collected and established against the indicators
set to measure the outcomes and outputs. These two gaps led to the result that the
activities decided at a later stage in the annual plan are sometimes not effectively in
line with the set outputs and outcomes. At the same time, without a base line data,
the M&E may easily become blind and inconsistent.

4. The operation and management model of the seed fund need improvement. We
suggest consulting the “village mutual help fund model” initiated by the poverty
alleviation office under the state council.

5. The self-management and development capacity of the women’s group need to be

consolidated.

6. The experiences gained in sustainability awareness raising, cross planting, green

fertilizer utilization etc. deserve to be promoted to other areas.

7. To maximize project sustainability and impact, some components of the project,

such as the seed fund and women’s group, need to be consolidated before handing
over to the local community. And it is reasonable to leave about one year for this
 purpose.

A. Introduction

In 2007, NMA（Norwegian Mission Alliance; hereinafter referred to as Buer）funded a

5-year project aiming at poverty reduction in Longchuan county (2007-2011). In 2011, Buer
completed the final evaluation and a new 5-year project was proposed on the base of this
evaluation.

The focus of the new project (2012-2016) shifted to sustainable livelihood, as it was found
out that the project communities mainly depends on sugarcane planting for their livelihood.
The monoculture of sugarcane on the one hand were causing increasing deterioration of the
soil and environment, due to application of increasing quantity of chemical fertilizers and
pesticide; on the other hand, were limiting fodder sources to develop animal husbandry
which is the source of green fertilizer. At the same time, the mono-income structure is
putting the villagers in a very vulnerable position, as the sugarcane price is not stable.
Another issue faced by the community was marginalization of women, especially the
Myanmar wives, in the social life. Due to various reasons like language, citizenship, and
traditions, women, especially the Myanmar wives, were excluded in major social life, and
did not have the opportunity to demonstrate their potential to gain respect in their families
and in the communities.

At the end of 2016, the 5-year project comes to the end. To learn the project effect and the
actual realization of the expected project outcomes and outputs while collecting lessons and
experiences for future improvement, Buer engaged two external evaluation specialists to
conduct the final evaluation in early December.

Goal and objectives of the project

Long term goal:

To 2016, in the project communities, pilot farmers have established the model of multiple
livelihoods that gradually reduces their dependence on sugarcane monoculture. In the
meantime, local government and farmers have raised their environmental-home awareness
based upon the environment value, so that they can change the use of local natural
resources in agriculture to become more sustainable with the improved participation as well
as the advocacy from local government, finally, the project will supply a model of
sustainable livelihood to local government and farmers for reference.
Sub-objectives:

 1. In 5 years, more natural agricultural practices are utilized to reduce the impact of
sugarcane monoculture.
 2. In 5 years, farmers take less risk of monoculture by the development of other
agricultural income channels.
  3. In 5 years, farmers take less risk of monoculture by the development of
non-agricultural income channels.
 4. In 5 years, local people have rational awareness and knowledge of sustainability.

Intervention strategies

In local communities, the farmers use quantities of chemicals for sugarcane planting, which
is their major income channel. Local farmers also lack alternative income channels. They
have to take high risk of sugar price variation. Therefore, the unsustainable livelihood is the
focal problem in this project.

Accordingly, the project designs 4 aspects to solve that problem. Namely:

1. Through the development of natural agricultural practices, for example, cross

planting, and green fertilizer supplying from husbandry development, the overuse of
chemical is expected to be reduced.

2. Through the development of husbandry and forestry, farmers will receive more
agricultural income channels so that they will take less risk of monoculture.

3. Through the development of non-agricultural income channels, engage local women

more in the community development, especially for those most vulnerable people,
like Myanmar wives.

4. Through sustainability awareness rising, local farmers promote the attitude and
knowledge about the sustainable development, and through environment value
education, young generations receive the knowledge about homeland environment
caring and protection.

B. Evaluation Purpose and Questions to Be Answered

The main purpose of the final evaluation is to have an overall review of the performance and
lessons of the project within the five-year project period and to bring out valuable
information/recommendations for local partner to continue the project when project is
finished.

Objectives of the final evaluation:

• To assess the achievement of project objectives at an outcome and output level as

defined in the Log-Frame of the project. Baseline data should be used.
 • To identify lessons which can be learnt from the project. In particular, to conclude
the model of sustainable development based on the project experience in the
communities.
• To analyze relevance, effect, impact sustainability, efficiency and coherence of the
project
• To provide recommendations to Buer and local partner to continue the project in a
sustainable way.

Evaluation questions

Project relevance

• Are project objectives relevant to needs of target people? How?

• Are the project approaches, strategies, and activities relevant/appropriate/helpful to
the project objectives? How?
• How has local context been taken into consideration when planning the project?
• Is the project experience applicable to other villages of Long Chuan County?

Project effect:

• In which level, the sub-goals and outputs are achieved, or not achieved?
• How are target groups benefited by the project? Has the project solved their problem
defined by the project analysis?
• In general, does this project solve the problem of unsustainable livelihood in local
communities? How?

Project impact:

• What impact has the project had to farmers including women?

• How did the project impact to community development and relevant government?
• Which aspects could be strengthened to reach better impact? How?
• How does the project make impact towards the local education system about
environment education? How does the project make impact on target farmers’ mind
about sustainable development?

Project effectiveness:

• Is the scale of beneficiary reasonable regarding the project input? If not, how to
promote the project result to more beneficiaries?
 • Comparing with achievements, Has the project input (including local and fund from
MA) been used effectively?

Project efficiency:

• How many stakeholders are involved in the project? How does the project make
them function?
• Do project strategies lead to sustainable and inclusive development in terms of
environment education, empowerment to women, and sustainable livelihood?

Project management:

• In the whole project cycle, which part is the strongest, and which is the weakest?
Why?
• Who participated in the process of plan making and management, and How?
• Is the monitor system including tools functioning and practical?
• Does the local project team have sufficient capacity and tools to carry over the
project? How?
• Is the financial management in accordance with law and regulation, as well as the
requirements of donors’ organization?

Project implementation:

• What kinds of factors have made obstacles for the project implementation? What
have been done to extinguish/reduce the difficulties?
• What kinds of resources and capacity building are provided by the project to the
partners and target people for the smooth implementation?
• What kinds of risks have the project come across? What have been done to anticipate
and reduce the risk?

Project sustainability:

• Has the project reached sustainability in terms of funding, human resource,

management and technique? If not, how to reach it?

Besides the above, the evaluation team is also expected to make recommendations on the
followings:

• In the organizational level, what kind of experience about the project shall be
concluded and shared by both country office and head quarter?
 • Experience and knowledge/tools about revolving fund management.
• Other external experience about sustainable development in the rural area.
• How to pass over the project experience to the local government.

C. Evluation Methodology and Process

1. Evaluation plan and guideline for interview

Before developing the evaluation plan and guideline for interview, we studied all important
project documents ranging from the base line study report (actually it is a need assessment
report), project document/proposal, project annual plans and reports, and analyzed the
logic relation between the need of and major issue faced by the villagers in their livelihood,
the project strategy and approach, and the project objective and outputs as well as their
measuring indicators. Based on this understanding and analysis, we followed the evaluation
TOR requirement to base our plan and guideline on the outcome and output and their
measuring indicators specified in the logic framework.

2. Target group interviewed in the evaluation

Following the evaluation TOR requirement and our interview guideline, we interviewed the

following groups：

Villagers representatives in Lvliang and Mushui villages.

Women representatives in Lvliang and Mushui villages, especially the Myanmar wives.
Representatives from the county educational bureau, and the primary school teachers
and students.
Representatives from the county agriculture bureau.
Representatives from the VDC and the seed fund management group.
Project officers from FAO and Buer Kunming office.

3. Evaluation methodology and process

Project file study: we studied all important project documents ranging from the base line
study report(actually it is a need assessment report), project document/proposal, project
annual plans and reports, and analyzed the logic relation between the need of and major
issue faced by the villagers in their livelihood, the project strategy and approach, and the
project objective and outputs as well as their measuring indicators. The purpose of this
study is to collect understanding and information of the project design, implementation,
and achievement so that we can have cross check during the field interview.

Field evaluation process and methodology

Following the requirement of evaluation TOR, and considering the time availability, we
adopted the methodology of participatory focal group discussion in the field interview.
Questions were asked in line with the interview guideline, and the focal groups discussed
and agreed on their responses. These methodology, though require experiences and skills
from the evaluators, and is kind of time consuming, has the advantage of covering more
participants, and promoting mutual sharing and learning , and cross checking among the
participants to achieve a more objective and representative result.

4. Evaluation strategy

In line with the evaluation objective, questions, and methodology specified in the TOR, the

following strategies were used in the evaluation：

1.The evaluation on project relevance was conducted through analysis

of the logic relations between the major development issues faced by the community, the
project objective, the project strategy and approaches, and the supporting activities.

2.The evaluation on project effect was conducted through checking the

results, changes, and outputs after the project implementation in line with the set targets
and measuring indicators specified in the project document. Considering that the project
has just come to the completion, impact evaluation was not a focus.

3.The evaluation on project management was conducted through checking the monitoring
system, the communication/supporting/and decision making process and mechanism, and
ways of learning and reflection etc.

4.The concept of participation was integrated into the whole evaluation process. Meaning
the groups were at the center for reflecting, sharing, analyzing, and summarizing; and the
role of evaluators is to facilitate the effect discussion in line with the interview guideline.

5.Based on the above-mentioned ideas, in-depth discussions were conducted with the
project personnel and the villagers respectively.

Discussions with the villagers mainly focus on realization of the outcomes and outputs
specified in the project design document. Based on the measuring indicators, the evaluation
team checked the change of the knowledge, attitude, behavior, and capacity of the target
group, as well as the relevant result before and after the project implementation. But
considering that different aspects need to be examined from different groups, we
re-organized the indicators in line characteristics of the different target groups, rather than
mechanically following the order in the logic framework.

The participants were put in 5 different groups according to the project activities they
involved. Namely a general group, a seed fund group, and women’s business group, a group
for Myanmar wives, and an environment education group. We also tried to strike a balance
in terms of age, gender, economic and social status when grouping.

Discussions with project personnel were mainly from the macro perspective to examine the
achievements, lesions, and experiences in realizing the outcomes and outputs; And to
self-assess in answering the evaluation questions. Project personnel from Buer Kunming
office, FAO, the county education bureau and agriculture bureau participated in the
discussion.

Evaluation conclusions and suggestions were reached on the base of these discussions,
project file studies, and the evaluators’ own observation and experience.

The whole evaluation was in 3 phases, namely:

Phase No.1: from November 11 to 24. The task in this phase was to study the project files
and prepare the evaluation plan and interview guideline.

Phase No.2: from November 27 to December 2. This was the field evaluation phase. Specific
schedule is reflected in the following table:

Date task venue Participants

Nov 27 travel to Kunming from Kunmin Peter Liu
Beijing g
Nov 28 1， Kunming to Longchu Project personnel from Buer,
Longchuan an FAO, and relevant government
county agencies, evaluation team
2， In-depth discussion town
with the project
personnel
Nov 29 Community interview Lvliang Project personnel from Buer,
FAO, and education bureau,
schools, students ,VDC
members and
villagers ,evaluation team
Nov 30 Community interview Mushui Project personnel from Buer,
FAO,villagers, VDC members ,
Myanmar wives, evaluation
team
Dec 1 Community interview Mushui Later changed to project
management discussion due to
availability of the villagers.
Dec 2 1， Feedback of the initial Kunmig Project personnel from Buer,
evaluation result FAO, evaluation team
 2， Back to Kunming

Phase No.3: from December 3 to 16.evlaution report writing and submission.

D. Evaluation Findings

The Longchuan Sustainable livelihood project(2012-2016) is made up of four components,

namely promotion of more natural agricultural practices, development of other agricultural
income channels, development of non-agricultural income channels (in some project
documents also named as “women’s development”), and awareness and knowledge
promotion of sustainability.Through evaluation, we found the following results about the

outputs of the activities：

1. Promotion of more natural agricultural practices:

This component is made up of 4 activities,namely animal husbandry seed fund

activity,forestry seed fund activity, animal husbandry related training, and forestry related
training. Findings of the result from these activities

1.1 Animal husbandry seed fund

The expected output of this activity is to increase the income from animal husbandry to
30%-35% in the total family income structure for the involved households. The actual result
is only 10%-20%. One of the reasons for the lower income is that the market price of goat
went down; another reason is that some deaths occurred due to improperness in fodder
provision and epidemic prevention.

At the same time, the repayment rate of the seed fund loan is low. Some villagers cannot
understand why they have to repay the loan in cash while they got it in kind(goat). The low
repayment rate is causing the low efficiency of the loan in its rotation to cover more
households.

Thus, this activity failed to achieve the expected output. However, the evaluation team think
that the idea and strategy is correct to include a revolving fund activity in the project; and
the experience gained is also valuable. Another benefit of the animal husbandry seed fund is
that it is promoting the green fertilizer which is one of the crucial element to improve the
deteriorating soil.

Therefore, the project team should not give up, but rather to continue to strengthen this
activity to win long term gain.
1.2 Forestry seed fund

The expected output of this activity is to increase the income from fruit tree planting to
about 10%-20% in the total family income structure for the involved households. The actual
result is that by the time of this evaluation, no income has been gained from this activity, as
the fruit trees have not started to bear fruit yet. The expected fruit bearing time is 3 years
after planting. But it did not happen because the villagers did not carefully look after their
trees as required in fertilizer providing, and pruning etc. Hopefully some harvest can be
available in 2017.

The repayment rate of this seed fund loan is also low. One of the reasons is late harvest as
described as above; another reason is that the local government delivered the same kind of
fruit tree saplings for free to the project community in 2014.Thus those involved households
think it unfair for them to pay for the sapling from this project.

Another finding about the seed fund is that it covers only a small number of households.
The inclusiveness and justice of the activity is of a concern.

1.3 Animal husbandry related training

The expected output of this activity is to build up knowledge and skills of the villagers in

animal husbandry to promote this industry. By the time of this evaluation，the villagers
have been trained in breed selection, sheepfold construction, fodder matching, epidemic
prevention etc. After the training, large scale of epidemic disappeared, and the villagers
have got the capacity to assess themselves the suitability to join the activity. At the same
time, some of the villagers could not shift from their traditional way of animal husbandry.
And the training effect to them is not ideal. But on the whole, all the planned trainings have
been conducted and the expected output has basically been achieved.

1.4 Forestry related training

The expected output of this activity is to build up knowledge and skills of the villagers in
fruit tree planting. After the training, the villagers involved have upgraded their
understanding and capacity, and basically are able to apply the knowledge and skills to their

practice. However，some of them could not strictly follow the requirement in looking after
their fruit trees. And this neglect results in late fruit bearing. But on the whole, all the
planned trainings have been conducted and the expected output has basically been
achieved.

2. Development of other agricultural income channels:

This component is made up 3 activities, namely rational application of fertilizers, cross

planting promotion, and training on soil improvement. If carefully examining the logic
relation between the planned outcome (promoting other agriculture income channels) and
the above 3 actual activities as well as their expected output, you will find that they are not
strictly in line with each other. This is an issue in project design, and we will discuss it at a
later stage.

2.1 Rational fertilizer application

The expected output of this activity is to build up understanding and skills of the villagers in
using green fertilizer to reduce the excess chemical fertilization by rational fertilizer
application in line with need of the soil. After the training, the participants understand the
harm of over chemical fertilization and are willing to adopt rational way of fertilizer
application. Almost 100% of the participants confirmed they can understand the
importance of using green fertilizer and will reduce using chemical fertilizer whenever
possible. So, from the aspect of awareness and willingness, this activity has got the expected
output.

However, the yet small scale of animal husbandry limited the green fertilizer supply; and
the inconvenience of green fertilizer application also prevented some people from making
full use of it.

2.2 Cross planting promotion

The expected output of this activity is to make about 50% of the households adopt cross
planting. The actual result is much better. Almost all the households applied, and the total
land area for cross planting is consistently increasing.

Maize, potatoes, red pepper, peanuts, and soy beans were cross planted in the sugarcane
field and fruit tree field. While providing fodders and green fertilizers, the cross planting
also increase income of the villagers. This activity is very successful, and it will sustain after
the project as it is both feasible and well accepted.

2.3 Training on soil improvement

The expected output of this activity is to improve the soil by increasing green fertilizer. The
actual result is that 2 sessions of training were conducted in the two project villages
respectively, and 4 session were conducted in other neighboring villagers. After the
trainings, the villagers got to know how to improve the soil. Some even bought the residual
of sugarcane from the sugar factory to put to their land. Thus, we can say that the expected
output has been achieved.

This activity has a huge room to expand. On the one hand, leaves and tops of the sugarcane
can be better handled to enrich the soil than the current practice of burning which also adds
to pollution. On the other hand, the sugarcane planting land area is very huge in Longchuan
and its neighboring counties, the successful experiences should be greatly promoted to
benefit a much wider population.
3. Development of non-agricultural income channels:

This component is made up 2 activities, namely Chinese training for Myanmar wives, and
community banquet service. When looking at the outcome and output targets in the logic
framework, this component should be one of the priorities. However, the two activities
planned for this outcome is too weak to support an effective result, and the Chinese training
activity did not have a direct logic relation with the set outcome, neither did it directly
contribute to changing the non-agriculture income channels.

3.1 Chinese training for Myanmar wives

The expected output of this activity is to provide opportunity for the Myanmar wives to be
better integrated into the mainstream community by improving their language skill. The
result is that basically all Myanmar wives participated in the Chinese training classes. And
with better language skill, they can more actively engage in community affairs such as
dancing, banquet service, skill trainings etc. During the evaluation, we found that 1/4 of
the 8-member dancing team are Myanmar wives. They told us that they are better
recognized by both their families and in the community; and their relation and interaction
with other community members are improving, though some of them still need further
upgrading in their Chinese. They said that they feel happier and more confident after these
activities. The calmness and confidence in their faces also verified what they said. Thus, we
think that the expected output of this activity has been achieved.

In the cross-border marriage, the Myanmar wives usually did not get the legal immigration
status, thus they cannot enjoy state benefit as their Chinese counterpart. This is a long
-existing issue for many counties, and it is unrealistic for a project to change the situation.

3.2 Community banquet service

The expected output of this activity is to promote women’s income generation, making the
income about 5% in the total family income structure. The evaluation found out that
women’s groups were formed to provide service to the community banquets (such as
wedding, funeral, festivals etc.) by dancing and table & benches leasing. Women increased
their opportunities and skills to engage in community affairs through the process, but the
income is only enough for operating the activities. Not meaningful income contribution is
made to their families. Therefore, the evaluation team think that its social effect is much
bigger than the economic one.

Through this activity, the women got more opportunity to participate in community affairs,
and their communication skill and confidence has been upgraded. At the same time, they
got more recognition in both their families and from the community. For example, some
women we interviewed said that their husband become more supportive to them to engage
in public affairs, and their family relation become more harmonious.
4. Awareness and knowledge promotion of sustainability:

This component is made up 6 activities, namely TOT training on environment education,

environment education textbook development, environment education summer camp,
students’ activities on environment education, teachers’ activities on environment
education, and sustainability awareness raising training for the villagers.

The evaluation team think that this component is the most successful part of the project and
almost all the outputs were achieved or even overachieved.

4.1 TOT training on environment education

The expected output for this activity is to upgrade knowledge and skills of the local teachers
in environment education in order that they can do their job in this area more effectively.
The actual result is that 3 sessions of TOT were conducted to cover all the central primary
schools in the county. 1-2 trainers from each central primary school participated in the
training. They were trained in knowledge and concept about environment and sustainability,
and in interactive teaching skills in environment education etc. Through the trainings, the
teachers got much deeper understanding about environment and sustainability; and their
environment education become much more effective. They also become more proactive to
integrate environment education in their daily teaching activities. In 2016, the two central
primary schools in Longba township and Husa township organized summary camp on
environment education subject by themselves. This initiative was highly praised by the
county educational bureau, and quite a few schools want to follow their ways.

4.2 Environment education textbook development

The expected output for this activity is to develop and publish environment education
textbooks and deliver them to the educational system. 500 copies were planned. In 2014,
this activity was completed, and 950 copies of environment education textbooks were
delivered to the local educational system, 50 more copies were delivered to other relevant
agencies.

4.3, Environment education summer camp

The expected output for this activity is to promote social atmosphere of environment
protection through summer camp. The activity was very successfully. Both the students and
teachers were impressed by this style of environment education. And some of them tried to
organize follow up summer camps by themselves. And some of the parents also participated
the activities. Video records were developed and shared with their friends and the idea of
environment protection was widely and effectively disseminated.

4.4 Students’ activities on environment education

The expected output for this activity is to encourage students to think about relation
between themselves and their hometown, and about their responsibilities in environment
protection. A series of activities were conducted in and out of the schools. While out of
school activities ranged from summer camps to environment education to parents by their
kids, etc. in-school activities included situational plays, blackboard newspaper, sharing
between students and teachers etc. These activities got a wide coverage and almost all
students participated. The evaluation team think that the activities were very successful and
effective.

4.5 Teachers’ activities on environment education

The expected output for this activity is to cause attention of the teachers in environment
education in order that they can integrate environment education into their daily teaching.
The activity was implemented in the form of knowledge contest on environment education.
The activity helped the teachers to have in-depth study on knowledge, theories, and
approached on environment education and share what they learned with their peers. The
result is very impactful and recognized by the county educational bureau. And the award
has become a reference indicator in the performance assessment of the teachers. The
evaluation team think that the outputs of the activity are also overachieved.

4.6 Sustainability awareness raising training for the villagers

The expected output for this activity is to increase the awareness and understanding of the
Villagers in sustainability and diversified production, so that they can proactively change
their unsustainable production model. After the training, the villagers understand the
reason why their current production model is not sustainable and why to adopt diversified
production and to improve the soil. The training have laid a solid base for the villagers to
understand project concept and implement the project activities such as cross planting,
rational fertilizer application, animal husbandry development, and fruit tree planting etc.
The expected outputs were achieved.

5. Project planning and design

The evaluation team did not find activity plan and their implementation time table in the
approved project proposal (or project document); Neither did we find a complete base line
data which was established against the measuring indicators of the outcomes and output
specified in the project proposal (logic framework part). The document titled “base line
report” though has some data relevant to base line, it is more a need assessment report.

At the same time, the evaluation team learnt that the project activities were developed in the
annual plans in different years. So, we assume that project activities were not specified at
the time of project design; and the outcomes and outputs as well as their measuring
indicators were decided after the need assessment was completed, and without a base or
support of project activities.
This way of project design though can leave room and power for the project implementation
stakeholders to decide specific project activities in line with the changing environment
during the project implementation process. However, project activities decided at a later
stage may not be strict in line with the set outcomes and their indicators; and outcomes and
their measuring indicators may not be realistic if they are not based on project activities. At
the same time, without specific activities and their implementing time tables, it is almost
impossible to establish a sound monitoring and evaluation system.

All the above-mentioned phenomena occurred in this project. The Evaluation team strictly
follow the evaluation TOR to design the evaluation discussions in line with the measuring
indicators of the outcomes and outputs specified in the approved project logic framework.
But later we found out that some of the actual activities are not very relevant to these
indicators. For example, some actual activities for outcome 2 and 3 are actually not so
relevant to income; and it is hard to find very relevant measuring indicators for some of the
activities actually carried out. So, it is hard to say that change of these measuring indicators
is caused by this project. We had to adjust the evaluation framework to continue the
evaluation.

However, the evaluation team realized that even agencies specialized in development work
are having challenges to ensure professional project design. For an agency like Buer, whose
project staff did not have enough technical training and support before designing the
project, it is understandable that the above-mentioned shortcomings appeared.

On the other hand, the design of the project need to serve the purpose the project want to
achieve. If the major purpose of the project is to empower the community to decide what to
do, and not so interested in measuring its result and effect, the current practice also make
sense.

Except for the above-mentioned issues. The project design is very professional and rigorous.
For example, the baseline study did an excellent job in analyzing the issues and their causes,
the setting of the project objective on promoting sustainable livelihood is very correct and
accurate, and the strategy and approaches also make good sense.

6. Limitations of the evaluation

We think that this evaluation has basically met the requirement of the TOR. However, some
limitations still exist. For example:

1）The required scope of evaluation is very wide. The evaluation questions need to be
answered range from implementation, management, output, outcome to impact, covering
almost all aspects. However, the time arranged for field evaluation is very short. So, the
evaluation could only focus on output and outcome level, and did not have time to examine
the project in a wider and deeper way; neither could we have time to conduct some in-depth
individual interview.

2）As no base line data was collected and established against the measuring indicators of
the outputs and outcome, we had to ask the participants to recall the situations of the
indicators in 2012 and compare those with the current ones in 2016. Due to fact that it is a
long past, the accuracy of the recalled numbers maybe a problem. And the recalling process
was very time consuming.

However, this regret does not affect us to make trend judgement of the change brought
about by the project; neither can it affect us in making suggestions for future project
practice.

E. Evaluation Conclusion

1. Project relevance

Through project documents review and field dialogue with the relevant stakeholders, the
evaluation team concluded that there is good relevance between the project objectives and
the needs of as well as the development issues faced by the project communities. Same
relevance is also obvious between project objectives and its strategies and approaches.

At the time of designing this project, the project communities mainly depend on sugarcane
planting for their livelihood. The monoculture of sugarcane on the one hand were
causing increasing deterioration of the soil and environment, due to application of
increasing quantity of chemical fertilizers and pesticide; on the other hand, were limiting
fodder sources to develop animal husbandry which is the source of green fertilizer. At the
same time, the mono-income structure is putting the villagers in a very vulnerable position,
as the sugarcane price is not stable. Thus, the monoculture of sugarcane is the leading cause
of the unsustainable livelihood of the project communities. Another issue faced by the
community was marginalization of women, especially the Myanmar wives, in the social life.
Due to various reasons like language, citizenship, and traditions, women, especially the
Myanmar wives, were excluded in major social life, and did not have the opportunity to
demonstrate their potential to gain respect in their families and in the communities.

To change this situation, the project aimed to establish sustainable livelihood for the
community by promoting diversified agricultural production and income structure, while
creating opportunities for women to participate in social life and income generation. Along
with this objective and strategy, the project took the approaches of raising sustainability
awareness, encouraging cross planting and fruit tree planting, supporting animal husbandry,
forming and supporting women’s groups, and training the farmers in in green fertilizer
utilization and soil improvement etc.
The evaluation found out that these objective, strategy and approaches are in good logic
among themselves, and are very relevant to the need of and issue faced by the local
villagers.

The two selected villages can also represent majority of the communities in the county in
terms of production and income structure, culture, and development level. Therefore, the
experiences, strategy and approaches of this project can also be introduced to the
neighboring communities.

2. Project effect

From perspective of project effect, the evaluation result shows that

Basically, the project is very effective, and many of the outcome targets have been achieved.
The valuation team think that these unrealized outcome targets such as changing of the
production and income structures are too ambitious for a 4-year project of this scale,
especially under the context that the sugarcane planting is both traditionally and currently
the leading industry in the local communities.

According to the project document, there are four expected sub-goals or outcomes for this
project namely:

1, In 5 years, more natural agricultural practices are utilized to reduce the impact of
sugarcane monoculture

2, In 5 years, farmers take less risk of monoculture by the development of other agricultural
income channels.

3, In 5 years, farmers take less risk of monoculture by the development of non-agricultural

income channels.

4, In 5 years, local people have rational awareness and knowledge of sustainability.

To the first sub-goal or outcome. Evaluation of the measuring indicators before and
after the project indicates that:

1）Change of the fertilizer structure：The average number of pigs raided by each

household increases by 100%. Though per unit land utilization of green fertilizer does not
change meaningfully, the total area of land covered by green fertilizer, or total quantity of
green fertilizers utilized doubles. However, though 100% of the villagers interviewed
expressed recognition and understanding on the importance of using green fertilizer, the
use of chemical fertilizer per unit land also doubles. The reason behind this is that the land
fertility decreased and the sugar factory increases the supply of chemical fertilizer.
2）Change of household percentage practicing cross planting： The evaluation
found that almost all households adopt cross planting at some degree. Far above the target
of 50%. From perspective of land area, cross planting in flat land increased from about 5%
in 2012 to about 20% in 2016, while cross planting in upland increase from about 5% in
2012 to about 85% in 2016.

While increasing income, the cross planting also generates much straw to enrich the soil,
and especially, the soybean root has the function of collecting and keeping nitrogen, which
is very helpful for the soil improvement.

3）Change of household percentage practicing crop rotation： The situation is:

each cycle of sugar cane planting will last 3-4years, and only in the interval between two
planting cycles can rotation be available; meanwhile, rotation in the intervals was already
commonly practiced before the project. Therefore, there is not much room for change for
this indicator.

To the second and third sub-goal or outcome. Evaluation of the measuring indicators
before and after the project indicates that:

1) Change of income structure：

The project has meaningfully promoted animal husbandry and fruit tree planting in the
project communities. And a more diversified production and income structure is on the way
to change the mono ones.

However, due to lack of effective care, the fruit trees take longer time than expected to bear
fruit and thus have not generated any income by the time of this evaluation. Hopefully they
can bear fruit next year. At the same time, income from the animal husbandry is not stable,
as they are affected by market price and epidemic situation. Moreover, the seed fund
supported animal husbandry by now only covers a small percentage of the households and
its income contribution to the community level is not meaningful.

The project also intended to promote women’s development and women’s income
generation in the non-agriculture sector through the women’s business group. This is a
good idea. However, the evaluation did not find meaningful income contribution by the
groups while their social effect is obvious.

Therefore, the evaluation team can only say that the project has not achieved this target.
But we believe that with income from the fruit tree and animal husbandry improved, the
income structure in the project community will change positively toward the objective.
2 ） The change of local governmental resources invested on diversified
agricultures:

Promoted by this project, relevant country government agencies such as the forestry bureau,
and agriculture bureau, and the county poverty alleviation office proactively contributed
their technology and resources in promoting the diversified agriculture in the project
communities. For example, the forestry bureau has provided technical support in the fruit
tree planting; The agriculture bureau has provided training in rational fertilizer application,
cross planting in sugarcane field, soil improvement, and animal epidemic prevention
training; The country poverty alleviation office provided free fruit tree seed to the
community though causing some negative effect to the repayment of the seed fund of this
project.

Therefore, the evaluation can conclude that this target has been achieved, though we think
that more work can be done to ensure support from these agencies coming in a more
coordinated and sustainable way.

3）The change of technique resources and channels to which farmers can

access.

Through trainings, both the villagers and the existing technical persons such as the vet, have
enriched their knowledge and skills in animal husbandry and planting. While the local
technical persons can provide better service, the training also provided connection between
these people and the trainers.

So, the evaluation team can say that local technical support has been improved though we
are not clear about the degree of the improvement.

4） People’s recognition change about women and men’s contribution and

value for the family income：

The evaluation found that women’s contribution in pig raising and cross planting is well
recognized. But due to physical requirement, men are still taking the major role in goat
raising and sugarcane planting.

The outcome, output, and their measuring indicators in the logic framework demonstrate
that women’s development and income generation was one of the priority focus of this
project, especially in the non-agriculture income sector. However, the evaluation found that
project activities arranged for this area were marginalized, and mostly limited to language
training and banquet dancing and table and bench providing. While these activities,
especially the dancing, had gained good recognition to women’s value from the social aspect,
the income contribution is not meaningful. And we are not sure whether the women’s group
has got enough capacity to sustain their operation or not after the project.

So, the evaluation conclude that this target has been achieved at some degree, but it is still
hard to say a full achievement.

To the forth sub-goal or outcome. Evaluation of the measuring indicators before and
after the project indicates that:

1) The awareness change of people’s recognition about nature and

environment; and the young generation’s awareness change about the
hometown and environment:

Through the project, all the stakeholders, including relevant government agencies, schools,
teachers, students, and villagers have improved their understanding and recognition about
nature and environment, as well as about the relation between environment and their
livelihood. And they are more willing to apply sustainable approaches to their livelihood
when possible. More discussions and analysis is reflected in part F and point No.4.

Therefore, the evaluation conclude that this target has been fully achieved or even over
achieved.

2）The application of sustainable approaches; and the change of farmers’ own

plan, as well as the governmental plan:

Just as comment and analysis in part F and point No.4, green fertilizer utilization and cross
planting have been commonly accepted and adopted by the villagers. And understanding,
recognition, and practices from the schools and relevant government agencies are also
changing positively.

In addition to the positive involvement from the county agriculture bureau and forestry
bureau as previously stated, the county educational bureau also work positively to make the
change. For Example, in their environment education project, they compiled and published
reference book on interactive environment education and shared them with all schools in
the county; they also organized county-wide knowledge contest among the teachers on
hometown and homeland; and environment education was included in the routine teaching
plan and performance assessment of teachers. These initiatives are very helpful build up the
awareness of the young generation on environment and sustainability.

Therefore, the evaluation conclude that this target has been achieved.
3. Project Impact

Sugarcane planting is the leading industry in LongChuan county, covering more than half of
the total farmland. Just as the situation in the two project villages, the increasing use of
chemical fertilizer and pesticide are reducing the fertility of the land and causing
environment pollution. In the face of reducing soil fertility, more chemical fertilizer is used
to ensure the yield, putting the production into a vicious circle while making the
environment pollution more and more serious. This practice is leading the livelihood to an
unsustainable and vulnerable model.

In the context, the project which aims to change to the mono-livelihood model by building a
diversified production and income stricture is very meaningful and important. Success of
the project will have very important and positive impact to the county as well as its
neighboring counties.

Though it is unrealistic for a project of this scale to have meaningful change within just 4
years, and it is too early to make rational judgement on the impact soon after the project
completion, the evaluation team can conclude that the project effect is very positive towards
its goals. The project has built solid understanding and recognition of both the villagers and
other stakeholders such as schools and relevant government agencies on a diversified and
sustainable production and income structure, and they are trying what they can to change
the current mono-model by environment education, green fertilizer application, cross
planting, rational application of fertilizers, fruit tree planting, developing animal husbandry
etc. Though the effect from animal husbandry and fruit tree planting is not obvious at
current stage, it leaves room for change in the future. We hope that income from the animal
husbandry and fruit tree will increase to a meaningful level in a few years to help the
villagers to move from depending on the monoculture of sugarcane. And the increasing
green fertilizer brought about by the increasing animal husbandry will reduce the
dependency of the villagers on chemical fertilizer. If this happens, a virtuous circle will
begin to form.

At the same time, the project has laid the good base for community women’s group and
financial cooperative. After some consolidation, these two organizations can play a big role
in social and economic development in the community in the future.

4. Project Effectiveness and Efficiency

Direct project input to the two villages is about CNY 3 million. If divided by the number of
the population in these two villages, the per capita input to the community is about CNY
1200. So we can say that with the input of about CNY 1200 per capita and in 4 years’ time,
the project has raised a villager’s awareness of sustainable livelihood, built up the capacity
in planting and animal husbandry, taught and convinced the villager to practice cross
planting and green fertilizer application, helped a Myanmar wife to integrate into the
mainstream, and left a seed fund for future development. From this angle, the project is
very effective and efficient.

At the same time, the project also promoted communication and interaction among
different players including the villagers, the women’s group, the villager committee, the
schools and the relevant government agencies. These communication and interaction is
very helpful to build the sense of diversity and inclusiveness in the community, while
promoting multi-sectoral cooperation for its benefit.

The project also helped to promote women’s development. With the project, and male
dominated social model in the community is changing, and women are more and more
accepted in the decision making both at family level and in community affairs. And women
are having more and more opportunity to demonstrate their value and capacity.

However, only a few people are involving in making the rules and can benefit from the seed
fund. And the repayment rate is low. Therefore, the efficiency, effectiveness, and
inclusiveness of the seed fund need to improve. At the same time, activity of the women’s
group is limited to community banquet service, and activity for the Myanmar wives is
limited to language teaching. Therefore, more work need to be done to make the women’s
group more effective and efficient in both women’s development and income generation.

5. Project Management

Just as discussed in Part E and point No.5, the establishment of the outcome and output
targets and their measuring indicators were not based on project activities. This practice has
caused some challenges to keep the project activities decided at a later stage fully in line
with the previously set targets and measuring indicators. This weakness when put together
with the lack of a base line data, is making the project monitoring and evaluation sometimes
blind and confused in consistency and direction.

Except for this weakness, the project management is quite normative, well organized, and
effective. It is really unusual for an organization like Buer which does not fully focus on
development work, and whose project staff could not get enough technical training and
support before and during the project implementation, to have achieved this.

For example, the project design was based on a sound need assessment; annual planning
and review were going on well; and all the important stakeholders, including the FAO staff,
village leaders, and villagers’ representatives, were participatorily involved in the discussion
and decision making; these involvement not only ensured local ownership and feasibility of
the plans for implementation, but also provided opportunities for the stakeholders to build
up their capacity by learning from and sharing with each other during the review and
reflection.

From the angle of monitoring and support, field visits were going on regularly, and progress
reports on project implementation and expenditure were timely prepared by the county
office, and received and reviewed by Buer Kunming office. Daily communication between
project communities and project offices at both county and provincial level, and between
FAO and Buer Kunming office are smooth and effective. For example, during our evaluation,
we found that intimacy was obvious between Buer Kunming office and the FAO, and
between project staff and the communities. We are kind of surprised that the Buer project
manager Mr Liu is quite clear about all the details of the project and familiar with almost all
the villagers we met. He and his counterpart at FAO are regarded as trustful old friends by
the project communities.

6. Project implementation

The project implementation in environment education, green fertilizer utilization, and cross
planting is quite successful. However, project implementation in the following areas
experienced challenges and difficulties.

1)，The sugar cane has been the leading industry in the communities for many years, and it
has formed a tradition for the villagers to depend on this monoculture for livelihood. In this
context, it is very challengeable to change the current mono-model and promote diversified
production and income structure in a few years. Though through the sustainability
awareness raising, the villagers understand and recognize the importance to shift to a
sustainable livelihood model, and are active in cross planting and green fertilizer
application, it is hard for some of the villagers to apply the trained new approaches to their
animal husbandry and fruit tree planting. And this reluctance has ended in slow growth of
the fruit trees and death of some of the goats supported by the seed fund. At the same time,
it will take time for meaningful income growth from the animal husbandry, fruit tree
planting etc. Before the replacing income source is fully formed, it is not realistic to
meaningfully change the current mono-model.

2)，The seed fund operation and management is also experiencing difficulties.

Qeensland nut planting: After the project Qeensland nut planting, the county poverty
alleviation office delivered Qeensland nut saplings for free to the project communities. This
free delivery has caused very negative impact to the repayment of the seed fund loan used to
by Qeensland nut saplings. Those households think it unfair for them to pay for the saplings
while other households got free ones from the government, and thus reluctant to repay the
loan. This lesson indicates the importance of learning government plans when designing
project activities, and keeping close coordination and communication with the relevant
government agencies.

At the same time, some of the households failed to follow the trained new approaches to
take care of their Qeensland trees. And this failure ends in late fruit bearing time than
expected.
Waxberry planting: Due to lack of timely technical training, the survival rate of waxberry
sapling after panting is low.

Animal husbandry: Due to traditional impact, some of the households are reluctant to
adopt the new approaches trained to their practice. And this reluctance cause death of the
goat raised by some households.

The above-mentioned reasons and phenomena end in low repayment rate of the seed fund
loans.

After finding out the problems, the county project office (FAO), together with the village
development committee (VDC) conducted series of discussions with relevant households,
making them understand the impact to other villagers if their loans are not repaid. After
this effort, the seed fund loan repayment began to change positively.

However, the evaluation team thinks that in addition to the abovementioned reasons, the
immature operation and management model is another cause, maybe the major cause of the
low repayment rate. This is especially true when only a few people were involved in
discussing and deciding the operational rules, and only a few households were covered by or
benefiting from seed fund.

7. Project sustainability

Villagers’ awareness and practice: The Villagers have reached common understanding
and recognition on the importance of sustainability, application of green fertilizer, animal
husbandry development, and cross planting etc. The evaluation team believe that these
understanding and practice will continue after the project.

Awareness and practice at the schools: The teachers and school leaders we visited
have got enough understanding and skills to continue the environment education in the
schools. They also have got enough understanding on the importance of environment to
their livelihood and thus have got the passion to continue the environment education. Some
of them even expressed that after this project, they will try to look for fund to continue some
of the activities such as summer camp, environment knowledge contests etc. Therefore, we
believe that environment education will at some degree continue in the schools after the
project.

Management capacity: Both the two village-project management group and the county
project office (FAO) have got enough knowledge and skills to management the project.
However, the FAO may be closed after this project; and the operational model of the villager
project management group is vulnerable. Leaving of a key member may end in collapse of
the group. This is already happening in Moshui village when the leader left.
At the same time, as suggested previously, the women’s group and the seed fund
management group are also weak in their self-management and development capacity.

Funding resource: The village can ask for help from their linking government agency.
Therefore, to ensure project sustainability, building the self-management capacity and
mechanism at the village level is needed and important. For this purpose, the women’s
group, and the seed fund management group should be prioritized.

F. Follow up Suggestions and Comments

1.Suggestions and comments on project planning and design

1），Project activity specification is the base for developing other steps in the
designing framework.

Project objectives and outcomes need to base on outputs from the activities; and project
activities need to be in line with the root causes of the issue identified. Only following this
logic flow, can the logic link between issue and project activities, between issue and project
objectives, and between objectives and activities be clear and convincing. At the same time,
only after the project activities are specified can implementation plan and time table be
developed and M&E system be established

Of course, the activities can be adjusted when the need base changes. Many projects include
a mid-term evaluation, and one major purpose of this evaluation is to review whether it is
necessary to do some adjustment to the activities, output, and objectives if the base for
designing the project changed. However, usually no big change will occur if the design is
based on solid need assessment and consultation.

2）Base line data is the base to rigid measurement of project result

After the project document or proposal is approved, if the project wants to have rigid
assessment of its result and impact, the base line data against the measuring indicators
need to be collected before project implementation. Some project even set up a reference
group and the same base line data is collected in the reference group. However, the practice
of reference group is very complicated and expensive, and generally it is only for large scale
project.

Nonetheless, if a project designer does not intend to have rigid measurement of the result
but only want to have a judgement on the trend or direction, it is not necessary to establish
the baseline data. And actually, project implementation without a base line data is the
general practice for quite a few NGOs. But the measuring indicators need to be simplified in
this case.
2.Suggestions and comments on the seed fund

Actually, in addition to the previously mentioned narrow coverage of beneficiaries and low
repayment rate, there is risk of loss of the fund if the current mechanism is not improved.

To solve the problem and avoid the risk, the evaluation team make suggestions for both
governance level and operation level.

1) Governance level: Make all the households shareholders of this fund, and then form a
shareholders’ cooperative in each village. On this base, all shareholders jointly discuss and
agree on mechanism, policies and rules in operating and managing the fund, including
loan delivery, repayment, interest etc.. It is also suggested consulting the “village mutual
help fund model” initiated by the poverty alleviation office under the state council when
deciding the policies and mechanism. At the same time, external technical person on
finance and cooperative management need to be invited to facilitate and guide the model
and policy establishment; and representatives from the villages need to have the
opportunity to visit one or two mature and successful cases to build up their experience and
confidence. And there are already quite a few very successful cases in Sichuan, Congqing,
Heibei and Guangxi.

At the same time, to continue promoting women’s development in the communities, there
should be reasonable number of women members in the management team, and women’s
income generation should be among the priorities supported by the fund.

2) Operational level: We suggest that the following areas need to be carefully considered
for sustainability reasons:

(1) There should be interest to the loan: while ensuring that the operational cost can be
covered, the remaining interest income can also help to expand the scale of the loan to
benefit more households. At the same time, paying interest can present misuse of the loan
by the powerful group in the communities.

(2) Strengthen the self-management and development capacity of the management team.
Currently the team is mainly driven by a crucial member. In case this member leaves, the
management and operation cannot sustain. The management goal is to ensure that
operation can sustain even when the most crucial member leaves.

(3) There should be flexibility in the way of loan delivery and repayment. For example, the
present-passing model from Heifer international sometimes is very effective, especially for
animal husbandry project.

At the same time, when delivering the loan, the production cycle and local production
tradition should be considered. E.g. long-term loan will make the repayment more risky and
the rotation less effective to cover more needy people.
(4) Participatory decision making and operation transparency should be ensured. If all the
villagers participate in discussing and deciding the rules and mechanisms and are aware of
the rules of loan delivery and repayment, their sense of ownership and belonging can be
assured, and the risk of loan misuse will be greatly reduced.

3. Suggestions and comments on experience promotion

Sugarcane planting is the leading industry as well as leading income source for most
villagers in Longchuan, as well as in its neighboring counties. And increasing quantity of
chemical fertilizer and pesticide are used to promote the yield in the face of the declining
land fertility caused by the chemical elements. Thus, sustainability of the land become quite
crucial for the future of the local agriculture and livelihood.

Under this context, it is quite meaningful to promote experiences gained from this project
to its neighboring communities to promote change in a wider range. Therefore, the
evaluation team suggest conducting some advocacy and experience promotion activity at
policy making level before the project handover, to consolidate effect and impact of this
project.

4. Suggestions and comments on the women’s group

The women’s groups established in this project has generated quite positive social result
and even impact at some degree. However, the self-management and development capacity
of the groups are not mature enough to sustain after the project. At the same time, the
income generation function of the group need further support before it can generate
meaningful result.

Therefore, the evaluation team suggest that before the handover, some more work need to
be done to consolidate the groups; and it is helpful to link the income generation function as
well as women’s development with the seed fund.

5. Suggestions on the handover

To maximize project sustainability and impact, the evaluation team suggest that Buer leave
another year to consolidate some components of the project before the handover. For
example, effective operation and management model and rules of the seed fund need to be
established; the self-management and development capacity of the women’s group need to
be strengthened; some experiences gained in the project such as in sustainability awareness
raising, green fertilizer utilization, cross planting, and rational application of fertilizers etc.
are worthwhile to be shared with the neighboring communities and advocated to the local
policy makers.
G. Annex (only for Chinese version)
1, The evaluation plan
2, Project output self assessment form (for project staff)
3, Self assessment form on the evaluation questions (for project staff)