SMART CITIES

#2 - citizen

COLLECTIVE INTELLIGENCE
#1 - MOBILITY

TO CO-CREATE THE CITIES

OF THE FUTURE
PROPOSAL OF AN EVALUATION TOOL FOR CITIZEN
INITIATIVES
 sustainability

Article
Collective Intelligence to Co-Create the Cities of the Future:
Proposal of an Evaluation Tool for Citizen Initiatives
Fanny E. Berigüete 1, * , Inma Rodriguez Cantalapiedra 1 , Mariana Palumbo 2 and Torsten Masseck 2, *

1 Department of Physics, Universitat Politècnica de Catalunya UPC, 08028 Barcelona, Spain

2 Department of Architectural Technology, Universitat Politècnica de Catalunya UPC,
08173 Sant Cugat del Vallès, Spain
* Correspondence: [email protected] (F.E.B.); [email protected] (T.M.)

Abstract: Citizen initiatives (CIs), through their activities, have become a mechanism to promote
empowerment, social inclusion, change of habits, and the transformation of neighbourhoods, influ-
encing their sustainability, but how can this impact be measured? Currently, there are no tools that
directly assess this impact, so our research seeks to describe and evaluate the contributions of CIs in a
holistic and comprehensive way, respecting the versatility of their activities. This research proposes
an evaluation system of 33 indicators distributed in 3 blocks: social cohesion, urban metabolism,
and transformation potential, which can be applied through a questionnaire. This research applied
different methods such as desk study, literature review, and case study analysis. The evaluation of
case studies showed that the developed evaluation system well reflects the individual contribution
of CIs to sensitive and important aspects of neighbourhoods, with a lesser or greater impact accord-
ing to the activities they carry out and the holistic conception they have of sustainability. Further
implementation and validation of the system in different contexts is needed, but it is a novel and
interesting proposal that will favour decision making for the promotion of one or another type of
initiative according to its benefits and the reality and needs of the neighbourhood.

Keywords: social innovation; sustainable cities; bottom-up process; citizen participation; tool for
Citation: Berigüete, F.E.; Rodriguez assessing sustainability; sustainable living habits; resilience; neighbourhoods; community living
Cantalapiedra, I.; Palumbo, M.;
Masseck, T. Collective Intelligence to
Co-Create the Cities of the Future:
Proposal of an Evaluation Tool for 1. Introduction
Citizen Initiatives. Sustainability 2023,
Today’s cities require a change in urban planning to address the complex relationships
15, 7956. https://doi.org/10.3390/
su15107956
between citizens’ interests and needs, urban transformations, environmental degradation,
urban policies, and economic interests in a sustainable approach [1]. Social innovation
Academic Editor: Víctor Jesús is one of the pillars of sustainability that is becoming increasingly important as one of
García-Morales the dimensions that merit a multidisciplinary and transversal approach [2–9], where the
Received: 6 March 2023 active participation of citizens is required not only as recipients [10] but also as co-creators
Revised: 5 May 2023 of their reality. In this sense, a series of mechanisms and strategies have emerged that
Accepted: 10 May 2023 seek to involve citizens in decision-making processes in the city, such as participatory
Published: 12 May 2023 budgets, urban centres, living labs [11,12], and innovation centres, in which synergies are
created between civil society, public administration, academia, and companies [13] to seek
solutions to social challenges; these strategies and platforms are usually guided by public
administration. On the other hand, another type of participation and appropriation of space
Copyright: © 2023 by the authors. mechanism called Citizens’ Initiatives (CIs) are born in a spontaneous but conscientious way.
Licensee MDPI, Basel, Switzerland. CIs can have different natures, guided through top-down processes or self-managed
This article is an open access article
through bottom-up processes. This article focuses on CIs aligned with bottom-up processes,
distributed under the terms and
which can be defined as “informal processes of citizen practice that resiliently and adap-
conditions of the Creative Commons
tively modify the urban environment. They are self-organized, collective practices that
Attribution (CC BY) license (https://
work for the urban empowerment of citizens and develop critical processes on the current
creativecommons.org/licenses/by/
city” [14,15] (this citation was translated from Spanish by the authors).
4.0/).

Sustainability 2023, 15, 7956. https://doi.org/10.3390/su15107956 https://www.mdpi.com/journal/sustainability

Sustainability 2023, 15, 7956 2 of 28

These initiatives arise from a variety of problems and perspectives in the city, such
as the recovery of public spaces, the creation of gardens, the use of renewable energies,
and the protection of the city’s cultural and historical legacy, among others. As Cámara
Menoyo pointed out, these are spontaneous processes with heterogeneous results, which,
although they do not have a similar scope to the urban planning processes promoted by
local authorities [16] (this citation was translated from Spanish by the authors), have a
positive impact on local problems and on raising citizens’ awareness.
The role of CIs in the transition to sustainability is a topic of growing interest [17,18]
due to recent evidence that they can in one way or another contribute to improving their
environment and people’s quality of life, yet there is little evaluation and assessment of
their level of contribution [19].
CIs have the potential to be less constrained by top-down structural processes and
stimulate small-scale changes that positively impact the urban space and its citizens, as
they are not subject to institutional policies and bureaucracies. That said, little is known
about the potential of CIs to generate beneficial impacts for the Sustainable Development
Goals (SDGs), as well as their leadership in creating fertile ground and contributing to a
more sustainable urban city model [20].
This article reports on the work carried out on the basis of the collaboration agreement
between InnoEnergy SE and Renault Spain “Development and Piloting of Assessment
Model and Certification Scheme for Sustainable City Neighborhoods and Citizen Initia-
tives_FeliZiudad”. Within the framework of this project, the Barcelona Urban Ecology
Agency (BCNecología) and the Universitat Politècnica de Catalunya (UPC) were contacted
for the development of an assessment system for neighbourhoods and CIs; the UPC team
was in charge of the development of an assessment system for CIs, with reference to their
degree and contribution to the level of sustainability of the neighbourhood in which they
are developed [21].
The main objective of the project, in relation to the evaluation of the CIs, was to create a
system to measure the impact of the CIs on the sustainability of the neighbourhood, through
an automatic evaluation process hosted on the FeliZiudad platform. This procedure would
be carried out by a CI manager and its results would have an impact on the overall
evaluation of the neighbourhood [21].
In order to understand this tool design process and the results obtained, we structured
this article as follows: (i) the introduction discusses the concept of CIs, highlighting their
role in the transition towards sustainability; (ii) a bibliographical review is framed mainly
in the European context, where mention is made of a review article analysing evaluation
models in the European context and how these differ from our proposal; (iii) the methodol-
ogy used to create the evaluation system is detailed; (iv) the development of the evaluation
proposal is given; (v) the evaluation process of CIs in Spain is discussed; (vi) the results
of the case studies and discussion around them are given; and (vii) the conclusions of the
evaluation process of CIs are presented.

2. Background
The first widespread and organised collective movement arose as a reaction to exces-
sive prices and poor-quality products, especially in the food sector. The modern cooperative
movement emerged in Rochdale, in the northwest of England, in 1844, in the midst of
industrialisation. Although the first cooperatives date back to an earlier period, at the hands
of craftsmen, it was at this time that a wave of cooperative movements emerged, based
on the principle of “self-help by the people” without distinguishing between consumers
and producers [22].
The history of cooperative movements and community development initiatives are
intertwined, as both movements have grown out of people’s need to collectively address
issues affecting their daily lives. Cooperatives are an example of community-driven efforts
to address economic and social inequality, while community development initiatives
aim to promote social change by empowering communities to take charge of their own
Sustainability 2023, 15, 7956 3 of 28

development. Both movements have been instrumental in addressing issues of poverty,

access to goods and services, and social exclusion [23].
For instance, research has shown that cooperatives can have positive impacts on
poverty reduction, food security, and women’s empowerment [23,24]. Similarly, commu-
nity development initiatives have been found to be effective in promoting community
participation, social cohesion, and sustainable development [25,26].
Over the past 30 years, tools have been developed to support the development of
these integrated initiatives for community change and provide visibility for them. In 1995,
a team of experts from the University of Kansas created an online tool to guide community
processes on their path to sustainability, called the Community Tool Box. This tool provides
a range of content and information within the framework of community development and
health, enabling the management and evaluation of community initiatives. Although this
tool does not focus on assessing the environmental impact of initiatives, it helps commu-
nities overcome the barriers that may arise in their search for community development,
creating consolidated groups that work together to generate solutions to common problems.
As such, cooperative movements and community development initiatives can benefit from
each other’s experiences and practices to further promote sustainable development and
social change [27].
On the other hand, in Europe, the Treaty of Lisbon, which amends the Treaty of
European Union (EU) and the Treaty establishing the European Community, which entered
into force on 1 December 2009, opens a new window for citizen participation by giving
citizens the opportunity to participate directly in legislative development [28]. In the
same vein, cohesion policy is the main investment instrument of the European Union [29],
whose objective is to “support job creation, business competitiveness, economic growth,
sustainable development and the improvement of the quality of life of citizens in all regions
and cities of the EU” [30].
In 2020, the European Commission developed two pilot schemes to facilitate and
enhance the active participation of citizens in social cohesion projects. In the first pilot
scheme, the Organization for Economic Cooperation and Development (OECD) provided
advice and training to the authorities managing the funds for the implementation of new
initiatives to encourage citizen participation, in a coworking model where the needs and
opinions of all the groups involved were considered. In the second pilot scheme, the
European Commission earmarked EUR 250,000 to fund and support innovative CIs from
small local civil society organizations. This favoured the inclusion of small and local
civil society institutions in European funding programs, an action that generates citizens’
ownership of the changes brought about by the cohesion policies developed [31].
Simultaneously, in recent decades, CIs have emerged in Europe to address the many
early 21st-century challenges facing society at the local level. Through them, citizens
collectively strive to achieve common goals, based on the self-management of the partic-
ipants. As a result, new collectives emerge and the associative fabric gains strength in
the management of community needs, as well as in the improvement of their immediate
environment [32].
In Spain in recent years, there has been a proliferation of such collectives and as-
sociations that empower citizens in the promotion and implementation of sustainable
practices [33]. In this type of association, citizens play an active role in the self-management
of resources and in the generation of collective spaces for the debate of local problems, lead-
ing projects to reconfigure the physical and social context in which they are located [34]. In
this sense, several research studies attempt to analyse CIs, whether from the context [35,36],
the typology [37,38], or the way they are managed [17,39].
The evaluation of these types of collectives and associations is scarce. Our review of
the existing literature identified only three systems in Europe aiming at the measurement of
the level of empowerment of citizens to modify and improve their environment. These are:
Sustainability 2023, 15, 7956 4 of 28

• Towards European Sustainable Societies (TESS)—United Kingdom, Italy, Spain, Finland,

and Romania (2013–2016).
• Multidimensional assessment of the environmental and socioeconomic performance
of community-based sustainability initiatives (MDA)—Italia (2019).
• Criteria for assessing the transformation potential of sustainability initiatives (CATPSI)—
Germany (2019).
An in-depth review analysing these three evaluation models can be found in [40].
In summary:
In 2019, Celata and Sanna created an evaluation system assessing the impact of
initiatives on their surroundings in four areas: environmental, social, economic, and
community development. They evaluated 37 initiatives and classified them according to
their typology: food cooperatives, solidarity shopping collectives, community gardens,
community energy, recycling, and mobility initiatives [18].
In 2013, the European research project TESS (Towards European Social Sustainability)
was launched, which analysed the role of community initiatives on the road to sustainability
in cities, through the creation of an evaluation system and the subsequent analysis of 63 case
studies, which made it possible to monitor and report on the social, political, economic,
technological, and environmental impacts of community initiatives, as well as their savings
in carbon emissions [20].
Finally, the project Kriterien zur Bewertung des Transformationspotenzials von Nach-
haltigkeitsinitiativen (criteria for assessing the transformation potential of sustainability
initiatives) developed a system for assessing the sustainability and transformation potential
of initiatives in order to reveal their value and facilitate recognition and support by public
administrations [41].
The analysed systems share a common structure comprising criteria, dimensions, indi-
cators, and a questionnaire. However, they differ in their weighting of parameters, namely,
innovation, resilient communities, recommendations to administrations, organizational
capacity, scalability, transformation potential, and political dimension. The evaluation of
initiatives in these systems is time-consuming due to the need for extensive interviews
involving up to 130 highly specific and lengthy questions. Notably, while sustainability is
a primary focus, these systems do not prioritize the improvement and transformation of
urban space.
In our system, we incorporate the spatial dimension into the evaluation of citizen
initiatives, recognizing its significance in influencing urban dynamics. Our aim is to develop
an accessible tool that all initiatives can use to self-assess and reflect on sustainability. To
ensure robustness and inclusivity, we are currently collaborating with experts from various
fields and stakeholders to create a more holistic and comprehensive tool. We aim to improve
the design to be scalable and replicable, with flexibility to adjust the catalogue of criteria,
indicators, and questions based on its application.

3. Methodology
The evaluation system for CIs created has followed a number of phases over the course
of the project (about 2 years), which are depicted in Figure 1 and described below.

Changes
to the
Results
Initial Selection Tool Case system Dissemination
and
approach of criteria design studies and new of results
feedback
case
studies

Figure 1. Phases of CIs evaluation system FeliZiudad. Source: prepared by the authors.
Sustainability 2023, 15, 7956 5 of 28

3.1. Step 1. Initial Approach

The first phase was to identify previous relevant works by analysing the publications
in the databases and repositories (Scopus, Web of Science, Dialnet, Oxford Journals, UPCom-
mons, and Recyt) as well as consulting the social academic networks (Academia.edu and
ResearchGate) and the Google Scholar search engine, using, initially, two keywords: “Citi-
zen initiative” and “community participation”. Due to the limited information obtained,
other keywords were included in the search: “Self-assessment + neighbourhood impact”,
“Community participation”, “Community initiatives”, “Indicators + neighbourhood im-
pact”, “Community participation + evaluation”, “Neighbourhood impact + sustainability”,
“Community self-management + neighbourhood improvement”, “Citizen self-management
+ appropriation of public space”, “Citizen self-participation”, “Community empowerment
+ self-assessment”, and “Citizen empowerment”. All of these were searched in Spanish,
English, French, and German over a period of two months. This search resulted in the
identification of the three evaluation models mentioned before.

3.2. Step 2. Criteria Selection

For the selection of the criteria, FeliZiudad researchers (professionals in architecture,
urban sociology, and materials) made a review of the existing literature and identified
the most relevant evaluation criteria among all the evaluation systems reviewed (the
three specific ones but also other relevant systems, both theoretical and applied). In the case
of all three systems, the criteria had been selected in processes of consultation, literature
review, surveys, and validation with experts and stakeholders of the initiatives, for example:
In TESS, consultations and interviews were conducted among the project researchers
(natural and social scientists) and the actors closest to the initiatives, and a ranking of
criteria was collaboratively elaborated according to their impact and importance in the
sustainable development of the community.
In MDA, surveys were carried out in which the initiatives themselves identified
those fundamental aspects that mark their objectives. From all the aspects identified,
two exclusion criteria were established: on the one hand, the criteria had to have been
considered important by a minimum of 65% of the initiatives surveyed and to have obtained,
in terms of importance, an average of 7–10 points; on the other hand, these criteria had to
be able to be converted into measurable evaluation indicators.
Finally, CATPSI analysed various scientific articles and 14 existing sustainability evalu-
ation systems, mainly in the German context, in order to define the evaluation criteria [41].
From there, the FeliZiudad team set up a discussion table to select those criteria
that were most related to the CIs and those that have a potential to be worked on by
them. Among the criteria selected were the impact of the initiative on the public space,
networking, the robustness or maturity of the initiative, change through learning and
training, organisational structure, creation of quality employment, sustainable commitment,
human resource orientation, and the quality of the initiative. In this way, three general
blocks were established, which contemplated these criteria.

3.3. Step 3. Tool Design

For the creation of the evaluation tool, three levels were structured: blocks, topics
(evaluated by indicators), and, finally, a series of questions responding to these indicators.
In order to establish this catalogue of questions, indicators were defined and objectives
were established for each indicator, so that the questions would be as coherent as possible
with the indicator and could be evaluated correctly.

3.3.1. Evaluation Blocks

The proposed sustainability assessment system is organised in three blocks, each
covering a potential area of impact of a sustainable CI: metabolic efficiency, social cohesion,
and transformational potential, allowing us to address social, environmental, economic,
spatial, and urban metabolism aspects. In addition, similar to the TESS system [40], a fourth
Sustainability 2023, 15, 7956 6 of 28

domain was included, which is not scored but serves to filter those initiatives that meet the
minimum evaluation criteria, as detailed in Section 4.1.1.

3.3.2. Topics and Indicators

Once the blocks were established, these were split into topics, which, in turn, were
characterized using a range of relevant indicators that had to be able to measure the
performance of CIs in that specific topic. These indicators were chosen on the basis of
existing literature and a process of analysis by FeliZiudad project members.
In the evaluation system, 33 indicators were established and defined. The indicators
were distributed so that each block had a similar number of indicators.

3.3.3. Questionnaire and Weighting System

Once the indicators were defined, the data collection method was determined, which
usually takes the form of a questionnaire.
A total of 79 basic questions (objective) and 40 supplementary questions (subjective,
depending on the typology of the initiative) were established. These questions were
carefully selected, trying to be coherent with the indicator to be evaluated, collecting as
much information as possible and using concepts that can be understood by the initiative.
This resulted in a total of 119 questions distributed in the defined blocks. The rating
system allowed scores between 0 and 100 points for the basic questions, and between 0 and
50 points for the supplementary ones.

3.4. Step 4. Selection of Case Studies

To verify the evaluation system, a series of case studies were selected to provide
feedback on the process and to detect its strengths, weaknesses, and possible improvements.
First, five CIs were selected from those previously registered on the FeliZiudad website
(this page is currently unavailable) (project page). The selection was made, aiming at the
greatest diversity of cases in terms of activity, size, area of influence, and age of the CIs.
This evaluation was carried out based on the information available on the internet.

3.5. Step 5. Results and Feedback

Then, an analysis of the results was carried out and improvements were made to
the system. Therefore, the application of the evaluation system was carried out in a peer
process, with two to three researchers (from the fields of architecture, materials and urban
sociology) evaluating each initiative independently, and in a collective review process
afterward, agreeing on a consensus regarding the final results for each case. This process
allowed a critical analysis on the indicators and questions and minor adjustments in the
weighting system.

3.6. Step 6. Changes to the System and New Case Studies

In this step, the questionnaire was simplified, reducing it to around 75% of the original
questionnaire. The simplification was motivated by the evaluation time needed to answer
the 119 questions, which was in the range of 2 h, so one of the main objectives was to
achieve a more reasonable time, around 25–35 min. Details of the process can be found in
Section 6.2.

3.7. Step 7. Dissemination of Results

The project sought to disseminate the results through presentations, conferences,
seminars, and congresses [42]. Of particular note is the article reviewing the current systems
for evaluating community-based initiatives, analysing their creation process, barriers, and
future prospects [40].
Sustainability 2023, 15, 7956 7 of 28

4. Evaluation Tool
The proposed evaluation tool consists of four blocks. The first block defines the
minimum criteria that an initiative must meet to be considered for evaluation, with a total
of five indicators, while the following three evaluation blocks contain 33 specific indicators
to be evaluated and scored through 119 questions and their predefined answers.
The questions require single- or multiple-choice answers, which facilitates the au-
tomation of the system in an online format. The few additional questions that allow for
open answers do not score in the evaluation system but allow for qualitative information
relevant to the understanding of the activities carried out by the evaluated initiatives.

4.1. Evaluation Blocks

The system for assessing the sustainability of initiatives is organized into four blocks.
The first one (block A) is not scored but serves to identify in a first step those initiatives
that meet the minimum criteria for assessment from those who do not (Figure 2). The
three other blocks are the main evaluation blocks, identified from the review of existing
assessment systems. Each covers an area of impact: (1) Social Cohesion, closely linked
to the Social Solidarity Economy (SSE) and the level of integration in the neighbourhood,
(2) Metabolic Efficiency, related to resource management and environmental quality, and
(3) Transformation Potential, linked to the capacity of CIs to influence new models, practices,
and sustainable habits.

Figure 2. Evaluation blocks. Block A: Minimum criteria. Block B, C, D: Main evaluation blocks.
Source: prepared by the authors.

4.1.1. Block A. Minimum Criteria

The first block of the evaluation system incorporates the minimum criteria that ini-
tiatives must meet in order to be evaluated, both in relation to the type of organization
and the activities they carry out. Thus, in order to be evaluated in the FeliZiudad system,
initiatives must fulfil all the conditions mentioned below:
• To have, as their main objective, the promotion of the sustainability of the neighbour-
hood and/or the community in which they operate.
• To be a nonprofit organization (formal or informal).
• To be made up of at least two people (single-person initiatives are not included).
• To have carried out activities with tangible results prior to the evaluation process.
Sustainability 2023, 15, 7956 8 of 28

The results in this block will not add points to the overall evaluation of the initiative
but will serve as a filter for its participation or exclusion in the evaluation.

4.1.2. Block B. Social Cohesion

The social dimension is often the most relevant impact of CIs. These are often based
on actions that generate collaborative networks between people with diverse realities and
common objectives or visions. The human factor is the main resource of the initiatives
and, for this reason, the initiatives generally pay special attention to the inclusion of
disadvantaged groups and to the creation and care of the community, regardless of the
specific objective they pursue.
The Social Cohesion block includes 10 indicators with 44 questions that evaluate the
impact of the initiative on the strengthening and care of the community and the physical
environment in which it is developed.

4.1.3. Block C. Metabolic Efficiency

The second block refers to the contribution of initiatives to the improvement of the
metabolic efficiency of the neighbourhood, both at the household level and at the level
of urban metabolism. This block analyses the capacity of the initiatives to influence
aspects such as the circular economy, waste reduction, energy consumption, energy self-
sufficiency, etc., of the neighbourhood.
The Metabolic Efficiency block includes 11 indicators with 31 questions that evaluate
the impact of the initiative on the efficient management of resources and the reduction of
waste and emissions.

4.1.4. Block D. Transformation Potential

The transformative potential generated by the initiatives is one of the most valuable
contributions of this type of citizen organization in the holistic processes of transforming
our society and our cities towards more sustainable models [16]. Transformation potential
is defined as the capacity of an initiative to create a sustained impact over time in the
community in which it participates, whether through influencing people’s daily habits,
generating spaces for debate, or activism aimed at transforming existing policies and
regulations in terms of social rights and environmental protection.
The Transformation Potential block is made up of 12 indicators with 44 questions
that evaluate the impact of these in terms of changes in habits and policies, collaboration
networks, the dynamism of public space, the creation of new facilities, etc.
This distribution in blocks makes it possible to organize both the topics and the
indicators so that it is possible to choose the degree of aggregation in which the evaluation
results are presented: overall score (a single value), score by blocks (3 values), score by
topics (13 values), or score by indicator (33 values).

4.2. Indicators
An indicator is defined as a measuring tool that allows us to evaluate in a simple and
reliable way the fulfilment of the defined objectives and the achievements or development
of an initiative or intervention proposal [17].
For the evaluation of the level of sustainability of the CIs, 33 indicators were defined.
These indicators were selected because, according to the literature review carried out, they
are essential in the study of the proposed evaluation blocks (Social Cohesion, Metabolic
Efficiency, and Transformation Potential). The assessment system is designed to break
down information or areas of analysis into manageable “blocks” or “topics” and use
“indicators” to qualitatively measure CIs (Table 1). The use of a modular framework offers
several advantages, such as flexibility, scalability, and ease of use. The system can be
adapted to the realities of different CIs. The use of indicators also provides a standardised
way of measuring progress or performance in different areas, allowing for more effective
monitoring and evaluation.
Sustainability 2023, 15, 7956 9 of 28

Table 1. Evaluation blocks, topics, and indicators of the methodology.

Topics Indicators
BLOCK A. MINIMUM CRITERIA
AA1.1 Type of organization
AA1 Organization AA1.2 People
AA1.3 Track record
AA2 Activity AA2.1 ActivitiesAA2.2 Impact
BLOCK B. SOCIAL COHESION
BB1.1 Social inclusion index
BB1 Coexistence and social participation BB1.2 Gender parity
BB1.3 Accessibility and change of mobility model
BB2.1 Level of self-management
BB2 Empowerment
BB2.2 Level of transparency
BB3.1 Level of participation in the social fabric of the neighbourhood
BB3 Relationship with the environment BB3.2 Revaluation of public space
BB3.3 Protection of the existing sociocultural fabric
BB4.1 Participation in the SSE (social and solidarity economy)
BB4 Employment and social economy
BB4.2 Quality job creation
BLOCK C. METABOLIC EFFICIENCY
BC1.1 Reduction of material consumption
BC1 Materials BC1.2 Circularity of materials
BC1.3 Low material impact
BC2.1 Energy efficiency effort
BC2 Energy
BC2.2 Level of energy self-sufficiency through renewable energies
BC3.1 Reduction of water consumption
BC3 Water
BC3.2 Circularity of water
BC4.1 Reducing the carbon footprint of mobility
BC4 Air quality
BC4.2 Creation of urban green areas
BC5.1 Waste reduction
BC5 Waste
BC5.2 Waste management
BLOCK D. TRANSFORMATION POTENTIAL
BD1.1 Encouraging behavioural change
BD1 Social innovation BD1.2 Promotion of new policies
BD1.3 Holistic view of sustainability
BD2.1 Transformative learning
BD2 Creating debate in society BD2.2 Dissemination and replicability
BD2.3 Collaborative networking
BD3.1 Consistent objectives and results
BD3.2 Support network
BD3 Robustness of the initiative
BD3.3 Financial sustainability
BD3.4 Continuity of the initiative
BD4.1 Dynamism of public space
BD4 Activation of public space
BD4.2 Creation of new facilities

Overall, this allows users to better understand and manage the data derived from this
evaluation system.

4.3. Rating System

A catalogue of 119 questions (79 basic and 40 supplementary questions) was estab-
lished for the three blocks B, C, and D, allowing scores between 0 and 100 points (basic)
Sustainability 2023, 15, 7956 10 of 28

and between 0–50 points (supplementary). The questions are divided into basic questions
and supplementary questions, with different ratings and weighting systems.
The basic questions are distinguished by the objectivity with which their answers can
be interpreted and assessed in terms of their contribution to a greater degree of sustainability
of the neighbourhood and its social fabric.
The supplementary questions are considered to be of great interest for the evaluation
of initiatives, but the objectivity with which they can be evaluated is conditioned by the
type of initiative. Therefore, for the final score, a weighting of this score (0–50 points) with
a factor of 0.4 is proposed, which allows for an additional score of 20 points in the final
calculation, totalling a maximum of 120 points (100 points for basic questions and 20 points
for supplementary questions).

4.4. Evaluation Workflow

The evaluation workflow is structured in a series of steps that require the intervention
of a key actor of the initiative or a key actor of the FeliZiudad project. Figure 3 details the
steps to carry out this evaluation. In the data entry process (create user profile and fill in the
minimum criteria), a person in charge of the initiative will fill in the data, which will take
2 min and 5 min, respectively; if the initiative complies with the minimum criteria detailed
in Section 4.1.1 “Block A. Minimum criteria”, the person in charge of the initiative will be
able to fill in the questionnaire corresponding to blocks B, C and D, which will take 2 h (this
time corresponds to the questionnaire of 119 questions); from this moment on, a whole
validation process will be carried out by an external evaluator who will check the veracity
of the information entered in the questionnaires through the documentation provided by
the initiatives.

Figure 3. Flowchart of the proposed evaluation process. Prepared by the authors.

5. Evaluation of CIs in Spain

Figure 4 shows the flowchart of the system evaluation process with case studies, which
lasted approximately one year, distributed as follows: 1.5 months in the evaluation of the

ff tt
Sustainability 2023, 15, 7956 11 of 28

initial five initiatives (this includes the collection of information on the initiative); 2 months
for the simplification process detailed in Section 6.2; 4 months in the evaluation of 25 CIs,
as in the evaluation of the five CIs, this contemplates the time for information collection
and, finally, the data analysis and discussion which merited 4.5 months.

Figure 4. Flowchart of the case studies. Prepared by the authors.

The data collection process was carried out by analysing and searching for information
through digital media (the initiative’s website, social networks, publications on different
pages, etc.). An attempt was made to gather reliable information that could answer all the
questions evaluated.
For the evaluation of citizen initiatives, teams of three people (FeliZiudad researchers)
were formed to evaluate the same initiative, which made it possible to compare the results
of each of them and calculate an average.

5.1. Evaluation of 5 Initiatives

Five CIs were selected from those previously registered on the FeliZiudad website
(project page). The selection was made looking for the greatest diversity of these with
respect to their activity, size, area of influence, and age. This evaluation was carried out by
members of the FeliZiudad team on the basis of the information available on the Internet,
because we wanted to have this first approximation and check the feasibility and suitability
of the system.
The selected CIs focus on (1) sustainable mobility, (2) neighbourhood improvement,
(3) waste management, (4) energy management, and (5) cohousing (this classification was
made by the authors according to the main activity carried out by the initiative; the names
are not mentioned by copyright).
This evaluation was carried out using the 119-question questionnaire, with a maximum
score of 120 points.

5.1.1. Sustainable Mobility Initiative

This CI was foundedff in 1987 by a group of urban and touring cyclists who sought to
promote the use of bicycles as a means of transport in Barcelona.
It was constituted as a people-centred working cooperative with a social and envi-
ronmental commitment in the area of sustainable mobility, promoting the bicycle as an
economical, ecological, and healthy means of transport. Its activities are centred around the
use of bicycles, offering training courses in cycling, bicycle repair, safety, physical spaces
ff ff
Sustainability 2023, 15, 7956 12 of 28

for the self-repair of bicycles, excursions, and rides, as well as strengthening the social and
associative fabric of the neighbourhood (Figure 5).
The CI has established a working network in collaboration with the social fabric of
the neighbourhood, the local productive fabric, educational institutions (schools, institutes,
educational centres, universities, etc.), public administrations, companies, etc. [43,44].
Figure 5 represents the different positive impact of the initiative in different categories.
The central grey circle represents the evaluated initiative, from which four coloured circles
correspond to the general benefits of the initiatives, and, finally, the more specific benefits
are derived. This explanation of the configuration of this figure is the same for Figures 6–9.

Figure 5. Multidimensional benefit of the sustainable mobility CI. Prepared by the authors.

5.1.2. Neighbourhood Improvement Initiative

This initiative was born out of the neighbourhood’s struggle to obtain public facilities,
such as a school, a health centre, a park, and a library, for a neighbourhood that lacked
these spaces, but where there were unused spaces that could serve this purpose and that
were rooted in the neighbourhood and in the city’s architecture. After years of waiting
Sustainability 2023, 15, 7956 13 of 28

and the delay in the execution of the General Metropolitan Plan of Barcelona (MPGM) of
1975, which stipulated that this area would be used for facilities, social housing, and green
zones [45], neighbours organised themselves and set a deadline for the start of the works,
1 June 2011, otherwise they would have to occupy the site in a movement they called “ticking
clock”, which began on 24 January 2010. Before the date was reached, the neighbours were
called to a dialogue table that ended in the cession of one of the spaces [46–48].
Since then, the initiative has been working to create spaces and facilities for social
housing, parks, green areas, and urban allotments, promoting cooperatives that strengthen
social cohesion, empower the social fabric of the neighbourhood, and create a solid and
common neighbourhood network. It is a completely open organisation, which is consti-
tuted as an intergenerational and heterogeneous meeting space and common facility [32,33].
This self-managed initiative has given back to the neighbourhood an identity, influenc-
ing essential aspects such as social welfare, the environment, neighbourhood and urban
development, and creating new management models (Figure 6) [49,50].

Figure 6. Multidimensional benefit of the neighbourhood improvement CI. Prepared by the authors.
Sustainability 2023, 15, 7956 14 of 28

5.1.3. Waste Management Initiative

This initiative was created in 2015 by four women at risk of exclusion, as a result of a
training and social work process initiated by the organisation “Architects without borders”.
Their work is based on the management of used household oil (collection, storage, recovery,
and reuse), to be used as an energy resource. This initiative brings social benefits (inclusion
of vulnerable groups and social cohesion), economic benefits (creation of new jobs), and
environmental benefits (reduction of waste and pollutants) (Figure 7) [51].

Figure 7. Multidimensional benefit of the waste management CI. Prepared by the authors.

5.1.4. Energy Management Initiative

This initiative stems from an interest in promoting renewable energies, specifically
solar panels. It seeks to influence the economic and energy management model, providing
environmental, social, and economic benefits (Figure 8).
Sustainability 2023, 15, 7956 15 of 28

Figure 8. Multidimensional benefit of the energy management CI. Prepared by the authors.

5.1.5. Cohousing Initiative

This initiative arose from a neighbourhood movement to recover an area of the neigh-
bourhood for collective use. Its first steps date back to 2012, although it was formally
established in 2014 when a group of people of different ages and disciplines sought solu-
ff
tions to solve the housing problem, although they encountered the vicissitudes of finding
decent housing.
They considered the idea of creating a housing cooperative, for which they investigated
different models around the world. Once they had decided to do so, they began the process
with the aim of meeting the need for decent housing in the neighbourhood, which in turn
would beff affordable, sustainable, and strengthen community ties through the equitable
sharing of these facilities (Figure 9) [45,52].
ff
Sustainability 2023, 15, 7956 16 of 28

Figure 9. Multidimensional benefit of the cohousing CI. Prepared by the authors.

6. Results and Discussion

6.1. Results of the Evaluation of 5 Initiatives
Table 2 and Figure 10 present the results of the evaluation of each of the selected initia-
tives obtained after applying the procedures and criteria described in the previous sections.
Sustainability 2023, 15, 7956 17 of 28

Table 2. Results of the evaluation of 5 CIs.

Blocks

Block B Block C Block D

Basic Suppl. Basic Suppl. Basic Suppl. Final
CIs Results Results Results
score score (0.4) score score (0.4) score score (0.4) Score
Sustainable Medium
19.0 10.5 23.2 12.5 7.0 15.3 23.5 13.5 28.9 67.4
mobility contribution
Neighbourhood Medium
15.0 7.0 17.8 10.5 5.0 12.5 19.0 11.0 23.4 53.7
improvement contribution
Medium-low
Waste reduction 12.5 4.5 14.3 5.0 1.5 5.6 19.5 10.5 23.7 43.6
contribution
Energy Low
4.0 4.5 5.8 7.0 2.5 8 9.5 6.5 12.1 25.9
management contribution
Medium
Cohousing 14.5 11.0 18.9 19.0 6.0 21.4 22 11.0 26.4 66.7
contribution

RESULT OF EVALUATION
Final score Block D Block C Block B

66.7
26.4
Cohousing 21.4
18.9

25.9
12.1
Energy management 8
5.8

43.60
23.70
CIs

Waste reduction 5.60

14.30

53.7
23.4
Neighbourhood improvement 12.5
17.8

67.4
28.9
Sustainable mobility 15.3
23.2

0 20 40 60 80 100 120
Score

Figure 10. Results of the first evaluation. Prepared by the authors.

In order to assess the level of contribution of the initiatives, the following ranking was
established:
• <24.9 points: CIs with low contribution.
• 25–49.9 points: CIs with medium–low contribution.
• 50–79.9 points: CIs with medium contribution.
• 80–99.9 points: CIs with medium–high contribution.
• >100 points: CIs with high contribution.
The results of this first assessment round show that according to the established
ranking categories, three of the five initiatives have reached a medium contribution (scoring
Sustainability 2023, 15, 7956 18 of 28

67.4 points, 53.7 points, and 66.7 points, respectively). One of the initiatives is rated with
43.5 points, considered as a medium–low contribution, and therefore some improvements
would be recommended.
One of the initiatives is rated with 25.9 points, considered as an initiative with a low
contribution and therefore would require many improvements for considering ff it as a CI
with a high contribution to the sustainability of its neighbourhood.
The average scoring of all five CIs was 51.46, so if we evaluated the initiatives as a
whole, they would pass the system being rated as an overall medium tt contribution. From
this, we could draw the conclusion that if there are several initiatives working on different
sustainability issues in a neighbourhood, their average score could be an indicator in order
to understand their overall impact. Alliances and cooperation networks would probably
allow them to strengthen and diversify their activities and possibly achieve better results.
ffi
6.2. Simplification of the System
One of the aspects that was identified in the evaluation using the 119-question cata-
logue is the fact that it required an execution period of
ffi2 h, with more difficulties in block
B, due to a greater degree of subjectivity in this block with respect to the others. All this
added to the time needed to search for information on the initiative, which depended
on how accessible the information was, as well as the degree of sufficiency and validity
of this information to carry out the evaluation. In the case of information, these disad-
vantages would be eliminated when the process is outsourced, as the person completing
the evaluation would be someone who knows or is familiar with the initiative (manager,
coordinator, etc.). On the other hand, in terms of time, we decided to speed up the process,
reducing the catalogue, in order to guarantee a more user-friendly evaluation. In this sense,
some guidelines were established for this reduction:
• Maintain the same number of indicators.
• The questions should be clear and coherent.
• The contribution of an initiative to the improvement of sustainability and quality of
life in the neighbourhood can be deduced from them.
• There remains a balance between the blocks and indicators:
• The total number of questions remains between 30 and 35 questions.
• Finally, the information can be verified via the web (at that time we could not contact
the initiatives directly due to data protection).
At the end of the simplification process, 33 questions were retained, which corresponds
to the number of indicators in the system. The score for each question ranges from 0 to 1,
so the maximum score to be obtained is 33 points (Figure 11).

Figure 11. Outcome of the simplification process. Prepared by the authors.

Sustainability 2023, 15, 7956 19 of 28

Once the simplification of the system was completed, a verification of the results ob-
tained in the previous evaluation with 119 questions and the new simplified questionnaire
of 33 questions was carried out. For this verification, one of the previously evaluated
initiatives, called “Sustainable Mobility”, was selected. The objective of the comparison
was to determine whether there were significant differences between the results obtained
with the new set of questions and those obtained previously. The selection of this initiative
was based on the availability of detailed information on its website, as well as on news and
other third-party sites.
Table 3 shows the results obtained, in which it can be seen that in the section on the
blocks a similar trend is maintained in the distribution of points, with block D having the
best score, followed by block B, and finally block C. In relation to the results in general, we
can see that in the evaluation with the simplified questionnaire the values are increasing.
This is due to the fact that in the questionnaire with 119 questions, there are some that
generate inconsistent answers and that there is a greater degree of ambiguity, which affects
the result obtained, while in the simplified questionnaire, we tried to address each indicator
with clearer and more concise questions.

Table 3. Comparison between the evaluation system with 119 questions and the simplified system.

Sustainable Mobility CI
Questionnaire of 119 Questions Simplified Questionnaire
Supplementary score
Blocks Basic scores Results Percentages Results Percentages Differences
Factor 0.4
Block B 19.0 10.5 4.2 23.2 23.20% 8.5 25.76% 2.56%
Block C 12.5 7.0 2.8 15.3 15.30% 5.23 15.85% 0.55%
Block D 23.5 13.5 5.4 28.9 28.90% 10.75 32.58% 3.68%

6.3. Evaluation of the 25 Initiatives

Following the same procedure as in Section 5.1, 25 initiatives were selected from the
FeliZiudad project website. Initiatives from different Spanish cities were included.
As in the previous process, this evaluation was carried out by three members of the
team and using exclusively information available in the digital media, due to the pandemic
situation in that moment and data protection issues that did not allow contacting the
initiatives directly. For this evaluation, the simplified questionnaire of 33 questions, with a
maximum score of 33 points, was applied.
Figure 12 details the results of the initiatives evaluated in Bilbao, Zaragoza, Valladolid,
Barcelona, and Madrid.
In the city of Zaragoza, five CIs in the areas of animal care, art and culture, food and
well-being, social model, and food sovereignty were evaluated. The initiative that achieved
the highest score was the social model, which is a social market present in several locations
in Zaragoza, with a strong social component and solid results, encouraging changes in
habits and community support. It was followed by the food and well initiative, which is
an urban organic garden with a marked impact on the social fabric and which promotes
food sovereignty. With respect to the blocks, all CIs showed a good performance in block
D, while in block C, the results were the poorest, except in the case of the social model
initiative, which obtained 54.45% of the maximum score.
In the city of Valladolid, three initiatives were evaluated (water management, social
cohesion, and awareness raising and education). The initiative focused on water manage-
ment only obtained around 30% of the total score, as it is focused on a very specific action.
In the case of the initiative focused on social cohesion, it achieved a good score in block D,
as it is seen as an initiative with some robustness (funding sources, support networks, clear
objectives, and continuity plan), and, finally, the initiative focused on awareness raising
and education also achieved a good score in block D, but in block C, it obtained a score of 0,
because they do not work and have no impact on aspects of metabolic flows efficiency.
Sustainability 2023, 15, 7956 20 of 28

RESULT OF EVALUATION
Final score Block D Block C Block B

14.8
Food sovereignty 9.25
0.50
5.0
25.3
Social model 10.75
6
8.5
ZARAGOZA

20.0
Food and well-being 9.5
3.5
7
15.8
Art and culture 9.75
0
5.75
8.3
Animal care 5.8
0
2.5
13.5
Awareness raising and Education 8.83
0.0
4.67
VALLADOLID

14.5
Social cohesion 8.33
2.0
4.2
CIs

9.8
Water management 5
1.0
3.8
24.5
Sustainable mobility 10.75
5.25
8.50
BARCELONA

25.0
Neighbourhood improvement 11
5.25
8.75
16.0
Awareness raising and Education 10
0.67
5.33
18.8
Sustainable mobility 10
3.0
5.8
MADRID

15.0
Neighbourhood improvement 8
0.5
6.5
15.0
Sustainable mobility 7.25
2.5
5.25

0.00 3.00 6.00 9.00 12.00 15.00 18.00 21.00 24.00 27.00 30.00 33.00
Score

Figure 12. Results of the evaluation of CIs in Zaragoza, Valladolid, Barcelona, and Madrid.

In the case of Barcelona, three initiatives were evaluated, focused on awareness raising
and education, neighbourhood improvement, and sustainable mobility. In general terms, we
observed that, although these initiatives address different issues, their incidence in block C
Sustainability 2023, 15, 7956 21 of 28

always remained below the other blocks., In contrast, in the case of block D, a good trend was
maintained, practically approaching the maximum score possible, which shows that the CIs
evaluated are robust and have the support of the neighbourhood and other entities.
In Madrid, three CIs were evaluated, two in the field of sustainable mobility and one
focused on neighbourhood improvement. Two of them (second and third in Figure 12)
obtained 15 points out of 33. The sustainable mobility initiative had more impact in block C,
since its activities focus on improving environmental quality and boosting new mobility
models. The second obtained a higher score in block B, as its activities have a direct impact
on the cohesion of the neighbourhood.
Another aspect to highlight is that although the first and the third initiative work in
the same area, the latter obtained a higher score since it has implemented strategies that
seek to influence not only mobility but also other aspects, such as reduction of material
consumption, waste reduction, dissemination, continuity of the initiative, and financial
sustainability. Finally, the three initiatives performed well in block D, since they actively
promote habit change, have the support of the community and other institutions, have a
good level of transparency, and have strategies for their continuity.
Figure 13 details the results of the initiatives evaluated in Zamora, León, Palencia,
Badajoz, Valencia, Córdoba, A Coruña, Canary Islands, Sevilla, Gijón, and Bilbao.
In the cases of Palencia (energy management CI), León (art and culture CI), and
Zamora (art and culture) CI), these initiatives obtained low scores, for two general reasons,
because they have an almost null incidence in block C, and in the rest of the blocks their
incidence is low; this is due to the fact that these initiatives are one-off projects, which,
although they have had favourable results, have not remained active to increase these
benefits and have a greater impact on the indicators under study.
In Valencia and Badajoz, only one was evaluated for each city, in the area of the food
and wellbeing and social model. The Valencia initiative, which is a vegetable garden,
achieved a good score, in contrast to the Badajoz initiative, which is a soup kitchen. These
differences may be due to the fact that although the soup kitchen is a solution that provides
support to vulnerable groups, it does not have an impact on a change of habits and
awareness, where groups are empowered and change their way of conceiving food, in
addition to the fact that the vegetable gardens are a solution for renaturalising urban space.
If we look at the score obtained by the Zaragoza garden and compare it with the Valencia
garden, we realise that it is relatively the same, which gives us a clue that a garden is an
interesting proposal to promote the sustainability of neighbourhoods.
In A Coruña and Cordoba, a waste management initiative and another focused on
awareness-raising and education, respectively, were analysed. The two cases obtained
almost the same total score and maintained similar scores in each of the blocks. It is
interesting how waste management initiatives can have a greater impact on the social
domain than CIs focused on raising awareness and educating. In this regard, the inclusion
of groups at risk of exclusion and the participation in the social and solidarity economy
were aspects that scored high in the evaluation tool.
One CI in Seville and one in the Canary Islands were evaluated. The CI in the Canary
Islands has a broad scope as it works in several cities in the archipelago (La Palma, Tenerife,
Gran Canaria, Fuerteventura, and Lanzarote), which focuses on blurring art and culture
in the cities where it is present. In the case of Seville, the CI analysed focuses on the
renaturalisation of spaces, creating a new leisure and wellbeing area. Block C was in the
minimum ranges in both initiatives. This is due to the fact that in the case of Seville and the
Canary Islands, these initiatives evaluated are very specific and do not affect aspects such
as waste, water, materials, and energy management.
In the cases of Bilbao and Gijón, only one single initiative was assessed for each city,
but with different areas of action: on the one hand, health and healthy living and, on the
other hand, sustainable mobility, successively. Both initiatives obtained a higher score in
block D, reaching a final score of 16 and 17.4, successively. In the case of block B, the Gijón
initiative had a lower score than that of Bilbao, which may be due to the fact that the latter
Sustainability 2023, 15, 7956 22 of 28

is more rooted in public space and community life, with a greater impact on the social
cohesion of the neighbourhood.

RESULT OF EVALUATION
Final score Block D Block C Block B
ZAMORA

7.3
Art and culture 4.25
0
3

5.8
LEÓN

Art and culture 3.75

0
2
ISLANDS A CORUÑA CÓRDOBA VALENCIA BADAJOZ PALENCIA

4.5
Energy management 2.00
2
0.50

12.0
Social model 8.50
0
3.50

20.3
Food and well-being 9
4.0
7.25

19.0
Awareness raising and Education 10.25
3
CIs

5.75

18.9
Waste management 8.25
3.8
6.8
CANARY

16.3
Art and culture 8.83
0.7
6.8

12.0
SEVILLA

Renaturalisation of spaces 8.50

0.00
3.50

17.4
GIJÓN

Sustainable mobility 9.75

3.1
4.50

16.0
BILBAO

Health and healthy living 8

0.50
7.5

0.0 3.0 6.0 9.0 12.0 15.0 18.0 21.0 24.0 27.0 30.0 33.0
Score

Figure 13. Results of the evaluation of CIs in Zamora, León, Palencia, Badajoz, Valencia, Córdoba, A
Coruña, Canary Islands, Sevilla, Gijón, and Bilbao. All of them are cities with smaller populations
than the ones presented in the previous table.
Sustainability 2023, 15, 7956 23 of 28

6.4. Performance of CIs by Blocks and by Typology

According to Figure 14, it is noticeable that the block that reported the highest degree
of compliance is block D (potential for transformation), as most of these CIs have been
well welcomed in their cities and have received the support of the community, the city
council, and other initiatives or related organisations. Most of the CIs have an active
role in creating the basis for a change in thinking and reflection on sustainability, either
from a specific area or from a set of actions dispersed among the areas that arise from the
concept of sustainability (social, environmental, and economic). The second block in which
most of the CIs perform well is block B (social cohesion). CIs, being the result of citizen
self-management, understand the need to create neighbourhood collaboration networks
and create transparent management mechanisms. In contrast, in the case of block C (urban
metabolism), the initiatives do not tend to cover all aspects (water, energy, materials, and
waste management), but, rather, they tend to focus on specific aspects with small actions.

Figure 14. General summary of the evaluated CIs. Prepared by the authors.

On the other hand, with regard to the typologies of initiatives, it can be inferred from
the analysis that those with the best performance (20–33 p.) are those initiatives that focus
on issues such as sustainable mobility, neighbourhood improvement, social model, food and
wellbeing; those with intermediate impact (10–199 p.) initiatives focusing on food sovereignty,
awareness raising and education, waste management, social cohesion, naturalisation of spaces,
and health and healthy living; and, finally, low impact (0–9.9 p.) initiatives focusing on water
management, animal care, energy management, and art and culture.
From a scientific perspective, it is important to highlight that the analysis of the
effectiveness of CIs should be approached from a holistic viewpoint that considers multiple
factors that may influence their overall performance. In this sense, the previous paragraph
identified the impact according to the typologies of CIs evaluated, but it is necessary to
point out that the evaluation of their success also depends on aspects such as:
Sustainability 2023, 15, 7956 24 of 28

• The consistency of their objectives with the real needs of the community.
• The support received from the community, other organisations, and the administration.
• The public policies of the specific city.
• The spatial and beneficiary coverage.
• The versatility of its activities.
• Its trajectory.
• The degree of dissemination.
• The level of awareness and empowerment of citizens.
• The level of the city’s relationship with other contexts (cities) that can serve as examples
or promoters of the same.
• Socioeconomic condition of the population.
• Depopulation.
• The ageing of the population.
Other aspects not mentioned in this article may be conditioning factors that hinder the
development of the CIs but must be analysed in depth.

6.5. Evaluation of Activities vs. Evaluation of Organization

The developed evaluation system pretends to adapt to the diversity of initiatives and
assess the different areas of sustainability they contribute to. Being a holistic assessment,
the tool favours initiatives that have diversified actions and a broad range of impact on
a neighbourhood. On the other hand, it evaluates aspects not only directly related to
the activities but also, and above all, to the organisation, so that in reality what we are
evaluating is a cultural change in the way things are performed: inclusive, taking into
account the resources used and the waste generated, egalitarian, democratic, etc.

6.6. Full Questionnaire vs. Reduced Questionnaire

With regard to the system, several aspects could be ascertained from the study carried
out with the general questionnaire with 119 questions. On the one hand, the tool provides
us with valuable information both on the organisational level and on the activities carried
out, which have a direct or indirect impact on the achievement of the objectives and aspects
of sustainability that we have set out for the system. On the other hand, with regard to the
time associated with its application, a long time is required for the search for information
and the evaluation by the evaluators. A key aspect in this process is to reduce subjectivity
on the part of the evaluators as much as possible, trying to work with the same information,
and checking for elevated divergence in the different evaluations.
Once the system was simplified and a larger number of initiatives spread throughout
Spain and of different natures were selected, the process was more agile, although more
detailed information was lost regarding specific aspects of the activities they carry out,
specific quantified outcomes, specific strategies for waste, water, energy management,
aspects of language, etc.

6.7. Single Topic CIs vs. Diverse Topics CIs

Regarding the topic coverage and the evaluation results obtained, the following can
be stated:
The evaluation tool, by its nature, tends to value initiatives that act in a diversified
way, while initiatives that are very good and specific to a single topic (topics (energy
management, waste reduction, etc.) have a very low evaluation.
On the other hand, more diverse CIs (neighbourhood improvement, sustainable
mobility, social model, and food and wellbeing) tend to score higher, as the positive impact
is usually distributed among the three blocks of the assessment tool in an ascending manner.
Other aspects that can enhance these impacts of more diverse CIs even more are the support
received from local administrations (local sessions, grants, etc.) and partnerships with
other social entities, which create a range of new possibilities and a more comprehensive
and cross-cutting approach to problems.
Sustainability 2023, 15, 7956 25 of 28

6.8. Potential Applications of the Tool

The evaluation model proposed for the CIs can be understood as an evaluation tool
and, at the same time, as a continuous assessment tool for learning and improvement over
time for the promoters of initiatives.
Detailed results of the evaluation can serve as inspiration for the continuous im-
provement of social, environmental, and energy-sustainability initiatives, both at the or-
ganisational level of the initiatives and at the level of their activities. For this reason, a
questionnaire in digital format is proposed in which best practices and areas for improve-
ment can be reported.
Given that this evaluation can be renewed periodically, the improvements can be
incorporated and even serve as an incentive or condition to obtain the support of the
administration or other interested parties, as well as to assess and disseminate the positive
impacts of initiatives to the citizens.

7. Conclusions
The path towards sustainability is closely related to new institutional and political
paradigms and emerging models of practices, technologies, lifestyles, attitudes, and val-
ues [53]. In this sense, the CIs, collectives where citizens self-organise through collaborative
practices, networking, and knowledge transfer, come into their own, creating spaces for
coexistence, reflection, and meeting where social innovation is the backbone. These multi-
disciplinary collaborative practices are developed on the basis of active citizen participation,
improving individual and community life in the neighbourhood.
The main objective of the CIs is to tackle neighbourhood problems through self-
management and neighbourhood empowerment, adopting more sustainable lifestyles [54].
These citizen models are transformed into solid platforms based on bottom-up processes
where the social fabric is dynamised and, on many occasions, neighbourhood dynamics are
transformed. Although this impact is known by the beneficiaries, it remains diffuse within
the context of the city and even in some cases for the social fabric of the neighbourhood
where the initiative is located.
These social dynamics require monitoring, reflection, and analysis of their activities
and organisational process and the creation of political strategies, in order to understand
and take advantage of the potential of the CIs to generate improvements in their environ-
ment, in the quality of life of citizens, and, therefore, in the sustainability of the community.
This is where it makes sense to make these experiences visible and quantify their Impact
through systemic evaluations that allow confirming their diverse positive effects on the
sustainable development of communities.
In order to establish an evaluation framework, it is necessary to have a holistic vision
of all the dimensions addressed to a greater or lesser extent by the CIs and to build an inter-
disciplinary, integral, and collective knowledge that helps to understand these processes
and, at the same time, promote them. In this process, academia, public administration,
and civil society are called upon to participate as promoters and final beneficiaries. To
this end, channels of interaction must be opened between civil society and the admin-
istration, with the support of experts who can mediate and channel these learning and
decision-making processes.
A simple, robust, and inclusive evaluation model for CIs, on the one hand, allows for a
better understanding of their holistic contribution to solving neighbourhood issues and pro-
moting healthy lifestyles, and, on the other hand, enables benchmarking between different
initiatives and fosters their continuous improvement. Undoubtedly, this evaluation system
should be inclusive, highlighting the role of the evaluated initiatives as good practices.
The presented evaluation system was developed based on the in-depth analysis
of existing evaluation systems, with special reference to three international proposals.
Moreover, the system was designed within the framework of a collaboration agreement in
Spain, focusing on the evaluation of Spanish CIs. However, although this proposal takes
into account European references in its development, it is intended to be applicable to all
Sustainability 2023, 15, 7956 26 of 28

types of CIs. Indicators with their universality, in most cases, can allow quantifying this
aggregate contribution to the overall sustainability of a neighbourhood, but it is interesting
to evaluate the sociopolitical context of the environment where the initiative is developed
as an input to the process of identifying barriers and opportunities that can promote this
social innovation [55]. For example, it may happen that in some countries, public policies
are not oriented to promote and support this type of participation or that citizens are not
aware or do not have the means to implement them. Another aspect to consider is the
adaptation of the catalogue of questions depending on the reality of the initiative and its
environment, i.e., it does not make sense to ask about waste management in a country
where this is very well solved, but rather to focus on those sensitive aspects of the city.
In a next step, the presented system needs to be implemented and validated in different
contexts, creating a framework for dialogue with the different agents involved and experts
in the different areas addressed in the CIs. This will allow the generation of an even more
balanced tool that encourages open and plural citizen processes, highlighting the activities
it carries out as good practices, since the contribution of one initiative cannot be assessed as
better than another one without taking into account a holistic analysis of the local context,
the social structures, the level of citizen participation, the decision-making structures,
and its potential to create new habits and learning, among many other factors. A series
of optimization processes are currently underway in order to achieve this balance. We
consider this implementation in countries with other socioeconomic, cultural, and political
particularities in Latin America.
To address subjectivity, we involve external agents in the evaluation process, ensuring
an objective and fair evaluation of initiatives. Our aim is to create a simpler and more
compact methodology that facilitates a friendly and comfortable interaction between the
interviewer and the user, and between the platform and the user in digital format.
Finally, continuous feedback is prioritized to ensure the effectiveness and relevance
of the system over time, promoting a culture of sustainability and innovation in urban
development. The authors are convinced that the presented evaluation tool for CIs is a
novel and interesting proposal with the potential to support decision-making processes
for the promotion of CIs based on a holistic analysis of their benefits and the match with
identified needs of the neighbourhood.

Author Contributions: Conceptualization, F.E.B., I.R.C., M.P. and T.M.; Methodology, F.E.B., M.P.
and T.M.; Validation, F.E.B., T.M., I.R.C. and M.P.; Investigation, F.E.B., T.M. and M.P.; Data curation,
F.E.B., I.R.C., T.M. and M.P.; Writing—original draft, F.E.B.; Writing—review & editing, F.E.B., I.R.C.,
T.M. and M.P.; Visualization, F.E.B.; Project administration, T.M.; Funding acquisition project, T.M.;
Funding acquisition article, F.E.B., I.R.C., T.M. and M.P. All authors have read and agreed to the
published version of the manuscript.
Funding: This research was funded by the European Institute of Innovation and Technology (EIT)
which is supported by the European Union’s Horizon 2020 research and innovation program through
the project “FeliZiudad” (E-01498). This article was funded by AGAUR (2021 SGR 01405–research
group GICITED). F.E.B. has received the support of the FI AGAUR 2019 PhD grant funded by the
Secretaria d’Universitats i Recerca de la Generalitat de Catalunya and the European Social Fund.
Institutional Review Board Statement: Not applicable.
Informed Consent Statement: Not applicable.
Data Availability Statement: The data are contained within the article. Additional data are available
upon request from the corresponding author.
Acknowledgments: This research work is the result of a collaboration agreement between Renault
Spain and Innoenergy SE. We would like to thank the entire UPC and Innoenergy team for the
collaboration and synergies to be part of the project “Development and Piloting of Assessment Model
and Certification Scheme for Sustainable City Neighborhoods and Citizen Initiatives (FeliZiudad)”.
Conflicts of Interest: The authors declare no conflict of interest.
Sustainability 2023, 15, 7956 27 of 28

References
1. Gómez, M.V.; Álvarez, J. El Cambio Social en la Era de la Incertidumbre. Una Reflexión Sobre Teoría Social; Talasa Ediciones: Madrid,
Spain, 2013; ISBN 978-84-96266-44-5.
2. Wu, J.; Li, W.; Xu, W.; Yuan, L. Measuring Resident Participation in the Renewal of Older Residential Communities in China
under Policy Change. Sustainability 2023, 15, 2751. [CrossRef]
3. Bouw, K.; Wiekens, C.J.; Tigchelaar, C.; Faaij, A. Involving Citizens in Heat Planning: A Participatory Process Design for Informed
Decision-Making. Sustainability 2023, 15, 1937. [CrossRef]
4. de Freitas, A.C.; do Nascimento, L.A.; de Castro, R.G.; Motokane, M.T.; Reis, P. Biodiversity and Citizenship in an Argumentative
Socioscientific Process. Sustainability 2023, 15, 2987. [CrossRef]
5. Navickien, O.; Meidutė-Kavaliauskienė, I.; Činčikaitė, R.; Morkūnas, M.; Valackienė, A. Modernisation of a Country in the
Context of Social Environmental Sustainability: Example of Lithuania. Sustainability 2023, 15, 3689. [CrossRef]
6. Yamin, P.; Artavia-mora, L.; Martunaite, B.; Lahiri, S. Installations for Civic Culture: Behavioral Policy Interventions to Promote
Social Sustainability. Sustainability 2023, 15, 3825. [CrossRef]
7. Pinto, H.; Nogueira, C.; Guerreiro, J.A.; Sampaio, F. Social Innovation and the Role of the State: Learning from the Portuguese
Experience on Multi-Level Interactions. World 2021, 2, 62–80. [CrossRef]
8. Semanjski, I.; Gautama, S. A Collaborative Stakeholder Decision-Making Approach for Sustainable Urban Logistics. Sustainability
2019, 11, 234. [CrossRef]
9. Almeida, F. The Contribution of Local Agents and Citizens to Sustainable Development: The Portuguese Experience. Sustainability
2022, 14, 12696. [CrossRef]
10. MacRae, R.; Szabo, M.; Anderson, K.; Louden, F.; Trillo, S. Empowering the Citizen-Consumer: Re-Regulating Consumer Information
to Support the Transition to Sustainable and Health Promoting Food Systems in Canada. Sustainability 2012, 4, 2146–2175. [CrossRef]
11. Bouwma, I.; Wigboldus, S.; Potters, J.; Selnes, T.; van Rooij, S.; Westerink, J. Sustainability Transitions and the Contribution
of Living Labs: A Framework to Assess Collective Capabilities and Contextual Performance. Sustainability 2022, 14, 15628.
[CrossRef]
12. Kesselring, S.; Simon-philipp, C.; Bansen, J.; Hefner, B.; Minnich, L.; Schreiber, J. Sustainable Mobilities in the Neighborhood:
Methodological Innovation for Social Change. Sustainability 2023, 15, 3583. [CrossRef]
13. Lepore, D.; Testi, N.; Pasher, E. Building Inclusive Smart Cities through Innovation Intermediaries. Sustainability 2023, 15, 4024.
[CrossRef]
14. Acosta Pérez, E.; Gil-Fournier Esquerra, M.; Jaenicke Fontao, M. Vivero de iniciativas ciudadanas: Transferencia de la innovación
ciudadana al espacio público. Educ. Soc. Rev. Interv. Socioeduc. 2013, 55, 94–102, ISSN 1135-8629.
15. Vivero de Iniciativas Ciudadanas. Pero ¿Qué es una Iniciativa Ciudadana? 2012. Available online: http://viveroiniciativasciudadanas.
net/2012/11/21/pero-que-es-una-iniciativa-ciudadana/ (accessed on 12 January 2020).
16. Cámara Menoyo, C. Las iniciativas de participación ciudadana en el urbanismo. El urbanismo participativo, una nueva forma de
entender la ciudad y la ciudadanía en la configuración de espacios públicos. URBS Rev. Estud. Urbanos Cienc. Soc. 2012, 2, 19–32,
ISSN 2014-2714.
17. Celata, F.; Sanna, V.S. A multi-dimensional assessment of the environmental and socioeconomic performance of community-based
sustainability initiatives in Europe. Rev. Reg. Environ. Chang. 2019, 19, 939–952. [CrossRef]
18. Cespedes Avalos, M. Evaluación y Participaci ón Ciudadana. Aportes Para una Estrategia de Integración, San José, Costa Rica.
2017. Available online: http://foceval.org/wp-content/uploads/2016/12/2017_Evaluaci{\protect\edefT1{T5}\let\enc@update\
relaxó}n-y-participaci{\protect\edefT1{T5}\let\enc@update\relaxó}n-ciudadana.pdf (accessed on 25 May 2019).
19. Marradi, C.; Mulder, I. Scaling Local Bottom-Up Innovations through Value Co-Creation. Sustainability 2022, 14, 11678. [CrossRef]
20. Kropp, J.P.; Dinnie, E.; Anguelovski, I.; Tikkanen, J. Towards European Sustainable Societies (TESS): Final Publishable Summary Report
of the European FP7 Project; Potsdam Institute for Climate Impact Research: Potsdam, Germany, 2016. [CrossRef]
21. InnoEnergy. Desarrollo y Pilotaje del Modelo de Evaluación y Esquema de Certificación para Vecindarios de la Ciudad Sostenible e Iniciativas
Ciudadanas; InnoEnergy: Barcelona, Spain, 2019.
22. Gabriel, Y.; Lang, T. A Brief History of Consumer Activism. In The Ethical Consumer; Harrison, R., Newholm, T., Shaw, D., Eds.;
Sage: London, UK, 2005. [CrossRef]
23. Birchall, J. Building Communities: The Co-Operative Way; Routledge: London, UK, 1998; ISBN 978-13-17-70351-8. [CrossRef]
24. International Labour Organization. World Employment and Social Outlook: Trends; ILO: Geneva, Switzerland, 2018; ISBN 978-92-2-131535-3.
25. Matarrita-Cascante, D.; Brennan, M.A. Conceptualizing community development in the twenty-first century. Community Dev.
2012, 43, 293–305. [CrossRef]
26. Wallerstein, I. El Mundo Está Desencajado. Interpretaciones Histórico-Mundiales de las Continuas Polarizaciones; Siglo XXI: Mexico City,
Mexico, 2016; ISBN 978-607-03-0790-4.
27. Fawcett, S.B.; Schultz, J.A.; Thomas Francisco, V. The community tool box: A web-based resource for building healthier
communities. Public Health Rep. 2000, 115, 274–278. [CrossRef]
28. Comisión Europea. Guía Para La Iniciativa Ciudadana Europea; Oficina de Publicaciones de la Unión Europea: Luxembourg, 2011;
ISBN 978-92-79-21773-9. [CrossRef]
29. Comisión Europea. Política de cohesión 2014–2020. 2014. Available online: http://ec.europa.eu/regional_policy/informing/
index_es.cfm%0APortales (accessed on 15 May 2020).
Sustainability 2023, 15, 7956 28 of 28

30. Comisión Europea, Dirección General de Política Regional y Urbana. Una Introducción a la Política de Cohesión de la UE 2014–2020;
Oficina de Publicaciones: Luxembourg, 2014.
31. Centro de Documentación Europea. Política de Cohesión: Hacer que los Ciudadanos Participen Para Obtener Mejores Resultados.
7 February 2020. Available online: http://cde.ugr.es/index.php/union-europea/noticias-ue/916-politica-de-cohesion-hacer-
que-los-ciudadanos-participen-para-obtener-mejores-resultados (accessed on 10 January 2022).
32. López-Sánchez, M.P.; Alberich, T.; Aviñó, D.; Francés García, F.; Ruiz-Azarola, A.; Villasante, T. Herramientas y métodos
participativos para la acción comunitaria. Informe SESPAS 2018. Gac. Sanit. 2018, 32, 32–40. [CrossRef]
33. Morales Morales, E. Empoderamiento y Transformación de las Relaciones de Poder. Un Análisis Crítico de los Procesos
Institucionales de Participación Ciudadana. Ph.D. Thesis, Universitat Autònoma de Barcelona, Barcelona, Spain, 2016.
34. Gomà, R.; Blanco, I. Gobiernos locales y redes participativas: Retos e innovaciones. In Proceedings of the VII Congreso Internacional del
CLAD sobre la Reforma del Estado y de la Administración Pública, Lisboa, Portugal, 8–11 October 2002; pp. 8–11.
35. Asociación Simbioisis. Alcalá con Iniciativas. 2016. Available online: https://asociacionsimbiosis.org/alcala-con-iniciativas/
(accessed on 13 April 2023).
36. De Jorge-Huertas, V. Collaborative designing of communities. Helsinki and Zurich pioneers. ACE Archit. City Environ. 2020, 15, 1–22.
[CrossRef]
37. CERAI. Sistemas Alimentarios Territorializados en España. 100 Iniciativas Locales Para una Alimentación Responsable y Sostenible.
Available online: https://cerai.org/wordpress/wp-content/uploads/2019/05/publicaci%C3%B3n_100iniciativasSATenEspa%C3
%B1a-CERAI.pdf (accessed on 16 June 2022).
38. Gómez Lucena, Á. Consolidar el tejido social a través de la participación comunitaria en salud. Rev. Salud 2000 2008, 116, 21–22.
39. Guillén Marcos, E. Iniciativas ciudadanas. Experiencias, Revista E-Rph 2010, 7, 126–166.
40. Berigüete Alcántara, F.; Cantalapiedra, I.R.; Palumbo Fernández, M.; Masseck, T. ¿Cómo medir el impacto de las iniciativas
ciudadanas en la sostenibilidad urbana? ACE Archit. City Environ. 2022, 17, 10413. [CrossRef]
41. Wunder, S.; Albrecht, S.; Porsch, L.; Öhler, L. Kriterien zur Bewertung des Transformationspotentials von Nachhaltigkeitsinitiativen;
Umweltbundesamt: Dessau-Roßlau, Germany, 2019.
42. Berigüete, F.E.; Masseck, T.; Palumbo, M.; García, B. Iniciativas ciudadanas y su contribución a la sostenibilidad en las ciudades.
In Proceedings of the III Jornada TAEU 2020, Barcelona, Spain, 26 November 2020; pp. 11–15.
43. Barcelona + Sostenible. Accions per la Sostenibilitat. 2016. Available online: https://www.barcelona.cat/barcelonasostenible/
sites/default/files/documents/plans/1316.pdf (accessed on 16 June 2022).
44. Fitó, M.Á.; Corrons, A. Values of the Social and Solidarity Economy: Biciclot Cooperative Case Study. In New Trends in the Creation
and Management of Social Enterprises; Oliveras, E., Bastida, R., Mas-Machuca, M., Eds.; Economics and Business Studies: Barcelona,
España, 2018.
45. Joanpere Foraster, M.; Morlà Folch, T. Liderazgo creador de nuevas realidades. Respuestas cooperativas a los desafíos de la
vivienda. Rev. Int. Organ. 2018, 21, 127–147. [CrossRef]
46. Droege, F. Variaciones en Procesos Participativos. Estudio de la Participación Ciudadana en la Renovación Urbana del Recinto de
Can Batlló. Master’s Thesis, Universitat Politècnica de Catalunya, Barcelona, Spain, 2021.
47. Graus, R.; Rosell, J. La fàbrica Batlló, una obra influent en l’arquitectura catalana. In Proceedings of the VIII Jornades d’Arqueologia
Industrial de Catalunya (Barcelona i les Grans Fàbriques dels Segles XIX-XX), Barcelona, Spain, 29–31 November 2009.
48. Domínguez Santana, R. Participación en el Urbanismo. La Iniciativa Ciudadana de Can Batlló. Master’s Thesis, Universitat
Autònoma de Barcelona, Barcelona, Spain, 2012.
49. Baiges Campruvi, C. Can Batlló: Cuando la ciudadanía reutiliza el patrimonio industrial. Butlletí D’arqueologia Ind. I Mus. Tècnica
I Ciència 2015, 76, 2–6.
50. Dalmau, I.; Torvà, M. Can Batlló: De la degradación planificada a la construcción comunitaria. Quad. L’institut Catala D’antropologia
2014, 19, 143–159, ISSN 1696-8298.
51. Arza Arza, N.; Cruz López, L.; Serantes-Pazos, A. “Mulleres colleiteiras” e alumnado de Educación Social: Unha experiencia de
aprendizaje e servizo. Ambient. SustenTable 2018, 1, 393–400. [CrossRef]
52. Romero Siscar, M. Cooperativas de Viviendas en Cesion de Uso: La Borda Y Entrepatios. Final Degree Project, Universidad
Politécnica de Madrid, Madrid, Spain, 2020.
53. Brohmann, B.; Grießhammer, R. Wie Transformationen und Gesellschaftliche Innovationen Gelingen Können: Transformationsstrategien
und Models of Change für Nachhaltigen Gesellschaftlichen Wandel, 1st ed.; Nomos Verlagsgesellschaft mbH & Co. KG: Baden-Baden,
Germany, 2015.
54. Gómez, M.V.; Lebrusán, I. Urban Ageing, Gender and the Value of the Local Environment: The Experience of Older Women in a
Central Neighbourhood of Madrid, Spain. Land 2022, 11, 1456. [CrossRef]
55. Warbroek, B.; Hoppe, T.; Coenen, F.; Bressers, H. The Role of Intermediaries in Supporting Local Low-Carbon Energy Initiatives.
Sustainability 2018, 10, 2450. [CrossRef]

Disclaimer/Publisher’s Note: The statements, opinions and data contained in all publications are solely those of the individual
author(s) and contributor(s) and not of MDPI and/or the editor(s). MDPI and/or the editor(s) disclaim responsibility for any injury to
people or property resulting from any ideas, methods, instructions or products referred to in the content.