See discussions, stats, and author profiles for this publication at: https://www.researchgate.

net/publication/259078971

Performance measurement in the nonprofit sector

Conference Paper · January 2013

CITATIONS READS

0 1,838

3 authors:

Ishani Soysa Nihal Palitha Jayamaha

Massey University Massey University
14 PUBLICATIONS   27 CITATIONS    45 PUBLICATIONS   167 CITATIONS   

SEE PROFILE SEE PROFILE

Nigel P. Grigg
Massey University
108 PUBLICATIONS   637 CITATIONS   

SEE PROFILE

Some of the authors of this publication are also working on these related projects:

The Impact of ICT Usage on Collaborative Product Development Performance View project

Developing methods for improved decision making in Quality Function Deployment View project

All content following this page was uploaded by Ishani Soysa on 31 July 2017.

The user has requested enhancement of the downloaded file.

 PERFORMANCE MEASUREMENT IN THE NONPROFIT SECTOR

Ishani B. Soysa, School of Engineering and Advanced Technology, Massey University,

PB11222, Palmerston North, New Zealand, +6463569099 Ext: 2455,
[email protected]
Nihal P. Jayamaha, School of Engineering and Advanced Technology, Massey University,
PB11222, Palmerston North, New Zealand, +6463569099 Ext: 81731,
[email protected]
Nigel P. Grigg, School of Engineering and Advanced Technology, Massey University,
PB11222, Palmerston North, New Zealand, +6463569099 Ext: 81732,
[email protected]

ABSTRACT

Nonprofit management is a unique endeavour, which can be distinguished from for-profit or

government sector management. Measurement of strategic performance is important to any
organisation, irrespective of the sector to which the organisation belongs to. Comparative to
the for-profit sector, less attention has been given to performance measurement in the
nonprofit sector. This paper describes how an empirically validated performance
measurement (PM) model will be developed for the Australian and New Zealand nonprofit
organisations (NPOs) belonging to the “development and housing” category. The current
state of knowledge about PM in the nonprofit sector, how the data collection tool (survey
instrument) will be developed and how the PM model will be derived and tested using factor
analysis methods and structural equation modelling have been covered. The paper also
describes the extent to which PM in NPOs reflect principles of strategic quality management
and compare the difference between PM in NPOs and PM in for-profit organisations.

Key words: Performance measurement (PM); Nonprofit sector; Balanced Scorecard (BSC);
Data Analysis

INTRODUCTION

Theories on organisations and management do change as the world around us (demographics,

technology, social and political forces, legal forces, economic forces etc.) evolves [14] . Quite
a number of management theories and practices that were commonplace 4 or 5 decades ago
no longer hold good today, in most management situations. However, there are a few
fundamental axioms on organisations that have stood the test of time. For example, no matter
what the organisation is, it is still a “collection of people” who aim to achieve a “common
objective” for the survival and growth of the organisation [38]. If an organisation has
objectives—and high (strategic) level objectives —then, there is a logical need to measure
whether or not the organisation is performing the way it should to achieve its strategic
objectives [29]. Therefore performance measurement (PM) is a perennial requirement of any
organisation, for its survival and growth. The term PM can variously apply to people in the

-1-
organisation, the processes of the organisation, and the organisation as a whole [29]. In this
paper, the term PM refers to the organisation as a whole.

Whether an organisation should be managed so as to maximise the shareholder wealth

(shareholder view) [11] or whether an organisation should be managed to jointly maximise
the expectations of all the key stakeholders (stakeholder view) [10], measurement of financial
performance of the organisation remains a key priority for the health of the organisation [19].
For this reason, organisational PM has traditionally revolved around assessing how an
organisation performs on financial measures [26].

Although financial reporting remains an integral part of PM, one must not lose sight of the
fact that financial measures on organisational performance (e.g. return on the capital invested,
profit margin ratio, asset turnover ratio) actually do reflect the outcomes of past actions [19]
[27] [40]. Therefore it is inadvisable to measure strategic performance of the organisation
based on the single dimension “financial performance” [19]. In the literature financial
measures are referred to as “lagging indicators” because they actually do lag behind (changes
after) an economic event or an organisational action [19] [27]. To offset the shortcomings of
using lagging indicators alone, Kaplan and Norton proposed a “Balanced Scorecard” (BSC)
of a business system which looks at the organisational performance from four dimensions
(perspectives): customer, internal processes, innovation and learning, and financial [19]. They
viewed the four dimensions/perspectives as the key to future financial performance.
Furthermore, using the BSC, they argued that it is possible to answer four basic questions
which are very important for the sustainability of an organisation: “how do customers see us?
(Customer perspective), what must we excel at? (Internal perspective), can we continue to
improve and create value? (Innovation), how do we look to shareholders? (Financial)”. The
BSC reduces the load of information that managers have to focus on because the BSC
requires managers to look for those few measures that are actually related to the
organisational strategy [19] [27]. The BSC aligns organisational mission, vision, goals and
strategies to focus on the “financial success” (the “exclusive focus” of strategic performance
measurement) through the four perspectives; however, these four perspectives are more
suitable for for-profit organisations [18] [23]. Having realised that the format of the BSC is
not fully appropriate to nonprofit organisations (NPOs) because of the nature of these
organisations (the nature of NPOs is described later), Kaplan proposed a revised version of
the BSC framework for NPOs [18]; the revised version places the organisational mission on
top (as opposed to the financial perspective in the BSC meant for for-profits) to align the
organisational mission with both the financial perspective (donor expectations) and customer
perspective, which are deemed equally important; these two perspectives are followed by the
internal focus and innovation respectively, in the pecking order [18]. Apart from rearranging
the hierarchical nature of the four perspectives, the revised BSC for NPOs does not provide
insights (descriptive or prescriptive criteria) as to what types of measures in each perspective
are appropriate for nonprofit organisations.

-2-
Defining an NPO, PM challenges and the importance of NPOs

An NPO is a tax-exempt organisation, a group organised for purposes other than generating
profit and in which “no part of the organisation's income is distributed to its members,
directors, or officers” [34].

There are many other terms (e.g. the third sector, voluntary sector, charitable sector) that are
used to refer to the nonprofit sector. However, each of these terms emphasises one aspect of
the NPOs making light of what could be accepted as the reality [34]. For example, the term
‘voluntary’ emphasises the importance of the role being played by the volunteers in the sector
but the reality is that quite a number of paid employees do work in the sector [34].

One of the most exhaustive studies launched by scholars over 20 countries to understand the
structure of NPOs in the United States and elsewhere (Australia and New Zealand included)
is the “Johns Hopkins Comparative Nonprofit Sector Project”. This project, whose country-
specific findings were published through a series of working papers, identified the following
five key features of an NPO, giving regard to the basic structure and operation of an NPO
[34]:
• “Organised”: The term ‘organised’ is used to mean that NPOs are highly structured in
that they are methodical and efficient in arrangement and operations, whether or not
they are legally registered.
• “Private”: The term private is used to mean that the nonprofit sector is not another
arm of the government (although NPOs are meant to provide some form of a public
service) although they may receive funds from governmental sources.
• “Not profit-distributing”: Of all the five characteristics of an NPO, the not profit
distribution feature is arguably the most pervasive characteristic; an NPO should exist
for purposes other than generating profit and in which no part of the organisation's
income is distributed to its members, directors, or officers.
• “Self-governing”: The term self-governing is used to mean that NPOs (e.g. churches,
schools) have their own ways and means to pursue their operations and achieve
targets.
• “Voluntary”: The term voluntary is used to mean that membership and donations of
money or time are not essential, in the sense, not required by law.

The aforesaid “structural-operational definition” has been tested (and validated) in every
country (41 countries) included in the Johns Hopkins project to determine whether it is wider
enough a definition to embrace the different range of organisations considered to be part of
the nonprofit sector and to distinguish these organisations from for-profit and government
organisations.

Measuring the performance of any kind of organisation is not an easy task because all
measurements need to be aimed at achieving the strategic objectives of the organisation [20]
[5]; while the use of strategy maps have found to be useful in PM [21], it becomes even more
difficult to measure performance in NPOs. The complication of measuring performance in
the nonprofit sector has long been recognised [24] [37]. While the for-profit sector

-3-
organisations successfully use proven conventional business models to measure performance
in the areas of strategic planning, marketing, finance, information systems and organisational
development, the NPOs too use the same models and associated measurement areas from the
very outset, in the hope to improve their effectiveness and efficiency [23]. Since for-profit
sector’s success is mainly associated with profit and loss accounts figures and availability of
many quantitative data to measure profits, the same performance approaches do not create the
same results for nonprofits [23] [17].

The major difference between NPOs and for-profit organisations is that the former have goals
and services which are more intangible and harder to define [17] and thus measuring
performance provides a greater challenge for nonprofits [8] [28]. Donors, people receiving
the services (beneficiaries) and workers have different views of nonprofits’ quality of
services. In PM this translates to measuring social values than measuring financial values.
Allocation of resources within for-profit organisations becomes much flexible in number of
ways if they satisfy the criteria of financial performance. On the other hand, NPOs face
limitations on their flexibility to follow suit, due to difficulty of achieving the desired social
values [17]. For a mission-driven NPO, measuring “success” is very difficult [4] [8] [17].
Also, it appears that the technical difficulties of measuring complex missions, or the political
difficulties of designing measurement systems that accommodate the various stakeholders in
nonprofits, has dampened the level of “mission measurement” within the sector [32]. This is
not the case for the for-profit sector because sufficient freedom is available for these
organisations to redefine their missions so as to make profit. Also, in a free enterprise system
devoid of entry and exist barriers, for-profit organisations can move in and out of the business
at any time, depending on the conditions of the external environment [31]. However, NPOs
have to stick their missions to sustain their existence [17].

Studying orgaisational effeciveness in NPOs through standard PM tools also becomes

complex due to their legal and financial status. NPOs also often have unstrctured goals
pertaining to delivery of intangibale servies which limits the use of quantitave measures used
in the PM in the for-profit sector. Finally, as Kanter and Summer [17] note, the strategic
objectives of NPOs are often based on “societal values about which there may be little or no
consensus”, which makes PM very challenging. For these reasons, NPOs need to develop
measurement systems that serve their own strategic needs.

Given that the strategic needs of NPOs can differ based on the subsector in which they
operate, it becomes important to look for an internationally accepted classification scheme of
NPOs. According to the classification system “International Classification of Nonprofit
Organizations” (ICNPO), NPOs are classified under twelve (12) main activity sets (groups)
as shown below (Table 1) [34].

TABLE 1: The International Classification of Nonprofit Organisations [34]

Groups Subgroups
Culture and recreation Culture and arts
Sports

-4-
 Groups Subgroups
Other recreation and social clubs
Education and research Primary and secondary education
Higher education
Other education Research
Health Hospitals and rehabilitation
Nursing homes
Mental health and crisis intervention
Other health services
Social services Social services
Emergency and relief
Income support and maintenance
Environment Environment
Animal protection
Development and housing Economic, social and community
development
Housing
Employment and training
Law, advocacy and politics Civic and advocacy organizations
Law and legal services
Political organizations
Philanthropic intermediaries and Grant-making Foundations
voluntarism promotion Other philanthropic intermediaries and
voluntarism promotion
International International activities
Religion Religious congregations and associations
Business and professional associations, Business associations
unions Professional associations
Labour unions
Not elsewhere classified Not elsewhere classified

While the ICPNO is a useful classification scheme that groups NPOs based on a plausible
criterion, Salamon and Anheier opine that the inherent attributes and the function of the
nonprofit sector in any country is eventually shaped by the entire pattern of social, economic
and political development of that country [34]. This may impose a limitation on the
generalisability of a PM instrument, even if the instrument is focused on a specific group
referred to in the ICPNO. For this reason, the study referred to in this paper focuses on
Australia and New Zealand only; the two Tasman neighbours share many things (social and
cultural characteristics) in common.

Based on the basic structure, size of the sector, volunteer participation, revenue, countries’
levels of development, their social and political histories, their geographical regions, culture
and traditions, the researchers of the Johns Hopkins Comparative Nonprofit Sector Project
established eight country clusters—Welfare Partnership, Anglo-Saxon, Nordic, Eastern
European, Asian Industrialised, Latin American, African, and other developing—for the 41
countries that participated in the project [33]. According to the said cluster classification,

-5-
unsurprisingly, both New Zealand and Australia fall into the same cluster, namely “Anglo-
Saxon”. Anglo-Saxon countries are defined as those countries that are highly developed and
share a common historical connection with the Anglo-Saxon political and legal tradition [33].
Salient features of the nonprofit sector in Anglo-Saxon countries are a very large nonprofit
sector (on average) yet very low contribution by the government (only about a third of total
revenue on average), grater voluntary participation (twice as much as other country
averages), and greater philanthropic support in the form of cash and revenue [35]. The other
important feature of the nonprofit sector in the Anglo-Saxon countries is the greater emphasis
on community service functions (e.g. development and housing) relative to other service
functions.

While the challenges and complexities of measuring performance in NPOs and a dearth of
scholarly work are justifiable reasons in their own right to undertake an academic study, the
motivation for the study reported in this paper were triggered by two other reasons also.
Firstly, there is a phenomenal growth in the nonprofit sector (relative to the government
sector and the private sector), particularly in the Anglo-Saxon countries, over the last fifty
years. According to the Australian Bureau of Statistics [2], from virtually a handful of entities
fifty years ago, the Australian nonprofit sector has soared to 41,008 organisations at the end
of June 2007 (latest published statistics available) employing some 889,919 people (in
addition to over 2.1 million volunteers) with an industry value addition of $35.9 billion.
Similar patterns exist in New Zealand [39]. Therefore, PM in the NPOs has practical
relevance and significance to the region. Secondly, judging by the organisations that vie for
national quality awards in countries such as the US and Australia, it seems that more and
more NPOs are applying for quality awards—both for-profit organisations and NPOs are
assessed using the exact same quality criteria for performance excellence—although very
little (if any) is known about the extent to which principles and practices of quality
management suit NPOs [15] [30]. Even though not all scholars accept so, assessment of
organisations using a strategic quality model such as the Baldrige Criteria for Performance
Excellence (used in the US and many other countries including New Zealand) or the
Australian Business Excellence Framework— for whatever purpose it may be (e.g. vying for
a national quality award, benchmarking)—equates to performance measurement [7] [16].
Therefore it is quite fitting to study to what extent PM in NPOs reflects principals of “quality
management”, as operationally defined by a strategic quality model.

Existing performance measurement frameworks for NPOs and the key issues

Kaplan and Norton’s BSC is the most common performance measurement system that is used
by NPOs [25]. As mentioned earlier, the BSC entails four measurement perspectives
(dimensions)—financial, customer, operational, and learning and growth—which are aligned
in a priority order. The financial perspective receives the highest priority because the BSC is
built on the premise that the ultimate aim of a business is achieving shareholder expectations
[25]. As mentioned earlier, even though perfunctory adjustments have been made to the BSC
to appeal NPOs, there are several aspects in a BSC that do not seem to fit NPOs.

Firstly, in NPOs, the financial measures do not play the same role as they do in for-profit
organisations. The BSC does not attempt to convey measures of “social responsibility”,

-6-
which is a primary reason for the very existence of an NPO. The ultimate goal of an NPO is
to create value to the society in which it operates (without sustaining profitability) and not to
create wealth for those who provide the funding (e.g. private philanthropists, government);
the BSC takes the view that the aim of running the business is to create wealth to its owners
(through customer and employee satisfaction by creating a learning environment) [19] [23].
This does not allude to the fact that financial measures are incongruent to NPOs; the financial
measures do serve as a tool in handling/controlling the cost of operations of NPOs more
effectively. However, financial measures alone cannot determine whether an NPO has
created the intended social value which it wishes to achieve [23]. In NPOs it is important that
social value measurements do align with non-financial measures to become consistent with
their “social missions” [17]. While for-profit organisations need non-profit measures to
retrieve a path to accomplish profit missions, NPOs seek non-profit measures to ensure that
they have effectively used financial resources to create ultimate social goals [23].

Secondly, the customer perspective in the BSC imposes some difficulties in adopting the
BSC to NPOs. Unlike the for-profit sector, the nonprofit sector attracts different kinds of
customers. One customer segment consumes goods and services without paying
(beneficiaries) whilst a third party, which is also a customer segment, acts as charitable
donors who supply the resources to serve the first customer segment. Both customer
segments are equally important to NPOs because fulfilling the aspirations and requirements
of these customers are crucial for their sustainability. Thus the main difficulty in employing
the BSC framework to NPOs is the difficulty in identifying who the actual customers are and
whether the customer segment ‘beneficiaries’ should be given priority or otherwise over the
customer segment ‘donors’ [23].

Finally, the two remaining perspectives—internal processes and learning—are within-

organisational dimensions that are meant to increase the quality and productivity of the
system to gain competitive advantage [23]. Applying competitive advantage notions to NPOs
may bring up the question whether the primary goal of an NPO is to explicate competitive
advantage and sustain the market share in its operating markets [23].

The literature review revealed that apart from the BSC and strategic quality models (e.g.
Baldrige Criteria for Performance Excellence), there are only a limited number of PM
models that have been prescribed for NPOs. These include Tableau de Bord Scorecard,
SMART Pyramid, Performance Prism, The Production of Welfare Framework and the
Logical Framework. Some of these are described in turn.

The prime focus of Tableau de Bord Scorecard is to command physical and financial
standards to gauge and anticipate performance. This model has been developed considering
the organisational vision and mission as the foundation, followed by the strategic objectives,
critical success factors and key performance indicators [9].

Unlike the Tableau de Bord Scorecard, SMART Pyramid is a four layer system that consists
of objectives and measurements that are formulated to “stitch-up” management control
systems with performance standards to facilitate benchmarking [9]. Measures connected with

-7-
intangible assets, differentiation between internal and external measures, efforts to
incorporate the objectives using operational measures are the positive elements of the
SMART Pyramid.

Neely [25] designed the PM framework the Performance Prism using an accessory that was
termed the “success map”; the success map embraces the organisation strategy into the
management system. Five interrelated faces: Stakeholder satisfaction, Strategies, Processes,
Capabilities, and Stakeholder Contribution are embodied in the prism. The most favourable
facet of this model is “stakeholder satisfaction” which is not considered in many other PM
systems [9].

Five key features are embraced in the Production Welfare Framework: “resource inputs”
(employees, capital); “cost or budget”, “non-resource inputs” (ideas or attitudes of internal
participants), “intermediate outputs or outcomes” (services provided and quality of it), and
the “final outcomes” (improvement in the beneficiaries within the period) [22]. The “meso
and macro context” are also entailed in the framework, which embraces the internal and
external environment respectively. The framework is further developed using eight
performance domains and indicators: “economy, effectiveness, efficiency, pluralism, equity,
participation, advocacy and innovation” [22]. However, all eight domains may not be
relevant to all types of NPOs.

The Logical Framework is a management tool that is used for project planning. The
framework has been developed using nine key features: “context, stakeholder analysis,
problem analysis, objective analysis, activities, resource planning, goal achievement
indicators identification, risk analysis and assumptions”; PM is embedded in some of the nine
features [3]. The Logical Framework does not prescribe a priority order (or a logical order) in
which the nine features should be arranged; flexibility is given to contingency factors such as
the organisational requirements. This may mean that the use of the framework may differ
from one organisation to another. The Logical Framework consists of a 4x4 matrix, where the
rows refer to level of objectives and the columns refer to achievements of the objectives.

The shortcomings of the existing PM frameworks become apparent when one attempts to
summarise their weaknesses (the shortcomings of the BSC was covered earlier). The main
pitfall in using the Tableau de Bord Scorecard is that this scorecard mainly focuses on
financial indicators. Moreover, the financial measures are mostly used introspectively to track
how the measures are performing, without attempting to make “external comparisons” to
identify external environmental factors that may inhibit or hasten the growth of the
organisation [4]. Furthermore, it is harder to recognise a distinguishable difference between
strategic and operational indicators and the scorecard; the scorecard also seems to encourage
the senior hierarchy to keep their distance from the operational processes [4]. Strategy
determination is crucial for organisations to achieve its goals but unfortunately, the so called
“Integrated Measurement and Performance System” in the scorecard does not provide a clear
picture about the organisational strategy that underpins the performance measurement
system; also, there is no proper way to capture either the customers or the other stakeholders
in the scorecard [4].

-8-
Compared to other PM models, the “Performance Prism” directs PM efforts towards main
business features. The main disadvantage of the prism is lack of emphasis on having to
provide evidence (through appropriate performance measures) on continuous improvement
and the strategy—two ingredients which are very crucial for organisational sustainability [4].
Furthermore, the prism does not form link between the strategic indicators and the
operational indicators; the prism also does not provide insights as to how the prism should be
practically implemented for performance improvement [4].

Compared to most other models, the theoretical framework of the “Smart Pyramid” does not
seem to be well developed. Although the pyramid shows a causal link between objectives to
operational benchmarks, again, how to implement pyramid is not clearly defined.
Performance feedback is also not added to the model.

Of all the PM models, the Production and Welfare Framework seems to be the most difficult
model to adopt as there is virtually no explanation as to how the model could be used in
practice.

The Logical Framework approach on the other hand seems to be too simple, lacking essential
PM dimensions that are considered to be important in managing organisational performance
[3].

How to measure the performance of NPOs is the main focus of the study reported in this
paper. There is no consensus in the literature as to what guidelines should be used to
accomplish PM unlike in the for-profit sector. The diversity of the nonprofit sector may be
the reason for this, which could be handled by confining the research to a well identified
group belonging to the nonprofit sector (Table 1). Through a better understanding of the
sector, researches and practitioners can contribute their knowledge towards developing a
suitable PM model for NPOs, although to date, not much has been done towards this cause
[24]. Thus, the ultimate goal of this study is to establish a performance measurement
instrument for a specific nonprofit group (development and housing) using appropriate
methods (see the methodology section).

Aims and Objectives of the Study

The aim of the study is to develop a generalisable measurement instrument empirically, in

order to assess the strategic performance of nonprofit organisations for an identified group
(development and housing). The general objective of the study is to critically review the
existing literature on performance measurement.

The specific objectives of the doctoral study are as follows:

• Propose a scientifically validated performance measurement instrument for the
Australasian development and housing sector (objective 1).

-9-
 • Formulate and test the theoretical model that underpins the performance measurement
instrument (objective 2).
• Understand and explain in what respect performance measurement in the nonprofit sector
differs from that in the for-profit sector (objective 3).
• Understand and explain to what extent PM in the nonprofit sector reflects the principals
of strategic quality management (objective 4).

METHODOLOGY

Objective #1

Objective 1 shall be achieved as follows. A survey instrument will be developed to capture

respondent agreement or disagreement to specific statements pertaining to a range of
performance measures using a seven-point Likert type scale (ranging from ‘strongly disagree’
to ‘strongly agree’). The survey instrument shall initially be developed using an extensive
literature review on PM. 1 This instrument shall then be reviewed by a panel of content
experts who are knowledgeable about PM, particularly in the nonprofit sector. Following the
content review, the survey instrument shall be ‘pilot tested’ for clarity, using a sample of 20
respondents (10 each from Australia and New Zealand) belonging to the Australian and New
Zealand sampling frames. The sampling frames listing the NPOs shall be developed based on
records maintained by the Australian Bureau of Statistics and Statistics New Zealand
respectively. Only organisations belonging to the development and housing group (Table 1)
shall be included in the sampling frames. Furthermore, a constraint shall be imposed for the
organisational size—only organisations that employ more than a certain number of staff (the
number to be decided later) will be included in the two sampling frames on the assumption
that smaller NPOs in general are less likely to be interested on PM, compared to the larger
NPOs. The survey responses shall be collected by administering the questionnaire to the
NPOs in the sampling frames using the World Wide Web. All the directors in each NPO in
the sampling frame will be invited to respond to the questionnaire. It is hoped that at least 300
usable responses will be collected from the respondents. The responses will be randomly
assigned to two subsamples: the training sample and the cross-validation sample (both
samples will be equal in size).

The training sample (n ≈ 150) will be used to conduct an exploratory factor analysis (EFA) to
identify the factor structure (the dimensions/perspectives of PM) underlying PM in the
nonprofit sector (a greater sample size will be required if the questionnaire contains a large
number of questionnaire items). SPSS 20 will be used to perform EFA. It is hoped that a
meaningful factor structure will emerge (after factor rotation) and that a practically useful set
of performance measures will be identified under each factor (dimension/perspective).

One can question why two different samples (a training sample and a cross-validation
sample) will be used in the study. The reason is quite simple: a model that has been derived

1
Given the scarcity of literature available pertaining to PM in the nonprofit sector, literature review alone might
be insufficient to develop the survey instrument. This matter is covered in the expected outcomes section.

- 10 -
in an exploratory fashion should never be tested again (objective 2) for conformance by the
very dataset that was used to derive the model [1]. This is because empirically derived
models are notorious for fitting some datasets well at the detriment of under-fitting to a
different dataset, a condition known as “over-fitting” in statistics [36].

Objective #2

Conceptually, objective 2 is closely related to objective 1. An empirically validated

theoretical model underpinning PM in NPOs (objective 2) will further strengthen the
existence of PM dimensions previously identified through EFA (objective 1). In order to
validate the factor structure identified through EFA, the posited factor structure shall be
tested in confirmatory factor analysis (CFA) mode for factorial validity, using a covariance
structure analysis program such as AMOS 20, using the cross-validation sample. In addition
to factorial validity, the measurement model shall also be tested for convergent validity and
discriminant validity. A convergent validity test will be used to show evidence in support of
the fact that measures belonging to each measurement perspective (dimension) do belong (in
a correlational sense) to their assigned measurement perspective. A discriminant validity test
will be used to show evidence in support of the fact that measures belonging to a given
measurement perspective (dimension) do not belong to other measurement perspectives (in a
correlational sense) [12]. This completes the first phase of formulating and testing the
theoretical model underpinning PM in NPOs.

In the second phase of testing, an attempt will be made to identify (causal) structural
relationships between constructs (measurement perspectives/dimensions) based on the
literature. The posited causal model will then be tested as a full structural equation model
using a covariance structure analysis program such as AMOS 20, using the cross-validation
sample. This completes the testing phase related to objective 2. In addition, the empirical
model shall be compared (in descriptive terms) with the BSC and other major PM
frameworks to widen the understanding on PM in NPOs.

Objective #3

In order to achieve objective 3, two ten-member focus groups shall be established. The first
focus group shall consist of directors belonging to NPOs while the second focus group shall
consist of senior members belonging to for-profit organisations. The lead author of this paper
will act as the convenor/moderator in both focus groups. The task of each member in the
focus group (initially) is to rate the relevance of each measure (in a scale of 1 to 10) in the
final questionnaire (objective 1) for the organisation the member is associated with (the final
questionnaire means the questionnaire that will be finalised after the EFA, where poorly
loading questionnaire items will be excluded). Based on the response patterns, it will become
possible to make tentative conclusions on the similarities and differences between
performance measures used in the respective sectors (nonprofit versus for-profit). In the next
stage of mediation, the tentative conclusions shall further be discussed with the focus groups
to make final conclusions related to objective 3.

- 11 -
Objective #4

Objective 4 will be accomplished by triangulating both qualitative evidence and quantitative

evidence. The respondents for quantitative and qualitative data collection are PhD students
and academics (within and outside Australasia) engaged in research on quality management.
A screening test will be done to ensure that the respondents are knowledgeable about at least
one major national quality award model. The sample size required for this part of the study
will be dictated by statistical power calculations related to the quantitative component of the
study.

In the qualitative phase of data collection, each respondent will be asked, through an
unstructured interview (using the telephone or by Skype), to assess the theoretical model
derived for objective 2 in descriptive terms. The responses will then be recorded and coded
using conventions used in qualitative research. In the quantitate phase of data collection, a
suitable method will be devised to test the extent to which the performance measures of the
PM instrument developed relate to strategic quality management concepts (convergent
validity) and the extent to which they do not relate to other management concepts
(discriminant validity). This convergent-discriminant validity analogy was first used by
Campbell and Fiske over 50 years ago [6], which has since been used by management
researchers (e.g. [13]) to answer queries similar to the one related to objective 4.

EXPECTED OUTCOMES

Given the research is still at its early stages, not to mention the highly empirical nature of the
methodology, it would be difficult to make concrete predictions about the expected outcomes
without data. However, even though statistics will be heavily involved in deriving the
measurement model (data analysis in EFA mode followed up in CFA mode using a different
data set) for measuring performance in NPOs, the survey instrument that initiates the analysis
will be grounded in theory, in that the said instrument will be developed based on published
literature. For this reason it would be very surprising if a meaningful factor structure does not
emerge from the EFA (objective 1) and that the factor structure thus derived would not be
confirmed in the CFA (objective 2). Arguably, developing the questionnaire will be one of
the most challenging tasks in the study. Given the limited number of publications available
on PM in the nonprofit sector, the researchers may also have to rely on ‘expert opinion’ to
formulate the questionnaire in a focus group like setting.

While it is reasonable for one to expect the PM framework derived for the NPOs to possess
some features that could be regarded as unique to the sector (objective 3), it would be naïve
to believe that this framework would be vastly different from a PM framework meant for the
for-profit sector. For example, it could be quite possible that a factor which could be labelled
as ‘financial performance’ will emerge from then EFA and that this factor will be placed as
the ‘driver’ that drives the other factors. NPOs (e.g. churches, schools and many other
community service organisations) also do compete with each other for customers and tangible
resources (e.g. donor funding and employees)—even though this competition is not being
dictated by the market forces as in the for-profit sector—and thence a theoretical framework

- 12 -
that underpins PM in the NPOs need not be vastly different from a theoretical framework
meant for the for-profit sector (e.g. the BSC). For similar reasons it would not be surprising
to observe a great degree of similarity between a theoretical model that underpins PM in
NPOs and a strategic quality management model such as the Baldrige Criteria for
Performance Excellence or the Australian Business Excellence Framework (objective 4).

CONCLUSIONS

The nonprofit sector is emerging as a very important sector in our society. Not all public
services can be provided by the government. NPOs too need to measure their strategic
performance to ensure that they remain viable, in the face of the changing external
environment. Very little is known about what PM means in the nonprofit sector. The research
reported in this paper is undertaken to add new knowledge by way of deriving a scientifically
validated PM framework, based on data collected from a particular group of NPOs
(development and housing group). Comparing and contrasting the empirically derived PM
framework with PM frameworks meant for the for-profit sector and studying to what extent
the former reflects principals of strategic quality management are the other objectives of the
research. The research is at its early stages of development and hence no data are available at
the present point in time—the data collection will begin as soon as the survey instrument,
which acts as a pivotal accessory in this study, is developed—to conduct the analyses related
to the research objectives.

Future studies may be aimed at replicating the study in other settings such as in other groups
in the nonprofit sector (Table 1) and across other national or regional boundaries (e.g. in a
nonAnglo-Saxon setting) in order to increase the generalisability of the findings.

REFERENCES

[1] Abdi, H. & Williams, L.J Principal component analysis. Wiley Interdisciplinary Reviews:
Computational Statistics, 2010, 2(4), 433–459.

[2] Australian Bureau of Statistics Not-for-profit Organisations, Australia, 2006-07

(Re-Issue): Overview. 2009 [cited 2013 28 March]; Available from:
http://www.abs.gov.au/ausstats/[email protected]/Products/8106.0~2006-07+(Re-Issue)~Main+
Features~Overview?OpenDocument.

[3] Bakewell, O. & Garbutt, A The use and abuse of the logical framework approach.
Stockholm: Swedish International Development Cooperation Agency (Sida), 2005.

[4] Bryson, J.M. Strategic planning for public and nonprofit organizations: A guide to
strengthening and sustaining organizational achievement. Jossey-Bass, Vol. 1, 2011.

[5] Butler, A., Letza, S.R. & Neale, B Linking the balanced scorecard to strategy. Long
Range Planning, 1997, 30(2), 242-153.

- 13 -
[6] Campbell, D.T. & Fiske, D.W Convergent and discriminant validation by the multitrait-
multimethod matrix. Psychological Bulletin, 1959, 56, 81-105.

[7] Evans, J.R. An exploratory study of performance measurement systems and relationships
with performance results. Journal of Operations Management, 2004, 22(3), 219-232.

[8] Forbes, D.P. Measuring the unmeasurable: Empirical studies of nonprofit organization
effectiveness from 1977 to 1997.

[9] Forcada, M.Á.H., Ramis-Pujol, J. & Cusumano, J.L.G Comparative Analysis of Nine
Performance Measurement Systems. The theories and practices of organization excellence:
New Perspectives, 359.

[10] Friedman, A.L. & Miles, S Developing Stakeholder Theory. Journal of Management
Studies, 2002, 39(1), 1-21.

[11] Friedman, M. & Friedman, R.D Capitalism and freedom / Milton Friedman ; with the
assistance of Rose D. Friedman ; with a new preface by the author 2002. Chicago : University
of Chicago Press, 2002, 40th anniversary ed.

[12] Gefen, D. & Straub, D.A practical guide to factorial validity using PLS-GRAPH: tutorial
and annotated example. Communications of the Association for Information Systems, 2005,
16, 91-109.

[13] Hackman, R.J. & Wageman, R Total quality management: empirical, conceptual, and
practical issues. Administrative Science Quarterly, 1995, 40(2), 309-342.

[14] Hellriegel, D., Jackson, S.E. & Slocum, J.W Managing : a competency-based approach /
Don Hellriegel, Susan E. Jackson, John W. Slocum, Jr2008. Cincinnati, Ohio: Thomson/
South-Western, c2008, 11th ed.

[15] Jayamaha, N.P., Grigg, N.P. & Mann, R.S Empirical analysis of the Baldrige Criteria as
both an organisational performance measure and a theoretical model. Measuring Business
Excellence, 2011,15(1), 20-33.

[16] Johnson, C.C. Introduction to the Balanced Scorecard and performance measurement
systems, in Balanced Scorecard for state-owned enterprises: driving performance and
corporate governance, C.C. Johnson and I. Beiman, Editors. Asian Development Bank:
Manila, Philippines, 2007, 1-13.

[17] Kanter, R.M. & Summers, D.V Doing well while doing good: Dilemmas of performance
measurement in nonprofit organizations and the need for a multiple-constituency
approach1994: Sage publication, Londres.

[18] Kaplan, R.S. Strategic performance measurement and management in nonprofit

organizations. Nonprofit Management and Leadership, 2001, 11(3), 353-370.

[19] Kaplan, R.S. & Norton, D.P The balanced scorecard–measures that drive performance.
Harvard business review, 1992, 70(1), 71-79.

- 14 -
[20] Kaplan, R.S. & Norton, D.P Linking the Balanced Scorecard to strategy. California
Management Review, 1996, 39(1), 53-79.

[21] Kaplan, R.S. & Norton, D.P Strategy maps: converting intangible assets into tangible
outcomes. Boston, MA: Harvard Business School Press, 2004.

[22] Kendall, J. & Knapp, M Measuring the performance of voluntary organizations. Public
Management Review, 2000, 2(1),105-132.

[23] Moore, M.H. The public value scorecard: a rejoinder and an alternative. Review of
strategic performance measurement and management in non-profit organizations, by Robert
Kaplan. HCNO Working Paper Series, May, 2003.

[24] Moxham, C. Performance measurement: Examining the applicability of the existing

body of knowledge to nonprofit organisations. International Journal of Operations &
Production Management, 2009, 29(7), 740-763.

[25] Neely, A.D. Business Performance Measurement : Theory and Practice. Cambridge
University Press, 2002.

[26] Neely, A.D. Business performance measurement : unifying theories and integrating
practice / edited by Andy Neely 2007. Cambridge , New York : Cambridge University Press,
2007 , 2nd ed.

[27] Northcott, D. & Taulapapa, T.M.A Using the balanced scorecard to manage performance
in public sector organizations: Issues and challenges. International Journal of Public Sector
Management, 2012, 25(3), 166 - 191.

[28] O'Neill, M. & Young, D.R Educating managers of nonprofit organizations / edited by
Michael O'Neill and Dennis R. Young ; with a foreword by Paul Ylvisaker. New York :
Praeger, 1988.

[29] Oakland, J.S. Total organizational excellence. Routledge,2001.

[30] Paton, R., Foot, J. & Payne, G What happens when nonprofits use quality models for
self-sssessment? Nonprofit Management and Leadership, 2000, 11(1), 21-34.

[31] Porter, M.E. The five competitive forces that shape strategy. Harvard Business Review,
2008, 86(1), 78-93.

[32] Ritchie, W.J. & Kolodinsky, R.W Nonprofit organization financial performance
measurement: An evaluation of new and existing financial performance measures. Nonprofit
Management and Leadership, 2003, 13(4), 367-381.

[33] Salamon, L.M. Global civil society. Vol. 2, Dimensions of the nonprofit sector / Lester
M. Salamon, S. Wojciech Sokolowski, and associates. Bloomfield, Conn. : Kumarian ;
London : Eurospan, 2004.

[34] Salamon, L.M. & Anheier, H.K Defining the nonprofit sector : a cross-national analysis /
Lester M. Salamon and Helmut K. Anheier. Johns Hopkins non-profit sector series: 41997.

- 15 -
 New York : Manchester University Press, 1997.

[35] Sanders, J., et al. The New Zealand non-profit sector in comparative perspective2008:
Office for the Community and Voluntary Sector Wellington.

[36] SAS InstituteInc., S. Partial least squares, 2013.

[37] Sawhill, J.C. & Williamson, D Mission impossible?: Measuring success in nonprofit
organizations. Nonprofit Management and Leadership, 2001, 11(3), 371-386.

[38] Tyagi, K. & Misra, P Professional Communication. PHI Learning Pvt. Ltd,2011.

[39] Welch, D. Counting Non-profit Institutions in New Zealand. Sector Highlights, 2005.

[40] Zhang, C., Pan, F. & Lin, T.W Market competition and the use of performance measures
in Chinese firms Journal of Corporate Accounting & Finance, 2013, 24(3), 65–70.

- 16 -

View publication stats