This article is (c) Emerald Group Publishing and permission has been granted for this version to appear

here
([Link] Emerald does not grant permission for this article to be
further copied/distributed or hosted elsewhere without the express permission from Emerald Group Publishing
Limited.'
 THE IMPACT OF ERP-SYSTEMS ON FIRM AND BUSINESS
PROCESS PERFORMANCE

Authors:

Bernhard Wieder*
Peter Booth
Zoltan P. Matolcsy
Maria-Luise Ossimitz

Affiliation:

University of Technology, Sydney

Faculty of Business, School of Accounting

Address:

POBOX 123, Broadway

NSW 2007, AUSTRALIA

Tel/Fax/Email:
Tel: ++61 2 9514 3569
Fax: ++61 2 9514 3669
[Link]@[Link]

* Author of Correspondence
 THE IMPACT OF ERP-SYSTEMS ON
FIRM AND BUSINESS PROCESS PERFORMANCE

ABSTRACT

The purpose of this article is to provide further insights into the adoption of enterprise
resource planning systems (ERPS) and the impacts on organisational performance. It
aims at challenging existing claims of ERP vendors with regards to the benefits of their
products and at providing evidence of the benefits of bundling ERPS with supply chain
management systems (SCMS).

A survey was conducted to collect data on several aspects of organisational perform-

ance in companies which adopted ERPS, SCMS and the respective control groups. Fi-
nancial key performance indicators were used to measure overall firm performance and
the Supply-Chain Operations Reference (SCOR) model to operationalise performance
at the business process (supply chain) level.

Our key results contradict the claims of ERPS vendors insofar as we found no signifi-
cant performance differences between ERPS adopters and non-adopters, neither at the
core business process level, nor at the overall firm level. While we could confirm that,
the longer the experience of firms with ERPS, the higher their overall performance, we
found no evidence of a similar effect on business process (supply chain) performance.
Only those ERPS adopters which also adopted SCMS achieved significantly higher per-
formance at the business process level.

KEYWORDS:
Enterprise Resource Planning (ERP), Enterprise Systems, SCMS, IS-evaluation

CLASSIFICATION:
Conceptual article

ii
1. INTRODUCTION
The global success of Enterprise Resource Planning Systems (ERPS) has not only at-
tracted the interest of researchers from the IT/IS-discipline, but from all major disciplines
in business research – including accounting. The emergence of ERPS has moved the
topic 'computerised business information systems' increasingly from the IT/IS domain to
the business domain; from systems design and programming to business configuration,
process mapping and reengineering. Using the systems development life cycle (SDLC)
as conceptual reference, we observe that the efforts and weight of the critical success
factors of ERPS projects (Sumner, M.,1999) moved from 'systems design' to 'systems
implementation', and implementation teams are now usually led and dominated by clas-
sic business roles rather than IT-staff. ERPS impose their logic on organisations and of-
ten force employees to think in terms of integrated processes and to change the way
they do accounting, production planning and control, etc.

All these changes brought about by ERPS made them a very ‘attractive’ research object
of many business disciplines, and accounting is by nature the one from which we expect
an answer to the question which ultimately counts in business: Are those systems worth
the money?

Earlier approaches to this question identified non-financial performance measures as

proxies for financial value (Bhatt, 2000, Beretta, 2002, Irani, 2002). Only relatively re-
cently, the accounting discipline has come up with studies which for the first time pro-
vided evidence of the economic impacts of ERPS (Poston and Grabski, 2001, Matolcsy
et al., 2002, Hunton et al., 2003, Nicolaou et al., 2003, Matolcsy et al., 2005). The re-
sults of those studies point into the same direction, but are not entirely consistent. They
all find some evidence of immediate or delayed increases in firm performance after
ERPS adoptions, but the performance increases identified are evident only in very few
KPIs.

We argue that one of the reasons for these inconsistencies is that just relating 'ERPS-
adoption' to publicly available 'financial key performance indicators' (and controlling for
some other dependent variables) fails to explain the increasing diversity and complexity

1
of the life cycle of enterprise systems. In the emerging generation of enterprise applica-
tion integration (EAI) enabled enterprise systems, traditional ERP-functionality cannot
be isolated from the rest of the whole system and has to be evaluated in the broader
context using a multi-dimensional system of performance measurement.

The purpose of this article is to provide further insights into the impacts of several as-
pects of ERPS adoption and ERPS use on firm performance and business process per-
formance. It aims at challenging existing claims of ERP vendors with regards to the
benefits of their products, and at providing evidence of the impact of bundling ERPS
with supply chain management systems on performance.

We use ERPS adoption, ERPS history and ERPS extension (with a supply chain man-
agement system) as independent variables and KPIs for supply chain performance and
firm performance as dependent variables.

Our key results contradict the claims of ERPS vendors insofar as we found no signifi-
cant performance differences between ERPS adopters and non-adopters, neither at the
supply chain level, nor at the overall firm level. While we could confirm that the longer
the experience of firms with ERPS, the higher their overall performance, we could not
confirm this effect with regards to business process (supply chain) performance. The
latter result contradicts benefits widely associated with ERPS and supports the argu-
ments in favour of the adoption of supply chain management systems (SCMS), because
the latter proved to have a positive impact on business process (supply chain) perform-
ance in our study.

The main part of the article is structured as follows: First, we develop our research
model and predictions (section 2), then we explain our research method and sample
(section 3) and provide the results of our statistical analysis (section 4). The interpreta-
tions of the results and limitations are outlined in the concluding section (5).

2
2. RESEARCH FRAMEWORK AND MODEL DEVELOPMENT

2.1. Review of Literature on the Impact of ERPS on Performance

Recent research on the impacts of ERPS on financial performance has found some evi-
dence supporting the claim of all major ERPS vendors that their products enable com-
panies to achieve “faster return on investment” (SAP, 2003a, and similar: SAP, 2003b,
ORACLE, 2003a/b, and PEOPLESOFT, 2003).

Poston and Grabski (2001), for example, analyse four financial characteristics before
and after ERP adoption using univariate tests. Their results indicate that ERP adoption
leads to efficiency increase in terms of a reduction in employee numbers and in the ratio
of employees to revenues for each year following the ERP implementation.

Hunton et al. (2003) provide evidence on the impact of ERPS adoption and overall firm
performance by comparing return on assets, return on investments and asset turnover
for ERP adopters and non-adopters. Their key results do not indicate a performance
improvement for ERP adopters. However, they find that the financial performance of
adopters has not declined during their test period, whilst the performance of non-
adopters has declined during the same period.

Nicolaou et al. (2003) compare financial data of companies adopting enterprise wide
systems and of a matched control group of firms. The results from a univariate analysis
of performance differences across time periods show that firms adopting enterprise sys-
tems have significantly higher differential performance in their second year after the
completion of the system than the control group.

All these recent examples of accounting research on the impact of ERPS on the per-
formance of organisations provide valuable insights into a previously under-researched
topic and they are similar in their approach in several ways: (a) They use only aggre-
gated, publicly available financial accounting data to measure performance, (b) they do
not clearly distinguish between overall firm performance and business process perform-
ance, and (c) the dependent variables used are essentially only ERPS adoption and the
time of adoption.

3
More recently, Matolcsy et al. (2005) performed a study using the same approach with
regards to (a) and (c), however they extended the analysis of performance by also ex-
plicitly investigating financial performance indicators of core business processes in the
value-chain. They found that the adoption of ERP systems leads to sustained opera-
tional efficiencies and improved overall liquidity. In addition, some support was found for
increased profitability some time after the adoption of ERP, and for improvements in ac-
counts receivable management.

While the approach in all these studies is similar, the results differ substantially. In the
following sections, we elaborate a model which – as we argue – can provide deeper in-
sights into some of the relationships between ERPS and performance by further refining
the dependant variable ‘ERPS’ (see section 2.3) and by combining financial and non-
financial performance measures in our analysis (see section 2.4).

2.2. Model Foundations

Like Matolcsy et al. (2005), we use the generic framework suggested by Dehning and
Richardson (2002) as guidance for the development of our ERPS performance meas-
urement model. Their framework classifies existing research on IT-performance meas-
urement using 5 paths as shown in Figure 1. Based on a comprehensive review of the
financial accounting and finance literature on IT-performance measurement, they dem-
onstrate that most research in this field focuses on only one path (usually path 1), by-
passing many other important factors. They also argue convincingly that this main-
stream one-dimensional approach fails to explain IT-performance and leaves a gap be-
tween anecdotal evidence of IT-success and results of mainstream empirical research.
The 'IT-productivity paradox' discussion and the often conflicting results of research fol-
lowing alternative paths in their framework can be quoted as the most important exam-
ples supporting their call for a more comprehensive 'multi-path' approach.

4
 1

Business
Information Firm
Process
Technology 2 3 Performance
Performance
Measures Measures
Measures

Contextual
5
Factors

Figure 1: Framework for Evaluating Research on the Effects of IT Investments

(Dehning and Richardson, 2002; simplified)

In the subsequent sections, we first operationalise the three objects in our model in the
context of ERPS performance measurement: IT-measures, business process perform-
ance measures and firm performance measures. Then we make predictions about rela-
tionships (paths) between those objects.

2.3. IT (ERPS) Measures Refined

The definitions of ERPS used in literature vary slightly, but they all have one common
theme: enterprise-wide integration of data/information and business processes. For
Markus et al. (2000), for example, “ERP-systems are commercial software packages
that enable the integration of transaction-oriented data and business processes
throughout an organization”, and according to Shang and Seddon (2002) “ERP software
integrates management information and processes, such as financial, manufacturing,
distribution and human resources, for the purpose of enterprise-wide management of
resources”. They also emphasise that ERPS are an example, but not the only form of
‘enterprise systems’ (ES). They discuss common features of enterprise software which
are often associated with ERPS,1 such as ‘set of packaged application software mod-

5
ules’, real-time information processing, based on deep knowledge of business practices
gained by the vendors from their customers, semi-finished product which has to be con-
figured, etc.

The first comprehensive empirical analysis of the performance of ERPS uses 'ERPS
adoption' as the only independent variable, and then compares the performance of or-
ganisations pre and post-adoption (Poston and Grabski, 2001). Later research consid-
ers and investigates the time-factor in more detail, or more precisely: the time-difference
between the adoption of an ERPS and when performance is measured (e.g. Hunton et
al., 2003, Matolcsy et al., 2002, Nicolaou et al., 2003, Matolcsy et al., 2005). The results
of this more recent research suggest the existence of a time-lag between the adoption
of an ERPS and the realisation of benefits. We therefore include the time factor in our
analysis of performance impacts of ERPS.

The second aspect of ERPS to be considered is the co-existence of other components

of enterprise systems (ES), in particular specialised supply chain management systems
(SCMS). Already back in 2001, a survey (N = 89) in the US found that 59% of the re-
spondents either had already or were planning to extend their ERPS with a SCM-
solution. 84% of the respondents indicated that for any such expansion they would pre-
fer to use the SCMS provided by their ERP-vendor (CIO magazine, 2001). SCM-
software packages are more specialised forms of ES which integrate all logistics func-
tions within organisations and usually also support inter-organisational business and
systems integration (Tarn et al., 2002). In contrast to ERPS, SCMS usually comprise
Advanced Planning Systems (APS) which support mathematical optimisation proce-
dures and heuristics. Used in combination with ERPS, they partly substitute and partly
complement ERPS functionality, with the level of substitution/complementation being
primarily determined by the functionality of either package and partly by the preferences
of the organisation using them.

We expect that the advanced features of SCMS and their functional focus lead to higher
performance along the primary activities of the value chain (Porter, 1985), and used in
conjunction with ERPS, to higher overall firm performance.

6
2.4. Business Process Performance Measures
All major ERP-products on the market support ‘logistics’ (supply chain) processes and
methods in operations management as part of their core-functionality. Therefore, we
expect that the adoption of ERPS leads to better performance in terms of both primary
and secondary2 (support) business processes in the value chain.

Financial KPIs are doubtlessly the most important and most widely used KPIs for firm
performance, and they can also provide some valuable insights into business process
performance (Dehning and Richardson, 2002, Matolcsy et al., 2005). These insights
are, however, limited, especially if only publicly available data (from annual reports, etc.)
are used. These highly aggregated data can only be used for a very general analysis of
core processes in the value chain, e.g. the ability to turn over inventory quickly, but do
not provide deeper insights into performance at the business process level (Kaplan and
Norton, 1992). Obtaining these more detailed data on business process performance,
however, requires research beyond secondary data analysis, i.e. a field study.

The operations management and supply chain management (SCM) literatures provide
many examples of SC-performance measurement, and the Supply-Chain Operations
Reference model (SCOR-model) published by the Supply Chain Council3 is considered
to be an established model for measuring performance along this dimension; it will
therefore be used for identifying and measuring business process performance in this
study. The model comprises a mix of financial and non-financial measures for flexibility,
reliability, responsiveness and costs/assets along the supply chain (see appendix).4

2.5. Firm Performance Measures

The empirical studies investigating the financial performance implications of ERPS use
primarily financial ratios (and some limited market data). However, there is little overlap
in terms of the precise measures used, as Table 1 illustrates.

7
Study Poston and Hunton et al., Nicolaou et al., Matolcsy et al.
Grabski, 2001 2003 2003 2005
Residual Income
Return on Assets Return on Assets Net Profit Margin
at 12%
Return on
SG&A/Revenues Return on Sales Current Ratio
Investments
Fixed Asset
COGS/Revenues Asset Turnover Asset Turnover
Turnover
Measures
Number of Em- Sales Days
used
ployees/Revenues Outstanding
Accounts
Payable Days
Inventory
Turnover
Sales Change
SG&A = Selling, General and Administrative Expenses; COGS = Costs of Goods Sold

Table 1: Financial and Market Performance Measurements used in Prior Literature

Interestingly, there is no agreement in the accounting literature with regards to which (of
those) KPIs measure overall firm performance or business process performance best in
the context of ERPS evaluation. There are even substantially different views as to
whether certain KPIs measure firm performance or business process performance.

For classification purposes, follow the approach taken by Matolcsy et al. (2005), who
use net profit margins and the current ratio as indicators of firm performance and fixed
asset turnover, sales days outstanding, accounts payable days, inventory turnover and
sales change as examples of process measures. Accordingly, we classify residual in-
come, return on assets, return on assets, return on investments and return on sales as
firm performance measures (underlined in Table 1), and all remaining measures as
business process performance measures.

In contrast to the studies mentioned above, we use a field study (survey), which gives
us more flexibility with regards to the selection of financial performance measures, be-
cause we do not have to rely on publicly available financial data. In addition, we are
able to include respondents perceptions about their companys’ performance compared
to other firm's in their industry. We use ROI as indicator of the profitability of the capital
employed and operating profits as an absolute5 KPI of profitability. Non-operating re-

8
sults are excluded, because we do not think that they are influenced by ERPS. We also
include the dynamic components of operating profits, viz. sales growth rate and cost re-
duction. In addition to these accrual performance measures, we include cash flow as an
indicator of a firm’s performance in terms of liquidity (Matolcsy et al., 2002).

2.6. Predictions
As mentioned in the introduction, our key objective is to investigate impacts of several
aspects of ERPS adoption on firm performance and business process performance. We
use ERPS adoption, ERPS history and ‘ERPS extension with a SCMS’ as independent
variables and KPIs for supply chain performance and firm performance as dependent
variables.

P1a-c

ERPS-related
Measures:

ERPS Adoption KPIs of Firm Key

P2a-c Supply Chain P3 Performance
ERPS History Processes Indicators
(KPIs)
ERPS
Extension
(SCMS)

Figure 2: Research Model

The model shown in Figure 2 comprises all main relationships investigated in our study.
Our model development leads to the following predictions:

P1a: We predict better firm performance for ERPS users than for non-ERPS users.
P1b: For ERPS users we predict that the longer the experience with ERPS, the better
their firm performance.
P1c: We predict better firm performance for ERPS users extending their ERPS with
SCMS than for ERPS users that do not.

9
P2a: We predict better supply chain performance for ERPS users than for non-ERPS
users.
P2b: For ERPS users we predict that the longer the experience with ERPS, the better
their supply chain performance.
P2c: We predict better supply chain performance for ERPS users extending their
ERPS with SCMS than for ERPS users that do not.
P3 (control): We predict a positive impact of supply chain performance on firm perform-
ance.

3. RESEARCH METHOD

3.1. Sample and Descriptive Statistics

In March 2001, a questionnaire was mailed out to the Chief Information Officer (or
equivalent) of 2,170 Australian companies sourced from the Australian Business Jour-
nal 'BRW' top-500 list and a 'Connect4' database. The large target sample was used to
identify a sufficiently comprehensive set of firms who were either using or intending to
use both ERPS and SCMS. A follow up letter was sent to all non-respondents in April.
68 responses to this first stage of the survey were received by the third week of April.
As 296 questionnaires were returned by Australia Post due to incorrect addresses, the
actual sample size was 1,874 firms. The low response rate might have been due to
‘over-surveying’ in Australia’s industry, the length of survey, but also to errors in the un-
derlying address databases.6

In this second stage, a telephone survey, we had to exclude several sections from our
questionnaire which was too long for a telephone survey. Accordingly, we had to drop
some research questions we had initially set up. The sample for stage 2 was selected
from the initial large sample reduced by all companies which had either responded al-
ready or had moved location. Out of this sample, we selected those 49 firms who had
responded to a previous survey on similar topics.7 38 of those firms participated in the
second stage of the survey. The final sample of respondents comprised 106 companies
or 5.7% of the actual sample size.

10
During the process of data entry, another 4 companies had to be eliminated due to
clearly inconsistent data or irrelevance for this study (non-profit organizations). So the
final sample used for our analysis comprised 102 companies from all sectors of the
economy (see Table 2).
Industry Percent SCMS
Diversified Resources/Mining 15.7%
Others (Primary Sector) 1.0%
Energy 2.9%
Total: Primary Sector 19.6%
Assembly-Line Production 5.9%
Batch Production 13.7%
Process Production 10.8%
Project/Unit Production 1.0%
Total: Secondary Sector 31.4%
Retail/Wholesale and Transport 18.6%
Consulting 2.0%
Financial Services, Banking/Finance, Insurance 11.8%
Healthcare, Tourism and Leisure 6.9%
Media, Telecommunications, Software, IT-Services 2.9%
Property, Infrastructure and Facilities 1.0%
Others (Services Sector) 5.9%
Total: Tertiary Sector 49.0%

Table 2: Industry Classification of Respondents

All primary sector, manufacturing sector and retail/wholesale companies (69.6%) were
considered to be eligible for testing predictions 1c, 2c and 3. In this article, we refer to
this sample as ‘SCM-companies’.

The average operating revenue in the financial year preceding the survey was Mio
A$7,506 and the average number of full-time employees (or equivalents) was 6,683.
The average number of regular suppliers (2,832) and customers (276,688) were rela-
tively large, considering the relatively small size of the Australian market and economy.

48% of the respondents indicated that their company was fully Australian owned, 20%
were majority Australian owned, and 23% were either dominated or fully owned by a
foreign entity. The remaining 9% of respondents to the survey did not answer this ques-
tion.

11
3.2. Measures of Key Variables
The selection criteria for the key variables used in our study have already been dis-
cussed in detail. In this section, their actual measurement is explained.

Respondents were provided with a short version of our definition of ERPS and were
asked whether they had ever considered implementing such a system. If so, their ERPS
status was categorised as follows: (1) ‘System analysis is still in progress’, (2) ‘rejec-
tion’, (3) ‘implementation is still in progress’ or (4) ‘have adopted and ‘gone-live’. Only
go-live companies (4) were included in the test group for P1a and P2a. Implementers
(3) were excluded from our analysis, because we considered them to be in a transition
state which qualified them neither as ERPS users nor as non-users. Respondents who
ticked either (1) or (2) were treated as non-users. The same set of questions was used
to categorise respondents in terms of SCMS-use.8

We also asked ERPS users to provide us with the go-live year and month, the ERPS
vendor and product name, the current version/release and details on the modules used
at the time the survey was conducted. This information was used to verify the ERPS
status and to provide the basis for generating a measure of ERPS history.

All ERP products from vendors other than the ‘big-3’ were verified to ensure that they
really met the definition of ERPS used in our research. 3 respondents who claimed to
be ERPS users had to be eliminated from this group after this investigation. The final
sample contained 49 (48% of 102) ERPS users and 40 (39.2%) non-users. 9 respon-
dents were just in the implementation process and 4 did not answer this question.
Hence, the ratio of companies in the test-group for P1a and P2a to those in the control
group was 55.1 to 44.9.

As mentioned earlier in this article, five firm KPIs were selected with reference to earlier
accounting literature. Respondents were asked to rate their comparative firms’ perform-
ance on those criteria on a 7-point Likert scale (1 = not at all satisfactory, 7 = out-
standing).

The same scale was used to measure the performance at the business process/supply
chain level. Those questions were derived from the SCOR-model, which contains sup-

12
ply chain measures from four areas: Flexibility, reliability, responsiveness and
costs/assets. Only the responses of SCM-companies were used in our tests of P1c, P2c
and P3 (see appendix for details).

4. STATISTICAL ANALYSIS AND RESULTS

4.1. Statistical Analysis

4.1.1. Performance of ERPS users and non-ERPS users (P1a & P2a)

In our first analysis, we tested for differences between ERPS users (49 valid responses)
and non-users (40) with regards to the two levels of performance identified in our model
(prediction 1a & 2a). As the test for normal distribution of the test group and control
group was negative, nonparametric two-independent sample tests (Mann-Whitney U
Test and Two-Sample Kolmogorov-Smirnov Test) were used.

Table 3 summarises the results of our tests of P1a & 2a. Despite the fact that ERPS us-
ers on average scored clearly higher in all dimensions of performance, we could not find
any significant differences between the two samples, neither in terms of firm perform-
ance, nor in terms of performance along the supply chain. Even a 1-tailed interpretation
of our tests does not change this fact.

Grouping Variable: ERPS use Firm (Financial KPIs) Supply Chain

KPIsx)
Mann-Whitney U 841.5 242.5
Wilcoxon W 1,661.5 395.5
Z -0.841 -1.073
Asymp. Sig. (2-tailed) 0.400 0.283
N 87 52

13
 Firm (Financial KPIs) Supply Chain
KPIsx)
Kolmogorov-Smirnov Z 0.717 0.847
Asymp. Sig. (2-tailed) 0.683 0.470
N 87 52
x)
Only SCM-companies included.
Table 3: Mann-Whitney U Test and Two-Sample Kolmogorov-Smirnov Test
for ERPS/non-ERPS

These overall results are partly contradicting those of recent studies quoted in the litera-
ture review, which actually concluded that there are some significant differences in fi-
nancial performance between ERPS adopters and non-adopters (Hunton et al., 2003,
Nicolaou et al., 2003, Matolcsy et al., 2005). What was even more surprising was the
fact that ERPS users did not score significantly higher than the control group in the area
of logistics/supply chain management, one of the core domains of ERPS. A closer look
at all individual scores within the aggregate measure supply chain KPIs revealed that
ERPS users did actually score higher in all 23 individual KPIs, but it also shoed that
these differences were only significant in the following 4 individual categories (at the
0.05-level): Relative level of manufacturing labour costs, relative level of finished goods
inventory holding costs, time required to produce an item and percentage of shipping
errors.

In the case of firm performance indicators, the ERPS users also showed higher means
in all individual categories, but none of these differences were significant. We expected
further insights into the reasons for the partly surprising results by testing for perform-
ance differences with regards to ERPS history and ERPS adoption (use).

4.1.2. ERPS History and Performance (P1b and P2b)

Two types of tests were performed to identify differences between companies with
longer ERP-experience and more recent adopters. In the first test, an ERPS history
scale was used which measures the number of months between the go-live date and
the day the survey was completed. As the observations on this scale were normally dis-
tributed, we used standard correlations (Pearson, Kendall and Spearman) to test our
predications.

14
The results of this first test confirm our predictions with regards to the relationship be-
tween ERPS history and firm performance, but not about the impact of ERPS history on
supply chain performance.

Firm (Financial Supply Chain

ERPS use in months KPIs) KPIsx)
Correlation Coefficient 0.394** 0.173
Pearson
Sig. (1-tailed) 0.004 0.143
Correlation
N 44 40
Correlation Coefficient 0.243* 0.073
Kendall's tau_b Sig. (1-tailed) 0.012 0.278
N 44 33
Correlation Coefficient 0.382** 0.061
Spearman's rho Sig. (1-tailed) 0.005 0.368
N 44 33
* Correlation is significant at the 0.05 level (1-tailed).
** Correlation is significant at the 0.01 level (1-tailed).
x)
Only SCM-companies included.

Table 4: Correlations for ERPS history and Performance

In the second test, we split the group of ERPS users into 2 sub-groups of ‘mature ERPS
users’ and ‘recent ERPS adopters’. We used the median (5 years) as a cut-off point,
which also complies with recent research which suggests that there is a time-lag of
benefit realisation and that the positive impact on performance starts to become signifi-
cant after 2-3 years and tends to increase over years. The results of the nonparametric
two sample tests confirmed the results of the correlations shown in Table 4.

For the reasons mentioned in section 2.4, the weak impact of ERPS history on supply
chain performance was surprising. So we hoped to gain further insights into this result
by comparing the performance scores of ERPS users with SCMS (test group) and with-
out (control group).

4.1.3. ERPS Extensions (SCMS) and Supply Chain Performance (P1c & P2c)

Due to the short history of SCMS, especially in Australia, only 7 organisations in our
sample had adopted SCMS by the time the survey was conducted, 6 of which also used
ERPS. This very small sample size eliminates statistical explanatory power from this
analysis. However, the results of our statistics might still at least give useful indications

15
and provide paths for future research. The results of the independent sample statistics
are depicted in Table 5. In the absence of positive results from tests for normal distribu-
tion, nonparametric two-independent sample tests were used again.

Grouping Variable: Firm (Financial KPIs) Supply Chain

SCMS & ERPS use KPIs
Mann-Whitney U 130.0 34.5
Wilcoxon W 964.0 737.5
Z -0.640 -2.253
Asymp. Sig. (2-tailed) 0.522 0.024*
N 6 6
Kolmogorov-Smirnov Z 0.670 1.418
Asymp. Sig. (2-tailed) 0.761 0.036*
N 6 6
* Significant at the 0.05 level (1-tailed).
** Significant at the 0.01 level (1-tailed).

Table 5: Extended ERPS (ERPS + SCMS)

These results provide an indication that ERPS users that also adopted SCMS perform
by far better along the supply chain (mean 5.0) than ERPS users which did not use ad-
ditional SCMS (mean 4.24). However, we could not confirm prediction 1c, viz that the
adoption of a SCMS in a ERPS user company also directly leads to firm performance
increases.

4.1.4. Supply-Chain Performance and Firm Performance (P3)

Prediction 3 was tested only for control purposes rather than for gaining new insights,
because unless important additional factors impact negatively on firm performance, high
performance in the core business processes would very likely lead to higher firm per-
formance. Correlation results indeed confirmed out predication (1-tailed):
• Pearson Correlation: 0.363, p = 0.007
• Kendall's tau_b: 0.213, p = 0.024
• Spearman's rho: 0.304, p = 0.022

4.2. Summary of Main Results

Figure 3 summarises the main results of our analysis. We found no support for our pre-
diction that ERPS users will have significantly higher performance scores at both the

16
core business process level and at firm level (P1a and P2a). However, we confirmed
that longer experience with ERPS has a positive impact on firm performance and that
ERPS extended by SCMS positively impact on business process performance (supply
chain KPIs). The latter result, however, is only indicative due to a very small sample
size.

ERPS-related
Measures:
P1a P2a
ERPS Adoption
Firm Key KPIs of
Performance P1b P2b Supply Chain
ERPS History
Indicators Processes
P1c ERPS P2c
(KPIs)
Extension
(SCMS)

P3
Prediction confirmed Prediction no confirmed

Figure 3: Summary of Main Results

17
5. CONCLUSIONS, LIMITATIONS AND FUTURE RESEARCH
The main objective of this article was to investigate impacts of several aspects of ERPS
adoption and use on firm performance and business process performance. We use
ERPS adoption, ERPS history and ERPS extension (with a SCMS) as independent
variables and KPIs for supply chain performance and firm performance as dependent
variables.

Perception-based measures were used to capture both financial and non-financial per-
formance indicators. We used independent samples tests to identify performance differ-
ences between ERPS users and non-ERP users, and we then analysed performance
differences within the groups of ERPS users based on two criteria: ERPS history and
the extension of the ERPS using SCM-software. This study is the first empirical study
investigating the impact of several aspects of ERPS-adoption on both core business
processes (supply chain performance) and overall firm performance.

Probably the most striking result was that we did not find any significant performance
differences between ERPS adopters and the control group – neither at the supply chain
level, nor at the overall firm level. However we found that the longer ago an ERPS was
initially implemented, the higher the overall firm performance, but – again surprisingly –
not the supply chain performance.

One of the explanations for the absence of a positive impact on firm performance in the
shorter run offered in literature is that the costs of the ERPS investment tend to exceed
short and medium-term productivity increases (Dehning and Richardson, 2002). While
our findings partly support this argument, they also reveal that even companies who
have implemented ERPS long ago do not perform better in their core business proc-
esses (supply chain) than companies with a shorter ERPS history (and non-ERPS
adopters). We can summarise that the ERPS-users in our study failed to achieve higher
supply chain performance, both in the short and long-term, and they failed to achieve
higher overall firm performance, although the latter improved with time since the adop-
tion; as this improvement is not due to improvements in core business processes, it is
supposedly due to improvements in supporting functions in the value chain (Porter,

18
1985). Finally, we found some evidence supporting our prediction that extending ERPS
with additional solutions for supply chain management actually has a positive impact on
an organisation’s performance along the supply chain.

We acknowledge that there are several limitations to our research, the most important
one being the low response rate to our survey and the small sample size. We are aware
that this limits explanatory power of our results.

We also made some observations which are not explicitly included in our analysis, but
which might also create incentives for further research. We noticed, for example, that
there is some confusion about the term ERPS and its relation to other forms of enter-
prise systems. We assume this confusion is stimulated by the ERP-vendors in 2 ways:
On the one hand, we found that more and more vendors of small functional software
packages seem to use the label ‘ERP’ inappropriately, e.g. for small accounting pack-
ages with integrated inventory management. On the other hand it is observed that the
large ERP-vendors SAP, Oracle and Peoplesoft start to change their interpretation and
use of the term ERP and package their traditional products in a variety of solutions,
which make it increasingly difficult to classify them as ERP, SCM, CRM, etc. In fact, fu-
ture research in ‘enterprise systems’ will have to redefine its underlying typologies of en-
terprise systems based on this ongoing changes in the business systems landscape.
This will open a pathway for research into e.g. the ‘optimum’ scope of integration and
advantages and disadvantages of various integration approaches (ERPS, SCMS,
CRMS, DW, Enterprise Application Integration, etc.).

19
6. REFERENCES

Beretta, S. (2002), “Unleashing the integration potential of ERP systems: the role of
process-based performance measurement systems”, Business Process
Management Journal, Vol. 8 No. 3, pp. 254-277.

Bhatt, G.D. (2000), “An empirical examination of the effects of information systems inte-
gration on business process improvement”, International Journal of Operations
and Production Management, Vol. 20 No. 11, pp. 1331-1359.

Booth, P., Matolcsy, Z. and Wieder, B. (2000), “The impacts of enterprise resource
planning systems on accounting practice - the Australian experience”, Australian
Accounting Review, Vol. 16, pp. 4-18.

CIO Magazine (2001), “ERP progress report”, [Link] (accessed

on 13/05/2004).

Dehning, B. and Richardson, D.J. (2002), “Returns of investments in information tech-

nology: a research synthesis”, Journal of Information Systems, Vol. 16 No. 1, pp.
7-30.

Hunton, J.E., Lippincott, B. and Reck, J.L. (2003), “Enterprise resource planning (ERP)
systems: comparing firm performance of adopters and non-adopters”, Interna-
tional Journal of Accounting Information Systems, Vol. 4, pp. 165-184.

Irani, Z. (2002), “Information systems evaluation: navigating through the problem do-
main”, Information and Management, Vol. 40, pp. 11-24.

Kaplan, R.S. and Norton, D.P. (1992), “The Balance Scorecard - measures that drive
performance”, Harvard Business Review, January-February, pp. 71-79.

Lee, Z. and Lee, J. (2000), “An ERP implementation case study from a knowledge
transfer perspective”, Journal of Information Technology, Vol. 15, pp. 281-288.

Markus, M.L., Axline, S., Petrie, D. and Tanis, C. (2000), “Learning from adopters’ ex-
perience with ERP problems encountered and success achieved”, Journal of In-
formation Technology, Vol. 15 No. 4, pp. 245-265.

Matolcsy, Z., Booth, P. and Wieder, B. (2002), “The economic benefits of enterprise re-
source planning systems: some empirical evidence”, AAANZ Conference, Perth,
July.

20
Matolcsy, Z., Booth, P. and Wieder, B. (2005), “The economic benefits of enterprise re-
source planning systems: some empirical evidence”, Journal of Accounting and
Finance, 2005 (forthcoming).

Nicolaou, A.I., Stratopoulos, T. and Dehning, B. (2003), “Financial analysis of potential

benefits from ERP systems adoption”, Journal of Business and Information
Technology, pp. 40-50.

Oracle (2003a), “E-Business Suite”, [Link]/applications/ (accessed on

8/8/03).

Oracle (2003b), “Oracle Customers”,

[Link]/customers/[Link]?[Link]/ (accessed on 8/8/03).

Peoplesoft (2003), [Link]/corp/en/products/line/scm/inv_plan/ (accessed

on 8/8/03).

Porter M.E. (1985), Competitive advantage: creating and sustaining superior perform-
ance, Free Press, New York/ London.

Poston, R. and Grabski, S. (2001), “Financial impact of enterprise resource planning

implementations”, International Journal of Accounting Information Systems, Vol.
2 No. 4, pp. 271-294.

SAP AG (2003a), “Benefits of ERP”, [Link]/solutions/erp/businessbenefits/ (ac-

cessed on 8/8/03).

SAP AG (2003b), “ROI of integrated SAP solutions (Netweaver)”,

[Link]/solutions/netweaver/businessbenefits/[Link] (accessed on
8/8/03).

Shang, S. and Seddon, P. (2002), “Assessing and managing the benefits of enterprise
systems: the business manager's perspective”, Information Systems Journal, Vol.
12, pp. 271-299.

Sumner, M. (1999), “Critical success factors in enterprise wide information management

systems projects”, Americas Conference on Information Systems, Milwaukee,
USA.

Tarn, M., Yen, D. and Beaumont, M. (2002), “Exploring the rationales for ERP and SCM
integration, Industrial Management and Data Systems”, Vol. 102 No. 1, pp. 26-
34.

21
7. APPENDIX: COMPONENTS OF AGGREGATED KPIS
The scores for firm performance were obtained asking the following question:

Please rate your firm’s performance on the criterion (1 = not at all satisfactory, 4 = satisfactory,
7 = outstanding).
Return on investment
Operating profits
Sales growth rate
Cost reduction programs
Cash-flow
(etc.)

The measurement of the SCM-KPIs was based on the following question:

Please rate your firm’s performance in the following attributes compared to other similar firms (1
= not at all satisfactory, 4 = satisfactory, 7 = outstanding).
Relative level of logistics and distribution costs
Relative level of manufacturing labour costs
Relative level of maintenance costs
Relative level of re-work costs
Relative level of materials inventory holding costs
Relative level of obsolete inventory costs
Relative level of work in progress inventory costs
Relative level of finished goods inventory holding costs
Speed at filling orders
Percentage of deliveries made on time
Stockout probability
Backorder level
Time required producing an item
Percentage of shipping errors
Percentage of customers who complain
The extent to which we can change the output level of products produced
The ease with which we can change the output level of products produced
The extent to which we can change planned delivery dates
The ease with which we can change planned delivery dates
The extent to which we can change the variety of products produced
The ease with which we can change the variety of products produced
The extent to which we can introduce and produce new products
The ease with which we can introduce and produce new products

22
1 Similar examples are provided by Lee and Lee (2000) and Tarn et al. (2002).
2 Secondary (support) processes such as accounting, IT, human resource management, are not
considered in this study.
3 [Link]
4 The individual performance indicators are listed in the appendix.
5 The measure was, however, relative in the sense that we asked respondents to benchmark their
company against the industry.
6 The fact that 269 questionnaires were returned by the receivers because of the addressed com-
panies having moved location gave reason to assume that a very large number of other question-
naires has not reached the addressee, but was not ‘voluntarily’ returned by the receiver. This as-
sumption was confirmed in the second stage of the survey, a telephone survey, which revealed
that many of the addressees selected in this second stage had never received the questionnaire
by mail, although we had not got their letters back marked ‘return to sender’.
7 See Booth et al. (2000) for details.
8 We also tried to identify CRMS-users and DW-users into our survey, but had to later exclude this
section due to many incomplete answers.

23