Time Series Prediction with Multilayer
Perceptron (MLP): A New Generalized Error
Based Approach
Md. Shiblee1 , P.K. Kalra1, and B. Chandra2,
1
Department of Electrical Engineering
Indian Institute of Technology Kanpur, India
2
Department of Industrial Engineering and Managment
Indian Institute of Technology Kanpur, India
{shiblee,kalra}@iitk.ac.in,
[email protected] Abstract. The paper aims at training multilayer perceptron with dif-
ferent new error measures. Traditionally in MLP, Least Mean Square
error (LMSE) based on Euclidean distance measure is used. However
Euclidean distance measure is optimal distance metric for Gaussian dis-
tribution. Often in real life situations, data does not follow the Gaussian
distribution. In such a case, one has to resort to error measures other
than LMSE which are based on different distance metrics [7,8]. It has
been illustrated in this paper on wide variety of well known time se-
ries prediction problems that generalized geometric and harmonic error
measures perform better than LMSE for wide class of problems.
1 Introduction
Time series prediction and forecasting are key problems of function approxima-
tion. In the existing literature, various neural network learning algorithms have
been used for these problems [1,2,3]. In this paper, we have used multilayer per-
ceptron (MLP) neural network for time series prediction. MLP is composed of a
hierarchy of processing units, organized in series of two or more mutually exclu-
sive sets of neurons or layers. The input layer serves as the holding site for the
input applied to the network. The output layer is the point at which the overall
mapping of the network input is available [4,5]. Most widely used algorithm for
learning MLP is the back-propagation algorithm [6]. In Back-propagation algo-
rithm, error between target value and observed value is minimized. Typically
Euclidean distance is used to for the error measure. It has been proven based
on Maximum Likelihood criterion that Euclidean distance is optimal distance
metric for Gaussian distribution [7,8].Since the distribution of data is unknown,
using LMSE for training of MLP may not give the actual approximation of the
functions.In this paper, Back-propagation algorithm with some new error mea-
sures based on distance metrics (for similarity measures) given by Jie Yu et al
Permanent Add: Department of Mathematics, IIT Delhi, India.
M. Köppen et al. (Eds.): ICONIP 2008, Part II, LNCS 5507, pp. 37–44, 2009.
c Springer-Verlag Berlin Heidelberg 2009
38 Md. Shiblee, P.K. Kalra, and B. Chandra
[7,8] has been used for training MLP for time series prediction. Comparative
performance for each of the error measures has been done on a wide variety of
well known time series prediction problems. The performance measures used are
chi-square goodness of fit test, AIC and training and testing error.
Rest of the paper is organized as follows, Section 2, gives the overview of
distance metrics used as various error measures. The network architecture of
feed-forward (MLP) neural network is presented in section 3. In section 4, learn-
ing rule of MLP using Back-propagation (BP) algorithm with different error
measures has been discussed. In section 5, results have been tested on several
widely known time series prediction and forecasting data sets. In section 6, we
have concluded our work.
2 New Error Measures
Traditionally Back-propagation (BP) algorithm is used in training Artificial Neu-
ral Network (ANN). The performance of Neural Network depends on the method
of error computation. It has been proven that when the underlying distribution
of data is Gaussian, Least mean square error (LMSE) [7,8] is best for training
the network. %begincenter Since the distribution of data is unknown, it may be
possible that error computed based on Euclidean distance may not be suitable
for training of neural network. It will be reasonable to assume that there may
be some distance metric which will give new error measure to fit the unknown
data better. Some new error measures based on distance metric for similarity
measures [7,8] is given in Table 1. In Table 1, yi denotes the desired value of
neuron, ti denotes the target value and N denotes number of pattern.In error
estimation, if the target value is far away from the desired value, error measure
Table 1.
Error Measure
N
LMS E = 12 (ti − yi )2
i=1
N
Geometric E= 1
2
[log(ti /yi )]2
i=1
N
Harmonic E= 1
2
[ti (yi /ti ) − 1]2
i=1
N
Generalized Geometric E= 1
2
[ti r log(ti /yi )]2
i=1
N
Generalized Harmonic Type-1 E = 1
2
[ti p (yi /ti ) − 1]2
i=1
N
Generalized Harmonic Type-2 E = 12 (ti q − y i q )2
i=1
