254 IEEE TRANSACTIONS ON SYSTEMS, MAN, AND CYBERNETICS—PART B: CYBERNETICS, VOL. 29, NO.

2, APRIL 1999

Correspondence
Identification of Nonlinear Dynamic Systems Using and control of nonlinear systems using a FLANN [13]. Chen and
Functional Link Artificial Neural Networks Billings [6] have reported nonlinear dynamic system modeling and
identification using three different ANN structures. They have studied
Jagdish C. Patra, Ranendra N. Pal, this problem using an MLP structure, a radial basis function (RBF)
B. N. Chatterji, and Ganapati Panda network and a FLANN and have obtained satisfactory results with
all the three networks.
Several research works have been reported on system identification
Abstract—In this paper, we have presented an alternate ANN structure using MLP networks in [1], [2], and [18] and using RBF networks
called functional link ANN (FLANN) for nonlinear dynamic system
identification using the popular back propagation algorithm. In contrast
in [5] and [7]. Recently, Yang and Tseng have reported function
to a feed forward ANN structure, i.e., a multilayer perceptron (MLP), approximation with an orthonormal ANN using Legendre functions
the FLANN is basically a single layer structure in which nonlinearity [19]. Nonlinear system identification using a FLANN structure has
is introduced by enhancing the input pattern with nonlinear functional been reported in [16]. Besides system identification, some other
expansion. With proper choice of functional expansion in a FLANN, this applications of FLANN in digital communications may be found in
network performs as good as and in some cases even better than the MLP
structure for the problem of nonlinear system identification. [14] and [15].
In this paper, we propose a novel alternate FLANN structure for
Index Terms— Artificial neural networks, computational complexity, identification of nonlinear static and dynamic systems. The proposed
nonlinear dynamic system identification.
approach is different from that of [6] and [13]. Here, we have
considered trigonometric polynomials for functional expansion, and
I. INTRODUCTION the output node contains a tan-hyperbolic nonlinearity. Where as,
Because of nonlinear signal processing and learning capability, in [13], Pao et al. have taken products of a random vector with
artificial neural networks (ANN’s) have become a powerful tool the input vector for the same purpose. Chen and Billings [6] have
for many complex applications including functional approximation, utilized a FLANN structure with polynomial expansion in terms of
nonlinear system identification and control, pattern recognition and outer product of the elements of the input vector for this purpose,
classification, and optimization. The ANN’s are capable of generating and the output node has linear characteristics. The performance of
complex mapping between the input and the output space and thus, the proposed FLANN structure has been compared with that of an
arbitrarily complex nonlinear decision boundaries can be formed by MLP structure with simulation by taking system model examples of
these networks. Narendra and Parthasarathy [9] and [10]. This type of performance
In contrast to the static systems that are described by algebraic comparison has not been attempted so far.
equations, the dynamic systems are described by difference or differ-
ential equations. It has been reported that even if only the outputs are
available for measurement, under certain assumptions, it is possible II. CHARACTERIZATION AND IDENTIFICATION OF SYSTEMS
to identify the dynamic system from the delayed inputs and outputs In system theory characterization and identification are fundamen-
using an multilayer perceptron (MLP) structure [4]. The problem of tal problems. When the plant behavior is completely unknown, it may
nonlinear dynamic system identification using MLP structure trained be characterized using certain model and then, it’s identification may
by BP algorithm was proposed by Narendra and Parthasarathy [9], be carried out with some networks like MLP or FLANN using some
[10]. Nguyen and Widrow have shown that satisfactory results can be learning rules such as BP algorithm.
obtained in the case of identification and control of highly nonlinear The primary concern of the problem of characterization is the
Truck-Backer-Upper system using an MLP [11]. mathematical representation of the system under study. Let us express
Originally, the functional link ANN (FLANN) was proposed by the model of a system by an operator P from an input space U into
Pao [12]. He has shown that, this network may be conveniently used an output space Y : The objective is to categorize the class P to which
for function approximation and pattern classification with faster con- P belongs. For a given class P ; P 2 P ; the identification problem
vergence rate and lesser computational load than an MLP structure. is to determine a class P^  P and P^ 2 P^ such that P^ approximates
The FLANN is basically a flat net and the need of the hidden layer is P in some desired sense. The spaces U and Y are subsets of R
n

removed and hence, the BP learning algorithm used in this network m

and R ; respectively in a static system. Where as, in the case of
becomes very simple. The functional expansion effectively increases dynamic systems they are assumed to be bounded Lebesgue integrable
the dimensionality of the input vector and hence the hyperplanes functions in the interval [0; T ] or [0; 1]: However, in both cases, the
generated by the FLANN provides greater discrimination capability operator P is defined implicitly by the specified input–output pairs
in the input pattern space. Pao et al. have reported identification [9].
A typical example of identification of static system is the problem
Manuscript received February 8, 1997; revised July 1, 1998. This paper
was recommended by Associate Editor P. Borne. of pattern recognition. By a decision function P ; compact input sets
n
J. C. Patra and G. Panda are with the Department of Applied Electronics Ui  R are mapped into elements yi 2 Rm for i = 1; 2; 1 1 1 ;
and Instrumentation Engineering, Regional Engineering College, Rourkela, in the output space. The elements of Ui denote the pattern vectors
Orissa 769 008, India. corresponding to class yi : Where as, in the case of a dynamic system,
R. N. Pal and B. N. Chatterji are with the Department of Electronics
and Electrical Communication Engineering, Indian Institute of Technology,
the input–output pairs of the time function u(t); y (t); t 2 [0; T ];
Kharagpur, W.B. 721 302, India. implicitly define the operator P describing the dynamic plant. The
Publisher Item Identifier S 1083-4419(99)02293-1. main objective in both type of identifications is to determine P^ such
1083–4419/99$10.00  1999 IEEE

Authorized licensed use limited to: MALAVIYA NATIONAL INSTITUTE OF TECHNOLOGY. Downloaded on March 30,2022 at 10:12:57 UTC from IEEE Xplore. Restrictions apply.
 IEEE TRANSACTIONS ON SYSTEMS, MAN, AND CYBERNETICS—PART B: CYBERNETICS, VOL. 29, NO. 2, APRIL 1999 255

III. THE MULTILAYER PERCEPTRON

The MLP is a feed forward network with one or more layer
of nodes between its input and output layers. Consider an L-layer
MLP [12] as shown in Fig. 2. This network may be represented
by NnL ;n ;111;n ; where nl ; l =01 ; ; 1 1 1 ; L denote the number of
nodes (excluding the threshold unit) in the input layer l ( = 0) ; the
hidden layers l ( =12 1)
; ; 1 1 1 ; L 0 and the output layer l L : ( = )
Let X (0) =[ 0 u1 u2 1 1 1 un
x(0) ] T
and X (l) =[ ]
x(0l) x(1l) 1 1 1 x(nl) T ;
represent the input vector and the lth layer output vector of the MLP,
respectively. Here, fuj g; j =12 ; ; 1 1 1 ; n0 denote input pattern and
x(jl) denote the output of the j th node of lth layer. The threshold
input is denoted by x0 and its value is fixed at
(l)
+1 : The synaptic
weight of the j th node of lth layer from the ith node of l 0 th ( 1)
layer is denoted by wji : The activation function associated with
Fig. 1. Schematic of identification of static and dynamic systems. (l)

all the nodes of the network (except the input layer) is the tanh
that function given by  S ( ) = tanh( ) = (1 S 0 e02S = e020 S : The
1+ )
ky 0 y^k = kP (u) 0 P^ (u)k < ; ( )
partial derivative of  S with respect to S is denoted by  S and ( )
(1) is given by 0 S ( ) = (1 ( ))
0 2(l)S : The linear sum of the j th node
0
where u 2 U ;  is some desired small value > ; and k1k is a defined of lth layer is denoted by Sj :
norm on the output space. In (1), P u ^( ) = ^ ( )=
y and P u y denote In the forward phase, at the kth time instant (here, the time index
the output of the identified model and the plant, respectively. The is omitted for simplicity of notation), the input pattern vector X (0)
= ^
error e y 0 y is the difference between the observed plant output is applied to the network. Let the corresponding desired output be
and the output generated by P : ^ =1 2
fyj g; for j ; ; 1 1 1 ; nL : Since no computation takes place in the
In Fig. 1, a schematic diagram of system identification of a time input layer the outputs of the input layer of the MLP is given by
invariant, causal discrete-time dynamic plant is shown. The input x(0) = =12
uj for j ; ; 1 1 1 ; n0 : For other layers, l =12 ; ; 1 1 1 ; L;
( )(= ) =12 ; ; 1 1 1 ; nl ; the outputs are computed as
j

and output of the plant is given by u and P u yp respectively, and j

where u is assumed to be an uniformly bounded function of time. The n
stability of the plant is assumed with a known parameterization but
with unknown parameter values. The objective of the identification
x(jl) = (Sj(l) ) and Sj(l) = wji(l) x(il01) : (3)
problem is to construct a suitable model generating an output P u ^ ( )(= i=0

^)
yp which approximates the plant output yp when subjected to the ^
The estimated output is denoted by fyj g and is given by fyj g ^ =
same input u so that the error e is minimum. fx g
(L)
j for all j = 12
; ; 1 1 1 ; nL : The mean square error (MSE)
Four models for representation of SISO plants are introduced is given by e2 =nL where the error signal for the j th output is
which may also be generalized for multivariable case. The nonlinear ej = ^
yj 0 yj and the instantaneous squared error is given by
difference equations describing the four models are as follows: e2 =6 n
j =1 ej2 :
• Model 1: In the learning phase, the BP algorithm minimizes the squared error
01 by recursively altering fwji g based on the gradient search technique.
(l)
n

yp (k + 1) = i yp (k 0 i) The squared error derivative associated with the j th node in layer l

i=0 is defined as
+ g[u(k); u(k 0 1); 1 1 1 ; u(k 0 m + 1)]:
j(l) = 0 1 @e(l) :
2

• Model 2: 2 @Sj (4)

yp (k + 1) = f [yp (k); yp (k 0 1); 1 1 1 ; yp (k 0 n + 1)]

m01 These derivatives may be found out as
+ i u(k 0 i):
0 (Sj(l) ) 1 ej for l = L
(2)

=
i=0
j 0 (Sj(l) ) 1 ni=1 wij(l+1) 1 i(l+1)
(l)
(5)
for l = L 0 1; L 0 2; 1 1 1 ; 1:
• Model 3:
yp (k + 1) = f [yp (k); yp (k 0 1); 1 1 1 ; yp (k 0 n + 1)]
+ g[u(k); u(k 0 1); 1 1 1 ; u(k 0 m + 1)]: Finally, at the kth instant the weights of the MLP are updated as
follows:
• Model 4:
yp (k + 1) = f [yp (k); yp (k 0 1); 1 1 1 ; yp (k 0 n + 1); wji(l) (k + 1) = wji(l) (k) + 1(jil) (k) and
+ u(k); u(k 0 1); 1 1 1 ; u(k 0 m + 1)]: 1(jil) (k) = j(l) x(il01) + 1(jil) (k 0 1) (6)
Here, u(k) and yp (k) represent the input and output of the SISO where  and
plant respectively at the kth time instant and m  n: In this study
denote learning rate and momentum rate parameters,
respectively.
an MLP and a FLANN structure have been used to construct the
functions f and/or g in (2) so as to approximate such mappings over
compact sets. For the identification problem discussed in this paper, IV. MATHEMATICAL ANALYSIS OF FLANN
a series-parallel scheme has been utilized in which, the output of the The learning of an ANN may be considered as approximating
plant, instead of the ANN model is fed back into the model during or interpolating a continuous, multivariate function f X by an ( )
the training period for stability reasons [9]. ( )
approximating function fW X : In the FLANN, a set of basis

Authorized licensed use limited to: MALAVIYA NATIONAL INSTITUTE OF TECHNOLOGY. Downloaded on March 30,2022 at 10:12:57 UTC from IEEE Xplore. Restrictions apply.
 256 IEEE TRANSACTIONS ON SYSTEMS, MAN, AND CYBERNETICS—PART B: CYBERNETICS, VOL. 29, NO. 2, APRIL 1999

Fig. 2. The structure of an multilayer perceptron.

functions 8 and a fixed number of weight parameters W are used to

represent fW (X ): With a specific choice of a set of basis functions,
the problem is then to find the weight parameters W that provides the
best possible approximation of f on the set of input–output examples.
The theory behind the FLANN for the purpose of multidimensional
function approximation has been discussed in [14] and [17] and is
analyzed below.

A. Structure of the FLANN

Let us consider a set of basis functions B = fi 2 L(A)gi2I with
the following properties: 1) 1 = 1; 2) the subset Bj = fi 2 Bgji=1
(a)
is linearly independent set, i.e., if 6N i=1 wi i = 0; then wi = 0 for all
i = 1; 2; 1 1 1 ; j; and 3) supj [6ji=1 ki k2A ]1=2 < 1: Let BN = fgiN=1
be a set of basis functions to be considered for the FLANN as
shown in Fig. 3(a). Thus, the FLANN consists of N basis functions
f1 ; 2 ; 1 1 1 ; N g 2 BN with the following input–output relationship
for the j th output
N
y^j = (Sj ); Sj = wji i (X ); (7)
i=1

where X 2 A  Rn ; i.e., X = [x1 x2 1 1 1 xn ]T is the input pattern

vector, y^ 2 Rm ; i.e., y^ = [^ y1 y^2 1 1 1 y^m ]T is the output vector and (b)
wj = [wj 1 wj 2 1 1 1 wjN ] is the weight vector associated with the j th
output of the FLANN. The nonlinear function (1) = tanh(1):
Fig. 3. The structure of a functional link ANN (a) general structure and (b)
with a single output node.
Considering the m-dimensional output vector, (7) may be written
as
through a functional expansion block producing a corresponding N -
S = W8 (8) dimensional (N  n) expanded vector. In this case, the dimension
of the weight matrix is of 1 2 N and hence, the individual weights
where W is a (m 2 N ) weight matrix of the FLANN given by
are represented by a single subscript. Let w = [w1 w2 1 1 1 wN ] be the
W = [w1 w2 1 1 1 wm ]T ;  = [1 (X )2 (X ) 1 1 1 N (X )]T is the basis
weight vector of this FLANN. The linear weighted sum, Sk is passed
function vector, and S = [S1 S2 1 1 1 SN ]T is a matrix of linear outputs
through the tanh(1) nonlinear function to produce the output y^k with
of the FLANN. The m-dimensional output vector y^ may be given by
the following relationship:
y^ = (S ) = fW (X ): (9)
y^k = tanh(Sk ); or; Sk = 12 loge 11 0
+ y^k
y^k
: (10)
B. Learning with the FLANN
Referring to Fig. 3(b), let there be K number of input–output
pattern pairs to be learned by the FLANN. Let the input pattern The FLANN with a set of basis functions  attempts to approximate
vector X is of dimension n and for ease of understanding, let the the nonlinear function f : A  Rn ! R using a set of K training
output, y be a scalar, i.e., m = 1: Each of the input pattern is passed patterns.

Authorized licensed use limited to: MALAVIYA NATIONAL INSTITUTE OF TECHNOLOGY. Downloaded on March 30,2022 at 10:12:57 UTC from IEEE Xplore. Restrictions apply.
 IEEE TRANSACTIONS ON SYSTEMS, MAN, AND CYBERNETICS—PART B: CYBERNETICS, VOL. 29, NO. 2, APRIL 1999 257

Considering all K patterns, input–output relationship may be TABLE I

expressed as COMPARISON OF COMPUTATIONAL COMPLEXITY BETWEEN AN L-LAYER
MLP AND A FLANN IN ONE ITERATION WITH BP ALGORITHM
8wT = S; (11)

where 8 is a K 2 N -dimensional matrix given by 8 =

[(X1 )(X2 ) 1 1 1 (XK )]T and S is a K -dimensional vector given
by S = [S1 S2 1 1 1 SK ]T : Thus, from (11) it is evident that finding
the weights of the FLANN requires the solution of K number of
simultaneous equations. The number of basis functions, N is so
chosen that K  N: Now, depending on the value of K and N;
two cases may arise.
Case I: K = N: If the determinant of 8; i.e., Det8 6= 0; the
weight solution is given by wT = 801 S:
Case II: K < N: The matrix 8 may be partitioned to obtain a compact representation of the function in the mean square sense.
matrix of 8F of dimension K 2 K: Let w be modified to wF such However, when the outer product terms were used along with the
that wi = 0 for i > K: If Det8F 6= 0; then the weight solution trigonometric polynomials for functional expansion, better results
is given by wFT = 8F 01 S: The FLANN obtains the weight solution were obtained in the case of learning of a two-variable function [12].
iteratively by using the training algorithm to be described below.
E. Computational Complexity
C. The Learning Algorithm Here, we present a comparison of computational complexity be-
Let K number of patterns be applied to the network in a sequence tween an MLP and a FLANN structure trained by the BP algorithm.
repeatedly. Let the training sequence be denoted by fXk ; yk g and the Let us consider an L-layer MLP with nl number of nodes (excluding
weight of the network be W (k); where k is the discrete time index the threshold unit) in layer l; l = 0; 1; 1 1 1 ; L; where n0 and nL are
given by k =  + K; for  = 0; 1; 2; 1 1 1 ; and  = 1; 2; 1 1 1 ; K: the number of nodes in the input layer and output layer, respectively.
Referring to (7) the j th output of the FLANN at time k is given by Three basic computations, i.e., the addition, the multiplication and the
computation of tanh(1) are involved for updating the weights of an
N MLP. In the case of FLANN, in addition, computations of cos(1) and
^ ( ) =
yj k wji (k)i (Xk ) sin(1) are also involved. The computations in the network are due to
i=1
= (wj (k)T (Xk )) (12)
1) forward calculation to find the activation value of all nodes of
the entire network;
for all X 2 A and j = 1; 2; 1 1 1 ; m; where (Xk ) =
2) back error propagation for calculation of square error deriva-
[1 (Xk )2 (Xk ) 1 1 1 N (Xk )]: Let the corresponding error be tives;
denoted by ej (k) = yj (k) 0 y^j (k):
3) updating of the weights of the entire network.
The total number of weights to be updated in one iteration in an MLP
Using the BP algorithm (6) for a single layer, the update rule for 01 (nl + 1)nl+1 ): Whereas in the case of a
structure is given by (6lL=0
all the weights of the FLANN is given by
FLANN the same is only (n0 +1): Since hidden layer does not exist
W k( + 1) = W (k) + (k)(Xk ) (13) in a FLANN, the computational complexity is drastically reduced in
comparison to that of an MLP. A comparison of computational load
where W (k) = [w1 (k)w2 (k) 1 1 1 wm (k)]T is the m 2 N dimensional in one iteration, for an MLP and a FLANN structure is provided in
weight matrix of the FLANN at the kth time instant,  (k) = Table I.
[1 (k)2 (k) 1 1 1 m (k)]T ; and [j (k) = (1 0 y^j (k)2 )ej (k)]:
V. SIMULATION STUDIES
D. Motivation for Using Trigonometric Polynomials Extensive simulation studies were carried out with several ex-
From (11) it may be seen that the condition for existence of amples for static as well as nonlinear dynamic systems. We have
weight solution depends on the existence of inverse of the matrix compared the performance of the proposed FLANN structure with
8: This can be assured only if the matrix equation (11) is linearly that of the MLP structure for this problem mainly by taking system
independent and this may be achieved by the use of suitable orthog- examples reported by Narendra and Parthasarathy [9], [10].
onal polynomials for functional expansion. The examples of which
include Legendre, Chebyshev and trigonometric polynomials. Besides A. Static Systems
orthogonal functions, other functions which have been used success- Here, different nonlinear static systems are chosen to examine the
fully for the purpose of multidimensional functional approximation approximation capabilities of the MLP and the FLANN. In all the
include sigmoid functions [8] and Gaussian functions [3]. Basically, simulation studies reported in this paper, a three-layer MLP structure
an MLP uses sigmoid functions for nonlinear mapping between the with 20 and ten nodes (excluding the threshold unit) in the first
input–output space. and second layers respectively and one input node and one output
Some of the advantages of using trigonometric polynomials for node was chosen for the purpose of identification of both static and
use in the functional expansion are explained below. Of all the dynamic systems. The same MLP structure was utilized for simulation
polynomials of N th order with respect to an orthonormal system studies as reported in [9] and it has total of 261 weights, which
fi (u)giN=1 the best approximation in the metric space L2 is given by are to be updated in one iteration during learning. Where as, in a
the N th partial sum of its Fourier series with respect this system Thus, FLANN, the number of input nodes differ depending on system model
the trigonometric polynomial basis functions given by f1; cos(u); chosen. In static identification, the FLANN structure has 14 number
sin(u); cos(2u); sin(2u); 1 1 1 ; cos(N u); sin(N u)g provides a of input nodes. Thus, it has only 15 weights including the threshold

Authorized licensed use limited to: MALAVIYA NATIONAL INSTITUTE OF TECHNOLOGY. Downloaded on March 30,2022 at 10:12:57 UTC from IEEE Xplore. Restrictions apply.
 258 IEEE TRANSACTIONS ON SYSTEMS, MAN, AND CYBERNETICS—PART B: CYBERNETICS, VOL. 29, NO. 2, APRIL 1999

(a) (b)

(c) (d)
Fig. 4. Identification of static maps: (a) f1 using MLP, (b) f2 using MLP, (c) f1 using FLANN, and (d) f2 using FLANN.

unit, which are to be updated in one iteration. The input pattern a random signal drawn from an uniform distribution in the interval
was expanded by using trigonometric polynomials, i.e., by using [01, 1]. Both the convergence parameter  and the momentum term
cos(nu) and sin(nu); for n = 0; 1; 2; 1 1 1 : In some cases, the were set to 0.1. Both the MLP and the FLANN were trained for
cross product terms were also included in the functional expansion. 50 000 iterations, after which the weights of the ANN were stored
The nonlinearity used in a node of the MLP and the FLANN is the for testing.
tanh(1) function. The four functions considered for this study are as The results of identification of f1 and f2 (14) are shown in Fig. 4.
follows [10]: Here, the system output and the model output are represented by
f (u) and f^(u) and marked in these figures as “true” and “estimated”
(a) f1 (u) = u3 + 0:3u2 0 0:4u; respectively. From these figures, it may be seen that the performance
(b) f2 (u) = 0:6 sin(u) + 0:3 sin(3u) + 0:1 sin(5u); of the MLP with f1 (u) is quite satisfactory, where as, with f2 (u) it
4:0u 0 1:2u 0 3:0u + 1:2
3 2 is not very good. For the FLANN structure, quite close agreement
(c) f3 (u) = ;
0:4u + 0:8u 0 1:2u3 + 0:2u2 0 3:0
5 4 between the system output and the model output is evident. In fact,
2:0 the modeling error of the FLANN structure is found to be much less
(d) f4 (u) = 0:5 sin3 (u) 0 3 0 0:1 cos(4u) than that of the MLP structure for all the four nonlinear functions
u + 2:0
+ 1:125: (14) considered.

The scheme for the identification of static and dynamic systems is

shown in Fig. 1. Here, the system P; belongs to either a static map or B. Dynamic Systems
a dynamic system. The output of the ANN model P^ (u) and the output In the following we have undertaken simulation studies of nonlin-
of the system P (u) is compared to produce an error e; which is then ear dynamic systems with the help of several examples using Fig. 1.
utilized to update the weights of the model. The BP algorithm was The nonlinear functions given in (14) was used in the characterization
used to adapt the weights of both the ANN structures. The input u was of the dynamic plants. In each example, one particular model of the

Authorized licensed use limited to: MALAVIYA NATIONAL INSTITUTE OF TECHNOLOGY. Downloaded on March 30,2022 at 10:12:57 UTC from IEEE Xplore. Restrictions apply.
 IEEE TRANSACTIONS ON SYSTEMS, MAN, AND CYBERNETICS—PART B: CYBERNETICS, VOL. 29, NO. 2, APRIL 1999 259

(a) (b)

(c) (d)
Fig. 5. Identification of the second order plant (Example 1): (a) with f3 using MLP, (b) with f4 using MLP, (c) with f3 using FLANN, (d) with
f4 using FLANN.

unknown system is considered. The input to the plant was taken from equation of Model 1 given in (2). The plant is assumed to be of
an uniformly distributed random signal over the interval [01; 1]: The second order and is described by the following difference equation:
convergence factor  and momentum factor were chosen differently
for different examples. The adaptation continues for 50 000 iterations yp (k + 1) = 0:3yp (k) + 0:6yp (k 0 1) + g[u(k)]: (16)
during which the series-parallel scheme of identification was used.
where the nonlinear function g is unknown but 0 = 0:3 and
Then, the adaptation was stopped and the network was used for
1 = 0:6 are assumed to be known. The unknown function g was
testing for identification using the parallel scheme. This procedure of
taken from the nonlinear functions of (14). To identify the plant,
training and testing was carried out for all examples illustrated here.
a series-parallel model was considered which is governed by the
The testing of the network models was undertaken by presenting a
difference equation
sinusoidal input to the identified model given by
2k
y^p (k + 1) = 0:3yp (k) + 0:6yp (k 0 1) + N [u(k)]: (17)
sin
250 The MLP used for this purpose has f1–20–10–1g structure. The
u(k) = for k  250 (15)
2k 2k input was expanded to 14 terms by the trigonometric polynomials
0:8 sin + 0:2 sin and used in the FLANN. Both  and were chosen to be 0.1 for the
250 25
for k > 250: two ANN structures. The results of identification (16) with nonlinear
functions f3 and f4 of (14) are shown in Fig. 5.
Performance comparison between an MLP and a FLANN structure Example 2: In this example, the plant to be identified is of Model
in terms of, estimated output of the unknown plant and modeling 2 of (2) and is described by the following second order difference
error has been carried out. equation
Example 1: In the first example of identification of nonlinear
dynamic systems, we consider a system described by the difference yp (k + 1) = f [yp (k); yp (k 0 1)] + u(k): (18)

Authorized licensed use limited to: MALAVIYA NATIONAL INSTITUTE OF TECHNOLOGY. Downloaded on March 30,2022 at 10:12:57 UTC from IEEE Xplore. Restrictions apply.
 260 IEEE TRANSACTIONS ON SYSTEMS, MAN, AND CYBERNETICS—PART B: CYBERNETICS, VOL. 29, NO. 2, APRIL 1999

(a) (a)

(b) (b)
Fig. 6. Identification of nonlinear plant (Example 2) (a) using MLP and (b) Fig. 7. Identification of nonlinear plant (Example 3) (a) using MLP and (b)
using FLANN. using FLANN.

It is known a priori that the output of the plant depends only on the where the unknown functions f and g have the following form given
past two values of the output and input to the plant at the previous by
instant. The function f is unknown and is given by
y (y + 0:3)
y1 y2 (y1 + 2:5)(y1 0 1:0) f (y ) = ; and g (u) = u(u + 0:8)(u 0 0:5):
f1 (y1 ; y2 ) = : (19) 1:0 + y 2
1:0 + y12 + y22 (22)
A series-parallel scheme was adopted for the identification of this The series-parallel scheme for this plant is given by the difference
plant and is described by the difference equation equation
y^p (k + 1) = N [yp (k); yp (k 0 1)] + u(k): (20) y^p (k + 1) = N1 [yp (k)] + N2 [u(k)]; (23)

An MLP with f2–20–10–1g structure was used in this example. In where N1 [1] and N2 [1] are the two ANN’s used to approximate the
the FLANN structure, the two-dimensional input vector was expanded nonlinear functions f and g; respectively.
to a dimension of 24 by using trigonometric functions. The values In the MLP, both N1 and N2 were of f1–20–10–1g structure.
of  and used were 0.05 and 0.1 respectively for both the ANN In the FLANN, the expanded input vector dimensions of 14 and 24
structures. The outputs of both the plant and ANN model and were used for N1 and N2 respectively using trigonometric functions.
corresponding error with the nonlinear functions f1 (19) are shown Both  and were chosen as 0.1 for both the ANN structures in this
in Fig. 6. example. Both the plant output and ANN model output and modeling
Example 3: Here, the plant model chosen belong to the Model 3 error, using the MLP and the FLANN structure are depicted in Fig. 7.
as given in (2) and is described by the following difference equation: Example 4: The example of the plant chosen here, is the most
general of all the examples described so far and belongs to Model 4
yp (k + 1) = f [yp (k)] + g [u(k)]; (21) of (2). The difference equation governing the plant which is used in

Authorized licensed use limited to: MALAVIYA NATIONAL INSTITUTE OF TECHNOLOGY. Downloaded on March 30,2022 at 10:12:57 UTC from IEEE Xplore. Restrictions apply.
 IEEE TRANSACTIONS ON SYSTEMS, MAN, AND CYBERNETICS—PART B: CYBERNETICS, VOL. 29, NO. 2, APRIL 1999 261

this simulation is given by

yp (k + 1) = f [y (k); y (k 0 1); y (k 0 2); u(k); u(k 0 1)]

(24)

where the unknown nonlinear function f is given by

a1 a2 a3 a5 (a3 0 1:0) + a4
f [a 1 ; a2 ; a3 ; a4 ; a 5 ] = : (25)
1:0 + a22 + a23

The series-parallel model for identification of this plant is given by

y^p (k + 1) = N [yp (k); yp (k 0 1); yp (k 0 2); u(k); u(k 0 1)]:

(26)

In the case of MLP, N represents a f5–20–10–1g structure. Where

as, in the FLANN structure, the input u(1) was expanded by ten terms
and the output was also expanded by ten terms by using trigonometric
polynomials and some cross-product terms and then used for the
identification problem. Thus, the FLANN used for this purpose had (a)
20 input nodes and a single output node. For the two ANN structures,
both the convergence parameter  and the momentum factor were
set at 0.1.
The outputs of the plant and the model along with their correspond-
ing error, are shown in Fig. 8. the MLP and the FLANN structure
respectively. From the simulation results (Figs. 5–8), it may be seen
that the model outputs closely agree with the plant output for both the
MLP and the FLANN based structures. However, the performance of
the FLANN structure is superior to that of MLP as in the former
ANN structure the modeling error is less in several examples.
Comparison of computational complexity between the MLP and
the FLANN using BP learning algorithm is provided in Table II.
Here, number of additions, multiplications, etc., needed per iteration
during training period using BP algorithm are indicated for different
examples studied in this paper. From this table it may be inferred that
for all the selected examples in this study, the computational load on
the FLANN is much less than that of the MLP.

VI. CONCLUSIONS (b)

In this study of identification of nonlinear dynamic systems, we
Fig. 8. Identification of nonlinear plant (Example 4) (a) using MLP and (b)
have proposed a novel ANN structure based on FLANN. Here, the using FLANN.
input pattern is expanded using trigonometric polynomials and cross-
product terms of the input vector. The functional expansion may be
thought of analogous to the nonlinear processing of signals in the TABLE II
EXAMPLE-WISE COMPARISON OF COMPUTATIONAL COMPLEXITY
hidden layer of an MLP. This functional expansion increases the
dimensionality of the input pattern and thus, creation of nonlinear
decision boundaries in the multidimensional space and identification
of complex nonlinear functions become simple with this network.
Since, the hidden layer is absent in this structure, the computational
complexity is less and thus, the learning is faster in comparison to
an MLP. Therefore, this structure may be implemented for on-line
applications.
Four models of nonlinear systems of increasing order of complexity
have been considered here for identification purpose. Mainly by
taking examples from [9] and [10], extensive simulation studies
were carried out. System identification with the FLANN structure
is found to be quite effective for all the four models considered here.
Performance comparison between an MLP and a FLANN structure
in terms of computational complexity, and modeling error between REFERENCES
the plant and model outputs has been carried out. It is shown that
[1] S. Bhama and H. Singh, “Single layer neural networks for linear system
the overall performance of a suitably chosen FLANN structure is
identification using gradient descent technique,” IEEE Trans. Neural
superior to an MLP structure for identification of nonlinear static as Networks, vol. 4, pp. 884–888, Sept. 1993.
well as dynamic systems. The FLANN structure may also be applied [2] N. V. Bhat et al., “Modeling chemical process systems via neural
to other nonlinear signal processing applications. computation, IEEE Contr. Syst. Mag., pp. 24–29, Apr. 1990, .

Authorized licensed use limited to: MALAVIYA NATIONAL INSTITUTE OF TECHNOLOGY. Downloaded on March 30,2022 at 10:12:57 UTC from IEEE Xplore. Restrictions apply.
 262 IEEE TRANSACTIONS ON SYSTEMS, MAN, AND CYBERNETICS—PART B: CYBERNETICS, VOL. 29, NO. 2, APRIL 1999

[3] D. S. Broomhead and D. H. Lowe, “Multivariable functional inter- Nonlinear Channel Equalization for QAM Signal
polation and adaptive networks,” Complex Syst., vol. 2, pp. 321–355, Constellation Using Artificial Neural Networks
1988.
[4] S. Chen, S. A. Billings and P. M. Grant, “Nonlinear system identification
using neural networks,” Int. J. Contr., vol. 51, no. 6, pp. 1191–1214, Jagdish C. Patra, Ranendra N. Pal,
1990. , “Recursive hybrid algorithm for nonlinear system identifi- Rameswar Baliarsingh, and Ganapati Panda
cation using radial basis function networks,” Int. J. Contr., vol. 55, no.
5, pp. 1051–1070, 1992.
[5] S. Chen and S. A. Billings, “Neural networks for nonlinear dynamic Abstract—Application of artificial neural networks (ANN’s) to adaptive
system modeling and identification,” Int. J. Contr., vol. 56, no. 2, pp. channel equalization in a digital communication system with 4-QAM
319–346, 1992. signal constellation is reported in this paper. A novel computationally
[6] S. V. T. Elanayar and Y. C. Shin, “Radial basis function neural net- efficient single layer functional link ANN (FLANN) is proposed for this
work for approximation and estimation of nonlinear stochastic dynamic purpose. This network has a simple structure in which the nonlinearity is
systems,” IEEE Trans. Neural Networks, vol. 5, pp. 594–603, July 1994. introduced by functional expansion of the input pattern by trigonometric
[7] L. K. Jones, “Constructive approximations for neural networks by polynomials. Because of input pattern enhancement, the FLANN is capa-
sigmoidal functions,” Proc. IEEE, vol. 78, pp. 1586–1589, Oct. 1990. ble of forming arbitrarily nonlinear decision boundaries and can perform
[8] K. S. Narendra and K. Parthasarathy, “Identification and control of dy- complex pattern classification tasks. Considering channel equalization
namical systems using neural networks,” IEEE Trans. Neural Networks, as a nonlinear classification problem, the FLANN has been utilized
vol. 1, pp. 4–27, Mar. 1990. for nonlinear channel equalization. The performance of the FLANN is
[9] , “Neural networks and dynamical systems, Part II: Identification,” compared with two other ANN structures [a multilayer perceptron (MLP)
Tech. Rep. 8902, Center Syst. Sci., Dept. Elect. Eng., Yale Univ., New and a polynomial perceptron network (PPN)] along with a conventional
Haven, CT, Feb. 1989. linear LMS-based equalizer for different linear and nonlinear channel
[10] D. H. Nguyen and B. Widrow, “Neural networks for self-learning control models. The effect of eigenvalue ratio (EVR) of input correlation matrix
systems,” Int. J. Contr., vol. 54, no. 6, pp. 1439–1451, 1991. on the equalizer performance has been studied. The comparison of
[11] Y.-H. Pao, Adaptive Pattern Recognition and Neural Networks. Read- computational complexity involved for the three ANN structures is also
ing, MA: Addison-Wesley, 1989. provided.
[12] Y.-H. Pao, S. M. Phillips and D. J. Sobajic, “Neural-net computing and
intelligent control systems,” Int. J. Contr., vol. 56, no. 2, pp. 263–289, Index Terms— Functional link artificial neural networks, multilayer
1992. perceptron, nonlinear channel equalization, polynomial perceptron, QAM
[13] J. C. Patra, “Some studies on artificial neural networks for signal pro- signals.
cessing applications,” Ph.D. dissertation, Indian Inst. Technol., Kharag-
pur, Dec. 1996.
[14] J. C. Patra and R. N. Pal, “A functional link artificial neural network for I. INTRODUCTION
adaptive channel equalization,” Signal Process., vol. 43, pp. 181–195,
May 1995. For high-speed digital data transmission over a communication
[15] J. C. Patra, R. N. Pal and B. N. Chatterji, “FLANN-based identification channel effectively, the adverse effects of the dispersive channel
of nonlinear systems,” Proc. 5th Eur. Congr. Intelligent Techniques causing intersymbol interference (ISI), the nonlinearities introduced
Soft Computing (EUFIT’97), Sept. 1997, Aachen, Germany, vol. 1, pp. by the modulation/demodulation process and the noise generated
454–458.
[16] N. Sadegh, “A perceptron based neural network for identification and in the system are to be suitably compensated. The performance of
control of nonlinear systems,” IEEE Trans. Neural Networks, vol. 4, pp. the linear channel equalizers employing a linear filter with FIR or
982–988, Nov. 1993. lattice structure and using a least mean square (LMS) or recursive
[17] T. Yamada and T. Yabuta, “Dynamic system identification using neural least-squares (RLS) algorithm is limited specially when the nonlinear
networks,” IEEE Trans. Syst., Man, Cybern., vol. 23, pp. 204–211,
Jan./Feb. 1993.
distortion is severe. In such cases, nonlinear equalizer structures may
[18] S. S. Yang and C. S. Tseng, “An orthonormal neural network for be conveniently employed with added advantage in terms of lower
function approximation,” IEEE Trans. Syst., Man, Cybern. B, vol. 26, bit error rate (BER), lower mean square error (MSE) and higher
pp. 779–785, Oct. 1996. convergence rate than those of a linear equalizer.
Artificial neural networks (ANN’s) can perform complex mapping
between its input and output space and are capable of forming
complex decision regions with nonlinear decision boundaries. Further,
because of nonlinear characteristics of the ANN’s, these networks of
different architecture have found successful application in channel
equalization problem. One of the earliest applications of the ANN in
digital communication channel equalization is reported by Siu et al.
[13]. They have proposed a multilayer perceptron (MLP) structure for
channel equalization with decision feedback and have shown that the
performance of this network is superior to that of a linear equalizer
trained with LMS algorithm. Using MLP structures in the problem

Manuscript received February 8, 1997; revised July 10, 1998. This paper
was recommended by Associate Editor P. Borne.
J. C. Patra and G. Panda are with the Department of Applied Electronics
and Instrumentation Engineering, Regional Engineering College, Rourkela 769
008, India.
R. N. Pal is with the Department of Electronics and Electrical Communica-
tion Engineering, Indian Institute of Technology, Kharagpur 721 302, India.
R. Baliarsingh is with the Department of Computer Science, Engineering
and Applications, Regional Engineering College, Rourkela 769 008, India.
Publisher Item Identifier S 1083-4419(99)02294-3.

1083–4419/99$10.00  1999 IEEE

Authorized licensed use limited to: MALAVIYA NATIONAL INSTITUTE OF TECHNOLOGY. Downloaded on March 30,2022 at 10:12:57 UTC from IEEE Xplore. Restrictions apply.